Kubernetes集群 服务暴露 Traefik¶
一、认识traefik¶
1.1 traefik简介¶
-
参考链接: https://traefik.cn/
-
是一个为了让部署微服务更加便捷而诞生的现代HTTP反向代理、负载均衡工具。
-
它支持多种后台 (Docker, Swarm, Kubernetes, Marathon, Mesos, Consul, Etcd, Zookeeper, BoltDB, Rest API, file…) 来自动化、动态的应用它的配置文件设置。
1.2 traefix 特性¶
- 非常快
- 无需安装其他依赖，通过Go语言编写的单一可执行文件
- 支持 Rest API
- 多种后台支持：Docker, Swarm, Kubernetes, Marathon, Mesos, Consul, Etcd, 并且还会更多
- 后台监控, 可以监听后台变化进而自动化应用新的配置文件设置
- 配置文件热更新。无需重启进程
- 正常结束http连接
- 后端断路器
- 轮询，rebalancer 负载均衡
- Rest Metrics
- 支持最小化官方docker 镜像
- 前、后台支持SSL
- 清爽的AngularJS前端页面
- 支持Websocket
- 支持HTTP/2
- 网络错误重试
- 支持Let’s Encrypt (自动更新HTTPS证书)
- 高可用集群模式
1.3 traefik与nginx ingress对比¶
1.4 traefik核心概念及能力¶
Traefik是一个边缘路由器，它会拦截外部的请求并根据逻辑规则选择不同的操作方式，这些规则决定着这些请求到底该如何处理。Traefik提供自动发现能力，会实时检测服务，并自动更新路由规则。
从上图可知，请求首先会连接到entrypoints，然后分析这些请求是否与定义的rules匹配，如果匹配，则会通过一系列middlewares，再到对应的services上。
这就涉及到以下几个重要的核心组件。
- Providers
- Entrypoints
- Routers
- Services
- Middlewares
下面分别介绍一下：
- Providers
Providers是基础组件，Traefik的配置发现是通过它来实现的，它可以是协调器，容器引擎，云提供商或者键值存储。
Traefik通过查询Providers的API来查询路由的相关信息，一旦检测到变化，就会动态的更新路由。
- Entrypoints
Entrypoints是Traefik的网络入口，它定义接收请求的接口，以及是否监听TCP或者UDP。
- Routers
Routers主要用于分析请求，并负责将这些请求连接到对应的服务上去，在这个过程中，Routers还可以使用Middlewares来更新请求，比如在把请求发到服务之前添加一些Headers。
- Services
Services负责配置如何到达最终将处理传入请求的实际服务。
- Middlewares
Middlewares用来修改请求或者根据请求来做出一些判断（authentication, rate limiting, headers, …），中间件被附件到路由上，是一种在请求发送到你的服务之前（或者在服务的响应发送到客户端之前）调整请求的一种方法。
二、traefik部署¶
2.1 获取traefik部署前置资源清单文件¶
2.1.1 创建CRD资源¶
本次Traefik 是部署在 kube-system Namespace 下，如果不想部署到配置的 Namespace，需要修改下面部署文件中的 Namespace 参数。
此yaml资源清单文件可在traefik.io网站直接复制使用：https://doc.traefik.io/traefik/v2.5/reference/dynamic-configuration/kubernetes-crd/#definitions
# vim traefik-crd.yaml
---
apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
annotations:
controller-gen.kubebuilder.io/version: v0.6.2
creationTimestamp: null
name: ingressroutes.traefik.containo.us
spec:
group: traefik.containo.us
names:
kind: IngressRoute
listKind: IngressRouteList
plural: ingressroutes
singular: ingressroute
scope: Namespaced
versions:
- name: v1alpha1
schema:
openAPIV3Schema:
description: IngressRoute is an Ingress CRD specification.
properties:
apiVersion:
description: 'APIVersion defines the versioned schema of this representation
of an object. Servers should convert recognized schemas to the latest
internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
type: string
kind:
description: 'Kind is a string value representing the REST resource this
object represents. Servers may infer this from the endpoint the client
submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
type: string
metadata:
type: object
spec:
description: IngressRouteSpec is a specification for a IngressRouteSpec
resource.
properties:
entryPoints:
items:
type: string
type: array
routes:
items:
description: Route contains the set of routes.
properties:
kind:
enum:
- Rule
type: string
match:
type: string
middlewares:
items:
description: MiddlewareRef is a ref to the Middleware resources.
properties:
name:
type: string
namespace:
type: string
required:
- name
type: object
type: array
priority:
type: integer
services:
items:
description: Service defines an upstream to proxy traffic.
properties:
kind:
enum:
- Service
- TraefikService
type: string
name:
description: Name is a reference to a Kubernetes Service
object (for a load-balancer of servers), or to a TraefikService
object (service load-balancer, mirroring, etc). The
differentiation between the two is specified in the
Kind field.
type: string
namespace:
type: string
passHostHeader:
type: boolean
port:
anyOf:
- type: integer
- type: string
x-kubernetes-int-or-string: true
responseForwarding:
description: ResponseForwarding holds configuration for
the forward of the response.
properties:
flushInterval:
type: string
type: object
scheme:
type: string
serversTransport:
type: string
sticky:
description: Sticky holds the sticky configuration.
properties:
cookie:
description: Cookie holds the sticky configuration
based on cookie.
properties:
httpOnly:
type: boolean
name:
type: string
sameSite:
type: string
secure:
type: boolean
type: object
type: object
strategy:
type: string
weight:
description: Weight should only be specified when Name
references a TraefikService object (and to be precise,
one that embeds a Weighted Round Robin).
type: integer
required:
- name
type: object
type: array
required:
- kind
- match
type: object
type: array
tls:
description: "TLS contains the TLS certificates configuration of the
routes. To enable Let's Encrypt, use an empty TLS struct, e.g. in
YAML: \n \t tls: {} # inline format \n \t tls: \t secretName:
# block format"
properties:
certResolver:
type: string
domains:
items:
description: Domain holds a domain name with SANs.
properties:
main:
type: string
sans:
items:
type: string
type: array
type: object
type: array
options:
description: Options is a reference to a TLSOption, that specifies
the parameters of the TLS connection.
properties:
name:
type: string
namespace:
type: string
required:
- name
type: object
secretName:
description: SecretName is the name of the referenced Kubernetes
Secret to specify the certificate details.
type: string
store:
description: Store is a reference to a TLSStore, that specifies
the parameters of the TLS store.
properties:
name:
type: string
namespace:
type: string
required:
- name
type: object
type: object
required:
- routes
type: object
required:
- metadata
- spec
type: object
served: true
storage: true
status:
acceptedNames:
kind: ""
plural: ""
conditions: []
storedVersions: []
---
apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
annotations:
controller-gen.kubebuilder.io/version: v0.6.2
creationTimestamp: null
name: ingressroutetcps.traefik.containo.us
spec:
group: traefik.containo.us
names:
kind: IngressRouteTCP
listKind: IngressRouteTCPList
plural: ingressroutetcps
singular: ingressroutetcp
scope: Namespaced
versions:
- name: v1alpha1
schema:
openAPIV3Schema:
description: IngressRouteTCP is an Ingress CRD specification.
properties:
apiVersion:
description: 'APIVersion defines the versioned schema of this representation
of an object. Servers should convert recognized schemas to the latest
internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
type: string
kind:
description: 'Kind is a string value representing the REST resource this
object represents. Servers may infer this from the endpoint the client
submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
type: string
metadata:
type: object
spec:
description: IngressRouteTCPSpec is a specification for a IngressRouteTCPSpec
resource.
properties:
entryPoints:
items:
type: string
type: array
routes:
items:
description: RouteTCP contains the set of routes.
properties:
match:
type: string
middlewares:
description: Middlewares contains references to MiddlewareTCP
resources.
items:
description: ObjectReference is a generic reference to a Traefik
resource.
properties:
name:
type: string
namespace:
type: string
required:
- name
type: object
type: array
services:
items:
description: ServiceTCP defines an upstream to proxy traffic.
properties:
name:
type: string
namespace:
type: string
port:
anyOf:
- type: integer
- type: string
x-kubernetes-int-or-string: true
proxyProtocol:
description: ProxyProtocol holds the ProxyProtocol configuration.
properties:
version:
type: integer
type: object
terminationDelay:
type: integer
weight:
type: integer
required:
- name
- port
type: object
type: array
required:
- match
type: object
type: array
tls:
description: "TLSTCP contains the TLS certificates configuration of
the routes. To enable Let's Encrypt, use an empty TLS struct, e.g.
in YAML: \n \t tls: {} # inline format \n \t tls: \t secretName:
# block format"
properties:
certResolver:
type: string
domains:
items:
description: Domain holds a domain name with SANs.
properties:
main:
type: string
sans:
items:
type: string
type: array
type: object
type: array
options:
description: Options is a reference to a TLSOption, that specifies
the parameters of the TLS connection.
properties:
name:
type: string
namespace:
type: string
required:
- name
type: object
passthrough:
type: boolean
secretName:
description: SecretName is the name of the referenced Kubernetes
Secret to specify the certificate details.
type: string
store:
description: Store is a reference to a TLSStore, that specifies
the parameters of the TLS store.
properties:
name:
type: string
namespace:
type: string
required:
- name
type: object
type: object
required:
- routes
type: object
required:
- metadata
- spec
type: object
served: true
storage: true
status:
acceptedNames:
kind: ""
plural: ""
conditions: []
storedVersions: []
---
apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
annotations:
controller-gen.kubebuilder.io/version: v0.6.2
creationTimestamp: null
name: ingressrouteudps.traefik.containo.us
spec:
group: traefik.containo.us
names:
kind: IngressRouteUDP
listKind: IngressRouteUDPList
plural: ingressrouteudps
singular: ingressrouteudp
scope: Namespaced
versions:
- name: v1alpha1
schema:
openAPIV3Schema:
description: IngressRouteUDP is an Ingress CRD specification.
properties:
apiVersion:
description: 'APIVersion defines the versioned schema of this representation
of an object. Servers should convert recognized schemas to the latest
internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
type: string
kind:
description: 'Kind is a string value representing the REST resource this
object represents. Servers may infer this from the endpoint the client
submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
type: string
metadata:
type: object
spec:
description: IngressRouteUDPSpec is a specification for a IngressRouteUDPSpec
resource.
properties:
entryPoints:
items:
type: string
type: array
routes:
items:
description: RouteUDP contains the set of routes.
properties:
services:
items:
description: ServiceUDP defines an upstream to proxy traffic.
properties:
name:
type: string
namespace:
type: string
port:
anyOf:
- type: integer
- type: string
x-kubernetes-int-or-string: true
weight:
type: integer
required:
- name
- port
type: object
type: array
type: object
type: array
required:
- routes
type: object
required:
- metadata
- spec
type: object
served: true
storage: true
status:
acceptedNames:
kind: ""
plural: ""
conditions: []
storedVersions: []
---
apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
annotations:
controller-gen.kubebuilder.io/version: v0.6.2
creationTimestamp: null
name: middlewares.traefik.containo.us
spec:
group: traefik.containo.us
names:
kind: Middleware
listKind: MiddlewareList
plural: middlewares
singular: middleware
scope: Namespaced
versions:
- name: v1alpha1
schema:
openAPIV3Schema:
description: Middleware is a specification for a Middleware resource.
properties:
apiVersion:
description: 'APIVersion defines the versioned schema of this representation
of an object. Servers should convert recognized schemas to the latest
internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
type: string
kind:
description: 'Kind is a string value representing the REST resource this
object represents. Servers may infer this from the endpoint the client
submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
type: string
metadata:
type: object
spec:
description: MiddlewareSpec holds the Middleware configuration.
properties:
addPrefix:
description: AddPrefix holds the AddPrefix configuration.
properties:
prefix:
type: string
type: object
basicAuth:
description: BasicAuth holds the HTTP basic authentication configuration.
properties:
headerField:
type: string
realm:
type: string
removeHeader:
type: boolean
secret:
type: string
type: object
buffering:
description: Buffering holds the request/response buffering configuration.
properties:
maxRequestBodyBytes:
format: int64
type: integer
maxResponseBodyBytes:
format: int64
type: integer
memRequestBodyBytes:
format: int64
type: integer
memResponseBodyBytes:
format: int64
type: integer
retryExpression:
type: string
type: object
chain:
description: Chain holds a chain of middlewares.
properties:
middlewares:
items:
description: MiddlewareRef is a ref to the Middleware resources.
properties:
name:
type: string
namespace:
type: string
required:
- name
type: object
type: array
type: object
circuitBreaker:
description: CircuitBreaker holds the circuit breaker configuration.
properties:
expression:
type: string
type: object
compress:
description: Compress holds the compress configuration.
properties:
excludedContentTypes:
items:
type: string
type: array
type: object
contentType:
description: ContentType middleware - or rather its unique `autoDetect`
option - specifies whether to let the `Content-Type` header, if
it has not been set by the backend, be automatically set to a value
derived from the contents of the response. As a proxy, the default
behavior should be to leave the header alone, regardless of what
the backend did with it. However, the historic default was to always
auto-detect and set the header if it was nil, and it is going to
be kept that way in order to support users currently relying on
it. This middleware exists to enable the correct behavior until
at least the default one can be changed in a future version.
properties:
autoDetect:
type: boolean
type: object
digestAuth:
description: DigestAuth holds the Digest HTTP authentication configuration.
properties:
headerField:
type: string
realm:
type: string
removeHeader:
type: boolean
secret:
type: string
type: object
errors:
description: ErrorPage holds the custom error page configuration.
properties:
query:
type: string
service:
description: Service defines an upstream to proxy traffic.
properties:
kind:
enum:
- Service
- TraefikService
type: string
name:
description: Name is a reference to a Kubernetes Service object
(for a load-balancer of servers), or to a TraefikService
object (service load-balancer, mirroring, etc). The differentiation
between the two is specified in the Kind field.
type: string
namespace:
type: string
passHostHeader:
type: boolean
port:
anyOf:
- type: integer
- type: string
x-kubernetes-int-or-string: true
responseForwarding:
description: ResponseForwarding holds configuration for the
forward of the response.
properties:
flushInterval:
type: string
type: object
scheme:
type: string
serversTransport:
type: string
sticky:
description: Sticky holds the sticky configuration.
properties:
cookie:
description: Cookie holds the sticky configuration based
on cookie.
properties:
httpOnly:
type: boolean
name:
type: string
sameSite:
type: string
secure:
type: boolean
type: object
type: object
strategy:
type: string
weight:
description: Weight should only be specified when Name references
a TraefikService object (and to be precise, one that embeds
a Weighted Round Robin).
type: integer
required:
- name
type: object
status:
items:
type: string
type: array
type: object
forwardAuth:
description: ForwardAuth holds the http forward authentication configuration.
properties:
address:
type: string
authRequestHeaders:
items:
type: string
type: array
authResponseHeaders:
items:
type: string
type: array
authResponseHeadersRegex:
type: string
tls:
description: ClientTLS holds TLS specific configurations as client.
properties:
caOptional:
type: boolean
caSecret:
type: string
certSecret:
type: string
insecureSkipVerify:
type: boolean
type: object
trustForwardHeader:
type: boolean
type: object
headers:
description: Headers holds the custom header configuration.
properties:
accessControlAllowCredentials:
description: AccessControlAllowCredentials is only valid if true.
false is ignored.
type: boolean
accessControlAllowHeaders:
description: AccessControlAllowHeaders must be used in response
to a preflight request with Access-Control-Request-Headers set.
items:
type: string
type: array
accessControlAllowMethods:
description: AccessControlAllowMethods must be used in response
to a preflight request with Access-Control-Request-Method set.
items:
type: string
type: array
accessControlAllowOriginList:
description: AccessControlAllowOriginList is a list of allowable
origins. Can also be a wildcard origin "*".
items:
type: string
type: array
accessControlAllowOriginListRegex:
description: AccessControlAllowOriginListRegex is a list of allowable
origins written following the Regular Expression syntax (https://golang.org/pkg/regexp/).
items:
type: string
type: array
accessControlExposeHeaders:
description: AccessControlExposeHeaders sets valid headers for
the response.
items:
type: string
type: array
accessControlMaxAge:
description: AccessControlMaxAge sets the time that a preflight
request may be cached.
format: int64
type: integer
addVaryHeader:
description: AddVaryHeader controls if the Vary header is automatically
added/updated when the AccessControlAllowOriginList is set.
type: boolean
allowedHosts:
items:
type: string
type: array
browserXssFilter:
type: boolean
contentSecurityPolicy:
type: string
contentTypeNosniff:
type: boolean
customBrowserXSSValue:
type: string
customFrameOptionsValue:
type: string
customRequestHeaders:
additionalProperties:
type: string
type: object
customResponseHeaders:
additionalProperties:
type: string
type: object
featurePolicy:
description: 'Deprecated: use PermissionsPolicy instead.'
type: string
forceSTSHeader:
type: boolean
frameDeny:
type: boolean
hostsProxyHeaders:
items:
type: string
type: array
isDevelopment:
type: boolean
permissionsPolicy:
type: string
publicKey:
type: string
referrerPolicy:
type: string
sslForceHost:
description: 'Deprecated: use RedirectRegex instead.'
type: boolean
sslHost:
description: 'Deprecated: use RedirectRegex instead.'
type: string
sslProxyHeaders:
additionalProperties:
type: string
type: object
sslRedirect:
description: 'Deprecated: use EntryPoint redirection or RedirectScheme
instead.'
type: boolean
sslTemporaryRedirect:
description: 'Deprecated: use EntryPoint redirection or RedirectScheme
instead.'
type: boolean
stsIncludeSubdomains:
type: boolean
stsPreload:
type: boolean
stsSeconds:
format: int64
type: integer
type: object
inFlightReq:
description: InFlightReq limits the number of requests being processed
and served concurrently.
properties:
amount:
format: int64
type: integer
sourceCriterion:
description: SourceCriterion defines what criterion is used to
group requests as originating from a common source. If none
are set, the default is to use the request's remote address
field. All fields are mutually exclusive.
properties:
ipStrategy:
description: IPStrategy holds the ip strategy configuration.
properties:
depth:
type: integer
excludedIPs:
items:
type: string
type: array
type: object
requestHeaderName:
type: string
requestHost:
type: boolean
type: object
type: object
ipWhiteList:
description: IPWhiteList holds the ip white list configuration.
properties:
ipStrategy:
description: IPStrategy holds the ip strategy configuration.
properties:
depth:
type: integer
excludedIPs:
items:
type: string
type: array
type: object
sourceRange:
items:
type: string
type: array
type: object
passTLSClientCert:
description: PassTLSClientCert holds the TLS client cert headers configuration.
properties:
info:
description: TLSClientCertificateInfo holds the client TLS certificate
info configuration.
properties:
issuer:
description: TLSClientCertificateDNInfo holds the client TLS
certificate distinguished name info configuration. cf https://tools.ietf.org/html/rfc3739
properties:
commonName:
type: boolean
country:
type: boolean
domainComponent:
type: boolean
locality:
type: boolean
organization:
type: boolean
province:
type: boolean
serialNumber:
type: boolean
type: object
notAfter:
type: boolean
notBefore:
type: boolean
sans:
type: boolean
serialNumber:
type: boolean
subject:
description: TLSClientCertificateDNInfo holds the client TLS
certificate distinguished name info configuration. cf https://tools.ietf.org/html/rfc3739
properties:
commonName:
type: boolean
country:
type: boolean
domainComponent:
type: boolean
locality:
type: boolean
organization:
type: boolean
province:
type: boolean
serialNumber:
type: boolean
type: object
type: object
pem:
type: boolean
type: object
plugin:
additionalProperties:
x-kubernetes-preserve-unknown-fields: true
type: object
rateLimit:
description: RateLimit holds the rate limiting configuration for a
given router.
properties:
average:
format: int64
type: integer
burst:
format: int64
type: integer
period:
anyOf:
- type: integer
- type: string
x-kubernetes-int-or-string: true
sourceCriterion:
description: SourceCriterion defines what criterion is used to
group requests as originating from a common source. If none
are set, the default is to use the request's remote address
field. All fields are mutually exclusive.
properties:
ipStrategy:
description: IPStrategy holds the ip strategy configuration.
properties:
depth:
type: integer
excludedIPs:
items:
type: string
type: array
type: object
requestHeaderName:
type: string
requestHost:
type: boolean
type: object
type: object
redirectRegex:
description: RedirectRegex holds the redirection configuration.
properties:
permanent:
type: boolean
regex:
type: string
replacement:
type: string
type: object
redirectScheme:
description: RedirectScheme holds the scheme redirection configuration.
properties:
permanent:
type: boolean
port:
type: string
scheme:
type: string
type: object
replacePath:
description: ReplacePath holds the ReplacePath configuration.
properties:
path:
type: string
type: object
replacePathRegex:
description: ReplacePathRegex holds the ReplacePathRegex configuration.
properties:
regex:
type: string
replacement:
type: string
type: object
retry:
description: Retry holds the retry configuration.
properties:
attempts:
type: integer
initialInterval:
anyOf:
- type: integer
- type: string
x-kubernetes-int-or-string: true
type: object
stripPrefix:
description: StripPrefix holds the StripPrefix configuration.
properties:
forceSlash:
type: boolean
prefixes:
items:
type: string
type: array
type: object
stripPrefixRegex:
description: StripPrefixRegex holds the StripPrefixRegex configuration.
properties:
regex:
items:
type: string
type: array
type: object
type: object
required:
- metadata
- spec
type: object
served: true
storage: true
status:
acceptedNames:
kind: ""
plural: ""
conditions: []
storedVersions: []
---
apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
annotations:
controller-gen.kubebuilder.io/version: v0.6.2
creationTimestamp: null
name: middlewaretcps.traefik.containo.us
spec:
group: traefik.containo.us
names:
kind: MiddlewareTCP
listKind: MiddlewareTCPList
plural: middlewaretcps
singular: middlewaretcp
scope: Namespaced
versions:
- name: v1alpha1
schema:
openAPIV3Schema:
description: MiddlewareTCP is a specification for a MiddlewareTCP resource.
properties:
apiVersion:
description: 'APIVersion defines the versioned schema of this representation
of an object. Servers should convert recognized schemas to the latest
internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
type: string
kind:
description: 'Kind is a string value representing the REST resource this
object represents. Servers may infer this from the endpoint the client
submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
type: string
metadata:
type: object
spec:
description: MiddlewareTCPSpec holds the MiddlewareTCP configuration.
properties:
ipWhiteList:
description: TCPIPWhiteList holds the TCP ip white list configuration.
properties:
sourceRange:
items:
type: string
type: array
type: object
type: object
required:
- metadata
- spec
type: object
served: true
storage: true
status:
acceptedNames:
kind: ""
plural: ""
conditions: []
storedVersions: []
---
apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
annotations:
controller-gen.kubebuilder.io/version: v0.6.2
creationTimestamp: null
name: serverstransports.traefik.containo.us
spec:
group: traefik.containo.us
names:
kind: ServersTransport
listKind: ServersTransportList
plural: serverstransports
singular: serverstransport
scope: Namespaced
versions:
- name: v1alpha1
schema:
openAPIV3Schema:
description: ServersTransport is a specification for a ServersTransport resource.
properties:
apiVersion:
description: 'APIVersion defines the versioned schema of this representation
of an object. Servers should convert recognized schemas to the latest
internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
type: string
kind:
description: 'Kind is a string value representing the REST resource this
object represents. Servers may infer this from the endpoint the client
submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
type: string
metadata:
type: object
spec:
description: ServersTransportSpec options to configure communication between
Traefik and the servers.
properties:
certificatesSecrets:
description: Certificates for mTLS.
items:
type: string
type: array
disableHTTP2:
description: Disable HTTP/2 for connections with backend servers.
type: boolean
forwardingTimeouts:
description: Timeouts for requests forwarded to the backend servers.
properties:
dialTimeout:
anyOf:
- type: integer
- type: string
description: The amount of time to wait until a connection to
a backend server can be established. If zero, no timeout exists.
x-kubernetes-int-or-string: true
idleConnTimeout:
anyOf:
- type: integer
- type: string
description: The maximum period for which an idle HTTP keep-alive
connection will remain open before closing itself.
x-kubernetes-int-or-string: true
responseHeaderTimeout:
anyOf:
- type: integer
- type: string
description: The amount of time to wait for a server's response
headers after fully writing the request (including its body,
if any). If zero, no timeout exists.
x-kubernetes-int-or-string: true
type: object
insecureSkipVerify:
description: Disable SSL certificate verification.
type: boolean
maxIdleConnsPerHost:
description: If non-zero, controls the maximum idle (keep-alive) to
keep per-host. If zero, DefaultMaxIdleConnsPerHost is used.
type: integer
peerCertURI:
description: URI used to match against SAN URI during the peer certificate
verification.
type: string
rootCAsSecrets:
description: Add cert file for self-signed certificate.
items:
type: string
type: array
serverName:
description: ServerName used to contact the server.
type: string
type: object
required:
- metadata
- spec
type: object
served: true
storage: true
status:
acceptedNames:
kind: ""
plural: ""
conditions: []
storedVersions: []
---
apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
annotations:
controller-gen.kubebuilder.io/version: v0.6.2
creationTimestamp: null
name: tlsoptions.traefik.containo.us
spec:
group: traefik.containo.us
names:
kind: TLSOption
listKind: TLSOptionList
plural: tlsoptions
singular: tlsoption
scope: Namespaced
versions:
- name: v1alpha1
schema:
openAPIV3Schema:
description: TLSOption is a specification for a TLSOption resource.
properties:
apiVersion:
description: 'APIVersion defines the versioned schema of this representation
of an object. Servers should convert recognized schemas to the latest
internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
type: string
kind:
description: 'Kind is a string value representing the REST resource this
object represents. Servers may infer this from the endpoint the client
submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
type: string
metadata:
type: object
spec:
description: TLSOptionSpec configures TLS for an entry point.
properties:
alpnProtocols:
items:
type: string
type: array
cipherSuites:
items:
type: string
type: array
clientAuth:
description: ClientAuth defines the parameters of the client authentication
part of the TLS connection, if any.
properties:
clientAuthType:
description: ClientAuthType defines the client authentication
type to apply.
enum:
- NoClientCert
- RequestClientCert
- RequireAnyClientCert
- VerifyClientCertIfGiven
- RequireAndVerifyClientCert
type: string
secretNames:
description: SecretName is the name of the referenced Kubernetes
Secret to specify the certificate details.
items:
type: string
type: array
type: object
curvePreferences:
items:
type: string
type: array
maxVersion:
type: string
minVersion:
type: string
preferServerCipherSuites:
type: boolean
sniStrict:
type: boolean
type: object
required:
- metadata
- spec
type: object
served: true
storage: true
status:
acceptedNames:
kind: ""
plural: ""
conditions: []
storedVersions: []
---
apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
annotations:
controller-gen.kubebuilder.io/version: v0.6.2
creationTimestamp: null
name: tlsstores.traefik.containo.us
spec:
group: traefik.containo.us
names:
kind: TLSStore
listKind: TLSStoreList
plural: tlsstores
singular: tlsstore
scope: Namespaced
versions:
- name: v1alpha1
schema:
openAPIV3Schema:
description: TLSStore is a specification for a TLSStore resource.
properties:
apiVersion:
description: 'APIVersion defines the versioned schema of this representation
of an object. Servers should convert recognized schemas to the latest
internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
type: string
kind:
description: 'Kind is a string value representing the REST resource this
object represents. Servers may infer this from the endpoint the client
submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
type: string
metadata:
type: object
spec:
description: TLSStoreSpec configures a TLSStore resource.
properties:
defaultCertificate:
description: DefaultCertificate holds a secret name for the TLSOption
resource.
properties:
secretName:
description: SecretName is the name of the referenced Kubernetes
Secret to specify the certificate details.
type: string
required:
- secretName
type: object
required:
- defaultCertificate
type: object
required:
- metadata
- spec
type: object
served: true
storage: true
status:
acceptedNames:
kind: ""
plural: ""
conditions: []
storedVersions: []
---
apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
annotations:
controller-gen.kubebuilder.io/version: v0.6.2
creationTimestamp: null
name: traefikservices.traefik.containo.us
spec:
group: traefik.containo.us
names:
kind: TraefikService
listKind: TraefikServiceList
plural: traefikservices
singular: traefikservice
scope: Namespaced
versions:
- name: v1alpha1
schema:
openAPIV3Schema:
description: TraefikService is the specification for a service (that an IngressRoute
refers to) that is usually not a terminal service (i.e. not a pod of servers),
as opposed to a Kubernetes Service. That is to say, it usually refers to
other (children) services, which themselves can be TraefikServices or Services.
properties:
apiVersion:
description: 'APIVersion defines the versioned schema of this representation
of an object. Servers should convert recognized schemas to the latest
internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
type: string
kind:
description: 'Kind is a string value representing the REST resource this
object represents. Servers may infer this from the endpoint the client
submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
type: string
metadata:
type: object
spec:
description: ServiceSpec defines whether a TraefikService is a load-balancer
of services or a mirroring service.
properties:
mirroring:
description: Mirroring defines a mirroring service, which is composed
of a main load-balancer, and a list of mirrors.
properties:
kind:
enum:
- Service
- TraefikService
type: string
maxBodySize:
format: int64
type: integer
mirrors:
items:
description: MirrorService defines one of the mirrors of a Mirroring
service.
properties:
kind:
enum:
- Service
- TraefikService
type: string
name:
description: Name is a reference to a Kubernetes Service
object (for a load-balancer of servers), or to a TraefikService
object (service load-balancer, mirroring, etc). The differentiation
between the two is specified in the Kind field.
type: string
namespace:
type: string
passHostHeader:
type: boolean
percent:
type: integer
port:
anyOf:
- type: integer
- type: string
x-kubernetes-int-or-string: true
responseForwarding:
description: ResponseForwarding holds configuration for
the forward of the response.
properties:
flushInterval:
type: string
type: object
scheme:
type: string
serversTransport:
type: string
sticky:
description: Sticky holds the sticky configuration.
properties:
cookie:
description: Cookie holds the sticky configuration based
on cookie.
properties:
httpOnly:
type: boolean
name:
type: string
sameSite:
type: string
secure:
type: boolean
type: object
type: object
strategy:
type: string
weight:
description: Weight should only be specified when Name references
a TraefikService object (and to be precise, one that embeds
a Weighted Round Robin).
type: integer
required:
- name
type: object
type: array
name:
description: Name is a reference to a Kubernetes Service object
(for a load-balancer of servers), or to a TraefikService object
(service load-balancer, mirroring, etc). The differentiation
between the two is specified in the Kind field.
type: string
namespace:
type: string
passHostHeader:
type: boolean
port:
anyOf:
- type: integer
- type: string
x-kubernetes-int-or-string: true
responseForwarding:
description: ResponseForwarding holds configuration for the forward
of the response.
properties:
flushInterval:
type: string
type: object
scheme:
type: string
serversTransport:
type: string
sticky:
description: Sticky holds the sticky configuration.
properties:
cookie:
description: Cookie holds the sticky configuration based on
cookie.
properties:
httpOnly:
type: boolean
name:
type: string
sameSite:
type: string
secure:
type: boolean
type: object
type: object
strategy:
type: string
weight:
description: Weight should only be specified when Name references
a TraefikService object (and to be precise, one that embeds
a Weighted Round Robin).
type: integer
required:
- name
type: object
weighted:
description: WeightedRoundRobin defines a load-balancer of services.
properties:
services:
items:
description: Service defines an upstream to proxy traffic.
properties:
kind:
enum:
- Service
- TraefikService
type: string
name:
description: Name is a reference to a Kubernetes Service
object (for a load-balancer of servers), or to a TraefikService
object (service load-balancer, mirroring, etc). The differentiation
between the two is specified in the Kind field.
type: string
namespace:
type: string
passHostHeader:
type: boolean
port:
anyOf:
- type: integer
- type: string
x-kubernetes-int-or-string: true
responseForwarding:
description: ResponseForwarding holds configuration for
the forward of the response.
properties:
flushInterval:
type: string
type: object
scheme:
type: string
serversTransport:
type: string
sticky:
description: Sticky holds the sticky configuration.
properties:
cookie:
description: Cookie holds the sticky configuration based
on cookie.
properties:
httpOnly:
type: boolean
name:
type: string
sameSite:
type: string
secure:
type: boolean
type: object
type: object
strategy:
type: string
weight:
description: Weight should only be specified when Name references
a TraefikService object (and to be precise, one that embeds
a Weighted Round Robin).
type: integer
required:
- name
type: object
type: array
sticky:
description: Sticky holds the sticky configuration.
properties:
cookie:
description: Cookie holds the sticky configuration based on
cookie.
properties:
httpOnly:
type: boolean
name:
type: string
sameSite:
type: string
secure:
type: boolean
type: object
type: object
type: object
type: object
required:
- metadata
- spec
type: object
served: true
storage: true
status:
acceptedNames:
kind: ""
plural: ""
conditions: []
storedVersions: []
2.1.2 创建RBAC权限¶
基于角色的访问控制（RBAC）策略，方便对Kubernetes资源和API进行细粒度控制
traefik需要一定的权限，需要提前创建ServiceAccount并分配一定的权限。
# vim traefik-rbac.yaml
apiVersion: v1
kind: ServiceAccount
metadata:
namespace: kube-system
name: traefik-ingress-controller
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: traefik-ingress-controller
rules:
- apiGroups:
- ""
resources:
- services
- endpoints
- secrets
verbs:
- get
- list
- watch
- apiGroups:
- extensions
- networking.k8s.io
resources:
- ingresses
- ingressclasses
verbs:
- get
- list
- watch
- apiGroups:
- extensions
resources:
- ingresses/status
verbs:
- update
- apiGroups:
- traefik.containo.us
resources:
- middlewares
- middlewaretcps
- ingressroutes
- traefikservices
- ingressroutetcps
- ingressrouteudps
- tlsoptions
- tlsstores
- serverstransports
verbs:
- get
- list
- watch
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: traefik-ingress-controller
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: traefik-ingress-controller
subjects:
- kind: ServiceAccount
name: traefik-ingress-controller
namespace: kube-system
2.1.3 创建traefik配置文件¶
由traefik配置很多，通过CLI定义不方便，一般都通过配置文件对traefik进行参数配置，例如使用ConfigMap将配置挂载至traefik中
# vim traefik-configmap.yaml
apiVersion: v1
kind: ConfigMap
metadata:
name: traefik
namespace: kube-system
data:
traefik.yaml: |-
serversTransport:
insecureSkipVerify: true ## 略验证代理服务的 TLS 证书
api:
insecure: true ## 允许 HTTP 方式访问 API
dashboard: true ## 启用 Dashboard
debug: true ## 启用 Debug 调试模式
metrics:
prometheus: "" ## 配置 Prometheus 监控指标数据，并使用默认配置
entryPoints:
web:
address: ":80" ## 配置 80 端口，并设置入口名称为 web
websecure:
address: ":443" ## 配置 443 端口，并设置入口名称为 websecure
metrics:
address: ":8082" ## 配置 8082端口，并设置入口名称为 metrics
tcpep:
address: ":8083" ## 配置 8083端口，并设置入口名称为 tcpep,做为tcp入口
udpep:
address: ":8084/udp" ## 配置 8084端口，并设置入口名称为 udpep，做为udp入口
providers:
kubernetesCRD: "" ## 启用 Kubernetes CRD 方式来配置路由规则
kubernetesingress: "" ## 启用 Kubernetes Ingress 方式来配置路由规则
kubernetesGateway: "" ## 启用 Kubernetes Gateway API
experimental:
kubernetesGateway: true ## 允许使用 Kubernetes Gateway API
log:
filePath: "" ## 设置调试日志文件存储路径，如果为空则输出到控制台
level: error ## 设置调试日志级别
format: json ## 设置调试日志格式
accessLog:
filePath: "" ## 设置访问日志文件存储路径，如果为空则输出到控制台
format: json ## 设置访问调试日志格式
bufferingSize: 0 ## 设置访问日志缓存行数
filters:
retryAttempts: true ## 设置代理访问重试失败时，保留访问日志
minDuration: 20 ## 设置保留请求时间超过指定持续时间的访问日志
fields: ## 设置访问日志中的字段是否保留（keep 保留、drop 不保留）
defaultMode: keep ## 设置默认保留访问日志字段
names:
ClientUsername: drop
headers:
defaultMode: keep ## 设置 Header 中字段是否保留,设置默认保留 Header 中字段
names: ## 针对 Header 中特别字段特别配置保留模式
User-Agent: redact
Authorization: drop
Content-Type: keep
2.1.4 应用上述资源清单文件¶
# kubectl apply -f traefik-crd.yaml
# kubectl apply -f traefik-rbac.yaml
# kubectl apply -f traefik-config.yaml
2.1.5 设置节点Label¶
由于使用DaemonSet方式部署Traefik，所以需要为节点设置label，当应用部署时会根据节点Label进行选择。
设置节点标签
# kubectl label nodes --all IngressProxy=true
查看节点标签
# kubectl get nodes --show-labels
如需要取消时，可执行下述命令
# kubectl label nodes --all IngressProxy-
2.2 部署Traefik¶
2.2.1 deploy资源清单文件准备¶
本次将用Daemonset方式部署traefik,便于后期扩展
本次部署通过hostport方式把Pod中容器内的80、443映射到物理机，方便集群外访问。
# vim traefik-deploy.yaml
apiVersion: apps/v1
kind: DaemonSet
metadata:
namespace: kube-system
name: traefik
labels:
app: traefik
spec:
selector:
matchLabels:
app: traefik
template:
metadata:
labels:
app: traefik
spec:
serviceAccountName: traefik-ingress-controller
containers:
- name: traefik
image: traefik:v2.5.7
args:
- --configfile=/config/traefik.yaml
volumeMounts:
- mountPath: /config
name: config
ports:
- name: web
containerPort: 80
hostPort: 80 ## 将容器端口绑定所在服务器的 80 端口
- name: websecure
containerPort: 443
hostPort: 443 ## 将容器端口绑定所在服务器的 443 端口
- name: admin
containerPort: 8080 ## Traefik Dashboard 端口
- name: tcpep
containerPort: 8083
hostPort: 8083 ## 将容器端口绑定所在服务器的 8083 端口
- name: udpep
containerPort: 8084
hostPort: 8084 ## 将容器端口绑定所在服务器的 8084 端口
protocol: UDP
volumes:
- name: config
configMap:
name: traefik
tolerations: ## 设置容忍所有污点，防止节点被设置污点
- operator: "Exists"
nodeSelector: ## 设置node筛选器，在特定label的节点上启动
IngressProxy: "true"
验证端口配置是否正确
# kubectl get daemonset traefik -n kube-system -o yaml
2.2.2 应用deploy资源清单文件¶
# kubectl apply -f traefix-deploy.yaml
2.2.3 service资源清单文件准备¶
# vim traefix-service.yaml
apiVersion: v1
kind: Service
metadata:
name: traefik
namespace: kube-system
spec:
ports:
- protocol: TCP
name: web
port: 80
- protocol: TCP
name: admin
port: 8080
- protocol: TCP
name: websecure
port: 443
- protocol: TCP
name: tcpep
port: 8083
- protocol: UDP
name: udpep
port: 8084
selector:
app: traefik
2.2.4 应用service资源清单文件¶
# kubectl apply -f traefik-service.yaml
2.3 配置访问traefik dashboard路由规则¶
Traefik 应用已经部署完成，但是想让外部访问 Kubernetes 内部服务，还需要配置路由规则，上面部署 Traefik 时开启了 Traefik Dashboard，这是 Traefik 提供的视图看板，所以，首先配置基于 HTTP 的 Traefik Dashboard 路由规则，使外部能够访问 Traefik Dashboard。这里使用 IngressRoute方式进行演示。
2.3.1 Traefik创建路由规则方法¶
- 原生ingress
- CRD IngressRoute
- Gateway API
2.3.2 通过原生ingress方式暴露traefik dashboard¶
# kubectl get svc -n kube-system
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
traefik ClusterIP 10.96.174.88 <none> 80/TCP,8080/TCP,443/TCP,8083/TCP,8084/UDP 6h44m
# kubectl get endpoints -n kube-system
NAME ENDPOINTS AGE
traefik 10.244.135.204:80,10.244.159.141:80,10.244.194.92:80 + 17 more... 6h44m
# cat traefik-dashboard-native-ingress.yaml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: traefik-dashboard-ingress
namespace: kube-system
annotations:
kubernetes.io/ingress.class: traefik
traefik.ingress.kubernetes.io/router.entrypoints: web
spec:
rules:
- host: tfni.kubemsb.com
http:
paths:
- pathType: Prefix
path: /
backend:
service:
name: traefik
port:
number: 8080
# kubectl apply -f traefik-dashboard-native-ingress.yaml
# kubectl get ingress -n kube-system
NAME CLASS HOSTS ADDRESS PORTS AGE
traefik-dashboard-ingress <none> tfni.kubemsb.com 80 56m
# kubectl describe ingress traefik-dashboard-ingress -n kube-system
Name: traefik-dashboard-ingress
Namespace: kube-system
Address:
Default backend: default-http-backend:80 (<error: endpoints "default-http-backend" not found>)
Rules:
Host Path Backends
---- ---- --------
tfni.kubemsb.com
/ traefik:8080 (10.244.135.204:8080,10.244.159.141:8080,10.244.194.92:8080 + 1 more...)
Annotations: kubernetes.io/ingress.class: traefik
traefik.ingress.kubernetes.io/router.entrypoints: web
Events: <none>
在集群之外主机访问
# vim /etc/hosts
......
192.168.10.12 tfni.kubemsb.com
2.3.3 创建dashboard ingress route资源清单文件¶
# vim traefik-dashboard-ingress-route.yaml
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:
name: traefik
namespace: kube-system
spec:
entryPoints:
- web
routes:
- match: Host(`traefik.kubemsb.com`) && PathPrefix(`/`)
kind: Rule
services:
- name: traefik
port: 8080
2.3.4 应用资源清单文件¶
# kubectl apply -f traefik-dashboard-ingress-route.yaml
2.3.5 在集群内或集群外主机配置域名解析¶
把域名解析为kubernetes集群任意节点IP既可。
# vim /etc/hosts
192.168.10.12 traefik.kubemsb.com
2.3.6 通过域名访问traefik dashboard¶
# firefox http://traefik.kubemsb.com &
三、traefik基础应用¶
3.1 配置kubernetes dashboard路由规则¶
必须使用SSL证书创建secret密钥
3.1.1 查看kubernetes dashboard service状态¶
# kubectl get svc -n kubernetes-dashboard
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
dashboard-metrics-scraper ClusterIP 10.96.100.4 <none> 8000/TCP 21d
kubernetes-dashboard NodePort 10.96.19.1 <none> 443:30000/TCP 21d
3.1.2 编写访问kubernetes dashboard 路由规则¶
# ls
6864844_kubemsb.com_nginx.zip kubemsb.key kubemsb.pem
# kubectl create secret tls kubemsb-tls --cert=kubemsb.pem --key=kubemsb.key
secret/kubemsb-tls created
# kubectl get secret
NAME TYPE DATA AGE
default-token-x4qbc kubernetes.io/service-account-token 3 24d
kubemsb-tls kubernetes.io/tls 2 11s
# vim kubernetes-dashboard-ir.yaml
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:
name: kubernetes-dashboard
namespace: kubernetes-dashboard
spec:
entryPoints:
- websecure
routes:
- match: Host(`dashboard.kubemsb.com`)
kind: Rule
services:
- name: kubernetes-dashboard
port: 443
tls:
secretName: kubemsb-tls
3.1.3 应用上述路由规则文件¶
# kubectl apply -f kubernetes-dashboard-ir.yaml
3.1.4 配置域名解析及访问¶
# vim /etc/hosts
192.168.10.12 dashboard.kubemsb.com
# kubectl get secret -n kubernetes-dashboard
NAME TYPE DATA AGE
......
kubernetes-dashboard-token-8tm5n kubernetes.io/service-account-token 3 21d
# kubectl describe secret kubernetes-dashboard-token-8tm5n -n kubernetes-dashboard
Name: kubernetes-dashboard-token-8tm5n
Namespace: kubernetes-dashboard
Labels: <none>
Annotations: kubernetes.io/service-account.name: kubernetes-dashboard
kubernetes.io/service-account.uid: 47d19e14-e3ed-4733-9799-02396dfb436a
Type: kubernetes.io/service-account-token
Data
====
ca.crt: 1359 bytes
namespace: 20 bytes
token: eyJhbGciOiJSUzI1NiIsImtpZCI6ImVGc2xjT05uekl0MlVOZ0VCSlhHSURfOXd6WGFvVnZFZmNwREwtVk1STlEifQ.eyJpc3MiOiJrdWJlcm5ldGVzL3NlcnZpY2VhY2NvdW50Iiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9uYW1lc3BhY2UiOiJrdWJlcm5ldGVzLWRhc2hib2FyZCIsImt1YmVybmV0ZXMuaW8vc2VydmljZWFjY291bnQvc2VjcmV0Lm5hbWUiOiJrdWJlcm5ldGVzLWRhc2hib2FyZC10b2tlbi04dG01biIsImt1YmVybmV0ZXMuaW8vc2VydmljZWFjY291bnQvc2VydmljZS1hY2NvdW50Lm5hbWUiOiJrdWJlcm5ldGVzLWRhc2hib2FyZCIsImt1YmVybmV0ZXMuaW8vc2VydmljZWFjY291bnQvc2VydmljZS1hY2NvdW50LnVpZCI6IjQ3ZDE5ZTE0LWUzZWQtNDczMy05Nzk5LTAyMzk2ZGZiNDM2YSIsInN1YiI6InN5c3RlbTpzZXJ2aWNlYWNjb3VudDprdWJlcm5ldGVzLWRhc2hib2FyZDprdWJlcm5ldGVzLWRhc2hib2FyZCJ9.rY5YFoJD3ZUg1e31uxrPhKl0a_mslAmVG5r1ZDia0z7WKKtGjs8XwSu5cIDqKa-1FUV6ixxgfDwm5Rd-nK8TCMVRrDY_7r2I5-u2ebh4rEAXCBvQw0gtJu16-e1Z_TQqNuc73E9fDS0AqHr2qGOWAQcz_FjWGAGjZKzYlKPcA3mFI2wsAIRgxh29-S2f4SxB2zgWyYQdbW-fiHDHWK6dQ-a3glIlCk4jnPtzrX1HNK3BSRPoKaZg_9Ot0dABex2ro5QkQ0wyuT3NLio-n8v21MbhKG5ehBEFRNrTcTPM2CLt4XaUJezphKHShdc3VbUlizPge5DleAJcp9JrFzyqBQ
3.2 配置HTTP路由规则¶
3.2.1 创建应用及服务资源清单文件并应用¶
# vim whoami.yaml
kind: Deployment
apiVersion: apps/v1
metadata:
name: whoami
namespace: default
labels:
app: traefiklabs
name: whoami
spec:
replicas: 2
selector:
matchLabels:
app: traefiklabs
task: whoami
template:
metadata:
labels:
app: traefiklabs
task: whoami
spec:
containers:
- name: whoami
image: traefik/whoami
ports:
- containerPort: 80
---
apiVersion: v1
kind: Service
metadata:
name: whoami
namespace: default
spec:
ports:
- name: http
port: 80
selector:
app: traefiklabs
task: whoami
---
kind: Deployment
apiVersion: apps/v1
metadata:
name: whoamitcp
namespace: default
labels:
app: traefiklabs
name: whoamitcp
spec:
replicas: 2
selector:
matchLabels:
app: traefiklabs
task: whoamitcp
template:
metadata:
labels:
app: traefiklabs
task: whoamitcp
spec:
containers:
- name: whoamitcp
image: traefik/whoamitcp
ports:
- containerPort: 8080
---
apiVersion: v1
kind: Service
metadata:
name: whoamitcp
namespace: default
spec:
ports:
- protocol: TCP
port: 8080
selector:
app: traefiklabs
task: whoamitcp
---
kind: Deployment
apiVersion: apps/v1
metadata:
name: whoamiudp
namespace: default
labels:
app: traefiklabs
name: whoamiudp
spec:
replicas: 2
selector:
matchLabels:
app: traefiklabs
task: whoamiudp
template:
metadata:
labels:
app: traefiklabs
task: whoamiudp
spec:
containers:
- name: whoamiudp
image: traefik/whoamiudp:latest
ports:
- containerPort: 8080
---
apiVersion: v1
kind: Service
metadata:
name: whoamiudp
namespace: default
spec:
ports:
- port: 8080
protocol: UDP
selector:
app: traefiklabs
task: whoamiudp
# kubectl apply -f whoami.yaml
# kubectl get all
NAME READY STATUS RESTARTS AGE
pod/whoami-7d666f84d8-ffbdv 1/1 Running 0 35m
pod/whoami-7d666f84d8-n75wb 1/1 Running 0 35m
pod/whoamitcp-744cc4b47-g98fv 1/1 Running 0 35m
pod/whoamitcp-744cc4b47-s2m7h 1/1 Running 0 35m
pod/whoamiudp-58f6cf7b8-54552 1/1 Running 0 35m
pod/whoamiudp-58f6cf7b8-dzxpg 1/1 Running 0 35m
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 25d
service/nginx-metallb LoadBalancer 10.96.109.6 192.168.10.90 8090:30259/TCP 14d
service/whoami ClusterIP 10.96.251.213 <none> 80/TCP 35m
service/whoamitcp ClusterIP 10.96.20.1 <none> 8080/TCP 35m
service/whoamiudp ClusterIP 10.96.85.175 <none> 8080/UDP 35m
NAME READY UP-TO-DATE AVAILABLE AGE
deployment.apps/whoami 2/2 2 2 35m
deployment.apps/whoamitcp 2/2 2 2 35m
deployment.apps/whoamiudp 2/2 2 2 35m
NAME DESIRED CURRENT READY AGE
replicaset.apps/whoami-7d666f84d8 2 2 2 35m
replicaset.apps/whoamitcp-744cc4b47 2 2 2 35m
replicaset.apps/whoamiudp-58f6cf7b8 2 2 2 35m
3.2.2 创建whoami应用ingress route资源清单文件并应用¶
# vim whoami-ingressroute.yaml
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:
name: myingressroute
namespace: default
spec:
entryPoints:
- web
routes:
- match: Host(`whoami.kubemsb.com`) && PathPrefix(`/`)
kind: Rule
services:
- name: whoami
port: 80
# kubectl apply -f whoami-ingressroute.yaml
# kubectl get ingressroute
NAME AGE
myingressroute 29m
3.3 配置HTTPS路由规则¶
如果我们需要用 HTTPS 来访问我们这个应用的话，就需要监听 websecure 这个入口点，也就是通过 443 端口来访问，同样用 HTTPS 访问应用必然就需要证书
3.3.1 自签名证书¶
# openssl req -x509 -nodes -days 365 -newkey rsa:2048 -keyout tls.key -out tls.crt -subj "/CN=whoamissl.kubemsb.com"
3.3.2 创建secret¶
# kubectl create secret tls who-tls --cert=tls.crt --key=tls.key
3.3.3 创建https应用路由规则¶
# vim whoamissl-ingressroute.yaml
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:
name: ingressroutetls
spec:
entryPoints:
- websecure
routes:
- match: Host(`whoamissl.kubemsb.com`)
kind: Rule
services:
- name: whoami
port: 80
tls:
secretName: who-tls
# kubectl apply -f whoamissl-ingressroute.yaml
# kubectl get ingressroute
NAME AGE
ingressroutetls 17s
# cat /etc/hosts
192.168.10.12 whoamissl.kubemsb.com
3.4 配置TCP路由规则¶
SNI为服务名称标识，是 TLS 协议的扩展。因此，只有 TLS 路由才能使用该规则指定域名。但是，非 TLS 路由必须使用带有
*的规则（每个域）来声明每个非 TLS 请求都将由路由进行处理。
3.4.1 实验案例配置¶
# vim whoami-ingressroutetcp.yaml
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRouteTCP
metadata:
name: ingressroutetcpwho
spec:
entryPoints:
- tcpep
routes:
- match: HostSNI(`*`)
services:
- name: whoamitcp
port: 8080
# kubectl get ingressroutetcp
NAME AGE
ingressroutetcpwho 17s
3.4.2 生产案例配置 MySQL部署及traefik代理¶
3.4.2.1 修改traefik-configmap.yaml¶
# cat traefik-configmap.yaml
apiVersion: v1
kind: ConfigMap
metadata:
name: traefik
namespace: kube-system
data:
traefik.yaml: |-
serversTransport:
insecureSkipVerify: true
api:
insecure: true
dashboard: true
debug: true
metrics:
prometheus: ""
entryPoints:
web:
address: ":80"
websecure:
address: ":443"
metrics:
address: ":8082"
tcpep:
address: ":8083"
udpep:
address: ":8084/udp"
mysql:
address: ":3312"
providers:
kubernetesCRD: ""
kubernetesingress: ""
log:
filePath: ""
level: error
format: json
accessLog:
filePath: ""
format: json
bufferingSize: 0
filters:
retryAttempts: true
minDuration: 20
fields:
defaultMode: keep
names:
ClientUsername: drop
headers:
defaultMode: keep
names:
User-Agent: redact
Authorization: drop
Content-Type: keep
3.4.2.2 修改traefik-deploy.yaml¶
# vim traefik-deploy.yaml
apiVersion: apps/v1
kind: DaemonSet
metadata:
namespace: kube-system
name: traefik
labels:
app: traefik
spec:
selector:
matchLabels:
app: traefik
template:
metadata:
labels:
app: traefik
spec:
serviceAccountName: traefik-ingress-controller
containers:
- name: traefik
image: traefik:v2.5.7
args:
- --configfile=/config/traefik.yaml
volumeMounts:
- mountPath: /config
name: config
ports:
- name: web
containerPort: 80
hostPort: 80
- name: websecure
containerPort: 443
hostPort: 443
- name: admin
containerPort: 8080
- name: tcpep
containerPort: 8083
hostPort: 8083
- name: udpep
containerPort: 8084
hostPort: 8084
protocol: UDP
- name: mysql
containerPort: 3312
hostPort: 3306
volumes:
- name: config
configMap:
name: traefik
# vim traefik-service.yaml
apiVersion: v1
kind: Service
metadata:
name: traefik
namespace: kube-system
spec:
ports:
- protocol: TCP
name: web
port: 80
- protocol: TCP
name: admin
port: 8080
- protocol: TCP
name: websecure
port: 443
- protocol: TCP
name: tcpep
port: 8083
- protocol: UDP
name: udpep
port: 8084
- protocol: TCP
name: mysql
port: 3312
selector:
app: traefik
删除以前的配置及应用
# kubectl delete -f traefix-configmap.yaml
# kubectl delete -f traefix-deploy.yaml
# kubectl delete -f traefix-service.yaml
重新部署
# kubectl apply -f traefix-configmap.yaml
# kubectl apply -f traefix-deploy.yaml
# kubectl apply -f traefix-service.yaml
3.4.2.3 部署mysql应用¶
关于端口的说明： traefik Pod：3312:3306（traefik Pod:k8s Node），3312是traefik配置的mysql入口点的端口，3306是k8s Node的端口，traefik请求入口
# vim mysql-configmap.yaml
apiVersion: v1
kind: ConfigMap
metadata:
name: mysql
labels:
app: mysql
namespace: default
data:
my.cnf: |
[mysqld]
character-set-server = utf8mb4
collation-server = utf8mb4_unicode_ci
skip-character-set-client-handshake = 1
default-storage-engine = INNODB
max_allowed_packet = 500M
explicit_defaults_for_timestamp = 1
long_query_time = 10
# vim mysql-deploy.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
labels:
app: mysql
name: mysql
namespace: default
spec:
selector:
matchLabels:
app: mysql
template:
metadata:
labels:
app: mysql
spec:
containers:
- name: mysql
image: mysql:5.7
imagePullPolicy: IfNotPresent
env:
- name: MYSQL_ROOT_PASSWORD
value: abc123
ports:
- containerPort: 3306
volumeMounts:
- mountPath: /var/lib/mysql
name: pv
- mountPath: /etc/mysql/conf.d/my.cnf
subPath: my.cnf
name: cm
volumes:
- name: pv
hostPath:
path: /opt/mysqldata
- name: cm
configMap:
name: mysql
# vim mysql-service.yaml
apiVersion: v1
kind: Service
metadata:
name: mysql
namespace: default
spec:
ports:
- port: 3306
protocol: TCP
targetPort: 3306
selector:
app: mysql
# kubectl apply -f mysql-configmap.yaml
configmap/mysql created
# kubectl apply -f mysql-deploy.yaml
deployment.apps/mysql created
# kubectl apply -f mysql-service.yaml
service/mysql created
3.4.2.4 为mysql应用创建ingressroute¶
# vim mysql-ingressroutetcp.yaml
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRouteTCP
metadata:
name: mysql
namespace: default
spec:
entryPoints:
- mysql
routes:
- match: HostSNI(`*`)
services:
- name: mysql
port: 3306
# kubectl apply -f mysql-ingressroutetcp.yaml
ingressroutetcp.traefik.containo.us/mysql created
# kubectl get ingressroutetcp
NAME AGE
mysql 8s
3.4.2.5 验证¶
在集群之外主机上执行如下操作
# cat /etc/hosts
......
192.168.10.12 mysql.kubemsb.com
# mysql -h mysql.kubemsb.com -uroot -pabc123 -P3306
mysql: [Warning] Using a password on the command line interface can be insecure.
Welcome to the MySQL monitor. Commands end with ; or \g.
Your MySQL connection id is 3
Server version: 5.7.36 MySQL Community Server (GPL)
Copyright (c) 2000, 2022, Oracle and/or its affiliates.
Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
mysql>
3.4.3 生产案例 Redis部署及traefik代理¶
# vim redis.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: redis
namespace: default
spec:
selector:
matchLabels:
app: redis
template:
metadata:
labels:
app: redis
spec:
containers:
- name: redis
image: redis:6.2.6
ports:
- containerPort: 6379
protocol: TCP
---
apiVersion: v1
kind: Service
metadata:
name: redis
namespace: default
spec:
ports:
- port: 6379
targetPort: 6379
selector:
app: redis
# kubectl apply -f redis.yaml
deployment.apps/redis created
service/redis created
# vim traefik-configmap.yaml
apiVersion: v1
kind: ConfigMap
metadata:
name: traefik
namespace: kube-system
data:
traefik.yaml: |-
serversTransport:
insecureSkipVerify: true
api:
insecure: true
dashboard: true
debug: true
metrics:
prometheus: ""
entryPoints:
web:
address: ":80"
websecure:
address: ":443"
metrics:
address: ":8082"
tcpep:
address: ":8083"
udpep:
address: ":8084/udp"
mysql:
address: ":3312"
redis:
address: ":6379"
providers:
kubernetesCRD: ""
kubernetesingress: ""
log:
filePath: ""
level: error
format: json
accessLog:
filePath: ""
format: json
bufferingSize: 0
filters:
retryAttempts: true
minDuration: 20
fields:
defaultMode: keep
names:
ClientUsername: drop
headers:
defaultMode: keep
names:
User-Agent: redact
Authorization: drop
Content-Type: keep
删除原configmap,再重新创建
# kubectl delete -f traefik-configmap.yaml
# kubectl apply -f traefik-configmap.yaml
# vim traefix-deploy.yaml
apiVersion: apps/v1
kind: DaemonSet
metadata:
namespace: kube-system
name: traefik
labels:
app: traefik
spec:
selector:
matchLabels:
app: traefik
template:
metadata:
labels:
app: traefik
spec:
serviceAccountName: traefik-ingress-controller
containers:
- name: traefik
image: traefik:v2.5.7
args:
- --configfile=/config/traefik.yaml
volumeMounts:
- mountPath: /config
name: config
ports:
- name: web
containerPort: 80
hostPort: 80
- name: websecure
containerPort: 443
hostPort: 443
- name: admin
containerPort: 8080
- name: tcpep
containerPort: 8083
hostPort: 8083
- name: udpep
containerPort: 8084
hostPort: 8084
protocol: UDP
- name: mysql
containerPort: 3312
hostPort: 3306
- name: redis
containerPort: 6379
hostPort: 6379
volumes:
- name: config
configMap:
name: traefik
删除原deploy,再重新创建
# kubectl delete -f traefik-deploy.yaml
# kubectl apply -f traefik-deploy.yaml
验证是否添加成功
# kubectl get daemonset traefik -n kube-system -o yaml
...
- containerPort: 6379
hostPort: 6379
name: redis
protocol: TCP
...
# vim redis-ingressroutetcp.yaml
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRouteTCP
metadata:
name: redis
namespace: default
spec:
entryPoints:
- redis
routes:
- match: HostSNI(`*`)
services:
- name: redis
port: 6379
# kubectl get ingressroutetcp
NAME AGE
redis 8s
在集群之外主机添加域名解析
# cat /etc/hosts
192.168.10.15 redis.kubemsb.com
# wget http://download.redis.io/releases/redis-3.2.8.tar.gz
# tar xf redis-3.2.8.tar.gz
# make
# ./src/redis-cli -h redis.kubemsb.com -p 6379
redis.kubemsb.com:6379> ping
PONG
redis.kubemsb.com:6379> set hello world
OK
redis.kubemsb.com:6379> get hello
"world"
3.5 配置UDP路由规则¶
# vim whoami-ingressrouteudp.yaml
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRouteUDP
metadata:
name: ingressrouteudpwho
spec:
entryPoints:
- udpep
routes:
- services:
- name: whoamiudp
port: 8080
# kubectl get ingressrouteudp
NAME AGE
ingressrouteudpwho 11s
验证可有性
# kubectl get svc
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
whoamiudp ClusterIP 10.96.85.175 <none> 8080/TCP 6h35m
echo "WHO" | socat - udp4-datagram:10.96.85.175:8080
echo "othermessage" | socat - udp4-datagram:10.96.85.175:8080
输出
# echo "WHO" | socat - udp4-datagram:192.168.10.12:8084
Hostname: whoamiudp-58f6cf7b8-54552
IP: 127.0.0.1
IP: ::1
IP: 10.244.159.187
IP: fe80::1cb4:e1ff:fe66:d9b
# echo "othermessage" | socat - udp4-datagram:192.168.10.12:8084
Received: othermessage
四、traefik中间件 MiddleWare¶
4.1 traefik中间件介绍 MiddleWare¶
中间件是 Traefik2.0 中一个非常有特色的功能，我们可以根据自己的各种需求去选择不同的中间件来满足服务，Traefik 官方已经内置了许多不同功能的中间件，其中包括修改请求头信息；重定向；身份验证等等，而且中间件还可以通过链式组合的方式来适用各种情况。例如：强制跳转https、去除访问前缀、访问白名单等。
4.2 traefik 中间件应用案例 ipWhiteList¶
在工作 中，有一些URL并不希望对外暴露，比如prometheus、grafana等，我们就可以通过白名单IP来过到要求，可以使用Traefix中的ipWhiteList中间件来完成。
# vim deploy-service.yaml
kind: Deployment
apiVersion: apps/v1
metadata:
name: nginx-web-middleware
namespace: default
spec:
replicas: 1
selector:
matchLabels:
app: middle
template:
metadata:
labels:
app: middle
spec:
containers:
- name: nginx-web-c
image: nginx:latest
ports:
- containerPort: 80
---
apiVersion: v1
kind: Service
metadata:
name: service-middle
namespace: default
spec:
ports:
- name: http
port: 80
selector:
app: middle
# kubectl apply -f deploy-service.yaml
deployment.apps/nginx-web-middleware created
service/service-middle created
# vim middleware-ipwhitelist.yaml
apiVersion: traefik.containo.us/v1alpha1
kind: Middleware
metadata:
name: gs-ipwhitelist
spec:
ipWhiteList:
sourceRange:
- 127.0.0.1
- 10.244.0.0/16
- 10.96.0.0/12
- 192.168.10.0/24
# kubectl apply -f middleware-ipwhitelist.yaml
middleware.traefik.containo.us/gs-ipwhitelist created
# vim deploy-service-middle.yaml
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:
name: ingressroutemiddle
namespace: default
spec:
entryPoints:
- web
routes:
- match: Host(`middleware.kubemsb.com`) && PathPrefix(`/`)
kind: Rule
services:
- name: service-middle
port: 80
namespace: default
middlewares:
- name: gs-ipwhitelist
# kubectl apply -f deploy-service-middle.yaml
ingressroute.traefik.containo.us/ingressroutemiddle created
在集群之外的主机上访问
# vim /etc/hosts
192.168.10.15 middleware.kubemsb.com
# curl http://middleware.kubemsb.com
把集群外主机所在的网段从白名单中删除，发现无法访问。
# cat middleware-ipwhitelist.yaml
apiVersion: traefik.containo.us/v1alpha1
kind: Middleware
metadata:
name: gs-ipwhitelist
spec:
ipWhiteList:
sourceRange:
- 127.0.0.1
- 10.244.0.0/16
- 10.96.0.0/12
# kubectl apply -f middleware-ipwhitelist.yaml
middleware.traefik.containo.us/gs-ipwhitelist configured
在集群外主机访问
# curl http://middleware.kubemsb.com
Forbidden
五、traefik高级应用¶
在实际的生产环境，除了上线业务之外，还有更复杂的使用要求。 在开始traefik的高级用法之前，还需要了解一个TraefikService，通过把TraefikService注册到CRD来实现更复杂的请求设置。
TraefikService 目前能用于以下功能
servers load balancing.（负载均衡）
services Weighted Round Robin load balancing.（权重轮询）
services mirroring.（镜像）
5.1 负载均衡¶
5.1.1 创建deployment控制器类型应用¶
# vim loadbalancer-deploy.yaml
kind: Deployment
apiVersion: apps/v1
metadata:
name: nginx-web1
namespace: default
spec:
replicas: 1
selector:
matchLabels:
app: v1
template:
metadata:
labels:
app: v1
spec:
containers:
- name: nginx-web-c
image: nginx:latest
ports:
- containerPort: 80
---
kind: Deployment
apiVersion: apps/v1
metadata:
name: nginx-web2
namespace: default
spec:
replicas: 1
selector:
matchLabels:
app: v2
template:
metadata:
labels:
app: v2
spec:
containers:
- name: nginx-web-c
image: nginx:latest
ports:
- containerPort: 80
# kubectl apply -f loadbalancer-deploy.yaml
修改容器中网站页面信息
# kubectl get pods
NAME READY STATUS RESTARTS AGE
nginx-web1-856c759646-58snd 1/1 Running 0 2m34s
nginx-web2-5d547f7c5f-vd8n7 1/1 Running 0 2m34s
# kubectl exec -it nginx-web1-856c759646-58snd -- sh
# echo "svc1" > /usr/share/nginx/html/index.html
# exit
# kubectl exec -it nginx-web2-5d547f7c5f-vd8n7 -- sh
# echo "svc2" > /usr/share/nginx/html/index.html
# exit
# vim loadbalancer-deploy.yaml
kind: Deployment
apiVersion: apps/v1
metadata:
name: nginx-web1
namespace: default
spec:
replicas: 1
selector:
matchLabels:
app: v1
template:
metadata:
labels:
app: v1
spec:
containers:
- name: nginx-web-c
image: nginx:latest
lifecycle:
postStart:
exec:
command: ["/bin/sh", "-c", "echo svc1 > /usr/share/nginx/html/index.html"]
ports:
- containerPort: 80
---
kind: Deployment
apiVersion: apps/v1
metadata:
name: nginx-web2
namespace: default
spec:
replicas: 1
selector:
matchLabels:
app: v2
template:
metadata:
labels:
app: v2
spec:
containers:
- name: nginx-web-c
image: nginx:latest
lifecycle:
postStart:
exec:
command: ["/bin/sh", "-c", "echo svc2 > /usr/share/nginx/html/index.html"]
ports:
- containerPort: 80
5.1.2 创建service¶
# vim loadbalancer-deploy-service.yaml
apiVersion: v1
kind: Service
metadata:
name: svc1
namespace: default
spec:
ports:
- name: http
port: 80
selector:
app: v1
---
apiVersion: v1
kind: Service
metadata:
name: svc2
namespace: default
spec:
ports:
- name: http
port: 80
selector:
app: v2
# kubectl apply -f loadbalancer-deploy-service.yaml
# kubectl get svc
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
svc1 ClusterIP 10.96.136.245 <none> 80/TCP 3s
svc2 ClusterIP 10.96.2.4 <none> 80/TCP 3s
# curl http://10.96.136.245
svc1
# curl http://10.96.2.4
svc2
5.1.3 创建ingressroute¶
# vim loadbalancer-deploy-service-ingressroute.yaml
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:
name: ingressrouteweblb
namespace: default
spec:
entryPoints:
- web
routes:
- match: Host(`lb.kubemsb.com`) && PathPrefix(`/`)
kind: Rule
services:
- name: svc1
port: 80
namespace: default
- name: svc2
port: 80
namespace: default
# kubectl apply -f loadbalancer-deploy-service-ingressroute.yaml
# kubectl get ingressroute
NAME AGE
ingressrouteweblb 9s
5.1.4 访问验证¶
在集群之外的主机上访问
# cat /etc/hosts
192.168.10.15 lb.kubemsb.com
# curl http://lb.kubemsb.com
svc1
# curl http://lb.kubemsb.com
svc2
5.2 灰度发布¶
基于上述负载均衡案例基础之上实施。
灰度发布也称为金丝雀发布，让一部分即将上线的服务发布到线上，观察是否达到上线要求，主要通过权重轮询的方式实现。
5.2.1 创建TraefikService¶
# vim traefikservice.yaml
apiVersion: traefik.containo.us/v1alpha1
kind: TraefikService
metadata:
name: wrr
namespace: default
spec:
weighted:
services:
- name: svc1
port: 80
weight: 3 # 定义权重
kind: Service # 可选，默认就是 Service
- name: svc2
port: 80
weight: 1
# kubectl apply -f traefikservice.yaml
traefikservice.traefik.containo.us/wrr created
5.2.2 创建ingressroute¶
需要注意的是现在我们配置的 Service 不再是直接的 Kubernetes 对象了，而是上面我们定义的 TraefikService 对象
# vim traefik-wrr.yaml
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:
name: ingressroutewrr
namespace: default
spec:
entryPoints:
- web
routes:
- match: Host(`lb.kubemsb.com`) && PathPrefix(`/`)
kind: Rule
services:
- name: wrr
namespace: default
kind: TraefikService
# kubectl apply -f traefik-wrr.yaml
ingressroute.traefik.containo.us/ingressroutewrr created
第一次
# curl http://lb.kubemsb.com
svc1
第二次
# curl http://lb.kubemsb.com
svc1
第三次
# curl http://lb.kubemsb.com
svc2
第四次
# curl http://lb.kubemsb.com
svc1
5.3 流量复制¶
在负责均衡案例基础之上实施
所谓的流量复制，也称为镜像服务是指将请求的流量按规则复制一份发送给其它服务，并且会忽略这部分请求的响应，这个功能在做一些压测或者问题复现的时候很有用。
5.3.1 指定流量来自己于kubernetes service对象¶
# vim mirror_from_service.yaml
apiVersion: traefik.containo.us/v1alpha1
kind: TraefikService
metadata:
name: mirror-from-service
namespace: default
spec:
mirroring:
name: svc1 # 发送 100% 的请求到 K8S 的 Service "v1"
port: 80
mirrors:
- name: svc2 # 然后复制 20% 的请求到 v2
port: 80
percent: 20
# kubectl apply -f mirror_from_service.yaml
traefikservice.traefik.containo.us/mirror-from-service created
# vim mirror-from-service-ingressroute.yaml
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:
name: ingressroute-mirror
namespace: default
spec:
entryPoints:
- web
routes:
- match: Host(`lb.kubemsb.com`) && PathPrefix(`/`)
kind: Rule
services:
- name: mirror-from-service
namespace: default
kind: TraefikService
# kubectl apply -f mirror-from-service-ingressroute.yaml
ingressroute.traefik.containo.us/ingressroute-mirror created
# while true
do
curl http://lb.kubemsb.com
sleep 1
done
5.3.2 通过traefikservice导入流量¶
# vim mirror-from-traefikservice.yaml
apiVersion: traefik.containo.us/v1alpha1
kind: TraefikService
metadata:
name: mirror-from-traefikservice
namespace: default
spec:
mirroring:
name: mirror-from-service #流量入口从TraefikService 来
kind: TraefikService
mirrors:
- name: svc2
port: 80
percent: 20
# kubectl apply -f mirror-from-traefikservice.yaml
traefikservice.traefik.containo.us/mirror-from-traefikservice created
# vim mirror-from-traefikservice-ingressroute.yaml
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:
name: ingressroute-mirror-traefikservice
namespace: default
spec:
entryPoints:
- web
routes:
- match: Host(`lb.kubemsb.com`) && PathPrefix(`/`)
kind: Rule
services:
- name: mirror-from-traefikservice
namespace: default
kind: TraefikService
# kubectl apply -f mirror-from-tradfikservice-ingressroute.yaml
ingressroute.traefik.containo.us/ingressroute-mirror-traefikservice created
# while true; do curl http://lb.kubemsb.com; sleep 1; done
5.3.3 流量复制小结¶
通过上述的演示我们会发现所有的流量100%发送了svc1,有20%的流量被复制到svc2，且用户收到响应均来自svc1，svc2并没有响应，可通过查看svc1及svc2应用日志获取访问日志。
六、Kubernetes Gateway API¶
6.1 Gateway API介绍¶
6.1.1 Gateway API架构¶
Gateway API（之前叫 Service API）是由 SIG-NETWORK 社区管理的开源项目，项目地址：https://gateway-api.sigs.k8s.io/。主要原因是 Ingress 资源对象不能很好的满足网络需求，很多场景下 Ingress 控制器都需要通过定义 annotations 或者 crd 来进行功能扩展，这对于使用标准和支持是非常不利的，新推出的 Gateway API 旨在通过可扩展的面向角色的接口来增强服务网络。
Gateway API 是 Kubernetes 中的一个 API 资源集合，包括 GatewayClass、Gateway、HTTPRoute、TCPRoute、Service 等，这些资源共同为各种网络用例构建模型。
Gateway API 的改进比当前的 Ingress 资源对象有很多更好的设计：
- 面向角色 - Gateway 由各种 API 资源组成，这些资源根据使用和配置 Kubernetes 服务网络的角色进行建模。
- 通用性 - 和 Ingress 一样是一个具有众多实现的通用规范，Gateway API 是一个被设计成由许多实现支持的规范标准。
- 更具表现力 - Gateway API 资源支持基于 Header 头的匹配、流量权重等核心功能，这些功能在 Ingress 中只能通过自定义注解才能实现。
- 可扩展性 - Gateway API 允许自定义资源链接到 API 的各个层，这就允许在 API 结构的适当位置进行更精细的定制。
还有一些其他值得关注的功能：
- GatewayClasses - GatewayClasses 将负载均衡实现的类型形式化，这些类使用户可以很容易了解到通过 Kubernetes 资源可以获得什么样的能力。
- 共享网关和跨命名空间支持 - 它们允许共享负载均衡器和 VIP，允许独立的路由资源绑定到同一个网关，这使得团队可以安全地共享（包括跨命名空间）基础设施，而不需要直接协调。
- 规范化路由和后端 - Gateway API 支持类型化的路由资源和不同类型的后端，这使得 API 可以灵活地支持各种协议（如 HTTP 和 gRPC）和各种后端服务（如 Kubernetes Service、存储桶或函数）。
6.1.2 面向角色设计¶
无论是道路、电力、数据中心还是 Kubernetes 集群，基础设施都是为了共享而建的，然而共享基础设施提供了一个共同的挑战，那就是如何为基础设施用户提供灵活性的同时还能被所有者控制。
Gateway API 通过对 Kubernetes 服务网络进行面向角色的设计来实现这一目标，平衡了灵活性和集中控制。它允许共享的网络基础设施（硬件负载均衡器、云网络、集群托管的代理等）被许多不同的团队使用，所有这些都受到集群运维设置的各种策略和约束。下面的例子显示了是如何在实践中运行的。
一个集群运维人员创建了一个基于 GatewayClass 的 Gateway 资源，这个 Gateway 部署或配置了它所代表的基础网络资源，集群运维和特定的团队必须沟通什么可以附加到这个 Gateway 上来暴露他们的应用。集中的策略，如 TLS，可以由集群运维在 Gateway 上强制执行，同时，Store 和 Site 应用在他们自己的命名空间中运行，但将他们的路由附加到相同的共享网关上，允许他们独立控制他们的路由逻辑。
这种关注点分离的设计可以使不同的团队能够管理他们自己的流量，同时将集中的策略和控制留给集群运维。
6.1.3 Gateway API概念¶
在整个 Gateway API 中涉及到3个角色：基础设施提供商、集群管理员、应用开发人员，在某些场景下可能还会涉及到应用管理员等角色。Gateway API 中定义了3种主要的资源模型：GatewayClass、Gateway、Route。
6.1.3.1 GatewayClass¶
GatewayClass 定义了一组共享相同配置和动作的网关。每个GatewayClass 由一个控制器处理，是一个集群范围的资源，必须至少有一个 GatewayClass 被定义。
这与 Ingress 的 IngressClass 类似，在 Ingress v1beta1 版本中，与 GatewayClass 类似的是 ingress-class 注解，而在Ingress V1 版本中，最接近的就是 IngressClass 资源对象。
6.1.3.2 Gateway¶
Gateway 网关描述了如何将流量转化为集群内的服务，也就是说，它定义了一个请求，要求将流量从不了解 Kubernetes 的地方转换到集群内的服务。例如，由云端负载均衡器、集群内代理或外部硬件负载均衡器发送到 Kubernetes 服务的流量。
它定义了对特定负载均衡器配置的请求，该配置实现了 GatewayClass 的配置和行为规范，该资源可以由管理员直接创建，也可以由处理 GatewayClass 的控制器创建。
Gateway 可以附加到一个或多个路由引用上，这些路由引用的作用是将流量的一个子集导向特定的服务。
6.1.3.3 Route 资源¶
路由资源定义了特定的规则，用于将请求从网关映射到 Kubernetes 服务。
从 v1alpha2 版本开始，API 中包含四种 Route 路由资源类型，对于其他未定义的协议，鼓励采用特定实现的自定义路由类型，当然未来也可能会添加新的路由类型。
6.1.3.3.1 HTTPRoute¶
HTTPRoute 适用于 HTTP 或 HTTPS 连接，适用于我们想要检查 HTTP 请求并使用 HTTP 请求进行路由或修改的场景，比如使用 HTTP Headers 头进行路由，或在请求过程中对它们进行修改。
6.1.3.3.2 TLSRoute¶
TLSRoute 用于 TLS 连接，通过 SNI 进行区分，它适用于希望使用 SNI 作为主要路由方法的地方，并且对 HTTP 等更高级别协议的属性不感兴趣，连接的字节流不经任何检查就被代理到后端。
6.1.3.3.3 TCPRoute 和 UDPRoute¶
TCPRoute（和UDPRoute）旨在用于将一个或多个端口映射到单个后端。在这种情况下，没有可以用来选择同一端口的不同后端的判别器，所以每个 TCPRoute 在监听器上需要一个不同的端口。你可以使用 TLS，在这种情况下，未加密的字节流会被传递到后端，当然也可以不使用 TLS，这样加密的字节流将传递到后端。
6.1.4 Gateway API概念之间关系¶
GatewayClass、Gateway、xRoute 和 Service 的组合定义了一个可实施的负载均衡器。下图说明了不同资源之间的关系:
使用反向代理实现的网关的典型客户端/网关 API 请求流程如下所示：
- 客户端向 http://foo.example.com 发出请求
- DNS 将域名解析为 Gateway 网关地址
- 反向代理在监听器上接收请求，并使用 Host Header 来匹配HTTPRoute
- (可选)反向代理可以根据 HTTPRoute 的匹配规则进行路由
- (可选)反向代理可以根据 HTTPRoute 的过滤规则修改请求，即添加或删除 headers
- 最后，反向代理根据 HTTPRoute 的 forwardTo 规则，将请求转发给集群中的一个或多个对象，即服务。
6.2 kubernetes gateway CRD安装¶
要在 Traefik 中使用 Gateway API，首先我们需要先手动安装 Gateway API 的 CRDs，使用如下命令即可安装，这将安装包括 GatewayClass、Gateway、HTTPRoute、TCPRoute 等 CRDs：
# kubectl apply -k "github.com/kubernetes-sigs/service-apis/config/crd?ref=v0.3.0"
输出：
customresourcedefinition.apiextensions.k8s.io/gatewayclasses.gateway.networking.k8s.io created
customresourcedefinition.apiextensions.k8s.io/gateways.gateway.networking.k8s.io created
customresourcedefinition.apiextensions.k8s.io/httproutes.gateway.networking.k8s.io created
customresourcedefinition.apiextensions.k8s.io/referencepolicies.gateway.networking.k8s.io created
customresourcedefinition.apiextensions.k8s.io/tcproutes.gateway.networking.k8s.io created
customresourcedefinition.apiextensions.k8s.io/tlsroutes.gateway.networking.k8s.io created
customresourcedefinition.apiextensions.k8s.io/udproutes.gateway.networking.k8s.io created
6.3 为traefik授权（RBAC）¶
# vim traefik-gatewayapi-rbac.yaml
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: gateway-role
rules:
- apiGroups:
- ""
resources:
- services
- endpoints
- secrets
verbs:
- get
- list
- watch
- apiGroups:
- networking.x-k8s.io
resources:
- gatewayclasses
- gateways
- httproutes
- tcproutes
- tlsroutes
verbs:
- get
- list
- watch
- apiGroups:
- networking.x-k8s.io
resources:
- gatewayclasses/status
- gateways/status
- httproutes/status
- tcproutes/status
- tlsroutes/status
verbs:
- update
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: gateway-controller
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: gateway-role
subjects:
- kind: ServiceAccount
name: traefik-ingress-controller
namespace: kube-system
6.4 Traefik开启Gateway api支持¶
# vim traefik-configmap.yaml
......
providers:
kubernetesCRD: ""
kubernetesingress: ""
kubernetesGateway: "" 添加此行
experimental: 添加此行
kubernetesGateway: true 添加此行
......
删除
# kubectl delete -f traefik-configmap.yaml
# kubectl delete -f traefik-deploy.yaml
重新运行
# kubectl apply -f traefik-configmap.yaml
# kubectl apply -f traefik-deploy.yaml
6.5 创建Gateway API的GatewayClass¶
# vim gatewayclass.yaml
apiVersion: networking.x-k8s.io/v1alpha1
kind: GatewayClass
metadata:
name: traefik
spec:
controller: traefik.io/gateway-controller
# kubectl apply -f gatewayclass.yaml
# kubectl get gatewayclass
NAME CONTROLLER AGE
traefik traefik.io/gateway-controller 2m59s
6.6 Gateway API应用案例¶
6.6.1 通过Gateway API方式暴露traefik dashboard¶
6.6.1.1 创建gateway¶
# vim gateway.yaml
apiVersion: networking.x-k8s.io/v1alpha1
kind: Gateway
metadata:
name: http-gateway
namespace: kube-system
spec:
gatewayClassName: traefik
listeners:
- protocol: HTTP
port: 80
routes:
kind: HTTPRoute
namespaces:
from: All
selector:
matchLabels:
app: traefik
# kubectl apply -f gateway.yaml
gateway.networking.x-k8s.io/http-gateway created
# kubectl get gateway
NAME CLASS AGE
http-gateway traefik 6s
6.5.1.2 创建HTTPRoute¶
# vim httproute.yaml
apiVersion: networking.x-k8s.io/v1alpha1
kind: HTTPRoute
metadata:
name: traefix-dashboard-gateway-api-route
namespace: kube-system
labels:
app: traefik
spec:
hostnames:
- "traefikdashboard.kubemsb.com"
rules:
- matches:
- path:
type: Prefix
value: /
forwardTo:
- serviceName: traefik
port: 8080
weight: 1
# kubectl apply -f httproute.yaml
httproute.networking.x-k8s.io/traefix-dashboard-gateway-api-route created
# kubectl get httproute
NAME HOSTNAMES AGE
traefix-dashboard-gateway-api-route ["traefikdashboard.kubemsb.com"] 6s
6.5.1.3 在集群之外主机访问¶
# vim /etc/hosts
192.168.10.12 traefikdashboard.kubemsb.com
6.6.2 通过Gateway API方式暴露WEB应用¶
# cat gatewayapi-web.yaml
kind: Deployment
apiVersion: apps/v1
metadata:
name: nginx-web-gatewayapi
namespace: default
spec:
replicas: 1
selector:
matchLabels:
app: gatewayweb
template:
metadata:
labels:
app: gatewayweb
spec:
containers:
- name: nginx-web
image: nginx:latest
lifecycle:
postStart:
exec:
command: ["/bin/sh", "-c", "echo gatewayweb > /usr/share/nginx/html/index.html"]
ports:
- containerPort: 80
---
apiVersion: v1
kind: Service
metadata:
name: nginx-web-gatewayapi-svc
namespace: default
spec:
ports:
- name: http
port: 80
selector:
app: gatewayweb
# kubectl apply -f gatewayapi-web.yaml
deployment.apps/nginx-web-gatewayapi created
service/nginx-web-gatewayapi-svc created
# vim gatewayapi-web-gateway.yaml
apiVersion: networking.x-k8s.io/v1alpha1
kind: Gateway
metadata:
name: nginx-web-gateway
spec:
gatewayClassName: traefik
listeners:
- protocol: HTTP
port: 80
routes:
kind: HTTPRoute
namespaces:
from: All
selector:
matchLabels:
app: gatewayweb
# kubectl apply -f gatewayapi-web-gateway.yaml
gateway.networking.x-k8s.io/nginx-web-gateway created
# kubectl get gateway
NAME CLASS AGE
nginx-web-gateway traefik 7s
# kubectl describe gateway nginx-web-gateway
# cat gatewayapi-web-httproute.yaml
apiVersion: networking.x-k8s.io/v1alpha1
kind: HTTPRoute
metadata:
name: nginx-web-gateway-api-route
labels:
app: gatewayweb
spec:
hostnames:
- "nginx.kubemsb.com"
rules:
- matches:
- path:
type: Prefix
value: /
forwardTo:
- serviceName: nginx-web-gatewayapi-svc
port: 80
weight: 1
# kubectl apply -f gatewayapi-web-httproute.yaml
httproute.networking.x-k8s.io/nginx-web-gateway-api-route created
kubectl get httproute
NAME HOSTNAMES AGE
nginx-web-gateway-api-route ["nginx.kubemsb.com"] 6s
在集群之外主机访问
# vim /etc/hosts
192.168.10.12 nginx.kubemsb.com
# curl http://nginx.kubemsb.com
gatewayweb
6.6.3 金丝雀发布¶
Gateway APIs 规范可以支持的另一个功能是金丝雀发布，假设你想在一个端点上运行两个不同的服务（或同一服务的两个版本），并将一部分请求路由到每个端点，则可以通过修改你的 HTTPRoute 来实现。
# vim gateway-cn-deploy.yaml
kind: Deployment
apiVersion: apps/v1
metadata:
name: nginx-web1
namespace: kube-system
spec:
replicas: 1
selector:
matchLabels:
app: v1
template:
metadata:
labels:
app: v1
spec:
containers:
- name: nginx-web-c
image: nginx:latest
lifecycle:
postStart:
exec:
command: ["/bin/sh", "-c", "echo svc1 > /usr/share/nginx/html/index.html"]
ports:
- containerPort: 80
---
kind: Deployment
apiVersion: apps/v1
metadata:
name: nginx-web2
namespace: kube-system
spec:
replicas: 1
selector:
matchLabels:
app: v2
template:
metadata:
labels:
app: v2
spec:
containers:
- name: nginx-web-c
image: nginx:latest
lifecycle:
postStart:
exec:
command: ["/bin/sh", "-c", "echo svc2 > /usr/share/nginx/html/index.html"]
ports:
- containerPort: 80
# kubectl apply -f gateway-cn-deploy.yaml
# vim gateway-cn-deploy-svc.yaml
apiVersion: v1
kind: Service
metadata:
name: svc1
namespace: kube-system
spec:
ports:
- name: http
port: 80
selector:
app: v1
---
apiVersion: v1
kind: Service
metadata:
name: svc2
namespace: kube-system
spec:
ports:
- name: http
port: 80
selector:
app: v2
# kubectl apply -fgateway-cn-deploy-svc.yaml
# vim gateway-cn-httproute.yaml
apiVersion: networking.x-k8s.io/v1alpha1
kind: HTTPRoute
metadata:
labels:
app: traefik
name: nginxweb-app
namespace: kube-system
spec:
hostnames:
- canary.kubemsb.com
rules:
- forwardTo:
- port: 80
serviceName: svc1
weight: 3 # 3/4 的请求到svc1
- port: 80
serviceName: svc2
weight: 1 # 1/4 的请求到svc2
# kubectl apply -f gateway-cn-httproute.yaml
httproute.networking.x-k8s.io/nginxweb-app created
# kubectl get httproute -n kube-system
NAME HOSTNAMES AGE
nginxweb-app ["canary.kubemsb.com"] 7s
在 集群之外主机访问
# vim /etc/hosts
192.168.10.12 canary.kubemsb.com
[root@dockerhost ~]# curl http://canary.kubemsb.com
svc1
[root@dockerhost ~]# curl http://canary.kubemsb.com
svc1
[root@dockerhost ~]# curl http://canary.kubemsb.com
svc1
[root@dockerhost ~]# curl http://canary.kubemsb.com
svc2
以上使用 Traefik 来测试了 Kubernetes Gateway APIs 的使用。目前，Traefik 对 Gateway APIs 的实现是基于 v1alpha1 版本的规范，目前最新的规范是 v1alpha2，所以和最新的规范可能有一些出入的地方。