Download to read offline
Kubernetes Controller for Pull Request Based Environment
- 1. k8s Controller forPull Request based Environment
- 2.
- 3. It means whenevera developer creates a new Pull Request to their microservice, it will be automatically deployed in development environment. What is Pull Request Based Environment?
- 4. Simply, it’s greatfor development, debugging and testing. How it is useful?
- 5. Simply, it’s greatfor development, debugging and testing. If your branch is deployed in cluster and can talk with other services then its really helpful for debugging, development and end to end testing. How it is useful?
- 6. How it isuseful? ... In microservices architecture, services depend on other services and as it grows dependency graph gets more complicated. It is responsibility of upstream developers to keep their master branch stable so that downstream service development does not halt.
- 7. Simple Microservices ArchitectureExample (BookInfo) Product Page Review Service Detail Service Rating Service
- 8. Simple Microservices ArchitectureExample (BookInfo) Product Page (frontend) Review Service Detail Service Rating Service frontend
- 9. Simple Microservices ArchitectureExample (BookInfo) Product Page (frontend) Review Service Detail Service Rating Service backend (microservices)
- 11.
- 12.
- 13.
- 14.
- 15. Kubernetes Deployment Product Page ReviewService Detail Service Rating Service
- 16. # deployment.yaml apiVersion: apps/v1 kind:Deployment metadata: name: rating spec: template: containers: - name: rating image: rating:v1 Kubernetes Deployment (Rating) Product Page Review Service Detail Service Rating Service # service.yaml apiVersion: v1 kind: Service metadata: name: rating spec: ports: - name: TCP port: 80 targetPort: 8080 Rating Service is available at: rating.default.svc.cluster.local
- 17. # deployment.yaml apiVersion: apps/v1 kind:Deployment metadata: name: review spec: template: containers: - name: review image: review:v1 env: RATING_URL: rating.default.svc.cluster.local Kubernetes Deployment (Review) Product Page Review Service Detail Service Rating Service # service.yaml apiVersion: v1 kind: Service metadata: name: review spec: ports: - name: TCP port: 80 targetPort: 8080 Review Service is available at: review.default.svc.cluster.local
- 18. # deployment.yaml apiVersion: apps/v1 kind:Deployment metadata: name: product-page spec: template: containers: - name: product-page image: product-page:v1 env: REVIEW_URL: review.default.svc.cluster.local DETAIL_URL: detail.default.svc.cluster.local Kubernetes Deployment (ProductPage) Product Page Review Service Detail Service Rating Service # service.yaml apiVersion: v1 kind: Service metadata: name: product-page spec: ports: - name: TCP port: 80 targetPort: 8080 type: LoadBalancer Review Service is available at: product-page.example.com
- 19. apiVersion: apps/v1 kind: Deployment metadata: name:product-page spec: template: containers: - name: product-page image: product-page:v1 env: REVIEW_URL: review.default.svc.cluster.local DETAIL_URL: detail.default.svc.cluster.local Kubernetes Deployment (ProductPage) Product Page Review Service Detail Service Rating Service ● Although ProductPage does not have any direct dependency on Rating service but it depends on it. ● If Rating Service breaks, ProductPage might also break.
- 20. apiVersion: apps/v1 kind: Deployment metadata: name:product-page spec: template: containers: - name: product-page image: product-page:v1 env: REVIEW_URL: review.default.svc.cluster.local DETAIL_URL: detail.default.svc.cluster.local Kubernetes Deployment (ProductPage) Product Page Review Service Detail Service Rating Service ● Although ProductPage does not have any direct dependency on Rating service but it depends on it. ● If Rating Service breaks, ProductPage might also break. So, it is responsibility of RatingService owners to keep default branch of ratingService tested, so that downstream service development does not halt.
- 21. How to makesure Rating Service is stable?
- 22. End to EndTesting!
- 23.
- 24. But, you first needan environment to test your feature branch
- 25. But, you first needan environment to test your feature branch https://product-page-{feature}.example.com
- 26. Development Workflow Product Page ReviewService Detail Service Rating Service
- 27. Workflow Product Page Review Service DetailService Rating Service New Feature Requirement: 6-star-rating
- 28. Workflow Product Page Review Service DetailService Rating Service 6-star-rating (pr-222) rating-222.default.svc.cluster.local
- 29. Workflow Product Page Review Service DetailService Rating Service 6-star-rating (pr-222) rating-222.default.svc.cluster.local 6-star-rating (pr-333) review-333.default.svc.cluster.local
- 30. Workflow Product Page Review Service DetailService Rating Service 6-star-rating (pr-222) rating-222.default.svc.cluster.local 6-star-rating (pr-333) review-333.default.svc.cluster.local 6-star-rating (pr-500) product-page-500.example.com
- 31. Workflow Product Page Review Service DetailService Rating Service 6-star-rating (pr-222) rating-222.default.svc.cluster.local 6-star-rating (pr-333) review-333.default.svc.cluster.local 6-star-rating (pr-333) product-page-500.example.com
- 32. k8s Pull RequestReplication Controller
- 33. Current State ->Desired State Kubernetes Controller Philosophy
- 34. Current State ->Desired State Kubernetes Controller Philosophy for { desired := getDesiredState() current := getCurentState() makeChanges(desired, current) }
- 35. Desired State There shouldbe a Deployment and a Service for all Pull Requests.
- 36. Usage (Rating Service) apiVersion:apps/v1 kind: Deployment metadata: name: rating annotation: pr-rc.alpha.mercari.com/repository: https://github.com/example/rating-service spec: template: containers: - name: rating image: rating:v1 ● Developer needs to add an annotation to let controller know which repository to look.
- 37. Pull Request ReplicationController ... func (c *Controller) Run(ctx context.Context) { for { c.RunOnce(ctx) select { case <-time.After(60 * time.Second): } } }
- 38.
- 39.
- 40.
- 41.
- 42. RunOnce() kubernetes API Server fordeploy := range diff(desired - current) { // New Pull Request CreateDuplicateDeployment(deplo y) }
- 43. RunOnce() kubernetes API Server fordeploy := range diff(current - desired) { // Closed Pull Request DeleteDuplicateDeployment(deplo y) }
- 44. How to changeenvironment variables?
- 45. .pr-rc.yaml (ProductPage) func CreateDuplicateDeployment(deployv1.Deployment) { // Get .pr-rc.yaml file for branch prRCYaml := github.GetPrRCYaml() // Override environmnet variable for duplicate deployment duplicateDeploy := mergeDeploy(deploy, prRCYaml) // Create Duplicate Deployment clientset.AppsV1().Deployments().Create(duplicateDeplo y) } # .pr-rc.yaml apiVersion: apps/v1 kind: Deployment spec: replicas: 1 template: spec: containers: - name: product-page image: product-page:pr-{PR_NUM}-{COMM_REF} env: - name: "REVIEW_URL" value: "review- 123.default.svc.cluster.local"
- 46. Future Work ● UseCRDs to manage desired state instead of pull requests, so that developers do not need to create pull request of downstream repository ● Allow QA to create complicated environment through UI
- 47. References which canhelp in writing controllers ● https://www.youtube.com/watch?v=_BuqPMlXfpE ● https://engineering.bitnami.com/articles/kubewatch-an-example-of-kubernetes-custom- controller.html ● https://borismattijssen.github.io/articles/kubernetes-informers-controllers-reectors-stores ● https://github.com/kubernetes/sample-controller ● https://github.com/kubernetes/community/blob/master/contributors/devel/controllers.m ● https://github.com/kubernetes/client-go/tree/master/tools/cache