DEV Community

Cover image for osm-edge: Using access control policies to access services with the service mesh
Flomesh profile image Ali Naqvi
Ali Naqvi for Flomesh

Posted on

osm-edge: Using access control policies to access services with the service mesh

#servicemesh #accesscontrol #osmedge #pipyproxy

Deploying a service mesh in a complex brownfield environment is a lengthy and gradual process requiring upfront planning, or there may exist use cases where you have a specific set of services that either aren't yet ready for migration or for some reason can not be migrated to service mesh.

This blog post will talk about the approaches which can be used to enable services outside of the service mesh to communicate with services within the osm-edge service mesh.

osm-edge forked from Open Service Mesh is a lightweight, extensible, cloud-native, SMI-compatible service mesh built purposely for Edge computing. osm-edge uses lightweight programmable proxy Pipy as a sidecar proxy.

Accessing services within service mesh

osm-edge offers two ways to allow accessing services within the service mesh:

  • via Ingress
    • FSM Ingress controller
    • Nginx Ingress controller
  • Access Control
    • Service
    • IPRange

The first method to access the services in the service mesh is via Ingress controller, and treat the services outside the mesh as the services inside the cluster. The advantage of this approach is that the setup is simple and straightforward and the disadvantages are also apparent, as you cannot achieve fine-grained access control, and all services outside the mesh can access services within the mesh.

This article will focus on the second approach, which allows support for fine-grained access control on who can access services within the service mesh. This feature is newly added and available in release 1.2.0 osm-edge v1.2.0.

Access Control can be configured via two resource types: Service and IP range. In terms of data transmission, it supports plaintext transmission and mTLS-encrypted traffic.

Let's get started with a demo.

Demo environment preparation

Kubernetes cluster

Using the minimalist Kubernetes distribution k8e:

curl -sfL https://getk8e.com/install.sh | K8E_TOKEN=k8e-mesh INSTALL_K8E_EXEC="server --cluster-init --write-kubeconfig-mode 644 --write-kubeconfig ~/.kube/config" sh -
Enter fullscreen mode Exit fullscreen mode

osm-edge CLI 

system=$(uname -s | tr [:upper:] [:lower:]) arch=$(dpkg --print-architecture) release=v1.2.0 curl -L https://github.com/flomesh-io/osm-edge/releases/download/${release}/osm-edge-${release}-${system}-${arch}.tar.gz | tar -vxzf - ./${system}-${arch}/osm version cp ./${system}-${arch}/osm /usr/local/bin/
Enter fullscreen mode Exit fullscreen mode

Install osm-edge

Execute the following commands to install the related components of osm-edge.

export osm_namespace=osm-system export osm_mesh_name=osm osm install \ --mesh-name "$osm_mesh_name" \ --osm-namespace "$osm_namespace" \ --set=osm.image.pullPolicy=Always
Enter fullscreen mode Exit fullscreen mode

Check to make sure all pods are up and running properly.

Deploy the sample application 

#Mock target service kubectl create namespace httpbin osm namespace add httpbin kubectl apply -n httpbin -f https://raw.githubusercontent.com/flomesh-io/osm-edge-docs/main/manifests/samples/httpbin/httpbin.yaml #Mock external service kubectl create namespace curl kubectl apply -n curl -f https://raw.githubusercontent.com/flomesh-io/osm-edge-docs/main/manifests/samples/curl/curl.yaml #Wait for the dependent POD to start normally kubectl wait --for=condition=ready pod -n httpbin -l app=httpbin --timeout=180s kubectl wait --for=condition=ready pod -n curl -l app=curl --timeout=180s
Enter fullscreen mode Exit fullscreen mode

Demo

At this point, we send a request from service curl to target service httpbin by executing the following command.

kubectl exec "$(kubectl get pod -n curl -l app=curl -o jsonpath='{.items..metadata.name}')" -n curl -- curl -sI http://httpbin.httpbin:14001/get command terminated with exit code 56
Enter fullscreen mode Exit fullscreen mode

The access fails because by default the services outside the mesh cannot access the services inside the mesh and we need to apply an access control policy.

Before applying the policy, you need to enable the access control feature, which is disabled by default.

kubectl patch meshconfig osm-mesh-config -n "$osm_namespace" -p '{"spec":{"featureFlags":{"enableAccessControlPolicy":true}}}' --type= merge
Enter fullscreen mode Exit fullscreen mode

Plaintext transfer

Data can be transferred in plaintext or with two-way TLS encryption. Plaintext transfer is relatively simple, so let's demonstrate the plaintext transfer scenario first.

Service-based access control

First, create a Service for service curl.

kubectl apply -n curl -f - <<EOF apiVersion: v1 kind: Service metadata: name: curl labels: app: curl service: curl spec: ports: - name: http port: 80 selector: app: curl EOF
Enter fullscreen mode Exit fullscreen mode

Next, create an access control policy with the source Service curl and the target service httpbin.

kubectl apply -f - <<EOF kind: AccessControl apiVersion: policy.openservicemesh.io/v1alpha1 metadata: name: httpbin namespace: httpbin spec: backends: - name: httpbin port: number: 14001 # targetPort of httpbin service protocol: http sources: - kind: Service namespace: curl name: curl EOF
Enter fullscreen mode Exit fullscreen mode

Execute the command again to send the authentication request, and you can see that this time an HTTP 200 response is received.

kubectl exec "$(kubectl get pod -n curl -l app=curl -o jsonpath='{.items..metadata.name}')" -n curl -- curl -sI http://httpbin.httpbin:14001/get HTTP/1.1 200 OK server: gunicorn/19.9.0 date: Mon, 07 Nov 2022 08:47:55 GMT content-type: application/json content-length: 267 access-control-allow-origin: * access-control-allow-credentials: true osm-stats-namespace: httpbin osm-stats-kind: Deployment osm-stats-name: httpbin osm-stats-pod: httpbin-69dc7d545c-qphrh connection: keep-alive
Enter fullscreen mode Exit fullscreen mode

Before continuing with the rest of the demonstration, execute the command kubectl delete accesscontrol httpbin -n httpbin to delete the policy you just created.

IP range-based access control, plaintext transfer

First, get the pod IP address of the service curl.

curl_pod_ip="$(kubectl get pod -n curl -l app=curl -o jsonpath='{.items[0].status.podIP}')"
Enter fullscreen mode Exit fullscreen mode

Access control using IP ranges is simple, just set the access source type to IPRange and configure the IP address you just obtained.

kubectl apply -f - <<EOF kind: AccessControl apiVersion: policy.openservicemesh.io/v1alpha1 metadata: name: httpbin namespace: httpbin spec: backends: - name: httpbin port: number: 14001 # targetPort of httpbin service protocol: http sources: - kind: IPRange name: ${curl_pod_ip}/32 EOF
Enter fullscreen mode Exit fullscreen mode

Execute the command again to test if the control policy is in effect.

kubectl exec "$(kubectl get pod -n curl -l app=curl -o jsonpath='{.items..metadata.name}')" -n curl -- curl -sI http://httpbin.httpbin:14001/get HTTP/1.1 200 OK server: gunicorn/19.9.0 date: Mon, 07 Nov 2022 09:20:57 GMT content-type: application/json content-length: 267 access-control-allow-origin: * access-control-allow-credentials: true osm-stats-namespace: httpbin osm-stats-kind: Deployment osm-stats-name: httpbin osm-stats-pod: httpbin-69dc7d545c-qphrh connection: keep-alive
Enter fullscreen mode Exit fullscreen mode

Remember to execute kubectl delete accesscontrol httpbin -n httpbin to clean up the policy.

The previous ones we used were plaintext transfers, next we look at encrypted transfers.

Encrypted transfers

The default access policy certificate feature is off, turn it on by executing the following command.

kubectl patch meshconfig osm-mesh-config -n "$osm_namespace" -p '{"spec":{"featureFlags":{"enableAccessCertPolicy":true}}}' --type=merge
Enter fullscreen mode Exit fullscreen mode

Create AccessCert for the access source to assign a certificate for data encryption. The controller will store the certificate information in Secret curl-mtls-secret under the namespace curl, and here also assign SAN curl.curl.cluster.local for the access source.

kubectl apply -f - <<EOF kind: AccessCert apiVersion: policy.openservicemesh.io/v1alpha1 metadata: name: curl-mtls-cert namespace: httpbin spec: subjectAltNames: - curl.curl.cluster.local secret: name: curl-mtls-secret namespace: curl EOF
Enter fullscreen mode Exit fullscreen mode

Redeploy curl and mount the system-assigned Secret to the pod.

kubectl apply -n curl -f - <<EOF apiVersion: apps/v1 kind: Deployment metadata: name: curl spec: replicas: 1 selector: matchLabels: app: curl template: metadata: labels: app: curl spec: serviceAccountName: curl nodeSelector: kubernetes.io/os: linux containers: - image: curlimages/curl imagePullPolicy: IfNotPresent name: curl command: ["sleep", "365d"] volumeMounts: - name: curl-mtls-secret mountPath: "/certs" readOnly: true volumes: - name: curl-mtls-secret secret: secretName: curl-mtls-secret EOF
Enter fullscreen mode Exit fullscreen mode

Service-based Access Control

The next step is to create an access control policy that uses encrypted transport. When configuring the target service, enable client certificate checking by specifying tls.skipClientCertValidation = false. For the access source, in addition to the Service type of access source, the SAN curl.curl.cluster.local should be specified via AuthenticatedPrincipal.

kubectl apply -f - <<EOF kind: AccessControl apiVersion: policy.openservicemesh.io/v1alpha1 metadata: name: httpbin namespace: httpbin spec: backends: - name: httpbin port: number: 14001 # targetPort of httpbin service protocol: http tls: skipClientCertValidation: false sources: - kind: Service namespace: curl name: curl - kind: AuthenticatedPrincipal name: curl.curl.cluster.local EOF
Enter fullscreen mode Exit fullscreen mode

Test whether the access policy is effective. When sending the request, we have to specify the CA certificate, key and certificate to be used for the curl command of the access source, which will be responded to by HTTP 200 normally.

kubectl exec "$(kubectl get pod -n curl -l app=curl -o jsonpath='{.items..metadata.name}')" -n curl -- curl -ksI https://httpbin.httpbin:14001/get --cacert /certs/ca.crt --key /certs/tls.key --cert /certs/tls.crt HTTP/2 200 server: gunicorn/19.9.0 date: Mon, 07 Nov 2022 10:44:05 GMT content-type: application/json content-length: 267 access-control-allow-origin: * access-control-allow-credentials: true osm-stats-namespace: httpbin osm-stats-kind: Deployment osm-stats-name: httpbin osm-stats-pod: httpbin-69dc7d545c-qphrh
Enter fullscreen mode Exit fullscreen mode

IP Range-Based Access Control

After service-based access control, IP range-based control is simple. You just need to specify the type of access source as IPRange and specify the IP address of the access source. As the application curl is redeployed, its IP address needs to be retrieved (perhaps you have discovered the drawbacks of IP range-based access control).

curl_pod_ip="$(kubectl get pod -n curl -l app=curl -o jsonpath='{.items[0].status.podIP}')"
Enter fullscreen mode Exit fullscreen mode 
kubectl apply -f - <<EOF kind: AccessControl apiVersion: policy.openservicemesh.io/v1alpha1 metadata: name: httpbin namespace: httpbin spec: backends: - name: httpbin port: number: 14001 # targetPort of httpbin service protocol: http tls: skipClientCertValidation: false sources: - kind: IPRange name: ${curl_pod_ip}/32 - kind: AuthenticatedPrincipal name: curl.curl.cluster.local EOF
Enter fullscreen mode Exit fullscreen mode

Send the request again for testing.

kubectl exec "$(kubectl get pod -n curl -l app=curl -o jsonpath='{.items..metadata.name}')" -n curl -- curl -ksI https://httpbin.httpbin:14001/get --cacert /certs/ca.crt --key /certs/tls.key --cert /certs/tls.crt HTTP/2 200 server: gunicorn/19.9.0 date: Mon, 07 Nov 2022 10:58:55 GMT content-type: application/json content-length: 267 access-control-allow-origin: * access-control-allow-credentials: true osm-stats-namespace: httpbin osm-stats-kind: Deployment osm-stats-name: httpbin osm-stats-pod: httpbin-69dc7d545c-qphrh
Enter fullscreen mode Exit fullscreen mode

Bingo! The access is successful, indicating that our policy has taken effect

Summary

This blog post will talk about the approaches which can be used to enable services outside of the service mesh to communicate with services within the osm-edge service mesh. There may exist use cases where you have a specific set of services that aren't yet ready for migration to service mesh or for some reason can't be migrated yet.

The access control approaches presented in this article are suitable for solving such problems, and allow you to choose the appropriate type of access source and whether to transfer data in plaintext or encrypted, depending on your needs. The control is more granular than using a unified Ingress for access.

Top comments (0)