When you need to setup Grafana Private Data source Connect in Kubernetes you need to apply some tricks in order to make it work.
Here I will describe what I did in order to use it.
Used links:
I created my custom helm chart that contains next deployment.yaml:
apiVersion: apps/v1 kind: Deployment metadata: labels: app: {{ .Release.Name }} name: {{ .Release.Name }} name: {{ .Release.Name }} spec: replicas: {{ .Values.minReplicas }} selector: matchLabels: name: {{ .Release.Name }} strategy: rollingUpdate: maxSurge: 1 maxUnavailable: 0 type: RollingUpdate template: metadata: labels: name: {{ .Release.Name }} annotations: proxy.istio.io/config: | holdApplicationUntilProxyStarts: true spec: containers: - name: {{ .Release.Name }} env: - name: CLUSTER valueFrom: secretKeyRef: key: cluster name: {{ .Release.Name }} - name: HOSTED_GRAFANA_ID valueFrom: secretKeyRef: key: hostedGrafanaId name: {{ .Release.Name }} - name: TOKEN valueFrom: secretKeyRef: key: token name: {{ .Release.Name }} args: - -cluster - "$(CLUSTER)" - -gcloud-hosted-grafana-id - "$(HOSTED_GRAFANA_ID)" - -token - "$(TOKEN)" - -ssh-key-file - "/home/pdc/.ssh/grafana_pdc_v3" image: grafana/pdc-agent:{{ .Values.version }} imagePullPolicy: Always resources: limits: cpu: 1024m memory: 1Gi requests: cpu: 1024m memory: 1Gi securityContext: allowPrivilegeEscalation: false privileged: false runAsNonRoot: true capabilities: drop: - all securityContext: runAsUser: 30000 runAsGroup: 30000 fsGroup: 30000 topologySpreadConstraints: - labelSelector: matchLabels: app: {{ .Release.Name }} maxSkew: 1 minDomains: {{ .Values.minReplicas }} topologyKey: "kubernetes.io/hostname" whenUnsatisfiable: DoNotSchedule matchLabelKeys: - pod-template-hash nodeAffinityPolicy: Honor nodeTaintsPolicy: Honor - labelSelector: matchLabels: app: {{ .Release.Name }} maxSkew: 1 minDomains: {{ .Values.minReplicas }} topologyKey: "topology.kubernetes.io/zone" whenUnsatisfiable: DoNotSchedule matchLabelKeys: - pod-template-hash nodeAffinityPolicy: Honor nodeTaintsPolicy: Honor Nuances
Istio
Sidecar
Set holdApplicationUntilProxyStarts: true for the pods, so they will not start until istio sidecar not starts.
Access (optional)
If you not allow outbound traffic - set ServiceEntry that will allow several urls.
What I have for API access:
apiVersion: networking.istio.io/v1alpha3 kind: ServiceEntry metadata: name: {{ .Values.name }}-api spec: hosts: - private-datasource-connect-api-<cluster>.grafana.net location: MESH_EXTERNAL ports: - name: https number: 443 protocol: HTTPS resolution: DNS What I have for ssh access:
apiVersion: networking.istio.io/v1alpha3 kind: ServiceEntry metadata: name: {{ .Values.name }}-ssh spec: hosts: - private-datasource-connect-<cluster>.grafana.net location: MESH_EXTERNAL ports: - name: tcp number: 22 protocol: TCP resolution: DNS Grafana PDC config
Key Pair force regeneration
I set -ssh-key-file to /home/pdc/.ssh/grafana_pdc_v3 because if there already host in allowed list(for ssh access) - then it not starts and fail in constant restarts.
This should be addressed in GitHub issue
Log level
Currently, in PDC agent log level set to debug level.
Unfortunately, as of today, when you use -ssh-key-file parameter you cannot change it.
This should be addressed in GitHub issue
Top comments (0)