kube-arangodb/docs/metrics.md

Metrics collection

Operator provides metrics of its operations in a format supported by Prometheus.

The metrics are exposed through HTTPS on port 8528 under path /metrics.

For a full list of available metrics, see here.

Check out examples directory examples/metrics for Services and ServiceMonitors definitions you can use to integrate with Prometheus through the Prometheus-Operator by CoreOS.

Contents

• Integration with standard Prometheus installation (no TLS)
• Integration with standard Prometheus installation (TLS)
• Integration with Prometheus Operator
• Exposing ArangoDB metrics

Integration with standard Prometheus installation (no TLS)

After creating operator deployment, you must configure Prometheus using a configuration file that instructs it about which targets to scrape. To do so, add a new scrape job to your prometheus.yaml config:

scrape_configs:
  - job_name:       'arangodb-operator'

    scrape_interval: 10s # scrape every 10 seconds.

    scheme: 'https'
    tls_config:
      insecure_skip_verify: true

    static_configs:
      - targets:
          - "<operator-endpoint-ip>:8528"

Integration with standard Prometheus installation (TLS)

By default, the operator uses self-signed certificate for its server API. To use your own certificate, you need to create k8s secret containing certificate and provide secret name to operator.

Create k8s secret (in same namespace where the operator is running):

kubectl create secret tls my-own-certificate --cert ./cert.crt  --key ./cert.key

Then edit the operator deployment definition (kubectl edit deployments.apps) to use your secret for its server API:

spec:
  # ...
  containers:
  # ...
    args:
    - --server.tls-secret-name=my-own-certificate
    # ...

Wait for operator pods to restart.

Now update Prometheus config to use your certificate for operator scrape job:

tls_config:
  # if you are using self-signed certificate, just specify CA certificate:
  ca_file: /etc/prometheus/rootCA.crt
  
  # otherwise, specify the generated client certificate and key:
  cert_file: /etc/prometheus/cert.crt
  key_file: /etc/prometheus/cert.key

Integration with Prometheus Operator

Assuming that you have Prometheus Operator installed in your cluster (monitoring namespace), and kube-arangodb installed in default namespace, you can easily configure the integration with ArangoDB operator.

The easiest way to do that is to create new a ServiceMonitor:

apiVersion: monitoring.coreos.com/v1
kind: ServiceMonitor
metadata:
  name: arango-deployment-operator
  namespace: monitoring
  labels:
    prometheus: kube-prometheus
spec:
  selector:
    matchLabels:
      app.kubernetes.io/name: kube-arangodb
  namespaceSelector:
    matchNames:
    - default
  endpoints:
  - port: server
    scheme: https
    tlsConfig:
      insecureSkipVerify: true

You also can see the example of Grafana dashboard at examples/metrics folder of this repo.

ArangoDB metrics

The operator can run sidecar containers for ArangoDB deployments of type Cluster which expose metrics in Prometheus format. Edit your ArangoDeployment resource, setting spec.metrics.enabled to true to enable ArangoDB metrics:

spec:
  metrics:
    enabled: true

The operator will run a sidecar container for every cluster component. In addition to the sidecar containers the operator will deploy a Service to access the exporter ports (from within the k8s cluster), and a resource of type ServiceMonitor, provided the corresponding custom resource definition is deployed in the k8s cluster. If you are running Prometheus in the same k8s cluster with the Prometheus operator, this will be the case. The ServiceMonitor will have the following labels set:

app: arangodb
arango_deployment: YOUR_DEPLOYMENT_NAME
context: metrics
metrics: prometheus

This makes it possible to configure your Prometheus deployment to automatically start monitoring on the available Prometheus feeds. To this end, you must configure the serviceMonitorSelector in the specs of your Prometheus deployment to match these labels. For example:

serviceMonitorSelector:
    matchLabels:
      metrics: prometheus

would automatically select all pods of all ArangoDB cluster deployments which have metrics enabled.

By default, the sidecar metrics exporters are using TLS for all connections. You can disable the TLS by specifying

spec:
  metrics:
    enabled: true
    tls: false

You can fine-tune the monitored metrics by specifying ArangoDeployment annotations. Example:

spec:
  annotations:
    prometheus.io/scrape: 'true'
    prometheus.io/port: '9101'
    prometheus.io/scrape_interval: '5s'

See the Metrics HTTP API documentation for the metrics exposed by ArangoDB deployments.