Monitoring
Monitoring IBM Verify Identity Access OIDC Provider
IBM Verify Identity Access OIDC provider supports Instana, Dynatrace, and Prometheus to monitor the performance of some application runtime components.
Instana
- CPU
- Memory
- Response time for individual HTTP calls
Dynatrace
- CPU
- Memory
- Response time for individual HTTP calls
- Detailed SQL queries for each HTTP call using OpenTelemetry libraries for PostgreSQL database
Prometheus
- http_requests_total
- response_status
- node_memory_usage_bytes
- http_response_time_seconds
- Go lang based metrics
Configuration for Instana
- The Instana agent can be installed on a Kubernetes platform by using the instructions provided here
- A helm chart example
helm install instana-agent \
--repo https://agents.instana.io/helm \
--namespace instana-agent \
--create-namespace \
--set agent.key=QHAvLwgRSH11111zGGGnTA \
--set agent.downloadKey=QHAvLwgRSH11111zGGGnTA \
--set agent.endpointHost=ingress-test.instana.io \
--set agent.endpointPort=443 \
--set cluster.name='IVIAOP' \
--set zone.name='jp-tok' \
instana-agent
- To configure sending metrics to an Instana agent, some environmental variables must be set at the IVIAOP deployment.
- A Kubernetes environment example
env:
- name: INSTANA_ENDPOINT_URL
value: https://<INSTANA_ENDPOINT_URL>
- name: INSTANA_AGENT_KEY
value: uBp4GXpZQp11111XNcvInQ
- name: INSTANA_AGENT_HOST
value: 10.67.92.81
- name: INSTANA_ENABLEMENT
value: 'true'
Environment Variable | Description | |
---|---|---|
INSTANA_ENDPOINT_URL | Instana endpoint URL, it can be retrieved from the Instana tenant | |
INSTANA_AGENT_KEY | Instana agent key which can be retrieved from the tenant | |
INSTANA_AGENT_HOST | The IP address on the Instana agent running in the same Kubernetes cluster | |
INSTANA_ENABLEMENT | Boolean flag that control pushing metric to the Instana agent |
- After the connection is successful, the Kubernetes cluster and the HTTP calls can be monitored in the Instana dashboard.
Configuration for Dynatrace
- The Dynatrace One Agent can be installed on a Kubernetes platform by using the instructions provided here.
- A command example
$kubectl create namespace dynatrace
$kubectl apply -f https://github.com/Dynatrace/dynatrace-operator/releases/download/v1.0.0/kubernetes.yaml
$kubectl apply -f csi.yaml
$kubectl -n dynatrace wait pod --for=condition=ready --selector=app.kubernetes.io/name=dynatrace-operator,app.kubernetes.io/component=webhook --timeout=300s
$kubectl -n dynatrace create secret generic dynakube --from-literal="apiToken=<apiToken>" --from-literal="dataIngestToken=<dataIngestToken>"
$kubectl apply -f cloudnative.yaml
- Example of csi.yaml
---
# Source: dynatrace-operator/templates/Common/csi/priority-class.yaml
# Copyright 2021 Dynatrace LLC
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
# http://www.apache.org/licenses/LICENSE-2.0
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
kind: PriorityClass
apiVersion: scheduling.k8s.io/v1
metadata:
name: dynatrace-high-priority
value: 1000000
globalDefault: false
description: "This priority class is used for Dynatrace Components in order to make sure they are not evicted in favor of other pods"
---
# Source: dynatrace-operator/templates/Common/csi/serviceaccount-csi.yaml
# Copyright 2021 Dynatrace LLC
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
# http://www.apache.org/licenses/LICENSE-2.0
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
apiVersion: v1
kind: ServiceAccount
metadata:
name: dynatrace-oneagent-csi-driver
namespace: dynatrace
labels:
app.kubernetes.io/name: dynatrace-operator
app.kubernetes.io/version: "1.0.0"
app.kubernetes.io/component: csi-driver
---
# Source: dynatrace-operator/templates/Common/csi/clusterrole-csi.yaml
# Copyright 2021 Dynatrace LLC
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
# http://www.apache.org/licenses/LICENSE-2.0
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: dynatrace-oneagent-csi-driver
labels:
app.kubernetes.io/name: dynatrace-operator
app.kubernetes.io/version: "1.0.0"
app.kubernetes.io/component: csi-driver
rules:
- apiGroups:
- ""
resources:
- namespaces
verbs:
- get
- list
- watch
- apiGroups:
- ""
resources:
- events
verbs:
- list
- watch
- create
- update
- patch
- apiGroups:
- storage.k8s.io
resources:
- csinodes
verbs:
- get
- list
- watch
- apiGroups:
- ""
resources:
- nodes
verbs:
- get
- list
- watch
- apiGroups:
- ""
resources:
- pods
verbs:
- get
- list
- watch
---
# Source: dynatrace-operator/templates/Common/csi/clusterrole-csi.yaml
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: dynatrace-oneagent-csi-driver
labels:
app.kubernetes.io/name: dynatrace-operator
app.kubernetes.io/version: "1.0.0"
app.kubernetes.io/component: csi-driver
subjects:
- kind: ServiceAccount
name: dynatrace-oneagent-csi-driver
namespace: dynatrace
roleRef:
kind: ClusterRole
name: dynatrace-oneagent-csi-driver
apiGroup: rbac.authorization.k8s.io
---
# Source: dynatrace-operator/templates/Common/csi/role-csi.yaml
# Copyright 2021 Dynatrace LLC
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
# http://www.apache.org/licenses/LICENSE-2.0
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
kind: Role
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: dynatrace-oneagent-csi-driver
namespace: dynatrace
labels:
app.kubernetes.io/name: dynatrace-operator
app.kubernetes.io/version: "1.0.0"
app.kubernetes.io/component: csi-driver
rules:
- apiGroups:
- ""
resources:
- endpoints
verbs:
- get
- watch
- list
- delete
- update
- create
- apiGroups:
- coordination.k8s.io
resources:
- leases
verbs:
- get
- watch
- list
- delete
- update
- create
- apiGroups:
- dynatrace.com
resources:
- dynakubes
verbs:
- get
- list
- watch
- apiGroups:
- ""
resources:
- secrets
verbs:
- get
- list
- watch
- apiGroups:
- ""
resources:
- configmaps
verbs:
- get
- list
- watch
---
# Source: dynatrace-operator/templates/Common/csi/role-csi.yaml
kind: RoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: dynatrace-oneagent-csi-driver
namespace: dynatrace
labels:
app.kubernetes.io/name: dynatrace-operator
app.kubernetes.io/version: "1.0.0"
app.kubernetes.io/component: csi-driver
subjects:
- kind: ServiceAccount
name: dynatrace-oneagent-csi-driver
namespace: dynatrace
roleRef:
kind: Role
name: dynatrace-oneagent-csi-driver
apiGroup: rbac.authorization.k8s.io
---
# Source: dynatrace-operator/templates/Common/csi/daemonset.yaml
# Copyright 2021 Dynatrace LLC
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
# http://www.apache.org/licenses/LICENSE-2.0
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
apiVersion: apps/v1
kind: DaemonSet
metadata:
labels:
app.kubernetes.io/name: dynatrace-operator
app.kubernetes.io/version: "1.0.0"
app.kubernetes.io/component: csi-driver
name: dynatrace-oneagent-csi-driver
namespace: dynatrace
spec:
revisionHistoryLimit: 10
selector:
matchLabels:
internal.oneagent.dynatrace.com/app: csi-driver
internal.oneagent.dynatrace.com/component: csi-driver
template:
metadata:
annotations:
dynatrace.com/inject: "false"
kubectl.kubernetes.io/default-container: provisioner
cluster-autoscaler.kubernetes.io/enable-ds-eviction: "false"
labels:
app.kubernetes.io/name: dynatrace-operator
app.kubernetes.io/version: "1.0.0"
app.kubernetes.io/component: csi-driver
internal.oneagent.dynatrace.com/app: csi-driver
internal.oneagent.dynatrace.com/component: csi-driver
spec:
initContainers:
- name: csi-init
image: public.ecr.aws/dynatrace/dynatrace-operator:v1.0.0@sha256:f742a1856f61ad127e9130e44e20dfa4d9990d1d472cbd5ce815b3599205e2e3
imagePullPolicy: Always
args:
- csi-init
terminationMessagePath: /dev/termination-log
terminationMessagePolicy: File
resources:
limits:
cpu: 50m
memory: 100Mi
requests:
cpu: 50m
memory: 100Mi
securityContext:
allowPrivilegeEscalation: false
privileged: false
readOnlyRootFilesystem: true
runAsNonRoot: false
runAsUser: 0
seLinuxOptions:
level: s0
seccompProfile:
type: RuntimeDefault
volumeMounts:
- mountPath: /data
name: data-dir
containers:
# Used to receive/execute gRPC requests (NodePublishVolume/NodeUnpublishVolume) from kubelet to mount/unmount volumes for a pod
# - Needs access to the csi socket, needs to read/write to it, needs root permissions to do so.
# - Needs access to the filesystem of pods on the node, and mount stuff to it,needs to read/write to it, needs root permissions to do so
# - Needs access to a dedicated folder on the node to persist data, needs to read/write to it.
- name: server
image: public.ecr.aws/dynatrace/dynatrace-operator:v1.0.0@sha256:f742a1856f61ad127e9130e44e20dfa4d9990d1d472cbd5ce815b3599205e2e3
imagePullPolicy: Always
args:
- csi-server
- --endpoint=unix://csi/csi.sock
- --node-id=$(KUBE_NODE_NAME)
- --health-probe-bind-address=:10080
env:
- name: POD_NAMESPACE
valueFrom:
fieldRef:
apiVersion: v1
fieldPath: metadata.namespace
- name: KUBE_NODE_NAME
valueFrom:
fieldRef:
apiVersion: v1
fieldPath: spec.nodeName
livenessProbe:
failureThreshold: 3
httpGet:
path: /livez
port: livez
scheme: HTTP
initialDelaySeconds: 5
periodSeconds: 5
successThreshold: 1
timeoutSeconds: 1
ports:
- containerPort: 10080
name: livez
protocol: TCP
resources:
limits:
cpu: 50m
memory: 100Mi
requests:
cpu: 50m
memory: 100Mi
securityContext:
allowPrivilegeEscalation: true
privileged: true
readOnlyRootFilesystem: true
runAsNonRoot: false
runAsUser: 0
seLinuxOptions:
level: s0
seccompProfile:
type: RuntimeDefault
terminationMessagePath: /dev/termination-log
terminationMessagePolicy: File
volumeMounts:
- mountPath: /csi
name: plugin-dir
- mountPath: /var/data/kubelet/pods/
mountPropagation: Bidirectional
name: mountpoint-dir
- mountPath: /data
name: data-dir
mountPropagation: Bidirectional
- name: tmp-dir
mountPath: /tmp
- name: provisioner
image: public.ecr.aws/dynatrace/dynatrace-operator:v1.0.0@sha256:f742a1856f61ad127e9130e44e20dfa4d9990d1d472cbd5ce815b3599205e2e3
imagePullPolicy: Always
args:
- csi-provisioner
- --health-probe-bind-address=:10090
env:
- name: POD_NAMESPACE
valueFrom:
fieldRef:
apiVersion: v1
fieldPath: metadata.namespace
startupProbe:
exec:
command:
- /usr/local/bin/dynatrace-operator
- startup-probe
periodSeconds: 10
timeoutSeconds: 5
failureThreshold: 1
livenessProbe:
failureThreshold: 3
httpGet:
path: /livez
port: livez
scheme: HTTP
initialDelaySeconds: 5
periodSeconds: 5
successThreshold: 1
timeoutSeconds: 1
ports:
- containerPort: 10090
name: livez
protocol: TCP
resources:
limits:
cpu: 300m
memory: 100Mi
requests:
cpu: 300m
memory: 100Mi
securityContext:
allowPrivilegeEscalation: true
privileged: true
readOnlyRootFilesystem: true
runAsNonRoot: false
runAsUser: 0
seLinuxOptions:
level: s0
seccompProfile:
type: RuntimeDefault
terminationMessagePath: /dev/termination-log
terminationMessagePolicy: File
volumeMounts:
- mountPath: /data
name: data-dir
mountPropagation: Bidirectional
- mountPath: /tmp
name: tmp-dir
# Used to make a gRPC request (GetPluginInfo()) to the driver to get driver name and driver contain
# - Needs access to the csi socket, needs to read/write to it, needs root permissions to do so.
# Used for registering the driver with kubelet
# - Needs access to the registration socket, needs to read/write to it, needs root permissions to do so.
- name: registrar
image: public.ecr.aws/dynatrace/dynatrace-operator:v1.0.0@sha256:f742a1856f61ad127e9130e44e20dfa4d9990d1d472cbd5ce815b3599205e2e3
imagePullPolicy: Always
env:
- name: DRIVER_REG_SOCK_PATH
value: /var/data/kubelet/plugins/csi.oneagent.dynatrace.com/csi.sock
args:
- --csi-address=/csi/csi.sock
- --kubelet-registration-path=$(DRIVER_REG_SOCK_PATH)
command:
- csi-node-driver-registrar
resources:
limits:
cpu: 20m
memory: 30Mi
requests:
cpu: 20m
memory: 30Mi
securityContext:
privileged: false
readOnlyRootFilesystem: true
runAsNonRoot: false
runAsUser: 0
seccompProfile:
type: RuntimeDefault
terminationMessagePath: /dev/termination-log
terminationMessagePolicy: File
volumeMounts:
- mountPath: /csi
name: plugin-dir
- mountPath: /registration
name: registration-dir
- mountPath: /var/data/kubelet/plugins/csi.oneagent.dynatrace.com/
name: lockfile-dir
# Used to make a gRPC request (Probe()) to the driver to check if its running
# - Needs access to the csi socket, needs to read/write to it, needs root permissions to do so.
- name: liveness-probe
image: public.ecr.aws/dynatrace/dynatrace-operator:v1.0.0@sha256:f742a1856f61ad127e9130e44e20dfa4d9990d1d472cbd5ce815b3599205e2e3
imagePullPolicy: Always
args:
- --csi-address=/csi/csi.sock
- --health-port=9898
command:
- livenessprobe
resources:
limits:
cpu: 20m
memory: 30Mi
requests:
cpu: 20m
memory: 30Mi
terminationMessagePath: /dev/termination-log
terminationMessagePolicy: File
securityContext:
allowPrivilegeEscalation: false
privileged: false
readOnlyRootFilesystem: true
runAsNonRoot: false
runAsUser: 0
seccompProfile:
type: RuntimeDefault
volumeMounts:
- mountPath: /csi
name: plugin-dir
dnsPolicy: ClusterFirst
restartPolicy: Always
schedulerName: default-scheduler
securityContext: {}
serviceAccountName: dynatrace-oneagent-csi-driver
terminationGracePeriodSeconds: 30
priorityClassName: dynatrace-high-priority
volumes:
# This volume is where the registrar registers the plugin with kubelet
- name: registration-dir
hostPath:
path: /var/data/kubelet/plugins_registry/
type: Directory
# This volume is where the socket for kubelet->driver communication is done
- name: plugin-dir
hostPath:
path: /var/data/kubelet/plugins/csi.oneagent.dynatrace.com/
type: DirectoryOrCreate
- name: data-dir
hostPath:
path: /var/data/kubelet/plugins/csi.oneagent.dynatrace.com/data
type: DirectoryOrCreate
# This volume is where the driver mounts volumes
- name: mountpoint-dir
hostPath:
path: /var/data/kubelet/pods/
type: DirectoryOrCreate
# Used by the registrar to create its lockfile
- name: lockfile-dir
emptyDir: {}
# A volume for the driver to write temporary files to
- name: tmp-dir
emptyDir: {}
tolerations:
- effect: NoSchedule
key: node-role.kubernetes.io/master
operator: Exists
- effect: NoSchedule
key: node-role.kubernetes.io/control-plane
operator: Exists
- key: kubernetes.io/arch
value: arm64
effect: NoSchedule
- key: kubernetes.io/arch
value: amd64
effect: NoSchedule
- key: kubernetes.io/arch
value: ppc64le
effect: NoSchedule
- key: ToBeDeletedByClusterAutoscaler
operator: Exists
effect: NoSchedule
updateStrategy:
rollingUpdate:
maxUnavailable: 1
type: RollingUpdate
---
# Source: dynatrace-operator/templates/Common/csi/csidriver.yaml
# Copyright 2021 Dynatrace LLC
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
# http://www.apache.org/licenses/LICENSE-2.0
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
apiVersion: storage.k8s.io/v1
kind: CSIDriver
metadata:
name: csi.oneagent.dynatrace.com
labels:
app.kubernetes.io/name: dynatrace-operator
app.kubernetes.io/version: "1.0.0"
app.kubernetes.io/component: csi-driver
spec:
attachRequired: false
podInfoOnMount: true
volumeLifecycleModes:
- Ephemeral
- Example of cloudnative.yaml, replace the TENANT with the Dynatrace tenant information.
apiVersion: dynatrace.com/v1beta1
kind: DynaKube
metadata:
name: dynakube
namespace: dynatrace
spec:
# Dynatrace apiUrl including the `/api` path at the end.
# For SaaS, set `ENVIRONMENTID` to your environment ID.
# For Managed, change the apiUrl address.
# For instructions on how to determine the environment ID and how to configure the apiUrl address, see https://www.dynatrace.com/support/help/reference/dynatrace-concepts/environment-id/.
apiUrl: https://<TENANT>/api
# Optional: Name of the secret holding the credentials required to connect to the Dynatrace tenant
# If unset, the name of this custom resource is used
#
# tokens: ""
# Optional: Defines a custom pull secret in case you use a private registry when pulling images from the Dynatrace environment
# The secret has to be of type 'kubernetes.io/dockerconfigjson' (see https://kubernetes.io/docs/tasks/configure-pod-container/pull-image-private-registry/)
#
# customPullSecret: "custom-pull-secret"
# Optional: Disable certificate validation checks for installer download and API communication
#
# skipCertCheck: false
# Optional: Set custom proxy settings either directly or from a secret with the field 'proxy'
#
# proxy:
# value: my-proxy-url.com
# valueFrom: name-of-my-proxy-secret
# Optional: Adds custom RootCAs from a configmap
# The key to the data must be "certs"
# This property only affects certificates used to communicate with the Dynatrace API.
# The property is not applied to the ActiveGate
#
# trustedCAs: name-of-my-ca-configmap
# Optional: Sets Network Zone for OneAgent and ActiveGate pods
# Make sure networkZones are enabled on your cluster before (see https://www.dynatrace.com/support/help/setup-and-configuration/network-zones/network-zones-basic-info/)
#
# networkZone: name-of-my-network-zone
# Optional: If enabled, and if Istio is installed on the Kubernetes environment, the
# Operator will create the corresponding VirtualService and ServiceEntry objects to allow access
# to the Dynatrace cluster from agents or activeGates. Disabled by default.
#
# enableIstio: false
# The namespaces which should be injected into
# If unset, all namespace will be injected into
# namespaceSelector has no effect on hostMonitoring or classicFullstack
# For examples regarding namespaceSelectors, see https://kubernetes.io/docs/concepts/overview/working-with-objects/labels/#resources-that-support-set-based-requirements
#
# namespaceSelector:
# matchLabels:
# app: my-app
# matchExpressions:
# - key: app
# operator: In
# values: [my-frontend, my-backend, my-database]
# Configuration for OneAgent instances
#
oneAgent:
# Enables cloud-native fullstack monitoring and changes its settings
# Cannot be used in conjunction with classic fullstack monitoring, application-only monitoring or host monitoring
#
cloudNativeFullStack:
# Optional: Sets a node selector to control on which nodes the OneAgent will be deployed.
# For more information on node selectors, see https://kubernetes.io/docs/tasks/configure-pod-container/assign-pods-nodes/
#
# nodeSelector: {}
# Optional: Sets the priority class assigned to the OneAgent Pods. No class is set by default.
# For more information on priority classes, see https://kubernetes.io/docs/concepts/configuration/pod-priority-preemption/
#
# priorityClassName: priority-class
# Optional: Specifies tolerations to include with the OneAgent DaemonSet.
# For more information on tolerations, see https://kubernetes.io/docs/concepts/configuration/taint-and-toleration/
#
tolerations:
- effect: NoSchedule
key: node-role.kubernetes.io/master
operator: Exists
- effect: NoSchedule
key: node-role.kubernetes.io/control-plane
operator: Exists
# Optional: Adds resource settings for OneAgent container
# Consumption of the OneAgent heavily depends on the workload to monitor
# The values should be adjusted according to the workload
#
# oneAgentResources:
# requests:
# cpu: 100m
# memory: 512Mi
# limits:
# cpu: 300m
# memory: 1.5Gi
# Optional: Adds custom arguments to the OneAgent installer
# For a list of available options, see https://www.dynatrace.com/support/help/shortlink/linux-custom-installation
# For a list of the limitations for OneAgents in Docker, see https://www.dynatrace.com/support/help/shortlink/oneagent-docker#limitations
#
# args: []
# Optional: Adds custom environment variables to OneAgent pods
#
# env: []
# Optional: Enables or disables automatic updates of OneAgent pods
# By default, if a new version is available, the OneAgent pods are restarted to apply the update
# If set to "false", this behavior is disabled
# Defaults to "true"
#
# autoUpdate: true
# Optional: Sets the DNS Policy for OneAgent pods
# Defaults to "ClusterFirstWithHostNet"
# For more information on DNS policies, see https://kubernetes.io/docs/concepts/services-networking/dns-pod-service/#pod-s-dns-policy
#
# dnsPolicy: "ClusterFirstWithHostNet"
# Optional: Adds custom annotations to OneAgent pods
#
# annotations:
# custom: annotation
# Optional: Adds custom labels to OneAgent pods
# Can be used to structure workloads
#
# labels:
# custom: label
# Optional: Sets the URI for the image containing the OneAgent installer used by the DaemonSet
# Defaults to the latest OneAgent image on the tenant's registry
#
# image: ""
# Optional: If specified, indicates the OneAgent version to use
# Defaults to the configured version on your Dynatrace environment
# The version is expected to be provided in the semver format
# Example: {major.minor.release}, e.g., "1.200.0"
#
# version: ""
# Optional: Defines resources requests and limits for the initContainer
# See more: https://kubernetes.io/docs/concepts/configuration/manage-resources-containers
#
# initResources:
# requests:
# cpu: 100m
# memory: 512Mi
# limits:
# cpu: 300m
# memory: 1.5Gi
# Optional: The URI of the image that contains the codemodules specific OneAgent that will be injected into pods and applications.
# For an example of a Dockerfile creating such an image, see https://dt-url.net/operator-docker-samples
#
# codeModulesImage: ""
# Configuration for ActiveGate instances.
#
activeGate:
# Specifies which capabilities will be enabled on ActiveGate instances
# The following capabilities can be set:
# - routing
# - kubernetes-monitoring
# - metrics-ingest
# - dynatrace-api
#
capabilities:
- routing
- kubernetes-monitoring
- dynatrace-api
# Optional: Sets how many ActiveGate pods are spawned by the StatefulSet
# Defaults to "1"
#
# replicas: 1
# Optional: Sets the image used to deploy ActiveGate instances
# Defaults to the latest ActiveGate image on the tenant's registry
# Example: "ENVIRONMENTID.live.dynatrace.com/linux/activegate:latest"
#
# image: ""
# Recommended: Sets the activation group for ActiveGate instances
#
# group: ""
# Optional: Defines a custom properties file, the file contents can be provided either as a value in this yaml or as a reference to a secret.
# If a reference to a secret is used, then the file contents must be stored under the 'customProperties' key within the secret.
#
# customProperties:
# value: |
# [connectivity]
# networkZone=
# valueFrom: myCustomPropertiesConfigMap
# Optional: Specifies resource settings for ActiveGate instances
# Consumption of the ActiveGate heavily depends on the workload to monitor
# The values should be adjusted according to the workload
#
resources:
requests:
cpu: 500m
memory: 512Mi
limits:
cpu: 1000m
memory: 1.5Gi
# Optional: Sets a node selector to control on which nodes the ActiveGate will be deployed.
# For more information on node selectors, see https://kubernetes.io/docs/tasks/configure-pod-container/assign-pods-nodes/
#
# nodeSelector: {}
# Optional: Specifies tolerations to include with the ActiveGate StatefulSet.
# For more information on tolerations, see https://kubernetes.io/docs/concepts/configuration/taint-and-toleration/
#
# tolerations:
# - effect: NoSchedule
# key: node-role.kubernetes.io/master
# operator: Exists
# Optional: Adds custom labels to ActiveGate pods
# Can be used to structure workloads
#
# labels:
# custom: label
# Optional: Adds custom environment variables to ActiveGate pods
#
# env: []
# Optional: Specifies the name of a secret containing a TLS certificate, a TLS key and the TLS key's password to be used by ActiveGate instances
# If unset, a self-signed certificate is used
# The secret is expected to have the following key-value pairs
# server.p12: TLS certificate and TLS key pair in pkcs12 format
# password: passphrase to decrypt the TLS certificate and TLS key pair
#
# tlsSecretName: "my-tls-secret"
# Optional: Sets the DNS Policy for ActiveGate pods
# Defaults to "Default"
# For more information on DNS policies, see https://kubernetes.io/docs/concepts/services-networking/dns-pod-service/#pod-s-dns-policy
#
# dnsPolicy: "Default"
# Optional: Specifies the priority class to assign to the ActiveGate Pods
# No class is set by default
# For more information on priority classes, see https://kubernetes.io/docs/concepts/configuration/pod-priority-preemption/
#
# priorityClassName: priority-class
# Optional: Adds custom annotations to ActiveGate pods
#
# annotations:
# custom: annotation
# Optional: Adds TopologySpreadConstraints to the ActiveGate pods
# For more information on TopologySpreadConstraints, see https://kubernetes.io/docs/concepts/scheduling-eviction/topology-spread-constraints/
#
# topologySpreadConstraints: []
- Access tokens can be generated with the right scopes by using this documentation link.
- Enable OpenTelemetry for Go by using the following link
- Enable Attribute Capturing by using the following link
- After the previous steps are configured successfully, all the HTTP calls and the respective DB queries are logged in Dynatrace Distributed Traces.
Note
Instrumentation is added only for the PostgreSQL database library.
Configuration for Prometheus
- To enable /metrics endpoint, the following environmental variable must be set. Example https://:/metrics.
- A Kubernetes environment example
env:
- name: PROMETHEUS_ENABLED
value: 'true'
- The following counters, gauges and historgram will be exposed
- http_requests_total
- response_status
- node_memory_usage_bytes
- http_response_time_seconds
- Go lang based metrics
- In Kubernetes a prometheus operator can be deployed in a different namespace where the IVIAOP is deployed.
- Example deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: prometheus-server
namespace: prometheus
spec:
replicas: 1
selector:
matchLabels:
app: prometheus-server
template:
metadata:
labels:
app: prometheus-server
spec:
containers:
- name: prometheus
image: prom/prometheus
ports:
- containerPort: 9090
volumeMounts:
- name: config-volume
mountPath: /etc/prometheus
volumes:
- name: config-volume
configMap:
name: prometheus-server-conf
defaultMode: 420
- Example service.yaml
apiVersion: v1
kind: Service
metadata:
name: prometheus-service
namespace: prometheus
spec:
selector:
app: prometheus-server
ports:
- protocol: TCP
port: 80
targetPort: 9090
type: LoadBalancer
- Example of the promethues configuration file
apiVersion: v1
kind: ConfigMap
metadata:
name: prometheus-server-conf
namespace: prometheus
data:
prometheus.yml: |
global:
scrape_interval: 20s
evaluation_interval: 10s
scrape_configs:
- job_name: iviaop
metrics_path: /metrics
static_configs:
- targets:
- <IP>:<IVIAOP Port>
- To understand more about Kubernetes SD Configuration, (see )[https://prometheus.io/docs/prometheus/latest/configuration/configuration/#kubernetes_sd_config].
Updated 3 months ago