Management Service MQTT Messaging
Send kubectl
This section describes the MQTT messages and topics the Management Service handles to perform the "send-kubectl" action. When using AnythingDB in Studio all of this is taken care of within the product.
Sending a Message to the Management Service
The MQTT topic the Management Service subscribes to, for running kubectl commands
is:
spaces/<space>/things/<thing-id>/actions/send-kubectlThe message format for publishing is as follows:
{
"send-kubectl": {
"input": {
"correlationId": "{create-a-unique-id-for-each-message}",
"command": "kubectl",
"arguments": [
"{each}",
"{new}",
"{argument}",
"{needs}",
"{its}",
"{own}",
"{array}",
"{entry}"
]
},
"status": "pending",
"timeRequested": "<datetime>",
"href": "/spaces/<space>/things/<thing-id>/actions/send-kubectl/<action-id>"
}
}
For example:
{
"send-kubectl": {
"input": {
"correlationId": "666",
"command": "kubectl",
"arguments": [
"get",
"nodes",
"-o",
"wide"
]
},
"status": "pending",
"timeRequested": "2020-05-26 15:37:46+0000",
"href": "/spaces/my-space/things/my-thing-id/actions/send-kubectl/1234"
}
}
It is important that each argument is put in a new array entry. If there is a space in the complete command then that means a new argument.
Status Update After Receiving Action Request
MQTT topic the message is sent
to:
spaces/<space>/things/<thing-id>/actionsMessage:
{
"send-kubectl": {
"href": "/spaces/<space>/things/<thing-id>/actions/send-kubectl/<action-id>",
"status": "received"
}
}
Event Publish After Running Action
MQTT topic the message is sent
to:
spaces/<space>/things/<thing-id>/eventsMessage:
{
"kubectl-logs": {
"data": {
"correlationId": "{the-unique-id-from-the-request}",
"statusCode": "{an-http-status-code}",
"statusMessage": "{optional descriptive message}",
"response": "{the output in plain text format}"
}
}
}
Example:
{
"kubectl-logs": {
"data": {
"correlationId": "666",
"statusCode": 200,
"statusMessage": "Command executed successfully",
"response": "NAME STATUS ROLES AGE VERSION INTERNAL-IP EXTERNAL-IP OS-IMAGE KERNEL-VERSION CONTAINER-RUNTIME\nmartin-ag150 Ready master 50d v1.18.2+k3s1 10.10.10.100 \u003cnone\u003e Ubuntu 20.04 LTS 5.4.0-40-generic containerd://1.3.3-k3s2\n"
}
}
}
You need to use the correlationId to match the response to the
request.
Status Update After Completing Action
MQTT topic the message is sent
to:
spaces/<space>/things/<thing-id>/actionsMessage:
{
"send-kubectl": {
"href": "/spaces/<space>/things/<thing-id>/actions/send-kubectl/<action-id>",
"status": "completed"
}
}
Send kubernetes API
This section describes the MQTT messages and topics the Management Service handles to
perform the send-kubernetes-api action. When using AnythingDB in Studio all
of this is taken care of within the product.
Sending a Message to the Management Service
The MQTT topic the Management Service subscribes to, for running kubernetes API
requests
is:
spaces/<space>/things/<thing-id>/actions/send-kubernetes-apiThe message format for publishing is as follows:
{
"send-kubernetes-api": {
"input": {
"correlationId": "{create-a-unique-id-for-each-message}",
"requestMethod": "{GET|POST|PUT|PATCH|DELETE}",
"href": "{see API docs}",
"requestBody": "{body if required}"
},
"status": "pending",
"timeRequested": "<datetime>",
"href": "/spaces/<space>/things/<thing-id>/actions/send-kubernetes-api/<action-id>"
}
}The requestBody is a string. For a JSON requestBody
this means it has to be stringified first. This results in "double
JSON-encoding".
For example:
{
"send-kubernetes-api":{
"input":{
"correlationId": "666",
"requestMethod": "GET",
"href": "/api/v1/nodes",
"requestBody": ""
},
"status": "pending",
"timeRequested": "2020-05-26 15:37:46+0000",
"href": "/spaces/may-space/things/my-thing-id/actions/send-kubernetes-api/1234"
}
}
Status Update After Receiving Action Request
MQTT topic the message is sent
to:
spaces/<space>/things/<thing-id>/actionsMessage:
{
"send-kubernetes-api": {
"href": "/spaces/<space>/things/<thing-id>/actions/send-kubernetes-api/<action-id>",
"status": "received"
}
}Event Publish After Running Action
MQTT topic the message is sent
to:
spaces/<space>/things/<thing-id>/eventsMessage:
{
"send-kubernetes-api": {
"data": {
"correlationId": "{the-unique-id-from-the-request}",
"statusCode": "{an-http-status-code}",
"statusMessage": "{optional descriptive message}",
"response": "{the output of the API request}"
}
}
}Example:
{
"send-kubernetes-api": {
"data": {
"correlationId": "666",
"statusCode": 200,
"statusMessage": "Command executed successfully",
"response": ""
}
}
}You need to use the correlationId to match the response to the
request.
Status Update After Completing Action
MQTT topic the message is sent
to:
spaces/<space>/things/<thing-id>/actionsMessage:
{
"send-kubernetes-api": {
"href": "/spaces/<space>/things/<thing-id>/actions/send-kubernetes-api/<action-id>",
"status": "completed"
}
}
Metrics
This section describes the MQTT messages and topics the Management Service handles to perform the "run-stats" and "stop-stats" actions. When using AnythingDB in Studio all of this is taken care of within the product.
Metrics / Statistics Data
The Management Service publishes metrics for the nodes and deployments of the cluster at a regular interval (default: 30 seconds). This data is obtained from the Kubernetes Metrics Server which is installed by default in a K3s installation.
Start the Metrics / Update the Interval
The metrics publishing can be started, or the interval can be updated, by sending a
message to the following
topic:
status/<space>/things/<thing-id>/actions/run-statsThe message format for publishing is as follows:
{
"run-stats": {
"input": {
"interval": {in-seconds}
},
"status": "pending",
"timeRequested": "<datetime>",
"href": "<space>/things/<thing-id>/actions/run-stats/<action-id>"
}
}For example:
{
"run-stats": {
"input": {
"interval": 15
},
"status": "pending",
"timeRequested": "2020-06-18 10:32:46+0000",
"href": "martin/things/ag150/actions/run-stats/1234"
}
}Stop the Metrics
The metrics publishing can be stopped by sending a message to the following
topic:
status/<space>/things/<thing-id>/actions/stop-statsThe message format for publishing is as follows:
{
"stop-stats": {
"status": "pending",
"timeRequested": "<datetime>",
"href": "<space>/things/<thing-id>/actions/stop-stats/<action-id>"
}
}For example:
{
"stop-stats": {
"status": "pending",
"timeRequested": "2020-06-18 10:36:46+0000",
"href": "martin/things/ag150/actions/stop-stats/1234"
}
}Example of Metrics
{
"nodes": [
{
"capacities": {
"cpu": 4000000000,
"memory": 16608059392,
"pods": 110,
"storage": 105089261568
},
"conditions": [
{
"lastHeartbeatTime": 1724660414,
"lastTransitionTime": 1715072919,
"message": "kubelet has sufficient memory available",
"reason": "KubeletHasSufficientMemory",
"status": "False",
"type": "MemoryPressure"
},
{
"lastHeartbeatTime": 1724660414,
"lastTransitionTime": 1715072919,
"message": "kubelet has no disk pressure",
"reason": "KubeletHasNoDiskPressure",
"status": "False",
"type": "DiskPressure"
},
{
"lastHeartbeatTime": 1724660414,
"lastTransitionTime": 1715072919,
"message": "kubelet has sufficient PID available",
"reason": "KubeletHasSufficientPID",
"status": "False",
"type": "PIDPressure"
},
{
"lastHeartbeatTime": 1724660414,
"lastTransitionTime": 1715072919,
"message": "kubelet is posting ready status",
"reason": "KubeletReady",
"status": "True",
"type": "Ready"
}
],
"daemonsets": [
{
"name": "ase-support-cloud-storage",
"namespace": "ecp",
"pods": [
{
"containers": [
{
"name": "ase-support-cloud-storage",
"stats": {
"cpu": 265892,
"cpuPercentage": 0.0066473,
"memory": 9846784,
"memoryPercentage": 0.05928919067295205,
"timestamp": 1724660596,
"window": 11
}
}
],
"name": "ase-support-cloud-storage-pcr9t",
"namespace": "ecp"
}
],
"status": {
"currentNumberScheduled": 1,
"desiredNumberScheduled": 1,
"numberAvailable": 1,
"numberReady": 1,
"observedGeneration": 1,
"updatedNumberScheduled": 1
}
}
],
"deployments": [
{
"name": "ase-core-export-mqtt",
"namespace": "ecp",
"pods": [
{
"containers": [
{
"name": "ase-core-export-mqtt",
"stats": {
"cpu": 2021938,
"cpuPercentage": 0.05054845,
"memory": 33472512,
"memoryPercentage": 0.20154378792818806,
"timestamp": 1724660600,
"window": 12
}
}
],
"name": "ase-core-export-mqtt-844c67d4f7-878kz",
"namespace": "ecp"
}
],
"status": {
"availableReplicas": 1,
"conditions": [
{
"lastTransitionTime": 1715073169,
"lastUpdateTime": 1720074799,
"message": "ReplicaSet \"ase-core-export-mqtt-844c67d4f7\" has successfully progressed.",
"reason": "NewReplicaSetAvailable",
"status": "True",
"type": "Progressing"
},
{
"lastTransitionTime": 1723539614,
"lastUpdateTime": 1723539614,
"message": "Deployment has minimum availability.",
"reason": "MinimumReplicasAvailable",
"status": "True",
"type": "Available"
}
],
"observedGeneration": 3,
"readyReplicas": 1,
"replicas": 1,
"updatedReplicas": 1
}
},
{
"name": "ase-virtual-meter",
"namespace": "ecp",
"pods": [
{
"containers": [
{
"name": "ase-virtual-meter",
"stats": {
"cpu": 605035,
"cpuPercentage": 0.015125874999999999,
"memory": 7507968,
"memoryPercentage": 0.04520677475188066,
"timestamp": 1724660599,
"window": 17
}
}
],
"name": "ase-virtual-meter-5f487f7787-fltpl",
"namespace": "ecp"
}
],
"status": {
"availableReplicas": 1,
"conditions": [
{
"lastTransitionTime": 1715073162,
"lastUpdateTime": 1715073190,
"message": "ReplicaSet \"ase-virtual-meter-5f487f7787\" has successfully progressed.",
"reason": "NewReplicaSetAvailable",
"status": "True",
"type": "Progressing"
},
{
"lastTransitionTime": 1722360862,
"lastUpdateTime": 1722360862,
"message": "Deployment has minimum availability.",
"reason": "MinimumReplicasAvailable",
"status": "True",
"type": "Available"
}
],
"observedGeneration": 1,
"readyReplicas": 1,
"replicas": 1,
"updatedReplicas": 1
}
},
{
"name": "ase-cluster-upgrade",
"namespace": "ecp",
"pods": [
{
"containers": [
{
"name": "ase-cluster-upgrade",
"stats": {
"cpu": 0,
"memory": 12972032,
"memoryPercentage": 0.07810684977588982,
"timestamp": 1724660597,
"window": 29
}
}
],
"name": "ase-cluster-upgrade-765b94587c-w2cfc",
"namespace": "ecp"
}
],
"status": {
"availableReplicas": 1,
"conditions": [
{
"lastTransitionTime": 1715073165,
"lastUpdateTime": 1715073173,
"message": "ReplicaSet \"ase-cluster-upgrade-765b94587c\" has successfully progressed.",
"reason": "NewReplicaSetAvailable",
"status": "True",
"type": "Progressing"
},
{
"lastTransitionTime": 1722360558,
"lastUpdateTime": 1722360558,
"message": "Deployment has minimum availability.",
"reason": "MinimumReplicasAvailable",
"status": "True",
"type": "Available"
}
],
"observedGeneration": 1,
"readyReplicas": 1,
"replicas": 1,
"updatedReplicas": 1
}
},
{
"name": "ase-core-updater",
"namespace": "ecp",
"pods": [
{
"containers": [
{
"name": "ase-core-updater",
"stats": {
"cpu": 115982,
"cpuPercentage": 0.00289955,
"memory": 9703424,
"memoryPercentage": 0.058425995301257645,
"timestamp": 1724660600,
"window": 14
}
}
],
"name": "ase-core-updater-55b47fbb74-5txhn",
"namespace": "ecp"
}
],
"status": {
"availableReplicas": 1,
"conditions": [
{
"lastTransitionTime": 1723507248,
"lastUpdateTime": 1723507248,
"message": "Deployment has minimum availability.",
"reason": "MinimumReplicasAvailable",
"status": "True",
"type": "Available"
},
{
"lastTransitionTime": 1715073143,
"lastUpdateTime": 1724284830,
"message": "ReplicaSet \"ase-core-updater-55b47fbb74\" has successfully progressed.",
"reason": "NewReplicaSetAvailable",
"status": "True",
"type": "Progressing"
}
],
"observedGeneration": 8,
"readyReplicas": 1,
"replicas": 1,
"updatedReplicas": 1
}
},
{
"name": "ase-core-cloud",
"namespace": "ecp",
"pods": [
{
"containers": [
{
"name": "ase-core-cloud",
"stats": {
"cpu": 2493175,
"cpuPercentage": 0.06232937499999999,
"memory": 21626880,
"memoryPercentage": 0.1302191875013256,
"timestamp": 1724660598,
"window": 11
}
}
],
"name": "ase-core-cloud-6bf4dcbf7b-nbdpr",
"namespace": "ecp"
}
],
"status": {
"availableReplicas": 1,
"conditions": [
{
"lastTransitionTime": 1715073169,
"lastUpdateTime": 1720074798,
"message": "ReplicaSet \"ase-core-cloud-6bf4dcbf7b\" has successfully progressed.",
"reason": "NewReplicaSetAvailable",
"status": "True",
"type": "Progressing"
},
{
"lastTransitionTime": 1723746270,
"lastUpdateTime": 1723746270,
"message": "Deployment has minimum availability.",
"reason": "MinimumReplicasAvailable",
"status": "True",
"type": "Available"
}
],
"observedGeneration": 3,
"readyReplicas": 1,
"replicas": 1,
"updatedReplicas": 1
}
},
{
"name": "ase-core-meta",
"namespace": "ecp",
"pods": [
{
"containers": [
{
"name": "ase-core-meta",
"stats": {
"cpu": 3273851,
"cpuPercentage": 0.081846275,
"memory": 18644992,
"memoryPercentage": 0.11226472377008224,
"timestamp": 1724660609,
"window": 14
}
}
],
"name": "ase-core-meta-648f5bb74c-zrjbs",
"namespace": "ecp"
}
],
"status": {
"availableReplicas": 1,
"conditions": [
{
"lastTransitionTime": 1715073169,
"lastUpdateTime": 1720074800,
"message": "ReplicaSet \"ase-core-meta-648f5bb74c\" has successfully progressed.",
"reason": "NewReplicaSetAvailable",
"status": "True",
"type": "Progressing"
},
{
"lastTransitionTime": 1722360937,
"lastUpdateTime": 1722360937,
"message": "Deployment has minimum availability.",
"reason": "MinimumReplicasAvailable",
"status": "True",
"type": "Available"
}
],
"observedGeneration": 3,
"readyReplicas": 1,
"replicas": 1,
"updatedReplicas": 1
}
},
{
"name": "ase-core-cache",
"namespace": "ecp",
"pods": [
{
"containers": [
{
"name": "ase-core-cache",
"stats": {
"cpu": 2071880,
"cpuPercentage": 0.051796999999999996,
"memory": 4472832,
"memoryPercentage": 0.026931695596865073,
"timestamp": 1724660602,
"window": 13
}
}
],
"name": "ase-core-cache-76dbfbb696-hdflc",
"namespace": "ecp"
}
],
"status": {
"availableReplicas": 1,
"conditions": [
{
"lastTransitionTime": 1715073169,
"lastUpdateTime": 1715073180,
"message": "ReplicaSet \"ase-core-cache-76dbfbb696\" has successfully progressed.",
"reason": "NewReplicaSetAvailable",
"status": "True",
"type": "Progressing"
},
{
"lastTransitionTime": 1722360587,
"lastUpdateTime": 1722360587,
"message": "Deployment has minimum availability.",
"reason": "MinimumReplicasAvailable",
"status": "True",
"type": "Available"
}
],
"observedGeneration": 1,
"readyReplicas": 1,
"replicas": 1,
"updatedReplicas": 1
}
},
{
"name": "ase-core-health",
"namespace": "ecp",
"pods": [
{
"containers": [
{
"name": "ase-core-health",
"stats": {
"cpu": 886369,
"cpuPercentage": 0.022159225,
"memory": 17387520,
"memoryPercentage": 0.10469326722407712,
"timestamp": 1724660604,
"window": 19
}
}
],
"name": "ase-core-health-797fdf9d77-gmgb9",
"namespace": "ecp"
}
],
"status": {
"availableReplicas": 1,
"conditions": [
{
"lastTransitionTime": 1715073169,
"lastUpdateTime": 1720074789,
"message": "ReplicaSet \"ase-core-health-797fdf9d77\" has successfully progressed.",
"reason": "NewReplicaSetAvailable",
"status": "True",
"type": "Progressing"
},
{
"lastTransitionTime": 1722360937,
"lastUpdateTime": 1722360937,
"message": "Deployment has minimum availability.",
"reason": "MinimumReplicasAvailable",
"status": "True",
"type": "Available"
}
],
"observedGeneration": 3,
"readyReplicas": 1,
"replicas": 1,
"updatedReplicas": 1
}
},
{
"name": "ase-core-management",
"namespace": "ecp",
"pods": [
{
"containers": [
{
"name": "ase-core-management",
"stats": {
"cpu": 172030,
"cpuPercentage": 0.00430075,
"memory": 14680064,
"memoryPercentage": 0.08839120606150588,
"timestamp": 1724660601,
"window": 11
}
}
],
"name": "ase-core-management-587f89b769-vk8h4",
"namespace": "ecp"
}
],
"status": {
"availableReplicas": 1,
"conditions": [
{
"lastTransitionTime": 1722360557,
"lastUpdateTime": 1722360557,
"message": "Deployment has minimum availability.",
"reason": "MinimumReplicasAvailable",
"status": "True",
"type": "Available"
},
{
"lastTransitionTime": 1715073144,
"lastUpdateTime": 1724284825,
"message": "ReplicaSet \"ase-core-management-587f89b769\" has successfully progressed.",
"reason": "NewReplicaSetAvailable",
"status": "True",
"type": "Progressing"
}
],
"observedGeneration": 4,
"readyReplicas": 1,
"replicas": 1,
"updatedReplicas": 1
}
},
{
"name": "ase-core-message",
"namespace": "ecp",
"pods": [
{
"containers": [
{
"name": "ase-core-message",
"stats": {
"cpu": 13169991,
"cpuPercentage": 0.329249775,
"memory": 141828096,
"memoryPercentage": 0.853971512579716,
"timestamp": 1724660602,
"window": 13
}
}
],
"name": "ase-core-message-849668cf76-55l48",
"namespace": "ecp"
}
],
"status": {
"availableReplicas": 1,
"conditions": [
{
"lastTransitionTime": 1715073169,
"lastUpdateTime": 1715073223,
"message": "ReplicaSet \"ase-core-message-849668cf76\" has successfully progressed.",
"reason": "NewReplicaSetAvailable",
"status": "True",
"type": "Progressing"
},
{
"lastTransitionTime": 1722360926,
"lastUpdateTime": 1722360926,
"message": "Deployment has minimum availability.",
"reason": "MinimumReplicasAvailable",
"status": "True",
"type": "Available"
}
],
"observedGeneration": 1,
"readyReplicas": 1,
"replicas": 1,
"updatedReplicas": 1
}
},
{
"name": "ase-support-iot-studio",
"namespace": "ecp",
"pods": [
{
"containers": [
{
"name": "ase-support-iot-studio",
"stats": {
"cpu": 2701762,
"cpuPercentage": 0.06754405,
"memory": 21422080,
"memoryPercentage": 0.12898605125604792,
"timestamp": 1724660601,
"window": 13
}
}
],
"name": "ase-support-iot-studio-658f7d8f95-56t5x",
"namespace": "ecp"
}
],
"status": {
"availableReplicas": 1,
"conditions": [
{
"lastTransitionTime": 1715073164,
"lastUpdateTime": 1715073251,
"message": "ReplicaSet \"ase-support-iot-studio-658f7d8f95\" has successfully progressed.",
"reason": "NewReplicaSetAvailable",
"status": "True",
"type": "Progressing"
},
{
"lastTransitionTime": 1722360967,
"lastUpdateTime": 1722360967,
"message": "Deployment has minimum availability.",
"reason": "MinimumReplicasAvailable",
"status": "True",
"type": "Available"
}
],
"observedGeneration": 1,
"readyReplicas": 1,
"replicas": 1,
"updatedReplicas": 1
}
},
{
"name": "ase-core-api",
"namespace": "ecp",
"pods": [
{
"containers": [
{
"name": "ase-core-api",
"stats": {
"cpu": 957308,
"cpuPercentage": 0.0239327,
"memory": 14315520,
"memoryPercentage": 0.08619622354491156,
"timestamp": 1724660604,
"window": 17
}
}
],
"name": "ase-core-api-5cbf864fd-dq9dg",
"namespace": "ecp"
}
],
"status": {
"availableReplicas": 1,
"conditions": [
{
"lastTransitionTime": 1715073169,
"lastUpdateTime": 1720074798,
"message": "ReplicaSet \"ase-core-api-5cbf864fd\" has successfully progressed.",
"reason": "NewReplicaSetAvailable",
"status": "True",
"type": "Progressing"
},
{
"lastTransitionTime": 1722360937,
"lastUpdateTime": 1722360937,
"message": "Deployment has minimum availability.",
"reason": "MinimumReplicasAvailable",
"status": "True",
"type": "Available"
}
],
"observedGeneration": 3,
"readyReplicas": 1,
"replicas": 1,
"updatedReplicas": 1
}
}
],
"name": "k3d-dev-01-server-0",
"role": [
"control-plane",
"master"
],
"stats": {
"cpu": 208512731,
"cpuPercentage": 5.212818275,
"memory": 1401962496,
"memoryPercentage": 8.441458829773433,
"timestamp": 1724660605,
"window": 20
}
}
]
}