Skip to main content
Version: Next

Advanced debugging

Enabling debug shell access​

In some cases, additional information from within a pod VM is necessary to diagnose issues. To facilitate this, the Contrast pod VM can be configured for debugging via initdata that's passed to the guest. Services in the guest will then enable an SSH based debug service that can be accessed from within the pod sandbox.

Debug shell architecture

To enable debug shell access, add the insecure-enable-debug-shell-access flag when generating annotations:

contrast generate --insecure-enable-debug-shell-access
danger

Enabling debug access is inherently insecure. Never use this option on production workloads or with sensitive data involved.

This will add an additional field to the initdata passed to the guest, instructing the guest components to enable debug shell access:

[data]
'contrast.insecure-debug' = 'true'

It will also inject an additional debug-shell container into pods with a contrast-cc runtime class, which can be used as access point into the pod VM and can help to collect debug information.

Notice that enabling debug features via initdata is covered by the measurements of a pod VM and thus detectable via remote attestation.

Collecting pod VM logs​

The debug-shell container exposes the journal logs of the pod VM into Kubernetes pod logs by default. These logs can then be collected via standard Kubernetes tooling:

kubectl logs <pod-name> -c debug-shell

Ensure the settings.json used for generate includes the following settings so that access to the pod logs are allowed:

{
    ...
    "request_defaults": {
        ...
        "ReadStreamRequest": true,

Accessing the pod VM via debug shell​

Ensure the settings.json used for generate includes the following settings to allow interactive exec in Contrast pods:

{
    ...
    "request_defaults": {
        ...
        "ExecProcessRequest": {
            "allowed_commands": [],
            "regex": [ ".*" ]
        },
        ...
        "ReadStreamRequest": true,
        ...
        "WriteStreamRequest": true

Then execute the following the get a shell within the pod VM:

kubectl exec -it <pod-name> -c debug-shell -- debugshell

You should see something like the following output:

Warning: Permanently added '[localhost]:2222' (RSA) to the list of known hosts.

[root@nixos:/]#