HomeUse CasesConceptsConfig GuidesDeveloper GuidesAPI Reference
Content

Troubleshooting

IBM Verify MCP Server - Troubleshooting Guide

This guide helps you diagnose and resolve common issues when using IBM Verify MCP Server with MCP clients like Claude Desktop and IBM Bob.

Table of Contents

  1. Connection Issues
  2. Authentication Problems
  3. Server Not Running
  4. Tool Execution Failures
  5. Performance Issues
  6. Configuration Problems
  7. Docker-Specific Issues
  8. Kubernetes-Specific Issues
  9. Collecting Diagnostics
  10. Getting Support

Connection Issues

MCP Client Cannot Connect to Server

Symptoms:

  • Connection timeout errors
  • "Server not reachable" messages
  • HTTP 502/503/504 errors
  • Client cannot discover tools

Diagnostic Steps:

  1. Verify server is running:

    # Docker
docker ps | grep verify-mcp-server

# Kubernetes
kubectl get pods -n verify-mcp-server -l app=verify-mcp-server

  2. Test health endpoint:

    # Docker (local)
curl http://localhost:8000/health

# Kubernetes (external)
curl https://your-mcp-server-url.com/health

  3. Check MCP endpoint:

    # Docker
curl http://localhost:8000/mcp

# Kubernetes
curl https://your-mcp-server-url.com/mcp

Common Solutions:

  1. Incorrect URL in client configuration:

    • Verify the URL includes the /mcp path
    • Example: https://verify-mcp-server.example.com/mcp
    • For local Docker: http://localhost:8000/mcp

  2. Incorrect client configuration — ensure your config matches the correct format. Example for local development (OAuth, end user):

    {
  "mcpServers": {
    "ibm-verify": {
      "command": "uvx",
      "args": [
        "[email protected]",
        "http://localhost:8000/mcp",
        "--header",
        "persona: end_user"
      ]
    }
  }
}

    For the full set of configuration options, see the Configuring MCP Clients guide.

  3. Firewall or network policy blocking traffic:

    • Check firewall rules
    • Verify network policies in Kubernetes
    • Test from different network

  4. TLS certificate issues:

Client Not Registered Error

Symptoms:

  • Error message: "Client Not Registered"
  • Error message: "The client ID was not found in the server's client registry"
  • MCP client fails to connect after initial setup
  • Connection works initially but fails after some time

Diagnostic Steps:

  1. Check for stale authentication cache:

    # Check if mcp-auth cache exists
ls -la ~/.mcp-auth

  2. Review client logs:

    • Check MCP client logs for "Client Not Registered" errors
    • Look for client ID mismatches

Common Solutions:

  1. Clear MCP authentication cache:

    # On MCP Client system (IBM Bob or Claude Desktop)
rm -rf ~/.fastmcp

  2. Restart MCP client:

    • After clearing the cache, restart your MCP client (IBM Bob or Claude Desktop)
    • The client re-register with the server on next connection

  3. Verify client configuration:

    • Ensure the client configuration hasn't changed
    • Check that the server URL is still correct
    • Verify OAuth client metadata is properly configured

  4. Check server-side client registry:

    # Docker
docker logs verify-mcp-server | grep -i "client.*register"

# Kubernetes
kubectl logs -n verify-mcp-server <pod-name> | grep -i "client.*register"

Note: This issue commonly occurs when:

  • The MCP server has been restarted or redeployed
  • Client authentication tokens have expired
  • There's a mismatch between cached client data and server registry

OAuth Discovery Fails Due to Incorrect MCP Base URL

Symptoms:

  • Connection fails during OAuth discovery
  • 401 Unauthorized with WWW-Authenticate header
  • Error fetching Protected Resource Metadata
  • Error fetching authorization server metadata
  • "The operation was aborted due to timeout"
  • "Connection error: fetch failed"

Example logs:

Received 401 with WWW-Authenticate header

resource_metadata="https://verify-mcp-server.svc.cluster.local:8000/.well-known/oauth-protected-resource/mcp"

Error fetching Protected Resource Metadata
The operation was aborted due to timeout

Error fetching authorization server metadata
The operation was aborted due to timeout

Connection error: fetch failed

Root Cause:

The MCP server is advertising OAuth metadata by using an internal Kubernetes or OpenShift service URL (*.svc.cluster.local) instead of the externally accessible route.

Example of problematic URL:

https://verify-mcp-server.svc.cluster.local:8000

The OAuth discovery process uses the URLs returned by the MCP server. If those URLs are only reachable from inside the cluster, external MCP clients cannot complete OAuth discovery, Dynamic Client Registration (DCR), or token acquisition.

Diagnostic Steps:

  1. Check the configured MCP Base URL:

    # Kubernetes
kubectl get configmap -n verify-mcp-server verify-mcp-config -o yaml | grep BASE_URL

# OpenShift
oc get configmap -n verify-mcp-server verify-mcp-config -o yaml | grep BASE_URL

  2. Verify OAuth Protected Resource Metadata endpoint:

    curl -s https://<mcp-base-url>/.well-known/oauth-protected-resource/mcp

  3. Check authorization server metadata endpoint:

    curl -s https://<mcp-base-url>/.well-known/oauth-authorization-server

Common Solutions:

  1. Configure correct MCP Base URL:

    The MCP Base URL should point to the public route exposed by OpenShift or Kubernetes, for example:

    https://verify-mcp-server-dev.5le8.p1.openshiftapps.com

    NOT an internal service URL such as:

    https://verify-mcp-server.svc.cluster.local:8000

  2. Update ConfigMap with correct URL:

    # Kubernetes
kubectl edit configmap -n verify-mcp-server verify-mcp-config

# OpenShift
oc edit configmap -n verify-mcp-server verify-mcp-config

    Set IBM_VERIFY_BASE_URL to the public route URL.

  3. Restart deployment to apply changes:

    # Kubernetes
kubectl rollout restart deployment -n verify-mcp-server verify-mcp-server

# OpenShift
oc rollout restart deployment -n verify-mcp-server verify-mcp-server

Validation Steps:

  1. Verify OAuth Protected Resource Metadata returns public URLs:

    curl -s https://<public-route>/.well-known/oauth-protected-resource/mcp

    Expected response:

    {
  "resource": "https://<public-route>/mcp",
  "authorization_servers": [
    "https://<public-route>"
  ]
}

    NOT expected (internal URLs):

    {
  "resource": "https://<service-name>.<namespace>.svc.cluster.local:8000/mcp",
  "authorization_servers": [
    "https://<service-name>.<namespace>.svc.cluster.local:8000"
  ]
}

  2. Verify authorization server metadata:

    curl -s https://<public-route>/.well-known/oauth-authorization-server

    Ensure all advertised endpoints use the public route and are reachable from the client machine.

  3. Test MCP client connection:

    • Restart your MCP client (IBM Bob or Claude Desktop)
    • Attempt to connect to the MCP server
    • OAuth discovery should now succeed

Important Notes:

  • A 401 Unauthorized response from the MCP endpoint is expected during OAuth discovery and does not indicate a problem by itself.
  • The issue occurs when the OAuth metadata returned by the server contains internal cluster URLs that cannot be reached by the MCP client.
  • Always use the externally accessible route URL for IBM_VERIFY_BASE_URL when deploying in Kubernetes or OpenShift.

Authentication Problems

Login to IBM Verify Fails

Symptoms:

  • Authentication failures
  • 401 Unauthorized or 403 Forbidden errors
  • Login redirect loops
  • "Invalid client" errors

Diagnostic Steps:

  1. Verify tenant URL is accessible:

    curl https://your-tenant.verify.ibm.com/oidc/endpoint/default/.well-known/openid-configuration

  2. Check environment variables:

    # Docker
docker exec verify-mcp-server env | grep IBM_VERIFY

# Kubernetes
kubectl exec -n verify-mcp-server <pod-name> -- env | grep IBM_VERIFY

  3. Review authentication logs:

    # Docker
docker logs verify-mcp-server | grep -i "auth\|token\|oauth"

# Kubernetes
kubectl logs -n verify-mcp-server <pod-name> | grep -i "auth\|token\|oauth"

Common Solutions:

  1. Incorrect OAuth credentials:

    • Verify IBM_VERIFY_CLIENT_ID matches Subject application client ID
    • Verify IBM_VERIFY_CLIENT_SECRET is correct
    • Check credentials haven't expired

  2. Wrong redirect URI:

    • Ensure IBM_VERIFY_BASE_URL matches the actual server URL
    • For Docker: http://localhost:8000
    • For Kubernetes: https://your-mcp-server-url.com
    • Verify redirect URI is configured in IBM Verify application

  3. User not entitled to application:

    • Go to IBM Verify Admin Console
    • Navigate to Applications → MCP Server Subject
    • Go to Entitlement tab
    • Add the user or group

  4. Missing or incorrect scopes:

    • Minimum required scope: openid
    • Verify scope in client configuration matches application configuration

Token Exchange Fails

Symptoms:

  • "Token exchange failed" errors
  • Tools cannot be executed after login
  • invalid_grant errors

Diagnostic Steps:

  1. Verify STS client configuration:

    # Check STS credentials
docker exec verify-mcp-server env | grep STS

  2. Check Actor application credentials:

    docker exec verify-mcp-server env | grep ACTOR

Common Solutions:

  1. Incorrect STS credentials:

    • Verify IBM_VERIFY_STS_CLIENT_ID is correct
    • Verify IBM_VERIFY_STS_CLIENT_SECRET is correct

  2. Actor application not configured:

    • Verify IBM_VERIFY_ACTOR_CLIENT_ID is set
    • Verify IBM_VERIFY_ACTOR_CLIENT_SECRET is set

  3. STS client not enabled:

    • Check STS client is enabled in IBM Verify admin console
    • Verify token exchange grant type is enabled

OAuth Error - Token Exchange with Identity Provider Failed: invalid_client (CSIAQ0155E)

Symptoms:

  • Error: OAuth error - Token exchange with identity provider failed: invalid_client
  • Error code CSIAQ0155E: Client could not be authenticated
  • Login completes but subsequent token exchange fails

Cause:

One or more of the OAuth client credentials configured in the server deployment (Subject, Actor, or STS) do not match the corresponding application definitions in IBM Verify.

Resolution:

Review and correct the following environment variables in your server deployment configuration:

VariableIBM Verify ApplicationField
IBM_VERIFY_CLIENT_IDMCP Server SubjectClient ID
IBM_VERIFY_CLIENT_SECRETMCP Server SubjectClient secret
IBM_VERIFY_ACTOR_CLIENT_IDMCP Server ActorClient ID
IBM_VERIFY_ACTOR_CLIENT_SECRETMCP Server ActorClient secret
IBM_VERIFY_STS_CLIENT_IDMCP Server STSClient ID
IBM_VERIFY_STS_CLIENT_SECRETMCP Server STSClient secret

Steps to verify:

  1. In the IBM Verify Admin Console, navigate to Applications and open each of the three applications (Subject, Actor, STS).
  2. Cross-check the Client ID and Client secret values against those set in your deployment.
  3. Correct any mismatches, then redeploy or restart the server.

Redirect URI Mismatch - CSIAQ0167E

Symptoms:

  • Error: Your request cannot be processed
  • Error code CSIAQ0167E: Redirection URI provided in request is either invalid or doesn't match with any of the OAuth 2.0 client pre-registered redirect URIs
  • User is shown an error page after being redirected from IBM Verify

Cause:

The IBM_VERIFY_BASE_URL (application URL) set in the server deployment does not match a redirect URI registered in the MCP Server Subject application in IBM Verify.

Resolution:

  1. In the IBM Verify Admin Console, navigate to Applications → MCP Server Subject → Sign-on.
  2. Under Redirect URIs, add the public URL of your MCP server deployment. Examples:
    • https://verify-mcp-server.openshiftapps.com
    • https://<hostname>/mcp
  3. Ensure IBM_VERIFY_BASE_URL in your deployment config is set to the same base URL (without a trailing slash).
  4. Save the application and retry the login flow.

Note: Both the hostname and path must match exactly. A redirect to https://example.com/mcp fails if only https://example.com is registered, and vice versa.

Server Not Running

Pod/Container in CrashLoopBackOff

Symptoms:

  • Container repeatedly restarts
  • Pod status shows CrashLoopBackOff or Error
  • Server endpoint unavailable

Diagnostic Steps:

  1. Check container/pod status:

    # Docker
docker ps -a | grep verify-mcp-server

# Kubernetes
kubectl get pods -n verify-mcp-server

  2. Review logs:

    # Docker
docker logs verify-mcp-server

# Kubernetes
kubectl logs -n verify-mcp-server <pod-name>
kubectl logs -n verify-mcp-server <pod-name> --previous

  3. Check for missing environment variables:

    # Docker
docker inspect verify-mcp-server | grep -A 20 Env

# Kubernetes
kubectl describe pod -n verify-mcp-server <pod-name>

Common Solutions:

  1. Missing required environment variables:

    • Ensure all variables from setup-env.sh are set
    • Required variables:
      • IBM_VERIFY_TENANT_URL
      • IBM_VERIFY_CLIENT_ID
      • IBM_VERIFY_CLIENT_SECRET
      • IBM_VERIFY_STS_CLIENT_ID
      • IBM_VERIFY_STS_CLIENT_SECRET
      • IBM_VERIFY_ACTOR_CLIENT_ID
      • IBM_VERIFY_ACTOR_CLIENT_SECRET

  2. Invalid configuration:

    • Check for typos in environment variables
    • Verify URLs don't have trailing slashes
    • Ensure secrets don't have extra spaces or newlines

  3. Resource constraints:

    # Kubernetes - check resource usage
kubectl top pod -n verify-mcp-server <pod-name>

# Increase memory/CPU limits if needed
kubectl edit deployment -n verify-mcp-server verify-mcp-server

  4. Image pull errors:

    # Verify image exists
docker pull icr.io/ibm-verify/verify-mcp-server:latest

# Check image pull secrets in Kubernetes
kubectl get secrets -n verify-mcp-server

Tool Execution Failures

Tools Discovered But Cannot Execute

Symptoms:

  • Tools appear in client but fail when invoked
  • Timeout errors during execution
  • Permission denied errors
  • Partial results returned

Diagnostic Steps:

  1. Verify tool appears in discovery:

    • Check MCP client shows the tool
    • Verify tool parameters are correct

  2. Check execution logs:

    # Docker
docker logs verify-mcp-server --tail=100 | grep -i "tool\|error"

# Kubernetes
kubectl logs -n verify-mcp-server <pod-name> --tail=100 | grep -i "tool\|error"

  3. Test IBM Verify API connectivity:

    # Docker
docker exec verify-mcp-server curl -v https://your-tenant.verify.ibm.com/v2.0/Users

# Kubernetes
kubectl exec -n verify-mcp-server <pod-name> -- \
  curl -v https://your-tenant.verify.ibm.com/v2.0/Users

Common Solutions:

  1. Invalid tool parameters:

    • Verify required parameters are provided
    • Check parameter formats match documentation
    • See Using MCP Tools for parameter details

  2. User lacks permissions:

    • Verify user has required entitlements in IBM Verify
    • Check user roles and permissions
    • Ensure STS client has correct entitlements configured

  3. IBM Verify API unavailable:

    • Check tenant is accessible
    • Verify network connectivity
    • Check for IBM Verify service outages

  4. Rate limiting:

    • Reduce request frequency
    • Implement delays between requests
    • Check IBM Verify rate limits

Tools Not Appearing

Symptoms:

  • Expected tools don't appear in client
  • Empty tool catalog
  • Tools appear intermittently

Diagnostic Steps:

  1. Check server startup logs:

    # Docker
docker logs verify-mcp-server | grep -i "tool\|register"

# Kubernetes
kubectl logs -n verify-mcp-server <pod-name> | grep -i "tool\|register"

  2. Verify MCP endpoint:

    curl http://localhost:8000/mcp

Common Solutions:

  1. Server not fully started:

    • Wait for server to complete initialization
    • Check health endpoint returns 200 OK

  2. Client configuration error:

    • Verify client configuration is correct
    • Restart MCP client
    • Check client logs for errors

  3. Authorization issues:

    • Ensure user is authenticated
    • Verify OAuth flow completed successfully

Performance Issues

Slow Response Times

Symptoms:

  • High latency
  • Timeouts
  • Slow tool execution

Diagnostic Steps:

  1. Check resource usage:

    # Docker
docker stats verify-mcp-server

# Kubernetes
kubectl top pod -n verify-mcp-server <pod-name>

  2. Test network latency:

    # Test IBM Verify API response time
time curl -s https://your-tenant.verify.ibm.com/v2.0/Users > /dev/null

  3. Review logs for slow operations:

    docker logs verify-mcp-server | grep -i "slow\|timeout\|latency"

Common Solutions:

  1. Insufficient resources:

    # Docker - increase memory
docker stop verify-mcp-server
docker run -d --name verify-mcp-server \
  --memory="2g" --cpus="2" \
  # ... other options

# Kubernetes - edit deployment
kubectl edit deployment -n verify-mcp-server verify-mcp-server

  2. Network latency:

    • Check network connectivity to IBM Verify
    • Consider deploying closer to IBM Verify region
    • Use CDN or caching if applicable

  3. Scale horizontally (Kubernetes):

    kubectl scale deployment -n verify-mcp-server verify-mcp-server --replicas=3

Memory Leak — OOM Crash (verify-mcp-server Pod Hits Memory Limit)

Symptoms:

  • The verify-mcp-server pod is killed with an OOMKilled status after sustained concurrent usage.
  • Memory usage grows steadily as more users access the MCP Server simultaneously and is never released between sessions.
  • Pod restarts automatically (CrashLoopBackOff or OOMKilled) once it hits its configured memory limit (default: 1 GB).
  • All active user sessions are lost when the pod restarts.

Root Cause:

Memory allocated per user session is not released when sessions are idle or when a user disconnects. Under concurrent load, cumulative in-memory session state accumulates until the pod exhausts its memory limit and is killed by the OS/Kubernetes OOM killer.

Solution / Workaround:

There is currently no automatic session cleanup between user disconnections. When the pod restarts after an OOMKilled event, all users must re-authenticate by logging in to the MCP Server again.

Note: A permanent fix requires implementing server-side session lifecycle management so that memory is released when a user session ends. Until that fix is released, the provided workaround applies.

Configuration Problems

Invalid Configuration Errors

Symptoms:

  • Server fails to start
  • Configuration validation errors
  • Missing required parameters

Diagnostic Steps:

  1. Verify all environment variables are set:

    # Docker
docker exec verify-mcp-server env | grep IBM_VERIFY

# Kubernetes
kubectl get configmap -n verify-mcp-server verify-mcp-config -o yaml
kubectl get secret -n verify-mcp-server verify-mcp-credentials -o yaml

  2. Check for typos:

    • Review environment variable names
    • Verify URLs are correct
    • Check for extra spaces or newlines

Common Solutions:

  1. Re-run setup script:

    cd docs/users/verify-mcp-server
./setup-verify-tenant.sh
source setup-env.sh

  2. Manually verify each variable:

    echo $IBM_VERIFY_TENANT_URL
echo $IBM_VERIFY_CLIENT_ID
# ... check all variables

  3. Update configuration:

    # Docker - restart with correct environment
docker stop verify-mcp-server
docker rm verify-mcp-server
source setup-env.sh
./docker-run-mcp-server.sh

# Kubernetes - update ConfigMap/Secret
kubectl edit configmap -n verify-mcp-server verify-mcp-config
kubectl edit secret -n verify-mcp-server verify-mcp-credentials
kubectl rollout restart deployment -n verify-mcp-server verify-mcp-server

Docker-Specific Issues

Container Won't Start

Diagnostic Steps:

  1. Check Docker is running:

    docker info

  2. Verify image exists:

    docker images | grep verify-mcp-server

  3. Check port conflicts:

    lsof -i :8000
netstat -an | grep 8000

Common Solutions:

  1. Port already in use:

    # Stop conflicting service or use different port
docker run -d --name verify-mcp-server \
  -p 8001:8000 \
  # ... other options

  2. Pull latest image:

    docker pull icr.io/ibm-verify/verify-mcp-server:latest

  3. Check Docker logs:

    docker logs verify-mcp-server

Platform Mismatch — no matching manifest for linux/arm64/v8

Symptoms:

docker pull icr.io/ibm-verify/verify-mcp-server:latest
latest: Pulling from ibm-verify/verify-mcp-server
no matching manifest for linux/arm64/v8 in the manifest list entries

This error occurs on Apple Silicon (M1/M2/M3) or other ARM64 machines because the image is published for linux/amd64 only.

Solution:

Add --platform linux/amd64 to your docker pull or docker run command:

# Pull with explicit platform
docker pull --platform linux/amd64 icr.io/ibm-verify/verify-mcp-server:latest

# Run with explicit platform
docker run --platform linux/amd64 -d --name verify-mcp-server \
  -p 8000:8000 \
  # ... other options
  icr.io/ibm-verify/verify-mcp-server:latest

Note: Docker Desktop on Apple Silicon automatically enables Rosetta 2 emulation for linux/amd64 images, so performance is generally acceptable for development and testing.

Cannot Access from Host

Symptoms:

  • curl http://localhost:8000/health fails
  • Connection refused errors

Common Solutions:

  1. Verify port mapping:

    docker ps | grep verify-mcp-server
# Should show: 0.0.0.0:8000->8000/tcp

  2. Check container is running:

    docker ps -a | grep verify-mcp-server

  3. Test from inside container:

    docker exec verify-mcp-server curl http://localhost:8000/health

Kubernetes-Specific Issues

Pod Not Scheduling

Symptoms:

  • Pod stuck in Pending state
  • No nodes available

Diagnostic Steps:

  1. Check pod status:

    kubectl describe pod -n verify-mcp-server <pod-name>

  2. Check node resources:

    kubectl top nodes
kubectl describe nodes

Common Solutions:

  1. Insufficient resources:

    • Reduce resource requests
    • Add more nodes to cluster
    • Scale down other workloads

  2. Node selector issues:

    • Remove or update node selectors
    • Add required labels to nodes

Service Not Accessible

Symptoms:

  • Cannot reach service from outside cluster
  • Route/Ingress not working

Diagnostic Steps:

  1. Check service:

    kubectl get svc -n verify-mcp-server verify-mcp-server
kubectl describe svc -n verify-mcp-server verify-mcp-server

  2. Check endpoints:

    kubectl get endpoints -n verify-mcp-server verify-mcp-server

  3. Check Route/Ingress:

    # OpenShift
oc get route -n verify-mcp-server verify-mcp-server

# Kubernetes
kubectl get ingress -n verify-mcp-server verify-mcp-server

Common Solutions:

  1. Service selector mismatch:

    kubectl edit svc -n verify-mcp-server verify-mcp-server
# Verify selector matches pod labels

  2. Create or fix Route/Ingress:

  3. Test from within cluster:

    kubectl run -it --rm debug --image=curlimages/curl --restart=Never -n verify-mcp-server -- \
  curl http://verify-mcp-server:8000/health

TLS Certificate Problems

Symptoms:

  • Certificate verification errors
  • x509: certificate signed by unknown authority
  • TLS handshake failures

Diagnostic Steps:

  1. Check certificate:

    openssl s_client -connect your-mcp-server-url.com:443 -showcerts

  2. Verify certificate in pod:

    kubectl exec -n verify-mcp-server <pod-name> -- \
  openssl s_client -connect your-tenant.verify.ibm.com:443 -showcerts

Common Solutions:

  1. Install CA certificates:

    # Create ConfigMap with CA cert
kubectl create configmap ca-certificates \
  --from-file=ca-bundle.crt=/path/to/ca-cert.pem \
  -n verify-mcp-server

# Mount in deployment
kubectl edit deployment -n verify-mcp-server verify-mcp-server

  2. Use valid certificate:

    • Obtain certificate from trusted CA
    • Update Secret with new certificate
    • Restart pods

  3. For testing only - disable verification:

    # NOT RECOMMENDED for production
kubectl set env deployment/verify-mcp-server \
  -n verify-mcp-server \
  PYTHONHTTPSVERIFY=0

Collecting Diagnostics

Information to Collect

When troubleshooting or reporting issues, collect the following:

  1. Server logs:

    # Docker
docker logs verify-mcp-server > mcp-server-logs.txt

# Kubernetes
kubectl logs -n verify-mcp-server <pod-name> > mcp-server-logs.txt
kubectl logs -n verify-mcp-server <pod-name> --previous > mcp-server-logs-previous.txt

  2. Configuration:

    # Docker
docker inspect verify-mcp-server > docker-inspect.json

# Kubernetes
kubectl get deployment -n verify-mcp-server verify-mcp-server -o yaml > deployment.yaml
kubectl get configmap -n verify-mcp-server verify-mcp-config -o yaml > configmap.yaml
kubectl get secret -n verify-mcp-server verify-mcp-credentials -o yaml > secret.yaml

  3. Support bundle script:

    Docker / local container:

    Save the following as collect-support-info-docker.sh and run it from your host machine. It connects to the running container, collects logs, stats, health-check output, and environment variables (all secrets and UUIDs are redacted automatically), then packages everything into a .tar.gz.

    #!/bin/bash
# collect-support-info-docker.sh
# Support information collection for IBM Verify MCP Server (standalone Docker)
# Collects logs, metrics, and configuration WITHOUT sensitive data

set -euo pipefail

DEFAULT_CONTAINER="verify-mcp-server-local"
CONTAINER="${1:-$DEFAULT_CONTAINER}"
OUTPUT_DIR="support-bundle-$(date +%Y%m%d-%H%M%S)"

RED='\033[0;31m'; GREEN='\033[0;32m'; YELLOW='\033[1;33m'; NC='\033[0m'
log_info()  { echo -e "${GREEN}[INFO]${NC}  $1"; }
log_warn()  { echo -e "${YELLOW}[WARN]${NC}  $1"; }
log_error() { echo -e "${RED}[ERROR]${NC} $1"; }

redact() {
    sed -E \
        -e 's/(SECRET|PASSWORD|TOKEN|KEY|client_secret|authorization|bearer)[=:][^ ]*/\1=<REDACTED>/gi' \
        -e 's/[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}/00000000-0000-0000-0000-000000000000/gi'
}

cexec() { docker exec "$CONTAINER" sh -c "$1" 2>/dev/null || true; }

# ---- Preflight -----------------------------------------------------------
if ! command -v docker &>/dev/null; then
    log_error "docker not found."; exit 1
fi
if ! docker inspect "$CONTAINER" &>/dev/null; then
    log_error "Container '$CONTAINER' not found."
    echo "Running containers:"; docker ps --format "  {{.Names}}  ({{.Status}})"
    exit 1
fi

# ---- Setup ---------------------------------------------------------------
mkdir -p "$OUTPUT_DIR"
log_info "Output directory : $OUTPUT_DIR"
log_info "Container        : $CONTAINER"

# ---- 1. Docker host metadata ---------------------------------------------
log_info "Collecting Docker host info..."
docker version                        > "$OUTPUT_DIR/docker-version.txt"     2>/dev/null || true
docker info                           > "$OUTPUT_DIR/docker-info.txt"         2>/dev/null || true
docker inspect "$CONTAINER" | redact  > "$OUTPUT_DIR/container-inspect.json" 2>/dev/null || true

# ---- 2. Container status & stats -----------------------------------------
log_info "Collecting container status..."
docker ps -a --filter "name=$CONTAINER"           > "$OUTPUT_DIR/container-status.txt" 2>/dev/null || true
docker stats "$CONTAINER" --no-stream --no-trunc  > "$OUTPUT_DIR/container-stats.txt"  2>/dev/null || \
    log_warn "Stats unavailable (container may be stopped)"

# ---- 3. Application logs -------------------------------------------------
log_info "Collecting stdout/stderr logs (last 5000 lines)..."
docker logs "$CONTAINER" --tail 5000 2>&1 | redact > "$OUTPUT_DIR/logs-stdout.txt" || \
    log_warn "Failed to collect stdout logs"

log_info "Collecting file-based logs (/app/logs/mcp.log)..."
cexec "cat /app/logs/mcp.log 2>/dev/null" | redact > "$OUTPUT_DIR/logs-mcp.log"

for i in 1 2 3 4 5; do
    content=$(cexec "cat /app/logs/mcp.log.$i 2>/dev/null")
    if [ -n "$content" ]; then
        echo "$content" | redact > "$OUTPUT_DIR/logs-mcp.log.$i"
        log_info "Collected rotated log: mcp.log.$i"
    fi
done

# ---- 4. Health check -----------------------------------------------------
log_info "Checking health endpoint..."
cexec "curl -sf http://localhost:8000/health" > "$OUTPUT_DIR/health-check.json" || \
    log_warn "Health endpoint did not respond"

# ---- 5. Environment variables (redacted) ---------------------------------
log_info "Collecting env vars (redacted)..."
docker inspect "$CONTAINER" \
    --format '{{range .Config.Env}}{{println .}}{{end}}' 2>/dev/null | \
    redact | sort > "$OUTPUT_DIR/env-vars.txt" || \
    log_warn "Failed to collect env vars"

# ---- 6. System diagnostics inside container ------------------------------
log_info "Collecting system diagnostics from container..."
cexec "free -h 2>/dev/null || cat /proc/meminfo"                                > "$OUTPUT_DIR/sys-memory.txt"
cexec "df -h"                                                                   > "$OUTPUT_DIR/sys-disk.txt"
cexec "uptime"                                                                  > "$OUTPUT_DIR/sys-uptime.txt"
cexec "ps aux 2>/dev/null || ps -ef"                                           > "$OUTPUT_DIR/sys-processes.txt"
cexec "netstat -tuln 2>/dev/null || ss -tuln 2>/dev/null || cat /proc/net/tcp" > "$OUTPUT_DIR/sys-netstat.txt"
cexec "cat /proc/cpuinfo | grep -E 'processor|model name|cpu MHz' | head -20"  > "$OUTPUT_DIR/sys-cpu.txt"

# ---- 7. Summary ----------------------------------------------------------
log_info "Generating summary..."
{
cat <<SUMMARY
IBM Verify MCP Server - Docker Support Bundle
=============================================
Generated    : $(date)
Container    : $CONTAINER
Collected by : $(whoami)@$(hostname)

CONTAINER STATUS
----------------
$(docker ps -a --filter "name=$CONTAINER" 2>/dev/null)

CONTAINER STATS
---------------
$(docker stats "$CONTAINER" --no-stream 2>/dev/null || echo "N/A")

HEALTH CHECK
------------
$(cat "$OUTPUT_DIR/health-check.json" 2>/dev/null || echo "N/A")

LAST 20 LOG LINES
-----------------
$(tail -20 "$OUTPUT_DIR/logs-stdout.txt" 2>/dev/null)

FILES COLLECTED
---------------
$(ls -lh "$OUTPUT_DIR")

NOTES
-----
- All secrets, tokens, and UUIDs have been redacted
- Logs capped at 5000 lines
- Share this bundle with your IBM Support case number
SUMMARY
} > "$OUTPUT_DIR/SUMMARY.txt"

# ---- 8. Compress ---------------------------------------------------------
log_info "Compressing bundle..."
tar -czf "$OUTPUT_DIR.tar.gz" "$OUTPUT_DIR"

BUNDLE_SIZE=$(du -h "$OUTPUT_DIR.tar.gz" | cut -f1)
echo ""
echo "================================================"
echo "  Bundle   : $OUTPUT_DIR.tar.gz"
echo "  Size     : $BUNDLE_SIZE"
echo "  Location : $(pwd)/$OUTPUT_DIR.tar.gz"
echo "================================================"
echo ""

read -rp "Remove uncompressed directory? (y/N) " reply
if [[ "$reply" == "y" || "$reply" == "Y" ]]; then
    rm -rf "$OUTPUT_DIR"
    log_info "Removed."
fi

log_info "Done."

    Run it:

    chmod +x collect-support-info-docker.sh

# default container name (verify-mcp-server-local)
./collect-support-info-docker.sh

# custom container name
./collect-support-info-docker.sh <your-container-name>

    Kubernetes:

    Option A — copy from a running container (preferred):

    docker cp verify-mcp-server:/app/support/scripts/collect-support-info.sh ./collect-support-info.sh
chmod +x ./collect-support-info.sh
./collect-support-info.sh <correct-namespace>

    Option B — stream via kubectl exec when docker cp is not available:

    kubectl exec -n <namespace> <pod-name> -- \
    cat /app/support/scripts/collect-support-info.sh > collect-support-info.sh
chmod +x ./collect-support-info.sh
./collect-support-info.sh <correct-namespace>

    The script generates a support bundle .tar.gz file in your current directory.

    Note: Before sharing the bundle with IBM Support, review its contents and remove or redact any sensitive information such as secrets, tokens, passwords, private keys, or internal hostnames that should not leave your environment.

  4. Environment details:

    # Docker version
docker version

# Kubernetes version
kubectl version

# OS information
uname -a

  5. Network diagnostics:

    # Test connectivity
curl -v http://localhost:8000/health
curl -v https://your-tenant.verify.ibm.com/oidc/endpoint/default/.well-known/openid-configuration

# DNS resolution
nslookup your-mcp-server-url.com

  6. Client configuration:

    • MCP client configuration file (redact secrets)
    • Client version
    • Client logs if available

Enable Debug Logging

  1. Docker:

    docker stop verify-mcp-server
docker rm verify-mcp-server

# Add debug environment variable
docker run -d --name verify-mcp-server \
  -e LOG_LEVEL=DEBUG \
  # ... other options

  2. Kubernetes:

    kubectl set env deployment/verify-mcp-server \
  -n verify-mcp-server \
  LOG_LEVEL=DEBUG

kubectl rollout restart deployment -n verify-mcp-server verify-mcp-server

  3. MCP Client (Claude Desktop/Bob): Add --debug to your existing config args to enable verbose client-side logging:

    {
  "mcpServers": {
    "ibm-verify": {
      "command": "uvx",
      "args": [
        "[email protected]",
        "http://localhost:8000/mcp",
        "--header",
        "persona: end_user",
        "--debug"
      ]
    }
  }
}

Getting Support

Before Contacting Support

Complete this checklist:

  • Reviewed this troubleshooting guide
  • Checked server is running and healthy
  • Verified all environment variables are set correctly
  • Tested network connectivity to IBM Verify
  • Collected server logs
  • Documented exact steps to reproduce the issue
  • Noted any error messages with timestamps

What to Include in Support Request

  1. Problem Description:

    • What were you trying to do?
    • What happened instead?
    • When did it start?
    • How often does it occur?

  2. Environment Details:

    • Deployment method (Docker or Kubernetes)
    • MCP Server version/image tag
    • IBM Verify tenant URL (without credentials)
    • MCP client (Claude Desktop, IBM Bob, etc.)
    • Operating system and version

  3. Reproduction Steps:

    • Exact steps to reproduce the issue
    • Expected behavior
    • Actual behavior

  4. Logs and Diagnostics:

    • Server logs (see Collecting Diagnostics)
    • Configuration files (redact secrets)
    • Error messages with timestamps
    • Screenshots if applicable

  5. Impact:

    • Number of users affected
    • Business impact
    • Workarounds attempted

Support Channels

Quick Reference

Common Commands

# Docker - View logs
docker logs verify-mcp-server

# Docker - Restart server
docker restart verify-mcp-server

# Docker - Check status
docker ps | grep verify-mcp-server

# Kubernetes - View logs
kubectl logs -n verify-mcp-server <pod-name>

# Kubernetes - Restart deployment
kubectl rollout restart deployment -n verify-mcp-server verify-mcp-server

# Kubernetes - Check status
kubectl get pods -n verify-mcp-server

# Test health endpoint
curl http://localhost:8000/health

# Test MCP endpoint
curl http://localhost:8000/mcp

Environment Variables Checklist

Required variables:

  • IBM_VERIFY_TENANT_URL - Your IBM Verify tenant URL
  • IBM_VERIFY_CLIENT_ID - Subject application client ID
  • IBM_VERIFY_CLIENT_SECRET - Subject application client secret
  • IBM_VERIFY_STS_CLIENT_ID - STS client ID
  • IBM_VERIFY_STS_CLIENT_SECRET - STS client secret
  • IBM_VERIFY_ACTOR_CLIENT_ID - Actor application client ID
  • IBM_VERIFY_ACTOR_CLIENT_SECRET - Actor application client secret

Optional variables:

  • LOG_LEVEL - Logging level (DEBUG, INFO, WARNING, ERROR)
  • MCP_TRANSPORT - Transport type (default: streamable-http)

Related Documentation

Last Updated: 2026-06-24
Version: 1.0.0
Maintained By: IBM Verify Team

Updated about 3 hours ago

Did this page help you?