A failed deployment pipeline is easier to fix when its symptoms are mapped to a small set of variables: source changes, runner environment, credentials, secrets, artifacts, health checks, and rollback state. This guide provides a reusable CI/CD deployment troubleshooting matrix, commands for collecting evidence, checkpoints for recurring maintenance, and an incident handoff template that works across common deployment platforms.
Overview
When a deployment fails, avoid treating the pipeline as one large system. Break it into stages and identify the first stage that produced an unexpected result. A later error is often only a consequence of an earlier failure. For example, a missing build artifact can cause a deployment step to fail, while an incorrect environment variable can make a healthy application appear to be unavailable.
Use this order when investigating a ci cd build failed event:
- Confirm the commit, branch, workflow, runner, and deployment target.
- Find the first failed step rather than the last visible error.
- Separate code, environment, authentication, and infrastructure causes.
- Reproduce the smallest failing command locally or in a controlled runner.
- Apply one change, rerun the relevant stage, and record the result.
Do not rotate credentials, rebuild infrastructure, or roll back immediately without preserving logs. A rushed fix can remove evidence and make a recurring problem harder to diagnose. For a complete deployment workflow, see How to Deploy a Web App with GitHub Actions.
What to track
Maintain a short record for every failed run. The goal is not to capture every log line; it is to preserve the variables that distinguish one failure from another.
Deployment pipeline troubleshooting matrix
| Symptom | Likely area | What to check first | Practical next step |
|---|---|---|---|
| Dependency installation fails | Build or runner | Lockfile, runtime version, package registry access, cache state | Run the package-manager install with a clean cache and compare the runner runtime to the project requirement. |
| Tests fail only in CI | Environment or test isolation | Environment variables, timezone, service dependencies, filesystem assumptions | Print safe configuration metadata, run the same test command locally, and use a disposable service dependency where required. |
| Permission denied or unauthorized | Authentication | Token scope, expiration, repository or cloud role, target account | Confirm the identity being used and test the narrowest read or deploy operation without exposing the token. |
| Deployment secrets are empty | Secret injection | Secret name, environment or branch scope, workflow context, masking behavior | Verify the secret is configured in the correct project and environment; never echo the secret value. |
| Artifact not found | Artifact handoff | Build output path, upload step, artifact name, retention or job dependency | List the output directory before upload and confirm the consuming job downloads the same artifact name. |
| Application deploys but is unhealthy | Runtime or health check | Port binding, process status, logs, DNS, reverse proxy, readiness endpoint | Check the service directly on the host or platform before investigating external DNS. |
| Rollback does not restore service | Release state | Rollback target, database compatibility, cached assets, configuration changes | Identify what changed outside the application image and verify the previous release is actually serving traffic. |
Evidence to capture
For each run, record the commit identifier, workflow or pipeline name, failing job and step, runner image or operating system, runtime versions, deployment target, and timestamp. Include whether the failure is reproducible and whether it affects one branch, environment, or service.
Collect logs without publishing credentials or sensitive configuration. Useful commands vary by platform, but these examples help establish the local state:
git rev-parse HEAD
git status --short
node --version
npm --version
printenv | sort | sed 's/=.*/=<redacted>/'
find dist build public -maxdepth 2 -type f 2>/dev/null | head -50For a container-based release, also inspect the image and running service:
docker images --format '{{.Repository}}:{{.Tag}} {{.ID}}'
docker ps --all
docker logs --tail=200 <container-name>Use equivalent commands for the language, package manager, hosting provider, or CI platform in use. The important practice is to capture versions and paths before changing the runner or rebuilding the application.
Cadence and checkpoints
Deployment troubleshooting is not only a response activity. Several pipeline variables change gradually and should be reviewed on a recurring schedule.
Per deployment
- Confirm the intended commit and target environment.
- Check that the artifact was produced, uploaded, and consumed by the expected job.
- Verify application logs and the health check after deployment.
- Record manual approvals, retries, overrides, and rollback decisions.
Monthly or quarterly
- Review failed-run patterns by stage, repository, branch, and environment.
- Remove obsolete secrets, tokens, deploy keys, and environment variables.
- Test that the documented rollback path still identifies a usable release.
- Review runner images, language versions, package-manager versions, and pinned actions or plugins.
- Check caches and artifact retention rules for stale or unexpectedly missing outputs.
- Confirm that DNS, reverse-proxy, and service configuration still match the deployment target.
Refresh this troubleshooting guide when a provider changes an action, runner image, authentication flow, environment interface, or log format. Commands and UI paths should be treated as implementation details that require periodic verification. If the application uses a Linux service, the Node.js systemd deployment guide provides a useful companion for service status, logs, and restarts.
How to interpret changes
A single failed run may be a transient event, but repeated failures in the same stage usually indicate a changed dependency or an undocumented assumption. Compare successful and failed runs rather than examining the failed run in isolation.
- Build failures after a runner change: compare operating-system packages, runtime versions, shell behavior, and cache contents. Pin or explicitly install required versions where practical.
- Authentication failures after credential rotation: verify the secret name and deployment context first, then check token scope, account, expiration, and target permissions.
- Secrets work on one branch but not another: inspect environment and branch restrictions. A secret may be available to a protected environment but intentionally absent from a different workflow context.
- Artifacts disappear between jobs: compare the producer's output path with the upload path and the consumer's download name. Confirm the consumer depends on the producer job.
- Deployment succeeds but traffic fails: distinguish application health from DNS or proxy routing. Test the service locally, then inspect the reverse proxy, TLS configuration, hostname, and DNS records. The Nginx reverse proxy setup guide covers common routing and health-check considerations.
- Rollback repeatedly fails: look for database migrations, configuration changes, external API changes, or persistent volumes that are not reverted with the application code.
For domain-related symptoms, use a DNS propagation check only after confirming that the application is listening and the proxy is routing correctly. DNS tools can show where a record resolves, but they cannot prove that the deployed process is healthy. See How to Diagnose Records That Are Not Updating for the DNS portion of that investigation.
Reusable incident handoff
When another person takes over, provide a compact handoff instead of forwarding an unfiltered log archive:
Summary: <what users or systems can no longer do>
First failed stage: <job and step>
Commit and environment: <identifier and target>
Started: <time and timezone>
Last known good run: <run or commit>
Observed error: <short redacted message>
Changes since last success: <code, dependency, runner, secret, or infrastructure>
Checks completed: <commands and results>
Current impact: <scope>
Safe next action: <one proposed test or rollback>
Owner and next update: <person and time>When to revisit
Revisit this matrix after any failed deployment that required a manual workaround, after a runner or dependency upgrade, during credential rotation, and after a rollback or incident review. A quarterly review is a practical baseline for teams with regular releases; teams with frequent infrastructure changes may need a monthly checkpoint.
Turn repeated discoveries into pipeline improvements. If a missing artifact is a recurring issue, add an explicit file-existence check before upload. If secrets are often mis-scoped, validate required variable names without printing values. If health checks are ambiguous, expose a documented readiness check and log the release identifier. If rollback depends on undocumented commands, automate and test the procedure.
Start with the latest failure today: identify the first failed step, save the evidence, compare it with the last successful run, and write down the smallest safe next action. Then schedule the next review date and update this article or your internal runbook whenever the pipeline's authentication, runner, artifact, deployment, or rollback behavior changes. For container releases that use GitHub Actions and Nginx, the Dockerized web app deployment guide can be used alongside this matrix.