Troubleshooting

Verify services

Once the deployment or upgrade is complete, there are a number of ways to verify that the services are running correctly.

Verify the Helm deployment

List the current helm deployments and verify that the <your domain> deployment is listed.

helm ls

Verify the Kubernetes pods

List the current pods and verify that all pods are in the Running state.

kubectl get pods -A

View logs

To view the logs for a specific service, you can use the kubectl logs command.

kubectl logs <pod name>

502 Bad Gateway

If you get a 502 Bad Gateway error when trying to access the application, it is likely that the application is still starting up but otherwise inaccessable. Ensure first that the pod for the affected service is running and shows 1/1 under the READY column.

$ kubectl get pods -A

NAME                                 READY   STATUS    RESTARTS      AGE
plex-xwz-123                         1/1     Running   0             5m

If you see 0/1 under the READY column, refer to the STATUS column to determine the cause of the failure.

If there is an issue you will likely see CrashLoopBackOff under the STATUS column. This indicates that the pod is crashing and restarting repeatedly, which is likely due to a configuration issue. To view the logs for the pod, use the kubectl logs <pod name> command.

If the STATUS reads READY and there is nothing in the container logs to indicate an issue, there is likely nothing wrong with the container itself. More likely, there is a misconfiguration between the container and the nginx config. Check to ensure the port defined for the service is correct. You may also want to check files/nginx to view the nginx configs.

ContainerCreating stuck

If you see ContainerCreating under the STATUS column, this indicates that the pod is still being created. This is normal during the initial deployment, but if you see this for an extended period of time, there may be an issue. After a few minutes, if the pod is still in the ContainerCreating state, you can describe the pod to get more information.

kubectl describe pod <pod name>

This will often be the result of mounting a volume that does not exist. If you see an error similar to the following, ensure that the volume exists and is spelled correctly.

Warning  FailedMount  2m (x4 over 3m)  kubelet, <node name>  MountVolume.SetUp failed for volume "config" : hostPath type check failed: /opt/<service>/config is not a directory

Create the missing directory and re-run the deployment.

mkdir -p /opt/<service>/config
chmod 660 /opt/<service>/config