Readiness vs Liveness Probe Kubernetes
So I have been having fun learning Kubernetes and recently learned that readiness and liveness probe shouldn’t be the exact same values. I was shocked I overlooked such an important setup. I’ll go over what readiness and liveness probe are and examples of how to set them in helm.
The liveness probe is used for general health checks and lets kubelet know when to restart the containers. This is a quick check to say the application is up and running, nothing more and nothing less.
Here below is a
httpGet of a liveness probe, I’d recommend using these generally unless the container isn’t a webserver that is running.
- name: Custom-Header
Straight from the documentation this is an example of liveness probe. As you can see here the liveness probe is generally used for health checks on the application. The liveness probe is used for determining if the application up and running.
Here’s some example endpoints that I would personally use:
- An information endpoint that displays information about the API
- A health endpoint that exists only for the purpose of ensuring the application is up and running. Note: This doesn’t mean the application is running properly or a connection to the database is working, but say the container is up and running.
- If this is a internal service, an endpoint that contains
Note: This endpoint shouldn’t require high level authorization (like administrator access), but general access. This should be the lowest level of access for the endpoint so less setup is required and shouldn’t affect any portion of the application.
If you wanted to do another type without an HTTP here’s an example:
This script you can run some custom checks to ensure the application is running as expected.
Here’s some example checks you can do:
- Ensure application logs are being written consistently
- Run a check to ensure resources aren’t used at near/max capacity
Whatever you do ensure this check is quick and can be ran several times quickly.
The readiness probe is a bit different and checks whether traffic is ready to hit the container. This essentially doesn’t mean the container is up and running, but the service within the container is ready to go. The readiness probe should check to ensure a user can use the service without receiving a 502 error and responses get dropped for the client.
Here’s an example call made of a simple web app:
Now this isn’t the strongest check but it details how similar both are (which can allow for mistakes to be made like I did). The readiness probe says this application is ready to go in a simple context, which is varying application to application.
Here’s some ideas I have for this endpoint to ensure the app is “ready to go”:
- In a simple web application that has a database I’d hit one of the endpoints that ensure the database connection works (a regular id lookup or even a endpoint dedicated to checking that connection to the database works)
- A simple endpoint that hits the web application if nothing else functions on it
- Similar to the above exec command, you can run a bit more complicated of a script that checks if a cache or other resources is working/exists
Now you can be a lot more creative than I can for how to setup these endpoints to increase confidence that the application is running as expected, but this is a great way to check endpoints.
Hopefully this article prevents you from making the same mistake as I did thinking both probes function similarly and are not needed. I’d recommend for a production application having both probes setup for another fail check.