How to gracefully kill Kubernetes Jobs with a sidecar?

Have you ever had a sidecar in your Kubernetes Job? If no, then trust me that you are lucky. If yes, then you will have the frustration of your life. The thing is Kubernetes Jobs are meant to exit on completion. But if you have a long-running sidecar, then that might twist things for Kubernetes and in turn of you. Why would you even want a sidecar for Job? Well, one of the most prevalent use case is when using service mesh proxy. There could be something else as well like metrics endpoint, log collection or whatever. Given the complexity and heterogeneity of the workloads, there could be any kind of use case that involves having sidecar for a Job pod. ...

August 29, 2020 · 6 min · Suraj Deshmukh

Use Configmap for Scripts

We generally use some sort of scripts in application container images. They serve various purposes. Some scripts might do an initial setup before the application starts, others may have the whole logic of the container image, etc. Whatever the goal may be the general pattern is to copy the script into the container image, build the image and then the script is available when you consume the image. Cons of the Traditional Method The round trip time during development and testing of such script is very long. You make some change to the script, you need to build the image, push it and then it is downloaded again. On an average for every change adds a couple of minutes to your feedback loop. Bash scripts are generally precarious in nature. You have to hammer it down, consider edge cases and thereby make it robust. This, of course, takes a lot of iterations. And with iterations comes the added time. So the question is, how do we reduce this feedback loop? ...

August 22, 2020 · 6 min · Suraj Deshmukh

Being Productive with Kubectl

This blog will showcase my productivity tips with kubectl . This does not venture into any plugins per se. But only using bash aliases to achieve it. Bash Aliases # k8s alias alias k=kubectl alias kg="kubectl get" alias kgp="kubectl get pods" alias kgs="kubectl get services" alias kge="kubectl get events" alias kgpvc="kubectl get pvc" alias kgpv="kubectl get pv" alias kd="kubectl describe" alias kl="kubectl logs -f" alias kc="kubectl create -f" I have above aliases setup in the ~/.bashrc file. The beauty of the aliases is that you can append more flags and parameters to the existing smaller alias. For, e.g. I have an alias for kubectl get pods as kgp, but if I want to get pods from all the namespaces, I use kgp -A. ...

August 2, 2020 · 3 min · Suraj Deshmukh

How to backup and restore Prometheus?

This blog will show you how to take a backup from a running Prometheus and restore it in some other Prometheus instance. You might ask why would you even want to do something like that? Well, sometimes you want the Prometheus metrics because they were collected for some particular purpose and you want to do some analysis later. Prerequisites/Assumptions This blog assumes that you have a Prometheus running that is deployed using prometheus-operator in monitoring namespace. But even if you have deployed it in some other way modify the commands in few places. ...

July 31, 2020 · 2 min · Suraj Deshmukh

Watch Container Traffic Without Exec

Introduction For the reasons of security, many container deployments nowadays run their workloads in a scratch based image. This form of implementation helps reduce the attack surface since there is no shell to gain access to, especially if someone were to break out of the application. But for the developers or operators of such applications, it is hard to debug. Since they lack essential tools or even bash for that matter, but the application’s debugging ability should not dictate its production deployment and compromise its security posture. ...

June 6, 2020 · 4 min · Suraj Deshmukh

Enabling Seccomp on your Prometheus Operator and related Pods

Seccomp helps us limit the system calls the process inside container can make. And PodSecurityPolicy is the way to enable it on pods in Kubernetes. Prometheus Operator Prometheus Operator makes it really easy to monitor your Kubernetes cluster. To deploy this behemoth, helm chart is the easiest way to do it. Almost all the pods that run as a part of Prometheus Operator viz. Prometheus Operator, Prometheus, Alertmanager, Grafana, Kube State Metrics don’t need to run with elevated privileges except Node Exporter. In your Kubernetes cluster if you are using PodSecurityPolicy to make sure that your cluster is secure, then you would want your Prometheus Operator pods to run securely as well. And the good news is, Prometheus Operator chart ships PodSecurityPolicy for each sub-component. We will look at how to enable seccomp for all the sub-components. ...

April 14, 2020 · 3 min · Suraj Deshmukh

Kubernetes Bangalore March 2019 Event Report

The Kubernetes Bangalore Meetup was organized at Arvind Internet on Feb 16th 2019. The agenda for the meetup was to teach Kubernetes to the beginners. Meetup agenda can be found here. The moments from Meetup: We go online in sometime here https://t.co/FkwgOx0Tm4 — Kubernetes Bangalore (@k8sBLR) March 16, 2019 .@pmishra1598 kick started the Meetup by explaining what #Kubernetes is! Currently clarifying what a pod is. pic.twitter.com/Ny7bN9c62x — Kubernetes Bangalore (@k8sBLR) March 16, 2019 Huge turnout at today's meetup it's on 🔥🔥 pic.twitter.com/YYbMBoumWw ...

March 21, 2019 · 1 min · Suraj Deshmukh