Jenkins operator

What is Jenkins operator and how to deploy Jenkins operator on Kubernetes [2022]

As per the Jenkins operator’s official site, The Jenkins Operator is a Kubernetes Native Operator who manages operations for Jenkins on Kubernetes. It has been built with Immutability and declarative Configuration as Code in mind.

In this blog post, we are going to talk about Custom Resource Definitions and will go over:

  • Why is the operator required?
  • What is Kubernetes operator
  • Features of operator
  • working of Jenkins operator.
  • we will set up a Jenkins operator
  • A few extra configurations.

To follow along, I assume you have prior knowledge of Kubernetes,kubectl, and Jenkins.

The operator manages the entire life cycle of Jenkins on a cloud-native application like Kubernetes. The operator added additional capabilities to Jenkins like

  • backups
  • observability
  • cloud security.

Before going deep dive into this, let’s first understand the issue with the previous Jenkins deployments.

Why jenkins operator is required?

Before the operator came into the picture, people installed Jenkins as a standalone machine. When using the Jenkins on a standalone server, you don’t have to worry about multiple servers and nodes. But as the no of jobs increases, people start facing issues with scalability. Sometimes jobs were failing in production because the resource was not adequate to fulfill the requirement.

To resolve this issue, people moved their Jenkins to the Kubernetes cluster. Running Jenkins on Kubernetes enables us to scale up our application to handle the additional load. Jenkins provides scaling features out-of-the-box by default. Jenkins on Kubernetes provides us following advantages as compared to traditional setup:

  • We don’t have to worry about corrupted agents, as Jenkins will automatically remove the unhealthy instance and spin up a new one.
  • We can easily run multiple jobs in parallel without worrying about executers and resources.
  • We don’t have to worry about the load distribution as Jenkins will automatically distribute the load on the best available server.

But we can not manage the entire life cycle of Jenkins by simply deploying it on Kubernetes.

To resolve the above limitations, the Kubernetes operator framework came into the picture in 2016 by engineers at CoreOS.A Jenkins operator monitors its application as it runs and can back up data, recover from failures, and automatically upgrade the application over time.

What is Kubernetes operator

As per Kubernetes operator official doc, A Kubernetes operator is a method of packaging, deploying, and managing a Kubernetes application. A Kubernetes Operator is an application-specific controller that extends the functionality of the Kubernetes API to create, configure, and manage instances of complex applications on behalf of a Kubernetes user.

Features of jenkins operator

How jenkins operator work

Jenkins operator watches for any manifest changes and maintains the desired state according to deployed custom resource(CR) manifest.

The base loop will take care of the base Jenkins configuration, which are:

  • Monitor change in state.
  • Monitor the status of the pod as take action accordingly.
  • Hardening, initial configuration, etc.

The base loop will take care of user-provided configuration, which are:

  • Create backup and restore jobs. Ensure backup and restore have been successfully performed.
  • Create seed job and ensure that all of them is successfully executed

Setup jenkins operator

Now we have a basic understanding of how the operator work, now let’s set up a basic Jenkins operator in the Kubernetes cluster. Here we will be installing the operator version:0.5. You should have admin access to the cluster to deploy the operator.

create a namespace called Jenkins-operator

➜  ~ kubectl create ns jenkins-operator
namespace/jenkins-operator created

Check if the namespace gets successfully created

➜  ~ kubectl get ns
NAME               STATUS   AGE
default            Active   7d14h
jenkins-operator   Active   3s

Now let’s create a Custom resource(CR). The code can be found here

custom resource(CR) are the resources that are by default not present in the Kubernetes cluster. We call them the custom resources because we manually create them. once you create a CR, a new endpoint will be added in the Kubernetes cluster. Admin access to a Kubernetes cluster is required to create a custom resource.

General yaml syntax for a custom resource(CR)

apiVersion: apiextensions.k8s.io/v1beta1
kind: CustomResourceDefinition
metadata:
  name: <name>
spec:
  group: <group>
  names:
  ......
  .....

apiextensions.k8s.ioendpoint API creates the CR

➜  ~ kubectl -n jenkins-operator create -f ./jenkins_v1alpha2_jenkins_crd.yaml
v1.22+; use apiextensions.k8s.io/v1 CustomResourceDefinition
customresourcedefinition.apiextensions.k8s.io/jenkins.jenkins.io created
customresourcedefinition.apiextensions.k8s.io/jenkinsimages.jenkins.io created
➜  ~

Let’s verify the Custom resource(CR) by typing the below command.

➜  ~ kubectl -n jenkins-operator get crd
NAME                          CREATED AT
jenkins.jenkins.io            2021-05-08T05:26:36Z
jenkinsimages.jenkins.io      2021-05-08T05:26:36Z

Now let’s proceed and deploy the Jenkins operator. The code for creating the Jenkins operator can be found here.

➜  ~ kubectl -n jenkins-operator create -f jenkins-operator.yaml
serviceaccount/jenkins-operator created
role.rbac.authorization.k8s.io/jenkins-operator created
rolebinding.rbac.authorization.k8s.io/jenkins-operator created
deployment.apps/jenkins-operator created
service/jenkins-operator-metrics created

Check for pod and other services in the jenkins-operator namespace

➜  ~ kubectl -n jenkins-operator get all
NAME                                   READY   STATUS    RESTARTS   AGE
pod/jenkins-operator-cfbklmfd-abk   1/1     Running   0          2m23s

NAME                               TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)             AGE
service/jenkins-operator-metrics   ClusterIP   00.00.00.123   <none>        8383/TCP,8686/TCP   2m24s

NAME                               READY   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/jenkins-operator   1/1     1            1           2m25s

NAME                                         DESIRED   CURRENT   READY   AGE
replicaset.apps/jenkins-operator-cf99b6ffd   1         1         1       2m25s

From the above o/p, it is clear that all the resources mentioned in YAML file are successfully created.

Now let’s deploy the actual Jenkins.

apiVersion: jenkins.io/v1alpha2
kind: Jenkins
metadata:
  name: example
spec:
  service:
    type: LoadBalancer
    port: 8080
  master:
    basePlugins:
    - name: kubernetes
      version: "1.29.2"
    - name: workflow-job
      version: "2.40"
    - name: workflow-aggregator
      version: "2.6"
    - name: git
      version: "4.5.0"
    - name: job-dsl
      version: "1.77"
    - name: configuration-as-code
      version: "1.47"
    - name: kubernetes-credentials-provider
      version: "0.15"
    containers:
    - name: jenkins-master
      image: jenkins/jenkins:2.263.3-lts-alpine
      imagePullPolicy: Always
      livenessProbe:
        failureThreshold: 12
        httpGet:
          path: /login
          port: http
          scheme: HTTP
        initialDelaySeconds: 80
        periodSeconds: 10
        successThreshold: 1
        timeoutSeconds: 5
      readinessProbe:
        failureThreshold: 3
        httpGet:
          path: /login
          port: http
          scheme: HTTP
        initialDelaySeconds: 30
        periodSeconds: 10
        successThreshold: 1
        timeoutSeconds: 1
      resources:
        limits:
          cpu: 1500m
          memory: 3Gi
        requests:
          cpu: "1"
          memory: 500Mi
      securityContext:
        runAsUser: 1000
        fsGroup: 1000
  seedJobs:
  - id: jenkins-operator
    targets: "cicd/jobs/*.jenkins"
    description: "Jenkins Operator repository"
    repositoryBranch: master
    repositoryUrl: https://github.com/jenkinsci/kubernetes-operator.git

Copy the above yaml into a file simple-Jenkins-deploy.yaml and run the below command to deploy Jenkins

➜  ~ kubectl -n jenkins-operator create -f simple-jenkins-deploy.yaml 
jenkins.jenkins.io/example created

Check if pods get created.

➜  ~ kubectl -n jenkins-operator get pods
NAME                                     READY   STATUS    RESTARTS   AGE
jenkins-example                          1/1     Running   0          8m30s
jenkins-operator-cfbklmfd-abk            1/1     Running   0          33m
seed-job-agent-example-6c456dd86-bwddj   1/1     Running   0          7m38s

You might also observe that a seed pod gets created along with a Jenkins pod. Now, let’s understand what is a seed job before proceeding further.

What is seed job

The seed job is a Jenkins job that runs a DSL script and generates a new job. The seed job is a normal freestyle Jenkins job that you add the “Process Job DSL” build step. Jenkins operator deployment using a groovy script to create a seed job. User have to prepare pipelines and job definitions in their GitHub repository using the following structure:

cicd/
├── jobs
│   └── k8s.jenkins
└── pipelines
    └── k8s.jenkins

Please refer to this doc for more information on seed jobs.

get Jenkins credential

➜  ~ kubectl -n jenkins-operator get secrets  jenkins-operator-credentials-example -o 'jsonpath={.data.user}' | base64 -d
jenkins-operator 
➜  ~ kubectl -n jenkins-operator get secrets  jenkins-operator-credentials-example -o 'jsonpath={.data.password}' | base64 -d
x6F5DqYRUZnhRLD73UtI 

Now let’s connect to Jenkins to URL

http://${LoadBalancerIp}:8080

Let’s try to connect to Jenkins by using the above credentials

jenkins operator

Congrats, you have successfully installed the Jenkins operator and deployed Jenkins on top of that. Now let’s try to run a sample job and see if everything is working

jenkins operator

Here you can see the job is successfully run.

More Configurations for jenkins operator

Backup and restore

Please follow this link to set up backup and restore for your Jenkins operator. This step is optional.

LDAP configuration

Please follow this link to integrate Jenkins with LDAP.

Conclusion

We have successfully set up a working Jenkins by using the operator. I hope you enjoyed this article. Happy learning 🙂

More to read

Kubernetes custom resource

Leave a Comment

Your email address will not be published.

Scroll to Top