As per the Jenkins operator’s official site, The Jenkins Operator is a Kubernetes Native Operator who manages operations for Jenkins on Kubernetes. It has been built with Immutability and declarative Configuration as Code in mind.
In this blog post, we are going to talk about Custom Resource Definitions and will go over:
- Why is the operator required?
- What is the Kubernetes operator
- Features of operator
- working of Jenkins operator.
- we will set up a Jenkins operator
- A few extra configurations.
To follow along, I assume you have prior knowledge of Kubernetes,kubectl, and Jenkins.
The operator manages the entire life cycle of Jenkins on a cloud-native application like Kubernetes. The operator added additional capabilities to Jenkins like
- backups
- observability
- cloud security.
Before going deep dive into this, let’s first understand the issue with the previous Jenkins deployments.
Why Jenkins operator is required?
Before the operator came into the picture, people installed Jenkins as a standalone machine. When using the Jenkins on a standalone server, you don’t have to worry about multiple servers and nodes. But as the no of jobs increases, people start facing issues with scalability. Sometimes jobs were failing in production because the resource was not adequate to fulfill the requirement.
To resolve this issue, people moved their Jenkins to the Kubernetes cluster. Running Jenkins on Kubernetes enables us to scale up our application to handle the additional load. Jenkins provides scaling features out of the box by default. Jenkins on Kubernetes provides us following advantages as compared to traditional setup:
- We don’t have to worry about corrupted agents, as Jenkins will automatically remove the unhealthy instance and spin up a new one.
- We can easily run multiple jobs in parallel without worrying about executors and resources.
- We don’t have to worry about load distribution as Jenkins will automatically distribute the load on the best available server.
But we can not manage the entire life cycle of Jenkins by simply deploying it on Kubernetes.
To resolve the above limitations, the Kubernetes operator framework came into the picture in 2016 by engineers at CoreOS.A Jenkins operator monitors its application as it runs and can back up data, recover from failures, and automatically upgrade the application over time.
What is the Kubernetes operator
As per the Kubernetes operator official doc, A Kubernetes operator is a method of packaging, deploying, and managing a Kubernetes application. A Kubernetes Operator is an application-specific controller that extends the functionality of the Kubernetes API to create, configure, and manage instances of complex applications on behalf of a Kubernetes user.
Features of Jenkins operator
- Maintain the full life cycle of Jenkins using operator SDK.
- Out-of-box connectivity with Jenkins Kubernetes plugin and pipeline as code plugin
- Performs an initial security hardening of the Jenkins instance.
- Enable us to extend Jenkins via a groovy script.
How Jenkins operators work
The Jenkins operator watches for any manifest changes and maintains the desired state according to deployed custom resource(CR) manifest.
The base loop will take care of the base Jenkins configuration, which are:
- Monitor change in state.
- Monitor the status of the pod and take action accordingly.
- Hardening, initial configuration, etc.
- The base loop will take care of user-provided configurations, which are:
- Create backup and restore jobs. Ensure backup and restore have been successfully performed.
- Create seed job and ensure that all of them is successfully executed
Setup Jenkins operator
Now we have a basic understanding of how the operator work, now let’s set up a basic Jenkins operator in the Kubernetes cluster. Here we will be installing the operator version:0.5. You should have admin access to the cluster to deploy the operator.
create a namespace called Jenkins-operator
➜ ~ kubectl create ns jenkins-operator namespace/jenkins-operator created
Check if the namespace gets successfully created
➜ ~ kubectl get ns NAME STATUS AGE default Active 7d14h jenkins-operator Active 3s
Now let’s create a Custom resource(CR). The code can be found here
custom resource(CR) are the resources that are by default not present in the Kubernetes cluster. We call them the custom resources because we manually create them. once you create a CR, a new endpoint will be added to the Kubernetes cluster. Admin access to a Kubernetes cluster is required to create a custom resource.
General yaml syntax for a custom resource(CR)
apiVersion: apiextensions.k8s.io/v1beta1 kind: CustomResourceDefinition metadata: name: <name> spec: group: <group> names: ...... .....
apiextensions.k8s.ioendpoint API creates the CR
➜ ~ kubectl -n jenkins-operator create -f ./jenkins_v1alpha2_jenkins_crd.yaml v1.22+; use apiextensions.k8s.io/v1 CustomResourceDefinition customresourcedefinition.apiextensions.k8s.io/jenkins.jenkins.io created customresourcedefinition.apiextensions.k8s.io/jenkinsimages.jenkins.io created ➜ ~
Let’s verify the Custom resource(CR) by typing the below command.
➜ ~ kubectl -n jenkins-operator get crd NAME CREATED AT jenkins.jenkins.io 2021-05-08T05:26:36Z jenkinsimages.jenkins.io 2021-05-08T05:26:36Z
Now let’s proceed and deploy the Jenkins operator. The code for creating the Jenkins operator can be found here.
➜ ~ kubectl -n jenkins-operator create -f jenkins-operator.yaml serviceaccount/jenkins-operator created role.rbac.authorization.k8s.io/jenkins-operator created rolebinding.rbac.authorization.k8s.io/jenkins-operator created deployment.apps/jenkins-operator created service/jenkins-operator-metrics created
Check for pod and other services in the Jenkins-operator namespace
➜ ~ kubectl -n jenkins-operator get all NAME READY STATUS RESTARTS AGE pod/jenkins-operator-cfbklmfd-abk 1/1 Running 0 2m23s NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE service/jenkins-operator-metrics ClusterIP 00.00.00.123 <none> 8383/TCP,8686/TCP 2m24s NAME READY UP-TO-DATE AVAILABLE AGE deployment.apps/jenkins-operator 1/1 1 1 2m25s NAME DESIRED CURRENT READY AGE replicaset.apps/jenkins-operator-cf99b6ffd 1 1 1 2m25s
From the above o/p, it is clear that all the resources mentioned in the YAML file are successfully created.
Now let’s deploy the actual Jenkins.
apiVersion: jenkins.io/v1alpha2 kind: Jenkins metadata: name: example spec: service: type: LoadBalancer port: 8080 master: basePlugins: - name: kubernetes version: "1.29.2" - name: workflow-job version: "2.40" - name: workflow-aggregator version: "2.6" - name: git version: "4.5.0" - name: job-dsl version: "1.77" - name: configuration-as-code version: "1.47" - name: kubernetes-credentials-provider version: "0.15" containers: - name: jenkins-master image: jenkins/jenkins:2.263.3-lts-alpine imagePullPolicy: Always livenessProbe: failureThreshold: 12 httpGet: path: /login port: http scheme: HTTP initialDelaySeconds: 80 periodSeconds: 10 successThreshold: 1 timeoutSeconds: 5 readinessProbe: failureThreshold: 3 httpGet: path: /login port: http scheme: HTTP initialDelaySeconds: 30 periodSeconds: 10 successThreshold: 1 timeoutSeconds: 1 resources: limits: cpu: 1500m memory: 3Gi requests: cpu: "1" memory: 500Mi securityContext: runAsUser: 1000 fsGroup: 1000 seedJobs: - id: jenkins-operator targets: "cicd/jobs/*.jenkins" description: "Jenkins Operator repository" repositoryBranch: master repositoryUrl: https://github.com/jenkinsci/kubernetes-operator.git
Copy the above yaml into a file simple-Jenkins-deploy.yaml and run the below command to deploy Jenkins
➜ ~ kubectl -n jenkins-operator create -f simple-jenkins-deploy.yaml jenkins.jenkins.io/example created
Check if pods get created.
➜ ~ kubectl -n jenkins-operator get pods NAME READY STATUS RESTARTS AGE jenkins-example 1/1 Running 0 8m30s jenkins-operator-cfbklmfd-abk 1/1 Running 0 33m seed-job-agent-example-6c456dd86-bwddj 1/1 Running 0 7m38s
You might also observe that a seed pod gets created along with a Jenkins pod. Now, let’s understand what is a seed job before proceeding further.
What is a seed job
The seed job is a Jenkins job that runs a DSL script and generates a new job. The seed job is a normal freestyle Jenkins job that you add the “Process Job DSL” build step. Jenkins operator deployment using a groovy script to create a seed job. User have to prepare pipelines and job definitions in their GitHub repository using the following structure:
cicd/ ├── jobs │ └── k8s.jenkins └── pipelines └── k8s.jenkins
Please refer to this doc for more information on seed jobs.
get Jenkins credential
➜ ~ kubectl -n jenkins-operator get secrets jenkins-operator-credentials-example -o 'jsonpath={.data.user}' | base64 -d jenkins-operator ➜ ~ kubectl -n jenkins-operator get secrets jenkins-operator-credentials-example -o 'jsonpath={.data.password}' | base64 -d x6F5DqYRUZnhRLD73UtI
Now let’s connect to Jenkins to URL
http://${LoadBalancerIp}:8080
Let’s try to connect to Jenkins by using the above credentials
Congrats, you have successfully installed the Jenkins operator and deployed Jenkins on top of that. Now let’s try to run a sample job and see if everything is working
Here you can see the job is successfully run.
More Configurations for Jenkins operator
Backup and restore
Please follow this link to set up a backup and restore for your Jenkins operator. This step is optional.
LDAP configuration
Please follow this link to integrate Jenkins with LDAP.
Conclusion
We have successfully set up a working Jenkins by using the operator. I hope you enjoyed this article. Happy learning :)