Jenkins End to End CI/CD Pipeline. Jenkins Pipeline for Java based application using Maven, SonarQube, Argo CD and Kubernetes
1. Create an EC2 instance in AWS:
Log in to the AWS Management Console and launch instance as Ubuntu t2.large instance type.
add inbound rules for-
Port 8080: for Jenkins
Port 9000: for SonarQube
2. Install and configure Jenkins:
SSH into the EC2 instance using a terminal or SSH client.
Update the system packages: sudo apt update
Install Jenkins by following the official Jenkins documentation for Ubuntu: jenkins.io/doc/book/installing/linux
Once installed, access Jenkins by navigating to http://<public-ip>:8080 in a web browser.
Follow the on-screen instructions to complete the Jenkins setup, including installing suggested plugins.
3. Fork the GitHub repository and create a webhook:
Fork the repository at https://github.com/krishkprawat/Jenkins-Zero-To-Hero/
In your forked repository, go to "Settings" > "Webhooks" > "Add webhook".
Set the Payload URL to http://<public-ip>:8080/github-webhook/
Select "Just the push event" under "Which events would you like to trigger this webhook".
Save the webhook configuration.
4. Build a new item in Jenkins:
5. Installing plugins:
Install the docker pipeline plugin:
Install SonarQube Scanner plugin:
6. Install SonarQube in Ubuntu and configure it in Jenkins:
SSH into the EC2 instance.
Follow the official SonarQube documentation for Ubuntu installation: docs.sonarqube.org/latest/setup/install-ser..
# Install the 'unzip' package
apt install unzip
# Create a new user named 'sonarqube'
adduser sonarqube
# Switch to the 'sonarqube' user
sudo su - sonarqube
# Download SonarQube zip package
wget https://binaries.sonarsource.com/Distribution/sonarqube/sonarqube-9.4.0.54424.zip
# Unzip the downloaded package
unzip *.zip
# Set appropriate permissions for the SonarQube directory
chmod -R 755 /home/sonarqube/sonarqube-9.4.0.54424
# Change ownership of the SonarQube directory to the 'sonarqube' user
chown -R sonarqube:sonarqube /home/sonarqube/sonarqube-9.4.0.54424
# Change the current directory to the SonarQube bin directory
cd sonarqube-9.4.0.54424/bin/linux-x86-64/
# Start SonarQube
./sonar.sh start
Once SonarQube is installed and running, access it through
http://<public-ip>:9000
7. Adding Credentials in Jenkins
Connecting Jenkins and SonarQube:
Generate token in SonarQube:
Copy the token.
Create credentials in jenkins:
Paste the token in secret.
Adding credentials for DockerHub in Jenkins:
Adding credentials for github in Jenkins:
Generate a token:
Copy the token and paste on secret:
Restart Jenkins.
8. Install Docker and configure it in Jenkins:
SSH into the EC2 instance.
Follow the official Docker documentation for Ubuntu installation: docs.docker.com/engine/install/ubuntu
# Update the package lists
sudo apt update
# Install Docker
sudo apt install docker.io
# Switch to the root user
sudo su -
# Add the 'jenkins' user to the 'docker' group
usermod -aG docker jenkins
# Add the 'ubuntu' user to the 'docker' group
usermod -aG docker ubuntu
# Grant permissions to the Docker socket
sudo chmod 666 /var/run/docker.sock
# Restart the Docker service
systemctl restart docker
9. Writing Jenkinsfile and manifest file:
pipeline {
agent {
docker {
image 'abhishekf5/maven-abhishek-docker-agent:v1'
args '--user root -v /var/run/docker.sock:/var/run/docker.sock' // mount Docker socket to access the host's Docker daemon
}
}
stages {
stage('Checkout') {
steps {
git branch: 'main', url: 'https://github.com/krishkprawat/Jenkins-Zero-To-Hero.git'
}
}
stage('Build and Test') {
steps {
sh 'ls -ltr'
// build the project and create a JAR file
sh 'cd java-maven-sonar-argocd-helm-k8s/spring-boot-app && mvn clean package'
}
}
stage('Static Code Analysis') {
environment {
SONAR_URL = "http://3.95.165.9:9000"
}
steps {
withCredentials([string(credentialsId: 'sonarqube', variable: 'SONAR_AUTH_TOKEN')]) {
sh 'cd java-maven-sonar-argocd-helm-k8s/spring-boot-app && mvn sonar:sonar -Dsonar.login=$SONAR_AUTH_TOKEN -Dsonar.host.url=${SONAR_URL}'
}
}
}
stage('Build and Push Docker Image') {
environment {
DOCKER_IMAGE = "krishpauri/ultimate-cicd:${BUILD_NUMBER}"
// DOCKERFILE_LOCATION = "java-maven-sonar-argocd-helm-k8s/spring-boot-app/Dockerfile"
REGISTRY_CREDENTIALS = credentials('docker-cred')
}
steps {
script {
sh 'cd java-maven-sonar-argocd-helm-k8s/spring-boot-app && docker build -t ${DOCKER_IMAGE} .'
def dockerImage = docker.image("${DOCKER_IMAGE}")
docker.withRegistry('https://index.docker.io/v1/', "docker-cred") {
dockerImage.push()
}
}
}
}
stage('Update Deployment File') {
environment {
GIT_REPO_NAME = "Jenkins-Zero-To-Hero"
GIT_USER_NAME = "krishkprawat"
}
steps {
withCredentials([string(credentialsId: 'github', variable: 'GITHUB_TOKEN')]) {
sh '''
git config user.email "krishkprawat@gmail.com"
git config user.name "krishnapal singh rawat"
BUILD_NUMBER=${BUILD_NUMBER}
sed -i "s/replaceImageTag/${BUILD_NUMBER}/g" java-maven-sonar-argocd-helm-k8s/spring-boot-app-manifests/deployment.yml
git add java-maven-sonar-argocd-helm-k8s/spring-boot-app-manifests/deployment.yml
git commit -m "Update deployment image to version ${BUILD_NUMBER}"
git push https://${GITHUB_TOKEN}@github.com/${GIT_USER_NAME}/${GIT_REPO_NAME} HEAD:main
'''
}
}
}
}
}
Pipeline -
1st is checkout
2nd is maven clean package
Maven clean package find pom.xml and this(pom) is responsible for describing the dependencies. Maven clean package will download all the packages which are listed on pom file. This action wil create a jar file .and this jar file we copying and executing in the dockerfile as well.
3rd- static code analysis - sonarqube
Give url and give maven target for sonar.
4th- build and push docker image
5th- run a shell script and update the image repository with new image tag then use argo cd to pull this repo and deploy to k8 cluster.
"In jenkins - add secrets for docker cred-name and pass, sonartoken, github access token as secret text .and restart jenkins."
Manifest: deployment.yml file
apiVersion: apps/v1
kind: Deployment
metadata:
name: spring-boot-app
labels:
app: spring-boot-app
spec:
replicas: 2 # Number of replicas for the deployment
selector:
matchLabels:
app: spring-boot-app
template:
metadata:
labels:
app: spring-boot-app
spec:
containers:
- name: spring-boot-app
image: krishpauri/ultimate-cicd:replaceImageTag # Docker image for the container
ports:
- containerPort: 8080 # Port on which the container listens
10. Build the pipeline:
Create a repository in your dockerhub: ultimate-cicd or as per the changes you made in your groovy script.
Then, build your pipeline.
After the build is successful, an image is pushed to dockerhub on build and push docker image stage. We can check it from dockerhub:
Now let’s perform continuous deployment using ArgoCD in Kubernetes in the local system
To install and use Minikube on Ubuntu, you can follow these steps:
1. Install Dependencies:
- Open a terminal/EC2 instance of t2.medium then install kubectl and start a minikube- see this document for reference https://www.fosstechnix.com/how-to-install-argocd-on-minikube/
2. Installing and configuring Argo CD
Goto: operatorhub.io/operator/argocd-operator
we can install argo cd by different methods also.
Create: 1 file (eg: argocd.yml) and paste the following code into the file:
From: argocd-operator.readthedocs.io/en/latest/us..
apiVersion: argoproj.io/v1alpha1
kind: ArgoCD
metadata:
name: example-argocd # Name of the Argo CD instance
labels:
example: basic # Labels for identification or categorization
spec: {} # Empty specification, no additional configuration provided
# Apply the configuration in the argocd.yml file
kubectl apply -f argocd.yml
# Get the pods in the cluster
kubectl get pods
get services and expose the argocd-server by changing the type :nodeport
# Get the list of services in the cluster
kubectl get svc
# Edit the example-argocd-server service
kubectl edit svc example-argocd-server
Change type: from ClusterIP to NodePort
# Get the list of services in the cluster
kubectl get svc
# List the services exposed by minikube
minikube service list
You'll get the URL from where you can access the ArgoCD dashboard.
Username: admin
for password:
# Get the list of secrets in the cluster
kubectl get secret
# Edit the example-argocd-cluster secret
kubectl edit secret example-argocd-cluster
copy admin.password
The password is base64 encrypted, so in terminal:
# Deode the base64-encoded string
echo T21vcWJTS1VJOHd6WkdjanAzYUZDRUIxSm5OUjJ2bDQ= | base64 -d
Copy the output and use it as a password.
4. Setting up Argo CD for deployment in K8s
Application deployed successfully:
Checking from terminal:
Two pods are running as the replica is set to 2 in deployment.yml file.
5. Testing the deployment from the browser:
# kubectl describe pod <pod_name>
# Describe the details of the pod with the name spring-boot-app-5878ccfc4-gbzfj
kubectl describe pod <pod name>
# kubectl port-forward pod/<pod_name> <local_port>:<application_port>
# Forward the local port 8010 to the port 8080 of the pod spring-boot-app-5878ccfc4-gbzfj
kubectl port-forward pod/spring-boot-app-5878ccfc4-gbzfj 8010:8080
Checking the application from the browser:
To summarize, this tutorial offers a detailed and hands-on walkthrough for establishing a resilient CI/CD pipeline utilizing AWS, Jenkins, Docker, SonarQube, and ArgoCD. By following the step-by-step instructions, you can successfully automate the build, test, and deployment processes of your applications. This tutorial equips you with the knowledge and hands-on experience necessary to streamline your software development workflow, increase efficiency, and ensure consistent delivery of high-quality software.
Thank you for reading and Happy Learning! 🎉 thanks Abhishek Veeramalla for this tutorial.- https://www.youtube.com/watch?v=JGQI5pkK82w&t=604s