Jenkins Agents: Your Path to Smooth CI/CD Scaling

👤Jenkins Master (Server)

• Jenkins’s server or master node holds all key configurations. The Jenkins master server is like a control server that orchestrates all the workflow defined in the pipelines. For example, scheduling a job, monitoring the jobs, etc.

👤Jenkins Agent

• An agent is typically a machine or container that connects to a Jenkins master and this agent actually executes all the steps mentioned in a Job. When you create a Jenkins job, you have to assign an agent to it. Every agent has a label as a unique identifier.

• When you trigger a Jenkins job from the master, the actual execution happens on the agent node that is configured in the job.

• A single, monolithic Jenkins installation can work great for a small team with a relatively small number of projects. As your needs grow, however, it often becomes necessary to scale up. Jenkins provides a way to do this called “master to agent connection.” Instead of serving the Jenkins UI and running build jobs all on a single system, you can provide Jenkins with agents to handle the execution of jobs while the master serves the Jenkins UI and acts as a control node.

  

👤Configure Jenkins Agent

Let's understand how to set up Jenkins master-agent setup.

• The first step is to create two EC2 instances and name them accordingly, I named them jenkins_master and Jenkins_agent.

  

• Now Go to the jenkins_master instance install docker and jenkins , add $USER and jenkins to docker group using sudo usermod -aG docker $USER & sudo usermod -aG docker jenkins . If you followd yesterdays blog then it will be more easy for you to underdstand this one.

• In jenkins_master : go to .ssh directory and generate key

    cd .ssh
    ssh-keygen 
    # or i would suggest to go with this
    ssh-keygen -t ed25519
  

  

• Copy the public key (id_rsa.pub) use cat id_rsa.pub to display the key

  

• Paste it in jenkins_agent instance : go to the jenkins_agent instance , copy the public key in .ssh/authorized_keys .

  

  

Come to the Jenkins_master instance and try to connect to the jenkins_agent via ssh

• This shows that I'm successfully connected to jenkins_agent (check the IP) , now we can proceed with the next step.

Setting-up Agent in Jenkins

• Go to Jenkins, just like we did in our previous blog ip:8080, go to set up an agent on the dashboard ( if you don't see that go to manage Jenkins> Nodes )

  

• It will ask to set node name, check mark on permanent agent and then create !!

Then on the next page, you will be asked to enter a few details such as description, remote directory, Host ip and credentials

• In the label, u can give any name, suppose I'll deploy a Django app I'll label it as Django, I named it a dev to deploy all development parts.

• In the launch method select it as Launch agents via SSH as we have shared the public key with jenkins_agents.

• Host: give IP of jenkins_agent (agent instance)

• Credentials: This is the major part, click on add

  

  

  • Select the SSH username with the private key, id to store the private key and scroll down and add your private key to this

    use cat id_rsa to display

    

• In the Host key verification strategy select Non-Verifying Verification strategy

  

• Click on Create and tadaaa agent node is created !!

  

• Refresh it once or click on the node and click on Launch node

  

Deploying the project

As we have seen in this blog we will be deploying the same in the agent node.

The only difference in the pipeline script is

pipeline {
    agent { label "dev-agent" }
    stages {
        stage("Clone Code") {
            steps {
                echo "Cloning the code"
                git url:"https://github.com/srahul0502/django-todo-cicd.git", branch: "develop"
            }
        }
        stage("Build") {
            steps {
                echo "Building the image"
                sh "docker build -t django-app ."
            }
        }
        stage("Push To Docker Hub") {
            steps {
                echo "Pushing the image to docker hub"
                withCredentials([usernamePassword(credentialsId:"dockerHub" ,passwordVariable:"dockerHubPass" ,usernameVariable:"dockerHubUser")]){
                sh "docker tag django-app ${env.dockerHubUser}/django-app:latest"
                sh "docker login -u ${env.dockerHubUser} -p ${env.dockerHubPass}"
                sh "docker push ${env.dockerHubUser}/django-app:latest"
                }
            }
        }
        stage("Deploy") {
            steps {
                echo "Deploying the container"
                sh "docker run -d -p 8000:8000 srahul0502/django-app:latest"
            }
        }
    }
}

• The only difference is in agents : we have specified the agent on which we want to deploy.

• You can see this on the lefthand side bottom

Error Handling💀:

• You might get an error while setting up the agent on Jenkins

• This might be due to extra spaces or incorrect key

• You can resolve this by generating another key as I have mentioned in the above steps ssh-keygen ed25519

• Difference between ed25519 and rsa :

  • Ed25519: Ed25519 uses 256-bit keys, making it relatively short but still secure due to the strength of the elliptic curve.

  • RSA: RSA keys are generally longer, often 2048 or 4096 bits, to achieve similar levels of security.

I hope you found this useful

kindly go through this to understand the deployment of the project

Happy Learning :D