Skip to Content
Software EngineeringBooksJenkins2 up and RunningChapter 1. Introducing Jenkins 2

Chapter 1. Introducing Jenkins 2

What Is Jenkins 2?

Jenkins 2 refers to newer versions of Jenkins that emphasize pipelines-as-code and provide enhanced features like Jenkinsfiles, which streamline the automation of CI/CD processes. While many of these features existed in earlier Jenkins versions (via plugins), Jenkins 2 signifies a core shift in how users interact with Jenkins. Key elements of Jenkins 2 include:

  1. Pipelines-as-Code:

    • Rather than configuring Jenkins jobs via the web interface, users can now define their CI/CD pipelines programmatically using Jenkinsfiles.
    • These files use a DSL (Domain-Specific Language) based on Groovy to define pipeline logic.

  2. Groovy Integration:

    • Jenkins has long included a Groovy engine, used for advanced scripting and access to additional Jenkins functionalities not available through the web UI.
    • In Jenkins 2, Groovy is tightly integrated into the pipeline DSL, allowing developers to define complex workflows programmatically.

  3. Node Keyword:

    • The node keyword is part of Jenkins’ DSL and is used to specify the node (formerly called “master” or “slave”) where the pipeline should execute tasks.

Key Features in Jenkins 2

  • Jenkinsfile: A file that resides in the project’s repository, which defines the pipeline stages and steps. This keeps the pipeline definition close to the code, ensuring consistency and better version control.

  • Declarative Pipelines: A more structured approach to defining pipelines, introduced in Jenkins 2, allowing for easier setup and maintenance compared to the older Scripted Pipelines.

  • Blue Ocean Interface: A modern UI introduced with Jenkins 2, offering a more visual and user-friendly way to view and manage pipelines.

The Jenkinsfile in Jenkins 2

In Jenkins 2, a Jenkinsfile is used to define the pipeline configuration in code, separating it from Jenkins’ internal storage. This file-based approach offers several key advantages over the previous Jenkins versions:

Key Features of Jenkinsfile

  1. Pipeline-as-Code:

    • In older versions, job definitions were stored within Jenkins’ configuration files in the Jenkins home directory, making them harder to track and modify.
    • Jenkins 2 allows you to store the pipeline configuration in a Jenkinsfile within your project’s source code repository, enabling version control, history tracking, and collaboration as with any other source code.

  2. Multiple Jenkinsfiles:

    • Each branch or project can have its own Jenkinsfile, allowing for different pipeline configurations per branch or project.

  3. Shared Libraries:

    • Jenkinsfiles can be extended to use external code or shared libraries, enabling you to reuse common pipeline steps across multiple pipelines.

  4. Marker File:

    • The presence of a Jenkinsfile in the project repository acts as a signal to Jenkins that the project is ready for pipeline execution. Jenkins can automatically detect and run jobs based on the contents of this file.

Advantages of Storing Pipelines in Jenkinsfile

  • Version Control: Since the Jenkinsfile is stored in the same repository as the code, it benefits from source control management (SCM) features like history tracking and diffs.
  • Collaborative Development: Multiple teams or developers can collaborate on the Jenkinsfile, tracking changes like any other code.
  • Flexibility: Allows pipelines to be updated and customized per project and branch, without needing direct access to Jenkins configuration.

Jenkinsfile as a Marker

  • The Jenkinsfile also serves a marker file purpose. Jenkins recognizes the file in your repository, which tells Jenkins:
    • What project and branch to use from source control.
    • How to execute the pipeline, based on the instructions in the Jenkinsfile, without additional configuration.

Comparison with Gradle’s build.gradle

  • Similar to how Gradle uses build.gradle files for project build definitions, Jenkins uses Jenkinsfile to define and manage CI/CD pipelines in code.

JobConfigHistory Plugin

  • For those still using the older XML-based configuration, Jenkins offers the JobConfigHistory plugin, which tracks changes to the XML configuration files. It allows users to view the history of changes made to jobs over time

Declarative Pipelines in Jenkins

Declarative Pipelines were introduced as a more structured and user-friendly alternative to the earlier Scripted Pipelines in Jenkins. The key difference lies in how the pipeline is written and structured. Here’s a detailed breakdown:

Scripted Pipelines

  • Structure: Scripted Pipelines are written primarily in Groovy and are highly flexible but require more manual control. The flow of the pipeline is governed by Groovy constructs like try-catch-finally blocks.
  • Error Handling: Error checking is less user-friendly; errors are often reported as Groovy tracebacks, which can make debugging difficult.
  • Use Case: Ideal for users familiar with Groovy programming who need more control over the pipeline.

Declarative Pipelines

Declarative Pipelines are designed to offer a more structured and simplified way of defining Jenkins pipelines.

Key Features

  1. Clear Structure:

    • Declarative Pipelines use a predefined syntax that enforces structure, making them easier to read, write, and maintain.
    • The overall structure is simpler and more similar to Freestyle projects (Jenkins’ older job configuration style).

  2. Enhanced DSL:

    • The DSL (Domain-Specific Language) for Declarative Pipelines adds new, higher-level constructs, making it easier to define pipelines without needing to write Groovy code.
    • Supports key Jenkins features like post-build actions (e.g., notifications) without additional scripting.

  3. Built-in Error Checking:

    • Declarative Pipelines offer better error checking, with clearer and more concise error messages that directly indicate where the issue lies, rather than providing Groovy tracebacks.

  4. Post-Build Processing:

    • The DSL now supports post-build steps (e.g., notifications, result processing) natively, making it simpler to define actions based on the build outcome (e.g., success or failure).
    • This eliminates the need for custom Groovy try-catch blocks, reducing complexity.

  5. Agent and Stage Definitions:

    • Declarative Pipelines use pipeline, agent, and stages blocks to define the flow:
      • agent specifies where the pipeline should run (e.g., any available node).
      • stages contain multiple stages such as build, test, deploy, etc.

  6. Error Messages:

    • Instead of Groovy traceback errors, Declarative Pipelines provide more targeted error messages that help developers quickly locate and fix issues in their pipelines.

Example

pipeline {
    agent any
    stages {
        stage('Source') { 
            steps {
                git branch: 'test', url: 'git@diyvb:repos/gradle-greetings'
                stash name: 'test-sources', includes: 'build.gradle, /src/test'
            }
        }
        stage('Build') {
            steps {
                // Build steps go here
            }
        }
    }
}
  • pipeline: The root block that defines the pipeline.
  • agent any: Tells Jenkins to run the pipeline on any available agent.
  • stages: Defines multiple stages. Each stage represents a distinct phase in the pipeline.
  • steps: Inside each stage, defines the specific tasks or steps (e.g., fetching code from a Git repository).

Benefits of Declarative Pipelines

  1. Simplicity: The formal structure and clear syntax make Declarative Pipelines easier to understand and maintain, even for less experienced users.
  2. Less Groovy: Users can focus on pipeline logic without needing to write Groovy code.
  3. Error Handling: Built-in error checking provides clearer, more specific feedback.
  4. Post-Build Actions: Supports notifications and other post-build processes without complex scripting.

Blue Ocean Interface in Jenkins 2

  • Blue Ocean is a modern, graphical user interface introduced in Jenkins 2 to improve the user experience, especially when working with Declarative Pipelines.
  • It provides an intuitive, visual representation of pipeline stages, allowing users to easily monitor progress and access detailed information about each stage.

Key Features of Blue Ocean

  1. Visual Pipeline Representation:

    • Blue Ocean displays each stage of a pipeline as a graphical block, showing the status of that stage (success, failure, in progress).
    • Users can quickly see which parts of the pipeline are working and which have failed.

  2. Stage Progress and Status Indicators:

    • The UI shows real-time progress for each stage of the pipeline, making it easier to track the overall execution.
    • Stages are color-coded to indicate their status: green for success, red for failure, and blue for in-progress.

  3. Log Access:

    • Each stage in Blue Ocean includes point-and-click access to logs specific to that stage.
    • Instead of searching through an entire pipeline log, users can view logs per stage, helping them quickly diagnose issues.

  4. Visual Editor:

    • Blue Ocean includes a basic visual editor for creating and editing Declarative Pipelines.
    • This allows users to define pipelines with minimal coding experience by using a drag-and-drop interface.

  5. Improved User Experience:

    • Blue Ocean simplifies pipeline management and monitoring, making it more accessible to a broader range of users, including those who may not be familiar with traditional Jenkins interfaces.

Reasons for the Shift to Jenkins 2

Jenkins 2 introduced major changes and improvements over the traditional Jenkins setup, driven by several internal and external factors. These reasons highlight why Jenkins evolved to focus on pipelines-as-code, Jenkinsfiles, and improved user experience.

1. The DevOps Movement

  • Continuous Integration (CI), Continuous Delivery (CD), and Continuous Deployment became essential practices in modern software development.
  • The rise of DevOps emphasized the need for tools that could support automation, pipelines, and efficient workflows from code to production.
  • Jenkins, as a major CI/CD tool, had to evolve to align with these practices by making pipelines a first-class feature, rather than a challenge to implement.

2. Assembling Pipelines

  • While creating individual jobs in Jenkins’ traditional Freestyle interface was manageable, building multi-job pipelines to handle end-to-end software delivery was complex.
  • Sharing data between jobs, handling workspaces, and chaining tasks required special configurations or plugins.
  • Jenkins 2 introduced pipelines-as-code to provide a native way to define complex pipelines in a unified, manageable format (Jenkinsfile).

3. Resumability

  • Durable pipelines became a key feature in Jenkins 2, allowing jobs to be paused, resumed, or even recovered after a master node restart.
  • In older Jenkins versions, users often had to restart jobs from scratch after a system failure or wade through logs to determine what needed to be restarted.
  • Jenkins 2’s pipelines ensure that jobs are serialized and can be resumed from where they left off.

4. Configurability

  • In Jenkins’ legacy setup, users relied on web-based forms to create and configure jobs, which involved multiple steps and interactions (clicking, dragging, typing).
  • Changing job workflows or reordering tasks was cumbersome, and performing iterative, logic-based operations was difficult through the GUI.
  • Jenkins 2 allowed users to configure jobs programmatically through text-based Jenkinsfiles, offering more flexibility, control, and ease of management.

5. Sharing Workspaces

  • Traditional Jenkins isolated workspaces for each job, which was beneficial for independent jobs but inefficient for pipelines where multiple jobs needed access to the same artifacts.
  • Sharing data between jobs often required complicated workarounds, such as defining custom workspaces or using additional plugins.
  • Jenkins 2 improved the ability to share workspaces and data between jobs, making pipelines more efficient without unnecessary repetition.

6. Specialized Knowledge

  • Legacy Jenkins often required users to have specialized knowledge or tricks to accomplish basic tasks, like transferring data between jobs or controlling job flow.
  • Jenkins 2 aimed to reduce this complexity by simplifying how pipelines are created and managed through a structured, DSL-based approach.

7. Access to Logic

  • With older Jenkins, job logic was hidden within web forms and XML configuration files, making it difficult to quickly grasp or share job setups.
  • Collaborating, debugging, or reviewing multijob pipelines was challenging without scrolling through several screens and configuration pages.
  • Jenkins 2 improved visibility and collaboration by storing pipeline definitions in Jenkinsfiles, making it easier to view, edit, and understand pipeline logic.

8. Pipeline Source Management

  • In legacy Jenkins, job definitions were stored in XML configuration files, separate from the source code. This separation made it difficult to manage, version, and audit job definitions.
  • Jenkins 2 introduced Jenkinsfiles stored alongside the source code in version control systems like Git, ensuring that pipeline configurations could be tracked, audited, and versioned just like any other part of the project.

9. Competition

  • Other tools that focused on pipelines-as-code, like Concourse (with pipelines described in YAML files and using containerization), began to emerge, offering competition to Jenkins.
  • To stay relevant and competitive, Jenkins needed to modernize and provide easier ways to define, manage, and execute pipelines.

Challenges

Jenkins 2 introduces several key features to address the challenges faced by legacy Jenkins.

1. Pipelines as First-Class Citizens

  • Core Focus on Pipelines: Jenkins 2 treats pipelines as a core feature, with native support throughout the platform, as opposed to older versions where pipelines were created by chaining individual jobs.
  • Direct Support: Pipelines are now a central part of Jenkins, providing native design and support to work with them as self-contained entities.

2. Programmatic Pipelines

  • Coding Pipelines: Users can define pipelines using code, allowing for greater flexibility in how they define workflows. This shift enables the use of programming logic, such as loops and conditionals, which were harder to implement in legacy Jenkins.
  • Advanced Logic: By allowing pipelines to be coded directly, users can define complex workflows and automation logic that was previously difficult to express through the configuration interface.

3. Structured DSL for Pipelines

  • Declarative DSL: Jenkins 2 introduces a structured Domain-Specific Language (DSL) specifically for pipelines. This allows users to easily program pipelines with defined syntax and structure, making the process more intuitive and organized.
  • Ease of Use: The DSL is designed to simplify pipeline creation and reduce the complexity of setting up workflows compared to traditional job configuration interfaces.

4. Pipeline Creation Without Web Forms

  • Script-Based Pipelines: Pipelines can be created as scripts without needing to fill out complex web forms, streamlining the process of setting up new pipelines. This reduces the need for multiple web interactions to define or modify jobs.
  • Jenkinsfiles: Pipelines can be defined entirely through Jenkinsfiles, which are stored alongside source code and versioned, making them easier to maintain and manage over time.

5. Jenkinsfiles Integrated with Source Code

  • Version Control Integration: Jenkinsfiles can be stored in the same repository as the source code, allowing them to be versioned and tracked like any other code file. This ensures better collaboration, auditing, and history tracking.
  • Separation from Jenkins: By decoupling pipeline definitions from Jenkins’ internal configuration, Jenkinsfiles make pipelines portable and easier to maintain across different environments.

6. File Sharing Across Workspaces

  • Workspace Sharing: The DSL now includes built-in functions that allow easy sharing of files across workspaces. This resolves one of the major pain points from legacy Jenkins, where sharing artifacts between jobs required complex configurations or plugins.

7. Advanced Docker Support

  • Docker Integration: Jenkins 2 includes advanced, built-in support for working with Docker containers. This makes it easier to define pipeline steps that run in containers, allowing for more consistent and isolated build environments.
  • Containerized Builds: With Docker support, Jenkins can use containers for building, testing, and deploying applications, simplifying the management of dependencies and environments.

Overall Benefits

  • Easier Maintenance: With pipelines stored as code and defined in Jenkinsfiles, they are easier to maintain, version, and update.
  • Better Testing and Resilience: The ability to handle exceptions through programmatic constructs, along with the resumability of pipelines, ensures that jobs can continue or restart even after failures or restarts of the master node.
  • Enhanced Flexibility: By allowing pipelines to be defined with code, Jenkins 2 opens up more flexible and powerful ways to manage continuous integration and delivery workflows, supporting complex workflows and logic that were previously hard to implement.

Compatibility

  • Jenkins 2 offers significant improvements over legacy Jenkins, but maintaining compatibility with traditional features and plugins remains a key aspect.
  • Jenkins 2 addresses compatibility through two pipeline styles—Scripted and Declarative Pipelines—while also ensuring plugins and global configurations adapt to the new pipeline model.

Pipeline Compatibility

Jenkins 2 supports two types of pipelines: Scripted and Declarative. Both aim to provide similar functionality to Freestyle jobs but through code-based pipelines.

Scripted Pipelines

  • Flexibility: Built using Groovy code, Scripted Pipelines offer flexibility but require users to handle certain tasks manually, like post-build actions.

  • Example: Sending post-build email notifications in a Scripted Pipeline requires using Groovy constructs like try-catch-finally, as there is no built-in support for post-build actions:

    node {
    try {
        // do some work
    } catch(e) {
        currentBuild.result = "FAILED"
        throw e
    } finally {
        mail to:"<buildAdmin@mycompany.com>",
             subject:"STATUS FOR PROJECT: ${currentBuild.fullDisplayName}",
             body: "RESULT: ${currentBuild.result}"
    }
}

  • Groovy-Based: Scripted Pipelines often require Groovy coding to mimic Jenkins features that are not available through DSL.

Declarative Pipelines

  • Structured Approach: Designed to provide a more user-friendly, structured syntax with built-in support for key Jenkins functionalities like post-build actions.

  • Post-Build Processing: In Declarative Pipelines, post-build steps are natively supported through the post block:

    pipeline {
    agent any
    stages {
        stage ("dowork") {
            steps {
                // do some work
            }
        }
    }
    post {
        always {
            mail to:"<buildAdmin@mycompany.com>",
                 subject:"STATUS FOR PROJECT: ${currentBuild.fullDisplayName}",
                 body: "RESULT: ${currentBuild.result}"
        }
    }
}

  • Ease of Use: Declarative Pipelines handle many tasks with simple, structured DSL constructs, reducing the need for Groovy coding.

Plugin Compatibility

Plugins are crucial in extending Jenkins’ functionality. With Jenkins 2, plugins need to meet new requirements to ensure they work seamlessly with pipelines.

Key Plugin Compatibility Requirements

  1. Surviving Restarts:

    • Pipelines in Jenkins 2 must be restartable, meaning they can continue or resume after a system restart.
    • Plugins need to ensure that any stateful objects are serializable so they can be preserved and resumed after a restart. Many legacy plugins may require updates to meet this requirement.

  2. Providing Scriptable APIs:

    • Plugins must provide APIs that can be used in pipeline scripts, replacing the traditional web-based form configurations.

    • For example, configuring an Artifactory plugin for pipelines involves calling API methods in the script:

      def server = Artifactory.server "LocalArtifactory"
def artifactoryGradle = Artifactory.newGradleBuild()
artifactoryGradle.tool = "gradle4"
artifactoryGradle.deployer repo:'libs-snapshot-local', server:server
artifactoryGradle.resolver repo:'remote-repos', server:server

    • Traditional checkbox-based configurations in Freestyle projects are now replaced by DSL API calls in pipeline scripts.

  3. Using Global Tools in Pipelines:

    • Jenkins allows pipeline scripts to reference global tools (like Git or Gradle) defined in the global configuration. For instance, using Gradle in a pipeline:

      stage('Compile') {
    sh "${tool 'gradle3.2'}/bin/gradle clean build"
}

    • Declarative Pipelines have a similar tool directive to integrate global tools.

Checking Plugin Compatibility

  • Jenkins users can check whether a plugin is pipeline-compatible through specific resources:
    • GitHub Repositories: Some sites on GitHub list pipeline-compatible plugins.
    • Pipeline Steps Reference: Available on Jenkins.io, this resource details pipeline-compatible plugins and their steps.

Global Configuration in Jenkins 2

In Jenkins 2, global configurations are split into two categories:

  1. Configure System:

    • Typically handles server setups or system-wide configurations.
    • Useful for setting up global parameters like security settings or server connections.

  2. Global Tool Configuration:

    • Manages standalone executable tools like Git, Gradle, or Maven.
    • These tools are referenced in pipeline scripts and can be configured globally for all projects.
Last updated on
Jenkins 2: Up and RunningChapters 2. The Foundations