Jenkins common pipeline

 

Section	Description
parameters	Accepts user input before build: branch, environment, flags.
environment	Defines environment variables, including credential bindings.
options	Controls build retention, timeout, log formatting.
triggers	Automatically starts pipeline on SCM change or schedule.
stages > parallel	Runs stages in parallel (e.g., lint & coverage).
when	Conditional execution of stages (e.g., test only if RUN_TESTS == true).
stash/unstash	Transfers files between stages/nodes.
input	Manual approval before deploy (great for production safety).
docker.withRegistry	Builds and pushes Docker image using Docker Hub credentials.
post	Sends email notifications and performs cleanup actions.

 

pipeline {

  agent any

  environment {

    PYTHON_ENV = 'venv'

    ARTIFACTORY_URL = 'https://your.jfrog.instance/artifactory'

    ARTIFACTORY_REPO = 'python-release-local'

    CHECKMARX_SERVER = 'your-checkmarx-server-url'

  }

  stages {

    stage('Checkout from Bitbucket') {

      steps {

        git credentialsId: 'bitbucket-creds-id', url: 'https://bitbucket.org/org/project.git', branch: 'main'

      }

    }

    stage('Setup Python Environment') {

      steps {

        sh '''

          python3 -m venv ${PYTHON_ENV}

          . ${PYTHON_ENV}/bin/activate

          pip install --upgrade pip

          pip install -r requirements.txt

          pip install pytest pytest-cov

        '''

      }

    }

    stage('Static Code Analysis - Checkmarx') {

      steps {

        script {

          def scanStatus = sh(

            script: '''

              cx scan --project-name "MyPythonApp" --src . --sast-host ${CHECKMARX_SERVER} --report-format json --report-output checkmarx-result.json

            ''',

            returnStatus: true

          )

          if (scanStatus != 0) {

            error "❌ Checkmarx scan failed."

          }

          def highIssues = sh(script: "grep -c 'High' checkmarx-result.json || true", returnStdout: true).trim()

          if (highIssues.toInteger() > 0) {

            error "❌ High severity vulnerabilities found in Checkmarx. Failing pipeline."

          }

        }

      }

    }

    stage('Run Unit Tests and Coverage Check') {

      steps {

        script {

          def testStatus = sh(

            script: '''

              . ${PYTHON_ENV}/bin/activate

              pytest --junitxml=test-results.xml --cov=. --cov-report=term --cov-report=xml > coverage-output.txt || exit 1

            ''',

            returnStatus: true

          )

          if (testStatus != 0) {

            error "❌ Unit tests failed. Failing pipeline."

          }

          // Parse coverage and warn if < 80%

          def coverageLine = sh(

            script: "tail -n 20 coverage-output.txt | grep TOTAL || true",

            returnStdout: true

          ).trim()

          if (!coverageLine) {

            echo "⚠️ WARNING: TOTAL line not found in coverage output."

          } else {

            def matcher = coverageLine =~ /TOTAL\\s+\\d+\\s+\\d+\\s+\\d+\\s+(\\d+)%/

            if (matcher) {

              def coverage = matcher[0][1].toInteger()

              if (coverage < 80) {

                echo "⚠️ WARNING: Code coverage is ${coverage}%, which is below the 80% threshold."

              } else {

                echo "✅ Code coverage is ${coverage}%."

              }

            } else {

              echo "⚠️ WARNING: Could not parse coverage percentage."

            }

          }

        }

      }

      post {

        always {

          junit 'test-results.xml'

        }

      }

    }

    stage('Package Application') {

      steps {

        sh '''

          . ${PYTHON_ENV}/bin/activate

          python setup.py sdist bdist_wheel

        '''

      }

    }

    stage('Upload to JFrog Artifactory') {

      steps {

        sh '''

          jfrog rt upload dist/*.whl ${ARTIFACTORY_REPO}/

          jfrog rt upload dist/*.tar.gz ${ARTIFACTORY_REPO}/

        '''

      }

    }

  }

  post {

    failure {

      mail to: 'team@example.com',

           subject: "❌ Build Failed: ${env.JOB_NAME} #${env.BUILD_NUMBER}",

           body: "Build failed at stage: ${env.STAGE_NAME}\n\nCheck Jenkins logs for details: ${env.BUILD_URL}"

    }

  }

}

Secret Management in Jenkins Pipelines

To keep things secure and clean, never hardcode secrets (e.g., API keys, passwords) into your Jenkinsfile.

✅ Use Jenkins Credentials:

• Store secrets in Jenkins > Manage Jenkins > Credentials

• Types:

  • Username & Password

  • Secret Text

  • SSH Key

  • File (e.g., service account JSON)

📌 Example:

environment {
  ARTIFACTORY_CRED = credentials('jfrog-cred-id')
}

stages {
  stage('Upload to Artifactory') {
    steps {
      sh '''
        jfrog rt config --url $ARTIFACTORY_URL --user $ARTIFACTORY_CRED_USR --apikey $ARTIFACTORY_CRED_PSW --interactive=false
        jfrog rt upload dist/*.whl ${ARTIFACTORY_REPO}/
      '''
    }
  }
}

✅ credentials('id') injects *_USR and *_PSW automatically.

📚 Shared Libraries for Reusable Logic

Jenkins Shared Libraries help avoid repeating code like:

• Python virtualenv setup

• Coverage parsing

• Artifactory upload

• Checkmarx scanning logic

📁 Directory Structure (Git Repo for Shared Lib):

python

(vars, src, resources are required structure)
.
└── vars/
    └── pythonSetup.groovy
    └── runCheckmarx.groovy

🧩 Sample vars/pythonSetup.groovy

groovy

def call() {
  sh '''
    python3 -m venv venv
    . venv/bin/activate
    pip install --upgrade pip
    pip install -r requirements.txt
    pip install pytest pytest-cov
  '''
}

🔧 Use in Jenkinsfile:

groovy

@Library('my-shared-library') _

pipeline {
  agent any
  stages {
    stage('Setup Python') {
      steps {
        pythonSetup()
      }
    }
  }
}

---

✅ How to Configure Shared Library in Jenkins:

1. Go to Jenkins → Manage Jenkins → Configure System

2. Scroll to Global Pipeline Libraries

3. Add:

   • Name: my-shared-library

   • Source: Git repo (public or private)

   • Default Version: main or master

---

📌 TL;DR to Include in Your Jenkinsfile:

You can include this note at the top of your Jenkinsfile:

groovy

// Jenkinsfile
// Secrets are managed via Jenkins Credentials and injected using `credentials()`
// Common logic (e.g., virtualenv setup, artifact upload) can be offloaded to a Shared Library
// Add Shared Library from Jenkins UI: Manage Jenkins > Configure System > Global Pipeline Librari




✅ How to Set Up Webhook from Bitbucket Server (On-Prem) to Jenkins 
🔧 Step 1: Prepare Jenkins 
✅ Option A: For Multibranch Pipelines (Recommended) 
Install these plugins: 
Bitbucket Branch Source Plugin 
Bitbucket Plugin 
Jenkins URL for Webhook: 
http://<your-jenkins>/bitbucket-hook/ 
 
This is a generic endpoint Jenkins exposes to listen for Bitbucket webhook events. 
 
🔧 Step 2: Configure Webhook in Bitbucket Server (on-prem) 
Go to the repository in Bitbucket Server. 
Navigate to: 
Repository settings → Webhooks 
 
Click Create webhook 
Fill in: 
Name: Jenkins CI Hook 
URL: 
For multibranch: http://<jenkins>/bitbucket-hook/ 
Events: 
✅ Push 
✅ Pull Request Created (optional) 
✅ Pull Request Merged (optional) 
Save the webhook. 
 
🧪 Step 3: Test the Webhook 
Push a commit or create a pull request in Bitbucket. 
Go to Jenkins and verify that the job was triggered. 
In Bitbucket, go to: 
Repository settings → Webhooks → View recent deliveries 
 
to debug payloads and responses. 
 
🛡️ Optional: Secure Jenkins with Credentials 
If your Jenkins requires authentication: 
Add credentials to the webhook URL like: 
http://username:token@jenkins.yourcompany.com/bitbucket-hook/ 
 
Or use Jenkins API tokens + credentials plugin. 
 
📌 Notes 
Component 
Required? 
Purpose 
bitbucket-hook/ 
✅ 
Listens for Bitbucket webhook payloads 
Jenkins token 
✅ (freestyle only) 
Authorizes remote build trigger 
Branch Source Plugin 
✅ (multibranch) 
Auto-discovers branches & PRs 
Public Jenkins URL 
✅ 
Bitbucket must reach Jenkins endpoint 
 


✅ How Jenkins Detects Unit Test Failures

🔍 Behavior



Most test runners (e.g. pytest, JUnit, mocha, etc.) return a non-zero exit code if tests fail.


Jenkins will mark the build as failed if any step in the pipeline returns a non-zero code.



🔧 Example 1: Simple Pipeline With Unit Test Step (e.g. Python/pytest)

groovy
pipeline {
  agent any

  stages {
    stage('Checkout') {
      steps {
        checkout scm
      }
    }

    stage('Install') {
      steps {
        sh 'pip install -r requirements.txt'
      }
    }

    stage('Run Unit Tests') {
      steps {
        sh 'pytest tests/' // will fail pipeline if tests fail
      }
    }
  }
}

✅ If pytest finds failing tests, it will return exit code 1, which fails the pipeline.