Inroductory Plugin Tutorial

Introduction

This tutorial provides instructions for the 4 step process to create a functional plugin called MyJenkins which retrieves the last build number from a jenkins job and stores it in an output parameter.

The 4 Step Plugin creation Process

  1. Create Plugin Workspace.

  2. Define plugin spec and generate plugin.

  3. Implement Additional logic.

  4. Build plugin and test.

Prerequisites

These are assumptions for this Tutorial.

  1. An active CloudBees Flow instance.

  2. A Machine with Docker installed which is registered as a pingable resource in CloudBees Flow.

  3. Internet connection.

  4. flowdpk installed and setup.

Install and Setup Jenkins

Pull the Docker Jenkins image that already has electricflow agent installed from Docker Hub.

docker pull electricflow/jenkins

Important: Preface sudo with docker commands if in case you have not setup Docker to run as a non-root user.

Start the Jenkins Docker Container as follows.

docker run  -p "8080:8080" -d electricflow/jenkins:latest

Do the following to confirm that your container is running:

ubuntu@ubuntu:~$ sudo docker ps
CONTAINER ID        IMAGE                         COMMAND                  CREATED             STATUS              PORTS                              NAMES
0d84b3b113a9        electricflow/jenkins:latest   "/bin/bash -c /work/…"   2 weeks ago         Up 2 weeks          8043/tcp, 0.0.0.0:8080->8080/tcp   sleepy_khayyam

The above snippet is provided for reference only. Your output would be similar, but not exactly the same as above.

Once you see that docker container is running, point your browser to your docker machine address with 8080 port. If you’re using localhost, just visit http://localhost:8080 . You should see a jenkins instance login screen. The credentials for this jenkins instance are admin/changeme.

If you can login to the above instance, you are done with your Jenkins environment setup.

Step 1 : Create Plugin Workspace

After making sure flowpdk is available in your PATH, you can create a plugin workspace.

Change to your working directory

cd ~/work

Call flowpdk as follows

flowpdk generate workspace

In the interactive prompt type MyJenkins as plugin name. For the rest of the options which are all optional, choose defaults if available or provide your own values.

Generated Workspace

Generated Workspace

The generated workspace would have the following 2 files:

Files in Workspace

Files in Workspace

Step 2: Define plugin spec and generate plugin

The pluginspec.yaml has 2 sections one for configuration and the other for procedures.

Update config section of YAML

To interact with Jenkins we need the following config values:

  1. Jenkins endpoint

  2. Username for authorization.

  3. Password for authorization.

Fill up the config section as follows.

configuration:
  # Shell to be used for checking connection
  shell: ec-perl
  parameters:
   - name: config
     type: entry
     label: Configuration Name
     required: true
     documentation: The name for the created configuration
   - name: desc
     type: entry
     label: Description
     required: false
     documentation: Description for the configuration
   - name: endpoint
     label: Endpoint
     type: entry
     required: true
     documentation: A jenkins URL with port.
   - name: basic_credential
     type: credential
     label: Credential
     userNameLabel: Username
     required: true
     passwordLabel: Password
   - name: debugLevel
     # This parameter name "debugLevel" is supported by ECPDF Logger out of the box
     label: Debug Level
     type: select
     required: 0
     value: 0
     options:
       - name: Info
         value: 0
       - name: Debug
         value: 1
       - name: Trace
         value: 2

In this tutorial we will assume that the Jenkins environment is protected using the “Prevent Cross Site Request Forgery exploits” security. This means when the plugin makes a POST request, a CSRF protection token should be sent as part of the HTTP request header.

Rename the auto-generated specification called credential to basic_credential.

Update procedure section

We will implement a procedure called “Get Last Build Number” with interface specification as follows.

procedures:
- name: Get Last Build Number
  description: This procedure gets last build number from provided Jenkins job.
  hasConfig: true # configuration field will be generated automatically
  parameters:
  - name: jenkinsJobName
    documentation: A name of the Jenkins job to get the latest build number.
    type: entry
    required: true
    label: Jenkins Job Name
  outputParameters:
    lastBuildNumber: A last build number for job.
  # Steps are not defined so we assume that this is one-step procedure with a single step named Deploy
  shell: ec-perl

Generate plugin

Execute the following command from the root level directory of the plugin workspace.

flowpdk generate plugin

After execution, it should look as follows.

Plugin Generation

Plugin Generation

With this step plugin generation is complete.

Go through the auto-generated code

This section is provided to give a perspective on how the boiler plate generated code looks like and what it means.

The dsl/properties folder has the following structure:

  • perl/core

  • perl/lib

Important: Do not modify any file under core folder. Core folder has plugin-related internal code, which should not be edited by plugin developer.

The only folder that could be modified is the perl/lib folder.

Notice that dsl/properties/perl/lib/FlowPlugin/MyJenkins.pm has been auto-generated and contains the following code.

package FlowPlugin::MyJenkins;
use strict;
use warnings;
use base qw/FlowPDF/;

use FlowPDF::Log;
# Feel free to use new libraries here, e.g. use File::Temp;

# Service function that is being used to set some metadata for a plugin.
sub pluginInfo {
    return {
        pluginName          => '@PLUGIN_KEY@',
        pluginVersion       => '@PLUGIN_VERSION@',
        configFields        => ['config'],
        configLocations     => ['ec_plugin_cfgs'],
        defaultConfigValues => {}
    };
}

# Auto-generated method for the procedure Get Last Build Number/Get Last Build Number
# Add your code into this method and it will be called when step runs
sub getLastBuildNumber {
    my ($pluginObject, $runtimeParameters, $stepResult) = @_;

    my $context = $pluginObject->newContext();
    logInfo("Current context is: ", $context->getRunContext());
    my $params = $context->getStepParameters();
    logInfo("Step parameters are: ", $params);

    my $configValues = $context->getConfigValues();
    logInfo("Config values are: ", $configValues);

    $stepResult->setJobStepOutcome('warning');
    $stepResult->setJobSummary("This is a job summary.");
}
## === step ends ===
# Please do not remove the marker above, it is used to place new procedures into this file.


1;

Each plugin procedure has one or more steps. The auto-generated code for the step “Get Last Build Number”. is located in dsl/procedures/GetLastBuildNumber/steps/GetLastBuildNumber.pl and looks like the following :

$[/myProject/perl/core/scripts/preamble.pl]

use FlowPlugin::MyJenkins;
# Auto generated code of plugin step
# Go to dsl/properties/perl/lib/EC/Plugin/MyJenkins.pm and change the function "getLastBuildNumber"
FlowPlugin::MyJenkins->runStep('Get Last Build Number', 'Get Last Build Number', 'getLastBuildNumber');

Step 3: Implement Additional logic

Add default config values:

Edit pluginInfo function to set a default values for parameters. Notice that authScheme is set to the value basic.

sub pluginInfo {
    return {
        pluginName          => '@PLUGIN_KEY@',
        pluginVersion       => '@PLUGIN_VERSION@',
        configFields        => ['config'],
        configLocations     => ['ec_plugin_cfgs'],
        defaultConfigValues => {
            authScheme => 'basic'
        },
    };
}

Add necessary library imports

Since Jenkins returns a JSON, add appropriate libraries that can be used to parse, write them to flow logs as necessary or throw exceptions.

package FlowPlugin::MyJenkins;
use strict;
use warnings;
use base qw/FlowPDF/;

use FlowPDF::Log;
use FlowPDF::Helpers qw/bailOut/;
use Data::Dumper;
use JSON;
use Try::Tiny;

Retrieve the Jenkins CSRF Token

Note that the CSRF Token is also called as the Jenkins crumb.

Implement a function that takes the endpoint returned by plugin configuration and appends an additional path to it.

sub getBaseUrl {
    my ($self, $url) = @_;

    if (!$url) {
        bailOut("URL is mandatory parameter");
    }
    # retrieving runtime parameters
    my $runtimeParameters = $self->getContext()->getRuntimeParameters();
    # endpoint is a field from configuration
    my $endpoint = $runtimeParameters->{endpoint};

    # removing trailing slash.
    $endpoint =~ s|\/+$||gs;

    # appending url
    my $retval = $endpoint . '/' . $url;
    return $retval;
}

For example if the endpoint is https://foobar crumbUrl in the snippet below would be https://foobar/crumbIssuer/api/json

my $crumbUrl = $self->getBaseUrl('crumbIssuer/api/json');

Implement crumb retrieval as follows.

sub addCrumbToRequest {
    my ($self, $request) = @_;

    # Creating base URL using previously implemented function
    my $crumbUrl = $self->getBaseUrl('crumbIssuer/api/json');
    # Creating REST client object
    my $rest = $self->getContext()->newRESTClient();
    # Creating crumb request object
    my $crumbRequest = $rest->newRequest(GET => $crumbUrl);
    # actual request.
    my $response = $rest->doRequest($crumbRequest);

    if ($response->code() > 399) {
        return $request;
    }

    my $decodedResponse;
    try {
        $decodedResponse = decode_json($response->decoded_content());
    };

    if ($decodedResponse->{crumb} && $decodedResponse->{crumbRequestField}) {
        $request->header($decodedResponse->{crumbRequestField} => $decodedResponse->{crumb});
    }
    return $request;
}

Note that in the above snippet the Rest Interaction with Jenkins is handled by FlowPDF.

Retrieve Jenkins Job Last Build Number

Introduce a function that retrieves Jenkins Job build number by first getting the CSRF token and then passing it to Jenkins by appending it to the jenkins endpoint set up in the plugin configuration.

sub retrieveLastBuildNumberFromJob {
    my ($self, $jobName) = @_;

    if (!$jobName) {
        bailOut "Missing jobName for retrieveLastBuildNumberFromJob";
    }
    # Retrieving base URL for jenkins job.
    my $baseUrl = $self->getBaseUrl(
        sprintf('job/%s/api/json', $jobName)
    );

    # getting rest client
    my $rest = $self->getContext()->newRESTClient();
    # creating request object
    my $request = $rest->newRequest(GET => $baseUrl);
    # augmenting this request with crumbs
    $request = $self->addCrumbToRequest($request);
    # performing request:
    my $response = $rest->doRequest($request);
    # decoding request content:
    my $json = decode_json($response->decoded_content());
    # finding and returning build number.
    if ($json->{lastBuild} && $json->{lastBuild}->{number}) {
        return $json->{lastBuild}->{number};
    }
    return undef;
}

Modify step code

The auto-generated function in step looks as follows :

sub getLastBuildNumber {
    my ($pluginObject, $runtimeParameters, $stepResult) = @_;

    my $context = $pluginObject->newContext();
    logInfo("Current context is: ", $context->getRunContext());
    my $params = $context->getStepParameters();
    logInfo("Step parameters are: ", $params);

    my $configValues = $context->getConfigValues();
    logInfo("Config values are: ", $configValues);

    $stepResult->setJobStepOutcome('warning');
    $stepResult->setJobSummary("This is a job summary.");
}

Modify the logic as follows using the function you built:

sub getLastBuildNumber {
    my ($pluginObject, $runtimeParameters, $stepResult) = @_;

    my $buildNumber = $pluginObject->retrieveLastBuildNumberFromJob($runtimeParameters->{jenkinsJobName});
    unless ($buildNumber) {
        bailOut("No buildNumber for jenkins job.");
    }
}

Set Output Parameters

An output parameter is set as part of the $stepResult object.

$stepResult->setOutputParameter(lastBuildNumber => $buildNumber);

Add the above code to the step function as follows:

sub getLastBuildNumber {
    my ($pluginObject, $runtimeParameters, $stepResult) = @_;

    my $buildNumber = $pluginObject->retrieveLastBuildNumberFromJob($runtimeParameters->{jenkinsJobName});
    unless ($buildNumber) {
        bailOut("No buildNumber for jenkins job.");
    }

    $stepResult->setOutputParameter(lastBuildNumber => $buildNumber);
}

Set pipeline, job and jobstep summary

Set all the 3 summaries - pipeline summary, job summary and jobstep summary.

sub getLastBuildNumber {
    my ($pluginObject, $runtimeParameters, $stepResult) = @_;

    my $buildNumber = $pluginObject->retrieveLastBuildNumberFromJob($runtimeParameters->{jenkinsJobName});
    unless ($buildNumber) {
        bailOut("No buildNumber for jenkins job.");
    }

    logInfo("Last Build Number for $runtimeParameters->{jenkinsJobName} is $buildNumber");
    $stepResult->setOutputParameter(lastBuildNumber => $buildNumber);
    $stepResult->setPipelineSummary("Build Number for $runtimeParameters->{jenkinsJobName}", $buildNumber);
    $stepResult->setJobSummary("Build Number for $runtimeParameters->{jenkinsJobName}: $buildNumber");
    $stepResult->setJobStepSummary("Build Number for $runtimeParameters->{jenkinsJobName}: $buildNumber");

}

Go through MyJenkins.pm

This is a summary of how MyJenkins.pm after you make the changes.

package FlowPlugin::MyJenkins;
use strict;
use warnings;
use base qw/FlowPDF/;

use FlowPDF::Log;
use FlowPDF::Helpers qw/bailOut/;
use Data::Dumper;
use JSON;
use Try::Tiny;

# Feel free to use new libraries here, e.g. use File::Temp;

# Service function that is being used to set some metadata for a plugin.
sub pluginInfo {
    return {
        pluginName          => '@PLUGIN_KEY@',
        pluginVersion       => '@PLUGIN_VERSION@',
        configFields        => ['config'],
        configLocations     => ['ec_plugin_cfgs'],
        defaultConfigValues => {
        authScheme => 'basic'
        },
    };
    }


# Auto-generated method for the procedure Get Last Build Number/Get Last Build Number
# Add your code into this method and it will be called when step runs
sub getLastBuildNumber {
    my ($pluginObject, $runtimeParameters, $stepResult) = @_;

    my $buildNumber = $pluginObject->retrieveLastBuildNumberFromJob($runtimeParameters->{jenkinsJobName});
    unless ($buildNumber) {
    bailOut("No buildNumber for jenkins job.");
}

    $stepResult->setOutputParameter(lastBuildNumber => $buildNumber);
    $stepResult->setPipelineSummary("Build Number for $runtimeParameters->{jenkinsJobName}", $buildNumber);
    $stepResult->setJobSummary("Build Number for $runtimeParameters->{jenkinsJobName}: $buildNumber");
    $stepResult->setJobStepSummary("Build Number for $runtimeParameters->{jenkinsJobName}: $buildNumber");

}
## === step ends ===
# Please do not remove the marker above, it is used to place new procedures into this file.

sub retrieveLastBuildNumberFromJob {
    my ($self, $jobName) = @_;

    if (!$jobName) {
    bailOut "Missing jobName for retrieveLastBuildNumberFromJob";
    }
    # Retrieving base URL for jenkins job.
    my $baseUrl = $self->getBaseUrl(
    sprintf('job/%s/api/json', $jobName)
    );

    # getting rest client
    my $rest = $self->getContext()->newRESTClient();
    # creating request object
    my $request = $rest->newRequest(GET => $baseUrl);
    # augmenting this request with crumbs
    $request = $self->addCrumbToRequest($request);
    # performing request:
    my $response = $rest->doRequest($request);
    # decoding request content:
    my $json = decode_json($response->decoded_content());
    # finding and returning build number.
    if ($json->{lastBuild} && $json->{lastBuild}->{number}) {
        return $json->{lastBuild}->{number};
    }
    return undef;
}


sub getBaseUrl {
    my ($self, $url) = @_;

    if (!$url) {
        bailOut("URL is mandatory parameter");
    }
    # retrieving runtime parameters
    my $runtimeParameters = $self->getContext()->getRuntimeParameters();
    # endpoint is a field from configuration
    my $endpoint = $runtimeParameters->{endpoint};

    # removing trailing slash.
    $endpoint =~ s|\/+$||gs;

    # appending url
    my $retval = $endpoint . '/' . $url;
    return $retval;
    }


sub addCrumbToRequest {
    my ($self, $request) = @_;

    # Creating base URL using previously implemented function
    my $crumbUrl = $self->getBaseUrl('crumbIssuer/api/json');
    # Creating REST client object
    my $rest = $self->getContext()->newRESTClient();
    # Creating crumb request object
    my $crumbRequest = $rest->newRequest(GET => $crumbUrl);
    # actual request.
    my $response = $rest->doRequest($crumbRequest);

    if ($response->code() > 399) {
        return $request;
    }

    my $decodedResponse;
    try {
        $decodedResponse = decode_json($response->decoded_content());
    };

    if ($decodedResponse->{crumb} && $decodedResponse->{crumbRequestField}) {
        $request->header($decodedResponse->{crumbRequestField} => $decodedResponse->{crumb});
    }
    return $request;
}

    1;

Step 4: Build, Install and Test

Build the plugin from the root directory of the MyJenkins plugin workspace.

flowpdk build

It should look something like this:

_images/pluginBuild.png

Congratulations! You have now created a functional plugin.

Install and Promote the plugin

Go to your CloudBees Flow instance, login and navigate to the Plugin Manager page which lists all plugins.

_images/pluginsListLink.png

Click on “Install from File/URL” tab, click Choose File, select your file and click upload:

_images/installPlugin.png

You will be redirected to the plugin manager page. Find your plugin MyJenkins, and click “Promote” link.

After promotion the page would be automatically refreshed.

Create a Pipeline and Configure the plugin

After your plugin is installed, navigate to the Pipelines page and click “New” in the top right corner of the web page.

_images/createNewPipeline.png

Click “Create New”, enter “MyJenkinsTestPipeline”, choose your project, click “Ok”.

Now you’re in your pipeline editor. Click “Add +” on the task explorer, enter “Get Last Build Number”, click “Select Task Type”, scroll down the plugin list and click on your MyPlugin procedure below the plugin name:

_images/selectProcedure.png

Now click on your procedure name and “Define” link. You will see:

_images/defineProcedureLink.png

Click on input parameters to confirm that this procedure see 2 parameters as per the YAML spec.

Enter Jenkins Job Name as HelloWorld, click on triangle and then click “New Configuration”

_images/inputParameters.png

Enter values relevant for your configuration now. In this example, Jenkins instance is located at 10.200.1.171:8080. Username and password are admin/changeme:

_images/configValues.png

Save configuration and select it on the Input Parameters form.

This concludes the plugin as well as the pipeline setup.

Running the pipeline

In the flow pipeline page click “Run” on the pipeline you have created. In the Popup menu, click “New Run” and in the modal dialog that shows up, click “Run” again. You will see the following:

_images/pipelineResult1.png

Click on the summary link:

_images/pipelineResult2.png

Click on the “1. Get Last Build Number Link”:

_images/pipelineResult3.png

Click on the “Parameters” tab:

_images/pipelineResult4.png

Click back on the “Steps” link and open the log link:

_images/pipelineResult5.png

We now have a minimal working plugin for Jenkins.

Summary

The following diagram is provided as a reference to enunciate the flow of control using flowpdf-perl.

_images/flowpdf-perl-call-stack.png