============= Introductory ============= Introduction ------------ This tutorial provides instructions for the 4 step process to create a functional plugin called **MyJenkins** which retrieves the last build number from a jenkins job and stores it in an output parameter. The 4 Step Plugin creation Process ---------------------------------- 1. Create Plugin Workspace. 2. Define plugin spec and generate plugin. 3. Implement Additional logic. 4. Build plugin and test. Prerequisites ------------- These are assumptions for this Tutorial. 1. An active CloudBees Flow instance. 2. A Machine with Docker installed which is registered as a pingable resource in CloudBees Flow. 3. Internet connection. 4. **flowdpk** installed and setup. Install and Setup Jenkins ~~~~~~~~~~~~~~~~~~~~~~~~~ Pull the Docker Jenkins image that already has electricflow agent installed from Docker Hub. :: docker pull electricflow/jenkins **Important:** Preface **sudo** with docker commands if in case you have not setup Docker to run as a non-root user. Start the Jenkins Docker Container as follows. :: docker run -p "8080:8080" -d electricflow/jenkins:latest Do the following to confirm that your container is running: :: ubuntu@ubuntu:~$ sudo docker ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 0d84b3b113a9 electricflow/jenkins:latest "/bin/bash -c /work/…" 2 weeks ago Up 2 weeks 8043/tcp, 0.0.0.0:8080->8080/tcp sleepy_khayyam The above snippet is provided for reference only. Your output would be similar, but not exactly the same as above. Once you see that docker container is running, point your browser to your docker machine address with 8080 port. If you’re using localhost, just visit http://localhost:8080 . You should see a jenkins instance login screen. The credentials for this jenkins instance are admin/changeme. If you can login to the above instance, you are done with your Jenkins environment setup. Step 1 : Create Plugin Workspace --------------------------------- After making sure **flowpdk** is available in your PATH, you can create a plugin workspace. Change to your working directory :: cd ~/work Call **flowpdk** as follows :: flowpdk generate workspace In the interactive prompt type **MyJenkins** as plugin name. For the rest of the options which are all optional, choose defaults if available or provide your own values. .. figure:: images/tutorial-introductory/workspaceGeneration.png :alt: Generated Workspace Generated Workspace The generated workspace would have the following 2 files: .. figure:: images/tutorial-introductory/filesInWorkspace.png :alt: Files in Workspace Files in Workspace Step 2: Define plugin spec and generate plugin ---------------------------------------------- The pluginspec.yaml has 2 sections one for configuration and the other for procedures. Update config section of YAML ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ To interact with Jenkins we need the following config values: 1. Jenkins endpoint 2. Username for authorization. 3. Password for authorization. Fill up the config section as follows. .. code:: yaml configuration: # Shell to be used for checking connection shell: ec-perl parameters: - name: config type: entry label: Configuration Name required: true documentation: The name for the created configuration - name: desc type: entry label: Description required: false documentation: Description for the configuration - name: endpoint label: Endpoint type: entry required: true documentation: A jenkins URL with port. - name: basic_credential type: credential label: Credential userNameLabel: Username required: true passwordLabel: Password - name: debugLevel # This parameter name "debugLevel" is supported by ECPDF Logger out of the box label: Debug Level type: select required: 0 value: 0 options: - name: Info value: 0 - name: Debug value: 1 - name: Trace value: 2 In this tutorial we will assume that the Jenkins environment is protected using the "Prevent Cross Site Request Forgery exploits" security. This means when the plugin makes a POST request, a CSRF protection token should be sent as part of the HTTP request header. Rename the auto-generated specification called credential to basic_credential. Update procedure section ~~~~~~~~~~~~~~~~~~~~~~~~ We will implement a procedure called “Get Last Build Number” with interface specification as follows. .. code:: yaml procedures: - name: Get Last Build Number description: This procedure gets last build number from provided Jenkins job. hasConfig: true # configuration field will be generated automatically parameters: - name: jenkinsJobName documentation: A name of the Jenkins job to get the latest build number. type: entry required: true label: Jenkins Job Name outputParameters: lastBuildNumber: A last build number for job. # Steps are not defined so we assume that this is one-step procedure with a single step named Deploy shell: ec-perl Generate plugin ~~~~~~~~~~~~~~~ Execute the following command from the root level directory of the plugin workspace. :: flowpdk generate plugin After execution, it should look as follows. .. figure:: images/tutorial-introductory/pluginGeneration.png :alt: Plugin Generation Plugin Generation With this step plugin generation is complete. Review the auto-generated code ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ This section is provided to give a perspective on how the boiler plate generated code looks like and what it means. The dsl/properties folder has the following structure: - perl/core - perl/lib **Important: Do not modify any file under core folder. Core folder has plugin-related internal code, which should not be edited by plugin developer.** The only folder that could be modified is the perl/lib folder. Notice that dsl/properties/perl/lib/FlowPlugin/MyJenkins.pm has been auto-generated and contains the following code. .. code:: perl package FlowPlugin::MyJenkins; use strict; use warnings; use base qw/FlowPDF/; use FlowPDF::Log; # Feel free to use new libraries here, e.g. use File::Temp; # Service function that is being used to set some metadata for a plugin. sub pluginInfo { return { pluginName => '@PLUGIN_KEY@', pluginVersion => '@PLUGIN_VERSION@', configFields => ['config'], configLocations => ['ec_plugin_cfgs'], defaultConfigValues => {} }; } # Auto-generated method for the procedure Get Last Build Number/Get Last Build Number # Add your code into this method and it will be called when step runs sub getLastBuildNumber { my ($pluginObject, $runtimeParameters, $stepResult) = @_; my $context = $pluginObject->newContext(); logInfo("Current context is: ", $context->getRunContext()); my $params = $context->getStepParameters(); logInfo("Step parameters are: ", $params); my $configValues = $context->getConfigValues(); logInfo("Config values are: ", $configValues); $stepResult->setJobStepOutcome('warning'); $stepResult->setJobSummary("This is a job summary."); } ## === step ends === # Please do not remove the marker above, it is used to place new procedures into this file. 1; Each plugin procedure has one or more steps. The auto-generated code for the step “Get Last Build Number”. is located in dsl/procedures/GetLastBuildNumber/steps/GetLastBuildNumber.pl and looks like the following : .. code:: perl $[/myProject/perl/core/scripts/preamble.pl] use FlowPlugin::MyJenkins; # Auto generated code of plugin step # Go to dsl/properties/perl/lib/EC/Plugin/MyJenkins.pm and change the function "getLastBuildNumber" FlowPlugin::MyJenkins->runStep('Get Last Build Number', 'Get Last Build Number', 'getLastBuildNumber'); Step 3: Implement Additional logic ---------------------------------- Add default config values: ~~~~~~~~~~~~~~~~~~~~~~~~~~ Edit pluginInfo function to set a default values for parameters. Notice that authScheme is set to the value **basic**. .. code:: perl sub pluginInfo { return { pluginName => '@PLUGIN_KEY@', pluginVersion => '@PLUGIN_VERSION@', configFields => ['config'], configLocations => ['ec_plugin_cfgs'], defaultConfigValues => { authScheme => 'basic' }, }; } Add necessary library imports ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Since Jenkins returns a JSON, add appropriate libraries that can be used to parse, write them to flow logs as necessary or throw exceptions. .. code:: perl package FlowPlugin::MyJenkins; use strict; use warnings; use base qw/FlowPDF/; use FlowPDF::Log; use FlowPDF::Helpers qw/bailOut/; use Data::Dumper; use JSON; use Try::Tiny; Retrieve the Jenkins CSRF Token ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Note that the CSRF Token is also called as the Jenkins crumb. Implement a function that takes the endpoint returned by plugin configuration and appends an additional path to it. .. code:: perl sub getBaseUrl { my ($self, $url) = @_; if (!$url) { bailOut("URL is mandatory parameter"); } # retrieving runtime parameters my $runtimeParameters = $self->getContext()->getRuntimeParameters(); # endpoint is a field from configuration my $endpoint = $runtimeParameters->{endpoint}; # removing trailing slash. $endpoint =~ s|\/+$||gs; # appending url my $retval = $endpoint . '/' . $url; return $retval; } For example if the endpoint is https://foobar crumbUrl in the snippet below would be https://foobar/crumbIssuer/api/json .. code:: perl my $crumbUrl = $self->getBaseUrl('crumbIssuer/api/json'); Implement crumb retrieval as follows. .. code:: perl sub addCrumbToRequest { my ($self, $request) = @_; # Creating base URL using previously implemented function my $crumbUrl = $self->getBaseUrl('crumbIssuer/api/json'); # Creating REST client object my $rest = $self->getContext()->newRESTClient(); # Creating crumb request object my $crumbRequest = $rest->newRequest(GET => $crumbUrl); # actual request. my $response = $rest->doRequest($crumbRequest); if ($response->code() > 399) { return $request; } my $decodedResponse; try { $decodedResponse = decode_json($response->decoded_content()); }; if ($decodedResponse->{crumb} && $decodedResponse->{crumbRequestField}) { $request->header($decodedResponse->{crumbRequestField} => $decodedResponse->{crumb}); } return $request; } Note that in the above snippet the Rest Interaction with Jenkins is handled by FlowPDF. Retrieve Jenkins Job Last Build Number ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Introduce a function that retrieves Jenkins Job build number by first getting the CSRF token and then passing it to Jenkins by appending it to the jenkins endpoint set up in the plugin configuration. .. code:: perl sub retrieveLastBuildNumberFromJob { my ($self, $jobName) = @_; if (!$jobName) { bailOut "Missing jobName for retrieveLastBuildNumberFromJob"; } # Retrieving base URL for jenkins job. my $baseUrl = $self->getBaseUrl( sprintf('job/%s/api/json', $jobName) ); # getting rest client my $rest = $self->getContext()->newRESTClient(); # creating request object my $request = $rest->newRequest(GET => $baseUrl); # augmenting this request with crumbs $request = $self->addCrumbToRequest($request); # performing request: my $response = $rest->doRequest($request); # decoding request content: my $json = decode_json($response->decoded_content()); # finding and returning build number. if ($json->{lastBuild} && $json->{lastBuild}->{number}) { return $json->{lastBuild}->{number}; } return undef; } Modify step code ~~~~~~~~~~~~~~~~ The auto-generated function in step looks as follows : .. code:: perl sub getLastBuildNumber { my ($pluginObject, $runtimeParameters, $stepResult) = @_; my $context = $pluginObject->newContext(); logInfo("Current context is: ", $context->getRunContext()); my $params = $context->getStepParameters(); logInfo("Step parameters are: ", $params); my $configValues = $context->getConfigValues(); logInfo("Config values are: ", $configValues); $stepResult->setJobStepOutcome('warning'); $stepResult->setJobSummary("This is a job summary."); } Modify the logic as follows using the function you built: .. code:: perl sub getLastBuildNumber { my ($pluginObject, $runtimeParameters, $stepResult) = @_; my $buildNumber = $pluginObject->retrieveLastBuildNumberFromJob($runtimeParameters->{jenkinsJobName}); unless ($buildNumber) { bailOut("No buildNumber for jenkins job."); } } Set Output Parameters ~~~~~~~~~~~~~~~~~~~~~ An output parameter is set as part of the $stepResult object. .. code:: perl $stepResult->setOutputParameter(lastBuildNumber => $buildNumber); Add the above code to the step function as follows: .. code:: perl sub getLastBuildNumber { my ($pluginObject, $runtimeParameters, $stepResult) = @_; my $buildNumber = $pluginObject->retrieveLastBuildNumberFromJob($runtimeParameters->{jenkinsJobName}); unless ($buildNumber) { bailOut("No buildNumber for jenkins job."); } $stepResult->setOutputParameter(lastBuildNumber => $buildNumber); } Set pipeline, job and jobstep summary ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Set all the 3 summaries - pipeline summary, job summary and jobstep summary. :: sub getLastBuildNumber { my ($pluginObject, $runtimeParameters, $stepResult) = @_; my $buildNumber = $pluginObject->retrieveLastBuildNumberFromJob($runtimeParameters->{jenkinsJobName}); unless ($buildNumber) { bailOut("No buildNumber for jenkins job."); } logInfo("Last Build Number for $runtimeParameters->{jenkinsJobName} is $buildNumber"); $stepResult->setOutputParameter(lastBuildNumber => $buildNumber); $stepResult->setPipelineSummary("Build Number for $runtimeParameters->{jenkinsJobName}", $buildNumber); $stepResult->setJobSummary("Build Number for $runtimeParameters->{jenkinsJobName}: $buildNumber"); $stepResult->setJobStepSummary("Build Number for $runtimeParameters->{jenkinsJobName}: $buildNumber"); } Review MyJenkins.pm ~~~~~~~~~~~~~~~~~~~~~~~ This is a summary of how MyJenkins.pm after you make the changes. .. code:: perl package FlowPlugin::MyJenkins; use strict; use warnings; use base qw/FlowPDF/; use FlowPDF::Log; use FlowPDF::Helpers qw/bailOut/; use Data::Dumper; use JSON; use Try::Tiny; # Feel free to use new libraries here, e.g. use File::Temp; # Service function that is being used to set some metadata for a plugin. sub pluginInfo { return { pluginName => '@PLUGIN_KEY@', pluginVersion => '@PLUGIN_VERSION@', configFields => ['config'], configLocations => ['ec_plugin_cfgs'], defaultConfigValues => { authScheme => 'basic' }, }; } # Auto-generated method for the procedure Get Last Build Number/Get Last Build Number # Add your code into this method and it will be called when step runs sub getLastBuildNumber { my ($pluginObject, $runtimeParameters, $stepResult) = @_; my $buildNumber = $pluginObject->retrieveLastBuildNumberFromJob($runtimeParameters->{jenkinsJobName}); unless ($buildNumber) { bailOut("No buildNumber for jenkins job."); } $stepResult->setOutputParameter(lastBuildNumber => $buildNumber); $stepResult->setPipelineSummary("Build Number for $runtimeParameters->{jenkinsJobName}", $buildNumber); $stepResult->setJobSummary("Build Number for $runtimeParameters->{jenkinsJobName}: $buildNumber"); $stepResult->setJobStepSummary("Build Number for $runtimeParameters->{jenkinsJobName}: $buildNumber"); } ## === step ends === # Please do not remove the marker above, it is used to place new procedures into this file. sub retrieveLastBuildNumberFromJob { my ($self, $jobName) = @_; if (!$jobName) { bailOut "Missing jobName for retrieveLastBuildNumberFromJob"; } # Retrieving base URL for jenkins job. my $baseUrl = $self->getBaseUrl( sprintf('job/%s/api/json', $jobName) ); # getting rest client my $rest = $self->getContext()->newRESTClient(); # creating request object my $request = $rest->newRequest(GET => $baseUrl); # augmenting this request with crumbs $request = $self->addCrumbToRequest($request); # performing request: my $response = $rest->doRequest($request); # decoding request content: my $json = decode_json($response->decoded_content()); # finding and returning build number. if ($json->{lastBuild} && $json->{lastBuild}->{number}) { return $json->{lastBuild}->{number}; } return undef; } sub getBaseUrl { my ($self, $url) = @_; if (!$url) { bailOut("URL is mandatory parameter"); } # retrieving runtime parameters my $runtimeParameters = $self->getContext()->getRuntimeParameters(); # endpoint is a field from configuration my $endpoint = $runtimeParameters->{endpoint}; # removing trailing slash. $endpoint =~ s|\/+$||gs; # appending url my $retval = $endpoint . '/' . $url; return $retval; } sub addCrumbToRequest { my ($self, $request) = @_; # Creating base URL using previously implemented function my $crumbUrl = $self->getBaseUrl('crumbIssuer/api/json'); # Creating REST client object my $rest = $self->getContext()->newRESTClient(); # Creating crumb request object my $crumbRequest = $rest->newRequest(GET => $crumbUrl); # actual request. my $response = $rest->doRequest($crumbRequest); if ($response->code() > 399) { return $request; } my $decodedResponse; try { $decodedResponse = decode_json($response->decoded_content()); }; if ($decodedResponse->{crumb} && $decodedResponse->{crumbRequestField}) { $request->header($decodedResponse->{crumbRequestField} => $decodedResponse->{crumb}); } return $request; } 1; Step 4: Build, Install and Test ------------------------------- Build the plugin from the root directory of the MyJenkins plugin workspace. :: flowpdk build It should look something like this: .. figure:: images/tutorial-introductory/pluginBuild.png Congratulations! You have now created a functional plugin. Install and Promote the plugin ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Go to your CloudBees Flow instance, login and navigate to the Plugin Manager page which lists all plugins. .. figure:: images/tutorial-introductory/pluginsListLink.png Click on "Install from File/URL" tab, click Choose File, select your file and click upload: .. figure:: images/tutorial-introductory/installPlugin.png You will be redirected to the plugin manager page. Find your plugin MyJenkins, and click "Promote" link. After promotion the page would be automatically refreshed. Create a Pipeline and Configure the plugin ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ After your plugin is installed, navigate to the Pipelines page and click "New" in the top right corner of the web page. .. figure:: images/tutorial-introductory/createNewPipeline.png Click "Create New", enter "MyJenkinsTestPipeline", choose your project, click "Ok". Now you're in your pipeline editor. Click "Add +" on the task explorer, enter "Get Last Build Number", click "Select Task Type", scroll down the plugin list and click on your MyPlugin procedure below the plugin name: .. figure:: images/tutorial-introductory/selectProcedure.png Now click on your procedure name and "Define" link. You will see: .. figure:: images/tutorial-introductory/defineProcedureLink.png Click on input parameters to confirm that this procedure see 2 parameters as per the YAML spec. Enter Jenkins Job Name as HelloWorld, click on triangle and then click "New Configuration" .. figure:: images/tutorial-introductory/inputParameters.png Enter values relevant for your configuration now. In this example, Jenkins instance is located at 10.200.1.171:8080. Username and password are admin/changeme: .. figure:: images/tutorial-introductory/configValues.png Save configuration and select it on the Input Parameters form. This concludes the plugin as well as the pipeline setup. Running the pipeline ~~~~~~~~~~~~~~~~~~~~ In the flow pipeline page click "Run" on the pipeline you have created. In the Popup menu, click "New Run" and in the modal dialog that shows up, click "Run" again. You will see the following: .. figure:: images/tutorial-introductory/pipelineResult1.png Click on the summary link: .. figure:: images/tutorial-introductory/pipelineResult2.png Click on the "1. Get Last Build Number Link": .. figure:: images/tutorial-introductory/pipelineResult3.png Click on the "Parameters" tab: .. figure:: images/tutorial-introductory/pipelineResult4.png Click back on the "Steps" link and open the log link: .. figure:: images/tutorial-introductory/pipelineResult5.png We now have a minimal working plugin for Jenkins. Summary ------- The following diagram is provided as a reference to enunciate the flow of control using **flowpdf-perl**. .. figure:: images/tutorial-introductory/flowpdf-perl-call-stack.png