My Thai Star in Production Line

What is PL?

The Production Line Project is a set of server-side collaboration tools for Capgemini engagements. It has been developed for supporting project engagements with individual tools like issue tracking, continuous integration, continuous deployment, documentation, binary storage and much more!

pl logo
Introduction

Although the PL Project is a wide set of tools, only 3 are going to be mainly used for My Thai Star projects to build a Continuous Integration and Continuos Delivery environment. All three are available in the PL instance used for this project.

  1. Jenkins

    This is going to be the "main tool". Jenkins helps to automate the non-human part of the development with Continuos Integration and is going to host all Pipelines (and, obviously, execute them).

  2. Nexus

    Nexus manages software "artifacts" required for development. It is possible to both download dependencies from Nexus and publish artifacts as well. It allows to share resources within an organization.

  3. SonarQube

    It is a platform for continuous inspection of the code. It is going to be used for the Java back-end.

Where can I find all My Thai Star Pipelines?

They are located under the MTS folder of the PL instance:

mts pipelines

Those Jenkins Pipelines will not have any code to execute. They’re just pointing to all Jenkinsfiles under the /jenkins folder of the repository. They can be found here.

CI in My Thai Star stack

How to configure everything out of the box

Production Line currently has a template to integrate My Thai Star. All information can be found at devonfw production line repository

Angular CI

The Angular client-side of My Thai Star is going to have some specific needs for the CI-CD Pipeline to perform mandatory operations.

Pipeline

The Pipeline for the Angular client-side is going to be called MyThaiStar_FRONTEND_BUILD. It is located in the PL instance, under the MTS folder (as previously explained). It is going to follow a process flow like this one:

angular pipeline flow

Each of those steps are called stages in the Jenkins context.Let’s see what those steps mean in the context of the Angular application:

  1. Declarative: Checkout SCM

    Retrieves the project from the GitHub repository which it’s located. This step is not defined directly in our pipeline, but as it is loaded from the repository this step should always be done at the beginning.

    pipeline config
  2. Declarative: Tool Install

    The Pipeline needs some Tools to perform some operations with the Angular project. These tool is a correct version of NodeJS (10.17.0 LTS) with Yarn installed as global package.

    tools {
        nodejs "NodeJS 10.14.0"
    }
  3. Loading Custom Tools

    The Pipeline also needs a browser in order to execute the tests, so in this step the chrome-stable will be loaded. We will use it in a headless mode.

    tool chrome
  4. Fresh Dependency Installation

    The script $ yarn does a package installation. As we always clean the workspace after the pipeline, all packages must be installed in every execution.

  5. Code Linting

    This script executes a linting process of TypeScript. Rules can be defined in the tslint.json file of the project. It throws an exception whenever a file contains a non-compliant piece of code.

  6. Execute Angular tests

    The CI testing of the Angular client is different than the standard local testing (adapted to CI environments, as specified in the Adaptation section of document). This script just executes the following commands:

    ng test --browsers ChromeHeadless --watch=false
  7. Check dependencies

    Before continue, we print the result of yarn audit. It shows the vulnerabilities in the dependencies. It do not process the reponse. The purpose is only to track the result of the command.

    yarn audit
  8. SonarQube code analysis

    The script load and execute the tool sonar-scanner. This tool is loaded here because it’s not used in any other part of the pipeline. The sonar-scanner will take all code, upload it to sonarQube and wait until sonarQube send us a response with the quality of our code. If the code do not pass the quality gate, the pipeline will stop at this point.

  9. Build Application

    The building process of the Angular client would result in a folder called /dist in the main Angular’s directory. That folder is the one that is going to be served afterwards as an artifact. This process has also been adapted to some Deployment needs. This building script executes the following:

    ng build --configuration=docker
  10. Deliver application into Nexus

    Once the scripts produce the Angular artifact (/dist folder), it’s time to package it and store into nexus.

  11. Declarative: Post Actions

    At the end, this step is always executed, even if a previous stage fail. We use this step to clean up the workspace for future executions

    post {
        always {
            cleanWs()
        }
    }
Adjustments

The Angular project Pipeline needed some "extra" features to complete all planned processes. Those features resulted in some additions to the project.

Pipeline Environment

In order to easily reuse the pipeline in other angular projects, all variables have been defined in the block environment. All variables have the default values that Production Line uses, so if you’re going to work in production line you won’t have to change anything. Example:

environment {
    // Script for build the application. Defined at package.json
    buildScript = 'build --configuration=docker'
    // Script for lint the application. Defined at package.json
    lintScript = 'lint'
    // Script for test the application. Defined at package.json
    testScript = 'test:ci'
    // Angular directory
    angularDir = 'angular'
    // SRC folder. It will be angularDir/srcDir
    srcDir = 'src'
    // Name of the custom tool for chrome stable
    chrome = 'Chrome-stable'

    // sonarQube
    // Name of the sonarQube tool
    sonarTool = 'SonarQube'
    // Name of the sonarQube environment
    sonarEnv = "SonarQube"

    // Nexus
    // Artifact groupId
    groupId = 'com.devonfw.mythaistar'
    // Nexus repository ID
    repositoryId = 'pl-nexus'
    // Nexus internal URL
    repositoryUrl = 'http://nexus3-core:8081/nexus3/repository/maven-snapshots'
    // Maven global settings configuration ID
    globalSettingsId = 'MavenSettings'
    // Maven tool id
    mavenInstallation = 'Maven3'
}
Description
  • buildScript: script for build the application. It must be defined at package.json.

    Example (package.json):

    {
        "name": "mythaistar-restaurant",
        ...
        "scripts": {
            ...
            "build": "ng build",
            ...
        }
        ...
    }

    This will be used as follows:

    sh """yarn ${buildScript}"""
  • lintScript: Script for lint the application. Defined at package.json

    Example (package.json):

    {
        "name": "mythaistar-restaurant",
        ...
        "scripts": {
            ...
            "lint": "ng lint",
            ...
        }
        ...
    }

    This will be used as follows:

    sh """yarn ${lintScript}"""
  • testScript: Script for test the application. Defined at package.json

    Example (package.json):

    {
        "name": "mythaistar-restaurant",
        ...
        "scripts": {
            ...
            "test:ci": "npm run postinstall:web && ng test --browsers ChromeHeadless --watch=false",
            ...
        }
        ...
    }

    This will be used as follows:

    sh """yarn ${testScript}"""
  • angularDir: Relative route to angular application. In My Thai Star this is the angular folder. The actual directory (.) is also allowed.

    angular directory
  • srcDir: Directory where you store the source code. For angular applications the default value is src

    src directory
  • chrome: Since you need a browser to run your tests, we must provide one. This variable contains the name of the custom tool for google chrome.

    chrome installation
  • sonarTool: Name of the sonarQube scanner installation.

    sonar scanner
  • sonarEnv: Name of the sonarQube environment. SonarQube is the default value for PL.

    sonar env
  • groupId: Group id of the application. It will be used to storage the application in nexus3

    nexus3 groupid
  • repositoryId: Id of the nexus3 repository. It must be defined at maven global config file.

    nexus3 id
  • repositoryUrl: The url of the repository.

  • globalSettingsId: The id of the global settings file.

    nexus3 global config
  • mavenInstallation: The name of the maven tool.

    maven tool

Java CI

The Java server-side of My Thai Star is an devon4j-based application. As long as Maven and a Java 8 are going to be needed, the Pipeline should have those tools available as well.

Pipeline

This Pipeline is called MyThaiStar_SERVER_BUILD, and it is located exactly in the same PL instance’s folder than MyThaiStar_FRONTEND_BUILD. Let’s see how the Pipeline’s flow behaves.

java pipeline flow

Check those Pipeline stages with more detail:

  1. Declarative: Checkout SCM

    Gets the code from https://github.com/devonfw/my-thai-star . This step is not defined directly in our pipeline, but as it is loaded from the repository this step should always be done at the beginning.

  2. Declarative: Tool Install

    The My Thai Star application works with JDK11. In this step, if JDK11 is not installed, we install it and then put the JDK folder into PATH.

    tools {
      jdk 'OpenJDK11'
    }
  3. Loading Custom Tools

    In this step we load the tools that can not be loaded in the previous step. As My Thai Star is delivered as docker container, in this step we load docker as custom tool.

    tool dockerTool
  4. Install dependencies

    This step will download all project dependencies.

    mvn clean install -Dmaven.test.skip=true
  5. Unit Tests

    This step will execute the project unit test with maven.

    mvn clean test
  6. Dependeny Checker

    Execute the OWASP Dependency Checker in order to validate the project dependencies. It will generate a report that can be used in SonarQube

    dependencyCheck additionalArguments: '--project "mtsj" --scan java/mtsj --format XML', odcInstallation: 'dependency-check'
    dependencyCheckPublisher pattern: ''
  7. SonarQube analysis

    The code is evaluated using the integrated PL instance’s SonarQube. Also, it will wait for the quality gate status. If the status is failing, the pipeline execution will be stopped.

    withSonarQubeEnv(sonarEnv) {
        sh "mvn sonar:sonar"
    }
    
    def qg = waitForQualityGate()
    if (qg.status != 'OK') {
        error "Pipeline aborted due to quality gate failure: ${qg.status}"
    }
  8. Deliver application into Nexus

    Store all artifacts into nexus.

    mvn deploy -Dmaven.test.skip=true
  9. Create the Docker image

    Create the docker image and then publish the image into a docker registry.

Adjustments
Pipeline Environment

In order to easily reuse the pipeline in other java projects, all variables have been defined in the block environment. All variables have the default values that Production Line uses, so if you’re going to work in production line you won’t have to change anything. Example:

environment {
    // Directory with java project
    javaDir = 'java/mtsj'

    // sonarQube
    // Name of the sonarQube environment
    sonarEnv = "SonarQube"

    // Nexus 3
    // Maven global settings configuration ID
    globalSettingsId = 'MavenSettings'
    // Maven tool id
    mavenInstallation = 'Maven3'

    // Docker
    dockerRegistryCredentials = 'nexus-api'
    dockerRegistryProtocol = 'https://'
    dockerTool = 'docker-global
}
Description
  • javaDir: Relative route to java application. In My Thai Star this is the java/mtsj folder. The actual directory (.) is also allowed.

    java directory
  • sonarEnv: Name of the sonarQube environment. SonarQube is the default value for PL.

  • globalSettingsId: The id of the global settings file. MavenSettings is the default value for PL.

    nexus3 global config
  • mavenInstallation: The name of the maven tool. Maven3 is the default value for PL.

    maven tool
Distribution management

The only extra thing that needs to be added to the Java server-side is some information that determines where the artifact of the project is going to be stored in Nexus. This is going to be a section in the main pom.xml file called <distributionManagement>. This section will point to the PL instance’s Nexus. Let’s have a look at it. It’s already configured with the PL default values.

<distributionManagement>
    <repository>
      <id>pl-nexus</id>
      <name>PL Releases</name>
      <url>http://nexus3-core:8081/nexus/content/repositories/maven-releases/</url>
    </repository>
    <snapshotRepository>
      <id>pl-nexus</id>
      <name>PL Snapshots</name>
      <url>http://nexus3-core:8081/nexus3/repository/maven-snapshots</url>
    </snapshotRepository>
</distributionManagement>
Last updated 2021-10-21 17:17:16 UTC