Reading License Information with Readers

Different Readers are available to import raw component / license information for different technologies. This chapter describes how to setup the different build / dependency management systems to create the required input and how to configure the corresponding reader.

Maven

For the export of the licenses from a maven based project the license-maven-plugin is used, which can directly be called without the need to change anything in the pom.xml.

To generate the input file required for Solicitor the License Plugin needs to be executed with the following command:

mvn org.codehaus.mojo:license-maven-plugin:1.14:aggregate-download-licenses -Dlicense.excludedScopes=test,provided

The generated output file named licenses.xml (in the directory specified in the plugin config) should look like the following:

<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<licenseSummary>
    <dependency>
      <groupId>org.hsqldb</groupId>
      <artifactId>hsqldb</artifactId>
      <version>2.4.1</version>
      <licenses>
        <license>
          <name>HSQLDB License, a BSD open source license</name>
          <url>http://hsqldb.org/web/hsqlLicense.html</url>
          <distribution>repo</distribution>
        </license>
      </licenses>
    </dependency>
    <dependency>
	...
    <dependency>
      <groupId>org.apache.maven</groupId>
      <artifactId>maven-compat</artifactId>
      <version>3.3.9</version>
      <licenses>
        <license>
          <name>Apache License, Version 2.0</name>
          <url>http://www.apache.org/licenses/LICENSE-2.0.txt</url>
          <distribution>repo</distribution>
        </license>
      </licenses>
    </dependency>
  </dependencies>
</licenseSummary>

In Solicitor the data is read with the following reader config:

"readers" : [ {
  "type" : "maven",
  "source" : "file:target/generated-resouces/licenses.xml",
  "usagePattern" : "DYNAMIC_LINKING"
} ]

(the above assumes that Solicitor is executed in the maven projects main directory)

CSV

The CSV input is normally manually generated and should look like this:

org.eclipse;albireo;0.0.3;Eclipse Public License - v 1.0;https://www.eclipse.org/legal/epl-v10.html
org.antlr.runtime;antlr;4.6.0;BSD;https://www.antlr.org/license.html
org.eclipse.equinox.p2;Equinox p2 Provisioning for IDEs;2.3.2;Eclipse Public License - v 1.0;http://www.eclipse.org/legal/epl-v10.html
org.springframework;spring;5.0.6;Apache 2;https://www.apache.org/licenses/LICENSE-2.0
org.eclipse.xtend;xtend;2.2.0;MIT;https://spdx.org/licenses/MIT#licenseText

In Solicitor the data is read with the following part of the config

"readers" : [ {
  "type" : "csv",
  "source" : "file:path/to/the/file.csv",
  "usagePattern" : "DYNAMIC_LINKING"
} ]

The following 5 columns need to be contained in order (separated with ";"):

  • groupId

  • artifactId

  • version

  • license name

  • license URL

Additionally, an optional configuration can be set in order to customize the given structure of the csv file e.g.:

"readers" : [ {
  "type" : "csv",
  "source" : "file:path/to/the/file.csv",
  "usagePattern" : "DYNAMIC_LINKING",
  "configuration" : {
	"charset" = "UTF-8",
	"artifactId" : "0",
	"version" : "1",
	"format" : "EXCEL",
	"skipHeaderRecord" : "true",
	"delimiter" : ";"
  }
} ]

The minimum of following 2 configuration settings need to be contained:

  • artifactId

  • version

With these settings one can specify the position of the value within the csv file. Additional positional settings include:

  • groupId

  • license

  • licenseUrl

If a charset needs to be specified, one can use the following option:

  • charset (string, specified charset for reader e.g. UTF-8)

Furthermore, one can configure a range of other csv structure options based on the Apache Commons CSV API:

  • allowDuplicateHeaderNames (boolean)

  • allowMissingColumnNames (boolean)

  • autoFlush (boolean)

  • commentMarker (char)

  • delimiter (string)

  • escape (char)

  • ignoreEmptyLines (boolean)

  • ignoreHeaderCase (boolean)

  • ignoreSurroundingSpaces (boolean)

  • nullString (string)

  • quote (char)

  • recordSeparator (string)

  • skipHeaderRecord (boolean)

  • trailingDelimiter (boolean)

  • trim (boolean)

These configurations may also be used to overwrite options of a predefined format, which can be set with:

  • format (string, predefined format e.g. EXCEL)

Important: In case that a component has multiple licenses attached, there needs to be a separate line in the csv file for each license.

The CSV reader currently does not fill the attribute packageUrl. Any functionality/reporting based on this attribute will be disfunctional for data read bythe CSV reader.

NPM

For NPM based projects either the NPM License Crawler (https://www.npmjs.com/package/npm-license-crawler) or the NPM License Checker (https://www.npmjs.com/package/license-checker) might be used. The NPM License Crawler can process several node packages in one run.

NPM License Crawler

To install the NPM License Crawler the following command needs to be executed.

npm i npm-license-crawler -g

To get the licenses, the crawler needs to be executed like the following example

npm-license-crawler --dependencies --csv licenses.csv

The export should look like the following (The csv file is "," separated)

"module name","licenses","repository","licenseUrl","parents"
"@angular-devkit/architect@0.10.6","MIT","https://github.com/angular/angular-cli","https://github.com/angular/angular-cli/raw/master/LICENSE","mythaistar-restaurant"
"@types/selenium-webdriver@2.53.43","MIT","https://github.com/DefinitelyTyped/DefinitelyTyped","https://github.com/DefinitelyTyped/DefinitelyTyped/raw/master/LICENSE","protractor:mythaistar-restaurant"
"aws-sign2@0.6.0","Apache-2.0","https://github.com/mikeal/aws-sign","https://github.com/mikeal/aws-sign/raw/master/LICENSE","loggly:mythaistar-restaurant"
"is-posix-bracket@0.1.1","MIT","https://github.com/jonschlinkert/is-posix-bracket","https://github.com/jonschlinkert/is-posix-bracket/raw/master/LICENSE","mythaistar-restaurant"
"npm-package-arg@5.1.2","ISC","https://github.com/npm/npm-package-arg","https://github.com/npm/npm-package-arg/raw/master/LICENSE","npm:npx:mythaistar-restaurant"
"readable-stream@2.0.6","MIT","https://github.com/nodejs/readable-stream","https://github.com/nodejs/readable-stream/raw/master/LICENSE","bl:mythaistar-restaurant"

In Solicitor the data is read with the following part of the config

"readers" : [ {
  "type" : "npm-license-crawler-csv",
  "source" : "file:path/to/licenses.csv",
  "usagePattern" : "DYNAMIC_LINKING",
  "repoType" : "npm"
} ]
NPM License Checker

To install the NPM License Checker the following command needs to be executed.

npm i license-checker -g

To get the licenses, the checker needs to be executed like the following example (we require JSON output here)

license-checker --json > /path/to/licenses.json

The export should look like the following

{
  "foo@0.0.1": {
    "licenses": "MIT",
    "repository": "https://github.com/somebody/foo",
    "url": "http://www.somebody.com/",
    "path": "C:\\Users\\me\\foo",
    "licenseFile": "C:\\Users\\me\\foo\\LICENSE"
  },
  ...
  "foo-bar@1.0.0": {
    "licenses": [
      "AFLv2.1",
      "BSD"
    ],
    "repository": "https://github.com/nobody/foo-bar",
    "path": "C:\\Users\\me\\foo-bar"
  }
}

In Solicitor the data is read with the following part of the config

"readers" : [ {
  "type" : "npm-license-checker",
  "source" : "file:path/to/licenses.json",
  "usagePattern" : "DYNAMIC_LINKING",
  "repoType" : "npm"
} ]

Yarn

To generate the input file required for Solicitor yarn needs to be executed with the following command within the directory that contains the project’s package.json (we require JSON output here):

yarn licenses list --json > /path/to/yarnlicenses.json

The export should look like the following

{"type":"progressStart","data":{"id":0,"total":2}}
{"type":"progressTick","data":{"id":0,"current":1}}
{"type":"progressTick","data":{"id":0,"current":2}}
{"type":"progressFinish","data":{"id":0}}
{"type":"table","data":{"head":["Name","Version","License","URL","VendorUrl","VendorName"],"body":[["test","11.0.0","Apache-2.0","https://github.com/mrtest/test","http://test.com","Mr.Test"],["@test/testing","7.16.3","MIT","https://yarnpkg.com/package/@test/testing","TestCase"]]}}

In Solicitor the data is read with the following part of the config

"readers" : [ {
  "type" : "yarn",
  "source" : "file:path/to/yarnlicenses.json",
  "usagePattern" : "DYNAMIC_LINKING",
  "repoType" : "yarn"
} ]

Pip

To generate the input file required for Solicitor one has follow two steps:

  • Capsulate software with all relevant dependencies/requirements in a virtual environment (venv)

  • Install the pip-licenses plugin within this virtual environment

After that, we execute following command within the virtual environment to extract the input file (we require JSON output here):

pip-licenses --from=all --format=json --with-urls --with-license-file > piplicenses.json

The export should look like the following

[
  {
    "License-Classifier": "MIT License",
    "License-Metadata": "MIT",
    "LicenseFile": "c:\\users\\path\\LICENSE.txt",
    "LicenseText": "Permission...",
    "Name": "test",
    "URL": "http://test.org",
    "Version": "2021.1"
  },
  {
    "License-Classifier": "BSD License",
    "License-Metadata": "BSD-3-Clause",
    "LicenseFile": "c:\\users\\path\\LICENSE",
    "LicenseText": "Redistribution...",
    "Name": "test2",
    "URL": "https://github.com/test/test2",
    "Version": "0.4.2"
  }
]

In Solicitor the data is read with the following part of the config

"readers" : [ {
  "type" : "pip",
  "source" : "file:path/to/piplicenses.json",
  "usagePattern" : "DYNAMIC_LINKING",
  "repoType" : "pip"
} ]

Gradle (Windows)

For the export of the licenses from a Gradle based project the Gradle License Plugin is used.

To install the plugin some changes need to be done in build.gradle, like following example

buildscript {
  repositories {
    maven { url 'https://oss.jfrog.org/artifactory/oss-snapshot-local/' }
  }

  dependencies {
    classpath 'com.jaredsburrows:gradle-license-plugin:0.8.5-SNAPSHOT'
  }
}

apply plugin: 'java-library'
apply plugin: 'com.jaredsburrows.license'

Afterwards execute the following command in the console:

For Windows (Java Application)

gradlew licenseReport

The Export should look like this:

[
    {
        "project": "Apache Commons Codec",
        "description": "The Apache Commons Codec package contains simple encoder and decoders for\n     various formats such as Base64 and Hexadecimal.  In addition to these\n     widely used encoders and decoders, the codec package also maintains a\n     collection of phonetic encoding utilities.",
        "version": "1.11",
        "developers": [
			...
        ],
        "url": "http://commons.apache.org/proper/commons-codec/",
        "year": "2002",
        "licenses": [
            {
                "license": "Apache License, Version 2.0",
                "license_url": "https://www.apache.org/licenses/LICENSE-2.0.txt"
            }
        ],
        "dependency": "commons-codec:commons-codec:1.11"
    },
    ...
    {
        "project": "XmlBeans",
        "description": "XmlBeans main jar",
        "version": "3.0.2",
        "developers": [
			...
        ],
        "url": "https://xmlbeans.apache.org/",
        "year": null,
        "licenses": [
            {
                "license": "The Apache Software License, Version 2.0",
                "license_url": "http://www.apache.org/licenses/LICENSE-2.0.txt"
            }
        ],
        "dependency": "org.apache.xmlbeans:xmlbeans:3.0.2"
    }
]

In Solicitor the data is read with the following part of the config

"readers" : [ {
  "type" : "gradle2",
  "source" : "file:path/to/licenses.json",
  "usagePattern" : "DYNAMIC_LINKING"
} ]
The former reader of type gradle is deprecated and should no longer be used. See List of Deprecated Features.

Gradle (Android)

For the Export of the the Licenses from a Gradle based Android Projects the Gradle License Plugin is used.

To install the Plugin some changes need to be done in the build.gradle of the Project, like following example

buildscript {
  repositories {
    jcenter()
  }

  dependencies {
    classpath 'com.jaredsburrows:gradle-license-plugin:0.8.5'
  }
}

Also there is a change in the build.gradle of the App. Add the line in the second line

apply plugin: 'com.android.application'

Afterwards execute the following command in the Terminal of Android studio: For Windows(Android Application)

gradlew licenseDebugReport

The Export is in the following folder

$Projectfolder\app\build\reports\licenses

It should look like this:

[
    {
        "project": "Java Client for Google Maps Services",
        "description": "Use the Google Maps API Web Services in Java! https://developers.google.com/maps/documentation/webservices/",
        "version": "0.9.4",
        "developers": [
            "Mark McDonald",
            ...
        ],
        "url": "https://github.com/googlemaps/google-maps-services-java",
        "year": null,
        "licenses": [
            {
                "license": "The Apache Software License, Version 2.0",
                "license_url": "http://www.apache.org/licenses/LICENSE-2.0.txt"
            }
        ],
        "dependency": "com.google.maps:google-maps-services:0.9.4"
    },
    {
        "project": "Retrofit",
        "description": null,
        "version": "2.6.1",
        "developers": [],
        "url": null,
        "year": null,
        "licenses": [
            {
                "license": "Apache 2.0",
                "license_url": "https://www.apache.org/licenses/LICENSE-2.0.txt"
            }
        ],
        "dependency": "com.squareup.retrofit2:retrofit:2.6.1"
    }
]

In Solicitor the Data is read with the following part of the config

"readers" : [ {
      "type" : "gradle2",
      "source" : "file:$/input/licenses.json",
      "usagePattern" : "DYNAMIC_LINKING"
   	} ]
The former reader of type gradle is deprecated and should no longer be used. See List of Deprecated Features.
Last updated 2022-07-01 13:40:31 UTC