Reading License Information with Readers

Different Readers are available to import raw component / license information for different technologies. This chapter describes how to setup the different build / dependency management systems to create the required input and how to configure the corresponding reader.

Maven

For the export of the licenses from a maven based project the license-maven-plugin is used, which can directly be called without the need to change anything in the pom.xml.

To generate the input file required for Solicitor the License Plugin needs to be executed with the following command:

mvn org.codehaus.mojo:license-maven-plugin:1.14:aggregate-download-licenses -Dlicense.excludedScopes=test,provided

The generated output file named licenses.xml (in the directory specified in the plugin config) should look like the following:

<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<licenseSummary>
    <dependency>
      <groupId>org.hsqldb</groupId>
      <artifactId>hsqldb</artifactId>
      <version>2.4.1</version>
      <licenses>
        <license>
          <name>HSQLDB License, a BSD open source license</name>
          <url>http://hsqldb.org/web/hsqlLicense.html</url>
          <distribution>repo</distribution>
        </license>
      </licenses>
    </dependency>
    <dependency>
	...
    <dependency>
      <groupId>org.apache.maven</groupId>
      <artifactId>maven-compat</artifactId>
      <version>3.3.9</version>
      <licenses>
        <license>
          <name>Apache License, Version 2.0</name>
          <url>http://www.apache.org/licenses/LICENSE-2.0.txt</url>
          <distribution>repo</distribution>
        </license>
      </licenses>
    </dependency>
  </dependencies>
</licenseSummary>

In Solicitor the data is read with the following reader config:

"readers" : [ {
  "type" : "maven",
  "source" : "file:target/generated-resouces/licenses.xml",
  "usagePattern" : "DYNAMIC_LINKING"
} ]

(the above assumes that Solicitor is executed in the maven projects main directory)

CSV

The CSV input is normally manually generated and should look like this (The csv File is ";" separated):

org.eclipse;albireo;0.0.3;Eclipse Public License - v 1.0;https://www.eclipse.org/legal/epl-v10.html
org.antlr.runtime;antlr;4.6.0;BSD;https://www.antlr.org/license.html
org.eclipse.equinox.p2;Equinox p2 Provisioning for IDEs;2.3.2;Eclipse Public License - v 1.0;http://www.eclipse.org/legal/epl-v10.html
org.springframework;spring;5.0.6;Apache 2;https://www.apache.org/licenses/LICENSE-2.0
org.eclipse.xtend;xtend;2.2.0;MIT;https://spdx.org/licenses/MIT#licenseText

In Solicitor the data is read with the following part of the config

"readers" : [ {
  "type" : "csv",
  "source" : "file:path/to/the/file.csv",
  "usagePattern" : "DYNAMIC_LINKING"
} ]

The following 5 columns need to be contained:

  • groupId

  • artifactId

  • version

  • license name

  • license URL

In case that a component has multiple licenses attached, there needs to be a separate line in the file for each license.

NPM

For NPM based projects either the NPM License Crawler (https://www.npmjs.com/package/npm-license-crawler) or the NPM License Checker (https://www.npmjs.com/package/license-checker) might be used. The NPM License Crawler can process several node packages in one run.

NPM License Crawler

To install the NPM License Crawler the following command needs to be executed.

npm i npm-license-crawler -g

To get the licenses, the crawler needs to be executed like the following example

npm-license-crawler --dependencies --csv licenses.csv

The export should look like the following (The csv file is "," separated)

"module name","licenses","repository","licenseUrl","parents"
"@angular-devkit/architect@0.10.6","MIT","https://github.com/angular/angular-cli","https://github.com/angular/angular-cli/raw/master/LICENSE","mythaistar-restaurant"
"@types/selenium-webdriver@2.53.43","MIT","https://github.com/DefinitelyTyped/DefinitelyTyped","https://github.com/DefinitelyTyped/DefinitelyTyped/raw/master/LICENSE","protractor:mythaistar-restaurant"
"aws-sign2@0.6.0","Apache-2.0","https://github.com/mikeal/aws-sign","https://github.com/mikeal/aws-sign/raw/master/LICENSE","loggly:mythaistar-restaurant"
"is-posix-bracket@0.1.1","MIT","https://github.com/jonschlinkert/is-posix-bracket","https://github.com/jonschlinkert/is-posix-bracket/raw/master/LICENSE","mythaistar-restaurant"
"npm-package-arg@5.1.2","ISC","https://github.com/npm/npm-package-arg","https://github.com/npm/npm-package-arg/raw/master/LICENSE","npm:npx:mythaistar-restaurant"
"readable-stream@2.0.6","MIT","https://github.com/nodejs/readable-stream","https://github.com/nodejs/readable-stream/raw/master/LICENSE","bl:mythaistar-restaurant"

In Solicitor the data is read with the following part of the config

"readers" : [ {
  "type" : "npm-license-crawler-csv",
  "source" : "file:path/to/licenses.csv",
  "usagePattern" : "DYNAMIC_LINKING",
  "repoType" : "npm"
} ]
NPM License Checker

To install the NPM License Checker the following command needs to be executed.

npm i license-checker -g

To get the licenses, the checker needs to be executed like the following example (we require JSON output here)

license-checker --json > /path/to/licenses.json

The export should look like the following

{
  "foo@0.0.1": {
    "licenses": "MIT",
    "repository": "https://github.com/somebody/foo",
    "url": "http://www.somebody.com/",
    "path": "C:\\Users\\me\\foo",
    "licenseFile": "C:\\Users\\me\\foo\\LICENSE"
  },
  ...
  "foo-bar@1.0.0": {
    "licenses": [
      "AFLv2.1",
      "BSD"
    ],
    "repository": "https://github.com/nobody/foo-bar",
    "path": "C:\\Users\\me\\foo-bar"
  }
}

In Solicitor the data is read with the following part of the config

"readers" : [ {
  "type" : "npm-license-checker",
  "source" : "file:path/to/licenses.json",
  "usagePattern" : "DYNAMIC_LINKING",
  "repoType" : "npm"
} ]

Yarn

To generate the input file required for Solicitor yarn needs to be executed with the following command within the directory that contains the project’s package.json (we require JSON output here):

yarn licenses list --json > /path/to/yarnlicenses.json

The export should look like the following

{"type":"progressStart","data":{"id":0,"total":2}}
{"type":"progressTick","data":{"id":0,"current":1}}
{"type":"progressTick","data":{"id":0,"current":2}}
{"type":"progressFinish","data":{"id":0}}
{"type":"table","data":{"head":["Name","Version","License","URL","VendorUrl","VendorName"],"body":[["test","11.0.0","Apache-2.0","https://github.com/mrtest/test","http://test.com","Mr.Test"],["@test/testing","7.16.3","MIT","https://yarnpkg.com/package/@test/testing","TestCase"]]}}

In Solicitor the data is read with the following part of the config

"readers" : [ {
  "type" : "yarn",
  "source" : "file:path/to/yarnlicenses.json",
  "usagePattern" : "DYNAMIC_LINKING",
  "repoType" : "yarn"
} ]

Pip

To generate the input file required for Solicitor one has follow two steps:

  • Capsulate software with all relevant dependencies/requirements in a virtual environment (venv)

  • Install the pip-licenses plugin within this virtual environment

After that, we execute following command within the virtual environment to extract the input file (we require JSON output here):

pip-licenses --from=all --format=json --with-urls --with-license-file > piplicenses.json

The export should look like the following

[
  {
    "License-Classifier": "MIT License",
    "License-Metadata": "MIT",
    "LicenseFile": "c:\\users\\path\\LICENSE.txt",
    "LicenseText": "Permission...",
    "Name": "test",
    "URL": "http://test.org",
    "Version": "2021.1"
  },
  {
    "License-Classifier": "BSD License",
    "License-Metadata": "BSD-3-Clause",
    "LicenseFile": "c:\\users\\path\\LICENSE",
    "LicenseText": "Redistribution...",
    "Name": "test2",
    "URL": "https://github.com/test/test2",
    "Version": "0.4.2"
  }
]

In Solicitor the data is read with the following part of the config

"readers" : [ {
  "type" : "pip",
  "source" : "file:path/to/piplicenses.json",
  "usagePattern" : "DYNAMIC_LINKING",
  "repoType" : "pip"
} ]

Gradle (Windows)

For the export of the licenses from a Gradle based project the Gradle License Plugin is used.

To install the plugin some changes need to be done in build.gradle, like following example

buildscript {
  repositories {
    maven { url 'https://oss.jfrog.org/artifactory/oss-snapshot-local/' }
  }

  dependencies {
    classpath 'com.jaredsburrows:gradle-license-plugin:0.8.5-SNAPSHOT'
  }
}

apply plugin: 'java-library'
apply plugin: 'com.jaredsburrows.license'

Afterwards execute the following command in the console:

For Windows (Java Application)

gradlew licenseReport

The Export should look like this:

[
    {
        "project": "Apache Commons Codec",
        "description": "The Apache Commons Codec package contains simple encoder and decoders for\n     various formats such as Base64 and Hexadecimal.  In addition to these\n     widely used encoders and decoders, the codec package also maintains a\n     collection of phonetic encoding utilities.",
        "version": "1.11",
        "developers": [
			...
        ],
        "url": "http://commons.apache.org/proper/commons-codec/",
        "year": "2002",
        "licenses": [
            {
                "license": "Apache License, Version 2.0",
                "license_url": "https://www.apache.org/licenses/LICENSE-2.0.txt"
            }
        ],
        "dependency": "commons-codec:commons-codec:1.11"
    },
    ...
    {
        "project": "XmlBeans",
        "description": "XmlBeans main jar",
        "version": "3.0.2",
        "developers": [
			...
        ],
        "url": "https://xmlbeans.apache.org/",
        "year": null,
        "licenses": [
            {
                "license": "The Apache Software License, Version 2.0",
                "license_url": "http://www.apache.org/licenses/LICENSE-2.0.txt"
            }
        ],
        "dependency": "org.apache.xmlbeans:xmlbeans:3.0.2"
    }
]

In Solicitor the data is read with the following part of the config

"readers" : [ {
  "type" : "gradle2",
  "source" : "file:path/to/licenses.json",
  "usagePattern" : "DYNAMIC_LINKING"
} ]
The former reader of type gradle is deprecated and should no longer be used. See List of Deprecated Features.

Gradle (Android)

For the Export of the the Licenses from a Gradle based Android Projects the Gradle License Plugin is used.

To install the Plugin some changes need to be done in the build.gradle of the Project, like following example

buildscript {
  repositories {
    jcenter()
  }

  dependencies {
    classpath 'com.jaredsburrows:gradle-license-plugin:0.8.5'
  }
}

Also there is a change in the build.gradle of the App. Add the line in the second line

apply plugin: 'com.android.application'

Afterwards execute the following command in the Terminal of Android studio: For Windows(Android Application)

gradlew licenseDebugReport

The Export is in the following folder

$Projectfolder\app\build\reports\licenses

It should look like this:

[
    {
        "project": "Java Client for Google Maps Services",
        "description": "Use the Google Maps API Web Services in Java! https://developers.google.com/maps/documentation/webservices/",
        "version": "0.9.4",
        "developers": [
            "Mark McDonald",
            ...
        ],
        "url": "https://github.com/googlemaps/google-maps-services-java",
        "year": null,
        "licenses": [
            {
                "license": "The Apache Software License, Version 2.0",
                "license_url": "http://www.apache.org/licenses/LICENSE-2.0.txt"
            }
        ],
        "dependency": "com.google.maps:google-maps-services:0.9.4"
    },
    {
        "project": "Retrofit",
        "description": null,
        "version": "2.6.1",
        "developers": [],
        "url": null,
        "year": null,
        "licenses": [
            {
                "license": "Apache 2.0",
                "license_url": "https://www.apache.org/licenses/LICENSE-2.0.txt"
            }
        ],
        "dependency": "com.squareup.retrofit2:retrofit:2.6.1"
    }
]

In Solicitor the Data is read with the following part of the config

"readers" : [ {
      "type" : "gradle2",
      "source" : "file:$/input/licenses.json",
      "usagePattern" : "DYNAMIC_LINKING"
   	} ]
The former reader of type gradle is deprecated and should no longer be used. See List of Deprecated Features.
Last updated 2022-01-24 21:30:19 UTC