Skip to content

Plug-in Development

Hoppr's core architecture utilizes plug-ins to perform most tasks. Plug-ins are specified (and optionally configured) in the transfer configuration file.

A number of default plug-ins are included with the Hoppr framework. Additional plugins are available at the Hoppr open source project and also by installing via pip install or poetry add

Basic Starter Plugin

Clone the Hoppr Starter Plugin as a basis for your first Hoppr plugin:

[~]$ git clone https://gitlab.com/lmco/hoppr/plugins/starter-plugin.git

Assuming you have already installed Hoppr, execute a bundle from your starter plugin directory:

[starter-plugin]$ hopctl bundle starter_files/manifest.yml --transfer starter_files/transfer.yml --credentials starter_files/credentials.yml

This plugin simply logs output to show the Hoppr flow of operations across a complete set of config files (manifest/transfer/credentials) and how it processes a mini SBOM. We strongly recommend that any plug-in class inherits from the HopprPlugin class (in the hoppr.base_plugins.hoppr module), which provides the framework for:

  1. The __init__ constructor, which takes two arguments:
    • The Hoppr context, which includes information about the current state of the process
    • (Optional) the config to be used (as a python dict object). Since the configuration is specific to each plug-in, there are no requirements on its structure. The starter plugin demonstrates use of config inside the pre_stage_process method
  2. Reporting version number as a string via the get_version() method. This is implemented as an abstract method in the HopprPlugin base class, and must be overridden.
  3. Methods to handle SBOM processing. Each should return a Result object (defined in hoppr.result). When sub-classing HopprPlugin, each by default returns a Result object with the status of SKIP, meaning that the process did nothing, and logs for that process will be deleted. Some notes on these methods:
    • pre_stage_process: Runs once at the beginning of each stage.
    • process_component: Takes a Component as an argument, and will be called for every component of every SBOM to be processed.
    • post_stage_process: Runs once after all Components have been processed by process_component.

Make sure to read through the comments and NOTE: sections in the Starter Plugin's plugin.py to get a better feel for how a plugin operates.

Collector Plugins

Collectors (of type CollectorPlugin) are specific HopprPlugin classes used to copy artifacts from wherever they are maintained (a URL, or a specialized repository, for example), and place them in a directory structure. That directory structure, in turn, will be used by other plug-ins (in particular, Bundlers) to create bundles that can be transferred to isolated networks.

An SBOM does not specify the source repositories from which components are to be copied. That information is stored in the manifest configuration file(s). Often, there will be more than one repository for a given PURL type. Therefore, a collector plug-in is expected to search through all appropriate repositories until the requested component is found.

We strongly recommend that any plug-in class intended to act as a collector inherit from the CollectorPlugin class (in the hoppr.base_plugins.collector module), which overrides the HopprPlugin class processes_component as a @final method. Subclasses of CollectorPlugin should not override processes_component. In CollectorPlugin, the processes_component method:

  • Loops through each applicable repository, and finds any credentials necessary for access.

  • Calls the abstract method collect for the specified component, repository, and credentials

  • If the collect method succeeds, processing exits without checking any more repositories.

  • If no repository returns a successful collect response, the last failed response is returned to the caller.

Sample Collector Plugin Implementation

import requests
from typing import Any

from hoppr.base_plugins.hoppr import hoppr_rerunner
from hoppr.base_plugins.collector import CollectorPlugin
from hoppr.result import Result
from hoppr.context import Context

class MyCollectorPlugin(CollectorPlugin):

    supported_purl_types = ["generic"] # This plug-in will only run its process_component method for 
                                       # components with the generic purl type

    def get_version(self): # Required
        return "1.0.7"

    @hoppr_rerunner
    def collect(self, comp: Any, repo_url: str, creds: CredObject = None) -> Result:
        authentication = None
        if creds is not None:
          authentication = HTTPBasicAuth(creds.userid, creds.password)

        response = requests.get(repo_url), auth = authentication)
        return Result.from_http_response(response) # Result class convenience method

This collector retrieves an artifact directly from the repo_url. This is not realistic. To be useful, there would need to be at least two changes:

  • it would need to specify a path or query paramter on the url, likely based on the component purl

  • it does not save the downloaded file in the directory structure specified in the context.

The Result returned is based on the request response, using a convenience function that sets the response type based on the HTTP response code (< 300: SUCCESS; 300-499: FAIL; > 500: RETRY).

Hoppr Decorators

The hoppr framework provides two decorators to simplify the development of plug-in processes.

@hoppr_process

The @hoppr_process decorator handles several tasks that any implementation of pre_stage_process, process_component, or post_stage_process. These include:

  • Creation of a logger (accessible via the get_logger() method) to which information may be written regarding the execution of the process. This is an "in-memory" logger, which is copied to stdout at the completion of the processes, thereby preventing interleving of log messages from multiple processes run in parallel.

  • Logging of certain standard events: Start and completion of the process, run time, and result.

  • For process_component (or any method that passes an argument of type Component), check that the process is appropriate to the purl type of the component.

  • At process completion, close the logger, and dump its contents to stdout.

@hoppr_rerunner

The @hoppr_rerunner decorator runs a process repeatedly, up to the value max_attempts specified in the Context, waiting for retry_wait_seconds (also from the Context) seconds between attempts.

Combining the decorators

The two decorators can (and often will be) combined for a single method. We recommend that the order be:

    @hoppr_process
    @hoppr_rerunner
    def process_component(....

The reverse order will also work, but will be slightly less efficient, creating and destroying loggers and logging data for each attempt.


Last update: 2022-09-01