macaron.build_spec_generator.common_spec package

Submodules

macaron.build_spec_generator.common_spec.base_spec module

This module includes base build specification and helper classes.

class macaron.build_spec_generator.common_spec.base_spec.BaseBuildSpecDict

Bases: TypedDict

Initialize base build specification.

It supports multiple languages, build tools, and additional metadata for enhanced traceability.

ecosystem: Required[str]: The package ecosystem.

purl: Required[str]: The package identifier.

language: Required[str]: The programming language, e.g., ‘java’, ‘python’, ‘javascript’.

build_tools: Required[list[str]]: The build tools or package managers, e.g., ‘maven’, ‘gradle’, ‘pip’, ‘poetry’, ‘npm’, ‘yarn’.

macaron_version: Required[str]: The version of Macaron used for generating the spec.

group_id: NotRequired[str | None]: The group identifier for the project/component.

artifact_id: Required[str]: The artifact identifier for the project/component.

version: Required[str]: The version of the package or component.

git_repo: NotRequired[str]: The remote path or URL of the git repository.

git_tag: NotRequired[str]: The commit SHA or tag in the VCS repository.

newline: NotRequired[str]: The type of line endings used (e.g., ‘lf’, ‘crlf’).

language_version: Required[list[str]]: The version of the programming language or runtime, e.g., ‘11’ for JDK, ‘3.11’ for Python.

dependencies: NotRequired[list[str]]: List of release dependencies.

build_dependencies: NotRequired[list[str]]: List of build dependencies, which includes tests.

build_commands: NotRequired[list[list[str]]]: List of shell commands to build the project.

test_commands: NotRequired[list[str]]: List of shell commands to test the project.

environment: NotRequired[dict[str, str]]: Environment variables required during build or test.

artifact_path: NotRequired[str | None]: Path or location of the build artifact/output.

entry_point: NotRequired[str | None]: Entry point script, class, or binary for running the project.

build_requires: NotRequired[dict[str, str]]: The build_requires is the required packages that need to be available in the build environment.

build_backends: NotRequired[list[str]]: A “back end” is tool that a “front end” (such as pip/build) would call to package the source distribution into the wheel format. build_backends would be a list of these that were used in building the wheel alongside their version.

class macaron.build_spec_generator.common_spec.base_spec.BaseBuildSpec

Bases: ABC

Abstract base class for build specification behavior and field resolution.

abstractmethod resolve_fields(purl)

Resolve fields that require special logic for a specific build ecosystem.

Return type:: None

Notes

This method should be implemented by subclasses to handle logic specific to a given package ecosystem, such as Maven or PyPI.

abstractmethod get_default_build_commands(build_tool_names)

Return the default build commands for the build tools.

Parameters:: build_tool_names (list[str]) – The build tools to get the default build command.
Returns:: The build command as a list[list[str]].
Return type:: list[list[str]]
Raises:: GenerateBuildSpecError – If there is no default build command available for the specified build tool.

macaron.build_spec_generator.common_spec.core module

This module contains the logic to generate a build spec in a generic format that can be transformed if needed.

class macaron.build_spec_generator.common_spec.core.ECOSYSTEMS(value)

Bases: Enum

This Enum provides implementation mappings for supported ecosystems.

MAVEN(data) = <class 'macaron.build_spec_generator.common_spec.maven_spec.MavenBuildSpec'>: The Maven build specification.

PYPI(data) = <class 'macaron.build_spec_generator.common_spec.pypi_spec.PyPIBuildSpec'>: The PyPI build specification.

class macaron.build_spec_generator.common_spec.core.LANGUAGES(value)

Bases: Enum

This Enum provides mappings for supported languages.

MAVEN = 'java': The language used in the Maven build ecosystem.

PYPI = 'python': The language used in the PyPI build ecosystem.

class macaron.build_spec_generator.common_spec.core.MacaronBuildToolName(value)

Bases: str, Enum

Represent the name of a build tool that Macaron stores in the database.

This doesn’t cover all build tools that Macaron supports, and ONLY includes the ones that we support generating build spec for.

MAVEN = 'maven'

GRADLE = 'gradle'

PIP = 'pip'

POETRY = 'poetry'

FLIT = 'flit'

HATCH = 'hatch'

CONDA = 'conda'

macaron.build_spec_generator.common_spec.core.format_build_command_info(build_command_info)

Return the prettified str format for a list of GenericBuildCommandInfo instances.

Parameters:: build_command_info (GenericBuildCommandInfo) – A list of GenericBuildCommandInfo instances.
Returns:: The prettified output.
Return type:: str

macaron.build_spec_generator.common_spec.core.remove_shell_quote(cmd)

Remove shell quotes from a shell command.

Parameters:: cmd (list[str]) – The shell command as list[str]ing.
Returns:: The shell command with all quote removed.
Return type:: list[str]

Examples

>>> cmd = "mvn -f fit/core-reference/pom.xml verify '-Dit.test=RESTITCase' '-Dmodernizer.skip=true' '-Drat.skip=true'"
>>> remove_shell_quote(cmd.split())
['mvn', '-f', 'fit/core-reference/pom.xml', 'verify', '-Dit.test=RESTITCase', '-Dmodernizer.skip=true', '-Drat.skip=true']

macaron.build_spec_generator.common_spec.core.compose_shell_commands(cmds_sequence)

Combine a sequence of command fragments into a single shell command suitable for a build spec.

Parameters:: cmds_sequence (list[list[str]]) – The sequence of build command fragments.
Returns:: A shell command string to be used in the build specification’s command field.
Return type:: str

macaron.build_spec_generator.common_spec.core.get_macaron_build_tool_names(build_tool_facts, target_language)

Retrieve the Macaron build tool names for supported projects from the database facts.

Iterates over the provided build tool facts and returns the list of valid MacaronBuildToolName for a supported language.

Parameters:

build_tool_facts (Sequence[BuildToolFacts]) – A sequence of build tool fact records to be searched.
target_language (str) – The target build language.

Returns:

The corresponding Macaron build tool names, or None otherwise.

Return type:

list[MacaronBuildToolName] None

macaron.build_spec_generator.common_spec.core.get_build_tool_names(component_id, session, target_language)

Retrieve the Macaron build tool names for a given component.

Queries the database for build tool facts associated with the specified component ID. It returns the corresponding list of MacaronBuildToolName if found.

Parameters:

component_id (int) – The ID of the component for which to retrieve the build tool name.
session (sqlalchemy.orm.Session) – The SQLAlchemy session used to access the database.
target_language (str) – The target build language.

Returns:

The corresponding build tool name for the component if available, otherwise None.

Return type:

list[MacaronBuildToolName] | None

macaron.build_spec_generator.common_spec.core.get_build_command_info(component_id, session)

Return the highest confidence build command information from the database for a component.

The build command is found by looking up CheckFacts for build-related checks.

Parameters:

component_id (int) – The id of the component we are finding the build command for.
session (sqlalchemy.orm.Session) – The SQLAlchemy Session opened for the database to extract build information.

Returns:

The GenericBuildCommandInfo object for the highest confidence build command; or None if there was an error, or no build command is found from the database.

Return type:

GenericBuildCommandInfo | None

macaron.build_spec_generator.common_spec.core.get_language_version(build_command_info)

Retrieve the language version from a GenericBuildCommandInfo object.

If available, returns a language version from the language_versions list associated with the provided GenericBuildCommandInfo object. Currently, this function returns the last element in the list. If the list is empty, returns None.

Notes

The selection of the last element from language_versions is a temporary strategy, as more robust selection logic may be implemented in the future depending on requirements for specific language/runtime versions (e.g., multiple JDK versions).

Parameters:: build_command_info (GenericBuildCommandInfo) – The object containing language version information.
Returns:: The selected language version as a string, or None if not available.
Return type:: str | None

macaron.build_spec_generator.common_spec.core.gen_generic_build_spec(purl, session)

Generate and return the Buildspec file.

Parameters:

purl (PackageURL) – The PackageURL to generate build spec for.
session (sqlalchemy.orm.Session) – The SQLAlchemy Session opened for the database to extract build information.

Returns:

The generated build spec.

Return type:

BaseBuildSpecDict

Raises:

GenerateBuildSpecError – Raised if generation of the build spec fails due to any of the following reasons: 1. The input PURL is invalid. 2. There is no supported build tool for this PURL. 3. Failed to patch the build commands using the provided patches. 4. The database from session doesn’t contain enough information.

macaron.build_spec_generator.common_spec.jdk_finder module

This module includes the functions for obtaining the JDK version from a Java artifact.

class macaron.build_spec_generator.common_spec.jdk_finder.JavaArtifactExt(value)

Bases: str, Enum

The extensions for Java artifacts.

JAR = '.jar'

class macaron.build_spec_generator.common_spec.jdk_finder.CacheStrategy(value)

Bases: Enum

The strategy for caching the downloaded artifacts for JDK version finding.

DISABLE = 0

MAVEN_LAYOUT = 1

macaron.build_spec_generator.common_spec.jdk_finder.download_file(url, dest)

Stream a file into a local destination.

Parameters:

url (str) – The URL of the file to stream from.
dest (str) – The path to the destination file in the local file system. This path includes the file name.

Raises:

InvalidHTTPResponseError – If an error happens while streaming the file.
OSError – If the parent directory of dest doesn’t exist.

Return type:

None

macaron.build_spec_generator.common_spec.jdk_finder.join_remote_maven_repo_url(remote_maven_url, maven_repo_path)

Join the base remote maven URL with a maven repository path.

Parameters:

remote_maven_url (str) – The url to a remove maven layout repository. For example: https://repo1.maven.org/maven2
maven_repo_path (str) – The maven repository path for a GAV coordinate or an artifact from the root of the remote maven layout repository.

Returns:

The joined path.

Return type:

str

Examples

>>> remote_maven_repo = "https://repo1.maven.org/maven2"
>>> artifact_path = "io/liftwizard/liftwizard-checkstyle/2.1.22/liftwizard-checkstyle-2.1.22.jar"
>>> join_remote_maven_repo_url(remote_maven_repo, artifact_path)
'https://repo1.maven.org/maven2/io/liftwizard/liftwizard-checkstyle/2.1.22/liftwizard-checkstyle-2.1.22.jar'
>>> join_remote_maven_repo_url(remote_maven_repo, "io/liftwizard/liftwizard-checkstyle/2.1.22/")
'https://repo1.maven.org/maven2/io/liftwizard/liftwizard-checkstyle/2.1.22/'
>>> join_remote_maven_repo_url(f"{remote_maven_repo}/", artifact_path)
'https://repo1.maven.org/maven2/io/liftwizard/liftwizard-checkstyle/2.1.22/liftwizard-checkstyle-2.1.22.jar'

macaron.build_spec_generator.common_spec.jdk_finder.get_jdk_version_from_jar(artifact_path)

Return the JDK version obtained from a Java artifact.

Parameters:: artifact_path (str) – The path to the artifact to extract the jdk version.
Returns:: The version string extract from the artifact (as is) or None if there is an error, or if we couldn’t find any jdk version.
Return type:: str | None

macaron.build_spec_generator.common_spec.jdk_finder.find_jdk_version_from_remote_maven_repo_standalone(group_id, artifact_id, version, asset_name, remote_maven_repo_url)

Return the JDK version string from an artifact matching a given GAV from a remote maven layout repository.

This function doesn’t cache the downloaded artifact, and removes it after the function exits. We assume that the remote maven layout repository supports downloading a file through a HTTPS URL.

Parameters:

group_id (str) – The group ID part of the GAV coordinate.
artifact_id (str) – The artifact ID part of the GAV coordinate.
version (str) – The version part of the GAV coordinate.
asset_name (str) – The name of artifact to download and extract the jdk version.
remote_maven_repo_url (str) – The URL to the remote maven layout repository.

Returns:

The version string extracted from the artifact (as is); or None if there is an error, or if we couldn’t find any jdk version.

Return type:

str | None

macaron.build_spec_generator.common_spec.jdk_finder.find_jdk_version_from_remote_maven_repo_cache(group_id, artifact_id, version, asset_name, remote_maven_repo_url, local_cache_repo)

Return the JDK version string from an artifact matching a given GAV from a remote maven layout repository.

This function caches the downloaded artifact in a maven layout https://maven.apache.org/repository/layout.html under local_cache_repo. We assume that the remote maven layout repository supports downloading a file through a HTTPS URL.

Parameters:

group_id (str) – The group ID part of the GAV coordinate.
artifact_id (str) – The artifact ID part of the GAV coordinate.
version (str) – The version part of the GAV coordinate.
asset_name (str) – The name of artifact to download and extract the jdk version.
remote_maven_repo_url (str) – The URL to the remote maven layout repository.
local_cache_repo (str) – The path to a local directory for caching the downloaded artifact used in JDK version extraction.

Returns:

The version string extracted from the artifact (as is); or None if there is an error, or if we couldn’t find any jdk version.

Return type:

str | None

macaron.build_spec_generator.common_spec.jdk_finder.find_jdk_version_from_central_maven_repo(group_id, artifact_id, version, cache_strat=CacheStrategy.MAVEN_LAYOUT)

Return the JDK version string from an artifact matching a given GAV from Maven Central repository.

The artifacts will be downloaded from https://repo1.maven.org/maven2/ for JDK version extraction.

We now only support JAR files.

Parameters:

group_id (str) – The group ID part of the GAV coordinate.
artifact_id (str) – The artifact ID part of the GAV coordinate.
version (str) – The version part of the GAV coordinate.
cache_strat (CacheStrategy) – Specify how artifacts from maven central are persisted.

Returns:

The version string extract from the artifact (as is); or None if there is an error, or if we couldn’t find any jdk version.

Return type:

str | None

macaron.build_spec_generator.common_spec.jdk_version_normalizer module

This module contains the logic to normalize a JDK version string as a major version number.

macaron.build_spec_generator.common_spec.jdk_version_normalizer.normalize_jdk_version(jdk_version_str)

Return the major JDK version number.

We assume that the JDK version string is already valid (e.g Using a JDK version that is available in the real world).

For 1.x versions, we return the major version as x.

Parameters:: jdk_version_str (str) – The JDK version string.
Returns:: The major JDK version number as a string, or None if there is an error.
Return type:: str | None

Examples

>>> normalize_jdk_version("19")
'19'
>>> normalize_jdk_version("19-ea")
'19'
>>> normalize_jdk_version("11.0.1")
'11'
>>> normalize_jdk_version("1.8")
'8'
>>> normalize_jdk_version("25.0.1")

macaron.build_spec_generator.common_spec.maven_spec module

This module includes build specification and helper classes for Maven packages.

class macaron.build_spec_generator.common_spec.maven_spec.MavenBuildSpec(data)

Bases: BaseBuildSpec

This class implements build spec inferences for Maven packages.

__init__(data)

Initialize the object.

Parameters:: data (BaseBuildSpecDict) – The data object containing the build configuration fields.

get_default_build_commands(build_tool_names)

Return the default build commands for the build tools.

Parameters:: build_tool_names (list[str]) – The build tools to get the default build command.
Returns:: The build command as a list[list[str]].
Return type:: list[list[str]]
Raises:: GenerateBuildSpecError – If there is no default build command available for the specified build tool.

resolve_fields(purl)

Resolve Maven-specific fields in the build specification.

Parameters:: purl (str) – The target software component Package URL.
Return type:: None

macaron.build_spec_generator.common_spec.pypi_spec module

This module includes build specification and helper classes for PyPI packages.

class macaron.build_spec_generator.common_spec.pypi_spec.PyPIBuildSpec(data)

Bases: BaseBuildSpec

This class implements build spec inferences for PyPI packages.

__init__(data)

Initialize the object.

Parameters:: data (BaseBuildSpecDict) – The data object containing the build configuration fields.

get_default_build_commands(build_tool_names)

Return the default build commands for the build tools.

Parameters:: build_tool_names (list[str]) – The build tools to get the default build command.
Returns:: The build command as a list[list[str]].
Return type:: list[list[str]]
Raises:: GenerateBuildSpecError – If there is no default build command available for the specified build tool.

resolve_fields(purl)

Resolve PyPI-specific fields in the build specification.

Parameters:: purl (str) – The target software component Package URL.
Return type:: None

read_directory(wheel_path, purl)

Read in the WHEEL and METADATA file from the .dist_info directory.

Parameters:

wheel_path (str) – Path to the temporary directory where the wheel was downloaded into.
purl (PackageURL) – PURL corresponding to the package being analyzed.

Returns:

Tuple where the first element is a string of the .dist-info/WHEEL contents and the second element is a string of the .dist-info/METADATA contents

Return type:

tuple[str, str]

read_generator_line(wheel_contents)

Parse through the “Generator: {build backend} {version}” line of .dist_info/WHEEL.

Parameters:: wheel_contents (str) – String of the contents of the .dist_info/WHEEL file
Returns:: Tuple where the first element is the generating build backend and the second element is its version.
Return type:: tuple[str, str]