Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Conan package identifier #94

Merged
merged 19 commits into from
Nov 3, 2023
Merged
Show file tree
Hide file tree
Changes from 17 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions CA.nuspec
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@
<package >
<metadata>
<id>continuous-clearing</id>
<version>4.0.0</version>
karthika-g marked this conversation as resolved.
Show resolved Hide resolved
<version>6.0.0</version>
<authors>Siemens AG</authors>
<owners>continuous-clearing contributors</owners>
<projectUrl>https://github.com/siemens/continuous-clearing</projectUrl>
Expand All @@ -13,7 +13,7 @@
<requireLicenseAcceptance>false</requireLicenseAcceptance>
<description>
The License clearing tool helps the Project Manager/Developer, to reduce the manual effort and enable the faster license clearing process,
by automatically identifying the third party oss components used in their project(i.e., npm, nuget, maven and Debian type) and it creates them in the sw360 and fossology
by automatically identifying the third party oss components used in their project(i.e., npm, nuget, maven, python, conan and Debian type) and it creates them in the sw360 and fossology
for clearing license
</description>
<releaseNotes></releaseNotes>
Expand Down
32 changes: 26 additions & 6 deletions doc/UsageDoc/CA_UsageDocument.md
Original file line number Diff line number Diff line change
Expand Up @@ -46,20 +46,20 @@
<!--te-->
# Introduction

The Continuous Clearing Tool helps the Project Manager/Developer to automate the sw360 clearing process of 3rd party components. This tool scans and identifies the third-party components used in a NPM, NUGET, MAVEN and Debian projects and makes an entry in SW360, if it is not present. Continuous Clearing Tool links the components to the respective project and creates job for code scan in FOSSology.The output is an SBOM file which has a nested description of software artifact components and metadata.
The Continuous Clearing Tool helps the Project Manager/Developer to automate the sw360 clearing process of 3rd party components. This tool scans and identifies the third-party components used in a NPM, NUGET, MAVEN, PYTHON, CONAN and Debian projects and makes an entry in SW360, if it is not present. Continuous Clearing Tool links the components to the respective project and creates job for code scan in FOSSology.The output is an SBOM file which has a nested description of software artifact components and metadata.

Continuous Clearing Tool reduces the effort in creating components in SW360 and identifying the matching source codes from the public repository. Tool eliminates the manual error while creating component and identifying correct version of source code from public repository. Continuous Clearing Tool harmonize the creation of 3P components in SW360 by filling necessary information.

# Continuous Clearing Tool workflow diagram

- Package Identifier
- [NPM/NUGET/MAVEN](../usagedocimg/packageIdentifiernpmnuget.PNG)
- [NPM/NUGET/MAVEN/PYTHON/CONAN](../usagedocimg/packageIdentifiernpmnuget.PNG)
- [Debian](../usagedocimg/packageIdentifierdebian.PNG)
- SW360 Package Creator
- [NPM/NUGET/MAVEN](../usagedocimg/packageCreatirnpmnuget.PNG)
- [NPM/NUGET/MAVEN/PYTHON/CONAN](../usagedocimg/packageCreatirnpmnuget.PNG)
- [Debian](../usagedocimg/packagecreatordebian.PNG)
- Artifactory Uploader
- [NPM/NUGET/MAVEN](../usagedocimg/artifactoryuploader.PNG)
- [NPM/NUGET/MAVEN/PYTHON/CONAN](../usagedocimg/artifactoryuploader.PNG)

# Prerequisite

Expand Down Expand Up @@ -158,13 +158,19 @@ Continuous Clearing Tool reduces the effort in creating components in SW360 and

mvn clean install -DskipTests=true

- **Project Type :** **Python**
- **Project Type :** **Python**

* Input file repository should contain **poetry.lock** file.

`Note : Python package support in clearing tool is currently only for SBOM discovery and classification.Component Creation and Source code identification is not supported currently`

- **Project Type :** **Conan**

* Input file repository should contain **conan.lock** file.

`Note : Conan package support in clearing tool is currently only for SBOM discovery and classification.Component Creation and Source code identification is not supported currently`

- **Project Type :** **Debian**
- **Project Type :** **Debian**

**Note** : below steps is required only if you have `tar` file to process , otherwise you can keep `CycloneDx.json` file in the InputDirectory.
* Create `InputImage` directory for keeping `tar` images and `InputDirectory` for resulted file storing .
Expand Down Expand Up @@ -258,8 +264,22 @@ Continuous Clearing Tool reduces the effort in creating components in SW360 and
"Python": {
"Include": [ "poetry.lock", "*.cdx.json" ],
"Exclude": [],
"JfrogPythonRepoList": [
"<Python Remote Cache Repo Name>",
"<Python Release Repo Name>",//This should be the release repo in JFrog
],
"ExcludedComponents": []
},
"Conan": {
"Include": [ "conan.lock"],
"Exclude": [],
"JfrogConanRepoList": [
"<Conan Remote Cache Repo Name>",
"<Conan Release Repo Name>",
],
"ExcludedComponents": []
}

}
```

Expand Down
1 change: 1 addition & 0 deletions src/LCT.APICommunications/ApiConstant.cs
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,7 @@ public static class ApiConstant
public const string ComponentNameUrl = "?name=";
public const string NPMExternalID = "pkg:npm/";
public const string NugetExternalID = "pkg:nuget/";
public const string ConanExternalID = "pkg:conan/";
public const string NpmExtension = ".tgz";
public const string NugetExtension = ".nupkg";
public const string MavenExtension = "-sources.jar";
Expand Down
1 change: 1 addition & 0 deletions src/LCT.Common/CommonAppSettings.cs
Original file line number Diff line number Diff line change
Expand Up @@ -65,6 +65,7 @@ public CommonAppSettings(IFolderAction iFolderAction)
public Config Maven { get; set; }
public Config Debian { get; set; }
public Config Python { get; set; }
public Config Conan { get; set; }
public string CaVersion { get; set; }
public string CycloneDxSBomTemplatePath { get; set; }
public string[] InternalRepoList { get; set; }
Expand Down
14 changes: 14 additions & 0 deletions src/LCT.Common/CommonHelper.cs
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@
// --------------------------------------------------------------------------------------------------------------------

using CycloneDX.Models;
using LCT.Common.Constants;
using LCT.Common.Model;
using log4net;
using log4net.Core;
Expand Down Expand Up @@ -228,5 +229,18 @@ public static bool ComponentPropertyCheck(Component component, string constant)
}
return component.Properties.Exists(x => x.Name == constant);
}

public static void GetDetailsforManuallyAdded(List<Component> componentsForBOM, List<Component> listComponentForBOM)
{
foreach (var component in componentsForBOM)
{
component.Properties = new List<Property>();
Property isDev = new() { Name = Dataconstant.Cdx_IsDevelopment, Value = "false" };
Property identifierType = new() { Name = Dataconstant.Cdx_IdentifierType, Value = Dataconstant.ManullayAdded };
component.Properties.Add(isDev);
component.Properties.Add(identifierType);
listComponentForBOM.Add(component);
}
}
}
}
1 change: 1 addition & 0 deletions src/LCT.Common/Constants/Dataconstant.cs
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,7 @@ public static class Dataconstant
{"DEBIAN", "pkg:deb/debian"},
{"MAVEN", "pkg:maven"},
{"PYTHON", "pkg:pypi"},
{"CONAN", "pkg:conan"},
};

//Identified types
Expand Down
4 changes: 2 additions & 2 deletions src/LCT.Common/CycloneDXBomParser.cs
Original file line number Diff line number Diff line change
Expand Up @@ -72,8 +72,8 @@ public static Bom ExtractSBOMDetailsFromTemplate(Bom template)
}

//Taking SBOM Template Metadata
bom.Metadata = template?.Metadata;
bom.Dependencies = template?.Dependencies;
bom.Metadata = template.Metadata;
bom.Dependencies = template.Dependencies;
return bom;
}

Expand Down
1 change: 1 addition & 0 deletions src/LCT.Common/Model/Config.cs
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,7 @@ public class Config
public string[] JfrogNugetRepoList { get; set; }
public string[] JfrogMavenRepoList { get; set; }
public string[] JfrogPythonRepoList { get; set; }
public string[] JfrogConanRepoList { get; set; }
public string[] DevDependentScopeList { get; set; }
karthika-g marked this conversation as resolved.
Show resolved Hide resolved

}
Expand Down
9 changes: 9 additions & 0 deletions src/LCT.Common/appSettings.json
Original file line number Diff line number Diff line change
Expand Up @@ -74,5 +74,14 @@
"<Python Release Repo Name>" //This should be the release pypi in JFrog
],
"ExcludedComponents": []
},
"Conan": {
"Include": [ "conan.lock" ],
"Exclude": [],
"JfrogConanRepoList": [
"<Conan Remote Cache Repo Name>", //This is a mirror repo for conan in JFrog
"<Conan Release Repo Name>" //This should be the release repo in JFrog
],
"ExcludedComponents": []
}
}
Loading
Loading