Skip to content

Commit bae49b3

Browse files
henrikplatehenrikplate
authored
Merge pull request #422 from eclipse/kaybee-importer-doc
KB-Importer Docs
2 parents fe8c6af + 58e9414 commit bae49b3

File tree

3 files changed

+122
-1
lines changed

3 files changed

+122
-1
lines changed

‎docs/mkdocs.yml

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -65,6 +65,7 @@ nav:
6565
- Tutorial: vuln_db/tutorials/vuln_db_tutorial.md
6666
- Manual:
6767
- 'Intro': vuln_db/manuals/index.md
68+
- 'KB Importer': vuln_db/manuals/kb_importer.md
6869
- 'Patch Analyzer' : vuln_db/manuals/patch_analyzer.md
6970
- 'Patch Lib Analyzer' : vuln_db/manuals/patch_lib_analyzer.md
7071
- 'Manual Assessment' : vuln_db/manuals/manual_assessment.md

‎docs/public/content/vuln_db/manuals/index.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -10,6 +10,6 @@ An up-to-date list of all vulnerabilities currently comprised in the database ca
1010

1111
Detailed information for a given vulnerability can be obtained using the following link, whereby `<foo>` has to be replaced by the vulnerability identifier: `@@ADDRESS@@/backend/bugs/<foo>`
1212

13-
The [Patch Analyzer](../../vuln_db/manuals/patch_analyzer/) module can be used to add vulnerabilities to the knowledge base.
13+
The [Patch Analyzer](../../vuln_db/manuals/patch_analyzer/) and [KB Importer](../../vuln_db/manuals/kb_importer/) module can be used to add vulnerabilities to the knowledge base.
1414

1515
In the current implementation, the vulnerability detection is based on the containment of the fully-qualified name of a construct changed to fix a vulnerability in a library. The information whether the library contains the vulnerable or fixed version of the construct is computed asynchronously by the [Patch Lib Analyzer](../../vuln_db/manuals/patch_lib_analyzer/).

‎docs/public/content/vuln_db/manuals/kb_importer.md

Lines changed: 120 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,120 @@
1+
# KB(Knowledge Base) Importer
2+
3+
KB-Importer is a java application which is used to import vulnerabilities in Steady Backend from [Project KB](https://github.com/SAP/project-kb).
4+
5+
!!! info "Project KB"
6+
[Project KB](https://github.com/SAP/project-kb) supports the creation, management and aggregation of a distributed, collaborative knowledge base of vulnerabilities that affect open-source software.
7+
8+
This guide talks about gathering the data in Steady from [Project KB](https://github.com/SAP/project-kb).
9+
10+
## Setup
11+
1. Use the following link to setup and configure Project KB - [installing-the-kaybee-tool](https://sap.github.io/project-kb/getting_started/#installation).
12+
2. Update the variables in `kaybeeconf.yaml` configuration file. The variables are
13+
* `USER_TOKEN` - Token specified in your Steady .env file.
14+
* `BACKEND_URL` - URL of the rest-backend service (Mandatory).
15+
* `CIA_URL` - URL of the deployed rest-lib-util service (This is used if you are uploading the affected artifacts).
16+
* `KB_IMPORTER_PATH` - Jar path of your kb-importer module.
17+
18+
## Usage
19+
1. Run `kaybee merge` to get the vulnerablity information from Project-KB.
20+
2. Run `kaybee export -t steady` which outputs a shell script `steady.sh` which is is then run to update all vulnerability data in Steady.
21+
3. To get specific vulnerability information run `kaybee export -t steady -f .kaybee/merged/CVE-2019-0191/statement.yaml` which outputs a shell script `steady.sh` which is is then run to update `CVE-2019-0191` vulnerability data in Steady.
22+
23+
## Technical Information
24+
This information is to know how KB-Importer works in detail. The directory structure and the files are automatically generated by the shell script `steady.sh` as shown above.
25+
26+
To add a vulnerability manually using KB-Importer the folloing folder structure and command has to be used:
27+
28+
Command Line arguments for KB-Importer:
29+
30+
* `directory (-d) <arg>` : A directory containing commits information for a vulnerability. More details regarding the directory structure is shown below.
31+
* `overwrite (-o)` : Overwrite the analysis of a vulnerability if it already exists in the backend.
32+
* `upload (-u)` : Upload construct changes. Without the -u option, the resulting JSON will be saved to file for you to review (e.g., to check that the list of construct changes is not empty).
33+
* `verbose (-v)` : Verbose mode to get more detailed logs.
34+
35+
Other useful/required configurations:
36+
37+
* `vulas.shared.backend.serviceUrl` : URL of the deployed rest-backend service (Mandatory).
38+
* `vulas.shared.cia.serviceUrl` : URL of the deployed rest-lib-util service (This is used if you are uploading the affected artifacts as shown in the below example).
39+
40+
Directory structure should be as follows:
41+
```
42+
vulnerability_dir #vulnerability directory specified with option -d. Note: The directory name need not be the same as the vulnerability id
43+
├── commit1_dir #commit directory. Note: The directory name need not be the same as the commit id
44+
│   ├── before #directory containing the files/folders before the vulnerability fix
45+
│   ├── after #directory containing the files/folders after the vulnerability fix
46+
│   └── metadata.json #commit metadata file for each commit. More info can be found below
47+
├── commit2_dir #mutiple commit directories if there are mutiple commits for a vulnerability fix
48+
│   ├── before
49+
│   ├── after
50+
│   └── metadata.json
51+
└── metadata.json #vulnerability metadata file
52+
```
53+
54+
Vulnerability `metadata.json` example:
55+
```json
56+
{
57+
"vulnerability_id": "CVE-2016-1000338",
58+
"aliases": ["VULN-1000338","VULN-0002"],
59+
"affected_artifacts": [
60+
{
61+
"id": "pkg:maven/org.bouncycastle/bcprov-jdk15on@1.47",
62+
"reason": "Manual review",
63+
"affected": true
64+
},
65+
{
66+
"id": "pkg:maven/org.bouncycastle/bcprov-jdk15@1.43",
67+
"reason": "Manual review",
68+
"affected": false
69+
}
70+
],
71+
"notes": [
72+
{
73+
"links": [
74+
"https://www.bouncycastle.org/releasenotes.html",
75+
"https://ubuntu.com/security/notices/USN-3727-1"
76+
],
77+
"text": "In Bouncy Castle JCE Provider version 1.55 and earlier the DSA does not fully validate ASN.1 encoding of signature on verification. It is possible to inject extra elements in the sequence making up the signature and still have it validate, which in some cases may allow the introduction of 'invisible' data into a signed structure."
78+
},
79+
{
80+
"links": [
81+
"https://lists.debian.org/debian-lts-announce/2018/07/msg00009.html"
82+
],
83+
"text": "It was discovered that Bouncy Castle incorrectly handled certain crypto algorithms. A remote attacker could possibly use these issues to obtain sensitive information, including private keys."
84+
}
85+
]
86+
}
87+
```
88+
89+
Information about the fields of vulnerability `metadata.json` file:
90+
91+
* `vulnerability_id`: Vulnerability ID.
92+
* `aliases`: Aliases used for the vulnerability id.
93+
* `affected_artifacts`: Affected Libraries for the vulnerability.
94+
* `notes`: This is an array of `links` and `text`. Here, the links array consists of links which provide comprehensive vulnerability information and the text is the vulnerability description. (optional, it must be provided for vulnerabilities not available from the NVD).
95+
96+
97+
Commit metadata.json example file:
98+
```json
99+
{
100+
"repository" : "https://github.com/bcgit/bc-java",
101+
"branch" : "DEFAULT_BRANCH",
102+
"timestamp" : "1476490865",
103+
"commit_id" : "843c2e60f67d71faf81d236f448ebbe56c62c647"
104+
}
105+
```
106+
107+
Information about the fields of commit metadata.json file:
108+
109+
* `repository`: URL of the VCS repository hosting the library project.
110+
* `branch`: Branch name.
111+
* `timestamp`: Timestamp when the commit was made.
112+
* `commit_id`: VCS commit id.
113+
114+
Example run for kb-importer:
115+
```sh
116+
java -Dvulas.shared.backend.header.X-Vulas-Client-Token=<token> -Dvulas.shared.cia.serviceUrl=<cia-url> -Dvulas.shared.backend.serviceUrl=<backend-url> -jar kb-importer-3.1.14-SNAPSHOT-jar-with-dependencies.jar -d ./CVEdata/CVE-2016-1000338 -v -u -o
117+
```
118+
119+
## Limitation
120+
[Project KB](https://github.com/SAP/project-kb) only supports Java specific vulnerabilities. Python vulnerabilities will be supported once [For python, __init__ files are to be checked out](https://github.com/SAP/project-kb/issues/36) is fixed.

0 commit comments

Comments
 (0)