Merge pull request #50 from misje/dev

0.3.0

Author: misje

CHANGELOG.md

@@ -7,6 +7,67 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ## Unreleased
 
+## 0.3.0 - 2024-06-09
+
+### Added
+
+- Search docker URLs when searching for URL SCOs
+- Ignore observables with the empty SHA-256 hash
+  (e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855)
+- Create file SCOs from Office 365 logs in enrichment
+- Create directory SCOs from Office 365 logs in enrichment
+- Use vulnerability score for vulnerability_incident_cvss3_score_threshold
+  check if CVSS3 score is unavailable
+- Search directories in Office 365
+- Add a small sleep() of 100 ms, potentially solving
+  [#11](https://github.com/misje/opencti-wazuh-connector/issues/11)
+- New setting *rule_exclude_list* that allows for ignoring certain alert rules
+  altogether
+- New setting *incident_rule_exclude_list* that prevents incident creation for
+  certain alert rules
+- Mention in *event creation* in docs that incidents from sighted
+  vulnerabilities are not created by default unless configured
+- Document alert rules in glossary, with screenshots of the rule viewer in
+  Wazuh
+- Add a timeout to OpenSearch queries (default 20 s), preventing a complete
+  freeze if OpenSearch fails to reply
+- Add new setting *vulnerability_incident_active_only* that allows for only
+  creating incidents for sighted vulnerabilities if they are no longer active
+
+### Changed
+
+- OpenCTI 6.1.10 is used
+- No longer enrich URLs without host and scheme by default (e.g. "/",
+  "/foo/bar"), but leave the possibility as a new configuration option,
+  *enrich_urls_without_host".
+- If the vulnerability being enriched does not contain any CVSS3 information,
+  extract this from alerts before running the logic in
+  *vulnerability_incident_cvss3_score_threshold*. This allows for creating
+  incidents based on CVSS score threshold even if this information is not
+  present in the source entity.
+
+### Fixed
+
+- Avoid crashing when enriching untriaged vulnerabilities (when *published* is
+  not set)
+- Set confidence explicitly for sightings as a workaround for OpenCTI bug
+  #6835. This ensures that sightings now get the correct confidence (that of
+  the user/group running the connector).
+- Fix bug in vulnerability_incident_cvss3_score_threshold logic
+- Fix a number of typos and bugs in documentation
+- Do not use months in timedeltas in tests, causing issues with 30/31 days in a
+  month
+- Remove "Observable" from incident description, since not all enriched
+  entities are observables
+- Do not match file names partially (regex mistake)
+
+### Removed
+
+- Remove all traces of the Wazuh API. It was only partially implemented, and
+  will be added back when development of this as a separate enhancement is
+  completed.
+- Remove some debug output
+
 ## 0.2.1 - 2024-05-24
 
 ### Changed

Dockerfile

@@ -14,6 +14,5 @@ RUN apk --no-cache add build-base libmagic && \
 RUN pip3 install --no-cache-dir -r requirements.txt
 COPY src /opt/opencti-connector-wazuh
 COPY entrypoint.sh /
-RUN chmod +x /entrypoint.sh && \
-   mkdir -p /var/cache/wazuh
+RUN chmod +x /entrypoint.sh
 ENTRYPOINT ["/entrypoint.sh"]

docs/source/alert_search.rst

@@ -25,9 +25,10 @@ create incidents depending your configuration.
 Configuration
 ~~~~~~~~~~~~~
 
-Use CONNECTOR_SCOPE to select which entities to search for. Use the various
-settings in :attr:`~wazuh.search_config.SearchConfig` to determine how searches
-are performed.
+Use :attr:`CONNECTOR_SCOPE <wazuh.connector_config.ConnectorConfig.scope>` to
+select which entities to search for. Use the various settings in
+:attr:`~wazuh.search_config.SearchConfig` to determine how searches are
+performed.
 
 Observables that have been created by the connector through :ref:`enrichment
 <enrichment>` are not looked up by default (determined by

docs/source/conf.py

@@ -5,7 +5,7 @@
 project = "opencti-wazuh-connector"
 copyright = "2024, Andreas Misje"  # pylint: disable=redefined-builtin
 author = "Andreas Misje"
-release = "0.1.0"
+release = "0.3.0"
 ## The full version, including alpha/beta/rc tags
 # with open("../../../version.txt", "r") as f:
 #    release = f.readline().rstrip()

docs/source/conf_other_considerations.rst

@@ -50,6 +50,8 @@ settings determines when to create incidents:
 - :attr:`~wazuh.config.Config.require_indicator_detection`
 - :attr:`~wazuh.config.Config.ignore_revoked_indicators`
 - :attr:`~wazuh.config.Config.indicator_score_threshold`
+- :attr:`~wazuh.config.Config.vulnerability_incident_cvss3_score_threshold`
+- :attr:`~wazuh.config.Config.incident_rule_exclude_list`
 
 Playbooks
 ~~~~~~~~~

docs/source/config_reference.rst

@@ -31,12 +31,16 @@ CONNECTOR settings
 .. automodule:: wazuh.connector_config
    :members:
 
+.. _opensearch-config:
+
 OpenSearch configuration
 ------------------------
 
 .. automodule:: wazuh.opensearch_config
    :members:
 
+.. _search-config:
+
 Search configuration
 --------------------
 
@@ -52,11 +56,3 @@ Look at :ref:`the enrichment topic <enrichment>` for details.
 
 .. autopydantic_settings:: wazuh.enrich_config.EnrichmentConfig
    :settings-show-json-error-strategy: coerce
-
-Wazuh API configuration
------------------------
-
-Wazuh API is only partially supported.
-
-.. automodule:: wazuh.wazuh_api_config
-   :members:

docs/source/connector-compose.yml

@@ -1,6 +1,6 @@
 services:
   connector-wazuh:
-    image: ghcr.io/misje/opencti-wazuh-connector:0.2.1
+    image: ghcr.io/misje/opencti-wazuh-connector:0.3.0
     restart: always
     environment:
       # A timezone is needed for datetime tools to work as expected:
@@ -10,12 +10,11 @@ services:
       - OPENCTI_TOKEN=84387577-27ac-4751-b268-6893045aa73c
       - CONNECTOR_ID=81f9d582-2b4e-45f1-98b6-f33492d66b6e
       - CONNECTOR_NAME=Wazuh
-      - CONNECTOR_SCOPE=Artifact,Directory,Domain-Name,Email-Addr,Hostname,IPv4-Addr,IPv6-Addr,Mac-Addr,Network-Traffic,Process,Software,StixFile,Url,User-Account,User-Agent,Windows-Registry-Key,Windows-Registry-Value-Type,Vulnerability,Indicator
+      - CONNECTOR_SCOPE=Artifact,Directory,Domain-Name,Email-Addr,Hostname,IPv4-Addr,IPv6-Addr,Mac-Addr,Network-Traffic,Process,StixFile,Url,User-Account,User-Agent,Windows-Registry-Key,Windows-Registry-Value-Type,Vulnerability,Indicator # You may also just use "all" if you want to search available entities
       - CONNECTOR_AUTO=true
       - CONNECTOR_LOG_LEVEL=warning
       - CONNECTOR_EXPOSE_METRICS=true
       - AGENTS_AS_SYSTEMS=true
-      - WAZUH_API_ENABLED=false # This API is still in development
       - WAZUH_APP_URL=https://mywazuh.example.org
       - WAZUH_AUTHOR_NAME=Wazuh
       - WAZUH_BUNDLE_ABORT_LIMIT=500
@@ -25,10 +24,15 @@ services:
       - WAZUH_CREATE_INCIDENT_RESPONSE=true
       - WAZUH_CREATE_INCIDENT_SUMMARY=true
       - WAZUH_CREATE_INCIDENT_SUMMARY=true
+      - WAZUH_VULNERABILITY_INCIDENT_CVSS3_SCORE_THRESHOLD= # CVSS3 score (0–10, or empty)
+      - WAZUH_VULNERABILITY_INCIDENT_ACTIVE_ONLY=true
       - WAZUH_CREATE_INCIDENT_THRESHOLD=1
+      - WAZUH_RULE_EXCLUDE_LIST=
+      - WAZUH_INCIDENT_RULE_EXCLUDE_LIST=
       - WAZUH_CREATE_OBS_SIGHTINGS=true
       - WAZUH_CREATE_SIGHTING_SUMMARY=true
       - WAZUH_ENRICH_FILENAME_BEHAVIOUR=create-dir,remove-path
+      - WAZUH_ENRICH_URLS_WITHOUT_HOST=false
       - WAZUH_ENRICH_TYPES=all
       - WAZUH_ENRICH_AGENT=true
       - WAZUH_ENRICH_LABELS=wazuh_ignore
@@ -44,6 +48,7 @@ services:
       - WAZUH_MAX_TLP=TLP:RED
       - WAZUH_OPENSEARCH_EXCLUDE_MATCH=data.integration:opencti
       - WAZUH_OPENSEARCH_FILTER=
+      - "WAZUH_OPENSEARCH_TIMEOUT=20 seconds"
       - WAZUH_OPENSEARCH_INCLUDE_MATCH=
       - WAZUH_OPENSEARCH_INDEX=wazuh-alerts-*
       - WAZUH_OPENSEARCH_LIMIT=50
@@ -68,8 +73,6 @@ services:
       - WAZUH_SYSTEM_NAME="Wazuh SIEM"
       - WAZUH_TLPS=TLP:AMBER+STRICT
       - WAZUH_VULNERABILITY_INCIDENT_CVSS3_SCORE_THRESHOLD=
-    volumes:
-      - /var/cache/wazuh
     links:
       - opencti:opencti
     # Set a limit on logs:

docs/source/env-config.rst

@@ -45,9 +45,10 @@ whitespace or special characters, it must be quoted like this:
 Special characters
 ^^^^^^^^^^^^^^^^^^
 
-Dollar signs, **$**, but be escaped by an additional dollar sign, **$$**. This
+Dollar signs, **$**, must be escaped by an additional dollar sign, **$$**. This
 may be necessary in passphrases that contain these characters, e.g.:
 
+   "WAZUH_PASSWORD=MyPa$word" → "WAZUH_PASSWORD=MyPa$$word"
    "WAZUH_PASSWORD=MyPa$$word" → "WAZUH_PASSWORD=MyPa$$$$word"
 
 Enumerators
@@ -61,6 +62,18 @@ any of the following values are accepted:
 - ipv4-addr
 - Ipv4addr
 
+Date and time
+^^^^^^^^^^^^^
+
+Dates/times and relative times can be represented in almost any conceivable
+format (and in any langauge if locales are set up correctly), thanks to the
+Python library *dateparser*. Examples:
+
+- 2024-01-02 03:04:05
+- January 24, 2029 10:00 PM EST
+- In two months
+- Three weeks ago
+
 Complex data types
 ^^^^^^^^^^^^^^^^^^
 
@@ -74,7 +87,7 @@ values may be specified as a comma-separated list, e.g.:
 .. note::
 
    All sets of enums accept the special string "all", which will include every
-   defined enumerator.
+   defined enumerator, if the data type is a *set*.
 
 Other complex data types have their own environment variable-friendly syntax,
 documented in the settings reference.

docs/source/example_summary_note.rst

@@ -31,15 +31,19 @@ The meaning of all the fields in the summary table are as follows:
    * - Total hits
      - The total number of matches
    * - Max hits
-     - Maximum number of results to return, as per :py:attr:`~wazuh.config.Config.hits_limit`
+     - Maximum number of results to return, as per
+       :py:attr:`~wazuh.opensearch_config.OpenSearchConfig.limit`
    * - Dropped
      - Results dropped (Total hits - Max hits)
    * - Search since
-     - Time filter used, as per :py:attr:`~wazuh.config.Config.search_after`
+     - Time filter used, as per
+       :py:attr:`~wazuh.opensearch_config.OpenSearchConfig.search_after`
    * - Include filter
-     - Additional search filters used, as per :py:attr:`~wazuh.config.Config.search_include`
+     - Additional search filters used, as per
+       :py:attr:`~wazuh.config.OpenSearchConfig.include_match`
    * - Exclude filter
-     - Additional search filters used, as per :py:attr:`~wazuh.config.Config.search_exclude`
+     - Additional search filters used, as per
+       :py:attr:`~wazuh.config.OpenSearchConfig.exclude_match`
    * - Connector v.
      - The connector version at enrichment time
 

docs/source/full-demo-compose.yml

@@ -1,6 +1,6 @@
 services:
   redis:
-    image: redis:7.2.4
+    image: redis:7.2.5
     restart: always
     volumes:
       - redisdata:/data
@@ -33,7 +33,7 @@ services:
         max-size: 50m
 
   minio:
-    image: minio/minio:RELEASE.2024-01-16T16-07-38Z
+    image: minio/minio:RELEASE.2024-05-28T17-19-04Z
     volumes:
       - s3data:/data
     ports:
@@ -63,7 +63,7 @@ services:
         max-size: 50m
 
   opencti:
-    image: opencti/platform:6.1.4
+    image: opencti/platform:6.1.10
     environment:
       - NODE_OPTIONS=--max-old-space-size=8096
       - APP__PORT=8080
@@ -103,7 +103,7 @@ services:
         max-size: 50m
 
   worker:
-    image: opencti/worker:6.1.4
+    image: opencti/worker:6.1.10
     environment:
       - OPENCTI_URL=http://opencti:8080
       - OPENCTI_TOKEN=${OPENCTI_ADMIN_TOKEN}
@@ -120,7 +120,7 @@ services:
         max-size: 50m
 
   connector-export-file-stix:
-    image: opencti/connector-export-file-stix:6.1.4
+    image: opencti/connector-export-file-stix:6.1.10
     environment:
       - OPENCTI_URL=http://opencti:8080
       - OPENCTI_TOKEN=${OPENCTI_ADMIN_TOKEN}
@@ -139,7 +139,7 @@ services:
         max-size: 50m
 
   connector-import-file-stix:
-    image: opencti/connector-import-file-stix:6.1.4
+    image: opencti/connector-import-file-stix:6.1.10
     environment:
       - OPENCTI_URL=http://opencti:8080
       - OPENCTI_TOKEN=${OPENCTI_ADMIN_TOKEN}
@@ -180,8 +180,6 @@ services:
       - WAZUH_OPENSEARCH_USERNAME=cti_connector
       - WAZUH_OPENSEARCH_VERIFY_TLS=true
       - WAZUH_TLPS=TLP:AMBER+STRICT
-    volumes:
-      - /var/cache/wazuh
     # Set a limit on logs:
     logging:
       options: