v0.2.0

v0.2.0 includes quite a number of changes and improvements over v0.1.8. There were new features added and overall code quality improvements. The most important changes are outlined below.

Breaking Changes

• Addition of dialerGroup to the Scan() module API. This will let ZGrab be used as a library and let clients bring their own dialers/network connections. This theoretically would let you run any application protocol over any transport you want. In practice, several protocols are more tightly coupled and need both a TCP dialer, they'll exchange some info, and then establish a TLS connection. STARTTLS is an example of this. More info in #506.
• Made the default behavior for smtp more reasonable where it can auto-detect support for EHLO vs. HELO in the banner as well as STARTTLS support. Adds CLI override flags to specify behavior if user wishes to override. Removes --send-ehlo, --send-start-tls and --send-helo since these now auto-detect. Adds --send-X-override flags for overriding. #509

Major Changes/Improvements

• Categorizes the CLI flags for modules to group them into Application Options that affect all modules, Basic Options for a module that they all share, and module-specific options. Will let users find the flag they need faster. See #479
• Fixed timeout behavior which would let scans run far longer than expected. Provides the user with --connect-timeout to control initial connection establishment and --target-timeout to bound the runtime of a scan against a given input target. See #537
• User can specify --local-addr and --local-port to bind 1+ IPs to for outgoing connections. See #519
• Changes IPv6 behavior to assume IPv4 and have IPv6 as an optional CLI flag
• Adds support for specifying 1+ DNS resolvers to use for lookups. Default behavior remains to use the OS configured defaults (/etc/resolv.conf on UNIX) See #556
• Add a ZMap-style blocklist. See #553
• Adds a ZMap-style per-second scan status print-out to give constant user feedback on how a scan is proceeding. See #528

New Modules

• AMQP 0.9.1
• PPTP
• MQTT
• socks5

All Changes

• Bump google.golang.org/protobuf from 1.30.0 to 1.33.0 by @dependabot in #414
• build: bump alpine from 3.19 to 3.20 by @dependabot in #442
• New Protocol: AMQP 0.9.1 by @developStorm in #426
• fixed typo in cleanup script by @phillip-stephens in #445
• docs: docker usage by @developStorm in #438
• Fix non-compilation in /lib/ssh/messages_test.go by @phillip-stephens in #446
• style: reformat all go files by @developStorm in #437
• Try to align golang around a single version by @Seanstoppable in #461
• Upgrade golang.org/x/crypto from 0.23.0 to 0.31.0 by @Seanstoppable in #476
• fix: failing ci tests by @developStorm in #478
• Group Basic/TLS Flag Groupings in Modules and interface{} -> any by @phillip-stephens in #479
• build(deps): bump github.com/prometheus/client_golang from 1.14.0 to 1.20.5 by @dependabot in #482
• build(deps): bump golang.org/x/crypto from 0.31.0 to 0.32.0 by @dependabot in #480
• Document the PORT field for CSV-format input by @dlenskiSB in #456
• Upgrade to go 1.23 by @thecsw in #466
• build(deps): bump alpine from 3.20 to 3.21 by @dependabot in #484
• build(deps): bump actions/attest-build-provenance from 1 to 2 by @dependabot in #483
• build(deps): bump docker/build-push-action from 5 to 6 by @dependabot in #453
• Update zgrab's python2 dependency to python3 by @phillip-stephens in #485
• Use bodyTextLen instead of readLen for FailHTTPToHTTPS logic by @Nirusu in #345
• Add support for PPTP by @AlexAQ972 in #470
• Make banner read options configurable via CLI by @svbatalov in #468
• Add support for MQTT v3.1.1 and v5.0 by @AlexAQ972 in #467
• Add support for socks5 by @AlexAQ972 in #471
• Fix ssh by @espegro in #335
• ssh: Port lib/ssh from x/crypto v0.7.0 by @TrueSkrillor in #399
• Bump ubuntu from 16.04 to 24.04 in /docker-runner by @dependabot in #430
• Update Golang minimum version/installation instructions by @phillip-stephens in #488
• Test Automation Performance Improvements + Streamlining by @phillip-stephens in #489
• build(deps): bump golang.org/x/sys from 0.29.0 to 0.30.0 by @dependabot in #490
• build(deps): bump golang.org/x/text from 0.21.0 to 0.22.0 by @dependabot in #491
• build(deps): bump golang.org/x/crypto from 0.32.0 to 0.33.0 by @dependabot in #492
• build(deps): bump golang.org/x/net from 0.34.0 to 0.35.0 by @dependabot in #493
• build(deps): bump github.com/prometheus/client_golang from 1.20.5 to 1.21.0 by @dependabot in #494
• Bump all dependencies by @phillip-stephens in #497
• Generate server certs for MQTT test container by @Seanstoppable in #505
• Don't disable http tee after first header, so that headers of re-directs can be captured and we don't panic by @phillip-stephens in #504
• Upgrade the prometheus dependency by @phillip-stephens in #508
• Fix issues with running subset of integration tests by @phillip-stephens in #510
• Improve SMTP usability by @phillip-stephens in #509
• Add back in EHLO, HELO, and STARTTLS Overrides to SMTP by @phillip-stephens in #512
• Update the zcrypto dependency to bring in TLS 1.3 support by @phillip-stephens in #507
• build(deps): bump github.com/zmap/zcrypto to 4f0ea0eaccac by @developStorm in #513
• modbus - avoid array indexing out-of-bounds by @phillip-stephens in #514
• Fix SHA256 typo by @hegga in #515
• New DialerGroupConfig interface for modules to given greater control over connection (L4/TLS/etc) establishment by @phillip-stephens in #506
• WIP - Add large HTTP file integration test and remove --maxbytes by @phillip-stephens in #516
• Fix redirects not getting followed by @phillip-stephens in #518
• Use context.Deadline in addition to specified timeouts for connections by @phillip-stephens in #517
• build(deps): bump golang.org/x/crypto from 0.36.0 to 0.37.0 by @dependabot in #521
• Throw an error is all we got was a bunch of empty hashes for JARM by @Seanstoppable in #525
• Improve ZGrab Docs/--help by @phillip-stephens in #524
• Let user specify local addr(s) and/or local port(s) to be used for making connections by @phillip-stephens in #519
• Remove min() re-declarations by @phillip-stephens in #527
• Add a per-second status similar to ZDNS by @phillip-stephens in #528
• build(deps): bump golang.org/x/net from 0.38.0 to 0.39.0 by @dependabot in #529
• build(deps): bump github.com/prometheus/client_golang from 1.21.1 to 1.22.0 by @dependabot in #530
• Move setting global TLS parameters to init() so it's run once and -race won't complain by @phillip-stephens in #532
• Remove unused single local addr by @phillip-stephens in #531
• Stabilize conn timeout unit tests by @phillip-stephens in #536
• Fix Multiple Module regression by @phillip-stephens in #534
• Make timeout behavior user-changeable and easy-to-understand by @phillip-stephens in #537
• Downgrade to using v7 of redis, since v8 is buggy right now by @phillip-stephens in #539
• Give user IP that domain resolves to by @phillip-stephens in #533
• Fix timeout issue with setting ReadDeadline after conn creation by @phillip-stephens in #538
• Parse CAs file only if the file is different from last parsed by @phillip-stephens in #540
• Give accurate status for io-timeout by @phillip-stephens in #541
• build(deps): bump golang.org/x/sys from 0.32.0 to 0.33.0 by @dependabot in #543
• build(deps): bump golang.org/x/net from 0.39.0 to 0.40.0 by @dependabot in #545
• Add several linters to CI and fix found issues by @phillip-stephens in #542
• Fixes a few issues with redirects by @Seanstoppable in #462
• [HTTP] base64 encode binary data by @phillip-stephens in #549
• Set http default port to 443 if using --use-https by @phillip-stephens in #548
• Improve IP capability detection performance and over VPNs by @phillip-stephens in #552
• Add a ZMap-style Blocklist by @phillip-stephens in #553
• Bugfix - allow user to set the --dns option by @phillip-stephens in #556
• Remove IP capability detection, enable IPv4 by default by @phillip-stephens in #557
• build(deps): bump golang.org/x/text from 0.25.0 to 0.26.0 by @dependabot in #561
• build(deps): bump golang.org/x/crypto from 0.38.0 to 0.39.0 by @dependabot in #562
• build(deps): bump golang.org/x/net from 0.40.0 to 0.41.0 by @dependabot in #563
• Don't error if you can't close the connection by @phillip-stephens in #559
• Fix for status reporting not being complete and bound time of DNS resolutions by @phillip-stephens in #558
• [bug-fix] - Fix not including the body of a re-directed HTTP request by @phillip-stephens in #564
• Have default make target build, not install by @phillip-stephens in #566

New Contributors

• @phillip-stephens made their first contribution in #445
• @dlenskiSB made their first contribution in #456
• @thecsw made their first contribution in #466
• @AlexAQ972 made their first contribution in #470
• @espegro made their first contribution in #335
• @TrueSkrillor made their first contribution in #399
• @hegga made their first contribution in #515

Full Changelog: v0.1.8...v0.2.0