First version of the cc-downloader library

[pjox]

Description

This PR contains the first version of the cc-downloader library with some light documentation.

Breaking Changes

None

Notes & open questions

We need to discuss the API, this code makes cc-downloader usable as a library but setting the download options remains a bit convoluted. Here is an example on how to import and use the library:

use cc_downloader::{
    download::{DownloadOptions, download, download_paths},
    errors::DownloadError,
};

#[tokio::main]
async fn main() -> Result<(), DownloadError> {
    let options = DownloadOptions::new(
        "CC-MAIN-2025-18",
        "wet",
        "test-download/wet.paths.gz",
        "test-download/",
        10,
        1000,
        false,
        false,
        true,
    );
    download_paths(options.clone()).await?;
    download(options).await?;
    Ok(())
}

As you can see, the DownloadOptions struct contains far too many parameters, so maybe it would be a good idea to split this into multiple structures? All feedback is welcome. Also, once the API is stabilized, writing bindings should be easier.

[jenenglish]

@wumpus Just highlighting this is ready for review.

[wumpus]

I don't know much about Rust library interfaces. Is there a reason why the DownloadOptions don't have names and are a list?