Skip to content

Home

Jump to bottom
Aurelie Herbelot edited this page Jan 4, 2021 · 2 revisions

Benchmarking information

Processing .wet files

For your information, the sequential processing a single Common Crawl file on one core takes around 20 mins. Expect an output file of 150M uncompressed.

Clone this wiki locally