Skip to content

Steam review parsing CLI application that processes reviews in CSV format in a concurrent and multithreaded manner using the fork join model [1Q2025]

Notifications You must be signed in to change notification settings

gabrieldiem/steam-review-parallel-parsing

Repository files navigation

Review Assignment Due Date

Alumno: Diem, Walter Gabriel

Padrón: 105618

Video:

Video Programación Concurrente TP1 Fork+Join: Steam Review Parsing

Enlace: youtu.be/vpgKecMkcTk

Instrucciones

Ejecución

cargo run --release <input-path> <num-threads> <output-file-name>

por ejemplo

cargo run --release ~/Downloads/dataset 4 output.json

Pruebas

  • La salida de la ejecución con el dataset completo debe ser igual a la del archivo expected_output.json, sin importar el orden de aparición de las keys en los mapas.

Table of contents

  1. Run tests
    1. Heavy full scope test
  2. Documentation
  3. "Bench" utility
  4. Contribute
    1. Pre-commit setup
    2. Build documentation

Run tests

To run all tests (unit tests and integration tests):

cargo test

Heavy full scope test

There is an integration test marked as ignored because it does not use the testfiles but rather the full CSV from the dataset. It is located at tests/tests_over_full_csv.rs.

To run this test you will need to update the variable FULL_CSV_DIRPATH with the directory path that contains a single full CSV from the dataset. The output will be checked against the provided expected JSON output.

To run it execute:

cargo test --release -- --ignored

Documentation

You can read the project documentation that is located in the docs folder in this path:

docs/doc/steam_review_parser_tp1/index.html

"Bench" utility

Small utility to run batches of timed runs for different file sizes of the CsvParser library. Was used along the project to track changes in performance.

It is heavy to run because it will run various tests with different number of threads with fixed locations of datasets that are stores in variables in the code. You can run it, after configuring the folders with the next command:

cargo run --bin bench --release

Contribute

Pre-commit setup

Install pre-commit hooks:

pre-commit install

To run all pre-commit hooks without making a commit run:

pre-commit run --all-files

To create a commit without running the pre-commit hooks run:

git commit --no-verify

Build documentation

To build the documentation for the project run:

cargo doc --no-deps --document-private-items --verbose --open --color always --release --target-dir ./docs

About

Steam review parsing CLI application that processes reviews in CSV format in a concurrent and multithreaded manner using the fork join model [1Q2025]

Topics

Resources

Stars

Watchers

Forks

Contributors 2

  •  
  •  

Languages