fixing github workflows, updating .gitignore and pre-commit config

rafaelgreca · rafaelgreca · commit a27ab8fc92e6 · 2024-11-13T16:22:17.000-03:00
diff --git a/.github/workflows/pylint.yaml b/.github/workflows/pylint.yaml
@@ -17,7 +17,8 @@ jobs:
     - name: Install dependencies
       run: |
         python -m pip install --upgrade pip
-        pip install pylint
+        if [ -f requirements.txt ]; then pip install -r requirements.txt; fi
     - name: Analysing the code with pylint
       run: |
-        pylint $(git ls-files '*.py') --rcfile=.pylintrc
+        pip install pylint
+        pylint $(git ls-files '*.py') --rcfile=.pylintrc
diff --git a/.github/workflows/test.yaml b/.github/workflows/test.yaml
@@ -10,19 +10,15 @@ jobs:
         python-version: ["3.10"]
 
     steps:
-    - uses: actions/checkout@v3
+    - uses: actions/checkout@v4
     - name: Set up Python ${{ matrix.python-version }}
-      uses: actions/setup-python@v4
+      uses: actions/setup-python@v3
       with:
         python-version: ${{ matrix.python-version }}
     - name: Install dependencies
       run: |
         python -m pip install --upgrade pip
         if [ -f requirements.txt ]; then pip install -r requirements.txt; fi
-    - name: Python Black
-      run: |
-        pip install black==24.10.0
-        black . --check
     - name: Test with pytest
       run: |
-        pytest --cov-report html:./results/cov_html --cov=src tests/
+        pytest --cov-report html:./results/cov_html tests/
diff --git a/.gitignore b/.gitignore
@@ -2,7 +2,7 @@ logs/
 ipynb_checkpoints/
 mlruns
 mlartifacts
-*.csv
+# *.csv
 
 # Byte-compiled / optimized / DLL files
 __pycache__/
diff --git a/.pre-commit-config.yaml b/.pre-commit-config.yaml
@@ -6,7 +6,6 @@ repos:
       -   id: check-yaml
       -   id: end-of-file-fixer
       -   id: trailing-whitespace
-      -   id: check-added-large-files
       -   id: debug-statements
           language_version: python3
 
diff --git a/README.md b/README.md
@@ -61,4 +61,4 @@ dvc pull
 Reproduces the pipeline using DVC
 ```
 dvc repro
-```
+```
diff --git a/config/metadata.yaml b/config/metadata.yaml
@@ -1,4 +1,4 @@
 author: Rafael Greca Vieira
 model_type: scikit-learn_0.23
 project_name: e2e-mlops-project
-project_version: v0.1
+project_version: v0.1
diff --git a/config/settings.yaml b/config/settings.yaml
@@ -7,11 +7,11 @@ EXPERIMENT_ID: '708342276728582022'
 VERSION: '1.0'
 
 # GENERAL SETTINGS
-DATA_PATH: '/media/greca/HD/GitHub/e2e-mlops-project/data/'
+DATA_PATH: './data/'
 RAW_FILE_NAME: 'Original_ObesityDataSet.csv'
-ARTIFACTS_PATH: '/media/greca/HD/GitHub/e2e-mlops-project/models/artifacts/'
-FEATURES_PATH: '/media/greca/HD/GitHub/e2e-mlops-project/models/features/'
-RESEARCH_ENVIRONMENT_PATH: '/media/greca/HD/GitHub/e2e-mlops-project/notebooks/' 
+ARTIFACTS_PATH: './models/artifacts/'
+FEATURES_PATH: './models/features/'
+RESEARCH_ENVIRONMENT_PATH: './notebooks/'
 TARGET_COLUMN: 'NObeyesdad'
 LOG_LEVEL: 'INFO'
-LOG_PATH: '/media/greca/HD/GitHub/e2e-mlops-project/'
+LOG_PATH: './'
diff --git a/data/Original_ObesityDataSet.csv b/data/Original_ObesityDataSet.csv
diff --git a/data/Preprocessed_ObesityDataSet.csv b/data/Preprocessed_ObesityDataSet.csv
diff --git a/models/README.md b/models/README.md
@@ -2,4 +2,4 @@
 
 The models and artifacts will not be stored locally but rather in an AWS S3 Bucket to simulate a real-world scenario where models will have different versions (model versioning).
 
-This folder will be used temporarily to save the models and artifacts locally and then transfer it to your AWS S3 bucket. After that, the files will be deleted. If you choose to not use an AWS S3 Bucket and an AWS RDS Databaset, then the `artifacts` and the `features` will be stored into the `models` folder.
+This folder will be used temporarily to save the models and artifacts locally and then transfer it to your AWS S3 bucket. After that, the files will be deleted. If you choose to not use an AWS S3 Bucket and an AWS RDS Databaset, then the `artifacts` and the `features` will be stored into the `models` folder.
diff --git a/requirements.txt b/requirements.txt
@@ -1,6 +1,8 @@
-boto3==1.24.28
+awscli==1.34.0
+boto3==1.35.0
 fastapi==0.115.5
 joblib==1.3.2
+kaggle==1.6.17
 loguru==0.7.2
 mlflow==2.15.1
 numpy==1.21.5
diff --git a/results/README.md b/results/README.md
@@ -1,3 +1,3 @@
 # Results
 
-Here goes the results of evaluates, like metrics, graphs or others.
+Here goes the results of evaluates, like metrics, graphs or others.
diff --git a/tests/integration/test_model_inference.py b/tests/integration/test_model_inference.py
@@ -47,7 +47,6 @@ def test_model_inference_pipeline() -> None:
     assert predictions.shape[0] == features.shape[0]
     assert isinstance(predictions.dtype, type(np.dtype("float64")))
 
-    # FIXME: fix this
     # predictions = loaded_model.predict(x, transform_to_str=True)
 
     # assert isinstance(predictions, List)
diff --git a/tests/unit/test_model_functions.py b/tests/unit/test_model_functions.py
@@ -89,7 +89,6 @@ def test_prediction() -> None:
 #     Unit case to test the model performance on training and validation sets
 #     (making sure that are the same values as mentioned in MLflow's UI).
 #     """
-#     # FIXME: fix this
 #     indexes = [FEATURES_NAME.index(i) for i in model_settings.FEATURES]
 
 #     loaded_model = ModelServe(

Original file line number	Diff line number	Diff line change
`@@ -2,4 +2,4 @@`
`2`	`2`
`3`	`3`	`The models and artifacts will not be stored locally but rather in an AWS S3 Bucket to simulate a real-world scenario where models will have different versions (model versioning).`
`4`	`4`
`5`		-This folder will be used temporarily to save the models and artifacts locally and then transfer it to your AWS S3 bucket. After that, the files will be deleted. If you choose to not use an AWS S3 Bucket and an AWS RDS Databaset, then the `artifacts` and the `features` will be stored into the `models` folder.
	`5`	+This folder will be used temporarily to save the models and artifacts locally and then transfer it to your AWS S3 bucket. After that, the files will be deleted. If you choose to not use an AWS S3 Bucket and an AWS RDS Databaset, then the `artifacts` and the `features` will be stored into the `models` folder.