Skip to content

Commit 5e14949

Browse files
authored
Update demo.ipynb
1 parent c5aac63 commit 5e14949

File tree

1 file changed

+3
-3
lines changed

1 file changed

+3
-3
lines changed

tests/demo.ipynb

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -315,7 +315,7 @@
315315
"from hinteval.evaluation.relevance import Rouge\n",
316316
"from hinteval.evaluation.readability import MachineLearningBased\n",
317317
"from hinteval.evaluation.convergence import LlmBased\n",
318-
"from hinteval.evaluation.familiarity import Wikipedia\n",
318+
"from hinteval.evaluation.familiarity import WordFrequency\n",
319319
"from hinteval.evaluation.answer_leakage import ContextualEmbeddings"
320320
],
321321
"metadata": {
@@ -332,7 +332,7 @@
332332
"1. **Relevance (_Rouge_)**: Measures the relevance of the hints to the question using ROUGE-L algorithms.\n",
333333
"2. **Readability (_MachineLearningBased_)**: Uses a Random Forest algorithm to measure the readability of the hints and questions.\n",
334334
"3. **Convergence (_LLmBased_)**: Assesses how well the hints help eliminate incorrect answers using the Meta LLaMA-3.1-70b-Instruct-Turbo model.\n",
335-
"4. **Familiarity (_Wikipedia_)**: Evaluates the familiarity of the information in the hints, questions, and answers based on Wikipedia view counts.\n",
335+
"4. **Familiarity (_WordFrequency_)**: Evaluates the familiarity of the information in the hints, questions, and answers based on word occurancy.\n",
336336
"5. **AnswerLeakage (_ContextualEmbeddings_)**: Measures how much the hints reveal the answers by calculating the similarity between the hints and answers using contextual embeddings.\n",
337337
"\n",
338338
"To explore other metrics, check the [metrics guide](https://hinteval.readthedocs.io/en/latest/concepts/metrics/index.html)."
@@ -384,7 +384,7 @@
384384
"Rouge('rougeL', enable_tqdm=True).evaluate(instances)\n",
385385
"MachineLearningBased('random_forest', enable_tqdm=True).evaluate(questions + hints)\n",
386386
"LlmBased('llama-3-70b', together_ai_api_key=api_key, enable_tqdm=True).evaluate(instances)\n",
387-
"Wikipedia(enable_tqdm=True).evaluate(questions + hints + answers)\n",
387+
"WordFrequency(enable_tqdm=True).evaluate(questions + hints + answers)\n",
388388
"ContextualEmbeddings(enable_tqdm=True).evaluate(instances)\n",
389389
"pass"
390390
],

0 commit comments

Comments
 (0)