Skip to content

Feature Engineering: Word Difficulty Scoring

Shakleen Ishfar edited this page Jun 8, 2024 · 2 revisions

tldr; Sum of scores appears to work the best.

Code Commit

sum_of_word_difficulty_scores

Higher scored essays seem to have higher sum of difficult words. Exact calculation details in code.

After training LGBM model the importance increased quite a bit as shown below:

image

Clone this wiki locally