Feature Engineering: Cohesion Per Paragraph

tldr; Effective (CV: 0.7212 -> 0.7224) Commit

Experiment

Measure cohesion as similarity between sentences of the same paragraph.

1. Pairwise: Between each pair of sentences.
2. Dual: Between contiguous two sentences
3. Triple: Between contiguous three sentences.

Scorewise Relationship

Pairwise

Dual

Triple

Feature importance

Pairwise is the most important across 7 LGBM folds.