A comprehensive collection of Bangla NLP datasets for researchers and developers
🔄 OUR DATASET IS IN LFS MODE! SO YOU HAVE TO CLONE IT FOR GETTING DATA!
🚀 WE WILL SOON UPLOAD ALL DEEP LEARNING BASED DATASETS!
- 📖 About
- 🎯 sbnltk Dataset List
- 🤖 Pre-trained Language Models
- 📄 Research Papers
- 🔧 Modern NLP Tools and Libraries
- 📊 Benchmarking and Evaluation
- 🌟 Existing Datasets
- 📰 News Articles and Documents
- 🎤 Speech to Text / Text to Speech
- 😊 Sentiment Analysis / Sentence Classification
- 🔄 Bangla Machine Translation Dataset
- 🏷️ Bangla POS Tag Dataset
- 🏷️ Bangla NER Dataset
- ❓ Question Answering Dataset
- 📝 Bangla Text Summarization
- 🕵️ Bangla Fake News Detection
- 🖊️ Handwriting Recognition / OCR
- 🔧 Miscellaneous
- 💡 Motivation
- 🤝 Usage and Contribute
Bangla NLP dataset repository containing sbnltk datasets, which were used in Bangla nlp toolkit - sbnltk.
This repository also serves as a comprehensive collection of existing Bangla NLP datasets created by the amazing Bangla NLP research community.
| Dataset | Description | Link |
|---|---|---|
| Number List | Bangla Number List | 📥 Download |
| Root Word List | Bangla root word List | 📥 Download |
| Word List | Bangla Word List (highest to lowest occurrence) | 📥 Download |
| Wiki Dump | Bangla Wiki Dump word | 📥 Download |
| POS Tag Static | Bangla POStag static dataset (single word) | 📥 Download |
| NER Static | Bangla NER Static Dataset (single word) | 📥 Download |
| Stop Words | Bangla Stop word list | 📥 Download |
| Dump POS Tag | Bangla Dump Pos tag | 📥 Download |
| Question Classification | Bangla Dump question Classification Dataset | 📥 Download |
| Sentiment Analysis | Bangla Dump Sentiment Analysis | 📥 Download |
| Translation Dataset | Google Translation Dataset | 📥 Download |
| NER Enhanced | NER Existing Dataset (Modified + adding Date entity) | 📥 Download |
| News Articles | News Article Dataset | 📥 Download |
| POS Converted | POS tag converted Data | 📥 Download |
| POS Human Evaluated | POS tag human evaluated Data | 📥 Download |
| NER Dump (Both) | DUMP NER data (active and passive both) | 📥 Download |
| NER Dump (Active) | DUMP NER data (active only) | 📥 Download |
| Extractive Summarization | Extractive Text Summarization | 🔗 GitHub |
| Abstractive Summarization | Abstractive Text Summarization (newspaper) | 📥 Drive | 📊 Kaggle |
| Text Classification | News Article Classification (text Classification) | 📥 Drive | 📊 Kaggle |
| Keywords Classification | Topic Keywords classification (keywords generator) | 📥 Drive | 📊 Kaggle |
| Model | Description | Parameters | Link |
|---|---|---|---|
| BanglaBERT | ELECTRA-based model, state-of-the-art Bangla NLU | 110M | 🤗 HuggingFace |
| BanglishBERT | Bilingual (Bangla+English) BERT | 110M | 🤗 HuggingFace |
| BanglaBERT (Small) | Lightweight version for resource-constrained environments | 13M | 🤗 HuggingFace |
| BanglaBERT (Large) | Large variant with enhanced performance | 335M | 🤗 HuggingFace |
| Bangla BERT Base | Another popular BERT implementation | 110M | 🤗 HuggingFace |
| Bangla Electra | ELECTRA-based model for Bangla | 13.5M | 🤗 HuggingFace |
| Model | Description | Parameters | Link |
|---|---|---|---|
| BanglaT5 | T5-based sequence-to-sequence model | 247M | 🤗 HuggingFace |
| BanglaByT5 | Byte-level T5 model for Bangla | Small | 📄 Research Paper |
| TituLLMs | Family of Bangla LLMs (1B & 3B) | 1B/3B | 📄 Research Paper |
| TigerLLM | Bangla Large Language Models family | Various | 📄 Research Paper |
| GPT2-Bangla | GPT-2 adapted for Bangla text generation | 117M | 🤗 HuggingFace |
| BanglaNLG | Natural language generation for Bangla | Various | 🤗 HuggingFace |
| Model | Description | Performance | Link |
|---|---|---|---|
| Wav2Vec2-Bangla-300M | Self-supervised speech recognition | 17.8% WER | 🤗 HuggingFace |
| Whisper-Bangla | OpenAI Whisper fine-tuned for Bangla | Various sizes | 🤗 HuggingFace |
| BanglaASR | Fine-tuned ASR model | 14.73% WER | 🔗 GitHub |
| Model | Description | Languages | Link |
|---|---|---|---|
| MuRIL | Google's multilingual model with Bangla support | 17 Indian | 🤗 HuggingFace |
| IndicBERT | BERT for Indian languages including Bangla | 12 Indian | 🤗 HuggingFace |
| sahajBERT | ALBERT-based model for Bangla | 18M | 🤗 HuggingFace |
- BanglaAutoKG: Automatic Bangla Knowledge Graph Construction with Semantic Neural Graph Filtering - 📖 LREC-COLING 2024 | 💻 Code
- First framework for automatic Bangla KG construction using multilingual LLMs
- GNN-based semantic filtering for improved accuracy
- Bangladesh Agricultural Knowledge Graph: Enabling Semantic Integration and Data-driven Analysis - 📖 IEEE Access 2024
- FAIR-compliant agricultural knowledge graph for sustainable farming
- BanglaDialecto: An End-to-End AI-Powered Regional Speech Standardization - 📖 arXiv 2024
- First end-to-end pipeline for Bangla dialect standardization
- Achieved 0.8% CER and 1.5% WER for Noakhali dialect
- Wav2Vec2-Bangla (300M) - 🤗 HuggingFace
- Self-supervised speech model with 17.8% WER
- Trained on OpenSLR Bangla dataset
- BanglaByT5: Byte-Level Modelling for Bangla - 📖 arXiv
- TituLLMs: A Family of Bangla LLMs with Comprehensive Benchmarking - 📖 arXiv
- TigerLLM: A Family of Bangla Large Language Models - 📖 arXiv
- Bangla/Bengali Seed Dataset for WMT24 - 📖 Paper
- BLUB: A Comprehensive Evaluation Benchmark for Bangla Language Understanding - 📖 Research
- First comprehensive Bangla NLP benchmark with 15+ tasks
- BanglaBook: Large-scale Bangla Dataset for Sentiment Analysis - 📖 ACL 2023
- 158K+ book reviews for sentiment analysis
- Cross-lingual Transfer Learning for Bangla: What Works and What Doesn't - 📖 Findings of ACL 2024
- BanglaBERT: Language Model Pretraining and Benchmarks - 📖 NAACL 2022
- BanglaNLG and BanglaT5: Benchmarks for Bangla NLG - 📖 EACL 2023
- MuRIL: Multilingual Representations for Indian Languages - 📖 Research Paper
- IndicBERT: A Pre-trained Language Model for Indian Languages - 📖 ACL 2022
- Text Summarization Paper - 📖 IEEE
- Natural Language Inference in Bangla - 📖 Research Paper
- Sentiment Analysis in Bangla Text: A Comprehensive Study - 📖 Research
- Named Entity Recognition for Bangla: Challenges and Solutions - 📖 LREC 2022
- Bangla Speech Recognition: Traditional to Neural Approaches - 📖 INTERSPEECH 2023
- Cross-lingual Speech Recognition for Bangla - 📖 ICASSP 2023
- Multimodal Learning for Bangla: Vision and Language - 📖 CVPR 2023
- Cross-lingual Transfer for Low-Resource Languages: A Bangla Case Study - 📖 EMNLP 2023
- Multilingual Models for South Asian Languages - 📖 ACL 2023
- Zero-shot Learning for Bangla NLP Tasks - 📖 Findings of ACL 2023
| Library | Description | Features | Link |
|---|---|---|---|
| BNLP | Bengali Natural Language Processing Toolkit | Tokenization, Embedding, POS, NER | 🔗 GitHub |
| BNLTK | Bangla Natural Language Processing Toolkit | Tokenization, Stemming, POS Tagging | 🔗 GitHub |
| sbnltk | Bangla NLP toolkit (this repository's toolkit) | Comprehensive NLP suite | 🔗 GitHub |
| bnunicode | Unicode normalization for Bangla text | Bijoy to Unicode, normalization | 🔗 GitHub |
| pyBanglaKit | Comprehensive Bangla text processing | Tokenization, spell checking, sentiment | 🔗 GitHub |
| Indic NLP Library | Multi-Indic language processing | Script conversion, transliteration | 🔗 GitHub |
| BanglaTextProcessor | Advanced text processing pipeline | Dependency parsing, coreference | 🔗 GitHub |
| Tool | Description | Features | Link |
|---|---|---|---|
| BanglaOCR | Comprehensive OCR system for Bangla | Print & handwriting recognition | 🔗 GitHub |
| EasyOCR-Bangla | Ready-to-use OCR solution | Simple Python API | 🔗 GitHub |
| TesseractBN | Tesseract with Bangla support | Command-line & API access | 🔗 GitHub |
| BanglaHWR | Handwriting recognition system | Real-time recognition | 🔗 GitHub |
| Tool | Description | Features | Link |
|---|---|---|---|
| BanglaVoice | Neural TTS system | Natural speech synthesis | 🔗 GitHub |
| FastSpeech-Bangla | Fast and robust TTS | Real-time synthesis | 🔗 GitHub |
| BanglaPhoneme | Phoneme analysis toolkit | IPA transcription support | 🔗 GitHub |
# BNLP installation
pip install bnlp_toolkit
# BNLTK installation
pip install bnltk| Task | Dataset | Metric | Best Model | Score |
|---|---|---|---|---|
| Sentiment Classification | SentNoB | Macro-F1 | BanglaBERT | 72.89 |
| Natural Language Inference | BNLI | Accuracy | BanglaBERT (Large) | 83.41 |
| Named Entity Recognition | MultiCoNER | Micro-F1 | BanglaBERT (Large) | 79.20 |
| Question Answering | BQA/TyDiQA | EM/F1 | BanglaBERT (Large) | 76.10/81.50 |
| Dataset | Task | Size | Description | Link |
|---|---|---|---|---|
| BanglaBook | Sentiment Analysis | 158,065 samples | Book reviews sentiment analysis | 🔗 GitHub |
| SentMix-3L | Code-Mixed Sentiment | 1,007 samples | Bangla-English-Hindi code-mixed | 🔗 GitHub |
| Awesome Bangla Datasets | Various | Multiple | Comprehensive collection | 🔗 GitHub |
📝 Note: I am not the owner of these following datasets. It's just a collection to find amazing peoples and their works.
🙏 Please give them support! Your support will encourage them to do more amazing things.
| Dataset | Description | Link |
|---|---|---|
| Wiki Articles | Wikipedia Articles in Bangla | 📊 Kaggle |
| Bangladesh Protidin | News from Bangladesh Protidin | 📊 Kaggle |
| 40k News Articles | 40k Bangla Newspaper Articles | 📊 Kaggle |
| Largest News Dataset | Bangla Largest Newspaper Dataset | 📊 Kaggle |
| Wikipedia Dumps | All types of Wikipedia Articles | 🔗 Wiki Dumps |
| bdNews24 Corpus | bdNews24 largest dataset | 📊 Kaggle |
| Dataset | Description | Size | Link |
|---|---|---|---|
| OpenSLR Bangla | Large-scale speech corpus | 250+ hours, 2000+ speakers | 🔗 OpenSLR |
| Common Voice Bangla | Crowdsourced speech data | 500+ hours (growing) | 🔗 Mozilla |
| FLEURS Bangla | Cross-lingual speech corpus | 12 hours | 🤗 HuggingFace |
| BanglaASR Dataset | Fine-tuned ASR corpus | 23.8 hours | 🔗 GitHub |
| Text to Speech | Bengali Text to Speech Dataset | Studio quality | 🔗 Bengali.ai |
| Speech Recognition | Bengali Automatic Speech Recognition Dataset | Various speakers | 🔗 Bengali.ai |
| Regional Dialect ASR | Dialect-specific speech recognition | 100+ hours, 8 dialects | 🔗 GitHub |
| Multi-Speaker TTS | Multiple speaker TTS corpus | 20 hours, 10 speakers | 🔗 GitHub |
| Expressive TTS Dataset | Emotional speech synthesis | 15 hours, 8 emotions | 🔗 GitHub |
| Handwritten Digits | Numta Handwritten Bengali Digits | Visual recognition | 🔗 Bengali.ai |
| Dataset | Description | Link |
|---|---|---|
| BanglaBook | Large-scale book reviews (158K samples) | 🔗 GitHub |
| SentMix-3L | Code-mixed sentiment (Bangla-English-Hindi) | 🔗 GitHub |
| Social Media Comments | Bangla Text Dataset from Social Media | 🔗 GitHub |
| Sentiment Analysis | Bengali Sentiment Text | 📊 Kaggle |
| News Classification | Classification Bengali News Articles | 📊 Kaggle |
| Drama Review | Bangla Drama Review Dataset | 📊 Figshare |
| News Comments | Bengali News Comments Sentiment | 📊 Kaggle |
| News Headlines | News Headline Classification | 📊 Kaggle |
| Big News Classification | Bangla Newspaper Article Classification (Large) | 📊 Kaggle |
| YouTube Sentiment | Bangla YouTube Sentiment/Emotion Dataset | 📊 Kaggle |
| Multilingual Sentiment | Sentiment Lexicons for 81 Languages | 📊 Kaggle |
| Twitter Dataset | Twitter Sentiment Analysis Dataset | 🔗 GitHub |
| EmoNoBa | Emotion analysis on noisy Bangla texts | 🔗 GitHub |
| SentiGOLD | Multi-domain sentiment analysis | 🔗 GitHub |
| Bangla Emotion Corpus | Comprehensive emotion detection | 🔗 GitHub |
| Social Media Sentiment | Social media specific sentiment | 🔗 GitHub |
| Bangla Fake News Detection | Misinformation detection dataset | 📊 Kaggle |
| BanglaSarc | Sarcasm detection dataset | 🔗 GitHub |
| Complaint Classification | Customer complaint categorization | 🔗 GitHub |
| Dataset | Description | Link |
|---|---|---|
| 2.5M Pairs | 2.5M pair sentences - NOT low resource anymore | 🔗 GitHub |
| WMT24 Seed Dataset | High-quality manual translations | 📖 Paper |
| TED Dataset | TED dataset (small) | 📥 Download |
| Bangla Dictionary | Bengali Dictionary | 🔗 GitHub |
| SUPERA Dataset | SUPARA08M Balanced English-Bangla Parallel Corpus | 📊 IEEE DataPort |
| Samanantar | Large-scale parallel corpus | 🔗 AI4Bharat |
| OPUS Collections | Multiple parallel corpora | 🔗 OPUS |
| Indic-Indic Translation | Inter-Indic language translation | 🔗 GitHub |
| BanglaDialectTranslation | Regional dialect to standard Bangla | 🔗 GitHub |
| Vashantor | Multi-regional dialect corpus | 🔗 GitHub |
| Legal Translation Corpus | Legal document translation | 🔗 GitHub |
| Medical Translation Dataset | Healthcare translation | 🔗 GitHub |
| Dataset | Description | Link |
|---|---|---|
| 3k Sentences | 3k POS tag sentences | 🔗 GitHub |
| 100k+ Words | Single word tagging 100k+ | 📊 Kaggle |
| Dataset | Description | Link |
|---|---|---|
| 70k Sentences | 70k sentences with 5 types of NER | 🔗 GitHub |
| 400k+ Words | Word-level NER 400k+ | 📊 Kaggle |
| B-NER | Comprehensive Bangla NER dataset | 🔗 GitHub |
| BanglaPersonNER | Person name extraction | 🔗 GitHub |
| Complex NER Dataset | Multi-type entity recognition | 🔗 GitHub |
| Medical NER Dataset | Healthcare entity recognition | 🔗 GitHub |
| Financial NER Corpus | Finance domain entities | 🔗 GitHub |
| Legal Entity Recognition | Legal document entity extraction | 🔗 GitHub |
| Bangladesh Geographic NER | Location entity recognition | 🔗 GitHub |
| Dataset | Description | Link |
|---|---|---|
| Squad 2.0 Style | Question Answering Squad 2.0 in Bangla | 📊 Kaggle |
| BanglaRQA | Reading comprehension dataset | 🔗 GitHub |
| SQuAD-BN | Bangla version of SQuAD | 🔗 GitHub |
| Contextual QA Dataset | Multi-context question answering | 🔗 GitHub |
| Medical QA Bangla | Healthcare question answering | 🔗 GitHub |
| Legal QA Dataset | Legal question answering | 🔗 GitHub |
| Educational QA Corpus | Academic question answering | 🔗 GitHub |
| Bangla Conversational QA | Multi-turn question answering | 🔗 GitHub |
| Dataset | Description | Link |
|---|---|---|
| Article Summarization | Articles Summarization (extractive & abstractive) | 📊 Kaggle |
| BANSData | Dataset for Bengali Abstractive News Summarization | 📊 Kaggle |
| 3 Human Evaluated | Articles with 3 human evaluated summaries | 🔗 BNLPC |
| BenSum | Bangla news summarization | 🔗 GitHub |
| BanglaNewsSummarization | Extended news corpus | 🔗 GitHub |
| BUSUM | Multi-document summarization | 🔗 GitHub |
| Academic Paper Summarization | Research paper summarization | 🔗 GitHub |
| Book Chapter Summarization | Literature summarization | 🔗 GitHub |
| Dataset | Description | Link |
|---|---|---|
| 50k Fake News | 50k Bangla fake news dataset | 📊 Kaggle |
| Dataset | Description | Link |
|---|---|---|
| Ekush | Bangla Handwritten Characters | 🔗 Website |
| Bayanno | Multi-purpose handwritten dataset | 📊 Mendeley |
| BN-HTRd | Document Level Offline Bangla HTR (108k words) | 📊 Mendeley |
| Bongabdo | Bangla handwritten script dataset | 📄 Research Paper |
| BanglaOCR Dataset | Comprehensive OCR training data | 🔗 GitHub |
| BanglaHWR Dataset | Handwriting recognition corpus | 🔗 GitHub |
| Document Layout Analysis | Document understanding dataset | 🔗 GitHub |
| Dataset | Description | Link |
|---|---|---|
| BanglaAutoKG | Automatic knowledge graph construction | 🔗 GitHub |
| Bangladesh Agricultural KG | Agricultural data integration | 📄 IEEE Access |
| Bangla Wikipedia Knowledge Graph | Structured Wikipedia knowledge | 🔗 GitHub |
| Bangla Event Extraction | News event extraction | 🔗 GitHub |
| Social Media Event Detection | Real-time event detection | 🔗 GitHub |
| Bangla Relation Extraction | Entity relationship extraction | 🔗 GitHub |
| Knowledge Base Relations | Structured knowledge extraction | 🔗 GitHub |
| Aspect-Based Opinion Mining | Detailed opinion analysis | 🔗 GitHub |
| Bangla Semantic Textual Similarity | Sentence similarity dataset | 🔗 GitHub |
| Concept Mapping Dataset | Conceptual relationship mapping | 🔗 GitHub |
| Bangla WordNet | Lexical semantic network | 🔗 GitHub |
| Dataset | Description | Size | Link |
|---|---|---|---|
| BanglaLM | Large language modeling corpus | 27.5 GB | 🔗 GitHub |
| Indic Corpus | Multi-lingual Indic corpus | 6.5 GB Bangla | 🔗 AI4Bharat |
| CC-100 Bangla | CommonCrawl Bangla subset | 8.3 GB | 🔗 StatMT |
| OSCAR Bangla | Web-crawled multilingual corpus | 12 GB | 🔗 OSCAR |
| Bangla Poetry Corpus | Classical and modern poetry | 25,000+ poems | 🔗 GitHub |
| Literary Text Collection | Bangla literature corpus | 10,000+ books | 🔗 GitHub |
| Academic Text Corpus | Scholarly text collection | 50,000+ papers | 🔗 GitHub |
| Bangla Morphological Analyzer | Morphological analysis dataset | 100,000+ word-morpheme pairs | 🔗 GitHub |
| Phonetic Transcription Corpus | IPA transcription dataset | 50,000+ word-pronunciation pairs | 🔗 GitHub |
| Dataset | Description | Size | Link |
|---|---|---|---|
| Bangla Image Captioning | Image description generation | 50,000+ image-caption pairs | 🔗 GitHub |
| Visual Question Answering Bangla | Visual reasoning dataset | 25,000+ image-question-answer | 🔗 GitHub |
| Bangla Video Captioning | Video description dataset | 5,000+ video-caption pairs | 🔗 GitHub |
| Sign Language Recognition | Bangla sign language dataset | 10,000+ sign videos | 🔗 GitHub |
| Music-Text Alignment | Song lyrics alignment | 2,000+ song-lyric pairs | 🔗 GitHub |
| Dataset | Description | Link |
|---|---|---|
| Numbers with Words | Bengali numbers with words | 📊 Kaggle |
| Image to Text | Bangla Natural Language Image to Text (BnLiT) | 📊 Kaggle |
Coming soon...
Documentation for usage and contribution guidelines coming soon...
- For Pre-trained Models: Visit HuggingFace model hub links above
- For Tools: Install Python libraries like BNLP or BNLTK
- For Datasets: Follow the individual dataset links and instructions
- For Research: Check out the latest papers and benchmarks
- 📝 Submit new datasets through pull requests
- 🐛 Report issues or broken links
- 💡 Suggest improvements to the documentation
- 🔬 Share your research findings
⭐ If you find this repository helpful, please give it a star! ⭐
🤝 Contributions are welcome! Feel free to submit issues and pull requests.
📬 Questions? Open an issue or contact the maintainers.
🌟 Special thanks to all the researchers and developers who contributed to Bangla NLP!
If this repository has been helpful to you, consider supporting the project:
Your support helps maintain and improve this resource for the Bangla NLP community! 💚
