Feat 23 add fast linkedin option #43
Conversation
Easier to connect to streamable http mcps if you can link remotely
- Add scraper_factory.py for backend selection between linkedin-scraper and fast-linkedin-scraper - Add scraper_adapter.py with unified interface for both scrapers - Add playwright_wrapper.py for Playwright session management (used by fast scraper) - Update configuration to support scraper_type selection - Update all tools (person, company, job) to use scraper adapter - Fix type checking issues in config loaders and playwright wrapper - Maintain backward compatibility with existing linkedin-scraper backend 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com>
|
Hey, really appreciate the effort you put into this.
This would avoid the threading complexity in the scraper adapter. Thanks for pushing this forward! |
|
Cool! Yea that would def make things easier lol I had to hard workaround the sync-async. Are you essentially deprecating the original linkedin-scraper? Eg. is the primary build path going forward the fast-linkedin-scraper? Cause if so I can start focusing on that too and likely help out on a few things as I build my workflow out. Would be good to be working on the one that's more likely to be kept up. Happy to push those as well should anything be of interest. Q |
|
Yes exactly, the plan is to remove the original selenium scraper entirely as a dependency. I'll finish the async conversion today. |
Optional implementation of the fast linkedin scraper library with a configurable switch between selenium and playwright.
If interested @stickerdaniel I can go back and clean it up to make it merge worthy but figured I'd share the raw one regardless.
RE: #23
(btw - thanks for the MCP server, it took a bit to get going but it's pretty slick)