AI Scraping & LLM Engineering Manager – Python
Aimleap
India (Remote)
May 22, 2026
AIMLEAP is Hiring:
AI Scraping & LLM Engineering Manager – Python
Experience: 9 to 15 Years
Location: Remote (Work from Home) / Bangalore / India
Mode of Engagement: Full-time
No of Positions: 2
Educational Qualification: B.E / B.Tech / M.Tech / MCA / Computer Science / IT
Industry: AI / Data Engineering / Enterprise Automation / SaaS / LegalTech
Notice Period: Immediate / 15 Days Preferred
Job Description:
– 9–15 years of experience in building large-scale web scraping, browser automation, and data extraction systems using Python.
– Proven expertise in Scrapy, Selenium, Playwright, Puppeteer, and modern browser automation frameworks for enterprise-grade scraping projects.
– Strong experience in building AI-powered scraping workflows using LLMs for intelligent extraction, data normalization, classification, and automation.
– Experience in designing scalable distributed scraping pipelines with proxy rotation, anti-bot handling, CAPTCHA solving, and high-concurrency crawling systems.
– Proven ability to lead engineering teams, manage delivery timelines, conduct code reviews, and drive architecture decisions for AI/data platforms.
Responsibilities:
– Lead the architecture and development of AI-driven scraping and browser automation platforms for structured and unstructured web data extraction.
– Build intelligent scraping pipelines leveraging LLMs, LangChain, LlamaIndex, OpenAI, Claude, or open-source AI models for automated extraction and enrichment.
– Design and manage scalable scraping infrastructure using Scrapy, Selenium, Playwright, Docker, Kubernetes, queues, and cloud-native deployment systems.
– Collaborate with AI, product, and engineering teams to create next-generation intelligent data acquisition and automation solutions.
– Mentor and manage scraping/data engineering teams while establishing coding standards, best practices, monitoring, and scalable engineering processes.
– Optimize system performance, reliability, scalability, and deployment workflows using AWS/GCP, CI/CD pipelines, databases, and distributed processing systems.
Qualifications:
– Bachelor’s or Master’s degree in Computer Science, Engineering, IT, or related field.
– Strong hands-on expertise in Python, Scrapy, Selenium, Playwright, FastAPI/Django, APIs, and scalable scraping architectures.
– Experience working with LLMs, AI automation frameworks, vector databases, semantic search, embeddings, and intelligent extraction systems.
– Proficiency in Docker, Kubernetes, AWS/GCP, PostgreSQL/MySQL/MongoDB, Redis, CI/CD, and cloud infrastructure management.
– Prior experience leading engineering teams and delivering enterprise-scale scraping, AI automation, or data intelligence products.
– Excellent analytical, communication, leadership, and problem-solving skills with experience working in Agile development environments.
About Us:
AIMLEAP is an ISO 9001:2015 and ISO/IEC 27001:2013 certified global technology consulting and service provider offering Digital IT, AI-augmented Data Solutions, Automation, and Research & Analytics Services.
AIMLEAP has been recognized as ‘The Great Place to Work®’. With focus on AI and automation-first approach, our services include end-to-end IT application management, Mobile App Development, Data Management, Data Mining Services, Web Data Scraping, Self-serving BI reporting solutions, Digital Marketing, and Analytics solutions.
We started in 2012 and successfully delivered projects in IT & digital transformation, automation driven data solutions, and digital marketing for more than 750 fast-growing companies in the USA, Europe, New Zealand, Australia, Canada; and more.
– An ISO 9001:2015 and ISO/IEC 27001:2013 certified
– Served 750+ customers
– 11+ Years of industry experience
– 98% Client Retention
– Great Place to Work® Certified
– Global Delivery Centers in the USA, Canada, India & Australia.