Data Pipelines Engineering Manager – Python
Aimleap
India (Remote)
April 30, 2026
AIMLEAP is Hiring:
Data Pipelines Engineering Manager – Python
Experience: 7 to 12 Years
Location: Remote (Work from Home) / Bangalore / India
Mode of Engagement: Full-time
No of Positions: 2
Educational Qualification: B.E / B.Tech / M.Tech / MCA / Computer Science / IT
Industry: IT / Data / AI / LegalTech / Enterprise Solutions
Notice Period: Immediate
Job Description:
What We Are Looking For
– 7–12 years of experience in managing end-to-end data systems, including data collection, API-driven data delivery, and AI model integrations.
– Proven experience using large language models to automate and scale web data extraction, including building custom AI scraping pipelines for structured data collection at scale
– Proven expertise in Python-based data engineering, data scraping, and pipeline orchestration for large-scale analytics and automation projects.
– Strong background in building scalable APIs, microservices, and data pipelines using Python, Docker, and Kubernetes.
– Experience in customizing open-source language models (LLMs) and integrating them into enterprise solutions.
– Proficiency in AWS cloud services, PostgreSQL/MySQL, CI/CD, and DevOps tools for continuous delivery and monitoring.
– Ability to lead distributed engineering teams, manage delivery timelines, and drive innovation through automation and AI-driven workflows.
Responsibilities:
– Lead and mentor engineering teams responsible for designing, implementing, and maintaining data ingestion, transformation, and delivery pipelines.
– Architect data systems combine scraping, big data processing, and AI-based enrichment to support analytics and product intelligence.
– Oversee API architecture and system scalability, ensuring high performance and secure data flow across services.
– Manage public data scraping and data integration projects that involve large, unstructured datasets and complex relational mappings.
– Drive AI model customization using open-source frameworks (LangChain, LlamaIndex, or custom LLM integrations) to improve automation and search intelligence.
– Collaborate with product managers, AI teams, and data analysts to translate requirements into actionable data engineering tasks.
– Implement and monitor data quality, governance, and compliance standards throughout the data lifecycle.
– Optimize deployment processes using Docker, Kubernetes, AWS Lambda, and CI/CD pipelines.
– Conduct regular technical reviews, mentor developers, and establish best practices for Python, cloud deployment, and automation.
Qualifications:
– Bachelor’s or master’s degree in computer science, Engineering, or related field.
– Hands-on expertise in Python, Django, FastAPI, SQL, and API-driven system design.
– Strong understanding of data pipelines, scraping frameworks (BeautifulSoup, Scrapy), and distributed data systems.
– Experience with AI/LLM integration, automation frameworks, and data-driven architecture design.
– Proficiency in AWS cloud ecosystem, Docker, Kubernetes, and Git-based CI/CD workflows.
– Demonstrated leadership in managing multi-member engineering teams and delivering enterprise-grade products.
– Excellent analytical, communication, and problem-solving abilities.
– Familiarity with Agile methodologies, Jira/ClickUp, and project management best practices.
About Us:
AIMLEAP is an ISO 9001:2015 and ISO/IEC 27001:2013 certified global technology consulting and service provider offering Digital IT, AI-augmented Data Solutions, Automation, and Research & Analytics Services.
AIMLEAP has been recognized as ‘The Great Place to Work®’. With focus on AI and automation-first approach, our services include end-to-end IT application management, Mobile App Development, Data Management, Data Mining Services, Web Data Scraping, Self-serving BI reporting solutions, Digital Marketing, and Analytics solutions.
We started in 2012 and successfully delivered projects in IT & digital transformation, automation driven data solutions, and digital marketing for more than 750 fast-growing companies in the USA, Europe, New Zealand, Australia, Canada; and more.
– An ISO 9001:2015 and ISO/IEC 27001:2013 certified
– Served 750+ customers
– 11+ Years of industry experience
– 98% Client Retention
– Great Place to Work® Certified
– Global Delivery Centers in the USA, Canada, India & Australia.