Lead Python Developer
iCapital Network · IN - Jaipur
About this role
## iCapital — Lead Python Developer ### Responsibilities - Design and implement advanced scraping solutions using **Python**, **Playwright**, and **AWS** services. - Automate interactions with **JavaScript-heavy** and **authentication-secured** websites, including **MFA**, **CAPTCHAs**, and session/token-based login flows. - Architect scraping pipelines using **serverless AWS components** such as **Lambda**, **Step Functions**, **S3**, **CloudWatch**, and **Secrets Manager**. - Build systems that scale for high-volume data extraction with **fault tolerance**, **retries**, and **intelligent logging**. - Integrate and manage complex workflows across multiple **portals**, **APIs**, and **data sources**. - Contribute to architectural decisions, tooling, and best practices. - Own **code reviews** and **deployments**. ### Qualifications - Bachelor’s or master’s degree in computer science/software engineering (or equivalent professional experience). - **10+ years** of Python development experience, with strong focus on **automation** and **data extraction**. - Proven web scraping expertise using **Playwright**, **Selenium**, **Scrapy**, **BeautifulSoup**, and **requests**. - Experience handling multi-step authentication flows, including **MFA**, **CAPTCHA solving**, and **session/cookie management**. - Proficiency deploying and managing scraping workloads in **AWS** (notably **Lambda**, **S3**, **IAM**, **CloudWatch**, **Secrets Manager**). - Experience with **asynchronous programming**, **headless browsers**, and **JavaScript-rendered** content. - Strong understanding of web protocols (**HTTP/HTTPS**, cookies, headers) and ability to reverse-engineer network calls/auth mechanisms. - Comfortable working with **APIs**, **JSON/XML**, and data transformation. - Experience with **CI/CD**, **Docker**, and **infrastructure as code** (e.g., **CloudFormation**, **Terraform**). - Familiarity with orchestration tools such as **Apache Airflow** or **AWS Step Functions**. - Experience scraping from secure/enterprise portals (e.g., **financial**, **healthcare**, **legal**). - Data engineering/ETL background is a plus. - Exposure to Python testing frameworks; ability to write unit/integration tests. *Note: The original posting text appears truncated after the office-work schedule line.*
Listing freshness
CronJobs last confirmed this listing 6h ago. If its source stops confirming the opening for seven days, this page is removed from active inventory.