# Python Developer

**Company**: iCapital Network
**Location**: Jaipur, IN
**Work arrangement**: hybrid
**Experience**: senior
**Job type**: full-time
**Category**: Engineering
**Industry**: Technology

**Apply**: https://job-boards.greenhouse.io/icapitalnetwork/jobs/8844041002?utm_source=yubhub.co&utm_medium=jobs_feed&utm_campaign=apply
**Canonical**: https://yubhub.co/jobs/job_9f790ed8-100

## Description

We are looking for an experienced Python Developer with expertise in web scraping, automated portal interactions, and cloud-native deployment using AWS.

You will assist in architecting and building robust, secure, and scalable scraping solutions that interact with complex web applications and secured portals.

Responsibilities:

- Design and implement advanced scraping solutions using Python, Playwright, and AWS services.

- Automate interactions with JavaScript-heavy and authentication-secured websites, including handling MFA, CAPTCHAs, and session/token-based login flows.

- Architect scraping pipelines using serverless AWS components such as Lambda, Step Functions, S3, CloudWatch, and Secrets Manager.

- Build systems that scale to support high volumes of data extraction with fault tolerance, retries, and intelligent logging.

- Integrate and manage complex workflows across multiple portals, APIs, and data sources.

- Contribute to architectural decisions, tooling, and best practices.

Qualifications:

- 5+ years of experience in Python development, with a strong focus on automation and data extraction.

- Proven expertise in web scraping using tools like Playwright, Selenium, Scrapy, BeautifulSoup, and requests.

- In-depth experience handling multi-step authentication flows, including multi-factor authentication (MFA), CAPTCHA solving, and session/cookie management.

- Proficient in deploying and managing scraping workloads in AWS, particularly Lambda, S3, IAM, CloudWatch, and Secrets Manager.

- Experience with asynchronous programming, headless browsers, and JavaScript-rendered content.

- Solid understanding of web protocols (HTTP, HTTPS, cookies, headers), and ability to reverse-engineer network calls and authentication mechanisms.

- Comfortable working with APIs, JSON/XML, and data transformation.

Preferred Qualifications:

- Experience with CI/CD pipelines, Docker, and infrastructure as code (e.g., CloudFormation, Terraform).

- Familiarity with data pipeline orchestration tools such as Apache Airflow or AWS Step Functions.

- Prior experience scraping from secure or enterprise-level portals (e.g., financial, healthcare, legal).

- Background in data engineering or ETL workflows is a plus.

- Exposure to Python testing frameworks and writing unit/integration tests.

Benefits:

- Competitive salary, annual performance bonus, and equity for all full-time employees.

- Healthcare with 100% employer-paid health and dental insurance.

- Generous paid time off (PTO).

## Skills

### Required
- Python
- Playwright
- AWS
- web scraping
- automated portal interactions
- cloud-native deployment
- JavaScript
- HTML
- CSS
- APIs
- JSON/XML
- data transformation
- asynchronous programming
- headless browsers
- JavaScript-rendered content
- web protocols
- HTTP
- HTTPS
- cookies
- headers

### Nice to have
- CI/CD pipelines
- Docker
- infrastructure as code
- CloudFormation
- Terraform
- Apache Airflow
- AWS Step Functions
- data pipeline orchestration
- data engineering
- ETL workflows
- Python testing frameworks
- unit/integration tests

---

Source: [Apply at job-boards.greenhouse.io](https://job-boards.greenhouse.io/icapitalnetwork/jobs/8844041002?utm_source=yubhub.co&utm_medium=jobs_feed&utm_campaign=apply)