Lead Python Engineer / Developer - Vice President
Stamford, Connecticut, United States · Posted 5 months agoDescription
About the Role
iCapital is looking to hire an experienced Python Developer with expertise in web scraping, automated portal interactions, and cloud-native deployment using AWS. The ideal candidate will have hands-on experience working with Playwright for browser automation, managing multi-factor authentication (MFA) flows, and deploying scalable scraping tasks via AWS Lambda and related services. This role will assist in architecting and building robust, secure, and scalable scraping solutions that interact with complex web applications and secured portals.
Responsibilities
- Assist with the design and implementation of advanced scraping solutions using Python, Playwright, and AWS services.
- Automate interactions with JavaScript-heavy and authentication-secured websites, including handling MFA, CAPTCHAs, and session or token-based login flows.
- Architect scraping pipelines using serverless AWS components such as Lambda, Step Functions, S3, CloudWatch, and Secrets Manager.
- Build systems that scale to support high volumes of data extraction with fault tolerance, retries, and intelligent logging.
- Integrate and manage complex workflows across multiple portals, APIs, and data sources.
- Contribute to architectural decisions, tooling, and best practices.
Qualifications
- Bachelor’s or master’s degree in computer science, software engineering, or related field, or equivalent professional experience
- 10+ years of experience in Python development, with a strong focus on automation and data extraction
- Proven expertise in web scraping using tools like Playwright, Selenium, Scrapy, BeautifulSoup, and requests
- In-depth experience handling multi-step authentication flows, including multi-factor authentication (MFA), CAPTCHA solving, and session or cookie management
- Proficient in deploying and managing scraping workloads in AWS, particularly Lambda, S3, IAM, CloudWatch, and Secrets Manager
- Experience with asynchronous programming, headless browsers, and JavaScript-rendered content
- Solid understanding of web protocols (i.e. HTTP, HTTPS, cookies, headers), and the ability to reverse-engineer network calls and authentication mechanisms
- Comfortable working with APIs, JSON/XML, and data transformation
- Experience with CI or CD pipelines, Docker, and infrastructure as code (i.e. CloudFormation, Terraform)
- Familiar with data pipeline orchestration tools such as Apache Airflow or AWS Step Functions
- Prior experience scraping from secure or enterprise-level portals (i.e. financial, healthcare, legal)
- Background in data engineering or ETL workflows is a plus
- Exposure to Python testing frameworks and writing unit or integration tests
Benefits
The base salary range for this role is $150,000 to $180,000. iCapital offers a compensation package which includes salary, equity for all full-time employees, and an annual performance bonus. Employees also receive a comprehensive benefits package that includes an employer matched retirement plan, generously subsidized healthcare with 100% employer paid dental, vision, telemedicine, and virtual mental health counseling, parental leave, and unlimited paid time off (PTO).
We believe the best ideas and innovation happen when we are together. Employees in this role will work in the office Monday-Thursday, with the flexibility to work remotely on Friday.
For additional information on iCapital, please visit https://www.icapitalnetwork.com/about-us Twitter: @icapitalnetwork | LinkedIn: https://www.linkedin.com/company/icapital-network-inc | Awards Disclaimer: https://www.icapitalnetwork.com/about-us/recognition/
iCapital is proud to be an Equal Employment Opportunity and Affirmative Action employer. We do not discriminate based upon race, religion, color, national origin, gender, sexual orientation, gender identity, age, status as a protected veteran, status as an individual with a disability, or other applicable legally protected characteristics.
Similar jobs
Est. 97,500 USD
About the Role iCapital is seeking an Associate Python Developer to join the growing Core Operations Technology team. This individual will have foundational experience in Python programming and a passion for building sca…
iCapital is looking to hire an experienced Python Developer with expertise in web scraping, automated portal interactions, and cloud-native deployment using AWS. The ideal candidate will have hands-on experience working…
Est. 150,000 USD
About the Role The Backend Engineer at iCapital will play an integral role in designing and implementing systems that power the iCapital platform. We are looking for engineers who can build APIs and applications that are…
Est. 205,000 USD
About the Role The Platform Infrastructure team at iCapital plays a critical role in ensuring that both production and development environments operate smoothly, securely, and reliably. This role leverages advanced cloud…
Est. 125,000 USD
About the Role iCapital is seeking a motivated and detail-oriented Associate Business Process Analyst, UiPath Developer to join our Intelligent Automation team. The ideal candidate will combine solid business analysis sk…
Est. 117,500 USD
About the Role iCapital is looking to hire a Backend Engineer Associate or Assistant Vice President to join the Software Engineering team. This individual will play an integral role in designing and implementing systems…
Est. 125,000 USD
About the Role iCapital is seeking a motivated and detail-oriented Associate Business Process Analyst, UiPath Developer to join our Intelligent Automation team. The ideal candidate will combine solid business analysis sk…
Est. 185,000 USD
About the Role iCapital is seeking a Vice President Artificial Intelligence Engineer to lead the design, development, and delivery of production-grade AI systems that drive measurable business outcomes across the firm. T…
Est. 150,000 USD
About the Role The Backend Engineer at iCapital will play an integral role in designing and implementing systems that power the iCapital platform. We are looking for engineers who can build APIs and applications that are…
Est. 140,000 USD
About the Role iCapital is seeking a strategic, hands-on, and results-driven Vice President Program Manager to join the Product Operations team, directly supporting the Pre-Investment vertical. This role will lead the pl…
Est. 115,000 USD
About the Role The Full Stack Engineer at iCapital will work end-to-end on new features, enhancements, and defect resolution on one or more of iCapital's software platforms. You will work closely with our Product Managem…
Est. 140,000 USD
About the Role The Full Stack Engineer at iCapital will work end-to-end on new features, enhancements, and defect resolution on one or more of iCapital's software platforms. This individual will work closely with our Pro…
Est. 175,000 USD
About the Role iCapital’s Distributed Ledger Technology team is looking to hire a Senior Blockchain Backend Engineer to develop an industry-wide distributed ledger solution for the alternative investment ecosystem to fac…
Est. 132,500 USD
About the Role iCapital’s Distributed Ledger Technology team is looking to hire a Senior Blockchain Backend Engineer to develop an industry-wide distributed ledger solution for the alternative investment ecosystem to fac…
Est. 65,000 USD
About the Role iCapital is looking to hire an Analyst to join its Platform Support team. The team provides service and technical support to its rapidly growing client base of financial advisors and individual investors,…
Est. 92,500 USD
About the Role We are seeking a highly motivated and results-driven Business Development Representative (BDR) to play a key role in driving the success of our Sales team. This individual will be responsible for the criti…
Est. 80,000 EUR
About the Role iCapital is looking for a Full-Stack Engineer to join the iCapital Identity Platform team. This role will help define and deliver new features for one or more of iCapital's software platforms. You will wor…
Est. 105,000 USD
About the Role iCapital is looking for a Client Delivery Specialist, Assistant Vice President to join the Data Solutions team. iCapital is looking to hire a Client Delivery Specialist, Assistant Vice President to join th…
Est. 127,500 USD
About the Role iCapital is seeking an Associate to join the Corporate Development and Strategy team. The individual will join a highly valued team responsible for evaluating, planning, and executing organic and inorganic…
Est. 120,000 USD
About the Role iCapital is seeking an experienced cybersecurity risk professional to serve as Assistant Vice President, Third-Party Risk & Client Assurance Manager. This individual will lead a team of approximately f…
Est. 140,000 USD
About the Role iCapital is looking to hire a Client Delivery Vice President to join the Annuities Platform team. This role is responsible for leading the successful delivery of client implementations and ongoing client d…
Est. 80,000 USD
About the Role iCapital is looking for a Client Delivery Specialist Associate to join the Data Solutions team. This role will set up and maintain the financial software applications we utilize, assist clients with issues…
Est. 160,000 USD
About the Role iCapital is looking to hire a Strategic Finance professional to join the Corporate Finance team. This role will involve large scale responsibilities across financial and managerial reporting, budgeting, fo…
Est. 120,000 USD
About the Role iCapital is looking to hire a Strategic Finance professional to join the Corporate Finance team. This role will involve large scale responsibilities across financial and managerial reporting, budgeting, fo…
Est. 185,000 USD
About the Role iCapital is looking to hire a Vice President to join the Structured Investments team. This role will support the Structured Investments team, within the firm’s technology platform, by owning, developing, a…
Est. 80,000 USD
About the Role iCapital is seeking a Client Delivery Specialist Associate to join the Automation team within the Data Solution department. This role will ensure a seamless flow of financial data for iCapital’s clients an…
Est. 80,000 USD
About the Role iCapital is seeking a Client Delivery Specialist Associate to join the Automation team within the Data Solution department. This role will ensure a seamless flow of financial data for iCapital’s clients an…
Est. 80,000 USD
About the Role iCapital is looking for a Client Delivery Specialist Associate to join the Data Solutions team. This role will set up and maintain the financial software applications we utilize, assist clients with issues…
Est. 72,250 USD
About the Role iCapital is seeking an Associate to join the Digital Marketing and Strategy team, reporting to the Assistant Vice President, Digital Marketing. This role is the execution engine for a portfolio of iCapital…
Est. 185,000 USD
About the Role iCapital is looking to hire a Vice President to join the Structured Investments team. This role will support the Structured Investments team, within the firm’s technology platform, by owning, developing, a…