Description
Job Title: Data Engineer - Python and Azure Data Bricks
About Us
“Capco, a Wipro company, is a global technology and management consulting firm. Awarded with Consultancy of the year in the British Bank Award and has been ranked Top 100 Best Companies for Women in India 2022 by Avtar & Seramount. With our presence across 32 cities across globe, we support 100+ clients across banking, financial and Energy sectors. We are recognized for our deep transformation execution and delivery.
WHY JOIN CAPCO?
You will work on engaging projects with the largest international and local banks, insurance companies, payment service providers and other key players in the industry. The projects that will transform the financial services industry.
MAKE AN IMPACT
Innovative thinking, delivery excellence and thought leadership to help our clients transform their business. Together with our clients and industry partners, we deliver disruptive work that is changing energy and financial services.
#BEYOURSELFATWORK
Capco has a tolerant, open culture that values diversity, inclusivity, and creativity.
CAREER ADVANCEMENT
With no forced hierarchy at Capco, everyone has the opportunity to grow as we grow, taking their career into their own hands.
DIVERSITY & INCLUSION
We believe that diversity of people and perspective gives us a competitive advantage.
Job Description
ole Overview
We are looking for a highly skilled Data Engineer with strong expertise in Python and Azure Databricks to design, build, and optimize scalable data pipelines and modern data platforms. The ideal candidate will have hands-on experience in cloud-based data engineering, big data processing, and data transformation using Azure services and Databricks.
Key Responsibilities
- Design, develop, and maintain scalable data pipelines using Python and Azure Databricks.
- Build and optimize ETL/ELT workflows for ingesting, transforming, and processing large datasets.
- Develop data solutions using Apache Spark within the Databricks environment.
- Integrate data from multiple structured and unstructured sources.
- Ensure data quality, reliability, security, and governance across data platforms.
- Collaborate with Data Scientists, Analysts, Architects, and Business stakeholders to deliver data solutions.
- Implement performance tuning and optimization for large-scale data processing workloads.
- Develop and maintain data models, metadata management, and documentation.
- Monitor and troubleshoot production data pipelines and resolve data-related issues.
- Support CI/CD implementation and deployment automation for data engineering solutions.
Required Skills & Qualifications
- Bachelor's or Master's degree in Computer Science, Engineering, Information Technology, or a related field.
- 4+ years of experience in Data Engineering.
- Strong hands-on programming experience in Python.
- Expertise in:
- Azure Databricks
- Apache Spark (PySpark)
- SQL
- Delta Lake
- Experience with Azure data services:
- Azure Data Factory (ADF)
- Azure Data Lake Storage (ADLS)
- Azure Synapse Analytics
- Azure Key Vault
- Strong understanding of data warehousing concepts and dimensional modeling.
- Experience building and maintaining batch and real-time data pipelines.
- Knowledge of Git, CI/CD pipelines, and DevOps practices.
- Experience working in Agile development environments.
If you are keen to join us, you will be part of an organization that values your contributions, recognizes your potential, and provides ample opportunities for growth. For more information, visit www.capco.com. Follow us on Twitter, Facebook, LinkedIn, and YouTube.
Similar jobs
About Us “Capco, a Wipro company, is a global technology and management consulting firm. Awarded with Consultancy of the year in the British Bank Award and has been ranked Top 100 Best Companies for Women in India 2022 b…
Est. 80,000 GBP
Senior Azure Data Engineer (Databricks) Location: London (Hybrid) | Practice Area: Technology & Engineering | Type: Permanent Engineer advanced data solutions at scale in a world-class consulting environment. The Rol…
Est. 120,000 GBP
Principal Azure Data Engineer (Databricks) Location: London (Hybrid) | Practice Area: Technology & Engineering | Type: Permanent Lead the architecture of data platforms driving the future of financial services. The R…
Est. 85,000 USD
Business Applications Specialist (Brazilian Portuguese Speaking) Location: Orlando, FL Practice Area: Corporate Services Department: Global Internal Applications Type: Permanent Support the systems that power our global…
Est. 650,000 DKK
Who We Are VML is a leading creative company that combines brand experience, customer experience, and commerce, creating connected brands to drive growth. VML is celebrated for its innovative and award-winning human-firs…
About Us “Capco, a Wipro company, is a global technology and management consulting firm. Awarded with Consultancy of the year in the British Bank Award and has been ranked Top 100 Best Companies for Women in India 2022 b…
Est. 90,000 GBP
Senior Data Engineer – AWS Location: London (Hybrid) | Practice Area: Technology & Engineering | Type: Permanent Engineer future-ready data platforms that power financial transformation. The Role We’re looking for a…
About Us “Capco, a Wipro company, is a global technology and management consulting firm. Awarded with Consultancy of the year in the British Bank Award and has been ranked Top 100 Best Companies for Women in India 2022 b…
Est. 120,000 GBP
Principal Data Engineer – AWS Location: London (Hybrid) | Practice Area: Technology & Engineering | Type: Permanent Engineer future-ready data platforms that power financial transformation. The Role We’re looking for…
About Agoda At Agoda, we bridge the world through travel. Our story began in 2005, when two lifelong friends and entrepreneurs, driven by their passion for travel, launched Agoda to make it easier for everyone to explore…
About Agoda At Agoda, we bridge the world through travel. Our story began in 2005, when two lifelong friends and entrepreneurs, driven by their passion for travel, launched Agoda to make it easier for everyone to explore…
Est. 120,000 USD
Key Skills: Java, Spring Boot, .NET Core, .NET Framework, GraphQL, Docker, Kubernetes, AWS, EKS, GitHub Actions, Azure DevOps, SonarQube, DataDog, Grafana, Prometheus, Microservices, REST APIs Experience: 8+ years of exp…
Est. 175,000 USD
Company Introduction We exist to wow our customers. We know we’re doing the right thing when we hear our customers say, “How did I ever live without Coupang?” Born out of an obsession to make shopping, eating, and living…
About WPP Media WPP is the trusted growth partner for the world’s leading brands. With exceptional talent, trusted data and intelligence, and world-class partnerships – all united by our pioneering agentic marketing plat…
Job Title: Business Intelligence Key Skills: SQL, Power BI, Tableau, or similar. Experience: 6+ years Location Mexico Mode: Remote We at Coforge are hiring Position (22739) with the following skill set. The Analytics Eng…
Est. 80,000 GBP
Senior Backend Engineer – Java Location: London (Hybrid) | Practice Area: Technology & Engineering | Type: Permanent Power the platforms behind modern finance. Shape cloud-native solutions with Java at their core. Th…
Est. 141,000 USD
Cloud Architect Location: Houston, TX This is a 4 month assignment with potential to extend Houston, local OR be able to travel to Houston, TX to be at customer site every other week for a 4-5days/week Company Overview R…
Est. 120,000 GBP
Principal Software Developer in Test (SDET) Location: London (Hybrid) | Practice Area: Technology & Engineering | Type: Permanent Drive test automation innovation and engineering excellence in financial technology Th…
Capco, a Wipro company, is a global technology and management consultancy specializing in driving digital transformation in the financial services industry. With a growing client portfolio comprising of over 100 global o…
Est. 80,000 GBP
Data Management Consultant (Senior Consultant/Principal Consultant) Location: London (Hybrid) | Practice Area: Data & Analytics | Type: Permanent Help financial institutions design intelligent, trusted and future-rea…
ABOUT CAPCO Capco is a global technology and management consultancy dedicated to the financial services industry. We combine innovative thinking with unrivalled industry knowledge, to offer our clients consulting experti…
Why Choose Bottomline? Are you ready to transform the way businesses pay and get paid? Bottomline is a global leader in business payments and cash management, with over 35 years of experience and moving more than $16 tri…
Est. 124,000 USD
Anduril Industries is a defense technology company with a mission to transform U.S. and allied military capabilities with advanced technology. By bringing the expertise, technology, and business model of the 21st century…
Est. 124,000 USD
Senior OpenLink Developer Location: Houston, Texas, US | Practice Area: Technology & Engineering | Type: Permanent Drive complex OpenLink transformations and help energy clients modernize critical trading and risk pl…
Est. 124,000 USD
Anduril Industries is a defense technology company with a mission to transform U.S. and allied military capabilities with advanced technology. By bringing the expertise, technology, and business model of the 21st century…
Welcome to the future of cloud networking and security! Cato Networks is the first company to converge enterprise networking and security into one centralized and global service that is delivered by cloud. It is led by n…
Est. 80,000 GBP
Who We Are VML is a leading creative company that combines brand experience, customer experience, and commerce, creating connected brands to drive growth. VML is celebrated for its innovative and award-winning human-firs…
Est. 85,000 GBP
Senior Software Development Engineer in Test (SDET) Location: London (Hybrid) | Practice Area: Technology & Engineering | Type: Permanent Advance your testing career while shaping digital finance with automation The…
Why Choose Bottomline? Are you ready to transform the way businesses pay and get paid? Bottomline is a global leader in business payments and cash management, with over 35 years of experience and moving more than $16 tri…
Job Title: Full stack developer - Bangalore and Gurgaon About Us “Capco, a Wipro company, is a global technology and management consulting firm. Awarded with Consultancy of the year in the British Bank Award and has been…