Description
We are looking for a Senior Data Engineer to build and evolve the data platform powering our global workforce management ecosystem. You will design, implement, and maintain scalable data pipelines that consolidate data from multiple operational systems, transform it into trusted analytical datasets, and make it available for reporting, product analytics, and business intelligence.
You should be comfortable working with modern cloud-native data architectures on AWS, building reliable ETL/ELT pipelines, and designing data models optimized for analytical workloads. This role requires a strong engineering mindset, balancing performance, scalability, data quality, and operational excellence while collaborating closely with software engineers, product teams, analysts, and data scientists.
What You Will Own
- Design, build, and maintain scalable batch and streaming data pipelines using AWS-native services and distributed processing frameworks
- Develop ETL/ELT workflows to ingest, consolidate, sanitize, enrich, and transform data from multiple internal and external systems
- Build and optimize AWS Data Lake solutions using Amazon S3, AWS Glue, Amazon Redshift, and Amazon Kinesis Firehose
- Design and implement distributed data processing jobs using Apache Spark, AWS Glue, Databricks, or equivalent technologies
- Develop orchestration workflows using Apache Airflow (MWAA), AWS Step Functions, or similar workflow orchestration platforms
- Design analytical data models including star schemas, snowflake schemas, dimensional models, and optimized reporting datasets
- Optimize Redshift performance through distribution strategies, sort keys, partitioning, workload tuning, and query optimization
- Build resilient pipelines supporting retries, idempotency, checkpointing, incremental processing, and partial failure recovery
- Implement automated data quality validation, schema evolution, lineage tracking, and governance controls
- Develop infrastructure and deployment automation using Infrastructure as Code and CI/CD pipelines
- Monitor, troubleshoot, and continuously improve the reliability, scalability, and performance of the data platform
- Collaborate with analysts, software engineers, data scientists, and product managers to translate business requirements into scalable data solutions
- Participate in architecture discussions and contribute technical documentation, standards, and best practices
What We Are Looking For
- 5+ years of professional experience building production data pipelines and cloud-based data platforms
- Strong experience with AWS data services including Amazon Redshift, AWS Glue, Amazon S3, and Amazon Kinesis Firehose
- Strong Python programming skills for ETL development, automation, event processing, and scripting
- Advanced SQL expertise including query optimization, window functions, analytical queries, versioned migrations, rollback strategies, and warehouse tuning
- Experience designing scalable ETL/ELT pipelines for both batch and streaming workloads
- Experience with distributed compute and storage using Apache Spark, AWS Glue, Databricks, or similar distributed processing frameworks
- Strong understanding of data warehousing concepts including dimensional modeling, star schemas, snowflake schemas, partitioning strategies, and analytical data structures
- Experience designing end-to-end data architectures including ingestion, transformation, orchestration, and consumption layers
- Experience implementing workflow orchestration using Apache Airflow (MWAA), AWS Step Functions, or equivalent orchestration tools
- Understanding of data governance, metadata management, security best practices, IAM, encryption, and regulatory compliance considerations
- Experience with Git-based collaborative development workflows, CI/CD pipelines, Infrastructure as Code, deployment approvals, versioned migrations, and safe rollback strategies
- Experience monitoring and maintaining production data infrastructure, ensuring high availability, observability, data quality, and operational reliability
- Strong communication skills with the ability to explain technical concepts to business stakeholders and collaborate effectively across engineering, analytics, and product teams
Nice to Have
- Experience with Apache Iceberg, Delta Lake, Apache Hudi, or modern open table formats
- Experience with dbt or SQL-based transformation frameworks
- Familiarity with Kafka, Amazon MSK, or other streaming platforms
- Experience with Lakehouse architectures and modern analytical data platforms
- Knowledge of Terraform or AWS CloudFormation
- Experience with containerized data workloads using Docker and ECS/EKS
- Experience implementing DataOps practices and automated testing for data pipelines
- Familiarity with BI platforms such as Tableau, Power BI, Looker, or QuickSight
- Experience implementing data catalogs, lineage, and governance solutions
- Exposure to machine learning feature pipelines or data science infrastructure
Tech Stack
Layer
Technology
Programming
Python, SQL, PySpark
Data Processing
Apache Spark, AWS Glue, Databricks
Data Storage
Amazon S3, Amazon Redshift, Parquet
Streaming
Amazon Kinesis Firehose, EventBridge
Orchestration
Apache Airflow (MWAA), AWS Step Functions
Data Modeling
Star Schema, Snowflake Schema, Dimensional Modeling
Infrastructure
AWS, IAM, CloudWatch
IaC/CI
Git, GitHub Actions, Terraform, CloudFormation
Observability
CloudWatch, Datadog (or equivalent observability platforms)
Governance
Data Catalog, Metadata Management, Data Lineage
Similar jobs
Staff Software Engineer (Data Platform)Join our dynamic team at the forefront of cutting-edge technology as we seek a seasoned Staff Data Engineer. Embark on a journey where your deep-rooted expertise in distributed syst…
Est. 140,000 USD
Are you driven by the challenge of turning complex, fragmented data into scalable solutions that people can trust and use? Do you enjoy working across architecture, engineering, analytics, and business teams to solve pro…
Est. 25,000 USD
The Job in short The Data Enablement team is here to enable every team in the organisation with their data needs. Our job starts the moment that data enters our platform and ends when it reaches whoever needs it. We are…
About Workato Workato delivers enterprise infrastructure for the agentic era, redefining iPaaS and helping enterprises unify data, applications, processes, and AI into a single, governed platform. A leader in Enterprise…
About Us Are you ready to build the future of the supply chain? At Gather AI, we're not just creating software; we're pioneering a new era of warehouse intelligence. We've developed a groundbreaking, vision-powered platf…
ROLE OVERVIEW We are looking for a seasoned Data Architect with deep expertise in enterprise data warehousing, ETL/ELT pipeline development, and business intelligence. You will be responsible for the end-to-end design an…
Est. 120,000 PLN
Xebia is a global AI-first, digital transformation, and engineering partner. With over 25 years of experience and a team of 5,000 professionals across 16 countries, we help organizations design and build scalable product…
Est. 165,000 USD
(ID: 2026-1572) Axle is a bioscience and information technology company that offers advancements in translational research, biomedical informatics, and data science applications to research centers and healthcare organiz…
Est. 84,000 PLN
Xebia is a global AI-first, digital transformation, and engineering partner. With over 25 years of experience and a team of 5,000 professionals across 16 countries, we help organizations design and build scalable product…
Who we are DigiCert is a global leader in intelligent trust. We protect the digital world by ensuring the security, privacy, and authenticity of every interaction. Our AI-powered DigiCert ONE platform unifies PKI, DNS, a…
Est. 90,000 GBP
Senior Data Engineer Who we are Artefact is a new generation of data service provider, specialising in data consulting and data-driven digital marketing, dedicated to transforming data into business impact across the ent…
Join Riverside Natural Foods Ltd., a Canadian-based, family-owned, and globally operating business, committed to leaving the world better than we found it. As a B-Corp certified, Triple-Bottom Line company, we proudly ma…
Analytical Wizards is part of the Definitive Healthcare family. We balance innovation with an open, friendly culture and the backing of a long-established parent company, known for its ethical reputation. We guide custom…
Senior Data Engineer Powering the future requires serious data infrastructure. We’re looking for a Senior Data Engineer to design, scale, and champion modern data solutions for us. About Sand Sand Technologies is a globa…
Est. 175,000 USD
Vectra® is the leader in AI-driven threat detection and response for hybrid and multi-cloud enterprises. The Vectra AI Platform delivers integrated signal across public cloud, SaaS, identity, and data center networks in…
Who we are DigiCert is a global leader in intelligent trust. We protect the digital world by ensuring the security, privacy, and authenticity of every interaction. Our AI-powered DigiCert ONE platform unifies PKI, DNS, a…
Est. 120,000 PLN
Company Background Our client is a clinician-founded medical and behavioral health organization providing integrated mental health care across the United States. Founded in 2011, the company combines psychotherapy, psych…
Est. 141,000 USD
Cloud Architect Location: Houston, TX This is a 4 month assignment with potential to extend Houston, local OR be able to travel to Houston, TX to be at customer site every other week for a 4-5days/week Company Overview R…
Join phData, a remote-first data and AI consultancy company with employees across the United States, Latin America, and India. We partner with industry leaders, including Snowflake, AWS, Anthropic, Glean, and dbt, to sol…
About us: Working at Tech Holding isn't just a job, it's an opportunity to be a part of something bigger. We are a full-service consulting firm that was founded on the premise of delivering predictable outcomes and high-…
Est. 114,000 USD
Data Engineer – Inspire11 Elevens, as we call ourselves here, are curiously smart, creative, and dedicated to the highest standard of quality. By being true to ourselves and others, we create a respectful environment tha…
Spaulding Ridge is an advisory and IT implementation firm. We help global organizations get financial clarity into the complex, daily sales, and operational decisions that impact profitable revenue generations, efficient…
Est. 140,000 USD
Spaulding Ridge is an advisory and IT implementation firm. We help global organizations get financial clarity into the complex, daily sales, and operational decisions that impact profitable revenue generations, efficient…
Est. 80,000 GBP
(ID: 2026-2740) Axle Informatics is a bioscience and information technology company advancing translational research through biomedical informatics, data science, artificial intelligence, and intelligent automation. We p…
Est. 120,000 PLN
Company Background Our client is a global leader in access solutions and smart home technologies. With over a decade of innovation in the IoT space, the company is a pioneer in connected home solutions, including garage…
Est. 120,000 USD
Join phData, a remote-first data and AI consultancy company with employees across the United States, Latin America, and India. We partner with industry leaders, including Snowflake, AWS, Anthropic, Glean, and dbt, to sol…
Est. 147,500 USD
Capital Technology Group provides expert consulting services software development, digital transformation, human-centered design, data analytics and visualization, and cybersecurity. Our multidisciplinary teams use agile…
Est. 120,000 USD
Senior Data Engineer Latin America | 100% Remote About CodeRoad CodeRoad provides end-to-end software development services, helping businesses scale with ideal infrastructure solutions. From staff augmentation to dedicat…
Est. 120,000 PLN
Xebia is a global AI-first, digital transformation, and engineering partner. With over 25 years of experience and a team of 5,000 professionals across 16 countries, we help organizations design and build scalable product…
Est. 140,000 USD
Kunai builds full-stack technology solutions for banks, credit and payment networks, infrastructure providers, and their customers. Together, we are changing the world’s relationship with financial services. At Kunai, we…