Description
Staff Software Engineer (Data Platform)
Join our dynamic team at the forefront of cutting-edge technology as we seek a seasoned Staff Data Engineer. Embark on a journey where your deep-rooted expertise in distributed systems, data architectures, and large-scale processing becomes the cornerstone of building high-performance data platforms. This pivotal role demands proficiency in designing and scaling compute and I/O-intensive data systems, ensuring reliability, efficiency, and cost optimization across the data lifecycle.
Responsibilities:
● Design and build scalable data platform components for batch and real-time data processing.
● Architect, develop, and operationalize large-scale data systems across ingestion, transformation, and serving layers.
● Build and manage robust data pipelines ensuring high reliability, scalability, and cost efficiency.
● Develop reusable frameworks and tooling to accelerate productivity for data engineers and data scientists.
● Leverage expertise in Python, Airflow, SQL, and cloud platforms to build production-grade data solutions.
● Optimize query performance and data models using strong understanding of columnar OLAP systems such as ClickHouse, Doris, and StarRocks.
● Implement streaming and near real-time data processing systems.
● Translate complex business requirements into scalable and efficient data platform solutions.
● Work collaboratively with cross-functional teams and provide technical leadership and mentorship.
● Drive architectural decisions by evaluating tradeoffs and selecting the right tools for the problem.
Requirements:
● Bachelor's Degree in Computer Science, Information Technology, or a similar discipline.
● 8+ years of professional experience in data engineering, backend systems, or distributed systems.
● Proven experience building scalable data platforms and large-scale data systems.
● Strong experience with ETL pipelines, data integration, and workflow orchestration systems such as Airflow or Temporal.
● Hands-on experience in Python and SQL with strong understanding of data warehouse concepts.
● Experience working with distributed OLTP/OLAP databases such as ClickHouse, PostgreSQL, Cassandra, or Elasticsearch.
● Knowledge of messaging and streaming systems such as Kafka.
● Experience with cloud platforms (AWS/GCP) and big data tools such as Spark.
● Strong understanding of columnar storage systems and query optimization techniques.
● Solid understanding of distributed systems fundamentals and associated tradeoffs.
● Experience working with containers and orchestration tools such as Docker and Kubernetes.
● Strong Linux fundamentals and system-level debugging skills.
● Familiarity with modern data architectures such as Lakehouse (Iceberg, Hudi, Delta) is a plus.
Similar jobs
Join our dynamic team at the forefront of cutting-edge technology as we seek a seasoned Senior Data Engineer. Embark on a journey where your deep-rooted expertise in computer science fundamentals, alongside an intricate…
Est. 95,000 EUR
At Sonatus, we’re driving the transformation to AI-enabled software-defined vehicles. Traditional automotive software methods can’t keep pace with consumer expectations shaped by the mobile industry—where features evolve…
About Workato Workato delivers enterprise infrastructure for the agentic era, redefining iPaaS and helping enterprises unify data, applications, processes, and AI into a single, governed platform. A leader in Enterprise…
Join our dynamic team at the forefront of cutting-edge technology as we seek a seasoned Staff/Lead Backend Engineer. Embark on a journey where your deep-rooted expertise in computer science fundamentals, alongside an int…
ROLE OVERVIEW We are looking for a seasoned Data Architect with deep expertise in enterprise data warehousing, ETL/ELT pipeline development, and business intelligence. You will be responsible for the end-to-end design an…
About us: Working at Tech Holding isn't just a job, it's an opportunity to be a part of something bigger. We are a full-service consulting firm that was founded on the premise of delivering predictable outcomes and high-…
Welcome to the future of cloud networking and security! Cato Networks is the first company to converge enterprise networking and security into one centralized and global service that is delivered by cloud. It is led by n…
Est. 120,000 PLN
Xebia is a global AI-first, digital transformation, and engineering partner. With over 25 years of experience and a team of 5,000 professionals across 16 countries, we help organizations design and build scalable product…
Who we are DigiCert is a global leader in intelligent trust. We protect the digital world by ensuring the security, privacy, and authenticity of every interaction. Our AI-powered DigiCert ONE platform unifies PKI, DNS, a…
Est. 90,000 GBP
Senior Data Engineer Who we are Artefact is a new generation of data service provider, specialising in data consulting and data-driven digital marketing, dedicated to transforming data into business impact across the ent…
Est. 84,000 PLN
Xebia is a global AI-first, digital transformation, and engineering partner. With over 25 years of experience and a team of 5,000 professionals across 16 countries, we help organizations design and build scalable product…
Join phData, a remote-first data and AI consultancy company with employees across the United States, Latin America, and India. We partner with industry leaders, including Snowflake, AWS, Anthropic, Glean, and dbt, to sol…
This position will be part of a growing team working towards building world class large scale Big Data architectures. This individual should have a sound understanding of programming principles, experience in programming…
Senior Data Engineer Powering the future requires serious data infrastructure. We’re looking for a Senior Data Engineer to design, scale, and champion modern data solutions for us. About Sand Sand Technologies is a globa…
Orion Innovation is a premier, award-winning, global business and technology services firm. Orion delivers game-changing business transformation and product development rooted in digital strategy, experience design, and…
Please complete the attached Internal Transfer Request Form and submit. Please make sure to apply with your Coupang e-mail address. Role Overview The Customer-Centric Planning team is looking for a Senior Data Analyst wi…
Est. 175,000 USD
Vectra® is the leader in AI-driven threat detection and response for hybrid and multi-cloud enterprises. The Vectra AI Platform delivers integrated signal across public cloud, SaaS, identity, and data center networks in…
Join phData, a remote-first data and AI consultancy company with employees across the United States, Latin America, and India. We partner with industry leaders, including Snowflake, AWS, Anthropic, Glean, and dbt, to sol…
Est. 25,000 USD
The Job in short The Data Enablement team is here to enable every team in the organisation with their data needs. Our job starts the moment that data enters our platform and ends when it reaches whoever needs it. We are…
Sigmoid empowers enterprises to make smarter, data-driven decisions by blending advanced data engineering with AI consulting. We collaborate with some of the world’s leading data-rich organizations across sectors such as…
As a Senior Staff Backend Engineer - Cloud Infrastructure, this role will focus on Cloud Platform Infrastructure, Services and processes at its best to enhance Cloud infrastructure reliability & system efficiencies,…
Who we are DigiCert is a global leader in intelligent trust. We protect the digital world by ensuring the security, privacy, and authenticity of every interaction. Our AI-powered DigiCert ONE platform unifies PKI, DNS, a…
Est. 140,000 USD
Who are we?RZR Global is an AI-driven company specializing in mobile advertising solutions designed to fuel revenue growth. We leverage AI to discover audiences in a privacy-first environment through trillions of context…
Est. 120,000 PLN
Xebia is a global AI-first, digital transformation, and engineering partner. With over 25 years of experience and a team of 5,000 professionals across 16 countries, we help organizations design and build scalable product…
Est. 237,500 USD
Backed by leading Silicon Valley investors, Peregrine helps public safety organizations, state and local and governments, federal agencies, and private-sector institutions address society’s challenges with unprecedented…
Est. 237,500 USD
Backed by leading Silicon Valley investors, Peregrine helps public safety organizations, state and local and governments, federal agencies, and private-sector institutions address society’s challenges with unprecedented…
Est. 237,500 USD
Backed by leading Silicon Valley investors, Peregrine helps public safety organizations, state and local and governments, federal agencies, and private-sector institutions address society’s challenges with unprecedented…
Senior Data (Platform) Engineer Location: Hyderabad | Department: Technology, Data About the Role Are you passionate about building reliable, scalable data platforms that make analytics and AI development easier? As a Se…
Est. 650,000 DKK
Who We Are VML is a leading creative company that combines brand experience, customer experience, and commerce, creating connected brands to drive growth. VML is celebrated for its innovative and award-winning human-firs…
Est. 80,000 GBP
Spaulding Ridge is an advisory and IT implementation firm. We help global organizations get financial clarity into the complex, daily sales, and operational decisions that impact profitable revenue generations, efficient…