inhousefyi
← Back to listings

Senior, ML Engineer - VLM

Torc RoboticsRemote · Posted 1 month ago
Full-timeRemoteEst. 141,000 USD
Apply now

Description

About the Company

At Torc, we have always believed that autonomous vehicle technology will transform how we travel, move freight, and do business. A leader in autonomous driving since 2007, Torc has spent over a decade commercializing our solutions with experienced partners. Now a part of the Daimler family, we are focused solely on developing software for automated trucks to transform how the world moves freight.

Join us and catapult your career with the company that helped pioneer autonomous technology, and the first AV software company with the vision to partner directly with a truck manufacturer.

Meet The Team

Torc is marching toward its AV 3.0 strategy, where end-to-end Vision-Language-Action (VLA) models perceive, reason, and act directly from sensor data. High-quality, semantically rich training data is the single biggest lever for that strategy, and this team owns it.

Sitting within Offline Perception, this team turns petabytes of logged multi-modal fleet data (images, kinematics) into VLM/VLA-ready datasets: geometric annotations, scenario-level semantic descriptions, action- and trajectory-grounded labels, and reasoning traces that explain why a maneuver was taken. We run a continuous data flywheel — mine long-tail and failure cases, auto-label at scale, validate quality, and feed curated datasets directly into Torc’s end-to-end VLM/VLA model development. You will own the dataset layer that those models learn from.

What You’ll Do

  • Own the offline dataset pipeline — design, implement, test, and deploy Cloud-based pipelines that convert logged multi-sensor data into VLM/VLA training datasets, spanning geometric labels (3D/2D detection, tracking, segmentation, depth) through semantic, scenario-level, and action/trajectory-grounded annotations.
  • Build VLM-assisted auto-labeling — develop open-vocabulary detection, dense captioning, semantic enrichment, and scene/scenario description generation that move beyond closed-set bounding boxes, using foundation models to scale annotation and cut manual labeling cost.
  • Generate reasoning-grounded labels — produce language-grounded reasoning and chain-of-causation style annotations, temporally aligned to ego-motion and trajectories, to support VLA training and explainable driving behavior.
  • Mine and curate the long tail — surface rare, difficult, and high-uncertainty scenarios, and build curated datasets that measurably improve downstream VLM/VLA model metrics rather than simply adding volume.
  • Close the data flywheel — define dataset schemas, quality metrics, and validation; track auto-labeling quality against model requirements; route model failures back into re-labeling and retraining loops.
  • Partner with the end-to-end model team — co-define dataset specifications with VLM/VLA model developers, own the quality bar and delivery cadence, and operationalize a continuous dataset delivery loop into their training pipelines.
  • Scale on cloud infrastructure — build distributed, reproducible pipelines using columnar data formats and distributed compute, with disciplined software practices, version control, and documentation.
  • Lead and mentor — serve as project lead, guide less-experienced engineers, run design reviews, set coding and annotation standards, and drive alignment across team interfaces to the rest of the organization.
  • Stay current — track the latest advances in multimodal models, auto-labeling, and end-to-end autonomous driving, and translate relevant research into production data systems.

What You’ll Need to Succeed:

  • Considered highly skilled and proficient in discipline; conducts complex, important work under minimal supervision and with wide latitude for independent judgment.
  • Scope of Influence: Expected to drive alignment across team interfaces to the rest of the organization. Designs, maintains, and owns team technical solutions and drives consensus. Mentors and guides engineers within the group.
  • Bachelor’s Degree in Computer Science, Robotics, Electrical Engineering, or related technical field plus competences typically acquired through 6+ years of experience; OR Master’s Degree in a related technical field plus competences typically acquired through 3+ years of experience.

Required Qualifications (some combination of the following skills):

  • Computer Vision & Deep Learning — model training and at least two of: 2D/3D Object Detection, Tracking, Sensor Fusion, Semantic Segmentation, BEV, Depth Estimation.
  • Multimodal / VLM experience — hands-on work with vision-language models, open-vocabulary or zero-shot recognition, dense captioning, or semantic embeddings / search applied to perception data.
  • Model Data Curation — building targeted datasets that measurably improve downstream model performance; large-scale Parquet data processing (Databricks, Daft, Pandas, etc.).
  • Distributed ML & data frameworks — PyTorch, Lightning, Ray, Spark, or equivalent for training and large-scale data processing.
  • Scaled MLOps & Tooling — experiment tracking, model registry, MLflow / Weights & Biases, and ML metrics, evaluation, and quality.
  • Development Tools & Eco-System (at scale) — strong Python software development, VDI and cloud-based development environments, CI systems (GitHub Actions), and Docker.

Bonus Points!

  • End-to-end / VLA driving — familiarity with VLM/VLA or end-to-end driving models, trajectory and action grounding, or chain-of-causation / reasoning-trace datasets.
  • Auto-labeling foundation models — experience with segmentation, open-vocabulary detectors, or VLM/LLM-driven data engines for annotation and verification.
  • High-throughput model serving — vLLM, SGLang, or similar for batch auto-labeling and inference at scale.
  • Semantic inference & retrieval — attribute mapping, semantic search, and vector databases (e.g., LanceDB) for automotive data.
  • AV data standards & tooling — scenario-description standards such as Pegasus layers; parsing robotics formats (ROS bags, MCAP) and optimizing columnar storage (Parquet, Arrow).
  • Cloud development & orchestration — Terraform and AWS managed services (S3, ECS, Lambda, DynamoDB, Step Functions, Athena); AWS HyperPod / Anyscale; inference orchestration.
  • Data visualization — Foxglove, FiftyOne (51), three.js, OpenGL, or similar for dataset inspection and accessibility.

Similar jobs

Judi HealthNew York, New York, United States

Est. 141,000 USD

About Judi Health Judi Health is a health technology company providing benefit administration solutions to employers, unions, health plans, and government entities. Judi Health replaces fragmented, outdated systems with…

Full-time
Zone 5 TechnologiesUnited States

Est. 124,000 USD

At Zone 5 Technologies, we're redefining what's possible in unmanned aircraft systems. Our team of engineers and innovators is developing cutting-edge autonomous solutions that push the boundaries of UAS technology - sol…

Full-time
PubMaticPune, India

About the Role At PubMatic, data operates at unmatched scale. As a Senior Machine Learning Platform Engineer, you will design and scale the infrastructure and frameworks that enable machine learning development, experime…

Full-time
AI Engineer6 months ago

Robots & Pencils is seeking an AI Engineer to design, build, and deploy production-grade AI systems that deliver measurable business value. This is a hands-on engineering role focused on implementation and delivery.…

Full-timeRemote
Vulcan ElementsDurham, North Carolina, United States

Est. 124,000 USD

Vulcan Elements is manufacturing American rare-earth permanent magnets for a secure, resilient future. With a focus on national security and economic resiliency, we serve critical industries such as defense, aerospace, a…

Full-time
Playtika LtdPoland

Est. 120,000 PLN

Ambition Needs a Place to Grow. At Playtika, we create mobile games enjoyed by millions worldwide. Combining the scale of a public company (NASDAQ: PLTK) with the speed and ownership of a startup, we are a global leader…

Full-time
Lucid MotorsNewark, California, United States

Est. 175,000 USD

About Lucid At Lucid, we are creating exceptional mobility experiences through innovation to drive the world forward. Built on Lucid’s proprietary technology and software-defined vehicle architecture, our award-winning v…

Full-time
CoupangMountain View, California, United States

Est. 175,000 USD

Company Introduction We exist to wow our customers. We know we’re doing the right thing when we hear our customers say, “How did I ever live without Coupang?” Born out of an obsession to make shopping, eating, and living…

Full-time
Zone 5 TechnologiesUnited States

Est. 141,000 USD

At Zone 5 Technologies, we're redefining what's possible in unmanned aircraft systems. Our team of engineers and innovators is developing cutting-edge autonomous solutions that push the boundaries of UAS technology - sol…

Full-time
West MonroeChicago, Illinois, United States

Est. 144,000 USD

Are you ready to make an impact?West Monroe is seeking a Senior Software Engineer to join the Tech Ex AI Assets Foundry Team, working with the internal Labs product team to develop proprietary software assets and AI capa…

Full-time
CongaBoston, Massachusetts, United States

Est. 140,000 USD

A career that’s the whole package! At Conga, we’ve built a community where our colleagues can thrive. Here you’ll find opportunities to innovate and support growth through individual and team development, all within an e…

Full-time
GHXHyderabad, Telangana, India

Software Engineer - III About the Role GHX is building a next-generation Intelligent Process Automation (IPA) platform powered by LLMs and AI-native document understanding. We extract structured data from complex healthc…

Full-time

Est. 250,000 USD

We’re looking for a Principal Architect to lead the design and delivery of complex, multi-domain systems spanning cloud, data, and AI. This role is ideal for a deeply experienced engineer who owns the hardest architectur…

Full-timeRemote
NICERichardson, Texas, United States

Est. 105,000 USD

At NiCE, we don’t limit our challenges. We challenge our limits. Always. We’re ambitious. We’re game changers. And we play to win. We set the highest standards and execute beyond them. And if you’re like us, we can offer…

Full-time
NICERichardson, Texas, United States

Est. 95,000 USD

At NiCE, we don’t limit our challenges. We challenge our limits. Always. We’re ambitious. We’re game changers. And we play to win. We set the highest standards and execute beyond them. And if you’re like us, we can offer…

Full-time

Est. 250,000 USD

We’re looking for a Principal Architect to lead the design and delivery of complex, multi-domain systems spanning cloud, data, and AI. This role is ideal for a deeply experienced engineer who owns the hardest architectur…

Full-timeRemote
Zone 5 TechnologiesUnited States

Est. 124,000 USD

At Zone 5 Technologies, we're redefining what's possible in unmanned aircraft systems. Our team of engineers and innovators is developing cutting-edge autonomous solutions that push the boundaries of UAS technology - sol…

Full-time
NavVisMunich, Germany

Est. 120,000 EUR

OUR VISION When people use our technology to "bridge the gap" between the physical and digital worlds, they don’t just capture reality - they create a new one. In this new reality, they are smarter, more productive, more…

Full-time
Anduril IndustriesCosta Mesa, California, United States

Est. 141,000 USD

Anduril Industries is a defense technology company with a mission to transform U.S. and allied military capabilities with advanced technology. By bringing the expertise, technology, and business model of the 21st century…

Full-time
AxleRemote

Est. 140,000 USD

(ID: 2026-3432) Axle is a bioscience and information technology company that offers advancements in translational research, biomedical informatics, and data science applications to research centers and healthcare organiz…

Full-timeRemote
Cloud Architect16 days ago
Robots and PencilsHouston, Texas, United States

Est. 141,000 USD

Cloud Architect Location: Houston, TX This is a 4 month assignment with potential to extend Houston, local OR be able to travel to Houston, TX to be at customer site every other week for a 4-5days/week Company Overview R…

Full-time
Zone 5 TechnologiesSan Luis Obispo, California, United States

Est. 124,000 USD

At Zone 5 Technologies, we're redefining what's possible in unmanned aircraft systems. Our team of engineers and innovators is developing cutting-edge autonomous solutions that push the boundaries of UAS technology - sol…

Full-time

APPLICATIONS FROM OUTSIDE COLOMBIA WILL NOT BE CONSIDERED FOR THIS ROLERobots & Pencils is an applied AI engineering firm building the next frontier of business architecture. We design and ship AI co-workers that int…

Full-timeRemote
Torc RoboticsAnn Arbor, Michigan, United States

Est. 124,000 USD

About the Company At Torc, we have always believed that autonomous vehicle technology will transform how we travel, move freight, and do business. A leader in autonomous driving since 2007, Torc has spent over a decade c…

Full-time
NavVisMunich, Germany

Est. 90,000 EUR

OUR VISION When people use our technology to "bridge the gap" between the physical and digital worlds, they don’t just capture reality - they create a new one. In this new reality, they are smarter, more productive, more…

Full-time

Est. 144,000 USD

About the Company At Torc, we have always believed that autonomous vehicle technology will transform how we travel, move freight, and do business. A leader in autonomous driving since 2007, Torc has spent over a decade c…

Full-timeRemote
NebiusRemote

Est. 140,000 USD

About Nebius: Nebius is leading a new era in cloud infrastructure for the global AI economy. We are building a full-stack AI cloud platform that supports developers and enterprises from data and model training through to…

Full-timeRemote
Alarm.comKraków, Lesser Poland Voivodeship, Poland

Est. 120,000 PLN

EBS is a rapidly expanding, entrepreneurial technology company and part of Alarm.com. Alarm.com is the leading cloud-based platform for smart security and the Internet of Things. More than 7.6+ million home and business…

Full-time
Five9Bengaluru, India

Join us in bringing joy to customer experience. Five9 is a leading provider of cloud contact center software, bringing the power of cloud innovation to customers worldwide. Living our values everyday results in our team-…

Full-time
Flagship Pioneering, Inc.Cambridge, Massachusetts, United States

Est. 140,000 USD

Who We Are Flagship Pioneering is a biotechnology company that invents and builds platform companies that change the world. We bring together the greatest scientific minds with entrepreneurial company builders and assemb…

Full-time