Description
As a Perception Engineer, you will own how LightSpeed's robots see. You will design, calibrate, and deploy the multi-camera systems on our robot cells and end-of-arm tooling, and build the perception that guides manipulation in real time.
Camera & Sensor Systems
- Select, specify, and deploy multi-camera systems (RGB, stereo, depth) for robot cells, including wrist- and tool-mounted cameras on end-of-arm tooling and dexterous hands
- Own intrinsic, extrinsic, and hand-eye calibration, and keep calibration reliable across tool changes and daily operation
- Synchronize cameras with robot state and teleoperation streams
- Work with mechanical engineers on camera placement for coverage, occlusion, and lighting
Perception Development
- Estimate and track robot and end-effector pose using vision systems to verify kinematics and identify drift
- Compare 3D scans against CAD or BIM models to check part placement and as-built accuracy
- Localize robots and tools in the workspace using fiducials (AprilTag or ChArUco) and multi-view fusion
- Build object detection, segmentation, and 6-DoF pose estimation for parts, fasteners, and tools in cluttered, confined, and poorly lit workspaces
- Build point cloud pipelines (filtering, segmentation, registration such as ICP, and meshing) for manipulation and scene understanding
- Track hands and objects in egocentric video of human demonstrations
- Build CUDA-optimized perception pipelines: GPU preprocessing, custom CUDA kernels where needed, TensorRT inference, and minimal host–device copies
- Profile and optimize end-to-end latency, from photon to action, using tools like Nsight Systems
Data & Robot Learning Support
- Deliver calibrated, time-synchronized camera data in the formats our teleoperation recording and training pipelines consume
- Partner with AI/ML engineers to define observation spaces for imitation learning and vision-language-action policies
- Set perception accuracy targets tied to task success, and build benchmarks to measure them
Integration & Deployment
- Integrate perception with ROS2-based robot control
- Harden systems for real-world conditions: lighting changes, reflective surfaces, occlusion, and vibration
- Collaborate closely with robotics, mechanical, and controls engineers
Required Qualifications
- 3+ years building and deploying perception systems on physical robots or real-time systems
- Hands-on experience calibrating multi-camera systems (intrinsic, extrinsic, hand-eye) and synchronizing sensors
- Experience with RGB-D and stereo cameras in real deployments
- Strong computer vision fundamentals: camera geometry, multi-view geometry, and 3D reconstruction
- Deep learning experience with detection, segmentation, or pose estimation using PyTorch
- Strong Python and C++
- Experience with OpenCV and point cloud libraries such as Open3D or PCL
- Experience with ROS or ROS2
- Proficiency with Linux, Docker, Git, and CI/CD workflows
Preferred Qualifications
- MS or PhD in Computer Vision, Robotics, Computer Science, or a related field
- Perception for robotic manipulation, especially dexterous or tool-based manipulation
- Experience with hardware video encoding (NVENC, Jetson Multimedia API), GStreamer, or WebRTC
- Experience with NVIDIA Isaac ROS / NITROS, VPI, or CV-CUDA
- Experience with point cloud registration and scan-to-CAD/BIM comparison
- Experience with vision foundation models (for example SAM, DINOv2, FoundationPose)
- Experience with egocentric video or hand pose estimation
- Experience producing sensor data for robot learning datasets (e.g., rosbag or MCAP)
- Edge deployment on NVIDIA Jetson with TensorRT
- Simulation and synthetic data generation with Isaac Sim, MuJoCo, or similar
- Background in manufacturing, industrial automation, or construction technology
Why Join LightSpeed
- Build the eyes of robots that are addressing the housing crisis
- Own the full perception stack, from sensor selection to deployed models
- Work on rare real-world challenges: confined-space manipulation, dexterous hands, learning from human demonstration
- Hands-on culture with direct access to robots, sensors, and manufacturing environments
- Competitive compensation and comprehensive benefits