Luai Abuelsamen

Hi! I’m a Master’s student in Robotics at UC Berkeley working on embodied AI systems that bridge foundation models with real-world control. My research spans from multimodal perception for robot manipulation to large-scale autonomous systems simulation.

Currently, I work as a Graduate Student Researcher at PATH, building traffic simulation tools to evaluate infrastructure impacts on autonomous vehicle behavior. I also collaborated with the Autonomy, Robotics, and Controls (ARC) Lab on manipulation research combining learning-based control with multimodal perception.

Previously, I studied Mechanical Engineering at McGill University and interned at Tesla, Beta Technologies, and Vention, where I worked on embedded systems, mechatronic design, and simulation tooling for robotic platforms.

Research Interests:

• Foundation models for robotics and embodied AI
• Multimodal perception and imitation learning
• Model-based and learning-based control
• Sim-to-real transfer and real-time planning
• Multi-agent systems and intelligent infrastructure

DexTrack-Style RL Tracking on Dexmate Vega

Work in progress on bringing DexTrack-style manipulation tracking to the Dexmate Vega humanoid in MuJoCo, starting with single-arm control and extending toward coordinated bimanual nonprehensile manipulation.

Built around cumulative residual action targets, object pose tracking, and physically consistent in-sim reference generation for pushing and reorientation tasks.

Multimodal Perception in Imitation Learning

Research collaboration analyzing how RGB-D, proprioceptive, and language inputs affect sample complexity and optimization landscapes in robot manipulation tasks, with theoretical insights and empirical validation.

📄 Paper · 🔗 Code

Imitation Learning with Residual RL Fine-Tuning

Trained imitation learning policies using LeRobot framework and fine-tuned them in simulation with residual reinforcement learning, combining behavior cloning with policy optimization for improved robotic manipulation performance.

🔗 Code

RVT for LeRobot — Multi-View Transformer Policy (WIP)

Bridging NVlabs/RVT into Hugging Face's LeRobot ecosystem, which currently ships only RGB / dense-action policies. Built a MuJoCo SO-ARM100 env with 4 RGBD cameras, a PerAct/RLBench-format episode writer (depth packed as 24-bit RGB PNGs), and a PreTrainedPolicy-shaped wrapper that delegates to RVT's agent. The hero figure shows the actual trick: 4 real RGBD cams → fused 60k-point cloud → 5 orthographic virtual views — the multi-view transformer attends over the bottom row, and the projection step is geometric, not learned.

🔗 Code

Sim-to-Sim PPO Fine-Tuning + Flow-Matching Distillation of a Quadruped Locomotion Policy

First, fine-tuned the pretrained walk-these-ways Go2 policy from Isaac Gym to MuJoCo on a Jetson Orin NX, porting the full CoRL gait-shaping reward (von-Mises smoothed contact targets, raibert heuristic, ji22-style multiplicative gating) so the trot survives the sim-to-sim physics gap. Then distilled the RL teacher into an 11M-param flow-matching student (transformer encoder + AdaRMSNorm action expert with a 4-step ODE solver), trained on 215k demo transitions.

🔗 Code

Hard-Equivariant Latent Dynamics for 3D World Models

Built a hard SO(3)-equivariant 3D point cloud world model — Vector Neurons encoder + closed-form group-action latent dynamics — to test whether architectural geometric priors give provable extrapolation to rotations outside the training distribution. Latent pose error sits at machine epsilon (~10⁻⁸) across every rollout step in both regimes, vs 10⁻¹ for a learned-MLP baseline; at large OOD rotations the decoded reconstruction is an order of magnitude better. End-to-end on a Jetson Orin.

🔗 Code

Industrial Robot Motion Planning with GPUs

Integrated NVIDIA cuRobo into modular automation systems for real-time, collision-free trajectory planning in multi-axis robotic platforms, achieving significant speedups for industrial applications.

🔗 Code · 📄 Paper

Rerouting Impacts of Auxiliary Lanes

Conducted microsimulation research using Aimsun Next to evaluate how auxiliary lane removal affects freeway and arterial network performance in mixed-autonomy scenarios. Work performed at UC Berkeley – California PATH.

📄 Paper (To be Presented at TRB 2026)

Coverage Control for Hybrid Aerial / Ground Robot Teams

Developed a two-layer Voronoi-based approach for multi-agent coverage in heterogeneous robot teams, improving emergency response coordination and resilience to sensor loss.

🔗 Code · 📄 Paper

Rocket Landing Trajectory Optimization

Implemented SOCP-based convex optimization in Python for soft landing guidance, accounting for vehicle dynamics, control limits, and environmental constraints with real-time performance.

🔗 Code · 📄 Report

Bicycle State Estimation with Unscented Kalman Filter

Developed a UKF-based state estimator for a nonlinear bicycle model with uncertain parameters, achieving competitive performance (rank 2/40) through optimized process/measurement noise modeling and parameter estimation.

🔗 Code · 📄 Report

LLM-Based Natural Language Drone Control

Rapid prototyping project integrating vision-language models, SLAM, and zero-shot action chunking for voice-controlled quadrotor navigation in real environments. Built for Eric Schmidt AI hackathon.

🔗 Code · 🎥 Video

PCB Design: Raspberry Pi Motor Controller Hat

Custom PCB enabling high-current servo control and force sensing for autonomous robots, with integrated power regulation and I²C multiplexing for research platform development.

🔗 Project Page

BLE-Controlled Mecanum Robot with FreeRTOS

Developed embedded firmware in C for a mecanum-drive mobile robot featuring dual-mode operation (autonomous/manual), BLE GATT server, ultrasonic obstacle detection, and PWM motor control—all orchestrated with real-time FreeRTOS task scheduling.

🔗 Code