Luai Abuelsamen
Hi! I’m a robotics engineer working on embodied AI systems that bridge foundation models with real-world control. My work spans multimodal perception for robot manipulation to large-scale autonomous systems simulation.
I hold an M.Eng from UC Berkeley in Mechanical Engineering, concentrating on control of robotic and autonomous systems. There I was a Graduate Student Researcher at PATH, building traffic simulation tools to evaluate infrastructure impacts on autonomous vehicle behavior, and collaborated with the Autonomy, Robotics, and Controls (ARC) Lab on manipulation research combining learning-based control with multimodal perception.
Previously, I studied Mechanical Engineering at McGill University and interned at Tesla, Beta Technologies, and Vention, where I worked on embedded systems, mechatronic design, and simulation tooling for robotic platforms.
Research Interests:
- Foundation models for robotics and embodied AI
- Multimodal perception and imitation learning
- Model-based and learning-based control
- Sim-to-real transfer and real-time planning
- Multi-agent systems and intelligent infrastructure
Humanoid Getup from Arbitrary Pose
One brax PPO policy stands a 21-DOF dm_control humanoid up from supine, prone, side, kneeling, or already standing. Each pose needs a different first move (sit-up, push-up to all fours, roll to supine) out of the same network.
The stock reward is zero below head height 1.05, so supine starts had no gradient to follow. A non-saturating head-height bonus, plus a kneeling snapshot in the starting-pose mix, gave the policy a gradient from every start. Trained with MJX on Modal H100s.
Two-Hand Retargeting for Dexterous Manipulation
An extension of DexTrack (Liu et al.), whose tracking policy, task and single right-hand references are theirs. My part: a left Allegro hand and two-hand references from GRAB clips in DexTrack's format, so a tracker can be trained on two-handed clips. Mirroring their hand turned up a bug in their left URDF: the ring finger's mount rotation isn't mirrored, which splays the finger the wrong way.
Measuring DexTrack's released references showed fingers sitting 3.2–8.8 mm inside the object across nine clips, so tracking the reference and not crushing the object pull against each other. Six retargeted clips shown here; the two-hand policy is still in progress.
libphys: GPU Tactile Simulation with Partial-Slip Contact
A GPU tactile simulator whose gel fingertip pads report pressure, shear and marker displacement, solved as an elastic contact problem rather than with penalty springs. A gripper must lift fragile balls of hidden mass and friction without crushing them.
In simulation, a policy that sees the gel's marker images succeeds 67% of the time against 36% for one that sees pad forces (3 seeds, 4,096 held-out episodes each). The marker field shows the rim of the contact starting to slip, which force readings cannot.
Imitation Learning with Residual RL Fine-Tuning
Trained imitation learning policies using LeRobot framework and fine-tuned them in simulation with residual reinforcement learning, combining behavior cloning with policy optimization for improved robotic manipulation performance.
RVT for LeRobot — Multi-View Transformer Policy (WIP)
Bridging NVlabs/RVT into Hugging Face's LeRobot ecosystem, which currently ships only RGB / dense-action policies. Built a MuJoCo SO-ARM100 env with 4 RGBD cameras, a PerAct/RLBench-format episode writer (depth packed as 24-bit RGB PNGs), and a PreTrainedPolicy-shaped wrapper that delegates to RVT's agent. The hero figure shows the actual trick: 4 real RGBD cams → fused 60k-point cloud → 5 orthographic virtual views — the multi-view transformer attends over the bottom row, and the projection step is geometric, not learned.
Sim-to-Sim PPO Fine-Tuning + Flow-Matching Distillation of a Quadruped Locomotion Policy
First, fine-tuned the pretrained walk-these-ways Go2 policy from Isaac Gym to MuJoCo on a Jetson Orin NX, porting the full CoRL gait-shaping reward (von-Mises smoothed contact targets, raibert heuristic, ji22-style multiplicative gating) so the trot survives the sim-to-sim physics gap. Then distilled the RL teacher into an 11M-param flow-matching student (transformer encoder + AdaRMSNorm action expert with a 4-step ODE solver), trained on 215k demo transitions.
Hard-Equivariant Latent Dynamics for 3D World Models
Built a hard SO(3)-equivariant 3D point cloud world model — Vector Neurons encoder + closed-form group-action latent dynamics — to test whether architectural geometric priors give provable extrapolation to rotations outside the training distribution. Latent pose error stays at the encoder's equivariance noise floor at every rollout step, inside and outside the training rotations. A learned-MLP baseline is orders of magnitude worse in the same metric, and its decoded reconstruction diverges at large out-of-distribution rotations while the equivariant model's stays flat. End-to-end on a Jetson Orin.
Multi-Car Parking with Distributed MPC
A dependency-free C++17 planner and controller: Hybrid A* routes, then one constrained iLQR controller per car, each optimizing against the others' broadcast predictions under a 150 ms decision budget, with a checked braking fallback. Kinematic simulation with perfectly shared state.
In the checked-in stress benchmark all 13 two-car cases (7 named, 6 seeded) finish with no collisions and no deadline misses, and all 6 three-car traffic cases pass.
GPU Motion Planning for a Humanoid Upper Body
cuRobo planning and MPC for the Dexmate Vega 1U in MuJoCo, both loading the same URDF: reaching, chasing moving goals with one or both arms as one kinematic chain, and pick-and-place with the held object attached to the plan. A geometric impedance controller (control law from Seo et al., arXiv:2504.17080) executes plans with torques under gravity.
Rerouting Impacts of Auxiliary Lanes
Conducted microsimulation research using Aimsun Next to evaluate how auxiliary lane removal affects freeway and arterial network performance in mixed-autonomy scenarios. Work performed at UC Berkeley – California PATH.
Rocket Landing Trajectory Optimization
6-DoF powered-descent guidance in Python by successive convexification: each iteration solves an SOCP subject to thrust, gimbal, glide-slope, tilt and body-rate limits, with free final time. The landing shown is the optimized controls propagated open-loop through the nonlinear dynamics, not the optimizer's nodes. Offline solver, not real time.
PCB Design: Raspberry Pi Motor Controller Hat
Custom PCB enabling high-current servo control and force sensing for autonomous robots, with integrated power regulation and I²C multiplexing for research platform development.
BLE-Controlled Mecanum Robot with FreeRTOS
Developed embedded firmware in C for a mecanum-drive mobile robot featuring dual-mode operation (autonomous/manual), BLE GATT server, ultrasonic obstacle detection, and PWM motor control—all orchestrated with real-time FreeRTOS task scheduling.