The original ACT assumes absolute joint control. After migration to Robomimic's
incremental operational-space control (Delta OSC), the semantics no longer align: the same numeric delta means
something different at each state, so temporal ensembling across prediction windows can amplify open-loop error.原版 ACT 建立在绝对关节控制之上,迁移到 Robomimic 的增量操作空间控制(Delta OSC)后语义失配:
同一数值增量在不同当前状态下物理意义不同,导致时序集成(TE)的跨窗口平均反而放大开环误差。
Approach做法
First, I connected the full path:
MuJoCo/robosuite → multi-camera + qpos → IK / ARCore demonstrations → Robomimic HDF5 → training → rollout → skill scheduling.
Then I adapted each mismatch: Sim-to-Sim alignment; zero-padding for asymmetric state/action dimensions without inventing physical signals;
Masked L1 and KL tuning; RHC in place of TE (predict 100 / execute 8 / replan); a frozen DINOv2 backbone for visual OOD;
and stabilization plus Home reset between skills.先打通整条链路 —
MuJoCo/robosuite → 多相机 + qpos → IK / ARCore 示教 → Robomimic HDF5 → 训练 → rollout → 技能调度
再逐项适配:Sim-to-Sim 环境对齐、Zero-Padding 处理状态/动作维度不对称(不伪造物理信息)、
Masked L1 + KL 调节避免有效动作信号被平均稀释、用 RHC(预测 100 步 / 执行 8 步 / 滚动重规划)替代 TE、
冻结 DINOv2 主干替换 ResNet18 应对视觉 OOD、技能切换加缓冲与 Home 复位。
40 – 60%transport_ph dual-arm coordinationtransport_ph 双臂协同Preliminary and unstable初步可行,不稳定
Boundary边界
tool_hang_ph remained at 0% across multiple batches: high-precision contact is a clear boundary,
and lower training loss does not imply task success. All results are from simulation. Real deployment still requires a hardware control stack,
hand-eye and frame calibration, dynamics identification, and safety limits. This is a migration and incremental-control adaptation of ACT,
not a new algorithm.tool_hang_ph 多批次 0% —— 高精度接触任务是本工作的能力边界,训练 loss 下降不等于任务成功。
全部结论基于仿真;策略未做实机验证,真实部署还需要厂商控制栈、手眼与坐标系标定、动力学参数标定和安全限幅。
本工作是 ACT 的迁移实现与增量控制适配,不是提出新算法。
National undergraduate innovation project · Project lead国家级大创 · 项目负责人
2023.12 – 2025.01ROSCartographer SLAMOpenCV / OCRMecanumTeam of 4
Problem问题
Automate parcel recognition, grasping, and sorting on an edge-computing platform. Two practical bottlenecks were
map drift in long corridors and the arm's limited reachable workspace on a fixed chassis.在边缘算力平台上做仓储包裹的自动识别、抓取与分拣。两个实际卡点:长走廊场景下建图漂移,
以及固定底盘导致机械臂有效抓取范围受限。
Approach做法
Integrated the system with ROS. I chose Cartographer over Gmapping for more reliable loop closure in corridors,
fused two LiDARs, configured global and local planners in move_base, and adopted a mecanum chassis for omnidirectional reach.
The perception path used OpenCV and OCR for tilted parcels, followed by visual-guided manipulation with a Dobot arm.ROS 全栈集成;建图选 Cartographer 而非 Gmapping——理由是长走廊环境闭环检测更可靠;
双激光雷达融合提升定位精度,move_base 配置全局 / 局部规划器;
底盘换成麦克纳姆轮换取全向移动与更大作业半径;
感知侧用 OpenCV + OCR 处理快件倾斜摆放下的识别,再按品类分拣;执行端为视觉引导的越疆机械臂。
Role职责
Led a four-person team and owned the arm-vision path plus chassis SLAM/navigation, including hardware debugging
across LiDAR, IMU, encoders, and the ROS deployment.项目负责人(4 人):整体统筹与分工,个人负责机械臂视觉与底盘 SLAM / 导航,
含激光雷达、IMU、编码器的硬件调试与 ROS 部署。
Boundary边界
The system did not fully meet the original proposal. Navigation robustness and grasp success remained unstable
in real stacked scenes. It was my first complete encounter with the gap between a lab algorithm and a deployed system,
and it shaped the direction I chose afterward.项目功能未完全达到立项预期,导航鲁棒性和抓取成功率在真实堆叠场景下仍不稳定。
这是我第一次完整体会到「实验室算法」与「真实场景落地」之间的差距,也直接决定了后面两年的方向选择。
UNIX Systems Programming2024Built a small shell and a myfind directory-tree walker; compared synchronous and asynchronous write throughput.从零实现简易 shell 与 myfind 目录树遍历;做过同步 / 异步 write 的吞吐对比实验。
Cyberry Potter — Electromagic Wand2024An embedded TinyML exercise in MCU + IMU gesture recognition, with on-device deployment and real-time constraints.MCU + IMU 手势识别的嵌入式 TinyML 实践,接触端侧模型部署与实时性约束。
Vision Coursework — Crosswalk & Gem Recognition2023Two classical-vision and classification exercises that became preparation for the later logistics-robot perception stack.学科实践中的两个传统视觉 + 分类任务,是后来大创感知模块的前置练习。