01

Work项目

ACT for Household Robots — Migration & Adaptation家务机器人 ACT — 迁移与增量控制适配

Undergraduate thesis · Xiamen University《家务机器人 ACT 算法的优化与实现》· 本科毕业设计
2025 – 2026 Imitation LearningMuJoCo / robosuite RobomimicPyTorchDINOv2 Source · private代码 · 私有整理中
Problem问题

The original ACT assumes absolute joint control. After migration to Robomimic's incremental operational-space control (Delta OSC), the semantics no longer align: the same numeric delta means something different at each state, so temporal ensembling across prediction windows can amplify open-loop error. 原版 ACT 建立在绝对关节控制之上,迁移到 Robomimic 的增量操作空间控制(Delta OSC)后语义失配: 同一数值增量在不同当前状态下物理意义不同,导致时序集成(TE)的跨窗口平均反而放大开环误差。

Approach做法

First, I connected the full path: MuJoCo/robosuite → multi-camera + qpos → IK / ARCore demonstrations → Robomimic HDF5 → training → rollout → skill scheduling. Then I adapted each mismatch: Sim-to-Sim alignment; zero-padding for asymmetric state/action dimensions without inventing physical signals; Masked L1 and KL tuning; RHC in place of TE (predict 100 / execute 8 / replan); a frozen DINOv2 backbone for visual OOD; and stabilization plus Home reset between skills. 先打通整条链路 — MuJoCo/robosuite → 多相机 + qpos → IK / ARCore 示教 → Robomimic HDF5 → 训练 → rollout → 技能调度 再逐项适配:Sim-to-Sim 环境对齐、Zero-Padding 处理状态/动作维度不对称(不伪造物理信息)、 Masked L1 + KL 调节避免有效动作信号被平均稀释、用 RHC(预测 100 步 / 执行 8 步 / 滚动重规划)替代 TE、 冻结 DINOv2 主干替换 ResNet18 应对视觉 OOD、技能切换加缓冲与 Home 复位。

Evidence证据
25 → 45 → 100%Open-loop → TE → RHC (stride=8) Same-task comparison, complete rollout records同任务对照,完整 rollout 记录
20 / 20lift_ph with a frozen DINOv2 backbonelift_ph,冻结 DINOv2 主干 Formal validation, strong evidence正式验证,强证据
40 → 94%Skill switch: direct → stabilized transition技能切换:直接切换 → 缓冲切换 Hierarchical-system evaluation分层系统验证
90% / 80%can_ph / square_ph Representative peaks, reported conservatively代表性峰值,保守口径
40 – 60%transport_ph dual-arm coordinationtransport_ph 双臂协同 Preliminary and unstable初步可行,不稳定
Boundary边界

tool_hang_ph remained at 0% across multiple batches: high-precision contact is a clear boundary, and lower training loss does not imply task success. All results are from simulation. Real deployment still requires a hardware control stack, hand-eye and frame calibration, dynamics identification, and safety limits. This is a migration and incremental-control adaptation of ACT, not a new algorithm. tool_hang_ph 多批次 0% —— 高精度接触任务是本工作的能力边界,训练 loss 下降不等于任务成功。 全部结论基于仿真;策略未做实机验证,真实部署还需要厂商控制栈、手眼与坐标系标定、动力学参数标定和安全限幅。 本工作是 ACT 的迁移实现与增量控制适配,不是提出新算法。

Logistics Robot on Ascend Atlas 200I DK A2基于昇腾 Atlas 200I DK A2 的物流机器人

National undergraduate innovation project · Project lead国家级大创 · 项目负责人
2023.12 – 2025.01 ROSCartographer SLAMOpenCV / OCR MecanumTeam of 4
Problem问题

Automate parcel recognition, grasping, and sorting on an edge-computing platform. Two practical bottlenecks were map drift in long corridors and the arm's limited reachable workspace on a fixed chassis. 在边缘算力平台上做仓储包裹的自动识别、抓取与分拣。两个实际卡点:长走廊场景下建图漂移, 以及固定底盘导致机械臂有效抓取范围受限。

Approach做法

Integrated the system with ROS. I chose Cartographer over Gmapping for more reliable loop closure in corridors, fused two LiDARs, configured global and local planners in move_base, and adopted a mecanum chassis for omnidirectional reach. The perception path used OpenCV and OCR for tilted parcels, followed by visual-guided manipulation with a Dobot arm. ROS 全栈集成;建图选 Cartographer 而非 Gmapping——理由是长走廊环境闭环检测更可靠; 双激光雷达融合提升定位精度,move_base 配置全局 / 局部规划器; 底盘换成麦克纳姆轮换取全向移动与更大作业半径; 感知侧用 OpenCV + OCR 处理快件倾斜摆放下的识别,再按品类分拣;执行端为视觉引导的越疆机械臂。

Role职责

Led a four-person team and owned the arm-vision path plus chassis SLAM/navigation, including hardware debugging across LiDAR, IMU, encoders, and the ROS deployment. 项目负责人(4 人):整体统筹与分工,个人负责机械臂视觉与底盘 SLAM / 导航, 含激光雷达、IMU、编码器的硬件调试与 ROS 部署。

Boundary边界

The system did not fully meet the original proposal. Navigation robustness and grasp success remained unstable in real stacked scenes. It was my first complete encounter with the gap between a lab algorithm and a deployed system, and it shaped the direction I chose afterward. 项目功能未完全达到立项预期,导航鲁棒性和抓取成功率在真实堆叠场景下仍不稳定。 这是我第一次完整体会到「实验室算法」与「真实场景落地」之间的差距,也直接决定了后面两年的方向选择。

  • UNIX Systems Programming2024 Built a small shell and a myfind directory-tree walker; compared synchronous and asynchronous write throughput.从零实现简易 shell 与 myfind 目录树遍历;做过同步 / 异步 write 的吞吐对比实验。
  • Cyberry Potter — Electromagic Wand2024 An embedded TinyML exercise in MCU + IMU gesture recognition, with on-device deployment and real-time constraints.MCU + IMU 手势识别的嵌入式 TinyML 实践,接触端侧模型部署与实时性约束。
  • Vision Coursework — Crosswalk & Gem Recognition2023 Two classical-vision and classification exercises that became preparation for the later logistics-robot perception stack.学科实践中的两个传统视觉 + 分类任务,是后来大创感知模块的前置练习。
02

Stack技术

Languages语言C · C++ · Python · Shell
Robotics机器人ROS · MuJoCo · robosuite / Robomimic · Cartographer · move_base · IK / OSC
Learning学习PyTorch · ACT · CVAE · DETR · DINOv2 · behavior cloning
Perception感知OpenCV · OCR · visual-guided grasping
Systems系统Linux / UNIX syscalls · OS · compilers · architecture · databases
Hardware硬件Ascend Atlas 200I DK A2 · mecanum chassis · LiDAR / IMU · Dobot arm · Raspberry Pi · MCU
Tooling工具Git · LaTeX · HDF5 · Vim / VS Code