100 lines
3.6 KiB
Markdown
100 lines
3.6 KiB
Markdown
# ACT 真机抓取操作
|
||
|
||
SO-101 机械臂上，用 **ACT** 策略做 **本地同步推理**，并在 **Rerun** 里看三路相机与六关节曲线。
|
||
**Python、LeRobot、相机与标定已在平台配好**，你只需按下面开两个终端、跑对应脚本。
|
||
|
||
---
|
||
|
||
|
||
|
||
<div align="center" style="line-height:1">
|
||
<a href="https://www.kimi.com" target="_blank"><img alt="Chat" src="https://img.shields.io/badge/🤖%20Chat-Kimi%20K3-ff6b6b?color=1783ff&logoColor=white"/></a>
|
||
<a href="https://www.moonshot.ai" target="_blank"><img alt="Homepage" src="https://img.shields.io/badge/Homepage-Moonshot%20AI-white?logo=Kimi&logoColor=white"/></a>
|
||
</div>
|
||
|
||
<div align="center" style="line-height: 1;">
|
||
<a href="https://huggingface.co/moonshotai" target="_blank"><img alt="Hugging Face" src="https://img.shields.io/badge/%F0%9F%A4%97%20Hugging%20Face-Moonshot%20AI-ffc107?color=ffc107&logoColor=white"/></a>
|
||
<a href="https://twitter.com/kimi_moonshot" target="_blank"><img alt="Twitter Follow" src="https://img.shields.io/badge/Twitter-Kimi.ai-white?logo=x&logoColor=white"/></a>
|
||
<a href="https://discord.gg/TYU2fdJykW" target="_blank"><img alt="Discord" src="https://img.shields.io/badge/Discord-Kimi.ai-white?logo=discord&logoColor=white"/></a>
|
||
<a href="https://modelscope.cn/organization/moonshotai" target="_blank"><img alt="ModelScope" src="https://img.shields.io/badge/ModelScope-Moonshot%20AI-white?labelColor=rgb(99%2C%2074%2C%255)"/></a>
|
||
</div>
|
||
<div align="center" style="line-height: 1;">
|
||
<a href="https://huggingface.co/moonshotai/Kimi-K3/blob/main/LICENSE"><img alt="License" src="https://img.shields.io/badge/License-Kimi_K3-f5de53?&color=f5de53"/></a>
|
||
</div>
|
||
|
||
|
||
|
||
## 一、ACT 模型说明
|
||
|
||
| 项目 | 说明 |
|
||
|------|------|
|
||
| 算法 | **ACT**（Action Chunking with Transformers）：模仿学习；输入 **3 路相机 + 6 关节**，输出一段未来动作再逐步执行 |
|
||
| 本仓权重 | Stage A **五积木联合训练 · 350 条**（青/黄/橙/红/紫，放入 tray） |
|
||
| 权重路径 | `models/act_stageA_single5_350/pretrained_model/`（已随仓库提供，约 198MB） |
|
||
|
||
---
|
||
|
||
## 二、仓库结构（你要用到的部分）
|
||
|
||
```text
|
||
act_rerun/
|
||
├── models/act_stageA_single5_350/pretrained_model/ # ACT 权重（已包含）
|
||
└── scripts/
|
||
├── env.sh # 环境变量（先 source）
|
||
├── run_act_sync_rerun.sh # 单色积木抓取
|
||
├── run_act_five_blocks.sh # 五积木随机顺序验证
|
||
|
||
```
|
||
|
||
---
|
||
|
||
## 三、真机验证怎么跑
|
||
|
||
|
||
### 跑 ACT 推理（先加载环境变量再推理）
|
||
|
||
```bash
|
||
conda activate lerobot_v30
|
||
source /workspace/scripts/env.sh
|
||
|
||
```
|
||
|
||
**五积木各测 1 次（随机顺序，推荐使用）：**
|
||
|
||
```bash
|
||
bash /workspace/scripts/run_act_five_blocks.sh
|
||
```
|
||
|
||
**只测一种颜色（例如青色）：**
|
||
|
||
```bash
|
||
bash /workspace/scripts/run_act_sync_rerun.sh cyan
|
||
```
|
||
|
||
可选颜色：`cyan` `yellow` `orange` `red` `purple`。
|
||
|
||
---
|
||
|
||
## 四、各脚本做什么
|
||
|
||
| 脚本 | 用途 |
|
||
|------|------|
|
||
| `scripts/env.sh` | 设置本仓路径、ACT 权重路径；**其它脚本前必须先 source** |
|
||
| `scripts/run_act_sync_rerun.sh` | **单色积木**同步推理 |
|
||
| `scripts/run_act_five_blocks.sh` | **五积木**按随机顺序各跑 1 episode |
|
||
|
||
|
||
---
|
||
|
||
## 五、Rerun 里看什么算正常
|
||
|
||
| 位置 | 内容 |
|
||
|------|------|
|
||
| 上排 | `camera/front`、`camera/side`、`camera/wrist` 实时图 |
|
||
| 下排 | 6 个关节：`sent_action` 与 `observed_qpos` 两条曲线应基本贴合 |
|
||
| 时间轴 | 必须选 **`robot_time`** |
|
||
|
||
应看到类似下图（上排三路相机，下排六关节曲线）：
|
||
|
||
|