Compare commits
2 Commits
| Author | SHA1 | Date |
|---|---|---|
u_fengyj8820
|
90690c00c9 | |
|
u_fengyj8820
|
118f8c4408 |
|
|
@ -5,3 +5,17 @@ __pycache__/
|
||||||
*.pyc
|
*.pyc
|
||||||
*.log
|
*.log
|
||||||
node_modules/
|
node_modules/
|
||||||
|
|
||||||
|
# PMS — 容器运行时产物（勿提交）
|
||||||
|
.env
|
||||||
|
.pms-task-env.json
|
||||||
|
.bashrc
|
||||||
|
.bash_profile
|
||||||
|
.config/
|
||||||
|
.local/
|
||||||
|
.vnc/
|
||||||
|
.cache/
|
||||||
|
.gz/
|
||||||
|
models/
|
||||||
|
Desktop/
|
||||||
|
.ICEauthority
|
||||||
|
|
|
||||||
111
README.md
111
README.md
|
|
@ -1,34 +1,99 @@
|
||||||
# 仓库 3D 场景
|
# ACT 真机抓取操作
|
||||||
|
|
||||||
|
SO-101 机械臂上，用 **ACT** 策略做 **本地同步推理**，并在 **Rerun** 里看三路相机与六关节曲线。
|
||||||
|
**Python、LeRobot、相机与标定已在平台配好**，你只需按下面开两个终端、跑对应脚本。
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
## 场景预览
|
<div align="center" style="line-height:1">
|
||||||
|
<a href="https://www.kimi.com" target="_blank"><img alt="Chat" src="https://img.shields.io/badge/🤖%20Chat-Kimi%20K3-ff6b6b?color=1783ff&logoColor=white"/></a>
|
||||||
|
<a href="https://www.moonshot.ai" target="_blank"><img alt="Homepage" src="https://img.shields.io/badge/Homepage-Moonshot%20AI-white?logo=Kimi&logoColor=white"/></a>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<div align="center" style="line-height: 1;">
|
||||||
|
<a href="https://huggingface.co/moonshotai" target="_blank"><img alt="Hugging Face" src="https://img.shields.io/badge/%F0%9F%A4%97%20Hugging%20Face-Moonshot%20AI-ffc107?color=ffc107&logoColor=white"/></a>
|
||||||
|
<a href="https://twitter.com/kimi_moonshot" target="_blank"><img alt="Twitter Follow" src="https://img.shields.io/badge/Twitter-Kimi.ai-white?logo=x&logoColor=white"/></a>
|
||||||
|
<a href="https://discord.gg/TYU2fdJykW" target="_blank"><img alt="Discord" src="https://img.shields.io/badge/Discord-Kimi.ai-white?logo=discord&logoColor=white"/></a>
|
||||||
|
<a href="https://modelscope.cn/organization/moonshotai" target="_blank"><img alt="ModelScope" src="https://img.shields.io/badge/ModelScope-Moonshot%20AI-white?labelColor=rgb(99%2C%2074%2C%255)"/></a>
|
||||||
|
</div>
|
||||||
|
<div align="center" style="line-height: 1;">
|
||||||
|
<a href="https://huggingface.co/moonshotai/Kimi-K3/blob/main/LICENSE"><img alt="License" src="https://img.shields.io/badge/License-Kimi_K3-f5de53?&color=f5de53"/></a>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
## 一、ACT 模型说明
|
||||||
|
|
||||||
|
| 项目 | 说明 |
|
||||||
|
|------|------|
|
||||||
|
| 算法 | **ACT**（Action Chunking with Transformers）：模仿学习；输入 **3 路相机 + 6 关节**，输出一段未来动作再逐步执行 |
|
||||||
|
| 本仓权重 | Stage A **五积木联合训练 · 350 条**（青/黄/橙/红/紫，放入 tray） |
|
||||||
|
| 权重路径 | `models/act_stageA_single5_350/pretrained_model/`（已随仓库提供，约 198MB） |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 二、仓库结构（你要用到的部分）
|
||||||
|
|
||||||
|
```text
|
||||||
|
act_rerun/
|
||||||
|
├── models/act_stageA_single5_350/pretrained_model/ # ACT 权重（已包含）
|
||||||
|
└── scripts/
|
||||||
|
├── env.sh # 环境变量（先 source）
|
||||||
|
├── run_act_sync_rerun.sh # 单色积木抓取
|
||||||
|
├── run_act_five_blocks.sh # 五积木随机顺序验证
|
||||||
|
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 三、真机验证怎么跑
|
||||||
|
|
||||||
|
|
||||||
|
### 跑 ACT 推理（先加载环境变量再推理）
|
||||||
|
|
||||||
|
```bash
|
||||||
|
conda activate lerobot_v30
|
||||||
|
source /workspace/scripts/env.sh
|
||||||
|
|
||||||
|
```
|
||||||
|
|
||||||
|
**五积木各测 1 次（随机顺序，推荐使用）：**
|
||||||
|
|
||||||
|
```bash
|
||||||
|
bash /workspace/scripts/run_act_five_blocks.sh
|
||||||
|
```
|
||||||
|
|
||||||
|
**只测一种颜色（例如青色）：**
|
||||||
|
|
||||||
|
```bash
|
||||||
|
bash /workspace/scripts/run_act_sync_rerun.sh cyan
|
||||||
|
```
|
||||||
|
|
||||||
|
可选颜色：`cyan` `yellow` `orange` `red` `purple`。
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 四、各脚本做什么
|
||||||
|
|
||||||
|
| 脚本 | 用途 |
|
||||||
|
|------|------|
|
||||||
|
| `scripts/env.sh` | 设置本仓路径、ACT 权重路径；**其它脚本前必须先 source** |
|
||||||
|
| `scripts/run_act_sync_rerun.sh` | **单色积木**同步推理 |
|
||||||
|
| `scripts/run_act_five_blocks.sh` | **五积木**按随机顺序各跑 1 episode |
|
||||||
|
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## 启动（唯一命令，无需编译）
|
## 五、Rerun 里看什么算正常
|
||||||
|
|
||||||
卡片内已包含预编译 `install/`，**直接运行即可**，不要 `colcon build`。
|
|
||||||
|
|
||||||
```bash
|
|
||||||
./gzsim_run.sh
|
|
||||||
|
|
||||||
|
|
||||||
| `./gzsim_stop.sh` | 停止仿真 |
|
|
||||||
```
|
|
||||||
|
|
||||||
**必须等到**终端出现：
|
|
||||||
|
|
||||||
```text
|
|
||||||
[autorun] GzWeb scene OK (... bytes), 9002 stable
|
|
||||||
[autorun] ✓ 全部就绪 — 现在刷新「仿真器」面板
|
|
||||||
```
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
| 位置 | 内容 |
|
||||||
|
|------|------|
|
||||||
|
| 上排 | `camera/front`、`camera/side`、`camera/wrist` 实时图 |
|
||||||
|
| 下排 | 6 个关节：`sent_action` 与 `observed_qpos` 两条曲线应基本贴合 |
|
||||||
|
| 时间轴 | 必须选 **`robot_time`** |
|
||||||
|
|
||||||
|
应看到类似下图（上排三路相机，下排六关节曲线）：
|
||||||
|
|
||||||
|
|
||||||
|
|
|
||||||
117
doc.md
117
doc.md
|
|
@ -1,40 +1,99 @@
|
||||||
# 仓库 3D 场景
|
# ACT 真机抓取操作
|
||||||
|
|
||||||
|
SO-101 机械臂上，用 **ACT** 策略做 **本地同步推理**，并在 **Rerun** 里看三路相机与六关节曲线。
|
||||||
|
**Python、LeRobot、相机与标定已在平台配好**，你只需按下面开两个终端、跑对应脚本。
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
## 场景预览
|
<div align="center" style="line-height:1">
|
||||||
|
<a href="https://www.kimi.com" target="_blank"><img alt="Chat" src="https://img.shields.io/badge/🤖%20Chat-Kimi%20K3-ff6b6b?color=1783ff&logoColor=white"/></a>
|
||||||
|
<a href="https://www.moonshot.ai" target="_blank"><img alt="Homepage" src="https://img.shields.io/badge/Homepage-Moonshot%20AI-white?logo=Kimi&logoColor=white"/></a>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<div align="center" style="line-height: 1;">
|
||||||
|
<a href="https://huggingface.co/moonshotai" target="_blank"><img alt="Hugging Face" src="https://img.shields.io/badge/%F0%9F%A4%97%20Hugging%20Face-Moonshot%20AI-ffc107?color=ffc107&logoColor=white"/></a>
|
||||||
|
<a href="https://twitter.com/kimi_moonshot" target="_blank"><img alt="Twitter Follow" src="https://img.shields.io/badge/Twitter-Kimi.ai-white?logo=x&logoColor=white"/></a>
|
||||||
|
<a href="https://discord.gg/TYU2fdJykW" target="_blank"><img alt="Discord" src="https://img.shields.io/badge/Discord-Kimi.ai-white?logo=discord&logoColor=white"/></a>
|
||||||
|
<a href="https://modelscope.cn/organization/moonshotai" target="_blank"><img alt="ModelScope" src="https://img.shields.io/badge/ModelScope-Moonshot%20AI-white?labelColor=rgb(99%2C%2074%2C%255)"/></a>
|
||||||
|
</div>
|
||||||
|
<div align="center" style="line-height: 1;">
|
||||||
|
<a href="https://huggingface.co/moonshotai/Kimi-K3/blob/main/LICENSE"><img alt="License" src="https://img.shields.io/badge/License-Kimi_K3-f5de53?&color=f5de53"/></a>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
## 一、ACT 模型说明
|
||||||
|
|
||||||
|
| 项目 | 说明 |
|
||||||
|
|------|------|
|
||||||
|
| 算法 | **ACT**（Action Chunking with Transformers）：模仿学习；输入 **3 路相机 + 6 关节**，输出一段未来动作再逐步执行 |
|
||||||
|
| 本仓权重 | Stage A **五积木联合训练 · 350 条**（青/黄/橙/红/紫，放入 tray） |
|
||||||
|
| 权重路径 | `models/act_stageA_single5_350/pretrained_model/`（已随仓库提供，约 198MB） |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 二、仓库结构（你要用到的部分）
|
||||||
|
|
||||||
|
```text
|
||||||
|
act_rerun/
|
||||||
|
├── models/act_stageA_single5_350/pretrained_model/ # ACT 权重（已包含）
|
||||||
|
└── scripts/
|
||||||
|
├── env.sh # 环境变量（先 source）
|
||||||
|
├── run_act_sync_rerun.sh # 单色积木抓取
|
||||||
|
├── run_act_five_blocks.sh # 五积木随机顺序验证
|
||||||
|
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 三、真机验证怎么跑
|
||||||
|
|
||||||
|
|
||||||
|
### 跑 ACT 推理（先加载环境变量再推理）
|
||||||
|
|
||||||
|
```bash
|
||||||
|
conda activate lerobot_v30
|
||||||
|
source /workspace/scripts/env.sh
|
||||||
|
|
||||||
|
```
|
||||||
|
|
||||||
|
**五积木各测 1 次（随机顺序，推荐使用）：**
|
||||||
|
|
||||||
|
```bash
|
||||||
|
bash /workspace/scripts/run_act_five_blocks.sh
|
||||||
|
```
|
||||||
|
|
||||||
|
**只测一种颜色（例如青色）：**
|
||||||
|
|
||||||
|
```bash
|
||||||
|
bash /workspace/scripts/run_act_sync_rerun.sh cyan
|
||||||
|
```
|
||||||
|
|
||||||
|
可选颜色：`cyan` `yellow` `orange` `red` `purple`。
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 四、各脚本做什么
|
||||||
|
|
||||||
|
| 脚本 | 用途 |
|
||||||
|
|------|------|
|
||||||
|
| `scripts/env.sh` | 设置本仓路径、ACT 权重路径；**其它脚本前必须先 source** |
|
||||||
|
| `scripts/run_act_sync_rerun.sh` | **单色积木**同步推理 |
|
||||||
|
| `scripts/run_act_five_blocks.sh` | **五积木**按随机顺序各跑 1 episode |
|
||||||
|
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## 启动（唯一命令，无需编译）
|
## 五、Rerun 里看什么算正常
|
||||||
|
|
||||||
卡片内已包含预编译 `install/`，**直接运行即可**，不要 `colcon build`。
|
|
||||||
|
|
||||||
```bash
|
|
||||||
./gzsim_run.sh
|
|
||||||
|
|
||||||
|
|
||||||
| `./gzsim_stop.sh` | 停止仿真 |
|
|
||||||
```
|
|
||||||
|
|
||||||
**必须等到**终端出现：
|
|
||||||
|
|
||||||
```text
|
|
||||||
[autorun] GzWeb scene OK (... bytes), 9002 stable
|
|
||||||
[autorun] ✓ 全部就绪 — 现在刷新「仿真器」面板
|
|
||||||
```
|
|
||||||
|
|
||||||
打开rviz，显示实时点云数据
|
|
||||||
|
|
||||||
```bash
|
|
||||||
source install/setup.bash
|
|
||||||
ros2 launch bcr_bot rviz.launch.py
|
|
||||||
```
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
| 位置 | 内容 |
|
||||||
|
|------|------|
|
||||||
|
| 上排 | `camera/front`、`camera/side`、`camera/wrist` 实时图 |
|
||||||
|
| 下排 | 6 个关节：`sent_action` 与 `observed_qpos` 两条曲线应基本贴合 |
|
||||||
|
| 时间轴 | 必须选 **`robot_time`** |
|
||||||
|
|
||||||
|
应看到类似下图（上排三路相机，下排六关节曲线）：
|
||||||
|
|
||||||
|
|
||||||
|
|
|
||||||
Loading…
Reference in New Issue