feat: Implement ONNX model inference for BEN2, Depth Anything 3, and MoGe
- Added `ben2_onnx.py` for BEN2 ONNX inference with preprocessing. - Introduced `da3_onnx.py` for Depth Anything 3 inference, including camera projection calculations. - Created `moge_onnx.py` for MoGe-2 ONNX inference and post-processing. - Implemented shared ONNX Runtime session helpers in `onnx_runtime.py`. - Added model synchronization script `sync_models.py` for downloading and verifying ONNX models. - Developed unit tests for BEN2, Depth Anything 3, and MoGe ONNX models to ensure functionality. - Updated `vendor.lock.toml` to remove unused artifacts.
This commit is contained in:
@@ -1,31 +1,29 @@
|
||||
# 输出格式
|
||||
|
||||
## 输出格式说明
|
||||
## DA3 ONNX
|
||||
|
||||
`Output Type` 提供以下结果:
|
||||
|
||||
- Point Cloud (GLB):DA3 `glb`
|
||||
- Gaussian Splats (PLY):DA3 `gs_ply`,同时传入 `infer_gs=True`
|
||||
- Depth Preview (Images):DA3 `depth_vis`
|
||||
- Depth Data (NPZ):DA3 `mini_npz`
|
||||
- Textured Surface (GLB):SnapForm 根据 DA3 Prediction 构建并输出 `mesh.glb`
|
||||
- Point Cloud (GLB):由 depth、confidence、相机参数和原图生成彩色点云
|
||||
- Depth Preview (Images):保存深度着色预览
|
||||
- Depth Data (NPZ):保存 depth、confidence、extrinsics、intrinsics 与处理后的图像
|
||||
- Textured Surface (GLB):SnapForm 根据 Prediction 构建纹理网格
|
||||
|
||||
Point Cloud 专属设置仅在对应类型下显示,`gs_views_interval` 仅用于 Gaussian Splats。
|
||||
Textured Surface 使用 0–100% 的 Confidence Threshold 删除置信度最低的一定比例:
|
||||
DA3 Small 与 Base 的 ONNX 图不包含 Gaussian head,因此不提供 Gaussian Splats。默认推理参数为:
|
||||
|
||||
- `process_res=504`
|
||||
- `process_res_method="upper_bound_resize"`
|
||||
- `use_ray_pose=False`
|
||||
- `ref_view_strategy="saddle_balanced"`
|
||||
- `conf_thresh_percentile=40.0`
|
||||
- `num_max_points=1_000_000`
|
||||
- `show_cameras=True`
|
||||
- `export_kwargs["gs_ply"]["gs_views_interval"]=1`
|
||||
|
||||
DA3 官方 `glb` 是彩色点云与可选相机线框;`gs_ply` 是带 Gaussian 属性的 PLY;
|
||||
`depth_vis` 是原图与着色深度图的并排预览;`mini_npz` 保存深度、置信度和可用的相机参数。
|
||||
Point Cloud GLB 任务会关闭其默认附带的深度预览,使五种输出产物保持分离。
|
||||
相机外参与内参由模型的 camera decoder 输出恢复,不运行 ray-pose 求解器。
|
||||
|
||||
官方格式与限制可参阅
|
||||
[DA3 Python API 文档](https://github.com/ByteDance-Seed/Depth-Anything-3/blob/main/docs/API.md)。
|
||||
## MoGe-2 ONNX
|
||||
|
||||
MoGe-2 Debug 可保存原始 NPZ、depth/normal/mask maps、纹理 GLB 或彩色 PLY。Surface 输出由模型 points、mask 与 depth edge 在内部构建三角网格。
|
||||
|
||||
## BEN2 ONNX
|
||||
|
||||
BEN2 输出 `foreground.png`,RGB 保留原始输入,Alpha 来自 ONNX mask 并恢复到输入分辨率。
|
||||
|
||||
Reference in New Issue
Block a user