Refactor and enhance DA3 and BEN2 debug panels, introduce mesh export functionality
- Renamed DA3 and BEN2 debug panels for clarity and consistency. - Added a new function to draw DA3 input settings to reduce code duplication. - Implemented mesh export functionality, including confidence thresholding and edge filtering. - Updated export format options to include new mesh type and adjusted related properties. - Enhanced job handling to move generated inputs into job root and manage temporary directories. - Improved error handling and file management during export processes. - Added unit tests for new mesh functionality and ensured existing tests cover changes.
This commit is contained in:
@@ -45,8 +45,10 @@ SnapForm 支持视频(mp4、mkv、mov 等)和序列帧,会逐帧处理并
|
|||||||
|
|
||||||
## 任务结果保存在哪里
|
## 任务结果保存在哪里
|
||||||
|
|
||||||
任务结果保存在 `~/.snapform/jobs/` 下,每个任务一个文件夹。视频或序列帧生成的面片会引用这些文件夹
|
任务结果保存在 `~/.snapform/jobs/` 下,每个任务一个文件夹,名称统一为
|
||||||
里的序列帧,请保留对应文件夹,不要随意删除。
|
`动作-对象_日期-时间-微秒`。Packed Image 和序列帧的输入副本会直接归档到对应任务根目录,
|
||||||
|
不再使用公共 `inputs/` 文件夹;导出产物也尽量平铺。视频或序列帧生成的面片会引用任务文件夹里的
|
||||||
|
序列帧,请保留对应文件夹,不要随意删除。
|
||||||
|
|
||||||
## 常见问题
|
## 常见问题
|
||||||
|
|
||||||
|
|||||||
@@ -23,13 +23,15 @@
|
|||||||
- 视频按顺序截取前若干帧,可设置参与重建的最大帧数
|
- 视频按顺序截取前若干帧,可设置参与重建的最大帧数
|
||||||
- `Host > Models` 默认折叠,展开后可逐项查看和下载所有可用模型
|
- `Host > Models` 默认折叠,展开后可逐项查看和下载所有可用模型
|
||||||
- Runtime 已就绪时不额外占用面板空间;模型列表每个模型只占一行
|
- Runtime 已就绪时不额外占用面板空间;模型列表每个模型只占一行
|
||||||
- `Debug` 默认折叠,内部的 DA3 与 BEN2 测试项也分别默认折叠
|
- `Debug` 默认折叠,内部的 `DA3 Output Test` 与 `BEN2 Removal Test` 也分别默认折叠
|
||||||
- 使用一个导出类型下拉菜单和一个 `Export` 按钮
|
- 使用一个输出类型下拉菜单和一个 `Generate Output` 按钮
|
||||||
- 只显示当前格式的专属参数;GLB 与 PLY 设置不会同时出现
|
- 只显示当前格式的专属参数;GLB 与 PLY 设置不会同时出现
|
||||||
- Debug 的模型选项按格式过滤;PLY 只提供两个带 Gaussian head 的 Giant 模型
|
- Debug 的模型选项按格式过滤;PLY 只提供两个带 Gaussian head 的 Giant 模型
|
||||||
- Debug 与 Convert to Depth Plane 的模型选项直接显示下载状态;未下载模型带
|
- Debug 与 Convert to Depth Plane 的模型选项直接显示下载状态;未下载模型带
|
||||||
`(Missing)`,已下载模型保持原名
|
`(Missing)`,已下载模型保持原名
|
||||||
- 分别调用 DA3 的 `glb`、`gs_ply`、`depth_vis` 和 `mini_npz` exporter
|
- 分别调用 DA3 的 `glb`、`gs_ply`、`depth_vis` 和 `mini_npz` exporter
|
||||||
|
- `Textured Mesh (GLB)` 直接使用 DA3 Prediction 中的深度、置信度和相机参数,把相邻像素
|
||||||
|
组成三角面并写入 `mesh.glb`;每个输入帧生成一张独立纹理网格,不做多视图融合
|
||||||
- Image Empty 右键菜单可直接运行 DA3,根据 `mini_npz` 深度生成面片
|
- Image Empty 右键菜单可直接运行 DA3,根据 `mini_npz` 深度生成面片
|
||||||
- Image Empty 右键菜单可直接运行 BEN2,保留原图并换用新建的透明背景图片
|
- Image Empty 右键菜单可直接运行 BEN2,保留原图并换用新建的透明背景图片
|
||||||
- Image Empty 支持视频与序列帧输入:`Convert to Depth Plane` 逐帧估计深度,
|
- Image Empty 支持视频与序列帧输入:`Convert to Depth Plane` 逐帧估计深度,
|
||||||
@@ -39,7 +41,7 @@
|
|||||||
- 使用浮点深度贴图、Simple Subdivision、Displace Modifier 和原图 Bump
|
- 使用浮点深度贴图、Simple Subdivision、Displace Modifier 和原图 Bump
|
||||||
构建立体面片
|
构建立体面片
|
||||||
- 转换对话框可选择推理模式、处理分辨率和面片网格细分
|
- 转换对话框可选择推理模式、处理分辨率和面片网格细分
|
||||||
- Debug 的 DA3 导出或 BEN2 背景剔除完成后只打开对应 job 文件夹,不向当前 Blender 文件添加任何数据
|
- Debug 的 DA3 输出或 BEN2 背景剔除完成后只打开对应 job 文件夹,不修改当前场景
|
||||||
- 完整保留每次任务生成的所有官方文件
|
- 完整保留每次任务生成的所有官方文件
|
||||||
- 安装和模型下载使用后台子进程;推理提交给常驻 AI Host,状态栏实时显示阶段、百分比进度条与取消按钮
|
- 安装和模型下载使用后台子进程;推理提交给常驻 AI Host,状态栏实时显示阶段、百分比进度条与取消按钮
|
||||||
- 后台任务运行时,Install、Download 和 Export 按钮保持原位置并变灰,任务完成后恢复
|
- 后台任务运行时,Install、Download 和 Export 按钮保持原位置并变灰,任务完成后恢复
|
||||||
@@ -69,10 +71,10 @@
|
|||||||
自行探测回退到 HF 镜像;需要手动指定镜像时可通过 `PIP_INDEX_URL` 环境变量覆盖。
|
自行探测回退到 HF 镜像;需要手动指定镜像时可通过 `PIP_INDEX_URL` 环境变量覆盖。
|
||||||
5. 在 3D View 按 `N`,打开 `SnapForm` 页签。
|
5. 在 3D View 按 `N`,打开 `SnapForm` 页签。
|
||||||
6. 如需其他模型,展开 `Host > Models`,找到模型并点击该行的 `Download`。
|
6. 如需其他模型,展开 `Host > Models`,找到模型并点击该行的 `Download`。
|
||||||
7. 展开默认收起的 `Debug`,选择 `Export Format` 和本次执行使用的模型。
|
7. 展开默认收起的 `Debug`,选择 `Output Type` 和本次执行使用的模型。
|
||||||
8. 选择一张图片或一个视频文件。视频可设置从头截取的帧数,然后点击 `Export`。
|
8. 选择一张图片或一个视频文件。视频可设置从头截取的帧数,然后点击 `Generate Output`。
|
||||||
9. 测试背景剔除时,确认 `Host > Models` 中的 `BEN2 Base` 已就绪,然后在 `Debug` 的
|
9. 测试背景剔除时,确认 `Host > Models` 中的 `BEN2 Base` 已就绪,然后在 `Debug` 的
|
||||||
`BEN2 Background Removal` 区域选择单张图片并点击 `Remove Background`。
|
`BEN2 Removal Test` 区域选择单张图片并点击 `Remove Background`。
|
||||||
完成后会打开 job 文件夹,其中的 `foreground.png` 带透明 Alpha;状态栏显示
|
完成后会打开 job 文件夹,其中的 `foreground.png` 带透明 Alpha;状态栏显示
|
||||||
BEN2 推理、模型加载与任务总耗时。当前测试固定关闭较慢的 foreground refinement。
|
BEN2 推理、模型加载与任务总耗时。当前测试固定关闭较慢的 foreground refinement。
|
||||||
|
|
||||||
@@ -218,33 +220,52 @@ Image Empty 使用 Packed Image 时,SnapForm 会通过 Blender 写出临时 PN
|
|||||||
`DA3 Nested Giant Large 1.1`。Small、Base 和 Large 没有 Gaussian head,无法由
|
`DA3 Nested Giant Large 1.1`。Small、Base 和 Large 没有 Gaussian head,无法由
|
||||||
DA3 生成 `gs_ply`。
|
DA3 生成 `gs_ply`。
|
||||||
|
|
||||||
每个任务在 `~/.snapform/jobs/` 下用独立文件夹包裹(命名格式为 `任务类型_时间号`,例如
|
每个任务在 `~/.snapform/jobs/` 下用独立文件夹包裹。文件夹统一采用
|
||||||
`depth-plane_20260807-153000`),日志、状态与输出产物都放在该文件夹内:
|
`动作-对象_日期-时间-微秒`,每种任务都有唯一名称,例如
|
||||||
|
`generate-depth-plane_20260807-153000-123456`。微秒用于避免同一秒启动的任务共用目录。
|
||||||
|
|
||||||
|
Blender 为 Packed Image 或序列帧生成的输入副本也归档到对应 job 根目录:单图命名为
|
||||||
|
`input.png`,序列帧命名为 `frame.0001.*`。它们以前暂存在 `jobs/inputs/`,现在不再创建这个
|
||||||
|
公共中转文件夹。普通磁盘文件和视频仍直接读取原文件,不额外复制。
|
||||||
|
|
||||||
|
日志、运行中的状态文件、归档输入与输出产物均直接放在 job 根目录。各类任务名称为:
|
||||||
|
|
||||||
|
- `install-runtime`
|
||||||
|
- `download-model`
|
||||||
|
- `export-reconstruction`
|
||||||
|
- `generate-depth-plane`
|
||||||
|
- `remove-background`
|
||||||
|
- `remove-image-background`
|
||||||
|
- `generate-original-depth-plane`
|
||||||
|
- `extract-original-frames`
|
||||||
|
|
||||||
|
DA3 官方 exporter 产生的 `gs_ply/`、`depth_vis/`、`exports/mini_npz/` 会在完成后自动平铺,
|
||||||
|
不会保留只含一层产物的子目录:
|
||||||
|
|
||||||
```text
|
```text
|
||||||
reconstruct_20260807-153000/
|
export-reconstruction_20260807-153000-123456/
|
||||||
├── reconstruct_20260807-153000.log
|
├── export-reconstruction_20260807-153000-123456.log
|
||||||
├── reconstruct_20260807-153000.json
|
├── export-reconstruction_20260807-153000-123456.json
|
||||||
└── scene.glb
|
└── scene.glb
|
||||||
|
|
||||||
reconstruct_20260807-153000/
|
export-reconstruction_20260807-153000-123456/
|
||||||
├── reconstruct_20260807-153000.log
|
├── export-reconstruction_20260807-153000-123456.log
|
||||||
├── reconstruct_20260807-153000.json
|
├── export-reconstruction_20260807-153000-123456.json
|
||||||
└── gs_ply/*.ply
|
└── *.ply
|
||||||
|
|
||||||
reconstruct_20260807-153000/
|
export-reconstruction_20260807-153000-123456/
|
||||||
├── reconstruct_20260807-153000.log
|
├── export-reconstruction_20260807-153000-123456.log
|
||||||
├── reconstruct_20260807-153000.json
|
├── export-reconstruction_20260807-153000-123456.json
|
||||||
└── depth_vis/*.jpg
|
└── *.jpg
|
||||||
|
|
||||||
reconstruct_20260807-153000/
|
export-reconstruction_20260807-153000-123456/
|
||||||
├── reconstruct_20260807-153000.log
|
├── export-reconstruction_20260807-153000-123456.log
|
||||||
├── reconstruct_20260807-153000.json
|
├── export-reconstruction_20260807-153000-123456.json
|
||||||
└── results.npz
|
└── results.npz
|
||||||
|
|
||||||
remove-background_20260807-153000/
|
remove-background_20260807-153000-123456/
|
||||||
├── remove-background_20260807-153000.log
|
├── remove-background_20260807-153000-123456.log
|
||||||
├── remove-background_20260807-153000.json
|
├── remove-background_20260807-153000-123456.json
|
||||||
├── foreground.png 单图结果
|
├── foreground.png 单图结果
|
||||||
├── foreground.0001.png… 视频/序列帧结果
|
├── foreground.0001.png… 视频/序列帧结果
|
||||||
├── frame.0001.jpg… 视频/序列帧的源帧副本
|
├── frame.0001.jpg… 视频/序列帧的源帧副本
|
||||||
@@ -256,15 +277,16 @@ CC BY-NC 4.0,仅限非商业用途。
|
|||||||
|
|
||||||
## 输出格式说明
|
## 输出格式说明
|
||||||
|
|
||||||
`Export Format` 下拉菜单分别把以下原始 `export_format` 传给 DA3:
|
`Output Type` 提供以下结果:
|
||||||
|
|
||||||
- GLB:`glb`
|
- Point Cloud (GLB):DA3 `glb`
|
||||||
- PLY:`gs_ply`,同时传入 `infer_gs=True`
|
- Gaussian Splats (PLY):DA3 `gs_ply`,同时传入 `infer_gs=True`
|
||||||
- Depth Images:`depth_vis`
|
- Depth Preview (Images):DA3 `depth_vis`
|
||||||
- Mini NPZ:`mini_npz`
|
- Depth Data (NPZ):DA3 `mini_npz`
|
||||||
|
- Textured Mesh (GLB):SnapForm 根据 DA3 Prediction 构建并输出 `mesh.glb`
|
||||||
|
|
||||||
界面使用 DA3 API 的参数名、选项和默认值。GLB 专属设置仅在选择 `GLB` 时显示,
|
Point Cloud 专属设置仅在对应类型下显示,`gs_views_interval` 仅用于 Gaussian Splats。
|
||||||
`gs_views_interval` 仅在选择 `PLY (3DGS)` 时显示:
|
Textured Mesh 使用 0–100% 的 Confidence Threshold 删除置信度最低的一定比例:
|
||||||
|
|
||||||
- `process_res=504`
|
- `process_res=504`
|
||||||
- `process_res_method="upper_bound_resize"`
|
- `process_res_method="upper_bound_resize"`
|
||||||
@@ -277,7 +299,7 @@ CC BY-NC 4.0,仅限非商业用途。
|
|||||||
|
|
||||||
DA3 官方 `glb` 是彩色点云与可选相机线框;`gs_ply` 是带 Gaussian 属性的 PLY;
|
DA3 官方 `glb` 是彩色点云与可选相机线框;`gs_ply` 是带 Gaussian 属性的 PLY;
|
||||||
`depth_vis` 是原图与着色深度图的并排预览;`mini_npz` 保存深度、置信度和可用的相机参数。
|
`depth_vis` 是原图与着色深度图的并排预览;`mini_npz` 保存深度、置信度和可用的相机参数。
|
||||||
GLB 任务会关闭其默认附带的深度预览,使四种 job 产物保持分离。
|
Point Cloud GLB 任务会关闭其默认附带的深度预览,使五种输出产物保持分离。
|
||||||
|
|
||||||
官方格式与限制可参阅
|
官方格式与限制可参阅
|
||||||
[DA3 Python API 文档](https://github.com/ByteDance-Seed/Depth-Anything-3/blob/main/docs/API.md)。
|
[DA3 Python API 文档](https://github.com/ByteDance-Seed/Depth-Anything-3/blob/main/docs/API.md)。
|
||||||
|
|||||||
@@ -17,8 +17,8 @@ from .operators import (
|
|||||||
close_active_job,
|
close_active_job,
|
||||||
)
|
)
|
||||||
from .panel import (
|
from .panel import (
|
||||||
Ben2DebugPanel,
|
Ben2RemovalDebugPanel,
|
||||||
Da3DebugPanel,
|
Da3OutputDebugPanel,
|
||||||
DebugPanel,
|
DebugPanel,
|
||||||
HostPanel,
|
HostPanel,
|
||||||
ModelsPanel,
|
ModelsPanel,
|
||||||
@@ -42,8 +42,8 @@ CLASSES = (
|
|||||||
HostPanel,
|
HostPanel,
|
||||||
ModelsPanel,
|
ModelsPanel,
|
||||||
DebugPanel,
|
DebugPanel,
|
||||||
Da3DebugPanel,
|
Da3OutputDebugPanel,
|
||||||
Ben2DebugPanel,
|
Ben2RemovalDebugPanel,
|
||||||
)
|
)
|
||||||
|
|
||||||
|
|
||||||
|
|||||||
@@ -122,6 +122,7 @@ def close_active_job():
|
|||||||
shutil.rmtree(temporary_path, ignore_errors=True)
|
shutil.rmtree(temporary_path, ignore_errors=True)
|
||||||
else:
|
else:
|
||||||
temporary_path.unlink(missing_ok=True)
|
temporary_path.unlink(missing_ok=True)
|
||||||
|
temporary_path.parent.rmdir()
|
||||||
except OSError:
|
except OSError:
|
||||||
pass
|
pass
|
||||||
_active_job = None
|
_active_job = None
|
||||||
@@ -221,6 +222,12 @@ class RunJob(bpy.types.Operator):
|
|||||||
self.report({"ERROR"}, str(error))
|
self.report({"ERROR"}, str(error))
|
||||||
return {"CANCELLED"}
|
return {"CANCELLED"}
|
||||||
|
|
||||||
|
export_format = ""
|
||||||
|
if self.job_type == RECONSTRUCTION_JOB:
|
||||||
|
export_format = settings.export_format
|
||||||
|
elif self.job_type == IMAGE_DEPTH_JOB:
|
||||||
|
export_format = "mini_npz"
|
||||||
|
|
||||||
job = ActiveJob(
|
job = ActiveJob(
|
||||||
kind=self.job_type,
|
kind=self.job_type,
|
||||||
process=None,
|
process=None,
|
||||||
@@ -228,11 +235,7 @@ class RunJob(bpy.types.Operator):
|
|||||||
log_path=log_path,
|
log_path=log_path,
|
||||||
status_path=paths["status"],
|
status_path=paths["status"],
|
||||||
result_directory=paths["result"],
|
result_directory=paths["result"],
|
||||||
export_format=(
|
export_format=export_format,
|
||||||
settings.export_format if self.job_type == RECONSTRUCTION_JOB
|
|
||||||
else "mini_npz" if self.job_type == IMAGE_DEPTH_JOB
|
|
||||||
else ""
|
|
||||||
),
|
|
||||||
source_object_name=self.snapform_source_object_name,
|
source_object_name=self.snapform_source_object_name,
|
||||||
depth_data_name=self.snapform_depth_data_name,
|
depth_data_name=self.snapform_depth_data_name,
|
||||||
process_res=self.snapform_process_res,
|
process_res=self.snapform_process_res,
|
||||||
@@ -310,6 +313,21 @@ class RunJob(bpy.types.Operator):
|
|||||||
|
|
||||||
def _prepare_command(self, _context, settings):
|
def _prepare_command(self, _context, settings):
|
||||||
paths = runtime.new_job_paths(self.job_type)
|
paths = runtime.new_job_paths(self.job_type)
|
||||||
|
if (
|
||||||
|
getattr(self, "snapform_delete_input", False)
|
||||||
|
and self.job_type in {
|
||||||
|
IMAGE_DEPTH_JOB,
|
||||||
|
IMAGE_BACKGROUND_JOB,
|
||||||
|
ORIGINAL_BACKGROUND_JOB,
|
||||||
|
ORIGINAL_FRAMES_JOB,
|
||||||
|
}
|
||||||
|
):
|
||||||
|
archived_input = runtime.move_input_into_job(
|
||||||
|
self.snapform_input_path,
|
||||||
|
paths["result"],
|
||||||
|
)
|
||||||
|
self.snapform_input_path = str(archived_input)
|
||||||
|
self.snapform_delete_input = False
|
||||||
background_depth_distance = float(
|
background_depth_distance = float(
|
||||||
getattr(self, "snapform_background_depth_distance", 0.0)
|
getattr(self, "snapform_background_depth_distance", 0.0)
|
||||||
)
|
)
|
||||||
@@ -389,13 +407,14 @@ class RunJob(bpy.types.Operator):
|
|||||||
),
|
),
|
||||||
paths,
|
paths,
|
||||||
)
|
)
|
||||||
if self.job_type not in {RECONSTRUCTION_JOB, IMAGE_DEPTH_JOB}:
|
if self.job_type not in {
|
||||||
|
RECONSTRUCTION_JOB,
|
||||||
|
IMAGE_DEPTH_JOB,
|
||||||
|
}:
|
||||||
raise RuntimeError(f"Unsupported task type: {self.job_type}")
|
raise RuntimeError(f"Unsupported task type: {self.job_type}")
|
||||||
export_format = (
|
export_format = settings.export_format
|
||||||
"mini_npz"
|
if self.job_type == IMAGE_DEPTH_JOB:
|
||||||
if self.job_type == IMAGE_DEPTH_JOB
|
export_format = "mini_npz"
|
||||||
else settings.export_format
|
|
||||||
)
|
|
||||||
use_ray_pose = None
|
use_ray_pose = None
|
||||||
process_res = None
|
process_res = None
|
||||||
model_key = selected_model_key(settings, export_format)
|
model_key = selected_model_key(settings, export_format)
|
||||||
@@ -614,7 +633,10 @@ class RunJob(bpy.types.Operator):
|
|||||||
self._redraw_ui(context, force=True)
|
self._redraw_ui(context, force=True)
|
||||||
return {"CANCELLED"}
|
return {"CANCELLED"}
|
||||||
|
|
||||||
if job.kind in {RECONSTRUCTION_JOB, BACKGROUND_REMOVAL_JOB}:
|
if job.kind in {
|
||||||
|
RECONSTRUCTION_JOB,
|
||||||
|
BACKGROUND_REMOVAL_JOB,
|
||||||
|
}:
|
||||||
try:
|
try:
|
||||||
open_job_directory(job.result_directory)
|
open_job_directory(job.result_directory)
|
||||||
except Exception as open_error:
|
except Exception as open_error:
|
||||||
|
|||||||
@@ -160,26 +160,8 @@ class DebugPanel(bpy.types.Panel):
|
|||||||
layout.prop(settings, "device")
|
layout.prop(settings, "device")
|
||||||
|
|
||||||
|
|
||||||
class Da3DebugPanel(bpy.types.Panel):
|
def draw_da3_input_settings(layout, settings, model_property):
|
||||||
bl_label = "DA3 Reconstruction"
|
|
||||||
bl_idname = "SNAPFORM_PT_debug_da3"
|
|
||||||
bl_parent_id = DebugPanel.bl_idname
|
|
||||||
bl_space_type = "VIEW_3D"
|
|
||||||
bl_region_type = "UI"
|
|
||||||
bl_category = "SnapForm"
|
|
||||||
bl_options = {"DEFAULT_CLOSED"}
|
|
||||||
bl_order = 0
|
|
||||||
|
|
||||||
def draw(self, context):
|
|
||||||
layout = self.layout
|
|
||||||
settings = context.scene.snapform_settings
|
|
||||||
|
|
||||||
layout.prop(settings, "export_format")
|
|
||||||
model_property = (
|
|
||||||
"gs_model" if settings.export_format == "gs_ply" else "model"
|
|
||||||
)
|
|
||||||
layout.prop(settings, model_property)
|
layout.prop(settings, model_property)
|
||||||
|
|
||||||
layout.prop(settings, "input_path")
|
layout.prop(settings, "input_path")
|
||||||
input_path = bpy.path.abspath(settings.input_path)
|
input_path = bpy.path.abspath(settings.input_path)
|
||||||
if input_path.lower().endswith(
|
if input_path.lower().endswith(
|
||||||
@@ -201,6 +183,27 @@ class Da3DebugPanel(bpy.types.Panel):
|
|||||||
layout.prop(settings, "use_ray_pose")
|
layout.prop(settings, "use_ray_pose")
|
||||||
layout.prop(settings, "ref_view_strategy")
|
layout.prop(settings, "ref_view_strategy")
|
||||||
|
|
||||||
|
|
||||||
|
class Da3OutputDebugPanel(bpy.types.Panel):
|
||||||
|
bl_label = "DA3 Output Test"
|
||||||
|
bl_idname = "SNAPFORM_PT_debug_da3_output"
|
||||||
|
bl_parent_id = DebugPanel.bl_idname
|
||||||
|
bl_space_type = "VIEW_3D"
|
||||||
|
bl_region_type = "UI"
|
||||||
|
bl_category = "SnapForm"
|
||||||
|
bl_options = {"DEFAULT_CLOSED"}
|
||||||
|
bl_order = 0
|
||||||
|
|
||||||
|
def draw(self, context):
|
||||||
|
layout = self.layout
|
||||||
|
settings = context.scene.snapform_settings
|
||||||
|
|
||||||
|
layout.prop(settings, "export_format")
|
||||||
|
model_property = (
|
||||||
|
"gs_model" if settings.export_format == "gs_ply" else "model"
|
||||||
|
)
|
||||||
|
draw_da3_input_settings(layout, settings, model_property)
|
||||||
|
|
||||||
if settings.export_format == "glb":
|
if settings.export_format == "glb":
|
||||||
glb_box = layout.box()
|
glb_box = layout.box()
|
||||||
glb_box.label(text="GLB Settings")
|
glb_box.label(text="GLB Settings")
|
||||||
@@ -211,7 +214,12 @@ class Da3DebugPanel(bpy.types.Panel):
|
|||||||
ply_box = layout.box()
|
ply_box = layout.box()
|
||||||
ply_box.label(text="PLY (3DGS) Settings")
|
ply_box.label(text="PLY (3DGS) Settings")
|
||||||
ply_box.prop(settings, "gs_views_interval")
|
ply_box.prop(settings, "gs_views_interval")
|
||||||
|
elif settings.export_format == "mesh":
|
||||||
|
mesh_box = layout.box()
|
||||||
|
mesh_box.label(text="Mesh Settings")
|
||||||
|
mesh_box.prop(settings, "conf_thresh_percentile")
|
||||||
|
mesh_box.prop(settings, "mesh_filter_edges")
|
||||||
|
mesh_box.prop(settings, "mesh_max_edge_length")
|
||||||
export_row = layout.row()
|
export_row = layout.row()
|
||||||
export_row.enabled = (
|
export_row.enabled = (
|
||||||
active_job() is None
|
active_job() is None
|
||||||
@@ -220,15 +228,15 @@ class Da3DebugPanel(bpy.types.Panel):
|
|||||||
)
|
)
|
||||||
operator = export_row.operator(
|
operator = export_row.operator(
|
||||||
"snapform.run_job",
|
"snapform.run_job",
|
||||||
text="Export",
|
text="Generate Output",
|
||||||
icon="EXPORT",
|
icon="EXPORT",
|
||||||
)
|
)
|
||||||
operator.job_type = "RECONSTRUCT"
|
operator.job_type = "RECONSTRUCT"
|
||||||
|
|
||||||
|
|
||||||
class Ben2DebugPanel(bpy.types.Panel):
|
class Ben2RemovalDebugPanel(bpy.types.Panel):
|
||||||
bl_label = "BEN2 Background Removal"
|
bl_label = "BEN2 Removal Test"
|
||||||
bl_idname = "SNAPFORM_PT_debug_ben2"
|
bl_idname = "SNAPFORM_PT_debug_ben2_removal"
|
||||||
bl_parent_id = DebugPanel.bl_idname
|
bl_parent_id = DebugPanel.bl_idname
|
||||||
bl_space_type = "VIEW_3D"
|
bl_space_type = "VIEW_3D"
|
||||||
bl_region_type = "UI"
|
bl_region_type = "UI"
|
||||||
|
|||||||
@@ -66,8 +66,8 @@ class SnapFormSettings(bpy.types.PropertyGroup):
|
|||||||
default="upper_bound_resize",
|
default="upper_bound_resize",
|
||||||
)
|
)
|
||||||
conf_thresh_percentile: bpy.props.FloatProperty(
|
conf_thresh_percentile: bpy.props.FloatProperty(
|
||||||
name="Conf Thresh Percentile",
|
name="Confidence Threshold",
|
||||||
description="DA3 GLB confidence threshold percentile",
|
description="Remove the lowest-confidence percentile of points or mesh faces",
|
||||||
default=40.0,
|
default=40.0,
|
||||||
min=0.0,
|
min=0.0,
|
||||||
max=100.0,
|
max=100.0,
|
||||||
@@ -110,24 +110,52 @@ class SnapFormSettings(bpy.types.PropertyGroup):
|
|||||||
default="saddle_balanced",
|
default="saddle_balanced",
|
||||||
)
|
)
|
||||||
export_format: bpy.props.EnumProperty(
|
export_format: bpy.props.EnumProperty(
|
||||||
name="Export Format",
|
name="Output Type",
|
||||||
description="Official DA3 export format",
|
description="Result generated by the DA3 debug task",
|
||||||
items=(
|
items=(
|
||||||
("glb", "GLB", "Export a colored point cloud as GLB"),
|
(
|
||||||
("gs_ply", "PLY (3DGS)", "Export Gaussian splats as PLY"),
|
"glb",
|
||||||
|
"Point Cloud (GLB)",
|
||||||
|
"Generate a colored point cloud and optional cameras",
|
||||||
|
),
|
||||||
|
(
|
||||||
|
"gs_ply",
|
||||||
|
"Gaussian Splats (PLY)",
|
||||||
|
"Generate a 3D Gaussian Splatting PLY",
|
||||||
|
),
|
||||||
(
|
(
|
||||||
"depth_vis",
|
"depth_vis",
|
||||||
"Depth Images",
|
"Depth Preview (Images)",
|
||||||
"Export image and depth visualization pairs",
|
"Generate image and colored-depth preview pairs",
|
||||||
),
|
),
|
||||||
(
|
(
|
||||||
"mini_npz",
|
"mini_npz",
|
||||||
"Mini NPZ",
|
"Depth Data (NPZ)",
|
||||||
"Export compact depth and camera data",
|
"Generate compact depth, confidence, and camera data",
|
||||||
|
),
|
||||||
|
(
|
||||||
|
"mesh",
|
||||||
|
"Textured Mesh (GLB)",
|
||||||
|
"Generate one filtered textured depth mesh per input image",
|
||||||
),
|
),
|
||||||
),
|
),
|
||||||
default="glb",
|
default="glb",
|
||||||
)
|
)
|
||||||
|
mesh_filter_edges: bpy.props.BoolProperty(
|
||||||
|
name="Filter Depth Edges",
|
||||||
|
description="Remove faces around sharp depth transitions",
|
||||||
|
default=True,
|
||||||
|
)
|
||||||
|
mesh_max_edge_length: bpy.props.FloatProperty(
|
||||||
|
name="Max Edge Length",
|
||||||
|
description=(
|
||||||
|
"Remove faces with an edge longer than this scene distance; "
|
||||||
|
"zero disables the test"
|
||||||
|
),
|
||||||
|
default=0.1,
|
||||||
|
min=0.0,
|
||||||
|
soft_max=2.0,
|
||||||
|
)
|
||||||
gs_views_interval: bpy.props.IntProperty(
|
gs_views_interval: bpy.props.IntProperty(
|
||||||
name="GS Views Interval",
|
name="GS Views Interval",
|
||||||
description="Export 3D Gaussians every N input views",
|
description="Export 3D Gaussians every N input views",
|
||||||
|
|||||||
@@ -6,6 +6,7 @@ import shutil
|
|||||||
import socket
|
import socket
|
||||||
import subprocess
|
import subprocess
|
||||||
import sys
|
import sys
|
||||||
|
import tempfile
|
||||||
import threading
|
import threading
|
||||||
import time
|
import time
|
||||||
import uuid
|
import uuid
|
||||||
@@ -34,9 +35,12 @@ DEFAULT_STORAGE_ROOT = Path.home() / ".snapform"
|
|||||||
JOB_FOLDER_NAMES = {
|
JOB_FOLDER_NAMES = {
|
||||||
"INSTALL_RUNTIME": "install-runtime",
|
"INSTALL_RUNTIME": "install-runtime",
|
||||||
"DOWNLOAD_MODEL": "download-model",
|
"DOWNLOAD_MODEL": "download-model",
|
||||||
"IMAGE_DEPTH": "depth-plane",
|
"RECONSTRUCT": "export-reconstruction",
|
||||||
"IMAGE_BACKGROUND": "remove-background",
|
"IMAGE_DEPTH": "generate-depth-plane",
|
||||||
"ORIGINAL_FRAMES": "depth-plane",
|
"REMOVE_BACKGROUND": "remove-background",
|
||||||
|
"IMAGE_BACKGROUND": "remove-image-background",
|
||||||
|
"ORIGINAL_BACKGROUND": "generate-original-depth-plane",
|
||||||
|
"ORIGINAL_FRAMES": "extract-original-frames",
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
||||||
@@ -196,15 +200,37 @@ def jobs_directory():
|
|||||||
|
|
||||||
|
|
||||||
def temporary_input_path():
|
def temporary_input_path():
|
||||||
path = jobs_directory() / "inputs"
|
directory = Path(tempfile.mkdtemp(prefix="snapform-input-"))
|
||||||
path.mkdir(parents=True, exist_ok=True)
|
return directory / "input.png"
|
||||||
return path / f"{uuid.uuid4().hex}.png"
|
|
||||||
|
|
||||||
|
|
||||||
def temporary_input_directory():
|
def temporary_input_directory():
|
||||||
path = jobs_directory() / "inputs" / f"{uuid.uuid4().hex}"
|
return Path(tempfile.mkdtemp(prefix="snapform-input-"))
|
||||||
path.mkdir(parents=True, exist_ok=True)
|
|
||||||
return path
|
|
||||||
|
def move_input_into_job(input_path, job_directory):
|
||||||
|
"""Move a Blender-generated input copy into the flat job directory."""
|
||||||
|
input_path = Path(input_path)
|
||||||
|
job_directory = Path(job_directory)
|
||||||
|
if input_path.is_file():
|
||||||
|
destination = job_directory / "input.png"
|
||||||
|
input_path.replace(destination)
|
||||||
|
try:
|
||||||
|
input_path.parent.rmdir()
|
||||||
|
except OSError:
|
||||||
|
pass
|
||||||
|
return destination
|
||||||
|
if not input_path.is_dir():
|
||||||
|
raise ValueError(f"Temporary input does not exist: {input_path}")
|
||||||
|
for source in input_path.iterdir():
|
||||||
|
if not source.is_file():
|
||||||
|
raise ValueError(f"Temporary input contains a directory: {source}")
|
||||||
|
destination = job_directory / source.name
|
||||||
|
if destination.exists():
|
||||||
|
raise ValueError(f"Job input filename already exists: {source.name}")
|
||||||
|
source.replace(destination)
|
||||||
|
input_path.rmdir()
|
||||||
|
return job_directory
|
||||||
|
|
||||||
|
|
||||||
def job_folder_name(job_type):
|
def job_folder_name(job_type):
|
||||||
@@ -213,7 +239,7 @@ def job_folder_name(job_type):
|
|||||||
|
|
||||||
|
|
||||||
def new_job_paths(job_type):
|
def new_job_paths(job_type):
|
||||||
timestamp = datetime.now().strftime("%Y%m%d-%H%M%S")
|
timestamp = datetime.now().strftime("%Y%m%d-%H%M%S-%f")
|
||||||
stem = f"{job_folder_name(job_type)}_{timestamp}"
|
stem = f"{job_folder_name(job_type)}_{timestamp}"
|
||||||
directory = jobs_directory() / stem
|
directory = jobs_directory() / stem
|
||||||
directory.mkdir(parents=True, exist_ok=True)
|
directory.mkdir(parents=True, exist_ok=True)
|
||||||
@@ -824,6 +850,12 @@ def reconstruct_request(
|
|||||||
"ref_view_strategy": settings.ref_view_strategy,
|
"ref_view_strategy": settings.ref_view_strategy,
|
||||||
"remove_background_model_dir": "",
|
"remove_background_model_dir": "",
|
||||||
"background_depth_distance": float(background_depth_distance),
|
"background_depth_distance": float(background_depth_distance),
|
||||||
|
"mesh_filter_edges": bool(
|
||||||
|
getattr(settings, "mesh_filter_edges", True)
|
||||||
|
),
|
||||||
|
"mesh_max_edge_length": float(
|
||||||
|
getattr(settings, "mesh_max_edge_length", 0.1)
|
||||||
|
),
|
||||||
}
|
}
|
||||||
if remove_background or background_depth_distance > 0.0:
|
if remove_background or background_depth_distance > 0.0:
|
||||||
parameters["remove_background_model_dir"] = str(
|
parameters["remove_background_model_dir"] = str(
|
||||||
|
|||||||
@@ -24,7 +24,7 @@ VIDEO_SUFFIXES = {
|
|||||||
".mpeg",
|
".mpeg",
|
||||||
}
|
}
|
||||||
SCENE_FILENAME = "scene.glb"
|
SCENE_FILENAME = "scene.glb"
|
||||||
EXPORT_FORMATS = ("glb", "gs_ply", "depth_vis", "mini_npz")
|
EXPORT_FORMATS = ("glb", "gs_ply", "depth_vis", "mini_npz", "mesh")
|
||||||
HF_ENDPOINT = "https://huggingface.co"
|
HF_ENDPOINT = "https://huggingface.co"
|
||||||
MODELSCOPE_ENDPOINT = "https://modelscope.cn"
|
MODELSCOPE_ENDPOINT = "https://modelscope.cn"
|
||||||
MODELSCOPE_REVISION = "master"
|
MODELSCOPE_REVISION = "master"
|
||||||
@@ -340,10 +340,211 @@ def copy_frames(images, output_directory):
|
|||||||
return
|
return
|
||||||
for index, image_path in enumerate(images):
|
for index, image_path in enumerate(images):
|
||||||
suffix = Path(image_path).suffix.lower() or ".jpg"
|
suffix = Path(image_path).suffix.lower() or ".jpg"
|
||||||
shutil.copy2(
|
destination = (
|
||||||
image_path,
|
Path(output_directory) / f"frame.{index + 1:04d}{suffix}"
|
||||||
Path(output_directory) / f"frame.{index + 1:04d}{suffix}",
|
|
||||||
)
|
)
|
||||||
|
if Path(image_path).resolve() == destination.resolve():
|
||||||
|
continue
|
||||||
|
shutil.copy2(image_path, destination)
|
||||||
|
|
||||||
|
|
||||||
|
def unproject_depth_frame(depth, intrinsics, extrinsics):
|
||||||
|
import numpy as np
|
||||||
|
|
||||||
|
height, width = depth.shape
|
||||||
|
x, y = np.meshgrid(
|
||||||
|
np.arange(width, dtype=np.float32),
|
||||||
|
np.arange(height, dtype=np.float32),
|
||||||
|
)
|
||||||
|
pixels = np.stack((x, y, np.ones_like(x)), axis=-1).reshape(-1, 3)
|
||||||
|
rays = (np.linalg.inv(intrinsics) @ pixels.T).T
|
||||||
|
camera_points = rays * depth.reshape(-1, 1)
|
||||||
|
camera_points = np.column_stack(
|
||||||
|
(camera_points, np.ones(camera_points.shape[0], dtype=np.float32))
|
||||||
|
)
|
||||||
|
world_to_camera = np.eye(4, dtype=np.float32)
|
||||||
|
world_to_camera[: extrinsics.shape[0], :4] = extrinsics
|
||||||
|
world_points = (np.linalg.inv(world_to_camera) @ camera_points.T).T
|
||||||
|
return world_points[:, :3].reshape(height, width, 3)
|
||||||
|
|
||||||
|
|
||||||
|
def depth_edge_mask(depth, threshold=12.0 / 255.0):
|
||||||
|
import numpy as np
|
||||||
|
|
||||||
|
padded = np.pad(depth.astype(np.float32), 1, mode="edge")
|
||||||
|
grad_x = (
|
||||||
|
-padded[:-2, :-2]
|
||||||
|
+ padded[:-2, 2:]
|
||||||
|
- 2.0 * padded[1:-1, :-2]
|
||||||
|
+ 2.0 * padded[1:-1, 2:]
|
||||||
|
- padded[2:, :-2]
|
||||||
|
+ padded[2:, 2:]
|
||||||
|
)
|
||||||
|
grad_y = (
|
||||||
|
-padded[:-2, :-2]
|
||||||
|
- 2.0 * padded[:-2, 1:-1]
|
||||||
|
- padded[:-2, 2:]
|
||||||
|
+ padded[2:, :-2]
|
||||||
|
+ 2.0 * padded[2:, 1:-1]
|
||||||
|
+ padded[2:, 2:]
|
||||||
|
)
|
||||||
|
magnitude = np.hypot(grad_x, grad_y)
|
||||||
|
finite = np.isfinite(magnitude)
|
||||||
|
if not finite.any():
|
||||||
|
return np.ones(depth.shape, dtype=bool)
|
||||||
|
minimum = float(magnitude[finite].min())
|
||||||
|
maximum = float(magnitude[finite].max())
|
||||||
|
if maximum <= minimum:
|
||||||
|
return np.zeros(depth.shape, dtype=bool)
|
||||||
|
normalized = (magnitude - minimum) / (maximum - minimum)
|
||||||
|
return normalized >= threshold
|
||||||
|
|
||||||
|
|
||||||
|
def confidence_percentile_threshold(depth, confidence, percentile):
|
||||||
|
import numpy as np
|
||||||
|
|
||||||
|
valid = np.isfinite(depth) & (depth > 0.0) & np.isfinite(confidence)
|
||||||
|
if not valid.any():
|
||||||
|
raise ValueError("DA3 returned no valid depth confidence values")
|
||||||
|
return float(np.percentile(confidence[valid], percentile))
|
||||||
|
|
||||||
|
|
||||||
|
def mesh_triangle_faces(
|
||||||
|
points,
|
||||||
|
depth,
|
||||||
|
confidence,
|
||||||
|
confidence_threshold,
|
||||||
|
filter_edges,
|
||||||
|
max_edge_length,
|
||||||
|
):
|
||||||
|
import numpy as np
|
||||||
|
|
||||||
|
height, width = depth.shape
|
||||||
|
indices = np.arange(height * width, dtype=np.int32).reshape(height, width)
|
||||||
|
quads = np.stack(
|
||||||
|
(
|
||||||
|
indices[:-1, :-1],
|
||||||
|
indices[1:, :-1],
|
||||||
|
indices[1:, 1:],
|
||||||
|
indices[:-1, 1:],
|
||||||
|
),
|
||||||
|
axis=-1,
|
||||||
|
).reshape(-1, 4)
|
||||||
|
valid = (
|
||||||
|
np.isfinite(points).all(axis=-1)
|
||||||
|
& np.isfinite(depth)
|
||||||
|
& (depth > 0.0)
|
||||||
|
& (confidence >= confidence_threshold)
|
||||||
|
)
|
||||||
|
if filter_edges:
|
||||||
|
valid &= ~depth_edge_mask(depth)
|
||||||
|
face_valid = valid.reshape(-1)[quads].all(axis=1)
|
||||||
|
if max_edge_length > 0.0:
|
||||||
|
flat_points = points.reshape(-1, 3)
|
||||||
|
quad_points = flat_points[quads]
|
||||||
|
next_points = np.roll(quad_points, -1, axis=1)
|
||||||
|
edge_lengths = np.linalg.norm(quad_points - next_points, axis=-1)
|
||||||
|
face_valid &= (edge_lengths <= max_edge_length).all(axis=1)
|
||||||
|
quads = quads[face_valid]
|
||||||
|
if not len(quads):
|
||||||
|
return np.empty((0, 3), dtype=np.int32)
|
||||||
|
return np.concatenate(
|
||||||
|
(quads[:, (0, 1, 2)], quads[:, (0, 2, 3)]),
|
||||||
|
axis=0,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def export_textured_mesh_glb(
|
||||||
|
prediction,
|
||||||
|
image_paths,
|
||||||
|
output_directory,
|
||||||
|
confidence_percentile,
|
||||||
|
filter_edges,
|
||||||
|
max_edge_length,
|
||||||
|
):
|
||||||
|
import numpy as np
|
||||||
|
import trimesh
|
||||||
|
from PIL import Image
|
||||||
|
|
||||||
|
required = ("depth", "conf", "intrinsics", "extrinsics")
|
||||||
|
missing = [name for name in required if getattr(prediction, name) is None]
|
||||||
|
if missing:
|
||||||
|
raise ValueError(f"DA3 mesh output is missing: {', '.join(missing)}")
|
||||||
|
depth = np.asarray(prediction.depth, dtype=np.float32)
|
||||||
|
confidence = np.asarray(prediction.conf, dtype=np.float32)
|
||||||
|
threshold = confidence_percentile_threshold(
|
||||||
|
depth,
|
||||||
|
confidence,
|
||||||
|
confidence_percentile,
|
||||||
|
)
|
||||||
|
meshes = []
|
||||||
|
referenced_points = []
|
||||||
|
for index in range(depth.shape[0]):
|
||||||
|
points = unproject_depth_frame(
|
||||||
|
depth[index],
|
||||||
|
prediction.intrinsics[index],
|
||||||
|
prediction.extrinsics[index],
|
||||||
|
)
|
||||||
|
faces = mesh_triangle_faces(
|
||||||
|
points,
|
||||||
|
depth[index],
|
||||||
|
confidence[index],
|
||||||
|
threshold,
|
||||||
|
filter_edges,
|
||||||
|
max_edge_length,
|
||||||
|
)
|
||||||
|
if not len(faces):
|
||||||
|
continue
|
||||||
|
height, width = depth[index].shape
|
||||||
|
u = np.linspace(0.0, 1.0, width, dtype=np.float32)
|
||||||
|
v = np.linspace(1.0, 0.0, height, dtype=np.float32)
|
||||||
|
uu, vv = np.meshgrid(u, v)
|
||||||
|
uv = np.stack((uu, vv), axis=-1).reshape(-1, 2)
|
||||||
|
with Image.open(image_paths[index]) as source_image:
|
||||||
|
texture_image = source_image.convert("RGB").copy()
|
||||||
|
visual = trimesh.visual.texture.TextureVisuals(
|
||||||
|
uv=uv,
|
||||||
|
image=texture_image,
|
||||||
|
)
|
||||||
|
mesh = trimesh.Trimesh(
|
||||||
|
vertices=points.reshape(-1, 3),
|
||||||
|
faces=faces,
|
||||||
|
visual=visual,
|
||||||
|
process=False,
|
||||||
|
)
|
||||||
|
mesh.remove_unreferenced_vertices()
|
||||||
|
meshes.append(mesh)
|
||||||
|
referenced_points.append(mesh.vertices)
|
||||||
|
if not meshes:
|
||||||
|
raise ValueError(
|
||||||
|
"Mesh filters removed every face; lower Confidence Threshold or disable filters"
|
||||||
|
)
|
||||||
|
|
||||||
|
world_to_camera = np.asarray(prediction.extrinsics[0], dtype=np.float64)
|
||||||
|
if world_to_camera.shape == (3, 4):
|
||||||
|
homogeneous = np.eye(4, dtype=np.float64)
|
||||||
|
homogeneous[:3, :4] = world_to_camera
|
||||||
|
world_to_camera = homogeneous
|
||||||
|
cv_to_gltf = np.eye(4, dtype=np.float64)
|
||||||
|
cv_to_gltf[1, 1] = -1.0
|
||||||
|
cv_to_gltf[2, 2] = -1.0
|
||||||
|
alignment = cv_to_gltf @ world_to_camera
|
||||||
|
aligned_points = trimesh.transform_points(
|
||||||
|
np.concatenate(referenced_points, axis=0),
|
||||||
|
alignment,
|
||||||
|
)
|
||||||
|
center = np.median(aligned_points, axis=0)
|
||||||
|
alignment[:3, 3] -= center
|
||||||
|
|
||||||
|
scene = trimesh.Scene()
|
||||||
|
for index, mesh in enumerate(meshes):
|
||||||
|
mesh.apply_transform(alignment)
|
||||||
|
scene.add_geometry(mesh, node_name=f"DepthMesh.{index + 1:04d}")
|
||||||
|
output_directory = Path(output_directory)
|
||||||
|
output_directory.mkdir(parents=True, exist_ok=True)
|
||||||
|
output_path = output_directory / "mesh.glb"
|
||||||
|
scene.export(output_path)
|
||||||
|
return output_path
|
||||||
|
|
||||||
|
|
||||||
class DownloadProgressReporter:
|
class DownloadProgressReporter:
|
||||||
@@ -856,7 +1057,7 @@ def remove_background(args, model_cache=None, cancel_check=None):
|
|||||||
images, temporary_directory = prepare_background_input(
|
images, temporary_directory = prepare_background_input(
|
||||||
args.input,
|
args.input,
|
||||||
video_frames,
|
video_frames,
|
||||||
output_directory.parent,
|
output_directory,
|
||||||
start_frame=start_frame,
|
start_frame=start_frame,
|
||||||
cancel_check=cancel_check,
|
cancel_check=cancel_check,
|
||||||
)
|
)
|
||||||
@@ -933,7 +1134,7 @@ def extract_frames(args, cancel_check=None):
|
|||||||
images, temporary_directory = prepare_input(
|
images, temporary_directory = prepare_input(
|
||||||
args.input,
|
args.input,
|
||||||
getattr(args, "video_frames", 16),
|
getattr(args, "video_frames", 16),
|
||||||
output_directory.parent,
|
output_directory,
|
||||||
start_frame=getattr(args, "start_frame", 1),
|
start_frame=getattr(args, "start_frame", 1),
|
||||||
cancel_check=cancel_check,
|
cancel_check=cancel_check,
|
||||||
)
|
)
|
||||||
@@ -956,6 +1157,9 @@ def exported_paths(output_directory, export_format):
|
|||||||
return sorted((output_directory / "gs_ply").glob("*.ply"))
|
return sorted((output_directory / "gs_ply").glob("*.ply"))
|
||||||
if export_format == "depth_vis":
|
if export_format == "depth_vis":
|
||||||
return sorted((output_directory / "depth_vis").glob("*.jpg"))
|
return sorted((output_directory / "depth_vis").glob("*.jpg"))
|
||||||
|
if export_format == "mesh":
|
||||||
|
result_path = output_directory / "mesh.glb"
|
||||||
|
return [result_path] if result_path.is_file() else []
|
||||||
if export_format == "mini_npz":
|
if export_format == "mini_npz":
|
||||||
result_path = output_directory / "results.npz"
|
result_path = output_directory / "results.npz"
|
||||||
if result_path.is_file():
|
if result_path.is_file():
|
||||||
@@ -1002,6 +1206,70 @@ def wait_for_exported_paths(
|
|||||||
return []
|
return []
|
||||||
|
|
||||||
|
|
||||||
|
def move_exported_file(
|
||||||
|
source,
|
||||||
|
destination,
|
||||||
|
timeout=10.0,
|
||||||
|
cancel_check=None,
|
||||||
|
):
|
||||||
|
"""Move an async export after Windows releases its file handle."""
|
||||||
|
source = Path(source)
|
||||||
|
destination = Path(destination)
|
||||||
|
deadline = time.monotonic() + timeout
|
||||||
|
while True:
|
||||||
|
check_cancelled(cancel_check)
|
||||||
|
try:
|
||||||
|
source.replace(destination)
|
||||||
|
return destination
|
||||||
|
except OSError as error:
|
||||||
|
is_file_lock = isinstance(error, PermissionError) or getattr(
|
||||||
|
error,
|
||||||
|
"winerror",
|
||||||
|
None,
|
||||||
|
) in {32, 33}
|
||||||
|
if not is_file_lock or time.monotonic() >= deadline:
|
||||||
|
raise
|
||||||
|
time.sleep(0.1)
|
||||||
|
|
||||||
|
|
||||||
|
def flatten_exported_paths(
|
||||||
|
output_directory,
|
||||||
|
result_paths,
|
||||||
|
cancel_check=None,
|
||||||
|
):
|
||||||
|
"""Move files produced in exporter subdirectories to the job root."""
|
||||||
|
output_directory = Path(output_directory)
|
||||||
|
flattened_paths = []
|
||||||
|
for source in result_paths:
|
||||||
|
source = Path(source)
|
||||||
|
check_cancelled(cancel_check)
|
||||||
|
if source.parent == output_directory:
|
||||||
|
flattened_paths.append(source)
|
||||||
|
continue
|
||||||
|
destination = output_directory / source.name
|
||||||
|
if destination.exists():
|
||||||
|
raise RuntimeError(f"Export filename already exists: {destination.name}")
|
||||||
|
flattened_paths.append(
|
||||||
|
move_exported_file(
|
||||||
|
source,
|
||||||
|
destination,
|
||||||
|
cancel_check=cancel_check,
|
||||||
|
)
|
||||||
|
)
|
||||||
|
for directory in sorted(
|
||||||
|
{path.parent for path in map(Path, result_paths)},
|
||||||
|
key=lambda path: len(path.parts),
|
||||||
|
reverse=True,
|
||||||
|
):
|
||||||
|
while directory != output_directory:
|
||||||
|
try:
|
||||||
|
directory.rmdir()
|
||||||
|
except OSError:
|
||||||
|
break
|
||||||
|
directory = directory.parent
|
||||||
|
return flattened_paths
|
||||||
|
|
||||||
|
|
||||||
def reconstruct(args, model_cache=None, cancel_check=None):
|
def reconstruct(args, model_cache=None, cancel_check=None):
|
||||||
write_status(args.status, 0.02, "Starting AI runtime")
|
write_status(args.status, 0.02, "Starting AI runtime")
|
||||||
check_cancelled(cancel_check)
|
check_cancelled(cancel_check)
|
||||||
@@ -1019,7 +1287,7 @@ def reconstruct(args, model_cache=None, cancel_check=None):
|
|||||||
images, temporary_directory = prepare_input(
|
images, temporary_directory = prepare_input(
|
||||||
args.input,
|
args.input,
|
||||||
args.video_frames,
|
args.video_frames,
|
||||||
output_directory.parent,
|
output_directory,
|
||||||
start_frame=getattr(args, "start_frame", 1),
|
start_frame=getattr(args, "start_frame", 1),
|
||||||
cancel_check=cancel_check,
|
cancel_check=cancel_check,
|
||||||
)
|
)
|
||||||
@@ -1124,6 +1392,16 @@ def reconstruct(args, model_cache=None, cancel_check=None):
|
|||||||
background_depth_distance,
|
background_depth_distance,
|
||||||
)
|
)
|
||||||
write_status(args.status, 0.88, f"Exporting {args.export_format}")
|
write_status(args.status, 0.88, f"Exporting {args.export_format}")
|
||||||
|
if args.export_format == "mesh":
|
||||||
|
export_textured_mesh_glb(
|
||||||
|
prediction,
|
||||||
|
images,
|
||||||
|
output_directory,
|
||||||
|
getattr(args, "conf_thresh_percentile", 40.0),
|
||||||
|
bool(getattr(args, "mesh_filter_edges", True)),
|
||||||
|
float(getattr(args, "mesh_max_edge_length", 0.1)),
|
||||||
|
)
|
||||||
|
else:
|
||||||
export(
|
export(
|
||||||
prediction,
|
prediction,
|
||||||
args.export_format,
|
args.export_format,
|
||||||
@@ -1141,19 +1419,13 @@ def reconstruct(args, model_cache=None, cancel_check=None):
|
|||||||
raise RuntimeError(
|
raise RuntimeError(
|
||||||
f"No {args.export_format} files were exported"
|
f"No {args.export_format} files were exported"
|
||||||
)
|
)
|
||||||
if args.export_format == "mini_npz":
|
result_paths = flatten_exported_paths(
|
||||||
nested_path = (
|
output_directory,
|
||||||
output_directory / "exports" / "mini_npz" / "results.npz"
|
result_paths,
|
||||||
|
cancel_check=cancel_check,
|
||||||
)
|
)
|
||||||
|
if args.export_format == "mini_npz":
|
||||||
result_path = output_directory / "results.npz"
|
result_path = output_directory / "results.npz"
|
||||||
if nested_path.is_file():
|
|
||||||
nested_path.replace(result_path)
|
|
||||||
result_paths = [result_path]
|
|
||||||
for directory in (nested_path.parent, nested_path.parent.parent):
|
|
||||||
try:
|
|
||||||
directory.rmdir()
|
|
||||||
except OSError:
|
|
||||||
pass
|
|
||||||
if reference_depth is not None:
|
if reference_depth is not None:
|
||||||
write_npz_reference_depth(result_path, reference_depth)
|
write_npz_reference_depth(result_path, reference_depth)
|
||||||
check_cancelled(cancel_check)
|
check_cancelled(cancel_check)
|
||||||
|
|||||||
@@ -1279,7 +1279,7 @@ class AddonTest(unittest.TestCase):
|
|||||||
self.assertEqual(paths["status"].parent, paths["result"])
|
self.assertEqual(paths["status"].parent, paths["result"])
|
||||||
self.assertEqual(paths["status"].stem, paths["result"].name)
|
self.assertEqual(paths["status"].stem, paths["result"].name)
|
||||||
|
|
||||||
def test_new_job_paths_uses_type_time_folder_name(self):
|
def test_new_job_paths_uses_unique_action_object_names(self):
|
||||||
runtime = importlib.import_module("snapform.runtime")
|
runtime = importlib.import_module("snapform.runtime")
|
||||||
with TemporaryDirectory() as directory:
|
with TemporaryDirectory() as directory:
|
||||||
jobs_root = Path(directory)
|
jobs_root = Path(directory)
|
||||||
@@ -1300,15 +1300,41 @@ class AddonTest(unittest.TestCase):
|
|||||||
|
|
||||||
self.assertRegex(
|
self.assertRegex(
|
||||||
depth_paths["result"].name,
|
depth_paths["result"].name,
|
||||||
r"^depth-plane_\d{8}-\d{6}$",
|
r"^generate-depth-plane_\d{8}-\d{6}-\d{6}$",
|
||||||
)
|
)
|
||||||
self.assertTrue(
|
self.assertTrue(
|
||||||
background_paths["result"].name.startswith(
|
background_paths["result"].name.startswith(
|
||||||
"remove-background_"
|
"remove-image-background_"
|
||||||
)
|
)
|
||||||
)
|
)
|
||||||
self.assertTrue(
|
self.assertTrue(
|
||||||
frames_paths["result"].name.startswith("depth-plane_")
|
frames_paths["result"].name.startswith(
|
||||||
|
"extract-original-frames_"
|
||||||
|
)
|
||||||
|
)
|
||||||
|
|
||||||
|
def test_generated_inputs_are_moved_into_job_root(self):
|
||||||
|
runtime = importlib.import_module("snapform.runtime")
|
||||||
|
with TemporaryDirectory() as directory:
|
||||||
|
root = Path(directory)
|
||||||
|
staging = root / "staging"
|
||||||
|
staging.mkdir()
|
||||||
|
(staging / "frame.0001.jpg").write_bytes(b"first")
|
||||||
|
(staging / "frame.0002.jpg").write_bytes(b"second")
|
||||||
|
job_directory = root / "job"
|
||||||
|
job_directory.mkdir()
|
||||||
|
|
||||||
|
result = runtime.move_input_into_job(staging, job_directory)
|
||||||
|
|
||||||
|
self.assertEqual(result, job_directory)
|
||||||
|
self.assertFalse(staging.exists())
|
||||||
|
self.assertEqual(
|
||||||
|
(job_directory / "frame.0001.jpg").read_bytes(),
|
||||||
|
b"first",
|
||||||
|
)
|
||||||
|
self.assertEqual(
|
||||||
|
(job_directory / "frame.0002.jpg").read_bytes(),
|
||||||
|
b"second",
|
||||||
)
|
)
|
||||||
|
|
||||||
def test_empty_image_start_frame_honors_image_user(self):
|
def test_empty_image_start_frame_honors_image_user(self):
|
||||||
@@ -1468,9 +1494,83 @@ class AddonTest(unittest.TestCase):
|
|||||||
|
|
||||||
self.assertEqual(
|
self.assertEqual(
|
||||||
identifiers,
|
identifiers,
|
||||||
["glb", "gs_ply", "depth_vis", "mini_npz"],
|
["glb", "gs_ply", "depth_vis", "mini_npz", "mesh"],
|
||||||
)
|
)
|
||||||
self.assertEqual(export_format["default"], "glb")
|
self.assertEqual(export_format["default"], "glb")
|
||||||
|
self.assertEqual(export_format["name"], "Output Type")
|
||||||
|
self.assertEqual(
|
||||||
|
[item[1] for item in export_format["items"]],
|
||||||
|
[
|
||||||
|
"Point Cloud (GLB)",
|
||||||
|
"Gaussian Splats (PLY)",
|
||||||
|
"Depth Preview (Images)",
|
||||||
|
"Depth Data (NPZ)",
|
||||||
|
"Textured Mesh (GLB)",
|
||||||
|
],
|
||||||
|
)
|
||||||
|
|
||||||
|
def test_mesh_is_a_da3_output_type(self):
|
||||||
|
settings = SimpleNamespace(
|
||||||
|
input_path="",
|
||||||
|
export_format="mesh",
|
||||||
|
model="BASE",
|
||||||
|
gs_model="GIANT_1_1",
|
||||||
|
process_res=504,
|
||||||
|
process_res_method="upper_bound_resize",
|
||||||
|
conf_thresh_percentile=40.0,
|
||||||
|
num_max_points=1_000_000,
|
||||||
|
show_cameras=True,
|
||||||
|
gs_views_interval=1,
|
||||||
|
video_frames=16,
|
||||||
|
device="CPU",
|
||||||
|
use_ray_pose=False,
|
||||||
|
ref_view_strategy="saddle_balanced",
|
||||||
|
)
|
||||||
|
operator = self.addon.RunJob()
|
||||||
|
operator.job_type = self.addon.operators.RECONSTRUCTION_JOB
|
||||||
|
|
||||||
|
with TemporaryDirectory() as directory:
|
||||||
|
input_path = Path(directory) / "input.png"
|
||||||
|
input_path.touch()
|
||||||
|
settings.input_path = str(input_path)
|
||||||
|
paths = {
|
||||||
|
"log": Path(directory) / "job.log",
|
||||||
|
"status": Path(directory) / "job.json",
|
||||||
|
"result": Path(directory) / "result",
|
||||||
|
}
|
||||||
|
with patch.object(
|
||||||
|
self.addon.runtime,
|
||||||
|
"new_job_paths",
|
||||||
|
return_value=paths,
|
||||||
|
), patch.object(
|
||||||
|
self.addon.runtime,
|
||||||
|
"runtime_ready",
|
||||||
|
return_value=True,
|
||||||
|
), patch.object(
|
||||||
|
self.addon.runtime,
|
||||||
|
"model_ready",
|
||||||
|
return_value=True,
|
||||||
|
), patch.object(
|
||||||
|
self.addon.runtime,
|
||||||
|
"reconstruct_request",
|
||||||
|
return_value=self.addon.runtime.HostJobRequest(
|
||||||
|
{"job_id": "test"}
|
||||||
|
),
|
||||||
|
) as reconstruct:
|
||||||
|
operator._prepare_command(None, settings)
|
||||||
|
|
||||||
|
self.assertEqual(reconstruct.call_args.args[1], "mesh")
|
||||||
|
self.assertEqual(reconstruct.call_args.kwargs["model_key"], "BASE")
|
||||||
|
|
||||||
|
def test_confidence_threshold_is_a_bounded_percentage(self):
|
||||||
|
confidence = self.addon.SnapFormSettings.__annotations__[
|
||||||
|
"conf_thresh_percentile"
|
||||||
|
]
|
||||||
|
|
||||||
|
self.assertEqual(confidence["name"], "Confidence Threshold")
|
||||||
|
self.assertEqual(confidence["min"], 0.0)
|
||||||
|
self.assertEqual(confidence["max"], 100.0)
|
||||||
|
self.assertEqual(confidence["subtype"], "PERCENTAGE")
|
||||||
|
|
||||||
def test_gs_model_property_only_includes_gaussian_models(self):
|
def test_gs_model_property_only_includes_gaussian_models(self):
|
||||||
gs_model = self.addon.SnapFormSettings.__annotations__[
|
gs_model = self.addon.SnapFormSettings.__annotations__[
|
||||||
@@ -1586,9 +1686,17 @@ class AddonTest(unittest.TestCase):
|
|||||||
"DEFAULT_CLOSED",
|
"DEFAULT_CLOSED",
|
||||||
self.addon.DebugPanel.bl_options,
|
self.addon.DebugPanel.bl_options,
|
||||||
)
|
)
|
||||||
for panel in (self.addon.Da3DebugPanel, self.addon.Ben2DebugPanel):
|
panels = (
|
||||||
|
self.addon.Da3OutputDebugPanel,
|
||||||
|
self.addon.Ben2RemovalDebugPanel,
|
||||||
|
)
|
||||||
|
for panel in panels:
|
||||||
self.assertEqual(panel.bl_parent_id, self.addon.DebugPanel.bl_idname)
|
self.assertEqual(panel.bl_parent_id, self.addon.DebugPanel.bl_idname)
|
||||||
self.assertIn("DEFAULT_CLOSED", panel.bl_options)
|
self.assertIn("DEFAULT_CLOSED", panel.bl_options)
|
||||||
|
self.assertEqual(
|
||||||
|
[panel.bl_label for panel in panels],
|
||||||
|
["DA3 Output Test", "BEN2 Removal Test"],
|
||||||
|
)
|
||||||
|
|
||||||
def test_host_panel_exposes_shared_host_state_and_controls(self):
|
def test_host_panel_exposes_shared_host_state_and_controls(self):
|
||||||
self.assertIn(self.addon.HostPanel, self.addon.CLASSES)
|
self.assertIn(self.addon.HostPanel, self.addon.CLASSES)
|
||||||
|
|||||||
@@ -474,6 +474,38 @@ class WorkerTest(unittest.TestCase):
|
|||||||
)
|
)
|
||||||
self.assertEqual(result, [npz_path])
|
self.assertEqual(result, [npz_path])
|
||||||
|
|
||||||
|
def test_move_exported_file_retries_windows_file_lock(self):
|
||||||
|
with tempfile.TemporaryDirectory() as directory:
|
||||||
|
root = Path(directory)
|
||||||
|
source = root / "source.npz"
|
||||||
|
destination = root / "destination.npz"
|
||||||
|
source.touch()
|
||||||
|
path_type = type(source)
|
||||||
|
original_replace = path_type.replace
|
||||||
|
calls = 0
|
||||||
|
|
||||||
|
def locked_once(path, target):
|
||||||
|
nonlocal calls
|
||||||
|
calls += 1
|
||||||
|
if calls == 1:
|
||||||
|
error = PermissionError("file is in use")
|
||||||
|
error.winerror = 32
|
||||||
|
raise error
|
||||||
|
return original_replace(path, target)
|
||||||
|
|
||||||
|
with patch.object(path_type, "replace", new=locked_once), patch.object(
|
||||||
|
self.worker.time,
|
||||||
|
"sleep",
|
||||||
|
) as sleep:
|
||||||
|
result = self.worker.move_exported_file(
|
||||||
|
source,
|
||||||
|
destination,
|
||||||
|
)
|
||||||
|
|
||||||
|
self.assertEqual(result, destination)
|
||||||
|
self.assertTrue(destination.is_file())
|
||||||
|
sleep.assert_called_once_with(0.1)
|
||||||
|
|
||||||
def test_rejects_unsupported_input_file(self):
|
def test_rejects_unsupported_input_file(self):
|
||||||
with tempfile.TemporaryDirectory() as directory:
|
with tempfile.TemporaryDirectory() as directory:
|
||||||
input_path = Path(directory) / "input.txt"
|
input_path = Path(directory) / "input.txt"
|
||||||
@@ -850,9 +882,10 @@ class WorkerTest(unittest.TestCase):
|
|||||||
self.assertEqual(status["progress"], 1.0)
|
self.assertEqual(status["progress"], 1.0)
|
||||||
self.assertIn("3 frame", status["message"])
|
self.assertIn("3 frame", status["message"])
|
||||||
|
|
||||||
def test_uses_official_da3_export_parameters_for_all_four_formats(self):
|
def test_uses_da3_export_parameters_for_formats_and_debug_mesh(self):
|
||||||
inference_options = []
|
inference_options = []
|
||||||
export_calls = []
|
export_calls = []
|
||||||
|
mesh_calls = []
|
||||||
|
|
||||||
class FakeModel:
|
class FakeModel:
|
||||||
@classmethod
|
@classmethod
|
||||||
@@ -910,6 +943,27 @@ class WorkerTest(unittest.TestCase):
|
|||||||
fake_package = ModuleType("depth_anything_3")
|
fake_package = ModuleType("depth_anything_3")
|
||||||
fake_package.__path__ = []
|
fake_package.__path__ = []
|
||||||
|
|
||||||
|
def fake_mesh_export(
|
||||||
|
prediction,
|
||||||
|
images,
|
||||||
|
output_directory,
|
||||||
|
confidence_percentile,
|
||||||
|
filter_edges,
|
||||||
|
max_edge_length,
|
||||||
|
):
|
||||||
|
mesh_calls.append(
|
||||||
|
(
|
||||||
|
prediction,
|
||||||
|
images,
|
||||||
|
confidence_percentile,
|
||||||
|
filter_edges,
|
||||||
|
max_edge_length,
|
||||||
|
)
|
||||||
|
)
|
||||||
|
output_path = Path(output_directory) / "mesh.glb"
|
||||||
|
output_path.touch()
|
||||||
|
return output_path
|
||||||
|
|
||||||
with tempfile.TemporaryDirectory() as directory:
|
with tempfile.TemporaryDirectory() as directory:
|
||||||
root = Path(directory)
|
root = Path(directory)
|
||||||
input_path = root / "input.png"
|
input_path = root / "input.png"
|
||||||
@@ -923,6 +977,10 @@ class WorkerTest(unittest.TestCase):
|
|||||||
"depth_anything_3.utils": fake_utils,
|
"depth_anything_3.utils": fake_utils,
|
||||||
"depth_anything_3.utils.export": fake_export_module,
|
"depth_anything_3.utils.export": fake_export_module,
|
||||||
},
|
},
|
||||||
|
), patch.object(
|
||||||
|
self.worker,
|
||||||
|
"export_textured_mesh_glb",
|
||||||
|
side_effect=fake_mesh_export,
|
||||||
):
|
):
|
||||||
for export_format in self.worker.EXPORT_FORMATS:
|
for export_format in self.worker.EXPORT_FORMATS:
|
||||||
args = SimpleNamespace(
|
args = SimpleNamespace(
|
||||||
@@ -941,6 +999,8 @@ class WorkerTest(unittest.TestCase):
|
|||||||
num_max_points=12345,
|
num_max_points=12345,
|
||||||
show_cameras=0,
|
show_cameras=0,
|
||||||
gs_views_interval=2,
|
gs_views_interval=2,
|
||||||
|
mesh_filter_edges=True,
|
||||||
|
mesh_max_edge_length=0.2,
|
||||||
)
|
)
|
||||||
self.worker.reconstruct(args)
|
self.worker.reconstruct(args)
|
||||||
status = json.loads(
|
status = json.loads(
|
||||||
@@ -948,9 +1008,17 @@ class WorkerTest(unittest.TestCase):
|
|||||||
)
|
)
|
||||||
self.assertEqual(status["progress"], 1.0)
|
self.assertEqual(status["progress"], 1.0)
|
||||||
self.assertIn(export_format, status["message"])
|
self.assertIn(export_format, status["message"])
|
||||||
|
result_directory = Path(args.output_dir)
|
||||||
|
if export_format == "gs_ply":
|
||||||
|
self.assertTrue((result_directory / "0000.ply").is_file())
|
||||||
|
self.assertFalse((result_directory / "gs_ply").exists())
|
||||||
|
if export_format == "depth_vis":
|
||||||
|
self.assertTrue((result_directory / "0000.jpg").is_file())
|
||||||
|
self.assertFalse((result_directory / "depth_vis").exists())
|
||||||
|
|
||||||
self.assertEqual(len(inference_options), 4)
|
self.assertEqual(len(inference_options), 5)
|
||||||
self.assertEqual(len(export_calls), 4)
|
self.assertEqual(len(export_calls), 4)
|
||||||
|
self.assertEqual(len(mesh_calls), 1)
|
||||||
for images, options in inference_options:
|
for images, options in inference_options:
|
||||||
self.assertEqual(images, [str(input_path)])
|
self.assertEqual(images, [str(input_path)])
|
||||||
self.assertEqual(options["process_res"], 518)
|
self.assertEqual(options["process_res"], 518)
|
||||||
@@ -960,8 +1028,15 @@ class WorkerTest(unittest.TestCase):
|
|||||||
)
|
)
|
||||||
self.assertTrue(options["use_ray_pose"])
|
self.assertTrue(options["use_ray_pose"])
|
||||||
self.assertEqual(options["ref_view_strategy"], "middle")
|
self.assertEqual(options["ref_view_strategy"], "middle")
|
||||||
for _prediction, export_format, _export_dir, options in export_calls:
|
for (
|
||||||
expected_depth = -1.0 if export_format == "mini_npz" else 2.0
|
_prediction,
|
||||||
|
export_format,
|
||||||
|
_export_dir,
|
||||||
|
options,
|
||||||
|
) in export_calls:
|
||||||
|
expected_depth = (
|
||||||
|
-1.0 if export_format == "mini_npz" else 2.0
|
||||||
|
)
|
||||||
np.testing.assert_array_equal(
|
np.testing.assert_array_equal(
|
||||||
_prediction.depth,
|
_prediction.depth,
|
||||||
np.full((1, 2, 2), expected_depth, dtype=np.float32),
|
np.full((1, 2, 2), expected_depth, dtype=np.float32),
|
||||||
@@ -985,9 +1060,14 @@ class WorkerTest(unittest.TestCase):
|
|||||||
)
|
)
|
||||||
else:
|
else:
|
||||||
self.assertEqual(options, {})
|
self.assertEqual(options, {})
|
||||||
|
self.assertEqual(mesh_calls[0][1], [input_path])
|
||||||
|
self.assertEqual(mesh_calls[0][2:], (35.0, True, 0.2))
|
||||||
|
|
||||||
infer_gs_values = [options["infer_gs"] for _, options in inference_options]
|
infer_gs_values = [options["infer_gs"] for _, options in inference_options]
|
||||||
self.assertEqual(infer_gs_values, [False, True, False, False])
|
self.assertEqual(
|
||||||
|
infer_gs_values,
|
||||||
|
[False, True, False, False, False],
|
||||||
|
)
|
||||||
|
|
||||||
def test_reconstruct_copies_frames_for_multi_frame_input(self):
|
def test_reconstruct_copies_frames_for_multi_frame_input(self):
|
||||||
class FakeModel:
|
class FakeModel:
|
||||||
@@ -1080,6 +1160,74 @@ class WorkerTest(unittest.TestCase):
|
|||||||
status = json.loads(Path(args.status).read_text(encoding="utf-8"))
|
status = json.loads(Path(args.status).read_text(encoding="utf-8"))
|
||||||
self.assertEqual(status["progress"], 1.0)
|
self.assertEqual(status["progress"], 1.0)
|
||||||
|
|
||||||
|
def test_mesh_confidence_uses_percentile(self):
|
||||||
|
depth = np.ones((1, 2, 2), dtype=np.float32)
|
||||||
|
confidence = np.array([[[1.0, 2.0], [3.0, 4.0]]])
|
||||||
|
|
||||||
|
threshold = self.worker.confidence_percentile_threshold(
|
||||||
|
depth,
|
||||||
|
confidence,
|
||||||
|
50.0,
|
||||||
|
)
|
||||||
|
|
||||||
|
self.assertEqual(threshold, 2.5)
|
||||||
|
|
||||||
|
def test_mesh_faces_apply_confidence_threshold_and_face_direction(self):
|
||||||
|
points = np.array(
|
||||||
|
[
|
||||||
|
[[0.0, 0.0, 1.0], [1.0, 0.0, 1.0], [2.0, 0.0, 1.0]],
|
||||||
|
[[0.0, 1.0, 1.0], [1.0, 1.0, 1.0], [2.0, 1.0, 1.0]],
|
||||||
|
],
|
||||||
|
dtype=np.float32,
|
||||||
|
)
|
||||||
|
depth = np.ones((2, 3), dtype=np.float32)
|
||||||
|
confidence = np.ones((2, 3), dtype=np.float32)
|
||||||
|
confidence[:, 0] = 0.0
|
||||||
|
|
||||||
|
faces = self.worker.mesh_triangle_faces(
|
||||||
|
points,
|
||||||
|
depth,
|
||||||
|
confidence,
|
||||||
|
confidence_threshold=0.5,
|
||||||
|
filter_edges=False,
|
||||||
|
max_edge_length=0.0,
|
||||||
|
)
|
||||||
|
|
||||||
|
np.testing.assert_array_equal(faces, [[1, 4, 5], [1, 5, 2]])
|
||||||
|
vertices = points.reshape(-1, 3)[faces[0]]
|
||||||
|
normal = np.cross(
|
||||||
|
vertices[1] - vertices[0],
|
||||||
|
vertices[2] - vertices[0],
|
||||||
|
)
|
||||||
|
self.assertLess(normal[2], 0.0)
|
||||||
|
|
||||||
|
def test_mesh_percentile_reduces_face_count(self):
|
||||||
|
depth = np.ones((3, 3), dtype=np.float32)
|
||||||
|
confidence = np.arange(1, 10, dtype=np.float32).reshape(3, 3)
|
||||||
|
x, y = np.meshgrid(
|
||||||
|
np.arange(3, dtype=np.float32),
|
||||||
|
np.arange(3, dtype=np.float32),
|
||||||
|
)
|
||||||
|
points = np.stack((x, y, depth), axis=-1)
|
||||||
|
face_counts = []
|
||||||
|
for percentile in (0.0, 50.0, 100.0):
|
||||||
|
threshold = self.worker.confidence_percentile_threshold(
|
||||||
|
depth,
|
||||||
|
confidence,
|
||||||
|
percentile,
|
||||||
|
)
|
||||||
|
faces = self.worker.mesh_triangle_faces(
|
||||||
|
points,
|
||||||
|
depth,
|
||||||
|
confidence,
|
||||||
|
confidence_threshold=threshold,
|
||||||
|
filter_edges=False,
|
||||||
|
max_edge_length=0.0,
|
||||||
|
)
|
||||||
|
face_counts.append(len(faces))
|
||||||
|
|
||||||
|
self.assertEqual(face_counts, [8, 2, 0])
|
||||||
|
|
||||||
|
|
||||||
if __name__ == "__main__":
|
if __name__ == "__main__":
|
||||||
unittest.main()
|
unittest.main()
|
||||||
|
|||||||
Reference in New Issue
Block a user