icrdr/AnyImage
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Refactor image handling and update foreground processing

Browse Source 
- Changed the logic in `remove_background_faces` to keep only faces with all foreground vertices.
- Updated background job to save foreground images with a new naming convention (`color.0001.png`).
- Replaced `copy_frames` with `copy_color_images` in various jobs to ensure consistent image naming.
- Introduced a new function `copy_color_images` to handle the copying of images with a specific naming format.
- Updated the BEN2 model to save output images with a new prefix for better clarity.
- Modified tests to reflect changes in image naming and processing logic.
- Added a new shadeless node group for improved material handling in geometry nodes.
- Enhanced validation scripts to check for new image naming conventions and ensure proper functionality.
This commit is contained in:
马南
2026-08-19 01:45:59 +08:00
parent 835225d264
commit 20b653bb30
28 changed files with 663 additions and 383 deletions
Download Patch File Download Diff File Expand all files Collapse all files

16
README.md
View File

@@ -40,9 +40,9 @@ Environment 未安装时，同一位置显示两个安装按钮。Environment
- `Depth from`：`Original` 直接用原图，`DA3` 使用现有 DA3 流程，`MoGe-2` 生成 metric depth、normal 与有效区域 mask - `Depth from`：`Original` 直接用原图，`DA3` 使用现有 DA3 流程，`MoGe-2` 生成 metric depth、normal 与有效区域 mask
- `Resolution Scale` 使用 `1–100%`，按原图长边计算 Process Res；结果会限制在 `252–1008` 并自动对齐到 14 的倍数 - `Resolution Scale` 使用 `1–100%`，按原图长边计算 Process Res；结果会限制在 `252–1008` 并自动对齐到 14 的倍数
- `Remove Background` 默认关闭；开启后在转换期间使用 BEN2 删除背景 - `Remove Background` 默认关闭；开启后在转换期间使用 BEN2 删除背景
- `Unshaded` 保留为直接显示贴图颜色的选项 - `Shadeless` 使用 `O Shadeless` 节点组显示不受场景灯光影响的贴图颜色
- `Keep Original` 默认关闭，位于分割线下方；开启后保留源 Image Empty - `Keep Original` 默认关闭，位于分割线下方；开启后保留源 Image Empty
4. 点击确认，稍等片刻就得到带位移的立体面片 4. 点击确认，稍等片刻就得到带位移的立体面片。`O DA3 Depth Plane` 与 `O MoGe2 Depth Plane` 都使用 `.color` 的 `Alpha > 0.5` 划分前景范围，范围外由 `Background Depth` 控制。
### 去除背景 ### 去除背景
@@ -57,8 +57,8 @@ ViT-S、ViT-B、ViT-L 或 DA3 Small、Base，其中 MoGe-2 ViT-S 为默认模型
Surface 输出流程构建 DA3 与 MoGe-2 源网格：MoGe-2 使用深度断层 mask，DA3 保留 confidence 与边拉伸 Surface 输出流程构建 DA3 与 MoGe-2 源网格：MoGe-2 使用深度断层 mask，DA3 保留 confidence 与边拉伸
数据。两者再由独立的 Geometry Nodes 数据。两者再由独立的 Geometry Nodes
修改器完成剔面、快速重拓扑、Relax 与 UV 重投；顶层 `Iterations` 可设为 `0`。开启 Remove Background 修改器完成剔面、快速重拓扑、Relax 与 UV 重投；顶层 `Iterations` 可设为 `0`。开启 Remove Background
时，BEN2 Alpha 只要大于 `0` 就视为前景；三角面有任一前景点即保留，只有全背景面会删除并且不再交给 时，BEN2 Alpha 只要大于 `0` 就视为前景；三角形三个顶点必须全部为前景，任一顶点 Alpha 为 `0`
Geometry Nodes；Surface 材质使用 `.foreground`。匹配相机默认不创建；开启后会作为 Surface 的子级。 都会删除该面并且不再交给 Geometry Nodes；Surface 材质使用 `.color`。匹配相机默认不创建；开启后会作为 Surface 的子级。
MoGe-2 的像素级 normal 会作为 Object Space normal 贴图接入 `O MoGe2 Surface Layer`，并与原图 MoGe-2 的像素级 normal 会作为 Object Space normal 贴图接入 `O MoGe2 Surface Layer`，并与原图
Bump 合并；网格本身不写入 custom normal，因此重拓扑降低顶点数后仍能保留细腻的法线细节。 Bump 合并；网格本身不写入 custom normal，因此重拓扑降低顶点数后仍能保留细腻的法线细节。
当前此功能仅支持单张图片。 当前此功能仅支持单张图片。
@@ -67,20 +67,20 @@ MoGe-2 ViT-S、ViT-B、ViT-L、DA3 Small、DA3 Base 与 BEN2 全部使用 ONNX
点云重投影和网格生成均不依赖 PyTorch。Windows 使用 DirectML 加速，provider 不可用时会回退到 ONNX Runtime CPU， 点云重投影和网格生成均不依赖 PyTorch。Windows 使用 DirectML 加速，provider 不可用时会回退到 ONNX Runtime CPU，
并在任务状态中显示实际 provider。 并在任务状态中显示实际 provider。
转换对话框提供模型与 `Create Matching Camera`、`Remove Background`、`Unshaded`；DA3 显示 转换对话框提供模型与 `Create Matching Camera`、`Remove Background`、`Shadeless`；DA3 显示
`Resolution Scale`，MoGe-2 显示 `Resolution Level`， `Resolution Scale`，MoGe-2 显示 `Resolution Level`，
以及分割线下方的 `Keep Original`。 以及分割线下方的 `Keep Original`。
界面会显示计算所得的实际 Process Res。剔除和重拓扑参数直接在 界面会显示计算所得的实际 Process Res。剔除和重拓扑参数直接在
Geometry Nodes 修改器中调整；`Unshaded` 可使用与 Depth Plane 相同的 Emission 材质。普通材质的 Geometry Nodes 修改器中调整；`Shadeless` 与 Depth Plane 共用 `O Shadeless` 材质节点组。颜色贴图始终使用统一的 `.color` 转换产物；Remove Background 只决定是否由 BEN2 修改 Alpha，并使用 `AgX Base sRGB`。普通材质的
Bump Scale 会在材质组内乘以 `0.001`，并按透视表面的深度空间缩放。Surface 与 Depth Plane 的 Bump Scale 会在材质组内乘以 `0.001`，并按透视表面的深度空间缩放。Surface 与 Depth Plane 的
Layer Color 同时驱动 Principled BSDF 的 Base Color、Subsurface Radius 和 Emission Color，Layer Normal Layer Color 同时驱动 Principled BSDF 的 Base Color、Subsurface Radius 和 Emission Color，Layer Normal
同时驱动 Normal 与 Coat Normal；Subsurface、Emission、Coat 的其他设置保持默认。Surface attributes 同时驱动 Normal 与 Coat Normal；Subsurface、Emission、Coat 的其他设置保持默认。Surface attributes
统一使用 `o_` 前缀；Server 准备过程显示 统一使用 `o_` 前缀；Server 准备过程显示
`Preparing surface data n/6`，可看到当前阶段与总步骤。 `Preparing surface data n/6`，可看到当前阶段与总步骤；Surface 单帧模型推理不显示多余的 `1/1`。
### 视频与序列帧 ### 视频与序列帧
SnapForm 支持视频（mp4、mkv、mov 等）和序列帧，会逐帧处理并生成逐帧对应的立体面片。处理范围默认 SnapForm 支持视频（mp4、mkv、mov 等）和序列帧，会逐帧处理并生成逐帧对应的立体面片。`.color` 与 `.depth` 按源 Empty Image 的类型载入：普通单图为 `IMAGE`，Movie/Sequence 即使只处理一帧也仍为 `SEQUENCE`。处理范围默认
跟随 Empty 上设置的 `Image Frames` 与 `Offset`，不需要额外配置。DA3 的转换任务复用同一个 ONNX 跟随 Empty 上设置的 `Image Frames` 与 `Offset`，不需要额外配置。DA3 的转换任务复用同一个 ONNX
Session 逐帧推理，显存峰值不随视频帧数平方增长。Debug 只保存模型 API 的直接返回：DA3 多帧输入 Session 逐帧推理，显存峰值不随视频帧数平方增长。Debug 只保存模型 API 的直接返回：DA3 多帧输入
保留联合推理，MoGe-2 与 BEN2 按各自的单图 API 逐帧保存。 保留联合推理，MoGe-2 与 BEN2 按各自的单图 API 逐帧保存。

27
docs/depth-plane.md
View File

@@ -15,11 +15,11 @@
- `Original`：直接把原图作为深度贴图，不显示估算参数。仅在开启 `Remove Background` 时运行 BEN2 - `Original`：直接把原图作为深度贴图，不显示估算参数。仅在开启 `Remove Background` 时运行 BEN2
取得前景 Alpha；视频或序列帧按需同时抽取原始帧与前景序列，把原始帧作为 `.depth` 序列。 取得前景 Alpha；视频或序列帧按需同时抽取原始帧与前景序列，把原始帧作为 `.depth` 序列。
`MoGe-2 ViT-S` 是默认深度来源。选择 `DA3` 时，其 Model 与 Resolution Scale `MoGe-2 ViT-S` 是默认深度来源。选择 `DA3` 时，其 Model 与 Resolution Scale
参数会集中显示在独立边框中。`Remove Background` 与 `Unshaded` 直接显示，不再使用 Options 折叠； 参数会集中显示在独立边框中。`Remove Background` 与 `Shadeless` 直接显示，不再使用 Options 折叠；
`Keep Original` 默认关闭，并在分割线下方保持为最后一项。仅当开启 `Remove Background` 时才需要 `Keep Original` 默认关闭，并在分割线下方保持为最后一项。仅当开启 `Remove Background` 时才需要
`BEN2 Base`，并把 `.foreground` 的 Alpha 作为独立 `BEN2 Base`，并把 `.color` 的 Alpha 作为独立 Mask，不在 Server 中改写深度值。无论颜色层是直接
Mask，不在 Server 中改写深度值。保存 `.foreground` 时保留 BEN2 Alpha，但 RGB 使用未经修改的原始输入； 复制原图，还是由 BEN2 为单图或序列修改 Alpha，Server 都统一写为 `color.0001.*`；Blender 只导入
因此材质直接用 `.foreground` 同时提供颜色和 Mask，不再额外加载原图 Image Texture。 这一份 `.color`，不复用源 Empty 的 Image，Color Space 固定为 `AgX Base sRGB`。
Image Empty 引用视频或序列帧时，DA3 与 MoGe-2 会按 Empty 的 Image Frames 数值逐帧估计 Image Empty 引用视频或序列帧时，DA3 与 MoGe-2 会按 Empty 的 Image Frames 数值逐帧估计
深度（未设置时为完整帧数，显式设为 1 时只处理 1 帧），起始帧跟随 Empty 的 Offset，与背景蒙版范围一致； 深度（未设置时为完整帧数，显式设为 1 时只处理 1 帧），起始帧跟随 Empty 的 Offset，与背景蒙版范围一致；
DA3 会复用同一个 ONNX Session 对每帧独立推理，再合并为原有序列输出，避免多帧 Attention 的平方级显存占用。 DA3 会复用同一个 ONNX Session 对每帧独立推理，再合并为原有序列输出，避免多帧 Attention 的平方级显存占用。
@@ -27,10 +27,10 @@
该行为只用于 Depth Plane 转换；Surface 仅支持单张图片。Debug 中 DA3 多帧输入联合推理， 该行为只用于 Depth Plane 转换；Surface 仅支持单张图片。Debug 中 DA3 多帧输入联合推理，
MoGe-2 与 BEN2 按单图 API 逐帧运行。 MoGe-2 与 BEN2 按单图 API 逐帧运行。
Offset 超出视频长度时同样把窗口收进视频内。材质节点与置换贴图的 Start 保持与原 Empty Offset 超出视频长度时同样把窗口收进视频内。材质节点与置换贴图的 Start 保持与原 Empty
一致；Original 模式的视频或序列帧使用 `frame.0001.jpg` 等原图序列作为颜色与深度， 一致；Original 模式的视频或序列帧使用 `color.0001.*` 颜色层同时作为颜色与深度输入；开启
`foreground.0001.png` 只提供前景 Alpha。 Remove Background 时同名颜色层包含 BEN2 修改后的 Alpha。
DA3 的深度仍以 EXR 序列帧（`depth.0001.exr` 起）保存在 job 目录根下；MoGe-2 分别使用 `moge_depth`、`moge_mask` 与 `moge_normal` EXR 序列。 DA3 的深度仍以 EXR 序列帧（`depth.0001.exr` 起）保存在 job 目录根下；MoGe-2 分别使用 `moge_depth`、`moge_mask` 与 `moge_normal` EXR 序列。
`Unshaded`、`Remove Background` 与 `Keep Original` 均默认关闭，因此默认删除原对象。 `Shadeless`、`Remove Background` 与 `Keep Original` 均默认关闭，因此默认删除原对象。`Shadeless` 会以 `O Shadeless` 节点组替代普通 Principled BSDF 路径，并继续通过 Alpha 与 Transparent BSDF 混合。
3. DA3 或 MoGe-2 完成后会自动创建面片。DA3 面片几何先由 `mini_npz` 的相机内参与深度确定，再通过对象 3. DA3 或 MoGe-2 完成后会自动创建面片。DA3 面片几何先由 `mini_npz` 的相机内参与深度确定，再通过对象
Transform 匹配原 Image Empty 的可见尺寸、位置、旋转和 Offset。Original 模式直接使用源 Empty Transform 匹配原 Image Empty 的可见尺寸、位置、旋转和 Offset。Original 模式直接使用源 Empty
的可见尺寸、Offset 与 Transform。DA3 会把用于匹配画面尺寸的缩放预乘到 `.depth` 数值， 的可见尺寸、Offset 与 Transform。DA3 会把用于匹配画面尺寸的缩放预乘到 `.depth` 数值，
@@ -40,9 +40,10 @@
同名冲突，由 Blender 自动为生成面片追加编号。 同名冲突，由 Blender 自动为生成面片追加编号。
4. Blender 只创建保留图像纵横比的最低密度基础网格，再把 `O DA3 Depth Plane` 作为 Geometry Nodes 4. Blender 只创建保留图像纵横比的最低密度基础网格，再把 `O DA3 Depth Plane` 作为 Geometry Nodes
修改器添加到对象。节点组与 `O DA3 Surface` 一同保存在 `O_SnapForm.blend`，关闭 修改器添加到对象。节点组与 `O DA3 Surface` 一同保存在 `O_SnapForm.blend`，关闭
`show_group_selector`，不为每个对象复制节点组。MoGe-2 使用单独的 `O MoGe2 Depth Plane`，只保留 `show_group_selector`，不为每个对象复制节点组。MoGe-2 使用单独的 `O MoGe2 Depth Plane`，处理
网格细分、metric depth 位移、mask 删点和材质设置；材质使用 `O MoGe2 Depth Plane Layer`，直接接入 网格细分、metric depth 位移和材质设置；材质使用 `O MoGe2 Depth Plane Layer`，直接接入模型 normal。
模型 normal，不携带 DA3 的材质置换与背景深度分支。DA3 修改器公开以下参数： `O MoGe2 Depth Plane` 与 DA3 一样通过隐藏的 Color Image 读取 `.color` Alpha，以 `Alpha > 0.5`
划分前景范围，并在范围外改用 `Background Depth`。两个修改器公开以下参数：
- `Displacement`：位于 Geometry 之后的第一项，菜单包含 `Geometry`（默认）与 `Material`。`Geometry` - `Displacement`：位于 Geometry 之后的第一项，菜单包含 `Geometry`（默认）与 `Material`。`Geometry`
在节点组内沿法线修改网格，`Material` 保持网格平面并启用材质置换； 在节点组内沿法线修改网格，`Material` 保持网格平面并启用材质置换；
- `Subdivision`：位于第二项，默认 `5`。节点读取基础平面的 Bounding Box，短边分段数为 `2^Subdivision`， - `Subdivision`：位于第二项，默认 `5`。节点读取基础平面的 Bounding Box，短边分段数为 `2^Subdivision`，
@@ -51,16 +52,16 @@
材质置换使用该值；是否存在背景只取决于转换所得色彩贴图的 `Alpha > 0.5` 二值结果； 材质置换使用该值；是否存在背景只取决于转换所得色彩贴图的 `Alpha > 0.5` 二值结果；
- `Depth Scale`：默认 `1`，允许正负值，Structure Type 为 `Single`，同时控制 Geometry 与 Material - `Depth Scale`：默认 `1`，允许正负值，Structure Type 为 `Single`，同时控制 Geometry 与 Material
两条置换路径。 两条置换路径。
Grid 重建后通过隐藏的 Material 输入重新赋予原图材质，并对所有面执行 Shade Smooth。 Grid 重建后通过隐藏的 Material 输入重新赋予转换材质，并对所有面执行 Shade Smooth。
材质使用显式指向 `UVMap` 的 UV Map 节点连接颜色、前景 Mask 与深度 Image Texture，不依赖活动 UV。 材质使用显式指向 `UVMap` 的 UV Map 节点连接颜色、前景 Mask 与深度 Image Texture，不依赖活动 UV。
前景深度先与 `Background Depth` 取 Maximum，再按色彩贴图 Alpha 与背景深度混合， 前景深度先与 `Background Depth` 取 Maximum，再按色彩贴图 Alpha 与背景深度混合，
深度分支在混合前使用 `Alpha > 0.5` 转为二值 Mask，避免半透明前景被背景深度拉扯； 深度分支在混合前使用 `Alpha > 0.5` 转为二值 Mask，避免半透明前景被背景深度拉扯；
材质表面 Alpha 同样使用二值 Mask。材质置换使用相同的二值和钳制顺序。 材质表面 Alpha 同样使用二值 Mask。材质置换使用相同的二值和钳制顺序。
节点组还会把 `o_material_displacement` 与 `o_background_depth` 写入点域，材质通过 节点组还会把 `o_material_displacement` 与 `o_background_depth` 写入点域，材质通过
`Geometry` 类型的 Attribute 节点读取。点域会为顶点保存重复值，但能在应用修改器后继续保留参数； `Geometry` 类型的 Attribute 节点读取。点域会为顶点保存重复值，但能在应用修改器后继续保留参数；
相比自适应网格本身，其占用很小。Depth Image 与 Foreground Image 是修改器隐藏数据输入， 相比自适应网格本身，其占用很小。Depth Image 与 Color Image 是修改器隐藏数据输入，
由导入流程自动设置。 由导入流程自动设置。
视频与序列帧还会为 Depth 与 Foreground 共同写入隐藏的 `Frame`、`Frame Start`、`Frames` 与 `Cyclic`。 `.color` 与 `.depth` 的 Blender Image 类型只依据源 Empty Image：普通单图使用 `IMAGE`，Movie 或 Sequence 使用 `SEQUENCE`；即使动画源这次只处理一帧也仍为 `SEQUENCE`，不按输出文件数量推断。视频与序列帧还会为 Depth 与 Color 共同写入隐藏的 `Frame`、`Frame Start`、`Frames` 与 `Cyclic`。
节点组内部不再使用 Scene Time。只有原 Image Empty 是视频或序列时，导入流程才直接给修改器的 `Frame` 节点组内部不再使用 Scene Time。只有原 Image Empty 是视频或序列时，导入流程才直接给修改器的 `Frame`
socket 添加 Scripted Expression Driver `frame`；静态图保持默认值 `1`，不创建 Driver。导入内容已经 socket 添加 Scripted Expression Driver `frame`；静态图保持默认值 `1`，不创建 Driver。导入内容已经
是截取后的帧，因此 Offset 在节点组内恒定为 `0`。 是截取后的帧，因此 Offset 在节点组内恒定为 `0`。

12
docs/features.md
View File

@@ -89,18 +89,18 @@
2. 在 3D View 选中 Image Empty，右键打开 `SnapForm` 子菜单并选择 `Remove Background`。 2. 在 3D View 选中 Image Empty，右键打开 `SnapForm` 子菜单并选择 `Remove Background`。
3. 完成后以 Fake User 保留原 Image datablock，新建并 Pack 一张透明 PNG 到当前 `.blend`，然后只把 3. 完成后以 Fake User 保留原 Image datablock，新建并 Pack 一张透明 PNG 到当前 `.blend`，然后只把
当前 Empty 的 `data` 换成新图；Empty 的 Transform、显示尺寸、Offset 及其他设置保持不变。 当前 Empty 的 `data` 换成新图；Empty 的 Transform、显示尺寸、Offset 及其他设置保持不变。
4. 新 Image 使用原图去掉格式后再追加 `.foreground` 的名称。例如 `img.png` 生成 4. 新 Image 使用原图去掉格式后再追加 `.color` 的名称。例如 `img.png` 生成
`img.foreground`，`img.png.001` 生成 `img.001.foreground`。已有同名 datablock 时由 `img.color`，`img.png.001` 生成 `img.001.color`。已有同名 datablock 时由
Blender 自动追加编号。 Blender 自动追加编号。
原图已 Pack 时会先写出临时 PNG 交给 BEN2，任务结束后删除临时输入；原 Image 仍保持 Pack 原图已 Pack 时会先写出临时 PNG 交给 BEN2，任务结束后删除临时输入；原 Image 仍保持 Pack
状态和原始内容。BEN2 的 `foreground.png` 也会保留在对应 job 目录中。 状态和原始内容。正式任务的 `color.0001.png` 也会保留在对应 job 目录中。
Image Empty 引用视频或序列帧时，Remove Background 直接执行，默认取 Empty 的 Image Frames Image Empty 引用视频或序列帧时，Remove Background 直接执行，默认取 Empty 的 Image Frames
数值（未设置时为完整帧数，显式设为 1 时只处理 1 帧），起始帧跟随 Empty 的 Offset；若 Offset 加 Frames 超出视频长度， 数值（未设置时为完整帧数，显式设为 1 时只处理 1 帧），起始帧跟随 Empty 的 Offset；若 Offset 加 Frames 超出视频长度，
会把窗口整体收进视频内（保证实际处理 Frames 帧数），Offset 过小时从第 1 帧开始。输出 会把窗口整体收进视频内（保证实际处理 Frames 帧数），Offset 过小时从第 1 帧开始。输出
`foreground.0001.png` 等透明序列帧（帧号从 0001 开始，与 Blender 场景帧一一对应），并把 `color.0001.png` 等透明序列帧（帧号从 0001 开始，与 Blender 场景帧一一对应），并把
源帧以 `frame.0001.jpg` 等命名复制到 job 目录根下供材质颜色导入。替换 Empty 的 Image 为 关闭去背景时，原始颜色帧同样以 `color.0001.*` 起始的统一名称写入 job 目录。替换 Empty 的 Image 为
加载这些帧的 SEQUENCE Image，从原 Empty 的 Start 位置开始播放（Start 原样保留）， 加载这些帧的 SEQUENCE Image，从原 Empty 的 Start 位置开始播放（Start 原样保留）。图片类型只取决于源 Empty：普通单图的单个 `color.0001.*` 仍作为 `IMAGE`，Movie/Sequence 即使只导出一帧也作为 `SEQUENCE`；`.depth` 同样遵循该规则。
Cyclic 沿用原 Empty 的设置。序列帧无法被 Blender 整体 Pack，因此保持外部引用 job 目录； Cyclic 沿用原 Empty 的设置。序列帧无法被 Blender 整体 Pack，因此保持外部引用 job 目录；
请保留对应 job 目录，否则序列会失效。 请保留对应 job 目录，否则序列会失效。

12
docs/outputs.md
View File

@@ -22,11 +22,13 @@ Depth Plane 导入所需的贴图。这里的文件转换属于正式 Depth 工
## Surface ## Surface
generate-surface 统一写出 mesh.npz 与 texture.png。DA3 与 MoGe-2 都使用 generate-surface 统一写出 `mesh.npz`，不再生成 `texture.png`。DA3 与 MoGe-2 都使用
geometry/surface.py 的同一套 Surface 构建算法；序列化位于 outputs/surface.py。 `geometry/surface.py` 的同一套 Surface 构建算法；序列化位于 `outputs/surface.py`。MoGe-2 normal
MoGe-2 normal 另存为 normal.png 并由 Blender 材质加载，不写入几何 NPZ。 仍由 Server 单独导出为 `normal.png`，Blender 只负责加载并用于材质，不写入几何 NPZ。未去背景时
无论颜色来自原图复制还是 BEN2，Server 都统一写为 `color.0001.*` 起始的颜色层。Blender 不依据文件数量判断图片类型：源 Empty 为普通单图时载入为 `IMAGE`，源 Empty 为 Movie/Sequence 时即使只有一帧也载入为 `SEQUENCE`；`.depth` 使用相同规则。材质只在 Blender
端创建。MoGe-2 的 `normal.png` 仍由 Server 生成，Blender 只负责加载使用。
## BEN2 ## BEN2
正式 remove-background 工作流输出 foreground.png 或透明前景序列。BEN2 ONNX API 返回的 Alpha 正式 remove-background 工作流输出 `color.0001.png` 起始的透明颜色层。BEN2 ONNX API 返回的 Alpha
恢复到输入分辨率，RGB 保留输入图像颜色。 恢复到输入分辨率，RGB 保留输入图像颜色。Debug BEN2 仍保留模型 API 语义，使用 `foreground` 文件名。

40
docs/surface.md
View File

@@ -9,7 +9,7 @@
Resolution Scale 与 Depth Plane 使用相同设计；MoGe-2 改为显示共享的 Resolution Level。 Resolution Scale 与 Depth Plane 使用相同设计；MoGe-2 改为显示共享的 Resolution Level。
2. Server 先用未经抠图的原始 RGB 请求 DA3 的 `depth`、`conf` 与 `intrinsics`，随后仅在开启 2. Server 先用未经抠图的原始 RGB 请求 DA3 的 `depth`、`conf` 与 `intrinsics`，随后仅在开启
`Remove Background` 时运行 BEN2 取得前景 Alpha；Alpha 经目标网格分辨率缩放与轻微模糊后，只要 `Remove Background` 时运行 BEN2 取得前景 Alpha；Alpha 经目标网格分辨率缩放与轻微模糊后，只要
大于 `0` 就把对应点视为前景；每个三角面只要有一个前景点就保留，三个点均为背景时才在输出前删除； 大于 `0` 就把对应点视为前景；每个三角形的三个顶点必须全部为前景，任一顶点 Alpha 为 `0` 就在输出前删除该面；
关闭时不加载 BEN2。Depth 不做 smoothing，Server 反投影深度网格，建立源三角面， 关闭时不加载 BEN2。Depth 不做 smoothing，Server 反投影深度网格，建立源三角面，
计算 confidence percentile 与 3×3 相对深度不连续度，并写入 计算 confidence percentile 与 3×3 相对深度不连续度，并写入
基于原图像素坐标的源 `UVMap`。这些步骤只准备修改器的输入网格，不执行 Relax、拓扑修复、平滑、 基于原图像素坐标的源 `UVMap`。这些步骤只准备修改器的输入网格，不执行 Relax、拓扑修复、平滑、
@@ -44,11 +44,13 @@
Group Input，并隐藏没有连线的接口，避免跨越整个节点图的输入连线。 Group Input，并隐藏没有连线的接口，避免跨越整个节点图的输入连线。
`O DA3 Surface` 与 `O MoGe2 Surface` 的 `Iterations` 默认值均为 `2`、最小值为 `0`， `O DA3 Surface` 与 `O MoGe2 Surface` 的 `Iterations` 默认值均为 `2`、最小值为 `0`，
`Resolution` 默认值均为 `0.1`，所有输入 socket 的 Structure Type 均固定为 `Single`。 `Resolution` 默认值均为 `0.1`，所有输入 socket 的 Structure Type 均固定为 `Single`。
5. Surface Job 统一按输入准备、模型推理、可选 BEN2、构建 SurfaceMesh、写出产物五个阶段更新进度； 5. Surface Job 统一按输入准备、模型推理、可选 BEN2、构建 SurfaceMesh、写出产物五个阶段更新进度；单帧推理只显示模型与阶段名称，不追加 `1/1`，仅多帧时显示当前帧与总数；
Surface Builder 是不接收 Job Context 或 progress callback 的纯数据转换。 Surface Builder 是不接收 Job Context 或 progress callback 的纯数据转换。
6. Server 在 job 根目录写出 `mesh.npz` 与 `texture.png`。NPZ 保存 camera-space `vertices`、 6. Server 在 job 根目录写出 `mesh.npz`。NPZ 保存 camera-space `vertices`、`faces`、`uv`、
`faces`、`uv`、`intrinsics`、`image_size` 与有效深度中位数 `reference_depth`。Blender 读取这些 `intrinsics`、`image_size` 与有效深度中位数 `reference_depth`。颜色层无论来自原图复制还是 BEN2，
数据后直接创建 Surface；材质默认使用原图，开启 Remove Background 时改用 `.foreground`，不需要 Open3D 或 OpenCV 进入 Blender Python 环境。 Server 都统一写为 `color.0001.*`。Blender 读取网格数据后额外导入为 `.color`，材质不复用源
Empty 的 Image，Color Space 固定为 `AgX Base sRGB`。Server 不生成 `texture.png`，材质只在
Blender 端创建。
DA3 Surface 只保留 Geometry Nodes 后处理实际使用的 attributes：点域 `o_depth`、面域 DA3 Surface 只保留 Geometry Nodes 后处理实际使用的 attributes：点域 `o_depth`、面域
`o_confidence` 与面域 `o_depth_discontinuity`。Remove Background 已在 Server `o_confidence` 与面域 `o_depth_discontinuity`。Remove Background 已在 Server
建面前直接删除背景三角面，不写入任何 background attribute。MoGe-2 的导入网格保留点域 `o_depth`， 建面前直接删除背景三角面，不写入任何 background attribute。MoGe-2 的导入网格保留点域 `o_depth`，
@@ -62,11 +64,11 @@
缩放及父级关系产生的世界变换保持一致。更近的顶点沿 Empty 局部 `+Z` 方向凸出，三角面正面朝向 缩放及父级关系产生的世界变换保持一致。更近的顶点沿 Empty 局部 `+Z` 方向凸出，三角面正面朝向
观察相机，启用 Backface Culling 时可从正面看到。 观察相机，启用 Backface Culling 时可从正面看到。
8. 转换对话框不再提供后处理参数，只保留模型与 Resolution Scale 设置、`Create Matching Camera`、 8. 转换对话框不再提供后处理参数，只保留模型与 Resolution Scale 设置、`Create Matching Camera`、
`Remove Background`、`Unshaded`，以及分割线下方的 `Keep Original`。背景移除默认关闭；开启后 Server `Remove Background`、`Shadeless`，以及分割线下方的 `Keep Original`。背景移除默认关闭；开启后 Server
使用 BEN2，并在写出 Surface 前直接删除背景三角面，不再由 Geometry Nodes 的 Background 开关处理。 使用 BEN2，并在写出 Surface 前直接删除背景三角面，不再由 Geometry Nodes 的 Background 开关处理。
该预删除同时适用于 DA3 与 MoGe-2；因此 MoGe-2 转换后即使关闭 `Mask`，BEN2 已删除的背景也不会恢复。 该预删除同时适用于 DA3 与 MoGe-2；因此 MoGe-2 转换后即使关闭 `Mask`，BEN2 已删除的背景也不会恢复。
开启 Remove Background 时，Surface 材质的 Color Image Texture 使用打包后的 `.foreground`，原 Empty 仍保持原图。 Surface 材质的 Color Image Texture 始终使用 `.color`；Remove Background 只决定该层是否带 BEN2 Alpha，原 Empty 始终保持原图。颜色层使用 `AgX Base sRGB`。
`Unshaded` 使用与 Depth Plane 相同的 Emission/Transparent 材质路径；普通材质的 `Shadeless` 使用与 Depth Plane 相同的 `O Shadeless` 节点组，并继续通过 Alpha 与 Transparent BSDF 混合；普通材质的
Bump Scale 按 `abs(object.scale.z) × camera scale_z` 设置，并在材质组内乘以固定基准 `0.001`， Bump Scale 按 `abs(object.scale.z) × camera scale_z` 设置，并在材质组内乘以固定基准 `0.001`，
与透视表面实际深度尺度一致；它允许负数以反转凹凸方向。 与透视表面实际深度尺度一致；它允许负数以反转凹凸方向。
相机默认关闭；开启后会在 Empty 局部 `+Z` 侧创建一台朝向局部 `-Z` 的透视相机，并成为生成 Surface 相机默认关闭；开启后会在 Empty 局部 `+Z` 侧创建一台朝向局部 `-Z` 的透视相机，并成为生成 Surface
@@ -122,7 +124,7 @@ Displacement 设置为 `Displacement Only`，DA3 与 Original 都使用 `Midleve
MoGe-2 Surface 使用独立的 `O MoGe2 Surface Layer`；它额外接收模型输出的 `Normal` 贴图，使用 MoGe-2 Surface 使用独立的 `O MoGe2 Surface Layer`；它额外接收模型输出的 `Normal` 贴图，使用
`Normal Map` 节点按 `Object Space` 解码后连接到 Bump 的 `Normal` 输入，使像素级模型法线与原图 `Normal Map` 节点按 `Object Space` 解码后连接到 Bump 的 `Normal` 输入，使像素级模型法线与原图
Bump 合并，不依赖重拓扑后的顶点密度；`Normal Strength` 默认值为 `1.0`，只调节模型 normal map， Bump 合并，不依赖重拓扑后的顶点密度；`Normal Strength` 默认值为 `1.0`，只调节模型 normal map，
原图 Bump 仍由独立的 `Bump Scale` 控制； 颜色贴图 Bump 仍由独立的 `Bump Scale` 控制；
Depth Plane 使用 `O DA3 Depth Plane Layer` 材质组，并把背景剔除、背景深度与材质置换 Attribute Depth Plane 使用 `O DA3 Depth Plane Layer` 材质组，并把背景剔除、背景深度与材质置换 Attribute
读取全部收进组内。两者都接收 `Color`、`Alpha` 与 `Bump Scale`，只有 `O DA3 Depth Plane Layer` 读取全部收进组内。两者都接收 `Color`、`Alpha` 与 `Bump Scale`，只有 `O DA3 Depth Plane Layer`
额外接收 Float 类型的 `Depth`。 额外接收 Float 类型的 `Depth`。
@@ -138,9 +140,10 @@ Depth Plane 的 Bump Height 直接使用材质组的 `Color` 输入，只受独
`0.001`，会乘以面片 Transform 的统一缩放倍率。原图 Color 还会经过 `Float Curve` 连接到 `0.001`，会乘以面片 Transform 的统一缩放倍率。原图 Color 还会经过 `Float Curve` 连接到
Principled BSDF 的 `Roughness`，曲线两端输出分别为 `0.5` 和 `0.1`；材质 IOR 保持 Principled BSDF 的 `Roughness`，曲线两端输出分别为 `0.5` 和 `0.1`；材质 IOR 保持
Principled BSDF 默认值 `1.5`。 Principled BSDF 默认值 `1.5`。
开启 `Unshaded` 后，色彩贴图 Color 连接 Emission，二值 Alpha 作为 Mix Shader 系数，在 开启 `Shadeless` 后，色彩贴图 Color 接入 `O Shadeless`。该组沿用 Blender Images as Planes 的
Transparent BSDF 与 Emission 之间混合后连接 Material Output；不创建 Principled BSDF、Bump 或 Diffuse、Emission、Light Path 与射线判断连接；二值 Alpha 仍作为 Mix Shader 系数，在 Transparent BSDF
Float Curve。开启 `Remove Background` 时色彩贴图使用 BEN2 输出；关闭时使用原图。 与 `O Shadeless` 之间混合后连接 Material Output。外层不再创建 Emission、Principled BSDF、Bump 或
Float Curve。开启 `Remove Background` 时色彩贴图使用 `.color`；关闭时使用 `.color`。
这里的“原始距离”是 DA3 模型输出的尺度。普通 Small、Base、Large 和 Giant 模型的深度不应 这里的“原始距离”是 DA3 模型输出的尺度。普通 Small、Base、Large 和 Giant 模型的深度不应
被解释为现实世界的米；需要现实尺度时应选择带 metric depth 的 被解释为现实世界的米；需要现实尺度时应选择带 metric depth 的
@@ -150,7 +153,7 @@ Float Curve。开启 `Remove Background` 时色彩贴图使用 BEN2 输出；关
生成完成后会直接创建面片，不提供缓存重建入口或单独的导入参数。重新选择 生成完成后会直接创建面片，不提供缓存重建入口或单独的导入参数。重新选择
`Convert to Depth Plane` 会按当前 Depth from 再次转换。 `Convert to Depth Plane` 会按当前 Depth from 再次转换。
面片、深度 Image、前景 Image、Material 与 Geometry Nodes 修改器会在同一个 Blender Undo 步骤中创建；转换完成后执行 面片、深度 Image、颜色 Image、Material 与 Geometry Nodes 修改器会在同一个 Blender Undo 步骤中创建；转换完成后执行
`Ctrl+Z` 会完整撤销本次转换，不会越过转换去回退更早的 Image Empty 操作。 `Ctrl+Z` 会完整撤销本次转换，不会越过转换去回退更早的 Image Empty 操作。
Image Empty 使用 Packed Image 时，SnapForm 会通过 Blender 写出临时 PNG 交给 DA3， Image Empty 使用 Packed Image 时，SnapForm 会通过 Blender 写出临时 PNG 交给 DA3，
@@ -161,7 +164,7 @@ Image Empty 使用 Packed Image 时，SnapForm 会通过 Blender 写出临时 PN
生成的 Object、Mesh 与 Material 仍命名为 `obj_img`。Pack 使用 OpenEXR，因而不会把大于 `1.0` 生成的 Object、Mesh 与 Material 仍命名为 `obj_img`。Pack 使用 OpenEXR，因而不会把大于 `1.0`
的原始深度截断。已有同名 datablock 时由 Blender 自动追加 `.001` 等编号。 的原始深度截断。已有同名 datablock 时由 Blender 自动追加 `.001` 等编号。
视频输入时 Blender 无法整体 Pack 序列帧，因此 `.depth` 与颜色序列改为引用 job 目录根下的 视频输入时 Blender 无法整体 Pack 序列帧，因此 `.depth` 与颜色序列改为引用 job 目录根下的
`depth.0001.exr`、`frame.0001.jpg`（`.frame`）或 `foreground.0001.png`，并在每帧保留原始 `depth.0001.exr`、`color.0001.jpg`（`.color`）或 `color.0001.png`，并在每帧保留原始
浮点深度；保存 `.blend` 后再次打开仍需对应 job 目录存在。 浮点深度；保存 `.blend` 后再次打开仍需对应 job 目录存在。
每个 Server Job 被接受时，Job Runtime 在 Runtime Storage Root 的 `jobs/` 下创建一个独立文件夹。文件夹名是 32 位 opaque Job ID，不包含任务类型或对象名称。执行逻辑从 `JobContext.directory` 获取路径，输出产物直接写入该目录。 每个 Server Job 被接受时，Job Runtime 在 Runtime Storage Root 的 `jobs/` 下创建一个独立文件夹。文件夹名是 32 位 opaque Job ID，不包含任务类型或对象名称。执行逻辑从 `JobContext.directory` 获取路径，输出产物直接写入该目录。
@@ -176,14 +179,13 @@ Job 目录中的典型正式产物如下：
```text ```text
<opaque-job-id>/ <opaque-job-id>/
├── results.npz、mesh.npz、texture.png 或 normal.png ├── results.npz、mesh.npz 或 normal.png
└── prediction.npz Debug 原始 API 返回 └── prediction.npz Debug 原始 API 返回
<opaque-job-id>/ <opaque-job-id>/
├── foreground.png 单图结果 ├── color.0001.jpg/png… 正式工作流颜色层（原图复制或 BEN2 Alpha）
├── foreground.0001.png… 视频/序列帧结果 ├── depth.0001.exr… AI 深度序列帧
├── frame.0001.jpg… 视频/序列帧的源帧副本 └── normal.png MoGe-2 Surface 法线贴图
└── depth.0001.exr… AI 深度序列帧
``` ```
建议从 `DA3 Base` 开始。它使用 Apache 2.0 模型权重。 建议从 `DA3 Base` 开始。它使用 Apache 2.0 模型权重。

BIN
src/snapform/assets/O_SnapForm.blend LFS
View File

Binary file not shown.

242
src/snapform/blender/depth_plane.py
View File

@@ -23,6 +23,8 @@ SURFACE_MATERIAL_NODE_GROUP_NAME = "O DA3 Surface Layer"
MOGE_SURFACE_MATERIAL_NODE_GROUP_NAME = "O MoGe2 Surface Layer" MOGE_SURFACE_MATERIAL_NODE_GROUP_NAME = "O MoGe2 Surface Layer"
DEPTH_PLANE_MATERIAL_NODE_GROUP_NAME = "O DA3 Depth Plane Layer" DEPTH_PLANE_MATERIAL_NODE_GROUP_NAME = "O DA3 Depth Plane Layer"
MOGE_MATERIAL_NODE_GROUP_NAME = "O MoGe2 Depth Plane Layer" MOGE_MATERIAL_NODE_GROUP_NAME = "O MoGe2 Depth Plane Layer"
SHADELESS_NODE_GROUP_NAME = "O Shadeless"
MATERIAL_COLOR_SPACE = "AgX Base sRGB"
DA3_PATCH_SIZE = 14 DA3_PATCH_SIZE = 14
DA3_MIN_PROCESS_RES = 252 DA3_MIN_PROCESS_RES = 252
DA3_MAX_PROCESS_RES = 1008 DA3_MAX_PROCESS_RES = 1008
@@ -183,8 +185,8 @@ def copy_depth_image(source_object):
return depth_image return depth_image
def removed_background_data_name(source_object): def color_data_name(source_object):
return f"{image_base_name(source_object.data)}.foreground" return f"{image_base_name(source_object.data)}.color"
def scaled_process_resolution(image, scale_percent): def scaled_process_resolution(image, scale_percent):
@@ -328,7 +330,7 @@ def configure_image_user(
def pack_sequence_if_possible(image, expected_frames): def pack_sequence_if_possible(image, expected_frames):
"""Pack a frame sequence when Blender supports it; otherwise keep external files.""" """Pack a frame sequence when Blender supports it; otherwise keep external files."""
if expected_frames <= 0: if expected_frames <= 1:
return False return False
try: try:
image.pack() image.pack()
@@ -455,81 +457,60 @@ def create_depth_sequence_image(name, depth_frames, scene, frame_directory):
return sequence_image return sequence_image
def create_removed_background_image( def create_color_image(
source_object, source_object,
output_path, output_directory,
assign_to_source=True, assign_to_source=True,
): ):
output_path = Path(output_path) output_directory = Path(output_directory)
if output_path.is_dir(): color_paths = sorted_frame_paths(output_directory, "color.")
foreground_paths = sorted_frame_paths(output_path, "foreground.") if not color_paths:
if not foreground_paths:
raise RuntimeError(
f"Background removal result has no foreground frames: "
f"{output_path}"
)
try:
result_image = bpy.data.images.load(
str(foreground_paths[0]),
check_existing=False,
)
result_image.alpha_mode = "CHANNEL_PACKED"
force_sequence_source(result_image)
except (OSError, RuntimeError) as error:
if "result_image" in locals():
bpy.data.images.remove(result_image, do_unlink=True)
raise RuntimeError("Unable to load the background result") from error
source_image = source_object.data
original_fake_user = source_image.use_fake_user
try:
result_image.name = removed_background_data_name(source_object)
pack_sequence_if_possible(result_image, len(foreground_paths))
if assign_to_source:
image_user = getattr(source_object, "image_user", None)
(
original_frame_start,
_original_frame_offset,
_original_frame_duration,
original_use_cyclic,
) = image_user_settings(source_object)
source_image.use_fake_user = True
source_object.data = result_image
configure_image_user(
image_user,
len(foreground_paths),
frame_start=original_frame_start,
use_cyclic=original_use_cyclic,
)
except (OSError, RuntimeError) as error:
source_image.use_fake_user = original_fake_user
bpy.data.images.remove(result_image)
raise RuntimeError("Unable to load the background result") from error
return result_image
if not output_path.is_file():
raise RuntimeError( raise RuntimeError(
f"Background removal result does not exist: {output_path}" f"Color result has no images: {output_directory}"
) )
source_image = source_object.data
animated = is_animated_image(source_image)
try: try:
result_image = bpy.data.images.load( result_image = bpy.data.images.load(
str(output_path), str(color_paths[0]),
check_existing=False, check_existing=False,
) )
result_image.alpha_mode = "CHANNEL_PACKED" result_image.alpha_mode = "CHANNEL_PACKED"
except RuntimeError as error: if animated:
raise RuntimeError("Unable to load the background result") from error force_sequence_source(result_image)
source_image = source_object.data except (OSError, RuntimeError) as error:
if "result_image" in locals():
bpy.data.images.remove(result_image, do_unlink=True)
raise RuntimeError("Unable to load the color result") from error
original_fake_user = source_image.use_fake_user original_fake_user = source_image.use_fake_user
try: try:
result_image.name = removed_background_data_name(source_object) result_image.name = color_data_name(source_object)
result_image.pack() if animated:
pack_sequence_if_possible(result_image, len(color_paths))
else:
result_image.pack()
if assign_to_source: if assign_to_source:
image_user = getattr(source_object, "image_user", None)
(
original_frame_start,
_original_frame_offset,
_original_frame_duration,
original_use_cyclic,
) = image_user_settings(source_object)
source_image.use_fake_user = True source_image.use_fake_user = True
source_object.data = result_image source_object.data = result_image
if animated:
configure_image_user(
image_user,
len(color_paths),
frame_start=original_frame_start,
use_cyclic=original_use_cyclic,
)
except (OSError, RuntimeError) as error: except (OSError, RuntimeError) as error:
source_image.use_fake_user = original_fake_user source_image.use_fake_user = original_fake_user
bpy.data.images.remove(result_image) bpy.data.images.remove(result_image, do_unlink=True)
raise RuntimeError("Unable to pack the background result") from error raise RuntimeError("Unable to prepare the color result") from error
return result_image return result_image
@@ -993,7 +974,7 @@ def create_moge_material(
color_image, color_image,
depth_image, depth_image,
depth_scale, depth_scale,
unshaded=False, shadeless=False,
frame_count=1, frame_count=1,
frame_start=1, frame_start=1,
use_cyclic=False, use_cyclic=False,
@@ -1009,6 +990,7 @@ def create_moge_material(
uv.uv_map = "UVMap" uv.uv_map = "UVMap"
uv.location = (-900.0, 200.0) uv.location = (-900.0, 200.0)
color = nodes.new("ShaderNodeTexImage") color = nodes.new("ShaderNodeTexImage")
color_image.colorspace_settings.name = MATERIAL_COLOR_SPACE
color.image = color_image color.image = color_image
color.interpolation = "Linear" color.interpolation = "Linear"
color.extension = "EXTEND" color.extension = "EXTEND"
@@ -1043,14 +1025,16 @@ def create_moge_material(
links.new(color.outputs["Alpha"], layer.inputs["Alpha"]) links.new(color.outputs["Alpha"], layer.inputs["Alpha"])
links.new(depth.outputs["Color"], layer.inputs["Depth"]) links.new(depth.outputs["Color"], layer.inputs["Depth"])
layer.inputs["Bump Scale"].default_value = depth_scale layer.inputs["Bump Scale"].default_value = depth_scale
if unshaded: if shadeless:
shader = nodes.new("ShaderNodeEmission") shader = nodes.new("ShaderNodeGroup")
shader.node_tree = load_shadeless_node_group()
shader.name = SHADELESS_NODE_GROUP_NAME
transparent = nodes.new("ShaderNodeBsdfTransparent") transparent = nodes.new("ShaderNodeBsdfTransparent")
mix = nodes.new("ShaderNodeMixShader") mix = nodes.new("ShaderNodeMixShader")
links.new(layer.outputs["Color"], shader.inputs["Color"]) links.new(layer.outputs["Color"], shader.inputs["Color"])
links.new(layer.outputs["Alpha"], mix.inputs["Fac"]) links.new(layer.outputs["Alpha"], mix.inputs["Fac"])
links.new(transparent.outputs["BSDF"], mix.inputs[1]) links.new(transparent.outputs["BSDF"], mix.inputs[1])
links.new(shader.outputs["Emission"], mix.inputs[2]) links.new(shader.outputs["Shader"], mix.inputs[2])
links.new(mix.outputs["Shader"], output.inputs["Surface"]) links.new(mix.outputs["Shader"], output.inputs["Surface"])
return material return material
shader = nodes.new("ShaderNodeBsdfPrincipled") shader = nodes.new("ShaderNodeBsdfPrincipled")
@@ -1307,16 +1291,13 @@ def create_material(
color_image, color_image,
depth_scale, depth_scale,
displacement_image=None, displacement_image=None,
foreground_image=None,
displacement_midlevel=0.0, displacement_midlevel=0.0,
displacement_scale=1.0, displacement_scale=1.0,
unshaded=False, shadeless=False,
frame_count=1, frame_count=1,
frame_start=1, frame_start=1,
color_frame_offset=0, color_frame_offset=0,
color_frame_duration=None, color_frame_duration=None,
foreground_frame_offset=0,
foreground_frame_duration=None,
depth_frame_offset=0, depth_frame_offset=0,
depth_frame_duration=None, depth_frame_duration=None,
use_cyclic=None, use_cyclic=None,
@@ -1336,7 +1317,8 @@ def create_material(
uv_map.uv_map = "UVMap" uv_map.uv_map = "UVMap"
uv_map.location = (-1250.0, 350.0) uv_map.location = (-1250.0, 350.0)
material_image = foreground_image or color_image material_image = color_image
material_image.colorspace_settings.name = MATERIAL_COLOR_SPACE
color_texture.image = material_image color_texture.image = material_image
color_texture.interpolation = "Linear" color_texture.interpolation = "Linear"
color_texture.extension = "EXTEND" color_texture.extension = "EXTEND"
@@ -1355,13 +1337,6 @@ def create_material(
) )
material_frames = color_frames material_frames = color_frames
material_frame_offset = color_frame_offset material_frame_offset = color_frame_offset
if foreground_image is not None:
material_frames = (
foreground_frame_duration
if foreground_frame_duration is not None
else extracted_frames
)
material_frame_offset = foreground_frame_offset
if frame_count > 1 or is_animated_image(material_image): if frame_count > 1 or is_animated_image(material_image):
configure_image_user( configure_image_user(
color_texture.image_user, color_texture.image_user,
@@ -1394,16 +1369,18 @@ def create_material(
normal_texture.location = (-1000.0, -50.0) normal_texture.location = (-1000.0, -50.0)
links.new(uv_map.outputs["UV"], normal_texture.inputs["Vector"]) links.new(uv_map.outputs["UV"], normal_texture.inputs["Vector"])
links.new(normal_texture.outputs["Color"], material_inputs.inputs["Normal"]) links.new(normal_texture.outputs["Color"], material_inputs.inputs["Normal"])
if unshaded: if shadeless:
emission = nodes.new("ShaderNodeEmission") shader = nodes.new("ShaderNodeGroup")
shader.node_tree = load_shadeless_node_group()
shader.name = SHADELESS_NODE_GROUP_NAME
transparent = nodes.new("ShaderNodeBsdfTransparent") transparent = nodes.new("ShaderNodeBsdfTransparent")
mix_shader = nodes.new("ShaderNodeMixShader") mix_shader = nodes.new("ShaderNodeMixShader")
emission.location = (0.0, 550.0) shader.location = (0.0, 550.0)
transparent.location = (0.0, 150.0) transparent.location = (0.0, 150.0)
mix_shader.location = (300.0, 350.0) mix_shader.location = (300.0, 350.0)
links.new( links.new(
material_inputs.outputs["Color"], material_inputs.outputs["Color"],
emission.inputs["Color"], shader.inputs["Color"],
) )
links.new( links.new(
material_inputs.outputs["Alpha"], material_inputs.outputs["Alpha"],
@@ -1414,7 +1391,7 @@ def create_material(
mix_shader.inputs[1], mix_shader.inputs[1],
) )
links.new( links.new(
emission.outputs["Emission"], shader.outputs["Shader"],
mix_shader.inputs[2], mix_shader.inputs[2],
) )
links.new( links.new(
@@ -1502,13 +1479,13 @@ def node_asset_path():
return Path(__file__).parent.parent / "assets" / NODE_ASSET_NAME return Path(__file__).parent.parent / "assets" / NODE_ASSET_NAME
def load_depth_plane_node_group(group_name=DEPTH_PLANE_NODE_GROUP_NAME): def load_asset_node_group(group_name):
existing = bpy.data.node_groups.get(group_name) existing = bpy.data.node_groups.get(group_name)
if existing is not None: if existing is not None:
return existing return existing
asset_path = node_asset_path() asset_path = node_asset_path()
if not asset_path.is_file(): if not asset_path.is_file():
raise RuntimeError(f"Missing Geometry Nodes asset: {asset_path}") raise RuntimeError(f"Missing node asset: {asset_path}")
use_pack = bpy.app.version >= (5, 0, 0) use_pack = bpy.app.version >= (5, 0, 0)
if not use_pack: if not use_pack:
directory = str(asset_path) + "/NodeTree/" directory = str(asset_path) + "/NodeTree/"
@@ -1520,18 +1497,15 @@ def load_depth_plane_node_group(group_name=DEPTH_PLANE_NODE_GROUP_NAME):
do_reuse_local_id=True, do_reuse_local_id=True,
) )
if "FINISHED" not in result: if "FINISHED" not in result:
raise RuntimeError("Unable to append the Geometry Nodes asset") raise RuntimeError("Unable to append the node asset")
node_group = bpy.data.node_groups.get(group_name) node_group = bpy.data.node_groups.get(group_name)
if node_group is None: if node_group is None:
raise RuntimeError("Appended Geometry Nodes asset is unavailable") raise RuntimeError("Appended node asset is unavailable")
node_group.use_fake_user = True node_group.use_fake_user = True
return node_group return node_group
with bpy.data.libraries.load(str(asset_path), link=True) as (source, target): with bpy.data.libraries.load(str(asset_path), link=True) as (source, target):
if group_name not in source.node_groups: if group_name not in source.node_groups:
raise RuntimeError( raise RuntimeError(f"Node asset is missing {group_name}")
"Geometry Nodes asset is missing "
f"{group_name}"
)
target.node_groups = [group_name] target.node_groups = [group_name]
linked_group = target.node_groups[0] linked_group = target.node_groups[0]
node_group = bpy.data.pack_linked_ids_hierarchy(root_id=linked_group) node_group = bpy.data.pack_linked_ids_hierarchy(root_id=linked_group)
@@ -1541,6 +1515,14 @@ def load_depth_plane_node_group(group_name=DEPTH_PLANE_NODE_GROUP_NAME):
return node_group return node_group
def load_shadeless_node_group():
return load_asset_node_group(SHADELESS_NODE_GROUP_NAME)
def load_depth_plane_node_group(group_name=DEPTH_PLANE_NODE_GROUP_NAME):
return load_asset_node_group(group_name)
def modifier_input_identifier(node_group, name): def modifier_input_identifier(node_group, name):
for item in node_group.interface.items_tree: for item in node_group.interface.items_tree:
if item.item_type != "SOCKET" or item.in_out != "INPUT": if item.item_type != "SOCKET" or item.in_out != "INPUT":
@@ -1600,7 +1582,7 @@ def drive_modifier_input(owner_object, modifier, identifier, expression="frame")
def add_depth_plane_processing( def add_depth_plane_processing(
plane_object, plane_object,
depth_image, depth_image,
foreground_image, color_image,
material, material,
frame_start=1, frame_start=1,
frame_duration=1, frame_duration=1,
@@ -1616,7 +1598,7 @@ def add_depth_plane_processing(
modifier.show_group_selector = False modifier.show_group_selector = False
for name, value in ( for name, value in (
("Depth Image", depth_image), ("Depth Image", depth_image),
("Foreground Image", foreground_image), ("Color Image", color_image),
("Material", material), ("Material", material),
("Frame", 1), ("Frame", 1),
("Frame Start", frame_start), ("Frame Start", frame_start),
@@ -1635,6 +1617,7 @@ def add_depth_plane_processing(
def add_moge_depth_plane_processing( def add_moge_depth_plane_processing(
plane_object, plane_object,
depth_image, depth_image,
color_image,
material, material,
frame_start=1, frame_start=1,
frame_duration=1, frame_duration=1,
@@ -1652,6 +1635,7 @@ def add_moge_depth_plane_processing(
modifier.show_group_selector = False modifier.show_group_selector = False
for name, value in ( for name, value in (
("Depth Image", depth_image), ("Depth Image", depth_image),
("Color Image", color_image),
("Material", material), ("Material", material),
("Frame", 1), ("Frame", 1),
("Frame Start", frame_start), ("Frame Start", frame_start),
@@ -1688,9 +1672,8 @@ def build_image_depth_plane(
context, context,
source_object, source_object,
delete_source, delete_source,
unshaded=False, shadeless=False,
color_image=None, color_image=None,
foreground_image=None,
depth_frames_directory=None, depth_frames_directory=None,
depth_image=None, depth_image=None,
): ):
@@ -1701,9 +1684,10 @@ def build_image_depth_plane(
frames_image = create_sequence_image( frames_image = create_sequence_image(
depth_data_name(source_object), depth_data_name(source_object),
depth_frames_directory, depth_frames_directory,
frame_prefix="color.",
) )
frame_count = len( frame_count = len(
sorted_frame_paths(depth_frames_directory, "frame.") sorted_frame_paths(depth_frames_directory, "color.")
) )
( (
source_frame_start, source_frame_start,
@@ -1722,9 +1706,9 @@ def build_image_depth_plane(
depth_source_image = frames_image depth_source_image = frames_image
elif not is_animated_image(source_object.data): elif not is_animated_image(source_object.data):
depth_source_image = copy_depth_image(source_object) depth_source_image = copy_depth_image(source_object)
foreground_image = ( material_image = color_image
foreground_image or color_image or source_object.data if material_image is None:
) raise RuntimeError("Depth Plane material image is required")
mesh = create_plane_mesh(source_object, subdivision_level=None) mesh = create_plane_mesh(source_object, subdivision_level=None)
plane_object = bpy.data.objects.new(source_name, mesh) plane_object = bpy.data.objects.new(source_name, mesh)
context.collection.objects.link(plane_object) context.collection.objects.link(plane_object)
@@ -1733,19 +1717,16 @@ def build_image_depth_plane(
depth_scale = abs(float(plane_object.scale.z)) depth_scale = abs(float(plane_object.scale.z))
material = create_material( material = create_material(
source_name, source_name,
color_image if color_image is not None else source_object.data, material_image,
depth_scale, depth_scale,
displacement_image=depth_source_image, displacement_image=depth_source_image,
foreground_image=foreground_image,
displacement_midlevel=ORIGINAL_DISPLACEMENT_MIDLEVEL, displacement_midlevel=ORIGINAL_DISPLACEMENT_MIDLEVEL,
displacement_scale=ORIGINAL_DISPLACEMENT_STRENGTH, displacement_scale=ORIGINAL_DISPLACEMENT_STRENGTH,
unshaded=unshaded, shadeless=shadeless,
frame_count=frame_count, frame_count=frame_count,
frame_start=source_frame_start, frame_start=source_frame_start,
color_frame_offset=texture_frame_offset, color_frame_offset=texture_frame_offset,
color_frame_duration=texture_frame_count, color_frame_duration=texture_frame_count,
foreground_frame_offset=0,
foreground_frame_duration=frame_count,
depth_frame_offset=texture_frame_offset, depth_frame_offset=texture_frame_offset,
depth_frame_duration=texture_frame_count, depth_frame_duration=texture_frame_count,
use_cyclic=source_use_cyclic, use_cyclic=source_use_cyclic,
@@ -1754,7 +1735,7 @@ def build_image_depth_plane(
add_depth_plane_processing( add_depth_plane_processing(
plane_object, plane_object,
depth_source_image, depth_source_image,
foreground_image, material_image,
material, material,
frame_start=source_frame_start, frame_start=source_frame_start,
frame_duration=frame_count, frame_duration=frame_count,
@@ -1775,10 +1756,9 @@ def build_depth_plane(
source_object, source_object,
npz_path, npz_path,
delete_source, delete_source,
unshaded=False, shadeless=False,
depth_name="", depth_name="",
color_image=None, color_image=None,
foreground_image=None,
depth_frames_directory=None, depth_frames_directory=None,
): ):
source_name = source_object.name source_name = source_object.name
@@ -1797,9 +1777,9 @@ def build_depth_plane(
] ]
depth_name = depth_name or depth_data_name(source_object) depth_name = depth_name or depth_data_name(source_object)
frame_count = len(depth_arrays) frame_count = len(depth_arrays)
foreground_image = ( material_image = color_image
foreground_image or color_image or source_object.data if material_image is None:
) raise RuntimeError("Depth Plane material image is required")
( (
source_frame_start, source_frame_start,
source_frame_offset, source_frame_offset,
@@ -1825,7 +1805,10 @@ def build_depth_plane(
depth_name, depth_name,
scaled_depth_arrays[0], scaled_depth_arrays[0],
) )
if frame_count > 1 and depth_frames_directory is not None: if (
is_animated_image(source_object.data)
and depth_frames_directory is not None
):
bpy.data.images.remove(depth_image, do_unlink=True) bpy.data.images.remove(depth_image, do_unlink=True)
depth_image = create_depth_sequence_image( depth_image = create_depth_sequence_image(
depth_name, depth_name,
@@ -1836,21 +1819,18 @@ def build_depth_plane(
material = create_material( material = create_material(
source_name, source_name,
color_image if color_image is not None else source_object.data, material_image,
depth_scale, depth_scale,
displacement_image=depth_image, displacement_image=depth_image,
foreground_image=foreground_image,
displacement_midlevel=DISPLACEMENT_MIDLEVEL, displacement_midlevel=DISPLACEMENT_MIDLEVEL,
displacement_scale=DISPLACEMENT_STRENGTH, displacement_scale=DISPLACEMENT_STRENGTH,
unshaded=unshaded, shadeless=shadeless,
frame_count=frame_count, frame_count=frame_count,
frame_start=source_frame_start, frame_start=source_frame_start,
color_frame_offset=0 if color_image is not None else source_frame_offset, color_frame_offset=0 if color_image is not None else source_frame_offset,
color_frame_duration=( color_frame_duration=(
frame_count if color_image is not None else source_frame_duration frame_count if color_image is not None else source_frame_duration
), ),
foreground_frame_offset=0,
foreground_frame_duration=frame_count,
depth_frame_offset=0, depth_frame_offset=0,
depth_frame_duration=frame_count, depth_frame_duration=frame_count,
use_cyclic=source_use_cyclic, use_cyclic=source_use_cyclic,
@@ -1859,7 +1839,7 @@ def build_depth_plane(
add_depth_plane_processing( add_depth_plane_processing(
plane_object, plane_object,
depth_image, depth_image,
foreground_image, material_image,
material, material,
frame_start=source_frame_start, frame_start=source_frame_start,
frame_duration=frame_count, frame_duration=frame_count,
@@ -1885,9 +1865,8 @@ def build_moge2_depth_plane(
source_object, source_object,
npz_path, npz_path,
delete_source, delete_source,
unshaded=False, shadeless=False,
color_image=None, color_image=None,
foreground_image=None,
data_frames_directory=None, data_frames_directory=None,
): ):
import numpy as np import numpy as np
@@ -1935,7 +1914,10 @@ def build_moge2_depth_plane(
depth_data_name(source_object), depth_data_name(source_object),
displacement[0] * depth_scale, displacement[0] * depth_scale,
) )
if len(depth) > 1 and data_frames_directory is not None: if (
is_animated_image(source_object.data)
and data_frames_directory is not None
):
bpy.data.images.remove(depth_image, do_unlink=True) bpy.data.images.remove(depth_image, do_unlink=True)
depth_image = create_moge_sequence_image( depth_image = create_moge_sequence_image(
depth_data_name(source_object), depth_data_name(source_object),
@@ -1945,12 +1927,15 @@ def build_moge2_depth_plane(
"moge_depth", "moge_depth",
"depth", "depth",
) )
material_image = color_image
if material_image is None:
raise RuntimeError("Depth Plane material image is required")
material = create_moge_material( material = create_moge_material(
source_object.name, source_object.name,
foreground_image or color_image or source_object.data, material_image,
depth_image, depth_image,
depth_scale, depth_scale,
unshaded=unshaded, shadeless=shadeless,
frame_count=len(depth), frame_count=len(depth),
frame_start=source_frame_start, frame_start=source_frame_start,
use_cyclic=source_use_cyclic, use_cyclic=source_use_cyclic,
@@ -1959,6 +1944,7 @@ def build_moge2_depth_plane(
modifier = add_moge_depth_plane_processing( modifier = add_moge_depth_plane_processing(
plane_object, plane_object,
depth_image, depth_image,
material_image,
material, material,
frame_start=source_frame_start, frame_start=source_frame_start,
frame_duration=len(depth), frame_duration=len(depth),
@@ -2028,9 +2014,9 @@ class ConvertToDepthPlane(bpy.types.Operator):
bl_description = "Convert an image to a displaced depth plane" bl_description = "Convert an image to a displaced depth plane"
bl_options = {"REGISTER"} bl_options = {"REGISTER"}
unshaded: bpy.props.BoolProperty( shadeless: bpy.props.BoolProperty(
name="Unshaded", name="Shadeless",
description="Display the source color through an Emission shader", description="Display the source color without scene lighting",
default=False, default=False,
) )
remove_background: bpy.props.BoolProperty( remove_background: bpy.props.BoolProperty(
@@ -2081,7 +2067,7 @@ class ConvertToDepthPlane(bpy.types.Operator):
elif self.model.startswith("MOGE2_"): elif self.model.startswith("MOGE2_"):
layout.prop(self, "resolution_level") layout.prop(self, "resolution_level")
layout.prop(self, "remove_background") layout.prop(self, "remove_background")
layout.prop(self, "unshaded") layout.prop(self, "shadeless")
layout.separator(type="LINE") layout.separator(type="LINE")
layout.prop(self, "keep_original") layout.prop(self, "keep_original")
@@ -2110,7 +2096,7 @@ class ConvertToDepthPlane(bpy.types.Operator):
source_object_name=source_object.name, source_object_name=source_object.name,
delete_source=not self.keep_original, delete_source=not self.keep_original,
delete_input=delete_input, delete_input=delete_input,
unshaded=self.unshaded, shadeless=self.shadeless,
) )
if "CANCELLED" not in result: if "CANCELLED" not in result:
return {"FINISHED"} return {"FINISHED"}
@@ -2134,7 +2120,7 @@ class ConvertToDepthPlane(bpy.types.Operator):
model=self.model, model=self.model,
delete_source=not self.keep_original, delete_source=not self.keep_original,
delete_input=delete_input, delete_input=delete_input,
unshaded=self.unshaded, shadeless=self.shadeless,
remove_background=self.remove_background, remove_background=self.remove_background,
) )
if "CANCELLED" not in result: if "CANCELLED" not in result:
@@ -2169,7 +2155,7 @@ class ConvertToDepthPlane(bpy.types.Operator):
process_res=process_res, process_res=process_res,
delete_source=not self.keep_original, delete_source=not self.keep_original,
delete_input=delete_input, delete_input=delete_input,
unshaded=self.unshaded, shadeless=self.shadeless,
remove_background=self.remove_background, remove_background=self.remove_background,
) )
if "CANCELLED" in result: if "CANCELLED" in result:

18
src/snapform/blender/surface.py
View File

@@ -402,7 +402,7 @@ def build_image_mesh(
npz_path, npz_path,
delete_source, delete_source,
create_camera, create_camera,
unshaded, shadeless,
moge2=False, moge2=False,
color_image=None, color_image=None,
normal_path=None, normal_path=None,
@@ -437,6 +437,8 @@ def build_image_mesh(
context.collection.objects.link(mesh_object) context.collection.objects.link(mesh_object)
mesh_object.matrix_world = source_object.matrix_world.copy() mesh_object.matrix_world = source_object.matrix_world.copy()
depth_scale = material_depth_scale(mesh_object, camera_parameters) depth_scale = material_depth_scale(mesh_object, camera_parameters)
if color_image is None:
raise RuntimeError("Surface material image is required")
normal_image = ( normal_image = (
load_surface_normal_image( load_surface_normal_image(
normal_path, normal_path,
@@ -447,9 +449,9 @@ def build_image_mesh(
) )
material = create_material( material = create_material(
source_name, source_name,
color_image or source_object.data, color_image,
depth_scale, depth_scale,
unshaded=unshaded, shadeless=shadeless,
surface_group_name=( surface_group_name=(
MOGE_SURFACE_MATERIAL_NODE_GROUP_NAME if moge2 else None MOGE_SURFACE_MATERIAL_NODE_GROUP_NAME if moge2 else None
), ),
@@ -509,9 +511,9 @@ class ConvertToSurface(bpy.types.Operator):
description="Create a perspective camera matching the source projection", description="Create a perspective camera matching the source projection",
default=False, default=False,
) )
unshaded: bpy.props.BoolProperty( shadeless: bpy.props.BoolProperty(
name="Unshaded", name="Shadeless",
description="Display the source color through an Emission shader", description="Display the source color without scene lighting",
default=False, default=False,
) )
remove_background: bpy.props.BoolProperty( remove_background: bpy.props.BoolProperty(
@@ -550,7 +552,7 @@ class ConvertToSurface(bpy.types.Operator):
draw_process_resolution(layout, self, context.object.data) draw_process_resolution(layout, self, context.object.data)
layout.prop(self, "create_camera") layout.prop(self, "create_camera")
layout.prop(self, "remove_background") layout.prop(self, "remove_background")
layout.prop(self, "unshaded") layout.prop(self, "shadeless")
layout.separator(type="LINE") layout.separator(type="LINE")
layout.prop(self, "keep_original") layout.prop(self, "keep_original")
@@ -580,7 +582,7 @@ class ConvertToSurface(bpy.types.Operator):
"delete_source": not self.keep_original, "delete_source": not self.keep_original,
"delete_input": delete_input, "delete_input": delete_input,
"create_camera": self.create_camera, "create_camera": self.create_camera,
"unshaded": self.unshaded, "shadeless": self.shadeless,
"remove_background": self.remove_background, "remove_background": self.remove_background,
} }
if self.model.startswith("MOGE2_"): if self.model.startswith("MOGE2_"):

153
src/snapform/operators.py
View File

@@ -111,13 +111,12 @@ def build_da3_depth_response(
depth_data_name, depth_data_name,
remove_background, remove_background,
delete_source, delete_source,
unshaded, shadeless,
): ):
from .blender.depth_plane import ( from .blender.depth_plane import (
build_depth_plane, build_depth_plane,
create_removed_background_image, create_color_image,
create_sequence_image, is_animated_image,
image_base_name,
is_image_empty, is_image_empty,
) )
@@ -125,32 +124,21 @@ def build_da3_depth_response(
source_object = bpy.data.objects.get(source_object_name) source_object = bpy.data.objects.get(source_object_name)
if not is_image_empty(source_object): if not is_image_empty(source_object):
raise RuntimeError("The source image empty is no longer available") raise RuntimeError("The source image empty is no longer available")
foreground_paths = sorted(directory.glob("foreground.*.png")) animated = is_animated_image(source_object.data)
foreground_image = None color_image = create_color_image(
if remove_background: source_object,
foreground_image = create_removed_background_image( directory,
source_object, assign_to_source=False,
directory if foreground_paths else directory / "foreground.png", )
assign_to_source=False,
)
color_image = None
if list(directory.glob("frame.*")) and foreground_image is None:
color_image = create_sequence_image(
f"{image_base_name(source_object.data)}.frame",
directory,
)
build_depth_plane( build_depth_plane(
context, context,
source_object, source_object,
result.file("depth"), result.file("depth"),
delete_source, delete_source,
unshaded=unshaded, shadeless=shadeless,
depth_name=depth_data_name, depth_name=depth_data_name,
color_image=color_image, color_image=color_image,
foreground_image=foreground_image, depth_frames_directory=directory if animated else None,
depth_frames_directory=(
directory if list(directory.glob("frame.*")) else None
),
) )
push_undo("Convert to Depth Plane") push_undo("Convert to Depth Plane")
@@ -162,13 +150,11 @@ def build_moge2_depth_response(
source_object_name, source_object_name,
remove_background, remove_background,
delete_source, delete_source,
unshaded, shadeless,
): ):
from .blender.depth_plane import ( from .blender.depth_plane import (
build_moge2_depth_plane, build_moge2_depth_plane,
create_removed_background_image, create_color_image,
create_sequence_image,
image_base_name,
is_image_empty, is_image_empty,
) )
@@ -176,28 +162,18 @@ def build_moge2_depth_response(
source_object = bpy.data.objects.get(source_object_name) source_object = bpy.data.objects.get(source_object_name)
if not is_image_empty(source_object): if not is_image_empty(source_object):
raise RuntimeError("The source image empty is no longer available") raise RuntimeError("The source image empty is no longer available")
foreground_paths = sorted(directory.glob("foreground.*.png")) color_image = create_color_image(
foreground_image = None source_object,
if remove_background: directory,
foreground_image = create_removed_background_image( assign_to_source=False,
source_object, )
directory if foreground_paths else directory / "foreground.png",
assign_to_source=False,
)
color_image = None
if list(directory.glob("frame.*")) and foreground_image is None:
color_image = create_sequence_image(
f"{image_base_name(source_object.data)}.frame",
directory,
)
build_moge2_depth_plane( build_moge2_depth_plane(
context, context,
source_object, source_object,
result.file("depth"), result.file("depth"),
delete_source, delete_source,
unshaded=unshaded, shadeless=shadeless,
color_image=color_image, color_image=color_image,
foreground_image=foreground_image,
data_frames_directory=directory, data_frames_directory=directory,
) )
push_undo("Convert to Depth Plane") push_undo("Convert to Depth Plane")
@@ -210,13 +186,12 @@ def build_original_depth_response(
source_object_name, source_object_name,
background, background,
delete_source, delete_source,
unshaded, shadeless,
): ):
from .blender.depth_plane import ( from .blender.depth_plane import (
build_image_depth_plane, build_image_depth_plane,
create_removed_background_image, create_color_image,
create_sequence_image, is_animated_image,
image_base_name,
is_image_empty, is_image_empty,
) )
@@ -224,30 +199,19 @@ def build_original_depth_response(
source_object = bpy.data.objects.get(source_object_name) source_object = bpy.data.objects.get(source_object_name)
if not is_image_empty(source_object): if not is_image_empty(source_object):
raise RuntimeError("The source image empty is no longer available") raise RuntimeError("The source image empty is no longer available")
foreground_image = None animated = is_animated_image(source_object.data)
if background: color_image = create_color_image(
foreground_paths = sorted(directory.glob("foreground.*.png")) source_object,
foreground_image = create_removed_background_image( directory,
source_object, assign_to_source=False,
directory if foreground_paths else directory / "foreground.png", )
assign_to_source=False,
)
color_image = None
if list(directory.glob("frame.*")) and foreground_image is None:
color_image = create_sequence_image(
f"{image_base_name(source_object.data)}.frame",
directory,
)
build_image_depth_plane( build_image_depth_plane(
context, context,
source_object, source_object,
delete_source, delete_source,
unshaded=unshaded, shadeless=shadeless,
color_image=color_image, color_image=color_image,
foreground_image=foreground_image, depth_frames_directory=directory if animated else None,
depth_frames_directory=(
directory if list(directory.glob("frame.*")) else None
),
) )
push_undo("Convert to Depth Plane") push_undo("Convert to Depth Plane")
@@ -259,27 +223,19 @@ def build_surface_response(
source_object_name, source_object_name,
delete_source, delete_source,
create_camera, create_camera,
unshaded, shadeless,
moge2, moge2,
): ):
from .blender.depth_plane import ( from .blender.depth_plane import create_color_image, is_image_empty
create_removed_background_image,
is_image_empty,
)
from .blender.surface import build_image_mesh from .blender.surface import build_image_mesh
source_object = bpy.data.objects.get(source_object_name) source_object = bpy.data.objects.get(source_object_name)
if not is_image_empty(source_object): if not is_image_empty(source_object):
raise RuntimeError("The source Image Empty no longer exists") raise RuntimeError("The source Image Empty no longer exists")
foreground_path = result.directory / "foreground.png" color_image = create_color_image(
color_image = ( source_object,
create_removed_background_image( result.directory,
source_object, assign_to_source=False,
foreground_path,
assign_to_source=False,
)
if foreground_path.is_file()
else None
) )
build_image_mesh( build_image_mesh(
context, context,
@@ -287,7 +243,7 @@ def build_surface_response(
result.file("surface"), result.file("surface"),
delete_source, delete_source,
create_camera, create_camera,
unshaded, shadeless,
moge2, moge2,
color_image=color_image, color_image=color_image,
normal_path=( normal_path=(
@@ -300,19 +256,12 @@ def build_surface_response(
def apply_background_response(result, source_object_name): def apply_background_response(result, source_object_name):
from .blender.depth_plane import ( from .blender.depth_plane import create_color_image, is_image_empty
create_removed_background_image,
is_image_empty,
)
source_object = bpy.data.objects.get(source_object_name) source_object = bpy.data.objects.get(source_object_name)
if not is_image_empty(source_object): if not is_image_empty(source_object):
raise RuntimeError("The source image empty is no longer available") raise RuntimeError("The source image empty is no longer available")
foreground_paths = sorted(result.directory.glob("foreground.*.png")) create_color_image(source_object, result.directory)
create_removed_background_image(
source_object,
result.directory if foreground_paths else result.file("foreground"),
)
push_undo("Remove Background") push_undo("Remove Background")
@@ -461,7 +410,7 @@ class GenerateDa3Depth(JobOperatorBase, bpy.types.Operator):
process_res: bpy.props.IntProperty(default=1024, min=1, options=HIDDEN) process_res: bpy.props.IntProperty(default=1024, min=1, options=HIDDEN)
delete_source: bpy.props.BoolProperty(default=True, options=HIDDEN) delete_source: bpy.props.BoolProperty(default=True, options=HIDDEN)
delete_input: bpy.props.BoolProperty(default=False, options=HIDDEN) delete_input: bpy.props.BoolProperty(default=False, options=HIDDEN)
unshaded: bpy.props.BoolProperty(default=False, options=HIDDEN) shadeless: bpy.props.BoolProperty(default=False, options=HIDDEN)
remove_background: bpy.props.BoolProperty(default=False, options=HIDDEN) remove_background: bpy.props.BoolProperty(default=False, options=HIDDEN)
def request(self, context): def request(self, context):
@@ -492,7 +441,7 @@ class GenerateDa3Depth(JobOperatorBase, bpy.types.Operator):
depth_data_name=self.depth_data_name, depth_data_name=self.depth_data_name,
remove_background=self.remove_background, remove_background=self.remove_background,
delete_source=self.delete_source, delete_source=self.delete_source,
unshaded=self.unshaded, shadeless=self.shadeless,
) )
return "Depth plane is ready" return "Depth plane is ready"
@@ -512,7 +461,7 @@ class GenerateMoge2Depth(JobOperatorBase, bpy.types.Operator):
resolution_level: bpy.props.IntProperty(default=5, min=0, max=9, options=HIDDEN) resolution_level: bpy.props.IntProperty(default=5, min=0, max=9, options=HIDDEN)
delete_source: bpy.props.BoolProperty(default=True, options=HIDDEN) delete_source: bpy.props.BoolProperty(default=True, options=HIDDEN)
delete_input: bpy.props.BoolProperty(default=False, options=HIDDEN) delete_input: bpy.props.BoolProperty(default=False, options=HIDDEN)
unshaded: bpy.props.BoolProperty(default=False, options=HIDDEN) shadeless: bpy.props.BoolProperty(default=False, options=HIDDEN)
remove_background: bpy.props.BoolProperty(default=False, options=HIDDEN) remove_background: bpy.props.BoolProperty(default=False, options=HIDDEN)
def request(self, context): def request(self, context):
@@ -541,7 +490,7 @@ class GenerateMoge2Depth(JobOperatorBase, bpy.types.Operator):
source_object_name=self.source_object_name, source_object_name=self.source_object_name,
remove_background=self.remove_background, remove_background=self.remove_background,
delete_source=self.delete_source, delete_source=self.delete_source,
unshaded=self.unshaded, shadeless=self.shadeless,
) )
return "MoGe-2 depth plane is ready" return "MoGe-2 depth plane is ready"
@@ -562,7 +511,7 @@ class GenerateDa3Surface(JobOperatorBase, bpy.types.Operator):
delete_source: bpy.props.BoolProperty(default=True, options=HIDDEN) delete_source: bpy.props.BoolProperty(default=True, options=HIDDEN)
delete_input: bpy.props.BoolProperty(default=False, options=HIDDEN) delete_input: bpy.props.BoolProperty(default=False, options=HIDDEN)
create_camera: bpy.props.BoolProperty(default=False, options=HIDDEN) create_camera: bpy.props.BoolProperty(default=False, options=HIDDEN)
unshaded: bpy.props.BoolProperty(default=False, options=HIDDEN) shadeless: bpy.props.BoolProperty(default=False, options=HIDDEN)
remove_background: bpy.props.BoolProperty(default=False, options=HIDDEN) remove_background: bpy.props.BoolProperty(default=False, options=HIDDEN)
def request(self, context): def request(self, context):
@@ -591,7 +540,7 @@ class GenerateDa3Surface(JobOperatorBase, bpy.types.Operator):
source_object_name=self.source_object_name, source_object_name=self.source_object_name,
delete_source=self.delete_source, delete_source=self.delete_source,
create_camera=self.create_camera, create_camera=self.create_camera,
unshaded=self.unshaded, shadeless=self.shadeless,
moge2=False, moge2=False,
) )
return "Perspective surface is ready" return "Perspective surface is ready"
@@ -613,7 +562,7 @@ class GenerateMoge2Surface(JobOperatorBase, bpy.types.Operator):
delete_source: bpy.props.BoolProperty(default=True, options=HIDDEN) delete_source: bpy.props.BoolProperty(default=True, options=HIDDEN)
delete_input: bpy.props.BoolProperty(default=False, options=HIDDEN) delete_input: bpy.props.BoolProperty(default=False, options=HIDDEN)
create_camera: bpy.props.BoolProperty(default=False, options=HIDDEN) create_camera: bpy.props.BoolProperty(default=False, options=HIDDEN)
unshaded: bpy.props.BoolProperty(default=False, options=HIDDEN) shadeless: bpy.props.BoolProperty(default=False, options=HIDDEN)
remove_background: bpy.props.BoolProperty(default=False, options=HIDDEN) remove_background: bpy.props.BoolProperty(default=False, options=HIDDEN)
def request(self, context): def request(self, context):
@@ -642,7 +591,7 @@ class GenerateMoge2Surface(JobOperatorBase, bpy.types.Operator):
source_object_name=self.source_object_name, source_object_name=self.source_object_name,
delete_source=self.delete_source, delete_source=self.delete_source,
create_camera=self.create_camera, create_camera=self.create_camera,
unshaded=self.unshaded, shadeless=self.shadeless,
moge2=True, moge2=True,
) )
return "MoGe-2 surface is ready" return "MoGe-2 surface is ready"
@@ -691,7 +640,7 @@ class GenerateOriginalBackground(JobOperatorBase, bpy.types.Operator):
source_object_name: bpy.props.StringProperty(options=HIDDEN) source_object_name: bpy.props.StringProperty(options=HIDDEN)
delete_source: bpy.props.BoolProperty(default=True, options=HIDDEN) delete_source: bpy.props.BoolProperty(default=True, options=HIDDEN)
delete_input: bpy.props.BoolProperty(default=False, options=HIDDEN) delete_input: bpy.props.BoolProperty(default=False, options=HIDDEN)
unshaded: bpy.props.BoolProperty(default=False, options=HIDDEN) shadeless: bpy.props.BoolProperty(default=False, options=HIDDEN)
def request(self, context): def request(self, context):
require_environment() require_environment()
@@ -712,7 +661,7 @@ class GenerateOriginalBackground(JobOperatorBase, bpy.types.Operator):
source_object_name=self.source_object_name, source_object_name=self.source_object_name,
background=True, background=True,
delete_source=self.delete_source, delete_source=self.delete_source,
unshaded=self.unshaded, shadeless=self.shadeless,
) )
return "Depth plane is ready" return "Depth plane is ready"
@@ -730,7 +679,7 @@ class ExtractOriginalFrames(JobOperatorBase, bpy.types.Operator):
source_object_name: bpy.props.StringProperty(options=HIDDEN) source_object_name: bpy.props.StringProperty(options=HIDDEN)
delete_source: bpy.props.BoolProperty(default=True, options=HIDDEN) delete_source: bpy.props.BoolProperty(default=True, options=HIDDEN)
delete_input: bpy.props.BoolProperty(default=False, options=HIDDEN) delete_input: bpy.props.BoolProperty(default=False, options=HIDDEN)
unshaded: bpy.props.BoolProperty(default=False, options=HIDDEN) shadeless: bpy.props.BoolProperty(default=False, options=HIDDEN)
def request(self, context): def request(self, context):
require_environment() require_environment()
@@ -748,7 +697,7 @@ class ExtractOriginalFrames(JobOperatorBase, bpy.types.Operator):
source_object_name=self.source_object_name, source_object_name=self.source_object_name,
background=False, background=False,
delete_source=self.delete_source, delete_source=self.delete_source,
unshaded=self.unshaded, shadeless=self.shadeless,
) )
return "Depth plane is ready" return "Depth plane is ready"

4
src/snapform/server/geometry/grid_mesh.py
View File

@@ -69,9 +69,9 @@ def triangle_edge_stretch_values(points, triangles):
def remove_background_faces(faces, foreground_mask): def remove_background_faces(faces, foreground_mask):
"""Keep faces with at least one foreground vertex.""" """Keep only faces whose vertices are all foreground."""
foreground_points = foreground_mask.reshape(-1) foreground_points = foreground_mask.reshape(-1)
return faces[foreground_points[faces].any(axis=1)] return faces[foreground_points[faces].all(axis=1)]
def depth_edge_mask(depth, threshold=12.0 / 255.0): def depth_edge_mask(depth, threshold=12.0 / 255.0):

10
src/snapform/server/jobs/background.py
View File

@@ -1,7 +1,7 @@
import shutil import shutil
import time import time
from ..media import copy_frames, prepare_background_input from ..media import prepare_background_input
from ..models.ben2 import infer_foreground, infer_foreground_frames from ..models.ben2 import infer_foreground, infer_foreground_frames
@@ -39,7 +39,7 @@ def run(context, parameters):
) )
context.check_cancelled() context.check_cancelled()
context.progress(0.92, "Writing transparent PNG") context.progress(0.92, "Writing transparent PNG")
output_path = output_directory / "foreground.png" output_path = output_directory / "color.0001.png"
foreground.convert("RGBA").save( foreground.convert("RGBA").save(
output_path, output_path,
format="PNG", format="PNG",
@@ -58,8 +58,8 @@ def run(context, parameters):
0.7, 0.7,
model_cache=model_manager, model_cache=model_manager,
cancel_check=context.check_cancelled, cancel_check=context.check_cancelled,
output_prefix="color",
) )
copy_frames(images, output_directory)
finally: finally:
if temporary_directory is not None: if temporary_directory is not None:
shutil.rmtree(temporary_directory, ignore_errors=True) shutil.rmtree(temporary_directory, ignore_errors=True)
@@ -73,6 +73,4 @@ def run(context, parameters):
) )
print(message) print(message)
context.progress(1.0, message) context.progress(1.0, message)
if tuple(output_directory.glob("foreground.*.png")): return {"color": "."}
return {"foreground": "."}
return {"foreground": "foreground.png"}

7
src/snapform/server/jobs/depth.py
View File

@@ -1,4 +1,4 @@
from ..media import copy_frames from ..media import copy_color_images
from ..models import da3, geometry_adapter, moge2 from ..models import da3, geometry_adapter, moge2
from ..outputs.depth import write_da3_depth, write_moge_depth from ..outputs.depth import write_da3_depth, write_moge_depth
from .common import input_frames from .common import input_frames
@@ -36,14 +36,15 @@ def run(context, parameters):
cancel_check=context.check_cancelled, cancel_check=context.check_cancelled,
) )
write_moge_depth(context.directory, prediction) write_moge_depth(context.directory, prediction)
write_optional( color_paths = write_optional(
context, context,
parameters, parameters,
images, images,
progress_start=0.84, progress_start=0.84,
progress_span=0.08, progress_span=0.08,
) )
copy_frames(images, context.directory) if not color_paths:
copy_color_images(images, context.directory)
context.check_cancelled() context.check_cancelled()
context.progress(1.0, f"Exported depth for {len(images)} frame(s)") context.progress(1.0, f"Exported depth for {len(images)} frame(s)")
return {"depth": "results.npz"} return {"depth": "results.npz"}

5
src/snapform/server/jobs/foreground.py
View File

@@ -24,7 +24,7 @@ def write_optional(
model_cache=model_manager, model_cache=model_manager,
cancel_check=context.check_cancelled, cancel_check=context.check_cancelled,
) )
path = context.directory / "foreground.png" path = context.directory / "color.0001.png"
foreground.convert("RGBA").save( foreground.convert("RGBA").save(
path, path,
format="PNG", format="PNG",
@@ -42,8 +42,9 @@ def write_optional(
progress_span, progress_span,
model_cache=model_manager, model_cache=model_manager,
cancel_check=context.check_cancelled, cancel_check=context.check_cancelled,
output_prefix="color",
) )
paths = sorted(context.directory.glob("foreground.*.png")) paths = sorted(context.directory.glob("color.*.png"))
print( print(
f"BEN2 inference {inference_ms:.0f} ms; " f"BEN2 inference {inference_ms:.0f} ms; "
f"model load {load_ms:.0f} ms" f"model load {load_ms:.0f} ms"

4
src/snapform/server/jobs/frames.py
View File

@@ -1,6 +1,6 @@
import shutil import shutil
from ..media import copy_frames, prepare_input from ..media import copy_color_images, prepare_input
def run(context, parameters): def run(context, parameters):
@@ -13,7 +13,7 @@ def run(context, parameters):
cancel_check=context.check_cancelled, cancel_check=context.check_cancelled,
) )
try: try:
copy_frames(images, context.directory) copy_color_images(images, context.directory)
finally: finally:
if temporary_directory is not None: if temporary_directory is not None:
shutil.rmtree(temporary_directory, ignore_errors=True) shutil.rmtree(temporary_directory, ignore_errors=True)

33
src/snapform/server/jobs/surface.py
View File

@@ -1,15 +1,18 @@
from ..geometry import foreground_mesh_mask from ..geometry import build_surface, foreground_mesh_mask
from ..media import copy_color_images
from ..models import geometry_adapter, moge2 from ..models import geometry_adapter, moge2
from ..geometry import build_surface from ..outputs.surface import write_normal_texture, write_surface_archive
from ..outputs.surface import (
write_normal_texture,
write_surface_archive,
write_surface_texture,
)
from .common import input_frames from .common import input_frames
from .foreground import write_optional from .foreground import write_optional
def inference_progress_message(model_name, index, total):
message = f"Estimating {model_name} geometry"
if total > 1:
message += f" ({index + 1}/{total})"
return message
def run(context, parameters): def run(context, parameters):
context.progress(0.02, "Preparing surface input") context.progress(0.02, "Preparing surface input")
context.directory.mkdir(parents=True, exist_ok=True) context.directory.mkdir(parents=True, exist_ok=True)
@@ -27,7 +30,7 @@ def run(context, parameters):
include_points=True, include_points=True,
progress_callback=lambda index, total: context.progress( progress_callback=lambda index, total: context.progress(
0.12 + 0.58 * index / total, 0.12 + 0.58 * index / total,
f"Estimating MoGe-2 geometry ({index + 1}/{total})", inference_progress_message("MoGe-2", index, total),
), ),
cancel_check=context.check_cancelled, cancel_check=context.check_cancelled,
) )
@@ -38,13 +41,13 @@ def run(context, parameters):
images, images,
progress_callback=lambda index, total: context.progress( progress_callback=lambda index, total: context.progress(
0.12 + 0.58 * index / total, 0.12 + 0.58 * index / total,
f"Estimating DA3 geometry ({index + 1}/{total})", inference_progress_message("DA3", index, total),
), ),
cancel_check=context.check_cancelled, cancel_check=context.check_cancelled,
) )
context.check_cancelled() context.check_cancelled()
frame = adapter.surface_frame(prediction) frame = adapter.surface_frame(prediction)
foreground_paths = write_optional( color_paths = write_optional(
context, context,
parameters, parameters,
images, images,
@@ -52,19 +55,17 @@ def run(context, parameters):
progress_span=0.08, progress_span=0.08,
) )
foreground_mask = None foreground_mask = None
if foreground_paths: if color_paths:
foreground_mask = foreground_mesh_mask( foreground_mask = foreground_mesh_mask(
foreground_paths[0], color_paths[0],
frame.depth.shape, frame.depth.shape,
) )
context.progress(0.82, "Building surface") context.progress(0.82, "Building surface")
mesh = build_surface(frame, foreground_mask=foreground_mask) mesh = build_surface(frame, foreground_mask=foreground_mask)
context.progress(0.92, "Writing surface") context.progress(0.92, "Writing surface")
write_surface_archive(mesh, context.directory / "mesh.npz") write_surface_archive(mesh, context.directory / "mesh.npz")
write_surface_texture( if not color_paths:
foreground_paths[0] if foreground_paths else images[0], copy_color_images(images, context.directory)
context.directory / "texture.png",
)
if adapter is moge2: if adapter is moge2:
write_normal_texture( write_normal_texture(
prediction.normal[0], prediction.normal[0],

2
src/snapform/server/media/__init__.py
View File

@@ -4,6 +4,7 @@ from .input import (
TaskCancelled, TaskCancelled,
check_cancelled, check_cancelled,
collect_images, collect_images,
copy_color_images,
copy_frames, copy_frames,
extract_video_frames, extract_video_frames,
first_frames, first_frames,
@@ -18,6 +19,7 @@ __all__ = (
"TaskCancelled", "TaskCancelled",
"check_cancelled", "check_cancelled",
"collect_images", "collect_images",
"copy_color_images",
"copy_frames", "copy_frames",
"extract_video_frames", "extract_video_frames",
"first_frames", "first_frames",

20
src/snapform/server/media/input.py
View File

@@ -143,16 +143,26 @@ def prepare_background_input(
) )
def copy_frames(images, output_directory): def copy_color_images(images, output_directory):
"""Copy a multi-frame input as numbered frame files for the Blender sequence.""" """Copy input images as the canonical numbered color layer."""
if len(images) <= 1:
return
for index, image_path in enumerate(images): for index, image_path in enumerate(images):
suffix = Path(image_path).suffix.lower() or ".jpg" suffix = Path(image_path).suffix.lower() or ".jpg"
destination = ( destination = (
Path(output_directory) / f"frame.{index + 1:04d}{suffix}" Path(output_directory) / f"color.{index + 1:04d}{suffix}"
) )
if Path(image_path).resolve() == destination.resolve(): if Path(image_path).resolve() == destination.resolve():
continue continue
shutil.copy2(image_path, destination) shutil.copy2(image_path, destination)
def copy_frames(images, output_directory):
"""Copy multiple input images as numbered raw Debug frame files."""
if len(images) <= 1:
return
for index, image_path in enumerate(images):
suffix = Path(image_path).suffix.lower() or ".jpg"
destination = Path(output_directory) / f"frame.{index + 1:04d}{suffix}"
if Path(image_path).resolve() == destination.resolve():
continue
shutil.copy2(image_path, destination)

5
src/snapform/server/models/ben2.py
View File

@@ -48,8 +48,9 @@ def infer_foreground_frames(
progress_span, progress_span,
model_cache=None, model_cache=None,
cancel_check=None, cancel_check=None,
output_prefix="foreground",
): ):
"""Run BEN2 on every frame and write foreground.0001.png etc.""" """Run BEN2 on every frame and write numbered transparent PNG files."""
total_load_ms = 0.0 total_load_ms = 0.0
total_inference_ms = 0.0 total_inference_ms = 0.0
for index, image_path in enumerate(images): for index, image_path in enumerate(images):
@@ -65,7 +66,7 @@ def infer_foreground_frames(
model_cache=model_cache, model_cache=model_cache,
cancel_check=cancel_check, cancel_check=cancel_check,
) )
output_path = output_directory / f"foreground.{index + 1:04d}.png" output_path = output_directory / f"{output_prefix}.{index + 1:04d}.png"
foreground.convert("RGBA").save( foreground.convert("RGBA").save(
output_path, output_path,
format="PNG", format="PNG",

3
src/snapform/server/outputs/__init__.py
View File

@@ -1,5 +1,5 @@
from .depth import write_da3_depth, write_moge_depth from .depth import write_da3_depth, write_moge_depth
from .surface import write_normal_texture, write_surface_archive, write_surface_texture from .surface import write_normal_texture, write_surface_archive
__all__ = ( __all__ = (
@@ -7,5 +7,4 @@ __all__ = (
"write_moge_depth", "write_moge_depth",
"write_normal_texture", "write_normal_texture",
"write_surface_archive", "write_surface_archive",
"write_surface_texture",
) )

9
src/snapform/server/outputs/surface.py
View File

@@ -20,15 +20,6 @@ def write_surface_archive(mesh, output_path):
return output_path return output_path
def write_surface_texture(image_path, output_path):
from PIL import Image
output_path = Path(output_path)
with Image.open(image_path) as image:
image.convert("RGBA").save(output_path, format="PNG", compress_level=1)
return output_path
def write_normal_texture(normal, output_path): def write_normal_texture(normal, output_path):
import numpy as np import numpy as np
from PIL import Image from PIL import Image

6
tests/test_addon.py
View File

@@ -686,7 +686,7 @@ class AddonTest(unittest.TestCase):
operator.source_object_name = "Reference" operator.source_object_name = "Reference"
operator.remove_background = False operator.remove_background = False
operator.delete_source = True operator.delete_source = True
operator.unshaded = False operator.shadeless = False
result = object() result = object()
context = object() context = object()
@@ -701,7 +701,7 @@ class AddonTest(unittest.TestCase):
source_object_name="Reference", source_object_name="Reference",
remove_background=False, remove_background=False,
delete_source=True, delete_source=True,
unshaded=False, shadeless=False,
) )
def test_entry_operator_dispatches_plain_properties(self): def test_entry_operator_dispatches_plain_properties(self):
@@ -709,7 +709,7 @@ class AddonTest(unittest.TestCase):
operator.model = "MOGE2_VITS_NORMAL" operator.model = "MOGE2_VITS_NORMAL"
operator.resolution_level = 7 operator.resolution_level = 7
operator.keep_original = False operator.keep_original = False
operator.unshaded = True operator.shadeless = True
operator.remove_background = False operator.remove_background = False
source_object = SimpleNamespace( source_object = SimpleNamespace(
name="Reference", name="Reference",

16
tests/test_image_depth.py
View File

@@ -291,12 +291,12 @@ class ImageDepthTest(unittest.TestCase):
"微信视频2026-07-23_192756_414", "微信视频2026-07-23_192756_414",
) )
def test_removed_background_name_strips_video_extension(self): def test_color_name_strips_video_extension(self):
source_object = SimpleNamespace(data=SimpleNamespace(name="clip.mp4")) source_object = SimpleNamespace(data=SimpleNamespace(name="clip.mp4"))
self.assertEqual( self.assertEqual(
self.image_depth.removed_background_data_name(source_object), self.image_depth.color_data_name(source_object),
"clip.foreground", "clip.color",
) )
def test_force_sequence_source_keeps_loaded_sequence(self): def test_force_sequence_source_keeps_loaded_sequence(self):
@@ -634,19 +634,19 @@ class ImageDepthTest(unittest.TestCase):
self.assertEqual(depth_image.name, "img.depth") self.assertEqual(depth_image.name, "img.depth")
def test_removed_background_name_uses_source_image_base_name(self): def test_color_name_uses_source_image_base_name(self):
source_object = SimpleNamespace( source_object = SimpleNamespace(
data=SimpleNamespace(name="img.png"), data=SimpleNamespace(name="img.png"),
) )
self.assertEqual( self.assertEqual(
self.image_depth.removed_background_data_name(source_object), self.image_depth.color_data_name(source_object),
"img.foreground", "img.color",
) )
source_object.data.name = "img.png.001" source_object.data.name = "img.png.001"
self.assertEqual( self.assertEqual(
self.image_depth.removed_background_data_name(source_object), self.image_depth.color_data_name(source_object),
"img.001.foreground", "img.001.color",
) )
def test_surface_displacement_is_set_to_displacement_only(self): def test_surface_displacement_is_set_to_displacement_only(self):

2
tests/test_server.py
View File

@@ -376,7 +376,7 @@ class ServerTest(unittest.TestCase):
) )
status = self.wait_for_job(context) status = self.wait_for_job(context)
self.assertTrue( self.assertTrue(
(context.directory / "frame.0001.jpg").is_file() (context.directory / "color.0001.jpg").is_file()
) )
self.assertEqual(status["state"], "succeeded") self.assertEqual(status["state"], "succeeded")

69
tests/test_worker.py
View File

@@ -127,9 +127,24 @@ class WorkerTest(unittest.TestCase):
cls.foreground_job = load_server_module("jobs.foreground") cls.foreground_job = load_server_module("jobs.foreground")
cls.download = load_server_module("model_download") cls.download = load_server_module("model_download")
def test_surface_inference_progress_omits_single_frame_count(self):
self.assertEqual(
self.surface_job.inference_progress_message("MoGe-2", 0, 1),
"Estimating MoGe-2 geometry",
)
self.assertEqual(
self.surface_job.inference_progress_message("DA3", 0, 1),
"Estimating DA3 geometry",
)
self.assertEqual(
self.surface_job.inference_progress_message("DA3", 1, 3),
"Estimating DA3 geometry (2/3)",
)
def test_surface_has_one_model_independent_builder(self): def test_surface_has_one_model_independent_builder(self):
self.assertTrue(callable(self.surface.build_surface)) self.assertTrue(callable(self.surface.build_surface))
self.assertTrue(callable(self.surface_outputs.write_surface_archive)) self.assertTrue(callable(self.surface_outputs.write_surface_archive))
self.assertFalse(hasattr(self.surface_outputs, "write_surface_texture"))
self.assertFalse(hasattr(self.surface, "export_da3_surface")) self.assertFalse(hasattr(self.surface, "export_da3_surface"))
self.assertFalse(hasattr(self.surface, "export_moge2_surface")) self.assertFalse(hasattr(self.surface, "export_moge2_surface"))
@@ -226,6 +241,33 @@ class WorkerTest(unittest.TestCase):
) )
self.assertIsNone(temporary_directory) self.assertIsNone(temporary_directory)
def test_copy_color_images_numbers_a_still_image(self):
with tempfile.TemporaryDirectory() as directory:
root = Path(directory)
source = root / "source.png"
source.write_bytes(b"frame")
output = root / "result"
output.mkdir()
self.media.copy_color_images([source], output)
self.assertEqual(
(output / "color.0001.png").read_bytes(),
b"frame",
)
def test_copy_frames_skips_a_still_image_by_default(self):
with tempfile.TemporaryDirectory() as directory:
root = Path(directory)
source = root / "source.png"
source.touch()
output = root / "result"
output.mkdir()
self.media.copy_frames([source], output)
self.assertEqual(list(output.iterdir()), [])
def test_extract_video_frames_reads_frames_sequentially(self): def test_extract_video_frames_reads_frames_sequentially(self):
with tempfile.TemporaryDirectory() as directory: with tempfile.TemporaryDirectory() as directory:
root = Path(directory) root = Path(directory)
@@ -497,7 +539,7 @@ class WorkerTest(unittest.TestCase):
{"input": str(input_path), "model": "BEN2", "device": "cpu"}, {"input": str(input_path), "model": "BEN2", "device": "cpu"},
) )
self.assertTrue((root / "result" / "foreground.png").is_file()) self.assertTrue((root / "result" / "color.0001.png").is_file())
self.assertEqual( self.assertEqual(
inference_options[0][1], inference_options[0][1],
@@ -590,16 +632,10 @@ class WorkerTest(unittest.TestCase):
result_directory = context.directory result_directory = context.directory
self.assertTrue( self.assertTrue(
(result_directory / "foreground.0001.png").is_file() (result_directory / "color.0001.png").is_file()
) )
self.assertTrue( self.assertTrue(
(result_directory / "foreground.0002.png").is_file() (result_directory / "color.0002.png").is_file()
)
self.assertTrue(
(result_directory / "frame.0001.jpg").is_file()
)
self.assertTrue(
(result_directory / "frame.0002.jpg").is_file()
) )
self.assertEqual(len(inference_options), 2) self.assertEqual(len(inference_options), 2)
@@ -630,7 +666,7 @@ class WorkerTest(unittest.TestCase):
for path in result_directory.iterdir() for path in result_directory.iterdir()
if path.is_file() if path.is_file()
), ),
["frame.0001.jpg", "frame.0002.jpg", "frame.0003.jpg"], ["color.0001.jpg", "color.0002.jpg", "color.0003.jpg"],
) )
self.assertEqual(context.status["progress"], 1.0) self.assertEqual(context.status["progress"], 1.0)
self.assertIn("3 frame", context.status["message"]) self.assertIn("3 frame", context.status["message"])
@@ -889,10 +925,10 @@ class WorkerTest(unittest.TestCase):
result_directory = context.directory result_directory = context.directory
self.assertTrue( self.assertTrue(
(result_directory / "frame.0001.jpg").is_file() (result_directory / "color.0001.jpg").is_file()
) )
self.assertTrue( self.assertTrue(
(result_directory / "frame.0002.jpg").is_file() (result_directory / "color.0002.jpg").is_file()
) )
self.assertEqual(len(inference_inputs), 2) self.assertEqual(len(inference_inputs), 2)
self.assertTrue(all(len(paths) == 1 for paths in inference_inputs)) self.assertTrue(all(len(paths) == 1 for paths in inference_inputs))
@@ -1107,19 +1143,19 @@ class WorkerTest(unittest.TestCase):
) )
np.testing.assert_array_equal(compact_faces, [[0, 1, 2]]) np.testing.assert_array_equal(compact_faces, [[0, 1, 2]])
def test_surface_background_keeps_faces_with_any_foreground_vertex(self): def test_surface_background_requires_every_vertex_to_be_foreground(self):
faces = np.asarray( faces = np.asarray(
((0, 1, 4), (0, 4, 3), (1, 2, 5), (1, 5, 4)), ((0, 1, 4), (0, 4, 3), (1, 2, 5), (1, 5, 4)),
dtype=np.int32, dtype=np.int32,
) )
foreground = np.asarray( foreground = np.asarray(
((False, False, False), (False, True, False)), ((True, True, False), (False, True, False)),
dtype=bool, dtype=bool,
) )
filtered = self.geometry.remove_background_faces(faces, foreground) filtered = self.geometry.remove_background_faces(faces, foreground)
np.testing.assert_array_equal(filtered, faces[[0, 1, 3]]) np.testing.assert_array_equal(filtered, faces[[0]])
def test_surface_background_mask_keeps_any_nonzero_alpha(self): def test_surface_background_mask_keeps_any_nonzero_alpha(self):
from PIL import Image from PIL import Image
@@ -1255,6 +1291,3 @@ class WorkerTest(unittest.TestCase):
if __name__ == "__main__": if __name__ == "__main__":
unittest.main() unittest.main()

119
tools/build_geometry_nodes_asset.py
View File

@@ -7,6 +7,7 @@ GROUP_NAME = "O DA3 Surface"
MOGE_SURFACE_GROUP_NAME = "O MoGe2 Surface" MOGE_SURFACE_GROUP_NAME = "O MoGe2 Surface"
DEPTH_PLANE_GROUP_NAME = "O DA3 Depth Plane" DEPTH_PLANE_GROUP_NAME = "O DA3 Depth Plane"
MOGE_DEPTH_PLANE_GROUP_NAME = "O MoGe2 Depth Plane" MOGE_DEPTH_PLANE_GROUP_NAME = "O MoGe2 Depth Plane"
SHADELESS_GROUP_NAME = "O Shadeless"
MATERIAL_DISPLACEMENT_ATTRIBUTE = "o_material_displacement" MATERIAL_DISPLACEMENT_ATTRIBUTE = "o_material_displacement"
BACKGROUND_DEPTH_ATTRIBUTE = "o_background_depth" BACKGROUND_DEPTH_ATTRIBUTE = "o_background_depth"
DEPTH_SCALE_ATTRIBUTE = "o_depth_scale" DEPTH_SCALE_ATTRIBUTE = "o_depth_scale"
@@ -800,9 +801,9 @@ def build_depth_plane_group():
"NodeSocketImage", "NodeSocketImage",
parent=data_panel, parent=data_panel,
) )
foreground_image = new_socket( color_image = new_socket(
interface, interface,
"Foreground Image", "Color Image",
"INPUT", "INPUT",
"NodeSocketImage", "NodeSocketImage",
parent=data_panel, parent=data_panel,
@@ -835,7 +836,7 @@ def build_depth_plane_group():
socket.min_value = 1 socket.min_value = 1
frame_data.append(socket) frame_data.append(socket)
depth_image.hide_in_modifier = True depth_image.hide_in_modifier = True
foreground_image.hide_in_modifier = True color_image.hide_in_modifier = True
material.hide_in_modifier = True material.hide_in_modifier = True
for socket in frame_data: for socket in frame_data:
socket.hide_in_modifier = True socket.hide_in_modifier = True
@@ -848,7 +849,7 @@ def build_depth_plane_group():
images_input = new_group_input( images_input = new_group_input(
nodes, nodes,
(100.0, 1250.0), (100.0, 1250.0),
{"Depth Image", "Foreground Image"}, {"Depth Image", "Color Image"},
) )
frames_input = new_group_input( frames_input = new_group_input(
nodes, nodes,
@@ -973,7 +974,7 @@ def build_depth_plane_group():
foreground_texture.location = (350.0, 900.0) foreground_texture.location = (350.0, 900.0)
links.new(images_input.outputs["Depth Image"], depth_texture.inputs["Image"]) links.new(images_input.outputs["Depth Image"], depth_texture.inputs["Image"])
links.new( links.new(
images_input.outputs["Foreground Image"], images_input.outputs["Color Image"],
foreground_texture.inputs["Image"], foreground_texture.inputs["Image"],
) )
links.new(grid.outputs["UV Map"], depth_texture.inputs["Vector"]) links.new(grid.outputs["UV Map"], depth_texture.inputs["Vector"])
@@ -1141,10 +1142,13 @@ def build_moge_depth_plane_group():
depth_image = new_socket( depth_image = new_socket(
interface, "Depth Image", "INPUT", "NodeSocketImage", data_panel interface, "Depth Image", "INPUT", "NodeSocketImage", data_panel
) )
color_image = new_socket(
interface, "Color Image", "INPUT", "NodeSocketImage", data_panel
)
material = new_socket( material = new_socket(
interface, "Material", "INPUT", "NodeSocketMaterial", data_panel interface, "Material", "INPUT", "NodeSocketMaterial", data_panel
) )
for socket in (depth_image, material): for socket in (depth_image, color_image, material):
socket.hide_in_modifier = True socket.hide_in_modifier = True
frame_sockets = [] frame_sockets = []
for name, socket_type, default in ( for name, socket_type, default in (
@@ -1176,7 +1180,7 @@ def build_moge_depth_plane_group():
background_depth_value.inputs[0], background_depth_value.inputs[0],
) )
images = new_group_input( images = new_group_input(
nodes, (-500.0, 1100.0), {"Depth Image"} nodes, (-500.0, 1100.0), {"Depth Image", "Color Image"}
) )
frames = new_group_input( frames = new_group_input(
nodes, nodes,
@@ -1241,18 +1245,42 @@ def build_moge_depth_plane_group():
depth_texture.interpolation = "Linear" depth_texture.interpolation = "Linear"
depth_texture.extension = "EXTEND" depth_texture.extension = "EXTEND"
depth_texture.location = (-250.0, 1100.0) depth_texture.location = (-250.0, 1100.0)
color_texture = nodes.new("GeometryNodeImageTexture")
color_texture.interpolation = "Closest"
color_texture.extension = "EXTEND"
color_texture.location = (-250.0, 850.0)
links.new(images.outputs["Depth Image"], depth_texture.inputs["Image"]) links.new(images.outputs["Depth Image"], depth_texture.inputs["Image"])
links.new(images.outputs["Color Image"], color_texture.inputs["Image"])
links.new(grid.outputs["UV Map"], depth_texture.inputs["Vector"]) links.new(grid.outputs["UV Map"], depth_texture.inputs["Vector"])
links.new(grid.outputs["UV Map"], color_texture.inputs["Vector"])
frame = build_sequence_frame( frame = build_sequence_frame(
nodes, links, frames.outputs["Frame"], frames, 1800.0 nodes, links, frames.outputs["Frame"], frames, 1800.0
) )
links.new(frame, depth_texture.inputs["Frame"]) links.new(frame, depth_texture.inputs["Frame"])
links.new(frame, color_texture.inputs["Frame"])
clamp_depth = nodes.new("ShaderNodeMath") clamp_depth = nodes.new("ShaderNodeMath")
clamp_depth.operation = "MAXIMUM" clamp_depth.operation = "MAXIMUM"
clamp_depth.location = (0.0, 750.0) clamp_depth.location = (0.0, 750.0)
links.new(depth_texture.outputs["Color"], clamp_depth.inputs[0]) links.new(depth_texture.outputs["Color"], clamp_depth.inputs[0])
links.new(background_depth_value.outputs[0], clamp_depth.inputs[1]) links.new(background_depth_value.outputs[0], clamp_depth.inputs[1])
color_mask = new_compare(
nodes,
"FLOAT",
"GREATER_THAN",
(0.0, 1000.0),
0.5,
)
links.new(
color_texture.outputs["Alpha"],
input_socket(color_mask, "A", 0),
)
masked_depth = nodes.new("ShaderNodeMix")
masked_depth.data_type = "FLOAT"
masked_depth.location = (200.0, 850.0)
links.new(color_mask.outputs["Result"], masked_depth.inputs[0])
links.new(background_depth_value.outputs[0], masked_depth.inputs[2])
links.new(clamp_depth.outputs[0], masked_depth.inputs[3])
displacement_menu = nodes.new("GeometryNodeMenuSwitch") displacement_menu = nodes.new("GeometryNodeMenuSwitch")
displacement_menu.data_type = "FLOAT" displacement_menu.data_type = "FLOAT"
displacement_menu.enum_items.clear() displacement_menu.enum_items.clear()
@@ -1276,7 +1304,7 @@ def build_moge_depth_plane_group():
scaled_depth = nodes.new("ShaderNodeMath") scaled_depth = nodes.new("ShaderNodeMath")
scaled_depth.operation = "MULTIPLY" scaled_depth.operation = "MULTIPLY"
scaled_depth.location = (250.0, 750.0) scaled_depth.location = (250.0, 750.0)
links.new(clamp_depth.outputs[0], scaled_depth.inputs[0]) links.new(masked_depth.outputs["Result"], scaled_depth.inputs[0])
links.new(geometry_strength.outputs[0], scaled_depth.inputs[1]) links.new(geometry_strength.outputs[0], scaled_depth.inputs[1])
normal = nodes.new("GeometryNodeInputNormal") normal = nodes.new("GeometryNodeInputNormal")
normal.location = (0.0, 500.0) normal.location = (0.0, 500.0)
@@ -1320,6 +1348,80 @@ def build_moge_depth_plane_group():
return group return group
def build_shadeless_group():
group = bpy.data.node_groups.new(SHADELESS_GROUP_NAME, "ShaderNodeTree")
interface = group.interface
new_socket(interface, "Shader", "OUTPUT", "NodeSocketShader")
new_socket(interface, "Color", "INPUT", "NodeSocketColor")
nodes = group.nodes
links = group.links
group_input = nodes.new("NodeGroupInput")
group_input.location = (-900.0, -350.0)
group_output = nodes.new("NodeGroupOutput")
group_output.location = (850.0, 100.0)
diffuse = nodes.new("ShaderNodeBsdfDiffuse")
diffuse.location = (250.0, -150.0)
emission = nodes.new("ShaderNodeEmission")
emission.location = (250.0, -400.0)
links.new(group_input.outputs["Color"], diffuse.inputs["Color"])
links.new(group_input.outputs["Color"], emission.inputs["Color"])
light_path = nodes.new("ShaderNodeLightPath")
light_path.location = (-900.0, 150.0)
bounce_count = nodes.new("ShaderNodeMath")
bounce_count.operation = "SUBTRACT"
bounce_count.label = "Bounce Count"
bounce_count.location = (-500.0, -50.0)
links.new(light_path.outputs["Ray Depth"], bounce_count.inputs[0])
links.new(light_path.outputs["Transmission Depth"], bounce_count.inputs[1])
refracted = nodes.new("ShaderNodeMath")
refracted.operation = "SUBTRACT"
refracted.label = "Camera or Refracted"
refracted.inputs[0].default_value = 1.0
refracted.location = (-250.0, -50.0)
links.new(bounce_count.outputs[0], refracted.inputs[1])
reflection_limit = nodes.new("ShaderNodeMath")
reflection_limit.operation = "SUBTRACT"
reflection_limit.label = "Limit Reflections"
reflection_limit.inputs[0].default_value = 2.0
reflection_limit.location = (-650.0, 450.0)
links.new(light_path.outputs["Ray Depth"], reflection_limit.inputs[1])
camera_reflected = nodes.new("ShaderNodeMath")
camera_reflected.operation = "MULTIPLY"
camera_reflected.label = "Camera Ray to Glossy"
camera_reflected.location = (-400.0, 400.0)
links.new(reflection_limit.outputs[0], camera_reflected.inputs[0])
links.new(light_path.outputs["Is Glossy Ray"], camera_reflected.inputs[1])
shadow_or_reflect = nodes.new("ShaderNodeMath")
shadow_or_reflect.operation = "MAXIMUM"
shadow_or_reflect.label = "Shadow or Reflection?"
shadow_or_reflect.location = (-100.0, 350.0)
links.new(camera_reflected.outputs[0], shadow_or_reflect.inputs[0])
links.new(light_path.outputs["Is Shadow Ray"], shadow_or_reflect.inputs[1])
shader_factor = nodes.new("ShaderNodeMath")
shader_factor.operation = "MAXIMUM"
shader_factor.label = "Shadow, Reflect or Refract?"
shader_factor.location = (150.0, 300.0)
links.new(shadow_or_reflect.outputs[0], shader_factor.inputs[0])
links.new(refracted.outputs[0], shader_factor.inputs[1])
mix = nodes.new("ShaderNodeMixShader")
mix.location = (550.0, 100.0)
links.new(shader_factor.outputs[0], mix.inputs["Fac"])
links.new(diffuse.outputs["BSDF"], mix.inputs[1])
links.new(emission.outputs["Emission"], mix.inputs[2])
links.new(mix.outputs["Shader"], group_output.inputs["Shader"])
group.color_tag = "SHADER"
group.description = (
"Display color without scene lighting while preserving shadows "
"and reflections."
)
group.use_fake_user = True
return group
def main(): def main():
bpy.ops.wm.read_factory_settings(use_empty=True) bpy.ops.wm.read_factory_settings(use_empty=True)
bpy.context.preferences.filepaths.save_version = 0 bpy.context.preferences.filepaths.save_version = 0
@@ -1336,6 +1438,7 @@ def main():
configure_moge_surface_group(moge_surface_group) configure_moge_surface_group(moge_surface_group)
build_depth_plane_group() build_depth_plane_group()
build_moge_depth_plane_group() build_moge_depth_plane_group()
build_shadeless_group()
OUTPUT_PATH.parent.mkdir(parents=True, exist_ok=True) OUTPUT_PATH.parent.mkdir(parents=True, exist_ok=True)
bpy.ops.wm.save_as_mainfile(filepath=str(OUTPUT_PATH), check_existing=False) bpy.ops.wm.save_as_mainfile(filepath=str(OUTPUT_PATH), check_existing=False)

44
tools/validate_depth_plane_asset.py
View File

@@ -13,6 +13,8 @@ sys.path.insert(
from snapform.blender.depth_plane import ( from snapform.blender.depth_plane import (
DEPTH_PLANE_MATERIAL_NODE_GROUP_NAME, DEPTH_PLANE_MATERIAL_NODE_GROUP_NAME,
MATERIAL_COLOR_SPACE,
SHADELESS_NODE_GROUP_NAME,
SURFACE_MATERIAL_NODE_GROUP_NAME, SURFACE_MATERIAL_NODE_GROUP_NAME,
add_depth_plane_processing, add_depth_plane_processing,
create_material, create_material,
@@ -133,7 +135,6 @@ def main():
foreground, foreground,
1.0, 1.0,
displacement_image=depth, displacement_image=depth,
foreground_image=foreground,
) )
mesh.materials.append(material) mesh.materials.append(material)
material_group = bpy.data.node_groups[DEPTH_PLANE_MATERIAL_NODE_GROUP_NAME] material_group = bpy.data.node_groups[DEPTH_PLANE_MATERIAL_NODE_GROUP_NAME]
@@ -232,6 +233,47 @@ def main():
for node in material.node_tree.nodes for node in material.node_tree.nodes
if node.bl_idname == "ShaderNodeTexImage" if node.bl_idname == "ShaderNodeTexImage"
) )
shadeless_material = create_material(
"Shadeless",
foreground,
1.0,
shadeless=True,
)
shadeless_group = bpy.data.node_groups[SHADELESS_NODE_GROUP_NAME]
shadeless_nodes = [
node
for node in shadeless_material.node_tree.nodes
if node.bl_idname == "ShaderNodeGroup"
and node.node_tree == shadeless_group
]
assert len(shadeless_nodes) == 1
assert not any(
node.bl_idname == "ShaderNodeEmission"
for node in shadeless_material.node_tree.nodes
)
assert [
item.name
for item in shadeless_group.interface.items_tree
if item.item_type == "SOCKET" and item.in_out == "INPUT"
] == ["Color"]
assert [
item.name
for item in shadeless_group.interface.items_tree
if item.item_type == "SOCKET" and item.in_out == "OUTPUT"
] == ["Shader"]
assert {
node.label: node.operation
for node in shadeless_group.nodes
if node.bl_idname == "ShaderNodeMath"
} == {
"Bounce Count": "SUBTRACT",
"Camera or Refracted": "SUBTRACT",
"Limit Reflections": "SUBTRACT",
"Camera Ray to Glossy": "MULTIPLY",
"Shadow or Reflection?": "MAXIMUM",
"Shadow, Reflect or Refract?": "MAXIMUM",
}
assert foreground.colorspace_settings.name == MATERIAL_COLOR_SPACE
surface_material = create_material("Surface", foreground, 1.0) surface_material = create_material("Surface", foreground, 1.0)
surface_group = bpy.data.node_groups[SURFACE_MATERIAL_NODE_GROUP_NAME] surface_group = bpy.data.node_groups[SURFACE_MATERIAL_NODE_GROUP_NAME]
surface_group_nodes = [ surface_group_nodes = [

156
tools/validate_moge_depth_plane.py
View File

@@ -1,6 +1,8 @@
import sys import sys
import tempfile import tempfile
from pathlib import Path from pathlib import Path
from types import SimpleNamespace
from unittest.mock import patch
import bpy import bpy
import numpy as np import numpy as np
@@ -13,13 +15,81 @@ sys.path.insert(
str(next((PROJECT_ROOT / "src" / "snapform" / "wheels").glob("blendjob-*.whl"))), str(next((PROJECT_ROOT / "src" / "snapform" / "wheels").glob("blendjob-*.whl"))),
) )
import snapform.blender.depth_plane as depth_plane
from snapform.blender.depth_plane import ( from snapform.blender.depth_plane import (
MOGE_DEPTH_PLANE_NODE_GROUP_NAME, MOGE_DEPTH_PLANE_NODE_GROUP_NAME,
MOGE_MATERIAL_NODE_GROUP_NAME, MOGE_MATERIAL_NODE_GROUP_NAME,
build_moge2_depth_plane, build_moge2_depth_plane,
create_color_image,
get_modifier_input,
modifier_input_identifier,
) )
def validate_single_frame_source_types():
with tempfile.TemporaryDirectory() as temporary:
directory = Path(temporary)
color_path = directory / "color.0001.png"
seed = bpy.data.images.new("Color Seed", width=2, height=2, alpha=True)
seed.pixels[:] = [0.2, 0.4, 0.8, 1.0] * 4
seed.filepath_raw = str(color_path)
seed.file_format = "PNG"
seed.save()
bpy.data.images.remove(seed, do_unlink=True)
still_source_image = bpy.data.images.load(
str(color_path), check_existing=False
)
still_source = bpy.data.objects.new("Still Source", None)
still_source.empty_display_type = "IMAGE"
still_source.data = still_source_image
bpy.context.collection.objects.link(still_source)
still_color = create_color_image(
still_source, directory, assign_to_source=False
)
assert still_color.source == "FILE"
animated_source = SimpleNamespace(
data=SimpleNamespace(
source="SEQUENCE",
name="Sequence Source.png",
use_fake_user=False,
),
image_user=None,
)
sequence_color = create_color_image(
animated_source, directory, assign_to_source=False
)
assert sequence_color.source == "SEQUENCE"
depth = np.ones((1, 2, 2), dtype=np.float32)
mask = np.ones((1, 2, 2), dtype=bool)
intrinsics = np.array(
[[[1.0, 0.0, 0.5], [0.0, 1.0, 0.5], [0.0, 0.0, 1.0]]],
dtype=np.float32,
)
path = directory / "single-frame-results.npz"
np.savez(path, depth=depth, mask=mask, intrinsics=intrinsics)
with patch.object(depth_plane, "is_animated_image", return_value=True):
plane = build_moge2_depth_plane(
bpy.context,
still_source,
path,
delete_source=False,
color_image=sequence_color,
data_frames_directory=directory,
)
textures = [
node.image
for node in plane.data.materials[0].node_tree.nodes
if node.bl_idname == "ShaderNodeTexImage"
]
assert sequence_color in textures
depth_images = [image for image in textures if image != sequence_color]
assert len(depth_images) == 1
assert depth_images[0].source == "SEQUENCE"
def main(): def main():
bpy.ops.wm.read_factory_settings(use_empty=True) bpy.ops.wm.read_factory_settings(use_empty=True)
image = bpy.data.images.new("MoGe Test", width=64, height=48, alpha=True) image = bpy.data.images.new("MoGe Test", width=64, height=48, alpha=True)
@@ -45,11 +115,14 @@ def main():
mask=mask, mask=mask,
intrinsics=intrinsics, intrinsics=intrinsics,
) )
material_image = image.copy()
material_image.name = "MoGe Test.color"
plane = build_moge2_depth_plane( plane = build_moge2_depth_plane(
bpy.context, bpy.context,
source, source,
path, path,
delete_source=True, delete_source=True,
color_image=material_image,
) )
assert plane.modifiers[0].node_group.name == MOGE_DEPTH_PLANE_NODE_GROUP_NAME assert plane.modifiers[0].node_group.name == MOGE_DEPTH_PLANE_NODE_GROUP_NAME
assert MOGE_MATERIAL_NODE_GROUP_NAME in bpy.data.node_groups assert MOGE_MATERIAL_NODE_GROUP_NAME in bpy.data.node_groups
@@ -73,6 +146,9 @@ def main():
assert sockets["Background Depth"].structure_type == "SINGLE" assert sockets["Background Depth"].structure_type == "SINGLE"
assert sockets["Background Depth"].subtype == "DISTANCE" assert sockets["Background Depth"].subtype == "DISTANCE"
assert sockets["Background Depth"].default_value == 1.0 assert sockets["Background Depth"].default_value == 1.0
assert sockets["Color Image"].hide_in_modifier
color_identifier = modifier_input_identifier(geometry_group, "Color Image")
assert get_modifier_input(plane.modifiers[0], color_identifier) == material_image
material_group = bpy.data.node_groups[MOGE_MATERIAL_NODE_GROUP_NAME] material_group = bpy.data.node_groups[MOGE_MATERIAL_NODE_GROUP_NAME]
assert not any(node.bl_idname == "ShaderNodeNormalMap" for node in material_group.nodes) assert not any(node.bl_idname == "ShaderNodeNormalMap" for node in material_group.nodes)
assert any(node.bl_idname == "ShaderNodeBump" for node in material_group.nodes) assert any(node.bl_idname == "ShaderNodeBump" for node in material_group.nodes)
@@ -84,14 +160,90 @@ def main():
] ]
assert image_textures assert image_textures
assert all(node.extension == "EXTEND" for node in image_textures) assert all(node.extension == "EXTEND" for node in image_textures)
assert any(node.image.name == "MoGe Test.depth" for node in image_textures) static_depth = next(
node.image
for node in image_textures
if node.image.name == "MoGe Test.depth"
)
assert static_depth.source != "SEQUENCE"
geometry_textures = [ geometry_textures = [
node node
for node in geometry_group.nodes for node in geometry_group.nodes
if node.bl_idname == "GeometryNodeImageTexture" if node.bl_idname == "GeometryNodeImageTexture"
] ]
assert geometry_textures assert len(geometry_textures) == 2
assert all(node.extension == "EXTEND" for node in geometry_textures) assert all(node.extension == "EXTEND" for node in geometry_textures)
color_texture = next(
node
for node in geometry_textures
if any(
link.to_node == node
and link.to_socket == node.inputs["Image"]
and link.from_socket.name == "Color Image"
for link in geometry_group.links
)
)
depth_texture = next(node for node in geometry_textures if node != color_texture)
assert color_texture.interpolation == "Closest"
assert depth_texture.interpolation == "Linear"
color_frame_link = next(
link
for link in geometry_group.links
if link.to_node == color_texture
and link.to_socket == color_texture.inputs["Frame"]
)
depth_frame_link = next(
link
for link in geometry_group.links
if link.to_node == depth_texture
and link.to_socket == depth_texture.inputs["Frame"]
)
assert color_frame_link.from_socket == depth_frame_link.from_socket
alpha_compare = next(
node
for node in geometry_group.nodes
if node.bl_idname == "FunctionNodeCompare"
and node.operation == "GREATER_THAN"
and any(
link.from_node == color_texture
and link.from_socket == color_texture.outputs["Alpha"]
and link.to_node == node
for link in geometry_group.links
)
)
masked_depth = next(
node
for node in geometry_group.nodes
if node.bl_idname == "ShaderNodeMix"
and any(
link.from_node == alpha_compare and link.to_node == node
for link in geometry_group.links
)
)
assert masked_depth.data_type == "FLOAT"
masked_sources = {
link.from_node
for link in geometry_group.links
if link.to_node == masked_depth
}
assert alpha_compare in masked_sources
assert any(
node.bl_idname == "ShaderNodeMath" and node.operation == "MAXIMUM"
for node in masked_sources
)
assert any(
node.bl_idname == "ShaderNodeMath"
and node.operation == "MULTIPLY"
and node.inputs[1].default_value == -1.0
for node in masked_sources
)
assert any(
link.from_node == masked_depth
and link.to_node.bl_idname == "ShaderNodeMath"
and link.to_node.operation == "MULTIPLY"
for link in geometry_group.links
)
validate_single_frame_source_types()
print("MoGe-2 depth plane validation passed") print("MoGe-2 depth plane validation passed")

8
tools/validate_moge_surface.py
View File

@@ -62,7 +62,7 @@ def main():
) )
image = bpy.data.images.new("MoGe Surface Test", width=width, height=height) image = bpy.data.images.new("MoGe Surface Test", width=width, height=height)
foreground = bpy.data.images.new( foreground = bpy.data.images.new(
"MoGe Surface Test.foreground", "MoGe Surface Test.color",
width=width, width=width,
height=height, height=height,
) )
@@ -197,7 +197,8 @@ def main():
node for node in surface.data.materials[0].node_tree.nodes node for node in surface.data.materials[0].node_tree.nodes
if node.bl_idname == "ShaderNodeTexImage" and node.image == foreground if node.bl_idname == "ShaderNodeTexImage" and node.image == foreground
) )
assert color_texture.image.name == "MoGe Surface Test.foreground" assert color_texture.image.name == "MoGe Surface Test.color"
assert color_texture.image.colorspace_settings.name == "AgX Base sRGB"
normal_texture = next( normal_texture = next(
node for node in surface.data.materials[0].node_tree.nodes node for node in surface.data.materials[0].node_tree.nodes
if node.bl_idname == "ShaderNodeTexImage" if node.bl_idname == "ShaderNodeTexImage"
@@ -220,6 +221,8 @@ def validate_real_result(results_path, image_path):
normal_path = results_path.with_name("normal.png") normal_path = results_path.with_name("normal.png")
if not normal_path.is_file(): if not normal_path.is_file():
raise ValueError(f"Surface normal texture is missing: {normal_path}") raise ValueError(f"Surface normal texture is missing: {normal_path}")
color_image = bpy.data.images.load(str(image_path), check_existing=False)
color_image.name = f"{image_path.stem}.color"
surface = build_image_mesh( surface = build_image_mesh(
bpy.context, bpy.context,
source, source,
@@ -228,6 +231,7 @@ def validate_real_result(results_path, image_path):
True, True,
False, False,
True, True,
color_image=color_image,
normal_path=normal_path, normal_path=normal_path,
) )
assert len(surface.data.vertices) > 100 assert len(surface.data.vertices) > 100