Refactor image handling and update foreground processing

- Changed the logic in `remove_background_faces` to keep only faces with all foreground vertices.
- Updated background job to save foreground images with a new naming convention (`color.0001.png`).
- Replaced `copy_frames` with `copy_color_images` in various jobs to ensure consistent image naming.
- Introduced a new function `copy_color_images` to handle the copying of images with a specific naming format.
- Updated the BEN2 model to save output images with a new prefix for better clarity.
- Modified tests to reflect changes in image naming and processing logic.
- Added a new shadeless node group for improved material handling in geometry nodes.
- Enhanced validation scripts to check for new image naming conventions and ensure proper functionality.
This commit is contained in:
2026-08-19 01:45:59 +08:00
parent 835225d264
commit 20b653bb30
28 changed files with 663 additions and 383 deletions

View File

@@ -40,9 +40,9 @@ Environment 未安装时,同一位置显示两个安装按钮。Environment
- `Depth from`:`Original` 直接用原图,`DA3` 使用现有 DA3 流程,`MoGe-2` 生成 metric depth、normal 与有效区域 mask
- `Resolution Scale` 使用 `1–100%`,按原图长边计算 Process Res;结果会限制在 `252–1008` 并自动对齐到 14 的倍数
- `Remove Background` 默认关闭;开启后在转换期间使用 BEN2 删除背景
- `Unshaded` 保留为直接显示贴图颜色的选项
- `Shadeless` 使用 `O Shadeless` 节点组显示不受场景灯光影响的贴图颜色
- `Keep Original` 默认关闭,位于分割线下方;开启后保留源 Image Empty
4. 点击确认,稍等片刻就得到带位移的立体面片
4. 点击确认,稍等片刻就得到带位移的立体面片。`O DA3 Depth Plane` 与 `O MoGe2 Depth Plane` 都使用 `.color` 的 `Alpha > 0.5` 划分前景范围,范围外由 `Background Depth` 控制。
### 去除背景
@@ -57,8 +57,8 @@ ViT-S、ViT-B、ViT-L 或 DA3 Small、Base,其中 MoGe-2 ViT-S 为默认模型
Surface 输出流程构建 DA3 与 MoGe-2 源网格:MoGe-2 使用深度断层 mask,DA3 保留 confidence 与边拉伸
数据。两者再由独立的 Geometry Nodes
修改器完成剔面、快速重拓扑、Relax 与 UV 重投;顶层 `Iterations` 可设为 `0`。开启 Remove Background
时,BEN2 Alpha 只要大于 `0` 就视为前景;三角面有任一前景点即保留,只有全背景面会删除并且不再交给
Geometry Nodes;Surface 材质使用 `.foreground`。匹配相机默认不创建;开启后会作为 Surface 的子级。
时,BEN2 Alpha 只要大于 `0` 就视为前景;三角形三个顶点必须全部为前景,任一顶点 Alpha 为 `0`
都会删除该面并且不再交给 Geometry Nodes;Surface 材质使用 `.color`。匹配相机默认不创建;开启后会作为 Surface 的子级。
MoGe-2 的像素级 normal 会作为 Object Space normal 贴图接入 `O MoGe2 Surface Layer`,并与原图
Bump 合并;网格本身不写入 custom normal,因此重拓扑降低顶点数后仍能保留细腻的法线细节。
当前此功能仅支持单张图片。
@@ -67,20 +67,20 @@ MoGe-2 ViT-S、ViT-B、ViT-L、DA3 Small、DA3 Base 与 BEN2 全部使用 ONNX
点云重投影和网格生成均不依赖 PyTorch。Windows 使用 DirectML 加速,provider 不可用时会回退到 ONNX Runtime CPU,
并在任务状态中显示实际 provider。
转换对话框提供模型与 `Create Matching Camera`、`Remove Background`、`Unshaded`;DA3 显示
转换对话框提供模型与 `Create Matching Camera`、`Remove Background`、`Shadeless`;DA3 显示
`Resolution Scale`,MoGe-2 显示 `Resolution Level`,
以及分割线下方的 `Keep Original`。
界面会显示计算所得的实际 Process Res。剔除和重拓扑参数直接在
Geometry Nodes 修改器中调整;`Unshaded` 可使用与 Depth Plane 相同的 Emission 材质。普通材质的
Geometry Nodes 修改器中调整;`Shadeless` 与 Depth Plane 共用 `O Shadeless` 材质节点组。颜色贴图始终使用统一的 `.color` 转换产物;Remove Background 只决定是否由 BEN2 修改 Alpha,并使用 `AgX Base sRGB`。普通材质的
Bump Scale 会在材质组内乘以 `0.001`,并按透视表面的深度空间缩放。Surface 与 Depth Plane 的
Layer Color 同时驱动 Principled BSDF 的 Base Color、Subsurface Radius 和 Emission Color,Layer Normal
同时驱动 Normal 与 Coat Normal;Subsurface、Emission、Coat 的其他设置保持默认。Surface attributes
统一使用 `o_` 前缀;Server 准备过程显示
`Preparing surface data n/6`,可看到当前阶段与总步骤。
`Preparing surface data n/6`,可看到当前阶段与总步骤;Surface 单帧模型推理不显示多余的 `1/1`。
### 视频与序列帧
SnapForm 支持视频(mp4、mkv、mov 等)和序列帧,会逐帧处理并生成逐帧对应的立体面片。处理范围默认
SnapForm 支持视频(mp4、mkv、mov 等)和序列帧,会逐帧处理并生成逐帧对应的立体面片。`.color` 与 `.depth` 按源 Empty Image 的类型载入:普通单图为 `IMAGE`,Movie/Sequence 即使只处理一帧也仍为 `SEQUENCE`。处理范围默认
跟随 Empty 上设置的 `Image Frames` 与 `Offset`,不需要额外配置。DA3 的转换任务复用同一个 ONNX
Session 逐帧推理,显存峰值不随视频帧数平方增长。Debug 只保存模型 API 的直接返回:DA3 多帧输入
保留联合推理,MoGe-2 与 BEN2 按各自的单图 API 逐帧保存。

View File

@@ -15,11 +15,11 @@
- `Original`:直接把原图作为深度贴图,不显示估算参数。仅在开启 `Remove Background` 时运行 BEN2
取得前景 Alpha;视频或序列帧按需同时抽取原始帧与前景序列,把原始帧作为 `.depth` 序列。
`MoGe-2 ViT-S` 是默认深度来源。选择 `DA3` 时,其 Model 与 Resolution Scale
参数会集中显示在独立边框中。`Remove Background` 与 `Unshaded` 直接显示,不再使用 Options 折叠;
参数会集中显示在独立边框中。`Remove Background` 与 `Shadeless` 直接显示,不再使用 Options 折叠;
`Keep Original` 默认关闭,并在分割线下方保持为最后一项。仅当开启 `Remove Background` 时才需要
`BEN2 Base`,并把 `.foreground` 的 Alpha 作为独立
Mask,不在 Server 中改写深度值。保存 `.foreground` 时保留 BEN2 Alpha,但 RGB 使用未经修改的原始输入;
因此材质直接用 `.foreground` 同时提供颜色和 Mask,不再额外加载原图 Image Texture。
`BEN2 Base`,并把 `.color` 的 Alpha 作为独立 Mask,不在 Server 中改写深度值。无论颜色层是直接
复制原图,还是由 BEN2 为单图或序列修改 Alpha,Server 都统一写为 `color.0001.*`;Blender 只导入
这一份 `.color`,不复用源 Empty 的 Image,Color Space 固定为 `AgX Base sRGB`。
Image Empty 引用视频或序列帧时,DA3 与 MoGe-2 会按 Empty 的 Image Frames 数值逐帧估计
深度(未设置时为完整帧数,显式设为 1 时只处理 1 帧),起始帧跟随 Empty 的 Offset,与背景蒙版范围一致;
DA3 会复用同一个 ONNX Session 对每帧独立推理,再合并为原有序列输出,避免多帧 Attention 的平方级显存占用。
@@ -27,10 +27,10 @@
该行为只用于 Depth Plane 转换;Surface 仅支持单张图片。Debug 中 DA3 多帧输入联合推理,
MoGe-2 与 BEN2 按单图 API 逐帧运行。
Offset 超出视频长度时同样把窗口收进视频内。材质节点与置换贴图的 Start 保持与原 Empty
一致;Original 模式的视频或序列帧使用 `frame.0001.jpg` 等原图序列作为颜色与深度,
`foreground.0001.png` 只提供前景 Alpha。
一致;Original 模式的视频或序列帧使用 `color.0001.*` 颜色层同时作为颜色与深度输入;开启
Remove Background 时同名颜色层包含 BEN2 修改后的 Alpha。
DA3 的深度仍以 EXR 序列帧(`depth.0001.exr` 起)保存在 job 目录根下;MoGe-2 分别使用 `moge_depth`、`moge_mask` 与 `moge_normal` EXR 序列。
`Unshaded`、`Remove Background` 与 `Keep Original` 均默认关闭,因此默认删除原对象。
`Shadeless`、`Remove Background` 与 `Keep Original` 均默认关闭,因此默认删除原对象。`Shadeless` 会以 `O Shadeless` 节点组替代普通 Principled BSDF 路径,并继续通过 Alpha 与 Transparent BSDF 混合。
3. DA3 或 MoGe-2 完成后会自动创建面片。DA3 面片几何先由 `mini_npz` 的相机内参与深度确定,再通过对象
Transform 匹配原 Image Empty 的可见尺寸、位置、旋转和 Offset。Original 模式直接使用源 Empty
的可见尺寸、Offset 与 Transform。DA3 会把用于匹配画面尺寸的缩放预乘到 `.depth` 数值,
@@ -40,9 +40,10 @@
同名冲突,由 Blender 自动为生成面片追加编号。
4. Blender 只创建保留图像纵横比的最低密度基础网格,再把 `O DA3 Depth Plane` 作为 Geometry Nodes
修改器添加到对象。节点组与 `O DA3 Surface` 一同保存在 `O_SnapForm.blend`,关闭
`show_group_selector`,不为每个对象复制节点组。MoGe-2 使用单独的 `O MoGe2 Depth Plane`,只保留
网格细分、metric depth 位移、mask 删点和材质设置;材质使用 `O MoGe2 Depth Plane Layer`,直接接入
模型 normal,不携带 DA3 的材质置换与背景深度分支。DA3 修改器公开以下参数:
`show_group_selector`,不为每个对象复制节点组。MoGe-2 使用单独的 `O MoGe2 Depth Plane`,处理
网格细分、metric depth 位移和材质设置;材质使用 `O MoGe2 Depth Plane Layer`,直接接入模型 normal。
`O MoGe2 Depth Plane` 与 DA3 一样通过隐藏的 Color Image 读取 `.color` Alpha,以 `Alpha > 0.5`
划分前景范围,并在范围外改用 `Background Depth`。两个修改器公开以下参数:
- `Displacement`:位于 Geometry 之后的第一项,菜单包含 `Geometry`(默认)与 `Material`。`Geometry`
在节点组内沿法线修改网格,`Material` 保持网格平面并启用材质置换;
- `Subdivision`:位于第二项,默认 `5`。节点读取基础平面的 Bounding Box,短边分段数为 `2^Subdivision`,
@@ -51,16 +52,16 @@
材质置换使用该值;是否存在背景只取决于转换所得色彩贴图的 `Alpha > 0.5` 二值结果;
- `Depth Scale`:默认 `1`,允许正负值,Structure Type 为 `Single`,同时控制 Geometry 与 Material
两条置换路径。
Grid 重建后通过隐藏的 Material 输入重新赋予原图材质,并对所有面执行 Shade Smooth。
Grid 重建后通过隐藏的 Material 输入重新赋予转换材质,并对所有面执行 Shade Smooth。
材质使用显式指向 `UVMap` 的 UV Map 节点连接颜色、前景 Mask 与深度 Image Texture,不依赖活动 UV。
前景深度先与 `Background Depth` 取 Maximum,再按色彩贴图 Alpha 与背景深度混合,
深度分支在混合前使用 `Alpha > 0.5` 转为二值 Mask,避免半透明前景被背景深度拉扯;
材质表面 Alpha 同样使用二值 Mask。材质置换使用相同的二值和钳制顺序。
节点组还会把 `o_material_displacement` 与 `o_background_depth` 写入点域,材质通过
`Geometry` 类型的 Attribute 节点读取。点域会为顶点保存重复值,但能在应用修改器后继续保留参数;
相比自适应网格本身,其占用很小。Depth Image 与 Foreground Image 是修改器隐藏数据输入,
相比自适应网格本身,其占用很小。Depth Image 与 Color Image 是修改器隐藏数据输入,
由导入流程自动设置。
视频与序列帧还会为 Depth 与 Foreground 共同写入隐藏的 `Frame`、`Frame Start`、`Frames` 与 `Cyclic`。
`.color` 与 `.depth` 的 Blender Image 类型只依据源 Empty Image:普通单图使用 `IMAGE`,Movie 或 Sequence 使用 `SEQUENCE`;即使动画源这次只处理一帧也仍为 `SEQUENCE`,不按输出文件数量推断。视频与序列帧还会为 Depth 与 Color 共同写入隐藏的 `Frame`、`Frame Start`、`Frames` 与 `Cyclic`。
节点组内部不再使用 Scene Time。只有原 Image Empty 是视频或序列时,导入流程才直接给修改器的 `Frame`
socket 添加 Scripted Expression Driver `frame`;静态图保持默认值 `1`,不创建 Driver。导入内容已经
是截取后的帧,因此 Offset 在节点组内恒定为 `0`。

View File

@@ -89,18 +89,18 @@
2. 在 3D View 选中 Image Empty,右键打开 `SnapForm` 子菜单并选择 `Remove Background`。
3. 完成后以 Fake User 保留原 Image datablock,新建并 Pack 一张透明 PNG 到当前 `.blend`,然后只把
当前 Empty 的 `data` 换成新图;Empty 的 Transform、显示尺寸、Offset 及其他设置保持不变。
4. 新 Image 使用原图去掉格式后再追加 `.foreground` 的名称。例如 `img.png` 生成
`img.foreground`,`img.png.001` 生成 `img.001.foreground`。已有同名 datablock 时由
4. 新 Image 使用原图去掉格式后再追加 `.color` 的名称。例如 `img.png` 生成
`img.color`,`img.png.001` 生成 `img.001.color`。已有同名 datablock 时由
Blender 自动追加编号。
原图已 Pack 时会先写出临时 PNG 交给 BEN2,任务结束后删除临时输入;原 Image 仍保持 Pack
状态和原始内容。BEN2 的 `foreground.png` 也会保留在对应 job 目录中。
状态和原始内容。正式任务的 `color.0001.png` 也会保留在对应 job 目录中。
Image Empty 引用视频或序列帧时,Remove Background 直接执行,默认取 Empty 的 Image Frames
数值(未设置时为完整帧数,显式设为 1 时只处理 1 帧),起始帧跟随 Empty 的 Offset;若 Offset 加 Frames 超出视频长度,
会把窗口整体收进视频内(保证实际处理 Frames 帧数),Offset 过小时从第 1 帧开始。输出
`foreground.0001.png` 等透明序列帧(帧号从 0001 开始,与 Blender 场景帧一一对应),并把
源帧以 `frame.0001.jpg` 等命名复制到 job 目录根下供材质颜色导入。替换 Empty 的 Image 为
加载这些帧的 SEQUENCE Image,从原 Empty 的 Start 位置开始播放(Start 原样保留),
`color.0001.png` 等透明序列帧(帧号从 0001 开始,与 Blender 场景帧一一对应),并把
关闭去背景时,原始颜色帧同样以 `color.0001.*` 起始的统一名称写入 job 目录。替换 Empty 的 Image 为
加载这些帧的 SEQUENCE Image,从原 Empty 的 Start 位置开始播放(Start 原样保留)。图片类型只取决于源 Empty:普通单图的单个 `color.0001.*` 仍作为 `IMAGE`,Movie/Sequence 即使只导出一帧也作为 `SEQUENCE`;`.depth` 同样遵循该规则。
Cyclic 沿用原 Empty 的设置。序列帧无法被 Blender 整体 Pack,因此保持外部引用 job 目录;
请保留对应 job 目录,否则序列会失效。

View File

@@ -22,11 +22,13 @@ Depth Plane 导入所需的贴图。这里的文件转换属于正式 Depth 工
## Surface
generate-surface 统一写出 mesh.npz 与 texture.png。DA3 与 MoGe-2 都使用
geometry/surface.py 的同一套 Surface 构建算法;序列化位于 outputs/surface.py。
MoGe-2 normal 另存为 normal.png 并由 Blender 材质加载,不写入几何 NPZ。
generate-surface 统一写出 `mesh.npz`,不再生成 `texture.png`。DA3 与 MoGe-2 都使用
`geometry/surface.py` 的同一套 Surface 构建算法;序列化位于 `outputs/surface.py`。MoGe-2 normal
仍由 Server 单独导出为 `normal.png`,Blender 只负责加载并用于材质,不写入几何 NPZ。未去背景时
无论颜色来自原图复制还是 BEN2,Server 都统一写为 `color.0001.*` 起始的颜色层。Blender 不依据文件数量判断图片类型:源 Empty 为普通单图时载入为 `IMAGE`,源 Empty 为 Movie/Sequence 时即使只有一帧也载入为 `SEQUENCE`;`.depth` 使用相同规则。材质只在 Blender
端创建。MoGe-2 的 `normal.png` 仍由 Server 生成,Blender 只负责加载使用。
## BEN2
正式 remove-background 工作流输出 foreground.png 或透明前景序列。BEN2 ONNX API 返回的 Alpha
恢复到输入分辨率,RGB 保留输入图像颜色。
正式 remove-background 工作流输出 `color.0001.png` 起始的透明颜色层。BEN2 ONNX API 返回的 Alpha
恢复到输入分辨率,RGB 保留输入图像颜色。Debug BEN2 仍保留模型 API 语义,使用 `foreground` 文件名。

View File

@@ -9,7 +9,7 @@
Resolution Scale 与 Depth Plane 使用相同设计;MoGe-2 改为显示共享的 Resolution Level。
2. Server 先用未经抠图的原始 RGB 请求 DA3 的 `depth`、`conf` 与 `intrinsics`,随后仅在开启
`Remove Background` 时运行 BEN2 取得前景 Alpha;Alpha 经目标网格分辨率缩放与轻微模糊后,只要
大于 `0` 就把对应点视为前景;每个三角面只要有一个前景点就保留,三个点均为背景时才在输出前删除;
大于 `0` 就把对应点视为前景;每个三角形的三个顶点必须全部为前景,任一顶点 Alpha 为 `0` 就在输出前删除该面;
关闭时不加载 BEN2。Depth 不做 smoothing,Server 反投影深度网格,建立源三角面,
计算 confidence percentile 与 3×3 相对深度不连续度,并写入
基于原图像素坐标的源 `UVMap`。这些步骤只准备修改器的输入网格,不执行 Relax、拓扑修复、平滑、
@@ -44,11 +44,13 @@
Group Input,并隐藏没有连线的接口,避免跨越整个节点图的输入连线。
`O DA3 Surface` 与 `O MoGe2 Surface` 的 `Iterations` 默认值均为 `2`、最小值为 `0`,
`Resolution` 默认值均为 `0.1`,所有输入 socket 的 Structure Type 均固定为 `Single`。
5. Surface Job 统一按输入准备、模型推理、可选 BEN2、构建 SurfaceMesh、写出产物五个阶段更新进度;
5. Surface Job 统一按输入准备、模型推理、可选 BEN2、构建 SurfaceMesh、写出产物五个阶段更新进度;单帧推理只显示模型与阶段名称,不追加 `1/1`,仅多帧时显示当前帧与总数;
Surface Builder 是不接收 Job Context 或 progress callback 的纯数据转换。
6. Server 在 job 根目录写出 `mesh.npz` 与 `texture.png`。NPZ 保存 camera-space `vertices`、
`faces`、`uv`、`intrinsics`、`image_size` 与有效深度中位数 `reference_depth`。Blender 读取这些
数据后直接创建 Surface;材质默认使用原图,开启 Remove Background 时改用 `.foreground`,不需要 Open3D 或 OpenCV 进入 Blender Python 环境。
6. Server 在 job 根目录写出 `mesh.npz`。NPZ 保存 camera-space `vertices`、`faces`、`uv`、
`intrinsics`、`image_size` 与有效深度中位数 `reference_depth`。颜色层无论来自原图复制还是 BEN2,
Server 都统一写为 `color.0001.*`。Blender 读取网格数据后额外导入为 `.color`,材质不复用源
Empty 的 Image,Color Space 固定为 `AgX Base sRGB`。Server 不生成 `texture.png`,材质只在
Blender 端创建。
DA3 Surface 只保留 Geometry Nodes 后处理实际使用的 attributes:点域 `o_depth`、面域
`o_confidence` 与面域 `o_depth_discontinuity`。Remove Background 已在 Server
建面前直接删除背景三角面,不写入任何 background attribute。MoGe-2 的导入网格保留点域 `o_depth`,
@@ -62,11 +64,11 @@
缩放及父级关系产生的世界变换保持一致。更近的顶点沿 Empty 局部 `+Z` 方向凸出,三角面正面朝向
观察相机,启用 Backface Culling 时可从正面看到。
8. 转换对话框不再提供后处理参数,只保留模型与 Resolution Scale 设置、`Create Matching Camera`、
`Remove Background`、`Unshaded`,以及分割线下方的 `Keep Original`。背景移除默认关闭;开启后 Server
`Remove Background`、`Shadeless`,以及分割线下方的 `Keep Original`。背景移除默认关闭;开启后 Server
使用 BEN2,并在写出 Surface 前直接删除背景三角面,不再由 Geometry Nodes 的 Background 开关处理。
该预删除同时适用于 DA3 与 MoGe-2;因此 MoGe-2 转换后即使关闭 `Mask`,BEN2 已删除的背景也不会恢复。
开启 Remove Background 时,Surface 材质的 Color Image Texture 使用打包后的 `.foreground`,原 Empty 仍保持原图。
`Unshaded` 使用与 Depth Plane 相同的 Emission/Transparent 材质路径;普通材质的
Surface 材质的 Color Image Texture 始终使用 `.color`;Remove Background 只决定该层是否带 BEN2 Alpha,原 Empty 始终保持原图。颜色层使用 `AgX Base sRGB`。
`Shadeless` 使用与 Depth Plane 相同的 `O Shadeless` 节点组,并继续通过 Alpha 与 Transparent BSDF 混合;普通材质的
Bump Scale 按 `abs(object.scale.z) × camera scale_z` 设置,并在材质组内乘以固定基准 `0.001`,
与透视表面实际深度尺度一致;它允许负数以反转凹凸方向。
相机默认关闭;开启后会在 Empty 局部 `+Z` 侧创建一台朝向局部 `-Z` 的透视相机,并成为生成 Surface
@@ -122,7 +124,7 @@ Displacement 设置为 `Displacement Only`,DA3 与 Original 都使用 `Midleve
MoGe-2 Surface 使用独立的 `O MoGe2 Surface Layer`;它额外接收模型输出的 `Normal` 贴图,使用
`Normal Map` 节点按 `Object Space` 解码后连接到 Bump 的 `Normal` 输入,使像素级模型法线与原图
Bump 合并,不依赖重拓扑后的顶点密度;`Normal Strength` 默认值为 `1.0`,只调节模型 normal map,
原图 Bump 仍由独立的 `Bump Scale` 控制;
颜色贴图 Bump 仍由独立的 `Bump Scale` 控制;
Depth Plane 使用 `O DA3 Depth Plane Layer` 材质组,并把背景剔除、背景深度与材质置换 Attribute
读取全部收进组内。两者都接收 `Color`、`Alpha` 与 `Bump Scale`,只有 `O DA3 Depth Plane Layer`
额外接收 Float 类型的 `Depth`。
@@ -138,9 +140,10 @@ Depth Plane 的 Bump Height 直接使用材质组的 `Color` 输入,只受独
`0.001`,会乘以面片 Transform 的统一缩放倍率。原图 Color 还会经过 `Float Curve` 连接到
Principled BSDF 的 `Roughness`,曲线两端输出分别为 `0.5` 和 `0.1`;材质 IOR 保持
Principled BSDF 默认值 `1.5`。
开启 `Unshaded` 后,色彩贴图 Color 连接 Emission,二值 Alpha 作为 Mix Shader 系数,在
Transparent BSDF 与 Emission 之间混合后连接 Material Output;不创建 Principled BSDF、Bump 或
Float Curve。开启 `Remove Background` 时色彩贴图使用 BEN2 输出;关闭时使用原图。
开启 `Shadeless` 后,色彩贴图 Color 接入 `O Shadeless`。该组沿用 Blender Images as Planes 的
Diffuse、Emission、Light Path 与射线判断连接;二值 Alpha 仍作为 Mix Shader 系数,在 Transparent BSDF
与 `O Shadeless` 之间混合后连接 Material Output。外层不再创建 Emission、Principled BSDF、Bump 或
Float Curve。开启 `Remove Background` 时色彩贴图使用 `.color`;关闭时使用 `.color`。
这里的“原始距离”是 DA3 模型输出的尺度。普通 Small、Base、Large 和 Giant 模型的深度不应
被解释为现实世界的米;需要现实尺度时应选择带 metric depth 的
@@ -150,7 +153,7 @@ Float Curve。开启 `Remove Background` 时色彩贴图使用 BEN2 输出;关
生成完成后会直接创建面片,不提供缓存重建入口或单独的导入参数。重新选择
`Convert to Depth Plane` 会按当前 Depth from 再次转换。
面片、深度 Image、前景 Image、Material 与 Geometry Nodes 修改器会在同一个 Blender Undo 步骤中创建;转换完成后执行
面片、深度 Image、颜色 Image、Material 与 Geometry Nodes 修改器会在同一个 Blender Undo 步骤中创建;转换完成后执行
`Ctrl+Z` 会完整撤销本次转换,不会越过转换去回退更早的 Image Empty 操作。
Image Empty 使用 Packed Image 时,SnapForm 会通过 Blender 写出临时 PNG 交给 DA3,
@@ -161,7 +164,7 @@ Image Empty 使用 Packed Image 时,SnapForm 会通过 Blender 写出临时 PN
生成的 Object、Mesh 与 Material 仍命名为 `obj_img`。Pack 使用 OpenEXR,因而不会把大于 `1.0`
的原始深度截断。已有同名 datablock 时由 Blender 自动追加 `.001` 等编号。
视频输入时 Blender 无法整体 Pack 序列帧,因此 `.depth` 与颜色序列改为引用 job 目录根下的
`depth.0001.exr`、`frame.0001.jpg`(`.frame`)或 `foreground.0001.png`,并在每帧保留原始
`depth.0001.exr`、`color.0001.jpg`(`.color`)或 `color.0001.png`,并在每帧保留原始
浮点深度;保存 `.blend` 后再次打开仍需对应 job 目录存在。
每个 Server Job 被接受时,Job Runtime 在 Runtime Storage Root 的 `jobs/` 下创建一个独立文件夹。文件夹名是 32 位 opaque Job ID,不包含任务类型或对象名称。执行逻辑从 `JobContext.directory` 获取路径,输出产物直接写入该目录。
@@ -176,14 +179,13 @@ Job 目录中的典型正式产物如下:
```text
<opaque-job-id>/
├── results.npz、mesh.npz、texture.png 或 normal.png
├── results.npz、mesh.npz 或 normal.png
└── prediction.npz Debug 原始 API 返回
<opaque-job-id>/
├── foreground.png 单图结果
├── foreground.0001.png… 视频/序列帧结果
├── frame.0001.jpg… 视频/序列帧的源帧副本
└── depth.0001.exr… AI 深度序列帧
├── color.0001.jpg/png… 正式工作流颜色层(原图复制或 BEN2 Alpha)
├── depth.0001.exr… AI 深度序列帧
└── normal.png MoGe-2 Surface 法线贴图
```
建议从 `DA3 Base` 开始。它使用 Apache 2.0 模型权重。

Binary file not shown.

View File

@@ -23,6 +23,8 @@ SURFACE_MATERIAL_NODE_GROUP_NAME = "O DA3 Surface Layer"
MOGE_SURFACE_MATERIAL_NODE_GROUP_NAME = "O MoGe2 Surface Layer"
DEPTH_PLANE_MATERIAL_NODE_GROUP_NAME = "O DA3 Depth Plane Layer"
MOGE_MATERIAL_NODE_GROUP_NAME = "O MoGe2 Depth Plane Layer"
SHADELESS_NODE_GROUP_NAME = "O Shadeless"
MATERIAL_COLOR_SPACE = "AgX Base sRGB"
DA3_PATCH_SIZE = 14
DA3_MIN_PROCESS_RES = 252
DA3_MAX_PROCESS_RES = 1008
@@ -183,8 +185,8 @@ def copy_depth_image(source_object):
return depth_image
def removed_background_data_name(source_object):
return f"{image_base_name(source_object.data)}.foreground"
def color_data_name(source_object):
return f"{image_base_name(source_object.data)}.color"
def scaled_process_resolution(image, scale_percent):
@@ -328,7 +330,7 @@ def configure_image_user(
def pack_sequence_if_possible(image, expected_frames):
"""Pack a frame sequence when Blender supports it; otherwise keep external files."""
if expected_frames <= 0:
if expected_frames <= 1:
return False
try:
image.pack()
@@ -455,81 +457,60 @@ def create_depth_sequence_image(name, depth_frames, scene, frame_directory):
return sequence_image
def create_removed_background_image(
def create_color_image(
source_object,
output_path,
output_directory,
assign_to_source=True,
):
output_path = Path(output_path)
if output_path.is_dir():
foreground_paths = sorted_frame_paths(output_path, "foreground.")
if not foreground_paths:
raise RuntimeError(
f"Background removal result has no foreground frames: "
f"{output_path}"
)
try:
result_image = bpy.data.images.load(
str(foreground_paths[0]),
check_existing=False,
)
result_image.alpha_mode = "CHANNEL_PACKED"
force_sequence_source(result_image)
except (OSError, RuntimeError) as error:
if "result_image" in locals():
bpy.data.images.remove(result_image, do_unlink=True)
raise RuntimeError("Unable to load the background result") from error
source_image = source_object.data
original_fake_user = source_image.use_fake_user
try:
result_image.name = removed_background_data_name(source_object)
pack_sequence_if_possible(result_image, len(foreground_paths))
if assign_to_source:
image_user = getattr(source_object, "image_user", None)
(
original_frame_start,
_original_frame_offset,
_original_frame_duration,
original_use_cyclic,
) = image_user_settings(source_object)
source_image.use_fake_user = True
source_object.data = result_image
configure_image_user(
image_user,
len(foreground_paths),
frame_start=original_frame_start,
use_cyclic=original_use_cyclic,
)
except (OSError, RuntimeError) as error:
source_image.use_fake_user = original_fake_user
bpy.data.images.remove(result_image)
raise RuntimeError("Unable to load the background result") from error
return result_image
if not output_path.is_file():
output_directory = Path(output_directory)
color_paths = sorted_frame_paths(output_directory, "color.")
if not color_paths:
raise RuntimeError(
f"Background removal result does not exist: {output_path}"
f"Color result has no images: {output_directory}"
)
source_image = source_object.data
animated = is_animated_image(source_image)
try:
result_image = bpy.data.images.load(
str(output_path),
str(color_paths[0]),
check_existing=False,
)
result_image.alpha_mode = "CHANNEL_PACKED"
except RuntimeError as error:
raise RuntimeError("Unable to load the background result") from error
source_image = source_object.data
if animated:
force_sequence_source(result_image)
except (OSError, RuntimeError) as error:
if "result_image" in locals():
bpy.data.images.remove(result_image, do_unlink=True)
raise RuntimeError("Unable to load the color result") from error
original_fake_user = source_image.use_fake_user
try:
result_image.name = removed_background_data_name(source_object)
result_image.pack()
result_image.name = color_data_name(source_object)
if animated:
pack_sequence_if_possible(result_image, len(color_paths))
else:
result_image.pack()
if assign_to_source:
image_user = getattr(source_object, "image_user", None)
(
original_frame_start,
_original_frame_offset,
_original_frame_duration,
original_use_cyclic,
) = image_user_settings(source_object)
source_image.use_fake_user = True
source_object.data = result_image
if animated:
configure_image_user(
image_user,
len(color_paths),
frame_start=original_frame_start,
use_cyclic=original_use_cyclic,
)
except (OSError, RuntimeError) as error:
source_image.use_fake_user = original_fake_user
bpy.data.images.remove(result_image)
raise RuntimeError("Unable to pack the background result") from error
bpy.data.images.remove(result_image, do_unlink=True)
raise RuntimeError("Unable to prepare the color result") from error
return result_image
@@ -993,7 +974,7 @@ def create_moge_material(
color_image,
depth_image,
depth_scale,
unshaded=False,
shadeless=False,
frame_count=1,
frame_start=1,
use_cyclic=False,
@@ -1009,6 +990,7 @@ def create_moge_material(
uv.uv_map = "UVMap"
uv.location = (-900.0, 200.0)
color = nodes.new("ShaderNodeTexImage")
color_image.colorspace_settings.name = MATERIAL_COLOR_SPACE
color.image = color_image
color.interpolation = "Linear"
color.extension = "EXTEND"
@@ -1043,14 +1025,16 @@ def create_moge_material(
links.new(color.outputs["Alpha"], layer.inputs["Alpha"])
links.new(depth.outputs["Color"], layer.inputs["Depth"])
layer.inputs["Bump Scale"].default_value = depth_scale
if unshaded:
shader = nodes.new("ShaderNodeEmission")
if shadeless:
shader = nodes.new("ShaderNodeGroup")
shader.node_tree = load_shadeless_node_group()
shader.name = SHADELESS_NODE_GROUP_NAME
transparent = nodes.new("ShaderNodeBsdfTransparent")
mix = nodes.new("ShaderNodeMixShader")
links.new(layer.outputs["Color"], shader.inputs["Color"])
links.new(layer.outputs["Alpha"], mix.inputs["Fac"])
links.new(transparent.outputs["BSDF"], mix.inputs[1])
links.new(shader.outputs["Emission"], mix.inputs[2])
links.new(shader.outputs["Shader"], mix.inputs[2])
links.new(mix.outputs["Shader"], output.inputs["Surface"])
return material
shader = nodes.new("ShaderNodeBsdfPrincipled")
@@ -1307,16 +1291,13 @@ def create_material(
color_image,
depth_scale,
displacement_image=None,
foreground_image=None,
displacement_midlevel=0.0,
displacement_scale=1.0,
unshaded=False,
shadeless=False,
frame_count=1,
frame_start=1,
color_frame_offset=0,
color_frame_duration=None,
foreground_frame_offset=0,
foreground_frame_duration=None,
depth_frame_offset=0,
depth_frame_duration=None,
use_cyclic=None,
@@ -1336,7 +1317,8 @@ def create_material(
uv_map.uv_map = "UVMap"
uv_map.location = (-1250.0, 350.0)
material_image = foreground_image or color_image
material_image = color_image
material_image.colorspace_settings.name = MATERIAL_COLOR_SPACE
color_texture.image = material_image
color_texture.interpolation = "Linear"
color_texture.extension = "EXTEND"
@@ -1355,13 +1337,6 @@ def create_material(
)
material_frames = color_frames
material_frame_offset = color_frame_offset
if foreground_image is not None:
material_frames = (
foreground_frame_duration
if foreground_frame_duration is not None
else extracted_frames
)
material_frame_offset = foreground_frame_offset
if frame_count > 1 or is_animated_image(material_image):
configure_image_user(
color_texture.image_user,
@@ -1394,16 +1369,18 @@ def create_material(
normal_texture.location = (-1000.0, -50.0)
links.new(uv_map.outputs["UV"], normal_texture.inputs["Vector"])
links.new(normal_texture.outputs["Color"], material_inputs.inputs["Normal"])
if unshaded:
emission = nodes.new("ShaderNodeEmission")
if shadeless:
shader = nodes.new("ShaderNodeGroup")
shader.node_tree = load_shadeless_node_group()
shader.name = SHADELESS_NODE_GROUP_NAME
transparent = nodes.new("ShaderNodeBsdfTransparent")
mix_shader = nodes.new("ShaderNodeMixShader")
emission.location = (0.0, 550.0)
shader.location = (0.0, 550.0)
transparent.location = (0.0, 150.0)
mix_shader.location = (300.0, 350.0)
links.new(
material_inputs.outputs["Color"],
emission.inputs["Color"],
shader.inputs["Color"],
)
links.new(
material_inputs.outputs["Alpha"],
@@ -1414,7 +1391,7 @@ def create_material(
mix_shader.inputs[1],
)
links.new(
emission.outputs["Emission"],
shader.outputs["Shader"],
mix_shader.inputs[2],
)
links.new(
@@ -1502,13 +1479,13 @@ def node_asset_path():
return Path(__file__).parent.parent / "assets" / NODE_ASSET_NAME
def load_depth_plane_node_group(group_name=DEPTH_PLANE_NODE_GROUP_NAME):
def load_asset_node_group(group_name):
existing = bpy.data.node_groups.get(group_name)
if existing is not None:
return existing
asset_path = node_asset_path()
if not asset_path.is_file():
raise RuntimeError(f"Missing Geometry Nodes asset: {asset_path}")
raise RuntimeError(f"Missing node asset: {asset_path}")
use_pack = bpy.app.version >= (5, 0, 0)
if not use_pack:
directory = str(asset_path) + "/NodeTree/"
@@ -1520,18 +1497,15 @@ def load_depth_plane_node_group(group_name=DEPTH_PLANE_NODE_GROUP_NAME):
do_reuse_local_id=True,
)
if "FINISHED" not in result:
raise RuntimeError("Unable to append the Geometry Nodes asset")
raise RuntimeError("Unable to append the node asset")
node_group = bpy.data.node_groups.get(group_name)
if node_group is None:
raise RuntimeError("Appended Geometry Nodes asset is unavailable")
raise RuntimeError("Appended node asset is unavailable")
node_group.use_fake_user = True
return node_group
with bpy.data.libraries.load(str(asset_path), link=True) as (source, target):
if group_name not in source.node_groups:
raise RuntimeError(
"Geometry Nodes asset is missing "
f"{group_name}"
)
raise RuntimeError(f"Node asset is missing {group_name}")
target.node_groups = [group_name]
linked_group = target.node_groups[0]
node_group = bpy.data.pack_linked_ids_hierarchy(root_id=linked_group)
@@ -1541,6 +1515,14 @@ def load_depth_plane_node_group(group_name=DEPTH_PLANE_NODE_GROUP_NAME):
return node_group
def load_shadeless_node_group():
return load_asset_node_group(SHADELESS_NODE_GROUP_NAME)
def load_depth_plane_node_group(group_name=DEPTH_PLANE_NODE_GROUP_NAME):
return load_asset_node_group(group_name)
def modifier_input_identifier(node_group, name):
for item in node_group.interface.items_tree:
if item.item_type != "SOCKET" or item.in_out != "INPUT":
@@ -1600,7 +1582,7 @@ def drive_modifier_input(owner_object, modifier, identifier, expression="frame")
def add_depth_plane_processing(
plane_object,
depth_image,
foreground_image,
color_image,
material,
frame_start=1,
frame_duration=1,
@@ -1616,7 +1598,7 @@ def add_depth_plane_processing(
modifier.show_group_selector = False
for name, value in (
("Depth Image", depth_image),
("Foreground Image", foreground_image),
("Color Image", color_image),
("Material", material),
("Frame", 1),
("Frame Start", frame_start),
@@ -1635,6 +1617,7 @@ def add_depth_plane_processing(
def add_moge_depth_plane_processing(
plane_object,
depth_image,
color_image,
material,
frame_start=1,
frame_duration=1,
@@ -1652,6 +1635,7 @@ def add_moge_depth_plane_processing(
modifier.show_group_selector = False
for name, value in (
("Depth Image", depth_image),
("Color Image", color_image),
("Material", material),
("Frame", 1),
("Frame Start", frame_start),
@@ -1688,9 +1672,8 @@ def build_image_depth_plane(
context,
source_object,
delete_source,
unshaded=False,
shadeless=False,
color_image=None,
foreground_image=None,
depth_frames_directory=None,
depth_image=None,
):
@@ -1701,9 +1684,10 @@ def build_image_depth_plane(
frames_image = create_sequence_image(
depth_data_name(source_object),
depth_frames_directory,
frame_prefix="color.",
)
frame_count = len(
sorted_frame_paths(depth_frames_directory, "frame.")
sorted_frame_paths(depth_frames_directory, "color.")
)
(
source_frame_start,
@@ -1722,9 +1706,9 @@ def build_image_depth_plane(
depth_source_image = frames_image
elif not is_animated_image(source_object.data):
depth_source_image = copy_depth_image(source_object)
foreground_image = (
foreground_image or color_image or source_object.data
)
material_image = color_image
if material_image is None:
raise RuntimeError("Depth Plane material image is required")
mesh = create_plane_mesh(source_object, subdivision_level=None)
plane_object = bpy.data.objects.new(source_name, mesh)
context.collection.objects.link(plane_object)
@@ -1733,19 +1717,16 @@ def build_image_depth_plane(
depth_scale = abs(float(plane_object.scale.z))
material = create_material(
source_name,
color_image if color_image is not None else source_object.data,
material_image,
depth_scale,
displacement_image=depth_source_image,
foreground_image=foreground_image,
displacement_midlevel=ORIGINAL_DISPLACEMENT_MIDLEVEL,
displacement_scale=ORIGINAL_DISPLACEMENT_STRENGTH,
unshaded=unshaded,
shadeless=shadeless,
frame_count=frame_count,
frame_start=source_frame_start,
color_frame_offset=texture_frame_offset,
color_frame_duration=texture_frame_count,
foreground_frame_offset=0,
foreground_frame_duration=frame_count,
depth_frame_offset=texture_frame_offset,
depth_frame_duration=texture_frame_count,
use_cyclic=source_use_cyclic,
@@ -1754,7 +1735,7 @@ def build_image_depth_plane(
add_depth_plane_processing(
plane_object,
depth_source_image,
foreground_image,
material_image,
material,
frame_start=source_frame_start,
frame_duration=frame_count,
@@ -1775,10 +1756,9 @@ def build_depth_plane(
source_object,
npz_path,
delete_source,
unshaded=False,
shadeless=False,
depth_name="",
color_image=None,
foreground_image=None,
depth_frames_directory=None,
):
source_name = source_object.name
@@ -1797,9 +1777,9 @@ def build_depth_plane(
]
depth_name = depth_name or depth_data_name(source_object)
frame_count = len(depth_arrays)
foreground_image = (
foreground_image or color_image or source_object.data
)
material_image = color_image
if material_image is None:
raise RuntimeError("Depth Plane material image is required")
(
source_frame_start,
source_frame_offset,
@@ -1825,7 +1805,10 @@ def build_depth_plane(
depth_name,
scaled_depth_arrays[0],
)
if frame_count > 1 and depth_frames_directory is not None:
if (
is_animated_image(source_object.data)
and depth_frames_directory is not None
):
bpy.data.images.remove(depth_image, do_unlink=True)
depth_image = create_depth_sequence_image(
depth_name,
@@ -1836,21 +1819,18 @@ def build_depth_plane(
material = create_material(
source_name,
color_image if color_image is not None else source_object.data,
material_image,
depth_scale,
displacement_image=depth_image,
foreground_image=foreground_image,
displacement_midlevel=DISPLACEMENT_MIDLEVEL,
displacement_scale=DISPLACEMENT_STRENGTH,
unshaded=unshaded,
shadeless=shadeless,
frame_count=frame_count,
frame_start=source_frame_start,
color_frame_offset=0 if color_image is not None else source_frame_offset,
color_frame_duration=(
frame_count if color_image is not None else source_frame_duration
),
foreground_frame_offset=0,
foreground_frame_duration=frame_count,
depth_frame_offset=0,
depth_frame_duration=frame_count,
use_cyclic=source_use_cyclic,
@@ -1859,7 +1839,7 @@ def build_depth_plane(
add_depth_plane_processing(
plane_object,
depth_image,
foreground_image,
material_image,
material,
frame_start=source_frame_start,
frame_duration=frame_count,
@@ -1885,9 +1865,8 @@ def build_moge2_depth_plane(
source_object,
npz_path,
delete_source,
unshaded=False,
shadeless=False,
color_image=None,
foreground_image=None,
data_frames_directory=None,
):
import numpy as np
@@ -1935,7 +1914,10 @@ def build_moge2_depth_plane(
depth_data_name(source_object),
displacement[0] * depth_scale,
)
if len(depth) > 1 and data_frames_directory is not None:
if (
is_animated_image(source_object.data)
and data_frames_directory is not None
):
bpy.data.images.remove(depth_image, do_unlink=True)
depth_image = create_moge_sequence_image(
depth_data_name(source_object),
@@ -1945,12 +1927,15 @@ def build_moge2_depth_plane(
"moge_depth",
"depth",
)
material_image = color_image
if material_image is None:
raise RuntimeError("Depth Plane material image is required")
material = create_moge_material(
source_object.name,
foreground_image or color_image or source_object.data,
material_image,
depth_image,
depth_scale,
unshaded=unshaded,
shadeless=shadeless,
frame_count=len(depth),
frame_start=source_frame_start,
use_cyclic=source_use_cyclic,
@@ -1959,6 +1944,7 @@ def build_moge2_depth_plane(
modifier = add_moge_depth_plane_processing(
plane_object,
depth_image,
material_image,
material,
frame_start=source_frame_start,
frame_duration=len(depth),
@@ -2028,9 +2014,9 @@ class ConvertToDepthPlane(bpy.types.Operator):
bl_description = "Convert an image to a displaced depth plane"
bl_options = {"REGISTER"}
unshaded: bpy.props.BoolProperty(
name="Unshaded",
description="Display the source color through an Emission shader",
shadeless: bpy.props.BoolProperty(
name="Shadeless",
description="Display the source color without scene lighting",
default=False,
)
remove_background: bpy.props.BoolProperty(
@@ -2081,7 +2067,7 @@ class ConvertToDepthPlane(bpy.types.Operator):
elif self.model.startswith("MOGE2_"):
layout.prop(self, "resolution_level")
layout.prop(self, "remove_background")
layout.prop(self, "unshaded")
layout.prop(self, "shadeless")
layout.separator(type="LINE")
layout.prop(self, "keep_original")
@@ -2110,7 +2096,7 @@ class ConvertToDepthPlane(bpy.types.Operator):
source_object_name=source_object.name,
delete_source=not self.keep_original,
delete_input=delete_input,
unshaded=self.unshaded,
shadeless=self.shadeless,
)
if "CANCELLED" not in result:
return {"FINISHED"}
@@ -2134,7 +2120,7 @@ class ConvertToDepthPlane(bpy.types.Operator):
model=self.model,
delete_source=not self.keep_original,
delete_input=delete_input,
unshaded=self.unshaded,
shadeless=self.shadeless,
remove_background=self.remove_background,
)
if "CANCELLED" not in result:
@@ -2169,7 +2155,7 @@ class ConvertToDepthPlane(bpy.types.Operator):
process_res=process_res,
delete_source=not self.keep_original,
delete_input=delete_input,
unshaded=self.unshaded,
shadeless=self.shadeless,
remove_background=self.remove_background,
)
if "CANCELLED" in result:

View File

@@ -402,7 +402,7 @@ def build_image_mesh(
npz_path,
delete_source,
create_camera,
unshaded,
shadeless,
moge2=False,
color_image=None,
normal_path=None,
@@ -437,6 +437,8 @@ def build_image_mesh(
context.collection.objects.link(mesh_object)
mesh_object.matrix_world = source_object.matrix_world.copy()
depth_scale = material_depth_scale(mesh_object, camera_parameters)
if color_image is None:
raise RuntimeError("Surface material image is required")
normal_image = (
load_surface_normal_image(
normal_path,
@@ -447,9 +449,9 @@ def build_image_mesh(
)
material = create_material(
source_name,
color_image or source_object.data,
color_image,
depth_scale,
unshaded=unshaded,
shadeless=shadeless,
surface_group_name=(
MOGE_SURFACE_MATERIAL_NODE_GROUP_NAME if moge2 else None
),
@@ -509,9 +511,9 @@ class ConvertToSurface(bpy.types.Operator):
description="Create a perspective camera matching the source projection",
default=False,
)
unshaded: bpy.props.BoolProperty(
name="Unshaded",
description="Display the source color through an Emission shader",
shadeless: bpy.props.BoolProperty(
name="Shadeless",
description="Display the source color without scene lighting",
default=False,
)
remove_background: bpy.props.BoolProperty(
@@ -550,7 +552,7 @@ class ConvertToSurface(bpy.types.Operator):
draw_process_resolution(layout, self, context.object.data)
layout.prop(self, "create_camera")
layout.prop(self, "remove_background")
layout.prop(self, "unshaded")
layout.prop(self, "shadeless")
layout.separator(type="LINE")
layout.prop(self, "keep_original")
@@ -580,7 +582,7 @@ class ConvertToSurface(bpy.types.Operator):
"delete_source": not self.keep_original,
"delete_input": delete_input,
"create_camera": self.create_camera,
"unshaded": self.unshaded,
"shadeless": self.shadeless,
"remove_background": self.remove_background,
}
if self.model.startswith("MOGE2_"):

View File

@@ -111,13 +111,12 @@ def build_da3_depth_response(
depth_data_name,
remove_background,
delete_source,
unshaded,
shadeless,
):
from .blender.depth_plane import (
build_depth_plane,
create_removed_background_image,
create_sequence_image,
image_base_name,
create_color_image,
is_animated_image,
is_image_empty,
)
@@ -125,32 +124,21 @@ def build_da3_depth_response(
source_object = bpy.data.objects.get(source_object_name)
if not is_image_empty(source_object):
raise RuntimeError("The source image empty is no longer available")
foreground_paths = sorted(directory.glob("foreground.*.png"))
foreground_image = None
if remove_background:
foreground_image = create_removed_background_image(
source_object,
directory if foreground_paths else directory / "foreground.png",
assign_to_source=False,
)
color_image = None
if list(directory.glob("frame.*")) and foreground_image is None:
color_image = create_sequence_image(
f"{image_base_name(source_object.data)}.frame",
directory,
)
animated = is_animated_image(source_object.data)
color_image = create_color_image(
source_object,
directory,
assign_to_source=False,
)
build_depth_plane(
context,
source_object,
result.file("depth"),
delete_source,
unshaded=unshaded,
shadeless=shadeless,
depth_name=depth_data_name,
color_image=color_image,
foreground_image=foreground_image,
depth_frames_directory=(
directory if list(directory.glob("frame.*")) else None
),
depth_frames_directory=directory if animated else None,
)
push_undo("Convert to Depth Plane")
@@ -162,13 +150,11 @@ def build_moge2_depth_response(
source_object_name,
remove_background,
delete_source,
unshaded,
shadeless,
):
from .blender.depth_plane import (
build_moge2_depth_plane,
create_removed_background_image,
create_sequence_image,
image_base_name,
create_color_image,
is_image_empty,
)
@@ -176,28 +162,18 @@ def build_moge2_depth_response(
source_object = bpy.data.objects.get(source_object_name)
if not is_image_empty(source_object):
raise RuntimeError("The source image empty is no longer available")
foreground_paths = sorted(directory.glob("foreground.*.png"))
foreground_image = None
if remove_background:
foreground_image = create_removed_background_image(
source_object,
directory if foreground_paths else directory / "foreground.png",
assign_to_source=False,
)
color_image = None
if list(directory.glob("frame.*")) and foreground_image is None:
color_image = create_sequence_image(
f"{image_base_name(source_object.data)}.frame",
directory,
)
color_image = create_color_image(
source_object,
directory,
assign_to_source=False,
)
build_moge2_depth_plane(
context,
source_object,
result.file("depth"),
delete_source,
unshaded=unshaded,
shadeless=shadeless,
color_image=color_image,
foreground_image=foreground_image,
data_frames_directory=directory,
)
push_undo("Convert to Depth Plane")
@@ -210,13 +186,12 @@ def build_original_depth_response(
source_object_name,
background,
delete_source,
unshaded,
shadeless,
):
from .blender.depth_plane import (
build_image_depth_plane,
create_removed_background_image,
create_sequence_image,
image_base_name,
create_color_image,
is_animated_image,
is_image_empty,
)
@@ -224,30 +199,19 @@ def build_original_depth_response(
source_object = bpy.data.objects.get(source_object_name)
if not is_image_empty(source_object):
raise RuntimeError("The source image empty is no longer available")
foreground_image = None
if background:
foreground_paths = sorted(directory.glob("foreground.*.png"))
foreground_image = create_removed_background_image(
source_object,
directory if foreground_paths else directory / "foreground.png",
assign_to_source=False,
)
color_image = None
if list(directory.glob("frame.*")) and foreground_image is None:
color_image = create_sequence_image(
f"{image_base_name(source_object.data)}.frame",
directory,
)
animated = is_animated_image(source_object.data)
color_image = create_color_image(
source_object,
directory,
assign_to_source=False,
)
build_image_depth_plane(
context,
source_object,
delete_source,
unshaded=unshaded,
shadeless=shadeless,
color_image=color_image,
foreground_image=foreground_image,
depth_frames_directory=(
directory if list(directory.glob("frame.*")) else None
),
depth_frames_directory=directory if animated else None,
)
push_undo("Convert to Depth Plane")
@@ -259,27 +223,19 @@ def build_surface_response(
source_object_name,
delete_source,
create_camera,
unshaded,
shadeless,
moge2,
):
from .blender.depth_plane import (
create_removed_background_image,
is_image_empty,
)
from .blender.depth_plane import create_color_image, is_image_empty
from .blender.surface import build_image_mesh
source_object = bpy.data.objects.get(source_object_name)
if not is_image_empty(source_object):
raise RuntimeError("The source Image Empty no longer exists")
foreground_path = result.directory / "foreground.png"
color_image = (
create_removed_background_image(
source_object,
foreground_path,
assign_to_source=False,
)
if foreground_path.is_file()
else None
color_image = create_color_image(
source_object,
result.directory,
assign_to_source=False,
)
build_image_mesh(
context,
@@ -287,7 +243,7 @@ def build_surface_response(
result.file("surface"),
delete_source,
create_camera,
unshaded,
shadeless,
moge2,
color_image=color_image,
normal_path=(
@@ -300,19 +256,12 @@ def build_surface_response(
def apply_background_response(result, source_object_name):
from .blender.depth_plane import (
create_removed_background_image,
is_image_empty,
)
from .blender.depth_plane import create_color_image, is_image_empty
source_object = bpy.data.objects.get(source_object_name)
if not is_image_empty(source_object):
raise RuntimeError("The source image empty is no longer available")
foreground_paths = sorted(result.directory.glob("foreground.*.png"))
create_removed_background_image(
source_object,
result.directory if foreground_paths else result.file("foreground"),
)
create_color_image(source_object, result.directory)
push_undo("Remove Background")
@@ -461,7 +410,7 @@ class GenerateDa3Depth(JobOperatorBase, bpy.types.Operator):
process_res: bpy.props.IntProperty(default=1024, min=1, options=HIDDEN)
delete_source: bpy.props.BoolProperty(default=True, options=HIDDEN)
delete_input: bpy.props.BoolProperty(default=False, options=HIDDEN)
unshaded: bpy.props.BoolProperty(default=False, options=HIDDEN)
shadeless: bpy.props.BoolProperty(default=False, options=HIDDEN)
remove_background: bpy.props.BoolProperty(default=False, options=HIDDEN)
def request(self, context):
@@ -492,7 +441,7 @@ class GenerateDa3Depth(JobOperatorBase, bpy.types.Operator):
depth_data_name=self.depth_data_name,
remove_background=self.remove_background,
delete_source=self.delete_source,
unshaded=self.unshaded,
shadeless=self.shadeless,
)
return "Depth plane is ready"
@@ -512,7 +461,7 @@ class GenerateMoge2Depth(JobOperatorBase, bpy.types.Operator):
resolution_level: bpy.props.IntProperty(default=5, min=0, max=9, options=HIDDEN)
delete_source: bpy.props.BoolProperty(default=True, options=HIDDEN)
delete_input: bpy.props.BoolProperty(default=False, options=HIDDEN)
unshaded: bpy.props.BoolProperty(default=False, options=HIDDEN)
shadeless: bpy.props.BoolProperty(default=False, options=HIDDEN)
remove_background: bpy.props.BoolProperty(default=False, options=HIDDEN)
def request(self, context):
@@ -541,7 +490,7 @@ class GenerateMoge2Depth(JobOperatorBase, bpy.types.Operator):
source_object_name=self.source_object_name,
remove_background=self.remove_background,
delete_source=self.delete_source,
unshaded=self.unshaded,
shadeless=self.shadeless,
)
return "MoGe-2 depth plane is ready"
@@ -562,7 +511,7 @@ class GenerateDa3Surface(JobOperatorBase, bpy.types.Operator):
delete_source: bpy.props.BoolProperty(default=True, options=HIDDEN)
delete_input: bpy.props.BoolProperty(default=False, options=HIDDEN)
create_camera: bpy.props.BoolProperty(default=False, options=HIDDEN)
unshaded: bpy.props.BoolProperty(default=False, options=HIDDEN)
shadeless: bpy.props.BoolProperty(default=False, options=HIDDEN)
remove_background: bpy.props.BoolProperty(default=False, options=HIDDEN)
def request(self, context):
@@ -591,7 +540,7 @@ class GenerateDa3Surface(JobOperatorBase, bpy.types.Operator):
source_object_name=self.source_object_name,
delete_source=self.delete_source,
create_camera=self.create_camera,
unshaded=self.unshaded,
shadeless=self.shadeless,
moge2=False,
)
return "Perspective surface is ready"
@@ -613,7 +562,7 @@ class GenerateMoge2Surface(JobOperatorBase, bpy.types.Operator):
delete_source: bpy.props.BoolProperty(default=True, options=HIDDEN)
delete_input: bpy.props.BoolProperty(default=False, options=HIDDEN)
create_camera: bpy.props.BoolProperty(default=False, options=HIDDEN)
unshaded: bpy.props.BoolProperty(default=False, options=HIDDEN)
shadeless: bpy.props.BoolProperty(default=False, options=HIDDEN)
remove_background: bpy.props.BoolProperty(default=False, options=HIDDEN)
def request(self, context):
@@ -642,7 +591,7 @@ class GenerateMoge2Surface(JobOperatorBase, bpy.types.Operator):
source_object_name=self.source_object_name,
delete_source=self.delete_source,
create_camera=self.create_camera,
unshaded=self.unshaded,
shadeless=self.shadeless,
moge2=True,
)
return "MoGe-2 surface is ready"
@@ -691,7 +640,7 @@ class GenerateOriginalBackground(JobOperatorBase, bpy.types.Operator):
source_object_name: bpy.props.StringProperty(options=HIDDEN)
delete_source: bpy.props.BoolProperty(default=True, options=HIDDEN)
delete_input: bpy.props.BoolProperty(default=False, options=HIDDEN)
unshaded: bpy.props.BoolProperty(default=False, options=HIDDEN)
shadeless: bpy.props.BoolProperty(default=False, options=HIDDEN)
def request(self, context):
require_environment()
@@ -712,7 +661,7 @@ class GenerateOriginalBackground(JobOperatorBase, bpy.types.Operator):
source_object_name=self.source_object_name,
background=True,
delete_source=self.delete_source,
unshaded=self.unshaded,
shadeless=self.shadeless,
)
return "Depth plane is ready"
@@ -730,7 +679,7 @@ class ExtractOriginalFrames(JobOperatorBase, bpy.types.Operator):
source_object_name: bpy.props.StringProperty(options=HIDDEN)
delete_source: bpy.props.BoolProperty(default=True, options=HIDDEN)
delete_input: bpy.props.BoolProperty(default=False, options=HIDDEN)
unshaded: bpy.props.BoolProperty(default=False, options=HIDDEN)
shadeless: bpy.props.BoolProperty(default=False, options=HIDDEN)
def request(self, context):
require_environment()
@@ -748,7 +697,7 @@ class ExtractOriginalFrames(JobOperatorBase, bpy.types.Operator):
source_object_name=self.source_object_name,
background=False,
delete_source=self.delete_source,
unshaded=self.unshaded,
shadeless=self.shadeless,
)
return "Depth plane is ready"

View File

@@ -69,9 +69,9 @@ def triangle_edge_stretch_values(points, triangles):
def remove_background_faces(faces, foreground_mask):
"""Keep faces with at least one foreground vertex."""
"""Keep only faces whose vertices are all foreground."""
foreground_points = foreground_mask.reshape(-1)
return faces[foreground_points[faces].any(axis=1)]
return faces[foreground_points[faces].all(axis=1)]
def depth_edge_mask(depth, threshold=12.0 / 255.0):

View File

@@ -1,7 +1,7 @@
import shutil
import time
from ..media import copy_frames, prepare_background_input
from ..media import prepare_background_input
from ..models.ben2 import infer_foreground, infer_foreground_frames
@@ -39,7 +39,7 @@ def run(context, parameters):
)
context.check_cancelled()
context.progress(0.92, "Writing transparent PNG")
output_path = output_directory / "foreground.png"
output_path = output_directory / "color.0001.png"
foreground.convert("RGBA").save(
output_path,
format="PNG",
@@ -58,8 +58,8 @@ def run(context, parameters):
0.7,
model_cache=model_manager,
cancel_check=context.check_cancelled,
output_prefix="color",
)
copy_frames(images, output_directory)
finally:
if temporary_directory is not None:
shutil.rmtree(temporary_directory, ignore_errors=True)
@@ -73,6 +73,4 @@ def run(context, parameters):
)
print(message)
context.progress(1.0, message)
if tuple(output_directory.glob("foreground.*.png")):
return {"foreground": "."}
return {"foreground": "foreground.png"}
return {"color": "."}

View File

@@ -1,4 +1,4 @@
from ..media import copy_frames
from ..media import copy_color_images
from ..models import da3, geometry_adapter, moge2
from ..outputs.depth import write_da3_depth, write_moge_depth
from .common import input_frames
@@ -36,14 +36,15 @@ def run(context, parameters):
cancel_check=context.check_cancelled,
)
write_moge_depth(context.directory, prediction)
write_optional(
color_paths = write_optional(
context,
parameters,
images,
progress_start=0.84,
progress_span=0.08,
)
copy_frames(images, context.directory)
if not color_paths:
copy_color_images(images, context.directory)
context.check_cancelled()
context.progress(1.0, f"Exported depth for {len(images)} frame(s)")
return {"depth": "results.npz"}

View File

@@ -24,7 +24,7 @@ def write_optional(
model_cache=model_manager,
cancel_check=context.check_cancelled,
)
path = context.directory / "foreground.png"
path = context.directory / "color.0001.png"
foreground.convert("RGBA").save(
path,
format="PNG",
@@ -42,8 +42,9 @@ def write_optional(
progress_span,
model_cache=model_manager,
cancel_check=context.check_cancelled,
output_prefix="color",
)
paths = sorted(context.directory.glob("foreground.*.png"))
paths = sorted(context.directory.glob("color.*.png"))
print(
f"BEN2 inference {inference_ms:.0f} ms; "
f"model load {load_ms:.0f} ms"

View File

@@ -1,6 +1,6 @@
import shutil
from ..media import copy_frames, prepare_input
from ..media import copy_color_images, prepare_input
def run(context, parameters):
@@ -13,7 +13,7 @@ def run(context, parameters):
cancel_check=context.check_cancelled,
)
try:
copy_frames(images, context.directory)
copy_color_images(images, context.directory)
finally:
if temporary_directory is not None:
shutil.rmtree(temporary_directory, ignore_errors=True)

View File

@@ -1,15 +1,18 @@
from ..geometry import foreground_mesh_mask
from ..geometry import build_surface, foreground_mesh_mask
from ..media import copy_color_images
from ..models import geometry_adapter, moge2
from ..geometry import build_surface
from ..outputs.surface import (
write_normal_texture,
write_surface_archive,
write_surface_texture,
)
from ..outputs.surface import write_normal_texture, write_surface_archive
from .common import input_frames
from .foreground import write_optional
def inference_progress_message(model_name, index, total):
message = f"Estimating {model_name} geometry"
if total > 1:
message += f" ({index + 1}/{total})"
return message
def run(context, parameters):
context.progress(0.02, "Preparing surface input")
context.directory.mkdir(parents=True, exist_ok=True)
@@ -27,7 +30,7 @@ def run(context, parameters):
include_points=True,
progress_callback=lambda index, total: context.progress(
0.12 + 0.58 * index / total,
f"Estimating MoGe-2 geometry ({index + 1}/{total})",
inference_progress_message("MoGe-2", index, total),
),
cancel_check=context.check_cancelled,
)
@@ -38,13 +41,13 @@ def run(context, parameters):
images,
progress_callback=lambda index, total: context.progress(
0.12 + 0.58 * index / total,
f"Estimating DA3 geometry ({index + 1}/{total})",
inference_progress_message("DA3", index, total),
),
cancel_check=context.check_cancelled,
)
context.check_cancelled()
frame = adapter.surface_frame(prediction)
foreground_paths = write_optional(
color_paths = write_optional(
context,
parameters,
images,
@@ -52,19 +55,17 @@ def run(context, parameters):
progress_span=0.08,
)
foreground_mask = None
if foreground_paths:
if color_paths:
foreground_mask = foreground_mesh_mask(
foreground_paths[0],
color_paths[0],
frame.depth.shape,
)
context.progress(0.82, "Building surface")
mesh = build_surface(frame, foreground_mask=foreground_mask)
context.progress(0.92, "Writing surface")
write_surface_archive(mesh, context.directory / "mesh.npz")
write_surface_texture(
foreground_paths[0] if foreground_paths else images[0],
context.directory / "texture.png",
)
if not color_paths:
copy_color_images(images, context.directory)
if adapter is moge2:
write_normal_texture(
prediction.normal[0],

View File

@@ -4,6 +4,7 @@ from .input import (
TaskCancelled,
check_cancelled,
collect_images,
copy_color_images,
copy_frames,
extract_video_frames,
first_frames,
@@ -18,6 +19,7 @@ __all__ = (
"TaskCancelled",
"check_cancelled",
"collect_images",
"copy_color_images",
"copy_frames",
"extract_video_frames",
"first_frames",

View File

@@ -143,16 +143,26 @@ def prepare_background_input(
)
def copy_frames(images, output_directory):
"""Copy a multi-frame input as numbered frame files for the Blender sequence."""
if len(images) <= 1:
return
def copy_color_images(images, output_directory):
"""Copy input images as the canonical numbered color layer."""
for index, image_path in enumerate(images):
suffix = Path(image_path).suffix.lower() or ".jpg"
destination = (
Path(output_directory) / f"frame.{index + 1:04d}{suffix}"
Path(output_directory) / f"color.{index + 1:04d}{suffix}"
)
if Path(image_path).resolve() == destination.resolve():
continue
shutil.copy2(image_path, destination)
def copy_frames(images, output_directory):
"""Copy multiple input images as numbered raw Debug frame files."""
if len(images) <= 1:
return
for index, image_path in enumerate(images):
suffix = Path(image_path).suffix.lower() or ".jpg"
destination = Path(output_directory) / f"frame.{index + 1:04d}{suffix}"
if Path(image_path).resolve() == destination.resolve():
continue
shutil.copy2(image_path, destination)

View File

@@ -48,8 +48,9 @@ def infer_foreground_frames(
progress_span,
model_cache=None,
cancel_check=None,
output_prefix="foreground",
):
"""Run BEN2 on every frame and write foreground.0001.png etc."""
"""Run BEN2 on every frame and write numbered transparent PNG files."""
total_load_ms = 0.0
total_inference_ms = 0.0
for index, image_path in enumerate(images):
@@ -65,7 +66,7 @@ def infer_foreground_frames(
model_cache=model_cache,
cancel_check=cancel_check,
)
output_path = output_directory / f"foreground.{index + 1:04d}.png"
output_path = output_directory / f"{output_prefix}.{index + 1:04d}.png"
foreground.convert("RGBA").save(
output_path,
format="PNG",

View File

@@ -1,5 +1,5 @@
from .depth import write_da3_depth, write_moge_depth
from .surface import write_normal_texture, write_surface_archive, write_surface_texture
from .surface import write_normal_texture, write_surface_archive
__all__ = (
@@ -7,5 +7,4 @@ __all__ = (
"write_moge_depth",
"write_normal_texture",
"write_surface_archive",
"write_surface_texture",
)

View File

@@ -20,15 +20,6 @@ def write_surface_archive(mesh, output_path):
return output_path
def write_surface_texture(image_path, output_path):
from PIL import Image
output_path = Path(output_path)
with Image.open(image_path) as image:
image.convert("RGBA").save(output_path, format="PNG", compress_level=1)
return output_path
def write_normal_texture(normal, output_path):
import numpy as np
from PIL import Image

View File

@@ -686,7 +686,7 @@ class AddonTest(unittest.TestCase):
operator.source_object_name = "Reference"
operator.remove_background = False
operator.delete_source = True
operator.unshaded = False
operator.shadeless = False
result = object()
context = object()
@@ -701,7 +701,7 @@ class AddonTest(unittest.TestCase):
source_object_name="Reference",
remove_background=False,
delete_source=True,
unshaded=False,
shadeless=False,
)
def test_entry_operator_dispatches_plain_properties(self):
@@ -709,7 +709,7 @@ class AddonTest(unittest.TestCase):
operator.model = "MOGE2_VITS_NORMAL"
operator.resolution_level = 7
operator.keep_original = False
operator.unshaded = True
operator.shadeless = True
operator.remove_background = False
source_object = SimpleNamespace(
name="Reference",

View File

@@ -291,12 +291,12 @@ class ImageDepthTest(unittest.TestCase):
"微信视频2026-07-23_192756_414",
)
def test_removed_background_name_strips_video_extension(self):
def test_color_name_strips_video_extension(self):
source_object = SimpleNamespace(data=SimpleNamespace(name="clip.mp4"))
self.assertEqual(
self.image_depth.removed_background_data_name(source_object),
"clip.foreground",
self.image_depth.color_data_name(source_object),
"clip.color",
)
def test_force_sequence_source_keeps_loaded_sequence(self):
@@ -634,19 +634,19 @@ class ImageDepthTest(unittest.TestCase):
self.assertEqual(depth_image.name, "img.depth")
def test_removed_background_name_uses_source_image_base_name(self):
def test_color_name_uses_source_image_base_name(self):
source_object = SimpleNamespace(
data=SimpleNamespace(name="img.png"),
)
self.assertEqual(
self.image_depth.removed_background_data_name(source_object),
"img.foreground",
self.image_depth.color_data_name(source_object),
"img.color",
)
source_object.data.name = "img.png.001"
self.assertEqual(
self.image_depth.removed_background_data_name(source_object),
"img.001.foreground",
self.image_depth.color_data_name(source_object),
"img.001.color",
)
def test_surface_displacement_is_set_to_displacement_only(self):

View File

@@ -376,7 +376,7 @@ class ServerTest(unittest.TestCase):
)
status = self.wait_for_job(context)
self.assertTrue(
(context.directory / "frame.0001.jpg").is_file()
(context.directory / "color.0001.jpg").is_file()
)
self.assertEqual(status["state"], "succeeded")

View File

@@ -127,9 +127,24 @@ class WorkerTest(unittest.TestCase):
cls.foreground_job = load_server_module("jobs.foreground")
cls.download = load_server_module("model_download")
def test_surface_inference_progress_omits_single_frame_count(self):
self.assertEqual(
self.surface_job.inference_progress_message("MoGe-2", 0, 1),
"Estimating MoGe-2 geometry",
)
self.assertEqual(
self.surface_job.inference_progress_message("DA3", 0, 1),
"Estimating DA3 geometry",
)
self.assertEqual(
self.surface_job.inference_progress_message("DA3", 1, 3),
"Estimating DA3 geometry (2/3)",
)
def test_surface_has_one_model_independent_builder(self):
self.assertTrue(callable(self.surface.build_surface))
self.assertTrue(callable(self.surface_outputs.write_surface_archive))
self.assertFalse(hasattr(self.surface_outputs, "write_surface_texture"))
self.assertFalse(hasattr(self.surface, "export_da3_surface"))
self.assertFalse(hasattr(self.surface, "export_moge2_surface"))
@@ -226,6 +241,33 @@ class WorkerTest(unittest.TestCase):
)
self.assertIsNone(temporary_directory)
def test_copy_color_images_numbers_a_still_image(self):
with tempfile.TemporaryDirectory() as directory:
root = Path(directory)
source = root / "source.png"
source.write_bytes(b"frame")
output = root / "result"
output.mkdir()
self.media.copy_color_images([source], output)
self.assertEqual(
(output / "color.0001.png").read_bytes(),
b"frame",
)
def test_copy_frames_skips_a_still_image_by_default(self):
with tempfile.TemporaryDirectory() as directory:
root = Path(directory)
source = root / "source.png"
source.touch()
output = root / "result"
output.mkdir()
self.media.copy_frames([source], output)
self.assertEqual(list(output.iterdir()), [])
def test_extract_video_frames_reads_frames_sequentially(self):
with tempfile.TemporaryDirectory() as directory:
root = Path(directory)
@@ -497,7 +539,7 @@ class WorkerTest(unittest.TestCase):
{"input": str(input_path), "model": "BEN2", "device": "cpu"},
)
self.assertTrue((root / "result" / "foreground.png").is_file())
self.assertTrue((root / "result" / "color.0001.png").is_file())
self.assertEqual(
inference_options[0][1],
@@ -590,16 +632,10 @@ class WorkerTest(unittest.TestCase):
result_directory = context.directory
self.assertTrue(
(result_directory / "foreground.0001.png").is_file()
(result_directory / "color.0001.png").is_file()
)
self.assertTrue(
(result_directory / "foreground.0002.png").is_file()
)
self.assertTrue(
(result_directory / "frame.0001.jpg").is_file()
)
self.assertTrue(
(result_directory / "frame.0002.jpg").is_file()
(result_directory / "color.0002.png").is_file()
)
self.assertEqual(len(inference_options), 2)
@@ -630,7 +666,7 @@ class WorkerTest(unittest.TestCase):
for path in result_directory.iterdir()
if path.is_file()
),
["frame.0001.jpg", "frame.0002.jpg", "frame.0003.jpg"],
["color.0001.jpg", "color.0002.jpg", "color.0003.jpg"],
)
self.assertEqual(context.status["progress"], 1.0)
self.assertIn("3 frame", context.status["message"])
@@ -889,10 +925,10 @@ class WorkerTest(unittest.TestCase):
result_directory = context.directory
self.assertTrue(
(result_directory / "frame.0001.jpg").is_file()
(result_directory / "color.0001.jpg").is_file()
)
self.assertTrue(
(result_directory / "frame.0002.jpg").is_file()
(result_directory / "color.0002.jpg").is_file()
)
self.assertEqual(len(inference_inputs), 2)
self.assertTrue(all(len(paths) == 1 for paths in inference_inputs))
@@ -1107,19 +1143,19 @@ class WorkerTest(unittest.TestCase):
)
np.testing.assert_array_equal(compact_faces, [[0, 1, 2]])
def test_surface_background_keeps_faces_with_any_foreground_vertex(self):
def test_surface_background_requires_every_vertex_to_be_foreground(self):
faces = np.asarray(
((0, 1, 4), (0, 4, 3), (1, 2, 5), (1, 5, 4)),
dtype=np.int32,
)
foreground = np.asarray(
((False, False, False), (False, True, False)),
((True, True, False), (False, True, False)),
dtype=bool,
)
filtered = self.geometry.remove_background_faces(faces, foreground)
np.testing.assert_array_equal(filtered, faces[[0, 1, 3]])
np.testing.assert_array_equal(filtered, faces[[0]])
def test_surface_background_mask_keeps_any_nonzero_alpha(self):
from PIL import Image
@@ -1255,6 +1291,3 @@ class WorkerTest(unittest.TestCase):
if __name__ == "__main__":
unittest.main()

View File

@@ -7,6 +7,7 @@ GROUP_NAME = "O DA3 Surface"
MOGE_SURFACE_GROUP_NAME = "O MoGe2 Surface"
DEPTH_PLANE_GROUP_NAME = "O DA3 Depth Plane"
MOGE_DEPTH_PLANE_GROUP_NAME = "O MoGe2 Depth Plane"
SHADELESS_GROUP_NAME = "O Shadeless"
MATERIAL_DISPLACEMENT_ATTRIBUTE = "o_material_displacement"
BACKGROUND_DEPTH_ATTRIBUTE = "o_background_depth"
DEPTH_SCALE_ATTRIBUTE = "o_depth_scale"
@@ -800,9 +801,9 @@ def build_depth_plane_group():
"NodeSocketImage",
parent=data_panel,
)
foreground_image = new_socket(
color_image = new_socket(
interface,
"Foreground Image",
"Color Image",
"INPUT",
"NodeSocketImage",
parent=data_panel,
@@ -835,7 +836,7 @@ def build_depth_plane_group():
socket.min_value = 1
frame_data.append(socket)
depth_image.hide_in_modifier = True
foreground_image.hide_in_modifier = True
color_image.hide_in_modifier = True
material.hide_in_modifier = True
for socket in frame_data:
socket.hide_in_modifier = True
@@ -848,7 +849,7 @@ def build_depth_plane_group():
images_input = new_group_input(
nodes,
(100.0, 1250.0),
{"Depth Image", "Foreground Image"},
{"Depth Image", "Color Image"},
)
frames_input = new_group_input(
nodes,
@@ -973,7 +974,7 @@ def build_depth_plane_group():
foreground_texture.location = (350.0, 900.0)
links.new(images_input.outputs["Depth Image"], depth_texture.inputs["Image"])
links.new(
images_input.outputs["Foreground Image"],
images_input.outputs["Color Image"],
foreground_texture.inputs["Image"],
)
links.new(grid.outputs["UV Map"], depth_texture.inputs["Vector"])
@@ -1141,10 +1142,13 @@ def build_moge_depth_plane_group():
depth_image = new_socket(
interface, "Depth Image", "INPUT", "NodeSocketImage", data_panel
)
color_image = new_socket(
interface, "Color Image", "INPUT", "NodeSocketImage", data_panel
)
material = new_socket(
interface, "Material", "INPUT", "NodeSocketMaterial", data_panel
)
for socket in (depth_image, material):
for socket in (depth_image, color_image, material):
socket.hide_in_modifier = True
frame_sockets = []
for name, socket_type, default in (
@@ -1176,7 +1180,7 @@ def build_moge_depth_plane_group():
background_depth_value.inputs[0],
)
images = new_group_input(
nodes, (-500.0, 1100.0), {"Depth Image"}
nodes, (-500.0, 1100.0), {"Depth Image", "Color Image"}
)
frames = new_group_input(
nodes,
@@ -1241,18 +1245,42 @@ def build_moge_depth_plane_group():
depth_texture.interpolation = "Linear"
depth_texture.extension = "EXTEND"
depth_texture.location = (-250.0, 1100.0)
color_texture = nodes.new("GeometryNodeImageTexture")
color_texture.interpolation = "Closest"
color_texture.extension = "EXTEND"
color_texture.location = (-250.0, 850.0)
links.new(images.outputs["Depth Image"], depth_texture.inputs["Image"])
links.new(images.outputs["Color Image"], color_texture.inputs["Image"])
links.new(grid.outputs["UV Map"], depth_texture.inputs["Vector"])
links.new(grid.outputs["UV Map"], color_texture.inputs["Vector"])
frame = build_sequence_frame(
nodes, links, frames.outputs["Frame"], frames, 1800.0
)
links.new(frame, depth_texture.inputs["Frame"])
links.new(frame, color_texture.inputs["Frame"])
clamp_depth = nodes.new("ShaderNodeMath")
clamp_depth.operation = "MAXIMUM"
clamp_depth.location = (0.0, 750.0)
links.new(depth_texture.outputs["Color"], clamp_depth.inputs[0])
links.new(background_depth_value.outputs[0], clamp_depth.inputs[1])
color_mask = new_compare(
nodes,
"FLOAT",
"GREATER_THAN",
(0.0, 1000.0),
0.5,
)
links.new(
color_texture.outputs["Alpha"],
input_socket(color_mask, "A", 0),
)
masked_depth = nodes.new("ShaderNodeMix")
masked_depth.data_type = "FLOAT"
masked_depth.location = (200.0, 850.0)
links.new(color_mask.outputs["Result"], masked_depth.inputs[0])
links.new(background_depth_value.outputs[0], masked_depth.inputs[2])
links.new(clamp_depth.outputs[0], masked_depth.inputs[3])
displacement_menu = nodes.new("GeometryNodeMenuSwitch")
displacement_menu.data_type = "FLOAT"
displacement_menu.enum_items.clear()
@@ -1276,7 +1304,7 @@ def build_moge_depth_plane_group():
scaled_depth = nodes.new("ShaderNodeMath")
scaled_depth.operation = "MULTIPLY"
scaled_depth.location = (250.0, 750.0)
links.new(clamp_depth.outputs[0], scaled_depth.inputs[0])
links.new(masked_depth.outputs["Result"], scaled_depth.inputs[0])
links.new(geometry_strength.outputs[0], scaled_depth.inputs[1])
normal = nodes.new("GeometryNodeInputNormal")
normal.location = (0.0, 500.0)
@@ -1320,6 +1348,80 @@ def build_moge_depth_plane_group():
return group
def build_shadeless_group():
group = bpy.data.node_groups.new(SHADELESS_GROUP_NAME, "ShaderNodeTree")
interface = group.interface
new_socket(interface, "Shader", "OUTPUT", "NodeSocketShader")
new_socket(interface, "Color", "INPUT", "NodeSocketColor")
nodes = group.nodes
links = group.links
group_input = nodes.new("NodeGroupInput")
group_input.location = (-900.0, -350.0)
group_output = nodes.new("NodeGroupOutput")
group_output.location = (850.0, 100.0)
diffuse = nodes.new("ShaderNodeBsdfDiffuse")
diffuse.location = (250.0, -150.0)
emission = nodes.new("ShaderNodeEmission")
emission.location = (250.0, -400.0)
links.new(group_input.outputs["Color"], diffuse.inputs["Color"])
links.new(group_input.outputs["Color"], emission.inputs["Color"])
light_path = nodes.new("ShaderNodeLightPath")
light_path.location = (-900.0, 150.0)
bounce_count = nodes.new("ShaderNodeMath")
bounce_count.operation = "SUBTRACT"
bounce_count.label = "Bounce Count"
bounce_count.location = (-500.0, -50.0)
links.new(light_path.outputs["Ray Depth"], bounce_count.inputs[0])
links.new(light_path.outputs["Transmission Depth"], bounce_count.inputs[1])
refracted = nodes.new("ShaderNodeMath")
refracted.operation = "SUBTRACT"
refracted.label = "Camera or Refracted"
refracted.inputs[0].default_value = 1.0
refracted.location = (-250.0, -50.0)
links.new(bounce_count.outputs[0], refracted.inputs[1])
reflection_limit = nodes.new("ShaderNodeMath")
reflection_limit.operation = "SUBTRACT"
reflection_limit.label = "Limit Reflections"
reflection_limit.inputs[0].default_value = 2.0
reflection_limit.location = (-650.0, 450.0)
links.new(light_path.outputs["Ray Depth"], reflection_limit.inputs[1])
camera_reflected = nodes.new("ShaderNodeMath")
camera_reflected.operation = "MULTIPLY"
camera_reflected.label = "Camera Ray to Glossy"
camera_reflected.location = (-400.0, 400.0)
links.new(reflection_limit.outputs[0], camera_reflected.inputs[0])
links.new(light_path.outputs["Is Glossy Ray"], camera_reflected.inputs[1])
shadow_or_reflect = nodes.new("ShaderNodeMath")
shadow_or_reflect.operation = "MAXIMUM"
shadow_or_reflect.label = "Shadow or Reflection?"
shadow_or_reflect.location = (-100.0, 350.0)
links.new(camera_reflected.outputs[0], shadow_or_reflect.inputs[0])
links.new(light_path.outputs["Is Shadow Ray"], shadow_or_reflect.inputs[1])
shader_factor = nodes.new("ShaderNodeMath")
shader_factor.operation = "MAXIMUM"
shader_factor.label = "Shadow, Reflect or Refract?"
shader_factor.location = (150.0, 300.0)
links.new(shadow_or_reflect.outputs[0], shader_factor.inputs[0])
links.new(refracted.outputs[0], shader_factor.inputs[1])
mix = nodes.new("ShaderNodeMixShader")
mix.location = (550.0, 100.0)
links.new(shader_factor.outputs[0], mix.inputs["Fac"])
links.new(diffuse.outputs["BSDF"], mix.inputs[1])
links.new(emission.outputs["Emission"], mix.inputs[2])
links.new(mix.outputs["Shader"], group_output.inputs["Shader"])
group.color_tag = "SHADER"
group.description = (
"Display color without scene lighting while preserving shadows "
"and reflections."
)
group.use_fake_user = True
return group
def main():
bpy.ops.wm.read_factory_settings(use_empty=True)
bpy.context.preferences.filepaths.save_version = 0
@@ -1336,6 +1438,7 @@ def main():
configure_moge_surface_group(moge_surface_group)
build_depth_plane_group()
build_moge_depth_plane_group()
build_shadeless_group()
OUTPUT_PATH.parent.mkdir(parents=True, exist_ok=True)
bpy.ops.wm.save_as_mainfile(filepath=str(OUTPUT_PATH), check_existing=False)

View File

@@ -13,6 +13,8 @@ sys.path.insert(
from snapform.blender.depth_plane import (
DEPTH_PLANE_MATERIAL_NODE_GROUP_NAME,
MATERIAL_COLOR_SPACE,
SHADELESS_NODE_GROUP_NAME,
SURFACE_MATERIAL_NODE_GROUP_NAME,
add_depth_plane_processing,
create_material,
@@ -133,7 +135,6 @@ def main():
foreground,
1.0,
displacement_image=depth,
foreground_image=foreground,
)
mesh.materials.append(material)
material_group = bpy.data.node_groups[DEPTH_PLANE_MATERIAL_NODE_GROUP_NAME]
@@ -232,6 +233,47 @@ def main():
for node in material.node_tree.nodes
if node.bl_idname == "ShaderNodeTexImage"
)
shadeless_material = create_material(
"Shadeless",
foreground,
1.0,
shadeless=True,
)
shadeless_group = bpy.data.node_groups[SHADELESS_NODE_GROUP_NAME]
shadeless_nodes = [
node
for node in shadeless_material.node_tree.nodes
if node.bl_idname == "ShaderNodeGroup"
and node.node_tree == shadeless_group
]
assert len(shadeless_nodes) == 1
assert not any(
node.bl_idname == "ShaderNodeEmission"
for node in shadeless_material.node_tree.nodes
)
assert [
item.name
for item in shadeless_group.interface.items_tree
if item.item_type == "SOCKET" and item.in_out == "INPUT"
] == ["Color"]
assert [
item.name
for item in shadeless_group.interface.items_tree
if item.item_type == "SOCKET" and item.in_out == "OUTPUT"
] == ["Shader"]
assert {
node.label: node.operation
for node in shadeless_group.nodes
if node.bl_idname == "ShaderNodeMath"
} == {
"Bounce Count": "SUBTRACT",
"Camera or Refracted": "SUBTRACT",
"Limit Reflections": "SUBTRACT",
"Camera Ray to Glossy": "MULTIPLY",
"Shadow or Reflection?": "MAXIMUM",
"Shadow, Reflect or Refract?": "MAXIMUM",
}
assert foreground.colorspace_settings.name == MATERIAL_COLOR_SPACE
surface_material = create_material("Surface", foreground, 1.0)
surface_group = bpy.data.node_groups[SURFACE_MATERIAL_NODE_GROUP_NAME]
surface_group_nodes = [

View File

@@ -1,6 +1,8 @@
import sys
import tempfile
from pathlib import Path
from types import SimpleNamespace
from unittest.mock import patch
import bpy
import numpy as np
@@ -13,13 +15,81 @@ sys.path.insert(
str(next((PROJECT_ROOT / "src" / "snapform" / "wheels").glob("blendjob-*.whl"))),
)
import snapform.blender.depth_plane as depth_plane
from snapform.blender.depth_plane import (
MOGE_DEPTH_PLANE_NODE_GROUP_NAME,
MOGE_MATERIAL_NODE_GROUP_NAME,
build_moge2_depth_plane,
create_color_image,
get_modifier_input,
modifier_input_identifier,
)
def validate_single_frame_source_types():
with tempfile.TemporaryDirectory() as temporary:
directory = Path(temporary)
color_path = directory / "color.0001.png"
seed = bpy.data.images.new("Color Seed", width=2, height=2, alpha=True)
seed.pixels[:] = [0.2, 0.4, 0.8, 1.0] * 4
seed.filepath_raw = str(color_path)
seed.file_format = "PNG"
seed.save()
bpy.data.images.remove(seed, do_unlink=True)
still_source_image = bpy.data.images.load(
str(color_path), check_existing=False
)
still_source = bpy.data.objects.new("Still Source", None)
still_source.empty_display_type = "IMAGE"
still_source.data = still_source_image
bpy.context.collection.objects.link(still_source)
still_color = create_color_image(
still_source, directory, assign_to_source=False
)
assert still_color.source == "FILE"
animated_source = SimpleNamespace(
data=SimpleNamespace(
source="SEQUENCE",
name="Sequence Source.png",
use_fake_user=False,
),
image_user=None,
)
sequence_color = create_color_image(
animated_source, directory, assign_to_source=False
)
assert sequence_color.source == "SEQUENCE"
depth = np.ones((1, 2, 2), dtype=np.float32)
mask = np.ones((1, 2, 2), dtype=bool)
intrinsics = np.array(
[[[1.0, 0.0, 0.5], [0.0, 1.0, 0.5], [0.0, 0.0, 1.0]]],
dtype=np.float32,
)
path = directory / "single-frame-results.npz"
np.savez(path, depth=depth, mask=mask, intrinsics=intrinsics)
with patch.object(depth_plane, "is_animated_image", return_value=True):
plane = build_moge2_depth_plane(
bpy.context,
still_source,
path,
delete_source=False,
color_image=sequence_color,
data_frames_directory=directory,
)
textures = [
node.image
for node in plane.data.materials[0].node_tree.nodes
if node.bl_idname == "ShaderNodeTexImage"
]
assert sequence_color in textures
depth_images = [image for image in textures if image != sequence_color]
assert len(depth_images) == 1
assert depth_images[0].source == "SEQUENCE"
def main():
bpy.ops.wm.read_factory_settings(use_empty=True)
image = bpy.data.images.new("MoGe Test", width=64, height=48, alpha=True)
@@ -45,11 +115,14 @@ def main():
mask=mask,
intrinsics=intrinsics,
)
material_image = image.copy()
material_image.name = "MoGe Test.color"
plane = build_moge2_depth_plane(
bpy.context,
source,
path,
delete_source=True,
color_image=material_image,
)
assert plane.modifiers[0].node_group.name == MOGE_DEPTH_PLANE_NODE_GROUP_NAME
assert MOGE_MATERIAL_NODE_GROUP_NAME in bpy.data.node_groups
@@ -73,6 +146,9 @@ def main():
assert sockets["Background Depth"].structure_type == "SINGLE"
assert sockets["Background Depth"].subtype == "DISTANCE"
assert sockets["Background Depth"].default_value == 1.0
assert sockets["Color Image"].hide_in_modifier
color_identifier = modifier_input_identifier(geometry_group, "Color Image")
assert get_modifier_input(plane.modifiers[0], color_identifier) == material_image
material_group = bpy.data.node_groups[MOGE_MATERIAL_NODE_GROUP_NAME]
assert not any(node.bl_idname == "ShaderNodeNormalMap" for node in material_group.nodes)
assert any(node.bl_idname == "ShaderNodeBump" for node in material_group.nodes)
@@ -84,14 +160,90 @@ def main():
]
assert image_textures
assert all(node.extension == "EXTEND" for node in image_textures)
assert any(node.image.name == "MoGe Test.depth" for node in image_textures)
static_depth = next(
node.image
for node in image_textures
if node.image.name == "MoGe Test.depth"
)
assert static_depth.source != "SEQUENCE"
geometry_textures = [
node
for node in geometry_group.nodes
if node.bl_idname == "GeometryNodeImageTexture"
]
assert geometry_textures
assert len(geometry_textures) == 2
assert all(node.extension == "EXTEND" for node in geometry_textures)
color_texture = next(
node
for node in geometry_textures
if any(
link.to_node == node
and link.to_socket == node.inputs["Image"]
and link.from_socket.name == "Color Image"
for link in geometry_group.links
)
)
depth_texture = next(node for node in geometry_textures if node != color_texture)
assert color_texture.interpolation == "Closest"
assert depth_texture.interpolation == "Linear"
color_frame_link = next(
link
for link in geometry_group.links
if link.to_node == color_texture
and link.to_socket == color_texture.inputs["Frame"]
)
depth_frame_link = next(
link
for link in geometry_group.links
if link.to_node == depth_texture
and link.to_socket == depth_texture.inputs["Frame"]
)
assert color_frame_link.from_socket == depth_frame_link.from_socket
alpha_compare = next(
node
for node in geometry_group.nodes
if node.bl_idname == "FunctionNodeCompare"
and node.operation == "GREATER_THAN"
and any(
link.from_node == color_texture
and link.from_socket == color_texture.outputs["Alpha"]
and link.to_node == node
for link in geometry_group.links
)
)
masked_depth = next(
node
for node in geometry_group.nodes
if node.bl_idname == "ShaderNodeMix"
and any(
link.from_node == alpha_compare and link.to_node == node
for link in geometry_group.links
)
)
assert masked_depth.data_type == "FLOAT"
masked_sources = {
link.from_node
for link in geometry_group.links
if link.to_node == masked_depth
}
assert alpha_compare in masked_sources
assert any(
node.bl_idname == "ShaderNodeMath" and node.operation == "MAXIMUM"
for node in masked_sources
)
assert any(
node.bl_idname == "ShaderNodeMath"
and node.operation == "MULTIPLY"
and node.inputs[1].default_value == -1.0
for node in masked_sources
)
assert any(
link.from_node == masked_depth
and link.to_node.bl_idname == "ShaderNodeMath"
and link.to_node.operation == "MULTIPLY"
for link in geometry_group.links
)
validate_single_frame_source_types()
print("MoGe-2 depth plane validation passed")

View File

@@ -62,7 +62,7 @@ def main():
)
image = bpy.data.images.new("MoGe Surface Test", width=width, height=height)
foreground = bpy.data.images.new(
"MoGe Surface Test.foreground",
"MoGe Surface Test.color",
width=width,
height=height,
)
@@ -197,7 +197,8 @@ def main():
node for node in surface.data.materials[0].node_tree.nodes
if node.bl_idname == "ShaderNodeTexImage" and node.image == foreground
)
assert color_texture.image.name == "MoGe Surface Test.foreground"
assert color_texture.image.name == "MoGe Surface Test.color"
assert color_texture.image.colorspace_settings.name == "AgX Base sRGB"
normal_texture = next(
node for node in surface.data.materials[0].node_tree.nodes
if node.bl_idname == "ShaderNodeTexImage"
@@ -220,6 +221,8 @@ def validate_real_result(results_path, image_path):
normal_path = results_path.with_name("normal.png")
if not normal_path.is_file():
raise ValueError(f"Surface normal texture is missing: {normal_path}")
color_image = bpy.data.images.load(str(image_path), check_existing=False)
color_image.name = f"{image_path.stem}.color"
surface = build_image_mesh(
bpy.context,
source,
@@ -228,6 +231,7 @@ def validate_real_result(results_path, image_path):
True,
False,
True,
color_image=color_image,
normal_path=normal_path,
)
assert len(surface.data.vertices) > 100