Refactor image handling and update foreground processing
- Changed the logic in `remove_background_faces` to keep only faces with all foreground vertices. - Updated background job to save foreground images with a new naming convention (`color.0001.png`). - Replaced `copy_frames` with `copy_color_images` in various jobs to ensure consistent image naming. - Introduced a new function `copy_color_images` to handle the copying of images with a specific naming format. - Updated the BEN2 model to save output images with a new prefix for better clarity. - Modified tests to reflect changes in image naming and processing logic. - Added a new shadeless node group for improved material handling in geometry nodes. - Enhanced validation scripts to check for new image naming conventions and ensure proper functionality.
This commit is contained in:
16
README.md
16
README.md
@@ -40,9 +40,9 @@ Environment 未安装时,同一位置显示两个安装按钮。Environment
|
||||
- `Depth from`:`Original` 直接用原图,`DA3` 使用现有 DA3 流程,`MoGe-2` 生成 metric depth、normal 与有效区域 mask
|
||||
- `Resolution Scale` 使用 `1–100%`,按原图长边计算 Process Res;结果会限制在 `252–1008` 并自动对齐到 14 的倍数
|
||||
- `Remove Background` 默认关闭;开启后在转换期间使用 BEN2 删除背景
|
||||
- `Unshaded` 保留为直接显示贴图颜色的选项
|
||||
- `Shadeless` 使用 `O Shadeless` 节点组显示不受场景灯光影响的贴图颜色
|
||||
- `Keep Original` 默认关闭,位于分割线下方;开启后保留源 Image Empty
|
||||
4. 点击确认,稍等片刻就得到带位移的立体面片
|
||||
4. 点击确认,稍等片刻就得到带位移的立体面片。`O DA3 Depth Plane` 与 `O MoGe2 Depth Plane` 都使用 `.color` 的 `Alpha > 0.5` 划分前景范围,范围外由 `Background Depth` 控制。
|
||||
|
||||
### 去除背景
|
||||
|
||||
@@ -57,8 +57,8 @@ ViT-S、ViT-B、ViT-L 或 DA3 Small、Base,其中 MoGe-2 ViT-S 为默认模型
|
||||
Surface 输出流程构建 DA3 与 MoGe-2 源网格:MoGe-2 使用深度断层 mask,DA3 保留 confidence 与边拉伸
|
||||
数据。两者再由独立的 Geometry Nodes
|
||||
修改器完成剔面、快速重拓扑、Relax 与 UV 重投;顶层 `Iterations` 可设为 `0`。开启 Remove Background
|
||||
时,BEN2 Alpha 只要大于 `0` 就视为前景;三角面有任一前景点即保留,只有全背景面会删除并且不再交给
|
||||
Geometry Nodes;Surface 材质使用 `.foreground`。匹配相机默认不创建;开启后会作为 Surface 的子级。
|
||||
时,BEN2 Alpha 只要大于 `0` 就视为前景;三角形三个顶点必须全部为前景,任一顶点 Alpha 为 `0`
|
||||
都会删除该面并且不再交给 Geometry Nodes;Surface 材质使用 `.color`。匹配相机默认不创建;开启后会作为 Surface 的子级。
|
||||
MoGe-2 的像素级 normal 会作为 Object Space normal 贴图接入 `O MoGe2 Surface Layer`,并与原图
|
||||
Bump 合并;网格本身不写入 custom normal,因此重拓扑降低顶点数后仍能保留细腻的法线细节。
|
||||
当前此功能仅支持单张图片。
|
||||
@@ -67,20 +67,20 @@ MoGe-2 ViT-S、ViT-B、ViT-L、DA3 Small、DA3 Base 与 BEN2 全部使用 ONNX
|
||||
点云重投影和网格生成均不依赖 PyTorch。Windows 使用 DirectML 加速,provider 不可用时会回退到 ONNX Runtime CPU,
|
||||
并在任务状态中显示实际 provider。
|
||||
|
||||
转换对话框提供模型与 `Create Matching Camera`、`Remove Background`、`Unshaded`;DA3 显示
|
||||
转换对话框提供模型与 `Create Matching Camera`、`Remove Background`、`Shadeless`;DA3 显示
|
||||
`Resolution Scale`,MoGe-2 显示 `Resolution Level`,
|
||||
以及分割线下方的 `Keep Original`。
|
||||
界面会显示计算所得的实际 Process Res。剔除和重拓扑参数直接在
|
||||
Geometry Nodes 修改器中调整;`Unshaded` 可使用与 Depth Plane 相同的 Emission 材质。普通材质的
|
||||
Geometry Nodes 修改器中调整;`Shadeless` 与 Depth Plane 共用 `O Shadeless` 材质节点组。颜色贴图始终使用统一的 `.color` 转换产物;Remove Background 只决定是否由 BEN2 修改 Alpha,并使用 `AgX Base sRGB`。普通材质的
|
||||
Bump Scale 会在材质组内乘以 `0.001`,并按透视表面的深度空间缩放。Surface 与 Depth Plane 的
|
||||
Layer Color 同时驱动 Principled BSDF 的 Base Color、Subsurface Radius 和 Emission Color,Layer Normal
|
||||
同时驱动 Normal 与 Coat Normal;Subsurface、Emission、Coat 的其他设置保持默认。Surface attributes
|
||||
统一使用 `o_` 前缀;Server 准备过程显示
|
||||
`Preparing surface data n/6`,可看到当前阶段与总步骤。
|
||||
`Preparing surface data n/6`,可看到当前阶段与总步骤;Surface 单帧模型推理不显示多余的 `1/1`。
|
||||
|
||||
### 视频与序列帧
|
||||
|
||||
SnapForm 支持视频(mp4、mkv、mov 等)和序列帧,会逐帧处理并生成逐帧对应的立体面片。处理范围默认
|
||||
SnapForm 支持视频(mp4、mkv、mov 等)和序列帧,会逐帧处理并生成逐帧对应的立体面片。`.color` 与 `.depth` 按源 Empty Image 的类型载入:普通单图为 `IMAGE`,Movie/Sequence 即使只处理一帧也仍为 `SEQUENCE`。处理范围默认
|
||||
跟随 Empty 上设置的 `Image Frames` 与 `Offset`,不需要额外配置。DA3 的转换任务复用同一个 ONNX
|
||||
Session 逐帧推理,显存峰值不随视频帧数平方增长。Debug 只保存模型 API 的直接返回:DA3 多帧输入
|
||||
保留联合推理,MoGe-2 与 BEN2 按各自的单图 API 逐帧保存。
|
||||
|
||||
@@ -15,11 +15,11 @@
|
||||
- `Original`:直接把原图作为深度贴图,不显示估算参数。仅在开启 `Remove Background` 时运行 BEN2
|
||||
取得前景 Alpha;视频或序列帧按需同时抽取原始帧与前景序列,把原始帧作为 `.depth` 序列。
|
||||
`MoGe-2 ViT-S` 是默认深度来源。选择 `DA3` 时,其 Model 与 Resolution Scale
|
||||
参数会集中显示在独立边框中。`Remove Background` 与 `Unshaded` 直接显示,不再使用 Options 折叠;
|
||||
参数会集中显示在独立边框中。`Remove Background` 与 `Shadeless` 直接显示,不再使用 Options 折叠;
|
||||
`Keep Original` 默认关闭,并在分割线下方保持为最后一项。仅当开启 `Remove Background` 时才需要
|
||||
`BEN2 Base`,并把 `.foreground` 的 Alpha 作为独立
|
||||
Mask,不在 Server 中改写深度值。保存 `.foreground` 时保留 BEN2 Alpha,但 RGB 使用未经修改的原始输入;
|
||||
因此材质直接用 `.foreground` 同时提供颜色和 Mask,不再额外加载原图 Image Texture。
|
||||
`BEN2 Base`,并把 `.color` 的 Alpha 作为独立 Mask,不在 Server 中改写深度值。无论颜色层是直接
|
||||
复制原图,还是由 BEN2 为单图或序列修改 Alpha,Server 都统一写为 `color.0001.*`;Blender 只导入
|
||||
这一份 `.color`,不复用源 Empty 的 Image,Color Space 固定为 `AgX Base sRGB`。
|
||||
Image Empty 引用视频或序列帧时,DA3 与 MoGe-2 会按 Empty 的 Image Frames 数值逐帧估计
|
||||
深度(未设置时为完整帧数,显式设为 1 时只处理 1 帧),起始帧跟随 Empty 的 Offset,与背景蒙版范围一致;
|
||||
DA3 会复用同一个 ONNX Session 对每帧独立推理,再合并为原有序列输出,避免多帧 Attention 的平方级显存占用。
|
||||
@@ -27,10 +27,10 @@
|
||||
该行为只用于 Depth Plane 转换;Surface 仅支持单张图片。Debug 中 DA3 多帧输入联合推理,
|
||||
MoGe-2 与 BEN2 按单图 API 逐帧运行。
|
||||
Offset 超出视频长度时同样把窗口收进视频内。材质节点与置换贴图的 Start 保持与原 Empty
|
||||
一致;Original 模式的视频或序列帧使用 `frame.0001.jpg` 等原图序列作为颜色与深度,
|
||||
`foreground.0001.png` 只提供前景 Alpha。
|
||||
一致;Original 模式的视频或序列帧使用 `color.0001.*` 颜色层同时作为颜色与深度输入;开启
|
||||
Remove Background 时同名颜色层包含 BEN2 修改后的 Alpha。
|
||||
DA3 的深度仍以 EXR 序列帧(`depth.0001.exr` 起)保存在 job 目录根下;MoGe-2 分别使用 `moge_depth`、`moge_mask` 与 `moge_normal` EXR 序列。
|
||||
`Unshaded`、`Remove Background` 与 `Keep Original` 均默认关闭,因此默认删除原对象。
|
||||
`Shadeless`、`Remove Background` 与 `Keep Original` 均默认关闭,因此默认删除原对象。`Shadeless` 会以 `O Shadeless` 节点组替代普通 Principled BSDF 路径,并继续通过 Alpha 与 Transparent BSDF 混合。
|
||||
3. DA3 或 MoGe-2 完成后会自动创建面片。DA3 面片几何先由 `mini_npz` 的相机内参与深度确定,再通过对象
|
||||
Transform 匹配原 Image Empty 的可见尺寸、位置、旋转和 Offset。Original 模式直接使用源 Empty
|
||||
的可见尺寸、Offset 与 Transform。DA3 会把用于匹配画面尺寸的缩放预乘到 `.depth` 数值,
|
||||
@@ -40,9 +40,10 @@
|
||||
同名冲突,由 Blender 自动为生成面片追加编号。
|
||||
4. Blender 只创建保留图像纵横比的最低密度基础网格,再把 `O DA3 Depth Plane` 作为 Geometry Nodes
|
||||
修改器添加到对象。节点组与 `O DA3 Surface` 一同保存在 `O_SnapForm.blend`,关闭
|
||||
`show_group_selector`,不为每个对象复制节点组。MoGe-2 使用单独的 `O MoGe2 Depth Plane`,只保留
|
||||
网格细分、metric depth 位移、mask 删点和材质设置;材质使用 `O MoGe2 Depth Plane Layer`,直接接入
|
||||
模型 normal,不携带 DA3 的材质置换与背景深度分支。DA3 修改器公开以下参数:
|
||||
`show_group_selector`,不为每个对象复制节点组。MoGe-2 使用单独的 `O MoGe2 Depth Plane`,处理
|
||||
网格细分、metric depth 位移和材质设置;材质使用 `O MoGe2 Depth Plane Layer`,直接接入模型 normal。
|
||||
`O MoGe2 Depth Plane` 与 DA3 一样通过隐藏的 Color Image 读取 `.color` Alpha,以 `Alpha > 0.5`
|
||||
划分前景范围,并在范围外改用 `Background Depth`。两个修改器公开以下参数:
|
||||
- `Displacement`:位于 Geometry 之后的第一项,菜单包含 `Geometry`(默认)与 `Material`。`Geometry`
|
||||
在节点组内沿法线修改网格,`Material` 保持网格平面并启用材质置换;
|
||||
- `Subdivision`:位于第二项,默认 `5`。节点读取基础平面的 Bounding Box,短边分段数为 `2^Subdivision`,
|
||||
@@ -51,16 +52,16 @@
|
||||
材质置换使用该值;是否存在背景只取决于转换所得色彩贴图的 `Alpha > 0.5` 二值结果;
|
||||
- `Depth Scale`:默认 `1`,允许正负值,Structure Type 为 `Single`,同时控制 Geometry 与 Material
|
||||
两条置换路径。
|
||||
Grid 重建后通过隐藏的 Material 输入重新赋予原图材质,并对所有面执行 Shade Smooth。
|
||||
Grid 重建后通过隐藏的 Material 输入重新赋予转换材质,并对所有面执行 Shade Smooth。
|
||||
材质使用显式指向 `UVMap` 的 UV Map 节点连接颜色、前景 Mask 与深度 Image Texture,不依赖活动 UV。
|
||||
前景深度先与 `Background Depth` 取 Maximum,再按色彩贴图 Alpha 与背景深度混合,
|
||||
深度分支在混合前使用 `Alpha > 0.5` 转为二值 Mask,避免半透明前景被背景深度拉扯;
|
||||
材质表面 Alpha 同样使用二值 Mask。材质置换使用相同的二值和钳制顺序。
|
||||
节点组还会把 `o_material_displacement` 与 `o_background_depth` 写入点域,材质通过
|
||||
`Geometry` 类型的 Attribute 节点读取。点域会为顶点保存重复值,但能在应用修改器后继续保留参数;
|
||||
相比自适应网格本身,其占用很小。Depth Image 与 Foreground Image 是修改器隐藏数据输入,
|
||||
相比自适应网格本身,其占用很小。Depth Image 与 Color Image 是修改器隐藏数据输入,
|
||||
由导入流程自动设置。
|
||||
视频与序列帧还会为 Depth 与 Foreground 共同写入隐藏的 `Frame`、`Frame Start`、`Frames` 与 `Cyclic`。
|
||||
`.color` 与 `.depth` 的 Blender Image 类型只依据源 Empty Image:普通单图使用 `IMAGE`,Movie 或 Sequence 使用 `SEQUENCE`;即使动画源这次只处理一帧也仍为 `SEQUENCE`,不按输出文件数量推断。视频与序列帧还会为 Depth 与 Color 共同写入隐藏的 `Frame`、`Frame Start`、`Frames` 与 `Cyclic`。
|
||||
节点组内部不再使用 Scene Time。只有原 Image Empty 是视频或序列时,导入流程才直接给修改器的 `Frame`
|
||||
socket 添加 Scripted Expression Driver `frame`;静态图保持默认值 `1`,不创建 Driver。导入内容已经
|
||||
是截取后的帧,因此 Offset 在节点组内恒定为 `0`。
|
||||
|
||||
@@ -89,18 +89,18 @@
|
||||
2. 在 3D View 选中 Image Empty,右键打开 `SnapForm` 子菜单并选择 `Remove Background`。
|
||||
3. 完成后以 Fake User 保留原 Image datablock,新建并 Pack 一张透明 PNG 到当前 `.blend`,然后只把
|
||||
当前 Empty 的 `data` 换成新图;Empty 的 Transform、显示尺寸、Offset 及其他设置保持不变。
|
||||
4. 新 Image 使用原图去掉格式后再追加 `.foreground` 的名称。例如 `img.png` 生成
|
||||
`img.foreground`,`img.png.001` 生成 `img.001.foreground`。已有同名 datablock 时由
|
||||
4. 新 Image 使用原图去掉格式后再追加 `.color` 的名称。例如 `img.png` 生成
|
||||
`img.color`,`img.png.001` 生成 `img.001.color`。已有同名 datablock 时由
|
||||
Blender 自动追加编号。
|
||||
|
||||
原图已 Pack 时会先写出临时 PNG 交给 BEN2,任务结束后删除临时输入;原 Image 仍保持 Pack
|
||||
状态和原始内容。BEN2 的 `foreground.png` 也会保留在对应 job 目录中。
|
||||
状态和原始内容。正式任务的 `color.0001.png` 也会保留在对应 job 目录中。
|
||||
|
||||
Image Empty 引用视频或序列帧时,Remove Background 直接执行,默认取 Empty 的 Image Frames
|
||||
数值(未设置时为完整帧数,显式设为 1 时只处理 1 帧),起始帧跟随 Empty 的 Offset;若 Offset 加 Frames 超出视频长度,
|
||||
会把窗口整体收进视频内(保证实际处理 Frames 帧数),Offset 过小时从第 1 帧开始。输出
|
||||
`foreground.0001.png` 等透明序列帧(帧号从 0001 开始,与 Blender 场景帧一一对应),并把
|
||||
源帧以 `frame.0001.jpg` 等命名复制到 job 目录根下供材质颜色导入。替换 Empty 的 Image 为
|
||||
加载这些帧的 SEQUENCE Image,从原 Empty 的 Start 位置开始播放(Start 原样保留),
|
||||
`color.0001.png` 等透明序列帧(帧号从 0001 开始,与 Blender 场景帧一一对应),并把
|
||||
关闭去背景时,原始颜色帧同样以 `color.0001.*` 起始的统一名称写入 job 目录。替换 Empty 的 Image 为
|
||||
加载这些帧的 SEQUENCE Image,从原 Empty 的 Start 位置开始播放(Start 原样保留)。图片类型只取决于源 Empty:普通单图的单个 `color.0001.*` 仍作为 `IMAGE`,Movie/Sequence 即使只导出一帧也作为 `SEQUENCE`;`.depth` 同样遵循该规则。
|
||||
Cyclic 沿用原 Empty 的设置。序列帧无法被 Blender 整体 Pack,因此保持外部引用 job 目录;
|
||||
请保留对应 job 目录,否则序列会失效。
|
||||
|
||||
@@ -22,11 +22,13 @@ Depth Plane 导入所需的贴图。这里的文件转换属于正式 Depth 工
|
||||
|
||||
## Surface
|
||||
|
||||
generate-surface 统一写出 mesh.npz 与 texture.png。DA3 与 MoGe-2 都使用
|
||||
geometry/surface.py 的同一套 Surface 构建算法;序列化位于 outputs/surface.py。
|
||||
MoGe-2 normal 另存为 normal.png 并由 Blender 材质加载,不写入几何 NPZ。
|
||||
generate-surface 统一写出 `mesh.npz`,不再生成 `texture.png`。DA3 与 MoGe-2 都使用
|
||||
`geometry/surface.py` 的同一套 Surface 构建算法;序列化位于 `outputs/surface.py`。MoGe-2 normal
|
||||
仍由 Server 单独导出为 `normal.png`,Blender 只负责加载并用于材质,不写入几何 NPZ。未去背景时
|
||||
无论颜色来自原图复制还是 BEN2,Server 都统一写为 `color.0001.*` 起始的颜色层。Blender 不依据文件数量判断图片类型:源 Empty 为普通单图时载入为 `IMAGE`,源 Empty 为 Movie/Sequence 时即使只有一帧也载入为 `SEQUENCE`;`.depth` 使用相同规则。材质只在 Blender
|
||||
端创建。MoGe-2 的 `normal.png` 仍由 Server 生成,Blender 只负责加载使用。
|
||||
|
||||
## BEN2
|
||||
|
||||
正式 remove-background 工作流输出 foreground.png 或透明前景序列。BEN2 ONNX API 返回的 Alpha
|
||||
恢复到输入分辨率,RGB 保留输入图像颜色。
|
||||
正式 remove-background 工作流输出 `color.0001.png` 起始的透明颜色层。BEN2 ONNX API 返回的 Alpha
|
||||
恢复到输入分辨率,RGB 保留输入图像颜色。Debug BEN2 仍保留模型 API 语义,使用 `foreground` 文件名。
|
||||
|
||||
@@ -9,7 +9,7 @@
|
||||
Resolution Scale 与 Depth Plane 使用相同设计;MoGe-2 改为显示共享的 Resolution Level。
|
||||
2. Server 先用未经抠图的原始 RGB 请求 DA3 的 `depth`、`conf` 与 `intrinsics`,随后仅在开启
|
||||
`Remove Background` 时运行 BEN2 取得前景 Alpha;Alpha 经目标网格分辨率缩放与轻微模糊后,只要
|
||||
大于 `0` 就把对应点视为前景;每个三角面只要有一个前景点就保留,三个点均为背景时才在输出前删除;
|
||||
大于 `0` 就把对应点视为前景;每个三角形的三个顶点必须全部为前景,任一顶点 Alpha 为 `0` 就在输出前删除该面;
|
||||
关闭时不加载 BEN2。Depth 不做 smoothing,Server 反投影深度网格,建立源三角面,
|
||||
计算 confidence percentile 与 3×3 相对深度不连续度,并写入
|
||||
基于原图像素坐标的源 `UVMap`。这些步骤只准备修改器的输入网格,不执行 Relax、拓扑修复、平滑、
|
||||
@@ -44,11 +44,13 @@
|
||||
Group Input,并隐藏没有连线的接口,避免跨越整个节点图的输入连线。
|
||||
`O DA3 Surface` 与 `O MoGe2 Surface` 的 `Iterations` 默认值均为 `2`、最小值为 `0`,
|
||||
`Resolution` 默认值均为 `0.1`,所有输入 socket 的 Structure Type 均固定为 `Single`。
|
||||
5. Surface Job 统一按输入准备、模型推理、可选 BEN2、构建 SurfaceMesh、写出产物五个阶段更新进度;
|
||||
5. Surface Job 统一按输入准备、模型推理、可选 BEN2、构建 SurfaceMesh、写出产物五个阶段更新进度;单帧推理只显示模型与阶段名称,不追加 `1/1`,仅多帧时显示当前帧与总数;
|
||||
Surface Builder 是不接收 Job Context 或 progress callback 的纯数据转换。
|
||||
6. Server 在 job 根目录写出 `mesh.npz` 与 `texture.png`。NPZ 保存 camera-space `vertices`、
|
||||
`faces`、`uv`、`intrinsics`、`image_size` 与有效深度中位数 `reference_depth`。Blender 读取这些
|
||||
数据后直接创建 Surface;材质默认使用原图,开启 Remove Background 时改用 `.foreground`,不需要 Open3D 或 OpenCV 进入 Blender Python 环境。
|
||||
6. Server 在 job 根目录写出 `mesh.npz`。NPZ 保存 camera-space `vertices`、`faces`、`uv`、
|
||||
`intrinsics`、`image_size` 与有效深度中位数 `reference_depth`。颜色层无论来自原图复制还是 BEN2,
|
||||
Server 都统一写为 `color.0001.*`。Blender 读取网格数据后额外导入为 `.color`,材质不复用源
|
||||
Empty 的 Image,Color Space 固定为 `AgX Base sRGB`。Server 不生成 `texture.png`,材质只在
|
||||
Blender 端创建。
|
||||
DA3 Surface 只保留 Geometry Nodes 后处理实际使用的 attributes:点域 `o_depth`、面域
|
||||
`o_confidence` 与面域 `o_depth_discontinuity`。Remove Background 已在 Server
|
||||
建面前直接删除背景三角面,不写入任何 background attribute。MoGe-2 的导入网格保留点域 `o_depth`,
|
||||
@@ -62,11 +64,11 @@
|
||||
缩放及父级关系产生的世界变换保持一致。更近的顶点沿 Empty 局部 `+Z` 方向凸出,三角面正面朝向
|
||||
观察相机,启用 Backface Culling 时可从正面看到。
|
||||
8. 转换对话框不再提供后处理参数,只保留模型与 Resolution Scale 设置、`Create Matching Camera`、
|
||||
`Remove Background`、`Unshaded`,以及分割线下方的 `Keep Original`。背景移除默认关闭;开启后 Server
|
||||
`Remove Background`、`Shadeless`,以及分割线下方的 `Keep Original`。背景移除默认关闭;开启后 Server
|
||||
使用 BEN2,并在写出 Surface 前直接删除背景三角面,不再由 Geometry Nodes 的 Background 开关处理。
|
||||
该预删除同时适用于 DA3 与 MoGe-2;因此 MoGe-2 转换后即使关闭 `Mask`,BEN2 已删除的背景也不会恢复。
|
||||
开启 Remove Background 时,Surface 材质的 Color Image Texture 使用打包后的 `.foreground`,原 Empty 仍保持原图。
|
||||
`Unshaded` 使用与 Depth Plane 相同的 Emission/Transparent 材质路径;普通材质的
|
||||
Surface 材质的 Color Image Texture 始终使用 `.color`;Remove Background 只决定该层是否带 BEN2 Alpha,原 Empty 始终保持原图。颜色层使用 `AgX Base sRGB`。
|
||||
`Shadeless` 使用与 Depth Plane 相同的 `O Shadeless` 节点组,并继续通过 Alpha 与 Transparent BSDF 混合;普通材质的
|
||||
Bump Scale 按 `abs(object.scale.z) × camera scale_z` 设置,并在材质组内乘以固定基准 `0.001`,
|
||||
与透视表面实际深度尺度一致;它允许负数以反转凹凸方向。
|
||||
相机默认关闭;开启后会在 Empty 局部 `+Z` 侧创建一台朝向局部 `-Z` 的透视相机,并成为生成 Surface
|
||||
@@ -122,7 +124,7 @@ Displacement 设置为 `Displacement Only`,DA3 与 Original 都使用 `Midleve
|
||||
MoGe-2 Surface 使用独立的 `O MoGe2 Surface Layer`;它额外接收模型输出的 `Normal` 贴图,使用
|
||||
`Normal Map` 节点按 `Object Space` 解码后连接到 Bump 的 `Normal` 输入,使像素级模型法线与原图
|
||||
Bump 合并,不依赖重拓扑后的顶点密度;`Normal Strength` 默认值为 `1.0`,只调节模型 normal map,
|
||||
原图 Bump 仍由独立的 `Bump Scale` 控制;
|
||||
颜色贴图 Bump 仍由独立的 `Bump Scale` 控制;
|
||||
Depth Plane 使用 `O DA3 Depth Plane Layer` 材质组,并把背景剔除、背景深度与材质置换 Attribute
|
||||
读取全部收进组内。两者都接收 `Color`、`Alpha` 与 `Bump Scale`,只有 `O DA3 Depth Plane Layer`
|
||||
额外接收 Float 类型的 `Depth`。
|
||||
@@ -138,9 +140,10 @@ Depth Plane 的 Bump Height 直接使用材质组的 `Color` 输入,只受独
|
||||
`0.001`,会乘以面片 Transform 的统一缩放倍率。原图 Color 还会经过 `Float Curve` 连接到
|
||||
Principled BSDF 的 `Roughness`,曲线两端输出分别为 `0.5` 和 `0.1`;材质 IOR 保持
|
||||
Principled BSDF 默认值 `1.5`。
|
||||
开启 `Unshaded` 后,色彩贴图 Color 连接 Emission,二值 Alpha 作为 Mix Shader 系数,在
|
||||
Transparent BSDF 与 Emission 之间混合后连接 Material Output;不创建 Principled BSDF、Bump 或
|
||||
Float Curve。开启 `Remove Background` 时色彩贴图使用 BEN2 输出;关闭时使用原图。
|
||||
开启 `Shadeless` 后,色彩贴图 Color 接入 `O Shadeless`。该组沿用 Blender Images as Planes 的
|
||||
Diffuse、Emission、Light Path 与射线判断连接;二值 Alpha 仍作为 Mix Shader 系数,在 Transparent BSDF
|
||||
与 `O Shadeless` 之间混合后连接 Material Output。外层不再创建 Emission、Principled BSDF、Bump 或
|
||||
Float Curve。开启 `Remove Background` 时色彩贴图使用 `.color`;关闭时使用 `.color`。
|
||||
|
||||
这里的“原始距离”是 DA3 模型输出的尺度。普通 Small、Base、Large 和 Giant 模型的深度不应
|
||||
被解释为现实世界的米;需要现实尺度时应选择带 metric depth 的
|
||||
@@ -150,7 +153,7 @@ Float Curve。开启 `Remove Background` 时色彩贴图使用 BEN2 输出;关
|
||||
|
||||
生成完成后会直接创建面片,不提供缓存重建入口或单独的导入参数。重新选择
|
||||
`Convert to Depth Plane` 会按当前 Depth from 再次转换。
|
||||
面片、深度 Image、前景 Image、Material 与 Geometry Nodes 修改器会在同一个 Blender Undo 步骤中创建;转换完成后执行
|
||||
面片、深度 Image、颜色 Image、Material 与 Geometry Nodes 修改器会在同一个 Blender Undo 步骤中创建;转换完成后执行
|
||||
`Ctrl+Z` 会完整撤销本次转换,不会越过转换去回退更早的 Image Empty 操作。
|
||||
|
||||
Image Empty 使用 Packed Image 时,SnapForm 会通过 Blender 写出临时 PNG 交给 DA3,
|
||||
@@ -161,7 +164,7 @@ Image Empty 使用 Packed Image 时,SnapForm 会通过 Blender 写出临时 PN
|
||||
生成的 Object、Mesh 与 Material 仍命名为 `obj_img`。Pack 使用 OpenEXR,因而不会把大于 `1.0`
|
||||
的原始深度截断。已有同名 datablock 时由 Blender 自动追加 `.001` 等编号。
|
||||
视频输入时 Blender 无法整体 Pack 序列帧,因此 `.depth` 与颜色序列改为引用 job 目录根下的
|
||||
`depth.0001.exr`、`frame.0001.jpg`(`.frame`)或 `foreground.0001.png`,并在每帧保留原始
|
||||
`depth.0001.exr`、`color.0001.jpg`(`.color`)或 `color.0001.png`,并在每帧保留原始
|
||||
浮点深度;保存 `.blend` 后再次打开仍需对应 job 目录存在。
|
||||
|
||||
每个 Server Job 被接受时,Job Runtime 在 Runtime Storage Root 的 `jobs/` 下创建一个独立文件夹。文件夹名是 32 位 opaque Job ID,不包含任务类型或对象名称。执行逻辑从 `JobContext.directory` 获取路径,输出产物直接写入该目录。
|
||||
@@ -176,14 +179,13 @@ Job 目录中的典型正式产物如下:
|
||||
|
||||
```text
|
||||
<opaque-job-id>/
|
||||
├── results.npz、mesh.npz、texture.png 或 normal.png
|
||||
├── results.npz、mesh.npz 或 normal.png
|
||||
└── prediction.npz Debug 原始 API 返回
|
||||
|
||||
<opaque-job-id>/
|
||||
├── foreground.png 单图结果
|
||||
├── foreground.0001.png… 视频/序列帧结果
|
||||
├── frame.0001.jpg… 视频/序列帧的源帧副本
|
||||
└── depth.0001.exr… AI 深度序列帧
|
||||
├── color.0001.jpg/png… 正式工作流颜色层(原图复制或 BEN2 Alpha)
|
||||
├── depth.0001.exr… AI 深度序列帧
|
||||
└── normal.png MoGe-2 Surface 法线贴图
|
||||
```
|
||||
|
||||
建议从 `DA3 Base` 开始。它使用 Apache 2.0 模型权重。
|
||||
|
||||
Binary file not shown.
@@ -23,6 +23,8 @@ SURFACE_MATERIAL_NODE_GROUP_NAME = "O DA3 Surface Layer"
|
||||
MOGE_SURFACE_MATERIAL_NODE_GROUP_NAME = "O MoGe2 Surface Layer"
|
||||
DEPTH_PLANE_MATERIAL_NODE_GROUP_NAME = "O DA3 Depth Plane Layer"
|
||||
MOGE_MATERIAL_NODE_GROUP_NAME = "O MoGe2 Depth Plane Layer"
|
||||
SHADELESS_NODE_GROUP_NAME = "O Shadeless"
|
||||
MATERIAL_COLOR_SPACE = "AgX Base sRGB"
|
||||
DA3_PATCH_SIZE = 14
|
||||
DA3_MIN_PROCESS_RES = 252
|
||||
DA3_MAX_PROCESS_RES = 1008
|
||||
@@ -183,8 +185,8 @@ def copy_depth_image(source_object):
|
||||
return depth_image
|
||||
|
||||
|
||||
def removed_background_data_name(source_object):
|
||||
return f"{image_base_name(source_object.data)}.foreground"
|
||||
def color_data_name(source_object):
|
||||
return f"{image_base_name(source_object.data)}.color"
|
||||
|
||||
|
||||
def scaled_process_resolution(image, scale_percent):
|
||||
@@ -328,7 +330,7 @@ def configure_image_user(
|
||||
|
||||
def pack_sequence_if_possible(image, expected_frames):
|
||||
"""Pack a frame sequence when Blender supports it; otherwise keep external files."""
|
||||
if expected_frames <= 0:
|
||||
if expected_frames <= 1:
|
||||
return False
|
||||
try:
|
||||
image.pack()
|
||||
@@ -455,81 +457,60 @@ def create_depth_sequence_image(name, depth_frames, scene, frame_directory):
|
||||
return sequence_image
|
||||
|
||||
|
||||
def create_removed_background_image(
|
||||
def create_color_image(
|
||||
source_object,
|
||||
output_path,
|
||||
output_directory,
|
||||
assign_to_source=True,
|
||||
):
|
||||
output_path = Path(output_path)
|
||||
if output_path.is_dir():
|
||||
foreground_paths = sorted_frame_paths(output_path, "foreground.")
|
||||
if not foreground_paths:
|
||||
raise RuntimeError(
|
||||
f"Background removal result has no foreground frames: "
|
||||
f"{output_path}"
|
||||
)
|
||||
try:
|
||||
result_image = bpy.data.images.load(
|
||||
str(foreground_paths[0]),
|
||||
check_existing=False,
|
||||
)
|
||||
result_image.alpha_mode = "CHANNEL_PACKED"
|
||||
force_sequence_source(result_image)
|
||||
except (OSError, RuntimeError) as error:
|
||||
if "result_image" in locals():
|
||||
bpy.data.images.remove(result_image, do_unlink=True)
|
||||
raise RuntimeError("Unable to load the background result") from error
|
||||
source_image = source_object.data
|
||||
original_fake_user = source_image.use_fake_user
|
||||
try:
|
||||
result_image.name = removed_background_data_name(source_object)
|
||||
pack_sequence_if_possible(result_image, len(foreground_paths))
|
||||
if assign_to_source:
|
||||
image_user = getattr(source_object, "image_user", None)
|
||||
(
|
||||
original_frame_start,
|
||||
_original_frame_offset,
|
||||
_original_frame_duration,
|
||||
original_use_cyclic,
|
||||
) = image_user_settings(source_object)
|
||||
source_image.use_fake_user = True
|
||||
source_object.data = result_image
|
||||
configure_image_user(
|
||||
image_user,
|
||||
len(foreground_paths),
|
||||
frame_start=original_frame_start,
|
||||
use_cyclic=original_use_cyclic,
|
||||
)
|
||||
except (OSError, RuntimeError) as error:
|
||||
source_image.use_fake_user = original_fake_user
|
||||
bpy.data.images.remove(result_image)
|
||||
raise RuntimeError("Unable to load the background result") from error
|
||||
return result_image
|
||||
|
||||
if not output_path.is_file():
|
||||
output_directory = Path(output_directory)
|
||||
color_paths = sorted_frame_paths(output_directory, "color.")
|
||||
if not color_paths:
|
||||
raise RuntimeError(
|
||||
f"Background removal result does not exist: {output_path}"
|
||||
f"Color result has no images: {output_directory}"
|
||||
)
|
||||
source_image = source_object.data
|
||||
animated = is_animated_image(source_image)
|
||||
try:
|
||||
result_image = bpy.data.images.load(
|
||||
str(output_path),
|
||||
str(color_paths[0]),
|
||||
check_existing=False,
|
||||
)
|
||||
result_image.alpha_mode = "CHANNEL_PACKED"
|
||||
except RuntimeError as error:
|
||||
raise RuntimeError("Unable to load the background result") from error
|
||||
source_image = source_object.data
|
||||
if animated:
|
||||
force_sequence_source(result_image)
|
||||
except (OSError, RuntimeError) as error:
|
||||
if "result_image" in locals():
|
||||
bpy.data.images.remove(result_image, do_unlink=True)
|
||||
raise RuntimeError("Unable to load the color result") from error
|
||||
|
||||
original_fake_user = source_image.use_fake_user
|
||||
try:
|
||||
result_image.name = removed_background_data_name(source_object)
|
||||
result_image.pack()
|
||||
result_image.name = color_data_name(source_object)
|
||||
if animated:
|
||||
pack_sequence_if_possible(result_image, len(color_paths))
|
||||
else:
|
||||
result_image.pack()
|
||||
if assign_to_source:
|
||||
image_user = getattr(source_object, "image_user", None)
|
||||
(
|
||||
original_frame_start,
|
||||
_original_frame_offset,
|
||||
_original_frame_duration,
|
||||
original_use_cyclic,
|
||||
) = image_user_settings(source_object)
|
||||
source_image.use_fake_user = True
|
||||
source_object.data = result_image
|
||||
if animated:
|
||||
configure_image_user(
|
||||
image_user,
|
||||
len(color_paths),
|
||||
frame_start=original_frame_start,
|
||||
use_cyclic=original_use_cyclic,
|
||||
)
|
||||
except (OSError, RuntimeError) as error:
|
||||
source_image.use_fake_user = original_fake_user
|
||||
bpy.data.images.remove(result_image)
|
||||
raise RuntimeError("Unable to pack the background result") from error
|
||||
bpy.data.images.remove(result_image, do_unlink=True)
|
||||
raise RuntimeError("Unable to prepare the color result") from error
|
||||
return result_image
|
||||
|
||||
|
||||
@@ -993,7 +974,7 @@ def create_moge_material(
|
||||
color_image,
|
||||
depth_image,
|
||||
depth_scale,
|
||||
unshaded=False,
|
||||
shadeless=False,
|
||||
frame_count=1,
|
||||
frame_start=1,
|
||||
use_cyclic=False,
|
||||
@@ -1009,6 +990,7 @@ def create_moge_material(
|
||||
uv.uv_map = "UVMap"
|
||||
uv.location = (-900.0, 200.0)
|
||||
color = nodes.new("ShaderNodeTexImage")
|
||||
color_image.colorspace_settings.name = MATERIAL_COLOR_SPACE
|
||||
color.image = color_image
|
||||
color.interpolation = "Linear"
|
||||
color.extension = "EXTEND"
|
||||
@@ -1043,14 +1025,16 @@ def create_moge_material(
|
||||
links.new(color.outputs["Alpha"], layer.inputs["Alpha"])
|
||||
links.new(depth.outputs["Color"], layer.inputs["Depth"])
|
||||
layer.inputs["Bump Scale"].default_value = depth_scale
|
||||
if unshaded:
|
||||
shader = nodes.new("ShaderNodeEmission")
|
||||
if shadeless:
|
||||
shader = nodes.new("ShaderNodeGroup")
|
||||
shader.node_tree = load_shadeless_node_group()
|
||||
shader.name = SHADELESS_NODE_GROUP_NAME
|
||||
transparent = nodes.new("ShaderNodeBsdfTransparent")
|
||||
mix = nodes.new("ShaderNodeMixShader")
|
||||
links.new(layer.outputs["Color"], shader.inputs["Color"])
|
||||
links.new(layer.outputs["Alpha"], mix.inputs["Fac"])
|
||||
links.new(transparent.outputs["BSDF"], mix.inputs[1])
|
||||
links.new(shader.outputs["Emission"], mix.inputs[2])
|
||||
links.new(shader.outputs["Shader"], mix.inputs[2])
|
||||
links.new(mix.outputs["Shader"], output.inputs["Surface"])
|
||||
return material
|
||||
shader = nodes.new("ShaderNodeBsdfPrincipled")
|
||||
@@ -1307,16 +1291,13 @@ def create_material(
|
||||
color_image,
|
||||
depth_scale,
|
||||
displacement_image=None,
|
||||
foreground_image=None,
|
||||
displacement_midlevel=0.0,
|
||||
displacement_scale=1.0,
|
||||
unshaded=False,
|
||||
shadeless=False,
|
||||
frame_count=1,
|
||||
frame_start=1,
|
||||
color_frame_offset=0,
|
||||
color_frame_duration=None,
|
||||
foreground_frame_offset=0,
|
||||
foreground_frame_duration=None,
|
||||
depth_frame_offset=0,
|
||||
depth_frame_duration=None,
|
||||
use_cyclic=None,
|
||||
@@ -1336,7 +1317,8 @@ def create_material(
|
||||
|
||||
uv_map.uv_map = "UVMap"
|
||||
uv_map.location = (-1250.0, 350.0)
|
||||
material_image = foreground_image or color_image
|
||||
material_image = color_image
|
||||
material_image.colorspace_settings.name = MATERIAL_COLOR_SPACE
|
||||
color_texture.image = material_image
|
||||
color_texture.interpolation = "Linear"
|
||||
color_texture.extension = "EXTEND"
|
||||
@@ -1355,13 +1337,6 @@ def create_material(
|
||||
)
|
||||
material_frames = color_frames
|
||||
material_frame_offset = color_frame_offset
|
||||
if foreground_image is not None:
|
||||
material_frames = (
|
||||
foreground_frame_duration
|
||||
if foreground_frame_duration is not None
|
||||
else extracted_frames
|
||||
)
|
||||
material_frame_offset = foreground_frame_offset
|
||||
if frame_count > 1 or is_animated_image(material_image):
|
||||
configure_image_user(
|
||||
color_texture.image_user,
|
||||
@@ -1394,16 +1369,18 @@ def create_material(
|
||||
normal_texture.location = (-1000.0, -50.0)
|
||||
links.new(uv_map.outputs["UV"], normal_texture.inputs["Vector"])
|
||||
links.new(normal_texture.outputs["Color"], material_inputs.inputs["Normal"])
|
||||
if unshaded:
|
||||
emission = nodes.new("ShaderNodeEmission")
|
||||
if shadeless:
|
||||
shader = nodes.new("ShaderNodeGroup")
|
||||
shader.node_tree = load_shadeless_node_group()
|
||||
shader.name = SHADELESS_NODE_GROUP_NAME
|
||||
transparent = nodes.new("ShaderNodeBsdfTransparent")
|
||||
mix_shader = nodes.new("ShaderNodeMixShader")
|
||||
emission.location = (0.0, 550.0)
|
||||
shader.location = (0.0, 550.0)
|
||||
transparent.location = (0.0, 150.0)
|
||||
mix_shader.location = (300.0, 350.0)
|
||||
links.new(
|
||||
material_inputs.outputs["Color"],
|
||||
emission.inputs["Color"],
|
||||
shader.inputs["Color"],
|
||||
)
|
||||
links.new(
|
||||
material_inputs.outputs["Alpha"],
|
||||
@@ -1414,7 +1391,7 @@ def create_material(
|
||||
mix_shader.inputs[1],
|
||||
)
|
||||
links.new(
|
||||
emission.outputs["Emission"],
|
||||
shader.outputs["Shader"],
|
||||
mix_shader.inputs[2],
|
||||
)
|
||||
links.new(
|
||||
@@ -1502,13 +1479,13 @@ def node_asset_path():
|
||||
return Path(__file__).parent.parent / "assets" / NODE_ASSET_NAME
|
||||
|
||||
|
||||
def load_depth_plane_node_group(group_name=DEPTH_PLANE_NODE_GROUP_NAME):
|
||||
def load_asset_node_group(group_name):
|
||||
existing = bpy.data.node_groups.get(group_name)
|
||||
if existing is not None:
|
||||
return existing
|
||||
asset_path = node_asset_path()
|
||||
if not asset_path.is_file():
|
||||
raise RuntimeError(f"Missing Geometry Nodes asset: {asset_path}")
|
||||
raise RuntimeError(f"Missing node asset: {asset_path}")
|
||||
use_pack = bpy.app.version >= (5, 0, 0)
|
||||
if not use_pack:
|
||||
directory = str(asset_path) + "/NodeTree/"
|
||||
@@ -1520,18 +1497,15 @@ def load_depth_plane_node_group(group_name=DEPTH_PLANE_NODE_GROUP_NAME):
|
||||
do_reuse_local_id=True,
|
||||
)
|
||||
if "FINISHED" not in result:
|
||||
raise RuntimeError("Unable to append the Geometry Nodes asset")
|
||||
raise RuntimeError("Unable to append the node asset")
|
||||
node_group = bpy.data.node_groups.get(group_name)
|
||||
if node_group is None:
|
||||
raise RuntimeError("Appended Geometry Nodes asset is unavailable")
|
||||
raise RuntimeError("Appended node asset is unavailable")
|
||||
node_group.use_fake_user = True
|
||||
return node_group
|
||||
with bpy.data.libraries.load(str(asset_path), link=True) as (source, target):
|
||||
if group_name not in source.node_groups:
|
||||
raise RuntimeError(
|
||||
"Geometry Nodes asset is missing "
|
||||
f"{group_name}"
|
||||
)
|
||||
raise RuntimeError(f"Node asset is missing {group_name}")
|
||||
target.node_groups = [group_name]
|
||||
linked_group = target.node_groups[0]
|
||||
node_group = bpy.data.pack_linked_ids_hierarchy(root_id=linked_group)
|
||||
@@ -1541,6 +1515,14 @@ def load_depth_plane_node_group(group_name=DEPTH_PLANE_NODE_GROUP_NAME):
|
||||
return node_group
|
||||
|
||||
|
||||
def load_shadeless_node_group():
|
||||
return load_asset_node_group(SHADELESS_NODE_GROUP_NAME)
|
||||
|
||||
|
||||
def load_depth_plane_node_group(group_name=DEPTH_PLANE_NODE_GROUP_NAME):
|
||||
return load_asset_node_group(group_name)
|
||||
|
||||
|
||||
def modifier_input_identifier(node_group, name):
|
||||
for item in node_group.interface.items_tree:
|
||||
if item.item_type != "SOCKET" or item.in_out != "INPUT":
|
||||
@@ -1600,7 +1582,7 @@ def drive_modifier_input(owner_object, modifier, identifier, expression="frame")
|
||||
def add_depth_plane_processing(
|
||||
plane_object,
|
||||
depth_image,
|
||||
foreground_image,
|
||||
color_image,
|
||||
material,
|
||||
frame_start=1,
|
||||
frame_duration=1,
|
||||
@@ -1616,7 +1598,7 @@ def add_depth_plane_processing(
|
||||
modifier.show_group_selector = False
|
||||
for name, value in (
|
||||
("Depth Image", depth_image),
|
||||
("Foreground Image", foreground_image),
|
||||
("Color Image", color_image),
|
||||
("Material", material),
|
||||
("Frame", 1),
|
||||
("Frame Start", frame_start),
|
||||
@@ -1635,6 +1617,7 @@ def add_depth_plane_processing(
|
||||
def add_moge_depth_plane_processing(
|
||||
plane_object,
|
||||
depth_image,
|
||||
color_image,
|
||||
material,
|
||||
frame_start=1,
|
||||
frame_duration=1,
|
||||
@@ -1652,6 +1635,7 @@ def add_moge_depth_plane_processing(
|
||||
modifier.show_group_selector = False
|
||||
for name, value in (
|
||||
("Depth Image", depth_image),
|
||||
("Color Image", color_image),
|
||||
("Material", material),
|
||||
("Frame", 1),
|
||||
("Frame Start", frame_start),
|
||||
@@ -1688,9 +1672,8 @@ def build_image_depth_plane(
|
||||
context,
|
||||
source_object,
|
||||
delete_source,
|
||||
unshaded=False,
|
||||
shadeless=False,
|
||||
color_image=None,
|
||||
foreground_image=None,
|
||||
depth_frames_directory=None,
|
||||
depth_image=None,
|
||||
):
|
||||
@@ -1701,9 +1684,10 @@ def build_image_depth_plane(
|
||||
frames_image = create_sequence_image(
|
||||
depth_data_name(source_object),
|
||||
depth_frames_directory,
|
||||
frame_prefix="color.",
|
||||
)
|
||||
frame_count = len(
|
||||
sorted_frame_paths(depth_frames_directory, "frame.")
|
||||
sorted_frame_paths(depth_frames_directory, "color.")
|
||||
)
|
||||
(
|
||||
source_frame_start,
|
||||
@@ -1722,9 +1706,9 @@ def build_image_depth_plane(
|
||||
depth_source_image = frames_image
|
||||
elif not is_animated_image(source_object.data):
|
||||
depth_source_image = copy_depth_image(source_object)
|
||||
foreground_image = (
|
||||
foreground_image or color_image or source_object.data
|
||||
)
|
||||
material_image = color_image
|
||||
if material_image is None:
|
||||
raise RuntimeError("Depth Plane material image is required")
|
||||
mesh = create_plane_mesh(source_object, subdivision_level=None)
|
||||
plane_object = bpy.data.objects.new(source_name, mesh)
|
||||
context.collection.objects.link(plane_object)
|
||||
@@ -1733,19 +1717,16 @@ def build_image_depth_plane(
|
||||
depth_scale = abs(float(plane_object.scale.z))
|
||||
material = create_material(
|
||||
source_name,
|
||||
color_image if color_image is not None else source_object.data,
|
||||
material_image,
|
||||
depth_scale,
|
||||
displacement_image=depth_source_image,
|
||||
foreground_image=foreground_image,
|
||||
displacement_midlevel=ORIGINAL_DISPLACEMENT_MIDLEVEL,
|
||||
displacement_scale=ORIGINAL_DISPLACEMENT_STRENGTH,
|
||||
unshaded=unshaded,
|
||||
shadeless=shadeless,
|
||||
frame_count=frame_count,
|
||||
frame_start=source_frame_start,
|
||||
color_frame_offset=texture_frame_offset,
|
||||
color_frame_duration=texture_frame_count,
|
||||
foreground_frame_offset=0,
|
||||
foreground_frame_duration=frame_count,
|
||||
depth_frame_offset=texture_frame_offset,
|
||||
depth_frame_duration=texture_frame_count,
|
||||
use_cyclic=source_use_cyclic,
|
||||
@@ -1754,7 +1735,7 @@ def build_image_depth_plane(
|
||||
add_depth_plane_processing(
|
||||
plane_object,
|
||||
depth_source_image,
|
||||
foreground_image,
|
||||
material_image,
|
||||
material,
|
||||
frame_start=source_frame_start,
|
||||
frame_duration=frame_count,
|
||||
@@ -1775,10 +1756,9 @@ def build_depth_plane(
|
||||
source_object,
|
||||
npz_path,
|
||||
delete_source,
|
||||
unshaded=False,
|
||||
shadeless=False,
|
||||
depth_name="",
|
||||
color_image=None,
|
||||
foreground_image=None,
|
||||
depth_frames_directory=None,
|
||||
):
|
||||
source_name = source_object.name
|
||||
@@ -1797,9 +1777,9 @@ def build_depth_plane(
|
||||
]
|
||||
depth_name = depth_name or depth_data_name(source_object)
|
||||
frame_count = len(depth_arrays)
|
||||
foreground_image = (
|
||||
foreground_image or color_image or source_object.data
|
||||
)
|
||||
material_image = color_image
|
||||
if material_image is None:
|
||||
raise RuntimeError("Depth Plane material image is required")
|
||||
(
|
||||
source_frame_start,
|
||||
source_frame_offset,
|
||||
@@ -1825,7 +1805,10 @@ def build_depth_plane(
|
||||
depth_name,
|
||||
scaled_depth_arrays[0],
|
||||
)
|
||||
if frame_count > 1 and depth_frames_directory is not None:
|
||||
if (
|
||||
is_animated_image(source_object.data)
|
||||
and depth_frames_directory is not None
|
||||
):
|
||||
bpy.data.images.remove(depth_image, do_unlink=True)
|
||||
depth_image = create_depth_sequence_image(
|
||||
depth_name,
|
||||
@@ -1836,21 +1819,18 @@ def build_depth_plane(
|
||||
|
||||
material = create_material(
|
||||
source_name,
|
||||
color_image if color_image is not None else source_object.data,
|
||||
material_image,
|
||||
depth_scale,
|
||||
displacement_image=depth_image,
|
||||
foreground_image=foreground_image,
|
||||
displacement_midlevel=DISPLACEMENT_MIDLEVEL,
|
||||
displacement_scale=DISPLACEMENT_STRENGTH,
|
||||
unshaded=unshaded,
|
||||
shadeless=shadeless,
|
||||
frame_count=frame_count,
|
||||
frame_start=source_frame_start,
|
||||
color_frame_offset=0 if color_image is not None else source_frame_offset,
|
||||
color_frame_duration=(
|
||||
frame_count if color_image is not None else source_frame_duration
|
||||
),
|
||||
foreground_frame_offset=0,
|
||||
foreground_frame_duration=frame_count,
|
||||
depth_frame_offset=0,
|
||||
depth_frame_duration=frame_count,
|
||||
use_cyclic=source_use_cyclic,
|
||||
@@ -1859,7 +1839,7 @@ def build_depth_plane(
|
||||
add_depth_plane_processing(
|
||||
plane_object,
|
||||
depth_image,
|
||||
foreground_image,
|
||||
material_image,
|
||||
material,
|
||||
frame_start=source_frame_start,
|
||||
frame_duration=frame_count,
|
||||
@@ -1885,9 +1865,8 @@ def build_moge2_depth_plane(
|
||||
source_object,
|
||||
npz_path,
|
||||
delete_source,
|
||||
unshaded=False,
|
||||
shadeless=False,
|
||||
color_image=None,
|
||||
foreground_image=None,
|
||||
data_frames_directory=None,
|
||||
):
|
||||
import numpy as np
|
||||
@@ -1935,7 +1914,10 @@ def build_moge2_depth_plane(
|
||||
depth_data_name(source_object),
|
||||
displacement[0] * depth_scale,
|
||||
)
|
||||
if len(depth) > 1 and data_frames_directory is not None:
|
||||
if (
|
||||
is_animated_image(source_object.data)
|
||||
and data_frames_directory is not None
|
||||
):
|
||||
bpy.data.images.remove(depth_image, do_unlink=True)
|
||||
depth_image = create_moge_sequence_image(
|
||||
depth_data_name(source_object),
|
||||
@@ -1945,12 +1927,15 @@ def build_moge2_depth_plane(
|
||||
"moge_depth",
|
||||
"depth",
|
||||
)
|
||||
material_image = color_image
|
||||
if material_image is None:
|
||||
raise RuntimeError("Depth Plane material image is required")
|
||||
material = create_moge_material(
|
||||
source_object.name,
|
||||
foreground_image or color_image or source_object.data,
|
||||
material_image,
|
||||
depth_image,
|
||||
depth_scale,
|
||||
unshaded=unshaded,
|
||||
shadeless=shadeless,
|
||||
frame_count=len(depth),
|
||||
frame_start=source_frame_start,
|
||||
use_cyclic=source_use_cyclic,
|
||||
@@ -1959,6 +1944,7 @@ def build_moge2_depth_plane(
|
||||
modifier = add_moge_depth_plane_processing(
|
||||
plane_object,
|
||||
depth_image,
|
||||
material_image,
|
||||
material,
|
||||
frame_start=source_frame_start,
|
||||
frame_duration=len(depth),
|
||||
@@ -2028,9 +2014,9 @@ class ConvertToDepthPlane(bpy.types.Operator):
|
||||
bl_description = "Convert an image to a displaced depth plane"
|
||||
bl_options = {"REGISTER"}
|
||||
|
||||
unshaded: bpy.props.BoolProperty(
|
||||
name="Unshaded",
|
||||
description="Display the source color through an Emission shader",
|
||||
shadeless: bpy.props.BoolProperty(
|
||||
name="Shadeless",
|
||||
description="Display the source color without scene lighting",
|
||||
default=False,
|
||||
)
|
||||
remove_background: bpy.props.BoolProperty(
|
||||
@@ -2081,7 +2067,7 @@ class ConvertToDepthPlane(bpy.types.Operator):
|
||||
elif self.model.startswith("MOGE2_"):
|
||||
layout.prop(self, "resolution_level")
|
||||
layout.prop(self, "remove_background")
|
||||
layout.prop(self, "unshaded")
|
||||
layout.prop(self, "shadeless")
|
||||
layout.separator(type="LINE")
|
||||
layout.prop(self, "keep_original")
|
||||
|
||||
@@ -2110,7 +2096,7 @@ class ConvertToDepthPlane(bpy.types.Operator):
|
||||
source_object_name=source_object.name,
|
||||
delete_source=not self.keep_original,
|
||||
delete_input=delete_input,
|
||||
unshaded=self.unshaded,
|
||||
shadeless=self.shadeless,
|
||||
)
|
||||
if "CANCELLED" not in result:
|
||||
return {"FINISHED"}
|
||||
@@ -2134,7 +2120,7 @@ class ConvertToDepthPlane(bpy.types.Operator):
|
||||
model=self.model,
|
||||
delete_source=not self.keep_original,
|
||||
delete_input=delete_input,
|
||||
unshaded=self.unshaded,
|
||||
shadeless=self.shadeless,
|
||||
remove_background=self.remove_background,
|
||||
)
|
||||
if "CANCELLED" not in result:
|
||||
@@ -2169,7 +2155,7 @@ class ConvertToDepthPlane(bpy.types.Operator):
|
||||
process_res=process_res,
|
||||
delete_source=not self.keep_original,
|
||||
delete_input=delete_input,
|
||||
unshaded=self.unshaded,
|
||||
shadeless=self.shadeless,
|
||||
remove_background=self.remove_background,
|
||||
)
|
||||
if "CANCELLED" in result:
|
||||
|
||||
@@ -402,7 +402,7 @@ def build_image_mesh(
|
||||
npz_path,
|
||||
delete_source,
|
||||
create_camera,
|
||||
unshaded,
|
||||
shadeless,
|
||||
moge2=False,
|
||||
color_image=None,
|
||||
normal_path=None,
|
||||
@@ -437,6 +437,8 @@ def build_image_mesh(
|
||||
context.collection.objects.link(mesh_object)
|
||||
mesh_object.matrix_world = source_object.matrix_world.copy()
|
||||
depth_scale = material_depth_scale(mesh_object, camera_parameters)
|
||||
if color_image is None:
|
||||
raise RuntimeError("Surface material image is required")
|
||||
normal_image = (
|
||||
load_surface_normal_image(
|
||||
normal_path,
|
||||
@@ -447,9 +449,9 @@ def build_image_mesh(
|
||||
)
|
||||
material = create_material(
|
||||
source_name,
|
||||
color_image or source_object.data,
|
||||
color_image,
|
||||
depth_scale,
|
||||
unshaded=unshaded,
|
||||
shadeless=shadeless,
|
||||
surface_group_name=(
|
||||
MOGE_SURFACE_MATERIAL_NODE_GROUP_NAME if moge2 else None
|
||||
),
|
||||
@@ -509,9 +511,9 @@ class ConvertToSurface(bpy.types.Operator):
|
||||
description="Create a perspective camera matching the source projection",
|
||||
default=False,
|
||||
)
|
||||
unshaded: bpy.props.BoolProperty(
|
||||
name="Unshaded",
|
||||
description="Display the source color through an Emission shader",
|
||||
shadeless: bpy.props.BoolProperty(
|
||||
name="Shadeless",
|
||||
description="Display the source color without scene lighting",
|
||||
default=False,
|
||||
)
|
||||
remove_background: bpy.props.BoolProperty(
|
||||
@@ -550,7 +552,7 @@ class ConvertToSurface(bpy.types.Operator):
|
||||
draw_process_resolution(layout, self, context.object.data)
|
||||
layout.prop(self, "create_camera")
|
||||
layout.prop(self, "remove_background")
|
||||
layout.prop(self, "unshaded")
|
||||
layout.prop(self, "shadeless")
|
||||
layout.separator(type="LINE")
|
||||
layout.prop(self, "keep_original")
|
||||
|
||||
@@ -580,7 +582,7 @@ class ConvertToSurface(bpy.types.Operator):
|
||||
"delete_source": not self.keep_original,
|
||||
"delete_input": delete_input,
|
||||
"create_camera": self.create_camera,
|
||||
"unshaded": self.unshaded,
|
||||
"shadeless": self.shadeless,
|
||||
"remove_background": self.remove_background,
|
||||
}
|
||||
if self.model.startswith("MOGE2_"):
|
||||
|
||||
@@ -111,13 +111,12 @@ def build_da3_depth_response(
|
||||
depth_data_name,
|
||||
remove_background,
|
||||
delete_source,
|
||||
unshaded,
|
||||
shadeless,
|
||||
):
|
||||
from .blender.depth_plane import (
|
||||
build_depth_plane,
|
||||
create_removed_background_image,
|
||||
create_sequence_image,
|
||||
image_base_name,
|
||||
create_color_image,
|
||||
is_animated_image,
|
||||
is_image_empty,
|
||||
)
|
||||
|
||||
@@ -125,32 +124,21 @@ def build_da3_depth_response(
|
||||
source_object = bpy.data.objects.get(source_object_name)
|
||||
if not is_image_empty(source_object):
|
||||
raise RuntimeError("The source image empty is no longer available")
|
||||
foreground_paths = sorted(directory.glob("foreground.*.png"))
|
||||
foreground_image = None
|
||||
if remove_background:
|
||||
foreground_image = create_removed_background_image(
|
||||
source_object,
|
||||
directory if foreground_paths else directory / "foreground.png",
|
||||
assign_to_source=False,
|
||||
)
|
||||
color_image = None
|
||||
if list(directory.glob("frame.*")) and foreground_image is None:
|
||||
color_image = create_sequence_image(
|
||||
f"{image_base_name(source_object.data)}.frame",
|
||||
directory,
|
||||
)
|
||||
animated = is_animated_image(source_object.data)
|
||||
color_image = create_color_image(
|
||||
source_object,
|
||||
directory,
|
||||
assign_to_source=False,
|
||||
)
|
||||
build_depth_plane(
|
||||
context,
|
||||
source_object,
|
||||
result.file("depth"),
|
||||
delete_source,
|
||||
unshaded=unshaded,
|
||||
shadeless=shadeless,
|
||||
depth_name=depth_data_name,
|
||||
color_image=color_image,
|
||||
foreground_image=foreground_image,
|
||||
depth_frames_directory=(
|
||||
directory if list(directory.glob("frame.*")) else None
|
||||
),
|
||||
depth_frames_directory=directory if animated else None,
|
||||
)
|
||||
push_undo("Convert to Depth Plane")
|
||||
|
||||
@@ -162,13 +150,11 @@ def build_moge2_depth_response(
|
||||
source_object_name,
|
||||
remove_background,
|
||||
delete_source,
|
||||
unshaded,
|
||||
shadeless,
|
||||
):
|
||||
from .blender.depth_plane import (
|
||||
build_moge2_depth_plane,
|
||||
create_removed_background_image,
|
||||
create_sequence_image,
|
||||
image_base_name,
|
||||
create_color_image,
|
||||
is_image_empty,
|
||||
)
|
||||
|
||||
@@ -176,28 +162,18 @@ def build_moge2_depth_response(
|
||||
source_object = bpy.data.objects.get(source_object_name)
|
||||
if not is_image_empty(source_object):
|
||||
raise RuntimeError("The source image empty is no longer available")
|
||||
foreground_paths = sorted(directory.glob("foreground.*.png"))
|
||||
foreground_image = None
|
||||
if remove_background:
|
||||
foreground_image = create_removed_background_image(
|
||||
source_object,
|
||||
directory if foreground_paths else directory / "foreground.png",
|
||||
assign_to_source=False,
|
||||
)
|
||||
color_image = None
|
||||
if list(directory.glob("frame.*")) and foreground_image is None:
|
||||
color_image = create_sequence_image(
|
||||
f"{image_base_name(source_object.data)}.frame",
|
||||
directory,
|
||||
)
|
||||
color_image = create_color_image(
|
||||
source_object,
|
||||
directory,
|
||||
assign_to_source=False,
|
||||
)
|
||||
build_moge2_depth_plane(
|
||||
context,
|
||||
source_object,
|
||||
result.file("depth"),
|
||||
delete_source,
|
||||
unshaded=unshaded,
|
||||
shadeless=shadeless,
|
||||
color_image=color_image,
|
||||
foreground_image=foreground_image,
|
||||
data_frames_directory=directory,
|
||||
)
|
||||
push_undo("Convert to Depth Plane")
|
||||
@@ -210,13 +186,12 @@ def build_original_depth_response(
|
||||
source_object_name,
|
||||
background,
|
||||
delete_source,
|
||||
unshaded,
|
||||
shadeless,
|
||||
):
|
||||
from .blender.depth_plane import (
|
||||
build_image_depth_plane,
|
||||
create_removed_background_image,
|
||||
create_sequence_image,
|
||||
image_base_name,
|
||||
create_color_image,
|
||||
is_animated_image,
|
||||
is_image_empty,
|
||||
)
|
||||
|
||||
@@ -224,30 +199,19 @@ def build_original_depth_response(
|
||||
source_object = bpy.data.objects.get(source_object_name)
|
||||
if not is_image_empty(source_object):
|
||||
raise RuntimeError("The source image empty is no longer available")
|
||||
foreground_image = None
|
||||
if background:
|
||||
foreground_paths = sorted(directory.glob("foreground.*.png"))
|
||||
foreground_image = create_removed_background_image(
|
||||
source_object,
|
||||
directory if foreground_paths else directory / "foreground.png",
|
||||
assign_to_source=False,
|
||||
)
|
||||
color_image = None
|
||||
if list(directory.glob("frame.*")) and foreground_image is None:
|
||||
color_image = create_sequence_image(
|
||||
f"{image_base_name(source_object.data)}.frame",
|
||||
directory,
|
||||
)
|
||||
animated = is_animated_image(source_object.data)
|
||||
color_image = create_color_image(
|
||||
source_object,
|
||||
directory,
|
||||
assign_to_source=False,
|
||||
)
|
||||
build_image_depth_plane(
|
||||
context,
|
||||
source_object,
|
||||
delete_source,
|
||||
unshaded=unshaded,
|
||||
shadeless=shadeless,
|
||||
color_image=color_image,
|
||||
foreground_image=foreground_image,
|
||||
depth_frames_directory=(
|
||||
directory if list(directory.glob("frame.*")) else None
|
||||
),
|
||||
depth_frames_directory=directory if animated else None,
|
||||
)
|
||||
push_undo("Convert to Depth Plane")
|
||||
|
||||
@@ -259,27 +223,19 @@ def build_surface_response(
|
||||
source_object_name,
|
||||
delete_source,
|
||||
create_camera,
|
||||
unshaded,
|
||||
shadeless,
|
||||
moge2,
|
||||
):
|
||||
from .blender.depth_plane import (
|
||||
create_removed_background_image,
|
||||
is_image_empty,
|
||||
)
|
||||
from .blender.depth_plane import create_color_image, is_image_empty
|
||||
from .blender.surface import build_image_mesh
|
||||
|
||||
source_object = bpy.data.objects.get(source_object_name)
|
||||
if not is_image_empty(source_object):
|
||||
raise RuntimeError("The source Image Empty no longer exists")
|
||||
foreground_path = result.directory / "foreground.png"
|
||||
color_image = (
|
||||
create_removed_background_image(
|
||||
source_object,
|
||||
foreground_path,
|
||||
assign_to_source=False,
|
||||
)
|
||||
if foreground_path.is_file()
|
||||
else None
|
||||
color_image = create_color_image(
|
||||
source_object,
|
||||
result.directory,
|
||||
assign_to_source=False,
|
||||
)
|
||||
build_image_mesh(
|
||||
context,
|
||||
@@ -287,7 +243,7 @@ def build_surface_response(
|
||||
result.file("surface"),
|
||||
delete_source,
|
||||
create_camera,
|
||||
unshaded,
|
||||
shadeless,
|
||||
moge2,
|
||||
color_image=color_image,
|
||||
normal_path=(
|
||||
@@ -300,19 +256,12 @@ def build_surface_response(
|
||||
|
||||
|
||||
def apply_background_response(result, source_object_name):
|
||||
from .blender.depth_plane import (
|
||||
create_removed_background_image,
|
||||
is_image_empty,
|
||||
)
|
||||
from .blender.depth_plane import create_color_image, is_image_empty
|
||||
|
||||
source_object = bpy.data.objects.get(source_object_name)
|
||||
if not is_image_empty(source_object):
|
||||
raise RuntimeError("The source image empty is no longer available")
|
||||
foreground_paths = sorted(result.directory.glob("foreground.*.png"))
|
||||
create_removed_background_image(
|
||||
source_object,
|
||||
result.directory if foreground_paths else result.file("foreground"),
|
||||
)
|
||||
create_color_image(source_object, result.directory)
|
||||
push_undo("Remove Background")
|
||||
|
||||
|
||||
@@ -461,7 +410,7 @@ class GenerateDa3Depth(JobOperatorBase, bpy.types.Operator):
|
||||
process_res: bpy.props.IntProperty(default=1024, min=1, options=HIDDEN)
|
||||
delete_source: bpy.props.BoolProperty(default=True, options=HIDDEN)
|
||||
delete_input: bpy.props.BoolProperty(default=False, options=HIDDEN)
|
||||
unshaded: bpy.props.BoolProperty(default=False, options=HIDDEN)
|
||||
shadeless: bpy.props.BoolProperty(default=False, options=HIDDEN)
|
||||
remove_background: bpy.props.BoolProperty(default=False, options=HIDDEN)
|
||||
|
||||
def request(self, context):
|
||||
@@ -492,7 +441,7 @@ class GenerateDa3Depth(JobOperatorBase, bpy.types.Operator):
|
||||
depth_data_name=self.depth_data_name,
|
||||
remove_background=self.remove_background,
|
||||
delete_source=self.delete_source,
|
||||
unshaded=self.unshaded,
|
||||
shadeless=self.shadeless,
|
||||
)
|
||||
return "Depth plane is ready"
|
||||
|
||||
@@ -512,7 +461,7 @@ class GenerateMoge2Depth(JobOperatorBase, bpy.types.Operator):
|
||||
resolution_level: bpy.props.IntProperty(default=5, min=0, max=9, options=HIDDEN)
|
||||
delete_source: bpy.props.BoolProperty(default=True, options=HIDDEN)
|
||||
delete_input: bpy.props.BoolProperty(default=False, options=HIDDEN)
|
||||
unshaded: bpy.props.BoolProperty(default=False, options=HIDDEN)
|
||||
shadeless: bpy.props.BoolProperty(default=False, options=HIDDEN)
|
||||
remove_background: bpy.props.BoolProperty(default=False, options=HIDDEN)
|
||||
|
||||
def request(self, context):
|
||||
@@ -541,7 +490,7 @@ class GenerateMoge2Depth(JobOperatorBase, bpy.types.Operator):
|
||||
source_object_name=self.source_object_name,
|
||||
remove_background=self.remove_background,
|
||||
delete_source=self.delete_source,
|
||||
unshaded=self.unshaded,
|
||||
shadeless=self.shadeless,
|
||||
)
|
||||
return "MoGe-2 depth plane is ready"
|
||||
|
||||
@@ -562,7 +511,7 @@ class GenerateDa3Surface(JobOperatorBase, bpy.types.Operator):
|
||||
delete_source: bpy.props.BoolProperty(default=True, options=HIDDEN)
|
||||
delete_input: bpy.props.BoolProperty(default=False, options=HIDDEN)
|
||||
create_camera: bpy.props.BoolProperty(default=False, options=HIDDEN)
|
||||
unshaded: bpy.props.BoolProperty(default=False, options=HIDDEN)
|
||||
shadeless: bpy.props.BoolProperty(default=False, options=HIDDEN)
|
||||
remove_background: bpy.props.BoolProperty(default=False, options=HIDDEN)
|
||||
|
||||
def request(self, context):
|
||||
@@ -591,7 +540,7 @@ class GenerateDa3Surface(JobOperatorBase, bpy.types.Operator):
|
||||
source_object_name=self.source_object_name,
|
||||
delete_source=self.delete_source,
|
||||
create_camera=self.create_camera,
|
||||
unshaded=self.unshaded,
|
||||
shadeless=self.shadeless,
|
||||
moge2=False,
|
||||
)
|
||||
return "Perspective surface is ready"
|
||||
@@ -613,7 +562,7 @@ class GenerateMoge2Surface(JobOperatorBase, bpy.types.Operator):
|
||||
delete_source: bpy.props.BoolProperty(default=True, options=HIDDEN)
|
||||
delete_input: bpy.props.BoolProperty(default=False, options=HIDDEN)
|
||||
create_camera: bpy.props.BoolProperty(default=False, options=HIDDEN)
|
||||
unshaded: bpy.props.BoolProperty(default=False, options=HIDDEN)
|
||||
shadeless: bpy.props.BoolProperty(default=False, options=HIDDEN)
|
||||
remove_background: bpy.props.BoolProperty(default=False, options=HIDDEN)
|
||||
|
||||
def request(self, context):
|
||||
@@ -642,7 +591,7 @@ class GenerateMoge2Surface(JobOperatorBase, bpy.types.Operator):
|
||||
source_object_name=self.source_object_name,
|
||||
delete_source=self.delete_source,
|
||||
create_camera=self.create_camera,
|
||||
unshaded=self.unshaded,
|
||||
shadeless=self.shadeless,
|
||||
moge2=True,
|
||||
)
|
||||
return "MoGe-2 surface is ready"
|
||||
@@ -691,7 +640,7 @@ class GenerateOriginalBackground(JobOperatorBase, bpy.types.Operator):
|
||||
source_object_name: bpy.props.StringProperty(options=HIDDEN)
|
||||
delete_source: bpy.props.BoolProperty(default=True, options=HIDDEN)
|
||||
delete_input: bpy.props.BoolProperty(default=False, options=HIDDEN)
|
||||
unshaded: bpy.props.BoolProperty(default=False, options=HIDDEN)
|
||||
shadeless: bpy.props.BoolProperty(default=False, options=HIDDEN)
|
||||
|
||||
def request(self, context):
|
||||
require_environment()
|
||||
@@ -712,7 +661,7 @@ class GenerateOriginalBackground(JobOperatorBase, bpy.types.Operator):
|
||||
source_object_name=self.source_object_name,
|
||||
background=True,
|
||||
delete_source=self.delete_source,
|
||||
unshaded=self.unshaded,
|
||||
shadeless=self.shadeless,
|
||||
)
|
||||
return "Depth plane is ready"
|
||||
|
||||
@@ -730,7 +679,7 @@ class ExtractOriginalFrames(JobOperatorBase, bpy.types.Operator):
|
||||
source_object_name: bpy.props.StringProperty(options=HIDDEN)
|
||||
delete_source: bpy.props.BoolProperty(default=True, options=HIDDEN)
|
||||
delete_input: bpy.props.BoolProperty(default=False, options=HIDDEN)
|
||||
unshaded: bpy.props.BoolProperty(default=False, options=HIDDEN)
|
||||
shadeless: bpy.props.BoolProperty(default=False, options=HIDDEN)
|
||||
|
||||
def request(self, context):
|
||||
require_environment()
|
||||
@@ -748,7 +697,7 @@ class ExtractOriginalFrames(JobOperatorBase, bpy.types.Operator):
|
||||
source_object_name=self.source_object_name,
|
||||
background=False,
|
||||
delete_source=self.delete_source,
|
||||
unshaded=self.unshaded,
|
||||
shadeless=self.shadeless,
|
||||
)
|
||||
return "Depth plane is ready"
|
||||
|
||||
|
||||
@@ -69,9 +69,9 @@ def triangle_edge_stretch_values(points, triangles):
|
||||
|
||||
|
||||
def remove_background_faces(faces, foreground_mask):
|
||||
"""Keep faces with at least one foreground vertex."""
|
||||
"""Keep only faces whose vertices are all foreground."""
|
||||
foreground_points = foreground_mask.reshape(-1)
|
||||
return faces[foreground_points[faces].any(axis=1)]
|
||||
return faces[foreground_points[faces].all(axis=1)]
|
||||
|
||||
|
||||
def depth_edge_mask(depth, threshold=12.0 / 255.0):
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
import shutil
|
||||
import time
|
||||
|
||||
from ..media import copy_frames, prepare_background_input
|
||||
from ..media import prepare_background_input
|
||||
from ..models.ben2 import infer_foreground, infer_foreground_frames
|
||||
|
||||
|
||||
@@ -39,7 +39,7 @@ def run(context, parameters):
|
||||
)
|
||||
context.check_cancelled()
|
||||
context.progress(0.92, "Writing transparent PNG")
|
||||
output_path = output_directory / "foreground.png"
|
||||
output_path = output_directory / "color.0001.png"
|
||||
foreground.convert("RGBA").save(
|
||||
output_path,
|
||||
format="PNG",
|
||||
@@ -58,8 +58,8 @@ def run(context, parameters):
|
||||
0.7,
|
||||
model_cache=model_manager,
|
||||
cancel_check=context.check_cancelled,
|
||||
output_prefix="color",
|
||||
)
|
||||
copy_frames(images, output_directory)
|
||||
finally:
|
||||
if temporary_directory is not None:
|
||||
shutil.rmtree(temporary_directory, ignore_errors=True)
|
||||
@@ -73,6 +73,4 @@ def run(context, parameters):
|
||||
)
|
||||
print(message)
|
||||
context.progress(1.0, message)
|
||||
if tuple(output_directory.glob("foreground.*.png")):
|
||||
return {"foreground": "."}
|
||||
return {"foreground": "foreground.png"}
|
||||
return {"color": "."}
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
from ..media import copy_frames
|
||||
from ..media import copy_color_images
|
||||
from ..models import da3, geometry_adapter, moge2
|
||||
from ..outputs.depth import write_da3_depth, write_moge_depth
|
||||
from .common import input_frames
|
||||
@@ -36,14 +36,15 @@ def run(context, parameters):
|
||||
cancel_check=context.check_cancelled,
|
||||
)
|
||||
write_moge_depth(context.directory, prediction)
|
||||
write_optional(
|
||||
color_paths = write_optional(
|
||||
context,
|
||||
parameters,
|
||||
images,
|
||||
progress_start=0.84,
|
||||
progress_span=0.08,
|
||||
)
|
||||
copy_frames(images, context.directory)
|
||||
if not color_paths:
|
||||
copy_color_images(images, context.directory)
|
||||
context.check_cancelled()
|
||||
context.progress(1.0, f"Exported depth for {len(images)} frame(s)")
|
||||
return {"depth": "results.npz"}
|
||||
|
||||
@@ -24,7 +24,7 @@ def write_optional(
|
||||
model_cache=model_manager,
|
||||
cancel_check=context.check_cancelled,
|
||||
)
|
||||
path = context.directory / "foreground.png"
|
||||
path = context.directory / "color.0001.png"
|
||||
foreground.convert("RGBA").save(
|
||||
path,
|
||||
format="PNG",
|
||||
@@ -42,8 +42,9 @@ def write_optional(
|
||||
progress_span,
|
||||
model_cache=model_manager,
|
||||
cancel_check=context.check_cancelled,
|
||||
output_prefix="color",
|
||||
)
|
||||
paths = sorted(context.directory.glob("foreground.*.png"))
|
||||
paths = sorted(context.directory.glob("color.*.png"))
|
||||
print(
|
||||
f"BEN2 inference {inference_ms:.0f} ms; "
|
||||
f"model load {load_ms:.0f} ms"
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
import shutil
|
||||
|
||||
from ..media import copy_frames, prepare_input
|
||||
from ..media import copy_color_images, prepare_input
|
||||
|
||||
|
||||
def run(context, parameters):
|
||||
@@ -13,7 +13,7 @@ def run(context, parameters):
|
||||
cancel_check=context.check_cancelled,
|
||||
)
|
||||
try:
|
||||
copy_frames(images, context.directory)
|
||||
copy_color_images(images, context.directory)
|
||||
finally:
|
||||
if temporary_directory is not None:
|
||||
shutil.rmtree(temporary_directory, ignore_errors=True)
|
||||
|
||||
@@ -1,15 +1,18 @@
|
||||
from ..geometry import foreground_mesh_mask
|
||||
from ..geometry import build_surface, foreground_mesh_mask
|
||||
from ..media import copy_color_images
|
||||
from ..models import geometry_adapter, moge2
|
||||
from ..geometry import build_surface
|
||||
from ..outputs.surface import (
|
||||
write_normal_texture,
|
||||
write_surface_archive,
|
||||
write_surface_texture,
|
||||
)
|
||||
from ..outputs.surface import write_normal_texture, write_surface_archive
|
||||
from .common import input_frames
|
||||
from .foreground import write_optional
|
||||
|
||||
|
||||
def inference_progress_message(model_name, index, total):
|
||||
message = f"Estimating {model_name} geometry"
|
||||
if total > 1:
|
||||
message += f" ({index + 1}/{total})"
|
||||
return message
|
||||
|
||||
|
||||
def run(context, parameters):
|
||||
context.progress(0.02, "Preparing surface input")
|
||||
context.directory.mkdir(parents=True, exist_ok=True)
|
||||
@@ -27,7 +30,7 @@ def run(context, parameters):
|
||||
include_points=True,
|
||||
progress_callback=lambda index, total: context.progress(
|
||||
0.12 + 0.58 * index / total,
|
||||
f"Estimating MoGe-2 geometry ({index + 1}/{total})",
|
||||
inference_progress_message("MoGe-2", index, total),
|
||||
),
|
||||
cancel_check=context.check_cancelled,
|
||||
)
|
||||
@@ -38,13 +41,13 @@ def run(context, parameters):
|
||||
images,
|
||||
progress_callback=lambda index, total: context.progress(
|
||||
0.12 + 0.58 * index / total,
|
||||
f"Estimating DA3 geometry ({index + 1}/{total})",
|
||||
inference_progress_message("DA3", index, total),
|
||||
),
|
||||
cancel_check=context.check_cancelled,
|
||||
)
|
||||
context.check_cancelled()
|
||||
frame = adapter.surface_frame(prediction)
|
||||
foreground_paths = write_optional(
|
||||
color_paths = write_optional(
|
||||
context,
|
||||
parameters,
|
||||
images,
|
||||
@@ -52,19 +55,17 @@ def run(context, parameters):
|
||||
progress_span=0.08,
|
||||
)
|
||||
foreground_mask = None
|
||||
if foreground_paths:
|
||||
if color_paths:
|
||||
foreground_mask = foreground_mesh_mask(
|
||||
foreground_paths[0],
|
||||
color_paths[0],
|
||||
frame.depth.shape,
|
||||
)
|
||||
context.progress(0.82, "Building surface")
|
||||
mesh = build_surface(frame, foreground_mask=foreground_mask)
|
||||
context.progress(0.92, "Writing surface")
|
||||
write_surface_archive(mesh, context.directory / "mesh.npz")
|
||||
write_surface_texture(
|
||||
foreground_paths[0] if foreground_paths else images[0],
|
||||
context.directory / "texture.png",
|
||||
)
|
||||
if not color_paths:
|
||||
copy_color_images(images, context.directory)
|
||||
if adapter is moge2:
|
||||
write_normal_texture(
|
||||
prediction.normal[0],
|
||||
|
||||
@@ -4,6 +4,7 @@ from .input import (
|
||||
TaskCancelled,
|
||||
check_cancelled,
|
||||
collect_images,
|
||||
copy_color_images,
|
||||
copy_frames,
|
||||
extract_video_frames,
|
||||
first_frames,
|
||||
@@ -18,6 +19,7 @@ __all__ = (
|
||||
"TaskCancelled",
|
||||
"check_cancelled",
|
||||
"collect_images",
|
||||
"copy_color_images",
|
||||
"copy_frames",
|
||||
"extract_video_frames",
|
||||
"first_frames",
|
||||
|
||||
@@ -143,16 +143,26 @@ def prepare_background_input(
|
||||
)
|
||||
|
||||
|
||||
def copy_frames(images, output_directory):
|
||||
"""Copy a multi-frame input as numbered frame files for the Blender sequence."""
|
||||
if len(images) <= 1:
|
||||
return
|
||||
def copy_color_images(images, output_directory):
|
||||
"""Copy input images as the canonical numbered color layer."""
|
||||
for index, image_path in enumerate(images):
|
||||
suffix = Path(image_path).suffix.lower() or ".jpg"
|
||||
destination = (
|
||||
Path(output_directory) / f"frame.{index + 1:04d}{suffix}"
|
||||
Path(output_directory) / f"color.{index + 1:04d}{suffix}"
|
||||
)
|
||||
if Path(image_path).resolve() == destination.resolve():
|
||||
continue
|
||||
shutil.copy2(image_path, destination)
|
||||
|
||||
|
||||
def copy_frames(images, output_directory):
|
||||
"""Copy multiple input images as numbered raw Debug frame files."""
|
||||
if len(images) <= 1:
|
||||
return
|
||||
for index, image_path in enumerate(images):
|
||||
suffix = Path(image_path).suffix.lower() or ".jpg"
|
||||
destination = Path(output_directory) / f"frame.{index + 1:04d}{suffix}"
|
||||
if Path(image_path).resolve() == destination.resolve():
|
||||
continue
|
||||
shutil.copy2(image_path, destination)
|
||||
|
||||
|
||||
@@ -48,8 +48,9 @@ def infer_foreground_frames(
|
||||
progress_span,
|
||||
model_cache=None,
|
||||
cancel_check=None,
|
||||
output_prefix="foreground",
|
||||
):
|
||||
"""Run BEN2 on every frame and write foreground.0001.png etc."""
|
||||
"""Run BEN2 on every frame and write numbered transparent PNG files."""
|
||||
total_load_ms = 0.0
|
||||
total_inference_ms = 0.0
|
||||
for index, image_path in enumerate(images):
|
||||
@@ -65,7 +66,7 @@ def infer_foreground_frames(
|
||||
model_cache=model_cache,
|
||||
cancel_check=cancel_check,
|
||||
)
|
||||
output_path = output_directory / f"foreground.{index + 1:04d}.png"
|
||||
output_path = output_directory / f"{output_prefix}.{index + 1:04d}.png"
|
||||
foreground.convert("RGBA").save(
|
||||
output_path,
|
||||
format="PNG",
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
from .depth import write_da3_depth, write_moge_depth
|
||||
from .surface import write_normal_texture, write_surface_archive, write_surface_texture
|
||||
from .surface import write_normal_texture, write_surface_archive
|
||||
|
||||
|
||||
__all__ = (
|
||||
@@ -7,5 +7,4 @@ __all__ = (
|
||||
"write_moge_depth",
|
||||
"write_normal_texture",
|
||||
"write_surface_archive",
|
||||
"write_surface_texture",
|
||||
)
|
||||
|
||||
@@ -20,15 +20,6 @@ def write_surface_archive(mesh, output_path):
|
||||
return output_path
|
||||
|
||||
|
||||
def write_surface_texture(image_path, output_path):
|
||||
from PIL import Image
|
||||
|
||||
output_path = Path(output_path)
|
||||
with Image.open(image_path) as image:
|
||||
image.convert("RGBA").save(output_path, format="PNG", compress_level=1)
|
||||
return output_path
|
||||
|
||||
|
||||
def write_normal_texture(normal, output_path):
|
||||
import numpy as np
|
||||
from PIL import Image
|
||||
|
||||
@@ -686,7 +686,7 @@ class AddonTest(unittest.TestCase):
|
||||
operator.source_object_name = "Reference"
|
||||
operator.remove_background = False
|
||||
operator.delete_source = True
|
||||
operator.unshaded = False
|
||||
operator.shadeless = False
|
||||
result = object()
|
||||
context = object()
|
||||
|
||||
@@ -701,7 +701,7 @@ class AddonTest(unittest.TestCase):
|
||||
source_object_name="Reference",
|
||||
remove_background=False,
|
||||
delete_source=True,
|
||||
unshaded=False,
|
||||
shadeless=False,
|
||||
)
|
||||
|
||||
def test_entry_operator_dispatches_plain_properties(self):
|
||||
@@ -709,7 +709,7 @@ class AddonTest(unittest.TestCase):
|
||||
operator.model = "MOGE2_VITS_NORMAL"
|
||||
operator.resolution_level = 7
|
||||
operator.keep_original = False
|
||||
operator.unshaded = True
|
||||
operator.shadeless = True
|
||||
operator.remove_background = False
|
||||
source_object = SimpleNamespace(
|
||||
name="Reference",
|
||||
|
||||
@@ -291,12 +291,12 @@ class ImageDepthTest(unittest.TestCase):
|
||||
"微信视频2026-07-23_192756_414",
|
||||
)
|
||||
|
||||
def test_removed_background_name_strips_video_extension(self):
|
||||
def test_color_name_strips_video_extension(self):
|
||||
source_object = SimpleNamespace(data=SimpleNamespace(name="clip.mp4"))
|
||||
|
||||
self.assertEqual(
|
||||
self.image_depth.removed_background_data_name(source_object),
|
||||
"clip.foreground",
|
||||
self.image_depth.color_data_name(source_object),
|
||||
"clip.color",
|
||||
)
|
||||
|
||||
def test_force_sequence_source_keeps_loaded_sequence(self):
|
||||
@@ -634,19 +634,19 @@ class ImageDepthTest(unittest.TestCase):
|
||||
|
||||
self.assertEqual(depth_image.name, "img.depth")
|
||||
|
||||
def test_removed_background_name_uses_source_image_base_name(self):
|
||||
def test_color_name_uses_source_image_base_name(self):
|
||||
source_object = SimpleNamespace(
|
||||
data=SimpleNamespace(name="img.png"),
|
||||
)
|
||||
|
||||
self.assertEqual(
|
||||
self.image_depth.removed_background_data_name(source_object),
|
||||
"img.foreground",
|
||||
self.image_depth.color_data_name(source_object),
|
||||
"img.color",
|
||||
)
|
||||
source_object.data.name = "img.png.001"
|
||||
self.assertEqual(
|
||||
self.image_depth.removed_background_data_name(source_object),
|
||||
"img.001.foreground",
|
||||
self.image_depth.color_data_name(source_object),
|
||||
"img.001.color",
|
||||
)
|
||||
|
||||
def test_surface_displacement_is_set_to_displacement_only(self):
|
||||
|
||||
@@ -376,7 +376,7 @@ class ServerTest(unittest.TestCase):
|
||||
)
|
||||
status = self.wait_for_job(context)
|
||||
self.assertTrue(
|
||||
(context.directory / "frame.0001.jpg").is_file()
|
||||
(context.directory / "color.0001.jpg").is_file()
|
||||
)
|
||||
|
||||
self.assertEqual(status["state"], "succeeded")
|
||||
|
||||
@@ -127,9 +127,24 @@ class WorkerTest(unittest.TestCase):
|
||||
cls.foreground_job = load_server_module("jobs.foreground")
|
||||
cls.download = load_server_module("model_download")
|
||||
|
||||
def test_surface_inference_progress_omits_single_frame_count(self):
|
||||
self.assertEqual(
|
||||
self.surface_job.inference_progress_message("MoGe-2", 0, 1),
|
||||
"Estimating MoGe-2 geometry",
|
||||
)
|
||||
self.assertEqual(
|
||||
self.surface_job.inference_progress_message("DA3", 0, 1),
|
||||
"Estimating DA3 geometry",
|
||||
)
|
||||
self.assertEqual(
|
||||
self.surface_job.inference_progress_message("DA3", 1, 3),
|
||||
"Estimating DA3 geometry (2/3)",
|
||||
)
|
||||
|
||||
def test_surface_has_one_model_independent_builder(self):
|
||||
self.assertTrue(callable(self.surface.build_surface))
|
||||
self.assertTrue(callable(self.surface_outputs.write_surface_archive))
|
||||
self.assertFalse(hasattr(self.surface_outputs, "write_surface_texture"))
|
||||
self.assertFalse(hasattr(self.surface, "export_da3_surface"))
|
||||
self.assertFalse(hasattr(self.surface, "export_moge2_surface"))
|
||||
|
||||
@@ -226,6 +241,33 @@ class WorkerTest(unittest.TestCase):
|
||||
)
|
||||
self.assertIsNone(temporary_directory)
|
||||
|
||||
def test_copy_color_images_numbers_a_still_image(self):
|
||||
with tempfile.TemporaryDirectory() as directory:
|
||||
root = Path(directory)
|
||||
source = root / "source.png"
|
||||
source.write_bytes(b"frame")
|
||||
output = root / "result"
|
||||
output.mkdir()
|
||||
|
||||
self.media.copy_color_images([source], output)
|
||||
|
||||
self.assertEqual(
|
||||
(output / "color.0001.png").read_bytes(),
|
||||
b"frame",
|
||||
)
|
||||
|
||||
def test_copy_frames_skips_a_still_image_by_default(self):
|
||||
with tempfile.TemporaryDirectory() as directory:
|
||||
root = Path(directory)
|
||||
source = root / "source.png"
|
||||
source.touch()
|
||||
output = root / "result"
|
||||
output.mkdir()
|
||||
|
||||
self.media.copy_frames([source], output)
|
||||
|
||||
self.assertEqual(list(output.iterdir()), [])
|
||||
|
||||
def test_extract_video_frames_reads_frames_sequentially(self):
|
||||
with tempfile.TemporaryDirectory() as directory:
|
||||
root = Path(directory)
|
||||
@@ -497,7 +539,7 @@ class WorkerTest(unittest.TestCase):
|
||||
{"input": str(input_path), "model": "BEN2", "device": "cpu"},
|
||||
)
|
||||
|
||||
self.assertTrue((root / "result" / "foreground.png").is_file())
|
||||
self.assertTrue((root / "result" / "color.0001.png").is_file())
|
||||
|
||||
self.assertEqual(
|
||||
inference_options[0][1],
|
||||
@@ -590,16 +632,10 @@ class WorkerTest(unittest.TestCase):
|
||||
|
||||
result_directory = context.directory
|
||||
self.assertTrue(
|
||||
(result_directory / "foreground.0001.png").is_file()
|
||||
(result_directory / "color.0001.png").is_file()
|
||||
)
|
||||
self.assertTrue(
|
||||
(result_directory / "foreground.0002.png").is_file()
|
||||
)
|
||||
self.assertTrue(
|
||||
(result_directory / "frame.0001.jpg").is_file()
|
||||
)
|
||||
self.assertTrue(
|
||||
(result_directory / "frame.0002.jpg").is_file()
|
||||
(result_directory / "color.0002.png").is_file()
|
||||
)
|
||||
|
||||
self.assertEqual(len(inference_options), 2)
|
||||
@@ -630,7 +666,7 @@ class WorkerTest(unittest.TestCase):
|
||||
for path in result_directory.iterdir()
|
||||
if path.is_file()
|
||||
),
|
||||
["frame.0001.jpg", "frame.0002.jpg", "frame.0003.jpg"],
|
||||
["color.0001.jpg", "color.0002.jpg", "color.0003.jpg"],
|
||||
)
|
||||
self.assertEqual(context.status["progress"], 1.0)
|
||||
self.assertIn("3 frame", context.status["message"])
|
||||
@@ -889,10 +925,10 @@ class WorkerTest(unittest.TestCase):
|
||||
|
||||
result_directory = context.directory
|
||||
self.assertTrue(
|
||||
(result_directory / "frame.0001.jpg").is_file()
|
||||
(result_directory / "color.0001.jpg").is_file()
|
||||
)
|
||||
self.assertTrue(
|
||||
(result_directory / "frame.0002.jpg").is_file()
|
||||
(result_directory / "color.0002.jpg").is_file()
|
||||
)
|
||||
self.assertEqual(len(inference_inputs), 2)
|
||||
self.assertTrue(all(len(paths) == 1 for paths in inference_inputs))
|
||||
@@ -1107,19 +1143,19 @@ class WorkerTest(unittest.TestCase):
|
||||
)
|
||||
np.testing.assert_array_equal(compact_faces, [[0, 1, 2]])
|
||||
|
||||
def test_surface_background_keeps_faces_with_any_foreground_vertex(self):
|
||||
def test_surface_background_requires_every_vertex_to_be_foreground(self):
|
||||
faces = np.asarray(
|
||||
((0, 1, 4), (0, 4, 3), (1, 2, 5), (1, 5, 4)),
|
||||
dtype=np.int32,
|
||||
)
|
||||
foreground = np.asarray(
|
||||
((False, False, False), (False, True, False)),
|
||||
((True, True, False), (False, True, False)),
|
||||
dtype=bool,
|
||||
)
|
||||
|
||||
filtered = self.geometry.remove_background_faces(faces, foreground)
|
||||
|
||||
np.testing.assert_array_equal(filtered, faces[[0, 1, 3]])
|
||||
np.testing.assert_array_equal(filtered, faces[[0]])
|
||||
|
||||
def test_surface_background_mask_keeps_any_nonzero_alpha(self):
|
||||
from PIL import Image
|
||||
@@ -1255,6 +1291,3 @@ class WorkerTest(unittest.TestCase):
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
|
||||
|
||||
|
||||
|
||||
@@ -7,6 +7,7 @@ GROUP_NAME = "O DA3 Surface"
|
||||
MOGE_SURFACE_GROUP_NAME = "O MoGe2 Surface"
|
||||
DEPTH_PLANE_GROUP_NAME = "O DA3 Depth Plane"
|
||||
MOGE_DEPTH_PLANE_GROUP_NAME = "O MoGe2 Depth Plane"
|
||||
SHADELESS_GROUP_NAME = "O Shadeless"
|
||||
MATERIAL_DISPLACEMENT_ATTRIBUTE = "o_material_displacement"
|
||||
BACKGROUND_DEPTH_ATTRIBUTE = "o_background_depth"
|
||||
DEPTH_SCALE_ATTRIBUTE = "o_depth_scale"
|
||||
@@ -800,9 +801,9 @@ def build_depth_plane_group():
|
||||
"NodeSocketImage",
|
||||
parent=data_panel,
|
||||
)
|
||||
foreground_image = new_socket(
|
||||
color_image = new_socket(
|
||||
interface,
|
||||
"Foreground Image",
|
||||
"Color Image",
|
||||
"INPUT",
|
||||
"NodeSocketImage",
|
||||
parent=data_panel,
|
||||
@@ -835,7 +836,7 @@ def build_depth_plane_group():
|
||||
socket.min_value = 1
|
||||
frame_data.append(socket)
|
||||
depth_image.hide_in_modifier = True
|
||||
foreground_image.hide_in_modifier = True
|
||||
color_image.hide_in_modifier = True
|
||||
material.hide_in_modifier = True
|
||||
for socket in frame_data:
|
||||
socket.hide_in_modifier = True
|
||||
@@ -848,7 +849,7 @@ def build_depth_plane_group():
|
||||
images_input = new_group_input(
|
||||
nodes,
|
||||
(100.0, 1250.0),
|
||||
{"Depth Image", "Foreground Image"},
|
||||
{"Depth Image", "Color Image"},
|
||||
)
|
||||
frames_input = new_group_input(
|
||||
nodes,
|
||||
@@ -973,7 +974,7 @@ def build_depth_plane_group():
|
||||
foreground_texture.location = (350.0, 900.0)
|
||||
links.new(images_input.outputs["Depth Image"], depth_texture.inputs["Image"])
|
||||
links.new(
|
||||
images_input.outputs["Foreground Image"],
|
||||
images_input.outputs["Color Image"],
|
||||
foreground_texture.inputs["Image"],
|
||||
)
|
||||
links.new(grid.outputs["UV Map"], depth_texture.inputs["Vector"])
|
||||
@@ -1141,10 +1142,13 @@ def build_moge_depth_plane_group():
|
||||
depth_image = new_socket(
|
||||
interface, "Depth Image", "INPUT", "NodeSocketImage", data_panel
|
||||
)
|
||||
color_image = new_socket(
|
||||
interface, "Color Image", "INPUT", "NodeSocketImage", data_panel
|
||||
)
|
||||
material = new_socket(
|
||||
interface, "Material", "INPUT", "NodeSocketMaterial", data_panel
|
||||
)
|
||||
for socket in (depth_image, material):
|
||||
for socket in (depth_image, color_image, material):
|
||||
socket.hide_in_modifier = True
|
||||
frame_sockets = []
|
||||
for name, socket_type, default in (
|
||||
@@ -1176,7 +1180,7 @@ def build_moge_depth_plane_group():
|
||||
background_depth_value.inputs[0],
|
||||
)
|
||||
images = new_group_input(
|
||||
nodes, (-500.0, 1100.0), {"Depth Image"}
|
||||
nodes, (-500.0, 1100.0), {"Depth Image", "Color Image"}
|
||||
)
|
||||
frames = new_group_input(
|
||||
nodes,
|
||||
@@ -1241,18 +1245,42 @@ def build_moge_depth_plane_group():
|
||||
depth_texture.interpolation = "Linear"
|
||||
depth_texture.extension = "EXTEND"
|
||||
depth_texture.location = (-250.0, 1100.0)
|
||||
color_texture = nodes.new("GeometryNodeImageTexture")
|
||||
color_texture.interpolation = "Closest"
|
||||
color_texture.extension = "EXTEND"
|
||||
color_texture.location = (-250.0, 850.0)
|
||||
links.new(images.outputs["Depth Image"], depth_texture.inputs["Image"])
|
||||
links.new(images.outputs["Color Image"], color_texture.inputs["Image"])
|
||||
links.new(grid.outputs["UV Map"], depth_texture.inputs["Vector"])
|
||||
links.new(grid.outputs["UV Map"], color_texture.inputs["Vector"])
|
||||
frame = build_sequence_frame(
|
||||
nodes, links, frames.outputs["Frame"], frames, 1800.0
|
||||
)
|
||||
links.new(frame, depth_texture.inputs["Frame"])
|
||||
links.new(frame, color_texture.inputs["Frame"])
|
||||
|
||||
clamp_depth = nodes.new("ShaderNodeMath")
|
||||
clamp_depth.operation = "MAXIMUM"
|
||||
clamp_depth.location = (0.0, 750.0)
|
||||
links.new(depth_texture.outputs["Color"], clamp_depth.inputs[0])
|
||||
links.new(background_depth_value.outputs[0], clamp_depth.inputs[1])
|
||||
color_mask = new_compare(
|
||||
nodes,
|
||||
"FLOAT",
|
||||
"GREATER_THAN",
|
||||
(0.0, 1000.0),
|
||||
0.5,
|
||||
)
|
||||
links.new(
|
||||
color_texture.outputs["Alpha"],
|
||||
input_socket(color_mask, "A", 0),
|
||||
)
|
||||
masked_depth = nodes.new("ShaderNodeMix")
|
||||
masked_depth.data_type = "FLOAT"
|
||||
masked_depth.location = (200.0, 850.0)
|
||||
links.new(color_mask.outputs["Result"], masked_depth.inputs[0])
|
||||
links.new(background_depth_value.outputs[0], masked_depth.inputs[2])
|
||||
links.new(clamp_depth.outputs[0], masked_depth.inputs[3])
|
||||
displacement_menu = nodes.new("GeometryNodeMenuSwitch")
|
||||
displacement_menu.data_type = "FLOAT"
|
||||
displacement_menu.enum_items.clear()
|
||||
@@ -1276,7 +1304,7 @@ def build_moge_depth_plane_group():
|
||||
scaled_depth = nodes.new("ShaderNodeMath")
|
||||
scaled_depth.operation = "MULTIPLY"
|
||||
scaled_depth.location = (250.0, 750.0)
|
||||
links.new(clamp_depth.outputs[0], scaled_depth.inputs[0])
|
||||
links.new(masked_depth.outputs["Result"], scaled_depth.inputs[0])
|
||||
links.new(geometry_strength.outputs[0], scaled_depth.inputs[1])
|
||||
normal = nodes.new("GeometryNodeInputNormal")
|
||||
normal.location = (0.0, 500.0)
|
||||
@@ -1320,6 +1348,80 @@ def build_moge_depth_plane_group():
|
||||
return group
|
||||
|
||||
|
||||
def build_shadeless_group():
|
||||
group = bpy.data.node_groups.new(SHADELESS_GROUP_NAME, "ShaderNodeTree")
|
||||
interface = group.interface
|
||||
new_socket(interface, "Shader", "OUTPUT", "NodeSocketShader")
|
||||
new_socket(interface, "Color", "INPUT", "NodeSocketColor")
|
||||
nodes = group.nodes
|
||||
links = group.links
|
||||
group_input = nodes.new("NodeGroupInput")
|
||||
group_input.location = (-900.0, -350.0)
|
||||
group_output = nodes.new("NodeGroupOutput")
|
||||
group_output.location = (850.0, 100.0)
|
||||
|
||||
diffuse = nodes.new("ShaderNodeBsdfDiffuse")
|
||||
diffuse.location = (250.0, -150.0)
|
||||
emission = nodes.new("ShaderNodeEmission")
|
||||
emission.location = (250.0, -400.0)
|
||||
links.new(group_input.outputs["Color"], diffuse.inputs["Color"])
|
||||
links.new(group_input.outputs["Color"], emission.inputs["Color"])
|
||||
|
||||
light_path = nodes.new("ShaderNodeLightPath")
|
||||
light_path.location = (-900.0, 150.0)
|
||||
bounce_count = nodes.new("ShaderNodeMath")
|
||||
bounce_count.operation = "SUBTRACT"
|
||||
bounce_count.label = "Bounce Count"
|
||||
bounce_count.location = (-500.0, -50.0)
|
||||
links.new(light_path.outputs["Ray Depth"], bounce_count.inputs[0])
|
||||
links.new(light_path.outputs["Transmission Depth"], bounce_count.inputs[1])
|
||||
refracted = nodes.new("ShaderNodeMath")
|
||||
refracted.operation = "SUBTRACT"
|
||||
refracted.label = "Camera or Refracted"
|
||||
refracted.inputs[0].default_value = 1.0
|
||||
refracted.location = (-250.0, -50.0)
|
||||
links.new(bounce_count.outputs[0], refracted.inputs[1])
|
||||
|
||||
reflection_limit = nodes.new("ShaderNodeMath")
|
||||
reflection_limit.operation = "SUBTRACT"
|
||||
reflection_limit.label = "Limit Reflections"
|
||||
reflection_limit.inputs[0].default_value = 2.0
|
||||
reflection_limit.location = (-650.0, 450.0)
|
||||
links.new(light_path.outputs["Ray Depth"], reflection_limit.inputs[1])
|
||||
camera_reflected = nodes.new("ShaderNodeMath")
|
||||
camera_reflected.operation = "MULTIPLY"
|
||||
camera_reflected.label = "Camera Ray to Glossy"
|
||||
camera_reflected.location = (-400.0, 400.0)
|
||||
links.new(reflection_limit.outputs[0], camera_reflected.inputs[0])
|
||||
links.new(light_path.outputs["Is Glossy Ray"], camera_reflected.inputs[1])
|
||||
shadow_or_reflect = nodes.new("ShaderNodeMath")
|
||||
shadow_or_reflect.operation = "MAXIMUM"
|
||||
shadow_or_reflect.label = "Shadow or Reflection?"
|
||||
shadow_or_reflect.location = (-100.0, 350.0)
|
||||
links.new(camera_reflected.outputs[0], shadow_or_reflect.inputs[0])
|
||||
links.new(light_path.outputs["Is Shadow Ray"], shadow_or_reflect.inputs[1])
|
||||
shader_factor = nodes.new("ShaderNodeMath")
|
||||
shader_factor.operation = "MAXIMUM"
|
||||
shader_factor.label = "Shadow, Reflect or Refract?"
|
||||
shader_factor.location = (150.0, 300.0)
|
||||
links.new(shadow_or_reflect.outputs[0], shader_factor.inputs[0])
|
||||
links.new(refracted.outputs[0], shader_factor.inputs[1])
|
||||
|
||||
mix = nodes.new("ShaderNodeMixShader")
|
||||
mix.location = (550.0, 100.0)
|
||||
links.new(shader_factor.outputs[0], mix.inputs["Fac"])
|
||||
links.new(diffuse.outputs["BSDF"], mix.inputs[1])
|
||||
links.new(emission.outputs["Emission"], mix.inputs[2])
|
||||
links.new(mix.outputs["Shader"], group_output.inputs["Shader"])
|
||||
group.color_tag = "SHADER"
|
||||
group.description = (
|
||||
"Display color without scene lighting while preserving shadows "
|
||||
"and reflections."
|
||||
)
|
||||
group.use_fake_user = True
|
||||
return group
|
||||
|
||||
|
||||
def main():
|
||||
bpy.ops.wm.read_factory_settings(use_empty=True)
|
||||
bpy.context.preferences.filepaths.save_version = 0
|
||||
@@ -1336,6 +1438,7 @@ def main():
|
||||
configure_moge_surface_group(moge_surface_group)
|
||||
build_depth_plane_group()
|
||||
build_moge_depth_plane_group()
|
||||
build_shadeless_group()
|
||||
OUTPUT_PATH.parent.mkdir(parents=True, exist_ok=True)
|
||||
bpy.ops.wm.save_as_mainfile(filepath=str(OUTPUT_PATH), check_existing=False)
|
||||
|
||||
|
||||
@@ -13,6 +13,8 @@ sys.path.insert(
|
||||
|
||||
from snapform.blender.depth_plane import (
|
||||
DEPTH_PLANE_MATERIAL_NODE_GROUP_NAME,
|
||||
MATERIAL_COLOR_SPACE,
|
||||
SHADELESS_NODE_GROUP_NAME,
|
||||
SURFACE_MATERIAL_NODE_GROUP_NAME,
|
||||
add_depth_plane_processing,
|
||||
create_material,
|
||||
@@ -133,7 +135,6 @@ def main():
|
||||
foreground,
|
||||
1.0,
|
||||
displacement_image=depth,
|
||||
foreground_image=foreground,
|
||||
)
|
||||
mesh.materials.append(material)
|
||||
material_group = bpy.data.node_groups[DEPTH_PLANE_MATERIAL_NODE_GROUP_NAME]
|
||||
@@ -232,6 +233,47 @@ def main():
|
||||
for node in material.node_tree.nodes
|
||||
if node.bl_idname == "ShaderNodeTexImage"
|
||||
)
|
||||
shadeless_material = create_material(
|
||||
"Shadeless",
|
||||
foreground,
|
||||
1.0,
|
||||
shadeless=True,
|
||||
)
|
||||
shadeless_group = bpy.data.node_groups[SHADELESS_NODE_GROUP_NAME]
|
||||
shadeless_nodes = [
|
||||
node
|
||||
for node in shadeless_material.node_tree.nodes
|
||||
if node.bl_idname == "ShaderNodeGroup"
|
||||
and node.node_tree == shadeless_group
|
||||
]
|
||||
assert len(shadeless_nodes) == 1
|
||||
assert not any(
|
||||
node.bl_idname == "ShaderNodeEmission"
|
||||
for node in shadeless_material.node_tree.nodes
|
||||
)
|
||||
assert [
|
||||
item.name
|
||||
for item in shadeless_group.interface.items_tree
|
||||
if item.item_type == "SOCKET" and item.in_out == "INPUT"
|
||||
] == ["Color"]
|
||||
assert [
|
||||
item.name
|
||||
for item in shadeless_group.interface.items_tree
|
||||
if item.item_type == "SOCKET" and item.in_out == "OUTPUT"
|
||||
] == ["Shader"]
|
||||
assert {
|
||||
node.label: node.operation
|
||||
for node in shadeless_group.nodes
|
||||
if node.bl_idname == "ShaderNodeMath"
|
||||
} == {
|
||||
"Bounce Count": "SUBTRACT",
|
||||
"Camera or Refracted": "SUBTRACT",
|
||||
"Limit Reflections": "SUBTRACT",
|
||||
"Camera Ray to Glossy": "MULTIPLY",
|
||||
"Shadow or Reflection?": "MAXIMUM",
|
||||
"Shadow, Reflect or Refract?": "MAXIMUM",
|
||||
}
|
||||
assert foreground.colorspace_settings.name == MATERIAL_COLOR_SPACE
|
||||
surface_material = create_material("Surface", foreground, 1.0)
|
||||
surface_group = bpy.data.node_groups[SURFACE_MATERIAL_NODE_GROUP_NAME]
|
||||
surface_group_nodes = [
|
||||
|
||||
@@ -1,6 +1,8 @@
|
||||
import sys
|
||||
import tempfile
|
||||
from pathlib import Path
|
||||
from types import SimpleNamespace
|
||||
from unittest.mock import patch
|
||||
|
||||
import bpy
|
||||
import numpy as np
|
||||
@@ -13,13 +15,81 @@ sys.path.insert(
|
||||
str(next((PROJECT_ROOT / "src" / "snapform" / "wheels").glob("blendjob-*.whl"))),
|
||||
)
|
||||
|
||||
import snapform.blender.depth_plane as depth_plane
|
||||
|
||||
from snapform.blender.depth_plane import (
|
||||
MOGE_DEPTH_PLANE_NODE_GROUP_NAME,
|
||||
MOGE_MATERIAL_NODE_GROUP_NAME,
|
||||
build_moge2_depth_plane,
|
||||
create_color_image,
|
||||
get_modifier_input,
|
||||
modifier_input_identifier,
|
||||
)
|
||||
|
||||
|
||||
def validate_single_frame_source_types():
|
||||
with tempfile.TemporaryDirectory() as temporary:
|
||||
directory = Path(temporary)
|
||||
color_path = directory / "color.0001.png"
|
||||
seed = bpy.data.images.new("Color Seed", width=2, height=2, alpha=True)
|
||||
seed.pixels[:] = [0.2, 0.4, 0.8, 1.0] * 4
|
||||
seed.filepath_raw = str(color_path)
|
||||
seed.file_format = "PNG"
|
||||
seed.save()
|
||||
bpy.data.images.remove(seed, do_unlink=True)
|
||||
|
||||
still_source_image = bpy.data.images.load(
|
||||
str(color_path), check_existing=False
|
||||
)
|
||||
still_source = bpy.data.objects.new("Still Source", None)
|
||||
still_source.empty_display_type = "IMAGE"
|
||||
still_source.data = still_source_image
|
||||
bpy.context.collection.objects.link(still_source)
|
||||
still_color = create_color_image(
|
||||
still_source, directory, assign_to_source=False
|
||||
)
|
||||
assert still_color.source == "FILE"
|
||||
|
||||
animated_source = SimpleNamespace(
|
||||
data=SimpleNamespace(
|
||||
source="SEQUENCE",
|
||||
name="Sequence Source.png",
|
||||
use_fake_user=False,
|
||||
),
|
||||
image_user=None,
|
||||
)
|
||||
sequence_color = create_color_image(
|
||||
animated_source, directory, assign_to_source=False
|
||||
)
|
||||
assert sequence_color.source == "SEQUENCE"
|
||||
|
||||
depth = np.ones((1, 2, 2), dtype=np.float32)
|
||||
mask = np.ones((1, 2, 2), dtype=bool)
|
||||
intrinsics = np.array(
|
||||
[[[1.0, 0.0, 0.5], [0.0, 1.0, 0.5], [0.0, 0.0, 1.0]]],
|
||||
dtype=np.float32,
|
||||
)
|
||||
path = directory / "single-frame-results.npz"
|
||||
np.savez(path, depth=depth, mask=mask, intrinsics=intrinsics)
|
||||
with patch.object(depth_plane, "is_animated_image", return_value=True):
|
||||
plane = build_moge2_depth_plane(
|
||||
bpy.context,
|
||||
still_source,
|
||||
path,
|
||||
delete_source=False,
|
||||
color_image=sequence_color,
|
||||
data_frames_directory=directory,
|
||||
)
|
||||
textures = [
|
||||
node.image
|
||||
for node in plane.data.materials[0].node_tree.nodes
|
||||
if node.bl_idname == "ShaderNodeTexImage"
|
||||
]
|
||||
assert sequence_color in textures
|
||||
depth_images = [image for image in textures if image != sequence_color]
|
||||
assert len(depth_images) == 1
|
||||
assert depth_images[0].source == "SEQUENCE"
|
||||
|
||||
def main():
|
||||
bpy.ops.wm.read_factory_settings(use_empty=True)
|
||||
image = bpy.data.images.new("MoGe Test", width=64, height=48, alpha=True)
|
||||
@@ -45,11 +115,14 @@ def main():
|
||||
mask=mask,
|
||||
intrinsics=intrinsics,
|
||||
)
|
||||
material_image = image.copy()
|
||||
material_image.name = "MoGe Test.color"
|
||||
plane = build_moge2_depth_plane(
|
||||
bpy.context,
|
||||
source,
|
||||
path,
|
||||
delete_source=True,
|
||||
color_image=material_image,
|
||||
)
|
||||
assert plane.modifiers[0].node_group.name == MOGE_DEPTH_PLANE_NODE_GROUP_NAME
|
||||
assert MOGE_MATERIAL_NODE_GROUP_NAME in bpy.data.node_groups
|
||||
@@ -73,6 +146,9 @@ def main():
|
||||
assert sockets["Background Depth"].structure_type == "SINGLE"
|
||||
assert sockets["Background Depth"].subtype == "DISTANCE"
|
||||
assert sockets["Background Depth"].default_value == 1.0
|
||||
assert sockets["Color Image"].hide_in_modifier
|
||||
color_identifier = modifier_input_identifier(geometry_group, "Color Image")
|
||||
assert get_modifier_input(plane.modifiers[0], color_identifier) == material_image
|
||||
material_group = bpy.data.node_groups[MOGE_MATERIAL_NODE_GROUP_NAME]
|
||||
assert not any(node.bl_idname == "ShaderNodeNormalMap" for node in material_group.nodes)
|
||||
assert any(node.bl_idname == "ShaderNodeBump" for node in material_group.nodes)
|
||||
@@ -84,14 +160,90 @@ def main():
|
||||
]
|
||||
assert image_textures
|
||||
assert all(node.extension == "EXTEND" for node in image_textures)
|
||||
assert any(node.image.name == "MoGe Test.depth" for node in image_textures)
|
||||
static_depth = next(
|
||||
node.image
|
||||
for node in image_textures
|
||||
if node.image.name == "MoGe Test.depth"
|
||||
)
|
||||
assert static_depth.source != "SEQUENCE"
|
||||
geometry_textures = [
|
||||
node
|
||||
for node in geometry_group.nodes
|
||||
if node.bl_idname == "GeometryNodeImageTexture"
|
||||
]
|
||||
assert geometry_textures
|
||||
assert len(geometry_textures) == 2
|
||||
assert all(node.extension == "EXTEND" for node in geometry_textures)
|
||||
color_texture = next(
|
||||
node
|
||||
for node in geometry_textures
|
||||
if any(
|
||||
link.to_node == node
|
||||
and link.to_socket == node.inputs["Image"]
|
||||
and link.from_socket.name == "Color Image"
|
||||
for link in geometry_group.links
|
||||
)
|
||||
)
|
||||
depth_texture = next(node for node in geometry_textures if node != color_texture)
|
||||
assert color_texture.interpolation == "Closest"
|
||||
assert depth_texture.interpolation == "Linear"
|
||||
color_frame_link = next(
|
||||
link
|
||||
for link in geometry_group.links
|
||||
if link.to_node == color_texture
|
||||
and link.to_socket == color_texture.inputs["Frame"]
|
||||
)
|
||||
depth_frame_link = next(
|
||||
link
|
||||
for link in geometry_group.links
|
||||
if link.to_node == depth_texture
|
||||
and link.to_socket == depth_texture.inputs["Frame"]
|
||||
)
|
||||
assert color_frame_link.from_socket == depth_frame_link.from_socket
|
||||
alpha_compare = next(
|
||||
node
|
||||
for node in geometry_group.nodes
|
||||
if node.bl_idname == "FunctionNodeCompare"
|
||||
and node.operation == "GREATER_THAN"
|
||||
and any(
|
||||
link.from_node == color_texture
|
||||
and link.from_socket == color_texture.outputs["Alpha"]
|
||||
and link.to_node == node
|
||||
for link in geometry_group.links
|
||||
)
|
||||
)
|
||||
masked_depth = next(
|
||||
node
|
||||
for node in geometry_group.nodes
|
||||
if node.bl_idname == "ShaderNodeMix"
|
||||
and any(
|
||||
link.from_node == alpha_compare and link.to_node == node
|
||||
for link in geometry_group.links
|
||||
)
|
||||
)
|
||||
assert masked_depth.data_type == "FLOAT"
|
||||
masked_sources = {
|
||||
link.from_node
|
||||
for link in geometry_group.links
|
||||
if link.to_node == masked_depth
|
||||
}
|
||||
assert alpha_compare in masked_sources
|
||||
assert any(
|
||||
node.bl_idname == "ShaderNodeMath" and node.operation == "MAXIMUM"
|
||||
for node in masked_sources
|
||||
)
|
||||
assert any(
|
||||
node.bl_idname == "ShaderNodeMath"
|
||||
and node.operation == "MULTIPLY"
|
||||
and node.inputs[1].default_value == -1.0
|
||||
for node in masked_sources
|
||||
)
|
||||
assert any(
|
||||
link.from_node == masked_depth
|
||||
and link.to_node.bl_idname == "ShaderNodeMath"
|
||||
and link.to_node.operation == "MULTIPLY"
|
||||
for link in geometry_group.links
|
||||
)
|
||||
validate_single_frame_source_types()
|
||||
print("MoGe-2 depth plane validation passed")
|
||||
|
||||
|
||||
|
||||
@@ -62,7 +62,7 @@ def main():
|
||||
)
|
||||
image = bpy.data.images.new("MoGe Surface Test", width=width, height=height)
|
||||
foreground = bpy.data.images.new(
|
||||
"MoGe Surface Test.foreground",
|
||||
"MoGe Surface Test.color",
|
||||
width=width,
|
||||
height=height,
|
||||
)
|
||||
@@ -197,7 +197,8 @@ def main():
|
||||
node for node in surface.data.materials[0].node_tree.nodes
|
||||
if node.bl_idname == "ShaderNodeTexImage" and node.image == foreground
|
||||
)
|
||||
assert color_texture.image.name == "MoGe Surface Test.foreground"
|
||||
assert color_texture.image.name == "MoGe Surface Test.color"
|
||||
assert color_texture.image.colorspace_settings.name == "AgX Base sRGB"
|
||||
normal_texture = next(
|
||||
node for node in surface.data.materials[0].node_tree.nodes
|
||||
if node.bl_idname == "ShaderNodeTexImage"
|
||||
@@ -220,6 +221,8 @@ def validate_real_result(results_path, image_path):
|
||||
normal_path = results_path.with_name("normal.png")
|
||||
if not normal_path.is_file():
|
||||
raise ValueError(f"Surface normal texture is missing: {normal_path}")
|
||||
color_image = bpy.data.images.load(str(image_path), check_existing=False)
|
||||
color_image.name = f"{image_path.stem}.color"
|
||||
surface = build_image_mesh(
|
||||
bpy.context,
|
||||
source,
|
||||
@@ -228,6 +231,7 @@ def validate_real_result(results_path, image_path):
|
||||
True,
|
||||
False,
|
||||
True,
|
||||
color_image=color_image,
|
||||
normal_path=normal_path,
|
||||
)
|
||||
assert len(surface.data.vertices) > 100
|
||||
|
||||
Reference in New Issue
Block a user