diff --git a/README.md b/README.md index 2c4366d..7a81def 100644 --- a/README.md +++ b/README.md @@ -42,15 +42,15 @@ DA3、BEN2 源码包和 uv 已内置在扩展里，安装时直接从本地解 ### 转换为透视网格 -选中单张图片的 Image Empty，右键打开 `SnapForm` 子菜单并选择 `Convert to Mesh`。SnapForm 会按 DA3 +选中单张图片的 Image Empty，右键打开 `SnapForm` 子菜单并选择 `Convert to Surface`。SnapForm 会按 DA3 相机内参与深度逐像素反投影，并把生成网格匹配到原 Empty 的可见尺寸、Offset 与 Transform。DA3 -confidence、边拉伸与 BEN2 背景结果会保存为 Mesh attributes，再由一个可复用的 Geometry Nodes -修改器完成剔面、快速重拓扑、Relax 与 UV 重投。匹配相机默认不创建；开启后会作为 Mesh 的子级。 +confidence、边拉伸与 BEN2 背景结果会保存为 Surface attributes，再由一个可复用的 Geometry Nodes +修改器完成剔面、快速重拓扑、Relax 与 UV 重投。匹配相机默认不创建；开启后会作为 Surface 的子级。 当前此功能仅支持单张图片。 转换对话框只保留模型设置、`Create Matching Camera`、`Unshaded` 与 `Delete Original`。剔除和重拓扑参数直接在 Geometry Nodes 修改器中调整；`Unshaded` 可使用与 Depth Plane 相同的 Emission 材质。普通材质的 -Bump Distance 会按透视网格的深度空间缩放。Mesh attributes 统一使用 `o_` 前缀；Host 准备过程显示 +Bump Scale 会在材质组内乘以 `0.001`，并按透视表面的深度空间缩放。Surface attributes 统一使用 `o_` 前缀；Host 准备过程显示 `Post-processing n/6`，可看到当前阶段与总步骤。 ### 视频与序列帧 diff --git a/docs/technical.md b/docs/technical.md index c9a9f92..030e28c 100644 --- a/docs/technical.md +++ b/docs/technical.md @@ -30,17 +30,16 @@ - Debug 与 Convert to Depth Plane 的模型选项直接显示下载状态；未下载模型带 `(Missing)`，已下载模型保持原名 - 分别调用 DA3 的 `glb`、`gs_ply`、`depth_vis` 和 `mini_npz` exporter -- `Textured Mesh (GLB)` 直接使用 DA3 Prediction 中的深度、置信度和相机参数，把相邻像素 +- `Textured Surface (GLB)` 直接使用 DA3 Prediction 中的深度、置信度和相机参数，把相邻像素 组成三角面并写入 `mesh.glb`；每个输入帧生成一张独立纹理网格，不做多视图融合 - Image Empty 右键菜单可直接运行 DA3，根据 `mini_npz` 深度生成面片 - Image Empty 右键菜单可直接运行 BEN2，保留原图并换用新建的透明背景图片 - Image Empty 支持视频与序列帧输入：`Convert to Depth Plane` 逐帧估计深度， `Remove Background` 逐帧抠图，入口保持不变 - 视频生成的序列帧以外部文件引用保留在 job 目录（Blender 无法整体 Pack 序列帧）； - Displace 修改器与材质 Image Texture 均可按场景帧播放序列帧 -- 使用浮点深度贴图、Simple Subdivision、Displace Modifier 和原图 Bump - 构建立体面片 -- 转换对话框可选择推理模式、处理分辨率和面片网格细分 + Geometry Nodes 与材质 Image Texture 均可按场景帧播放序列帧 +- 使用浮点深度贴图、`O DA3 Depth Plane` Geometry Nodes 和原图 Bump 构建立体面片 +- 转换对话框选择推理模式和处理分辨率；细分、背景与置换方式在 Geometry Nodes 修改器中调整 - Debug 的 DA3 输出或 BEN2 背景剔除完成后只打开对应 job 文件夹，不修改当前场景 - 完整保留每次任务生成的所有官方文件 - 安装和模型下载使用后台子进程；推理提交给常驻 AI Host，状态栏实时显示阶段、百分比进度条与取消按钮 @@ -111,27 +110,20 @@ Cyclic 沿用原 Empty 的设置。序列帧无法被 Blender 整体 Pack，因 `Host > Models` 下载所选模型，并固定使用 `process_res_method="upper_bound_resize"`；设备和参考视图策略 继续使用场景设置。Model 下拉菜单只列出已经下载的 DA3 模型；若没有可用模型，对话框会引导到 `Host > Models` 下载。 - - `Original`：直接把原图作为深度贴图，不显示估算参数。单张图片不启动 Runtime；视频或 - 序列帧会启动轻量抽帧任务，把帧抽出来作为 `.depth` 序列（比直接引用视频稳定）。 + - `Original`：直接把原图作为深度贴图，不显示估算参数。单张图片运行 BEN2 取得前景 Alpha；视频或 + 序列帧同时抽取原始帧与前景序列，把原始帧作为 `.depth` 序列。 `Original` 是默认深度来源。选择 `AI Generated` 时，其 Model、Use Ray Pose 与 Resolution Scale - 参数会集中显示在独立边框中。两种模式都有默认折叠的 Options，内部依次为 Displace via、 - Subdivision、Background Depth Distance 和 Unshaded；Delete Original 保持在折叠项外。 - Background Depth Distance 默认 `0`，表示不剔除背景。大于 `0` 时需要 `BEN2 Base`，并使用固定值 - 处理 alpha 背景。AI Generated 的 DA3 仍完全使用原图估算深度，推理后先把全部 depth 转换为 - `1 - DA3 depth`，再只把全透明区域设为 `-Background Depth Distance`；平面尺寸仍使用转换前的深度参考值， - 同时把比背景更远的前景值钳制到背景平面，并强制最外侧 1 像素为背景距离，以减少边缘墙面和毛刺； - 因此背景赋值不会缩放前景。Original 则把原图前景转为灰度深度，并把全透明区域设为该距离的负值， - 生成浮点 `.depth` 图片或 EXR 序列。两种模式都会使用 `.foreground` 作为材质颜色，不修改源 - Image Empty 的 Image datablock，也不根据每帧前景深度动态调整背景值。 + 参数会集中显示在独立边框中。两种模式都有默认折叠的 Options，内部只保留 `Unshaded`； + `Delete Original` 保持在折叠项外。转换固定需要 `BEN2 Base`，只把 `.foreground` 的 Alpha 作为独立 + Mask，不在 Host 中改写深度值。保存 `.foreground` 时保留 BEN2 Alpha，但 RGB 使用未经修改的原始输入； + 因此材质直接用 `.foreground` 同时提供颜色和 Mask，不再额外加载原图 Image Texture。 Image Empty 引用视频或序列帧时，AI Generated 会按 Empty 的 Image Frames 数值逐帧估计 深度（未设置时为完整帧数，显式设为 1 时只处理 1 帧），起始帧跟随 Empty 的 Offset，与背景蒙版范围一致； Offset 超出视频长度时同样把窗口收进视频内。材质节点与置换贴图的 Start 保持与原 Empty - 一致；Original 模式在 Distance 为 `0` 时，单帧复制原图作为 `.depth`，视频或序列帧则使用 - `frame.0001.jpg` 等原图序列。Distance 大于 `0` 时改为生成包含固定负背景值的浮点 `.depth` 或 - `depth.0001.exr` 序列。材质颜色在启用固定背景距离时使用 `foreground.0001.png` 序列。 + 一致；Original 模式的视频或序列帧使用 `frame.0001.jpg` 等原图序列作为颜色与深度， + `foreground.0001.png` 只提供前景 Alpha。 AI Generated 的深度仍以 EXR 序列帧（`depth.0001.exr` 起）保存在 job 目录根下。 - Displace via 可选 `Modifier`（默认）或 `Material`； - Unshaded 默认关闭；Subdivision 默认为 `5`，并默认删除原对象。 + `Unshaded` 默认关闭，并默认删除原对象。 3. AI Generated 完成后会自动创建面片。面片几何先由 `mini_npz` 的相机内参与深度确定，再通过对象 Transform 匹配原 Image Empty 的可见尺寸、位置、旋转和 Offset。Original 模式直接使用源 Empty 的可见尺寸、Offset 与 Transform。AI Generated 会把用于匹配画面尺寸的缩放预乘到 `.depth` 数值， @@ -139,23 +131,48 @@ Cyclic 沿用原 Empty 的设置。序列帧无法被 Blender 整体 Pack，因 开启 `Delete Original` 时，成功生成后 删除源 Empty。生成面片 Object 使用源 Empty 的原名；关闭 `Delete Original` 时不处理 同名冲突，由 Blender 自动为生成面片追加编号。 -4. 短边分段数为 `2^Subdivision`，长边分段数根据图像宽高比取最接近的整数。例如高:宽为 - `2:3`、Subdivision 为 `1` 时，直接生成 `2×3` 个面；Subdivision 为 `3` 时生成 - `8×12` 个面。随后的 Simple Subdivision 修改器使用原生名称，Viewport Levels 固定为 - `1`，Render Levels 固定为 `2`，不再随 Resolution Scale 或 Subdivision 动态变化。 +4. Blender 只创建保留图像纵横比的最低密度基础网格，再把 `O DA3 Depth Plane` 作为 Geometry Nodes + 修改器添加到对象。节点组与 `O Refine DA3 Surface` 一同保存在 `O_SnapForm.blend`，关闭 + `show_group_selector`，不为每个对象复制节点组。修改器公开以下参数： + - `Subdivision`：默认 `5`。节点读取基础平面的 Bounding Box，短边分段数为 `2^Subdivision`， + 长边按宽高比计算并四舍五入；例如 16:9 图片在等级 `5` 时生成 `57×32` 个面； + - `Remove Background`：默认关闭的 Panel Toggle。关闭时折叠隐藏 `Background Depth`，开启后才显示； + 它不删除几何，而是通过点域的 + `o_remove_background` Attribute 控制材质 Alpha 是否乘以前景 Mask； + - `Background Depth`：默认 `0`，允许正负值。只有开启 `Remove Background` 时，前景 Mask 外的 + 结构或材质置换才使用该值；关闭时所有像素继续使用原始深度； + - `Displacement`：菜单包含 `Geometry`（默认）与 `Material`。`Geometry` 在节点组内沿法线修改网格， + `Material` 保持网格平面并启用材质置换。 + - `Depth Scale`：默认 `1`，允许正负值，Structure Type 为 `Single`，同时控制 Geometry 与 Material + 两条置换路径。 + Grid 重建后通过隐藏的 Material 输入重新赋予原图材质，并对所有面执行 Shade Smooth。 + 材质使用显式指向 `UVMap` 的 UV Map 节点连接颜色、前景 Mask 与深度 Image Texture，不依赖活动 UV。 + 开启背景移除时，前景深度先与 `Background Depth` 取 Maximum，再按前景 Alpha 与背景深度混合， + 深度分支在混合前使用 `Alpha > 0.5` 转为二值 Mask，避免半透明前景被背景深度拉扯； + 材质表面 Alpha 仍保留柔和边缘。材质置换使用相同的二值和钳制顺序。 + 节点组还会把 `o_material_displacement` 与 `o_background_depth` 写入点域，材质通过 + `Geometry` 类型的 Attribute 节点读取。点域会为顶点保存重复值，但能在应用修改器后继续保留参数； + 相比自适应网格本身，其占用很小。Depth Image 与 Foreground Image 是修改器隐藏数据输入， + 由导入流程自动设置。 + 视频与序列帧还会为 Depth 与 Foreground 共同写入隐藏的 `Frame Start`、`Frames` 与 `Cyclic`。 + 导入内容已经是截取后的帧，因此 Offset 在节点组内恒定为 `0`，不再作为输入。节点组使用 Scene Time + 计算从 `1` 开始的 Image Texture Frame：场景帧与 Start 都为 `1` 时读取图片第 `1` 帧；非循环输入 + 钳制到有效范围；循环输入使用 Floored Modulo，让 `Frame Start` 前后的负、正相对帧都映射到 + `1…Frames`，因此开始帧之前也会循环播放。Geometry 置换会像材质 Image Texture 一样随场景帧刷新。 ## Image Empty 转透视网格 1. 在 3D View 选中仅包含一帧的 Image Empty，右键打开 `SnapForm` 子菜单并选择 - `Convert to Mesh`。该入口独立于 + `Convert to Surface`。该入口独立于 `Convert to Depth Plane`，当前不处理视频或序列帧。 2. Host 请求 DA3 的 `depth`、`conf` 与 `intrinsics`，同时固定运行 BEN2 取得前景 alpha。Depth 不做 smoothing，也不在 Host 中剔面或减面；Host 只反投影完整有效拓扑，并计算 confidence percentile、 camera-space edge stretch 与 background 标记。 -3. Blender 把勾选 `Usage > Modifier` 的 `O Refine DA3 Mesh` 节点组直接作为 Geometry Nodes 修改器 - 添加到 Mesh，不另设修改器名称。节点组内没有子节点组，依次完成： +3. Blender 把勾选 `Usage > Modifier` 的 `O Refine DA3 Surface` 节点组直接作为 Geometry Nodes 修改器 + 添加到 Surface，不另设修改器名称。节点组内没有子节点组，依次完成： - 修改器顶层依次显示 `Resolution`、`Relax Iterations`、`Smooth`；默认展开的 `Culling` Panel - 包含 `Confidence`、`Stretch`、`Background`。`Stretch = 100%` 明确选择全部面，剔面后同时删除 + 包含 `Confidence`、`Stretch`、`Background`。两个百分比输入都使用 `0–100` 的节点值，并在组内 + 转换为 `0–1` 阈值；`Stretch = 100%` 明确选择全部面，剔面后同时删除 Face Count 为零的孤立顶点； - 每轮 Remesh Iteration 依次执行 Connected Merge、循环拓扑修复与一次 Smooth，使新拓扑在进入下一轮 Merge 前先稳定；拓扑修复删除非流形面、单面拓扑和小面积 mesh islands； @@ -166,7 +183,7 @@ Cyclic 沿用原 Empty 的设置。序列帧无法被 Blender 整体 Pack，因 `Smooth` 程度系数；节点组不再额外执行 Flip Faces； - 从修改前的原始表面以 nearest-surface 重新采样 `UVMap`，最终明确写为 Corner 域 `2D Vector`。 4. 节点组随插件放在 `O_SnapForm.blend`。Blender 4.5 首次 Append 后复用同一 datablock； - Blender 5.0 以 linked ID 载入后调用 Pack，后续 Mesh 继续复用。插件不会为每个 Mesh 新建节点组， + Blender 5.0 以 linked ID 载入后调用 Pack，后续 Surface 继续复用。插件不会为每个 Surface 新建节点组， 也不会在添加后重命名 Geometry Nodes 修改器。节点组只启用 `Usage > Modifier`，不 Mark as Asset； 修改器关闭 `show_group_selector`。`Fix Topology` 位于默认折叠的 `Options` Panel。 节点图按 Culling、Remesh、Topology Fix、Island Cleanup 与 UV 分区摆放；各区就近使用独立的 @@ -175,8 +192,8 @@ Cyclic 沿用原 Empty 的设置。序列帧无法被 Blender 整体 Pack，因 拓扑、投影 UV/attributes、写出网格。状态栏显示 `Post-processing n/6: ...`。 6. Host 在 job 根目录写出 `mesh.npz` 与 `texture.png`。NPZ 保存 camera-space `vertices`、 `faces`、`uv`、`intrinsics`、`image_size` 与有效深度中位数 `reference_depth`。Blender 读取这些 - 数据后直接创建 Mesh 与原图材质，不需要 Open3D 或 OpenCV 进入 Blender Python 环境。 - Mesh 保留下列结构化 attributes： + 数据后直接创建 Surface 与原图材质，不需要 Open3D 或 OpenCV 进入 Blender Python 环境。 + Surface 保留下列结构化 attributes： - 点域：`o_depth`、`o_confidence`、`o_confidence_percentile`、`o_background`； - 面域：`o_face_confidence`、`o_edge_stretch`、`o_background_face`。 7. 网格以 `reference_depth` 截面匹配 Image Empty 的显示矩形：该截面的投影四角与 Empty 的可见 @@ -185,12 +202,13 @@ Cyclic 沿用原 Empty 的设置。序列帧无法被 Blender 整体 Pack，因 观察相机，启用 Backface Culling 时可从正面看到。 8. 转换对话框不再提供后处理参数，只保留模型设置、`Create Matching Camera`、`Unshaded` 与 `Delete Original`。`Unshaded` 使用与 Depth Plane 相同的 Emission/Transparent 材质路径；普通材质的 - Bump Distance 按 `abs(object.scale.z) × camera scale_z` 缩放，与透视网格实际深度尺度一致。 - 相机默认关闭；开启后会在 Empty 局部 `+Z` 侧创建一台朝向局部 `-Z` 的透视相机，并成为生成 Mesh + Bump Scale 按 `abs(object.scale.z) × camera scale_z` 设置，并在材质组内乘以固定基准 `0.001`， + 与透视表面实际深度尺度一致；它允许负数以反转凹凸方向。 + 相机默认关闭；开启后会在 Empty 局部 `+Z` 侧创建一台朝向局部 `-Z` 的透视相机，并成为生成 Surface 的子级，使二者移动时保持投影关系。开启 `Delete Original` 时，网格成功创建后删除源 Empty。 - 网格、材质与相机会归入同一个 `Convert to Mesh` Undo 步骤。 + 表面、材质与相机会归入同一个 `Convert to Surface` Undo 步骤。 -Debug 的 Textured Mesh GLB 会把 depth、confidence 与 confidence percentile 写入自定义顶点 +Debug 的 Textured Surface GLB 会把 depth、confidence 与 confidence percentile 写入自定义顶点 attributes。其他 DA3 Debug 三维输出会同时写出未压缩的 `prediction.attributes.npz`，保存完整 depth、confidence、intrinsics、extrinsics 与 confidence percentile。当前 trimesh 的 glTF 点图元 导出器只写 Position/Color，无法可靠把任意 attribute 嵌入点云 GLB，因此点云先以 companion NPZ @@ -224,16 +242,19 @@ Height = sqrt(Area / Aspect) `snapform_camera_plane_height`，便于检查实际采用的 参考深度和尺寸。 -选择 `Modifier` 时，AI Generated 的 Displace Modifier 深度贴图使用 `Extension = Extend`， -`Strength` 固定为 `1`、`Midlevel` 固定为 `0`。Original 把原图指定给 Displace Texture，并设置 -`Texture Coordinates = UV`、`Midlevel = 0`；其余修改器设置保留 Blender 默认值。 -视频与序列帧生成的深度贴图是 SEQUENCE Image；Displace 修改器的 Texture 与材质的 Image Texture -节点都支持序列帧，通过 image_user 的 Frames / Offset / Start 设置把第 i 帧映射到场景 -`Start+i` 帧（Start 沿用 Empty 的 Start），并启用 Auto Refresh，播放时随场景帧逐帧更新。 -选择 `Material` 时，不创建 Displace Modifier；深度 Image Texture 的 Color 连接到 Displacement -节点的 Height，再连接 Material Output 的 Displacement，并把材质 Surface Displacement 设置为 -`Displacement Only`。AI Generated 与 Original 都使用 `Midlevel = 0`、`Scale = 1`，与对应修改器值一致。 -无论选择哪种呈现方式，原始彩色贴图都会连接 +材质始终导入深度 Image Texture，并连接到 Displacement 节点和 Material Output；材质 Surface +Displacement 设置为 `Displacement Only`，AI Generated 与 Original 都使用 `Midlevel = 0`、`Scale = 1`。 +`o_material_displacement` 为 `False` 时，材质把 Height 乘以零，置换只在几何节点中生效；为 `True` 时， +几何节点关闭结构置换，材质 Height 恢复。开启 `Remove Background` 时，前景 Alpha 同时用于选择 +原始深度或有符号的 `Background Depth`，使两种置换路径保持一致；关闭时背景深度不参与计算。 +视频与序列帧的颜色、深度与前景 Image Texture +通过 image_user 的 Frames / Offset / Start 把第 i 帧映射到场景 `Start+i` 帧，并启用 Auto Refresh。 +颜色贴图到 Principled BSDF 之间使用独立的 Shader Node Group。Surface 使用 `O DA3 Surface`； +Depth Plane 使用显示名为 `O DA3 Depth Plane` 的材质组，并把背景剔除、背景深度与材质置换 Attribute +读取全部收进组内。两者都依次接收 `Color`、`Alpha`、Float 类型的 `Depth`、`Bump Scale`，输出 `Color`、 +`Roughness`、`Alpha`、`Normal`、`Displacement`。Surface 把原图 Color 同时送入 `Color` 与 `Depth`， +保持图像 Bump 效果。外层材质只保留贴图、材质组、Principled BSDF 与 Material Output，方便继续调整。 +无论选择哪种置换方式，原始彩色贴图都会连接 Principled BSDF 的 `Base Color`，并通过 `Bump.Height → Bump.Normal → Principled.Normal` 提供表面细节。Bump 的基础 Distance 为 `0.001`，会乘以面片 Transform 的统一缩放倍率。原图 Color 还会经过 `Float Curve` 连接到 @@ -241,21 +262,18 @@ Principled BSDF 的 `Roughness`，曲线两端输出分别为 `0.5` 和 `0.1`； Principled BSDF 默认值 `1.5`。 开启 `Unshaded` 后，原图 Color 连接 Emission，原图 Alpha 作为 Mix Shader 系数，在 Transparent BSDF 与 Emission 之间混合后连接 Material Output；不创建 Principled BSDF、Bump 或 -Float Curve。若 Displace via 选择 `Material`，深度贴图与 Displacement -节点仍会保留并连接 Material Output 的 Displacement。 +Float Curve。`Remove Background` 开启时，原图 Alpha 会再乘以前景 Mask Alpha；关闭时仍完整显示原图。 这里的“原始距离”是 DA3 模型输出的尺度。普通 Small、Base、Large 和 Giant 模型的深度不应 被解释为现实世界的米；需要现实尺度时应选择带 metric depth 的 `DA3 Nested Giant Large 1.1`。此外，当前结果仍是固定 XY 平面上的深度位移面，不是使用 -相机内参逐像素反投影得到的完整 camera-space 网格；中位深度截面为现有 Displace Modifier -工作流提供正确的整体高宽尺度。 +相机内参逐像素反投影得到的完整 camera-space 网格；中位深度截面为 Geometry Nodes +深度面片提供正确的整体高宽尺度。 生成完成后会直接创建面片，不提供缓存重建入口或单独的导入参数。重新选择 `Convert to Depth Plane` 会按当前 Depth from 再次转换。 -面片、深度 Image、Texture 与 Material 会在同一个 Blender Undo 步骤中创建；转换完成后执行 +面片、深度 Image、前景 Image、Material 与 Geometry Nodes 修改器会在同一个 Blender Undo 步骤中创建；转换完成后执行 `Ctrl+Z` 会完整撤销本次转换，不会越过转换去回退更早的 Image Empty 操作。 -Background Depth Distance 大于 `0` 时，`.foreground` Image、Original 浮点深度、Pack 和深度面片生成也属于 -同一个 Undo 步骤。 Image Empty 使用 Packed Image 时，SnapForm 会通过 Blender 写出临时 PNG 交给 DA3， 不会 Unpack 或修改源 Image；任务结束后自动删除临时文件。普通图片仍直接读取源文件。DA3 的原始 @@ -290,7 +308,6 @@ Blender 为 Packed Image 或序列帧生成的输入副本也归档到对应 job - `remove-background` - `remove-image-background` - `generate-original-depth-plane` -- `extract-original-frames` DA3 官方 exporter 产生的 `gs_ply/`、`depth_vis/`、`exports/mini_npz/` 会在完成后自动平铺， 不会保留只含一层产物的子目录： @@ -336,10 +353,10 @@ CC BY-NC 4.0，仅限非商业用途。 - Gaussian Splats (PLY)：DA3 `gs_ply`，同时传入 `infer_gs=True` - Depth Preview (Images)：DA3 `depth_vis` - Depth Data (NPZ)：DA3 `mini_npz` -- Textured Mesh (GLB)：SnapForm 根据 DA3 Prediction 构建并输出 `mesh.glb` +- Textured Surface (GLB)：SnapForm 根据 DA3 Prediction 构建并输出 `mesh.glb` Point Cloud 专属设置仅在对应类型下显示，`gs_views_interval` 仅用于 Gaussian Splats。 -Textured Mesh 使用 0–100% 的 Confidence Threshold 删除置信度最低的一定比例： +Textured Surface 使用 0–100% 的 Confidence Threshold 删除置信度最低的一定比例： - `process_res=504` - `process_res_method="upper_bound_resize"` diff --git a/src/snapform/assets/O_SnapForm.blend b/src/snapform/assets/O_SnapForm.blend index a7d1719..832e30f 100644 --- a/src/snapform/assets/O_SnapForm.blend +++ b/src/snapform/assets/O_SnapForm.blend @@ -1,3 +1,3 @@ version https://git-lfs.github.com/spec/v1 -oid sha256:59a1b7a1b2b07b04cab23c5ea4609dffb912d49b411d0798bb8a0852cb179b67 -size 695763 +oid sha256:59c0a27375f6a9efed02878cd82b70cd1e92a69253ac82bb90981fd18c410067 +size 950499 diff --git a/src/snapform/image_depth.py b/src/snapform/image_depth.py index 31ad165..322fa61 100644 --- a/src/snapform/image_depth.py +++ b/src/snapform/image_depth.py @@ -13,6 +13,14 @@ REFERENCE_DEPTH_KEY = "snapform_reference_depth" ORIGINAL_DISPLACEMENT_STRENGTH = 1.0 ORIGINAL_DISPLACEMENT_MIDLEVEL = 0.0 BUMP_BASE_DISTANCE = 0.001 +DEPTH_PLANE_NODE_GROUP_NAME = "O DA3 Depth Plane" +NODE_ASSET_NAME = "O_SnapForm.blend" +REMOVE_BACKGROUND_ATTRIBUTE = "o_remove_background" +MATERIAL_DISPLACEMENT_ATTRIBUTE = "o_material_displacement" +BACKGROUND_DEPTH_ATTRIBUTE = "o_background_depth" +DEPTH_SCALE_ATTRIBUTE = "o_depth_scale" +SURFACE_MATERIAL_NODE_GROUP_NAME = "O DA3 Surface" +DEPTH_PLANE_MATERIAL_NODE_GROUP_NAME = "O DA3 Depth Plane Material" IMAGE_NAME_SUFFIXES = ( ".jpeg", ".jpg", @@ -430,6 +438,7 @@ def create_removed_background_image( str(foreground_paths[0]), check_existing=False, ) + result_image.alpha_mode = "CHANNEL_PACKED" force_sequence_source(result_image) except (OSError, RuntimeError) as error: if "result_image" in locals(): @@ -471,6 +480,7 @@ def create_removed_background_image( str(output_path), check_existing=False, ) + result_image.alpha_mode = "CHANNEL_PACKED" except RuntimeError as error: raise RuntimeError("Unable to load the background result") from error source_image = source_object.data @@ -794,10 +804,13 @@ def create_plane_mesh(source_object, bounds=None, subdivision_level=5): x_min, x_max, y_min, y_max = ( bounds if bounds is not None else plane_bounds(source_object) ) - x_segments, y_segments = grid_segments( - (x_min, x_max, y_min, y_max), - subdivision_level, - ) + if subdivision_level is None: + x_segments, y_segments = 1, 1 + else: + x_segments, y_segments = grid_segments( + (x_min, x_max, y_min, y_max), + subdivision_level, + ) vertices = [] uv_coordinates = [] for y_index in range(y_segments + 1): @@ -836,11 +849,171 @@ def create_plane_mesh(source_object, bounds=None, subdivision_level=5): return mesh +def material_node_group(depth_plane=False): + group_name = ( + DEPTH_PLANE_MATERIAL_NODE_GROUP_NAME + if depth_plane + else SURFACE_MATERIAL_NODE_GROUP_NAME + ) + existing = bpy.data.node_groups.get(group_name) + if existing is not None: + return existing + group = bpy.data.node_groups.new(group_name, "ShaderNodeTree") + interface = group.interface + for name, socket_type in ( + ("Color", "NodeSocketColor"), + ("Alpha", "NodeSocketFloat"), + ("Depth", "NodeSocketFloat"), + ("Bump Scale", "NodeSocketFloat"), + ): + socket = interface.new_socket( + name=name, + in_out="INPUT", + socket_type=socket_type, + ) + if name == "Alpha": + socket.default_value = 1.0 + socket.min_value = 0.0 + socket.max_value = 1.0 + socket.subtype = "FACTOR" + elif name == "Bump Scale": + socket.default_value = 1.0 + socket.min_value = -10000.0 + socket.max_value = 10000.0 + for name, socket_type in ( + ("Color", "NodeSocketColor"), + ("Roughness", "NodeSocketFloat"), + ("Alpha", "NodeSocketFloat"), + ("Normal", "NodeSocketVector"), + ("Displacement", "NodeSocketVector"), + ): + interface.new_socket( + name=name, + in_out="OUTPUT", + socket_type=socket_type, + ) + + nodes = group.nodes + links = group.links + group_input = nodes.new("NodeGroupInput") + group_input.location = (-500.0, 100.0) + group_output = nodes.new("NodeGroupOutput") + group_output.location = (300.0, 100.0) + roughness_curve = nodes.new("ShaderNodeFloatCurve") + roughness_curve.mapping.initialize() + curve_points = roughness_curve.mapping.curves[0].points + curve_points[0].location = (0.0, 0.5) + curve_points[-1].location = (1.0, 0.1) + roughness_curve.mapping.update() + roughness_curve.location = (-250.0, 350.0) + bump = nodes.new("ShaderNodeBump") + bump.location = (-250.0, 0.0) + displacement = nodes.new("ShaderNodeDisplacement") + displacement.inputs["Midlevel"].default_value = 0.0 + displacement.inputs["Scale"].default_value = 1.0 + displacement.location = (-250.0, -300.0) + + processed_alpha = group_input.outputs["Alpha"] + processed_depth = group_input.outputs["Depth"] + if depth_plane: + group_input.location = (-1200.0, 100.0) + group_output.location = (1050.0, 100.0) + roughness_curve.location = (-300.0, 450.0) + bump.location = (750.0, 50.0) + displacement.location = (750.0, -300.0) + remove_attribute = nodes.new("ShaderNodeAttribute") + remove_attribute.attribute_type = "GEOMETRY" + remove_attribute.attribute_name = REMOVE_BACKGROUND_ATTRIBUTE + remove_attribute.location = (-950.0, 0.0) + background_attribute = nodes.new("ShaderNodeAttribute") + background_attribute.attribute_type = "GEOMETRY" + background_attribute.attribute_name = BACKGROUND_DEPTH_ATTRIBUTE + background_attribute.location = (-950.0, -500.0) + material_attribute = nodes.new("ShaderNodeAttribute") + material_attribute.attribute_type = "GEOMETRY" + material_attribute.attribute_name = MATERIAL_DISPLACEMENT_ATTRIBUTE + material_attribute.location = (-50.0, -650.0) + strength_attribute = nodes.new("ShaderNodeAttribute") + strength_attribute.attribute_type = "GEOMETRY" + strength_attribute.attribute_name = DEPTH_SCALE_ATTRIBUTE + strength_attribute.location = (200.0, -650.0) + alpha_mix = nodes.new("ShaderNodeMix") + alpha_mix.data_type = "FLOAT" + alpha_mix.inputs[2].default_value = 1.0 + alpha_mix.location = (-650.0, 50.0) + foreground_mask = nodes.new("ShaderNodeMath") + foreground_mask.operation = "GREATER_THAN" + foreground_mask.inputs[1].default_value = 0.5 + foreground_mask.location = (-950.0, -250.0) + clamp_foreground = nodes.new("ShaderNodeMath") + clamp_foreground.operation = "MAXIMUM" + clamp_foreground.location = (-650.0, -400.0) + masked_depth = nodes.new("ShaderNodeMix") + masked_depth.data_type = "FLOAT" + masked_depth.location = (-350.0, -350.0) + background_mix = nodes.new("ShaderNodeMix") + background_mix.data_type = "FLOAT" + background_mix.location = (-50.0, -350.0) + enabled_depth = nodes.new("ShaderNodeMath") + enabled_depth.operation = "MULTIPLY" + enabled_depth.location = (250.0, -350.0) + scaled_depth = nodes.new("ShaderNodeMath") + scaled_depth.operation = "MULTIPLY" + scaled_depth.location = (450.0, -350.0) + links.new(remove_attribute.outputs["Fac"], alpha_mix.inputs[0]) + links.new(group_input.outputs["Alpha"], alpha_mix.inputs[3]) + links.new(group_input.outputs["Alpha"], foreground_mask.inputs[0]) + links.new(group_input.outputs["Depth"], clamp_foreground.inputs[0]) + links.new(background_attribute.outputs["Fac"], clamp_foreground.inputs[1]) + links.new(foreground_mask.outputs[0], masked_depth.inputs[0]) + links.new(background_attribute.outputs["Fac"], masked_depth.inputs[2]) + links.new(clamp_foreground.outputs[0], masked_depth.inputs[3]) + links.new(remove_attribute.outputs["Fac"], background_mix.inputs[0]) + links.new(group_input.outputs["Depth"], background_mix.inputs[2]) + links.new(masked_depth.outputs["Result"], background_mix.inputs[3]) + links.new(background_mix.outputs["Result"], enabled_depth.inputs[0]) + links.new(material_attribute.outputs["Fac"], enabled_depth.inputs[1]) + links.new(enabled_depth.outputs[0], scaled_depth.inputs[0]) + links.new(strength_attribute.outputs["Fac"], scaled_depth.inputs[1]) + processed_alpha = alpha_mix.outputs["Result"] + processed_depth = scaled_depth.outputs[0] + + links.new(group_input.outputs["Color"], group_output.inputs["Color"]) + links.new(group_input.outputs["Color"], roughness_curve.inputs["Value"]) + links.new( + roughness_curve.outputs["Value"], + group_output.inputs["Roughness"], + ) + links.new(processed_alpha, group_output.inputs["Alpha"]) + links.new(processed_depth, bump.inputs["Height"]) + bump_distance = nodes.new("ShaderNodeMath") + bump_distance.operation = "MULTIPLY" + bump_distance.inputs[1].default_value = BUMP_BASE_DISTANCE + bump_distance.location = (500.0, 100.0) if depth_plane else (-250.0, 150.0) + links.new(group_input.outputs["Bump Scale"], bump_distance.inputs[0]) + links.new(bump_distance.outputs[0], bump.inputs["Distance"]) + links.new(bump.outputs["Normal"], group_output.inputs["Normal"]) + links.new(processed_depth, displacement.inputs["Height"]) + links.new( + displacement.outputs["Displacement"], + group_output.inputs["Displacement"], + ) + group.color_tag = "SHADER" + group.description = ( + "Prepare DA3 depth-plane material outputs." + if depth_plane + else "Prepare DA3 surface material outputs." + ) + group.use_fake_user = True + return group + + def create_material( name, color_image, depth_scale, displacement_image=None, + foreground_image=None, displacement_midlevel=0.0, displacement_scale=1.0, unshaded=False, @@ -848,6 +1021,8 @@ def create_material( frame_start=1, color_frame_offset=0, color_frame_duration=None, + foreground_frame_offset=0, + foreground_frame_duration=None, depth_frame_offset=0, depth_frame_duration=None, use_cyclic=None, @@ -860,13 +1035,17 @@ def create_material( nodes.clear() color_texture = nodes.new("ShaderNodeTexImage") + uv_map = nodes.new("ShaderNodeUVMap") output = nodes.new("ShaderNodeOutputMaterial") - color_texture.image = color_image + uv_map.uv_map = "UVMap" + uv_map.location = (-1250.0, 350.0) + material_image = foreground_image or color_image + color_texture.image = material_image color_texture.interpolation = "Linear" color_texture.extension = "EXTEND" - color_texture.location = (-640.0, 180.0) - output.location = (460.0, 180.0) + color_texture.location = (-1000.0, 350.0) + output.location = (850.0, 350.0) extracted_frames = frame_count if frame_count > 1 else None color_frames = ( color_frame_duration @@ -878,29 +1057,51 @@ def create_material( if depth_frame_duration is not None else extracted_frames ) - if frame_count > 1 or is_animated_image(color_image): + material_frames = color_frames + material_frame_offset = color_frame_offset + if foreground_image is not None: + material_frames = ( + foreground_frame_duration + if foreground_frame_duration is not None + else extracted_frames + ) + material_frame_offset = foreground_frame_offset + if frame_count > 1 or is_animated_image(material_image): configure_image_user( color_texture.image_user, - color_frames, + material_frames, frame_start=frame_start, - frame_offset=color_frame_offset, + frame_offset=material_frame_offset, use_cyclic=use_cyclic, ) links = material.node_tree.links + links.new(uv_map.outputs["UV"], color_texture.inputs["Vector"]) + alpha = color_texture.outputs["Alpha"] + material_inputs = nodes.new("ShaderNodeGroup") + is_depth_plane = displacement_image is not None + material_inputs.node_tree = material_node_group(is_depth_plane) + material_inputs.name = ( + "O DA3 Depth Plane" if is_depth_plane else "O DA3 Surface" + ) + material_inputs.label = material_inputs.name + material_inputs.location = (-300.0, 350.0) + material_inputs.inputs["Bump Scale"].default_value = depth_scale + links.new(color_texture.outputs["Color"], material_inputs.inputs["Color"]) + links.new(alpha, material_inputs.inputs["Alpha"]) if unshaded: emission = nodes.new("ShaderNodeEmission") transparent = nodes.new("ShaderNodeBsdfTransparent") mix_shader = nodes.new("ShaderNodeMixShader") - emission.location = (-100.0, 260.0) - transparent.location = (-100.0, 20.0) - mix_shader.location = (200.0, 180.0) + emission.location = (0.0, 550.0) + transparent.location = (0.0, 150.0) + mix_shader.location = (300.0, 350.0) links.new( - color_texture.outputs["Color"], + material_inputs.outputs["Color"], emission.inputs["Color"], ) links.new( - color_texture.outputs["Alpha"], + material_inputs.outputs["Alpha"], mix_shader.inputs["Fac"], ) links.new( @@ -917,39 +1118,22 @@ def create_material( ) else: shader = nodes.new("ShaderNodeBsdfPrincipled") - bump = nodes.new("ShaderNodeBump") - roughness_curve = nodes.new("ShaderNodeFloatCurve") - bump.inputs["Distance"].default_value = BUMP_BASE_DISTANCE * depth_scale - roughness_curve.mapping.initialize() - curve_points = roughness_curve.mapping.curves[0].points - curve_points[0].location = (0.0, 0.5) - curve_points[-1].location = (1.0, 0.1) - roughness_curve.mapping.update() - - roughness_curve.location = (-300.0, 320.0) - bump.location = (-300.0, -80.0) - shader.location = (80.0, 180.0) + shader.location = (250.0, 400.0) links.new( - color_texture.outputs["Color"], + material_inputs.outputs["Color"], shader.inputs["Base Color"], ) - links.new(color_texture.outputs["Alpha"], shader.inputs["Alpha"]) - links.new(color_texture.outputs["Color"], bump.inputs["Height"]) - links.new(bump.outputs["Normal"], shader.inputs["Normal"]) + links.new(material_inputs.outputs["Alpha"], shader.inputs["Alpha"]) + links.new(material_inputs.outputs["Normal"], shader.inputs["Normal"]) links.new( - color_texture.outputs["Color"], - roughness_curve.inputs["Value"], - ) - links.new( - roughness_curve.outputs["Value"], + material_inputs.outputs["Roughness"], shader.inputs["Roughness"], ) links.new(shader.outputs["BSDF"], output.inputs["Surface"]) if displacement_image is not None: depth_texture = nodes.new("ShaderNodeTexImage") - displacement = nodes.new("ShaderNodeDisplacement") depth_texture.image = displacement_image depth_texture.interpolation = "Linear" depth_texture.extension = "EXTEND" @@ -961,21 +1145,30 @@ def create_material( frame_offset=depth_frame_offset, use_cyclic=use_cyclic, ) - displacement.inputs["Midlevel"].default_value = ( - displacement_midlevel - ) - displacement.inputs["Scale"].default_value = displacement_scale - depth_texture.location = (-300.0, -360.0) - displacement.location = (80.0, -260.0) + depth_texture.location = (-1000.0, -400.0) + links.new(uv_map.outputs["UV"], depth_texture.inputs["Vector"]) + displacement_height = depth_texture.outputs["Color"] + if displacement_midlevel != 0.0 or displacement_scale != 1.0: + depth_adjust = nodes.new("ShaderNodeMath") + depth_adjust.operation = "SUBTRACT" + depth_adjust.inputs[1].default_value = displacement_midlevel + depth_adjust.location = (500.0, -550.0) + depth_scale_node = nodes.new("ShaderNodeMath") + depth_scale_node.operation = "MULTIPLY" + depth_scale_node.inputs[1].default_value = displacement_scale + depth_scale_node.location = (700.0, -550.0) + links.new(displacement_height, depth_adjust.inputs[0]) + links.new(depth_adjust.outputs[0], depth_scale_node.inputs[0]) + links.new(depth_scale_node.outputs[0], material_inputs.inputs["Depth"]) + else: + links.new(displacement_height, material_inputs.inputs["Depth"]) links.new( - depth_texture.outputs["Color"], - displacement.inputs["Height"], - ) - links.new( - displacement.outputs["Displacement"], + material_inputs.outputs["Displacement"], output.inputs["Displacement"], ) set_displacement_only(material) + else: + links.new(color_texture.outputs["Color"], material_inputs.inputs["Depth"]) return material @@ -990,6 +1183,84 @@ def set_displacement_only(material): return +def node_asset_path(): + return Path(__file__).with_name("assets") / NODE_ASSET_NAME + + +def load_depth_plane_node_group(): + existing = bpy.data.node_groups.get(DEPTH_PLANE_NODE_GROUP_NAME) + if existing is not None: + return existing + asset_path = node_asset_path() + if not asset_path.is_file(): + raise RuntimeError(f"Missing Geometry Nodes asset: {asset_path}") + use_pack = bpy.app.version >= (5, 0, 0) + if not use_pack: + directory = str(asset_path) + "/NodeTree/" + result = bpy.ops.wm.append( + filepath=directory + DEPTH_PLANE_NODE_GROUP_NAME, + directory=directory, + filename=DEPTH_PLANE_NODE_GROUP_NAME, + link=False, + do_reuse_local_id=True, + ) + if "FINISHED" not in result: + raise RuntimeError("Unable to append the Geometry Nodes asset") + node_group = bpy.data.node_groups.get(DEPTH_PLANE_NODE_GROUP_NAME) + if node_group is None: + raise RuntimeError("Appended Geometry Nodes asset is unavailable") + node_group.use_fake_user = True + return node_group + with bpy.data.libraries.load(str(asset_path), link=True) as (source, target): + if DEPTH_PLANE_NODE_GROUP_NAME not in source.node_groups: + raise RuntimeError( + "Geometry Nodes asset is missing " + f"{DEPTH_PLANE_NODE_GROUP_NAME}" + ) + target.node_groups = [DEPTH_PLANE_NODE_GROUP_NAME] + linked_group = target.node_groups[0] + node_group = bpy.data.pack_linked_ids_hierarchy(root_id=linked_group) + if linked_group != node_group and linked_group.users == 0: + bpy.data.node_groups.remove(linked_group) + node_group.use_fake_user = True + return node_group + + +def modifier_input_identifier(node_group, name): + for item in node_group.interface.items_tree: + if item.item_type != "SOCKET" or item.in_out != "INPUT": + continue + if item.name == name: + return item.identifier + raise RuntimeError(f"Geometry Nodes input is missing: {name}") + + +def add_depth_plane_processing( + plane_object, + depth_image, + foreground_image, + material, + frame_start=1, + frame_duration=1, + use_cyclic=False, +): + node_group = load_depth_plane_node_group() + modifier = plane_object.modifiers.new(name=node_group.name, type="NODES") + modifier.node_group = node_group + modifier.show_group_selector = False + for name, value in ( + ("Depth Image", depth_image), + ("Foreground Image", foreground_image), + ("Material", material), + ("Frame Start", frame_start), + ("Frames", max(int(frame_duration or 1), 1)), + ("Cyclic", use_cyclic), + ): + identifier = modifier_input_identifier(node_group, name) + modifier[identifier] = value + return modifier + + def finish_depth_plane( context, source_object, @@ -1010,11 +1281,10 @@ def finish_depth_plane( def build_image_depth_plane( context, source_object, - subdivision_level, delete_source, - displacement_via="MODIFIER", unshaded=False, color_image=None, + foreground_image=None, depth_frames_directory=None, depth_image=None, ): @@ -1046,48 +1316,21 @@ def build_image_depth_plane( depth_source_image = frames_image elif not is_animated_image(source_object.data): depth_source_image = copy_depth_image(source_object) - mesh = create_plane_mesh( - source_object, - subdivision_level=subdivision_level, + foreground_image = ( + foreground_image or color_image or source_object.data ) + mesh = create_plane_mesh(source_object, subdivision_level=None) plane_object = bpy.data.objects.new(source_name, mesh) context.collection.objects.link(plane_object) plane_object.matrix_world = source_object.matrix_world.copy() - subdivision = plane_object.modifiers.new("Subdivision", "SUBSURF") - subdivision.subdivision_type = "SIMPLE" - subdivision.levels = 1 - subdivision.render_levels = 2 - - if displacement_via == "MODIFIER": - texture = bpy.data.textures.new( - depth_data_name(source_object), - type="IMAGE", - ) - texture.image = depth_source_image - texture.extension = "EXTEND" - if uses_extracted or is_animated_image(source_object.data): - configure_image_user( - texture.image_user, - texture_frame_count, - frame_start=source_frame_start, - frame_offset=texture_frame_offset, - use_cyclic=source_use_cyclic, - ) - displace = plane_object.modifiers.new("Displace", "DISPLACE") - displace.texture = texture - displace.texture_coords = "UV" - displace.mid_level = ORIGINAL_DISPLACEMENT_MIDLEVEL - depth_scale = abs(float(plane_object.scale.z)) - displacement_image = ( - depth_source_image if displacement_via == "MATERIAL" else None - ) material = create_material( source_name, color_image if color_image is not None else source_object.data, depth_scale, - displacement_image=displacement_image, + displacement_image=depth_source_image, + foreground_image=foreground_image, displacement_midlevel=ORIGINAL_DISPLACEMENT_MIDLEVEL, displacement_scale=ORIGINAL_DISPLACEMENT_STRENGTH, unshaded=unshaded, @@ -1095,11 +1338,22 @@ def build_image_depth_plane( frame_start=source_frame_start, color_frame_offset=texture_frame_offset, color_frame_duration=texture_frame_count, + foreground_frame_offset=0, + foreground_frame_duration=frame_count, depth_frame_offset=texture_frame_offset, depth_frame_duration=texture_frame_count, use_cyclic=source_use_cyclic, ) mesh.materials.append(material) + add_depth_plane_processing( + plane_object, + depth_source_image, + foreground_image, + material, + frame_start=source_frame_start, + frame_duration=frame_count, + use_cyclic=source_use_cyclic, + ) plane_object["snapform_frame_count"] = frame_count return finish_depth_plane( context, @@ -1113,12 +1367,11 @@ def build_depth_plane( context, source_object, npz_path, - subdivision_level, delete_source, - displacement_via="MODIFIER", unshaded=False, depth_name="", color_image=None, + foreground_image=None, depth_frames_directory=None, ): source_name = source_object.name @@ -1136,13 +1389,16 @@ def build_depth_plane( ) depth_name = depth_name or depth_data_name(source_object) frame_count = len(depth_arrays) + foreground_image = ( + foreground_image or color_image or source_object.data + ) ( source_frame_start, source_frame_offset, source_frame_duration, source_use_cyclic, ) = image_user_settings(source_object) - mesh = create_plane_mesh(source_object, bounds, subdivision_level) + mesh = create_plane_mesh(source_object, bounds, None) plane_object = bpy.data.objects.new( source_name, mesh, @@ -1170,39 +1426,12 @@ def build_depth_plane( depth_frames_directory, ) - subdivision = plane_object.modifiers.new("Subdivision", "SUBSURF") - subdivision.subdivision_type = "SIMPLE" - subdivision.levels = 1 - subdivision.render_levels = 2 - - if displacement_via == "MODIFIER": - texture = bpy.data.textures.new( - depth_name, - type="IMAGE", - ) - texture.image = depth_image - texture.extension = "EXTEND" - if frame_count > 1: - configure_image_user( - texture.image_user, - frame_count, - frame_start=source_frame_start, - use_cyclic=source_use_cyclic, - ) - displace = plane_object.modifiers.new("Displace", "DISPLACE") - displace.texture = texture - displace.texture_coords = "UV" - displace.direction = "NORMAL" - displace.strength = DISPLACEMENT_STRENGTH - displace.mid_level = DISPLACEMENT_MIDLEVEL - material = create_material( source_name, color_image if color_image is not None else source_object.data, depth_scale, - displacement_image=( - depth_image if displacement_via == "MATERIAL" else None - ), + displacement_image=depth_image, + foreground_image=foreground_image, displacement_midlevel=DISPLACEMENT_MIDLEVEL, displacement_scale=DISPLACEMENT_STRENGTH, unshaded=unshaded, @@ -1212,11 +1441,22 @@ def build_depth_plane( color_frame_duration=( frame_count if color_image is not None else source_frame_duration ), + foreground_frame_offset=0, + foreground_frame_duration=frame_count, depth_frame_offset=0, depth_frame_duration=frame_count, use_cyclic=source_use_cyclic, ) mesh.materials.append(material) + add_depth_plane_processing( + plane_object, + depth_image, + foreground_image, + material, + frame_start=source_frame_start, + frame_duration=frame_count, + use_cyclic=source_use_cyclic, + ) plane_object["snapform_reference_depth"] = reference_depth plane_object["snapform_camera_plane_width"] = bounds[1] - bounds[0] plane_object["snapform_camera_plane_height"] = bounds[3] - bounds[2] @@ -1254,24 +1494,10 @@ class BuildDepthPlane(bpy.types.Operator): snapform_depth_frames_dir: bpy.props.StringProperty( options={"HIDDEN", "SKIP_SAVE"}, ) - snapform_subdivision_level: bpy.props.IntProperty( - default=5, - min=0, - max=10, - options={"HIDDEN", "SKIP_SAVE"}, - ) snapform_delete_source: bpy.props.BoolProperty( default=True, options={"HIDDEN", "SKIP_SAVE"}, ) - snapform_displacement_via: bpy.props.EnumProperty( - items=( - ("MODIFIER", "Modifier", "Use a Displace modifier"), - ("MATERIAL", "Material", "Use material displacement"), - ), - default="MODIFIER", - options={"HIDDEN", "SKIP_SAVE"}, - ) snapform_unshaded: bpy.props.BoolProperty( default=False, options={"HIDDEN", "SKIP_SAVE"}, @@ -1295,14 +1521,15 @@ class BuildDepthPlane(bpy.types.Operator): return {"CANCELLED"} try: - color_image = None + foreground_image = None if self.snapform_background_path: - color_image = create_removed_background_image( + foreground_image = create_removed_background_image( source_object, self.snapform_background_path, assign_to_source=False, ) - elif self.snapform_color_frames_dir: + color_image = None + if self.snapform_color_frames_dir: color_image = create_sequence_image( f"{image_base_name(source_object.data)}.frame", self.snapform_color_frames_dir, @@ -1311,12 +1538,11 @@ class BuildDepthPlane(bpy.types.Operator): context, source_object, npz_path, - self.snapform_subdivision_level, self.snapform_delete_source, - displacement_via=self.snapform_displacement_via, unshaded=self.snapform_unshaded, depth_name=self.snapform_depth_data_name, color_image=color_image, + foreground_image=foreground_image, depth_frames_directory=( self.snapform_depth_frames_dir or None ), @@ -1347,24 +1573,10 @@ class BuildOriginalDepthPlane(bpy.types.Operator): snapform_depth_npz_path: bpy.props.StringProperty( options={"HIDDEN", "SKIP_SAVE"}, ) - snapform_subdivision_level: bpy.props.IntProperty( - default=5, - min=0, - max=10, - options={"HIDDEN", "SKIP_SAVE"}, - ) snapform_delete_source: bpy.props.BoolProperty( default=True, options={"HIDDEN", "SKIP_SAVE"}, ) - snapform_displacement_via: bpy.props.EnumProperty( - items=( - ("MODIFIER", "Modifier", "Use a Displace modifier"), - ("MATERIAL", "Material", "Use material displacement"), - ), - default="MODIFIER", - options={"HIDDEN", "SKIP_SAVE"}, - ) snapform_unshaded: bpy.props.BoolProperty( default=False, options={"HIDDEN", "SKIP_SAVE"}, @@ -1379,14 +1591,15 @@ class BuildOriginalDepthPlane(bpy.types.Operator): ) return {"CANCELLED"} try: - color_image = None + foreground_image = None if self.snapform_background_path: - color_image = create_removed_background_image( + foreground_image = create_removed_background_image( source_object, self.snapform_background_path, assign_to_source=False, ) - elif self.snapform_color_frames_dir: + color_image = None + if self.snapform_color_frames_dir: color_image = create_sequence_image( f"{image_base_name(source_object.data)}.frame", self.snapform_color_frames_dir, @@ -1411,11 +1624,10 @@ class BuildOriginalDepthPlane(bpy.types.Operator): build_image_depth_plane( context, source_object, - self.snapform_subdivision_level, self.snapform_delete_source, - displacement_via=self.snapform_displacement_via, unshaded=self.snapform_unshaded, color_image=color_image, + foreground_image=foreground_image, depth_frames_directory=( self.snapform_depth_frames_dir or None ), @@ -1527,27 +1739,11 @@ class ConvertImageToDepthPlane(bpy.types.Operator): ), default="ORIGINAL", ) - snapform_displacement_via: bpy.props.EnumProperty( - name="Displace via", - description="How the depth displacement is presented", - items=( - ("MODIFIER", "Modifier", "Use a Displace modifier"), - ("MATERIAL", "Material", "Use material displacement"), - ), - default="MODIFIER", - ) snapform_unshaded: bpy.props.BoolProperty( name="Unshaded", description="Display the source color through an Emission shader", default=False, ) - snapform_background_depth_distance: bpy.props.FloatProperty( - name="Background Depth Distance", - description="Set excluded background to this fixed depth distance; zero disables background exclusion", - default=0.0, - min=0.0, - soft_max=10.0, - ) snapform_model: bpy.props.EnumProperty( name="Model", description="Model used to estimate image depth", @@ -1572,24 +1768,19 @@ class ConvertImageToDepthPlane(bpy.types.Operator): description="Delete the source Image Empty after conversion", default=True, ) - snapform_subdivision_level: bpy.props.IntProperty( - name="Subdivision", - description="Power of two for the short-edge segment count; the long edge follows the image aspect ratio", - default=5, - min=0, - max=10, - ) + @classmethod def poll(cls, context): from .operators import active_job from .runtime import host_busy, model_ready - from .models import MODELS + from .models import BEN2_MODEL, MODELS return ( is_image_empty(context.object) and active_job() is None and not host_busy() and any(model_ready(model_key) for model_key in MODELS) + and model_ready(BEN2_MODEL.key) ) def invoke(self, context, _event): @@ -1608,9 +1799,6 @@ class ConvertImageToDepthPlane(bpy.types.Operator): return context.window_manager.invoke_props_dialog(self, width=340) def draw(self, context): - from .models import BEN2_MODEL - from .runtime import model_ready - layout = self.layout layout.prop(self, "snapform_depth_source") if self.snapform_depth_source == "ESTIMATE": @@ -1637,14 +1825,6 @@ class ConvertImageToDepthPlane(bpy.types.Operator): ) settings_header.label(text="Options") if settings_body is not None: - settings_body.prop(self, "snapform_displacement_via") - settings_body.prop(self, "snapform_subdivision_level") - remove_background_row = settings_body.row() - remove_background_row.enabled = model_ready(BEN2_MODEL.key) - remove_background_row.prop( - self, - "snapform_background_depth_distance", - ) settings_body.prop(self, "snapform_unshaded") layout.prop(self, "snapform_delete_original") @@ -1652,7 +1832,6 @@ class ConvertImageToDepthPlane(bpy.types.Operator): from .operators import ( IMAGE_DEPTH_JOB, ORIGINAL_BACKGROUND_JOB, - ORIGINAL_FRAMES_JOB, ) source_object = context.object @@ -1660,57 +1839,6 @@ class ConvertImageToDepthPlane(bpy.types.Operator): context.scene.snapform_settings.video_frames = ( default_extract_frames(source_object) ) - if ( - self.snapform_depth_source == "ORIGINAL" - and self.snapform_background_depth_distance <= 0.0 - ): - if video_total_frames(source_object.data) > 1: - try: - source_path, delete_input = prepare_image_input( - source_object.data, - context.scene, - ) - except RuntimeError as error: - self.report({"ERROR"}, str(error)) - return {"CANCELLED"} - result = bpy.ops.snapform.run_job( - job_type=ORIGINAL_FRAMES_JOB, - snapform_input_path=str(source_path), - snapform_source_object_name=source_object.name, - snapform_subdivision_level=self.snapform_subdivision_level, - snapform_delete_source=self.snapform_delete_original, - snapform_delete_input=delete_input, - snapform_displacement_via=self.snapform_displacement_via, - snapform_unshaded=self.snapform_unshaded, - ) - if "CANCELLED" not in result: - return {"FINISHED"} - if delete_input: - remove_temporary_input(source_path) - return {"CANCELLED"} - try: - build_image_depth_plane( - context, - source_object, - self.snapform_subdivision_level, - self.snapform_delete_original, - self.snapform_displacement_via, - self.snapform_unshaded, - ) - except (OSError, RuntimeError, ValueError) as error: - self.report({"ERROR"}, str(error)) - return {"CANCELLED"} - undo_result = bpy.ops.ed.undo_push( - message="Convert to Depth Plane", - ) - if "FINISHED" not in undo_result: - self.report( - {"ERROR"}, - "Unable to create the depth plane Undo step", - ) - return {"CANCELLED"} - return {"FINISHED"} - if self.snapform_depth_source == "ORIGINAL": try: source_path, delete_input = prepare_image_input( @@ -1724,14 +1852,9 @@ class ConvertImageToDepthPlane(bpy.types.Operator): job_type=ORIGINAL_BACKGROUND_JOB, snapform_input_path=str(source_path), snapform_source_object_name=source_object.name, - snapform_subdivision_level=self.snapform_subdivision_level, snapform_delete_source=self.snapform_delete_original, snapform_delete_input=delete_input, - snapform_displacement_via=self.snapform_displacement_via, snapform_unshaded=self.snapform_unshaded, - snapform_background_depth_distance=( - self.snapform_background_depth_distance - ), ) if "CANCELLED" not in result: return {"FINISHED"} @@ -1765,14 +1888,10 @@ class ConvertImageToDepthPlane(bpy.types.Operator): snapform_model_key=self.snapform_model, snapform_use_ray_pose=self.snapform_use_ray_pose, snapform_process_res=process_res, - snapform_subdivision_level=self.snapform_subdivision_level, snapform_delete_source=self.snapform_delete_original, snapform_delete_input=delete_input, - snapform_displacement_via=self.snapform_displacement_via, snapform_unshaded=self.snapform_unshaded, - snapform_background_depth_distance=( - self.snapform_background_depth_distance - ), + snapform_remove_background=True, ) if "CANCELLED" in result: if delete_input: @@ -1797,7 +1916,7 @@ class SnapFormImageMenu(bpy.types.Menu): ) layout.operator( "snapform.convert_image_to_mesh", - text="Convert to Mesh", + text="Convert to Surface", icon="MESH_GRID", ) layout.separator() diff --git a/src/snapform/image_mesh.py b/src/snapform/image_mesh.py index 44be6ed..59b5f82 100644 --- a/src/snapform/image_mesh.py +++ b/src/snapform/image_mesh.py @@ -33,7 +33,7 @@ FACE_ATTRIBUTES = { "o_edge_stretch": ("FLOAT", "face_edge_stretch"), "o_background_face": ("BOOLEAN", "face_background"), } -NODE_GROUP_NAME = "O Refine DA3 Mesh" +NODE_GROUP_NAME = "O Refine DA3 Surface" NODE_ASSET_NAME = "O_SnapForm.blend" @@ -60,7 +60,7 @@ def load_image_mesh_data(npz_path): missing = [key for key in required if key not in archive.files] if missing: raise ValueError( - f"Mesh data is missing: {', '.join(missing)}" + f"Surface data is missing: {', '.join(missing)}" ) vertices = np.asarray(archive["vertices"], dtype=np.float32) faces = np.asarray(archive["faces"], dtype=np.int32) @@ -81,19 +81,19 @@ def load_image_mesh_data(npz_path): }.items() } except OSError as error: - raise ValueError(f"Unable to read mesh data: {npz_path}") from error + raise ValueError(f"Unable to read surface data: {npz_path}") from error if vertices.ndim != 2 or vertices.shape[1] != 3: raise ValueError(f"Unsupported vertices shape: {vertices.shape}") if faces.ndim != 2 or faces.shape[1] != 3: raise ValueError(f"Unsupported faces shape: {faces.shape}") if uv.shape != (len(vertices), 2): - raise ValueError("Mesh UV and vertex counts do not match") + raise ValueError("Surface UV and vertex counts do not match") if intrinsics.shape != (3, 3): raise ValueError(f"Unsupported intrinsics shape: {intrinsics.shape}") if len(image_size) != 2 or min(image_size) <= 0: raise ValueError(f"Unsupported image size: {image_size}") if not reference_depth > 0.0: - raise ValueError("Mesh reference depth must be positive") + raise ValueError("Surface reference depth must be positive") for _name, (_data_type, key) in POINT_ATTRIBUTES.items(): if len(attributes[key]) != len(vertices): raise ValueError(f"Point attribute has the wrong length: {key}") @@ -394,7 +394,7 @@ def build_image_mesh( class BuildImageMesh(bpy.types.Operator): bl_idname = "snapform.build_image_mesh" - bl_label = "Build Image Mesh" + bl_label = "Build Image Surface" bl_options = {"INTERNAL"} snapform_source_object_name: bpy.props.StringProperty( @@ -438,8 +438,8 @@ class BuildImageMesh(bpy.types.Operator): class ConvertImageToMesh(bpy.types.Operator): bl_idname = "snapform.convert_image_to_mesh" - bl_label = "Convert to Mesh" - bl_description = "Convert an Image Empty into a perspective textured mesh" + bl_label = "Convert to Surface" + bl_description = "Convert an Image Empty into a perspective textured surface" bl_options = {"REGISTER"} snapform_model: bpy.props.EnumProperty( @@ -529,7 +529,7 @@ class ConvertImageToMesh(bpy.types.Operator): source_object = context.object if video_total_frames(source_object.data) != 1: - self.report({"ERROR"}, "Convert to Mesh currently supports still images") + self.report({"ERROR"}, "Convert to Surface currently supports still images") return {"CANCELLED"} try: process_res = scaled_process_resolution( diff --git a/src/snapform/operators.py b/src/snapform/operators.py index c276c9c..93af2d0 100644 --- a/src/snapform/operators.py +++ b/src/snapform/operators.py @@ -73,12 +73,9 @@ class ActiveJob: source_object_name: str = "" depth_data_name: str = "" process_res: int = IMAGE_DEPTH_DEFAULT_PROCESS_RES - subdivision_level: int = 5 delete_source: bool = True - displacement_via: str = "MODIFIER" unshaded: bool = False remove_background: bool = False - background_depth_distance: float = 0.0 mesh_create_camera: bool = False temporary_input_path: Path | None = None cancelled: bool = False @@ -141,7 +138,7 @@ class RunJob(bpy.types.Operator): ("DOWNLOAD_MODEL", "Download Model", ""), (RECONSTRUCTION_JOB, "Export DA3 Result", ""), (IMAGE_DEPTH_JOB, "Convert Image to Depth Plane", ""), - (IMAGE_MESH_JOB, "Convert Image to Mesh", ""), + (IMAGE_MESH_JOB, "Convert Image to Surface", ""), (BACKGROUND_REMOVAL_JOB, "Test BEN2 Background Removal", ""), (IMAGE_BACKGROUND_JOB, "Remove Image Empty Background", ""), (ORIGINAL_BACKGROUND_JOB, "Build Original Background Plane", ""), @@ -183,29 +180,10 @@ class RunJob(bpy.types.Operator): default=False, options={"HIDDEN", "SKIP_SAVE"}, ) - snapform_subdivision_level: bpy.props.IntProperty( - default=5, - min=0, - max=10, - options={"HIDDEN", "SKIP_SAVE"}, - ) - snapform_displacement_via: bpy.props.EnumProperty( - items=( - ("MODIFIER", "Modifier", ""), - ("MATERIAL", "Material", ""), - ), - default="MODIFIER", - options={"HIDDEN", "SKIP_SAVE"}, - ) snapform_unshaded: bpy.props.BoolProperty( default=False, options={"HIDDEN", "SKIP_SAVE"}, ) - snapform_background_depth_distance: bpy.props.FloatProperty( - default=0.0, - min=0.0, - options={"HIDDEN", "SKIP_SAVE"}, - ) snapform_mesh_create_camera: bpy.props.BoolProperty( default=False, options={"HIDDEN", "SKIP_SAVE"}, @@ -214,6 +192,7 @@ class RunJob(bpy.types.Operator): default=False, options={"HIDDEN", "SKIP_SAVE"}, ) + def execute(self, context): global _active_job @@ -251,15 +230,11 @@ class RunJob(bpy.types.Operator): source_object_name=self.snapform_source_object_name, depth_data_name=self.snapform_depth_data_name, process_res=self.snapform_process_res, - subdivision_level=self.snapform_subdivision_level, delete_source=self.snapform_delete_source, - displacement_via=self.snapform_displacement_via, unshaded=self.snapform_unshaded, - remove_background=( - self.snapform_background_depth_distance > 0.0 - or bool(getattr(self, "snapform_remove_background", False)) + remove_background=bool( + getattr(self, "snapform_remove_background", False) ), - background_depth_distance=self.snapform_background_depth_distance, mesh_create_camera=getattr( self, "snapform_mesh_create_camera", @@ -350,10 +325,9 @@ class RunJob(bpy.types.Operator): ) self.snapform_input_path = str(archived_input) self.snapform_delete_input = False - background_depth_distance = float( - getattr(self, "snapform_background_depth_distance", 0.0) + remove_background = bool( + getattr(self, "snapform_remove_background", False) ) - remove_background = background_depth_distance > 0.0 if self.job_type == IMAGE_MESH_JOB: remove_background = True if self.job_type == "INSTALL_RUNTIME": @@ -413,11 +387,6 @@ class RunJob(bpy.types.Operator): BEN2_MODEL.key, start_frame=start_frame, device=self._task_device(settings), - background_depth_distance=( - background_depth_distance - if self.job_type == ORIGINAL_BACKGROUND_JOB - else 0.0 - ), ), paths, ) @@ -483,11 +452,6 @@ class RunJob(bpy.types.Operator): self.job_type in {IMAGE_DEPTH_JOB, IMAGE_MESH_JOB} and remove_background ), - background_depth_distance=( - background_depth_distance - if self.job_type == IMAGE_DEPTH_JOB - else 0.0 - ), device=self._task_device(settings), ), paths, @@ -520,7 +484,7 @@ class RunJob(bpy.types.Operator): if self.job_type == IMAGE_DEPTH_JOB: return "Starting depth estimation" if self.job_type == IMAGE_MESH_JOB: - return "Starting perspective mesh generation" + return "Starting perspective surface generation" if self.job_type == BACKGROUND_REMOVAL_JOB: return "Starting BEN2 background removal" if self.job_type == IMAGE_BACKGROUND_JOB: @@ -696,19 +660,19 @@ class RunJob(bpy.types.Operator): status["message"] = "Depth plane is ready" if job.kind == IMAGE_MESH_JOB: try: - settings.status = "Building perspective mesh" + settings.status = "Building perspective surface" settings.progress = 0.96 self._redraw_ui(context, force=True) self._build_image_mesh(job) except (OSError, RuntimeError, ValueError) as import_error: - settings.status = "Mesh import failed" + settings.status = "Surface import failed" settings.progress = 0.0 self.report({"ERROR"}, str(import_error)) runtime.mark_host_job_complete(job.status_path.stem) close_active_job() self._redraw_ui(context, force=True) return {"CANCELLED"} - status["message"] = "Perspective mesh is ready" + status["message"] = "Perspective surface is ready" if job.kind in {ORIGINAL_BACKGROUND_JOB, ORIGINAL_FRAMES_JOB}: try: settings.status = "Building depth plane" @@ -761,10 +725,10 @@ class RunJob(bpy.types.Operator): snapform_unshaded=job.unshaded, ) if "FINISHED" not in result: - raise RuntimeError("Unable to build the perspective mesh") - undo_result = bpy.ops.ed.undo_push(message="Convert to Mesh") + raise RuntimeError("Unable to build the perspective surface") + undo_result = bpy.ops.ed.undo_push(message="Convert to Surface") if "FINISHED" not in undo_result: - raise RuntimeError("Unable to create the mesh Undo step") + raise RuntimeError("Unable to create the surface Undo step") def _build_depth_plane(self, context, job): result_directory = job.result_directory @@ -784,18 +748,14 @@ class RunJob(bpy.types.Operator): snapform_depth_data_name=job.depth_data_name, snapform_background_path=background_path, snapform_color_frames_dir=( - str(result_directory) - if is_sequence and not foreground_paths - else "" + str(result_directory) if is_sequence else "" ), snapform_depth_frames_dir=( str(result_directory) if is_sequence else "" ), - snapform_subdivision_level=job.subdivision_level, snapform_delete_source=job.delete_source, - snapform_displacement_via=job.displacement_via, snapform_unshaded=job.unshaded, ) if "FINISHED" not in result: @@ -815,7 +775,7 @@ class RunJob(bpy.types.Operator): if is_sequence else result_directory / "foreground.png" ) - elif is_sequence: + if is_sequence: color_frames_dir = str(result_directory) result = bpy.ops.snapform.build_original_depth_plane( snapform_source_object_name=job.source_object_name, @@ -824,14 +784,8 @@ class RunJob(bpy.types.Operator): snapform_depth_frames_dir=( str(result_directory) if is_sequence else "" ), - snapform_depth_npz_path=( - str(result_directory / "original-depth.npz") - if job.kind == ORIGINAL_BACKGROUND_JOB - else "" - ), - snapform_subdivision_level=job.subdivision_level, + snapform_depth_npz_path="", snapform_delete_source=job.delete_source, - snapform_displacement_via=job.displacement_via, snapform_unshaded=job.unshaded, ) if "FINISHED" not in result: diff --git a/src/snapform/panel.py b/src/snapform/panel.py index 2df3ef8..6a64b55 100644 --- a/src/snapform/panel.py +++ b/src/snapform/panel.py @@ -216,7 +216,7 @@ class Da3OutputDebugPanel(bpy.types.Panel): ply_box.prop(settings, "gs_views_interval") elif settings.export_format == "mesh": mesh_box = layout.box() - mesh_box.label(text="Mesh Settings") + mesh_box.label(text="Surface Settings") mesh_box.prop(settings, "conf_thresh_percentile") mesh_box.prop(settings, "mesh_filter_edges") mesh_box.prop(settings, "mesh_max_edge_length") diff --git a/src/snapform/properties.py b/src/snapform/properties.py index 43c834b..f5ba62e 100644 --- a/src/snapform/properties.py +++ b/src/snapform/properties.py @@ -67,7 +67,7 @@ class SnapFormSettings(bpy.types.PropertyGroup): ) conf_thresh_percentile: bpy.props.FloatProperty( name="Confidence Threshold", - description="Remove the lowest-confidence percentile of points or mesh faces", + description="Remove the lowest-confidence percentile of points or surface faces", default=40.0, min=0.0, max=100.0, @@ -135,8 +135,8 @@ class SnapFormSettings(bpy.types.PropertyGroup): ), ( "mesh", - "Textured Mesh (GLB)", - "Generate one filtered textured depth mesh per input image", + "Textured Surface (GLB)", + "Generate one filtered textured depth surface per input image", ), ), default="glb", diff --git a/src/snapform/runtime.py b/src/snapform/runtime.py index 4aca9cf..6f41061 100644 --- a/src/snapform/runtime.py +++ b/src/snapform/runtime.py @@ -748,7 +748,6 @@ def remove_background_request( video_frames=None, start_frame=1, device=None, - background_depth_distance=0.0, ): return HostJobRequest( { @@ -766,9 +765,6 @@ def remove_background_request( else video_frames ), "start_frame": start_frame, - "background_depth_distance": float( - background_depth_distance - ), }, } ) @@ -812,7 +808,6 @@ def reconstruct_request( process_res_method=None, use_ray_pose=None, remove_background=False, - background_depth_distance=0.0, mesh_filter_edges=None, start_frame=1, device=None, @@ -851,7 +846,6 @@ def reconstruct_request( "use_ray_pose": 1 if snapform_selected_ray_pose else 0, "ref_view_strategy": settings.ref_view_strategy, "remove_background_model_dir": "", - "background_depth_distance": float(background_depth_distance), "mesh_filter_edges": bool( getattr(settings, "mesh_filter_edges", True) if mesh_filter_edges is None @@ -861,7 +855,7 @@ def reconstruct_request( getattr(settings, "mesh_max_edge_length", 0.1) ), } - if remove_background or background_depth_distance > 0.0: + if remove_background: parameters["remove_background_model_dir"] = str( model_directory(BEN2_MODEL.key) ) diff --git a/src/snapform/worker.py b/src/snapform/worker.py index 7f95c22..8b61a32 100644 --- a/src/snapform/worker.py +++ b/src/snapform/worker.py @@ -48,64 +48,6 @@ def check_cancelled(cancel_check): raise TaskCancelled("Task cancelled") -def apply_foreground_depth_distance( - prediction, - foreground_paths, - background_depth_distance, -): - """Set fully transparent pixels to a fixed negative displacement.""" - import numpy as np - from PIL import Image - - if len(foreground_paths) != len(prediction.depth): - raise RuntimeError("Foreground mask count does not match depth frames") - masks = [] - height, width = prediction.depth.shape[-2:] - for foreground_path in foreground_paths: - with Image.open(foreground_path) as source: - alpha = source.convert("RGBA").getchannel("A") - alpha = alpha.resize((width, height), resample=Image.Resampling.NEAREST) - masks.append(np.asarray(alpha) > 0) - apply_depth_background_array( - prediction, - np.stack(masks), - background_depth_distance, - ) - - -def apply_depth_background_array( - prediction, - foreground_mask, - background_depth_distance, -): - import numpy as np - - depth = np.asarray(prediction.depth) - masked_depth = depth.copy() - background_depth = -background_depth_distance - for frame_index in range(len(masked_depth)): - np.maximum( - masked_depth[frame_index], - background_depth, - out=masked_depth[frame_index], - ) - masked_depth[frame_index][~foreground_mask[frame_index]] = ( - background_depth - ) - set_depth_border(masked_depth[frame_index], background_depth) - prediction.depth = masked_depth - - -def set_depth_border(depth, background_depth): - """Set the outermost pixel ring to the fixed background depth.""" - if depth.ndim != 2 or not depth.size: - raise ValueError("Depth frame must be a non-empty 2D array") - depth[0, :] = background_depth - depth[-1, :] = background_depth - depth[:, 0] = background_depth - depth[:, -1] = background_depth - - def convert_ai_depth_for_displacement(prediction): """Convert DA3 distance into Blender's positive displacement direction.""" import numpy as np @@ -137,55 +79,6 @@ def write_npz_reference_depth(npz_path, reference_depth): temporary_path.replace(npz_path) -def write_original_background_depth( - image_paths, - foreground_paths, - background_depth_distance, - output_directory, -): - """Write original-image luminance with a fixed negative background depth.""" - import numpy as np - from PIL import Image - - if len(image_paths) != len(foreground_paths): - raise RuntimeError("Foreground mask count does not match source frames") - depth_frames = [] - for image_path, foreground_path in zip(image_paths, foreground_paths): - with Image.open(image_path) as source: - luminance = np.asarray(source.convert("L"), dtype=np.float32) / 255.0 - with Image.open(foreground_path) as foreground: - alpha = np.asarray( - foreground.convert("RGBA").getchannel("A"), - dtype=np.uint8, - ) - if luminance.shape != alpha.shape: - raise RuntimeError("Foreground mask size does not match source frame") - depth_frames.append( - compose_original_background_depth( - luminance, - alpha, - background_depth_distance, - ) - ) - np.savez_compressed( - Path(output_directory) / "original-depth.npz", - depth=np.stack(depth_frames), - ) - - -def compose_original_background_depth( - luminance, - alpha, - background_depth_distance, -): - import numpy as np - - background_depth = -background_depth_distance - depth = np.where(alpha > 0, luminance, background_depth) - set_depth_border(depth, background_depth) - return depth - - def write_status( path, progress, @@ -608,7 +501,7 @@ def export_image_mesh_data( depth_frames = np.asarray(prediction.depth, dtype=np.float32) confidence_frames = np.asarray(prediction.conf, dtype=np.float32) if depth_frames.shape[0] != 1: - raise ValueError("Convert to Mesh currently supports one image") + raise ValueError("Convert to Surface currently supports one image") raw_depth = depth_frames[0] confidence = confidence_frames[0] intrinsics = np.asarray(prediction.intrinsics[0], dtype=np.float32) @@ -636,7 +529,7 @@ def export_image_mesh_data( ) if not len(faces): raise ValueError( - "DA3 output does not contain any valid mesh faces" + "DA3 output does not contain any valid surface faces" ) progress(5, "Projecting UV and structured attributes") vertices, faces = compact_mesh( @@ -658,7 +551,7 @@ def export_image_mesh_data( reference_depth = float(np.median(valid_depth)) output_directory = Path(output_directory) output_directory.mkdir(parents=True, exist_ok=True) - progress(6, "Writing mesh data") + progress(6, "Writing surface data") result_path = output_directory / "mesh.npz" np.savez_compressed( result_path, @@ -876,7 +769,7 @@ def export_textured_mesh_glb( referenced_points.append(mesh.vertices) if not meshes: raise ValueError( - "Mesh filters removed every face; lower Confidence Threshold or disable filters" + "Surface filters removed every face; lower Confidence Threshold or disable filters" ) world_to_camera = np.asarray(prediction.extrinsics[0], dtype=np.float64) @@ -1343,6 +1236,7 @@ def infer_foreground( image, refine_foreground=False, ) + foreground = preserve_source_rgb(image, foreground) if device.type == "cuda": torch.cuda.synchronize() inference_ms = (time.perf_counter() - inference_started) * 1000.0 @@ -1354,6 +1248,18 @@ def infer_foreground( return foreground, load_ms, inference_ms +def preserve_source_rgb(source, foreground): + """Combine the untouched source RGB with BEN2's foreground alpha.""" + source_rgba = source.convert("RGBA") + alpha = foreground.convert("RGBA").getchannel("A") + if alpha.size != source_rgba.size: + from PIL import Image + + alpha = alpha.resize(source_rgba.size, Image.Resampling.LANCZOS) + source_rgba.putalpha(alpha) + return source_rgba + + def infer_foreground_frames( images, model_dir, @@ -1456,21 +1362,6 @@ def remove_background(args, model_cache=None, cancel_check=None): cancel_check=cancel_check, ) copy_frames(images, output_directory) - background_depth_distance = float( - getattr(args, "background_depth_distance", 0.0) - ) - if background_depth_distance > 0.0: - foreground_paths = ( - [output_directory / "foreground.png"] - if len(images) == 1 - else sorted(output_directory.glob("foreground.*.png")) - ) - write_original_background_depth( - images, - foreground_paths, - background_depth_distance, - output_directory, - ) finally: if temporary_directory is not None: shutil.rmtree(temporary_directory, ignore_errors=True) @@ -1662,9 +1553,6 @@ def reconstruct(args, model_cache=None, cancel_check=None): "remove_background_model_dir", "", ) - background_depth_distance = float( - getattr(args, "background_depth_distance", 0.0) - ) foreground_images = [] if remove_background_model_dir: write_status(args.status, 0.12, "Removing image background") @@ -1747,12 +1635,6 @@ def reconstruct(args, model_cache=None, cancel_check=None): if args.export_format == "mini_npz": reference_depth = depth_reference_value(prediction.depth[0]) convert_ai_depth_for_displacement(prediction) - if background_depth_distance > 0.0: - apply_foreground_depth_distance( - prediction, - foreground_images, - background_depth_distance, - ) write_status(args.status, 0.88, f"Exporting {args.export_format}") if args.export_format == "mesh": export_textured_mesh_glb( diff --git a/tests/test_addon.py b/tests/test_addon.py index b740689..70472ad 100644 --- a/tests/test_addon.py +++ b/tests/test_addon.py @@ -231,9 +231,30 @@ class AddonTest(unittest.TestCase): self.assertTrue(all(name.startswith("o_") for name in names)) self.assertTrue(image_mesh.node_asset_path().is_file()) - self.assertEqual(image_mesh.NODE_GROUP_NAME, "O Refine DA3 Mesh") + self.assertEqual(image_mesh.NODE_GROUP_NAME, "O Refine DA3 Surface") self.assertEqual(image_mesh.NODE_ASSET_NAME, "O_SnapForm.blend") + def test_depth_plane_uses_bundled_nodes_and_material_attributes(self): + image_depth = self.addon.image_depth + + self.assertTrue(image_depth.node_asset_path().is_file()) + self.assertEqual( + image_depth.DEPTH_PLANE_NODE_GROUP_NAME, + "O DA3 Depth Plane", + ) + self.assertEqual( + image_depth.REMOVE_BACKGROUND_ATTRIBUTE, + "o_remove_background", + ) + self.assertEqual( + image_depth.MATERIAL_DISPLACEMENT_ATTRIBUTE, + "o_material_displacement", + ) + self.assertEqual( + image_depth.BACKGROUND_DEPTH_ATTRIBUTE, + "o_background_depth", + ) + def test_image_mesh_material_depth_scale_matches_fitted_geometry(self): mesh_object = SimpleNamespace(scale=SimpleNamespace(z=-2.0)) @@ -315,6 +336,7 @@ class AddonTest(unittest.TestCase): (-0.25, -0.75), ) self.assertEqual(result_image.name, "img.foreground") + self.assertEqual(result_image.alpha_mode, "CHANNEL_PACKED") self.assertEqual(pack_calls, [True]) self.assertEqual(remove_calls, []) self.assertEqual( @@ -365,6 +387,7 @@ class AddonTest(unittest.TestCase): self.assertIs(source_object.data, original_image) self.assertFalse(original_image.use_fake_user) self.assertEqual(result_image.name, "img.foreground") + self.assertEqual(result_image.alpha_mode, "CHANNEL_PACKED") self.assertEqual(pack_calls, [True]) def test_sequence_background_result_preserves_empty_image_start(self): @@ -408,6 +431,7 @@ class AddonTest(unittest.TestCase): self.assertIs(created, result_image) self.assertIs(source_object.data, result_image) + self.assertEqual(result_image.alpha_mode, "CHANNEL_PACKED") self.assertEqual(image_user.frame_start, 4) self.assertEqual(image_user.frame_duration, 2) self.assertEqual(image_user.frame_offset, 0) @@ -456,10 +480,7 @@ class AddonTest(unittest.TestCase): def test_original_depth_background_starts_independent_job(self): operator = self.addon.ConvertImageToDepthPlane() operator.snapform_depth_source = "ORIGINAL" - operator.snapform_background_depth_distance = 2.0 - operator.snapform_displacement_via = "MODIFIER" operator.snapform_unshaded = False - operator.snapform_subdivision_level = 5 operator.snapform_delete_original = False source_object = SimpleNamespace( name="Reference", @@ -486,15 +507,12 @@ class AddonTest(unittest.TestCase): self.assertEqual(result, {"FINISHED"}) self.assertEqual(calls[0]["job_type"], "ORIGINAL_BACKGROUND") - self.assertEqual(calls[0]["snapform_background_depth_distance"], 2.0) + self.assertNotIn("snapform_background_depth_distance", calls[0]) - def test_original_depth_animation_extracts_frames_without_background(self): + def test_original_depth_animation_extracts_frames_and_background(self): operator = self.addon.ConvertImageToDepthPlane() operator.snapform_depth_source = "ORIGINAL" - operator.snapform_background_depth_distance = 0.0 - operator.snapform_displacement_via = "MODIFIER" operator.snapform_unshaded = False - operator.snapform_subdivision_level = 5 operator.snapform_delete_original = False source_object = SimpleNamespace( name="Reference", @@ -528,7 +546,7 @@ class AddonTest(unittest.TestCase): self.assertEqual(result, {"FINISHED"}) self.assertEqual(scene.snapform_settings.video_frames, 97) - self.assertEqual(calls[0]["job_type"], "ORIGINAL_FRAMES") + self.assertEqual(calls[0]["job_type"], "ORIGINAL_BACKGROUND") self.assertEqual(calls[0]["snapform_input_path"], str(input_path)) self.assertEqual(calls[0]["snapform_source_object_name"], "Reference") self.assertFalse(calls[0]["snapform_delete_input"]) @@ -552,27 +570,15 @@ class AddonTest(unittest.TestCase): self.assertFalse( operator.__annotations__["snapform_unshaded"]["default"] ) - background_offset = operator.__annotations__[ - "snapform_background_depth_distance" - ] - self.assertEqual(background_offset["default"], 0.0) - self.assertEqual(background_offset["min"], 0.0) - self.assertEqual( - operator.__annotations__["snapform_subdivision_level"]["default"], - 5, - ) self.assertEqual( set(operator.__annotations__), { "snapform_depth_source", - "snapform_displacement_via", "snapform_unshaded", - "snapform_background_depth_distance", "snapform_model", "snapform_use_ray_pose", "snapform_process_res_factor", "snapform_delete_original", - "snapform_subdivision_level", }, ) depth_sources = operator.__annotations__["snapform_depth_source"] @@ -583,15 +589,6 @@ class AddonTest(unittest.TestCase): ) self.assertEqual(depth_sources["items"][0][1], "AI Generated") self.assertEqual(depth_sources["name"], "Depth from") - displacement_via = operator.__annotations__[ - "snapform_displacement_via" - ] - self.assertEqual(displacement_via["default"], "MODIFIER") - self.assertEqual(displacement_via["name"], "Displace via") - self.assertEqual( - [item[0] for item in displacement_via["items"]], - ["MODIFIER", "MATERIAL"], - ) self.assertEqual( operator.__annotations__["snapform_unshaded"]["name"], "Unshaded", @@ -674,7 +671,7 @@ class AddonTest(unittest.TestCase): self.assertEqual(result, {"FINISHED"}) self.assertEqual(scene.snapform_settings.video_frames, 24) - def test_conversion_execute_uses_background_offset_frame_range(self): + def test_conversion_execute_uses_background_mask_frame_range(self): source_object = SimpleNamespace( name="Reference", data=SimpleNamespace( @@ -694,10 +691,7 @@ class AddonTest(unittest.TestCase): operator.snapform_model = "BASE" operator.snapform_use_ray_pose = True operator.snapform_process_res_factor = 1.0 - operator.snapform_displacement_via = "MODIFIER" operator.snapform_unshaded = False - operator.snapform_background_depth_distance = 1.0 - operator.snapform_subdivision_level = 5 operator.snapform_delete_original = False calls = [] self.fake_bpy.ops = SimpleNamespace( @@ -720,7 +714,8 @@ class AddonTest(unittest.TestCase): self.assertEqual(result, {"FINISHED"}) self.assertEqual(scene.snapform_settings.video_frames, 24) self.assertEqual(calls[0]["job_type"], "IMAGE_DEPTH") - self.assertEqual(calls[0]["snapform_background_depth_distance"], 1.0) + self.assertTrue(calls[0]["snapform_remove_background"]) + self.assertNotIn("snapform_background_depth_distance", calls[0]) def test_conversion_dialog_only_lists_installed_models(self): items_callback = self.addon.ConvertImageToDepthPlane.__annotations__[ @@ -823,10 +818,7 @@ class AddonTest(unittest.TestCase): operator.snapform_model = "GIANT_1_1" operator.snapform_use_ray_pose = True operator.snapform_process_res_factor = 0.5 - operator.snapform_displacement_via = "MATERIAL" operator.snapform_unshaded = True - operator.snapform_background_depth_distance = 2.5 - operator.snapform_subdivision_level = 5 operator.snapform_delete_original = True image = SimpleNamespace( name="img.png", @@ -865,12 +857,8 @@ class AddonTest(unittest.TestCase): self.assertEqual(calls[0]["snapform_input_path"], str(packed_path)) self.assertEqual(calls[0]["snapform_process_res"], 960) self.assertEqual(calls[0]["snapform_model_key"], "GIANT_1_1") - self.assertEqual( - calls[0]["snapform_displacement_via"], - "MATERIAL", - ) + self.assertTrue(calls[0]["snapform_remove_background"]) self.assertTrue(calls[0]["snapform_unshaded"]) - self.assertEqual(calls[0]["snapform_background_depth_distance"], 2.5) self.assertEqual(calls[0]["snapform_depth_data_name"], "img.depth") self.assertTrue(calls[0]["snapform_delete_input"]) self.assertEqual( @@ -1070,7 +1058,7 @@ class AddonTest(unittest.TestCase): snapform_ray_pose, ) - def test_image_depth_background_option_requires_ben2(self): + def test_image_depth_background_mask_requires_ben2(self): settings = SimpleNamespace( input_path="debug-input.png", export_format="mini_npz", @@ -1082,7 +1070,7 @@ class AddonTest(unittest.TestCase): operator.snapform_model_key = "BASE" operator.snapform_use_ray_pose = True operator.snapform_process_res = 1024 - operator.snapform_background_depth_distance = 1.0 + operator.snapform_remove_background = True with TemporaryDirectory() as directory: input_path = Path(directory) / "input.png" @@ -1140,16 +1128,15 @@ class AddonTest(unittest.TestCase): Path("status.json"), Path("result"), remove_background=True, - background_depth_distance=2.5, ) self.assertEqual( request.payload["parameters"]["remove_background_model_dir"], str(Path("ben2_base")), ) - self.assertEqual( - request.payload["parameters"]["background_depth_distance"], - 2.5, + self.assertNotIn( + "background_depth_distance", + request.payload["parameters"], ) def test_host_busy_snapshot_is_released_for_completed_local_job(self): @@ -1174,12 +1161,9 @@ class AddonTest(unittest.TestCase): operator.snapform_source_object_name = "Reference" operator.snapform_depth_data_name = "img.depth" operator.snapform_process_res = 1024 - operator.snapform_subdivision_level = 5 operator.snapform_delete_source = False operator.snapform_delete_input = False - operator.snapform_displacement_via = "MODIFIER" operator.snapform_unshaded = False - operator.snapform_background_depth_distance = 0.0 started = threading.Event() release = threading.Event() @@ -1368,13 +1352,12 @@ class AddonTest(unittest.TestCase): Path("result"), "BEN2_BASE", start_frame=70, - background_depth_distance=3.0, ) parameters = request.payload["parameters"] self.assertEqual(parameters["video_frames"], 24) self.assertEqual(parameters["start_frame"], 70) - self.assertEqual(parameters["background_depth_distance"], 3.0) + self.assertNotIn("background_depth_distance", parameters) def test_extract_frames_request_passes_frame_range(self): runtime = importlib.import_module("snapform.runtime") @@ -1683,7 +1666,7 @@ class AddonTest(unittest.TestCase): "Gaussian Splats (PLY)", "Depth Preview (Images)", "Depth Data (NPZ)", - "Textured Mesh (GLB)", + "Textured Surface (GLB)", ], ) @@ -1826,7 +1809,7 @@ class AddonTest(unittest.TestCase): ): self.assertTrue(self.addon.RemoveImageBackground.poll(context)) - def test_convert_to_depth_plane_poll_requires_da3_model(self): + def test_convert_to_depth_plane_poll_requires_da3_and_ben2_models(self): context = SimpleNamespace( object=SimpleNamespace( type="EMPTY", @@ -1841,7 +1824,7 @@ class AddonTest(unittest.TestCase): ), patch.object( self.addon.runtime, "model_ready", - side_effect=lambda key: key == "BASE", + side_effect=lambda key: key in {"BASE", "BEN2_BASE"}, ): self.assertTrue( self.addon.ConvertImageToDepthPlane.poll(context) @@ -2321,7 +2304,7 @@ class AddonTest(unittest.TestCase): self.assertFalse(calls[0][1]["snapform_delete_source"]) self.assertEqual( calls[0][1]["snapform_depth_npz_path"], - str(Path("result") / "original-depth.npz"), + "", ) self.assertEqual( calls[1], diff --git a/tests/test_worker.py b/tests/test_worker.py index a7655fe..a74aed5 100644 --- a/tests/test_worker.py +++ b/tests/test_worker.py @@ -30,6 +30,25 @@ class WorkerTest(unittest.TestCase): def setUpClass(cls): cls.worker = load_worker() + def test_foreground_preserves_source_rgb_and_ben2_alpha(self): + alpha = SimpleNamespace(size=(2, 1)) + source_rgba = SimpleNamespace(size=(2, 1), applied_alpha=None) + source_rgba.putalpha = lambda value: setattr( + source_rgba, + "applied_alpha", + value, + ) + source = SimpleNamespace(convert=lambda _mode: source_rgba) + foreground_rgba = SimpleNamespace( + getchannel=lambda _name: alpha, + ) + foreground = SimpleNamespace(convert=lambda _mode: foreground_rgba) + + result = self.worker.preserve_source_rgb(source, foreground) + + self.assertIs(result, source_rgba) + self.assertIs(result.applied_alpha, alpha) + def test_collect_images_is_sorted_and_filters_suffixes(self): with tempfile.TemporaryDirectory() as directory: root = Path(directory) @@ -138,49 +157,6 @@ class WorkerTest(unittest.TestCase): ) self.assertEqual(args.remove_background_model_dir, "ben2") - def test_background_distance_sets_fixed_background_depth(self): - prediction = SimpleNamespace( - depth=np.array( - [[ - [1.0, 2.0, 3.0, 4.0], - [5.0, -9.0, -2.0, 8.0], - [9.0, -3.0, 4.0, 12.0], - [13.0, 14.0, 15.0, 16.0], - ]] - ), - conf=np.arange(16, dtype=np.float32).reshape(1, 4, 4), - ) - foreground_mask = np.array( - [[ - [True, True, True, True], - [True, True, False, True], - [True, True, True, True], - [True, True, True, True], - ]], - dtype=bool, - ) - original_confidence = prediction.conf.copy() - - self.worker.apply_depth_background_array( - prediction, - foreground_mask, - 8.0, - ) - - np.testing.assert_array_equal( - prediction.depth, - [[ - [-8.0, -8.0, -8.0, -8.0], - [-8.0, -8.0, -8.0, -8.0], - [-8.0, -3.0, 4.0, -8.0], - [-8.0, -8.0, -8.0, -8.0], - ]], - ) - np.testing.assert_array_equal( - prediction.conf, - original_confidence, - ) - def test_ai_depth_is_converted_for_positive_displacement(self): prediction = SimpleNamespace( depth=np.array([[[0.25, 1.0], [1.5, 2.0]]]), @@ -215,26 +191,6 @@ class WorkerTest(unittest.TestCase): 1.5, ) - def test_original_background_uses_fixed_negative_depth(self): - luminance = np.full((3, 3), 0.5, dtype=np.float32) - alpha = np.full((3, 3), 255, dtype=np.uint8) - alpha[1, 0] = 0 - - depth = self.worker.compose_original_background_depth( - luminance, - alpha, - 3.0, - ) - - np.testing.assert_allclose( - depth, - [ - [-3.0, -3.0, -3.0], - [-3.0, 0.5, -3.0], - [-3.0, -3.0, -3.0], - ], - ) - def test_prepares_a_single_image_file(self): with tempfile.TemporaryDirectory() as directory: image_path = Path(directory) / "input.png" @@ -646,6 +602,8 @@ class WorkerTest(unittest.TestCase): save_options = [] class FakeImage: + size = (1, 1) + def __enter__(self): return self @@ -656,6 +614,12 @@ class WorkerTest(unittest.TestCase): self.mode = mode return self + def getchannel(self, _name): + return self + + def putalpha(self, _alpha): + return None + def save(self, path, **options): save_options.append(options) Path(path).touch() @@ -726,6 +690,8 @@ class WorkerTest(unittest.TestCase): inference_options = [] class FakeImage: + size = (1, 1) + def __enter__(self): return self @@ -736,6 +702,12 @@ class WorkerTest(unittest.TestCase): self.mode = mode return self + def getchannel(self, _name): + return self + + def putalpha(self, _alpha): + return None + def save(self, path, **options): Path(path).touch() diff --git a/tools/build_geometry_nodes_asset.py b/tools/build_geometry_nodes_asset.py index 03d3053..de0564b 100644 --- a/tools/build_geometry_nodes_asset.py +++ b/tools/build_geometry_nodes_asset.py @@ -3,7 +3,12 @@ from pathlib import Path import bpy -GROUP_NAME = "O Refine DA3 Mesh" +GROUP_NAME = "O Refine DA3 Surface" +DEPTH_PLANE_GROUP_NAME = "O DA3 Depth Plane" +REMOVE_BACKGROUND_ATTRIBUTE = "o_remove_background" +MATERIAL_DISPLACEMENT_ATTRIBUTE = "o_material_displacement" +BACKGROUND_DEPTH_ATTRIBUTE = "o_background_depth" +DEPTH_SCALE_ATTRIBUTE = "o_depth_scale" OUTPUT_PATH = ( Path(__file__).parents[1] / "src" @@ -356,7 +361,13 @@ def build_group(): group = bpy.data.node_groups.new(GROUP_NAME, "GeometryNodeTree") group.is_modifier = True interface = group.interface - new_socket(interface, "Geometry", "INPUT", "NodeSocketGeometry") + geometry_input_socket = new_socket( + interface, + "Geometry", + "INPUT", + "NodeSocketGeometry", + ) + geometry_input_socket.structure_type = "SINGLE" resolution = new_socket( interface, @@ -400,7 +411,7 @@ def build_group(): ) confidence.default_value = 0.0 confidence.min_value = 0.0 - confidence.max_value = 1.0 + confidence.max_value = 100.0 confidence.subtype = "PERCENTAGE" edge = new_socket( interface, @@ -409,9 +420,9 @@ def build_group(): "NodeSocketFloat", parent=culling_panel, ) - edge.default_value = 0.1 + edge.default_value = 10.0 edge.min_value = 0.0 - edge.max_value = 1.0 + edge.max_value = 100.0 edge.subtype = "PERCENTAGE" background = new_socket( interface, @@ -459,6 +470,10 @@ def build_group(): confidence_attribute.data_type = "FLOAT" confidence_attribute.inputs["Name"].default_value = "o_face_confidence" confidence_attribute.location = (-1800.0, 600.0) + confidence_scale = nodes.new("ShaderNodeMath") + confidence_scale.operation = "MULTIPLY" + confidence_scale.inputs[1].default_value = 0.01 + confidence_scale.location = (-1600.0, 800.0) confidence_compare = new_compare( nodes, "FLOAT", @@ -472,6 +487,10 @@ def build_group(): ) links.new( confidence_input.outputs["Confidence"], + confidence_scale.inputs[0], + ) + links.new( + confidence_scale.outputs[0], input_socket(confidence_compare, "B", 1), ) @@ -479,6 +498,10 @@ def build_group(): edge_attribute.data_type = "FLOAT" edge_attribute.inputs["Name"].default_value = "o_edge_stretch" edge_attribute.location = (-1800.0, 350.0) + edge_scale = nodes.new("ShaderNodeMath") + edge_scale.operation = "MULTIPLY" + edge_scale.inputs[1].default_value = 0.01 + edge_scale.location = (-1600.0, -50.0) edge_maximum = nodes.new("ShaderNodeMath") edge_maximum.operation = "MAXIMUM" edge_maximum.inputs[1].default_value = 0.0001 @@ -494,7 +517,8 @@ def build_group(): (-1200.0, 350.0), 1.0, ) - links.new(stretch_input.outputs["Stretch"], edge_maximum.inputs[0]) + links.new(stretch_input.outputs["Stretch"], edge_scale.inputs[0]) + links.new(edge_scale.outputs[0], edge_maximum.inputs[0]) links.new(edge_maximum.outputs[0], edge_reciprocal.inputs[1]) links.new(edge_attribute.outputs["Attribute"], input_socket(edge_compare, "A", 0)) links.new(edge_reciprocal.outputs[0], input_socket(edge_compare, "B", 1)) @@ -526,7 +550,7 @@ def build_group(): 0.999999, ) links.new( - stretch_input.outputs["Stretch"], + edge_scale.outputs[0], input_socket(full_stretch, "A", 0), ) culling_selection = nodes.new("FunctionNodeBooleanMath") @@ -621,10 +645,434 @@ def build_group(): return group +def build_sequence_frame(nodes, links, scene_frame, frames_input, y): + relative_frame = nodes.new("ShaderNodeMath") + relative_frame.operation = "SUBTRACT" + relative_frame.location = (300.0, y) + links.new(scene_frame, relative_frame.inputs[0]) + links.new(frames_input.outputs["Frame Start"], relative_frame.inputs[1]) + safe_frames = nodes.new("ShaderNodeMath") + safe_frames.operation = "MAXIMUM" + safe_frames.inputs[1].default_value = 1.0 + safe_frames.location = (300.0, y - 200.0) + links.new(frames_input.outputs["Frames"], safe_frames.inputs[0]) + first_frame = nodes.new("ShaderNodeMath") + first_frame.operation = "ADD" + first_frame.inputs[1].default_value = 1.0 + first_frame.location = (500.0, y) + links.new(relative_frame.outputs[0], first_frame.inputs[0]) + clamp_minimum = nodes.new("ShaderNodeMath") + clamp_minimum.operation = "MAXIMUM" + clamp_minimum.inputs[1].default_value = 1.0 + clamp_minimum.location = (700.0, y) + links.new(first_frame.outputs[0], clamp_minimum.inputs[0]) + clamp_maximum = nodes.new("ShaderNodeMath") + clamp_maximum.operation = "MINIMUM" + clamp_maximum.location = (900.0, y) + links.new(clamp_minimum.outputs[0], clamp_maximum.inputs[0]) + links.new(safe_frames.outputs[0], clamp_maximum.inputs[1]) + cyclic_frame = nodes.new("ShaderNodeMath") + cyclic_frame.operation = "FLOORED_MODULO" + cyclic_frame.location = (700.0, y - 200.0) + links.new(relative_frame.outputs[0], cyclic_frame.inputs[0]) + links.new(safe_frames.outputs[0], cyclic_frame.inputs[1]) + cyclic_first_frame = nodes.new("ShaderNodeMath") + cyclic_first_frame.operation = "ADD" + cyclic_first_frame.inputs[1].default_value = 1.0 + cyclic_first_frame.location = (900.0, y - 200.0) + links.new(cyclic_frame.outputs[0], cyclic_first_frame.inputs[0]) + frame_switch = nodes.new("GeometryNodeSwitch") + frame_switch.input_type = "INT" + frame_switch.location = (1100.0, y) + links.new(frames_input.outputs["Cyclic"], frame_switch.inputs["Switch"]) + links.new(clamp_maximum.outputs[0], frame_switch.inputs["False"]) + links.new(cyclic_first_frame.outputs[0], frame_switch.inputs["True"]) + return frame_switch.outputs["Output"] + + +def build_depth_plane_group(): + group = bpy.data.node_groups.new(DEPTH_PLANE_GROUP_NAME, "GeometryNodeTree") + group.is_modifier = True + interface = group.interface + geometry_input_socket = new_socket( + interface, + "Geometry", + "INPUT", + "NodeSocketGeometry", + ) + geometry_input_socket.structure_type = "SINGLE" + displacement = new_socket( + interface, + "Displacement", + "INPUT", + "NodeSocketMenu", + ) + displacement_strength = new_socket( + interface, + "Depth Scale", + "INPUT", + "NodeSocketFloat", + ) + displacement_strength.default_value = 1.0 + displacement_strength.min_value = -10000.0 + displacement_strength.max_value = 10000.0 + displacement_strength.structure_type = "SINGLE" + subdivision = new_socket( + interface, + "Subdivision", + "INPUT", + "NodeSocketInt", + ) + subdivision.default_value = 5 + subdivision.min_value = 0 + subdivision.max_value = 10 + background_panel = interface.new_panel( + name="Remove Background", + default_closed=True, + ) + remove_background = new_socket( + interface, + "Remove Background", + "INPUT", + "NodeSocketBool", + parent=background_panel, + ) + remove_background.default_value = False + remove_background.is_panel_toggle = True + background_depth = new_socket( + interface, + "Background Depth", + "INPUT", + "NodeSocketFloat", + parent=background_panel, + ) + background_depth.default_value = 0.0 + background_depth.min_value = -10000.0 + background_depth.max_value = 10000.0 + background_depth.subtype = "DISTANCE" + data_panel = interface.new_panel(name="Data", default_closed=True) + depth_image = new_socket( + interface, + "Depth Image", + "INPUT", + "NodeSocketImage", + parent=data_panel, + ) + foreground_image = new_socket( + interface, + "Foreground Image", + "INPUT", + "NodeSocketImage", + parent=data_panel, + ) + material = new_socket( + interface, + "Material", + "INPUT", + "NodeSocketMaterial", + parent=data_panel, + ) + frame_data = [] + for name, socket_type, default_value in ( + ("Frame Start", "NodeSocketInt", 1), + ("Frames", "NodeSocketInt", 1), + ("Cyclic", "NodeSocketBool", False), + ): + socket = new_socket( + interface, + name, + "INPUT", + socket_type, + parent=data_panel, + ) + socket.default_value = default_value + if name == "Frames": + socket.min_value = 1 + frame_data.append(socket) + depth_image.hide_in_modifier = True + foreground_image.hide_in_modifier = True + material.hide_in_modifier = True + for socket in frame_data: + socket.hide_in_modifier = True + new_socket(interface, "Geometry", "OUTPUT", "NodeSocketGeometry") + + nodes = group.nodes + links = group.links + bounds_input = new_group_input(nodes, (-2200.0, 700.0), {"Geometry"}) + grid_input = new_group_input(nodes, (-2200.0, 150.0), {"Subdivision"}) + images_input = new_group_input( + nodes, + (100.0, 1250.0), + {"Depth Image", "Foreground Image"}, + ) + frames_input = new_group_input( + nodes, + (50.0, 1750.0), + { + "Frame Start", + "Frames", + "Cyclic", + }, + ) + background_input = new_group_input( + nodes, + (350.0, 250.0), + {"Remove Background", "Background Depth"}, + ) + displacement_input = new_group_input( + nodes, + (950.0, -150.0), + {"Displacement", "Depth Scale"}, + ) + material_input = new_group_input(nodes, (1450.0, 250.0), {"Material"}) + attributes_input = new_group_input( + nodes, + (2150.0, 50.0), + {"Background Depth"}, + ) + group_output = nodes.new("NodeGroupOutput") + group_output.location = (3050.0, 650.0) + + bounds = nodes.new("GeometryNodeBoundBox") + bounds.location = (-1950.0, 700.0) + links.new(bounds_input.outputs["Geometry"], bounds.inputs["Geometry"]) + dimensions = nodes.new("ShaderNodeVectorMath") + dimensions.operation = "SUBTRACT" + dimensions.location = (-1750.0, 700.0) + links.new(bounds.outputs["Max"], dimensions.inputs[0]) + links.new(bounds.outputs["Min"], dimensions.inputs[1]) + separate_dimensions = nodes.new("ShaderNodeSeparateXYZ") + separate_dimensions.location = (-1550.0, 700.0) + links.new(dimensions.outputs["Vector"], separate_dimensions.inputs[0]) + + short_segments = nodes.new("ShaderNodeMath") + short_segments.operation = "POWER" + short_segments.inputs[0].default_value = 2.0 + short_segments.location = (-1950.0, 150.0) + links.new(grid_input.outputs["Subdivision"], short_segments.inputs[1]) + + vertex_counts = [] + for index, (axis, other_axis) in enumerate((("X", "Y"), ("Y", "X"))): + y = 500.0 - index * 300.0 + ratio = nodes.new("ShaderNodeMath") + ratio.operation = "DIVIDE" + ratio.location = (-1350.0, y) + links.new(separate_dimensions.outputs[axis], ratio.inputs[0]) + links.new(separate_dimensions.outputs[other_axis], ratio.inputs[1]) + at_least_one = nodes.new("ShaderNodeMath") + at_least_one.operation = "MAXIMUM" + at_least_one.inputs[1].default_value = 1.0 + at_least_one.location = (-1150.0, y) + links.new(ratio.outputs[0], at_least_one.inputs[0]) + scaled_segments = nodes.new("ShaderNodeMath") + scaled_segments.operation = "MULTIPLY" + scaled_segments.location = (-950.0, y) + links.new(short_segments.outputs[0], scaled_segments.inputs[0]) + links.new(at_least_one.outputs[0], scaled_segments.inputs[1]) + rounded_segments = nodes.new("ShaderNodeMath") + rounded_segments.operation = "ROUND" + rounded_segments.location = (-750.0, y) + links.new(scaled_segments.outputs[0], rounded_segments.inputs[0]) + vertices = nodes.new("ShaderNodeMath") + vertices.operation = "ADD" + vertices.inputs[1].default_value = 1.0 + vertices.location = (-550.0, y) + links.new(rounded_segments.outputs[0], vertices.inputs[0]) + vertex_counts.append(vertices.outputs[0]) + + grid = nodes.new("GeometryNodeMeshGrid") + grid.location = (-300.0, 650.0) + links.new(separate_dimensions.outputs["X"], grid.inputs["Size X"]) + links.new(separate_dimensions.outputs["Y"], grid.inputs["Size Y"]) + links.new(vertex_counts[0], grid.inputs["Vertices X"]) + links.new(vertex_counts[1], grid.inputs["Vertices Y"]) + store_uv = nodes.new("GeometryNodeStoreNamedAttribute") + store_uv.data_type = "FLOAT2" + store_uv.domain = "CORNER" + store_uv.inputs["Name"].default_value = "UVMap" + store_uv.location = (-50.0, 650.0) + links.new(grid.outputs["Mesh"], store_uv.inputs["Geometry"]) + links.new(grid.outputs["UV Map"], store_uv.inputs["Value"]) + + center = nodes.new("ShaderNodeVectorMath") + center.operation = "ADD" + center.location = (-550.0, 950.0) + links.new(bounds.outputs["Min"], center.inputs[0]) + links.new(bounds.outputs["Max"], center.inputs[1]) + half_center = nodes.new("ShaderNodeVectorMath") + half_center.operation = "SCALE" + half_center.inputs[3].default_value = 0.5 + half_center.location = (-300.0, 950.0) + links.new(center.outputs["Vector"], half_center.inputs[0]) + transform_grid = nodes.new("GeometryNodeTransform") + transform_grid.location = (200.0, 650.0) + links.new(store_uv.outputs["Geometry"], transform_grid.inputs["Geometry"]) + links.new(half_center.outputs["Vector"], transform_grid.inputs["Translation"]) + + depth_texture = nodes.new("GeometryNodeImageTexture") + depth_texture.interpolation = "Linear" + depth_texture.extension = "EXTEND" + depth_texture.location = (350.0, 1150.0) + foreground_texture = nodes.new("GeometryNodeImageTexture") + foreground_texture.interpolation = "Closest" + foreground_texture.extension = "EXTEND" + foreground_texture.location = (350.0, 900.0) + links.new(images_input.outputs["Depth Image"], depth_texture.inputs["Image"]) + links.new( + images_input.outputs["Foreground Image"], + foreground_texture.inputs["Image"], + ) + links.new(grid.outputs["UV Map"], depth_texture.inputs["Vector"]) + links.new(grid.outputs["UV Map"], foreground_texture.inputs["Vector"]) + scene_time = nodes.new("GeometryNodeInputSceneTime") + scene_time.location = (50.0, 2200.0) + sequence_frame = build_sequence_frame( + nodes, + links, + scene_time.outputs["Frame"], + frames_input, + 2200.0, + ) + links.new(sequence_frame, depth_texture.inputs["Frame"]) + links.new(sequence_frame, foreground_texture.inputs["Frame"]) + + clamp_foreground = nodes.new("ShaderNodeMath") + clamp_foreground.operation = "MAXIMUM" + clamp_foreground.location = (600.0, 650.0) + links.new(depth_texture.outputs["Color"], clamp_foreground.inputs[0]) + links.new( + background_input.outputs["Background Depth"], + clamp_foreground.inputs[1], + ) + foreground_mask = new_compare( + nodes, + "FLOAT", + "GREATER_THAN", + (600.0, 900.0), + 0.5, + ) + links.new( + foreground_texture.outputs["Alpha"], + input_socket(foreground_mask, "A", 0), + ) + masked_depth = nodes.new("ShaderNodeMix") + masked_depth.data_type = "FLOAT" + masked_depth.location = (800.0, 650.0) + links.new(foreground_mask.outputs["Result"], masked_depth.inputs[0]) + links.new( + background_input.outputs["Background Depth"], + masked_depth.inputs[2], + ) + links.new(clamp_foreground.outputs[0], masked_depth.inputs[3]) + selected_depth = nodes.new("ShaderNodeMix") + selected_depth.data_type = "FLOAT" + selected_depth.location = (1000.0, 650.0) + remove_background_value = background_input.outputs["Remove Background"] + links.new(remove_background_value, selected_depth.inputs[0]) + links.new(depth_texture.outputs["Color"], selected_depth.inputs[2]) + links.new(masked_depth.outputs["Result"], selected_depth.inputs[3]) + + displacement_menu = nodes.new("GeometryNodeMenuSwitch") + displacement_menu.data_type = "FLOAT" + displacement_menu.enum_items.clear() + displacement_menu.enum_items.new("Geometry") + displacement_menu.enum_items.new("Material") + displacement_menu.inputs["Geometry"].default_value = 1.0 + displacement_menu.inputs["Material"].default_value = 0.0 + displacement_menu.location = (1150.0, 200.0) + links.new( + displacement_input.outputs["Displacement"], + displacement_menu.inputs["Menu"], + ) + displacement.default_value = "Geometry" + geometry_mode = displacement_menu.outputs[0] + material_mode = nodes.new("ShaderNodeMath") + material_mode.operation = "SUBTRACT" + material_mode.inputs[0].default_value = 1.0 + material_mode.location = (1350.0, 100.0) + links.new(geometry_mode, material_mode.inputs[1]) + + displacement_scale = nodes.new("ShaderNodeMath") + displacement_scale.operation = "MULTIPLY" + displacement_scale.location = (1200.0, 650.0) + links.new(selected_depth.outputs["Result"], displacement_scale.inputs[0]) + geometry_strength = nodes.new("ShaderNodeMath") + geometry_strength.operation = "MULTIPLY" + geometry_strength.location = (1150.0, 450.0) + links.new(geometry_mode, geometry_strength.inputs[0]) + links.new(displacement_input.outputs["Depth Scale"], geometry_strength.inputs[1]) + links.new(geometry_strength.outputs[0], displacement_scale.inputs[1]) + normal = nodes.new("GeometryNodeInputNormal") + normal.location = (1200.0, 950.0) + offset = nodes.new("ShaderNodeVectorMath") + offset.operation = "SCALE" + offset.location = (1400.0, 750.0) + links.new(normal.outputs["Normal"], offset.inputs[0]) + links.new(displacement_scale.outputs[0], offset.inputs[3]) + set_position = nodes.new("GeometryNodeSetPosition") + set_position.location = (1450.0, 650.0) + links.new(transform_grid.outputs["Geometry"], set_position.inputs["Geometry"]) + links.new(offset.outputs["Vector"], set_position.inputs["Offset"]) + + set_material = nodes.new("GeometryNodeSetMaterial") + set_material.location = (1650.0, 650.0) + links.new(set_position.outputs["Geometry"], set_material.inputs["Geometry"]) + links.new(material_input.outputs["Material"], set_material.inputs["Material"]) + shade_smooth = nodes.new("GeometryNodeSetShadeSmooth") + shade_smooth.domain = "FACE" + shade_smooth.inputs["Shade Smooth"].default_value = True + shade_smooth.location = (1850.0, 650.0) + links.new(set_material.outputs["Geometry"], shade_smooth.inputs["Geometry"]) + attribute_values = ( + ( + REMOVE_BACKGROUND_ATTRIBUTE, + "BOOLEAN", + remove_background_value, + ), + ( + MATERIAL_DISPLACEMENT_ATTRIBUTE, + "BOOLEAN", + material_mode.outputs[0], + ), + ( + BACKGROUND_DEPTH_ATTRIBUTE, + "FLOAT", + attributes_input.outputs["Background Depth"], + ), + ( + DEPTH_SCALE_ATTRIBUTE, + "FLOAT", + displacement_input.outputs["Depth Scale"], + ), + ) + geometry = shade_smooth.outputs["Geometry"] + x = 2100.0 + for name, data_type, value in attribute_values: + store = nodes.new("GeometryNodeStoreNamedAttribute") + store.data_type = data_type + store.domain = "POINT" + store.inputs["Name"].default_value = name + store.location = (x, 650.0) + links.new(geometry, store.inputs["Geometry"]) + links.new(value, store.inputs["Value"]) + geometry = store.outputs["Geometry"] + x += 200.0 + links.new(geometry, group_output.inputs["Geometry"]) + + group.color_tag = "GEOMETRY" + group.description = ( + "Build an aspect-adaptive DA3 depth grid, clamp its foreground, " + "and pass background and displacement controls to the material." + ) + group.use_fake_user = True + return group + + def main(): bpy.ops.wm.read_factory_settings(use_empty=True) bpy.context.preferences.filepaths.save_version = 0 build_group() + build_depth_plane_group() OUTPUT_PATH.parent.mkdir(parents=True, exist_ok=True) bpy.ops.wm.save_as_mainfile(filepath=str(OUTPUT_PATH), check_existing=False) diff --git a/tools/validate_depth_plane_asset.py b/tools/validate_depth_plane_asset.py new file mode 100644 index 0000000..a22864c --- /dev/null +++ b/tools/validate_depth_plane_asset.py @@ -0,0 +1,249 @@ +from pathlib import Path +import sys + +import bpy + + +sys.path.insert(0, str(Path(__file__).parents[1] / "src")) + +from snapform.image_depth import ( + DEPTH_PLANE_MATERIAL_NODE_GROUP_NAME, + SURFACE_MATERIAL_NODE_GROUP_NAME, + add_depth_plane_processing, + create_material, + load_depth_plane_node_group, + modifier_input_identifier, +) + + +def main(): + bpy.ops.wm.read_factory_settings(use_empty=True) + group = load_depth_plane_node_group() + background_depth = next( + item + for item in group.interface.items_tree + if item.item_type == "SOCKET" + and item.in_out == "INPUT" + and item.name == "Background Depth" + ) + assert background_depth.subtype == "DISTANCE" + geometry_input = next( + item + for item in group.interface.items_tree + if item.item_type == "SOCKET" + and item.in_out == "INPUT" + and item.name == "Geometry" + ) + assert geometry_input.structure_type == "SINGLE" + all_inputs = [ + (item.name, getattr(item.parent, "name", None), item.hide_in_modifier) + for item in group.interface.items_tree + if item.item_type == "SOCKET" + and item.in_out == "INPUT" + ] + visible_inputs = [name for name, parent, hidden in all_inputs if not hidden] + assert visible_inputs[:6] == [ + "Geometry", + "Displacement", + "Depth Scale", + "Subdivision", + "Remove Background", + "Background Depth", + ] + remove_background = next( + item + for item in group.interface.items_tree + if item.item_type == "SOCKET" + and item.in_out == "INPUT" + and item.name == "Remove Background" + ) + assert remove_background.socket_type == "NodeSocketBool" + assert remove_background.is_panel_toggle + assert remove_background.parent.name == "Remove Background" + assert background_depth.parent == remove_background.parent + depth_scale = next( + item + for item in group.interface.items_tree + if item.item_type == "SOCKET" + and item.in_out == "INPUT" + and item.name == "Depth Scale" + ) + assert depth_scale.default_value == 1.0 + assert depth_scale.min_value < 0.0 + assert depth_scale.structure_type == "SINGLE" + stores = { + node.inputs["Name"].default_value: node + for node in group.nodes + if node.bl_idname == "GeometryNodeStoreNamedAttribute" + } + attribute_names = ( + "o_remove_background", + "o_material_displacement", + "o_background_depth", + "o_depth_scale", + ) + for name in attribute_names: + assert stores[name].domain == "POINT" + assert not any( + node.bl_idname == "GeometryNodeGeometryToInstance" + for node in group.nodes + ) + + mesh = bpy.data.meshes.new("Depth Plane") + mesh.from_pydata( + [(-1, -1, 0), (1, -1, 0), (-1, 1, 0), (1, 1, 0)], + [], + [(0, 1, 3, 2)], + ) + obj = bpy.data.objects.new("Depth Plane", mesh) + bpy.context.scene.collection.objects.link(obj) + depth = bpy.data.images.new("Depth", width=2, height=2, float_buffer=True) + depth.pixels[:] = [0.25, 0.25, 0.25, 1.0] * 4 + foreground = bpy.data.images.new("Foreground", width=2, height=2) + foreground.pixels[:] = [0.8, 0.2, 0.1, 1.0] * 4 + foreground.alpha_mode = "CHANNEL_PACKED" + material = create_material( + "Depth Plane", + foreground, + 1.0, + displacement_image=depth, + foreground_image=foreground, + ) + mesh.materials.append(material) + material_group = bpy.data.node_groups[DEPTH_PLANE_MATERIAL_NODE_GROUP_NAME] + interface_inputs = [ + item.name + for item in material_group.interface.items_tree + if item.item_type == "SOCKET" + and item.in_out == "INPUT" + ] + interface_outputs = [ + item.name + for item in material_group.interface.items_tree + if item.item_type == "SOCKET" + and item.in_out == "OUTPUT" + ] + assert interface_inputs == [ + "Color", + "Alpha", + "Depth", + "Bump Scale", + ] + assert interface_outputs == [ + "Color", + "Roughness", + "Alpha", + "Normal", + "Displacement", + ] + depth_input = next( + item + for item in material_group.interface.items_tree + if item.item_type == "SOCKET" + and item.in_out == "INPUT" + and item.name == "Depth" + ) + assert depth_input.socket_type == "NodeSocketFloat" + bump_scale = next( + item + for item in material_group.interface.items_tree + if item.item_type == "SOCKET" + and item.in_out == "INPUT" + and item.name == "Bump Scale" + ) + assert bump_scale.default_value == 1.0 + assert bump_scale.min_value < 0.0 + material_group_nodes = [ + node + for node in material.node_tree.nodes + if node.bl_idname == "ShaderNodeGroup" + and node.node_tree == material_group + ] + assert len(material_group_nodes) == 1 + assert material_group_nodes[0].name == "O DA3 Depth Plane" + attributes = [ + node + for node in material_group.nodes + if node.bl_idname == "ShaderNodeAttribute" + ] + assert len(attributes) == 4 + assert all(node.attribute_type == "GEOMETRY" for node in attributes) + assert not any( + node.bl_idname == "ShaderNodeAttribute" + for node in material.node_tree.nodes + ) + forbidden_outer_nodes = { + "ShaderNodeAttribute", + "ShaderNodeBump", + "ShaderNodeDisplacement", + "ShaderNodeFloatCurve", + "ShaderNodeMath", + "ShaderNodeMix", + } + assert not any( + node.bl_idname in forbidden_outer_nodes + for node in material.node_tree.nodes + ) + surface_material = create_material("Surface", foreground, 1.0) + surface_group = bpy.data.node_groups[SURFACE_MATERIAL_NODE_GROUP_NAME] + surface_group_nodes = [ + node + for node in surface_material.node_tree.nodes + if node.bl_idname == "ShaderNodeGroup" + and node.node_tree == surface_group + ] + assert len(surface_group_nodes) == 1 + assert surface_group_nodes[0].name == "O DA3 Surface" + surface_depth_input = next( + item + for item in surface_group.interface.items_tree + if item.item_type == "SOCKET" + and item.in_out == "INPUT" + and item.name == "Depth" + ) + assert surface_depth_input.socket_type == "NodeSocketFloat" + assert not any( + node.bl_idname in forbidden_outer_nodes + for node in surface_material.node_tree.nodes + ) + modifier = add_depth_plane_processing(obj, depth, foreground, material) + modifier[modifier_input_identifier(group, "Depth Scale")] = -2.0 + for name, expected in ( + ("Frame Start", 1), + ("Frames", 1), + ("Cyclic", False), + ): + assert modifier[modifier_input_identifier(group, name)] == expected + image_textures = [ + node + for node in group.nodes + if node.bl_idname == "GeometryNodeImageTexture" + ] + assert len(image_textures) == 2 + assert all(node.inputs["Frame"].is_linked for node in image_textures) + assert image_textures[0].inputs["Frame"].links[0].from_socket == ( + image_textures[1].inputs["Frame"].links[0].from_socket + ) + floored_modulo = [ + node + for node in group.nodes + if node.bl_idname == "ShaderNodeMath" + and node.operation == "FLOORED_MODULO" + ] + assert len(floored_modulo) == 1 + bpy.context.view_layer.objects.active = obj + obj.select_set(True) + result = bpy.ops.object.modifier_apply(modifier=modifier.name) + assert result == {"FINISHED"} + assert not obj.modifiers + assert obj.type == "MESH" + assert len(obj.data.vertices) > 4 + for name in attribute_names: + assert obj.data.attributes[name].domain == "POINT" + assert all(polygon.use_smooth for polygon in obj.data.polygons) + assert material in obj.data.materials[:] + print("Depth plane asset validation passed") + + +if __name__ == "__main__": + main() diff --git a/tools/validate_geometry_nodes_asset.py b/tools/validate_geometry_nodes_asset.py index 7c2c976..856ae3f 100644 --- a/tools/validate_geometry_nodes_asset.py +++ b/tools/validate_geometry_nodes_asset.py @@ -37,6 +37,14 @@ def main(): group = load_geometry_node_group() assert load_geometry_node_group() == group assert group.is_modifier + geometry_input = next( + item + for item in group.interface.items_tree + if item.item_type == "SOCKET" + and item.in_out == "INPUT" + and item.name == "Geometry" + ) + assert geometry_input.structure_type == "SINGLE" assert group.asset_data is None assert not any(node.bl_idname == "GeometryNodeGroup" for node in group.nodes) assert not any(node.bl_idname == "GeometryNodeTriangulate" for node in group.nodes) @@ -72,6 +80,16 @@ def main(): assert [item.name for item in panels["Options"].interface_items] == [ "Fix Topology", ] + for name, default_value in (("Confidence", 0.0), ("Stretch", 10.0)): + item = next( + item + for item in panels["Culling"].interface_items + if item.name == name + ) + assert item.min_value == 0.0 + assert item.max_value == 100.0 + assert item.default_value == default_value + assert item.subtype == "PERCENTAGE" if bpy.app.version >= (5, 0, 0): assert group.library is not None @@ -104,13 +122,13 @@ def main(): finally: evaluated.to_mesh_clear() - set_input(modifier, group, "Stretch", 1.0) + set_input(modifier, group, "Stretch", 100.0) stretch_item = next( item for item in group.interface.items_tree if item.item_type == "SOCKET" and item.name == "Stretch" ) - assert modifier[stretch_item.identifier] == 1.0 + assert modifier[stretch_item.identifier] == 100.0 obj.update_tag(refresh={"DATA"}) bpy.context.view_layer.update() culled = obj.evaluated_get(bpy.context.evaluated_depsgraph_get())