diff --git a/docs/getting-started/quick-start.md b/docs/getting-started/quick-start.md deleted file mode 100644 index a3fb43a..0000000 --- a/docs/getting-started/quick-start.md +++ /dev/null @@ -1,33 +0,0 @@ -# 五分钟上手 - -这个流程会把一张静态图片转换为可继续编辑的 Depth Plane。 - -## 1. 导入图片 - -在 3D View 中选择 **Add > Image > Reference**,导入一张主体清晰、前后关系明确的图片。Blender 会创建一个 Image Empty。 - -## 2. 检查深度设置 - -打开 **Edit > Preferences > Extensions > AnyImage**。第一次使用建议保持默认: - -- **Depth & Normal Model**:MoGe-2 ViT-S Normal -- **MoGe Resolution Level**:5 -- **Device**:使用自动列出的首选设备 - -转换直接使用这些扩展级默认值。 - -## 3. 开始转换 - -选中 Image Empty,在 3D View 中右键选择 **AnyImage > Convert to Depth Plane**。如果环境或必需模型尚未就绪,AnyImage 会先引导完成安装或下载。 - -任务运行时,状态栏会显示准备输入、模型推理和结果导入进度。转换完成后,源 Image Empty 会被新的网格对象替代。 - -## 4. 调整结果 - -打开对象的 `O Image Depth Plane` Geometry Nodes 修改器: - -- **Subdivide** 控制网格密度; -- **Thickness** 增加背面和侧壁,`0` 保持单面; -- **Depth Amount** 控制深度位移强度。 - -建议先用较低 Subdivide 调整 Depth Amount,再按成品需要增加细分。 diff --git a/docs/internals/common.md b/docs/internals/common.md index 3cc0ee3..1d089bd 100644 --- a/docs/internals/common.md +++ b/docs/internals/common.md @@ -17,13 +17,14 @@ | `image.default_extract_frames()` | 取得当前 Empty 默认应处理的帧数。 | | `image.source_start_frame()` | 从源 Image Empty 取得 Job 应处理的起始帧。 | | `image.prepare_image_input()` | 将 Packed、Dirty、Sequence 等输入准备为 Job 可读路径,并返回临时标记。 | -| `image.prepare_selection_input()` | 将 `SelectionMask` 裁切为带 Mask Alpha 的临时 RGBA Job 输入。 | -| `image.cleanup_image_input()` | 按临时标记清理由 `prepare_image_input()` 创建的文件或目录。 | +| `image.prepare_image_bounds_input()` | 复用调用方已读取的 RGBA,按几何 bounds 写出保留原始 Alpha 的矩形 Job 输入。 | +| `image.cleanup_image_input()` | 清理由图片输入 helper 创建的临时文件或目录。 | | `image.image_base_name()` | 去除图片或视频扩展名和 Blender 数字后缀,得到数据块基础名。 | | `image.depth_data_name()` | 生成 Depth 结果数据块文件名。 | | `image.color_data_name()` | 生成 Color 结果数据块文件名。 | | `image.load_color_result_image()` | 加载、命名、Pack 并配置 Job 的 Color 或 Sequence 结果;不修改 Empty。 | | `image.load_normal_result_image()` | 加载、命名、设为 Non-Color 并 Pack Normal Map。 | +| `image.load_packed_rgba_image()` | 加载已有 RGBA 文件,继承源图颜色空间和 Alpha mode,命名并 Pack。 | | `image.create_pixel_result_image()` | 从本地编辑后的 RGBA 像素创建 Packed Image;不修改 Empty。 | | `image.replace_empty_image()` | 执行 Image Empty 结果替换、共享引用分流、旧数据清理、帧设置继承和可选画布对齐。 | | `image.image_empty_bounds()` | 计算 Image Empty 在对象局部坐标中的实际显示边界。 | @@ -32,12 +33,14 @@ 这组公共能力使用固定术语: -- Selection 是用户在图片空间圈定的业务范围;`SelectionPath` 保存路径点、反选状态和填充规则。 -- `SelectionMask` 是 Selection 的栅格化结果,只包含局部像素权重 `values` 和源图 `bounds`。 +- Selection 是用户在图片空间圈定的业务范围;`SelectionPath` 保存路径点和反选状态,并提供完整值的 JSON 编解码。 +- `SelectionMask` 是 Selection 的纯几何栅格化结果,只包含局部像素权重 `values` 和源图几何 `bounds`。 - Mask 表示二维像素权重或 Alpha 运算。 - Blender 对象 Selection 由上下文明确区分;图片选区细化动作称为 Refine Selection。 -`viewport.py` 处理 View3D 中的点击、坐标投影、`ImageSelectionGesture`、工具设置和 Overlay 绘制;`selection.py` 负责图片空间 Selection 的值、栅格化和 Alpha 裁切。 +`viewport.py` 处理 View3D 中的点击、坐标投影、`ImageSelectionGesture`、工具设置和 Overlay 绘制;手势完成时只把内存中的 `SelectionPath` 交给实际工具。`selection.py` 负责图片空间 Selection 的值、栅格化和 Alpha 裁切;`SelectionPath → SelectionMask` 只在图片编辑或 Shape 生成阶段执行。 + +所有 Path 使用 non-zero winding 填充,同方向重复绕行的重叠区域保持选中。栅格器按路径边与像素中心扫描行的交点累计 winding:非反选只建立 Path Bounds 的局部画布,抗锯齿时增加 2 px 边缘;反选使用完整图片范围。栅格阶段不接收图片 Alpha 或颜色,内容相交、空内容和内容 bounds 由 Image Tool 或 Cutout 业务处理。 ### Viewport 交互 @@ -62,7 +65,7 @@ | 函数 | 职责 | | --- | --- | | `selection.selection_path_bounds()` | 校验 Selection Path 并返回整数像素包围框。 | -| `selection.rasterize_selection_path()` | 把 `SelectionPath` 栅格化为 `SelectionMask`,并可与源 Alpha 相交。 | +| `selection.rasterize_selection_path()` | 以局部 non-zero scanline 把 `SelectionPath` 栅格化为纯几何 `SelectionMask`。 | | `selection.crop_pixels_to_alpha()` | 按 Alpha 非零区域紧裁切 RGBA 像素。 | ## Depth 产物 diff --git a/docs/internals/operators/clipboard-image.md b/docs/internals/operators/clipboard-image.md index 54ff708..8defa37 100644 --- a/docs/internals/operators/clipboard-image.md +++ b/docs/internals/operators/clipboard-image.md @@ -38,4 +38,4 @@ flowchart TD 落点仍在 `actions.py` 中完成:3D View 创建 Reference 或 Plane,Node Editor 创建与当前节点树匹配的 Image 节点,绘制模式把 Image 配置为当前画笔的 Texture。鼠标位置和视图位置也在这里换算。 -Plane 路径调用 `convert_to_plane/mesh_plane.py` 创建基础网格,并从节点资产加载 `O Mesh Plane`。`actions.py` 创建 Blender 数据。 +Plane 路径调用 `convert_to_plane/mesh_plane.py` 创建中心原点基础网格,并从节点资产加载 `O Mesh Plane`。`actions.py` 创建图像材质并放入 Mesh 第 `0` 个材质槽,节点组只生成几何。 diff --git a/docs/internals/operators/convert-to-plane.md b/docs/internals/operators/convert-to-plane.md index ddedf20..f55c6e0 100644 --- a/docs/internals/operators/convert-to-plane.md +++ b/docs/internals/operators/convert-to-plane.md @@ -1,6 +1,6 @@ # Plane 转换 -Plane 转换从当前 Image Empty 创建普通 Plane 或 Depth Plane:普通 Plane 同步完成,Depth Plane 先取得 MoGe-2 Z Depth 文件再回到 Blender 创建对象。 +Plane 转换从当前 Image Empty 创建普通 Plane 或 Depth Plane:普通 Plane 同步完成,Depth Plane 先取得 MoGe-2 Z Depth 与 Tangent Normal 文件,再回到 Blender 创建对象。 ## 执行过程 @@ -10,12 +10,12 @@ flowchart TD Kind -- "Plane" --> Color["复制 Color Image"] Color --> Plane["创建 Plane Object"] Kind -- "Depth Plane" --> Job["generate-depth-plane-geometry"] - Job --> Result["Color Image + z-depth.exr + depth.json"] - Result --> Import["直接加载并 Pack Depth Image"] + Job --> Result["Color Image + z-depth.exr + depth.json + tangent-normal.png"] + Result --> Import["加载并 Pack Depth / Normal Image"] Import --> Depth["创建 Depth Plane Object"] ``` -两条路径共用 Image Empty 的显示范围、颜色材质和基础 Plane。Depth Plane 通过一次 MoGe-2 Server Job 生成 Z Depth,再由节点根据相机元数据转换为局部位移。 +两条路径共用 Image Empty 的显示范围、颜色材质和中心原点 Plane。Depth Plane 通过一次 MoGe-2 Server Job 同时生成 Z Depth 与 Tangent Normal;节点根据相机元数据生成单面地形体块,Normal Map 则进入材质着色。 ## 节点说明 @@ -25,18 +25,32 @@ flowchart TD ### 创建普通 Plane -`conversion.py` 先调用 `object.create_color_image()` 复制当前 Image,再由 `object.create_plane_object()` 创建对象。`object.py` 使用 `mesh_plane.py` 生成与 Image Empty 显示范围一致的基础四边形,并通过 `nodes.py` 加载 `O Mesh Plane` 修改器。 +`conversion.py` 先调用 `object.create_color_image()` 复制当前 Image,再由 `object.create_plane_object()` 创建对象。`mesh_plane.py` 把基础四边形顶点放在局部原点两侧,并把 Image Empty 显示范围中心补入对象矩阵,因此世界空间位置不变而对象原点位于面中心。随后从节点资产加载 `O Mesh Plane` 修改器。 -Movie 或 Sequence 的 Image User 设置会保留。转换完成后,新 Plane 替换源 Empty,并沿用原对象名称。 +材质存入 Mesh 第 `0` 个材质槽。`O Mesh Plane` 将带匿名标记的输入几何与生成几何临时合并,统一设置材质索引 `0` 后再删除输入几何,从而让纯生成面保留原 Mesh 的材质集合。Movie 或 Sequence 的 Image User 设置会保留。转换完成后,新 Plane 替换源 Empty,并沿用原对象名称。 -### 生成 Z Depth +### 生成 Depth 与 Normal Depth 路径中的 `GenerateDepthPlane` 位于 `conversion.py`。它从 Preferences 读取 **Depth & Normal Model**、**MoGe Resolution Level** 和 **Maximum AI Input Size**,提交 `generate-depth-plane-geometry` Job。 -`server/jobs/depth_plane.py` 编排任务,`server/geometry/prediction_artifacts.py` 把图片合成到黑底后调用 MoGe-2,并从 `GeometryFrame` 写出 `z-depth.exr` 与 `depth.json`。EXR 的 RGB 重复保存完整相机 Z,Alpha 保存模型 Validity;JSON 保存图片尺寸、相机内参与有效区域的参考深度。AI 输入上限只影响模型分析图,颜色副本仍保留原始分辨率。 +`server/jobs/depth_plane.py` 编排完整图片任务,`server/geometry/prediction_artifacts.py` 把图片合成到黑底后调用一次 MoGe-2,并从同一个 `GeometryFrame` 写出 `z-depth.exr`、`depth.json` 与 `tangent-normal.png`。该 Artifact 生成层也由 Cutout Job 复用;两种顶层 Job 继续各自负责完整图片与 Selection 的输入、结果协议。 + +EXR 的 RGB 重复保存完整相机 Z,Alpha 保存模型 Validity;JSON 保存图片尺寸、相机内参与有效区域的参考深度。Tangent Normal 使用 PNG 保存,只参与材质着色。AI 输入上限只影响模型分析图,颜色副本仍保留原始分辨率。 ### 创建 Depth Plane -Job 返回后,`object.create_depth_plane_from_result()` 直接加载并 Pack `z-depth.exr`,数据块仍由 `depth_data_name()` 命名。`depth.json` 用于计算统一尺度,`O Image Depth Plane` 根据参考深度、统一尺度和用户的 Depth Amount 把相机 Z 转换为局部位移。 +Job 返回后,`object.create_depth_plane_from_result()` 加载并 Pack `z-depth.exr` 与 `tangent-normal.png`。Depth 数据块仍由 `depth_data_name()` 命名;Normal 图片设为 Non-Color,并以 Tangent Space 接入 `O Image Layer`。结果材质与 Cutout 一样使用 Principled IOR `1.2`。`depth.json` 的参考深度乘以统一尺度后初始化 **Base Plane Depth**,使它和采样深度处于同一局部距离单位。 -随后 `object.py` 创建颜色材质和基础四边形,`nodes.py` 加载 `O Image Depth Plane` 修改器。`conversion.py` 最后用一个 Undo 步骤替换源 Empty。 +`O Image Depth Plane` 把 `O Mesh Plane` 的固定底面放在原图片平面,`Thickness` 只沿正法线移动可变面与中间环,再按各层位置应用深度位移: + +```mermaid +flowchart TD + Depth["Z Depth × Uniform Scale"] --> Offset["(Base Plane Depth - Depth) × Depth Scale"] + Offset --> Clamp["负向位移限制在固定底面之前"] + Block["O Mesh Plane 厚度体块"] --> Weight["固定底面 0 → 中间环 → 可变面 1"] + Clamp --> Apply["位移 × 层权重"] + Weight --> Apply + Apply --> Terrain["单面地形体块"] +``` + +固定底面位于原图片平面,未置换可变面位于正法线方向的 **Thickness** 距离处。**Depth Scale** 为 `0` 时成品厚度等于 **Thickness**;正向位移可以超出基础厚度,负向位移不会穿过固定底面。`conversion.py` 最后用一个 Undo 步骤替换源 Empty。 diff --git a/docs/internals/operators/cutout-tool.md b/docs/internals/operators/cutout-tool.md index 54e38fe..40f9536 100644 --- a/docs/internals/operators/cutout-tool.md +++ b/docs/internals/operators/cutout-tool.md @@ -6,36 +6,40 @@ Cutout 从图片上的一个 Selection 开始,可按需执行 Refine Selection ```mermaid flowchart TD - Select["圈定 Selection,选择 Shape"] - Raster["栅格化 Selection"] + Select["圈定 SelectionPath"] + Menu["打开 Shape Pie Menu"] + Shape["选择 Shape"] + Raster["SelectionPath → 几何 SelectionMask"] Route{"需要后台生成数据?"} - Local["沿用本地 SelectionMask"] + Local["读取局部 RGBA
源 Alpha × Selection 值"] + Bounds["写出几何 bounds 矩形 RGBA"] Job["执行 Refine Selection,生成 Depth / Normal 产物"] - Ready["汇总 Shape 输入"] + Encoded["加载并 Pack 输入或 Refine RGBA"] + Ready["汇总内容 Mask 与 Shape 输入"] Build["构建 BaseShape"] Create["创建 Blender 对象"] - Select --> Raster --> Route + Select --> Menu --> Shape --> Raster --> Route Route -- "否" --> Local --> Ready - Route -- "是" --> Job --> Ready + Route -- "是" --> Bounds --> Job --> Encoded --> Ready Ready --> Build --> Create ``` -Surface 与 Balloon 直接使用 `SelectionMask` 构建 `BaseShape`;Refine Selection、Depth Shape 和 Normal Map 先通过 Server Job 生成所需产物。 +Surface 与 Balloon 在本地组合源 Alpha 与几何 Selection 后构建 `BaseShape`;Refine Selection、Depth Shape 和 Normal Map 先通过同一个矩形图片协议生成所需产物。 ## 节点说明 ### 圈定 Selection,选择 Shape -Cutout 工具由 `__init__.py` 注册,交互从 `main.py` 中的 `SelectCutoutSelection` 开始。它接收 Lasso 或双击整图手势,把图片空间路径保存为 `SelectionPath`,然后调用 `interaction.py` 打开 Shape 饼菜单。 +Cutout 工具由 `__init__.py` 注册,交互从 `main.py` 中的 `SelectCutoutSelection` 开始。它接收 Lasso 或双击整图手势,只构建内存中的 `SelectionPath`,然后调用 `interaction.py` 打开 Shape 饼菜单。此时不读取源图像素、不栅格化 Mask,也不扫描可见 Alpha。 -`interaction.py` 完成 Selection 有效性检查并读取三个工具选项。**Mesh Detail** 的 Low、Medium、High、Ultra 分别映射到 32、16、8、4 px 的采样间距,再换算期望网格分辨率;档位越高,采样越密。菜单中的 Surface、Balloon、Depth Balloon 和 Depth Surface,以及它们对应的节点分支索引,统一来自 `shape.py`。 +`interaction.py` 读取工具选项,并在构建菜单按钮时把只含 Points 与 Invert 的完整 `SelectionPath` 编码一次,将同一个 JSON 值直接设置给各 `CutoutSelectionToShape`。这次编码只用于跨 Blender Operator 传值,不属于 Selection Mask 构建。**Mesh Detail** 的 Low、Medium、High、Ultra 分别映射到 32、16、8、4 px 的采样间距,再换算期望网格分辨率;档位越高,采样越密。菜单中的 Surface、Balloon、Depth Balloon 和 Depth Surface,以及它们对应的节点分支索引,统一来自 `shape.py`。 ### 栅格化 Selection -用户选定 Shape 后,`main.py` 中的 `CutoutSelectionToShape` 接过主线。它使用 `common/selection.py` 把 `SelectionPath` 栅格化为 `SelectionMask`:`values` 保存局部像素权重,`bounds` 保存它在源图中的位置。套索外和源图透明区域不会进入最终 Selection。 +用户选定 Shape 后,`main.py` 中的 `CutoutSelectionToShape` 接过主线。它从自身 Property 恢复 `SelectionPath`,再使用 `common/selection.py` 以 non-zero winding 栅格化为一次纯几何 `SelectionMask`:非反选只处理 Path Bounds 与 2 px 抗锯齿边缘,反选覆盖完整图片范围。`values` 保存局部几何权重,`bounds` 保存它在源图中的位置;栅格阶段不读取、接收或合并源 Alpha。 -此时也会应用 **Maximum Cutout Mesh Resolution**,统一限制四种 Shape 裁切区域的最长采样边。 +实际 Cutout 业务随后读取一次源 RGBA。无需 AI 时,它在几何 bounds 内显式计算 `source_alpha × selection_values`,由该内容 Mask 判断空内容并约束轮廓;原始 `SelectionMask` 不被修改。此时也会应用 **Maximum Cutout Mesh Resolution**,统一限制四种 Shape 几何范围的最长采样边。 ### 需要后台生成数据? @@ -45,11 +49,11 @@ Cutout 工具由 `__init__.py` 注册,交互从 `main.py` 中的 `SelectCutout - 选择 Depth Balloon 或 Depth Surface; - 开启 **Normal Map**。 -Surface 和 Balloon 的本地路径直接沿用 `SelectionMask`。 +Surface 和 Balloon 的本地路径直接使用本地内容 Mask。 ### 执行 Refine Selection,生成 Depth / Normal 产物 -`main.py` 把请求提交给 `server/jobs/cutout.py`,生成本次 Shape 需要的产物: +`main.py` 先通过 `common/image.py` 把 Selection 几何 bounds 内的原始 RGBA 写成矩形输入。该文件保留源 Alpha,不预乘 Lasso 值;文件、Packed 或内存图片都使用同一个入口。随后把请求提交给 `server/jobs/cutout.py`,生成本次 Shape 需要的产物: - **Refine Selection** 调用 `server/selection/refine_selection.py`,写出收紧后的 `selection.png` 和 `selection_bounds.json`; - 需要 Vector Depth 时写出相机空间 XYZ 组成的 `vector-depth.exr`,需要 Z Depth 时写出 `z-depth.exr`,两者都配套 `depth.json`; @@ -59,11 +63,13 @@ Depth 与 Normal Map 共享同一个 MoGe-2 `GeometryFrame`。模型输入先按 **Maximum AI Input Size** 约束 BEN2 和 MoGe-2 的分析图长边;网格密度使用 **Maximum Cutout Mesh Resolution**,最终颜色图使用源图裁切分辨率。 +Refine 把 Lasso 视为矩形上下文,因此 BEN2 可以保留 bounds 内、Lasso 外的内容。启用 Refine 时,最终内容 Alpha 完全来自 BEN2;只生成 Depth 或 Normal 时,最终几何仍以本地几何 Selection 值乘矩形输入 Alpha 约束。 + ### 构建 BaseShape -Job 完成后,`main.py` 在 Blender 主线程读取可选的 SelectionMask,加载并 Pack Depth 与 Normal 图片。`main.create_cutout_shape()` 调用 `geometry.py` 中的 `build_base_shape()`: +Job 完成后,`main.py` 在 Blender 主线程加载并 Pack 可选的 Depth 与 Normal 图片,并选取一份已有编码 RGBA:启用 Refine 时使用 BEN2 输出;只生成 Depth 或 Normal 时使用提交前的矩形输入。它从同一局部 Image 读取 Alpha 构建内容 Mask,并把这个 Image 直接交给颜色材质。`main.create_cutout_shape()` 调用 `geometry.py` 中的 `build_base_shape()`: -1. 从最终 `SelectionMask` 提取外轮廓和孔洞; +1. 从内容 Alpha 值提取外轮廓和孔洞; 2. 按 Mesh Detail 对应的采样间距生成轮廓与内部采样点; 3. 进行约束三角化; 4. 计算四种 Shape 共用的 `o_balloon`; @@ -73,7 +79,7 @@ Job 完成后,`main.py` 在 Blender 主线程读取可选的 SelectionMask, ### 创建 Blender 对象 -`main.create_cutout_shape()` 把 `BaseShape` 和可选产物交给 `object.py` 的 `create_shape_object()`。这里创建裁切后的 Color Image、Material、Mesh Object,并挂上统一的 `O Image Cutout` Modifier。材质通过 `common/material.py` 创建,Cutout 使用 `1.2` 的 IOR。Color Image 按最终 Selection 包围矩形裁切源图并保留 RGBA。 +`main.create_cutout_shape()` 把 `BaseShape`、颜色 Image 和可选产物交给 `object.py` 的 `create_shape_object()`。这里创建 Material、Mesh Object,并挂上统一的 `O Image Cutout` Modifier。材质通过 `common/material.py` 创建,Cutout 使用 `1.2` 的 IOR。本地 Cutout 创建一次几何 bounds 颜色图;AI Cutout 直接复用已编码并 Pack 的矩形输入或 BEN2 输出,不再重读完整源图并重新编码同一颜色内容。 | Shape | Modifier 如何形成结果 | 必需后台产物 | | --- | --- | --- | @@ -82,7 +88,7 @@ Job 完成后,`main.py` 在 Blender 主线程读取可选的 SelectionMask, | Depth Balloon | 在 Balloon 上叠加相对基准面的深度 | `z-depth.exr`、`depth.json` | | Depth Surface | 使用相机点 XYZ 重建透视表面 | `vector-depth.exr`、`depth.json` | -Depth Shape 的 Base Plane Depth,以及 Depth Balloon 的 Direction,都由 `object.py` 调用 `geometry.py` 的标定函数,根据最终 Selection Alpha 计算。透明像素达到 `5%` 时使用有效 Selection 自适应标定,否则使用稳定的固定基准。 +Depth Shape 的 Base Plane Depth,以及 Depth Balloon 的 Direction,都由 `object.py` 调用 `geometry.py` 的标定函数,根据最终内容 Alpha 计算。透明像素达到 `5%` 时使用有效内容区域自适应标定,否则使用稳定的固定基准。 Normal Map 关闭时材质使用几何法线;开启时,`object.py` 把生成的图片和已经确定的 Normal Space 交给材质节点。最后 `main.py` 提交 Undo,整条 Cutout 流程结束。 diff --git a/docs/internals/operators/image-tool.md b/docs/internals/operators/image-tool.md index 48f55fe..57fa87d 100644 --- a/docs/internals/operators/image-tool.md +++ b/docs/internals/operators/image-tool.md @@ -7,13 +7,15 @@ Image Tool 从 Box、Lasso、Line 或 Polyline 手势得到一个图片 Selectio ```mermaid flowchart TD Tool["Box / Lasso / Line / Polyline"] --> Gesture["采集图片空间手势"] - Gesture --> Selection["SelectionPath → SelectionMask"] + Gesture --> Path["构建 SelectionPath"] + Path --> Selection["SelectionPath → 几何 SelectionMask"] Selection --> Refine{"Refine Selection?"} - Refine -- "否" --> Mask["使用本地 SelectionMask"] - Refine -- "是" --> Job["refine-image-selection"] - Job --> Mask - Mask --> Alpha["源 Alpha × SelectionMask"] - Alpha --> Crop["紧裁切并写回 Empty"] + Refine -- "否" --> Alpha["读取局部 RGBA
源 Alpha × Selection 值"] + Alpha --> Crop["计算内容 bounds
创建并写回 Image"] + Refine -- "是" --> Bounds["读取 RGBA
写出几何 bounds 矩形"] + Bounds --> Job["refine-image-selection"] + Job --> Result["加载并 Pack BEN2 RGBA"] + Result --> Replace["按响应内容 bounds 替换 Empty"] ``` 四个工具共用 `EditImageMask`。Box 和 Lasso 可以执行 Refine Selection;Line 与 Polyline 始终保留几何边界,直接使用本地 `SelectionMask`。 @@ -28,18 +30,18 @@ flowchart TD ### SelectionPath → SelectionMask -`EditImageMask` 通过 `common/selection.py` 把手势整理为 `SelectionPath`,再栅格化为 `SelectionMask`。凹多边形使用三角剖分,自交路径使用 `EVEN_ODD` 填充。预览缓存只在轨迹变化时更新。 +`ImageSelectionGesture` 把手势整理为内存中的 `SelectionPath`,其中 Points 与 Invert 共同定义 Selection,并直接交给同一个 `EditImageMask` Operator,不经过 JSON。预览和最终 Mask 统一使用 non-zero winding,同方向重复绕行的重叠区域保持选中。实际图片编辑开始后,`EditImageMask` 通过 `common/selection.py` 在必要画布内把路径栅格化为一次纯几何 `SelectionMask`;这一阶段不读取或接收源图 Alpha。预览缓存只在轨迹变化时更新。 Box 和 Lasso 分别读取自己的 **Refine Selection** 设置。Line 与 Polyline 直接使用几何边界。 ### Refine Selection -未开启 Refine Selection 时,`edit_mask.py` 直接使用本地 `SelectionMask`。开启后,`RefineImageSelection` 把带 Selection Alpha 的局部 RGBA 提交给 `server/jobs/refine_image_selection.py`;Job 调用 `server/selection/refine_selection.py` 运行 BEN2,并返回收紧后的图片和 `bounds`。 +未开启 Refine Selection 时,`edit_mask.py` 在实际编辑阶段读取 RGBA 并使用本地几何 `SelectionMask`。开启后,它按 Selection 几何 bounds 写出完整矩形 RGBA 并提交给 `server/jobs/refine_image_selection.py`;矩形内、Lasso 外的源像素与 Alpha 保持原样。Job 调用 `server/selection/refine_selection.py` 运行 BEN2,并返回按结果 Alpha 收紧后的图片和局部内容 `bounds`。 -响应阶段仍由 `edit_mask.py` 把局部结果映射回完整画布。 +Lasso 在 Refine 中只提供矩形上下文,不再作为 BEN2 输出的硬 Mask。响应阶段把局部内容 bounds 映射回源图,并直接加载、配置和 Pack BEN2 已编码的 RGBA 后替换 Empty;不再提取输出 Alpha、重读源图或重新创建同一颜色图片。 ### 源 Alpha × SelectionMask -`edit_mask.py` 保留源 RGB,只把最终 `SelectionMask` 乘入源 Alpha,再按非零 Alpha 紧裁切。像素读取、结果 Image 创建和共享引用处理使用 `common/image.py`。 +`edit_mask.py` 在 `SelectionMask.bounds` 内复用源 RGBA,只把几何 Selection 值乘入源 Alpha,再按行列可见范围计算内容 bounds 并紧裁切。几何 SelectionMask 在合并前后保持不变。像素读取、结果 Image 创建和共享引用处理使用 `common/image.py`。 公共 Image Empty 替换入口同步更新 `empty_display_size` 和 `empty_image_offset`,使裁切后的内容对齐原图片平面并保持对象 `matrix_world`。 diff --git a/mkdocs.yml b/mkdocs.yml index 33bec50..cae13d6 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -11,7 +11,6 @@ nav: - index.md - 开始使用: - 安装与首次配置: getting-started/installation.md - - 五分钟上手: getting-started/quick-start.md - 内部实现: - 总览: internals/index.md - 程序架构: internals/architecture.md diff --git a/openspec/changes/process-image-selection-once/.openspec.yaml b/openspec/changes/process-image-selection-once/.openspec.yaml new file mode 100644 index 0000000..b4b3ece --- /dev/null +++ b/openspec/changes/process-image-selection-once/.openspec.yaml @@ -0,0 +1,2 @@ +schema: spec-driven +created: 2026-09-01 diff --git a/openspec/changes/process-image-selection-once/design.md b/openspec/changes/process-image-selection-once/design.md new file mode 100644 index 0000000..4afb5db --- /dev/null +++ b/openspec/changes/process-image-selection-once/design.md @@ -0,0 +1,72 @@ +## Context + +当前 Viewport 手势已经能够把屏幕路径转换为 `SelectionPath`,却会立即把它编码到 `selection_path_json` Property;Image Tool 随即反解同一个值。Cutout 还会在打开 Shape Pie Menu 前读取整图并将路径栅格化为 `SelectionMask` 做可见性验证;用户选定 Shape 后,`CutoutSelectionToShape` 再次读取和栅格化。Image Tool 的实际编辑阶段也会因 helper 隐式读取图片而重复取得同一 RGBA。 + +本文将用户所说的“序列化选区”明确为昂贵的 `SelectionPath → SelectionMask` 构建过程。`SelectionPath.to_json()` 只是 Blender Operator Property 之间传递轻量路径值的编码,不需要额外状态或缓存。 + +## Goals / Non-Goals + +**Goals:** + +- Image Tool 与 Cutout Tool 复用同一条手势主链,只构建一次 `SelectionPath`。 +- Image Tool 在同一 Operator 内直接传递 `SelectionPath`,不做 JSON 往返。 +- Image Tool 只在图片编辑执行阶段构建一次 `SelectionMask`。 +- Cutout 只在 `CutoutSelectionToShape` 执行阶段构建一次 `SelectionMask`。 +- 同一执行阶段显式复用源图 RGBA 与 Selection Mask,避免 helper 隐式重读或重建。 + +**Non-Goals:** + +- 不引入令牌、全局表、Window 临时状态或 Selection 缓存。 +- 不修改 Selection Path 的 JSON 格式、填充规则、抗锯齿核、Alpha 阈值或最终边界语义。 +- 不修改 Cutout 网格算法、Shape Pie Menu 布局或 AI Job 协议。 +- 本次不把全尺寸 Selection Mask 改为局部画布实现。 + +## Decisions + +### 1. 手势阶段只构建并传递 Selection Path + +`ImageSelectionGesture` 继续负责屏幕坐标到图片坐标的转换,并构建一次 `SelectionPath`。完成边界把该 Python 值交给可覆盖的实际操作提交方法,不再统一写入 `selection_path_json`。Image Tool 与 Cutout Tool 都复用该结果,不在手势完成回调中读取图片像素或调用 Selection Mask 栅格化。 + +Image Tool 在同一个 Operator 实例中保存内存 Selection Path,并直接进入图片编辑执行方法,因此删除自身的 `selection_path_json` Property 和 JSON 编解码。Cutout Tool 把同一个 Python 值交给 Shape Pie Menu;由于按钮会启动新的 Operator,菜单只在这里把 Path 编码一次,并将同一个 JSON 字符串设置到各 `CutoutSelectionToShape` 的 `selection_path_json` Property。菜单层不解析、验证或构建 Selection Mask,也不需要令牌中转。 + +相比令牌或临时表,直接传递现有 Operator Property 没有生命周期、失效、跨 Window 或菜单取消清理问题,并且已经满足 Blender UI Operator 的可序列化参数约束。 + +### 2. Selection Path 到 Selection Mask 的步骤只存在于实际执行阶段 + +Image Tool 的实际编辑方法直接接收手势阶段构建的 Selection Path、读取一次源图 RGBA,并将 Path 栅格化为一次 Selection Mask,再进入本地编辑或 Refine Selection 输入准备。`execute` 不再通过隐藏 JSON Property 恢复同一 Operator 已经持有的路径。 + +`CutoutSelectionToShape.execute` 是 Cutout 的实际生成边界:它直接接收 Pie Menu 传来的 Selection Path 值,读取一次源图 RGBA,并将 Path 栅格化为一次 Selection Mask,再进入 Surface、Balloon 或 AI Shape 主链。 + +Cutout 菜单前的 `validate_cutout_selection` 整体删除。路径结构仍由 `SelectionPath` 构建时验证;必须读取像素的可见 Alpha 与空 Selection 检查由实际执行阶段唯一一次栅格化完成。因此空 Selection 的提示会从打开菜单前移动到点击 Shape 后。 + +### 3. 同一执行阶段显式复用 RGBA 与 Selection Mask + +完成一次读取和栅格化后,Operator 把现成的 RGBA 与 Selection Mask 显式传给图片编辑、Selection 输入导出、Cutout 颜色图和几何 helper。相关 helper 不再从 Blender Image 隐式读取同一像素,也不再接收 Path 后自行构建 Mask。 + +本地同步流程在一个调用栈内直接复用数据。AI Job 响应产生的 refined Selection 属于新的 Job 结果,不与提交前的手势 Selection 合并为同一个缓存生命周期;本次只移除同一阶段内语义相同的重复处理。 + +### 4. JSON 只保留在 Cutout 跨 Operator 边界 + +Lasso 或双击完成后,`SelectCutoutSelection` 把内存中的 `SelectionPath` 直接交给 `open_shape_pie`。菜单调用一次 `to_json()`,将得到的同一个字符串写入每个 `CutoutSelectionToShape` 按钮。用户点击按钮后,新 Operator 通过 `from_json()` 恢复 Path 并进入实际 Shape 生成。 + +保留这一次 JSON 编码是因为 Blender Operator Property 不能直接保存 Python dataclass,也没有适合任意长度套索点的简单标量 Property。彻底删除 JSON 需要全局状态、令牌或额外 PropertyGroup,复杂度高于这次轻量编码。`to_json()` / `from_json()` 因此保留,但只服务 Cutout Pie Menu 到 Shape Operator 的传值,不再属于通用用户圈选主链。 + +### 5. 用阶段边界和调用次数验证 + +交互测试替换像素读取与 `rasterize_selection_path` 入口,验证 Lasso、双击和 Pie Menu 构建阶段调用次数均为零;点击 Shape 或提交 Image Tool 后,每次实际操作各调用一次。结果等价性继续由现有 Image 编辑、Selection 与 Cutout 几何测试覆盖,不设置依赖机器性能的耗时阈值。 + +## Risks / Trade-offs + +- [空 Selection 反馈推迟到点击 Shape 后] → 保留 Path 结构的即时验证,仅推迟必须读取整图的可见性检查。 +- [helper 继续隐式读取像素导致单次处理失效] → 删除隐式回退参数并迁移全部调用方,以调用次数测试固定边界。 +- [Path JSON 在菜单按钮间复制] → 只编码一次并复用同一个字符串;数据量仅与手势点数相关,远小于 RGBA 和 Mask。 +- [Image Tool 的私有 Path 状态被 `execute` 重入] → 把实际图片编辑拆为显式接收 Selection Path 的业务方法,测试直接覆盖该方法,不依赖 Operator 重建后的临时属性。 + +## Migration Plan + +1. 将共享手势完成边界改为传递内存 `SelectionPath`,让 Image Tool 直接进入实际编辑方法,删除其 JSON Property。 +2. 调整 Cutout 提交与 `open_shape_pie`,仅在创建 Shape 按钮时编码一次 Path,并由 `CutoutSelectionToShape` 接收。 +3. 删除菜单前 Selection 像素验证,让 `CutoutSelectionToShape.execute` 成为唯一 Path-to-Mask 边界。 +4. 调整 Image Tool 与共享 helper,显式传递并复用 RGBA 和 Selection Mask。 +5. 更新交互、调用次数、结果等价性测试和内部文档。 +6. 运行相关测试与全部测试;若回退,整体恢复原调用链,不保留双路径或兼容层。 diff --git a/openspec/changes/process-image-selection-once/proposal.md b/openspec/changes/process-image-selection-once/proposal.md new file mode 100644 index 0000000..b21c2b3 --- /dev/null +++ b/openspec/changes/process-image-selection-once/proposal.md @@ -0,0 +1,28 @@ +## Why + +高清图片提交 Image Tool 或 Cutout 选区时,当前主线程会重复读取整图像素、栅格化 Selection 并扫描 Alpha;Cutout 甚至在 Shape Pie Menu 打开前就完成一轮像素处理,导致未启用 AI 也出现明显卡顿。选区数据应在用户真正提交操作后只生成一次,并由后续步骤直接复用。 + +## What Changes + +- Image Tool 与 Cutout Tool 复用同一条 Viewport 手势主链,只负责从手势构建一次内存中的 `SelectionPath`,不在用户圈选阶段执行 JSON 往返、读取图片或构建 `SelectionMask`。 +- Image Tool 由同一个 Operator 在实际图片编辑阶段直接消费内存中的 `SelectionPath`,删除 `selection_path_json` Property,并在该阶段才构建一次 `SelectionMask`。 +- Cutout 手势完成和双击全选将内存中的 `SelectionPath` 交给 Shape Pie Menu;菜单仅为跨 Operator 传值调用一次 `to_json()`,`CutoutSelectionToShape` 从按钮 Property 接收该值,不引入令牌、缓存或额外 Selection 生命周期。 +- Cutout 在用户选定 Shape、实际执行生成时才把 `SelectionPath` 转换为一次 `SelectionMask`,并复用结果完成可见性检查、本地 Shape 创建或 AI 输入准备。 +- 增加调用次数与高分辨率选区测试,确保菜单前零像素处理、提交后单次处理,并同步内部文档。 + +## Capabilities + +### New Capabilities + +- `image-selection-processing`: 规定 Image Tool 与 Cutout 对 Selection Path、Selection Mask 和源图像素的单次处理时机与复用行为。 + +### Modified Capabilities + +无。 + +## Impact + +- 影响 `common/selection.py`、`common/image.py`、Image Tool、Cutout Tool 的交互与转换主链,以及相关测试和内部文档。 +- Image Tool 与 Cutout 圈选 Operator 不再保存 `selection_path_json`;Cutout Pie Menu 到 `CutoutSelectionToShape` 的跨 Operator 边界继续通过 String Property 传递 Selection Path JSON。 +- 不改变用户界面、Shape 候选项、AI Job 协议或最终图像与几何结果。 +- 不新增依赖;Image Tool 与圈选 Operator 的旧 Selection JSON 路径不保留兼容入口,Cutout Shape Operator 所需的跨 Operator JSON 传值继续保留。 diff --git a/openspec/changes/process-image-selection-once/specs/image-selection-processing/spec.md b/openspec/changes/process-image-selection-once/specs/image-selection-processing/spec.md new file mode 100644 index 0000000..d627ea7 --- /dev/null +++ b/openspec/changes/process-image-selection-once/specs/image-selection-processing/spec.md @@ -0,0 +1,75 @@ +## Purpose + +规定 Image Tool 与 Cutout 先共享构建轻量 Selection Path,并只在图片编辑或 Shape 生成的执行阶段将其转换为一次 Selection Mask,使高分辨率源图避免重复处理。 + +## ADDED Requirements + +### Requirement: Image editing prepares one Selection +系统 SHALL 在一次 Image Tool 编辑中只读取一次源图 RGBA 并只将 Selection Path 栅格化为一次 Selection Mask,后续本地编辑或 AI 输入准备 MUST 复用该次处理结果。 + +#### Scenario: User commits a local image edit +- **WHEN** 用户完成 Image Tool 手势且未启用 Refine Selection +- **THEN** 系统使用同一次源图读取和同一个 Selection Mask 完成 Alpha 编辑、裁剪与结果创建 + +#### Scenario: User commits a refined image edit +- **WHEN** 用户完成 Image Tool 手势且启用 Refine Selection +- **THEN** 系统使用同一次源图读取和同一个 Selection Mask 准备 AI 输入,不为输入导出重复读取或栅格化 + +### Requirement: Tools share Selection Path construction +系统 SHALL 让 Image Tool 与 Cutout Tool 复用同一条手势主链构建一次内存中的 Selection Path,用户圈选阶段不得为了工具执行而构建 Selection Mask。 + +#### Scenario: Image Tool gesture completes +- **WHEN** 用户完成任一 Image Tool 手势 +- **THEN** 同一个 Operator 将内存中的 Selection Path 直接交给图片编辑执行阶段,不执行 JSON 编解码 + +#### Scenario: Cutout gesture completes +- **WHEN** 用户完成 Cutout Lasso 或双击全选 +- **THEN** 手势主链将内存中的 Selection Path 交给 Shape Pie Menu,且尚未构建 Selection Mask + +### Requirement: Selection Path serialization is limited to the Cutout operator boundary +系统 SHALL 仅在 Shape Pie Menu 向 `CutoutSelectionToShape` 传递任意长度 Selection Path 时,将路径编码为一次可由 Blender Operator Property 保存的 JSON 值。 + +#### Scenario: Cutout Shape buttons are built +- **WHEN** Shape Pie Menu 为当前 Selection Path 创建 Shape 按钮 +- **THEN** 菜单将该 Path 编码一次,并把同一个值直接设置给各 `CutoutSelectionToShape` 按钮 + +#### Scenario: Cutout Shape operation starts +- **WHEN** 用户点击 Shape 按钮 +- **THEN** `CutoutSelectionToShape` 从自身 Property 取得并解析 Selection Path,且不依赖令牌、全局状态或 Selection 缓存 + +### Requirement: Cutout defers Selection Mask construction until Shape selection +系统 SHALL 在 Cutout Shape Pie Menu 打开前只构建并传递轻量的 Selection Path,不得读取源图像素、栅格化 Selection Mask 或扫描可见 Alpha。 + +#### Scenario: User finishes a Cutout lasso +- **WHEN** 用户松开 Cutout Lasso 并形成有效路径 +- **THEN** 系统立即打开 Shape Pie Menu,且尚未执行全分辨率 Selection 像素处理 + +#### Scenario: User double-clicks a Cutout image +- **WHEN** 用户双击 Image Empty 以选择完整图片 +- **THEN** 系统以完整图片的 Selection Path 打开 Shape Pie Menu,且尚未读取或扫描完整图片像素 + +#### Scenario: User abandons the Shape menu +- **WHEN** 用户关闭 Shape Pie Menu 而未选择 Shape +- **THEN** 系统不读取源图像素、不构建 Selection Mask,也不执行可见 Alpha 扫描 + +### Requirement: Selected Cutout Shape consumes one Selection +系统 SHALL 让 `CutoutSelectionToShape` 直接接收 Selection Path 的序列化值,并在用户选定 Cutout Shape 后只将其转换为一次可供本地或 AI 主链复用的 Selection Mask。 + +#### Scenario: User chooses a local Cutout Shape +- **WHEN** 用户选择不需要 AI 的 Surface 或 Balloon +- **THEN** 系统只读取一次源图 RGBA、栅格化一次 Selection Mask,并复用结果完成验证、颜色图和几何创建 + +#### Scenario: User chooses an AI Cutout Shape +- **WHEN** 用户选择需要 Refine Selection、Depth 或 Normal Map 的 Shape +- **THEN** 系统只读取一次源图 RGBA、栅格化一次 Selection Mask,并复用结果准备持久化的选区输入供后续 Job 阶段消费 + +#### Scenario: Selected Cutout contains no visible pixels +- **WHEN** 用户选定 Shape 后,Selection 与源图可见 Alpha 没有交集 +- **THEN** 系统在该 Shape 操作中报告空 Selection,且不启动 AI Job 或创建 Cutout 对象 + +### Requirement: Selection optimization preserves results +系统 MUST 保持现有 Image Tool 输出边界、Cutout Shape 候选项、AI 参数和最终图像及几何语义不变。 + +#### Scenario: Same Selection is processed after optimization +- **WHEN** 相同源图、Selection Path 和工具设置在优化前后提交 +- **THEN** 系统产生等价的 Selection 边界、Alpha 结果、AI 输入与 Cutout 几何 diff --git a/openspec/changes/process-image-selection-once/tasks.md b/openspec/changes/process-image-selection-once/tasks.md new file mode 100644 index 0000000..393d228 --- /dev/null +++ b/openspec/changes/process-image-selection-once/tasks.md @@ -0,0 +1,20 @@ +## 1. Selection Path 共享主链 + +- [x] 1.1 整理 `ImageSelectionGesture` 的完成边界,使其向实际操作提交方法传递一次内存中的 `SelectionPath`,并用 Viewport 手势测试验证各类手势与双击全选的路径值不变 +- [x] 1.2 删除 Image Tool 和 Cutout 圈选 Operator 的 `selection_path_json` Property,让 Image Tool 直接消费内存 Path,并用测试验证同一 Operator 内不发生 JSON 编解码 +- [x] 1.3 调整 Cutout 手势与 `open_shape_pie`,仅在创建 Shape 按钮时调用一次 `to_json()`,把同一个值设置给各 `CutoutSelectionToShape`,用交互测试验证没有令牌或额外状态 +- [x] 1.4 删除 Cutout Pie Menu 前的 Selection 像素验证,用 Lasso、双击与放弃菜单测试验证 Shape 选定前不读取 RGBA、不栅格化 Mask、不扫描 Alpha + +## 2. 执行阶段单次构建 Selection + +- [x] 2.1 将 Image Tool 实际编辑拆为显式接收 Selection Path 的业务方法,使其成为唯一 Path-to-Mask 边界,并用本地编辑与 Refine Selection 调用次数测试验证每次操作只读取和栅格化一次 +- [x] 2.2 让 `CutoutSelectionToShape.execute` 直接消费 Pie Menu 传入的 Selection Path 值并成为 Cutout 唯一的 Path-to-Mask 边界,用本地与 AI Shape 测试验证每次操作只读取和栅格化一次 +- [x] 2.3 将空 Selection 的可见性检查合并到实际执行阶段的唯一栅格化结果,用测试验证点击 Shape 后正确取消且不会启动 Job 或创建对象 +- [x] 2.4 修改图片编辑、Selection 输入导出与 Cutout 创建 helper,要求显式接收已读取的 RGBA 和 Selection Mask,并用像素及几何测试验证不存在内部重读或重建且结果保持等价 + +## 3. 清理与验证 + +- [x] 3.1 删除 `validate_cutout_selection`、圈选 Operator 的 JSON Property 和 helper 的隐式重读入口,使用定向 `rg` 验证 `to_json()` / `from_json()` 只剩 Cutout Pie Menu 到 Shape Operator 的跨边界用途 +- [x] 3.2 增加高分辨率尺寸的调用边界回归测试,验证菜单前读取与栅格化为零、实际 Image Tool 或 Cutout 操作为一次,且不使用机器相关的耗时阈值 +- [x] 3.3 更新 `docs/internals` 中 Image Tool、Cutout 与共享 Selection 流程,人工核对流程图明确“构建 SelectionPath”与“执行阶段构建 SelectionMask”的边界 +- [x] 3.4 运行 Image Selection、Image Tool、Cutout 交互与几何相关测试,再运行 `uv run pytest`,确认全部测试通过 diff --git a/openspec/changes/rework-plane-conversion/.openspec.yaml b/openspec/changes/rework-plane-conversion/.openspec.yaml new file mode 100644 index 0000000..b4b3ece --- /dev/null +++ b/openspec/changes/rework-plane-conversion/.openspec.yaml @@ -0,0 +1,2 @@ +schema: spec-driven +created: 2026-09-01 diff --git a/openspec/changes/rework-plane-conversion/design.md b/openspec/changes/rework-plane-conversion/design.md new file mode 100644 index 0000000..e236fbc --- /dev/null +++ b/openspec/changes/rework-plane-conversion/design.md @@ -0,0 +1,136 @@ +## Context + +Convert to Plane 当前使用 Image Empty 的局部显示边界直接创建 Mesh,再复制源对象矩阵。非居中的 `empty_image_offset` 因此被写入顶点坐标,结果外观位置正确,但对象原点不在面中心。 + +`O Image Depth Plane` 当前先让 `O Mesh Plane` 生成零厚度网格,再进行一次挤出并移动前侧。该拓扑没有沿厚度方向的中间环;运行时又把默认 `Subdivide` 写为 `0`,深度图通常没有足够顶点形成地形。它还使用与 Cutout 不一致的 `Depth Amount`、`Reference Depth` 和重复的 `Material` 输入。 + +对象创建代码已经把图像材质追加到 Mesh 第 0 个材质槽。`O Image Cutout` 直接依赖该槽,而 `O Mesh Plane` 额外通过 `Material` socket 和 Set Material 节点重复同一职责。 + +Depth Plane Job 与 Cutout Job 已共同调用 `generate_moge2_artifacts()`,但 Depth Plane 当前只请求 Z Depth。共享生成层能够从同一个 MoGe-2 `GeometryFrame` 同时写出 Z Depth 与 Tangent Normal,因此不需要第二次模型推理。 + +## Goals / Non-Goals + +**Goals:** + +- 让普通 Plane 和 Depth Plane 以面中心为对象原点,并严格保留原显示矩形的世界空间位置。 +- 把 Depth Plane 建模为一面置换、背面固定、侧壁分层渐变的地形体块。 +- 统一 Depth Plane 与 Cutout 的 `Depth Scale`、`Base Plane Depth` 术语。 +- 让 Mesh 材质槽承担材质归属,并让纯生成的 Plane 几何保留输入 Mesh 的材质集合。 +- 让 Depth Plane 的几何置换与材质法线共享一次 MoGe-2 预测。 +- 用节点接口、求值几何和世界空间位置测试固定行为。 + +**Non-Goals:** + +- 不修改 MoGe-2 推理、Z Depth EXR、Validity Alpha 或 `depth.json` 协议。 +- 不按每张深度图归一化位移,也不让 `Thickness` 限制正向地形高度。 +- 不增加侧壁环数量、插值曲线或位移方向等新控制项。 +- 不修改 Cutout 的几何或材质行为。 +- 不合并 Cutout 与 Depth Plane 的顶层 Job 编排。 + +## Decisions + +### 1. 通过局部重心与矩阵补偿移动对象原点 + +设 Image Empty 局部显示边界中心为 `C`、原矩阵为 `M`。新 Mesh 使用 `P - C` 作为顶点坐标,新对象矩阵使用 `M × Translation(C)`,因此: + +```text +(M × Translation(C)) × (P - C) = M × P +``` + +普通 Plane 和 Depth Plane 共用同一套居中网格与放置计算,UV 仍覆盖 `0–1`。该方法不调用依赖当前选择和 Context 的 Origin Operator,也能正确保留旋转、非均匀缩放和 Image Empty 偏移。 + +不选择先创建对象再调用 `origin_set`,因为该 Operator 依赖 Blender Context,并会把稳定的坐标变换变成隐式场景操作。 + +### 2. 纯生成几何显式保留 Mesh 材质集合 + +从 `O Mesh Plane` 删除 `Material` interface socket;`O Image Depth Plane` 同步删除对应 socket 与转发。运行时继续在添加 Modifier 前执行 `mesh.materials.append(material)`。 + +Mesh Grid 和 Mesh Cube 是完全生成的几何,仅设置材质索引 `0` 不会自动保留输入 Mesh 的 Material Set。因此 `O Mesh Plane` 先在输入几何上捕获恒真匿名属性,再把输入几何与生成几何临时 Join,统一设为材质索引 `0`,最后按捕获属性删除全部输入点。输出只剩生成几何,但仍携带输入 Mesh 的材质集合。验证脚本将求值普通 Plane、厚度体块和 Depth Plane,检查求值 Mesh 保留第 `0` 个材质槽且所有面使用索引 `0`。 + +不保留隐藏 `Material` socket,因为它仍会形成第二条材质来源,并允许 Modifier 输入与对象材质槽漂移。 + +### 3. Depth Plane 直接复用 O Mesh Plane 的厚度拓扑 + +`O Image Depth Plane` 将输入 Geometry、`Subdivide` 和 `Thickness` 交给 `O Mesh Plane`。`O Mesh Plane` 的 Cube 路径已经按表面网格步长计算厚度方向分段;该计算增加最少侧壁分段,保证固定底面与可变面之间存在中间环,但不增加公开控制项。 + +`O Mesh Plane` 的体块以原平面为中心,Depth Plane 在内部沿局部 Z 平移 `Thickness / 2`,得到: + +```text +固定底面 z = 0 +未置换可变面 z = Thickness +``` + +`Thickness = 0` 时保留单张置换表面,不生成重叠的背面或侧壁。 + +不继续使用单次 Extrude,因为它只生成跨越完整厚度的侧面,无法表达逐环渐变;也不复制一套 Grid/Cube 构建逻辑,避免与 `O Mesh Plane` 的纵横比分段和 UV 规则漂移。 + +### 4. 使用同一个标量场驱动正面和侧壁环 + +每个 XY 位置从 `z-depth.exr` 采样 Z Depth。隐藏的 `Uniform Scale` 把相机深度转换为对象局部距离,完整正面位移为: + +```text +depth = sampled_z × Uniform Scale +full_offset = (Base Plane Depth - depth) × Depth Scale +``` + +这使较近深度向局部正 Z 突出,较远深度向固定底面内缩,等于 `Base Plane Depth` 时位移为零。连续 Depth RGB 在 Validity 边界仍被采样,不用 Alpha 删除点,与当前 Artifact 协议保持一致。 + +体块顶点以未置换位置计算层权重: + +```text +layer_weight = clamp(z / Thickness, 0, 1) +final_z = z + safe_full_offset × layer_weight +``` + +固定底面权重为 `0`,可变面为 `1`,中间环线性递增。线性插值比 Smooth Step 更直接地表达各层置换比例,也让截面顺序容易验证。 + +### 5. 负向位移限幅,正向位移保持完整 + +`safe_full_offset` 的下限设为略大于 `-Thickness`。因为所有层使用同一限幅结果乘以单调层权重,侧壁环不会反序,可变面也不会与固定底面重合。 + +正向位移不钳制。`Thickness` 表示固定底面到未置换可变面的基础厚度;固定底面始终停在原图片平面,`Depth Scale = 0` 时实际厚度恰好等于 `Thickness`,启用正向置换后最终包围盒可以更高。 + +不把位移归一化到固定包围盒,因为这会让相同 `Depth Scale` 在不同图片上产生不同物理尺度,并破坏固定底面、基础厚度和自由 Depth Scale 的独立语义。 + +### 6. 一次性替换节点接口与默认值 + +`Depth Amount` 直接改为 `Depth Scale`,`Reference Depth` 直接改为 `Base Plane Depth`,不添加旧名称转发。创建 Depth Plane 时继续从 Metadata 初始化基础深度与 Uniform Scale,并将 Depth Plane 的 `Subdivide` 默认值设为 `6`;普通 Plane 的默认细分保持 `0`。 + +节点资产、Modifier 输入设置、验证、测试和文档在同一次变更中更新并重建,不保留旧 `.blend` 接口。 + +### 7. 一次预测同时生成几何深度与材质法线 + +Depth Plane 顶层 Job 继续负责完整图片的颜色复制和结果协议,并调用共享的 `generate_moge2_artifacts()`,同时传入: + +```text +depth_mode = Z +normal_mode = TANGENT +``` + +共享生成层只执行一次 MoGe-2 inference,并从同一个 `GeometryFrame` 写出 `z-depth.exr`、`depth.json` 和 `tangent-normal.png`。Cutout 顶层 Job 仍保留 Selection 裁切、可选 BEN2 Refine 和自身结果编排;两条工作流只共享与图片选择无关的 MoGe-2 Artifact 生成层,不把 Cutout 分支引入 Depth Plane。 + +Blender 响应阶段使用现有 Normal 图片加载入口把 `tangent-normal.png` 设为 Non-Color 并 Pack,再以 `normal_space="TANGENT"` 和 `ior=1.2` 传给 `create_image_material()`,使 Depth Plane 与 Cutout 使用相同的 Principled IOR。Normal Map 只参与材质着色,`z-depth.exr` 继续单独驱动 Geometry Nodes 置换;任一步失败时清理尚未被数据块使用的 Depth 与 Normal 图片。 + +## Risks / Trade-offs + +- [高默认细分与厚度环会增加求值几何量] → 继续使用 `O Mesh Plane` 的纵横比自适应分段,并让用户通过现有 `Subdivide` 降低精度。 +- [极端负向位移会形成非常薄的楔形侧壁] → 在固定底面前保留小间隙,并用求值测试检查层顺序和非穿透。 +- [删除 Material socket 后纯生成几何会丢失 Material Set] → 临时 Join 带匿名标记的输入几何,设置索引后删除输入点,并对普通 Plane、厚度体块、Depth Plane 和剪贴板 Plane 做真实 Blender 求值。 +- [Depth 与 Normal 分开生成会重复执行 MoGe-2] → 由共享 Artifact 生成层从同一个 `GeometryFrame` 写出两类文件,并测试 inference 只调用一次。 +- [Tangent Normal 可能未进入最终材质或使用错误色彩空间] → 检查图片为 Non-Color、已 Pack,材质 Normal Texture 使用 Tangent Space 且连接到现有 `O Image Layer` 输入。 +- [移动原点可能影响依赖旧局部坐标的外部脚本] → 这是预期的对象语义修正;世界空间几何保持不变,不提供兼容路径。 +- [节点资产文档与当前源码已有少量名称漂移] → 在本次变更中按最终接口同步更正对应 Plane 文档,不扩展到无关主题。 + +## Migration Plan + +1. 先修改中心原点的 Mesh 与对象矩阵计算,并固定普通 Plane、Depth Plane 的世界空间回归测试。 +2. 删除 Plane 节点组的材质输入和运行时 Modifier 赋值,增加第 0 材质槽的求值验证。 +3. 重建 Depth Plane 节点图、接口名称、默认细分和负向限幅测试。 +4. 扩展 Depth Plane Job,让一次 MoGe-2 预测同时生成 Z Depth 与 Tangent Normal,并把 Normal 接入材质。 +5. 重建并验证 `O_AnyImage.blend`,运行完整测试,再同步 Plane 与节点资产内部文档。 + +回退必须同时回退节点构建脚本、发布 `.blend`、运行时输入名称和对应测试,避免节点接口与 Python 写入不一致。 + +## Open Questions + +无。 diff --git a/openspec/changes/rework-plane-conversion/proposal.md b/openspec/changes/rework-plane-conversion/proposal.md new file mode 100644 index 0000000..4505ecb --- /dev/null +++ b/openspec/changes/rework-plane-conversion/proposal.md @@ -0,0 +1,33 @@ +## Why + +Plane 转换目前保留了 Image Empty 的原点而不是生成面的中心,Depth Plane 也只有单次挤出的侧壁,默认细分不足以表达深度地形。节点组还重复承担材质赋值,导致接口和对象数据的职责不一致。 + +## What Changes + +- 普通 Plane 和 Depth Plane 在保持世界空间位置不变的同时,把对象原点统一放到面的中心。 +- 将 Depth Plane 重建为单面地形体块:固定底面留在原图片平面,`Thickness` 沿正法线移动置换面,中间侧壁环按厚度位置渐变置换。 +- 明确 `Thickness` 为基础厚度:`Depth Scale` 为零时,体块厚度等于 `Thickness`。 +- 让 `Base Plane Depth` 决定深度图的零位移位置,并限制负向置换不穿过固定底面。 +- **BREAKING**:将 Depth Plane 的 `Depth Amount` 改为 `Depth Scale`,将 `Reference Depth` 改为 `Base Plane Depth`,不保留旧接口。 +- **BREAKING**:从 `O Mesh Plane` 和 `O Image Depth Plane` 删除 `Material` socket;`O Mesh Plane` 在内部保留输入 Mesh 的材质集合,生成面统一使用第 `0` 个材质槽。 +- Depth Plane 通过一次共享的 MoGe-2 预测同时生成 `z-depth.exr`、`depth.json` 和 `tangent-normal.png`,并把 Tangent Normal 接入图像材质。 +- Depth Plane 图像材质与 Cutout 统一使用 Principled IOR `1.2`。 +- 将 Depth Plane 的默认 `Subdivide` 设为 `6`,在地形细节和求值几何量之间保持平衡。 + +## Capabilities + +### New Capabilities + +- `plane-conversion`: 规定 Plane 转换的中心原点、材质职责,以及 Depth Plane 地形体块的几何和控制语义。 + +### Modified Capabilities + +无。 + +## Impact + +- 影响 Plane 转换与 Depth Plane 结果创建;删除 `Material` socket 同时简化剪贴板 Plane 的 Modifier 输入。 +- 扩展 Depth Plane Job 的返回产物与 Blender 端材质创建,但继续复用 Cutout 已使用的通用 MoGe-2 Artifact 生成层。 +- 重建 `O Mesh Plane` 与 `O Image Depth Plane` 节点资产、验证脚本和发布 `.blend`。 +- 更新节点组接口、运行时 Modifier 输入、Blender 求值测试和内部文档。 +- 不改变 `z-depth.exr`、Depth Metadata 或 Cutout Job 协议。 diff --git a/openspec/changes/rework-plane-conversion/specs/plane-conversion/spec.md b/openspec/changes/rework-plane-conversion/specs/plane-conversion/spec.md new file mode 100644 index 0000000..181e421 --- /dev/null +++ b/openspec/changes/rework-plane-conversion/specs/plane-conversion/spec.md @@ -0,0 +1,109 @@ +## ADDED Requirements + +### Requirement: Converted planes use a centered origin +系统 SHALL 让 Convert to Plane 和 Convert to Depth Plane 的结果对象以生成矩形面的中心为对象原点,同时保持源 Image Empty 所表示矩形的世界空间位置不变。 + +#### Scenario: Convert an offset Image Empty +- **WHEN** 用户转换一个 `empty_image_offset` 不以原点为中心的 Image Empty +- **THEN** 结果对象原点位于矩形面的中心 +- **AND** 结果面的四个世界空间角点与转换前 Image Empty 的显示范围一致 + +#### Scenario: Preserve transformed placement +- **WHEN** 源 Image Empty 带有平移、旋转或缩放 +- **THEN** 普通 Plane 和 Depth Plane 均通过对象矩阵保留这些世界空间变换 +- **AND** 结果 Mesh 顶点以对象原点为中心分布 + +### Requirement: Plane node groups retain the Mesh material set +`O Mesh Plane` 和 `O Image Depth Plane` SHALL 不公开 `Material` socket。创建对象的代码 SHALL 把图像材质放入 Mesh 的第 `0` 个材质槽;`O Mesh Plane` SHALL 在删除输入几何前把其材质集合传递给纯生成几何,并让生成面使用材质索引 `0`。 + +#### Scenario: Evaluate a generated Plane +- **WHEN** 带有第 0 个材质槽的对象通过 `O Mesh Plane` 生成平面或体块 +- **THEN** 求值后的 Mesh 保留该材质槽 +- **AND** 所有面使用材质索引 `0` +- **AND** Modifier 不包含 `Material` 输入 + +#### Scenario: Evaluate a generated Depth Plane +- **WHEN** 带有第 0 个材质槽的对象通过 `O Image Depth Plane` 生成地形体块 +- **THEN** 求值后的 Mesh 保留该材质槽 +- **AND** 所有面使用材质索引 `0` +- **AND** `O Image Depth Plane` 不向 `O Mesh Plane` 传递材质 + +### Requirement: Depth Plane exposes unified depth controls +`O Image Depth Plane` SHALL 使用 `Thickness`、`Depth Scale`、`Base Plane Depth` 和 `Subdivide` 作为用户控制项。`Depth Image` 与 `Uniform Scale` SHALL 继续作为隐藏数据输入。旧的 `Depth Amount`、`Reference Depth` 和 `Material` 输入 SHALL 不再存在。 + +#### Scenario: Create a Depth Plane modifier +- **WHEN** Convert to Depth Plane 创建 Modifier +- **THEN** `Depth Scale` 默认值为 `1.0` +- **AND** `Base Plane Depth` 取当前 Depth Metadata 的参考深度乘以 `Uniform Scale` +- **AND** `Subdivide` 默认值为 `6` + +#### Scenario: Inspect the node group interface +- **WHEN** 加载发布资产中的 `O Image Depth Plane` +- **THEN** 输入接口包含 `Geometry`、`Subdivide`、`Thickness`、`Depth Scale`、`Base Plane Depth`、`Uniform Scale` 和 `Depth Image` +- **AND** 输入接口不包含 `Depth Amount`、`Reference Depth` 或 `Material` + +### Requirement: Depth Plane shares one prediction for depth and material normal +系统 SHALL 让 Depth Plane 通过通用 MoGe-2 Artifact 生成层执行一次模型预测,并从同一个 `GeometryFrame` 生成 `z-depth.exr`、`depth.json` 与 `tangent-normal.png`。Blender SHALL 将 Tangent Normal 作为 Non-Color 图片加载、Pack 并接入结果材质;Normal Map SHALL 只影响材质着色,不参与 Geometry Nodes 置换。 + +#### Scenario: Generate Depth Plane artifacts +- **WHEN** Depth Plane Job 处理一张完整图片 +- **THEN** MoGe-2 inference 只执行一次 +- **AND** Job 返回 `z_depth`、`depth_metadata` 和 `tangent_normal` + +#### Scenario: Create the Depth Plane material +- **WHEN** Blender 接收成功的 Depth Plane Job 结果 +- **THEN** `tangent-normal.png` 以 Non-Color Tangent Space Normal 加载并 Pack +- **AND** 结果材质的 Normal 输入使用该图片 +- **AND** 结果材质的 Principled IOR 为 `1.2` +- **AND** `z-depth.exr` 继续单独驱动 Depth Plane 几何置换 + +#### Scenario: Keep workflow-specific job orchestration +- **WHEN** Depth Plane 与 Cutout 请求 MoGe-2 产物 +- **THEN** 两者复用同一个通用 Artifact 生成层 +- **AND** Depth Plane Job 不承担 Cutout 的 Selection、Refine Selection 或裁切结果协议 + +### Requirement: Depth Plane is a single-sided terrain block +系统 SHALL 从 `O Mesh Plane` 的有厚度几何生成 Depth Plane。固定底面 SHALL 位于原图片平面,未置换的可变面 SHALL 位于其正法线方向的 `Thickness` 距离处;只有可变面完整应用深度置换,固定底面 SHALL 完全不动,侧壁中间环 SHALL 按其厚度位置线性渐变置换。 + +#### Scenario: Disable depth displacement +- **WHEN** `Depth Scale` 为 `0` +- **THEN** 固定底面保持在原图片平面 +- **AND** 可变面位于正法线方向的 `Thickness` 距离处 + +#### Scenario: Change depth displacement +- **WHEN** 用户改变 `Depth Scale` 或 `Base Plane Depth` +- **THEN** 可变面按完整位移变化 +- **AND** 固定底面的所有顶点位置保持不变 + +#### Scenario: Interpolate side-wall rings +- **WHEN** `Thickness` 大于 `0` 且 Depth Plane 发生置换 +- **THEN** 侧壁包含位于正面与背面之间的细分环 +- **AND** 每个环的位移按其从固定底面到可变面的归一化位置在零位移与完整位移之间线性插值 + +### Requirement: Depth displacement uses the calibrated zero plane +系统 SHALL 以采样 Z Depth 乘以隐藏的 `Uniform Scale` 得到局部深度,并以 `Base Plane Depth` 为零位移位置。比零平面更近的采样 SHALL 向正法线方向置换,比零平面更远的采样 SHALL 向固定底面置换。 + +#### Scenario: Sample the base depth +- **WHEN** 某个正面顶点采样的缩放深度等于 `Base Plane Depth` +- **THEN** 该顶点的深度位移为零 + +#### Scenario: Sample a nearer depth +- **WHEN** 某个正面顶点采样的缩放深度小于 `Base Plane Depth` +- **THEN** 该顶点沿正法线方向产生与 `Depth Scale` 成比例的位移 + +#### Scenario: Sample a farther depth +- **WHEN** 某个正面顶点采样的缩放深度大于 `Base Plane Depth` +- **THEN** 该顶点向固定底面产生与 `Depth Scale` 成比例的位移 + +### Requirement: Inward displacement does not cross the fixed base +系统 SHALL 把负向可变面位移限制在固定底面之前,并保留非零间隙以避免可变面、侧壁环和底面折叠或相交。正向位移 SHALL 不受 `Thickness` 限制,因此最终包围盒深度可以大于基础厚度。 + +#### Scenario: Excessive inward displacement +- **WHEN** 深度公式产生绝对值大于或等于 `Thickness` 的负向位移 +- **THEN** 可变面停留在固定底面之前 +- **AND** 所有侧壁环仍按顺序位于固定底面与可变面之间 + +#### Scenario: Outward displacement exceeds the base thickness +- **WHEN** 正向深度位移大于 `Thickness` +- **THEN** 系统保留完整正向位移 +- **AND** 固定底面仍位于原图片平面 diff --git a/openspec/changes/rework-plane-conversion/tasks.md b/openspec/changes/rework-plane-conversion/tasks.md new file mode 100644 index 0000000..ba8ecde --- /dev/null +++ b/openspec/changes/rework-plane-conversion/tasks.md @@ -0,0 +1,43 @@ +## 1. Center Plane Origins + +- [x] 1.1 Refactor the shared Plane mesh placement so centered local vertices and the compensated object matrix preserve the source Image Empty rectangle in world space. +- [x] 1.2 Apply the centered placement to Convert to Plane and Convert to Depth Plane without changing their UV range, selection replacement, names, or custom shape markers. +- [x] 1.3 Add Blender tests for offset Image Empty bounds and transformed sources, verifying centered origins and unchanged world-space corners for both conversion paths. + +## 2. Unify Plane Material Ownership + +- [x] 2.1 Remove the `Material` interface, Group Input, and Set Material node from `O Mesh Plane`, leaving generated faces on material index `0`. +- [x] 2.2 Remove the `Material` interface and forwarding from `O Image Depth Plane`, and simplify runtime Modifier input assignment while keeping image materials in Mesh slot `0`. +- [x] 2.3 Update Plane, thickness block, Depth Plane, and clipboard Plane validations to prove evaluated faces resolve the Mesh slot `0` material without a Modifier material input. + +## 3. Rebuild Depth Plane Geometry + +- [x] 3.1 Replace the single-extrude Depth Plane graph with the `O Mesh Plane` thickness topology positioned from the original image plane toward the positive-normal displaced face. +- [x] 3.2 Implement Z Depth sampling, `Base Plane Depth` zero calibration, `Depth Scale`, and linear layer weighting from the fixed base to the fully displaced face. +- [x] 3.3 Clamp inward displacement before the fixed base with a nonzero safety gap, preserve unrestricted outward displacement, and retain a single displaced surface when Thickness is zero. +- [x] 3.4 Ensure positive Thickness produces adaptive intermediate side-wall rings with a minimum useful ring count and stable UV/material indices. + +## 4. Replace Interfaces and Defaults + +- [x] 4.1 Replace `Depth Amount` with `Depth Scale` and `Reference Depth` with `Base Plane Depth` across the node builder, runtime Modifier inputs, validators, and tests without compatibility aliases. +- [x] 4.2 Set Convert to Depth Plane and its generated job handoff to default `Subdivide` to `6`, while keeping ordinary Plane default `0` and Metadata-derived hidden data unchanged. +- [x] 4.3 Add evaluated-geometry tests for zero Depth Scale thickness, fixed base vertices, zero-plane calibration, displacement direction and scale, side-ring interpolation, negative non-crossing, and outward displacement beyond the base thickness. + +## 5. Rebuild Assets and Documentation + +- [x] 5.1 Update node interface and layout validations, then rebuild `src/anyimage/assets/O_AnyImage.blend` with the final `O Mesh Plane` and `O Image Depth Plane` graphs. +- [x] 5.2 Update the Plane conversion and node asset internal documentation to describe centered origins, Mesh-slot material ownership, terrain-block topology, and the final control names. +- [x] 5.3 Run the node asset build pipeline and complete test suite, fixing any interface, topology, packaging, registration, or documentation consistency failures. + +## 6. Correct Material Transfer and Thickness Anchor + +- [x] 6.1 Preserve the source Mesh material set in generated `O Mesh Plane` geometry by joining a captured source marker, setting material index `0`, and deleting the source geometry. +- [x] 6.2 Anchor the fixed Depth Plane base at the original image plane so `Thickness` only moves the displaced face and intermediate side-wall rings. +- [x] 6.3 Update evaluated-geometry tests, OpenSpec artifacts, and documentation, then rebuild the node asset and run the complete validation suite. + +## 7. Add Depth Plane Material Normal + +- [x] 7.1 Make the Depth Plane Job request Z Depth and Tangent Normal from one shared `generate_moge2_artifacts()` call, returning `z_depth`, `depth_metadata`, and `tangent_normal` without a second inference. +- [x] 7.2 Load and Pack the Tangent Normal as Non-Color data, connect it to the Depth Plane image material in Tangent Space, and clean up unused Depth and Normal images on failure. +- [x] 7.3 Add server and Blender tests for the combined artifacts, material node connection, default `Subdivide` value, and failure cleanup; update documentation, rebuild the node asset, and run the complete validation suite. +- [x] 7.4 Set the Depth Plane image material Principled IOR to `1.2`, matching Cutout, and update its response test and documentation. diff --git a/openspec/changes/separate-selection-from-image-content/.openspec.yaml b/openspec/changes/separate-selection-from-image-content/.openspec.yaml new file mode 100644 index 0000000..b4b3ece --- /dev/null +++ b/openspec/changes/separate-selection-from-image-content/.openspec.yaml @@ -0,0 +1,2 @@ +schema: spec-driven +created: 2026-09-01 diff --git a/openspec/changes/separate-selection-from-image-content/design.md b/openspec/changes/separate-selection-from-image-content/design.md new file mode 100644 index 0000000..3ef28f5 --- /dev/null +++ b/openspec/changes/separate-selection-from-image-content/design.md @@ -0,0 +1,82 @@ +## Context + +`rasterize_selection_path()` 当前同时完成路径填充、抗锯齿、源 Alpha 相交、空内容判断和内容 bounds 收紧。调用方因此必须先通过 Blender `Image.pixels` 取得 RGBA,导致 `SelectionMask` 随图片透明度变化,而不是稳定表达用户圈选。 + +Image Tool 与 Cutout 的实际业务确实需要图片内容,但目的不同:本地编辑要把 Selection 乘入源 Alpha,Cutout 几何要用内容 Mask,AI 操作要把局部图片写成 Job 输入。需要 AI 时,现有代码先把 Lasso Mask 乘入临时 RGBA 并编码 PNG;响应后又读取源图并新建颜色 Image,造成同一操作中的第二次大图编码。实测 6288×5920 RGBA PNG 的 Selection 栅格化约 0.17–0.55 秒,提交前 PNG 保存约 1.79 秒,新建颜色 Image 的 Pack 约 1.74 秒。图片编码成本不属于 Selection,并且本设计接受生成新图片所需的单次编码成本。 + +## Goals / Non-Goals + +**Goals:** + +- 让 SelectionPath 与 SelectionMask 只表达用户圈选几何,不受源图片 Alpha 或颜色影响。 +- 让 Image Tool、Cutout 与 AI 输入准备在各自业务边界显式消费源 Alpha 或 RGBA。 +- 所有 AI 操作统一使用 Selection 几何 bounds 内、未乘 Lasso Mask 的矩形局部 RGBA 文件。 +- 复用提交前输入或 Refine 输出的已有编码图片,避免 AI 响应阶段重新构建并编码相同颜色图。 +- 保持 non-zero winding、Invert、抗锯齿和局部画布语义。 + +**Non-Goals:** + +- 不优化生成新图片时不可避免的 PNG 编码、Pack 或磁盘写入耗时。 +- 不按 FILE、Packed、Generated 或 Dirty 来源建立不同的 Operator 主链。 +- 不引入 Pillow、GPU readback、后台 Blender 数据访问或新的运行时依赖。 +- 不改变 SelectionPath JSON、AI 模型、Cutout Shape 算法或 Job 进程边界。 + +## Decisions + +### 1. Selection 栅格化只产生几何 Mask + +`rasterize_selection_path(image_size, selection_path, *, antialias=False)` 只根据路径、Invert、图片尺寸和抗锯齿生成 `SelectionMask(values, bounds)`。删除 `source_alpha` 与 `alpha_threshold` 参数,栅格器不导入 Blender Image helper,也不执行内容可见性检查。 + +普通 Selection 的 bounds 来自路径与抗锯齿 padding 裁切到图片范围;Invert 仍使用整张图片。相同 Path、尺寸和抗锯齿设置必须在透明、半透明与不透明图片上得到完全相同的 SelectionMask。 + +相比保留可选 Alpha 参数,彻底删除内容入口可以固定职责并阻止调用方继续把内容 bounds 误当成 Selection bounds;不保留兼容调用方式。 + +### 2. 图片业务直接合并局部 Alpha 与 Selection 值 + +Image Tool 在 `SelectionMask.bounds` 内裁切源 RGBA,并直接执行 `local_alpha *= selection_values`。最终内容 bounds、空内容和结果图片创建都属于 Image 编辑业务。实现应在已有局部数组上计算行列可见范围,不为每个可见像素生成坐标数组,也不把内容相交结果写回 SelectionMask。 + +本地 Cutout 在构建 BaseShape 时显式组合局部源 Alpha 与 Selection 值;颜色、几何和空内容判断使用该业务结果。SelectionMask 本身在调用链中保持不变。相比新增第二种与 SelectionMask 同形的公共值类型,业务直接消费两个一通道局部数组更简单,也避免把图片内容重新包装成“Selection”。 + +### 3. AI 输入是 Selection bounds 的原始矩形 RGBA + +将当前“Selection 输入”改为“Bounds 图片输入”:公共图片 helper 接收源 RGBA 与几何 bounds,只裁切并写出矩形 RGBA,不把 SelectionMask 乘入 Alpha。Image Tool、Cutout Refine、Normal 与 Depth 都调用同一个入口;来源已经统一为 Blender Image,不在 Operator 中按文件、Packed 或生成图片分流。 + +Refine 把 Lasso 当作上下文范围。BEN2 可以在该矩形内保留 Lasso 外的内容,其输出 Alpha 是 Refine 业务结果,不再与原始 Lasso Mask 相交。Normal 或 Depth 也消费同一矩形输入,最终 Shape 仍由本地 Selection 几何约束。 + +Invert Selection 的几何 bounds 是全图,因此其 AI 输入自然退化为全图矩形,不增加特殊协议。 + +### 4. 已编码的 AI 图片直接成为颜色资产 + +AI 响应选择一个已有文件作为最终局部颜色图:启用 Refine 时使用 BEN2 输出;仅生成 Normal 或 Depth 时使用提交前的 Bounds 输入。Blender 直接加载该文件,继承源颜色空间和 Alpha 约定,Pack 已编码文件后交给 Image Empty 或 Cutout 材质。 + +Image Tool Refine 直接按响应 bounds 替换源 Image Empty,不再提取输出 Alpha、重新读取源 RGBA、创建 Pixel Result Image。Cutout 从同一个局部文件读取需要的 Alpha 构建几何,并把该已加载 Image 用作颜色纹理,不再调用 `_cropped_color_image()` 重建同一颜色内容。 + +输入临时文件必须保持到响应完成且 Blender 已加载并 Pack;随后由现有 Job 清理生命周期删除。相比继续从原图重建,这一方案保持颜色来源一致,并把已有文件 Pack 从大图重新编码变为文件收纳。 + +### 5. 内容错误由消费业务报告 + +纯 Selection 只在路径与图片范围没有几何交集时报告空 Selection。源 Alpha 全透明、Image 编辑结果为空、Refine 输出为空或 Cutout 内容 Mask 为空,分别由执行该内容操作的业务报告。这样错误发生位置与实际失败职责一致。 + +## Risks / Trade-offs + +- [Refine 结果扩展到 Lasso 外] → 这是明确接受的产品语义;测试固定 Lasso 只提供矩形上下文,不作为 Refine 硬 Mask。 +- [Selection bounds 不再等于内容 bounds] → 在类型和文档中只把 SelectionMask bounds 称为几何 bounds,内容业务独立计算并传递 placement bounds。 +- [AI 输入矩形包含额外背景] → 最终本地 Shape 仍受 Selection 几何约束;Refine 则有意允许模型利用并保留矩形上下文。 +- [复用 PNG 改变颜色空间或 Alpha 配置] → 加载后显式继承源 Image 的颜色空间与 Alpha mode,并用结果等价测试覆盖。 +- [临时文件被过早清理] → 保持现有 Job cleanup 边界,只有成功加载并 Pack 或操作取消后才删除输入。 +- [全图或 Invert 仍需处理大图] → 接受单次新图片编码成本,本次只删除 Selection 的内容依赖和 AI 响应阶段的重复编码。 + +## Migration Plan + +1. 修改 Selection 栅格 API 与测试,使其只产生几何 Mask,并迁移所有调用方。 +2. 将 Image Tool 与本地 Cutout 的源 Alpha 合并、内容 bounds 和空内容检查放入各自业务阶段。 +3. 将 `prepare_selection_input` 替换为统一的 Bounds 矩形 RGBA 输入,并更新 AI Job 输入语义和测试。 +4. 调整 Image Tool 与 Cutout AI 响应,直接加载并 Pack 输入或 Refine 输出,删除源图重读和颜色 Image 重建。 +5. 更新内部文档,运行相关测试、完整测试和真实高分辨率阶段基准。 + +回退时整体恢复旧栅格参数、Masked AI 输入和响应重建链,不保留双入口或兼容层。 + +## Open Questions + +无。 + diff --git a/openspec/changes/separate-selection-from-image-content/proposal.md b/openspec/changes/separate-selection-from-image-content/proposal.md new file mode 100644 index 0000000..27340d1 --- /dev/null +++ b/openspec/changes/separate-selection-from-image-content/proposal.md @@ -0,0 +1,29 @@ +## Why + +当前 `SelectionPath → SelectionMask` 在栅格化阶段读取并合并源图 Alpha,使选区几何依赖图片内容,也迫使 Image Tool 与 Cutout 在业务开始前读取整图。Selection 应客观表达用户圈选;源 Alpha、图片裁切、AI 输入和最终图片持久化应由实际业务分别处理。 + +## What Changes + +- **BREAKING**:Selection 栅格化只接收图片尺寸与 `SelectionPath`,删除源 Alpha、Alpha threshold 及内容可见性职责;`SelectionMask` 只保存 non-zero winding、Invert 与抗锯齿产生的几何值和几何 bounds。 +- Image Tool 与本地 Cutout 在实际操作中显式将局部源 Alpha 与 `SelectionMask.values` 相乘,并在需要时计算内容 bounds;公共 Selection 模块不读取 Blender Image。 +- 需要 AI 的 Image Tool 与 Cutout 统一按几何 bounds 写出一份未应用 Lasso Mask 的矩形局部 RGBA 文件。Refine 将 Lasso 作为上下文范围,允许 BEN2 结果保留矩形内、Lasso 外的内容。 +- AI 响应复用已经编码的局部图片:Refine 直接使用 BEN2 输出,只有 Normal 或 Depth 时直接使用提交前输入;不再重新读取源图、重建同一颜色 Image 后再次编码 Pack。 +- 删除由 Selection 栅格化隐式承担的空内容判断,改由消费源 Alpha 或 AI 结果的具体业务报告空内容。 +- 增加 Selection 纯几何等价、Alpha 合并边界、矩形 AI 输入和编码图片复用测试,并同步内部文档。 + +## Capabilities + +### New Capabilities + +- `selection-content-processing`: 规定纯几何 Selection Mask、业务侧源 Alpha 合并、Bounds 矩形 AI 输入及已有编码图片复用行为。 + +### Modified Capabilities + +无。 + +## Impact + +- 影响 `common/selection.py`、`common/image.py`、Image Tool、Cutout Tool、Cutout 颜色图创建、AI 输入/响应处理及相关测试和内部文档。 +- `rasterize_selection_path()` 的内部调用协议发生变化,不保留带 `source_alpha` 或 `alpha_threshold` 的兼容入口。 +- Cutout 与 Image Tool 的 AI 输入 Alpha 语义改变:输入为几何 bounds 内的原始矩形 RGBA,而不是预先乘入 Lasso Mask 的 RGBA。 +- 不新增依赖,不改变 SelectionPath JSON、填充规则、抗锯齿核、AI 模型或 Job Server 进程边界;不以本次变更优化不可避免的新图片编码耗时。 diff --git a/openspec/changes/separate-selection-from-image-content/specs/selection-content-processing/spec.md b/openspec/changes/separate-selection-from-image-content/specs/selection-content-processing/spec.md new file mode 100644 index 0000000..05b6554 --- /dev/null +++ b/openspec/changes/separate-selection-from-image-content/specs/selection-content-processing/spec.md @@ -0,0 +1,72 @@ +## ADDED Requirements + +### Requirement: Selection Mask is independent of image content +系统 SHALL 仅根据 Selection Path、Invert、图片尺寸与抗锯齿设置栅格化 Selection Mask,并且 Selection 栅格化不得读取、接收或合并源图片 Alpha、颜色或 Alpha threshold。 + +#### Scenario: Images have different Alpha +- **WHEN** 相同 Selection Path、图片尺寸与抗锯齿设置用于透明度内容不同的图片 +- **THEN** 系统产生逐值相同且 bounds 相同的 Selection Mask + +#### Scenario: Selection is inverted +- **WHEN** Selection Path 启用 Invert +- **THEN** 系统在完整图片画布上产生路径外的纯几何 Selection Mask,而不检查源图片可见内容 + +### Requirement: Image businesses combine source Alpha explicitly +系统 SHALL 由 Image Tool 与本地 Cutout 在实际业务阶段显式组合 Selection Mask 值和对应 Bounds 内的源 Alpha;Selection Mask MUST 在组合前后保持纯几何值不变。 + +#### Scenario: Image Tool edits a Selection +- **WHEN** 用户提交未启用 Refine 的 Image Tool Selection +- **THEN** Image Tool 在几何 bounds 内把 Selection 值乘入源 Alpha,并由编辑业务计算内容 bounds、空结果和最终图片 + +#### Scenario: Local Cutout builds a Shape +- **WHEN** 用户选择不需要 AI 的 Cutout Shape +- **THEN** Cutout 在构建 Shape 时组合局部源 Alpha 与 Selection 值,并使用业务结果约束几何和判断空内容 + +#### Scenario: Source content is transparent +- **WHEN** 几何 Selection 有效但对应源 Alpha 没有可见内容 +- **THEN** 消费该内容的 Image 或 Cutout 业务报告空内容,Selection 栅格化本身仍成功 + +### Requirement: AI receives one rectangular Bounds image +系统 SHALL 为需要 AI 的 Image Tool 或 Cutout 按 Selection 几何 bounds 写出一份矩形局部 RGBA 文件,并且该文件 MUST 保留 bounds 内的源 Alpha,不得预先乘入 Lasso Selection 值。 + +#### Scenario: Refine receives Lasso context +- **WHEN** 用户对 Lasso Selection 启用 Refine +- **THEN** BEN2 接收 Lasso 几何 bounds 对应的完整矩形 RGBA,矩形内、Lasso 外的源像素保持原样 + +#### Scenario: Normal or Depth requires AI +- **WHEN** Cutout 仅因 Normal 或 Depth 进入 AI 流程 +- **THEN** Job 使用与 Refine 相同的 Bounds 矩形输入协议,不建立另一套 Masked Selection 输入 + +#### Scenario: Blender Image source varies +- **WHEN** 源 Image 来自文件、Packed 数据或 Blender 内存 +- **THEN** Operator 通过同一个 Bounds 图片序列化入口产生 AI 输入,不按来源建立不同业务主链 + +### Requirement: Refine may expand within Selection Bounds +系统 SHALL 将 Refine 的 Lasso 解释为矩形上下文范围,而不是 BEN2 输出的硬 Mask;最终 Refine Alpha SHALL 由 BEN2 在该矩形内的结果决定。 + +#### Scenario: BEN2 preserves content outside Lasso +- **WHEN** BEN2 在 Selection bounds 内保留了 Lasso 路径外的像素 +- **THEN** 系统保留这些像素,不再与原始 Lasso Selection Mask 相交 + +#### Scenario: BEN2 returns no visible content +- **WHEN** Refine 输出在矩形 bounds 内没有可见 Alpha +- **THEN** Refine 业务报告空内容且不创建 Image 或 Cutout 结果 + +### Requirement: AI operations reuse encoded local images +系统 SHALL 复用 AI 流程中已经编码的局部 RGBA 文件作为最终颜色 Image,不得重新读取完整源图并创建包含相同颜色内容的新内存 Image 后再次编码。 + +#### Scenario: Image Tool Refine completes +- **WHEN** BEN2 返回已收紧 bounds 的 Refine RGBA 文件 +- **THEN** Image Tool 直接加载、配置并 Pack 该文件,然后按响应 bounds 替换源 Image Empty + +#### Scenario: Cutout Refine completes +- **WHEN** Cutout 的 BEN2 Refine 返回局部 RGBA 文件 +- **THEN** Cutout 使用该文件的 Alpha 构建结果几何,并使用同一个已加载 Image 作为颜色纹理 + +#### Scenario: Cutout generates only Normal or Depth +- **WHEN** Cutout AI 流程没有 Refine 输出 +- **THEN** Cutout 直接加载并 Pack 提交前的 Bounds 输入作为颜色纹理 + +#### Scenario: AI operation cleans up +- **WHEN** 已有输入或 Refine 输出成功加载并 Pack,或者操作取消 +- **THEN** 系统按 Job 生命周期清理临时文件,且 Blender 结果不再依赖临时路径 diff --git a/openspec/changes/separate-selection-from-image-content/tasks.md b/openspec/changes/separate-selection-from-image-content/tasks.md new file mode 100644 index 0000000..df26f34 --- /dev/null +++ b/openspec/changes/separate-selection-from-image-content/tasks.md @@ -0,0 +1,30 @@ +## 1. 纯几何 Selection + +- [x] 1.1 删除 `rasterize_selection_path()` 的源 Alpha 与 Alpha threshold 参数,使其只按 Path、Invert、图片尺寸和抗锯齿返回几何 `SelectionMask` +- [x] 1.2 调整 Selection 的空路径、越界、Invert、局部画布和 bounds 行为,确保栅格阶段不再承担图片内容可见性判断 +- [x] 1.3 更新 Selection 单元测试,验证不同源 Alpha 不参与接口且相同几何输入始终产生相同 values 与 bounds + +## 2. 业务侧 Alpha 合并 + +- [x] 2.1 调整 Image Tool,在几何 bounds 内直接将 Selection 值乘入局部源 Alpha,并由编辑业务计算空内容与最终 placement bounds +- [x] 2.2 移除 Image Tool 对已知局部结果的重复逐像素坐标扫描,使用按行列可见范围完成内容收紧 +- [x] 2.3 调整本地 Cutout,使 BaseShape、空内容与几何处理显式消费局部源 Alpha 和纯 Selection 值,不修改 SelectionMask +- [x] 2.4 补充 Image Tool 与 Cutout 的透明、半透明、空内容、Invert 和内容 bounds 回归测试 + +## 3. 矩形 AI 输入 + +- [x] 3.1 将 `prepare_selection_input` 替换为统一的 Bounds 图片输入 helper,只裁切源 RGBA 并写出矩形局部文件,不乘入 Lasso Mask +- [x] 3.2 迁移 Image Tool Refine 与 Cutout Refine、Normal、Depth 调用链,使其共享同一个几何 bounds 输入协议 +- [x] 3.3 更新 Server 与交互测试,验证 BEN2 可以保留 bounds 内、Lasso 外内容,Normal 与 Depth 不再使用另一套 Masked 输入 + +## 4. 复用 AI 编码图片 + +- [x] 4.1 建立加载、配置并 Pack 已编码局部 RGBA 的公共入口,继承源颜色空间、Alpha mode、命名和 placement bounds 所需信息 +- [x] 4.2 调整 Image Tool Refine 响应,直接使用 BEN2 输出替换源 Image Empty,删除输出 Alpha 提取、源 RGBA 重读和 Pixel Result Image 重建 +- [x] 4.3 调整 Cutout AI 响应:Refine 使用 BEN2 输出,只有 Normal 或 Depth 时使用提交前输入,并让同一局部 Image 同时服务几何 Alpha 与颜色材质 +- [x] 4.4 验证成功、取消与异常路径只在结果已加载并 Pack 后清理临时输入,且不残留无用户的 Blender Image + +## 5. 文档与验证 + +- [x] 5.1 更新 Selection、Image Tool、Cutout Tool 与 AI 输出内部文档,区分几何 bounds、内容 bounds 和矩形上下文语义 +- [x] 5.2 运行相关测试与完整测试,并用真实高分辨率 RGBA 图片复测 Selection、AI 提交前准备和 AI 响应阶段,确认没有新增第二次图片编码 diff --git a/openspec/changes/simplify-selection-rasterization/.openspec.yaml b/openspec/changes/simplify-selection-rasterization/.openspec.yaml new file mode 100644 index 0000000..b4b3ece --- /dev/null +++ b/openspec/changes/simplify-selection-rasterization/.openspec.yaml @@ -0,0 +1,2 @@ +schema: spec-driven +created: 2026-09-01 diff --git a/openspec/changes/simplify-selection-rasterization/design.md b/openspec/changes/simplify-selection-rasterization/design.md new file mode 100644 index 0000000..7da3929 --- /dev/null +++ b/openspec/changes/simplify-selection-rasterization/design.md @@ -0,0 +1,78 @@ +## Context + +Selection 手势当前构建带 `points`、`invert`、`fill_rule` 和 JSON 方法的 `SelectionPath`。Image Tool 默认使用 `EVEN_ODD`,Cutout 使用 `NONZERO`;两者只在自交或重复绕行时不同。产品决定统一保留同方向重叠区域,因此删除 `EVEN_ODD`,同时保留 `SelectionPath` 及其 JSON 方法,继续把路径点和反选状态作为不可分割的值。 + +当前栅格器已经取得 Path bounds,却仍创建整图 Mask,并对每条路径边构造“扫描行块 × 选区宽度”的二维布尔数组。实测 5120×2880、1000 点 Lasso 的完整处理约为 4.07 秒。局部 non-zero scanline 原型通过每条边只记录实际穿过扫描行的区间端点,再对每行累计一次,在 960 组路径、Invert、抗锯齿、Alpha 和 bounds 组合中与目标 non-zero 结果逐值一致,同一 5K 用例约为 0.16 秒。 + +Cutout Pie Menu 的按钮会启动新的 Blender Operator,普通 Python 对象不能直接跨该 RNA 边界。现有完整 Path JSON 已同时保存 Points 与 Invert,千点路径编码实测约 0.6 ms;保留 `to_json()` / `from_json()` 比拆分多个 Property 或增加 RNA Collection 更直接。 + +## Goals / Non-Goals + +**Goals:** + +- 所有 Selection 使用 non-zero winding,保留同方向重叠区域。 +- 运行时传递只包含普通二维点数组和 `invert` 的精简 `SelectionPath`,删除 `fill_rule`。 +- Cutout 只在 Pie Menu 到 Shape Operator 的必要边界编码一次完整 Selection Path。 +- 将 Path-to-Mask 的计算量从逐边二维像素广播改为扫描行交点累计,并尽可能限制在局部画布。 +- 保持现有像素中心、Invert、抗锯齿核、Alpha 阈值、空 Selection、最终 bounds 和下游输出语义。 + +**Non-Goals:** + +- 不缓存 Blender 图片像素或 Selection,不引入令牌和全局状态。 +- 不改变 Lasso 采样与简化容差,不增加 Pillow、OpenCV 或 GPU 路径。 +- 不优化 Blender `image.pixels.foreach_get()`、图片写回、Pack、Cutout 网格或 AI Job。 +- 不保留旧 `fill_rule` 或兼容填充分支。 + +## Decisions + +### 1. Selection Path 只保存 Points 与 Invert + +`SelectionPath` 保留为不可变 dataclass,只包含 `points: tuple[tuple[float, float], ...]` 与 `invert: bool`,以及完整值的 `to_json()` / `from_json()`。构建时继续把点规范化为有限浮点二维数组、检查至少三个点,并固定反选值。共享手势完成边界提交一个完整 `SelectionPath`,Image Tool 在同一 Operator 调用栈内直接消费它。 + +`invert` 决定路径内部还是外部属于 Selection,与 `points` 共同定义同一个 Selection Path。保留二者的值对象及现有序列化边界可以避免调用链漏传或错配反选状态;删除 `fill_rule` 后,该类型不再承担工具填充策略。 + +### 2. Cutout 序列化完整 Selection Path + +`open_shape_pie` 对 `selection_path` 调用一次 `to_json()`,把同一个字符串写入各 Shape Operator 的 `selection_path_json`。`CutoutSelectionToShape` 通过 `SelectionPath.from_json()` 恢复 Points 与 Invert 后交给栅格器。JSON 保留 `points` 和 `invert`,只删除 `fill_rule`。 + +相比拆分 Points JSON 与 Bool Property,此方案不会产生两个必须同步的传输值;相比 RNA Collection,它不增加 PropertyGroup、注册顺序和每按钮逐点复制;相比令牌或全局表,它没有生命周期与菜单取消清理问题。 + +### 3. 使用 non-zero scanline 端点累计 + +栅格器先裁切路径的几何 bounds,并以像素中心 `x + 0.5`、`y + 0.5` 判断边界。每条非水平边只处理其实际跨越的扫描行,计算交点对应的水平区间终点;按边方向在该行的起点和终点写入正负 winding 变化。全部边处理完成后,每行只执行一次水平累计,累计值非零的像素即为 Selection。 + +这与当前 `NONZERO` 的射线 crossing 定义相同,但不再为每条边生成二维 `selected` 数组。相比 Pillow,此实现可以明确固定 winding、严格不等号与像素中心语义,并且无需为自交行为依赖第三方未公开约定。 + +### 4. 非 Invert 使用局部画布 + +普通 Selection 只创建裁切到图片范围的 Path bounds;启用抗锯齿时在四周增加核半径 2 px,使局部卷积与整图卷积等价。源 Alpha 只切片到同一局部范围。由于 Invert 的有效区域可能覆盖路径外的整图,Invert 继续使用完整图片范围,但仍采用单次 scanline 累计。 + +最终可见 bounds 使用按行和按列的 `any()` 后查找首尾非空索引,不再通过 `np.nonzero()` 为每个命中像素创建两组坐标。返回值继续是局部 `SelectionMask(values, bounds)`。 + +### 5. 用结果等价和工作边界验证性能 + +小尺寸参考实现覆盖普通、凹、自交、重复绕行、越界、水平边、Invert、抗锯齿、Alpha 阈值和空 Selection,并逐值比较 Mask 与 bounds。高分辨率测试验证中间工作数组只按目标画布建立、每条边只产生其跨行事件,不设置依赖 CPU 的秒数断言。 + +开发验证继续记录代表性 5K/1000 点 Lasso 的基准,用于确认实际数量级改善,但不把机器耗时写成测试契约。 + +## Risks / Trade-offs + +- [Image Tool 自交路径结果改变] → 这是明确的产品决策;更新预览和最终 Mask 测试,使重叠区始终保留。 +- [局部抗锯齿在边缘与旧结果不同] → 按固定核半径扩展局部画布,并用逐值测试覆盖图片边缘和越界路径。 +- [Invert 无法取得同等局部内存收益] → 保留整图画布以维持语义,但仍消除逐边二维广播和巨大 `nonzero` 坐标数组。 +- [扫描线交点在整数顶点重复计数] → 沿用半开 crossing 条件和像素中心判定,以参考实现测试水平边、共顶点与自交路径。 +- [Path JSON 仍包含已删除的填充规则] → 更新序列化测试,验证载荷只包含 Points 与 Invert,且不保留旧填充分支。 + +## Migration Plan + +1. 将预览与 Mask 参考语义统一为 non-zero,调整差异测试为重叠区保留测试。 +2. 将 `SelectionPath` 精简为 Points 与 Invert,删除 `fill_rule` 并更新 JSON 载荷,迁移共享手势、Image Tool 与 Cutout 的参数边界。 +3. 保持 Cutout 的单一 Path JSON Property,由其恢复精简 Path,删除旧填充字段解析。 +4. 实现局部 scanline、局部抗锯齿/Alpha 和低内存 bounds,迁移所有调用方。 +5. 更新测试和内部文档,运行相关测试、完整测试及代表性 5K 基准。 + +回退时整体恢复旧 Selection Path 字段和栅格器,不保留两套入口或兼容层。 + +## Open Questions + +无。 diff --git a/openspec/changes/simplify-selection-rasterization/proposal.md b/openspec/changes/simplify-selection-rasterization/proposal.md new file mode 100644 index 0000000..3a6cb8c --- /dev/null +++ b/openspec/changes/simplify-selection-rasterization/proposal.md @@ -0,0 +1,28 @@ +## Why + +当前 Selection 栅格化按路径的每条边重复扫描二维像素区域,5K 图片上的千点 Lasso 单次处理约需 4 秒并产生大量整图临时数组。`SelectionPath` 需要共同保存路径点与反选状态,但其中的两种填充规则没有继续保留的价值;产品实际期望所有重叠圈选区域都被保留。 + +## What Changes + +- **BREAKING**:所有 Selection 统一使用 non-zero winding 填充,删除 `EVEN_ODD`、`fill_rule` 及其分支;自交或重复绕行的同方向重叠区域始终保留。 +- **BREAKING**:将 `SelectionPath` 精简为只包含普通二维点数组和 `invert` 的不可变值,删除 `fill_rule`,保留完整 Path 的 `to_json()` / `from_json()`。 +- Cutout Pie Menu 继续把完整 `SelectionPath` 编码一次并交给 Shape Operator,不拆分 Points 与 Invert,也不引入 RNA Collection、令牌或额外状态。 +- 使用按扫描行交点累计的局部 scanline 算法替换逐边二维像素广播,并保持现有像素中心判定、Invert、抗锯齿、Alpha 阈值、空 Selection 与最终 bounds 语义。 +- 抗锯齿、源 Alpha 合并和可见 bounds 计算只处理必要的局部画布;Invert Selection 仍按完整图片范围处理。 +- 增加逐像素等价性、不同路径规模的复杂度边界和高分辨率性能回归测试,不使用机器相关的固定耗时阈值。 + +## Capabilities + +### New Capabilities + +- `selection-rasterization`: 规定精简 Selection Path 的统一 non-zero 填充语义、局部 scanline 栅格化结果和高分辨率复杂度边界。 + +### Modified Capabilities + +无。 + +## Impact + +- 影响 `common/selection.py`、`common/viewport.py`、Image Tool、Cutout Tool 及其交互测试和内部文档。 +- Cutout Operator 的内部 Selection 参数会改变;它不是公开用户 API,但旧 Operator 参数不再兼容。 +- 不新增运行时依赖,不修改 AI Job 协议、Cutout Shape、Mask 抗锯齿核或最终图片与几何处理链。 diff --git a/openspec/changes/simplify-selection-rasterization/specs/selection-rasterization/spec.md b/openspec/changes/simplify-selection-rasterization/specs/selection-rasterization/spec.md new file mode 100644 index 0000000..06592e8 --- /dev/null +++ b/openspec/changes/simplify-selection-rasterization/specs/selection-rasterization/spec.md @@ -0,0 +1,64 @@ +## ADDED Requirements + +### Requirement: Selection uses one overlap-preserving fill +系统 SHALL 使用 non-zero winding 栅格化所有 Selection Path,使同方向自交或重复绕行的重叠区域保持选中,并且不得暴露可切换的填充规则。 + +#### Scenario: Lasso overlaps itself +- **WHEN** Image Tool 或 Cutout 的 Selection 路径以同一方向重复包围一个区域 +- **THEN** 该重叠区域在 Viewport 预览和最终 Selection Mask 中均保持选中 + +#### Scenario: Simple path is selected +- **WHEN** Selection 路径是普通矩形、半平面、凹多边形或不自交 Lasso +- **THEN** 系统产生与既有 non-zero 填充相同的选中区域 + +### Requirement: Selection Path contains its inversion state +系统 SHALL 使用只包含普通二维点数组和 Invert 的不可变 Selection Path 值在用户手势与实际 Image 或 Cutout 操作之间传递 Selection;该类型 SHALL 保留完整值的 JSON 编解码,但不得包含填充规则。 + +#### Scenario: Image Tool submits a Selection +- **WHEN** 用户完成 Image Tool 手势 +- **THEN** 同一 Operator 把包含 Points 与 Invert 的 Selection Path 直接交给实际图片编辑 + +#### Scenario: Cutout menu creates Shape operations +- **WHEN** Cutout Pie Menu 为当前点数组创建 Shape 按钮 +- **THEN** 菜单将包含 Points 与 Invert 的完整 Path 编码一次,并为每个 Shape Operator 复用同一字符串 + +#### Scenario: Cutout Shape operation restores the Selection +- **WHEN** 用户点击任一 Cutout Shape 按钮 +- **THEN** Shape Operator 从单一 Path JSON 恢复 Points 与 Invert 后再进行栅格化 + +### Requirement: Selection rasterization uses scanline events +系统 SHALL 通过路径边与扫描行的交点事件构建 non-zero winding Mask,每条路径边不得重复创建扫描行块乘以 Selection 宽度的二维像素比较数组。 + +#### Scenario: High-resolution Lasso is rasterized +- **WHEN** 系统处理高分辨率图片上的多点 Lasso +- **THEN** 每条边只为实际跨越的扫描行产生端点事件,所有端点完成后每行执行一次水平累计 + +#### Scenario: Path point count increases +- **WHEN** Selection 画布不变而路径点数增加 +- **THEN** 栅格器增加交点事件处理,不按每个新增点重新扫描完整二维 Selection 像素区域 + +### Requirement: Non-inverted Selection processing stays local +系统 SHALL 对非 Invert Selection 只在裁切后的路径范围内执行填充、抗锯齿、源 Alpha 合并和可见 bounds 计算;抗锯齿范围 MUST 包含固定卷积核所需的边缘 padding。 + +#### Scenario: Local Selection is rasterized +- **WHEN** 非 Invert Selection 只覆盖源图的一部分 +- **THEN** 中间 Mask 和 Alpha 运算范围限制在路径 bounds 及必要的抗锯齿 padding + +#### Scenario: Inverted Selection is rasterized +- **WHEN** Selection 启用 Invert +- **THEN** 系统使用完整图片范围表达路径外区域,但仍只累计一次 scanline 事件 + +### Requirement: Optimized rasterization preserves Mask semantics +系统 MUST 保持现有像素中心判定、Invert、抗锯齿核、Alpha 阈值、空 Selection 警告和最终局部 Selection Mask bounds 语义。 + +#### Scenario: Selection has source Alpha +- **WHEN** Selection Path 与源图 Alpha 一起栅格化 +- **THEN** 系统只保留 Alpha 高于阈值且 non-zero winding Mask 有效的像素,并返回其紧致 bounds + +#### Scenario: Selection has no visible pixels +- **WHEN** Selection 与图片范围或可见源 Alpha 没有交集 +- **THEN** 系统报告空 Selection,且不向后续 Image、Cutout 或 AI 操作提交结果 + +#### Scenario: Equivalent Selection is processed +- **WHEN** 相同 Selection Path、源 Alpha、阈值和抗锯齿设置由参考实现与优化实现处理 +- **THEN** 两者产生逐值一致的 Mask values 和相同 bounds diff --git a/openspec/changes/simplify-selection-rasterization/tasks.md b/openspec/changes/simplify-selection-rasterization/tasks.md new file mode 100644 index 0000000..2c9eaf1 --- /dev/null +++ b/openspec/changes/simplify-selection-rasterization/tasks.md @@ -0,0 +1,20 @@ +## 1. 简化 Selection 值与填充语义 + +- [x] 1.1 将 Viewport 预览和最终 Mask 统一为 non-zero winding,删除 `EVEN_ODD`、`fill_rule` 参数与分支,并用自交和重复绕行测试验证重叠区始终保留 +- [x] 1.2 将 `SelectionPath` 精简为只包含 Points 与 Invert 的不可变值,保留完整值的 `to_json()` / `from_json()`,使共享手势向 Image Tool 和 Cutout Tool 提交完整 Path,并迁移反选相关测试 +- [x] 1.3 保持 Cutout Pie Menu 到 `CutoutSelectionToShape` 的单一 `selection_path_json` 边界,用交互测试验证只编码一次、各 Shape 按钮复用同一字符串且恢复相同 Points 与 Invert +- [x] 1.4 删除 `fill_rule` 的序列化、解析和所有兼容入口,使用定向搜索验证业务代码与 Path JSON 均不再包含填充规则 + +## 2. 局部 Scanline 栅格化 + +- [x] 2.1 实现 non-zero scanline 端点累计,使每条边只处理实际跨越的像素中心扫描行,并用整数顶点、水平边、凹路径、自交与越界路径测试验证结果 +- [x] 2.2 对非 Invert Selection 只建立裁切后的局部画布,抗锯齿时按 5-tap 核增加 2 px padding;对 Invert 保留完整图片画布并验证路径外语义 +- [x] 2.3 将源 Alpha 合并限制到当前画布切片,并以按行、按列归约计算紧致 bounds,删除整图 `np.where` 与逐命中像素坐标数组 +- [x] 2.4 保持 `SelectionMask`、空 Selection 警告、Image 编辑、AI 输入和 Cutout 几何调用边界不变,用现有像素与几何测试验证下游结果 + +## 3. 等价性、性能边界与文档 + +- [x] 3.1 增加小尺寸参考栅格器测试,组合覆盖普通与随机路径、Invert、抗锯齿、随机 Alpha、阈值和空结果,逐值比较 Mask values 与 bounds +- [x] 3.2 增加 5K 多点 Lasso 的工作边界测试,验证不出现逐边的扫描行块乘以宽度数组、局部 Selection 不建立整图 Mask,且不使用机器相关耗时阈值 +- [x] 3.3 更新 `docs/internals` 的 Selection 值、Cutout 传输边界、统一 non-zero 语义与局部 scanline 流程,不记录旧兼容路径 +- [x] 3.4 运行 Image Selection、Image Tool、Cutout 交互与几何相关测试,运行代表性 5K 基准,再运行 `uv run pytest` 确认全部通过 diff --git a/src/anyimage/assets/O_AnyImage.blend b/src/anyimage/assets/O_AnyImage.blend index 5735c14..d0a8a3a 100644 --- a/src/anyimage/assets/O_AnyImage.blend +++ b/src/anyimage/assets/O_AnyImage.blend @@ -1,3 +1,3 @@ version https://git-lfs.github.com/spec/v1 -oid sha256:0c4ff876a695bbc1af419a6007b47cced337c611fae0c5c519618623280182af -size 1638353 +oid sha256:ac265c4cbd1e5ca1ad359e060ddb7d164474690db2176b1136828062b17aa5c6 +size 1642604 diff --git a/src/anyimage/common/image.py b/src/anyimage/common/image.py index 9fd157d..d632551 100644 --- a/src/anyimage/common/image.py +++ b/src/anyimage/common/image.py @@ -262,21 +262,29 @@ def prepare_image_input(image, scene): return source_path, False -def prepare_selection_input(image, selection_mask): - """Write one cropped RGBA Selection input owned by its temporary directory.""" +def prepare_image_bounds_input(image, source_rgba, bounds): + """Write one rectangular RGBA input owned by its temporary directory.""" import numpy as np - left, top, right, bottom = selection_mask.bounds - rgba = image_rgba(image) - cropped = rgba[top:bottom, left:right].copy() - if cropped.shape[:2] != selection_mask.values.shape: - raise ValueError("The Selection does not match the source image") - cropped[:, :, 3] *= selection_mask.values - cropped[cropped[:, :, 3] <= 0.0, :3] = 0.0 - output_directory = Path(tempfile.mkdtemp(prefix="anyimage-selection-")) + left, top, right, bottom = (int(value) for value in bounds) + rgba = np.asarray(source_rgba, dtype=np.float32) + expected_shape = (int(image.size[1]), int(image.size[0]), 4) + if rgba.shape != expected_shape: + raise ValueError("The source RGBA does not match the Blender Image") + if ( + left < 0 + or top < 0 + or right > expected_shape[1] + or bottom > expected_shape[0] + ): + raise ValueError("The image bounds are outside the Blender Image") + if right <= left or bottom <= top: + raise ValueError("The image bounds are empty") + cropped = rgba[top:bottom, left:right] + output_directory = Path(tempfile.mkdtemp(prefix="anyimage-bounds-")) output_path = output_directory / "input.png" temporary_image = bpy.data.images.new( - "AnyImage Selection Input", + "AnyImage Bounds Input", width=right - left, height=bottom - top, alpha=True, @@ -288,7 +296,7 @@ def prepare_selection_input(image, selection_mask): temporary_image.pixels.foreach_set(np.flipud(cropped).ravel()) temporary_image.file_format = "PNG" temporary_image.save(filepath=str(output_path), save_copy=True) - except (OSError, RuntimeError): + except Exception: cleanup_image_input(output_path, True) raise finally: @@ -296,10 +304,26 @@ def prepare_selection_input(image, selection_mask): bpy.data.images.remove(temporary_image, do_unlink=True) if not output_path.is_file(): cleanup_image_input(output_path, True) - raise RuntimeError("Blender did not export the Selection input") + raise RuntimeError("Blender did not export the bounds image input") return output_path +def load_packed_rgba_image(path, source_image, name): + """Load an encoded local RGBA file and make its Blender data self-contained.""" + result_image = None + try: + result_image = bpy.data.images.load(str(path), check_existing=False) + result_image.colorspace_settings.name = source_image.colorspace_settings.name + result_image.alpha_mode = source_image.alpha_mode + result_image.name = name + result_image.pack() + except (OSError, RuntimeError, TypeError, ValueError) as error: + if result_image is not None: + bpy.data.images.remove(result_image, do_unlink=True) + raise RuntimeError("Unable to prepare the RGBA image result") from error + return result_image + + def configure_image_user( image_user, frame_count=None, diff --git a/src/anyimage/common/selection.py b/src/anyimage/common/selection.py index cc5744f..837ce9d 100644 --- a/src/anyimage/common/selection.py +++ b/src/anyimage/common/selection.py @@ -13,7 +13,6 @@ class ImageEditWarning(ValueError): class SelectionPath: points: tuple[tuple[float, float], ...] invert: bool = False - fill_rule: str = "EVEN_ODD" def __post_init__(self): points = tuple( @@ -21,8 +20,6 @@ class SelectionPath: for point in self.points ) selection_path_bounds(points) - if self.fill_rule not in {"EVEN_ODD", "NONZERO"}: - raise ValueError(f"Unknown Selection Path fill rule: {self.fill_rule}") object.__setattr__(self, "points", points) object.__setattr__(self, "invert", bool(self.invert)) @@ -31,7 +28,6 @@ class SelectionPath: { "points": self.points, "invert": self.invert, - "fill_rule": self.fill_rule, }, separators=(",", ":"), ) @@ -39,12 +35,11 @@ class SelectionPath: @classmethod def from_json(cls, value): data = json.loads(value) - if not isinstance(data, dict): + if not isinstance(data, dict) or set(data) != {"points", "invert"}: raise ValueError("The Selection Path data is invalid") return cls( - points=tuple(data.get("points", ())), - invert=bool(data.get("invert", False)), - fill_rule=str(data.get("fill_rule", "EVEN_ODD")), + points=tuple(data["points"]), + invert=bool(data["invert"]), ) @@ -108,45 +103,46 @@ def selection_path_bounds(points): return left, top, right, bottom -def _rasterize_path(image_size, points, fill_rule): +def _rasterize_path(bounds, points): import numpy as np + left, top, right, bottom = bounds + width = right - left + height = bottom - top + points = np.asarray(points, dtype=np.float64) + y = np.arange(top, bottom, dtype=np.float64) + 0.5 + changes = np.zeros((height, width + 1), dtype=np.int16) + previous = points[-1] + for current in points: + rows = np.flatnonzero((current[1] > y) != (previous[1] > y)) + if len(rows): + sample_y = y[rows] + edge_x = (previous[0] - current[0]) * (sample_y - current[1]) / ( + previous[1] - current[1] + 1e-300 + ) + current[0] + ends = np.ceil(edge_x - left - 0.5).astype(np.int64) + ends = np.clip(ends, 0, width) + starts = np.zeros_like(rows) + winding = 1 if current[1] > previous[1] else -1 + np.add.at(changes, (rows, starts), winding) + np.add.at(changes, (rows, ends), -winding) + previous = current + return np.cumsum(changes[:, :width], axis=1, dtype=np.int16) != 0 + + +def _selection_canvas_bounds(image_size, selection_path, padding): width, height = (int(value) for value in image_size) if width <= 0 or height <= 0: raise ValueError("The source image has no readable dimensions") - left, top, right, bottom = selection_path_bounds(points) - left = max(left, 0) - top = max(top, 0) - right = min(right, width) - bottom = min(bottom, height) - mask = np.zeros((height, width), dtype=bool) - if right <= left or bottom <= top: - return mask - - points = np.asarray(points, dtype=np.float64) - x = np.arange(left, right, dtype=np.float64) + 0.5 - for row_start in range(top, bottom, 128): - row_end = min(row_start + 128, bottom) - y = np.arange(row_start, row_end, dtype=np.float64) + 0.5 - state = np.zeros( - (len(y), len(x)), - dtype=bool if fill_rule == "EVEN_ODD" else np.int16, - ) - previous = points[-1] - for current in points: - crosses = (current[1] > y) != (previous[1] > y) - edge_x = (previous[0] - current[0]) * (y - current[1]) / ( - previous[1] - current[1] + 1e-300 - ) + current[0] - selected = crosses[:, None] & (x[None, :] < edge_x[:, None]) - if fill_rule == "EVEN_ODD": - state ^= selected - else: - winding = 1 if current[1] > previous[1] else -1 - state += selected * winding - previous = current - mask[row_start:row_end, left:right] = state != 0 - return mask + if selection_path.invert: + return 0, 0, width, height + left, top, right, bottom = selection_path_bounds(selection_path.points) + return ( + max(left - padding, 0), + max(top - padding, 0), + min(right + padding, width), + min(bottom + padding, height), + ) def _antialias_mask(mask): @@ -170,40 +166,29 @@ def rasterize_selection_path( image_size, selection_path, *, - source_alpha=None, - alpha_threshold=0.0, antialias=False, ): import numpy as np width, height = (int(value) for value in image_size) - mask = _rasterize_path( + canvas_bounds = _selection_canvas_bounds( (width, height), - selection_path.points, - selection_path.fill_rule, + selection_path, + 2 if antialias else 0, ) + left, top, right, bottom = canvas_bounds + if right <= left or bottom <= top: + raise ImageEditWarning("The selection contains no visible image pixels") + mask = _rasterize_path(canvas_bounds, selection_path.points) if antialias: mask = _antialias_mask(mask) else: mask = mask.astype(np.float32) if selection_path.invert: mask = 1.0 - mask - if source_alpha is not None: - alpha = np.asarray(source_alpha) - if alpha.shape != (height, width): - raise ValueError("The source Alpha does not match the image selection") - mask = np.where(alpha > float(alpha_threshold), mask, 0.0) - visible_y, visible_x = np.nonzero(mask > float(alpha_threshold)) - if not len(visible_x): + if not np.any(mask > 0.0): raise ImageEditWarning("The selection contains no visible image pixels") - bounds = ( - int(visible_x.min()), - int(visible_y.min()), - int(visible_x.max()) + 1, - int(visible_y.max()) + 1, - ) - left, top, right, bottom = bounds - return SelectionMask(mask[top:bottom, left:right], bounds) + return SelectionMask(mask, canvas_bounds) def crop_pixels_to_alpha(pixels, bounds): diff --git a/src/anyimage/common/viewport.py b/src/anyimage/common/viewport.py index 1485b7d..07d0251 100644 --- a/src/anyimage/common/viewport.py +++ b/src/anyimage/common/viewport.py @@ -326,12 +326,10 @@ def triangulated_polygon_vertices(vertices): return _triangulated_polygon_vertices_python(vertices) -def scanline_fill_triangles(vertices, fill_rule="EVEN_ODD", step=2.0): +def scanline_fill_triangles(vertices, step=2.0): vertices = tuple(tuple(float(value) for value in vertex) for vertex in vertices) if len(vertices) < 3: return () - if fill_rule not in {"EVEN_ODD", "NONZERO"}: - raise ValueError(f"Unknown polygon fill rule: {fill_rule}") minimum_y = min(vertex[1] for vertex in vertices) maximum_y = max(vertex[1] for vertex in vertices) band_bottom = minimum_y @@ -358,17 +356,11 @@ def scanline_fill_triangles(vertices, fill_rule="EVEN_ODD", step=2.0): while index < len(events): x = events[index][0] delta = 0 - count = 0 while index < len(events) and abs(events[index][0] - x) <= 1e-8: delta += events[index][1] - count += 1 index += 1 was_inside = winding != 0 - if fill_rule == "EVEN_ODD": - if count % 2: - winding = 0 if was_inside else 1 - else: - winding += delta + winding += delta is_inside = winding != 0 if not was_inside and is_inside: interval_start = x @@ -454,12 +446,12 @@ def polygon_self_intersects(vertices, cell_size=32.0): return False -def preview_fill_triangles(vertices, fill_rule="EVEN_ODD"): +def preview_fill_triangles(vertices): if not polygon_self_intersects(vertices): triangles = triangulated_polygon_vertices(vertices) if triangles: return triangles - return scanline_fill_triangles(vertices, fill_rule) + return scanline_fill_triangles(vertices) def draw_polygon_fill( @@ -609,7 +601,6 @@ def image_line_half_plane_polygon(line, image_size): class ImageSelectionGesture: active_tool_id = None - preview_fill_rule = "EVEN_ODD" _path = None _cursor = None _handle = None @@ -658,9 +649,7 @@ class ImageSelectionGesture: if len(polygon) >= 3: if polygon != self._fill_polygon: self._fill_polygon = polygon - self._fill_triangles = preview_fill_triangles( - polygon, self.preview_fill_rule - ) + self._fill_triangles = preview_fill_triangles(polygon) draw_polygon_fill( polygon, self._fill_triangles, @@ -725,6 +714,9 @@ class ImageSelectionGesture: def cancel(self, context): self._finish(context) + def _submit_selection_path(self, _context, _selection_path): + raise NotImplementedError + def invoke(self, context, event): source_object, should_edit = resolve_image_edit_click( context, event, self.report @@ -740,7 +732,6 @@ class ImageSelectionGesture: self.invert = False self.source_object_name = source_object.name self.source_matrix_data = serialize_matrix(source_object.matrix_world) - self.selection_path_json = "" point = (float(event.mouse_region_x), float(event.mouse_region_y)) self._path = [point] self._cursor = point @@ -787,13 +778,12 @@ class ImageSelectionGesture: image_empty_bounds(source_object), tuple(source_object.data.size), ) - self.selection_path_json = SelectionPath( + selection_path = SelectionPath( points=selection_path, invert=self.invert, - fill_rule=self.preview_fill_rule, - ).to_json() + ) self._finish(context) - return self.execute(context) + return self._submit_selection_path(context, selection_path) def modal(self, context, event): active_tool = active_view3d_tool_id(context) diff --git a/src/anyimage/operators/convert_to_plane/conversion.py b/src/anyimage/operators/convert_to_plane/conversion.py index 67f1f59..51845e0 100644 --- a/src/anyimage/operators/convert_to_plane/conversion.py +++ b/src/anyimage/operators/convert_to_plane/conversion.py @@ -50,7 +50,7 @@ class GenerateDepthPlane(JobOperatorBase, bpy.types.Operator): input_path: bpy.props.StringProperty(options=_HIDDEN) source_object_name: bpy.props.StringProperty(options=_HIDDEN) - mesh_detail: bpy.props.IntProperty(default=0, min=0, max=10, options=_HIDDEN) + mesh_detail: bpy.props.IntProperty(default=6, min=0, max=10, options=_HIDDEN) model: bpy.props.StringProperty(default=DEFAULT_MOGE2_MODEL_KEY, options=_HIDDEN) resolution_level: bpy.props.IntProperty(default=5, min=0, max=9, options=_HIDDEN) max_input_size: bpy.props.IntProperty( @@ -160,7 +160,7 @@ class ConvertToDepthPlane(bpy.types.Operator): bl_description = "Convert the complete Image Empty into a front-projected relief" bl_options = {"REGISTER", "UNDO"} - mesh_detail: bpy.props.IntProperty(name="Subdivide", default=0, min=0, max=10) + mesh_detail: bpy.props.IntProperty(name="Subdivide", default=6, min=0, max=10) @classmethod def poll(cls, context): diff --git a/src/anyimage/operators/convert_to_plane/mesh_plane.py b/src/anyimage/operators/convert_to_plane/mesh_plane.py index cdfdfba..86332e3 100644 --- a/src/anyimage/operators/convert_to_plane/mesh_plane.py +++ b/src/anyimage/operators/convert_to_plane/mesh_plane.py @@ -9,18 +9,29 @@ from ...common.node import load_node_group MESH_PLANE_NODE_GROUP_NAME = "O Mesh Plane" +def build_mesh_plane_matrix(matrix_world, bounds): + from mathutils import Matrix + + x_min, x_max, y_min, y_max = bounds + center = ((x_min + x_max) * 0.5, (y_min + y_max) * 0.5, 0.0) + return matrix_world @ Matrix.Translation(center) + + def create_mesh_plane_mesh(name, bounds): x_min, x_max, y_min, y_max = bounds if x_max <= x_min or y_max <= y_min: raise ValueError("The plane has invalid dimensions") + half_width = (x_max - x_min) * 0.5 + half_height = (y_max - y_min) * 0.5 + mesh = bpy.data.meshes.new(name) mesh.from_pydata( ( - (x_min, y_min, 0.0), - (x_max, y_min, 0.0), - (x_max, y_max, 0.0), - (x_min, y_max, 0.0), + (-half_width, -half_height, 0.0), + (half_width, -half_height, 0.0), + (half_width, half_height, 0.0), + (-half_width, half_height, 0.0), ), (), ((0, 1, 2, 3),), @@ -37,7 +48,6 @@ def create_mesh_plane_mesh(name, bounds): def add_mesh_plane_processing( plane_object, - material, subdivisions=0, thickness=0.0, ): @@ -52,7 +62,6 @@ def add_mesh_plane_processing( for name, value in ( ("Subdivide", max(0, int(subdivisions))), ("Thickness", max(0.0, float(thickness))), - ("Material", material), ): identifier = modifier_input_identifier(node_group, name) set_modifier_input(modifier, identifier, value) @@ -72,11 +81,13 @@ def create_mesh_plane_object( mesh.materials.append(material) plane = bpy.data.objects.new(name, mesh) context.collection.objects.link(plane) - if matrix_world is not None: - plane.matrix_world = matrix_world.copy() + if matrix_world is None: + from mathutils import Matrix + + matrix_world = Matrix.Identity(4) + plane.matrix_world = build_mesh_plane_matrix(matrix_world, bounds) add_mesh_plane_processing( plane, - material, subdivisions=subdivisions, thickness=thickness, ) diff --git a/src/anyimage/operators/convert_to_plane/object.py b/src/anyimage/operators/convert_to_plane/object.py index bc734c9..16aa973 100644 --- a/src/anyimage/operators/convert_to_plane/object.py +++ b/src/anyimage/operators/convert_to_plane/object.py @@ -7,6 +7,7 @@ from ...common.image import ( image_empty_bounds, image_pixels, load_color_result_image, + load_normal_result_image, require_image_empty, ) from ...common.depth import ( @@ -21,7 +22,11 @@ from ...common.object import ( set_modifier_input, ) from ...common.node import load_node_group -from .mesh_plane import create_mesh_plane_mesh, create_mesh_plane_object +from .mesh_plane import ( + build_mesh_plane_matrix, + create_mesh_plane_mesh, + create_mesh_plane_object, +) DEPTH_PLANE_NODE_GROUP_NAME = "O Image Depth Plane" @@ -65,26 +70,27 @@ def create_depth_plane_object( depth_metadata, ): source_name = source_object.name - mesh = create_mesh_plane_mesh(source_name, image_empty_bounds(source_object)) + bounds = image_empty_bounds(source_object) + mesh = create_mesh_plane_mesh(source_name, bounds) for polygon in mesh.polygons: polygon.use_smooth = True mesh.materials.append(material) plane = bpy.data.objects.new(source_name, mesh) context.collection.objects.link(plane) - plane.matrix_world = source_object.matrix_world.copy() + plane.matrix_world = build_mesh_plane_matrix(source_object.matrix_world, bounds) plane["anyimage_mesh_shape"] = "DEPTH_PLANE" node_group = load_node_group(DEPTH_PLANE_NODE_GROUP_NAME) modifier = plane.modifiers.new(name=node_group.name, type="NODES") modifier.node_group = node_group modifier.show_group_selector = False + uniform_scale = depth_uniform_scale(source_object, depth_metadata) for name, value in ( ("Subdivide", max(0, min(10, int(subdivisions)))), - ("Material", material), ("Thickness", 0.1), - ("Depth Amount", 1.0), - ("Reference Depth", depth_metadata["reference_depth"]), - ("Uniform Scale", depth_uniform_scale(source_object, depth_metadata)), + ("Depth Scale", 1.0), + ("Base Plane Depth", depth_metadata["reference_depth"] * uniform_scale), + ("Uniform Scale", uniform_scale), ("Depth Image", depth_image), ): set_modifier_input( @@ -99,19 +105,29 @@ def create_depth_plane_from_result(context, result, operator): source_object = require_image_empty(operator.source_object_name) color_image = load_color_result_image(source_object, result.directory) color_image.name = color_data_name(source_object) - material = create_image_material( - source_object, - color_image, - 0.0, - scene=context.scene, - ) - depth_metadata = load_depth_metadata(result.file("depth_metadata")) - depth_image = load_depth_result_image( - result.file("z_depth"), - source_object, - depth_metadata, - ) + material = None + depth_image = None + normal_image = None try: + depth_metadata = load_depth_metadata(result.file("depth_metadata")) + depth_image = load_depth_result_image( + result.file("z_depth"), + source_object, + depth_metadata, + ) + normal_image = load_normal_result_image( + result.file("tangent_normal"), + source_object, + ) + material = create_image_material( + source_object, + color_image, + 0.0, + normal_image=normal_image, + normal_space="TANGENT", + ior=1.2, + scene=context.scene, + ) create_depth_plane_object( context, source_object, @@ -121,8 +137,11 @@ def create_depth_plane_from_result(context, result, operator): depth_metadata, ) except Exception: - if depth_image.users == 0: - bpy.data.images.remove(depth_image, do_unlink=True) + if material is not None and material.users == 0: + bpy.data.materials.remove(material, do_unlink=True) + for image in (depth_image, normal_image): + if image is not None and image.users == 0: + bpy.data.images.remove(image, do_unlink=True) raise if "FINISHED" not in bpy.ops.ed.undo_push(message="Convert to Depth Plane"): raise RuntimeError("Unable to create the Convert to Depth Plane Undo step") diff --git a/src/anyimage/operators/cutout_tool/geometry.py b/src/anyimage/operators/cutout_tool/geometry.py index 233b7c6..de8f419 100644 --- a/src/anyimage/operators/cutout_tool/geometry.py +++ b/src/anyimage/operators/cutout_tool/geometry.py @@ -202,10 +202,10 @@ def _resize_values(values, size): ) -def extract_selection_contours(selection_mask, max_resolution=512, threshold=0.5): +def extract_selection_contours(selection_values, max_resolution=512, threshold=0.5): import numpy as np - values = np.asarray(selection_mask.values) + values = np.asarray(selection_values) if values.ndim != 2: raise ValueError("Cutout Alpha must be two-dimensional") if np.issubdtype(values.dtype, np.floating): @@ -1341,13 +1341,13 @@ def crop_plane_bounds(source_object, crop_bounds, source_size): def extract_cutout_contours( - selection_mask, + content_values, mesh_resolution, threshold=CUTOUT_ALPHA_THRESHOLD, ): contour_resolution = max(512, min(int(mesh_resolution) * 8, 1024)) return extract_selection_contours( - selection_mask, + content_values, max_resolution=contour_resolution, threshold=threshold, ) @@ -1373,11 +1373,11 @@ def cutout_sampling_spacing(source_object, mesh_resolution): return max(width, height) / max(int(mesh_resolution), 8) -def selection_calibration(selection_mask, target_shape, threshold=CUTOUT_ALPHA_THRESHOLD): - """Return the final Selection mask and transparency ratio at geometry resolution.""" +def selection_calibration(content_values, target_shape, threshold=CUTOUT_ALPHA_THRESHOLD): + """Return Cutout validity and transparency ratio at geometry resolution.""" import numpy as np - values = np.asarray(selection_mask.values, dtype=np.float32) + values = np.asarray(content_values, dtype=np.float32) target_height, target_width = (int(value) for value in target_shape) source_height, source_width = values.shape y = np.minimum( @@ -1394,16 +1394,22 @@ def selection_calibration(selection_mask, target_shape, threshold=CUTOUT_ALPHA_T return valid, transparent_fraction -def build_base_shape(context, source_object, mesh_resolution, selection_mask): +def build_base_shape( + context, + source_object, + mesh_resolution, + content_values, + content_bounds, +): import numpy as np bounds = crop_plane_bounds( source_object, - selection_mask.bounds, + content_bounds, tuple(source_object.data.size), ) selection_contours = extract_cutout_contours( - selection_mask, + content_values, mesh_resolution, threshold=CUTOUT_ALPHA_THRESHOLD, ) diff --git a/src/anyimage/operators/cutout_tool/interaction.py b/src/anyimage/operators/cutout_tool/interaction.py index b84d7c8..a9eb206 100644 --- a/src/anyimage/operators/cutout_tool/interaction.py +++ b/src/anyimage/operators/cutout_tool/interaction.py @@ -5,9 +5,6 @@ from ...preferences import ( MIN_CUTOUT_MESH_RESOLUTION, ) from ...properties import ai_ready -from ...common.image import image_rgba -from ...common.selection import rasterize_selection_path -from .geometry import CUTOUT_ALPHA_THRESHOLD from .shape import CUTOUT_SHAPE_LABELS CUTOUT_SHAPE_BUTTONS = ( @@ -25,21 +22,6 @@ CUTOUT_MESH_DETAIL_SPACING = { } -def validate_cutout_selection( - source_object, - selection_path, -): - image = source_object.data - rgba = image_rgba(image) - rasterize_selection_path( - tuple(image.size), - selection_path, - source_alpha=rgba[:, :, 3], - alpha_threshold=CUTOUT_ALPHA_THRESHOLD, - antialias=True, - ) - - def cutout_mesh_resolution(image_size, point_spacing): longest_edge = max(int(value) for value in image_size) detail = round(longest_edge / max(int(point_spacing), 1)) diff --git a/src/anyimage/operators/cutout_tool/main.py b/src/anyimage/operators/cutout_tool/main.py index bc9b279..9430a9e 100644 --- a/src/anyimage/operators/cutout_tool/main.py +++ b/src/anyimage/operators/cutout_tool/main.py @@ -34,9 +34,11 @@ from ...common.ai import ( ) from ...common.image import ( cleanup_image_input, + color_data_name, image_rgba, load_normal_result_image, - prepare_selection_input, + load_packed_rgba_image, + prepare_image_bounds_input, require_image_empty, source_start_frame, video_total_frames, @@ -52,7 +54,6 @@ from ...common.viewport import ( ) from ...common.selection import ( ImageEditWarning, - SelectionMask, SelectionPath, rasterize_selection_path, ) @@ -61,8 +62,12 @@ from .geometry import ( build_base_shape, limit_cutout_mesh_resolution, ) -from .interaction import open_shape_pie, validate_cutout_selection -from .object import NormalMode, create_shape_object +from .interaction import open_shape_pie +from .object import ( + NormalMode, + create_cropped_color_image, + create_shape_object, +) from .shape import CUTOUT_SHAPES, DEPTH_CUTOUT_SHAPES _HIDDEN = {"HIDDEN", "SKIP_SAVE"} @@ -89,66 +94,99 @@ def create_cutout_shape( source_object, shape, mesh_resolution, - selection_mask, + content_values, + content_bounds, *, depth_image=None, depth_metadata=None, normal_image=None, normal_mode=NormalMode.NONE, - source_rgba=None, + color_image, ): - base_shape = build_base_shape( - context, - source_object, - mesh_resolution, - selection_mask, - ) - result = create_shape_object( - context, - source_object, - shape, - mesh_resolution, - selection_mask, - base_shape, - depth_image=depth_image, - depth_metadata=depth_metadata, - normal_image=normal_image, - normal_mode=normal_mode, - source_rgba=source_rgba, - ) + try: + base_shape = build_base_shape( + context, + source_object, + mesh_resolution, + content_values, + content_bounds, + ) + result = create_shape_object( + context, + source_object, + shape, + mesh_resolution, + content_values, + content_bounds, + base_shape, + depth_image=depth_image, + depth_metadata=depth_metadata, + normal_image=normal_image, + normal_mode=normal_mode, + color_image=color_image, + ) + except Exception: + if color_image.users == 0: + bpy.data.images.remove(color_image, do_unlink=True) + raise undo_message = f"Convert to {shape.replace('_', ' ').title()}" if "FINISHED" not in bpy.ops.ed.undo_push(message=undo_message): raise RuntimeError(f"Unable to create the {undo_message} Undo step") return result -def _selection_mask_from_image(path, bounds): - image = bpy.data.images.load(str(path), check_existing=False) +def _cutout_content_values(source_alpha, selection_values=None): + """Combine local image Alpha with optional pure Selection geometry.""" + import numpy as np + + alpha = np.asarray(source_alpha, dtype=np.float32) + if alpha.ndim != 2: + raise ValueError("The Cutout source Alpha must be two-dimensional") + if selection_values is None: + values = alpha.copy() + else: + selection = np.asarray(selection_values, dtype=np.float32) + if alpha.shape != selection.shape: + raise ValueError("The Cutout source Alpha does not match the Selection") + values = alpha * selection + if not np.any(values > CUTOUT_ALPHA_THRESHOLD): + raise ImageEditWarning( + "The Cutout selection contains no visible image pixels" + ) + return values + + +def _generated_color_result(result, input_path, source_object, source_bounds): + value = result.value if isinstance(result.value, dict) else {} + if "selection_image" not in value: + path = input_path + bounds = source_bounds + else: + path = result.file("selection_image") + local_bounds = tuple( + int(value) + for value in json.loads( + result.file("selection_bounds").read_text(encoding="utf-8") + ) + ) + bounds = ( + source_bounds[0] + local_bounds[0], + source_bounds[1] + local_bounds[1], + source_bounds[0] + local_bounds[2], + source_bounds[1] + local_bounds[3], + ) + image = load_packed_rgba_image( + path, + source_object.data, + color_data_name(source_object), + ) try: - return SelectionMask( - image_rgba(image)[:, :, 3], - tuple(int(value) for value in bounds), - ) - finally: - bpy.data.images.remove(image, do_unlink=True) - - -def _generated_selection_mask(result, input_path, source_bounds): - if not isinstance(result.value, dict) or "selection_image" not in result.value: - return _selection_mask_from_image(input_path, source_bounds) - local_bounds = tuple( - int(value) - for value in json.loads( - result.file("selection_bounds").read_text(encoding="utf-8") - ) - ) - bounds = ( - source_bounds[0] + local_bounds[0], - source_bounds[1] + local_bounds[1], - source_bounds[0] + local_bounds[2], - source_bounds[1] + local_bounds[3], - ) - return _selection_mask_from_image(result.file("selection_image"), bounds) + alpha = image_rgba(image)[:, :, 3] + except Exception: + if image.users == 0: + bpy.data.images.remove(image, do_unlink=True) + raise + return image, alpha, bounds def cutout_geometry_settings(context): @@ -213,14 +251,12 @@ class CutoutSelectionToShape(JobOperatorBase, bpy.types.Operator): if self.shape not in CUTOUT_SHAPES: raise ValueError(f"Unsupported Cutout shape: {self.shape}") selection_path = SelectionPath.from_json(self.selection_path_json) - source_rgba = image_rgba(source_object.data) selection_mask = rasterize_selection_path( tuple(source_object.data.size), selection_path, - source_alpha=source_rgba[:, :, 3], - alpha_threshold=CUTOUT_ALPHA_THRESHOLD, antialias=True, ) + source_rgba = image_rgba(source_object.data) context.view_layer.objects.active = source_object source_object.select_set(True) crop_width = selection_mask.bounds[2] - selection_mask.bounds[0] @@ -232,13 +268,24 @@ class CutoutSelectionToShape(JobOperatorBase, bpy.types.Operator): ) self.max_ai_input_size = configured_max_ai_input_size(context) if not self._needs_generation(): + left, top, right, bottom = selection_mask.bounds + content_values = _cutout_content_values( + source_rgba[top:bottom, left:right, 3], + selection_mask.values, + ) + color_image = create_cropped_color_image( + source_object, + selection_mask.bounds, + source_rgba, + ) create_cutout_shape( context, source_object, self.shape, self.mesh_resolution, - selection_mask, - source_rgba=source_rgba, + content_values, + selection_mask.bounds, + color_image=color_image, ) return {"FINISHED"} setup_result = invoke_ai_setup_if_needed() @@ -249,7 +296,14 @@ class CutoutSelectionToShape(JobOperatorBase, bpy.types.Operator): selection_mask.bounds, separators=(",", ":"), ) - self.input_path = str(prepare_selection_input(source_object.data, selection_mask)) + self._selection_mask = selection_mask + self.input_path = str( + prepare_image_bounds_input( + source_object.data, + source_rgba, + selection_mask.bounds, + ) + ) except ( ImageEditWarning, OSError, @@ -303,15 +357,31 @@ class CutoutSelectionToShape(JobOperatorBase, bpy.types.Operator): def response(self, context, result): source_object = require_image_empty(self.source_object_name) source_bounds = tuple(json.loads(self.selection_bounds_json)) - selection_mask = _generated_selection_mask( + color_image, source_alpha, color_bounds = _generated_color_result( result, self.input_path, + source_object, source_bounds, ) value = result.value if isinstance(result.value, dict) else {} normal_image = None depth_image = None + shape_started = False try: + if "selection_image" in value: + content_values = _cutout_content_values( + source_alpha, + ) + content_bounds = color_bounds + else: + selection_mask = getattr(self, "_selection_mask", None) + if selection_mask is None: + raise RuntimeError("The Cutout Selection is no longer available") + content_values = _cutout_content_values( + source_alpha, + selection_mask.values, + ) + content_bounds = selection_mask.bounds normal_mode = normal_mode_for_shape(self.shape, self.generate_normal) normal_key = f"{normal_mode.value.lower()}_normal" if normal_key in value: @@ -328,22 +398,27 @@ class CutoutSelectionToShape(JobOperatorBase, bpy.types.Operator): source_object, depth_metadata, ) + shape_started = True create_cutout_shape( context, source_object, self.shape, self.mesh_resolution, - selection_mask, + content_values, + content_bounds, depth_image=depth_image, depth_metadata=depth_metadata, normal_image=normal_image, normal_mode=normal_mode, + color_image=color_image, ) except Exception: if normal_image is not None and normal_image.users == 0: bpy.data.images.remove(normal_image, do_unlink=True) if depth_image is not None and depth_image.users == 0: bpy.data.images.remove(depth_image, do_unlink=True) + if not shape_started and color_image.users == 0: + bpy.data.images.remove(color_image, do_unlink=True) raise return f"{self.shape.replace('_', ' ').title()} is ready" @@ -361,13 +436,11 @@ class SelectCutoutSelection(ImageSelectionGesture, bpy.types.Operator): source_object_name: bpy.props.StringProperty(options={"HIDDEN", "SKIP_SAVE"}) source_matrix_data: bpy.props.StringProperty(options={"HIDDEN", "SKIP_SAVE"}) - selection_path_json: bpy.props.StringProperty(options={"HIDDEN", "SKIP_SAVE"}) refine_selection: refine_selection_property() invert: bpy.props.BoolProperty(default=False, options={"SKIP_SAVE"}) gesture: gesture_property() full_image: bpy.props.BoolProperty(default=False, options={"HIDDEN", "SKIP_SAVE"}) - preview_fill_rule = "NONZERO" _pie_event = None def _set_status(self, context): @@ -396,28 +469,21 @@ class SelectCutoutSelection(ImageSelectionGesture, bpy.types.Operator): self.source_object_name = source_object.name self.source_matrix_data = serialize_matrix(source_object.matrix_world) width, height = (int(value) for value in source_object.data.size) - self.selection_path_json = SelectionPath( + selection_path = SelectionPath( points=((0, 0), (width, 0), (width, height), (0, height)), - fill_rule="NONZERO", - ).to_json() - return self.execute(context) + ) + return self._submit_selection_path(context, selection_path) def modal(self, context, event): self._pie_event = event return super().modal(context, event) - def execute(self, context): + def _submit_selection_path(self, context, selection_path): try: source_object = require_image_empty(self.source_object_name) except RuntimeError as error: self.report({"ERROR"}, str(error)) return {"CANCELLED"} - try: - selection_path = SelectionPath.from_json(self.selection_path_json) - validate_cutout_selection(source_object, selection_path) - except (ImageEditWarning, TypeError, ValueError) as error: - report_image_edit_exception(self, error) - return {"CANCELLED"} open_shape_pie( context, self._pie_event, diff --git a/src/anyimage/operators/cutout_tool/object.py b/src/anyimage/operators/cutout_tool/object.py index 4dcc3b9..145adba 100644 --- a/src/anyimage/operators/cutout_tool/object.py +++ b/src/anyimage/operators/cutout_tool/object.py @@ -4,10 +4,7 @@ from enum import StrEnum import bpy -from ...common.image import ( - color_data_name, - image_rgba, -) +from ...common.image import color_data_name from ...common.depth import camera_depth_values, depth_uniform_scale from ...common.material import ( IMAGE_MATERIAL_NODE_GROUP_NAME, @@ -84,8 +81,8 @@ def _assign_float_point_attribute(mesh, name, values): attribute.data.foreach_set("value", np.asarray(values, dtype=np.float32)) -def _cropped_color_image(source_object, selection_mask, source_rgba): - crop_bounds = selection_mask.bounds +def create_cropped_color_image(source_object, crop_bounds, source_rgba): + """Create a packed color image for one local Cutout bounds rectangle.""" import numpy as np source = source_object.data @@ -95,17 +92,10 @@ def _cropped_color_image(source_object, selection_mask, source_rgba): height = bottom - top if width <= 0 or height <= 0: raise ValueError("The Cutout crop dimensions must be positive") - cropped = np.zeros((height, width, 4), dtype=np.float32) source_height, source_width = rgba.shape[:2] - source_left = max(0, left) - source_top = max(0, top) - source_right = min(source_width, right) - source_bottom = min(source_height, bottom) - if source_right > source_left and source_bottom > source_top: - cropped[ - source_top - top : source_bottom - top, - source_left - left : source_right - left, - ] = rgba[source_top:source_bottom, source_left:source_right] + if left < 0 or top < 0 or right > source_width or bottom > source_height: + raise ValueError("The Cutout crop is outside the source RGBA") + cropped = rgba[top:bottom, left:right] image = bpy.data.images.new( color_data_name(source_object), width=width, @@ -151,21 +141,22 @@ def create_shape_object( source_object, shape, mesh_resolution, - selection_mask, + content_values, + content_bounds, base_shape, *, depth_image=None, depth_metadata=None, normal_image=None, normal_mode=NormalMode.NONE, - source_rgba=None, + color_image, ): import numpy as np normal_mode = NormalMode(normal_mode) bounds = crop_plane_bounds( source_object, - selection_mask.bounds, + content_bounds, tuple(source_object.data.size), ) crop_center = ( @@ -177,11 +168,6 @@ def create_shape_object( cutout_sampling_spacing(source_object, mesh_resolution), world_metric, ) - color_image = _cropped_color_image( - source_object, - selection_mask, - image_rgba(source_object.data) if source_rgba is None else source_rgba, - ) material = None mesh = None mesh_object = None @@ -209,7 +195,7 @@ def create_shape_object( raise ValueError("Depth Shape requires a Depth texture and metadata") camera_depth, valid = camera_depth_values(depth_image) calibration_valid, transparent_fraction = selection_calibration( - selection_mask, valid.shape + content_values, valid.shape ) calibration_valid &= valid uniform_scale = depth_uniform_scale( @@ -311,6 +297,4 @@ def create_shape_object( bpy.data.meshes.remove(mesh) if material is not None and material.users == 0: bpy.data.materials.remove(material) - if color_image.users == 0: - bpy.data.images.remove(color_image, do_unlink=True) raise diff --git a/src/anyimage/operators/image_tool/edit_mask.py b/src/anyimage/operators/image_tool/edit_mask.py index 6702509..96f66a3 100644 --- a/src/anyimage/operators/image_tool/edit_mask.py +++ b/src/anyimage/operators/image_tool/edit_mask.py @@ -12,9 +12,10 @@ from ...common.ai import ( from ...common.image import ( cleanup_image_input, create_pixel_result_image, - image_pixels, + image_base_name, image_rgba, - prepare_selection_input, + load_packed_rgba_image, + prepare_image_bounds_input, replace_empty_image, require_image_empty, video_total_frames, @@ -27,69 +28,47 @@ from ...common.viewport import ( ) from ...common.selection import ( ImageEditWarning, - SelectionPath, - crop_pixels_to_alpha, rasterize_selection_path, ) _HIDDEN = {"HIDDEN", "SKIP_SAVE"} -def _refined_mask_from_pixels(mask_pixels, mask_size, bounds, source_size): - import numpy as np - - mask_width, mask_height = (int(value) for value in mask_size) - source_width, source_height = (int(value) for value in source_size) - local = np.flipud( - np.asarray(mask_pixels, dtype=np.float32).reshape((mask_height, mask_width, 4)) - )[:, :, 3] - result = np.zeros((source_height, source_width), dtype=np.float32) - left, top, right, bottom = (int(value) for value in bounds) - source_left = max(left, 0) - source_top = max(top, 0) - source_right = min(right, source_width) - source_bottom = min(bottom, source_height) - if source_right <= source_left or source_bottom <= source_top: - return result - result[source_top:source_bottom, source_left:source_right] = local[ - source_top - top : source_bottom - top, - source_left - left : source_right - left, - ] - return result - - def _edit_image_mask_pixels( - source_pixels, - source_size, + source_rgba, selection_mask, - invert=False, ): """Apply a mask while preserving source RGB, then tightly crop the canvas.""" import numpy as np - width, height = (int(value) for value in source_size) - rgba = np.flipud( - np.asarray(source_pixels, dtype=np.float32).reshape((height, width, 4)) - ) - mask = np.asarray(selection_mask, dtype=np.float32) - if invert: - mask = 1.0 - mask - mask = np.where(rgba[:, :, 3] > 0.0, mask, 0.0) - rgba[:, :, 3] *= mask - return crop_pixels_to_alpha( - np.flipud(rgba).ravel(), - (0, 0, width, height), + left, top, right, bottom = selection_mask.bounds + rgba = np.asarray(source_rgba, dtype=np.float32)[top:bottom, left:right].copy() + if rgba.shape[:2] != selection_mask.values.shape: + raise ValueError("The Selection does not match the source RGBA") + rgba[:, :, 3] *= selection_mask.values + visible = rgba[:, :, 3] > 0.0 + visible_y = np.flatnonzero(np.any(visible, axis=1)) + visible_x = np.flatnonzero(np.any(visible, axis=0)) + if not len(visible_x): + raise ImageEditWarning("The edit contains no visible image pixels") + content_left = int(visible_x[0]) + content_top = int(visible_y[0]) + content_right = int(visible_x[-1]) + 1 + content_bottom = int(visible_y[-1]) + 1 + output = rgba[content_top:content_bottom, content_left:content_right] + return np.flipud(output).ravel(), ( + left + content_left, + top + content_top, + left + content_right, + top + content_bottom, ) -def _edit_image_object(source_object, selection_mask, invert=False): +def _edit_image_object(source_object, source_rgba, selection_mask): image = source_object.data - source_size = tuple(image.size) pixels, crop_bounds = _edit_image_mask_pixels( - image_pixels(image), - source_size, + source_rgba, selection_mask, - invert, ) left, top, right, bottom = crop_bounds result_image = create_pixel_result_image( @@ -116,7 +95,6 @@ class EditImageMask(ImageSelectionGesture, bpy.types.Operator): source_object_name: bpy.props.StringProperty(options={"HIDDEN", "SKIP_SAVE"}) source_matrix_data: bpy.props.StringProperty(options={"HIDDEN", "SKIP_SAVE"}) - selection_path_json: bpy.props.StringProperty(options={"HIDDEN", "SKIP_SAVE"}) refine_selection: refine_selection_property() invert: bpy.props.BoolProperty( name="Invert", @@ -126,25 +104,25 @@ class EditImageMask(ImageSelectionGesture, bpy.types.Operator): ) gesture: gesture_property() - def execute(self, context): + def _submit_selection_path(self, context, selection_path): try: source_object = require_image_empty(self.source_object_name) except RuntimeError as error: self.report({"ERROR"}, str(error)) return {"CANCELLED"} try: - selection_path = SelectionPath.from_json(self.selection_path_json) if video_total_frames(source_object.data) > 1: raise RuntimeError("Image tools only support a still image") selection_mask = rasterize_selection_path( tuple(source_object.data.size), selection_path, - source_alpha=image_rgba(source_object.data)[:, :, 3], ) + source_rgba = image_rgba(source_object.data) if self.refine_selection: - source_path = prepare_selection_input( + source_path = prepare_image_bounds_input( source_object.data, - selection_mask, + source_rgba, + selection_mask.bounds, ) result = bpy.ops.anyimage.refine_image_selection( input_path=str(source_path), @@ -157,9 +135,10 @@ class EditImageMask(ImageSelectionGesture, bpy.types.Operator): return {"FINISHED"} _edit_image_object( source_object, - selection_mask.full_values(tuple(source_object.data.size)), + source_rgba, + selection_mask, ) - except (json.JSONDecodeError, RuntimeError, TypeError, ValueError) as error: + except (RuntimeError, TypeError, ValueError) as error: report_image_edit_exception(self, error) return {"CANCELLED"} return {"FINISHED"} @@ -197,24 +176,22 @@ class RefineImageSelection(JobOperatorBase, bpy.types.Operator): source_bounds[0] + local_bounds[2], source_bounds[1] + local_bounds[3], ) - refined_image = bpy.data.images.load( - str(result.file("selection_image")), check_existing=False + source_image = source_object.data + refined_image = load_packed_rgba_image( + result.file("selection_image"), + source_image, + image_base_name(source_image), ) try: - refined_pixels = image_pixels(refined_image) - mask = _refined_mask_from_pixels( - refined_pixels, - tuple(refined_image.size), - bounds, - tuple(source_object.data.size), + replace_empty_image( + source_object, + refined_image, + placement_bounds=bounds, ) - finally: - bpy.data.images.remove(refined_image, do_unlink=True) - try: - _edit_image_object(source_object, mask) - except ImageEditWarning as error: - self.report({"WARNING"}, str(error)) - return str(error) + except Exception: + if refined_image.users == 0: + bpy.data.images.remove(refined_image, do_unlink=True) + raise if "FINISHED" not in bpy.ops.ed.undo_push(message="Edit Image Mask"): raise RuntimeError("Unable to create the Edit Image Mask Undo step") return "Image Selection refined" diff --git a/src/anyimage/server/jobs/cutout.py b/src/anyimage/server/jobs/cutout.py index 89210fa..c73e7f2 100644 --- a/src/anyimage/server/jobs/cutout.py +++ b/src/anyimage/server/jobs/cutout.py @@ -1,4 +1,4 @@ -"""Generate optional AI files for one Blender-rasterized Cutout Selection.""" +"""Generate optional AI files from one rectangular Cutout image input.""" import json from pathlib import Path diff --git a/src/anyimage/server/jobs/depth_plane.py b/src/anyimage/server/jobs/depth_plane.py index 786910f..ad530de 100644 --- a/src/anyimage/server/jobs/depth_plane.py +++ b/src/anyimage/server/jobs/depth_plane.py @@ -16,6 +16,7 @@ def run(context, parameters): parameters, images[0], depth_mode="Z", + normal_mode="TANGENT", ) context.check_cancelled() context.progress(1.0, "Depth Plane data is ready") diff --git a/src/anyimage/server/selection/refine_selection.py b/src/anyimage/server/selection/refine_selection.py index 5038871..4eaea55 100644 --- a/src/anyimage/server/selection/refine_selection.py +++ b/src/anyimage/server/selection/refine_selection.py @@ -5,14 +5,16 @@ def _crop_to_alpha(image): import numpy as np pixels = np.asarray(image.convert("RGBA"), dtype=np.uint8) - visible_y, visible_x = np.nonzero(pixels[:, :, 3] > 0) + visible = pixels[:, :, 3] > 0 + visible_y = np.flatnonzero(np.any(visible, axis=1)) + visible_x = np.flatnonzero(np.any(visible, axis=0)) if not len(visible_x): raise ValueError("The generated Selection contains no visible pixels") bounds = ( - int(visible_x.min()), - int(visible_y.min()), - int(visible_x.max()) + 1, - int(visible_y.max()) + 1, + int(visible_x[0]), + int(visible_y[0]), + int(visible_x[-1]) + 1, + int(visible_y[-1]) + 1, ) return image.crop(bounds), bounds @@ -24,7 +26,7 @@ def refine_selection( device, output_path, ): - """Refine an image Selection from an already rasterized Selection.""" + """Refine foreground Alpha within one rectangular Selection context.""" model_manager = context.resource("model_manager") foreground, _load_ms, _inference_ms = infer_foreground( input_path, diff --git a/tests/test_blender_addon.py b/tests/test_blender_addon.py index fb9f416..1859fa3 100644 --- a/tests/test_blender_addon.py +++ b/tests/test_blender_addon.py @@ -6,7 +6,7 @@ import threading import unittest from pathlib import Path from types import ModuleType, SimpleNamespace -from unittest.mock import patch +from unittest.mock import Mock, patch PROJECT_ROOT = Path(__file__).resolve().parents[1] SOURCE_ROOT = PROJECT_ROOT / "src" @@ -90,6 +90,9 @@ class BlenderAddonTest(unittest.TestCase): cls.convert_to_plane = importlib.import_module( "anyimage.operators.convert_to_plane.conversion" ) + cls.convert_to_plane_object = importlib.import_module( + "anyimage.operators.convert_to_plane.object" + ) cls.cutout_main = importlib.import_module( "anyimage.operators.cutout_tool.main" ) @@ -851,6 +854,19 @@ class BlenderAddonTest(unittest.TestCase): self.assertNotIn("draw", operator.__dict__) self.assertIn("REGISTER", operator.bl_options) self.assertIn("UNDO", operator.bl_options) + self.assertEqual(operator.__annotations__["mesh_detail"]["default"], 0) + self.assertEqual( + self.convert_to_plane.ConvertToDepthPlane.__annotations__["mesh_detail"][ + "default" + ], + 6, + ) + self.assertEqual( + self.convert_to_plane.GenerateDepthPlane.__annotations__["mesh_detail"][ + "default" + ], + 6, + ) image_empty = SimpleNamespace( type="EMPTY", @@ -923,6 +939,128 @@ class BlenderAddonTest(unittest.TestCase): self.assertNotIn("shape", parameters) self.assertNotIn("polygon", parameters) + def test_depth_plane_result_connects_the_tangent_normal_material(self): + module = self.convert_to_plane_object + source = SimpleNamespace(data=SimpleNamespace(name="Source")) + color_image = SimpleNamespace(name="") + depth_image = SimpleNamespace(users=0) + normal_image = SimpleNamespace(users=0) + material = SimpleNamespace(users=0) + metadata = {"reference_depth": 1.0} + files = { + "depth_metadata": Path("depth.json"), + "z_depth": Path("z-depth.exr"), + "tangent_normal": Path("tangent-normal.png"), + } + result = SimpleNamespace( + directory=Path("result"), + file=lambda key: files[key], + ) + operator = SimpleNamespace(source_object_name="Source", mesh_detail=6) + context = SimpleNamespace(scene=object()) + ops = SimpleNamespace( + ed=SimpleNamespace(undo_push=Mock(return_value={"FINISHED"})) + ) + + with ( + patch.object(module, "require_image_empty", return_value=source), + patch.object(module, "load_color_result_image", return_value=color_image), + patch.object(module, "load_depth_metadata", return_value=metadata), + patch.object(module, "load_depth_result_image", return_value=depth_image), + patch.object( + module, + "load_normal_result_image", + return_value=normal_image, + ) as load_normal, + patch.object( + module, + "create_image_material", + return_value=material, + ) as create_material, + patch.object(module, "create_depth_plane_object") as create_object, + patch.object(module.bpy, "ops", ops, create=True), + ): + module.create_depth_plane_from_result(context, result, operator) + + load_normal.assert_called_once_with(files["tangent_normal"], source) + create_material.assert_called_once_with( + source, + color_image, + 0.0, + normal_image=normal_image, + normal_space="TANGENT", + ior=1.2, + scene=context.scene, + ) + create_object.assert_called_once_with( + context, + source, + 6, + material, + depth_image, + metadata, + ) + + def test_depth_plane_result_cleans_unused_depth_and_normal_on_failure(self): + module = self.convert_to_plane_object + source = SimpleNamespace(data=SimpleNamespace(name="Source")) + color_image = SimpleNamespace(name="") + depth_image = SimpleNamespace(users=0) + normal_image = SimpleNamespace(users=0) + material = SimpleNamespace(users=0) + files = { + "depth_metadata": Path("depth.json"), + "z_depth": Path("z-depth.exr"), + "tangent_normal": Path("tangent-normal.png"), + } + result = SimpleNamespace( + directory=Path("result"), + file=lambda key: files[key], + ) + remove_material = Mock() + remove_image = Mock() + data = SimpleNamespace( + materials=SimpleNamespace(remove=remove_material), + images=SimpleNamespace(remove=remove_image), + ) + + with ( + patch.object(module, "require_image_empty", return_value=source), + patch.object(module, "load_color_result_image", return_value=color_image), + patch.object( + module, + "load_depth_metadata", + return_value={"reference_depth": 1.0}, + ), + patch.object(module, "load_depth_result_image", return_value=depth_image), + patch.object(module, "load_normal_result_image", return_value=normal_image), + patch.object(module, "create_image_material", return_value=material), + patch.object( + module, + "create_depth_plane_object", + side_effect=RuntimeError("failed"), + ), + patch.object(module.bpy, "data", data), + self.assertRaisesRegex(RuntimeError, "failed"), + ): + module.create_depth_plane_from_result( + SimpleNamespace(scene=object()), + result, + SimpleNamespace(source_object_name="Source", mesh_detail=6), + ) + + remove_material.assert_called_once_with(material, do_unlink=True) + self.assertEqual( + [entry.args[0] for entry in remove_image.call_args_list], + [depth_image, normal_image], + ) + self.assertTrue( + all( + entry.kwargs == {"do_unlink": True} + for entry in remove_image.call_args_list + ) + ) + def test_surface_and_balloon_without_ai_build_locally(self): import numpy as np @@ -932,7 +1070,7 @@ class BlenderAddonTest(unittest.TestCase): context = SimpleNamespace( view_layer=SimpleNamespace(objects=SimpleNamespace(active=None)) ) - selection_mask = self.cutout_main.SelectionMask( + selection_mask = self.image_selection.SelectionMask( np.ones((8, 8), dtype=np.float32), (1, 1, 9, 9) ) source_rgba = np.ones((10, 10, 4), dtype=np.float32) @@ -941,7 +1079,10 @@ class BlenderAddonTest(unittest.TestCase): operator = SimpleNamespace( source_object_name="Source", selection_path_json=json.dumps( - {"points": [[1, 1], [9, 1], [9, 9], [1, 9]]} + { + "points": [[1, 1], [9, 1], [9, 9], [1, 9]], + "invert": False, + } ), shape=shape, mesh_resolution=64, @@ -961,12 +1102,12 @@ class BlenderAddonTest(unittest.TestCase): ), patch.object( self.cutout_main, "image_rgba", return_value=source_rgba - ), + ) as read_rgba, patch.object( self.cutout_main, "rasterize_selection_path", return_value=selection_mask, - ), + ) as rasterize, patch.object( self.cutout_main, "configured_max_ai_input_size", @@ -977,6 +1118,16 @@ class BlenderAddonTest(unittest.TestCase): "configured_max_cutout_mesh_resolution", return_value=1024, ), + patch.object( + self.cutout_main, + "_cutout_content_values", + return_value=selection_mask.values, + ), + patch.object( + self.cutout_main, + "create_cropped_color_image", + return_value="Color", + ), patch.object( self.cutout_main, "create_cutout_shape" ) as convert, @@ -986,15 +1137,164 @@ class BlenderAddonTest(unittest.TestCase): ) self.assertEqual(result, {"FINISHED"}) + read_rgba.assert_called_once_with(source.data) + rasterize.assert_called_once() convert.assert_called_once_with( context, source, shape, 64, - selection_mask, - source_rgba=source_rgba, + selection_mask.values, + selection_mask.bounds, + color_image="Color", ) + def test_ai_cutout_writes_the_selection_bounds_image(self): + import numpy as np + + source = SimpleNamespace( + data=SimpleNamespace(size=(4096, 2048)), + select_set=lambda _value: None, + ) + context = SimpleNamespace( + view_layer=SimpleNamespace(objects=SimpleNamespace(active=None)) + ) + source_rgba = np.ones((4, 8, 4), dtype=np.float32) + selection_mask = self.image_selection.SelectionMask( + np.ones((2, 6), dtype=np.float32), + (1, 1, 7, 3), + ) + operator = self.cutout_main.CutoutSelectionToShape() + operator.source_object_name = "Source" + operator.selection_path_json = json.dumps( + { + "points": [[1, 1], [7, 1], [7, 3], [1, 3]], + "invert": False, + } + ) + operator.shape = "DEPTH_SURFACE" + operator.mesh_resolution = 64 + operator.generate_normal = False + operator.refine_selection = False + operator.report = lambda *_args: None + + with ( + patch.object( + self.cutout_main, + "require_image_empty", + return_value=source, + ), + patch.object(self.cutout_main, "video_total_frames", return_value=1), + patch.object( + self.cutout_main, + "image_rgba", + return_value=source_rgba, + ) as read_rgba, + patch.object( + self.cutout_main, + "rasterize_selection_path", + return_value=selection_mask, + ) as rasterize, + patch.object( + self.cutout_main, + "configured_max_ai_input_size", + return_value=2048, + ), + patch.object( + self.cutout_main, + "configured_max_cutout_mesh_resolution", + return_value=1024, + ), + patch.object( + self.cutout_main, + "invoke_ai_setup_if_needed", + return_value=None, + ), + patch.object( + self.cutout_main, + "cutout_geometry_settings", + return_value=("MODEL", 5), + ), + patch.object( + self.cutout_main, + "prepare_image_bounds_input", + return_value=Path(__file__), + ) as prepare_input, + patch.object( + self.cutout_main.JobOperatorBase, + "execute", + return_value={"FINISHED"}, + ), + ): + result = operator.execute(context) + + self.assertEqual(result, {"FINISHED"}) + read_rgba.assert_called_once_with(source.data) + rasterize.assert_called_once() + prepare_input.assert_called_once_with( + source.data, + source_rgba, + selection_mask.bounds, + ) + + def test_empty_cutout_selection_is_rejected_after_shape_choice(self): + source = SimpleNamespace( + data=SimpleNamespace(size=(4096, 2048)), + select_set=lambda _value: None, + ) + context = SimpleNamespace( + view_layer=SimpleNamespace(objects=SimpleNamespace(active=None)) + ) + reports = [] + operator = SimpleNamespace( + source_object_name="Source", + selection_path_json=json.dumps( + { + "points": [[1, 1], [7, 1], [7, 3], [1, 3]], + "invert": False, + } + ), + shape="SURFACE", + mesh_resolution=64, + generate_normal=False, + refine_selection=False, + report=lambda level, message: reports.append((level, message)), + ) + + with ( + patch.object( + self.cutout_main, + "require_image_empty", + return_value=source, + ), + patch.object(self.cutout_main, "video_total_frames", return_value=1), + patch.object(self.cutout_main, "image_rgba"), + patch.object( + self.cutout_main, + "rasterize_selection_path", + side_effect=self.image_selection.ImageEditWarning( + "The selection contains no visible image pixels" + ), + ), + patch.object(self.cutout_main, "create_cutout_shape") as create_shape, + ): + result = self.cutout_main.CutoutSelectionToShape.execute( + operator, + context, + ) + + self.assertEqual(result, {"CANCELLED"}) + create_shape.assert_not_called() + self.assertEqual( + reports, + [ + ( + {"WARNING"}, + "The selection contains no visible image pixels", + ) + ], + ) + def test_cutout_request_describes_ai_capabilities(self): operator = self.cutout_main.CutoutSelectionToShape() operator.input_path = __file__ @@ -1343,34 +1643,27 @@ class BlenderAddonTest(unittest.TestCase): (1, 1), ) - even_odd = self.image_selection.rasterize_selection_path( + selection = self.image_selection.rasterize_selection_path( (12, 9), - self.image_selection.SelectionPath(points=path, fill_rule="EVEN_ODD"), - ).full_values((12, 9)) - nonzero = self.image_selection.rasterize_selection_path( - (12, 9), - self.image_selection.SelectionPath(points=path, fill_rule="NONZERO"), + self.image_selection.SelectionPath(points=path), ).full_values((12, 9)) - self.assertFalse(even_odd[3, 5]) - self.assertTrue(nonzero[3, 5]) + self.assertTrue(selection[3, 5]) - def preview_area(fill_rule): - triangles = self.image_interaction.scanline_fill_triangles(path, fill_rule) - return sum( - abs( - self.image_interaction._triangle_cross( - triangles[index], - triangles[index + 1], - triangles[index + 2], - ) + triangles = self.image_interaction.scanline_fill_triangles(path) + preview_area = sum( + abs( + self.image_interaction._triangle_cross( + triangles[index], + triangles[index + 1], + triangles[index + 2], ) - * 0.5 - for index in range(0, len(triangles), 3) ) + * 0.5 + for index in range(0, len(triangles), 3) + ) - self.assertAlmostEqual(preview_area("EVEN_ODD"), 36.0) - self.assertAlmostEqual(preview_area("NONZERO"), 54.0) + self.assertAlmostEqual(preview_area, 54.0) def test_expected_empty_image_edits_are_reported_as_warnings(self): reports = [] @@ -1405,6 +1698,7 @@ class BlenderAddonTest(unittest.TestCase): self.assertIn(operator, self.anyimage.CLASSES) self.assertIn(self.edit_mask.RefineImageSelection, self.anyimage.CLASSES) self.assertIn(self.edit_mask.ActivateImageTool, self.anyimage.CLASSES) + self.assertNotIn("selection_path_json", operator.__annotations__) tool_cases = ( (self.image_box.ImageBoxTool, "BOX", "ops.sculpt.border_mask"), (self.image_lasso.ImageLassoTool, "LASSO", "ops.sculpt.lasso_mask"), @@ -1438,6 +1732,14 @@ class BlenderAddonTest(unittest.TestCase): "anyimage.operators.cutout_tool" ).CutoutTool cutout_module = sys.modules[cutout_tool.__module__] + self.assertNotIn( + "selection_path_json", + cutout_module.SelectCutoutSelection.__annotations__, + ) + self.assertIn( + "selection_path_json", + cutout_module.CutoutSelectionToShape.__annotations__, + ) drawn_settings.clear() with ( patch.object( @@ -1894,7 +2196,7 @@ class BlenderAddonTest(unittest.TestCase): def test_image_mask_preview_fills_a_self_intersecting_lasso(self): polygon = ((0, 0), (4, 4), (0, 4), (4, 0)) - triangles = self.image_interaction.preview_fill_triangles(polygon, "NONZERO") + triangles = self.image_interaction.preview_fill_triangles(polygon) area = sum( abs( self.image_interaction._triangle_cross( @@ -2010,19 +2312,20 @@ class BlenderAddonTest(unittest.TestCase): (), ) - def test_image_box_completion_serializes_the_image_selection(self): + def test_image_box_completion_submits_the_image_selection_path(self): source = SimpleNamespace(data=SimpleNamespace(size=(64, 32))) finished = [] + submitted = [] operator = SimpleNamespace( source_object_name="Source", source_matrix_data="matrix", gesture="BOX", - selection_path_json="", invert=False, - preview_fill_rule="EVEN_ODD", _screen_path=lambda: ((1, 1), (8, 1), (8, 6), (1, 6)), _finish=lambda context: finished.append(context), - execute=lambda _context: {"FINISHED"}, + _submit_selection_path=lambda context, selection_path: ( + submitted.append((context, selection_path)) or {"FINISHED"} + ), ) context = SimpleNamespace(region=object(), region_data=object()) polygon = ((1, 2), (3, 4), (5, 6)) @@ -2042,12 +2345,296 @@ class BlenderAddonTest(unittest.TestCase): result = self.image_interaction.ImageSelectionGesture._complete(operator, context) self.assertEqual(result, {"FINISHED"}) - self.assertEqual( - json.loads(operator.selection_path_json)["points"], - [[1.0, 2.0], [3.0, 4.0], [5.0, 6.0]], - ) + self.assertEqual(submitted[0][0], context) + self.assertEqual(submitted[0][1].points, polygon) self.assertEqual(finished, [context]) + def test_local_image_edit_builds_one_selection(self): + import numpy as np + + source = SimpleNamespace(data=SimpleNamespace(size=(65536, 32768))) + source_rgba = np.ones((6, 8, 4), dtype=np.float32) + selection_path = self.image_selection.SelectionPath( + points=((1, 1), (7, 1), (7, 5), (1, 5)) + ) + selection_mask = self.image_selection.SelectionMask( + np.ones((4, 6), dtype=np.float32), + (1, 1, 7, 5), + ) + operator = SimpleNamespace( + source_object_name="Source", + refine_selection=False, + report=lambda *_args: None, + ) + + with ( + patch.object( + self.edit_mask, + "require_image_empty", + return_value=source, + ), + patch.object(self.edit_mask, "video_total_frames", return_value=1), + patch.object( + self.edit_mask, + "image_rgba", + return_value=source_rgba, + ) as read_rgba, + patch.object( + self.edit_mask, + "rasterize_selection_path", + return_value=selection_mask, + ) as rasterize, + patch.object(self.edit_mask, "_edit_image_object") as edit, + ): + result = self.edit_mask.EditImageMask._submit_selection_path( + operator, + SimpleNamespace(), + selection_path, + ) + + self.assertEqual(result, {"FINISHED"}) + read_rgba.assert_called_once_with(source.data) + rasterize.assert_called_once() + edit.assert_called_once_with(source, source_rgba, selection_mask) + + def test_image_edit_applies_the_local_selection_mask(self): + import numpy as np + + source_rgba = np.ones((6, 8, 4), dtype=np.float32) + source_rgba[:, :, 0] = np.arange(8, dtype=np.float32)[None, :] + values = np.ones((4, 6), dtype=np.float32) + values[0, :] = 0.0 + values[:, 0] = 0.0 + selection_mask = self.image_selection.SelectionMask( + values, + (1, 1, 7, 5), + ) + + pixels, bounds = self.edit_mask._edit_image_mask_pixels( + source_rgba, + selection_mask, + ) + + self.assertEqual(bounds, (2, 2, 7, 5)) + rgba = np.flipud(pixels.reshape((3, 5, 4))) + self.assertTrue(np.all(rgba[:, :, 3] == 1.0)) + self.assertTrue(np.all(rgba[:, :, 0] == np.arange(2, 7)[None, :])) + + def test_refined_image_edit_writes_the_selection_bounds_image(self): + import numpy as np + + source = SimpleNamespace(data=SimpleNamespace(size=(8, 6))) + source_rgba = np.ones((6, 8, 4), dtype=np.float32) + selection_path = self.image_selection.SelectionPath( + points=((1, 1), (7, 1), (7, 5), (1, 5)) + ) + selection_mask = self.image_selection.SelectionMask( + np.ones((4, 6), dtype=np.float32), + (1, 1, 7, 5), + ) + operator = SimpleNamespace( + source_object_name="Source", + refine_selection=True, + report=lambda *_args: None, + ) + refine = Mock(return_value={"FINISHED"}) + ops = SimpleNamespace( + anyimage=SimpleNamespace(refine_image_selection=refine) + ) + + with ( + patch.object( + self.edit_mask, + "require_image_empty", + return_value=source, + ), + patch.object(self.edit_mask, "video_total_frames", return_value=1), + patch.object( + self.edit_mask, + "image_rgba", + return_value=source_rgba, + ) as read_rgba, + patch.object( + self.edit_mask, + "rasterize_selection_path", + return_value=selection_mask, + ) as rasterize, + patch.object( + self.edit_mask, + "prepare_image_bounds_input", + return_value=Path(__file__), + ) as prepare_input, + patch.object(self.edit_mask.bpy, "ops", ops, create=True), + ): + result = self.edit_mask.EditImageMask._submit_selection_path( + operator, + SimpleNamespace(), + selection_path, + ) + + self.assertEqual(result, {"FINISHED"}) + read_rgba.assert_called_once_with(source.data) + rasterize.assert_called_once() + prepare_input.assert_called_once_with( + source.data, + source_rgba, + selection_mask.bounds, + ) + refine.assert_called_once() + + def test_refined_image_response_uses_the_encoded_result_directly(self): + source_image = SimpleNamespace( + name="Poster.png", + colorspace_settings=SimpleNamespace(name="sRGB"), + alpha_mode="STRAIGHT", + ) + source = SimpleNamespace(data=source_image) + refined = SimpleNamespace(users=0) + files = { + "selection_bounds": SimpleNamespace( + read_text=lambda encoding: "[2,1,5,4]" + ), + "selection_image": Path(__file__), + } + result = SimpleNamespace(file=lambda key: files[key]) + operator = self.edit_mask.RefineImageSelection() + operator.source_object_name = "Source" + operator.selection_bounds_json = "[10,20,18,26]" + ops = SimpleNamespace( + ed=SimpleNamespace(undo_push=Mock(return_value={"FINISHED"})) + ) + + with ( + patch.object( + self.edit_mask, + "require_image_empty", + return_value=source, + ), + patch.object( + self.edit_mask, + "load_packed_rgba_image", + return_value=refined, + ) as load_result, + patch.object(self.edit_mask, "replace_empty_image") as replace, + patch.object(self.edit_mask, "image_rgba") as read_rgba, + patch.object( + self.edit_mask, + "create_pixel_result_image", + ) as create_pixels, + patch.object(self.edit_mask.bpy, "ops", ops, create=True), + ): + message = operator.response(None, result) + + self.assertEqual(message, "Image Selection refined") + load_result.assert_called_once_with( + Path(__file__), + source_image, + "Poster", + ) + replace.assert_called_once_with( + source, + refined, + placement_bounds=(12, 21, 15, 24), + ) + read_rgba.assert_not_called() + create_pixels.assert_not_called() + + def test_cutout_ai_response_reuses_color_image_and_keeps_lasso_for_no_refine(self): + import numpy as np + + source = SimpleNamespace(data=SimpleNamespace()) + color_image = SimpleNamespace(users=0) + pure_values = np.ones((3, 4), dtype=np.float32) + pure_values[:, 0] = 0.0 + selection_mask = self.image_selection.SelectionMask( + pure_values, + (10, 20, 14, 23), + ) + operator = self.cutout_main.CutoutSelectionToShape() + operator.source_object_name = "Source" + operator.selection_bounds_json = "[10,20,14,23]" + operator.input_path = __file__ + operator.shape = "SURFACE" + operator.mesh_resolution = 32 + operator.generate_normal = True + operator.refine_selection = False + operator._selection_mask = selection_mask + result = SimpleNamespace(value={}, file=lambda _key: Path(__file__)) + + with ( + patch.object( + self.cutout_main, + "require_image_empty", + return_value=source, + ), + patch.object( + self.cutout_main, + "_generated_color_result", + return_value=( + color_image, + np.ones((3, 4), dtype=np.float32), + selection_mask.bounds, + ), + ), + patch.object(self.cutout_main, "create_cutout_shape") as create_shape, + ): + message = operator.response(SimpleNamespace(), result) + + self.assertEqual(message, "Surface is ready") + content_values = create_shape.call_args.args[4] + np.testing.assert_array_equal(content_values, pure_values) + self.assertEqual(create_shape.call_args.args[5], selection_mask.bounds) + self.assertIs(create_shape.call_args.kwargs["color_image"], color_image) + + def test_cutout_refine_keeps_ben2_content_outside_the_lasso(self): + import numpy as np + + source = SimpleNamespace(data=SimpleNamespace()) + color_image = SimpleNamespace(users=0) + operator = self.cutout_main.CutoutSelectionToShape() + operator.source_object_name = "Source" + operator.selection_bounds_json = "[10,20,16,24]" + operator.input_path = __file__ + operator.shape = "SURFACE" + operator.mesh_resolution = 32 + operator.generate_normal = False + operator.refine_selection = True + operator._selection_mask = self.image_selection.SelectionMask( + np.zeros((4, 6), dtype=np.float32), + (10, 20, 16, 24), + ) + result = SimpleNamespace( + value={"selection_image": "selection.png"}, + file=lambda _key: Path(__file__), + ) + + with ( + patch.object( + self.cutout_main, + "require_image_empty", + return_value=source, + ), + patch.object( + self.cutout_main, + "_generated_color_result", + return_value=( + color_image, + np.ones((2, 3), dtype=np.float32), + (12, 21, 15, 23), + ), + ), + patch.object(self.cutout_main, "create_cutout_shape") as create_shape, + ): + operator.response(SimpleNamespace(), result) + + content_values = create_shape.call_args.args[4] + np.testing.assert_array_equal( + content_values, + np.ones((2, 3), dtype=np.float32), + ) + self.assertEqual(create_shape.call_args.args[5], (12, 21, 15, 23)) + self.assertIs(create_shape.call_args.kwargs["color_image"], color_image) + def test_image_selection_uses_white_dashed_segments(self): segments = self.image_interaction.dashed_line_segments( ((0, 0), (10, 0)), dash_length=3.0, gap_length=2.0 @@ -2066,7 +2653,6 @@ class BlenderAddonTest(unittest.TestCase): _path=[polygon[0]], _area_pointer=1, _region_pointer=2, - preview_fill_rule="NONZERO", _fill_polygon=None, _fill_triangles=(), _outline_polygon=None, @@ -2091,7 +2677,7 @@ class BlenderAddonTest(unittest.TestCase): self.image_interaction.ImageSelectionGesture._draw_selection(operator) self.image_interaction.ImageSelectionGesture._draw_selection(operator) - build_fill.assert_called_once_with(polygon, "NONZERO") + build_fill.assert_called_once_with(polygon) build_outline.assert_called_once_with((*polygon, polygon[0])) self.assertEqual(draw_fill.call_count, 2) draw_fill.assert_called_with( diff --git a/tests/test_cutout_geometry.py b/tests/test_cutout_geometry.py index 8ec9c0d..3089185 100644 --- a/tests/test_cutout_geometry.py +++ b/tests/test_cutout_geometry.py @@ -283,7 +283,7 @@ def test_selection_contours_group_holes_with_their_connected_component(): alpha[5:15, 5:15] = 255 selection_contours = extract_selection_contours( - SelectionMask(alpha, (0, 0, 200, 120)), + alpha, max_resolution=200, ) @@ -306,7 +306,7 @@ def test_selection_contours_accept_blender_float_mask_with_a_hole(): alpha[45:75, 80:120] = 0.0 selection_contours = extract_selection_contours( - SelectionMask(alpha, (0, 0, 200, 120)), + alpha, max_resolution=200, ) @@ -337,7 +337,7 @@ def test_depth_surface_cutout_uses_half_alpha_boundary(): alpha[4:8, 5:11] = 0.51 selection_contours = cutout_geometry.extract_cutout_contours( - SelectionMask(alpha, (0, 0, 16, 12)), + alpha, 64, threshold=0.5, ) @@ -364,7 +364,7 @@ def test_blender_cutout_alpha_pipeline_does_not_import_pillow(): ) antialiased = selection_mask.full_values((48, 32)) selection_contours = extract_selection_contours( - SelectionMask(antialiased, (0, 0, 48, 32)), + antialiased, max_resolution=24, ) @@ -380,8 +380,8 @@ def test_blender_cutout_contour_resolution_tracks_mesh_detail(): ) selection_mask = SelectionMask(np.ones((2, 2), dtype=np.float32), (0, 0, 2, 2)) with patch.object(cutout_geometry, "extract_selection_contours", return_value=expected) as trace: - assert cutout_geometry.extract_cutout_contours(selection_mask, 32) == expected - assert cutout_geometry.extract_cutout_contours(selection_mask, 256) == expected + assert cutout_geometry.extract_cutout_contours(selection_mask.values, 32) == expected + assert cutout_geometry.extract_cutout_contours(selection_mask.values, 256) == expected assert trace.call_args_list[0].kwargs["max_resolution"] == 512 assert trace.call_args_list[1].kwargs["max_resolution"] == 1024 @@ -580,7 +580,10 @@ def test_selection_calibration_uses_the_final_selection_mask(): (0, 0, 2, 2), ) - valid, transparent_fraction = cutout_geometry.selection_calibration(selection_mask, (2, 2)) + valid, transparent_fraction = cutout_geometry.selection_calibration( + selection_mask.values, + (2, 2), + ) assert np.array_equal(valid, ((True, True), (True, False))) assert transparent_fraction == 0.25 diff --git a/tests/test_cutout_interaction.py b/tests/test_cutout_interaction.py index e8740cd..cdaa17a 100644 --- a/tests/test_cutout_interaction.py +++ b/tests/test_cutout_interaction.py @@ -1,9 +1,8 @@ +import json import sys from pathlib import Path from types import MethodType, SimpleNamespace -import bpy - SOURCE_ROOT = Path(__file__).parents[1] / "src" sys.path.insert(0, str(SOURCE_ROOT)) @@ -83,6 +82,15 @@ def test_cutout_pie_passes_image_lasso_options_to_all_ai_shapes(monkeypatch): context = cutout_context(window_manager, mesh_detail="MEDIUM", generate_normal=True) source = cutout_source(size=(4096, 2048)) monkeypatch.setattr(interaction, "ai_ready", lambda: True) + serialized = [] + original_to_json = SelectionPath.to_json + + def record_to_json(selection_path): + value = original_to_json(selection_path) + serialized.append(value) + return value + + monkeypatch.setattr(SelectionPath, "to_json", record_to_json) interaction.open_shape_pie( context, @@ -103,6 +111,12 @@ def test_cutout_pie_passes_image_lasso_options_to_all_ai_shapes(monkeypatch): ] assert all(call[2].refine_selection for call in window_manager.pie.calls) assert all(call[2].generate_normal for call in window_manager.pie.calls) + assert len(serialized) == 1 + assert set(json.loads(serialized[0])) == {"points", "invert"} + assert all( + call[2].selection_path_json == serialized[0] + for call in window_manager.pie.calls + ) assert all( SelectionPath.from_json(call[2].selection_path_json).invert for call in window_manager.pie.calls @@ -132,12 +146,7 @@ def test_cutout_mesh_detail_maps_to_denser_sampling(): def test_cutout_reuses_image_lasso_gesture_and_double_click_selects_all(monkeypatch): - bpy.ops.wm.read_factory_settings(use_empty=True) - image = bpy.data.images.new("Source", width=320, height=180) - source = bpy.data.objects.new("Source", None) - source.empty_display_type = "IMAGE" - source.data = image - bpy.context.collection.objects.link(source) + source = cutout_source(size=(65536, 32768)) window_manager = WindowManager() context = cutout_context(window_manager) @@ -145,15 +154,32 @@ def test_cutout_reuses_image_lasso_gesture_and_double_click_selects_all(monkeypa cutout_main, "resolve_image_edit_click", lambda *_args: (source, True) ) monkeypatch.setattr(cutout_main, "video_total_frames", lambda _image: 1) + monkeypatch.setattr(cutout_main, "require_image_empty", lambda _name: source) monkeypatch.setattr(cutout_main, "apply_selection_settings", lambda *_args: None) - monkeypatch.setattr(cutout_main, "validate_cutout_selection", lambda *_args: None) + monkeypatch.setattr(cutout_main, "serialize_matrix", lambda _matrix: "matrix") + monkeypatch.setattr( + cutout_main, + "image_rgba", + lambda *_args: (_ for _ in ()).throw(AssertionError("unexpected pixel read")), + ) + monkeypatch.setattr( + cutout_main, + "rasterize_selection_path", + lambda *_args, **_kwargs: (_ for _ in ()).throw( + AssertionError("unexpected Selection rasterization") + ), + ) operator = SimpleNamespace( full_image=True, refine_selection=False, report=lambda *_args: None, ) - operator.execute = MethodType(cutout_main.SelectCutoutSelection.execute, operator) + source.matrix_world = object() + operator._submit_selection_path = MethodType( + cutout_main.SelectCutoutSelection._submit_selection_path, + operator, + ) result = cutout_main.SelectCutoutSelection.invoke( operator, context, SimpleNamespace(value="DOUBLE_CLICK") ) @@ -161,8 +187,13 @@ def test_cutout_reuses_image_lasso_gesture_and_double_click_selects_all(monkeypa assert issubclass(cutout_main.SelectCutoutSelection, ImageSelectionGesture) assert result == {"FINISHED"} selection_path = SelectionPath.from_json(window_manager.pie.calls[0][2].selection_path_json) - assert selection_path.points == ((0, 0), (320, 0), (320, 180), (0, 180)) - assert window_manager.pie.calls[0][2].mesh_resolution == 10 + assert selection_path.points == ( + (0, 0), + (65536, 0), + (65536, 32768), + (0, 32768), + ) + assert window_manager.pie.calls[0][2].mesh_resolution == 1024 assert not hasattr(cutout_main, "pending_selection") diff --git a/tests/test_cutout_objects.py b/tests/test_cutout_objects.py index 3ad8747..6b8b62d 100644 --- a/tests/test_cutout_objects.py +++ b/tests/test_cutout_objects.py @@ -5,7 +5,8 @@ from unittest.mock import patch import bpy import numpy as np -from mathutils import Vector +import pytest +from mathutils import Matrix, Vector SOURCE_ROOT = Path(__file__).parents[1] / "src" sys.path.insert(0, str(SOURCE_ROOT)) @@ -16,10 +17,12 @@ from anyimage.common import material as material_data from anyimage.common import node as node_asset from anyimage.common import object as object_data from anyimage.common.selection import ( + ImageEditWarning, SelectionMask, SelectionPath, rasterize_selection_path, ) +from anyimage.operators.clipboard_image import actions as clipboard_actions from anyimage.operators.convert_to_plane import object as plane_object from anyimage.operators.cutout_tool import geometry as cutout_geometry from anyimage.operators.cutout_tool import main as cutout_main @@ -64,21 +67,30 @@ def _create_cutout_shape( if selection_values is None: selection_values = np.ones((bottom - top, right - left), dtype=np.float32) selection_mask = SelectionMask(selection_values, bounds) + source_rgba = image_data.image_rgba(source.data) + color_image = cutout_object.create_cropped_color_image( + source, + selection_mask.bounds, + source_rgba, + ) base_shape = cutout_geometry.build_base_shape( bpy.context, source, detail, - selection_mask, + selection_mask.values, + selection_mask.bounds, ) return cutout_object.create_shape_object( bpy.context, source, shape, detail, - selection_mask, + selection_mask.values, + selection_mask.bounds, base_shape, depth_image=depth_image, depth_metadata=depth_metadata, + color_image=color_image, ) @@ -177,16 +189,15 @@ def test_cutout_modifier_wraps_four_menu_controlled_shape_groups(): cutout_object.load_cutout_node_group() == cutout for shape in cutout_shape.CUTOUT_SHAPES ) - assert _inputs(plane) == ["Geometry", "Subdivide", "Thickness", "Material"] + assert _inputs(plane) == ["Geometry", "Subdivide", "Thickness"] assert _inputs(depth_plane) == [ "Geometry", "Subdivide", "Thickness", - "Depth Amount", - "Reference Depth", + "Depth Scale", + "Base Plane Depth", "Uniform Scale", "Depth Image", - "Material", ] assert _inputs(cutout) == [ "Geometry", @@ -391,6 +402,12 @@ def test_cutout_modifier_wraps_four_menu_controlled_shape_groups(): if item.item_type == "SOCKET" and item.name == "Thickness" ) assert thickness.subtype == "DISTANCE" + depth_subdivide = next( + item + for item in depth_plane.interface.items_tree + if item.item_type == "SOCKET" and item.name == "Subdivide" + ) + assert depth_subdivide.default_value == 6 for group in ( child_groups["BALLOON"], child_groups["DEPTH_BALLOON"], @@ -504,6 +521,31 @@ def test_build_plane_keeps_one_quad_and_generates_topology_in_geometry_nodes(): evaluated_mesh = evaluated.to_mesh() try: assert len(evaluated_mesh.polygons) > 0 + assert result.data.materials[0] == material + assert evaluated_mesh.materials[0].original == material + assert all(polygon.material_index == 0 for polygon in evaluated_mesh.polygons) + finally: + evaluated.to_mesh_clear() + + +def test_clipboard_plane_uses_the_mesh_material_slot(): + bpy.ops.wm.read_factory_settings(use_empty=True) + image = bpy.data.images.new("Clipboard Plane", width=128, height=64, alpha=True) + + with patch.object(clipboard_actions, "_place_in_view"): + result = clipboard_actions.add_image_plane(bpy.context, image) + + assert _inputs(result.modifiers[0].node_group) == [ + "Geometry", + "Subdivide", + "Thickness", + ] + assert len(result.data.materials) == 1 + evaluated = result.evaluated_get(bpy.context.evaluated_depsgraph_get()) + evaluated_mesh = evaluated.to_mesh() + try: + assert evaluated_mesh.materials[0].original == result.data.materials[0] + assert all(polygon.material_index == 0 for polygon in evaluated_mesh.polygons) finally: evaluated.to_mesh_clear() @@ -787,6 +829,192 @@ def _evaluated_positions(obj): evaluated.to_mesh_clear() +def test_converted_planes_center_the_origin_without_moving_world_corners(): + bpy.ops.wm.read_factory_settings(use_empty=True) + + def create_source(name): + image = bpy.data.images.new(name, width=200, height=100) + source = bpy.data.objects.new(name, None) + source.empty_display_type = "IMAGE" + source.data = image + source.empty_display_size = 2.0 + source.empty_image_offset = (0.25, -0.2) + source.matrix_world = ( + Matrix.Translation((3.0, -2.0, 5.0)) + @ Matrix.Rotation(0.4, 4, "Z") + @ Matrix.Diagonal((1.5, 0.75, 2.0, 1.0)) + ) + bpy.context.collection.objects.link(source) + return source + + def assert_centered(result, bounds, source_matrix): + x_min, x_max, y_min, y_max = bounds + center = Vector( + ((x_min + x_max) * 0.5, (y_min + y_max) * 0.5, 0.0) + ) + assert np.allclose( + tuple(result.matrix_world.translation), + tuple(source_matrix @ center), + atol=1e-6, + ) + local = np.asarray([tuple(vertex.co) for vertex in result.data.vertices]) + assert np.allclose(local[:, :2].min(axis=0), (-1.0, -0.5), atol=1e-6) + assert np.allclose(local[:, :2].max(axis=0), (1.0, 0.5), atol=1e-6) + expected = sorted( + tuple(source_matrix @ Vector((x, y, 0.0))) + for x in (x_min, x_max) + for y in (y_min, y_max) + ) + actual = sorted( + tuple(result.matrix_world @ vertex.co) + for vertex in result.data.vertices + ) + assert np.allclose(actual, expected, atol=1e-6) + + plane_source = create_source("Centered Plane") + plane_bounds = image_data.image_empty_bounds(plane_source) + plane_matrix = plane_source.matrix_world.copy() + plane_material = bpy.data.materials.new("Centered Plane Material") + plane = plane_object.create_plane_object( + bpy.context, + plane_source, + 0, + plane_material, + ) + assert_centered(plane, plane_bounds, plane_matrix) + assert plane["anyimage_mesh_shape"] == "PLANE" + + depth_source = create_source("Centered Depth Plane") + depth_bounds = image_data.image_empty_bounds(depth_source) + depth_matrix = depth_source.matrix_world.copy() + depth_values = np.ones((2, 2, 3), dtype=np.float32) + depth_image = _depth_image(depth_values, np.ones((2, 2), dtype=bool)) + depth_material = bpy.data.materials.new("Centered Depth Material") + depth_plane = plane_object.create_depth_plane_object( + bpy.context, + depth_source, + 0, + depth_material, + depth_image, + _depth_metadata( + ((0.5, 0.0, 0.5), (0.0, 0.5, 0.5), (0.0, 0.0, 1.0)), + (2, 2), + 1.0, + ), + ) + assert_centered(depth_plane, depth_bounds, depth_matrix) + assert depth_plane["anyimage_mesh_shape"] == "DEPTH_PLANE" + depth_modifier = depth_plane.modifiers[0] + uniform_scale = depth_modifier[ + object_data.modifier_input_identifier( + depth_modifier.node_group, + "Uniform Scale", + ) + ] + assert np.isclose( + depth_modifier[ + object_data.modifier_input_identifier( + depth_modifier.node_group, + "Base Plane Depth", + ) + ], + uniform_scale, + ) + + +def test_depth_plane_builds_a_fixed_base_terrain_block(): + bpy.ops.wm.read_factory_settings(use_empty=True) + image = bpy.data.images.new("Terrain", width=4, height=4) + source = bpy.data.objects.new("Terrain", None) + source.empty_display_type = "IMAGE" + source.data = image + source.empty_display_size = 2.0 + bpy.context.collection.objects.link(source) + material = bpy.data.materials.new("Terrain Material") + depth_image = _depth_image( + np.ones((4, 4, 3), dtype=np.float32), + np.ones((4, 4), dtype=bool), + ) + result = plane_object.create_depth_plane_object( + bpy.context, + source, + 2, + material, + depth_image, + _depth_metadata( + ((1.5, 0.0, 1.5), (0.0, 1.5, 1.5), (0.0, 0.0, 1.0)), + (4, 4), + 1.0, + ), + ) + modifier = result.modifiers[0] + group = modifier.node_group + + def set_input(name, value): + object_data.set_modifier_input( + modifier, + object_data.modifier_input_identifier(group, name), + value, + ) + result.update_tag(refresh={"DATA"}) + bpy.context.view_layer.update() + + set_input("Depth Scale", 0.0) + baseline = _evaluated_positions(result) + assert np.isclose(baseline[:, 2].min(), 0.0, atol=1e-6) + assert np.isclose(baseline[:, 2].max(), 0.1, atol=1e-6) + baseline_base = { + (round(float(x), 6), round(float(y), 6)) + for x, y, z in baseline + if np.isclose(z, 0.0, atol=1e-6) + } + + evaluated = result.evaluated_get(bpy.context.evaluated_depsgraph_get()) + evaluated_mesh = evaluated.to_mesh() + try: + assert result.data.materials[0] == material + assert evaluated_mesh.materials[0].original == material + assert all(polygon.material_index == 0 for polygon in evaluated_mesh.polygons) + assert "UVMap" in evaluated_mesh.uv_layers + finally: + evaluated.to_mesh_clear() + + set_input("Depth Scale", 1.0) + set_input("Base Plane Depth", 1.2) + outward = _evaluated_positions(result) + assert np.isclose(outward[:, 2].min(), 0.0, atol=1e-6) + assert np.isclose(outward[:, 2].max(), 0.3, atol=1e-5) + outward_base = { + (round(float(x), 6), round(float(y), 6)) + for x, y, z in outward + if np.isclose(z, 0.0, atol=1e-6) + } + assert outward_base == baseline_base + z_levels = np.unique(np.round(outward[:, 2], 6)) + assert len(z_levels) >= 4 + assert np.allclose(np.diff(z_levels), np.diff(z_levels)[0], atol=1e-5) + + set_input("Base Plane Depth", 0.95) + inward = _evaluated_positions(result) + assert np.isclose(inward[:, 2].min(), 0.0, atol=1e-6) + assert np.isclose(inward[:, 2].max(), 0.05, atol=1e-5) + + set_input("Base Plane Depth", 0.0) + limited = _evaluated_positions(result) + assert np.isclose(limited[:, 2].min(), 0.0, atol=1e-6) + assert limited[:, 2].max() > 0.0 + assert limited[:, 2].max() < 0.001 + + set_input("Base Plane Depth", 1.3) + outward_beyond_thickness = _evaluated_positions(result) + assert np.isclose(outward_beyond_thickness[:, 2].max(), 0.4, atol=1e-5) + + set_input("Thickness", 0.0) + set_input("Base Plane Depth", 1.2) + sheet = _evaluated_positions(result) + assert np.allclose(sheet[:, 2], 0.2, atol=1e-5) + + def _add_cutout_modifier(obj, shape): group = cutout_object.load_cutout_node_group() modifier = obj.modifiers.new(group.name, "NODES") @@ -1206,6 +1434,41 @@ def test_converted_color_images_are_new_and_use_blender_duplicate_suffix(): assert second.name == "Poster_color.png.001" +def test_bounds_input_preserves_rectangular_rgba_and_stays_packed_after_cleanup(): + bpy.ops.wm.read_factory_settings(use_empty=True) + image = bpy.data.images.new("Bounds Source", width=5, height=4, alpha=True) + rgba = np.arange(4 * 5 * 4, dtype=np.float32).reshape((4, 5, 4)) / 100.0 + rgba[:, :, 3] = np.asarray( + ( + (0.0, 0.1, 0.2, 0.3, 0.4), + (0.5, 0.6, 0.7, 0.8, 0.9), + (1.0, 0.9, 0.8, 0.7, 0.6), + (0.5, 0.4, 0.3, 0.2, 0.1), + ), + dtype=np.float32, + ) + path = image_data.prepare_image_bounds_input(image, rgba, (1, 1, 5, 4)) + result = None + try: + result = image_data.load_packed_rgba_image( + path, + image, + "Bounds Result", + ) + loaded = image_data.image_rgba(result) + np.testing.assert_allclose(loaded, rgba[1:4, 1:5], atol=1.0 / 255.0) + assert result.packed_file is not None + + image_data.cleanup_image_input(path, True) + + assert not path.exists() + assert result.packed_file is not None + finally: + image_data.cleanup_image_input(path, True) + if result is not None: + bpy.data.images.remove(result, do_unlink=True) + + def test_local_cutout_selection_matches_uint8_tenth_alpha_boundary(): top_down = np.ones((4, 4, 4), dtype=np.float32) top_down[1, 1, 3] = 25.0 / 255.0 @@ -1214,17 +1477,30 @@ def test_local_cutout_selection_matches_uint8_tenth_alpha_boundary(): (4, 4), SelectionPath( points=((0, 0), (4, 0), (4, 4), (0, 4)), - fill_rule="NONZERO", ), - source_alpha=top_down[:, :, 3], - alpha_threshold=0.1, ) - selection = selection_mask.full_values((4, 4)) + selection = cutout_main._cutout_content_values( + top_down[:, :, 3], + selection_mask.values, + ) - assert selection[1, 1] == 0.0 + assert selection[1, 1] <= cutout_geometry.CUTOUT_ALPHA_THRESHOLD assert selection[1, 2] > 0.1 +def test_cutout_content_merge_keeps_the_geometric_selection_unchanged(): + selection = np.asarray(((0.25, 1.0), (0.5, 0.75)), dtype=np.float32) + original = selection.copy() + alpha = np.asarray(((1.0, 0.5), (0.0, 0.25)), dtype=np.float32) + + content = cutout_main._cutout_content_values(alpha, selection) + + np.testing.assert_array_equal(selection, original) + np.testing.assert_allclose(content, alpha * original) + with pytest.raises(ImageEditWarning, match="no visible image pixels"): + cutout_main._cutout_content_values(np.zeros((2, 2)), selection) + + def test_local_cutout_builds_from_blender_image_pixels_without_a_job(): bpy.ops.wm.read_factory_settings(use_empty=True) image = bpy.data.images.new("Local Cutout", width=64, height=48, alpha=True) @@ -1239,7 +1515,6 @@ def test_local_cutout_builds_from_blender_image_pixels_without_a_job(): bpy.context.view_layer.objects.active = source selection_path = SelectionPath( points=((-8, 8), (72, 8), (72, 40), (-8, 40)), - fill_rule="NONZERO", ) source_bounds = image_data.image_empty_bounds(source) @@ -1252,17 +1527,26 @@ def test_local_cutout_builds_from_blender_image_pixels_without_a_job(): selection_mask = rasterize_selection_path( tuple(image.size), selection_path, - source_alpha=rgba[:, :, 3], - alpha_threshold=cutout_geometry.CUTOUT_ALPHA_THRESHOLD, antialias=True, ) + left, top, right, bottom = selection_mask.bounds + content_values = cutout_main._cutout_content_values( + rgba[top:bottom, left:right, 3], + selection_mask.values, + ) + color_image = cutout_object.create_cropped_color_image( + source, + selection_mask.bounds, + rgba, + ) cutout_main.create_cutout_shape( bpy.context, source, "SURFACE", 16, - selection_mask, - source_rgba=rgba, + content_values, + selection_mask.bounds, + color_image=color_image, ) read_rgba.assert_called_once_with(image) @@ -1315,10 +1599,10 @@ def test_local_cutout_builds_from_blender_image_pixels_without_a_job(): if node.bl_idname == "ShaderNodeTexImage" ) assert color_texture.image is not image - assert tuple(color_texture.image.size) == (64, 34) - copied = np.empty(34 * 64 * 4, dtype=np.float32) + assert tuple(color_texture.image.size) == (64, 36) + copied = np.empty(36 * 64 * 4, dtype=np.float32) color_texture.image.pixels.foreach_get(copied) - copied = copied.reshape((34, 64, 4)) + copied = copied.reshape((36, 64, 4)) assert np.allclose(copied[:, :, :3], 1.0) left, top, right, bottom = selection_mask.bounds assert np.allclose( diff --git a/tests/test_selection.py b/tests/test_selection.py new file mode 100644 index 0000000..b35ada9 --- /dev/null +++ b/tests/test_selection.py @@ -0,0 +1,232 @@ +import json +import sys +from pathlib import Path + +import numpy as np +import pytest + +SOURCE_ROOT = Path(__file__).parents[1] / "src" +sys.path.insert(0, str(SOURCE_ROOT)) + +from anyimage.common import selection + + +def reference_selection_mask( + image_size, + selection_path, + *, + antialias=False, +): + width, height = map(int, image_size) + points = np.asarray(selection_path.points, dtype=np.float64) + x = np.arange(width, dtype=np.float64) + 0.5 + y = np.arange(height, dtype=np.float64) + 0.5 + winding = np.zeros((height, width), dtype=np.int16) + previous = points[-1] + for current in points: + crosses = (current[1] > y) != (previous[1] > y) + edge_x = (previous[0] - current[0]) * (y - current[1]) / ( + previous[1] - current[1] + 1e-300 + ) + current[0] + selected = crosses[:, None] & (x[None, :] < edge_x[:, None]) + winding += selected * (1 if current[1] > previous[1] else -1) + previous = current + mask = winding != 0 + mask = ( + selection._antialias_mask(mask) + if antialias + else mask.astype(np.float32) + ) + if selection_path.invert: + mask = 1.0 - mask + bounds = selection._selection_canvas_bounds( + image_size, + selection_path, + 2 if antialias else 0, + ) + left, top, right, bottom = bounds + mask = mask[top:bottom, left:right] + if not np.any(mask > 0.0): + raise selection.ImageEditWarning( + "The selection contains no visible image pixels" + ) + return selection.SelectionMask(mask, bounds) + + +def assert_matches_reference(image_size, selection_path, **options): + try: + expected = reference_selection_mask(image_size, selection_path, **options) + except selection.ImageEditWarning: + with pytest.raises(selection.ImageEditWarning): + selection.rasterize_selection_path( + image_size, + selection_path, + **options, + ) + return + actual = selection.rasterize_selection_path( + image_size, + selection_path, + **options, + ) + assert actual.bounds == expected.bounds + np.testing.assert_array_equal(actual.values, expected.values) + + +def test_selection_path_json_contains_only_points_and_invert(): + selection_path = selection.SelectionPath( + points=((1, 2), (8, 2), (4, 9)), + invert=True, + ) + + value = selection_path.to_json() + + assert set(json.loads(value)) == {"points", "invert"} + assert selection.SelectionPath.from_json(value) == selection_path + with pytest.raises(ValueError, match="Selection Path data is invalid"): + selection.SelectionPath.from_json( + json.dumps( + { + "points": selection_path.points, + "invert": selection_path.invert, + "fill_rule": "NONZERO", + } + ) + ) + + +def test_nonzero_winding_keeps_same_direction_overlap(): + points = ( + (1, 1), + (7, 1), + (7, 7), + (1, 7), + (1, 1), + (4, 1), + (10, 1), + (10, 7), + (4, 7), + (4, 1), + (1, 1), + ) + + mask = selection.rasterize_selection_path( + (12, 9), + selection.SelectionPath(points), + ).full_values((12, 9)) + + assert mask[3, 5] == 1.0 + + +def test_scanline_matches_reference_for_edge_cases(): + paths = ( + ((1, 1), (15, 1), (15, 10), (1, 10)), + ((0.2, 0.3), (15.8, 2.1), (8.4, 11.7)), + ((1, 1), (15, 1), (15, 5), (7, 5), (7, 11), (1, 11)), + ((1, 1), (15, 11), (1, 11), (15, 1)), + ((-8, -3), (10, 1), (23, 8), (7, 18), (-4, 10)), + ) + for points in paths: + for invert in (False, True): + for antialias in (False, True): + assert_matches_reference( + (18, 14), + selection.SelectionPath(points, invert=invert), + antialias=antialias, + ) + + +def test_scanline_matches_reference_for_random_paths(): + random = np.random.default_rng(20260901) + for _index in range(40): + point_count = int(random.integers(3, 60)) + points = tuple( + map( + tuple, + random.uniform((-8, -6), (40, 30), size=(point_count, 2)), + ) + ) + for invert in (False, True): + for antialias in (False, True): + assert_matches_reference( + (32, 24), + selection.SelectionPath(points, invert=invert), + antialias=antialias, + ) + + +def test_rasterizer_has_no_image_content_parameters(): + import inspect + + parameters = inspect.signature(selection.rasterize_selection_path).parameters + + assert "source_alpha" not in parameters + assert "alpha_threshold" not in parameters + + +def test_non_inverted_antialias_uses_only_padded_local_canvas(monkeypatch): + selection_path = selection.SelectionPath( + ((100, 200), (500, 200), (500, 600), (100, 600)) + ) + calls = [] + rasterize = selection._rasterize_path + + def record(bounds, points): + calls.append(bounds) + return rasterize(bounds, points) + + monkeypatch.setattr(selection, "_rasterize_path", record) + + selection.rasterize_selection_path( + (5120, 2880), + selection_path, + antialias=True, + ) + + assert calls == [(98, 198, 502, 602)] + + +def test_inverted_selection_uses_full_canvas(monkeypatch): + selection_path = selection.SelectionPath( + ((100, 200), (500, 200), (500, 600), (100, 600)), + invert=True, + ) + calls = [] + rasterize = selection._rasterize_path + + def record(bounds, points): + calls.append(bounds) + return rasterize(bounds, points) + + monkeypatch.setattr(selection, "_rasterize_path", record) + + selection.rasterize_selection_path((5120, 2880), selection_path) + + assert calls == [(0, 0, 5120, 2880)] + + +def test_high_resolution_lasso_allocates_one_local_scanline_table(monkeypatch): + angles = np.arange(1000, dtype=np.float64) * (2.0 * np.pi / 1000.0) + points = tuple( + zip( + 2560.0 + 500.0 * np.cos(angles), + 1440.0 + 300.0 * np.sin(angles), + ) + ) + bounds = selection.selection_path_bounds(points) + allocations = [] + zeros = np.zeros + + def record(shape, *args, **kwargs): + allocations.append(shape) + return zeros(shape, *args, **kwargs) + + monkeypatch.setattr(np, "zeros", record) + + mask = selection._rasterize_path(bounds, points) + + width = bounds[2] - bounds[0] + height = bounds[3] - bounds[1] + assert mask.shape == (height, width) + assert allocations == [(height, width + 1)] + assert (height, width) != (2880, 5120) diff --git a/tests/test_server_jobs.py b/tests/test_server_jobs.py index 998ef35..3e692f6 100644 --- a/tests/test_server_jobs.py +++ b/tests/test_server_jobs.py @@ -126,6 +126,7 @@ class ServerJobsTest(unittest.TestCase): cls.debug_da3_outputs = load_server_module("outputs.debug_da3") cls.debug_moge2_outputs = load_server_module("outputs.debug_moge2") cls.debug_jobs = load_server_module("jobs.debug_predictions") + cls.depth_plane_job = load_server_module("jobs.depth_plane") cls.upscale_job = load_server_module("jobs.upscale") cls.download = load_server_module("model_download") @@ -139,6 +140,51 @@ class ServerJobsTest(unittest.TestCase): "Upscaling image (2/3)", ) + def test_depth_plane_job_requests_z_depth_and_tangent_normal_together(self): + with tempfile.TemporaryDirectory() as directory: + root = Path(directory) + input_path = root / "input.png" + input_path.touch() + context = FakeJobContext(root / "result", "generate-depth-plane-geometry") + z_depth = context.directory / "z-depth.exr" + metadata = context.directory / "depth.json" + tangent_normal = context.directory / "tangent-normal.png" + parameters = {"input": str(input_path)} + with ( + patch.object( + self.depth_plane_job, + "input_frames", + return_value=nullcontext([input_path]), + ), + patch.object(self.depth_plane_job, "copy_color_images"), + patch.object( + self.depth_plane_job, + "generate_moge2_artifacts", + return_value={ + "z_depth": z_depth, + "depth_metadata": metadata, + "tangent_normal": tangent_normal, + }, + ) as generate, + ): + result = self.depth_plane_job.run(context, parameters) + + self.assertEqual( + result, + { + "z_depth": "z-depth.exr", + "depth_metadata": "depth.json", + "tangent_normal": "tangent-normal.png", + }, + ) + generate.assert_called_once_with( + context, + parameters, + input_path, + depth_mode="Z", + normal_mode="TANGENT", + ) + def test_collect_images_is_sorted_and_filters_suffixes(self): with tempfile.TemporaryDirectory() as directory: root = Path(directory) diff --git a/tests/test_server_runtime.py b/tests/test_server_runtime.py index 739e701..7cf3613 100644 --- a/tests/test_server_runtime.py +++ b/tests/test_server_runtime.py @@ -691,6 +691,7 @@ class ServerRuntimeTest(unittest.TestCase): normal_mode="OBJECT", ) + infer.assert_called_once() self.assertEqual( set(result), {"vector_depth", "depth_metadata", "object_normal"}, @@ -749,7 +750,7 @@ class ServerRuntimeTest(unittest.TestCase): normal_mode="OBJECT", ) - def test_generate_cutout_artifacts_refine_an_already_rasterized_selection(self): + def test_generate_cutout_artifacts_refines_the_bounds_image(self): job = sys.modules["server.jobs.cutout"] input_path = self.storage_root / "input.png" input_path.write_bytes(b"selection") diff --git a/tools/node_assets/build_depth_plane.py b/tools/node_assets/build_depth_plane.py index f5c577f..f9d92c6 100644 --- a/tools/node_assets/build_depth_plane.py +++ b/tools/node_assets/build_depth_plane.py @@ -6,11 +6,12 @@ import bpy DEPTH_PLANE_NODE_GROUP_NAME = "O Image Depth Plane" -def _interface_socket(group, name, in_out, socket_type): +def _interface_socket(group, name, in_out, socket_type, parent=None): socket = group.interface.new_socket( name=name, in_out=in_out, socket_type=socket_type, + parent=parent, ) if in_out == "INPUT" and hasattr(socket, "structure_type"): socket.structure_type = "SINGLE" @@ -26,6 +27,13 @@ def _group_input(nodes, location, visible_names): return node +def _math(nodes, operation, location): + node = nodes.new("ShaderNodeMath") + node.operation = operation + node.location = location + return node + + def build_depth_plane_group(mesh_plane_group): group = bpy.data.node_groups.get(DEPTH_PLANE_NODE_GROUP_NAME) if group is not None: @@ -35,156 +43,167 @@ def build_depth_plane_group(mesh_plane_group): "GeometryNodeTree", ) group.is_modifier = True - _interface_socket(group, "Geometry", "INPUT", "NodeSocketGeometry") + geometry = _interface_socket(group, "Geometry", "INPUT", "NodeSocketGeometry") + geometry.structure_type = "SINGLE" subdivide = _interface_socket(group, "Subdivide", "INPUT", "NodeSocketInt") - subdivide.default_value = 7 + subdivide.default_value = 6 subdivide.min_value = 0 subdivide.max_value = 10 thickness = _interface_socket(group, "Thickness", "INPUT", "NodeSocketFloat") thickness.default_value = 0.1 thickness.min_value = 0.0 thickness.subtype = "DISTANCE" - depth_amount = _interface_socket(group, "Depth Amount", "INPUT", "NodeSocketFloat") - depth_amount.default_value = 1.0 - depth_amount.min_value = 0.0 - reference_depth = _interface_socket( - group, "Reference Depth", "INPUT", "NodeSocketFloat" + depth_scale = _interface_socket(group, "Depth Scale", "INPUT", "NodeSocketFloat") + depth_scale.default_value = 1.0 + depth_scale.min_value = 0.0 + options = group.interface.new_panel(name="Options", default_closed=True) + base_plane_depth = _interface_socket( + group, + "Base Plane Depth", + "INPUT", + "NodeSocketFloat", + parent=options, ) - reference_depth.hide_in_modifier = True + base_plane_depth.default_value = 1.0 + base_plane_depth.min_value = 0.0 + base_plane_depth.subtype = "DISTANCE" + data = group.interface.new_panel(name="Data", default_closed=True) uniform_scale = _interface_socket( - group, "Uniform Scale", "INPUT", "NodeSocketFloat" + group, + "Uniform Scale", + "INPUT", + "NodeSocketFloat", + parent=data, ) + uniform_scale.default_value = 1.0 + uniform_scale.min_value = 0.0 uniform_scale.hide_in_modifier = True - depth_image = _interface_socket(group, "Depth Image", "INPUT", "NodeSocketImage") + depth_image = _interface_socket( + group, + "Depth Image", + "INPUT", + "NodeSocketImage", + parent=data, + ) depth_image.hide_in_modifier = True - material = _interface_socket(group, "Material", "INPUT", "NodeSocketMaterial") - material.hide_in_modifier = True _interface_socket(group, "Geometry", "OUTPUT", "NodeSocketGeometry") nodes = group.nodes links = group.links geometry_input = _group_input( nodes, - (-1850.0, 450.0), - {"Geometry", "Subdivide", "Material"}, + (-1900.0, 500.0), + {"Geometry", "Subdivide"}, ) shape_input = _group_input( nodes, - (-1250.0, 850.0), - {"Thickness", "Depth Amount"}, + (-1550.0, 950.0), + {"Thickness", "Depth Scale", "Base Plane Depth"}, ) image_input = _group_input( nodes, - (-1050.0, 50.0), - {"Depth Image", "Reference Depth", "Uniform Scale"}, + (-1200.0, -350.0), + {"Depth Image", "Uniform Scale"}, ) - cleanup_input = _group_input(nodes, (650.0, -200.0), {"Thickness"}) group_output = nodes.new("NodeGroupOutput") - group_output.location = (1850.0, 450.0) + group_output.location = (1550.0, 500.0) mesh_plane = nodes.new("GeometryNodeGroup") mesh_plane.node_tree = mesh_plane_group - mesh_plane.inputs["Thickness"].default_value = 0.0 - mesh_plane.location = (-1550.0, 450.0) + mesh_plane.location = (-1600.0, 500.0) links.new(geometry_input.outputs["Geometry"], mesh_plane.inputs["Geometry"]) links.new(geometry_input.outputs["Subdivide"], mesh_plane.inputs["Subdivide"]) - links.new(geometry_input.outputs["Material"], mesh_plane.inputs["Material"]) + links.new(shape_input.outputs["Thickness"], mesh_plane.inputs["Thickness"]) - extrude = nodes.new("GeometryNodeExtrudeMesh") - extrude.mode = "FACES" - extrude.inputs["Offset"].default_value = (0.0, 0.0, -1.0) - extrude.inputs["Individual"].default_value = False - extrude.location = (-1200.0, 450.0) - links.new(mesh_plane.outputs["Geometry"], extrude.inputs["Mesh"]) - links.new(shape_input.outputs["Thickness"], extrude.inputs["Offset Scale"]) - geometry = extrude.outputs["Mesh"] + positive_half = _math(nodes, "MULTIPLY", (-1300.0, 300.0)) + positive_half.inputs[1].default_value = 0.5 + links.new(shape_input.outputs["Thickness"], positive_half.inputs[0]) + translation = nodes.new("ShaderNodeCombineXYZ") + translation.location = (-1100.0, 300.0) + links.new(positive_half.outputs[0], translation.inputs["Z"]) + place_block = nodes.new("GeometryNodeTransform") + place_block.location = (-900.0, 500.0) + links.new(mesh_plane.outputs["Geometry"], place_block.inputs["Geometry"]) + links.new(translation.outputs["Vector"], place_block.inputs["Translation"]) uv = nodes.new("GeometryNodeInputNamedAttribute") uv.data_type = "FLOAT_VECTOR" uv.inputs["Name"].default_value = "UVMap" - uv.location = (-1000.0, 50.0) + uv.location = (-950.0, -350.0) texture = nodes.new("GeometryNodeImageTexture") - texture.location = (-800.0, 50.0) + texture.interpolation = "Linear" + texture.extension = "EXTEND" + texture.location = (-700.0, -350.0) links.new(image_input.outputs["Depth Image"], texture.inputs["Image"]) links.new(uv.outputs["Attribute"], texture.inputs["Vector"]) - separate = nodes.new("ShaderNodeSeparateXYZ") - separate.location = (-600.0, 50.0) - links.new(texture.outputs["Color"], separate.inputs["Vector"]) - relative_depth = nodes.new("ShaderNodeMath") - relative_depth.operation = "SUBTRACT" - relative_depth.location = (-600.0, -150.0) - links.new(image_input.outputs["Reference Depth"], relative_depth.inputs[0]) - links.new(separate.outputs["X"], relative_depth.inputs[1]) - local_depth = nodes.new("ShaderNodeMath") - local_depth.operation = "MULTIPLY" - local_depth.location = (-350.0, -150.0) - links.new(relative_depth.outputs[0], local_depth.inputs[0]) - links.new(image_input.outputs["Uniform Scale"], local_depth.inputs[1]) - raw_depth = nodes.new("ShaderNodeMath") - raw_depth.operation = "MULTIPLY" - raw_depth.location = (-100.0, 150.0) - links.new(local_depth.outputs[0], raw_depth.inputs[0]) - links.new(shape_input.outputs["Depth Amount"], raw_depth.inputs[1]) - inward_limit = nodes.new("ShaderNodeMath") - inward_limit.operation = "MULTIPLY" - inward_limit.inputs[1].default_value = -0.98 - inward_limit.location = (-100.0, -50.0) - links.new(shape_input.outputs["Thickness"], inward_limit.inputs[0]) - clamp = nodes.new("ShaderNodeMath") - clamp.operation = "MAXIMUM" - clamp.location = (150.0, 50.0) - links.new(raw_depth.outputs[0], clamp.inputs[0]) - links.new(inward_limit.outputs[0], clamp.inputs[1]) - has_thickness = nodes.new("ShaderNodeMath") - has_thickness.operation = "GREATER_THAN" - has_thickness.inputs[1].default_value = 1e-6 - has_thickness.location = (-100.0, -350.0) - links.new(shape_input.outputs["Thickness"], has_thickness.inputs[0]) - choose_depth = nodes.new("GeometryNodeSwitch") - choose_depth.input_type = "FLOAT" - choose_depth.location = (400.0, 50.0) - links.new(has_thickness.outputs[0], choose_depth.inputs["Switch"]) - links.new(raw_depth.outputs[0], choose_depth.inputs["False"]) - links.new(clamp.outputs[0], choose_depth.inputs["True"]) - combine = nodes.new("ShaderNodeCombineXYZ") - combine.location = (600.0, 50.0) - links.new(choose_depth.outputs["Output"], combine.inputs["Z"]) - front = nodes.new("FunctionNodeBooleanMath") - front.operation = "NOT" - front.location = (400.0, 300.0) - links.new(extrude.outputs["Top"], front.inputs[0]) - set_position = nodes.new("GeometryNodeSetPosition") - set_position.location = (800.0, 450.0) - links.new(geometry, set_position.inputs["Geometry"]) - links.new(front.outputs[0], set_position.inputs["Selection"]) - links.new(combine.outputs["Vector"], set_position.inputs["Offset"]) - geometry = set_position.outputs["Geometry"] + depth_components = nodes.new("ShaderNodeSeparateXYZ") + depth_components.location = (-450.0, -350.0) + links.new(texture.outputs["Color"], depth_components.inputs["Vector"]) + metric_depth = _math(nodes, "MULTIPLY", (-200.0, -350.0)) + links.new(depth_components.outputs["X"], metric_depth.inputs[0]) + links.new(image_input.outputs["Uniform Scale"], metric_depth.inputs[1]) + relative_depth = _math(nodes, "SUBTRACT", (50.0, -350.0)) + links.new(shape_input.outputs["Base Plane Depth"], relative_depth.inputs[0]) + links.new(metric_depth.outputs[0], relative_depth.inputs[1]) + full_offset = _math(nodes, "MULTIPLY", (300.0, -350.0)) + links.new(relative_depth.outputs[0], full_offset.inputs[0]) + links.new(shape_input.outputs["Depth Scale"], full_offset.inputs[1]) - zero_thickness = nodes.new("ShaderNodeMath") - zero_thickness.operation = "LESS_THAN" - zero_thickness.inputs[1].default_value = 1e-6 - zero_thickness.location = (1000.0, -150.0) - links.new(cleanup_input.outputs["Thickness"], zero_thickness.inputs[0]) - back_or_side = nodes.new("FunctionNodeBooleanMath") - back_or_side.operation = "OR" - back_or_side.location = (1000.0, 150.0) - links.new(extrude.outputs["Top"], back_or_side.inputs[0]) - links.new(extrude.outputs["Side"], back_or_side.inputs[1]) - delete_selection = nodes.new("FunctionNodeBooleanMath") - delete_selection.operation = "AND" - delete_selection.location = (1200.0, 50.0) - links.new(zero_thickness.outputs[0], delete_selection.inputs[0]) - links.new(back_or_side.outputs[0], delete_selection.inputs[1]) - delete_geometry = nodes.new("GeometryNodeDeleteGeometry") - delete_geometry.domain = "FACE" - delete_geometry.location = (1350.0, 450.0) - links.new(geometry, delete_geometry.inputs["Geometry"]) - links.new(delete_selection.outputs[0], delete_geometry.inputs["Selection"]) + inward_limit = _math(nodes, "MULTIPLY", (50.0, -600.0)) + inward_limit.inputs[1].default_value = -0.999 + links.new(shape_input.outputs["Thickness"], inward_limit.inputs[0]) + limited_offset = _math(nodes, "MAXIMUM", (300.0, -600.0)) + links.new(full_offset.outputs[0], limited_offset.inputs[0]) + links.new(inward_limit.outputs[0], limited_offset.inputs[1]) + has_thickness = _math(nodes, "GREATER_THAN", (50.0, -850.0)) + has_thickness.inputs[1].default_value = 1e-6 + links.new(shape_input.outputs["Thickness"], has_thickness.inputs[0]) + safe_offset = nodes.new("GeometryNodeSwitch") + safe_offset.input_type = "FLOAT" + safe_offset.location = (550.0, -450.0) + links.new(has_thickness.outputs[0], safe_offset.inputs["Switch"]) + links.new(full_offset.outputs[0], safe_offset.inputs["False"]) + links.new(limited_offset.outputs[0], safe_offset.inputs["True"]) + + position = nodes.new("GeometryNodeInputPosition") + position.location = (-650.0, 50.0) + position_components = nodes.new("ShaderNodeSeparateXYZ") + position_components.location = (-400.0, 50.0) + links.new(position.outputs["Position"], position_components.inputs["Vector"]) + layer_ratio = _math(nodes, "DIVIDE", (-150.0, 50.0)) + links.new(position_components.outputs["Z"], layer_ratio.inputs[0]) + links.new(shape_input.outputs["Thickness"], layer_ratio.inputs[1]) + nonnegative_ratio = _math(nodes, "MAXIMUM", (350.0, 50.0)) + nonnegative_ratio.inputs[1].default_value = 0.0 + links.new(layer_ratio.outputs[0], nonnegative_ratio.inputs[0]) + clamped_ratio = _math(nodes, "MINIMUM", (600.0, 50.0)) + clamped_ratio.inputs[1].default_value = 1.0 + links.new(nonnegative_ratio.outputs[0], clamped_ratio.inputs[0]) + layer_weight = nodes.new("GeometryNodeSwitch") + layer_weight.input_type = "FLOAT" + layer_weight.location = (850.0, 50.0) + layer_weight.inputs["False"].default_value = 1.0 + links.new(has_thickness.outputs[0], layer_weight.inputs["Switch"]) + links.new(clamped_ratio.outputs[0], layer_weight.inputs["True"]) + + weighted_offset = _math(nodes, "MULTIPLY", (850.0, -350.0)) + links.new(safe_offset.outputs["Output"], weighted_offset.inputs[0]) + links.new(layer_weight.outputs["Output"], weighted_offset.inputs[1]) + offset = nodes.new("ShaderNodeCombineXYZ") + offset.location = (1050.0, -200.0) + links.new(weighted_offset.outputs[0], offset.inputs["Z"]) + set_position = nodes.new("GeometryNodeSetPosition") + set_position.location = (1050.0, 500.0) + links.new(place_block.outputs["Geometry"], set_position.inputs["Geometry"]) + links.new(offset.outputs["Vector"], set_position.inputs["Offset"]) shade_smooth = nodes.new("GeometryNodeSetShadeSmooth") shade_smooth.domain = "FACE" shade_smooth.inputs["Shade Smooth"].default_value = True - shade_smooth.location = (1600.0, 450.0) - links.new(delete_geometry.outputs["Geometry"], shade_smooth.inputs["Geometry"]) + shade_smooth.location = (1300.0, 500.0) + links.new(set_position.outputs["Geometry"], shade_smooth.inputs["Geometry"]) links.new(shade_smooth.outputs["Geometry"], group_output.inputs["Geometry"]) + group.color_tag = "GEOMETRY" + group.description = ( + "Build a single-sided depth terrain block with a fixed base." + ) group.use_fake_user = True return group - diff --git a/tools/node_assets/build_material_layers.py b/tools/node_assets/build_material_layers.py index dbd15ed..67c16f6 100644 --- a/tools/node_assets/build_material_layers.py +++ b/tools/node_assets/build_material_layers.py @@ -2,7 +2,7 @@ import bpy -from anyimage.common.object import ( +from anyimage.common.material import ( DEPTH_MATERIAL_NODE_GROUP_NAME, IMAGE_MATERIAL_NODE_GROUP_NAME, ) diff --git a/tools/node_assets/build_mesh_plane.py b/tools/node_assets/build_mesh_plane.py index 10661f5..a612d9b 100644 --- a/tools/node_assets/build_mesh_plane.py +++ b/tools/node_assets/build_mesh_plane.py @@ -42,15 +42,6 @@ def build_mesh_plane_group(): thickness.min_value = 0.0 thickness.max_value = 10000.0 thickness.subtype = "DISTANCE" - data_panel = interface.new_panel(name="Data", default_closed=True) - material = new_socket( - interface, - "Material", - "INPUT", - "NodeSocketMaterial", - parent=data_panel, - ) - material.hide_in_modifier = True new_socket(interface, "Geometry", "OUTPUT", "NodeSocketGeometry") nodes = group.nodes @@ -58,9 +49,8 @@ def build_mesh_plane_group(): geometry_input = new_group_input(nodes, (-1800.0, 500.0), {"Geometry"}) subdivide_input = new_group_input(nodes, (-1800.0, 0.0), {"Subdivide"}) thickness_input = new_group_input(nodes, (100.0, -250.0), {"Thickness"}) - material_input = new_group_input(nodes, (900.0, 0.0), {"Material"}) output = nodes.new("NodeGroupOutput") - output.location = (1350.0, 500.0) + output.location = (1750.0, 500.0) bounds = nodes.new("GeometryNodeBoundBox") bounds.location = (-1600.0, 500.0) @@ -196,7 +186,7 @@ def build_mesh_plane_group(): links.new(depth_ratio.outputs[0], rounded_depth.inputs[0]) depth_segments = nodes.new("ShaderNodeMath") depth_segments.operation = "MAXIMUM" - depth_segments.inputs[1].default_value = 1.0 + depth_segments.inputs[1].default_value = 3.0 depth_segments.location = (700.0, -100.0) links.new(rounded_depth.outputs[0], depth_segments.inputs[0]) depth_vertices = nodes.new("ShaderNodeMath") @@ -239,11 +229,27 @@ def build_mesh_plane_group(): links.new(place_grid.outputs["Geometry"], geometry_switch.inputs["False"]) links.new(place_cube.outputs["Geometry"], geometry_switch.inputs["True"]) - set_material = nodes.new("GeometryNodeSetMaterial") - set_material.location = (1050.0, 500.0) - links.new(geometry_switch.outputs["Output"], set_material.inputs["Geometry"]) - links.new(material_input.outputs["Material"], set_material.inputs["Material"]) - links.new(set_material.outputs["Geometry"], output.inputs["Geometry"]) + capture_source = nodes.new("GeometryNodeCaptureAttribute") + capture_source.domain = "POINT" + capture_source.capture_items.clear() + capture_source.capture_items.new("BOOLEAN", "Source") + capture_source.inputs["Source"].default_value = True + capture_source.location = (750.0, 750.0) + links.new(geometry_input.outputs["Geometry"], capture_source.inputs["Geometry"]) + join_material = nodes.new("GeometryNodeJoinGeometry") + join_material.location = (1000.0, 500.0) + links.new(geometry_switch.outputs["Output"], join_material.inputs["Geometry"]) + links.new(capture_source.outputs["Geometry"], join_material.inputs["Geometry"]) + set_material_index = nodes.new("GeometryNodeSetMaterialIndex") + set_material_index.inputs["Material Index"].default_value = 0 + set_material_index.location = (1250.0, 500.0) + links.new(join_material.outputs["Geometry"], set_material_index.inputs["Geometry"]) + delete_source = nodes.new("GeometryNodeDeleteGeometry") + delete_source.domain = "POINT" + delete_source.location = (1500.0, 500.0) + links.new(set_material_index.outputs["Geometry"], delete_source.inputs["Geometry"]) + links.new(capture_source.outputs["Source"], delete_source.inputs["Selection"]) + links.new(delete_source.outputs["Geometry"], output.inputs["Geometry"]) group.color_tag = "GEOMETRY" group.description = ( diff --git a/tools/node_assets/validate.py b/tools/node_assets/validate.py index f8f01c1..fdffe2a 100644 --- a/tools/node_assets/validate.py +++ b/tools/node_assets/validate.py @@ -10,7 +10,7 @@ sys.path.insert( str(next((PROJECT_ROOT / "src" / "anyimage" / "wheels").glob("blendjob-*.whl"))), ) -from anyimage.common.object import ( +from anyimage.common.material import ( DEPTH_MATERIAL_NODE_GROUP_NAME, IMAGE_MATERIAL_NODE_GROUP_NAME, ) @@ -62,7 +62,6 @@ def main(): "Geometry", "Subdivide", "Thickness", - "Material", ] assert depth_plane.bl_idname == "GeometryNodeTree" assert depth_plane.is_modifier @@ -70,12 +69,30 @@ def main(): "Geometry", "Subdivide", "Thickness", - "Depth Amount", - "Reference Depth", + "Depth Scale", + "Base Plane Depth", "Uniform Scale", "Depth Image", - "Material", ] + depth_panels = [ + item for item in depth_plane.interface.items_tree if item.item_type == "PANEL" + ] + assert [panel.name for panel in depth_panels] == ["Options", "Data"] + assert [item.name for item in depth_panels[0].interface_items] == [ + "Base Plane Depth" + ] + assert [item.name for item in depth_panels[1].interface_items] == [ + "Uniform Scale", + "Depth Image", + ] + assert all(panel.default_closed for panel in depth_panels) + assert all(item.hide_in_modifier for item in depth_panels[1].interface_items) + depth_subdivide = next( + item + for item in depth_plane.interface.items_tree + if item.item_type == "SOCKET" and item.name == "Subdivide" + ) + assert depth_subdivide.default_value == 6 assert image_layer.bl_idname == "ShaderNodeTree" assert input_names(image_layer) == [ "Color", diff --git a/tools/node_assets/validate_mesh_plane.py b/tools/node_assets/validate_mesh_plane.py index 72e3014..c29c84b 100644 --- a/tools/node_assets/validate_mesh_plane.py +++ b/tools/node_assets/validate_mesh_plane.py @@ -27,21 +27,15 @@ def evaluated_mesh(obj): return evaluated, evaluated.to_mesh() -def modifier_input_value(modifier, identifier): - inputs = getattr(getattr(modifier, "properties", None), "inputs", None) - if inputs is not None: - return getattr(inputs, identifier).value - return modifier[identifier] - - def main(): bpy.ops.wm.read_factory_settings(use_empty=True) mesh = create_mesh_plane_mesh("Mesh Plane", (-1.0, 1.0, -0.5, 0.5)) assert len(mesh.polygons) == 1 material = bpy.data.materials.new("Mesh Plane Material") + mesh.materials.append(material) obj = bpy.data.objects.new("Mesh Plane", mesh) bpy.context.scene.collection.objects.link(obj) - modifier = add_mesh_plane_processing(obj, material) + modifier = add_mesh_plane_processing(obj) group = modifier.node_group assert group.name == "O Mesh Plane" assert group.is_modifier @@ -50,15 +44,15 @@ def main(): for item in group.interface.items_tree if item.item_type == "SOCKET" and item.in_out == "INPUT" } - assert list(inputs) == ["Geometry", "Subdivide", "Thickness", "Material"] + assert list(inputs) == ["Geometry", "Subdivide", "Thickness"] assert inputs["Subdivide"].default_value == 0 assert inputs["Subdivide"].max_value == 10 assert inputs["Thickness"].default_value == 0.0 - assert inputs["Material"].hide_in_modifier - assert modifier_input_value(modifier, inputs["Material"].identifier) == material + assert obj.data.materials[0] == material evaluated, result = evaluated_mesh(obj) try: + assert result.materials[0].original == material assert len(result.polygons) == 2 assert len(result.vertices) == 6 assert "UVMap" in result.attributes @@ -75,6 +69,7 @@ def main(): (1.0, 1.0), } assert all(abs(vertex.co.z) < 1e-6 for vertex in result.vertices) + assert all(polygon.material_index == 0 for polygon in result.polygons) finally: evaluated.to_mesh_clear() @@ -92,6 +87,7 @@ def main(): obj.update_tag(refresh={"DATA"}) evaluated, result = evaluated_mesh(obj) try: + assert result.materials[0].original == material assert len(result.polygons) == 184 z_values = [vertex.co.z for vertex in result.vertices] assert abs(min(z_values) + 0.6) < 1e-6 @@ -102,6 +98,17 @@ def main(): for edge_key in polygon.edge_keys: edge_users[edge_key] += 1 assert all(users == 2 for users in edge_users.values()) + assert all(polygon.material_index == 0 for polygon in result.polygons) + finally: + evaluated.to_mesh_clear() + + set_modifier_input(modifier, modifier_input_identifier(group, "Subdivide"), 0) + set_modifier_input(modifier, modifier_input_identifier(group, "Thickness"), 0.01) + obj.update_tag(refresh={"DATA"}) + evaluated, result = evaluated_mesh(obj) + try: + z_values = {round(vertex.co.z, 6) for vertex in result.vertices} + assert len(z_values) >= 4 finally: evaluated.to_mesh_clear()