Add image selection refinement functionality with BEN2 integration
- Implemented viewport.py for handling 3D view projections, gestures, and overlay drawing. - Created refine_image_selection.py to manage the refinement of gesture masks and output image bounds. - Added selection module with refine_selection.py to process and crop images based on alpha channel visibility. - Introduced __init__.py for selection module to expose refine_selection function.
This commit is contained in:
@@ -25,7 +25,7 @@ AnyImage 的扩展偏好设置按功能分组。Environment 区包含：
|
||||
Environment 检测后显示 Diagnostics。Depth & Normal、Remove Background、Upscale 与 Cutout 是默认展开的可折叠功能区，各自包含对应设置和模型状态。其中：
|
||||
|
||||
- **Default Upscale Model**：Upscale 使用的模型，默认是 General WDN x4v3。
|
||||
- **Maximum AI Input Size**：Upscale、Depth、Normal Map 与 Fit to Foreground 的 AI 输入长边上限，默认 2048 px；颜色贴图保留原始分辨率。
|
||||
- **Maximum AI Input Size**：Upscale、Depth、Normal Map 与 Refine Selection 的 AI 输入长边上限，默认 2048 px；颜色贴图保留原始分辨率。
|
||||
- **Maximum Cutout Mesh Resolution**：所有 Cutout Shape 的网格采样长边上限，默认 1024，独立于 AI 输入尺寸。
|
||||
|
||||
Depth & Normal 提供三档 MoGe-2，Depth Plane 与 Cutout Depth / Normal Map 共用当前选择。可选模型的 **Ready** 状态或 **Download** 按钮位于选择项下方；Cutout 区提供独立网格上限与 Balloon Profile。
|
||||
|
||||
@@ -42,7 +42,7 @@ flowchart TD
|
||||
|
||||
## 同步与异步路径
|
||||
|
||||
无需 AI 或媒体处理的功能直接在 Blender 中完成，例如 Plane、普通 Image Mask、Perspective，以及未启用 Fit to Foreground / Generate Normal Map 的 Surface 和 Balloon。需要模型或独立文件处理时，功能 Operator 派生自 `JobOperatorBase`：
|
||||
无需 AI 或媒体处理的功能直接在 Blender 中完成，例如 Plane、普通 Image Mask、Perspective，以及未启用 Refine Selection / Normal Map 的 Surface 和 Balloon。需要模型或独立文件处理时，功能 Operator 派生自 `JobOperatorBase`：
|
||||
|
||||
```mermaid
|
||||
sequenceDiagram
|
||||
@@ -61,6 +61,8 @@ sequenceDiagram
|
||||
|
||||
`server/` 保持纯 Python 实现；Operator 在 Blender 主线程读取产物并创建 RNA 数据。
|
||||
|
||||
几何模型的原始输出先由 `server/models/` 适配为单帧 `GeometryFrame`。它统一保存 Depth、Validity、Intrinsics，以及可选的 Normal 和 Points；Depth、Normal 与 Metadata Writer 只消费该结构。MoGe-2 的 Mask 转为 `0/1` Validity，DA3 的 Confidence 作为连续 Validity。
|
||||
|
||||
## 共享能力
|
||||
|
||||
`common/` 提供 Blender 侧复用的 Image 数据、Region、Depth、Object 和 AI Job 原子能力。公开函数见[Common 公共函数](common.md)。
|
||||
`common/` 按 AI、Image、Selection、Viewport、Depth、Material、Object 和 Node 八个业务模块提供 Blender 侧复用能力。公开函数见[Common 公共函数](common.md)。
|
||||
|
||||
@@ -1,105 +1,107 @@
|
||||
# Common 公共函数
|
||||
|
||||
`src/anyimage/common/` 提供多个功能复用的 Blender 侧原子能力。
|
||||
`src/anyimage/common/` 按 AI、Image、Selection、Viewport、Depth、Material、Object 和 Node 八个业务模块提供 Blender 侧复用能力。调用方直接从对应模块导入函数。
|
||||
|
||||
## Image 数据块
|
||||
|
||||
`image_data.py` 处理 Blender `Image` 数据块及其在 Image Empty 上的使用方式，包括像素读取、动画帧、Job 输入、结果 Image 创建和 Empty 引用替换。
|
||||
`image.py` 处理 Blender `Image` 数据块及其在 Image Empty 上的完整生命周期，包括像素读取、动画帧、Job 输入、结果 Image 创建和 Empty 引用替换。
|
||||
|
||||
| 函数 | 职责 |
|
||||
| --- | --- |
|
||||
| `image_data.is_image_empty()` | 判断对象是否为带有效 Image 数据的 Empty。 |
|
||||
| `image_data.require_image_empty()` | 按名称取得 Image Empty；对象已失效时抛错。 |
|
||||
| `image_data.image_pixels()` | 以 Blender 原始顺序读取一维 float32 像素。 |
|
||||
| `image_data.image_rgba()` | 读取为 Top-down RGBA float32 数组。 |
|
||||
| `image_data.video_total_frames()` | 返回 Movie 或 Sequence 总帧数，静态图返回 1。 |
|
||||
| `image_data.image_user_settings()` | 读取 Image Empty 的帧起点、偏移、时长和循环设置。 |
|
||||
| `image_data.default_extract_frames()` | 取得当前 Empty 默认应处理的帧数。 |
|
||||
| `image_data.source_start_frame()` | 从源 Image Empty 取得 Job 应处理的起始帧。 |
|
||||
| `image_data.prepare_image_input()` | 将 Packed、Dirty、Sequence 等输入准备为 Job 可读路径，并返回临时标记。 |
|
||||
| `image_data.prepare_region_input()` | 将 `RegionMask` 裁切为带 Mask Alpha 的临时 RGBA Job 输入。 |
|
||||
| `image_data.cleanup_image_input()` | 按临时标记清理由 `prepare_image_input()` 创建的文件或目录。 |
|
||||
| `image_data.image_base_name()` | 去除图片或视频扩展名和 Blender 数字后缀，得到数据块基础名。 |
|
||||
| `image_data.depth_data_name()` | 生成 Depth 结果数据块文件名。 |
|
||||
| `image_data.color_data_name()` | 生成 Color 结果数据块文件名。 |
|
||||
| `image_data.load_color_result_image()` | 加载、命名、Pack 并配置 Job 的 Color 或 Sequence 结果；不修改 Empty。 |
|
||||
| `image_data.load_normal_result_image()` | 加载、命名、设为 Non-Color 并 Pack Normal Map。 |
|
||||
| `image_data.create_pixel_result_image()` | 从本地编辑后的 RGBA 像素创建 Packed Image；不修改 Empty。 |
|
||||
| `image_data.replace_empty_image()` | 执行 Image Empty 结果替换、共享引用分流、旧数据清理、帧设置继承和可选画布对齐。 |
|
||||
| `image_data.image_empty_bounds()` | 计算 Image Empty 在对象局部坐标中的实际显示边界。 |
|
||||
| `image.is_image_empty()` | 判断对象是否为带有效 Image 数据的 Empty。 |
|
||||
| `image.require_image_empty()` | 按名称取得 Image Empty；对象已失效时抛错。 |
|
||||
| `image.image_pixels()` | 以 Blender 原始顺序读取一维 float32 像素。 |
|
||||
| `image.image_rgba()` | 读取为 Top-down RGBA float32 数组。 |
|
||||
| `image.video_total_frames()` | 返回 Movie 或 Sequence 总帧数，静态图返回 1。 |
|
||||
| `image.image_user_settings()` | 读取 Image Empty 的帧起点、偏移、时长和循环设置。 |
|
||||
| `image.default_extract_frames()` | 取得当前 Empty 默认应处理的帧数。 |
|
||||
| `image.source_start_frame()` | 从源 Image Empty 取得 Job 应处理的起始帧。 |
|
||||
| `image.prepare_image_input()` | 将 Packed、Dirty、Sequence 等输入准备为 Job 可读路径，并返回临时标记。 |
|
||||
| `image.prepare_selection_input()` | 将 `SelectionMask` 裁切为带 Mask Alpha 的临时 RGBA Job 输入。 |
|
||||
| `image.cleanup_image_input()` | 按临时标记清理由 `prepare_image_input()` 创建的文件或目录。 |
|
||||
| `image.image_base_name()` | 去除图片或视频扩展名和 Blender 数字后缀，得到数据块基础名。 |
|
||||
| `image.depth_data_name()` | 生成 Depth 结果数据块文件名。 |
|
||||
| `image.color_data_name()` | 生成 Color 结果数据块文件名。 |
|
||||
| `image.load_color_result_image()` | 加载、命名、Pack 并配置 Job 的 Color 或 Sequence 结果；不修改 Empty。 |
|
||||
| `image.load_normal_result_image()` | 加载、命名、设为 Non-Color 并 Pack Normal Map。 |
|
||||
| `image.create_pixel_result_image()` | 从本地编辑后的 RGBA 像素创建 Packed Image；不修改 Empty。 |
|
||||
| `image.replace_empty_image()` | 执行 Image Empty 结果替换、共享引用分流、旧数据清理、帧设置继承和可选画布对齐。 |
|
||||
| `image.image_empty_bounds()` | 计算 Image Empty 在对象局部坐标中的实际显示边界。 |
|
||||
|
||||
## Image Region 与视图交互
|
||||
## Image Selection 与视图交互
|
||||
|
||||
这组公共能力使用固定术语：
|
||||
|
||||
- Region 是用户在图片空间圈定的业务范围；`RegionPath` 保存路径点、反选状态和填充规则。
|
||||
- `RegionMask` 是 Region 的栅格化结果，只包含局部像素权重 `values` 和源图 `bounds`。
|
||||
- Selection 是用户在图片空间圈定的业务范围；`SelectionPath` 保存路径点、反选状态和填充规则。
|
||||
- `SelectionMask` 是 Selection 的栅格化结果，只包含局部像素权重 `values` 和源图 `bounds`。
|
||||
- Mask 表示二维像素权重或 Alpha 运算。
|
||||
- Selection 表示 Blender 对象选择；图片业务范围称为 Region，细化动作称为 Fit to Foreground。
|
||||
- Blender 对象 Selection 由上下文明确区分；图片选区细化动作称为 Refine Selection。
|
||||
|
||||
`image_interaction.py` 处理 3D Viewport 中的点击、坐标投影、`ImageRegionGesture`、工具设置和预览绘制；`image_region.py` 负责图片空间 Region 的值、栅格化和 Alpha 裁切。前者负责用户如何圈定 Region，后者负责 Region 如何变成 Mask。
|
||||
`viewport.py` 处理 View3D 中的点击、坐标投影、`ImageSelectionGesture`、工具设置和 Overlay 绘制；`selection.py` 负责图片空间 Selection 的值、栅格化和 Alpha 裁切。
|
||||
|
||||
### Viewport 交互
|
||||
|
||||
| 函数 | 职责 |
|
||||
| --- | --- |
|
||||
| `image_interaction.report_image_edit_exception()` | 将可预期的图片编辑警告和异常映射为 Operator Report。 |
|
||||
| `image_interaction.image_edit_poll()` | 判断当前上下文是否允许启动 Image 编辑工具。 |
|
||||
| `image_interaction.resolve_image_edit_click()` | 解析点击到 Image Empty，并保持或恢复选择状态。 |
|
||||
| `image_interaction.active_view3d_tool_id()` | 取得当前 View3D Workspace Tool ID。 |
|
||||
| `image_interaction.apply_region_settings()` | 将当前 Region 工具的 Fit to Foreground 设置同步到手势 Operator。 |
|
||||
| `image_interaction.draw_region_settings()` | 绘制 Region 工具的公共 Fit to Foreground 设置。 |
|
||||
| `image_interaction.draw_dashed_line()` | 绘制手势或透视框虚线。 |
|
||||
| `image_interaction.draw_polygon_fill()` | 绘制手势或透视框填充预览。 |
|
||||
| `image_interaction.gesture_property()` | 声明统一的 Box、Lasso、Line 和 Polyline RNA 属性。 |
|
||||
| `image_interaction.screen_path_to_image_pixels()` | 将屏幕路径投影为 Image Empty 像素坐标。 |
|
||||
| `image_interaction.serialize_matrix()` | 将 4×4 Matrix 序列化为 Operator 可保存的字符串。 |
|
||||
| `image_interaction.deserialize_matrix()` | 恢复已序列化的 4×4 Matrix。 |
|
||||
| `image_interaction.drawing_in_region()` | 判断 Modal 绘制是否仍位于启动时的 Area 和 Region。 |
|
||||
| `viewport.report_image_edit_exception()` | 将可预期的 Selection 警告和异常映射为 Operator Report。 |
|
||||
| `viewport.image_edit_poll()` | 判断当前上下文是否允许启动 Image 工具。 |
|
||||
| `viewport.resolve_image_edit_click()` | 解析点击到 Image Empty，并保持或恢复选择状态。 |
|
||||
| `viewport.active_view3d_tool_id()` | 取得当前 View3D Workspace Tool ID。 |
|
||||
| `viewport.apply_selection_settings()` | 将当前 Selection 工具的 Refine Selection 设置同步到手势 Operator。 |
|
||||
| `viewport.draw_selection_settings()` | 绘制 Selection 工具的公共 Refine Selection 设置。 |
|
||||
| `viewport.draw_dashed_line()` | 绘制手势或透视框虚线。 |
|
||||
| `viewport.draw_polygon_fill()` | 绘制手势或透视框填充预览。 |
|
||||
| `viewport.gesture_property()` | 声明统一的 Box、Lasso、Line 和 Polyline RNA 属性。 |
|
||||
| `viewport.screen_path_to_image_pixels()` | 将屏幕路径投影为 Image Empty 像素坐标。 |
|
||||
| `viewport.serialize_matrix()` | 将 4×4 Matrix 序列化为 Operator 可保存的字符串。 |
|
||||
| `viewport.deserialize_matrix()` | 恢复已序列化的 4×4 Matrix。 |
|
||||
| `viewport.drawing_in_region()` | 判断 Modal 绘制是否仍位于启动时的 Area 和 Region。 |
|
||||
|
||||
### Region 栅格化
|
||||
### Selection 栅格化
|
||||
|
||||
| 函数 | 职责 |
|
||||
| --- | --- |
|
||||
| `image_region.region_path_bounds()` | 校验 Region Path 并返回整数像素包围框。 |
|
||||
| `image_region.rasterize_region_path()` | 把 `RegionPath` 栅格化为 `RegionMask`，并可与源 Alpha 相交。 |
|
||||
| `image_region.crop_pixels_to_alpha()` | 按 Alpha 非零区域紧裁切 RGBA 像素。 |
|
||||
| `selection.selection_path_bounds()` | 校验 Selection Path 并返回整数像素包围框。 |
|
||||
| `selection.rasterize_selection_path()` | 把 `SelectionPath` 栅格化为 `SelectionMask`，并可与源 Alpha 相交。 |
|
||||
| `selection.crop_pixels_to_alpha()` | 按 Alpha 非零区域紧裁切 RGBA 像素。 |
|
||||
|
||||
## Depth 产物
|
||||
|
||||
`depth_data.py` 负责 Blender 侧共享的 Depth 文件加载和相机尺度换算，直接加载 Server 写出的 EXR。
|
||||
`depth.py` 负责 Blender 侧共享的 Depth 文件加载和相机尺度换算，直接加载 Server 写出的 EXR。
|
||||
|
||||
| 函数 | 职责 |
|
||||
| --- | --- |
|
||||
| `depth_data.load_depth_metadata()` | 读取并校验 `depth.json` 中的图片尺寸、相机内参与参考深度。 |
|
||||
| `depth_data.load_depth_result_image()` | 直接加载、命名、设为 Non-Color 并 Pack `vector-depth.exr` 或 `z-depth.exr`。 |
|
||||
| `depth_data.camera_depth_values()` | 从已加载 Depth Image 读取用于本地标定的 Z 与有效区。 |
|
||||
| `depth_data.depth_uniform_scale()` | 根据相机内参、参考深度和 Image Empty 范围计算统一尺度。 |
|
||||
| `depth.load_depth_metadata()` | 读取并校验 `depth.json` 中的图片尺寸、相机内参与参考深度。 |
|
||||
| `depth.load_depth_result_image()` | 直接加载、命名、设为 Non-Color 并 Pack `vector-depth.exr` 或 `z-depth.exr`。 |
|
||||
| `depth.camera_depth_values()` | 从已加载 Depth Image 读取用于本地标定的 Z 与有效区。 |
|
||||
| `depth.depth_uniform_scale()` | 根据相机内参、参考深度和 Image Empty 范围计算统一尺度。 |
|
||||
|
||||
## Object 构建
|
||||
|
||||
`object.py` 集中 Convert to Plane 与 Cutout 共用的材质、Modifier 和结果 Object 构建能力；`node.py` 负责加载节点资产。
|
||||
`material.py` 构建 Image 材质；`object.py` 处理 Object 与 Geometry Nodes Modifier；`node.py` 加载节点资产。
|
||||
|
||||
| 函数 | 职责 |
|
||||
| --- | --- |
|
||||
| `node.load_node_group()` | 从随扩展分发的 Blend 资产加载指定 Node Group。 |
|
||||
| `object.create_image_material()` | 从 Image Empty 创建包含颜色、位移、Normal Map、Alpha 和动画设置的完整材质。 |
|
||||
| `material.create_image_material()` | 从 Image Empty 创建包含颜色、位移、Normal Map、Alpha 和动画设置的完整材质。 |
|
||||
| `object.modifier_input_identifier()` | 按接口名称解析 Geometry Nodes Modifier 输入 Identifier。 |
|
||||
| `object.set_modifier_input()` | 跨受支持 Blender RNA 形式设置 Modifier 输入。 |
|
||||
| `object.finalize_object_result()` | 激活结果对象，并按需替换或保留源 Empty。 |
|
||||
|
||||
|
||||
## AI 任务准备
|
||||
## AI 准备
|
||||
|
||||
`ai_job.py` 处理多个 AI 功能共同使用的任务准备，包括输入路径校验、Environment 与模型检查、Device 选择和模型请求参数组装。
|
||||
`ai.py` 统一处理 AI 功能的 Environment、模型、Device、输入校验、Job 参数和公共 UI。
|
||||
|
||||
| 函数 | 职责 |
|
||||
| --- | --- |
|
||||
| `ai_job.require_input_path()` | 解析 Blender 路径并要求其为现存文件或目录。 |
|
||||
| `ai_job.require_background_input_path()` | 在通用路径校验之外限制 BEN2 支持的媒体类型。 |
|
||||
| `ai_job.production_device()` | 读取正式任务使用的计算设备。 |
|
||||
| `ai_job.require_environment()` | 要求 AnyImage AI Environment 已就绪。 |
|
||||
| `ai_job.invoke_ai_setup_if_needed()` | Environment 未就绪时启动安装交互。 |
|
||||
| `ai_job.require_model()` | 校验模型标识和下载状态，并返回模型记录。 |
|
||||
| `ai_job.da3_parameters()` | 组装 `debug-da3` 参数。 |
|
||||
| `ai_job.moge2_parameters()` | 组装 MoGe-2 正式任务与 Debug 共用参数。 |
|
||||
| `ai.draw_ai_setup()` | 绘制 AI 未就绪时的安装入口，并返回当前状态。 |
|
||||
| `ai.draw_ai_property()` | 按 AI 就绪状态绘制并启用业务属性。 |
|
||||
| `ai.require_input_path()` | 解析 Blender 路径并要求其为现存文件或目录。 |
|
||||
| `ai.require_background_input_path()` | 在通用路径校验之外限制 BEN2 支持的媒体类型。 |
|
||||
| `ai.production_device()` | 读取正式任务使用的计算设备。 |
|
||||
| `ai.require_environment()` | 要求 AnyImage AI Environment 已就绪。 |
|
||||
| `ai.invoke_ai_setup_if_needed()` | Environment 未就绪时启动安装交互。 |
|
||||
| `ai.require_model()` | 校验模型标识和下载状态，并返回模型记录。 |
|
||||
| `ai.da3_parameters()` | 组装 `debug-da3` 参数。 |
|
||||
| `ai.moge2_parameters()` | 组装 MoGe-2 正式任务与 Debug 共用参数。 |
|
||||
|
||||
@@ -33,7 +33,7 @@ Movie 或 Sequence 的 Image User 设置会保留。转换完成后，新 Plane
|
||||
|
||||
Depth 路径中的 `GenerateDepthPlane` 位于 `conversion.py`。它从 Preferences 读取 **Depth & Normal Model**、**MoGe Resolution Level** 和 **Maximum AI Input Size**，提交 `generate-depth-plane-geometry` Job。
|
||||
|
||||
`server/jobs/depth_plane.py` 编排任务，`server/geometry/prediction_artifacts.py` 调用 MoGe-2，写出 `z-depth.exr` 与 `depth.json`。EXR 的 RGB 重复保存相机 Z，Alpha 保存有效区；JSON 保存图片尺寸、相机内参与参考深度。AI 输入上限只影响模型分析图，颜色副本仍保留原始分辨率。
|
||||
`server/jobs/depth_plane.py` 编排任务，`server/geometry/prediction_artifacts.py` 把图片合成到黑底后调用 MoGe-2，并从 `GeometryFrame` 写出 `z-depth.exr` 与 `depth.json`。EXR 的 RGB 重复保存完整相机 Z，Alpha 保存模型 Validity；JSON 保存图片尺寸、相机内参与有效区域的参考深度。AI 输入上限只影响模型分析图，颜色副本仍保留原始分辨率。
|
||||
|
||||
### 创建 Depth Plane
|
||||
|
||||
|
||||
@@ -1,16 +1,16 @@
|
||||
# Cutout Tool
|
||||
|
||||
Cutout 从图片上的一个 Region 开始，可按需执行 Fit to Foreground、生成 Depth 或 Normal Map，最后在 Blender 中创建带材质与 Modifier 的 Mesh Object。
|
||||
Cutout 从图片上的一个 Selection 开始，可按需执行 Refine Selection、生成 Depth 或 Normal Map，最后在 Blender 中创建带材质与 Modifier 的 Mesh Object。
|
||||
|
||||
## 执行过程
|
||||
|
||||
```mermaid
|
||||
flowchart TD
|
||||
Select["圈定 Region，选择 Shape"]
|
||||
Raster["栅格化 Region"]
|
||||
Select["圈定 Selection，选择 Shape"]
|
||||
Raster["栅格化 Selection"]
|
||||
Route{"需要后台生成数据？"}
|
||||
Local["沿用本地 RegionMask"]
|
||||
Job["执行 Fit to Foreground，生成 Depth / Normal 产物"]
|
||||
Local["沿用本地 SelectionMask"]
|
||||
Job["执行 Refine Selection，生成 Depth / Normal 产物"]
|
||||
Ready["汇总 Shape 输入"]
|
||||
Build["构建 BaseShape"]
|
||||
Create["创建 Blender 对象"]
|
||||
@@ -21,19 +21,19 @@ flowchart TD
|
||||
Ready --> Build --> Create
|
||||
```
|
||||
|
||||
Surface 与 Balloon 直接使用 `RegionMask` 构建 `BaseShape`；Fit to Foreground、Depth Shape 和 Normal Map 先通过 Server Job 生成所需产物。
|
||||
Surface 与 Balloon 直接使用 `SelectionMask` 构建 `BaseShape`；Refine Selection、Depth Shape 和 Normal Map 先通过 Server Job 生成所需产物。
|
||||
|
||||
## 节点说明
|
||||
|
||||
### 圈定 Region，选择 Shape
|
||||
### 圈定 Selection，选择 Shape
|
||||
|
||||
Cutout 工具由 `__init__.py` 注册，交互从 `main.py` 中的 `SelectCutoutRegion` 开始。它接收 Lasso 或双击整图手势，把图片空间路径保存为 `RegionPath`，然后调用 `interaction.py` 打开 Shape 饼菜单。
|
||||
Cutout 工具由 `__init__.py` 注册，交互从 `main.py` 中的 `SelectCutoutSelection` 开始。它接收 Lasso 或双击整图手势，把图片空间路径保存为 `SelectionPath`，然后调用 `interaction.py` 打开 Shape 饼菜单。
|
||||
|
||||
`interaction.py` 完成 Region 有效性检查、读取三个工具选项，并根据 **Mesh Sampling Spacing** 换算期望网格分辨率。菜单中的 Surface、Balloon、Depth Balloon 和 Depth Surface，以及它们对应的节点分支索引，统一来自 `shape.py`。
|
||||
`interaction.py` 完成 Selection 有效性检查并读取三个工具选项。**Mesh Detail** 的 Low、Medium、High、Ultra 分别映射到 32、16、8、4 px 的采样间距，再换算期望网格分辨率；档位越高，采样越密。菜单中的 Surface、Balloon、Depth Balloon 和 Depth Surface，以及它们对应的节点分支索引，统一来自 `shape.py`。
|
||||
|
||||
### 栅格化 Region
|
||||
### 栅格化 Selection
|
||||
|
||||
用户选定 Shape 后，`main.py` 中的 `CutoutRegionToShape` 接过主线。它使用 `common/image_region.py` 把 `RegionPath` 栅格化为 `RegionMask`：`values` 保存局部像素权重，`bounds` 保存它在源图中的位置。套索外和源图透明区域不会进入最终 Region。
|
||||
用户选定 Shape 后，`main.py` 中的 `CutoutSelectionToShape` 接过主线。它使用 `common/selection.py` 把 `SelectionPath` 栅格化为 `SelectionMask`：`values` 保存局部像素权重，`bounds` 保存它在源图中的位置。套索外和源图透明区域不会进入最终 Selection。
|
||||
|
||||
此时也会应用 **Maximum Cutout Mesh Resolution**，统一限制四种 Shape 裁切区域的最长采样边。
|
||||
|
||||
@@ -41,30 +41,30 @@ Cutout 工具由 `__init__.py` 注册，交互从 `main.py` 中的 `SelectCutout
|
||||
|
||||
只有以下情况需要提交 Job：
|
||||
|
||||
- 开启 **Fit to Foreground**；
|
||||
- 开启 **Refine Selection**；
|
||||
- 选择 Depth Balloon 或 Depth Surface；
|
||||
- 开启 **Generate Normal Map**。
|
||||
- 开启 **Normal Map**。
|
||||
|
||||
Surface 和 Balloon 的本地路径直接沿用 `RegionMask`。
|
||||
Surface 和 Balloon 的本地路径直接沿用 `SelectionMask`。
|
||||
|
||||
### 执行 Fit to Foreground，生成 Depth / Normal 产物
|
||||
### 执行 Refine Selection，生成 Depth / Normal 产物
|
||||
|
||||
`main.py` 把请求提交给 `server/jobs/cutout.py`，生成本次 Shape 需要的产物：
|
||||
|
||||
- **Fit to Foreground** 调用 `server/region/refine_region.py`，写出收紧后的 `region.png` 和 `region_bounds.json`；
|
||||
- **Refine Selection** 调用 `server/selection/refine_selection.py`，写出收紧后的 `selection.png` 和 `selection_bounds.json`；
|
||||
- 需要 Vector Depth 时写出相机空间 XYZ 组成的 `vector-depth.exr`，需要 Z Depth 时写出 `z-depth.exr`，两者都配套 `depth.json`；
|
||||
- **Generate Normal Map** 根据既定空间写出 `tangent-normal.png` 或 `object-normal.png`。
|
||||
- **Normal Map** 根据既定空间写出 `tangent-normal.png` 或 `object-normal.png`。
|
||||
|
||||
Depth 与 Normal Map 共享同一次 MoGe-2 Prediction。Depth Surface 的 Normal Map 编码为 Cutout 对象空间，其正面是对象局部 `-X`；其余 Shape 使用 Tangent Space。`main.normal_mode_for_shape()` 根据 Shape 确定 Normal Space。
|
||||
Depth 与 Normal Map 共享同一个 MoGe-2 `GeometryFrame`。模型输入先按 Alpha 合成到黑底；Vector/Z Depth 的 RGB 保留完整连续 Field，Alpha 保存模型 Validity。Depth Surface 的 Normal Map 编码为 Cutout 对象空间，其正面是对象局部 `-X`；其余 Shape 使用 Tangent Space。`main.normal_mode_for_shape()` 根据 Shape 确定 Normal Space。
|
||||
|
||||
**Maximum AI Input Size** 约束 BEN2 和 MoGe-2 的分析图长边；网格密度使用 **Maximum Cutout Mesh Resolution**，最终颜色图使用源图裁切分辨率。
|
||||
|
||||
### 构建 BaseShape
|
||||
|
||||
Job 完成后，`main.py` 在 Blender 主线程读取可选的 RegionMask，加载并 Pack Depth 与 Normal 图片。`main.create_cutout_shape()` 调用 `geometry.py` 中的 `build_base_shape()`：
|
||||
Job 完成后，`main.py` 在 Blender 主线程读取可选的 SelectionMask，加载并 Pack Depth 与 Normal 图片。`main.create_cutout_shape()` 调用 `geometry.py` 中的 `build_base_shape()`：
|
||||
|
||||
1. 从最终 `RegionMask` 提取外轮廓和孔洞；
|
||||
2. 按 Mesh Sampling Spacing 生成轮廓与内部采样点；
|
||||
1. 从最终 `SelectionMask` 提取外轮廓和孔洞；
|
||||
2. 按 Mesh Detail 对应的采样间距生成轮廓与内部采样点；
|
||||
3. 进行约束三角化；
|
||||
4. 计算四种 Shape 共用的 `o_balloon`；
|
||||
5. 汇总顶点、三角面、UV 和 Balloon 权重，得到内存中的 `BaseShape`。
|
||||
@@ -73,7 +73,7 @@ Job 完成后，`main.py` 在 Blender 主线程读取可选的 RegionMask，加
|
||||
|
||||
### 创建 Blender 对象
|
||||
|
||||
`main.create_cutout_shape()` 把 `BaseShape` 和可选产物交给 `object.py` 的 `create_shape_object()`。这里创建裁切后的 Color Image、Material、Mesh Object，并挂上统一的 `O Image Cutout` Modifier。材质通过 `common/object.py` 创建，Cutout 使用 `1.2` 的 IOR。Color Image 按最终 Region 包围矩形裁切源图并保留 RGBA。
|
||||
`main.create_cutout_shape()` 把 `BaseShape` 和可选产物交给 `object.py` 的 `create_shape_object()`。这里创建裁切后的 Color Image、Material、Mesh Object，并挂上统一的 `O Image Cutout` Modifier。材质通过 `common/material.py` 创建，Cutout 使用 `1.2` 的 IOR。Color Image 按最终 Selection 包围矩形裁切源图并保留 RGBA。
|
||||
|
||||
| Shape | Modifier 如何形成结果 | 必需后台产物 |
|
||||
| --- | --- | --- |
|
||||
@@ -82,7 +82,7 @@ Job 完成后，`main.py` 在 Blender 主线程读取可选的 RegionMask，加
|
||||
| Depth Balloon | 在 Balloon 上叠加相对基准面的深度 | `z-depth.exr`、`depth.json` |
|
||||
| Depth Surface | 使用相机点 XYZ 重建透视表面 | `vector-depth.exr`、`depth.json` |
|
||||
|
||||
Depth Shape 的 Base Plane Depth，以及 Depth Balloon 的 Direction，都由 `object.py` 调用 `geometry.py` 的标定函数，根据最终 Region Alpha 计算。透明像素达到 `5%` 时使用有效 Region 自适应标定，否则使用稳定的固定基准。
|
||||
Depth Shape 的 Base Plane Depth，以及 Depth Balloon 的 Direction，都由 `object.py` 调用 `geometry.py` 的标定函数，根据最终 Selection Alpha 计算。透明像素达到 `5%` 时使用有效 Selection 自适应标定，否则使用稳定的固定基准。
|
||||
|
||||
Normal Map 关闭时材质使用几何法线；开启时，`object.py` 把生成的图片和已经确定的 Normal Space 交给材质节点。最后 `main.py` 提交 Undo，整条 Cutout 流程结束。
|
||||
|
||||
|
||||
@@ -1,22 +1,22 @@
|
||||
# Image Tool
|
||||
|
||||
Image Tool 从 Box、Lasso、Line 或 Polyline 手势得到一个图片 Region，再用它修改当前 Image Empty 的 Alpha。
|
||||
Image Tool 从 Box、Lasso、Line 或 Polyline 手势得到一个图片 Selection，再用它修改当前 Image Empty 的 Alpha。
|
||||
|
||||
## 执行过程
|
||||
|
||||
```mermaid
|
||||
flowchart TD
|
||||
Tool["Box / Lasso / Line / Polyline"] --> Gesture["采集图片空间手势"]
|
||||
Gesture --> Region["RegionPath → RegionMask"]
|
||||
Region --> Refine{"Fit to Foreground？"}
|
||||
Refine -- "否" --> Mask["使用本地 RegionMask"]
|
||||
Refine -- "是" --> Job["refine-image-region"]
|
||||
Gesture --> Selection["SelectionPath → SelectionMask"]
|
||||
Selection --> Refine{"Refine Selection？"}
|
||||
Refine -- "否" --> Mask["使用本地 SelectionMask"]
|
||||
Refine -- "是" --> Job["refine-image-selection"]
|
||||
Job --> Mask
|
||||
Mask --> Alpha["源 Alpha × RegionMask"]
|
||||
Mask --> Alpha["源 Alpha × SelectionMask"]
|
||||
Alpha --> Crop["紧裁切并写回 Empty"]
|
||||
```
|
||||
|
||||
四个工具共用 `EditImageMask`。Box 和 Lasso 可以执行 Fit to Foreground；Line 与 Polyline 始终保留几何边界，直接使用本地 `RegionMask`。
|
||||
四个工具共用 `EditImageMask`。Box 和 Lasso 可以执行 Refine Selection；Line 与 Polyline 始终保留几何边界，直接使用本地 `SelectionMask`。
|
||||
|
||||
## 节点说明
|
||||
|
||||
@@ -24,22 +24,22 @@ flowchart TD
|
||||
|
||||
`operators/image_tool/__init__.py` 汇总注册类型；`box.py`、`lasso.py`、`line.py` 和 `polyline.py` 分别定义四个 WorkspaceTool。它们都把操作交给 `edit_mask.py` 中的 `EditImageMask`。
|
||||
|
||||
屏幕坐标投射、点击选择、轨迹预览和状态切换来自 `common/image_interaction.py`。Lasso 按移动距离采点并合并近似共线点；Line 扩展为有向半平面；Polyline 在确认时闭合路径。
|
||||
屏幕坐标投射、点击选择、轨迹预览和状态切换来自 `common/viewport.py`。Lasso 按移动距离采点并合并近似共线点；Line 扩展为有向半平面；Polyline 在确认时闭合路径。
|
||||
|
||||
### RegionPath → RegionMask
|
||||
### SelectionPath → SelectionMask
|
||||
|
||||
`EditImageMask` 通过 `common/image_region.py` 把手势整理为 `RegionPath`，再栅格化为 `RegionMask`。凹多边形使用三角剖分，自交路径使用 `EVEN_ODD` 填充。预览缓存只在轨迹变化时更新。
|
||||
`EditImageMask` 通过 `common/selection.py` 把手势整理为 `SelectionPath`，再栅格化为 `SelectionMask`。凹多边形使用三角剖分，自交路径使用 `EVEN_ODD` 填充。预览缓存只在轨迹变化时更新。
|
||||
|
||||
Box 和 Lasso 分别读取自己的 **Fit to Foreground** 设置。Line 与 Polyline 直接使用几何边界。
|
||||
Box 和 Lasso 分别读取自己的 **Refine Selection** 设置。Line 与 Polyline 直接使用几何边界。
|
||||
|
||||
### Fit to Foreground
|
||||
### Refine Selection
|
||||
|
||||
未开启 Fit to Foreground 时，`edit_mask.py` 直接使用本地 `RegionMask`。开启后，`RefineImageRegion` 把带 Region Alpha 的局部 RGBA 提交给 `server/jobs/refine_image_region.py`；Job 调用 `server/region/refine_region.py` 运行 BEN2，并返回收紧后的图片和 `bounds`。
|
||||
未开启 Refine Selection 时，`edit_mask.py` 直接使用本地 `SelectionMask`。开启后，`RefineImageSelection` 把带 Selection Alpha 的局部 RGBA 提交给 `server/jobs/refine_image_selection.py`；Job 调用 `server/selection/refine_selection.py` 运行 BEN2，并返回收紧后的图片和 `bounds`。
|
||||
|
||||
响应阶段仍由 `edit_mask.py` 把局部结果映射回完整画布。
|
||||
|
||||
### 源 Alpha × RegionMask
|
||||
### 源 Alpha × SelectionMask
|
||||
|
||||
`edit_mask.py` 保留源 RGB，只把最终 `RegionMask` 乘入源 Alpha，再按非零 Alpha 紧裁切。像素读取、结果 Image 创建和共享引用处理使用 `common/image_data.py`。
|
||||
`edit_mask.py` 保留源 RGB，只把最终 `SelectionMask` 乘入源 Alpha，再按非零 Alpha 紧裁切。像素读取、结果 Image 创建和共享引用处理使用 `common/image.py`。
|
||||
|
||||
公共 Image Empty 替换入口同步更新 `empty_display_size` 和 `empty_image_offset`，使裁切后的内容对齐原图片平面并保持对象 `matrix_world`。
|
||||
|
||||
@@ -8,7 +8,7 @@
|
||||
| `operators/ai_setup.py` | [AI 环境管理](ai-setup.md) | 环境安装、模型下载与缓存释放 |
|
||||
| `operators/clipboard_image/` | [Clipboard Image](clipboard-image.md) | 系统剪贴板读取、粘贴与快捷键 |
|
||||
| `operators/convert_to_plane/` | [Convert to Plane](convert-to-plane.md) | Mesh Plane 与 Depth Plane 创建 |
|
||||
| `operators/cutout_tool/` | [Cutout Tool](cutout-tool.md) | Region 选择、基础 Shape 构建与 Blender 对象创建 |
|
||||
| `operators/cutout_tool/` | [Cutout Tool](cutout-tool.md) | Selection、基础 Shape 构建与 Blender 对象创建 |
|
||||
| `operators/debug.py` | [Debug Operator](debug.md) | 模型接口与原始产物验证 |
|
||||
| `operators/image_tool/` | [Image Tool](image-tool.md) | Box、Lasso、Line 与 Polyline 编辑 |
|
||||
| `operators/perspective_tool/` | [Perspective Tool](perspective-tool.md) | 四点透视框与透视校正 |
|
||||
|
||||
@@ -23,7 +23,7 @@ flowchart TD
|
||||
|
||||
`operators/perspective_tool/__init__.py` 导出注册类型，交互和计算集中在 `edit_perspective.py`。`ImagePerspectiveTool` 启动 `EditImagePerspective`，后者接受任意顺序的四个图片角点，再根据源图 X/Y 方向整理成不交错的凸 Quad。
|
||||
|
||||
图片坐标投射和首次点击选择复用 `common/image_interaction.py`。四个角点在该流程中统一表示为 Quad。
|
||||
图片坐标投射和首次点击选择复用 `common/viewport.py`。四个角点在该流程中统一表示为 Quad。
|
||||
|
||||
### 预览并调整比例
|
||||
|
||||
@@ -37,4 +37,4 @@ flowchart TD
|
||||
|
||||
### 裁切并写回
|
||||
|
||||
Warp 结果按可见 Alpha 紧裁切，再通过 `common/image_data.py` 创建 Packed Image 并替换当前 Empty。写回时保持中心和覆盖尺度近似一致。
|
||||
Warp 结果按可见 Alpha 紧裁切，再通过 `common/image.py` 创建 Packed Image 并替换当前 Empty。写回时保持中心和覆盖尺度近似一致。
|
||||
|
||||
@@ -20,7 +20,7 @@ Packed 图片和内存中修改过的单图会先导出临时文件；文件图
|
||||
|
||||
### 准备媒体输入
|
||||
|
||||
入口是 `operators/remove_background.py` 中的 `RemoveImageBackground`。它检查当前对象、AI Environment 和帧数，再调用 `common/image_data.py` 的 `prepare_image_input()`。多帧输入会先写入 Scene 中的抽帧设置。
|
||||
入口是 `operators/remove_background.py` 中的 `RemoveImageBackground`。它检查当前对象、AI Environment 和帧数，再调用 `common/image.py` 的 `prepare_image_input()`。多帧输入会先写入 Scene 中的抽帧设置。
|
||||
|
||||
准备完成后，内部 Operator `RunBackgroundRemoval` 保存输入路径、源对象名和临时文件清理标记。
|
||||
|
||||
@@ -32,6 +32,6 @@ BEN2 推理前先用源 Alpha 屏蔽透明区域，模型输出的前景 Alpha
|
||||
|
||||
### 导入并替换结果
|
||||
|
||||
Job 返回后，`remove_background.py` 的 `apply_background_response()` 调用 `common/image_data.py` 创建单图或序列 Image，并恢复源 Image User 的帧起点、偏移和循环设置。
|
||||
Job 返回后，`remove_background.py` 的 `apply_background_response()` 调用 `common/image.py` 创建单图或序列 Image，并恢复源 Image User 的帧起点、偏移和循环设置。
|
||||
|
||||
`replace_empty_image()` 随后替换 `Empty.data`，提交一个 Remove Background Undo 步骤，并清理本次创建的临时输入。
|
||||
|
||||
@@ -25,7 +25,7 @@ flowchart TD
|
||||
|
||||
入口位于 `operators/upscale.py`。`UpscaleImage.poll()` 读取 `preferences.py` 中的 **Maximum AI Input Size**；当前 Image 长边必须严格小于该值。菜单显示的预计尺寸来自同一文件的 `upscale_output_size()`。
|
||||
|
||||
执行时，`upscale.py` 冻结当前模型和输入上限，并通过 `common/image_data.py` 准备单图、视频或序列输入。Packed Image 与内存修改结果会导出临时文件，多帧输入保留帧起点和抽帧设置。
|
||||
执行时，`upscale.py` 冻结当前模型和输入上限，并通过 `common/image.py` 准备单图、视频或序列输入。Packed Image 与内存修改结果会导出临时文件，多帧输入保留帧起点和抽帧设置。
|
||||
|
||||
### 分块放大 RGB
|
||||
|
||||
@@ -39,6 +39,6 @@ flowchart TD
|
||||
|
||||
### 导入并替换结果
|
||||
|
||||
响应阶段回到 `operators/upscale.py`，通过 `common/image_data.py` 读取 `upscale.*` 结果序列并替换当前 Empty。
|
||||
响应阶段回到 `operators/upscale.py`，通过 `common/image.py` 读取 `upscale.*` 结果序列并替换当前 Empty。
|
||||
|
||||
后续 Upscale 和 Depth Plane 都以当前结果 Image 为输入。
|
||||
|
||||
@@ -8,36 +8,38 @@ Server 在 Job 目录写入图片、JSON 或 Debug NPZ；Blender 根据 Job 返
|
||||
| | | |
|
||||
| `remove-background` | `image.0001.png` 起 | 替换 Image Empty 的颜色数据 |
|
||||
| `upscale-image` | `upscale.0001.png` 起 | 替换或新建高清颜色数据 |
|
||||
| `refine-image-region` | `image.png`、`bounds.json` | Image Box / Lasso 读取 Fit to Foreground 结果 |
|
||||
| `refine-image-selection` | `image.png`、`bounds.json` | Image Box / Lasso 读取 Refine Selection 结果 |
|
||||
| `generate-depth-plane-geometry` | `image.0001.*`、`z-depth.exr`、`depth.json` | 创建 Depth Plane |
|
||||
| `generate-cutout-artifacts` | 可选 Region、Depth 与 Normal 文件 | 创建 Cutout Shape 与材质 |
|
||||
| `generate-cutout-artifacts` | 可选 Selection、Depth 与 Normal 文件 | 创建 Cutout Shape 与材质 |
|
||||
|
||||
Depth Job 保存可由 Blender 直接加载的模型空间 EXR；Blender 主线程负责轮廓提取、三角化、Depth 标定和数据创建。
|
||||
|
||||
## Depth 与 Normal 文件
|
||||
|
||||
MoGe-2 Prediction 只存在于 Server 内存。`geometry/prediction_artifacts.py` 共享一次推理，并按请求写出以下文件：
|
||||
MoGe-2 的原始结果由模型适配器转换为单帧 `GeometryFrame`。它包含 Depth、Validity、Intrinsics，以及可选的 Normal 和 Points；`geometry/prediction_artifacts.py` 把同一 Frame 交给各 Writer，并按请求写出以下文件：
|
||||
|
||||
| 文件 | 内容 |
|
||||
| --- | --- |
|
||||
| `vector-depth.exr` | RGB 保存相机空间 XYZ，Alpha 保存有效区 |
|
||||
| `z-depth.exr` | RGB 重复保存相机 Z，Alpha 保存有效区 |
|
||||
| `depth.json` | 保存 `image_size`、3×3 `intrinsics` 与 `reference_depth` |
|
||||
| `vector-depth.exr` | RGB 保存完整相机空间 XYZ，Alpha 保存 Validity |
|
||||
| `z-depth.exr` | RGB 重复保存完整相机 Z，Alpha 保存 Validity |
|
||||
| `depth.json` | 保存 `image_size`、3×3 `intrinsics` 与有效区域的 `reference_depth` |
|
||||
| `tangent-normal.png` | Tangent Space Normal Map |
|
||||
| `object-normal.png` | Object Space Normal Map |
|
||||
|
||||
Blender 通过 `common/depth_data.py` 加载 EXR，设为 Non-Color 并 Pack。Depth 与 Normal 数据块分别由 `depth_data_name()` 和 `normal_data_name()` 命名。Blender 读取 Z 与 Alpha 进行最终 Region 的本地标定，Geometry Nodes 采样已加载的 EXR。
|
||||
MoGe-2 的布尔 Mask 以 `0/1` 写入 Validity。Mask 外的有限 Depth/XYZ 仍保留在 RGB，使 Geometry Nodes 的线性采样维持连续；输入 Alpha、正深度和有限性只参与 `depth.json` 的参考样本筛选。任一待写 Field 包含 NaN/Inf 时不生成 EXR。
|
||||
|
||||
## Cutout Region 产物
|
||||
Blender 通过 `common/depth.py` 加载 EXR，设为 Non-Color 并 Pack。Depth 与 Normal 数据块分别由 `depth_data_name()` 和 `normal_data_name()` 命名。Geometry Nodes 采样已加载的 EXR。
|
||||
|
||||
Blender 先把 `RegionPath` 栅格化为 `RegionMask`，再把局部 RGBA 作为 Server 输入。Server 不接收 Path 或 Mask 对象协议：
|
||||
## Cutout Selection 产物
|
||||
|
||||
- 开启 Fit to Foreground 时，`region.png` 保存 Fit to Foreground 后的局部 RGBA，`region_bounds.json` 保存它相对输入裁切图的范围；
|
||||
Blender 先把 `SelectionPath` 栅格化为 `SelectionMask`，再把局部 RGBA 作为 Server 输入。MoGe-2 输入统一按 `RGB × Alpha` 合成到黑底；完全透明像素下的隐藏 RGB 不影响预测。Server 不接收 Path 或 Mask 对象协议：
|
||||
|
||||
- 开启 Refine Selection 时，`selection.png` 保存细化后的局部 RGBA，`selection_bounds.json` 保存它相对输入裁切图的范围；
|
||||
- 需要 Vector Depth 时请求 `vector-depth.exr` 与 `depth.json`；
|
||||
- 需要 Z Depth 时请求 `z-depth.exr` 与 `depth.json`；
|
||||
- Normal Map 按空间请求 `tangent-normal.png` 或 `object-normal.png`。
|
||||
|
||||
Surface 与 Balloon 的本地路径直接使用 `RegionMask` 构建 Shape。最终颜色图由 Blender 按 Region 范围裁切源 Image，并保留源 RGBA。
|
||||
Surface 与 Balloon 的本地路径直接使用 `SelectionMask` 构建 Shape。最终颜色图由 Blender 按 Selection 范围裁切源 Image，并保留源 RGBA。
|
||||
|
||||
## Debug 产物
|
||||
|
||||
|
||||
@@ -14,13 +14,13 @@ src/anyimage/
|
||||
├── model_manager.py # 模型目录与 Session 生命周期
|
||||
├── jobs/ # 公开协议与流程编排
|
||||
├── media/ # 图片、视频、序列帧和尺寸处理
|
||||
├── region/ # 通用 BEN2 Region 细化
|
||||
├── selection/ # 通用 BEN2 Selection 细化
|
||||
├── geometry/ # Depth / Normal 文件产物与相机元数据
|
||||
├── models/ # 模型适配与 ONNX 推理
|
||||
└── outputs/ # Debug Prediction 序列化
|
||||
```
|
||||
|
||||
`jobs/` 编排任务，领域目录提供媒体、Region、Geometry、模型和输出能力。
|
||||
`jobs/` 编排任务，领域目录提供媒体、Selection、Geometry、模型和输出能力。
|
||||
|
||||
## 生命周期
|
||||
|
||||
@@ -43,16 +43,16 @@ Job 在阶段边界调用 `context.check_cancelled()`，并把稳定产物写入
|
||||
| `download-model`、`download-required-models` | 准备模型文件 |
|
||||
| `remove-background` | 生成 BEN2 透明前景 |
|
||||
| `upscale-image` | 4× 图片或帧序列 |
|
||||
| `refine-image-region` | 为 Image Box / Lasso 执行 Fit to Foreground |
|
||||
| `refine-image-selection` | 为 Image Box / Lasso 执行 Refine Selection |
|
||||
| `generate-depth-plane-geometry` | 为 Depth Plane 复制颜色并生成 Z Depth 与相机元数据 |
|
||||
| `generate-cutout-artifacts` | 按需执行 Fit to Foreground，并生成 Depth 或 Normal 文件产物 |
|
||||
| `generate-cutout-artifacts` | 按需执行 Refine Selection，并生成 Depth 或 Normal 文件产物 |
|
||||
| `debug-*` | 保存模型适配层的直接输出 |
|
||||
|
||||
`generate-depth-plane-geometry` 对应 `jobs/depth_plane.py`；`generate-cutout-artifacts` 对应 `jobs/cutout.py`。共享的 MoGe-2 推理编排位于 `geometry/prediction_artifacts.py`，Depth EXR 与元数据写入位于 `geometry/depth_texture.py`，Normal 编码位于 `geometry/normal_texture.py`，Fit to Foreground 的通用实现位于 `region/refine_region.py`。
|
||||
`generate-depth-plane-geometry` 对应 `jobs/depth_plane.py`；`generate-cutout-artifacts` 对应 `jobs/cutout.py`。共享的 MoGe-2 推理编排位于 `geometry/prediction_artifacts.py`，它把最终 Selection RGBA 合成到黑底后执行推理，并把单帧 `GeometryFrame` 交给 Depth、Normal 与 Metadata Writer。Depth EXR 与元数据写入位于 `geometry/depth_texture.py`，Normal 编码位于 `geometry/normal_texture.py`，Refine Selection 的通用实现位于 `selection/refine_selection.py`。
|
||||
|
||||
## 尺寸约束
|
||||
|
||||
`server/media/resolution.py` 提供 **Maximum AI Input Size**，默认 `2048 px`。它限制 Upscale 的可提交源图长边，以及 Depth、Normal Map、Fit to Foreground 的模型分析图长边；材质颜色图使用原始分辨率。
|
||||
`server/media/resolution.py` 提供 **Maximum AI Input Size**，默认 `2048 px`。它限制 Upscale 的可提交源图长边，以及 Depth、Normal Map、Refine Selection 的模型分析图长边；材质颜色图使用原始分辨率。
|
||||
|
||||
Blender 侧的 **Maximum Cutout Mesh Resolution** 统一限制所有 Cutout Shape 的网格采样密度。
|
||||
|
||||
@@ -60,7 +60,7 @@ Blender 侧的 **Maximum Cutout Mesh Resolution** 统一限制所有 Cutout Shap
|
||||
|
||||
## 模型
|
||||
|
||||
Depth Plane、Cutout Depth 与 Normal Map 使用 MoGe-2，Remove Background 与 Fit to Foreground 使用 BEN2，Upscale 使用所选高清模型。
|
||||
Depth Plane、Cutout Depth 与 Normal Map 使用 MoGe-2，Remove Background 与 Refine Selection 使用 BEN2，Upscale 使用所选高清模型。MoGe-2 生产推理直接返回 `GeometryFrame`；DA3 调试推理保留模型原始 Prediction，并可按帧适配到相同结构。
|
||||
|
||||
`ModelManager` 按模型族管理下载目录与 ONNX Session，同一族只保留最近使用的模型和 Device，避免同类模型重复占用资源。
|
||||
|
||||
|
||||
@@ -0,0 +1,2 @@
|
||||
schema: spec-driven
|
||||
created: 2026-08-31
|
||||
93
openspec/changes/normalize-moge-alpha-depth/design.md
Normal file
93
openspec/changes/normalize-moge-alpha-depth/design.md
Normal file
@@ -0,0 +1,93 @@
|
||||
## Context
|
||||
|
||||
Cutout Job 会把最终 Region 图像交给 MoGe-2。Remove Background 路径会清除全透明像素下的 RGB，Fit to Foreground 路径则可能保留隐藏 RGB；MoGe-2 只读取 RGB，因此两条业务路径当前并不等价。
|
||||
|
||||
几何预测的另一处边界也不统一：MoGe-2 先返回专用 `Prediction`，Normal Writer 直接读取它，Depth Writer 才临时转换为 `GeometryFrame`；DA3 则有另一套转换入口。这样 `GeometryFrame` 没有真正成为模型预测与几何产物之间的稳定协议。
|
||||
|
||||
MoGe-2 返回连续的 Depth/Points Field 和原始 Mask。当前 Depth Writer 会按有效性 Mask 清零 RGB，而 Depth Surface 线性采样 RGB、不读取 Alpha，因此 Mask 边界的零坐标会把网格拉向相机原点。
|
||||
|
||||
## Goals / Non-Goals
|
||||
|
||||
**Goals:**
|
||||
|
||||
- 以 MoGe-2 单帧预测的数据类型重建 `GeometryFrame`，并让 MoGe-2 与 DA3 都通过模型适配器产出该类型。
|
||||
- 让 Depth EXR、Normal Texture 和 `depth.json` 只消费 `GeometryFrame`，不直接依赖模型专用预测结构。
|
||||
- 让所有 MoGe-2 调用在同一个 Server 边界获得确定的黑底 RGB 输入。
|
||||
- 让 Depth EXR 分别表达连续数值场与原始模型 Mask，两者互不改写。
|
||||
- 保留严格的 Metadata 校准筛选，并阻止非有限数值进入 Blender。
|
||||
|
||||
**Non-Goals:**
|
||||
|
||||
- 不修改 BEN2、MoGe-2 或 DA3 模型实现及其推理分辨率。
|
||||
- 不改变 Depth EXR 文件名、尺寸、RGBA 通道布局或节点组接口。
|
||||
- 不用输入 Alpha 覆盖模型 Mask，也不在本次变更中让 Geometry Nodes 按 Alpha 丢弃点。
|
||||
- 不把 DA3 恢复为当前生产路径；仅保持其适配器与统一协议可用、可测试。
|
||||
|
||||
## Decisions
|
||||
|
||||
### 1. `GeometryFrame` 是单帧几何预测协议
|
||||
|
||||
`GeometryFrame` 按 MoGe-2 单帧结果定义：
|
||||
|
||||
- `depth: ndarray[H, W]`，必需的 `float32` 深度场；
|
||||
- `normal: ndarray[H, W, 3] | None`，MoGe-2 提供，DA3 可为空；
|
||||
- `validity: ndarray[H, W]`，必需的 `float32` 模型有效性信号；MoGe-2 的原始 Mask 转换为 `0/1`，DA3 保存原始连续 Confidence；
|
||||
- `intrinsics: ndarray[3, 3]`，必需的 `float32` 像素坐标内参；
|
||||
- `points: ndarray[H, W, 3] | None`，可选的 `float32` 相机空间点场；
|
||||
|
||||
构造时校验 dtype、维度和同帧空间尺寸，但不在容器内改写预测值。`validity` 是唯一存储的模型有效性信号；不再同时保存 `mask`、`confidence` 和派生的 `valid_mask`，避免相同职责出现多个可漂移来源。
|
||||
|
||||
MoGe-2 的生产推理入口直接返回一组 `GeometryFrame`，在适配器内完成 batch 拆分、Mask 到 Validity 的转换和内参像素化。用于调试 NPZ 的原始单次推理入口继续返回模型原始字典，因为它不是 Artifact 生产边界。DA3 保留原始调试推理，并提供同样的逐帧适配器：`normal=None`，从深度和内参生成 Points，并把 Confidence 作为连续 Validity。
|
||||
|
||||
不选择在通用层保留 MoGe-2 `Prediction` 再二次转换，因为这会继续允许 Writer 绕过统一协议。也不把 `GeometryFrame` 限定为 MoGe-2 类型，避免 DA3 和后续模型各自复制 Artifact 逻辑。
|
||||
|
||||
### 2. 所有几何 Artifact Writer 只消费 `GeometryFrame`
|
||||
|
||||
Vector Depth Writer 从 `frame.points` 和 `frame.validity` 写 EXR；Z Depth Writer 从 `frame.depth` 和 `frame.validity` 写 EXR；Normal Writer 从 `frame.normal` 生成贴图；Metadata Writer 从 `frame.depth`、`frame.intrinsics` 和调用方提供的业务 Region Mask 计算 `depth.json`。
|
||||
|
||||
Writer 只依赖通用 `GeometryFrame`，不导入 `moge2` 或 `da3`。缺少所需可选字段时明确失败，例如没有 `points` 不能写 Vector Depth，没有 `normal` 不能写 Normal Texture。Artifact 编排层从 MoGe-2 取得一帧 `GeometryFrame` 后，把同一对象传给各 Writer，不再读取模型专用属性。
|
||||
|
||||
Metadata 所需的校准有效性按使用点计算：正的 `frame.validity`、有限且为正的 Depth，以及业务 Region Mask 的交集。它不是 `GeometryFrame` 的第二个持久字段，也不反向影响 EXR Alpha 或 RGB。
|
||||
|
||||
### 3. 在最终 MoGe-2 输入边界统一黑底合成
|
||||
|
||||
读取最终 `region_image_path` 为 RGBA，按输入上限缩放，然后使用 `output_rgb = source_rgb * alpha` 合成到黑底，并始终写为 Job 内的 RGB PNG。即使无需缩放，也经过相同规范化步骤。
|
||||
|
||||
这会让半透明像素保留按 Alpha 衰减后的颜色，并彻底消除 Alpha 为零时隐藏 RGB 的差异。Metadata 和 Normal 输出所需的业务 Alpha 仍从原始 Region 图读取，不从已移除 Alpha 的模型输入读取。
|
||||
|
||||
不在 Blender 的 `prepare_region_input()` 提前处理，因为该入口还服务其他 Job，而且 Fit to Foreground 会在 Server 端重新生成 RGBA，无法形成唯一收口点。
|
||||
|
||||
### 4. Depth RGB 保留连续 Field，Alpha 只记录 `frame.validity`
|
||||
|
||||
Vector/Z Depth Writer 无条件把完整 Points/Depth Field 写入 RGB，并仅把 `frame.validity` 写入 Alpha。零 Validity 不再导致 RGB 清零，使 Depth Surface 在边缘线性采样时仍能得到连续坐标。
|
||||
|
||||
Writer 在创建文件前要求所需 Field 全部有限。Mask 为假不豁免该检查，因为 Geometry Nodes 仍会读取 Mask 外 RGB；NaN/Inf 会传播到网格。有限但非正的 Depth 仍忠实写入，正值条件只属于 Metadata 校准。
|
||||
|
||||
不把非有限值替换为零或最近有效值：前者会重新引入相机原点尖刺，后者不再忠实于模型结果。
|
||||
|
||||
### 5. 测试固定协议、工作流等价与连续性
|
||||
|
||||
测试分为四层：
|
||||
|
||||
1. `GeometryFrame` 校验，以及 MoGe-2/DA3 适配器的字段、形状、dtype、Validity 映射和批次拆分。
|
||||
2. 模型输入验证 Alpha 为 `0`、`0.5`、`1` 的黑底合成，并验证不同隐藏 RGB 得到相同透明区输入。
|
||||
3. Writer 验证只接收 `GeometryFrame`、Validity 为零时 RGB 保留原值、Alpha 等于 `frame.validity`、缺字段与 NaN/Inf 明确失败；Normal 和 Metadata 不再读取 MoGe-2 专用结果。
|
||||
4. Depth Surface 使用边缘 Mask 为假但 XYZ 连续的 Vector Depth，验证求值网格不会聚集到相机原点。
|
||||
|
||||
## Risks / Trade-offs
|
||||
|
||||
- [MoGe-2 返回类型变化会影响调用方] → 生产调用集中在 Artifact 编排层；一次性迁移并删除旧 `Prediction`/转换入口，不保留兼容层。
|
||||
- [GeometryFrame 严格 dtype 会暴露模型适配错误] → 在适配器边界显式转换并用模型级测试固定协议。
|
||||
- [黑底输入会改变当前 Fit to Foreground 的预测] → 这是统一语义的预期行为，用像素级测试固定合成规则。
|
||||
- [零 Validity 区域的有限 Field 可能精度较低] → Alpha 完整保留模型判断供消费者解释；连续 Field 用于避免几何塌缩。
|
||||
- [完整 Field 中一个非有限值会使 Job 失败] → 失败优于生成被 NaN/Inf 污染的网格，错误信息区分 Depth、Point 和 Normal Field。
|
||||
|
||||
## Migration Plan
|
||||
|
||||
1. 重建 `GeometryFrame` 与两种模型适配器，迁移 MoGe-2 生产推理返回值并删除旧专用中间结构。
|
||||
2. 将 Depth、Normal 和 Metadata Writer 改为只消费 `GeometryFrame`，再迁移 Artifact 编排层。
|
||||
3. 建立统一的 MoGe-2 黑底输入准备，并补充像素与工作流等价测试。
|
||||
4. 调整 EXR 通道写入与数值检查，增加 Depth Surface 回归测试。
|
||||
5. 更新 GeometryFrame、Artifact 和 Cutout 内部文档并运行全量测试。
|
||||
|
||||
现有 Job 目录中的旧 EXR 不迁移；重新运行对应功能即可生成新语义 Artifact。回退时必须同时回退协议、适配器、Writer 与测试。
|
||||
32
openspec/changes/normalize-moge-alpha-depth/proposal.md
Normal file
32
openspec/changes/normalize-moge-alpha-depth/proposal.md
Normal file
@@ -0,0 +1,32 @@
|
||||
## Why
|
||||
|
||||
同一前景图片通过“Remove Background 后生成 Depth Surface”和“原图启用 Fit to Foreground 后生成 Depth Surface”时，当前会向 MoGe-2 提交不同的隐藏 RGB，导致不同的 Mask 与 Depth 结果；Vector/Z Depth 又按有效 Mask 将 RGB 清零，使连续深度场在 Mask 边界退化到相机原点。与此同时，Depth 与 Normal 产物仍直接读取 MoGe-2 专用 Prediction，绕过了原本用于统一 MoGe-2 与 DA3 几何预测的 `GeometryFrame` 边界。
|
||||
|
||||
## What Changes
|
||||
|
||||
- 在最终 MoGe-2 推理入口统一将 RGBA 按 Alpha 合成到黑底；Alpha 为 `0.5` 的像素以一半原色、一半黑色进入模型，使 Remove Background 与 Fit to Foreground 工作流具有相同的输入语义。
|
||||
- 以 MoGe-2 的单帧预测数据设计统一 `GeometryFrame`：包含 Depth、Normal、Validity、相机内参与可选 Points。Validity 统一承载 MoGe-2 的 `0/1` Mask 或 DA3 的连续 Confidence。
|
||||
- MoGe-2 的生产推理直接返回 `GeometryFrame` 序列；DA3 保留原始 Debug Prediction，并提供到相同 `GeometryFrame` 的适配作为参考路径。
|
||||
- Depth EXR、Normal Map 与 `depth.json` Writer 只消费 `GeometryFrame`，不直接读取 MoGe-2 Prediction 或模型专用字段。
|
||||
- `vector-depth.exr` 的 RGB 对所有像素保存 MoGe-2 原始相机空间 XYZ，不再因 Validity 为零而写成零。
|
||||
- `z-depth.exr` 的 RGB 对所有像素保存 MoGe-2 原始 Depth，不再因 Validity 为零而写成零。
|
||||
- 两种 Depth EXR 的 Alpha 只保存模型原始 Validity，不与输入 Alpha、正深度检查或有限性检查合并。
|
||||
- 非有限 Depth 或 XYZ 不进行静默替换；Artifact 生成明确失败，避免 NaN/Inf 进入 Blender Geometry Nodes。
|
||||
- `GeometryFrame.validity` 是模型有效性信号的唯一事实；Depth Metadata 的参考深度按需组合输入 Alpha、Validity 与数值有效性，但不在 Frame 中保存第二套有效性字段。
|
||||
- **BREAKING（内部产物语义）**：Depth EXR 中 Alpha 为零像素的 RGB 将从固定零值改为模型原始结果；依赖旧清零行为的内部消费者必须同步更新。
|
||||
|
||||
## Capabilities
|
||||
|
||||
### New Capabilities
|
||||
|
||||
- `moge-depth-artifacts`: 规定统一 `GeometryFrame`、MoGe-2 黑底输入以及 GeometryFrame 驱动的 Depth/Normal/Metadata 产物语义。
|
||||
|
||||
### Modified Capabilities
|
||||
|
||||
无。
|
||||
|
||||
## Impact
|
||||
|
||||
- 影响 `server/models/geometry.py`、`server/models/moge2.py`、`server/models/da3.py`、Geometry Writer 与其测试。
|
||||
- Cutout Depth Surface 与 Convert to Depth Plane 将消费连续的 Depth RGB；现有 EXR 文件名、尺寸、通道布局和 Job 返回协议保持不变。
|
||||
- 不改变 BEN2 模型、MoGe-2 模型、Blender Operator 标识符、节点组接口或外部依赖。
|
||||
@@ -0,0 +1,113 @@
|
||||
## Purpose
|
||||
|
||||
统一透明 Region 的 MoGe-2 输入像素语义，并规定 Depth EXR 独立保存连续模型结果与模型 Validity，使不同前景工作流生成一致、可解释且不会在有效性边界坍缩的几何产物。
|
||||
|
||||
## ADDED Requirements
|
||||
|
||||
### Requirement: GeometryFrame 是统一几何预测边界
|
||||
|
||||
系统 SHALL 以单帧 `GeometryFrame` 表示供产物生成使用的几何预测。Frame SHALL 包含二维 float32 Depth、同尺寸 float32 Validity、3×3 float32 像素空间相机内参，以及可选的同尺寸三通道 float32 Normal 与 Points。Validity SHALL 统一保存模型提供的逐像素有效性信号。
|
||||
|
||||
#### Scenario: MoGe-2 预测适配
|
||||
- **WHEN** MoGe-2 完成一帧预测
|
||||
- **THEN** 系统生成包含 Depth、Normal、由原始 Mask 转换而来的 `0/1` Validity、像素空间相机内参与按需 Points 的 `GeometryFrame`
|
||||
|
||||
#### Scenario: DA3 预测适配
|
||||
- **WHEN** DA3 预测被转换为统一几何结果
|
||||
- **THEN** 系统生成包含 Depth、派生 Points、由原始 Confidence 提供的连续 Validity 与相机内参的 `GeometryFrame`，且 Normal 为空
|
||||
|
||||
#### Scenario: Frame 字段维度不一致
|
||||
- **WHEN** 任一像素 Field 的高宽与 Depth 不一致，或相机内参不是 3×3
|
||||
- **THEN** `GeometryFrame` 构建失败并报告字段维度错误
|
||||
|
||||
### Requirement: Geometry Artifact 只消费 GeometryFrame
|
||||
|
||||
系统 SHALL 从 `GeometryFrame` 生成 Vector Depth、Z Depth、Normal Map 与 Depth Metadata，Artifact Writer SHALL NOT 直接依赖 MoGe-2 或 DA3 专用 Prediction 类型。
|
||||
|
||||
#### Scenario: 生成 Vector Depth
|
||||
- **WHEN** 一个 `GeometryFrame` 包含 Points
|
||||
- **THEN** Vector Depth Writer 从 Frame 的 Points 与 Validity 生成 `vector-depth.exr`
|
||||
|
||||
#### Scenario: 生成 Z Depth 与 Metadata
|
||||
- **WHEN** 一个 `GeometryFrame` 包含 Depth、Validity 与 Intrinsics
|
||||
- **THEN** Z Depth 与 Metadata Writer 从该 Frame 生成 `z-depth.exr` 与 `depth.json`
|
||||
|
||||
#### Scenario: 生成 Normal Map
|
||||
- **WHEN** 一个 `GeometryFrame` 包含 Normal
|
||||
- **THEN** Normal Writer 从 Frame 的 Normal 及调用方提供的 Region 可见 Mask 生成 Normal Map
|
||||
|
||||
#### Scenario: 缺少所需 Field
|
||||
- **WHEN** Writer 请求的 Points 或 Normal 在 `GeometryFrame` 中为空
|
||||
- **THEN** 产物生成失败并报告缺少对应 Geometry Field
|
||||
|
||||
### Requirement: MoGe-2 使用统一黑底输入
|
||||
|
||||
系统 SHALL 在 MoGe-2 推理前将最终 Region RGBA 合成到黑底，模型 RGB SHALL 等于源 RGB 乘以归一化 Alpha。
|
||||
|
||||
#### Scenario: 完全透明像素
|
||||
- **WHEN** 最终 Region 像素的 Alpha 为 `0`
|
||||
- **THEN** MoGe-2 接收到的对应 RGB 为 `(0, 0, 0)`
|
||||
|
||||
#### Scenario: 半透明像素
|
||||
- **WHEN** 最终 Region 像素的 Alpha 为 `0.5`
|
||||
- **THEN** MoGe-2 接收到的对应 RGB 为源 RGB 的一半
|
||||
|
||||
#### Scenario: 完全不透明像素
|
||||
- **WHEN** 最终 Region 像素的 Alpha 为 `1`
|
||||
- **THEN** MoGe-2 接收到的对应 RGB 与源 RGB 相同
|
||||
|
||||
#### Scenario: 前景工作流输入一致
|
||||
- **WHEN** Remove Background 与 Fit to Foreground 产生相同的可见 RGB 和 Alpha，仅完全透明像素下的隐藏 RGB 不同
|
||||
- **THEN** 两条工作流提交给 MoGe-2 的模型 RGB 完全相同
|
||||
|
||||
### Requirement: Vector Depth 独立保存 Points 与 Validity
|
||||
|
||||
系统 SHALL 在 `vector-depth.exr` 的 RGB 中保存 `GeometryFrame.points` 的每个原始有限相机空间 XYZ，且 SHALL 不因 `GeometryFrame.validity` 为零而修改 XYZ；Alpha SHALL 只保存 `GeometryFrame.validity`。
|
||||
|
||||
#### Scenario: Validity 为零的有限 Point
|
||||
- **WHEN** MoGe-2 为一个像素返回有限 XYZ 且原始 Mask 转换得到的 Validity 为 `0`
|
||||
- **THEN** EXR RGB 等于该 XYZ，Alpha 等于 `0`
|
||||
|
||||
#### Scenario: Validity 为一的有限 Point
|
||||
- **WHEN** MoGe-2 为一个像素返回有限 XYZ 且原始 Mask 转换得到的 Validity 为 `1`
|
||||
- **THEN** EXR RGB 等于该 XYZ，Alpha 等于 `1`
|
||||
|
||||
#### Scenario: 输入 Alpha 不改变 Vector Depth Alpha
|
||||
- **WHEN** Region 输入 Alpha 与 MoGe-2 原始 Mask 不同
|
||||
- **THEN** `vector-depth.exr` Alpha 等于由 MoGe-2 原始 Mask 转换得到的 Validity
|
||||
|
||||
### Requirement: Z Depth 独立保存 Depth 与 Validity
|
||||
|
||||
系统 SHALL 在 `z-depth.exr` 的 RGB 三个通道中保存 `GeometryFrame.depth` 的每个原始有限 Depth，且 SHALL 不因 `GeometryFrame.validity` 为零而修改 Depth；Alpha SHALL 只保存 `GeometryFrame.validity`。
|
||||
|
||||
#### Scenario: Validity 为零的有限 Depth
|
||||
- **WHEN** MoGe-2 为一个像素返回有限 Depth 且原始 Mask 转换得到的 Validity 为 `0`
|
||||
- **THEN** EXR RGB 三个通道均等于该 Depth，Alpha 等于 `0`
|
||||
|
||||
#### Scenario: Validity 为一的有限 Depth
|
||||
- **WHEN** MoGe-2 为一个像素返回有限 Depth 且原始 Mask 转换得到的 Validity 为 `1`
|
||||
- **THEN** EXR RGB 三个通道均等于该 Depth，Alpha 等于 `1`
|
||||
|
||||
#### Scenario: 输入 Alpha 不改变 Z Depth Alpha
|
||||
- **WHEN** Region 输入 Alpha 与 MoGe-2 原始 Mask 不同
|
||||
- **THEN** `z-depth.exr` Alpha 等于由 MoGe-2 原始 Mask 转换得到的 Validity
|
||||
|
||||
### Requirement: Depth Artifact 拒绝非有限结果
|
||||
|
||||
系统 MUST 在写入 Depth EXR 前校验所有待写入 Depth 或 XYZ；任何通道包含 NaN 或 Inf 时 MUST 终止 Artifact 生成并报告错误，且 MUST NOT 以零值替换。
|
||||
|
||||
#### Scenario: Vector Depth 包含非有限值
|
||||
- **WHEN** 任意原始 XYZ 分量为 NaN 或 Inf
|
||||
- **THEN** `vector-depth.exr` 生成失败并报告无效 Point Field
|
||||
|
||||
#### Scenario: Z Depth 包含非有限值
|
||||
- **WHEN** 任意原始 Depth 为 NaN 或 Inf
|
||||
- **THEN** `z-depth.exr` 生成失败并报告无效 Depth Field
|
||||
|
||||
### Requirement: 校准筛选不改写 Depth Artifact
|
||||
|
||||
系统 SHALL 按需使用输入 Alpha、正的 `GeometryFrame.validity`、正 Depth 与有限性共同筛选 Depth Metadata 的参考样本，但 SHALL NOT 把该派生筛选结果存为 Frame 的第二套有效性字段，且 SHALL NOT 改写 Depth EXR 的 RGB 或 Alpha。
|
||||
|
||||
#### Scenario: 零 Validity 区域存在连续 Depth
|
||||
- **WHEN** Validity 为零的区域存在有限 Depth 或 XYZ
|
||||
- **THEN** 这些值保留在 EXR RGB 中，同时不进入参考深度计算
|
||||
23
openspec/changes/normalize-moge-alpha-depth/tasks.md
Normal file
23
openspec/changes/normalize-moge-alpha-depth/tasks.md
Normal file
@@ -0,0 +1,23 @@
|
||||
## 1. 统一 GeometryFrame 边界
|
||||
|
||||
- [x] 1.1 按单帧 MoGe-2 数据重建 `GeometryFrame`：定义 Depth、Normal、Validity、Intrinsics、Points 字段及 dtype/shape 校验，删除 `mask`、`confidence`、`valid_mask`、`model_mask` 多重状态并补充单元测试。
|
||||
- [x] 1.2 让 MoGe-2 生产推理直接返回 `GeometryFrame` 序列，在适配器中完成 batch 拆分、Mask 到 `0/1` Validity 的转换和内参像素化；删除旧 `Prediction` 与二次转换入口，并验证每个字段忠实来自原始预测。
|
||||
- [x] 1.3 为 DA3 保留原始调试推理并实现 `GeometryFrame` 适配器，验证 `normal=None`、派生 Points、Intrinsics 和由 Confidence 提供的连续 Validity，保持该参考路径可用。
|
||||
|
||||
## 2. 统一 MoGe-2 输入
|
||||
|
||||
- [x] 2.1 将模型输入准备改为始终读取最终 Region RGBA、按输入上限缩放并以 `RGB × Alpha` 合成到黑底；用像素测试验证 Alpha 为 `0`、`0.5`、`1` 的输出。
|
||||
- [x] 2.2 增加不同隐藏 RGB 的工作流等价测试，验证可见 RGB 与 Alpha 相同时生成完全一致的 MoGe-2 RGB 输入；运行相关 Server Job 测试。
|
||||
|
||||
## 3. 让几何产物只消费 GeometryFrame
|
||||
|
||||
- [x] 3.1 修改 Vector/Z Depth Writer，使其只接收 `GeometryFrame`，完整写入有限的 Points/Depth RGB，并把 `frame.validity` 单独写入 Alpha；用 EXR 回读测试验证 Validity 为零时 Field 仍保留。
|
||||
- [x] 3.2 为 Depth Writer 增加缺少 Points、NaN/Inf 失败测试，验证错误明确区分 Point Field 与 Depth Field，且失败时不生成 EXR。
|
||||
- [x] 3.3 修改 Normal Writer 只从 `GeometryFrame.normal` 读取预测，保留业务 Region Mask 的可见性处理，并增加缺少 Normal 与非有限值测试。
|
||||
- [x] 3.4 修改 Metadata Writer 从 `GeometryFrame` 按需计算正 Validity、正且有限 Depth、业务 Region Mask 的校准交集；迁移 Artifact 编排层，使其不再读取 MoGe-2 专用预测字段，并运行 Server Job/Runtime 测试。
|
||||
|
||||
## 4. Depth Surface 回归与文档
|
||||
|
||||
- [x] 4.1 增加边缘 Mask 为假但 XYZ 连续的 Depth Surface 测试，验证求值网格不会聚集到相机原点；运行 `uv run pytest tests/test_cutout_objects.py tests/test_node_assets.py`。
|
||||
- [x] 4.2 更新 GeometryFrame、模型适配、MoGe 输入、Depth/Normal/Metadata Artifact 语义及 Cutout Runtime 文档；运行 `uv run pytest tests/test_docs.py`，不构建文档产物。
|
||||
- [x] 4.3 运行 `uv run pytest` 完成全量回归，确认两条前景工作流、Depth Plane、Depth Surface、打包和文档检查全部通过。
|
||||
@@ -0,0 +1,3 @@
|
||||
schema: spec-driven
|
||||
created: 2026-08-31
|
||||
skip_specs: true
|
||||
97
openspec/changes/reorganize-common-modules/design.md
Normal file
97
openspec/changes/reorganize-common-modules/design.md
Normal file
@@ -0,0 +1,97 @@
|
||||
## Context
|
||||
|
||||
`src/anyimage/common/` 当前有七个模块，其中主要问题不是文件数量，而是少数函数位于不符合业务认知的位置：通用 AI UI 位于 `image_interaction.py`，`object.py` 中的大部分代码实际负责材质，多个文件名又使用宽泛的 `*_data` 或 `*_interaction` 后缀。
|
||||
|
||||
调用方直接从具体模块导入函数，`common/__init__.py` 不承担公共导出。迁移必须一次性更新所有调用方并删除旧路径，不增加转发模块或重导出。`server/` 仍保持独立，不能引用上级模块。
|
||||
|
||||
## Goals / Non-Goals
|
||||
|
||||
**Goals:**
|
||||
|
||||
- 使用少量、稳定的业务名词，使函数位置容易预测。
|
||||
- 修正明显错位的 AI UI 与材质函数，同时避免为了追求形式纯粹而拆出大量小文件。
|
||||
- 保持函数行为、Job 字典、Operator 行为与 Blender 数据结果不变。
|
||||
|
||||
**Non-Goals:**
|
||||
|
||||
- 不要求每个模块只有一种技术副作用；同一业务对象的读取、输入准备与结果应用可以集中存在。
|
||||
- 不重写算法、调整 UI、改变公开 Blender 标识符或 Job 协议。
|
||||
- 不增加 Facade、抽象基类、动态导入或 `common/__init__.py` 重导出。
|
||||
|
||||
## Decisions
|
||||
|
||||
### 1. 采用八个业务模块
|
||||
|
||||
目标目录如下：
|
||||
|
||||
| 模块 | 职责 | 来源与主要内容 |
|
||||
| --- | --- | --- |
|
||||
| `ai.py` | Blender 侧所有共享 AI 准备能力 | 合并 `ai_job.py` 全部函数，以及 `draw_ai_setup()`、`draw_ai_property()` |
|
||||
| `image.py` | Blender Image 与 Image Empty 的完整生命周期 | 原 `image_data.py` 全部函数 |
|
||||
| `region.py` | Region 值、序列化、栅格化与 Alpha 裁切 | 原 `image_region.py` 全部内容 |
|
||||
| `viewport.py` | View3D 坐标投影、点击解析、工具设置、手势状态与 Overlay 绘制 | 原 `image_interaction.py` 除 AI UI 外的全部内容 |
|
||||
| `depth.py` | Depth 产物导入、校验与尺度换算 | 原 `depth_data.py` 全部内容 |
|
||||
| `material.py` | Image 材质节点树构建与配置 | 从 `object.py` 迁移 `material_color_space_name()`、`set_material_color_space()`、`material_node_group()`、`create_image_material()`、`set_displacement_only()` |
|
||||
| `object.py` | Object 与 Modifier 的共享构建收尾 | 保留 `modifier_input_identifier()`、`set_modifier_input()`、`finalize_object_result()` 及其私有辅助函数 |
|
||||
| `node.py` | 随扩展分发的 Node Group 资产加载 | 原 `node.py` 保持不变 |
|
||||
|
||||
这套分类以开发者查找路径为优先：想处理 AI、Image、Region、Viewport、Depth、Material、Object 或 Node 时，只需选择对应名词；所有不修改 Image 数据、只负责 View3D 输入与显示反馈的函数统一位于 `viewport.py`。
|
||||
|
||||
`image.py` 虽然体量较大，但其中函数共同覆盖 Blender Image 从读取、输入准备、结果导入到 Empty 替换的完整生命周期。拆成 `image_input`、`image_result`、`image_empty` 会迫使多数 Image Operator 同时依赖三至四个文件，增加导航成本，因此不采用。
|
||||
|
||||
`viewport.py` 保留交互上下文、二维预览几何和 Modal 手势。它们共同描述 View3D 中“如何点击、投影、绘制和完成手势”，且共享大量私有辅助函数；拆成 `viewport_draw`、`viewport_gesture` 和 `viewport_context` 对当前规模收益有限。
|
||||
|
||||
### 2. AI 能力使用单一入口
|
||||
|
||||
`ai.py` 同时容纳环境/模型检查、输入校验、参数组装和两项通用 UI 绘制函数。它们都是 Operator 在启动或呈现 AI 功能时使用的共享能力，调用方无需判断某项能力属于 Job、状态还是 UI 子分类。
|
||||
|
||||
备选方案是保留 `ai_job.py` 并新增 `ai_ui.py`。该方案能按技术层分离 UI，却会为两个很短的函数增加模块，并且不能满足“所有 AI 使用方容易找到”的首要目标，因此不采用。
|
||||
|
||||
### 3. 只对 Object 做一次有价值的拆分
|
||||
|
||||
`create_image_material()` 及其辅助函数构成独立、体量较大的材质构建单元，迁移到 `material.py` 后，`object.py` 自然剩下 Object 与其 Modifier 的构建收尾。Modifier 不再单独拆文件，因为只有两个公共入口，且全部由 Object 构建流程消费。
|
||||
|
||||
`node.py` 不改名为 `node_asset.py`：当前文件只负责 Node Group 资产加载，名称已经足够明确，改名不会改善边界。
|
||||
|
||||
### 4. 文件名简化，但函数名和行为保持不变
|
||||
|
||||
`image_data.py`、`image_region.py`、`image_interaction.py`、`depth_data.py` 分别迁移为 `image.py`、`region.py`、`viewport.py`、`depth.py`。迁移只修正模块名和导入，不顺带重命名函数或修改算法。
|
||||
|
||||
完成后直接删除被替代文件，不保留兼容转发；所有生产代码、测试和文档在同一变更中更新。
|
||||
|
||||
### 5. 依赖关系保持简单
|
||||
|
||||
```mermaid
|
||||
flowchart TD
|
||||
Operator["operators/"] --> AI["ai"]
|
||||
Operator --> Viewport["viewport"]
|
||||
Operator --> Image["image"]
|
||||
Operator --> Depth["depth"]
|
||||
Operator --> Material["material"]
|
||||
Operator --> Object["object"]
|
||||
Viewport --> AI
|
||||
Viewport --> Image
|
||||
Viewport --> Region["region"]
|
||||
Depth --> Image
|
||||
Material --> Image
|
||||
Material --> Node["node"]
|
||||
```
|
||||
|
||||
`region.py` 保持与 Blender UI 无关；`image.py` 不反向依赖编辑、材质或 Object 构建；`common/__init__.py` 不隐藏函数来源。
|
||||
|
||||
## Risks / Trade-offs
|
||||
|
||||
- [直接删除旧路径会使遗漏导入立即失败] → 全仓库检查旧模块名并运行完整测试。
|
||||
- [较大的 `image.py` 与 `viewport.py` 不是技术上最纯的分层] → 优先保证业务内聚与导航效率；只有出现第二个独立消费者群或显著循环依赖时再拆分。
|
||||
- [机械移动可能夹带行为变化] → 保持函数体、签名和私有辅助函数，只修改导入与模块说明。
|
||||
- [Material 移出后可能遗漏 Object 调用方] → 对 `create_image_material()` 和相关辅助函数执行全仓库引用检查并运行 Object 测试。
|
||||
|
||||
## Migration Plan
|
||||
|
||||
1. 将 `ai_job.py` 迁移为 `ai.py`，并从 `image_interaction.py` 接收通用 AI UI 函数。
|
||||
2. 将四个宽泛文件名迁移为业务名称：`image.py`、`region.py`、`viewport.py`、`depth.py`。
|
||||
3. 从 `object.py` 提取 `material.py`，保留 Object 与 Modifier 函数；`node.py` 保持原状。
|
||||
4. 更新生产代码、测试和文档的直接导入，确认无旧路径引用后删除旧文件。
|
||||
5. 运行完整测试验证行为不变。
|
||||
|
||||
本变更不涉及持久化数据迁移。若实现阶段需要回退，应整体回退该变更，而不是恢复兼容转发模块。
|
||||
28
openspec/changes/reorganize-common-modules/proposal.md
Normal file
28
openspec/changes/reorganize-common-modules/proposal.md
Normal file
@@ -0,0 +1,28 @@
|
||||
## Why
|
||||
|
||||
`common/` 当前以功能演进过程形成分类，部分模块同时容纳数据、I/O、UI、交互与绘制职责，导致公共函数的位置不能由名称直接判断；例如通用 AI UI 函数 `draw_ai_setup()` 位于 `image_interaction.py`。重新建立稳定、可预测的模块边界，可以降低新增功能复用公共能力时的查找与依赖成本。
|
||||
|
||||
## What Changes
|
||||
|
||||
- 将 `common/` 收敛为八个稳定业务模块：`ai`、`image`、`region`、`viewport`、`depth`、`material`、`object` 和 `node`。
|
||||
- 将 `draw_ai_setup()`、`draw_ai_property()` 从 Image 交互分类移至 `ai.py`，与 AI 环境、模型检查和 Job 参数准备形成统一入口。
|
||||
- 保留完整的 Image 生命周期于 `image.py`，将 View3D 坐标、手势和 Overlay 绘制统一归入 `viewport.py`，避免把视口交互误称为图片编辑。
|
||||
- 仅从 `object.py` 独立出体量较大、职责明确的材质构建函数；Modifier 和结果 Object 处理继续归入 Object 分类。
|
||||
- 更新所有 Operator 与测试的直接导入，同时同步 `docs/internals/common.md` 和架构索引。
|
||||
- **BREAKING**：删除被替代的旧模块路径，不提供重导出、转发函数或兼容层；这是仅面向项目内部源码的导入变更，不改变扩展的用户功能与公开 Blender 标识符。
|
||||
|
||||
## Capabilities
|
||||
|
||||
### New Capabilities
|
||||
|
||||
无。本变更只重构 Blender 侧内部公共函数的模块边界，不新增用户可观察行为。
|
||||
|
||||
### Modified Capabilities
|
||||
|
||||
无。现有功能、Operator 协议、Job 参数和结果保持不变。
|
||||
|
||||
## Impact
|
||||
|
||||
- 主要影响 `src/anyimage/common/` 及所有直接导入这些模块的 `operators/`、`menu.py` 和测试。
|
||||
- `src/anyimage/server/` 的进程边界、Job 协议、模型实现和依赖均不改变。
|
||||
- 不新增依赖，不改变 Blender 类型注册、`bl_idname`、节点组名称或扩展打包内容。
|
||||
17
openspec/changes/reorganize-common-modules/tasks.md
Normal file
17
openspec/changes/reorganize-common-modules/tasks.md
Normal file
@@ -0,0 +1,17 @@
|
||||
## 1. AI 与业务模块命名
|
||||
|
||||
- [x] 1.1 将 `ai_job.py` 迁移为 `ai.py`，并从 `image_interaction.py` 迁入 `draw_ai_setup()` 与 `draw_ai_property()`；运行相关 Operator 导入测试并用 `rg` 确认 AI 公共函数只定义于 `ai.py`。
|
||||
- [x] 1.2 将 `image_data.py`、`image_region.py`、`depth_data.py` 分别迁移为 `image.py`、`region.py`、`depth.py`，保持函数体和签名不变；运行 `uv run pytest tests/test_image_data.py tests/test_cutout_geometry.py tests/test_cutout_objects.py` 验证行为。
|
||||
- [x] 1.3 将移除 AI UI 后的 `image_interaction.py` 迁移为 `viewport.py`，集中 View3D 坐标投影、点击解析、工具设置、手势与 Overlay 绘制；运行 `uv run pytest tests/test_cutout_interaction.py tests/test_blender_addon.py` 验证视口交互函数和模块导入。
|
||||
|
||||
## 2. Material 与 Object 边界
|
||||
|
||||
- [x] 2.1 建立 `material.py`，从 `object.py` 迁移材质颜色空间、Node Group、材质构建和 Displacement 配置函数；运行 `uv run pytest tests/test_image_data.py tests/test_cutout_objects.py` 验证材质结果。
|
||||
- [x] 2.2 收窄 `object.py` 为 Object 与 Modifier 构建收尾，并保持 `node.py` 不变；运行 Object、Convert to Plane 与 Cutout 相关测试验证 Modifier 输入和结果 Object。
|
||||
|
||||
## 3. 调用方、文档与验证
|
||||
|
||||
- [x] 3.1 更新 `operators/`、`menu.py` 和测试的全部直接导入，删除 `ai_job.py`、`image_data.py`、`image_region.py`、`image_interaction.py` 与 `depth_data.py`；使用 `rg` 确认不存在旧模块导入、重导出或兼容转发。
|
||||
- [x] 3.2 更新 Common 模块结构测试，验证八个目标模块可导入、AI UI 定义位置唯一且 `common/__init__.py` 不重导出；运行该测试确认边界规则生效。
|
||||
- [x] 3.3 按八个业务模块更新 `docs/internals/common.md` 和 `docs/internals/architecture.md`；人工核对文档索引与生产源码一致，不构建文档产物。
|
||||
- [x] 3.4 运行 `uv run pytest` 完成全量回归，确认所有 Operator、Job、节点资产、打包和文档检查通过。
|
||||
@@ -0,0 +1,2 @@
|
||||
schema: spec-driven
|
||||
created: 2026-08-31
|
||||
91
openspec/changes/unify-selection-terminology/design.md
Normal file
91
openspec/changes/unify-selection-terminology/design.md
Normal file
@@ -0,0 +1,91 @@
|
||||
## Context
|
||||
|
||||
当前图片选区业务横跨 Blender 侧数据结构、Viewport 手势、Image Tool、Cutout Tool 和 Server Job，但统一使用 `region`；与此同时 Blender API 也用 Region 表示编辑器窗口区域。两种含义出现在同一调用链中，增加了阅读和查找成本。
|
||||
|
||||
当前工作树已将 Blender 侧共享模块整理为 `common/region.py` 与 `common/viewport.py`。本次变更以该结构为迁移起点，并遵守项目不保留旧入口或兼容层的约束。Server 仍保持独立包边界，不能引用上级共享模块。
|
||||
|
||||
## Goals / Non-Goals
|
||||
|
||||
**Goals:**
|
||||
|
||||
- 让图片选区从数据类型到 Job 产物全程只使用 `selection`。
|
||||
- 明确区分图片 Selection 与 Blender Viewport Region。
|
||||
- 用面向用户的 Mesh Detail 代替实现导向的像素采样间距。
|
||||
- 一次性迁移所有内部调用方、测试和文档，不留下混合术语。
|
||||
|
||||
**Non-Goals:**
|
||||
|
||||
- 不修改路径栅格化、Mask 运算、BEN2 细化或 Normal Map 生成算法。
|
||||
- 不增加新的网格采样密度，也不改变四个既有间距的几何结果。
|
||||
- 不重命名 Blender API 规定的 Region 概念，也不替换与图片选区无关的普通区域表达。
|
||||
- 不改变 AI 可用性门控、Shape 候选项或 Normal Space 规则。
|
||||
|
||||
## Decisions
|
||||
|
||||
### 1. 按业务含义迁移，不做全局文本替换
|
||||
|
||||
业务选区主链统一采用以下名词：
|
||||
|
||||
| 当前表达 | 目标表达 |
|
||||
| --- | --- |
|
||||
| `common/region.py` | `common/selection.py` |
|
||||
| `RegionPath` / `RegionMask` / `RegionContours` | `SelectionPath` / `SelectionMask` / `SelectionContours` |
|
||||
| `region_path` / `region_mask` / `region_contours` | `selection_path` / `selection_mask` / `selection_contours` |
|
||||
| `ImageRegionGesture` | `ImageSelectionGesture` |
|
||||
| `SelectCutoutRegion` / `CutoutRegionToShape` | `SelectCutoutSelection` / `CutoutSelectionToShape` |
|
||||
| `RefineImageRegion` / `refine-image-region` | `RefineImageSelection` / `refine-image-selection` |
|
||||
| `refine_region` | `refine_selection` |
|
||||
| `region_image` / `region_bounds` | `selection_image` / `selection_bounds` |
|
||||
| `server/region/refine_region.py` | `server/selection/refine_selection.py` |
|
||||
|
||||
同一规则应用于相关函数、属性、序列化字段、进度消息、临时文件和测试名称。由于项目不保留兼容层，旧模块、Operator ID、Job 类型和字段在迁移完成后直接删除。
|
||||
|
||||
不纳入迁移的名称包括 `context.region`、`context.region_data`、`event.mouse_region_x/y`、Viewport Region 指针与尺寸，以及算法中确实表示一般连通区域而非用户 Selection 的术语。相比机械替换，按语义清单迁移可以避免破坏 Blender API，并使残留检查具有明确判断标准。
|
||||
|
||||
### 2. Selection 是业务名词，Refine 是动作
|
||||
|
||||
属性工厂和三个工具设置分别改为 `refine_selection_property`、`cutout_refine_selection`、`image_box_refine_selection` 与 `image_lasso_refine_selection`，Operator 参数使用 `refine_selection`。界面统一显示 **Refine Selection**，描述说明使用 BEN2 将手势产生的 Selection 收紧到前景。
|
||||
|
||||
相较只改界面文案，此方案让代码与界面保持同一概念；相较继续使用 `fit`，`refine` 更准确地表达已有 AI 操作，并与原名称一致。
|
||||
|
||||
### 3. Normal Map 只缩短用户界面名称
|
||||
|
||||
Cutout 布尔选项显示名从 **Generate Normal Map** 改为 **Normal Map**。`generate_normal` 等执行层标识继续表达“是否生成”这一布尔动作，不为追求表面一致而改成含义更弱的 `normal_map`。
|
||||
|
||||
### 4. Mesh Detail 映射到既有采样间距
|
||||
|
||||
Cutout 设置从 `cutout_point_spacing` 整数属性迁移为 `cutout_mesh_detail` 枚举属性，显示名为 **Mesh Detail**。枚举标识使用稳定字符串 `"LOW"`、`"MEDIUM"`、`"HIGH"`、`"ULTRA"`，使代码标识与界面档位直接对应；创建 Shape 前通过单一映射转换为实际采样间距：
|
||||
|
||||
| Mesh Detail | Sampling Spacing |
|
||||
| --- | --- |
|
||||
| Low | 32 px |
|
||||
| Medium | 16 px |
|
||||
| High | 8 px |
|
||||
| Ultra | 4 px |
|
||||
|
||||
默认值为 Low，与当前默认 32 px 保持一致。转换函数放在使用网格间距的 Cutout 业务边界，后续几何代码仍接收明确的像素间距，不让 UI 档位渗入采样算法。
|
||||
|
||||
相比保留 4–32 的任意整数输入，四档枚举只暴露实际支持和测试的密度；相比直接把 Level 数值传入几何代码，显式映射保留了算法参数的物理含义。
|
||||
|
||||
### 5. Server 协议与文件产物同步切换
|
||||
|
||||
Blender 和 Server 在同一次变更中切换 Job 类型、参数、结果字段及产物文件名。建议的细化产物为 `selection.png` 与 `selection_bounds.json`；独立 Image Selection Job 的通用输出仍可沿用 `image.png` 与 `bounds.json`，但结果字典键使用 `selection_image` 与 `selection_bounds`。
|
||||
|
||||
协议不做双读或双写。这样符合项目移除旧路径的规则，也避免长期维护两套字段；代价是旧会话中尚未完成的 Job 结果不能由新代码恢复。
|
||||
|
||||
## Risks / Trade-offs
|
||||
|
||||
- [遗漏某个业务 `region` 导致混合术语或运行时字段不匹配] → 按模块清单迁移，并用定向 `rg` 结合人工语义复核；测试覆盖 Blender 调用与 Server 返回值两端。
|
||||
- [机械重命名误伤 Blender Viewport Region] → 将 Blender API 名称列入明确保留清单，重点运行手势绘制、投影和 Perspective 相关测试。
|
||||
- [与当前 `reorganize-common-modules` 工作树重叠] → 以其整理后的 `common/selection.py` 目标结构为准，不恢复已删除的旧模块。
|
||||
- [旧 `.blend` 中保存的工具设置名失效] → 接受该迁移成本，不增加兼容属性；工具设置回到新属性的默认值。
|
||||
- [Mesh Detail 顺序与采样间距方向写反] → 使用单一常量映射，并对 Low、Medium、High、Ultra 四档添加参数化测试。
|
||||
- [旧 Job 或临时结果不可恢复] → 同版本同步发布 Blender 与 Server 代码，并接受更新时终止中的旧 Job。
|
||||
|
||||
## Migration Plan
|
||||
|
||||
1. 先迁移 Selection 数据模块、类型和纯函数，再更新 Blender 侧所有导入与调用。
|
||||
2. 迁移 Viewport 手势、工具属性和 Operator，同时切换 Mesh Detail 枚举与间距映射，更新注册与逆序注销。
|
||||
3. 同步迁移 Server 包、Job 类型、参数、结果字段和产物文件名。
|
||||
4. 更新测试与内部文档，使用搜索确认剩余 `region` 均属于保留语义。
|
||||
5. 运行相关测试后运行全部测试；若失败，整体回退本次重命名，不引入临时兼容层。
|
||||
28
openspec/changes/unify-selection-terminology/proposal.md
Normal file
28
openspec/changes/unify-selection-terminology/proposal.md
Normal file
@@ -0,0 +1,28 @@
|
||||
## Why
|
||||
|
||||
当前代码把用户圈定的图像选区称为 `region`，容易与 Blender 的 Viewport Region 混淆，也与界面中的选择概念不完全对应；Cutout 还直接暴露像素采样间距，用户难以判断数值与细节的关系。统一业务术语并改用 Mesh Detail 后，代码、公开协议和界面可以使用更直观且彼此对应的表达。
|
||||
|
||||
## What Changes
|
||||
|
||||
- **BREAKING**：将表示图像选区的模块、类型、函数、属性、Operator、Job 参数、返回字段和文件名从 `region` 重命名为 `selection`，不保留兼容别名或转发入口。
|
||||
- 保留 Blender API 固有的 `context.region`、`region_data`、`mouse_region_x` 等名称，以及与选区业务无关的普通空间区域表达。
|
||||
- 将界面选项 **Fit to Foreground** 恢复为 **Refine Selection**，并同步对应的代码标识。
|
||||
- 将界面选项 **Generate Normal Map** 缩写为 **Normal Map**；其布尔语义和生成行为不变。
|
||||
- **BREAKING**：将 Cutout 的连续整数选项 **Mesh Sampling Spacing** 改为四档 **Mesh Detail**；Low、Medium、High、Ultra 分别使用 32、16、8、4 px 的采样间距。
|
||||
- 同步测试和内部文档，使术语、界面名称与当前实现一致。
|
||||
|
||||
## Capabilities
|
||||
|
||||
### New Capabilities
|
||||
|
||||
- `image-selection`: 规定图像工具和 Cutout 工具对选区的统一表达，以及选区细化、Normal Map 与 Cutout Mesh Detail 选项。
|
||||
|
||||
### Modified Capabilities
|
||||
|
||||
无。
|
||||
|
||||
## Impact
|
||||
|
||||
- 影响 `common` 的选区数据与 Viewport 交互、Image Tool、Cutout Tool、Server Job、属性定义、注册入口和相关文件布局。
|
||||
- 影响 Blender 内部 Operator ID、Job 类型、序列化参数与结果字段；旧名称不再可用。
|
||||
- 需要同步相关测试与 `docs/internals` 文档；不新增依赖，不改变选区栅格化、AI 细化、Normal Map 生成或既有四种网格采样密度。
|
||||
@@ -0,0 +1,37 @@
|
||||
## Purpose
|
||||
|
||||
统一 Image Tool 与 Cutout Tool 对图片选区的用户表达和外部协议，并以简洁一致的名称呈现选区细化、Normal Map 与 Cutout 网格细节选项。
|
||||
|
||||
## ADDED Requirements
|
||||
|
||||
### Requirement: Image selection uses one business term
|
||||
系统 SHALL 在 Image Tool、Cutout Tool 及其对外 Job 协议中使用 Selection 表示用户在图片上圈定并栅格化的范围。
|
||||
|
||||
#### Scenario: Selection passes through a tool workflow
|
||||
- **WHEN** 用户使用图片手势圈定范围并提交工具操作
|
||||
- **THEN** 工具状态、Job 参数和 Job 结果中表示该范围的字段使用 Selection 术语
|
||||
|
||||
### Requirement: Selection refinement option is named Refine Selection
|
||||
支持 AI 选区细化的工具 SHALL 将对应布尔选项显示为 **Refine Selection**，且开启后只细化当前 Selection 中的前景内容。
|
||||
|
||||
#### Scenario: User enables selection refinement
|
||||
- **WHEN** 用户在 Image Box、Image Lasso 或 Cutout 中开启 **Refine Selection**
|
||||
- **THEN** 系统使用 AI 将当前 Selection 收紧到其中的前景内容
|
||||
|
||||
### Requirement: Cutout normal option is named Normal Map
|
||||
Cutout Tool SHALL 将控制法线贴图生成的布尔选项显示为 **Normal Map**，并保持现有 Normal Map 生成行为。
|
||||
|
||||
#### Scenario: User enables Normal Map
|
||||
- **WHEN** 用户在 Cutout 中开启 **Normal Map** 并选择可执行该功能的 Shape
|
||||
- **THEN** 系统按该 Shape 的既定 Normal Space 生成并应用 Normal Map
|
||||
|
||||
### Requirement: Cutout mesh detail uses named presets
|
||||
Cutout Tool SHALL 使用 **Mesh Detail** 档位选择控制网格采样密度，并将 Low、Medium、High、Ultra 分别映射为 32、16、8、4 px 的采样间距。
|
||||
|
||||
#### Scenario: User chooses mesh detail
|
||||
- **WHEN** 用户选择任一 **Mesh Detail** 档位并创建 Cutout Shape
|
||||
- **THEN** 系统使用该档位对应的采样间距构建网格，且从 Low 到 Ultra 逐档增加采样密度
|
||||
|
||||
#### Scenario: User keeps the default mesh detail
|
||||
- **WHEN** 用户未修改 **Mesh Detail**
|
||||
- **THEN** 系统使用 Low 对应的 32 px 采样间距
|
||||
22
openspec/changes/unify-selection-terminology/tasks.md
Normal file
22
openspec/changes/unify-selection-terminology/tasks.md
Normal file
@@ -0,0 +1,22 @@
|
||||
## 1. 迁移 Selection 基础类型
|
||||
|
||||
- [x] 1.1 将 `common/region.py` 迁移为 `common/selection.py`，把图片选区类型、函数、参数与返回值统一重命名为 Selection，并运行 `uv run pytest tests/test_cutout_geometry.py tests/test_cutout_objects.py` 验证栅格化和几何行为不变
|
||||
- [x] 1.2 更新 `common/image.py` 与 `common/viewport.py` 的选区输入、手势基类、设置辅助函数和所有调用方，同时保留 Blender Viewport Region API 名称，并运行相关 image data、clipboard 和手势测试验证交互与投影行为
|
||||
|
||||
## 2. 迁移 Blender 工具
|
||||
|
||||
- [x] 2.1 将属性工厂、三个工具设置和 Operator 参数统一为 `refine_selection`，把显示名改为 **Refine Selection**，把 Cutout 法线选项显示名改为 **Normal Map**，并运行属性与工具设置测试验证名称、独立设置和 AI 门控
|
||||
- [x] 2.2 将 `cutout_point_spacing` 迁移为默认 Low 的 `cutout_mesh_detail` 四档枚举，在 Cutout 业务边界将 Low、Medium、High、Ultra 映射为 32、16、8、4 px，并用参数化测试验证全部映射及采样密度顺序
|
||||
- [x] 2.3 将 Image Tool 的 Selection 数据流、细化 Operator、Operator ID、Job 类型和注册入口迁移到新名称，并运行 Image Tool 与注册测试验证本地和 AI 两条路径
|
||||
- [x] 2.4 将 Cutout Tool 的 Selection Operator、交互传参、Mask/Contour 使用和结果读取迁移到新名称，并运行 `uv run pytest tests/test_cutout_interaction.py tests/test_cutout_geometry.py tests/test_cutout_objects.py` 验证 Shape 菜单、Mesh Detail 与对象构建行为
|
||||
|
||||
## 3. 迁移 Server 协议
|
||||
|
||||
- [x] 3.1 将 `server/region/` 与 Image Selection Job 文件迁移到 Selection 命名，更新 `refine-image-selection` 注册、参数、进度消息、结果字段和细化产物文件名，并运行 Server runtime 测试验证 Job 输出
|
||||
- [x] 3.2 同步 Blender 与 Server 两端的 `selection_image`、`selection_bounds` 和 `refine_selection` 协议，更新打包断言，并运行 `uv run pytest tests/test_server_runtime.py tests/test_extension_packaging.py tests/test_blender_addon.py` 验证跨进程契约和扩展内容
|
||||
|
||||
## 4. 同步文档与完整验证
|
||||
|
||||
- [x] 4.1 更新 `docs/internals`、getting started 与相关索引，将图片业务范围正面描述为 Selection，并确认 **Refine Selection**、**Normal Map**、**Mesh Detail** 档位映射和新 Job/产物协议一致
|
||||
- [x] 4.2 使用 `rg` 检查源码、测试和文档中的 `region`，逐项确认剩余结果仅为 Blender Viewport Region 或与图片选区无关的一般区域表达，且不存在旧模块、旧类型、旧 Operator、旧 Job、旧字段或兼容转发
|
||||
- [x] 4.3 运行 `uv run pytest`，确认全部测试通过且 Selection 重命名未改变现有功能
|
||||
@@ -1 +1 @@
|
||||
"""Shared Blender-side image, node, object, interaction, and AI Job helpers."""
|
||||
"""Shared Blender-side AI, image, selection, viewport, and result helpers."""
|
||||
|
||||
@@ -1,9 +1,17 @@
|
||||
"""Shared Blender-side AI setup, validation, UI, and Job parameters."""
|
||||
|
||||
from pathlib import Path
|
||||
|
||||
import bpy
|
||||
|
||||
from ..preferences import addon_preferences
|
||||
from ..properties import ai_ready, model_ready, model_record
|
||||
from ..properties import (
|
||||
ai_ready,
|
||||
ai_setup_label,
|
||||
ai_status,
|
||||
model_ready,
|
||||
model_record,
|
||||
)
|
||||
from ..runtime import runtime
|
||||
|
||||
BEN2_IMAGE_SUFFIXES = {".jpg", ".jpeg", ".png", ".webp"}
|
||||
@@ -12,6 +20,23 @@ BEN2_VIDEO_SUFFIXES = {
|
||||
}
|
||||
|
||||
|
||||
def draw_ai_setup(layout):
|
||||
status = ai_status()
|
||||
if not status["ready"]:
|
||||
layout.operator(
|
||||
"anyimage.setup_ai_environment",
|
||||
text=ai_setup_label(status),
|
||||
icon="IMPORT",
|
||||
)
|
||||
return status
|
||||
|
||||
|
||||
def draw_ai_property(layout, owner, name, status):
|
||||
row = layout.row()
|
||||
row.enabled = status["ready"]
|
||||
row.prop(owner, name)
|
||||
|
||||
|
||||
def require_input_path(value):
|
||||
path = Path(bpy.path.abspath(value))
|
||||
if path.is_file() or path.is_dir():
|
||||
@@ -5,7 +5,7 @@ from pathlib import Path
|
||||
|
||||
import bpy
|
||||
|
||||
from .image_data import depth_data_name, image_empty_bounds
|
||||
from .image import depth_data_name, image_empty_bounds
|
||||
|
||||
|
||||
def load_depth_metadata(path):
|
||||
@@ -1,3 +1,5 @@
|
||||
"""Shared Blender Image and Image Empty lifecycle helpers."""
|
||||
|
||||
import re
|
||||
import shutil
|
||||
import tempfile
|
||||
@@ -260,21 +262,21 @@ def prepare_image_input(image, scene):
|
||||
return source_path, False
|
||||
|
||||
|
||||
def prepare_region_input(image, region_mask):
|
||||
"""Write one cropped RGBA Region input owned by its temporary directory."""
|
||||
def prepare_selection_input(image, selection_mask):
|
||||
"""Write one cropped RGBA Selection input owned by its temporary directory."""
|
||||
import numpy as np
|
||||
|
||||
left, top, right, bottom = region_mask.bounds
|
||||
left, top, right, bottom = selection_mask.bounds
|
||||
rgba = image_rgba(image)
|
||||
cropped = rgba[top:bottom, left:right].copy()
|
||||
if cropped.shape[:2] != region_mask.values.shape:
|
||||
raise ValueError("The Region does not match the source image")
|
||||
cropped[:, :, 3] *= region_mask.values
|
||||
if cropped.shape[:2] != selection_mask.values.shape:
|
||||
raise ValueError("The Selection does not match the source image")
|
||||
cropped[:, :, 3] *= selection_mask.values
|
||||
cropped[cropped[:, :, 3] <= 0.0, :3] = 0.0
|
||||
output_directory = Path(tempfile.mkdtemp(prefix="anyimage-region-"))
|
||||
output_directory = Path(tempfile.mkdtemp(prefix="anyimage-selection-"))
|
||||
output_path = output_directory / "input.png"
|
||||
temporary_image = bpy.data.images.new(
|
||||
"AnyImage Region Input",
|
||||
"AnyImage Selection Input",
|
||||
width=right - left,
|
||||
height=bottom - top,
|
||||
alpha=True,
|
||||
@@ -294,7 +296,7 @@ def prepare_region_input(image, region_mask):
|
||||
bpy.data.images.remove(temporary_image, do_unlink=True)
|
||||
if not output_path.is_file():
|
||||
cleanup_image_input(output_path, True)
|
||||
raise RuntimeError("Blender did not export the Region input")
|
||||
raise RuntimeError("Blender did not export the Selection input")
|
||||
return output_path
|
||||
|
||||
|
||||
261
src/anyimage/common/material.py
Normal file
261
src/anyimage/common/material.py
Normal file
@@ -0,0 +1,261 @@
|
||||
"""Build and configure Blender materials for Image results."""
|
||||
|
||||
import bpy
|
||||
|
||||
from .image import (
|
||||
configure_image_user,
|
||||
image_base_name,
|
||||
image_pixels,
|
||||
is_animated_image,
|
||||
)
|
||||
from .node import load_node_group
|
||||
|
||||
|
||||
DISPLACEMENT_STRENGTH = 1.0
|
||||
DISPLACEMENT_MIDLEVEL = 0.0
|
||||
IMAGE_MATERIAL_NODE_GROUP_NAME = "O Image Layer"
|
||||
DEPTH_MATERIAL_NODE_GROUP_NAME = "O Image Depth Layer"
|
||||
SHADELESS_NODE_GROUP_NAME = "O Shadeless"
|
||||
VIEW_TRANSFORM_COLOR_SPACES = (
|
||||
("ACES 1.3", "ACES 1.3 sRGB"),
|
||||
("ACES 2.0", "ACES 2.0 sRGB"),
|
||||
("AgX", "AgX Base sRGB"),
|
||||
("Filmic", "Filmic sRGB"),
|
||||
("Khronos PBR Neutral", "Khronos PBR Neutral sRGB"),
|
||||
)
|
||||
|
||||
|
||||
def material_color_space_name(scene):
|
||||
view_settings = getattr(scene, "view_settings", None)
|
||||
view_transform = getattr(view_settings, "view_transform", "")
|
||||
normalized = str(view_transform).strip().casefold()
|
||||
for view_name, color_space in VIEW_TRANSFORM_COLOR_SPACES:
|
||||
if normalized.startswith(view_name.casefold()):
|
||||
return color_space
|
||||
return "sRGB"
|
||||
|
||||
|
||||
def set_material_color_space(image, scene=None):
|
||||
if scene is None:
|
||||
scene = getattr(bpy.context, "scene", None)
|
||||
settings = image.colorspace_settings
|
||||
preferred = material_color_space_name(scene)
|
||||
original = settings.name
|
||||
pixels = None
|
||||
if preferred != original and (
|
||||
getattr(image, "source", None) == "GENERATED"
|
||||
or getattr(image, "is_dirty", False)
|
||||
):
|
||||
pixels = image_pixels(image)
|
||||
for color_space in dict.fromkeys((preferred, "sRGB", original)):
|
||||
if not color_space:
|
||||
continue
|
||||
try:
|
||||
settings.name = color_space
|
||||
except (TypeError, ValueError):
|
||||
continue
|
||||
if settings.name == color_space:
|
||||
if pixels is not None:
|
||||
image.pixels.foreach_set(pixels)
|
||||
image.update()
|
||||
return color_space
|
||||
return settings.name
|
||||
|
||||
|
||||
def material_node_group(depth_plane=False):
|
||||
group_name = (
|
||||
DEPTH_MATERIAL_NODE_GROUP_NAME
|
||||
if depth_plane
|
||||
else IMAGE_MATERIAL_NODE_GROUP_NAME
|
||||
)
|
||||
return load_node_group(group_name)
|
||||
|
||||
|
||||
def create_image_material(
|
||||
source_object,
|
||||
color_image,
|
||||
depth_scale,
|
||||
displacement_image=None,
|
||||
displacement_midlevel=0.0,
|
||||
displacement_scale=1.0,
|
||||
shadeless=False,
|
||||
frame_count=1,
|
||||
frame_start=1,
|
||||
color_frame_offset=0,
|
||||
color_frame_duration=None,
|
||||
depth_frame_offset=0,
|
||||
depth_frame_duration=None,
|
||||
use_cyclic=None,
|
||||
normal_image=None,
|
||||
normal_space="TANGENT",
|
||||
ior=None,
|
||||
scene=None,
|
||||
):
|
||||
material = bpy.data.materials.new(image_base_name(source_object.data))
|
||||
material.use_nodes = True
|
||||
if hasattr(material, "surface_render_method"):
|
||||
material.surface_render_method = "DITHERED"
|
||||
nodes = material.node_tree.nodes
|
||||
nodes.clear()
|
||||
|
||||
color_texture = nodes.new("ShaderNodeTexImage")
|
||||
uv_map = nodes.new("ShaderNodeUVMap")
|
||||
output = nodes.new("ShaderNodeOutputMaterial")
|
||||
|
||||
uv_map.uv_map = "UVMap"
|
||||
uv_map.location = (-1250.0, 350.0)
|
||||
material_image = color_image
|
||||
set_material_color_space(material_image, scene)
|
||||
color_texture.image = material_image
|
||||
color_texture.interpolation = "Linear"
|
||||
color_texture.extension = "EXTEND"
|
||||
color_texture.location = (-1000.0, 350.0)
|
||||
output.location = (850.0, 350.0)
|
||||
extracted_frames = frame_count if frame_count > 1 else None
|
||||
color_frames = (
|
||||
color_frame_duration if color_frame_duration is not None else extracted_frames
|
||||
)
|
||||
depth_frames = (
|
||||
depth_frame_duration if depth_frame_duration is not None else extracted_frames
|
||||
)
|
||||
material_frames = color_frames
|
||||
material_frame_offset = color_frame_offset
|
||||
if frame_count > 1 or is_animated_image(material_image):
|
||||
configure_image_user(
|
||||
color_texture.image_user,
|
||||
material_frames,
|
||||
frame_start=frame_start,
|
||||
frame_offset=material_frame_offset,
|
||||
use_cyclic=use_cyclic,
|
||||
)
|
||||
|
||||
links = material.node_tree.links
|
||||
links.new(uv_map.outputs["UV"], color_texture.inputs["Vector"])
|
||||
alpha = color_texture.outputs["Alpha"]
|
||||
material_inputs = nodes.new("ShaderNodeGroup")
|
||||
is_depth_plane = displacement_image is not None
|
||||
material_inputs.node_tree = material_node_group(is_depth_plane)
|
||||
material_inputs.location = (-300.0, 350.0)
|
||||
material_inputs.inputs["Bump Scale"].default_value = depth_scale
|
||||
if "Object Space" in material_inputs.inputs:
|
||||
if normal_space not in {"TANGENT", "OBJECT"}:
|
||||
raise ValueError(f"Unsupported normal space: {normal_space}")
|
||||
material_inputs.inputs["Object Space"].default_value = normal_space == "OBJECT"
|
||||
links.new(color_texture.outputs["Color"], material_inputs.inputs["Color"])
|
||||
links.new(alpha, material_inputs.inputs["Alpha"])
|
||||
if normal_image is not None and "Normal" in material_inputs.inputs:
|
||||
normal_texture = nodes.new("ShaderNodeTexImage")
|
||||
normal_texture.image = normal_image
|
||||
normal_texture.interpolation = "Linear"
|
||||
normal_texture.extension = "EXTEND"
|
||||
normal_texture.location = (-1000.0, -50.0)
|
||||
links.new(uv_map.outputs["UV"], normal_texture.inputs["Vector"])
|
||||
links.new(normal_texture.outputs["Color"], material_inputs.inputs["Normal"])
|
||||
if shadeless:
|
||||
shader = nodes.new("ShaderNodeGroup")
|
||||
shader.node_tree = load_node_group(SHADELESS_NODE_GROUP_NAME)
|
||||
shader.name = SHADELESS_NODE_GROUP_NAME
|
||||
transparent = nodes.new("ShaderNodeBsdfTransparent")
|
||||
mix_shader = nodes.new("ShaderNodeMixShader")
|
||||
shader.location = (0.0, 550.0)
|
||||
transparent.location = (0.0, 150.0)
|
||||
mix_shader.location = (300.0, 350.0)
|
||||
links.new(
|
||||
material_inputs.outputs["Color"],
|
||||
shader.inputs["Color"],
|
||||
)
|
||||
links.new(
|
||||
material_inputs.outputs["Alpha"],
|
||||
mix_shader.inputs["Fac"],
|
||||
)
|
||||
links.new(
|
||||
transparent.outputs["BSDF"],
|
||||
mix_shader.inputs[1],
|
||||
)
|
||||
links.new(
|
||||
shader.outputs["Shader"],
|
||||
mix_shader.inputs[2],
|
||||
)
|
||||
links.new(
|
||||
mix_shader.outputs["Shader"],
|
||||
output.inputs["Surface"],
|
||||
)
|
||||
else:
|
||||
shader = nodes.new("ShaderNodeBsdfPrincipled")
|
||||
shader.location = (250.0, 400.0)
|
||||
if ior is not None:
|
||||
shader.inputs["IOR"].default_value = float(ior)
|
||||
|
||||
links.new(
|
||||
material_inputs.outputs["Color"],
|
||||
shader.inputs["Base Color"],
|
||||
)
|
||||
links.new(
|
||||
material_inputs.outputs["Color"],
|
||||
shader.inputs["Subsurface Radius"],
|
||||
)
|
||||
links.new(
|
||||
material_inputs.outputs["Color"],
|
||||
shader.inputs["Emission Color"],
|
||||
)
|
||||
links.new(material_inputs.outputs["Alpha"], shader.inputs["Alpha"])
|
||||
links.new(material_inputs.outputs["Normal"], shader.inputs["Normal"])
|
||||
links.new(
|
||||
material_inputs.outputs["Normal"],
|
||||
shader.inputs["Coat Normal"],
|
||||
)
|
||||
links.new(
|
||||
material_inputs.outputs["Roughness"],
|
||||
shader.inputs["Roughness"],
|
||||
)
|
||||
links.new(shader.outputs["BSDF"], output.inputs["Surface"])
|
||||
|
||||
if displacement_image is not None:
|
||||
depth_texture = nodes.new("ShaderNodeTexImage")
|
||||
depth_texture.image = displacement_image
|
||||
depth_texture.interpolation = "Linear"
|
||||
depth_texture.extension = "EXTEND"
|
||||
if frame_count > 1 or is_animated_image(displacement_image):
|
||||
configure_image_user(
|
||||
depth_texture.image_user,
|
||||
depth_frames,
|
||||
frame_start=frame_start,
|
||||
frame_offset=depth_frame_offset,
|
||||
use_cyclic=use_cyclic,
|
||||
)
|
||||
depth_texture.location = (-1000.0, -400.0)
|
||||
links.new(uv_map.outputs["UV"], depth_texture.inputs["Vector"])
|
||||
displacement_height = depth_texture.outputs["Color"]
|
||||
if displacement_midlevel != 0.0 or displacement_scale != 1.0:
|
||||
depth_adjust = nodes.new("ShaderNodeMath")
|
||||
depth_adjust.operation = "SUBTRACT"
|
||||
depth_adjust.inputs[1].default_value = displacement_midlevel
|
||||
depth_adjust.location = (500.0, -550.0)
|
||||
depth_scale_node = nodes.new("ShaderNodeMath")
|
||||
depth_scale_node.operation = "MULTIPLY"
|
||||
depth_scale_node.inputs[1].default_value = displacement_scale
|
||||
depth_scale_node.location = (700.0, -550.0)
|
||||
links.new(displacement_height, depth_adjust.inputs[0])
|
||||
links.new(depth_adjust.outputs[0], depth_scale_node.inputs[0])
|
||||
links.new(depth_scale_node.outputs[0], material_inputs.inputs["Depth"])
|
||||
else:
|
||||
links.new(displacement_height, material_inputs.inputs["Depth"])
|
||||
links.new(
|
||||
material_inputs.outputs["Displacement"],
|
||||
output.inputs["Displacement"],
|
||||
)
|
||||
set_displacement_only(material)
|
||||
return material
|
||||
|
||||
|
||||
def set_displacement_only(material):
|
||||
for owner, property_name in (
|
||||
(material, "surface_displacement_method"),
|
||||
(material, "displacement_method"),
|
||||
(getattr(material, "cycles", None), "displacement_method"),
|
||||
):
|
||||
if owner is not None and hasattr(owner, property_name):
|
||||
setattr(owner, property_name, "DISPLACEMENT")
|
||||
return
|
||||
|
||||
|
||||
@@ -1,12 +1,6 @@
|
||||
import bpy
|
||||
"""Shared Blender Object and Geometry Nodes Modifier helpers."""
|
||||
|
||||
from .image_data import (
|
||||
configure_image_user,
|
||||
image_base_name,
|
||||
image_pixels,
|
||||
is_animated_image,
|
||||
)
|
||||
from .node import load_node_group
|
||||
import bpy
|
||||
|
||||
|
||||
def modifier_input_identifier(node_group, name):
|
||||
@@ -30,265 +24,15 @@ def set_modifier_input(modifier, identifier, value):
|
||||
"""Assign a value to a Geometry Nodes modifier input socket."""
|
||||
inputs = _modifier_input_slots(modifier)
|
||||
if inputs is not None:
|
||||
# Blender 5.2+ exposes inputs as runtime RNA keyed by socket identifier.
|
||||
input_slot = getattr(inputs, identifier)
|
||||
value_property = input_slot.bl_rna.properties["value"]
|
||||
if value_property.type == "ENUM" and type(value) is int:
|
||||
value = value_property.enum_items[value].identifier
|
||||
input_slot.value = value
|
||||
else:
|
||||
# Blender 4.x exposes inputs as modifier ID properties.
|
||||
modifier[identifier] = value
|
||||
|
||||
|
||||
DISPLACEMENT_STRENGTH = 1.0
|
||||
DISPLACEMENT_MIDLEVEL = 0.0
|
||||
IMAGE_MATERIAL_NODE_GROUP_NAME = "O Image Layer"
|
||||
DEPTH_MATERIAL_NODE_GROUP_NAME = "O Image Depth Layer"
|
||||
SHADELESS_NODE_GROUP_NAME = "O Shadeless"
|
||||
VIEW_TRANSFORM_COLOR_SPACES = (
|
||||
("ACES 1.3", "ACES 1.3 sRGB"),
|
||||
("ACES 2.0", "ACES 2.0 sRGB"),
|
||||
("AgX", "AgX Base sRGB"),
|
||||
("Filmic", "Filmic sRGB"),
|
||||
("Khronos PBR Neutral", "Khronos PBR Neutral sRGB"),
|
||||
)
|
||||
|
||||
|
||||
def material_color_space_name(scene):
|
||||
view_settings = getattr(scene, "view_settings", None)
|
||||
view_transform = getattr(view_settings, "view_transform", "")
|
||||
normalized = str(view_transform).strip().casefold()
|
||||
for view_name, color_space in VIEW_TRANSFORM_COLOR_SPACES:
|
||||
if normalized.startswith(view_name.casefold()):
|
||||
return color_space
|
||||
return "sRGB"
|
||||
|
||||
|
||||
def set_material_color_space(image, scene=None):
|
||||
if scene is None:
|
||||
scene = getattr(bpy.context, "scene", None)
|
||||
settings = image.colorspace_settings
|
||||
preferred = material_color_space_name(scene)
|
||||
original = settings.name
|
||||
pixels = None
|
||||
if preferred != original and (
|
||||
getattr(image, "source", None) == "GENERATED"
|
||||
or getattr(image, "is_dirty", False)
|
||||
):
|
||||
pixels = image_pixels(image)
|
||||
for color_space in dict.fromkeys((preferred, "sRGB", original)):
|
||||
if not color_space:
|
||||
continue
|
||||
try:
|
||||
settings.name = color_space
|
||||
except (TypeError, ValueError):
|
||||
continue
|
||||
if settings.name == color_space:
|
||||
if pixels is not None:
|
||||
image.pixels.foreach_set(pixels)
|
||||
image.update()
|
||||
return color_space
|
||||
return settings.name
|
||||
|
||||
|
||||
def material_node_group(depth_plane=False):
|
||||
group_name = (
|
||||
DEPTH_MATERIAL_NODE_GROUP_NAME
|
||||
if depth_plane
|
||||
else IMAGE_MATERIAL_NODE_GROUP_NAME
|
||||
)
|
||||
return load_node_group(group_name)
|
||||
|
||||
|
||||
def create_image_material(
|
||||
source_object,
|
||||
color_image,
|
||||
depth_scale,
|
||||
displacement_image=None,
|
||||
displacement_midlevel=0.0,
|
||||
displacement_scale=1.0,
|
||||
shadeless=False,
|
||||
frame_count=1,
|
||||
frame_start=1,
|
||||
color_frame_offset=0,
|
||||
color_frame_duration=None,
|
||||
depth_frame_offset=0,
|
||||
depth_frame_duration=None,
|
||||
use_cyclic=None,
|
||||
normal_image=None,
|
||||
normal_space="TANGENT",
|
||||
ior=None,
|
||||
scene=None,
|
||||
):
|
||||
material = bpy.data.materials.new(image_base_name(source_object.data))
|
||||
material.use_nodes = True
|
||||
if hasattr(material, "surface_render_method"):
|
||||
material.surface_render_method = "DITHERED"
|
||||
nodes = material.node_tree.nodes
|
||||
nodes.clear()
|
||||
|
||||
color_texture = nodes.new("ShaderNodeTexImage")
|
||||
uv_map = nodes.new("ShaderNodeUVMap")
|
||||
output = nodes.new("ShaderNodeOutputMaterial")
|
||||
|
||||
uv_map.uv_map = "UVMap"
|
||||
uv_map.location = (-1250.0, 350.0)
|
||||
material_image = color_image
|
||||
set_material_color_space(material_image, scene)
|
||||
color_texture.image = material_image
|
||||
color_texture.interpolation = "Linear"
|
||||
color_texture.extension = "EXTEND"
|
||||
color_texture.location = (-1000.0, 350.0)
|
||||
output.location = (850.0, 350.0)
|
||||
extracted_frames = frame_count if frame_count > 1 else None
|
||||
color_frames = (
|
||||
color_frame_duration if color_frame_duration is not None else extracted_frames
|
||||
)
|
||||
depth_frames = (
|
||||
depth_frame_duration if depth_frame_duration is not None else extracted_frames
|
||||
)
|
||||
material_frames = color_frames
|
||||
material_frame_offset = color_frame_offset
|
||||
if frame_count > 1 or is_animated_image(material_image):
|
||||
configure_image_user(
|
||||
color_texture.image_user,
|
||||
material_frames,
|
||||
frame_start=frame_start,
|
||||
frame_offset=material_frame_offset,
|
||||
use_cyclic=use_cyclic,
|
||||
)
|
||||
|
||||
links = material.node_tree.links
|
||||
links.new(uv_map.outputs["UV"], color_texture.inputs["Vector"])
|
||||
alpha = color_texture.outputs["Alpha"]
|
||||
material_inputs = nodes.new("ShaderNodeGroup")
|
||||
is_depth_plane = displacement_image is not None
|
||||
material_inputs.node_tree = material_node_group(is_depth_plane)
|
||||
material_inputs.location = (-300.0, 350.0)
|
||||
material_inputs.inputs["Bump Scale"].default_value = depth_scale
|
||||
if "Object Space" in material_inputs.inputs:
|
||||
if normal_space not in {"TANGENT", "OBJECT"}:
|
||||
raise ValueError(f"Unsupported normal space: {normal_space}")
|
||||
material_inputs.inputs["Object Space"].default_value = normal_space == "OBJECT"
|
||||
links.new(color_texture.outputs["Color"], material_inputs.inputs["Color"])
|
||||
links.new(alpha, material_inputs.inputs["Alpha"])
|
||||
if normal_image is not None and "Normal" in material_inputs.inputs:
|
||||
normal_texture = nodes.new("ShaderNodeTexImage")
|
||||
normal_texture.image = normal_image
|
||||
normal_texture.interpolation = "Linear"
|
||||
normal_texture.extension = "EXTEND"
|
||||
normal_texture.location = (-1000.0, -50.0)
|
||||
links.new(uv_map.outputs["UV"], normal_texture.inputs["Vector"])
|
||||
links.new(normal_texture.outputs["Color"], material_inputs.inputs["Normal"])
|
||||
if shadeless:
|
||||
shader = nodes.new("ShaderNodeGroup")
|
||||
shader.node_tree = load_node_group(SHADELESS_NODE_GROUP_NAME)
|
||||
shader.name = SHADELESS_NODE_GROUP_NAME
|
||||
transparent = nodes.new("ShaderNodeBsdfTransparent")
|
||||
mix_shader = nodes.new("ShaderNodeMixShader")
|
||||
shader.location = (0.0, 550.0)
|
||||
transparent.location = (0.0, 150.0)
|
||||
mix_shader.location = (300.0, 350.0)
|
||||
links.new(
|
||||
material_inputs.outputs["Color"],
|
||||
shader.inputs["Color"],
|
||||
)
|
||||
links.new(
|
||||
material_inputs.outputs["Alpha"],
|
||||
mix_shader.inputs["Fac"],
|
||||
)
|
||||
links.new(
|
||||
transparent.outputs["BSDF"],
|
||||
mix_shader.inputs[1],
|
||||
)
|
||||
links.new(
|
||||
shader.outputs["Shader"],
|
||||
mix_shader.inputs[2],
|
||||
)
|
||||
links.new(
|
||||
mix_shader.outputs["Shader"],
|
||||
output.inputs["Surface"],
|
||||
)
|
||||
else:
|
||||
shader = nodes.new("ShaderNodeBsdfPrincipled")
|
||||
shader.location = (250.0, 400.0)
|
||||
if ior is not None:
|
||||
shader.inputs["IOR"].default_value = float(ior)
|
||||
|
||||
links.new(
|
||||
material_inputs.outputs["Color"],
|
||||
shader.inputs["Base Color"],
|
||||
)
|
||||
links.new(
|
||||
material_inputs.outputs["Color"],
|
||||
shader.inputs["Subsurface Radius"],
|
||||
)
|
||||
links.new(
|
||||
material_inputs.outputs["Color"],
|
||||
shader.inputs["Emission Color"],
|
||||
)
|
||||
links.new(material_inputs.outputs["Alpha"], shader.inputs["Alpha"])
|
||||
links.new(material_inputs.outputs["Normal"], shader.inputs["Normal"])
|
||||
links.new(
|
||||
material_inputs.outputs["Normal"],
|
||||
shader.inputs["Coat Normal"],
|
||||
)
|
||||
links.new(
|
||||
material_inputs.outputs["Roughness"],
|
||||
shader.inputs["Roughness"],
|
||||
)
|
||||
links.new(shader.outputs["BSDF"], output.inputs["Surface"])
|
||||
|
||||
if displacement_image is not None:
|
||||
depth_texture = nodes.new("ShaderNodeTexImage")
|
||||
depth_texture.image = displacement_image
|
||||
depth_texture.interpolation = "Linear"
|
||||
depth_texture.extension = "EXTEND"
|
||||
if frame_count > 1 or is_animated_image(displacement_image):
|
||||
configure_image_user(
|
||||
depth_texture.image_user,
|
||||
depth_frames,
|
||||
frame_start=frame_start,
|
||||
frame_offset=depth_frame_offset,
|
||||
use_cyclic=use_cyclic,
|
||||
)
|
||||
depth_texture.location = (-1000.0, -400.0)
|
||||
links.new(uv_map.outputs["UV"], depth_texture.inputs["Vector"])
|
||||
displacement_height = depth_texture.outputs["Color"]
|
||||
if displacement_midlevel != 0.0 or displacement_scale != 1.0:
|
||||
depth_adjust = nodes.new("ShaderNodeMath")
|
||||
depth_adjust.operation = "SUBTRACT"
|
||||
depth_adjust.inputs[1].default_value = displacement_midlevel
|
||||
depth_adjust.location = (500.0, -550.0)
|
||||
depth_scale_node = nodes.new("ShaderNodeMath")
|
||||
depth_scale_node.operation = "MULTIPLY"
|
||||
depth_scale_node.inputs[1].default_value = displacement_scale
|
||||
depth_scale_node.location = (700.0, -550.0)
|
||||
links.new(displacement_height, depth_adjust.inputs[0])
|
||||
links.new(depth_adjust.outputs[0], depth_scale_node.inputs[0])
|
||||
links.new(depth_scale_node.outputs[0], material_inputs.inputs["Depth"])
|
||||
else:
|
||||
links.new(displacement_height, material_inputs.inputs["Depth"])
|
||||
links.new(
|
||||
material_inputs.outputs["Displacement"],
|
||||
output.inputs["Displacement"],
|
||||
)
|
||||
set_displacement_only(material)
|
||||
return material
|
||||
|
||||
|
||||
def set_displacement_only(material):
|
||||
for owner, property_name in (
|
||||
(material, "surface_displacement_method"),
|
||||
(material, "displacement_method"),
|
||||
(getattr(material, "cycles", None), "displacement_method"),
|
||||
):
|
||||
if owner is not None and hasattr(owner, property_name):
|
||||
setattr(owner, property_name, "DISPLACEMENT")
|
||||
return
|
||||
|
||||
|
||||
def finalize_object_result(
|
||||
context,
|
||||
source_object,
|
||||
|
||||
@@ -1,3 +1,5 @@
|
||||
"""Selection values, serialization, rasterization, and Alpha cropping."""
|
||||
|
||||
import json
|
||||
import math
|
||||
from dataclasses import dataclass
|
||||
@@ -8,7 +10,7 @@ class ImageEditWarning(ValueError):
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class RegionPath:
|
||||
class SelectionPath:
|
||||
points: tuple[tuple[float, float], ...]
|
||||
invert: bool = False
|
||||
fill_rule: str = "EVEN_ODD"
|
||||
@@ -18,9 +20,9 @@ class RegionPath:
|
||||
(float(point[0]), float(point[1]))
|
||||
for point in self.points
|
||||
)
|
||||
region_path_bounds(points)
|
||||
selection_path_bounds(points)
|
||||
if self.fill_rule not in {"EVEN_ODD", "NONZERO"}:
|
||||
raise ValueError(f"Unknown Region Path fill rule: {self.fill_rule}")
|
||||
raise ValueError(f"Unknown Selection Path fill rule: {self.fill_rule}")
|
||||
object.__setattr__(self, "points", points)
|
||||
object.__setattr__(self, "invert", bool(self.invert))
|
||||
|
||||
@@ -38,7 +40,7 @@ class RegionPath:
|
||||
def from_json(cls, value):
|
||||
data = json.loads(value)
|
||||
if not isinstance(data, dict):
|
||||
raise ValueError("The Region Path data is invalid")
|
||||
raise ValueError("The Selection Path data is invalid")
|
||||
return cls(
|
||||
points=tuple(data.get("points", ())),
|
||||
invert=bool(data.get("invert", False)),
|
||||
@@ -47,7 +49,7 @@ class RegionPath:
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class RegionMask:
|
||||
class SelectionMask:
|
||||
values: object
|
||||
bounds: tuple[int, int, int, int]
|
||||
|
||||
@@ -57,16 +59,16 @@ class RegionMask:
|
||||
values = np.asarray(self.values, dtype=np.float32)
|
||||
bounds = tuple(int(value) for value in self.bounds)
|
||||
if values.ndim != 2:
|
||||
raise ValueError("The region mask must be two-dimensional")
|
||||
raise ValueError("The selection mask must be two-dimensional")
|
||||
if len(bounds) != 4:
|
||||
raise ValueError("The region bounds are invalid")
|
||||
raise ValueError("The selection bounds are invalid")
|
||||
left, top, right, bottom = bounds
|
||||
if right <= left or bottom <= top:
|
||||
raise ValueError("The region bounds are empty")
|
||||
raise ValueError("The selection bounds are empty")
|
||||
if values.shape != (bottom - top, right - left):
|
||||
raise ValueError("The region mask does not match its bounds")
|
||||
raise ValueError("The selection mask does not match its bounds")
|
||||
if not np.isfinite(values).all():
|
||||
raise ValueError("The region mask contains invalid values")
|
||||
raise ValueError("The selection mask contains invalid values")
|
||||
object.__setattr__(self, "values", np.clip(values, 0.0, 1.0))
|
||||
object.__setattr__(self, "bounds", bounds)
|
||||
|
||||
@@ -89,20 +91,20 @@ class RegionMask:
|
||||
return result
|
||||
|
||||
|
||||
def region_path_bounds(points):
|
||||
def selection_path_bounds(points):
|
||||
import numpy as np
|
||||
|
||||
points = np.asarray(points, dtype=np.float64)
|
||||
if points.ndim != 2 or points.shape[0] < 3 or points.shape[1] != 2:
|
||||
raise ValueError("The region does not define a valid image path")
|
||||
raise ValueError("The selection does not define a valid image path")
|
||||
if not np.isfinite(points).all():
|
||||
raise ValueError("The region contains invalid image coordinates")
|
||||
raise ValueError("The selection contains invalid image coordinates")
|
||||
left = int(math.floor(float(points[:, 0].min())))
|
||||
top = int(math.floor(float(points[:, 1].min())))
|
||||
right = int(math.ceil(float(points[:, 0].max())))
|
||||
bottom = int(math.ceil(float(points[:, 1].max())))
|
||||
if right <= left or bottom <= top:
|
||||
raise ValueError("The region does not define a valid image area")
|
||||
raise ValueError("The selection does not define a valid image area")
|
||||
return left, top, right, bottom
|
||||
|
||||
|
||||
@@ -112,7 +114,7 @@ def _rasterize_path(image_size, points, fill_rule):
|
||||
width, height = (int(value) for value in image_size)
|
||||
if width <= 0 or height <= 0:
|
||||
raise ValueError("The source image has no readable dimensions")
|
||||
left, top, right, bottom = region_path_bounds(points)
|
||||
left, top, right, bottom = selection_path_bounds(points)
|
||||
left = max(left, 0)
|
||||
top = max(top, 0)
|
||||
right = min(right, width)
|
||||
@@ -164,9 +166,9 @@ def _antialias_mask(mask):
|
||||
return result
|
||||
|
||||
|
||||
def rasterize_region_path(
|
||||
def rasterize_selection_path(
|
||||
image_size,
|
||||
region_path,
|
||||
selection_path,
|
||||
*,
|
||||
source_alpha=None,
|
||||
alpha_threshold=0.0,
|
||||
@@ -177,23 +179,23 @@ def rasterize_region_path(
|
||||
width, height = (int(value) for value in image_size)
|
||||
mask = _rasterize_path(
|
||||
(width, height),
|
||||
region_path.points,
|
||||
region_path.fill_rule,
|
||||
selection_path.points,
|
||||
selection_path.fill_rule,
|
||||
)
|
||||
if antialias:
|
||||
mask = _antialias_mask(mask)
|
||||
else:
|
||||
mask = mask.astype(np.float32)
|
||||
if region_path.invert:
|
||||
if selection_path.invert:
|
||||
mask = 1.0 - mask
|
||||
if source_alpha is not None:
|
||||
alpha = np.asarray(source_alpha)
|
||||
if alpha.shape != (height, width):
|
||||
raise ValueError("The source Alpha does not match the image region")
|
||||
raise ValueError("The source Alpha does not match the image selection")
|
||||
mask = np.where(alpha > float(alpha_threshold), mask, 0.0)
|
||||
visible_y, visible_x = np.nonzero(mask > float(alpha_threshold))
|
||||
if not len(visible_x):
|
||||
raise ImageEditWarning("The region contains no visible image pixels")
|
||||
raise ImageEditWarning("The selection contains no visible image pixels")
|
||||
bounds = (
|
||||
int(visible_x.min()),
|
||||
int(visible_y.min()),
|
||||
@@ -201,7 +203,7 @@ def rasterize_region_path(
|
||||
int(visible_y.max()) + 1,
|
||||
)
|
||||
left, top, right, bottom = bounds
|
||||
return RegionMask(mask[top:bottom, left:right], bounds)
|
||||
return SelectionMask(mask[top:bottom, left:right], bounds)
|
||||
|
||||
|
||||
def crop_pixels_to_alpha(pixels, bounds):
|
||||
@@ -1,16 +1,19 @@
|
||||
"""View3D projection, gestures, tool settings, and overlay drawing."""
|
||||
|
||||
import json
|
||||
import math
|
||||
|
||||
import bpy
|
||||
|
||||
from ..properties import ai_ready, ai_setup_label, ai_status
|
||||
from .image_data import (
|
||||
from ..properties import ai_ready
|
||||
from .ai import draw_ai_property, draw_ai_setup
|
||||
from .image import (
|
||||
image_empty_bounds,
|
||||
is_image_empty,
|
||||
require_image_empty,
|
||||
video_total_frames,
|
||||
)
|
||||
from .image_region import ImageEditWarning, RegionPath
|
||||
from .selection import ImageEditWarning, SelectionPath
|
||||
|
||||
MIN_PATH_DISTANCE = 2.0
|
||||
|
||||
@@ -148,46 +151,29 @@ def active_view3d_tool_id(context):
|
||||
return getattr(tool, "idname", None)
|
||||
|
||||
|
||||
def refine_region_setting_name(operator):
|
||||
setting_names = getattr(operator, "refine_region_settings", {})
|
||||
def refine_selection_setting_name(operator):
|
||||
setting_names = getattr(operator, "refine_selection_settings", {})
|
||||
return setting_names.get(operator.gesture)
|
||||
|
||||
|
||||
def apply_region_settings(operator, context):
|
||||
def apply_selection_settings(operator, context):
|
||||
settings = context.scene.anyimage_settings
|
||||
setting_name = refine_region_setting_name(operator)
|
||||
operator.refine_region = (
|
||||
setting_name = refine_selection_setting_name(operator)
|
||||
operator.refine_selection = (
|
||||
setting_name is not None and ai_ready() and getattr(settings, setting_name)
|
||||
)
|
||||
|
||||
|
||||
def draw_ai_setup(layout):
|
||||
status = ai_status()
|
||||
if not status["ready"]:
|
||||
layout.operator(
|
||||
"anyimage.setup_ai_environment",
|
||||
text=ai_setup_label(status),
|
||||
icon="IMPORT",
|
||||
)
|
||||
return status
|
||||
|
||||
|
||||
def draw_ai_property(layout, owner, name, status):
|
||||
row = layout.row()
|
||||
row.enabled = status["ready"]
|
||||
row.prop(owner, name)
|
||||
|
||||
|
||||
def draw_region_operator_settings(layout, operator):
|
||||
def draw_selection_operator_settings(layout, operator):
|
||||
if operator.gesture not in {"BOX", "LASSO"}:
|
||||
layout.prop(operator, "invert")
|
||||
return
|
||||
status = draw_ai_setup(layout)
|
||||
draw_ai_property(layout, operator, "refine_region", status)
|
||||
draw_ai_property(layout, operator, "refine_selection", status)
|
||||
layout.prop(operator, "invert")
|
||||
|
||||
|
||||
def draw_region_settings(context, layout, setting_name):
|
||||
def draw_selection_settings(context, layout, setting_name):
|
||||
settings = context.scene.anyimage_settings
|
||||
status = draw_ai_setup(layout)
|
||||
draw_ai_property(layout, settings, setting_name, status)
|
||||
@@ -621,7 +607,7 @@ def image_line_half_plane_polygon(line, image_size):
|
||||
return line_half_plane_polygon(line[1], line[0], image_size)
|
||||
|
||||
|
||||
class ImageRegionGesture:
|
||||
class ImageSelectionGesture:
|
||||
active_tool_id = None
|
||||
preview_fill_rule = "EVEN_ODD"
|
||||
_path = None
|
||||
@@ -636,7 +622,7 @@ class ImageRegionGesture:
|
||||
_region_size = (0, 0)
|
||||
|
||||
def draw(self, _context):
|
||||
draw_region_operator_settings(self.layout, self)
|
||||
draw_selection_operator_settings(self.layout, self)
|
||||
|
||||
@classmethod
|
||||
def poll(cls, context):
|
||||
@@ -656,7 +642,7 @@ class ImageRegionGesture:
|
||||
vertices.append(self._cursor)
|
||||
return tuple(vertices)
|
||||
|
||||
def _draw_region(self):
|
||||
def _draw_selection(self):
|
||||
if not self._path or not drawing_in_region(
|
||||
self._area_pointer, self._region_pointer
|
||||
):
|
||||
@@ -750,11 +736,11 @@ class ImageRegionGesture:
|
||||
if video_total_frames(source_object.data) > 1:
|
||||
self.report({"WARNING"}, "Image tools only support a still image")
|
||||
return {"CANCELLED"}
|
||||
apply_region_settings(self, context)
|
||||
apply_selection_settings(self, context)
|
||||
self.invert = False
|
||||
self.source_object_name = source_object.name
|
||||
self.source_matrix_data = serialize_matrix(source_object.matrix_world)
|
||||
self.region_path_json = ""
|
||||
self.selection_path_json = ""
|
||||
point = (float(event.mouse_region_x), float(event.mouse_region_y))
|
||||
self._path = [point]
|
||||
self._cursor = point
|
||||
@@ -766,7 +752,7 @@ class ImageRegionGesture:
|
||||
self._region_pointer = context.region.as_pointer()
|
||||
self._region_size = (context.region.width, context.region.height)
|
||||
self._handle = bpy.types.SpaceView3D.draw_handler_add(
|
||||
self._draw_region, (), "WINDOW", "POST_PIXEL"
|
||||
self._draw_selection, (), "WINDOW", "POST_PIXEL"
|
||||
)
|
||||
self._set_status(context)
|
||||
context.window_manager.modal_handler_add(self)
|
||||
@@ -785,7 +771,7 @@ class ImageRegionGesture:
|
||||
image_empty_bounds(source_object),
|
||||
tuple(source_object.data.size),
|
||||
)
|
||||
region_path = image_line_half_plane_polygon(
|
||||
selection_path = image_line_half_plane_polygon(
|
||||
line, tuple(source_object.data.size)
|
||||
)
|
||||
else:
|
||||
@@ -793,7 +779,7 @@ class ImageRegionGesture:
|
||||
if len(path) < 3:
|
||||
self._finish(context)
|
||||
return {"CANCELLED"}
|
||||
region_path = screen_path_to_image_pixels(
|
||||
selection_path = screen_path_to_image_pixels(
|
||||
path,
|
||||
context.region,
|
||||
context.region_data,
|
||||
@@ -801,8 +787,8 @@ class ImageRegionGesture:
|
||||
image_empty_bounds(source_object),
|
||||
tuple(source_object.data.size),
|
||||
)
|
||||
self.region_path_json = RegionPath(
|
||||
points=region_path,
|
||||
self.selection_path_json = SelectionPath(
|
||||
points=selection_path,
|
||||
invert=self.invert,
|
||||
fill_rule=self.preview_fill_rule,
|
||||
).to_json()
|
||||
@@ -1,6 +1,6 @@
|
||||
import bpy
|
||||
|
||||
from .common.image_data import is_image_empty
|
||||
from .common.image import is_image_empty
|
||||
from .operators.upscale import upscale_output_size
|
||||
|
||||
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
from .remove_background import RemoveImageBackground, RunBackgroundRemoval
|
||||
from .cutout_tool import (
|
||||
CutoutRegionToShape,
|
||||
SelectCutoutRegion,
|
||||
CutoutSelectionToShape,
|
||||
SelectCutoutSelection,
|
||||
)
|
||||
from .debug import DebugBen2, DebugDa3, DebugMoge2, DebugUpscale
|
||||
from .image_tool import CLASSES as IMAGE_TOOL_CLASSES
|
||||
@@ -31,8 +31,8 @@ CLASSES = (
|
||||
DebugBen2,
|
||||
DebugUpscale,
|
||||
GenerateDepthPlane,
|
||||
CutoutRegionToShape,
|
||||
SelectCutoutRegion,
|
||||
CutoutSelectionToShape,
|
||||
SelectCutoutSelection,
|
||||
RunBackgroundRemoval,
|
||||
ClearModels,
|
||||
*IMAGE_TOOL_CLASSES,
|
||||
|
||||
@@ -8,7 +8,7 @@ from ..properties import (
|
||||
model_download_size,
|
||||
model_record,
|
||||
)
|
||||
from ..common.ai_job import require_environment
|
||||
from ..common.ai import require_environment
|
||||
|
||||
|
||||
def format_download_size(size):
|
||||
|
||||
@@ -8,6 +8,7 @@ import bpy
|
||||
CLIPBOARD_HASH_PROPERTY = "anyimage_clipboard_sha256"
|
||||
CLIPBOARD_TEXTURE_PROPERTY = "anyimage_clipboard_texture"
|
||||
DEFAULT_STENCIL_SIZE = 256.0
|
||||
REFERENCE_IMAGE_SIZE = 5.0
|
||||
NODE_HEADER_HEIGHT = 20.0
|
||||
NODE_TYPES = {
|
||||
"CompositorNodeTree": "CompositorNodeImage",
|
||||
@@ -173,7 +174,7 @@ def add_reference_image(context, image, location=None):
|
||||
reference = bpy.data.objects.new(image.name, None)
|
||||
reference.empty_display_type = "IMAGE"
|
||||
reference.data = image
|
||||
reference.empty_display_size = 2.0
|
||||
reference.empty_display_size = REFERENCE_IMAGE_SIZE
|
||||
context.collection.objects.link(reference)
|
||||
_place_in_view(context, reference, location)
|
||||
_select_only(context, reference)
|
||||
|
||||
@@ -8,7 +8,7 @@ from ...server.media.resolution import (
|
||||
MAX_MAX_AI_INPUT_SIZE,
|
||||
MIN_MAX_AI_INPUT_SIZE,
|
||||
)
|
||||
from ...common.ai_job import (
|
||||
from ...common.ai import (
|
||||
invoke_ai_setup_if_needed,
|
||||
moge2_parameters,
|
||||
production_device,
|
||||
@@ -16,7 +16,7 @@ from ...common.ai_job import (
|
||||
require_input_path,
|
||||
require_model,
|
||||
)
|
||||
from ...common.image_data import (
|
||||
from ...common.image import (
|
||||
cleanup_image_input,
|
||||
image_user_settings,
|
||||
is_image_empty,
|
||||
@@ -24,7 +24,7 @@ from ...common.image_data import (
|
||||
source_start_frame,
|
||||
video_total_frames,
|
||||
)
|
||||
from ...common.object import create_image_material
|
||||
from ...common.material import create_image_material
|
||||
from .object import (
|
||||
create_color_image,
|
||||
create_depth_plane_from_result,
|
||||
|
||||
@@ -2,20 +2,20 @@
|
||||
|
||||
import bpy
|
||||
|
||||
from ...common.image_data import (
|
||||
from ...common.image import (
|
||||
color_data_name,
|
||||
image_empty_bounds,
|
||||
image_pixels,
|
||||
load_color_result_image,
|
||||
require_image_empty,
|
||||
)
|
||||
from ...common.depth_data import (
|
||||
from ...common.depth import (
|
||||
depth_uniform_scale,
|
||||
load_depth_metadata,
|
||||
load_depth_result_image,
|
||||
)
|
||||
from ...common.material import create_image_material
|
||||
from ...common.object import (
|
||||
create_image_material,
|
||||
finalize_object_result,
|
||||
modifier_input_identifier,
|
||||
set_modifier_input,
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
from .main import CutoutRegionToShape, CutoutTool, SelectCutoutRegion
|
||||
from .main import CutoutSelectionToShape, CutoutTool, SelectCutoutSelection
|
||||
|
||||
__all__ = (
|
||||
"CutoutRegionToShape",
|
||||
"CutoutSelectionToShape",
|
||||
"CutoutTool",
|
||||
"SelectCutoutRegion",
|
||||
"SelectCutoutSelection",
|
||||
)
|
||||
|
||||
@@ -5,11 +5,11 @@ from dataclasses import dataclass
|
||||
|
||||
from ...preferences import addon_preferences
|
||||
from ...server.media.resolution import limited_image_size
|
||||
from ...common.image_data import image_empty_bounds
|
||||
from ...common.image import image_empty_bounds
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class RegionContours:
|
||||
class SelectionContours:
|
||||
components: tuple
|
||||
image_size: tuple[int, int]
|
||||
|
||||
@@ -22,11 +22,11 @@ class RegionContours:
|
||||
)
|
||||
image_size = tuple(int(value) for value in self.image_size)
|
||||
if not components or any(not component for component in components):
|
||||
raise ValueError("The Region has no usable contour components")
|
||||
raise ValueError("The Selection has no usable contour components")
|
||||
if any(len(contour) < 3 for component in components for contour in component):
|
||||
raise ValueError("A Region contour has fewer than three points")
|
||||
raise ValueError("A Selection contour has fewer than three points")
|
||||
if len(image_size) != 2 or min(image_size) <= 0:
|
||||
raise ValueError("The Region contour image size is invalid")
|
||||
raise ValueError("The Selection contour image size is invalid")
|
||||
object.__setattr__(self, "components", components)
|
||||
object.__setattr__(self, "image_size", image_size)
|
||||
|
||||
@@ -57,7 +57,7 @@ def _simplify_axis_aligned_contour(points):
|
||||
return simplified
|
||||
|
||||
|
||||
def _trace_region_contours(mask):
|
||||
def _trace_selection_contours(mask):
|
||||
import numpy as np
|
||||
|
||||
mask = np.asarray(mask, dtype=bool)
|
||||
@@ -89,7 +89,7 @@ def _trace_region_contours(mask):
|
||||
for y, x in np.argwhere(left):
|
||||
add_edge((int(x), int(y) + 1), (int(x), int(y)))
|
||||
if not edges:
|
||||
raise ValueError("The Cutout region has no usable boundary")
|
||||
raise ValueError("The Cutout selection has no usable boundary")
|
||||
|
||||
contours = []
|
||||
edge_count = sum(len(values) for values in edges.values())
|
||||
@@ -117,7 +117,7 @@ def _trace_region_contours(mask):
|
||||
if len(contour) >= 3:
|
||||
contours.append(contour)
|
||||
if not contours:
|
||||
raise ValueError("The Cutout region is too small")
|
||||
raise ValueError("The Cutout selection is too small")
|
||||
return contours
|
||||
|
||||
|
||||
@@ -143,13 +143,13 @@ def _point_inside_contour(point, contour):
|
||||
return inside
|
||||
|
||||
|
||||
def _group_region_contours(mask):
|
||||
contours = _trace_region_contours(mask)
|
||||
def _group_selection_contours(mask):
|
||||
contours = _trace_selection_contours(mask)
|
||||
outer = [contour for contour in contours if _signed_contour_area(contour) > 0.0]
|
||||
holes = [contour for contour in contours if _signed_contour_area(contour) < 0.0]
|
||||
if not outer:
|
||||
raise ValueError("The Cutout region has no outer boundary")
|
||||
regions = [[contour] for contour in outer]
|
||||
raise ValueError("The Cutout selection has no outer boundary")
|
||||
components = [[contour] for contour in outer]
|
||||
outer_areas = [abs(_signed_contour_area(contour)) for contour in outer]
|
||||
for hole in holes:
|
||||
start = hole[0]
|
||||
@@ -167,10 +167,10 @@ def _group_region_contours(mask):
|
||||
if _point_inside_contour(sample, contour)
|
||||
]
|
||||
if not owners:
|
||||
raise ValueError("The Cutout hole has no enclosing region")
|
||||
raise ValueError("The Cutout hole has no enclosing selection component")
|
||||
owner = min(owners, key=outer_areas.__getitem__)
|
||||
regions[owner].append(hole)
|
||||
return regions
|
||||
components[owner].append(hole)
|
||||
return components
|
||||
|
||||
|
||||
def _resize_values(values, size):
|
||||
@@ -202,10 +202,10 @@ def _resize_values(values, size):
|
||||
)
|
||||
|
||||
|
||||
def extract_region_contours(region_mask, max_resolution=512, threshold=0.5):
|
||||
def extract_selection_contours(selection_mask, max_resolution=512, threshold=0.5):
|
||||
import numpy as np
|
||||
|
||||
values = np.asarray(region_mask.values)
|
||||
values = np.asarray(selection_mask.values)
|
||||
if values.ndim != 2:
|
||||
raise ValueError("Cutout Alpha must be two-dimensional")
|
||||
if np.issubdtype(values.dtype, np.floating):
|
||||
@@ -218,10 +218,10 @@ def extract_region_contours(region_mask, max_resolution=512, threshold=0.5):
|
||||
if sampled_size != source_size:
|
||||
values = _resize_values(values, sampled_size)
|
||||
mask = values >= 255.0 * threshold
|
||||
components = _group_region_contours(mask)
|
||||
components = _group_selection_contours(mask)
|
||||
scale_x = source_size[0] / sampled_size[0]
|
||||
scale_y = source_size[1] / sampled_size[1]
|
||||
return RegionContours(
|
||||
return SelectionContours(
|
||||
components=tuple(
|
||||
tuple(
|
||||
tuple(
|
||||
@@ -257,17 +257,17 @@ def resample_closed_contour(contour, spacing):
|
||||
|
||||
points = np.asarray(contour, dtype=np.float64)
|
||||
if len(points) < 3:
|
||||
raise ValueError("The lasso does not define a valid image region")
|
||||
raise ValueError("The lasso does not define a valid image selection")
|
||||
segments = np.roll(points, -1, axis=0) - points
|
||||
lengths = np.linalg.norm(segments, axis=1)
|
||||
points = points[lengths > 1e-8]
|
||||
if len(points) < 3:
|
||||
raise ValueError("The lasso region is too small to create a Cutout")
|
||||
raise ValueError("The selection is too small to create a Cutout")
|
||||
segments = np.roll(points, -1, axis=0) - points
|
||||
lengths = np.linalg.norm(segments, axis=1)
|
||||
perimeter = float(lengths.sum())
|
||||
if perimeter <= spacing * 3.0:
|
||||
raise ValueError("The lasso region is too small to create a Cutout")
|
||||
raise ValueError("The selection is too small to create a Cutout")
|
||||
count = max(3, int(round(perimeter / spacing)))
|
||||
distances = np.arange(count, dtype=np.float64) * perimeter / count
|
||||
cumulative = np.concatenate(([0.0], np.cumsum(lengths)))
|
||||
@@ -276,7 +276,7 @@ def resample_closed_contour(contour, spacing):
|
||||
return points[indices] + segments[indices] * factors[:, None]
|
||||
|
||||
|
||||
def smooth_region_contour(contour, spacing, iterations=16):
|
||||
def smooth_selection_contour(contour, spacing, iterations=16):
|
||||
contour = resample_closed_contour(contour, spacing * 0.25)
|
||||
original_area = abs(contour_area(contour))
|
||||
for _index in range(iterations):
|
||||
@@ -509,30 +509,30 @@ def prepare_planar_contours(
|
||||
if perimeter <= 1e-12:
|
||||
continue
|
||||
contour_spacing = min(spacing, perimeter / 8.0)
|
||||
smoothed = smooth_region_contour(planar, contour_spacing)
|
||||
smoothed = smooth_selection_contour(planar, contour_spacing)
|
||||
smoothed_local = smoothed @ np.linalg.inv(transform)
|
||||
smoothed_local[:, 0] = np.clip(smoothed_local[:, 0], x_min, x_max)
|
||||
smoothed_local[:, 1] = np.clip(smoothed_local[:, 1], y_min, y_max)
|
||||
prepared.append(smoothed_local @ transform)
|
||||
if not prepared:
|
||||
raise ValueError("The Cutout has no region large enough for this resolution")
|
||||
raise ValueError("The Cutout has no selection large enough for this resolution")
|
||||
interior = uniform_interior_points_for_contours(prepared, spacing)
|
||||
return prepared, interior, spacing
|
||||
|
||||
|
||||
def _prepare_region_contours(
|
||||
region_contours,
|
||||
def _prepare_selection_contours(
|
||||
selection_contours,
|
||||
local_bounds,
|
||||
spacing=POINT_SPACING,
|
||||
world_metric=None,
|
||||
):
|
||||
prepared = []
|
||||
errors = []
|
||||
for component in region_contours.components:
|
||||
for component in selection_contours.components:
|
||||
try:
|
||||
contours, interior, _spacing = prepare_planar_contours(
|
||||
component,
|
||||
region_contours.image_size,
|
||||
selection_contours.image_size,
|
||||
local_bounds,
|
||||
spacing=spacing,
|
||||
world_metric=world_metric,
|
||||
@@ -544,7 +544,7 @@ def _prepare_region_contours(
|
||||
if not prepared:
|
||||
if errors:
|
||||
raise errors[0]
|
||||
raise ValueError("The Cutout has no usable region")
|
||||
raise ValueError("The Cutout has no usable selection")
|
||||
return prepared, spacing
|
||||
|
||||
|
||||
@@ -582,7 +582,7 @@ def _triangulate_contour_component(contours, interior, spacing):
|
||||
centers = planar_points[faces].mean(axis=1)
|
||||
faces = faces[points_inside_contours(centers, contours)]
|
||||
if not len(faces):
|
||||
raise ValueError("Unable to triangulate the Cutout regions")
|
||||
raise ValueError("Unable to triangulate the Cutout selection components")
|
||||
used = np.unique(faces)
|
||||
if len(used) != len(planar_points):
|
||||
remap = np.full(len(planar_points), -1, dtype=np.int32)
|
||||
@@ -614,8 +614,8 @@ def boundary_vertices_from_faces(vertex_count, faces):
|
||||
return boundary
|
||||
|
||||
|
||||
def triangulate_region_contours(
|
||||
region_contours,
|
||||
def triangulate_selection_contours(
|
||||
selection_contours,
|
||||
local_bounds,
|
||||
spacing=POINT_SPACING,
|
||||
world_metric=None,
|
||||
@@ -623,8 +623,8 @@ def triangulate_region_contours(
|
||||
):
|
||||
import numpy as np
|
||||
|
||||
prepared, spacing = _prepare_region_contours(
|
||||
region_contours,
|
||||
prepared, spacing = _prepare_selection_contours(
|
||||
selection_contours,
|
||||
local_bounds,
|
||||
spacing=spacing,
|
||||
world_metric=world_metric,
|
||||
@@ -635,7 +635,7 @@ def triangulate_region_contours(
|
||||
components = []
|
||||
offset = 0
|
||||
for contours, interior in prepared:
|
||||
region_points, region_faces, region_boundary = _triangulate_contour_component(
|
||||
component_points, component_faces, component_boundary = _triangulate_contour_component(
|
||||
contours,
|
||||
interior,
|
||||
spacing,
|
||||
@@ -643,17 +643,17 @@ def triangulate_region_contours(
|
||||
components.append(
|
||||
(
|
||||
offset,
|
||||
offset + len(region_points),
|
||||
region_faces,
|
||||
offset + len(component_points),
|
||||
component_faces,
|
||||
contours[0],
|
||||
)
|
||||
)
|
||||
points.append(region_points)
|
||||
faces.extend(tuple(offset + value for value in face) for face in region_faces)
|
||||
boundaries.append(region_boundary)
|
||||
offset += len(region_points)
|
||||
points.append(component_points)
|
||||
faces.extend(tuple(offset + value for value in face) for face in component_faces)
|
||||
boundaries.append(component_boundary)
|
||||
offset += len(component_points)
|
||||
if not points:
|
||||
raise ValueError("Unable to triangulate the Cutout regions")
|
||||
raise ValueError("Unable to triangulate the Cutout selection components")
|
||||
result = (
|
||||
np.concatenate(points, axis=0),
|
||||
faces,
|
||||
@@ -1065,9 +1065,9 @@ DEFAULT_DEPTH_BASELINE = 8.0
|
||||
DEFAULT_DEPTH_DIRECTION = (0.0, 0.0, 1.0)
|
||||
|
||||
|
||||
def use_adaptive_depth_calibration(region_transparent_fraction):
|
||||
"""Return whether the Region input has enough transparent context."""
|
||||
return float(region_transparent_fraction) >= ADAPTIVE_DEPTH_TRANSPARENCY_RATIO
|
||||
def use_adaptive_depth_calibration(selection_transparent_fraction):
|
||||
"""Return whether the Selection input has enough transparent context."""
|
||||
return float(selection_transparent_fraction) >= ADAPTIVE_DEPTH_TRANSPARENCY_RATIO
|
||||
|
||||
|
||||
def metric_depth_reference(camera_depth, valid, uniform_scale):
|
||||
@@ -1253,7 +1253,7 @@ def orient_cutout_vertices(vertices):
|
||||
|
||||
|
||||
def cutout_domain(
|
||||
region_contours,
|
||||
selection_contours,
|
||||
bounds,
|
||||
spacing,
|
||||
world_metric,
|
||||
@@ -1263,8 +1263,8 @@ def cutout_domain(
|
||||
|
||||
x_min, x_max, y_min, y_max = bounds
|
||||
|
||||
planar, planar_faces, boundary, components = triangulate_region_contours(
|
||||
region_contours,
|
||||
planar, planar_faces, boundary, components = triangulate_selection_contours(
|
||||
selection_contours,
|
||||
bounds,
|
||||
spacing=spacing,
|
||||
world_metric=world_metric,
|
||||
@@ -1341,13 +1341,13 @@ def crop_plane_bounds(source_object, crop_bounds, source_size):
|
||||
|
||||
|
||||
def extract_cutout_contours(
|
||||
region_mask,
|
||||
selection_mask,
|
||||
mesh_resolution,
|
||||
threshold=CUTOUT_ALPHA_THRESHOLD,
|
||||
):
|
||||
contour_resolution = max(512, min(int(mesh_resolution) * 8, 1024))
|
||||
return extract_region_contours(
|
||||
region_mask,
|
||||
return extract_selection_contours(
|
||||
selection_mask,
|
||||
max_resolution=contour_resolution,
|
||||
threshold=threshold,
|
||||
)
|
||||
@@ -1363,7 +1363,7 @@ def limit_cutout_mesh_resolution(mesh_resolution, crop_size, max_input_size):
|
||||
return max(4, min(1024, round(int(mesh_resolution) * scale)))
|
||||
|
||||
|
||||
def cutout_point_spacing(source_object, mesh_resolution):
|
||||
def cutout_sampling_spacing(source_object, mesh_resolution):
|
||||
import math
|
||||
|
||||
x_min, x_max, y_min, y_max = image_empty_bounds(source_object)
|
||||
@@ -1373,11 +1373,11 @@ def cutout_point_spacing(source_object, mesh_resolution):
|
||||
return max(width, height) / max(int(mesh_resolution), 8)
|
||||
|
||||
|
||||
def region_calibration(region_mask, target_shape, threshold=CUTOUT_ALPHA_THRESHOLD):
|
||||
"""Return the final Region mask and transparency ratio at geometry resolution."""
|
||||
def selection_calibration(selection_mask, target_shape, threshold=CUTOUT_ALPHA_THRESHOLD):
|
||||
"""Return the final Selection mask and transparency ratio at geometry resolution."""
|
||||
import numpy as np
|
||||
|
||||
values = np.asarray(region_mask.values, dtype=np.float32)
|
||||
values = np.asarray(selection_mask.values, dtype=np.float32)
|
||||
target_height, target_width = (int(value) for value in target_shape)
|
||||
source_height, source_width = values.shape
|
||||
y = np.minimum(
|
||||
@@ -1394,16 +1394,16 @@ def region_calibration(region_mask, target_shape, threshold=CUTOUT_ALPHA_THRESHO
|
||||
return valid, transparent_fraction
|
||||
|
||||
|
||||
def build_base_shape(context, source_object, mesh_resolution, region_mask):
|
||||
def build_base_shape(context, source_object, mesh_resolution, selection_mask):
|
||||
import numpy as np
|
||||
|
||||
bounds = crop_plane_bounds(
|
||||
source_object,
|
||||
region_mask.bounds,
|
||||
selection_mask.bounds,
|
||||
tuple(source_object.data.size),
|
||||
)
|
||||
region_contours = extract_cutout_contours(
|
||||
region_mask,
|
||||
selection_contours = extract_cutout_contours(
|
||||
selection_mask,
|
||||
mesh_resolution,
|
||||
threshold=CUTOUT_ALPHA_THRESHOLD,
|
||||
)
|
||||
@@ -1412,9 +1412,9 @@ def build_base_shape(context, source_object, mesh_resolution, region_mask):
|
||||
raise ValueError("The Image Empty has a degenerate world transform")
|
||||
preferences = addon_preferences(context)
|
||||
balloon_algorithm = getattr(preferences, "balloon_profile_algorithm", "POISSON")
|
||||
point_spacing = cutout_point_spacing(source_object, mesh_resolution)
|
||||
point_spacing = cutout_sampling_spacing(source_object, mesh_resolution)
|
||||
domain = cutout_domain(
|
||||
region_contours,
|
||||
selection_contours,
|
||||
bounds,
|
||||
point_spacing,
|
||||
world_metric,
|
||||
|
||||
@@ -1,12 +1,12 @@
|
||||
"""Cutout Region validation and Shape-menu interaction helpers."""
|
||||
"""Cutout Selection validation and Shape-menu interaction helpers."""
|
||||
|
||||
from ...preferences import (
|
||||
MAX_CUTOUT_MESH_RESOLUTION,
|
||||
MIN_CUTOUT_MESH_RESOLUTION,
|
||||
)
|
||||
from ...properties import ai_ready
|
||||
from ...common.image_data import image_rgba
|
||||
from ...common.image_region import rasterize_region_path
|
||||
from ...common.image import image_rgba
|
||||
from ...common.selection import rasterize_selection_path
|
||||
from .geometry import CUTOUT_ALPHA_THRESHOLD
|
||||
from .shape import CUTOUT_SHAPE_LABELS
|
||||
|
||||
@@ -17,16 +17,23 @@ CUTOUT_SHAPE_BUTTONS = (
|
||||
("DEPTH_SURFACE", "SURFACE_NCURVE"),
|
||||
)
|
||||
|
||||
CUTOUT_MESH_DETAIL_SPACING = {
|
||||
"LOW": 32,
|
||||
"MEDIUM": 16,
|
||||
"HIGH": 8,
|
||||
"ULTRA": 4,
|
||||
}
|
||||
|
||||
def validate_cutout_region(
|
||||
|
||||
def validate_cutout_selection(
|
||||
source_object,
|
||||
region_path,
|
||||
selection_path,
|
||||
):
|
||||
image = source_object.data
|
||||
rgba = image_rgba(image)
|
||||
rasterize_region_path(
|
||||
rasterize_selection_path(
|
||||
tuple(image.size),
|
||||
region_path,
|
||||
selection_path,
|
||||
source_alpha=rgba[:, :, 3],
|
||||
alpha_threshold=CUTOUT_ALPHA_THRESHOLD,
|
||||
antialias=True,
|
||||
@@ -42,26 +49,30 @@ def cutout_mesh_resolution(image_size, point_spacing):
|
||||
)
|
||||
|
||||
|
||||
def cutout_mesh_detail_spacing(mesh_detail):
|
||||
return CUTOUT_MESH_DETAIL_SPACING[str(mesh_detail)]
|
||||
|
||||
|
||||
def _pie_button(
|
||||
pie,
|
||||
shape,
|
||||
label,
|
||||
icon,
|
||||
source_name,
|
||||
region_path_json,
|
||||
refine_region,
|
||||
selection_path_json,
|
||||
refine_selection,
|
||||
generate_normal,
|
||||
mesh_resolution,
|
||||
):
|
||||
operator = pie.operator(
|
||||
"anyimage.cutout_region_to_shape",
|
||||
"anyimage.cutout_selection_to_shape",
|
||||
text=label,
|
||||
icon=icon,
|
||||
)
|
||||
operator.source_object_name = source_name
|
||||
operator.region_path_json = region_path_json
|
||||
operator.selection_path_json = selection_path_json
|
||||
operator.shape = shape
|
||||
operator.refine_region = refine_region
|
||||
operator.refine_selection = refine_selection
|
||||
operator.generate_normal = generate_normal
|
||||
operator.mesh_resolution = mesh_resolution
|
||||
|
||||
@@ -70,15 +81,15 @@ def open_shape_pie(
|
||||
context,
|
||||
event,
|
||||
source_object,
|
||||
region_path,
|
||||
refine_region=False,
|
||||
selection_path,
|
||||
refine_selection=False,
|
||||
):
|
||||
source_name = source_object.name
|
||||
region_path_json = region_path.to_json()
|
||||
selection_path_json = selection_path.to_json()
|
||||
settings = context.scene.anyimage_settings
|
||||
point_spacing = settings.cutout_point_spacing
|
||||
point_spacing = cutout_mesh_detail_spacing(settings.cutout_mesh_detail)
|
||||
ready = ai_ready()
|
||||
refine_region = ready and refine_region
|
||||
refine_selection = ready and refine_selection
|
||||
generate_normal = ready and bool(
|
||||
getattr(settings, "cutout_generate_normal", False)
|
||||
)
|
||||
@@ -94,8 +105,8 @@ def open_shape_pie(
|
||||
CUTOUT_SHAPE_LABELS[shape],
|
||||
icon,
|
||||
source_name,
|
||||
region_path_json,
|
||||
refine_region,
|
||||
selection_path_json,
|
||||
refine_selection,
|
||||
generate_normal,
|
||||
mesh_resolution,
|
||||
)
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
"""Convert one image Region into a Cutout Shape."""
|
||||
"""Convert one image Selection into a Cutout Shape."""
|
||||
|
||||
import json
|
||||
|
||||
@@ -14,7 +14,7 @@ from ...preferences import (
|
||||
from ...properties import (
|
||||
BEN2_MODEL_KEY,
|
||||
DEFAULT_MOGE2_MODEL_KEY,
|
||||
refine_region_property,
|
||||
refine_selection_property,
|
||||
)
|
||||
from ...runtime import JobOperatorBase
|
||||
from ...server.media.resolution import (
|
||||
@@ -22,7 +22,9 @@ from ...server.media.resolution import (
|
||||
MAX_MAX_AI_INPUT_SIZE,
|
||||
MIN_MAX_AI_INPUT_SIZE,
|
||||
)
|
||||
from ...common.ai_job import (
|
||||
from ...common.ai import (
|
||||
draw_ai_property,
|
||||
draw_ai_setup,
|
||||
invoke_ai_setup_if_needed,
|
||||
moge2_parameters,
|
||||
production_device,
|
||||
@@ -30,38 +32,36 @@ from ...common.ai_job import (
|
||||
require_input_path,
|
||||
require_model,
|
||||
)
|
||||
from ...common.image_data import (
|
||||
from ...common.image import (
|
||||
cleanup_image_input,
|
||||
image_rgba,
|
||||
load_normal_result_image,
|
||||
prepare_region_input,
|
||||
prepare_selection_input,
|
||||
require_image_empty,
|
||||
source_start_frame,
|
||||
video_total_frames,
|
||||
)
|
||||
from ...common.depth_data import load_depth_metadata, load_depth_result_image
|
||||
from ...common.image_interaction import (
|
||||
ImageRegionGesture,
|
||||
apply_region_settings,
|
||||
draw_ai_property,
|
||||
draw_ai_setup,
|
||||
from ...common.depth import load_depth_metadata, load_depth_result_image
|
||||
from ...common.viewport import (
|
||||
ImageSelectionGesture,
|
||||
apply_selection_settings,
|
||||
gesture_property,
|
||||
report_image_edit_exception,
|
||||
resolve_image_edit_click,
|
||||
serialize_matrix,
|
||||
)
|
||||
from ...common.image_region import (
|
||||
from ...common.selection import (
|
||||
ImageEditWarning,
|
||||
RegionMask,
|
||||
RegionPath,
|
||||
rasterize_region_path,
|
||||
SelectionMask,
|
||||
SelectionPath,
|
||||
rasterize_selection_path,
|
||||
)
|
||||
from .geometry import (
|
||||
CUTOUT_ALPHA_THRESHOLD,
|
||||
build_base_shape,
|
||||
limit_cutout_mesh_resolution,
|
||||
)
|
||||
from .interaction import open_shape_pie, validate_cutout_region
|
||||
from .interaction import open_shape_pie, validate_cutout_selection
|
||||
from .object import NormalMode, create_shape_object
|
||||
from .shape import CUTOUT_SHAPES, DEPTH_CUTOUT_SHAPES
|
||||
|
||||
@@ -89,7 +89,7 @@ def create_cutout_shape(
|
||||
source_object,
|
||||
shape,
|
||||
mesh_resolution,
|
||||
region_mask,
|
||||
selection_mask,
|
||||
*,
|
||||
depth_image=None,
|
||||
depth_metadata=None,
|
||||
@@ -101,14 +101,14 @@ def create_cutout_shape(
|
||||
context,
|
||||
source_object,
|
||||
mesh_resolution,
|
||||
region_mask,
|
||||
selection_mask,
|
||||
)
|
||||
result = create_shape_object(
|
||||
context,
|
||||
source_object,
|
||||
shape,
|
||||
mesh_resolution,
|
||||
region_mask,
|
||||
selection_mask,
|
||||
base_shape,
|
||||
depth_image=depth_image,
|
||||
depth_metadata=depth_metadata,
|
||||
@@ -122,10 +122,10 @@ def create_cutout_shape(
|
||||
return result
|
||||
|
||||
|
||||
def _region_mask_from_image(path, bounds):
|
||||
def _selection_mask_from_image(path, bounds):
|
||||
image = bpy.data.images.load(str(path), check_existing=False)
|
||||
try:
|
||||
return RegionMask(
|
||||
return SelectionMask(
|
||||
image_rgba(image)[:, :, 3],
|
||||
tuple(int(value) for value in bounds),
|
||||
)
|
||||
@@ -133,13 +133,13 @@ def _region_mask_from_image(path, bounds):
|
||||
bpy.data.images.remove(image, do_unlink=True)
|
||||
|
||||
|
||||
def _generated_region_mask(result, input_path, source_bounds):
|
||||
if not isinstance(result.value, dict) or "region_image" not in result.value:
|
||||
return _region_mask_from_image(input_path, source_bounds)
|
||||
def _generated_selection_mask(result, input_path, source_bounds):
|
||||
if not isinstance(result.value, dict) or "selection_image" not in result.value:
|
||||
return _selection_mask_from_image(input_path, source_bounds)
|
||||
local_bounds = tuple(
|
||||
int(value)
|
||||
for value in json.loads(
|
||||
result.file("region_bounds").read_text(encoding="utf-8")
|
||||
result.file("selection_bounds").read_text(encoding="utf-8")
|
||||
)
|
||||
)
|
||||
bounds = (
|
||||
@@ -148,7 +148,7 @@ def _generated_region_mask(result, input_path, source_bounds):
|
||||
source_bounds[0] + local_bounds[2],
|
||||
source_bounds[1] + local_bounds[3],
|
||||
)
|
||||
return _region_mask_from_image(result.file("region_image"), bounds)
|
||||
return _selection_mask_from_image(result.file("selection_image"), bounds)
|
||||
|
||||
|
||||
def cutout_geometry_settings(context):
|
||||
@@ -160,18 +160,18 @@ def cutout_geometry_settings(context):
|
||||
)
|
||||
|
||||
|
||||
class CutoutRegionToShape(JobOperatorBase, bpy.types.Operator):
|
||||
bl_idname = "anyimage.cutout_region_to_shape"
|
||||
bl_label = "Cutout Region to Shape"
|
||||
bl_description = "Build the selected image Region as a Cutout shape"
|
||||
class CutoutSelectionToShape(JobOperatorBase, bpy.types.Operator):
|
||||
bl_idname = "anyimage.cutout_selection_to_shape"
|
||||
bl_label = "Cutout Selection to Shape"
|
||||
bl_description = "Build the image selection as a Cutout shape"
|
||||
bl_options = {"INTERNAL"}
|
||||
job_type = "generate-cutout-artifacts"
|
||||
starting_message = "Generating Cutout artifacts"
|
||||
|
||||
input_path: bpy.props.StringProperty(options=_HIDDEN)
|
||||
source_object_name: bpy.props.StringProperty(options=_HIDDEN)
|
||||
region_path_json: bpy.props.StringProperty(options=_HIDDEN)
|
||||
region_bounds_json: bpy.props.StringProperty(options=_HIDDEN)
|
||||
selection_path_json: bpy.props.StringProperty(options=_HIDDEN)
|
||||
selection_bounds_json: bpy.props.StringProperty(options=_HIDDEN)
|
||||
shape: bpy.props.StringProperty(options=_HIDDEN)
|
||||
mesh_resolution: bpy.props.IntProperty(
|
||||
default=128,
|
||||
@@ -196,11 +196,11 @@ class CutoutRegionToShape(JobOperatorBase, bpy.types.Operator):
|
||||
options=_HIDDEN,
|
||||
)
|
||||
generate_normal: bpy.props.BoolProperty(default=False, options=_HIDDEN)
|
||||
refine_region: bpy.props.BoolProperty(default=False, options=_HIDDEN)
|
||||
refine_selection: bpy.props.BoolProperty(default=False, options=_HIDDEN)
|
||||
|
||||
def _needs_generation(self):
|
||||
return (
|
||||
self.refine_region
|
||||
self.refine_selection
|
||||
or self.shape in DEPTH_CUTOUT_SHAPES
|
||||
or self.generate_normal
|
||||
)
|
||||
@@ -212,19 +212,19 @@ class CutoutRegionToShape(JobOperatorBase, bpy.types.Operator):
|
||||
raise ValueError("Cutout currently supports still images")
|
||||
if self.shape not in CUTOUT_SHAPES:
|
||||
raise ValueError(f"Unsupported Cutout shape: {self.shape}")
|
||||
region_path = RegionPath.from_json(self.region_path_json)
|
||||
selection_path = SelectionPath.from_json(self.selection_path_json)
|
||||
source_rgba = image_rgba(source_object.data)
|
||||
region_mask = rasterize_region_path(
|
||||
selection_mask = rasterize_selection_path(
|
||||
tuple(source_object.data.size),
|
||||
region_path,
|
||||
selection_path,
|
||||
source_alpha=source_rgba[:, :, 3],
|
||||
alpha_threshold=CUTOUT_ALPHA_THRESHOLD,
|
||||
antialias=True,
|
||||
)
|
||||
context.view_layer.objects.active = source_object
|
||||
source_object.select_set(True)
|
||||
crop_width = region_mask.bounds[2] - region_mask.bounds[0]
|
||||
crop_height = region_mask.bounds[3] - region_mask.bounds[1]
|
||||
crop_width = selection_mask.bounds[2] - selection_mask.bounds[0]
|
||||
crop_height = selection_mask.bounds[3] - selection_mask.bounds[1]
|
||||
self.mesh_resolution = limit_cutout_mesh_resolution(
|
||||
self.mesh_resolution,
|
||||
(crop_width, crop_height),
|
||||
@@ -237,7 +237,7 @@ class CutoutRegionToShape(JobOperatorBase, bpy.types.Operator):
|
||||
source_object,
|
||||
self.shape,
|
||||
self.mesh_resolution,
|
||||
region_mask,
|
||||
selection_mask,
|
||||
source_rgba=source_rgba,
|
||||
)
|
||||
return {"FINISHED"}
|
||||
@@ -245,11 +245,11 @@ class CutoutRegionToShape(JobOperatorBase, bpy.types.Operator):
|
||||
if setup_result is not None:
|
||||
return setup_result
|
||||
self.model, self.resolution_level = cutout_geometry_settings(context)
|
||||
self.region_bounds_json = json.dumps(
|
||||
region_mask.bounds,
|
||||
self.selection_bounds_json = json.dumps(
|
||||
selection_mask.bounds,
|
||||
separators=(",", ":"),
|
||||
)
|
||||
self.input_path = str(prepare_region_input(source_object.data, region_mask))
|
||||
self.input_path = str(prepare_selection_input(source_object.data, selection_mask))
|
||||
except (
|
||||
ImageEditWarning,
|
||||
OSError,
|
||||
@@ -269,19 +269,19 @@ class CutoutRegionToShape(JobOperatorBase, bpy.types.Operator):
|
||||
source_path = require_input_path(self.input_path)
|
||||
depth_mode = depth_mode_for_shape(self.shape)
|
||||
normal_mode = normal_mode_for_shape(self.shape, self.generate_normal)
|
||||
if self.refine_region:
|
||||
if self.refine_selection:
|
||||
require_model(BEN2_MODEL_KEY)
|
||||
if depth_mode != "NONE" or normal_mode != NormalMode.NONE:
|
||||
require_model(self.model)
|
||||
parameters = {
|
||||
"input": str(source_path),
|
||||
"device": production_device().lower(),
|
||||
"refine_region": self.refine_region,
|
||||
"refine_selection": self.refine_selection,
|
||||
"depth_mode": depth_mode,
|
||||
"normal_mode": normal_mode.value,
|
||||
"max_input_size": int(self.max_ai_input_size),
|
||||
}
|
||||
if self.refine_region:
|
||||
if self.refine_selection:
|
||||
parameters["refine_model"] = BEN2_MODEL_KEY
|
||||
if depth_mode != "NONE" or normal_mode != NormalMode.NONE:
|
||||
start_frame = source_start_frame(self.source_object_name)
|
||||
@@ -302,8 +302,8 @@ class CutoutRegionToShape(JobOperatorBase, bpy.types.Operator):
|
||||
|
||||
def response(self, context, result):
|
||||
source_object = require_image_empty(self.source_object_name)
|
||||
source_bounds = tuple(json.loads(self.region_bounds_json))
|
||||
region_mask = _generated_region_mask(
|
||||
source_bounds = tuple(json.loads(self.selection_bounds_json))
|
||||
selection_mask = _generated_selection_mask(
|
||||
result,
|
||||
self.input_path,
|
||||
source_bounds,
|
||||
@@ -333,7 +333,7 @@ class CutoutRegionToShape(JobOperatorBase, bpy.types.Operator):
|
||||
source_object,
|
||||
self.shape,
|
||||
self.mesh_resolution,
|
||||
region_mask,
|
||||
selection_mask,
|
||||
depth_image=depth_image,
|
||||
depth_metadata=depth_metadata,
|
||||
normal_image=normal_image,
|
||||
@@ -351,18 +351,18 @@ class CutoutRegionToShape(JobOperatorBase, bpy.types.Operator):
|
||||
cleanup_image_input(self.input_path, True)
|
||||
|
||||
|
||||
class SelectCutoutRegion(ImageRegionGesture, bpy.types.Operator):
|
||||
bl_idname = "anyimage.select_cutout_region"
|
||||
bl_label = "Select Cutout Region"
|
||||
bl_description = "Lasso an Image Empty region, then choose its mesh shape"
|
||||
class SelectCutoutSelection(ImageSelectionGesture, bpy.types.Operator):
|
||||
bl_idname = "anyimage.select_cutout_selection"
|
||||
bl_label = "Start Cutout Selection"
|
||||
bl_description = "Lasso an Image Empty selection, then choose its mesh shape"
|
||||
bl_options = {"INTERNAL"}
|
||||
active_tool_id = "anyimage.cutout_lasso"
|
||||
refine_region_settings = {"LASSO": "cutout_refine_region"}
|
||||
refine_selection_settings = {"LASSO": "cutout_refine_selection"}
|
||||
|
||||
source_object_name: bpy.props.StringProperty(options={"HIDDEN", "SKIP_SAVE"})
|
||||
source_matrix_data: bpy.props.StringProperty(options={"HIDDEN", "SKIP_SAVE"})
|
||||
region_path_json: bpy.props.StringProperty(options={"HIDDEN", "SKIP_SAVE"})
|
||||
refine_region: refine_region_property()
|
||||
selection_path_json: bpy.props.StringProperty(options={"HIDDEN", "SKIP_SAVE"})
|
||||
refine_selection: refine_selection_property()
|
||||
invert: bpy.props.BoolProperty(default=False, options={"SKIP_SAVE"})
|
||||
gesture: gesture_property()
|
||||
full_image: bpy.props.BoolProperty(default=False, options={"HIDDEN", "SKIP_SAVE"})
|
||||
@@ -391,12 +391,12 @@ class SelectCutoutRegion(ImageRegionGesture, bpy.types.Operator):
|
||||
if video_total_frames(source_object.data) > 1:
|
||||
self.report({"WARNING"}, "Image tools only support a still image")
|
||||
return {"CANCELLED"}
|
||||
apply_region_settings(self, context)
|
||||
apply_selection_settings(self, context)
|
||||
self.invert = False
|
||||
self.source_object_name = source_object.name
|
||||
self.source_matrix_data = serialize_matrix(source_object.matrix_world)
|
||||
width, height = (int(value) for value in source_object.data.size)
|
||||
self.region_path_json = RegionPath(
|
||||
self.selection_path_json = SelectionPath(
|
||||
points=((0, 0), (width, 0), (width, height), (0, height)),
|
||||
fill_rule="NONZERO",
|
||||
).to_json()
|
||||
@@ -413,8 +413,8 @@ class SelectCutoutRegion(ImageRegionGesture, bpy.types.Operator):
|
||||
self.report({"ERROR"}, str(error))
|
||||
return {"CANCELLED"}
|
||||
try:
|
||||
region_path = RegionPath.from_json(self.region_path_json)
|
||||
validate_cutout_region(source_object, region_path)
|
||||
selection_path = SelectionPath.from_json(self.selection_path_json)
|
||||
validate_cutout_selection(source_object, selection_path)
|
||||
except (ImageEditWarning, TypeError, ValueError) as error:
|
||||
report_image_edit_exception(self, error)
|
||||
return {"CANCELLED"}
|
||||
@@ -422,8 +422,8 @@ class SelectCutoutRegion(ImageRegionGesture, bpy.types.Operator):
|
||||
context,
|
||||
self._pie_event,
|
||||
source_object,
|
||||
region_path,
|
||||
self.refine_region,
|
||||
selection_path,
|
||||
self.refine_selection,
|
||||
)
|
||||
return {"FINISHED"}
|
||||
|
||||
@@ -435,15 +435,15 @@ class CutoutTool(bpy.types.WorkSpaceTool):
|
||||
bl_label = "Cutout"
|
||||
bl_description = "Lasso an Image Empty and choose a mesh shape"
|
||||
bl_icon = "ops.sculpt.lasso_mask"
|
||||
bl_operator = SelectCutoutRegion.bl_idname
|
||||
bl_operator = SelectCutoutSelection.bl_idname
|
||||
bl_keymap = (
|
||||
(
|
||||
SelectCutoutRegion.bl_idname,
|
||||
SelectCutoutSelection.bl_idname,
|
||||
{"type": "LEFTMOUSE", "value": "DOUBLE_CLICK"},
|
||||
{"properties": [("gesture", "LASSO"), ("full_image", True)]},
|
||||
),
|
||||
(
|
||||
SelectCutoutRegion.bl_idname,
|
||||
SelectCutoutSelection.bl_idname,
|
||||
{"type": "LEFTMOUSE", "value": "PRESS"},
|
||||
{"properties": [("gesture", "LASSO")]},
|
||||
),
|
||||
@@ -452,6 +452,6 @@ class CutoutTool(bpy.types.WorkSpaceTool):
|
||||
def draw_settings(context, layout, _tool):
|
||||
settings = context.scene.anyimage_settings
|
||||
status = draw_ai_setup(layout)
|
||||
draw_ai_property(layout, settings, "cutout_refine_region", status)
|
||||
layout.prop(settings, "cutout_point_spacing")
|
||||
draw_ai_property(layout, settings, "cutout_refine_selection", status)
|
||||
draw_ai_property(layout, settings, "cutout_generate_normal", status)
|
||||
layout.prop(settings, "cutout_mesh_detail")
|
||||
|
||||
@@ -4,14 +4,16 @@ from enum import StrEnum
|
||||
|
||||
import bpy
|
||||
|
||||
from ...common.image_data import (
|
||||
from ...common.image import (
|
||||
color_data_name,
|
||||
image_rgba,
|
||||
)
|
||||
from ...common.depth_data import camera_depth_values, depth_uniform_scale
|
||||
from ...common.object import (
|
||||
from ...common.depth import camera_depth_values, depth_uniform_scale
|
||||
from ...common.material import (
|
||||
IMAGE_MATERIAL_NODE_GROUP_NAME,
|
||||
create_image_material,
|
||||
)
|
||||
from ...common.object import (
|
||||
finalize_object_result,
|
||||
modifier_input_identifier,
|
||||
set_modifier_input,
|
||||
@@ -22,10 +24,10 @@ from .geometry import (
|
||||
DEFAULT_DEPTH_DIRECTION,
|
||||
crop_plane_bounds,
|
||||
cutout_cleanup_value,
|
||||
cutout_point_spacing,
|
||||
cutout_sampling_spacing,
|
||||
depth_balloon_projection_parameters,
|
||||
metric_depth_reference,
|
||||
region_calibration,
|
||||
selection_calibration,
|
||||
use_adaptive_depth_calibration,
|
||||
world_plane_metric,
|
||||
)
|
||||
@@ -82,8 +84,8 @@ def _assign_float_point_attribute(mesh, name, values):
|
||||
attribute.data.foreach_set("value", np.asarray(values, dtype=np.float32))
|
||||
|
||||
|
||||
def _cropped_color_image(source_object, region_mask, source_rgba):
|
||||
crop_bounds = region_mask.bounds
|
||||
def _cropped_color_image(source_object, selection_mask, source_rgba):
|
||||
crop_bounds = selection_mask.bounds
|
||||
import numpy as np
|
||||
|
||||
source = source_object.data
|
||||
@@ -149,7 +151,7 @@ def create_shape_object(
|
||||
source_object,
|
||||
shape,
|
||||
mesh_resolution,
|
||||
region_mask,
|
||||
selection_mask,
|
||||
base_shape,
|
||||
*,
|
||||
depth_image=None,
|
||||
@@ -163,7 +165,7 @@ def create_shape_object(
|
||||
normal_mode = NormalMode(normal_mode)
|
||||
bounds = crop_plane_bounds(
|
||||
source_object,
|
||||
region_mask.bounds,
|
||||
selection_mask.bounds,
|
||||
tuple(source_object.data.size),
|
||||
)
|
||||
crop_center = (
|
||||
@@ -172,12 +174,12 @@ def create_shape_object(
|
||||
)
|
||||
world_metric, _world_z_scale = world_plane_metric(source_object.matrix_world)
|
||||
cleanup_value = cutout_cleanup_value(
|
||||
cutout_point_spacing(source_object, mesh_resolution),
|
||||
cutout_sampling_spacing(source_object, mesh_resolution),
|
||||
world_metric,
|
||||
)
|
||||
color_image = _cropped_color_image(
|
||||
source_object,
|
||||
region_mask,
|
||||
selection_mask,
|
||||
image_rgba(source_object.data) if source_rgba is None else source_rgba,
|
||||
)
|
||||
material = None
|
||||
@@ -206,8 +208,8 @@ def create_shape_object(
|
||||
if depth_image is None or depth_metadata is None:
|
||||
raise ValueError("Depth Shape requires a Depth texture and metadata")
|
||||
camera_depth, valid = camera_depth_values(depth_image)
|
||||
calibration_valid, transparent_fraction = region_calibration(
|
||||
region_mask, valid.shape
|
||||
calibration_valid, transparent_fraction = selection_calibration(
|
||||
selection_mask, valid.shape
|
||||
)
|
||||
calibration_valid &= valid
|
||||
uniform_scale = depth_uniform_scale(
|
||||
|
||||
@@ -4,7 +4,7 @@ import bpy
|
||||
|
||||
from ..runtime import JobOperatorBase
|
||||
from ..properties import BEN2_MODEL_KEY, REALESRGAN_GENERAL_WDN_MODEL_KEY
|
||||
from ..common.ai_job import (
|
||||
from ..common.ai import (
|
||||
require_background_input_path,
|
||||
da3_parameters,
|
||||
require_input_path,
|
||||
|
||||
@@ -1,12 +1,12 @@
|
||||
from .box import ImageBoxTool
|
||||
from .edit_mask import ActivateImageTool, EditImageMask, RefineImageRegion
|
||||
from .edit_mask import ActivateImageTool, EditImageMask, RefineImageSelection
|
||||
from .lasso import ImageLassoTool
|
||||
from .line import ImageLineTool
|
||||
from .polyline import ImagePolylineTool
|
||||
|
||||
CLASSES = (
|
||||
EditImageMask,
|
||||
RefineImageRegion,
|
||||
RefineImageSelection,
|
||||
ActivateImageTool,
|
||||
)
|
||||
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
import bpy
|
||||
|
||||
from ...common.image_interaction import draw_region_settings
|
||||
from ...common.viewport import draw_selection_settings
|
||||
from .edit_mask import EditImageMask
|
||||
|
||||
|
||||
@@ -21,4 +21,4 @@ class ImageBoxTool(bpy.types.WorkSpaceTool):
|
||||
)
|
||||
|
||||
def draw_settings(context, layout, _tool):
|
||||
draw_region_settings(context, layout, "image_box_refine_region")
|
||||
draw_selection_settings(context, layout, "image_box_refine_selection")
|
||||
|
||||
@@ -2,34 +2,34 @@ import json
|
||||
|
||||
import bpy
|
||||
|
||||
from ...properties import BEN2_MODEL_KEY, refine_region_property
|
||||
from ...properties import BEN2_MODEL_KEY, refine_selection_property
|
||||
from ...runtime import JobOperatorBase
|
||||
from ...common.ai_job import (
|
||||
from ...common.ai import (
|
||||
production_device,
|
||||
require_environment,
|
||||
require_model,
|
||||
)
|
||||
from ...common.image_data import (
|
||||
from ...common.image import (
|
||||
cleanup_image_input,
|
||||
create_pixel_result_image,
|
||||
image_pixels,
|
||||
image_rgba,
|
||||
prepare_region_input,
|
||||
prepare_selection_input,
|
||||
replace_empty_image,
|
||||
require_image_empty,
|
||||
video_total_frames,
|
||||
)
|
||||
from ...common.image_interaction import (
|
||||
ImageRegionGesture,
|
||||
from ...common.viewport import (
|
||||
ImageSelectionGesture,
|
||||
gesture_property,
|
||||
image_edit_poll,
|
||||
report_image_edit_exception,
|
||||
)
|
||||
from ...common.image_region import (
|
||||
from ...common.selection import (
|
||||
ImageEditWarning,
|
||||
RegionPath,
|
||||
SelectionPath,
|
||||
crop_pixels_to_alpha,
|
||||
rasterize_region_path,
|
||||
rasterize_selection_path,
|
||||
)
|
||||
|
||||
_HIDDEN = {"HIDDEN", "SKIP_SAVE"}
|
||||
@@ -61,7 +61,7 @@ def _refined_mask_from_pixels(mask_pixels, mask_size, bounds, source_size):
|
||||
def _edit_image_mask_pixels(
|
||||
source_pixels,
|
||||
source_size,
|
||||
region_mask,
|
||||
selection_mask,
|
||||
invert=False,
|
||||
):
|
||||
"""Apply a mask while preserving source RGB, then tightly crop the canvas."""
|
||||
@@ -71,7 +71,7 @@ def _edit_image_mask_pixels(
|
||||
rgba = np.flipud(
|
||||
np.asarray(source_pixels, dtype=np.float32).reshape((height, width, 4))
|
||||
)
|
||||
mask = np.asarray(region_mask, dtype=np.float32)
|
||||
mask = np.asarray(selection_mask, dtype=np.float32)
|
||||
if invert:
|
||||
mask = 1.0 - mask
|
||||
mask = np.where(rgba[:, :, 3] > 0.0, mask, 0.0)
|
||||
@@ -82,13 +82,13 @@ def _edit_image_mask_pixels(
|
||||
)
|
||||
|
||||
|
||||
def _edit_image_object(source_object, region_mask, invert=False):
|
||||
def _edit_image_object(source_object, selection_mask, invert=False):
|
||||
image = source_object.data
|
||||
source_size = tuple(image.size)
|
||||
pixels, crop_bounds = _edit_image_mask_pixels(
|
||||
image_pixels(image),
|
||||
source_size,
|
||||
region_mask,
|
||||
selection_mask,
|
||||
invert,
|
||||
)
|
||||
left, top, right, bottom = crop_bounds
|
||||
@@ -104,10 +104,10 @@ def _edit_image_object(source_object, region_mask, invert=False):
|
||||
)
|
||||
|
||||
|
||||
class EditImageMask(ImageRegionGesture, bpy.types.Operator):
|
||||
refine_region_settings = {
|
||||
"BOX": "image_box_refine_region",
|
||||
"LASSO": "image_lasso_refine_region",
|
||||
class EditImageMask(ImageSelectionGesture, bpy.types.Operator):
|
||||
refine_selection_settings = {
|
||||
"BOX": "image_box_refine_selection",
|
||||
"LASSO": "image_lasso_refine_selection",
|
||||
}
|
||||
bl_idname = "anyimage.edit_image_mask"
|
||||
bl_label = "Edit Image Mask"
|
||||
@@ -116,8 +116,8 @@ class EditImageMask(ImageRegionGesture, bpy.types.Operator):
|
||||
|
||||
source_object_name: bpy.props.StringProperty(options={"HIDDEN", "SKIP_SAVE"})
|
||||
source_matrix_data: bpy.props.StringProperty(options={"HIDDEN", "SKIP_SAVE"})
|
||||
region_path_json: bpy.props.StringProperty(options={"HIDDEN", "SKIP_SAVE"})
|
||||
refine_region: refine_region_property()
|
||||
selection_path_json: bpy.props.StringProperty(options={"HIDDEN", "SKIP_SAVE"})
|
||||
refine_selection: refine_selection_property()
|
||||
invert: bpy.props.BoolProperty(
|
||||
name="Invert",
|
||||
description="Keep the opposite side of the refined gesture mask",
|
||||
@@ -133,23 +133,23 @@ class EditImageMask(ImageRegionGesture, bpy.types.Operator):
|
||||
self.report({"ERROR"}, str(error))
|
||||
return {"CANCELLED"}
|
||||
try:
|
||||
region_path = RegionPath.from_json(self.region_path_json)
|
||||
selection_path = SelectionPath.from_json(self.selection_path_json)
|
||||
if video_total_frames(source_object.data) > 1:
|
||||
raise RuntimeError("Image tools only support a still image")
|
||||
region_mask = rasterize_region_path(
|
||||
selection_mask = rasterize_selection_path(
|
||||
tuple(source_object.data.size),
|
||||
region_path,
|
||||
selection_path,
|
||||
source_alpha=image_rgba(source_object.data)[:, :, 3],
|
||||
)
|
||||
if self.refine_region:
|
||||
source_path = prepare_region_input(
|
||||
if self.refine_selection:
|
||||
source_path = prepare_selection_input(
|
||||
source_object.data,
|
||||
region_mask,
|
||||
selection_mask,
|
||||
)
|
||||
result = bpy.ops.anyimage.refine_image_region(
|
||||
result = bpy.ops.anyimage.refine_image_selection(
|
||||
input_path=str(source_path),
|
||||
source_object_name=self.source_object_name,
|
||||
region_bounds_json=json.dumps(region_mask.bounds),
|
||||
selection_bounds_json=json.dumps(selection_mask.bounds),
|
||||
)
|
||||
if "CANCELLED" in result:
|
||||
cleanup_image_input(source_path, True)
|
||||
@@ -157,7 +157,7 @@ class EditImageMask(ImageRegionGesture, bpy.types.Operator):
|
||||
return {"FINISHED"}
|
||||
_edit_image_object(
|
||||
source_object,
|
||||
region_mask.full_values(tuple(source_object.data.size)),
|
||||
selection_mask.full_values(tuple(source_object.data.size)),
|
||||
)
|
||||
except (json.JSONDecodeError, RuntimeError, TypeError, ValueError) as error:
|
||||
report_image_edit_exception(self, error)
|
||||
@@ -165,15 +165,15 @@ class EditImageMask(ImageRegionGesture, bpy.types.Operator):
|
||||
return {"FINISHED"}
|
||||
|
||||
|
||||
class RefineImageRegion(JobOperatorBase, bpy.types.Operator):
|
||||
bl_idname = "anyimage.refine_image_region"
|
||||
bl_label = "Refine Image Region"
|
||||
job_type = "refine-image-region"
|
||||
class RefineImageSelection(JobOperatorBase, bpy.types.Operator):
|
||||
bl_idname = "anyimage.refine_image_selection"
|
||||
bl_label = "Refine Image Selection"
|
||||
job_type = "refine-image-selection"
|
||||
starting_message = "Refining image selection"
|
||||
|
||||
input_path: bpy.props.StringProperty(options=_HIDDEN)
|
||||
source_object_name: bpy.props.StringProperty(options={"HIDDEN", "SKIP_SAVE"})
|
||||
region_bounds_json: bpy.props.StringProperty(options={"HIDDEN", "SKIP_SAVE"})
|
||||
selection_bounds_json: bpy.props.StringProperty(options={"HIDDEN", "SKIP_SAVE"})
|
||||
|
||||
def request(self, _context):
|
||||
require_environment()
|
||||
@@ -187,9 +187,9 @@ class RefineImageRegion(JobOperatorBase, bpy.types.Operator):
|
||||
def response(self, _context, result):
|
||||
source_object = require_image_empty(self.source_object_name)
|
||||
|
||||
source_bounds = tuple(json.loads(self.region_bounds_json))
|
||||
source_bounds = tuple(json.loads(self.selection_bounds_json))
|
||||
local_bounds = tuple(
|
||||
json.loads(result.file("region_bounds").read_text(encoding="utf-8"))
|
||||
json.loads(result.file("selection_bounds").read_text(encoding="utf-8"))
|
||||
)
|
||||
bounds = (
|
||||
source_bounds[0] + local_bounds[0],
|
||||
@@ -198,7 +198,7 @@ class RefineImageRegion(JobOperatorBase, bpy.types.Operator):
|
||||
source_bounds[1] + local_bounds[3],
|
||||
)
|
||||
refined_image = bpy.data.images.load(
|
||||
str(result.file("region_image")), check_existing=False
|
||||
str(result.file("selection_image")), check_existing=False
|
||||
)
|
||||
try:
|
||||
refined_pixels = image_pixels(refined_image)
|
||||
@@ -217,7 +217,7 @@ class RefineImageRegion(JobOperatorBase, bpy.types.Operator):
|
||||
return str(error)
|
||||
if "FINISHED" not in bpy.ops.ed.undo_push(message="Edit Image Mask"):
|
||||
raise RuntimeError("Unable to create the Edit Image Mask Undo step")
|
||||
return "Image Region refined"
|
||||
return "Image Selection refined"
|
||||
|
||||
def cleanup(self):
|
||||
cleanup_image_input(self.input_path, True)
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
import bpy
|
||||
|
||||
from ...common.image_interaction import draw_region_settings
|
||||
from ...common.viewport import draw_selection_settings
|
||||
from .edit_mask import EditImageMask
|
||||
|
||||
|
||||
@@ -21,4 +21,4 @@ class ImageLassoTool(bpy.types.WorkSpaceTool):
|
||||
)
|
||||
|
||||
def draw_settings(context, layout, _tool):
|
||||
draw_region_settings(context, layout, "image_lasso_refine_region")
|
||||
draw_selection_settings(context, layout, "image_lasso_refine_selection")
|
||||
|
||||
@@ -3,14 +3,14 @@ import math
|
||||
|
||||
import bpy
|
||||
|
||||
from ...common.image_interaction import (
|
||||
from ...common.viewport import (
|
||||
deserialize_matrix,
|
||||
drawing_in_region,
|
||||
screen_path_to_image_pixels,
|
||||
serialize_matrix,
|
||||
)
|
||||
from ...common.image_region import crop_pixels_to_alpha
|
||||
from ...common.image_data import (
|
||||
from ...common.selection import crop_pixels_to_alpha
|
||||
from ...common.image import (
|
||||
create_pixel_result_image,
|
||||
image_base_name,
|
||||
image_pixels,
|
||||
@@ -19,7 +19,7 @@ from ...common.image_data import (
|
||||
require_image_empty,
|
||||
video_total_frames,
|
||||
)
|
||||
from ...common.image_interaction import (
|
||||
from ...common.viewport import (
|
||||
active_view3d_tool_id,
|
||||
draw_dashed_line,
|
||||
draw_polygon_fill,
|
||||
|
||||
@@ -2,7 +2,7 @@ import bpy
|
||||
|
||||
from ..properties import BEN2_MODEL_KEY
|
||||
from ..runtime import JobOperatorBase
|
||||
from ..common.ai_job import (
|
||||
from ..common.ai import (
|
||||
require_background_input_path,
|
||||
invoke_ai_setup_if_needed,
|
||||
production_device,
|
||||
@@ -10,7 +10,7 @@ from ..common.ai_job import (
|
||||
require_model,
|
||||
)
|
||||
|
||||
from ..common.image_data import (
|
||||
from ..common.image import (
|
||||
cleanup_image_input,
|
||||
default_extract_frames,
|
||||
is_image_empty,
|
||||
@@ -70,7 +70,7 @@ class RemoveImageBackground(bpy.types.Operator):
|
||||
|
||||
|
||||
def apply_background_response(result, source_object_name):
|
||||
from ..common.image_data import (
|
||||
from ..common.image import (
|
||||
load_color_result_image,
|
||||
replace_empty_image,
|
||||
require_image_empty,
|
||||
|
||||
@@ -8,14 +8,14 @@ from ..server.media.resolution import (
|
||||
MIN_MAX_AI_INPUT_SIZE,
|
||||
)
|
||||
from ..runtime import JobOperatorBase
|
||||
from ..common.ai_job import (
|
||||
from ..common.ai import (
|
||||
require_input_path,
|
||||
invoke_ai_setup_if_needed,
|
||||
production_device,
|
||||
require_environment,
|
||||
require_model,
|
||||
)
|
||||
from ..common.image_data import (
|
||||
from ..common.image import (
|
||||
cleanup_image_input,
|
||||
default_extract_frames,
|
||||
is_image_empty,
|
||||
|
||||
@@ -113,7 +113,7 @@ class AnyImagePreferences(bpy.types.AddonPreferences):
|
||||
name="Maximum AI Input Size",
|
||||
description=(
|
||||
"Maximum source long edge eligible for Upscale and maximum long edge "
|
||||
"used by Depth, Normal and Region refinement"
|
||||
"used by Depth, Normal and Selection refinement"
|
||||
),
|
||||
default=DEFAULT_MAX_AI_INPUT_SIZE,
|
||||
min=MIN_MAX_AI_INPUT_SIZE,
|
||||
|
||||
@@ -216,17 +216,17 @@ def upscale_model_label(model_key):
|
||||
return model.label if model is not None else model_key
|
||||
|
||||
|
||||
def refine_region_property():
|
||||
def refine_selection_property():
|
||||
return bpy.props.BoolProperty(
|
||||
name="Fit to Foreground",
|
||||
description="Use BEN2 to fit the gesture mask to foreground content",
|
||||
name="Refine Selection",
|
||||
description="Use BEN2 to refine the selection to foreground content",
|
||||
default=False,
|
||||
)
|
||||
|
||||
|
||||
def cutout_generate_normal_property():
|
||||
return bpy.props.BoolProperty(
|
||||
name="Generate Normal Map",
|
||||
name="Normal Map",
|
||||
description=(
|
||||
"Generate a MoGe-2 normal texture; Depth Surface uses Object Space "
|
||||
"and other shapes use Tangent Space"
|
||||
@@ -235,25 +235,26 @@ def cutout_generate_normal_property():
|
||||
)
|
||||
|
||||
|
||||
def cutout_point_spacing_property():
|
||||
return bpy.props.IntProperty(
|
||||
name="Mesh Sampling Spacing",
|
||||
description=(
|
||||
"Source image pixels between neighboring mesh points; smaller values "
|
||||
"create denser geometry"
|
||||
def cutout_mesh_detail_property():
|
||||
return bpy.props.EnumProperty(
|
||||
name="Mesh Detail",
|
||||
description="Mesh sampling density",
|
||||
items=(
|
||||
("LOW", "Low", "32 source image pixels between neighboring mesh points"),
|
||||
("MEDIUM", "Medium", "16 source image pixels between neighboring mesh points"),
|
||||
("HIGH", "High", "8 source image pixels between neighboring mesh points"),
|
||||
("ULTRA", "Ultra", "4 source image pixels between neighboring mesh points"),
|
||||
),
|
||||
default=32,
|
||||
min=4,
|
||||
max=32,
|
||||
default="LOW",
|
||||
)
|
||||
|
||||
|
||||
class AnyImageSettings(bpy.types.PropertyGroup):
|
||||
cutout_point_spacing: cutout_point_spacing_property()
|
||||
cutout_mesh_detail: cutout_mesh_detail_property()
|
||||
cutout_generate_normal: cutout_generate_normal_property()
|
||||
cutout_refine_region: refine_region_property()
|
||||
image_box_refine_region: refine_region_property()
|
||||
image_lasso_refine_region: refine_region_property()
|
||||
cutout_refine_selection: refine_selection_property()
|
||||
image_box_refine_selection: refine_selection_property()
|
||||
image_lasso_refine_selection: refine_selection_property()
|
||||
input_path: bpy.props.StringProperty(
|
||||
name="Image or Video",
|
||||
description="A single image or video file used for reconstruction",
|
||||
|
||||
@@ -16,7 +16,7 @@ from .jobs.debug_predictions import (
|
||||
)
|
||||
from .jobs.cutout import run as _generate_cutout_artifacts
|
||||
from .jobs.depth_plane import run as _generate_depth_plane_geometry
|
||||
from .jobs.refine_image_region import run as _refine_image_region
|
||||
from .jobs.refine_image_selection import run as _refine_image_selection
|
||||
from .jobs.remove_background import run as _remove_background
|
||||
from .jobs.upscale import run as _upscale_image
|
||||
from .model_manager import ModelManager
|
||||
@@ -41,9 +41,9 @@ def download_required_models(context, parameters):
|
||||
return manager.download_missing(context, parameters["models"])
|
||||
|
||||
|
||||
@server.job("refine-image-region")
|
||||
def refine_image_region(context, parameters):
|
||||
return _refine_image_region(context, parameters)
|
||||
@server.job("refine-image-selection")
|
||||
def refine_image_selection(context, parameters):
|
||||
return _refine_image_selection(context, parameters)
|
||||
|
||||
|
||||
@server.job("remove-background")
|
||||
|
||||
@@ -71,31 +71,33 @@ def _write_float_exr(rgba, output_path):
|
||||
return output_path
|
||||
|
||||
|
||||
def write_vector_depth_texture(points, valid, output_path):
|
||||
"""Write camera-space XYZ and validity as a float RGBA texture."""
|
||||
def write_vector_depth_texture(frame, output_path):
|
||||
"""Write a frame's camera-space XYZ and validity as float RGBA."""
|
||||
import numpy as np
|
||||
|
||||
points = np.asarray(points, dtype=np.float32)
|
||||
valid = np.asarray(valid, dtype=bool)
|
||||
if points.ndim != 3 or points.shape[2] != 3 or valid.shape != points.shape[:2]:
|
||||
raise ValueError("Vector depth dimensions do not match")
|
||||
rgba = np.zeros((*valid.shape, 4), dtype=np.float32)
|
||||
rgba[valid, :3] = points[valid]
|
||||
rgba[..., 3] = valid
|
||||
if frame.points is None:
|
||||
raise ValueError("Vector depth requires a point field")
|
||||
points = np.asarray(frame.points, dtype=np.float32)
|
||||
validity = np.asarray(frame.validity, dtype=np.float32)
|
||||
if not np.isfinite(points).all():
|
||||
raise ValueError("Point field contains non-finite values")
|
||||
rgba = np.empty((*validity.shape, 4), dtype=np.float32)
|
||||
rgba[..., :3] = points
|
||||
rgba[..., 3] = validity
|
||||
return _write_float_exr(rgba, output_path)
|
||||
|
||||
|
||||
def write_z_depth_texture(depth, valid, output_path):
|
||||
"""Write camera Z and validity as a float RGBA texture."""
|
||||
def write_z_depth_texture(frame, output_path):
|
||||
"""Write a frame's camera Z and validity as float RGBA."""
|
||||
import numpy as np
|
||||
|
||||
depth = np.asarray(depth, dtype=np.float32)
|
||||
valid = np.asarray(valid, dtype=bool)
|
||||
if depth.ndim != 2 or valid.shape != depth.shape:
|
||||
raise ValueError("Z depth dimensions do not match")
|
||||
rgba = np.zeros((*depth.shape, 4), dtype=np.float32)
|
||||
rgba[..., :3] = np.where(valid, depth, 0.0)[..., None]
|
||||
rgba[..., 3] = valid
|
||||
depth = np.asarray(frame.depth, dtype=np.float32)
|
||||
validity = np.asarray(frame.validity, dtype=np.float32)
|
||||
if not np.isfinite(depth).all():
|
||||
raise ValueError("Depth field contains non-finite values")
|
||||
rgba = np.empty((*depth.shape, 4), dtype=np.float32)
|
||||
rgba[..., :3] = depth[..., None]
|
||||
rgba[..., 3] = validity
|
||||
return _write_float_exr(rgba, output_path)
|
||||
|
||||
|
||||
@@ -104,15 +106,19 @@ def write_depth_metadata(frame, reference_valid, output_path):
|
||||
import numpy as np
|
||||
|
||||
depth = np.asarray(frame.depth, dtype=np.float32)
|
||||
valid = np.asarray(frame.valid_mask, dtype=bool)
|
||||
validity = np.asarray(frame.validity, dtype=np.float32)
|
||||
intrinsics = np.asarray(frame.intrinsics, dtype=np.float32)
|
||||
reference_valid = np.asarray(reference_valid, dtype=bool) & valid
|
||||
if depth.ndim != 2 or valid.shape != depth.shape:
|
||||
reference_valid = np.asarray(reference_valid, dtype=bool)
|
||||
if depth.ndim != 2 or validity.shape != depth.shape:
|
||||
raise ValueError("Depth metadata dimensions do not match")
|
||||
if reference_valid.shape != depth.shape:
|
||||
raise ValueError("Depth metadata reference mask dimensions do not match")
|
||||
if intrinsics.shape != (3, 3):
|
||||
raise ValueError("Depth metadata requires 3x3 camera intrinsics")
|
||||
valid = (validity > 0.0) & np.isfinite(depth) & (depth > 0.0)
|
||||
reference_valid &= valid
|
||||
if not valid.any():
|
||||
raise ValueError("MoGe-2 returned no valid depth pixels")
|
||||
raise ValueError("Geometry frame contains no valid depth pixels")
|
||||
if not reference_valid.any():
|
||||
raise ValueError("The image contains no valid depth pixels")
|
||||
metadata = {
|
||||
|
||||
@@ -3,14 +3,14 @@
|
||||
from pathlib import Path
|
||||
|
||||
|
||||
def resized_valid_mask(valid_mask, shape):
|
||||
def resized_visibility(visibility, shape):
|
||||
import numpy as np
|
||||
from PIL import Image
|
||||
|
||||
valid_mask = np.asarray(valid_mask, dtype=bool)
|
||||
if valid_mask.shape == tuple(shape):
|
||||
return valid_mask
|
||||
resized = Image.fromarray(valid_mask.astype(np.uint8) * 255, mode="L")
|
||||
visibility = np.asarray(visibility, dtype=bool)
|
||||
if visibility.shape == tuple(shape):
|
||||
return visibility
|
||||
resized = Image.fromarray(visibility.astype(np.uint8) * 255, mode="L")
|
||||
resized = resized.resize(
|
||||
(int(shape[1]), int(shape[0])),
|
||||
Image.Resampling.NEAREST,
|
||||
@@ -19,8 +19,8 @@ def resized_valid_mask(valid_mask, shape):
|
||||
|
||||
|
||||
def write_camera_normal_texture(
|
||||
normal,
|
||||
valid,
|
||||
frame,
|
||||
visibility,
|
||||
origin,
|
||||
crop_size,
|
||||
source_size,
|
||||
@@ -31,12 +31,14 @@ def write_camera_normal_texture(
|
||||
import numpy as np
|
||||
from PIL import Image
|
||||
|
||||
normal = np.asarray(normal, dtype=np.float32)
|
||||
valid = np.asarray(valid, dtype=bool)
|
||||
if normal.ndim != 3 or normal.shape[2] != 3:
|
||||
raise ValueError("Depth Balloon normal data must have three components")
|
||||
if valid.shape != normal.shape[:2]:
|
||||
valid = resized_valid_mask(valid, normal.shape[:2])
|
||||
if frame.normal is None:
|
||||
raise ValueError("Normal texture requires a normal field")
|
||||
normal = np.asarray(frame.normal, dtype=np.float32)
|
||||
visibility = np.asarray(visibility, dtype=bool)
|
||||
if not np.isfinite(normal).all():
|
||||
raise ValueError("Normal field contains non-finite values")
|
||||
if visibility.shape != normal.shape[:2]:
|
||||
visibility = resized_visibility(visibility, normal.shape[:2])
|
||||
normal_mode = str(normal_mode).upper()
|
||||
if normal_mode == "TANGENT":
|
||||
converted = normal * np.asarray((1.0, -1.0, -1.0), dtype=np.float32)
|
||||
@@ -55,7 +57,7 @@ def write_camera_normal_texture(
|
||||
0.0,
|
||||
255.0,
|
||||
).astype(np.uint8)
|
||||
encoded[~valid] = empty_normal
|
||||
encoded[~visibility] = empty_normal
|
||||
crop_width, crop_height = (int(value) for value in crop_size)
|
||||
crop = Image.fromarray(encoded, mode="RGB")
|
||||
if crop.size != (crop_width, crop_height):
|
||||
|
||||
@@ -14,10 +14,10 @@ def _limited_model_input(source_path, output_path, max_input_size):
|
||||
from PIL import Image
|
||||
|
||||
with Image.open(source_path) as opened:
|
||||
limited = limit_image(opened, max_input_size)
|
||||
if limited is opened:
|
||||
return source_path
|
||||
limited.save(output_path, format="PNG", compress_level=1)
|
||||
limited = limit_image(opened.convert("RGBA"), max_input_size)
|
||||
background = Image.new("RGBA", limited.size, (0, 0, 0, 255))
|
||||
normalized = Image.alpha_composite(background, limited).convert("RGB")
|
||||
normalized.save(output_path, format="PNG", compress_level=1)
|
||||
return output_path
|
||||
|
||||
|
||||
@@ -34,16 +34,14 @@ def _alpha_mask(image_path, shape):
|
||||
return np.asarray(alpha, dtype=np.uint8) >= 26
|
||||
|
||||
|
||||
def _write_normal(prediction, color_path, output_path, normal_mode):
|
||||
import numpy as np
|
||||
def _write_normal(frame, color_path, output_path, normal_mode):
|
||||
from PIL import Image
|
||||
|
||||
normal = np.asarray(prediction.normal[0], dtype=np.float32)
|
||||
valid = _alpha_mask(color_path, normal.shape[:2])
|
||||
valid = _alpha_mask(color_path, frame.depth.shape)
|
||||
with Image.open(color_path) as source:
|
||||
output_size = source.size
|
||||
return write_camera_normal_texture(
|
||||
normal,
|
||||
frame,
|
||||
valid,
|
||||
(0, 0),
|
||||
output_size,
|
||||
@@ -74,36 +72,34 @@ def generate_moge2_artifacts(
|
||||
parameters.get("max_input_size", DEFAULT_MAX_AI_INPUT_SIZE),
|
||||
)
|
||||
try:
|
||||
prediction = moge2.infer(
|
||||
frames = moge2.infer(
|
||||
context.resource("model_manager"),
|
||||
parameters,
|
||||
[model_path],
|
||||
include_points=depth_mode == "VECTOR",
|
||||
cancel_check=context.check_cancelled,
|
||||
)
|
||||
if len(frames) != 1:
|
||||
raise ValueError("Artifact generation requires one geometry frame")
|
||||
frame = frames[0]
|
||||
result = {}
|
||||
if depth_mode != "NONE":
|
||||
context.progress(0.45, "Generating depth texture")
|
||||
frame = moge2.geometry_frame(prediction)
|
||||
if depth_mode == "VECTOR":
|
||||
if frame.points is None:
|
||||
raise ValueError("Vector depth requires a MoGe-2 point field")
|
||||
depth_path = write_vector_depth_texture(
|
||||
frame.points,
|
||||
frame.valid_mask,
|
||||
frame,
|
||||
context.directory / "vector-depth.exr",
|
||||
)
|
||||
result["vector_depth"] = depth_path
|
||||
else:
|
||||
depth_path = write_z_depth_texture(
|
||||
frame.depth,
|
||||
frame.valid_mask,
|
||||
frame,
|
||||
context.directory / "z-depth.exr",
|
||||
)
|
||||
result["z_depth"] = depth_path
|
||||
result["depth_metadata"] = write_depth_metadata(
|
||||
frame,
|
||||
_alpha_mask(input_path, frame.valid_mask.shape),
|
||||
_alpha_mask(input_path, frame.depth.shape),
|
||||
context.directory / "depth.json",
|
||||
)
|
||||
|
||||
@@ -111,7 +107,7 @@ def generate_moge2_artifacts(
|
||||
context.progress(0.72, "Generating surface normal")
|
||||
key = f"{normal_mode.lower()}_normal"
|
||||
result[key] = _write_normal(
|
||||
prediction,
|
||||
frame,
|
||||
input_path,
|
||||
context.directory / f"{normal_mode.lower()}-normal.png",
|
||||
normal_mode,
|
||||
|
||||
@@ -1,35 +1,35 @@
|
||||
"""Generate optional AI files for one Blender-rasterized Cutout Region."""
|
||||
"""Generate optional AI files for one Blender-rasterized Cutout Selection."""
|
||||
|
||||
import json
|
||||
from pathlib import Path
|
||||
|
||||
from ..geometry.prediction_artifacts import generate_moge2_artifacts
|
||||
from ..region import refine_region
|
||||
from ..selection import refine_selection
|
||||
|
||||
|
||||
def run(context, parameters):
|
||||
context.progress(0.05, "Preparing Cutout Region")
|
||||
context.progress(0.05, "Preparing Cutout Selection")
|
||||
context.directory.mkdir(parents=True, exist_ok=True)
|
||||
input_path = Path(parameters["input"])
|
||||
region_image_path = input_path
|
||||
selection_image_path = input_path
|
||||
result = {}
|
||||
if parameters.get("refine_region"):
|
||||
context.progress(0.2, "Refining Region")
|
||||
region_image_path, bounds = refine_region(
|
||||
if parameters.get("refine_selection"):
|
||||
context.progress(0.2, "Refining Selection")
|
||||
selection_image_path, bounds = refine_selection(
|
||||
context,
|
||||
input_path,
|
||||
parameters["refine_model"],
|
||||
parameters["device"],
|
||||
context.directory / "region.png",
|
||||
context.directory / "selection.png",
|
||||
)
|
||||
bounds_path = context.directory / "region_bounds.json"
|
||||
bounds_path = context.directory / "selection_bounds.json"
|
||||
bounds_path.write_text(
|
||||
json.dumps(bounds, separators=(",", ":")),
|
||||
encoding="utf-8",
|
||||
)
|
||||
result.update(
|
||||
region_image=region_image_path.name,
|
||||
region_bounds=bounds_path.name,
|
||||
selection_image=selection_image_path.name,
|
||||
selection_bounds=bounds_path.name,
|
||||
)
|
||||
|
||||
depth_mode = str(parameters.get("depth_mode", "NONE"))
|
||||
@@ -38,7 +38,7 @@ def run(context, parameters):
|
||||
artifacts = generate_moge2_artifacts(
|
||||
context,
|
||||
parameters,
|
||||
region_image_path,
|
||||
selection_image_path,
|
||||
depth_mode=depth_mode,
|
||||
normal_mode=normal_mode,
|
||||
)
|
||||
|
||||
@@ -1,18 +1,18 @@
|
||||
import json
|
||||
import time
|
||||
|
||||
from ..region import refine_region
|
||||
from ..selection import refine_selection
|
||||
|
||||
|
||||
def run(context, parameters):
|
||||
"""Refine a gesture mask with BEN2 and return its source-image bounds."""
|
||||
started = time.perf_counter()
|
||||
context.progress(0.05, "Preparing image Region")
|
||||
context.progress(0.05, "Preparing image Selection")
|
||||
context.check_cancelled()
|
||||
output_directory = context.directory
|
||||
output_directory.mkdir(parents=True, exist_ok=True)
|
||||
context.progress(0.2, f"Loading BEN2 for {parameters['device'].upper()}")
|
||||
output_path, bounds = refine_region(
|
||||
output_path, bounds = refine_selection(
|
||||
context,
|
||||
parameters["input"],
|
||||
parameters["model"],
|
||||
@@ -22,6 +22,6 @@ def run(context, parameters):
|
||||
bounds_path = output_directory / "bounds.json"
|
||||
bounds_path.write_text(json.dumps(bounds, separators=(",", ":")), encoding="utf-8")
|
||||
elapsed_ms = (time.perf_counter() - started) * 1000.0
|
||||
message = f"Image Region refined in {elapsed_ms:.0f} ms"
|
||||
message = f"Image Selection refined in {elapsed_ms:.0f} ms"
|
||||
context.progress(1.0, message)
|
||||
return {"region_image": output_path.name, "region_bounds": bounds_path.name}
|
||||
return {"selection_image": output_path.name, "selection_bounds": bounds_path.name}
|
||||
@@ -71,24 +71,26 @@ def infer(
|
||||
return merge_frame_predictions(predictions)
|
||||
|
||||
|
||||
def geometry_frame(prediction):
|
||||
def geometry_frames(prediction):
|
||||
import numpy as np
|
||||
|
||||
depth = np.asarray(prediction.depth, dtype=np.float32)
|
||||
confidence = np.asarray(prediction.conf, dtype=np.float32)
|
||||
validity = np.asarray(prediction.conf, dtype=np.float32)
|
||||
intrinsics = np.asarray(prediction.intrinsics, dtype=np.float32)
|
||||
if depth.shape[0] != 1:
|
||||
raise ValueError("Image mesh generation currently supports one image")
|
||||
points = unproject_camera_depth(depth[0], intrinsics[0])
|
||||
valid = (
|
||||
np.isfinite(depth[0])
|
||||
& (depth[0] > 0.0)
|
||||
& np.isfinite(points).all(axis=-1)
|
||||
if validity.shape != depth.shape or intrinsics.shape != (len(depth), 3, 3):
|
||||
raise ValueError("DA3 prediction fields do not share one frame layout")
|
||||
return [
|
||||
GeometryFrame(
|
||||
depth=frame_depth,
|
||||
validity=frame_validity,
|
||||
intrinsics=frame_intrinsics,
|
||||
points=unproject_camera_depth(frame_depth, frame_intrinsics).astype(
|
||||
np.float32
|
||||
),
|
||||
)
|
||||
return GeometryFrame(
|
||||
depth=depth[0],
|
||||
points=points,
|
||||
intrinsics=intrinsics[0],
|
||||
valid_mask=valid,
|
||||
confidence=confidence[0],
|
||||
for frame_depth, frame_validity, frame_intrinsics in zip(
|
||||
depth,
|
||||
validity,
|
||||
intrinsics,
|
||||
)
|
||||
]
|
||||
|
||||
@@ -4,8 +4,29 @@ from dataclasses import dataclass
|
||||
@dataclass
|
||||
class GeometryFrame:
|
||||
depth: object
|
||||
points: object | None
|
||||
validity: object
|
||||
intrinsics: object
|
||||
valid_mask: object
|
||||
confidence: object | None = None
|
||||
model_mask: object | None = None
|
||||
normal: object | None = None
|
||||
points: object | None = None
|
||||
|
||||
def __post_init__(self):
|
||||
import numpy as np
|
||||
|
||||
depth = np.asarray(self.depth)
|
||||
validity = np.asarray(self.validity)
|
||||
intrinsics = np.asarray(self.intrinsics)
|
||||
if depth.dtype != np.float32 or depth.ndim != 2:
|
||||
raise ValueError("Geometry depth must be a 2D float32 field")
|
||||
if validity.dtype != np.float32 or validity.shape != depth.shape:
|
||||
raise ValueError("Geometry validity must match the depth as float32")
|
||||
if intrinsics.dtype != np.float32 or intrinsics.shape != (3, 3):
|
||||
raise ValueError("Geometry intrinsics must be a 3x3 float32 matrix")
|
||||
for name in ("normal", "points"):
|
||||
value = getattr(self, name)
|
||||
if value is None:
|
||||
continue
|
||||
field = np.asarray(value)
|
||||
if field.dtype != np.float32 or field.shape != (*depth.shape, 3):
|
||||
raise ValueError(
|
||||
f"Geometry {name} must match the depth as three-channel float32"
|
||||
)
|
||||
|
||||
@@ -1,26 +1,37 @@
|
||||
from dataclasses import dataclass
|
||||
|
||||
from .geometry import GeometryFrame
|
||||
|
||||
|
||||
@dataclass
|
||||
class Prediction:
|
||||
depth: object
|
||||
normal: object
|
||||
mask: object
|
||||
intrinsics: object
|
||||
points: object | None
|
||||
def _pixel_intrinsics(intrinsics, image_size):
|
||||
import numpy as np
|
||||
|
||||
def archive_values(self):
|
||||
values = {
|
||||
"depth": self.depth,
|
||||
"normal": self.normal,
|
||||
"mask": self.mask,
|
||||
"intrinsics": self.intrinsics,
|
||||
}
|
||||
if self.points is not None:
|
||||
values["points"] = self.points
|
||||
return values
|
||||
width, height = image_size
|
||||
result = np.asarray(intrinsics, dtype=np.float32).copy()
|
||||
if (
|
||||
0.0 <= float(result[0, 2]) <= 1.0
|
||||
and 0.0 <= float(result[1, 2]) <= 1.0
|
||||
):
|
||||
result[0] *= width
|
||||
result[1] *= height
|
||||
result[2] = (0.0, 0.0, 1.0)
|
||||
return result
|
||||
|
||||
|
||||
def _geometry_frame(prediction, include_points):
|
||||
import numpy as np
|
||||
|
||||
depth = np.asarray(prediction["depth"], dtype=np.float32)
|
||||
height, width = depth.shape
|
||||
return GeometryFrame(
|
||||
depth=depth,
|
||||
normal=np.asarray(prediction["normal"], dtype=np.float32),
|
||||
validity=np.asarray(prediction["mask"], dtype=np.float32),
|
||||
intrinsics=_pixel_intrinsics(prediction["intrinsics"], (width, height)),
|
||||
points=(
|
||||
np.asarray(prediction["points"], dtype=np.float32)
|
||||
if include_points
|
||||
else None
|
||||
),
|
||||
)
|
||||
|
||||
|
||||
def infer(
|
||||
@@ -32,41 +43,19 @@ def infer(
|
||||
progress_callback=None,
|
||||
cancel_check=None,
|
||||
):
|
||||
import numpy as np
|
||||
depths = []
|
||||
normals = []
|
||||
masks = []
|
||||
intrinsics = []
|
||||
points = []
|
||||
frames = []
|
||||
for index, image_path in enumerate(images):
|
||||
if cancel_check is not None:
|
||||
cancel_check()
|
||||
if progress_callback is not None:
|
||||
progress_callback(index, len(images))
|
||||
prediction = infer_one(
|
||||
model_manager,
|
||||
parameters,
|
||||
image_path,
|
||||
frames.append(
|
||||
_geometry_frame(
|
||||
infer_one(model_manager, parameters, image_path),
|
||||
include_points,
|
||||
)
|
||||
depths.append(prediction["depth"])
|
||||
normals.append(prediction["normal"])
|
||||
masks.append(prediction["mask"].astype(bool))
|
||||
intrinsics.append(prediction["intrinsics"])
|
||||
if include_points:
|
||||
points.append(prediction["points"])
|
||||
if len({array.shape for array in depths}) != 1:
|
||||
raise RuntimeError("MoGe-2 frame sizes must match")
|
||||
return Prediction(
|
||||
depth=np.stack(depths).astype(np.float32),
|
||||
normal=np.stack(normals).astype(np.float32),
|
||||
mask=np.stack(masks),
|
||||
intrinsics=np.stack(intrinsics).astype(np.float32),
|
||||
points=(
|
||||
np.stack(points).astype(np.float32)
|
||||
if include_points
|
||||
else None
|
||||
),
|
||||
)
|
||||
return frames
|
||||
|
||||
|
||||
def infer_one(model_manager, parameters, image_path):
|
||||
@@ -87,35 +76,3 @@ def infer_one(model_manager, parameters, image_path):
|
||||
image,
|
||||
int(parameters["resolution_level"]),
|
||||
)
|
||||
|
||||
|
||||
def geometry_frame(prediction):
|
||||
import numpy as np
|
||||
|
||||
if prediction.depth.shape[0] != 1:
|
||||
raise ValueError("Image depth generation requires one MoGe-2 frame")
|
||||
depth = prediction.depth[0]
|
||||
points = prediction.points[0] if prediction.points is not None else None
|
||||
intrinsics = prediction.intrinsics[0].copy()
|
||||
height, width = depth.shape
|
||||
if (
|
||||
0.0 <= float(intrinsics[0, 2]) <= 1.0
|
||||
and 0.0 <= float(intrinsics[1, 2]) <= 1.0
|
||||
):
|
||||
intrinsics[0] *= width
|
||||
intrinsics[1] *= height
|
||||
intrinsics[2] = (0.0, 0.0, 1.0)
|
||||
valid = (
|
||||
prediction.mask[0]
|
||||
& np.isfinite(depth)
|
||||
& (depth > 0.0)
|
||||
)
|
||||
if points is not None:
|
||||
valid &= np.isfinite(points).all(axis=-1)
|
||||
return GeometryFrame(
|
||||
depth=depth,
|
||||
points=points,
|
||||
intrinsics=intrinsics,
|
||||
valid_mask=valid,
|
||||
model_mask=prediction.mask[0],
|
||||
)
|
||||
|
||||
@@ -1,5 +0,0 @@
|
||||
"""AI generation for rasterized image Regions."""
|
||||
|
||||
from .refine_region import refine_region
|
||||
|
||||
__all__ = ("refine_region",)
|
||||
5
src/anyimage/server/selection/__init__.py
Normal file
5
src/anyimage/server/selection/__init__.py
Normal file
@@ -0,0 +1,5 @@
|
||||
"""AI generation for rasterized image Selections."""
|
||||
|
||||
from .refine_selection import refine_selection
|
||||
|
||||
__all__ = ("refine_selection",)
|
||||
@@ -7,7 +7,7 @@ def _crop_to_alpha(image):
|
||||
pixels = np.asarray(image.convert("RGBA"), dtype=np.uint8)
|
||||
visible_y, visible_x = np.nonzero(pixels[:, :, 3] > 0)
|
||||
if not len(visible_x):
|
||||
raise ValueError("The generated Region contains no visible pixels")
|
||||
raise ValueError("The generated Selection contains no visible pixels")
|
||||
bounds = (
|
||||
int(visible_x.min()),
|
||||
int(visible_y.min()),
|
||||
@@ -17,14 +17,14 @@ def _crop_to_alpha(image):
|
||||
return image.crop(bounds), bounds
|
||||
|
||||
|
||||
def refine_region(
|
||||
def refine_selection(
|
||||
context,
|
||||
input_path,
|
||||
model,
|
||||
device,
|
||||
output_path,
|
||||
):
|
||||
"""Refine an image Region from an already rasterized Region."""
|
||||
"""Refine an image Selection from an already rasterized Selection."""
|
||||
model_manager = context.resource("model_manager")
|
||||
foreground, _load_ms, _inference_ms = infer_foreground(
|
||||
input_path,
|
||||
@@ -84,7 +84,8 @@ class BlenderAddonTest(unittest.TestCase):
|
||||
cls.anyimage = importlib.import_module("anyimage")
|
||||
cls.runtime = importlib.import_module("anyimage.runtime")
|
||||
cls.ai_setup = importlib.import_module("anyimage.operators.ai_setup")
|
||||
cls.image_data = importlib.import_module("anyimage.common.image_data")
|
||||
cls.ai = importlib.import_module("anyimage.common.ai")
|
||||
cls.image_data = importlib.import_module("anyimage.common.image")
|
||||
cls.debug_operators = importlib.import_module("anyimage.operators.debug")
|
||||
cls.convert_to_plane = importlib.import_module(
|
||||
"anyimage.operators.convert_to_plane.conversion"
|
||||
@@ -96,15 +97,11 @@ class BlenderAddonTest(unittest.TestCase):
|
||||
"anyimage.operators.remove_background"
|
||||
)
|
||||
cls.upscale = importlib.import_module("anyimage.operators.upscale")
|
||||
cls.image_interaction = importlib.import_module(
|
||||
"anyimage.common.image_interaction"
|
||||
)
|
||||
cls.image_interaction = importlib.import_module("anyimage.common.viewport")
|
||||
cls.edit_mask = importlib.import_module(
|
||||
"anyimage.operators.image_tool.edit_mask"
|
||||
)
|
||||
cls.image_region = importlib.import_module(
|
||||
"anyimage.common.image_region"
|
||||
)
|
||||
cls.image_selection = importlib.import_module("anyimage.common.selection")
|
||||
cls.image_box = importlib.import_module("anyimage.operators.image_tool.box")
|
||||
cls.image_lasso = importlib.import_module("anyimage.operators.image_tool.lasso")
|
||||
cls.image_line = importlib.import_module("anyimage.operators.image_tool.line")
|
||||
@@ -218,8 +215,8 @@ class BlenderAddonTest(unittest.TestCase):
|
||||
for operator, identifier in (
|
||||
(self.anyimage.operators.GenerateDepthPlane, "anyimage.generate_depth_plane"),
|
||||
(
|
||||
self.anyimage.operators.CutoutRegionToShape,
|
||||
"anyimage.cutout_region_to_shape",
|
||||
self.anyimage.operators.CutoutSelectionToShape,
|
||||
"anyimage.cutout_selection_to_shape",
|
||||
),
|
||||
):
|
||||
self.assertEqual(operator.bl_idname, identifier)
|
||||
@@ -837,7 +834,7 @@ class BlenderAddonTest(unittest.TestCase):
|
||||
operator = self.convert_to_plane.ConvertToPlane
|
||||
self.assertIn(operator, self.anyimage.CLASSES)
|
||||
self.assertIn(self.convert_to_plane.ConvertToDepthPlane, self.anyimage.CLASSES)
|
||||
self.assertIn(self.cutout_main.CutoutRegionToShape, self.anyimage.CLASSES)
|
||||
self.assertIn(self.cutout_main.CutoutSelectionToShape, self.anyimage.CLASSES)
|
||||
self.assertEqual(operator.bl_idname, "anyimage.convert_to_plane")
|
||||
self.assertNotIn("shadeless", operator.__annotations__)
|
||||
self.assertNotIn("keep_original", operator.__annotations__)
|
||||
@@ -926,7 +923,7 @@ class BlenderAddonTest(unittest.TestCase):
|
||||
context = SimpleNamespace(
|
||||
view_layer=SimpleNamespace(objects=SimpleNamespace(active=None))
|
||||
)
|
||||
region_mask = self.cutout_main.RegionMask(
|
||||
selection_mask = self.cutout_main.SelectionMask(
|
||||
np.ones((8, 8), dtype=np.float32), (1, 1, 9, 9)
|
||||
)
|
||||
source_rgba = np.ones((10, 10, 4), dtype=np.float32)
|
||||
@@ -934,13 +931,13 @@ class BlenderAddonTest(unittest.TestCase):
|
||||
with self.subTest(shape=shape):
|
||||
operator = SimpleNamespace(
|
||||
source_object_name="Source",
|
||||
region_path_json=json.dumps(
|
||||
selection_path_json=json.dumps(
|
||||
{"points": [[1, 1], [9, 1], [9, 9], [1, 9]]}
|
||||
),
|
||||
shape=shape,
|
||||
mesh_resolution=64,
|
||||
generate_normal=False,
|
||||
refine_region=False,
|
||||
refine_selection=False,
|
||||
report=lambda *_args: None,
|
||||
_needs_generation=lambda: False,
|
||||
)
|
||||
@@ -958,8 +955,8 @@ class BlenderAddonTest(unittest.TestCase):
|
||||
),
|
||||
patch.object(
|
||||
self.cutout_main,
|
||||
"rasterize_region_path",
|
||||
return_value=region_mask,
|
||||
"rasterize_selection_path",
|
||||
return_value=selection_mask,
|
||||
),
|
||||
patch.object(
|
||||
self.cutout_main,
|
||||
@@ -975,7 +972,7 @@ class BlenderAddonTest(unittest.TestCase):
|
||||
self.cutout_main, "create_cutout_shape"
|
||||
) as convert,
|
||||
):
|
||||
result = self.cutout_main.CutoutRegionToShape.execute(
|
||||
result = self.cutout_main.CutoutSelectionToShape.execute(
|
||||
operator, context
|
||||
)
|
||||
|
||||
@@ -985,12 +982,12 @@ class BlenderAddonTest(unittest.TestCase):
|
||||
source,
|
||||
shape,
|
||||
64,
|
||||
region_mask,
|
||||
selection_mask,
|
||||
source_rgba=source_rgba,
|
||||
)
|
||||
|
||||
def test_cutout_request_describes_ai_capabilities(self):
|
||||
operator = self.cutout_main.CutoutRegionToShape()
|
||||
operator = self.cutout_main.CutoutSelectionToShape()
|
||||
operator.input_path = __file__
|
||||
operator.source_object_name = "Source"
|
||||
operator.shape = "DEPTH_SURFACE"
|
||||
@@ -998,7 +995,7 @@ class BlenderAddonTest(unittest.TestCase):
|
||||
operator.resolution_level = 5
|
||||
operator.max_ai_input_size = 2048
|
||||
operator.generate_normal = True
|
||||
operator.refine_region = True
|
||||
operator.refine_selection = True
|
||||
context = SimpleNamespace(
|
||||
scene=SimpleNamespace(anyimage_settings=SimpleNamespace())
|
||||
)
|
||||
@@ -1021,7 +1018,7 @@ class BlenderAddonTest(unittest.TestCase):
|
||||
[call.args[0] for call in require_model.call_args_list],
|
||||
[self.anyimage.properties.BEN2_MODEL_KEY, operator.model],
|
||||
)
|
||||
self.assertTrue(parameters["refine_region"])
|
||||
self.assertTrue(parameters["refine_selection"])
|
||||
self.assertEqual(parameters["depth_mode"], "VECTOR")
|
||||
self.assertEqual(parameters["normal_mode"], "OBJECT")
|
||||
self.assertNotIn("polygon", parameters)
|
||||
@@ -1290,7 +1287,7 @@ class BlenderAddonTest(unittest.TestCase):
|
||||
def test_cutout_contour_is_smooth_and_evenly_resampled(self):
|
||||
import numpy as np
|
||||
|
||||
contour = self.cutout_geometry.smooth_region_contour(
|
||||
contour = self.cutout_geometry.smooth_selection_contour(
|
||||
((0, 0), (3, 0), (4, 0), (4, 2), (0, 2)),
|
||||
0.25,
|
||||
)
|
||||
@@ -1309,7 +1306,7 @@ class BlenderAddonTest(unittest.TestCase):
|
||||
staircase.append((value + 1.0, float(value)))
|
||||
staircase.append((value + 1.0, value + 1.0))
|
||||
staircase.append((0.0, 16.0))
|
||||
smooth = self.cutout_geometry.smooth_region_contour(staircase, 2.0)
|
||||
smooth = self.cutout_geometry.smooth_selection_contour(staircase, 2.0)
|
||||
|
||||
def diagonal_noise(points):
|
||||
points = np.asarray(points)
|
||||
@@ -1322,7 +1319,7 @@ class BlenderAddonTest(unittest.TestCase):
|
||||
|
||||
self.assertLess(diagonal_noise(smooth), 0.05)
|
||||
|
||||
def test_region_mask_keeps_nonzero_winding_overlap(self):
|
||||
def test_selection_mask_keeps_nonzero_winding_overlap(self):
|
||||
path = (
|
||||
(1, 1),
|
||||
(7, 1),
|
||||
@@ -1337,13 +1334,13 @@ class BlenderAddonTest(unittest.TestCase):
|
||||
(1, 1),
|
||||
)
|
||||
|
||||
even_odd = self.image_region.rasterize_region_path(
|
||||
even_odd = self.image_selection.rasterize_selection_path(
|
||||
(12, 9),
|
||||
self.image_region.RegionPath(points=path, fill_rule="EVEN_ODD"),
|
||||
self.image_selection.SelectionPath(points=path, fill_rule="EVEN_ODD"),
|
||||
).full_values((12, 9))
|
||||
nonzero = self.image_region.rasterize_region_path(
|
||||
nonzero = self.image_selection.rasterize_selection_path(
|
||||
(12, 9),
|
||||
self.image_region.RegionPath(points=path, fill_rule="NONZERO"),
|
||||
self.image_selection.SelectionPath(points=path, fill_rule="NONZERO"),
|
||||
).full_values((12, 9))
|
||||
|
||||
self.assertFalse(even_odd[3, 5])
|
||||
@@ -1397,7 +1394,7 @@ class BlenderAddonTest(unittest.TestCase):
|
||||
def test_image_edit_tools_and_operators_are_registered(self):
|
||||
operator = self.edit_mask.EditImageMask
|
||||
self.assertIn(operator, self.anyimage.CLASSES)
|
||||
self.assertIn(self.edit_mask.RefineImageRegion, self.anyimage.CLASSES)
|
||||
self.assertIn(self.edit_mask.RefineImageSelection, self.anyimage.CLASSES)
|
||||
self.assertIn(self.edit_mask.ActivateImageTool, self.anyimage.CLASSES)
|
||||
tool_cases = (
|
||||
(self.image_box.ImageBoxTool, "BOX", "ops.sculpt.border_mask"),
|
||||
@@ -1422,12 +1419,12 @@ class BlenderAddonTest(unittest.TestCase):
|
||||
layout = SimpleNamespace(
|
||||
prop=lambda _settings, name: drawn_settings.append(name)
|
||||
)
|
||||
with patch.object(self.image_box, "draw_region_settings") as draw:
|
||||
with patch.object(self.image_box, "draw_selection_settings") as draw:
|
||||
self.image_box.ImageBoxTool.draw_settings(context, layout, None)
|
||||
draw.assert_called_once_with(context, layout, "image_box_refine_region")
|
||||
with patch.object(self.image_lasso, "draw_region_settings") as draw:
|
||||
draw.assert_called_once_with(context, layout, "image_box_refine_selection")
|
||||
with patch.object(self.image_lasso, "draw_selection_settings") as draw:
|
||||
self.image_lasso.ImageLassoTool.draw_settings(context, layout, None)
|
||||
draw.assert_called_once_with(context, layout, "image_lasso_refine_region")
|
||||
draw.assert_called_once_with(context, layout, "image_lasso_refine_selection")
|
||||
cutout_tool = importlib.import_module(
|
||||
"anyimage.operators.cutout_tool"
|
||||
).CutoutTool
|
||||
@@ -1453,9 +1450,9 @@ class BlenderAddonTest(unittest.TestCase):
|
||||
self.assertEqual(
|
||||
drawn_settings,
|
||||
[
|
||||
"cutout_refine_region",
|
||||
"cutout_point_spacing",
|
||||
"cutout_refine_selection",
|
||||
"cutout_generate_normal",
|
||||
"cutout_mesh_detail",
|
||||
],
|
||||
)
|
||||
self.assertNotIn("draw_settings", self.image_line.ImageLineTool.__dict__)
|
||||
@@ -1468,24 +1465,33 @@ class BlenderAddonTest(unittest.TestCase):
|
||||
self.assertNotIn("padding", operator.__annotations__)
|
||||
settings = self.anyimage.properties.AnyImageSettings.__annotations__
|
||||
for name in (
|
||||
"cutout_refine_region",
|
||||
"image_box_refine_region",
|
||||
"image_lasso_refine_region",
|
||||
"cutout_refine_selection",
|
||||
"image_box_refine_selection",
|
||||
"image_lasso_refine_selection",
|
||||
):
|
||||
self.assertFalse(settings[name]["default"])
|
||||
self.assertNotIn("refine_region", settings)
|
||||
self.assertNotIn("image_edit_refine_region", settings)
|
||||
self.assertEqual(settings["cutout_point_spacing"]["default"], 32)
|
||||
self.assertEqual(settings["cutout_point_spacing"]["min"], 4)
|
||||
self.assertEqual(settings["cutout_point_spacing"]["max"], 32)
|
||||
self.assertFalse(settings["cutout_generate_normal"]["default"])
|
||||
self.assertEqual(settings["cutout_refine_region"]["name"], "Fit to Foreground")
|
||||
self.assertNotIn("refine_selection", settings)
|
||||
self.assertNotIn("image_edit_refine_selection", settings)
|
||||
self.assertEqual(settings["cutout_mesh_detail"]["default"], "LOW")
|
||||
self.assertEqual(
|
||||
settings["cutout_point_spacing"]["name"], "Mesh Sampling Spacing"
|
||||
[item[:2] for item in settings["cutout_mesh_detail"]["items"]],
|
||||
[
|
||||
("LOW", "Low"),
|
||||
("MEDIUM", "Medium"),
|
||||
("HIGH", "High"),
|
||||
("ULTRA", "Ultra"),
|
||||
],
|
||||
)
|
||||
self.assertFalse(settings["cutout_generate_normal"]["default"])
|
||||
self.assertEqual(
|
||||
settings["cutout_refine_selection"]["name"], "Refine Selection"
|
||||
)
|
||||
self.assertEqual(
|
||||
settings["cutout_mesh_detail"]["name"], "Mesh Detail"
|
||||
)
|
||||
self.assertEqual(
|
||||
settings["cutout_generate_normal"]["name"],
|
||||
"Generate Normal Map",
|
||||
"Normal Map",
|
||||
)
|
||||
self.assertNotIn("image_extract_continuous", settings)
|
||||
self.assertNotIn("image_extract_padding", settings)
|
||||
@@ -1522,10 +1528,10 @@ class BlenderAddonTest(unittest.TestCase):
|
||||
"ready": False,
|
||||
}
|
||||
setup = ("operator", "anyimage.setup_ai_environment", True)
|
||||
with patch.object(self.image_interaction, "ai_status", return_value=missing):
|
||||
with patch.object(self.ai, "ai_status", return_value=missing):
|
||||
for tool, setting_name in (
|
||||
(self.image_box.ImageBoxTool, "image_box_refine_region"),
|
||||
(self.image_lasso.ImageLassoTool, "image_lasso_refine_region"),
|
||||
(self.image_box.ImageBoxTool, "image_box_refine_selection"),
|
||||
(self.image_lasso.ImageLassoTool, "image_lasso_refine_selection"),
|
||||
):
|
||||
layout = Layout()
|
||||
tool.draw_settings(context, layout, None)
|
||||
@@ -1543,71 +1549,71 @@ class BlenderAddonTest(unittest.TestCase):
|
||||
layout.events,
|
||||
[
|
||||
setup,
|
||||
("prop", "cutout_refine_region", False),
|
||||
("prop", "cutout_point_spacing", True),
|
||||
("prop", "cutout_refine_selection", False),
|
||||
("prop", "cutout_generate_normal", False),
|
||||
("prop", "cutout_mesh_detail", True),
|
||||
],
|
||||
)
|
||||
|
||||
layout = Layout()
|
||||
operator = SimpleNamespace(gesture="BOX")
|
||||
self.image_interaction.draw_region_operator_settings(layout, operator)
|
||||
self.image_interaction.draw_selection_operator_settings(layout, operator)
|
||||
self.assertEqual(
|
||||
layout.events,
|
||||
[
|
||||
setup,
|
||||
("prop", "refine_region", False),
|
||||
("prop", "refine_selection", False),
|
||||
("prop", "invert", True),
|
||||
],
|
||||
)
|
||||
|
||||
ready = {"environment_ready": True, "missing_models": (), "ready": True}
|
||||
with patch.object(self.image_interaction, "ai_status", return_value=ready):
|
||||
with patch.object(self.ai, "ai_status", return_value=ready):
|
||||
layout = Layout()
|
||||
cutout_tool.draw_settings(context, layout, None)
|
||||
self.assertNotIn(setup, layout.events)
|
||||
self.assertTrue(all(event[2] for event in layout.events))
|
||||
|
||||
def test_refine_region_is_independent_for_each_area_tool(self):
|
||||
def test_refine_selection_is_independent_for_each_area_tool(self):
|
||||
settings = SimpleNamespace(
|
||||
cutout_refine_region=True,
|
||||
image_box_refine_region=False,
|
||||
image_lasso_refine_region=True,
|
||||
cutout_refine_selection=True,
|
||||
image_box_refine_selection=False,
|
||||
image_lasso_refine_selection=True,
|
||||
)
|
||||
context = SimpleNamespace(scene=SimpleNamespace(anyimage_settings=settings))
|
||||
with patch.object(self.image_interaction, "ai_ready", return_value=True):
|
||||
for gesture in ("LINE", "POLYLINE"):
|
||||
operator = SimpleNamespace(
|
||||
gesture=gesture,
|
||||
refine_region=True,
|
||||
refine_selection=True,
|
||||
)
|
||||
self.image_interaction.apply_region_settings(operator, context)
|
||||
self.assertFalse(operator.refine_region)
|
||||
self.image_interaction.apply_selection_settings(operator, context)
|
||||
self.assertFalse(operator.refine_selection)
|
||||
|
||||
box = SimpleNamespace(
|
||||
gesture="BOX",
|
||||
refine_region=True,
|
||||
refine_region_settings={"BOX": "image_box_refine_region"},
|
||||
refine_selection=True,
|
||||
refine_selection_settings={"BOX": "image_box_refine_selection"},
|
||||
)
|
||||
self.image_interaction.apply_region_settings(box, context)
|
||||
self.assertFalse(box.refine_region)
|
||||
self.image_interaction.apply_selection_settings(box, context)
|
||||
self.assertFalse(box.refine_selection)
|
||||
|
||||
lasso = SimpleNamespace(
|
||||
gesture="LASSO",
|
||||
refine_region=False,
|
||||
refine_region_settings={"LASSO": "image_lasso_refine_region"},
|
||||
refine_selection=False,
|
||||
refine_selection_settings={"LASSO": "image_lasso_refine_selection"},
|
||||
)
|
||||
self.image_interaction.apply_region_settings(lasso, context)
|
||||
self.assertTrue(lasso.refine_region)
|
||||
self.image_interaction.apply_selection_settings(lasso, context)
|
||||
self.assertTrue(lasso.refine_selection)
|
||||
|
||||
cutout = SimpleNamespace(
|
||||
gesture="LASSO",
|
||||
active_tool_id="anyimage.cutout_lasso",
|
||||
refine_region=False,
|
||||
refine_region_settings={"LASSO": "cutout_refine_region"},
|
||||
refine_selection=False,
|
||||
refine_selection_settings={"LASSO": "cutout_refine_selection"},
|
||||
)
|
||||
self.image_interaction.apply_region_settings(cutout, context)
|
||||
self.assertTrue(cutout.refine_region)
|
||||
self.image_interaction.apply_selection_settings(cutout, context)
|
||||
self.assertTrue(cutout.refine_selection)
|
||||
|
||||
def test_image_edit_first_click_selects_another_image_without_editing(self):
|
||||
selected = []
|
||||
@@ -1981,7 +1987,7 @@ class BlenderAddonTest(unittest.TestCase):
|
||||
self.assertNotIn("image_perspective_aspect_ratio", settings)
|
||||
self.assertNotIn("aspect_mode", operator.__annotations__)
|
||||
self.assertIn("aspect_ratio", operator.__annotations__)
|
||||
self.assertNotIn("refine_region", operator.__annotations__)
|
||||
self.assertNotIn("refine_selection", operator.__annotations__)
|
||||
self.assertNotIn("continuous", operator.__annotations__)
|
||||
self.assertNotIn("padding", operator.__annotations__)
|
||||
|
||||
@@ -1995,14 +2001,14 @@ class BlenderAddonTest(unittest.TestCase):
|
||||
(),
|
||||
)
|
||||
|
||||
def test_image_box_completion_serializes_the_image_region(self):
|
||||
def test_image_box_completion_serializes_the_image_selection(self):
|
||||
source = SimpleNamespace(data=SimpleNamespace(size=(64, 32)))
|
||||
finished = []
|
||||
operator = SimpleNamespace(
|
||||
source_object_name="Source",
|
||||
source_matrix_data="matrix",
|
||||
gesture="BOX",
|
||||
region_path_json="",
|
||||
selection_path_json="",
|
||||
invert=False,
|
||||
preview_fill_rule="EVEN_ODD",
|
||||
_screen_path=lambda: ((1, 1), (8, 1), (8, 6), (1, 6)),
|
||||
@@ -2024,16 +2030,16 @@ class BlenderAddonTest(unittest.TestCase):
|
||||
return_value=polygon,
|
||||
),
|
||||
):
|
||||
result = self.image_interaction.ImageRegionGesture._complete(operator, context)
|
||||
result = self.image_interaction.ImageSelectionGesture._complete(operator, context)
|
||||
|
||||
self.assertEqual(result, {"FINISHED"})
|
||||
self.assertEqual(
|
||||
json.loads(operator.region_path_json)["points"],
|
||||
json.loads(operator.selection_path_json)["points"],
|
||||
[[1.0, 2.0], [3.0, 4.0], [5.0, 6.0]],
|
||||
)
|
||||
self.assertEqual(finished, [context])
|
||||
|
||||
def test_image_region_uses_white_dashed_segments(self):
|
||||
def test_image_selection_uses_white_dashed_segments(self):
|
||||
segments = self.image_interaction.dashed_line_segments(
|
||||
((0, 0), (10, 0)), dash_length=3.0, gap_length=2.0
|
||||
)
|
||||
@@ -2073,8 +2079,8 @@ class BlenderAddonTest(unittest.TestCase):
|
||||
patch.object(self.image_interaction, "draw_polygon_fill") as draw_fill,
|
||||
patch.object(self.image_interaction, "draw_dashed_line") as draw_outline,
|
||||
):
|
||||
self.image_interaction.ImageRegionGesture._draw_region(operator)
|
||||
self.image_interaction.ImageRegionGesture._draw_region(operator)
|
||||
self.image_interaction.ImageSelectionGesture._draw_selection(operator)
|
||||
self.image_interaction.ImageSelectionGesture._draw_selection(operator)
|
||||
|
||||
build_fill.assert_called_once_with(polygon, "NONZERO")
|
||||
build_outline.assert_called_once_with((*polygon, polygon[0]))
|
||||
@@ -2131,11 +2137,11 @@ class BlenderAddonTest(unittest.TestCase):
|
||||
((0, 5), (10, 5)), (10, 10)
|
||||
)
|
||||
self.assertEqual(
|
||||
self.image_region.region_path_bounds(preview),
|
||||
self.image_selection.selection_path_bounds(preview),
|
||||
(0, 5, 10, 10),
|
||||
)
|
||||
self.assertEqual(
|
||||
self.image_region.region_path_bounds(image),
|
||||
self.image_selection.selection_path_bounds(image),
|
||||
(0, 0, 10, 5),
|
||||
)
|
||||
|
||||
|
||||
@@ -453,6 +453,28 @@ class ClipboardImageTest(unittest.TestCase):
|
||||
|
||||
self.assertIs(result, expected_reference)
|
||||
|
||||
def test_reference_image_uses_drag_and_drop_size(self):
|
||||
reference = SimpleNamespace(select_set=Mock())
|
||||
objects = SimpleNamespace(new=Mock(return_value=reference))
|
||||
context = SimpleNamespace(
|
||||
mode="OBJECT",
|
||||
collection=SimpleNamespace(objects=SimpleNamespace(link=Mock())),
|
||||
selected_objects=(),
|
||||
view_layer=SimpleNamespace(objects=SimpleNamespace(active=None)),
|
||||
region_data=None,
|
||||
)
|
||||
image = SimpleNamespace(name="Clipboard", size=(1920, 1080))
|
||||
|
||||
with patch.object(self.actions.bpy.data, "objects", objects, create=True):
|
||||
result = self.actions.add_reference_image(
|
||||
context,
|
||||
image,
|
||||
location=(1.0, 2.0, 3.0),
|
||||
)
|
||||
|
||||
self.assertIs(result, reference)
|
||||
self.assertEqual(reference.empty_display_size, 5.0)
|
||||
|
||||
def test_only_import_as_and_shadeless_are_drawn_for_plane_paste(self):
|
||||
operator = self.addon.clipboard_image.PasteClipboardImage()
|
||||
drawn_properties = []
|
||||
|
||||
@@ -10,12 +10,12 @@ import numpy as np
|
||||
SOURCE_ROOT = Path(__file__).parents[1] / "src"
|
||||
sys.path.insert(0, str(SOURCE_ROOT))
|
||||
|
||||
from anyimage.common.image_region import (
|
||||
RegionMask,
|
||||
RegionPath,
|
||||
rasterize_region_path,
|
||||
from anyimage.common.selection import (
|
||||
SelectionMask,
|
||||
SelectionPath,
|
||||
rasterize_selection_path,
|
||||
)
|
||||
from anyimage.common.depth_data import (
|
||||
from anyimage.common.depth import (
|
||||
depth_uniform_scale,
|
||||
load_depth_metadata,
|
||||
)
|
||||
@@ -25,11 +25,13 @@ from anyimage.operators.cutout_tool import geometry as cutout_geometry
|
||||
from anyimage.operators.cutout_tool import object as cutout_object
|
||||
from anyimage.operators.cutout_tool import shape as cutout_shape
|
||||
from anyimage.operators.cutout_tool.geometry import (
|
||||
RegionContours,
|
||||
extract_region_contours,
|
||||
SelectionContours,
|
||||
extract_selection_contours,
|
||||
poisson_balloon_profile,
|
||||
)
|
||||
from anyimage.server.models import moge2 as moge2_model
|
||||
from anyimage.server.models import da3 as da3_model
|
||||
from anyimage.server.models.geometry import GeometryFrame
|
||||
|
||||
|
||||
def test_cutout_mesh_detail_uses_the_cropped_analysis_scale():
|
||||
@@ -193,53 +195,102 @@ def test_depth_surface_projection_uses_one_scale_for_all_camera_axes():
|
||||
assert np.isclose(reference_depth, 5.8, atol=1e-6)
|
||||
|
||||
|
||||
def test_moge_surface_field_uses_the_model_validity_mask():
|
||||
prediction = moge2_model.Prediction(
|
||||
depth=np.ones((1, 2, 2), dtype=np.float32),
|
||||
normal=np.zeros((1, 2, 2, 3), dtype=np.float32),
|
||||
mask=np.asarray((((True, False), (True, True)),)),
|
||||
intrinsics=np.asarray(
|
||||
(((1.0, 0.0, 0.5), (0.0, 1.0, 0.5), (0.0, 0.0, 1.0)),),
|
||||
dtype=np.float32,
|
||||
),
|
||||
points=np.ones((1, 2, 2, 3), dtype=np.float32),
|
||||
def test_geometry_frame_requires_matching_float32_fields():
|
||||
with np.testing.assert_raises_regex(ValueError, "validity"):
|
||||
GeometryFrame(
|
||||
depth=np.ones((2, 2), dtype=np.float32),
|
||||
validity=np.ones((1, 2), dtype=np.float32),
|
||||
intrinsics=np.eye(3, dtype=np.float32),
|
||||
)
|
||||
with np.testing.assert_raises_regex(ValueError, "normal"):
|
||||
GeometryFrame(
|
||||
depth=np.ones((2, 2), dtype=np.float32),
|
||||
validity=np.ones((2, 2), dtype=np.float32),
|
||||
intrinsics=np.eye(3, dtype=np.float32),
|
||||
normal=np.ones((2, 2, 2), dtype=np.float32),
|
||||
)
|
||||
|
||||
frame = moge2_model.geometry_frame(prediction)
|
||||
|
||||
assert np.array_equal(frame.valid_mask, prediction.mask[0])
|
||||
def test_moge_infer_returns_geometry_frames_with_model_validity():
|
||||
mask = np.asarray(((True, False), (True, True)))
|
||||
prediction = {
|
||||
"depth": np.ones((2, 2), dtype=np.float32),
|
||||
"normal": np.zeros((2, 2, 3), dtype=np.float32),
|
||||
"mask": mask,
|
||||
"intrinsics": np.asarray(
|
||||
((1.0, 0.0, 0.5), (0.0, 1.0, 0.5), (0.0, 0.0, 1.0)),
|
||||
dtype=np.float32,
|
||||
),
|
||||
"points": np.ones((2, 2, 3), dtype=np.float32),
|
||||
}
|
||||
|
||||
with patch.object(moge2_model, "infer_one", return_value=prediction):
|
||||
frames = moge2_model.infer(None, {}, ["image"], include_points=True)
|
||||
|
||||
assert len(frames) == 1
|
||||
frame = frames[0]
|
||||
assert np.array_equal(frame.validity, mask.astype(np.float32))
|
||||
assert np.array_equal(frame.normal, prediction["normal"])
|
||||
assert np.array_equal(frame.points, prediction["points"])
|
||||
assert np.array_equal(
|
||||
frame.intrinsics,
|
||||
np.asarray(((2.0, 0.0, 1.0), (0.0, 2.0, 1.0), (0.0, 0.0, 1.0))),
|
||||
)
|
||||
|
||||
|
||||
def test_moge_z_depth_frame_does_not_require_points():
|
||||
prediction = moge2_model.Prediction(
|
||||
depth=np.ones((1, 2, 2), dtype=np.float32),
|
||||
normal=np.zeros((1, 2, 2, 3), dtype=np.float32),
|
||||
mask=np.ones((1, 2, 2), dtype=bool),
|
||||
intrinsics=np.eye(3, dtype=np.float32)[None],
|
||||
points=None,
|
||||
)
|
||||
prediction = {
|
||||
"depth": np.ones((2, 2), dtype=np.float32),
|
||||
"normal": np.zeros((2, 2, 3), dtype=np.float32),
|
||||
"mask": np.ones((2, 2), dtype=bool),
|
||||
"intrinsics": np.eye(3, dtype=np.float32),
|
||||
"points": np.ones((2, 2, 3), dtype=np.float32),
|
||||
}
|
||||
|
||||
frame = moge2_model.geometry_frame(prediction)
|
||||
with patch.object(moge2_model, "infer_one", return_value=prediction):
|
||||
frame = moge2_model.infer(None, {}, ["image"], include_points=False)[0]
|
||||
|
||||
assert frame.points is None
|
||||
assert frame.valid_mask.all()
|
||||
assert frame.validity.all()
|
||||
|
||||
|
||||
def test_region_contours_group_holes_with_their_connected_component():
|
||||
def test_da3_geometry_frames_use_confidence_as_validity_and_derive_points():
|
||||
prediction = SimpleNamespace(
|
||||
depth=np.ones((2, 2, 3), dtype=np.float32),
|
||||
conf=np.asarray(
|
||||
(
|
||||
((0.25, 0.5, 0.75), (1.0, 0.5, 0.25)),
|
||||
((0.1, 0.2, 0.3), (0.4, 0.5, 0.6)),
|
||||
),
|
||||
dtype=np.float32,
|
||||
),
|
||||
intrinsics=np.repeat(np.eye(3, dtype=np.float32)[None], 2, axis=0),
|
||||
)
|
||||
|
||||
frames = da3_model.geometry_frames(prediction)
|
||||
|
||||
assert len(frames) == 2
|
||||
assert frames[0].normal is None
|
||||
assert np.array_equal(frames[1].validity, prediction.conf[1])
|
||||
assert frames[0].points.shape == (2, 3, 3)
|
||||
assert frames[0].points.dtype == np.float32
|
||||
|
||||
|
||||
def test_selection_contours_group_holes_with_their_connected_component():
|
||||
alpha = np.zeros((120, 200), dtype=np.uint8)
|
||||
alpha[20:100, 50:160] = 255
|
||||
alpha[45:75, 80:120] = 0
|
||||
alpha[5:15, 5:15] = 255
|
||||
|
||||
region_contours = extract_region_contours(
|
||||
RegionMask(alpha, (0, 0, 200, 120)),
|
||||
selection_contours = extract_selection_contours(
|
||||
SelectionMask(alpha, (0, 0, 200, 120)),
|
||||
max_resolution=200,
|
||||
)
|
||||
|
||||
assert region_contours.image_size == (200, 120)
|
||||
assert sorted(len(component) for component in region_contours.components) == [1, 2]
|
||||
assert selection_contours.image_size == (200, 120)
|
||||
assert sorted(len(component) for component in selection_contours.components) == [1, 2]
|
||||
contours = [
|
||||
contour for component in region_contours.components for contour in component
|
||||
contour for component in selection_contours.components for contour in component
|
||||
]
|
||||
bounds = [
|
||||
(*np.min(contour, axis=0), *np.max(contour, axis=0)) for contour in contours
|
||||
@@ -249,19 +300,19 @@ def test_region_contours_group_holes_with_their_connected_component():
|
||||
assert (5.0, 5.0, 15.0, 15.0) in bounds
|
||||
|
||||
|
||||
def test_region_contours_accept_blender_float_mask_with_a_hole():
|
||||
def test_selection_contours_accept_blender_float_mask_with_a_hole():
|
||||
alpha = np.zeros((120, 200), dtype=np.float32)
|
||||
alpha[20:100, 50:160] = 1.0
|
||||
alpha[45:75, 80:120] = 0.0
|
||||
|
||||
region_contours = extract_region_contours(
|
||||
RegionMask(alpha, (0, 0, 200, 120)),
|
||||
selection_contours = extract_selection_contours(
|
||||
SelectionMask(alpha, (0, 0, 200, 120)),
|
||||
max_resolution=200,
|
||||
)
|
||||
|
||||
assert region_contours.image_size == (200, 120)
|
||||
assert len(region_contours.components) == 1
|
||||
assert len(region_contours.components[0]) == 2
|
||||
assert selection_contours.image_size == (200, 120)
|
||||
assert len(selection_contours.components) == 1
|
||||
assert len(selection_contours.components[0]) == 2
|
||||
|
||||
|
||||
def test_cutout_crop_bounds_map_to_the_source_plane():
|
||||
@@ -285,14 +336,14 @@ def test_depth_surface_cutout_uses_half_alpha_boundary():
|
||||
alpha[2:10, 2:14] = 0.49
|
||||
alpha[4:8, 5:11] = 0.51
|
||||
|
||||
region_contours = cutout_geometry.extract_cutout_contours(
|
||||
RegionMask(alpha, (0, 0, 16, 12)),
|
||||
selection_contours = cutout_geometry.extract_cutout_contours(
|
||||
SelectionMask(alpha, (0, 0, 16, 12)),
|
||||
64,
|
||||
threshold=0.5,
|
||||
)
|
||||
contour = np.asarray(region_contours.components[0][0])
|
||||
contour = np.asarray(selection_contours.components[0][0])
|
||||
|
||||
assert region_contours.image_size == (16, 12)
|
||||
assert selection_contours.image_size == (16, 12)
|
||||
np.testing.assert_allclose(contour.min(axis=0), (5.0, 4.0))
|
||||
np.testing.assert_allclose(contour.max(axis=0), (11.0, 8.0))
|
||||
|
||||
@@ -306,37 +357,37 @@ def test_blender_cutout_alpha_pipeline_does_not_import_pillow():
|
||||
return original_import(name, *args, **kwargs)
|
||||
|
||||
with patch("builtins.__import__", side_effect=without_pillow):
|
||||
region_mask = rasterize_region_path(
|
||||
selection_mask = rasterize_selection_path(
|
||||
(48, 32),
|
||||
RegionPath(points=((6, 4), (42, 4), (42, 28), (6, 28))),
|
||||
SelectionPath(points=((6, 4), (42, 4), (42, 28), (6, 28))),
|
||||
antialias=True,
|
||||
)
|
||||
antialiased = region_mask.full_values((48, 32))
|
||||
region_contours = extract_region_contours(
|
||||
RegionMask(antialiased, (0, 0, 48, 32)),
|
||||
antialiased = selection_mask.full_values((48, 32))
|
||||
selection_contours = extract_selection_contours(
|
||||
SelectionMask(antialiased, (0, 0, 48, 32)),
|
||||
max_resolution=24,
|
||||
)
|
||||
|
||||
assert region_contours.image_size == (48, 32)
|
||||
assert region_contours.components
|
||||
assert selection_contours.image_size == (48, 32)
|
||||
assert selection_contours.components
|
||||
assert np.any((antialiased > 0.0) & (antialiased < 1.0))
|
||||
|
||||
|
||||
def test_blender_cutout_contour_resolution_tracks_mesh_detail():
|
||||
expected = RegionContours(
|
||||
expected = SelectionContours(
|
||||
components=(((((0, 0), (1, 0), (0, 1))),),),
|
||||
image_size=(1, 1),
|
||||
)
|
||||
region_mask = RegionMask(np.ones((2, 2), dtype=np.float32), (0, 0, 2, 2))
|
||||
with patch.object(cutout_geometry, "extract_region_contours", return_value=expected) as trace:
|
||||
assert cutout_geometry.extract_cutout_contours(region_mask, 32) == expected
|
||||
assert cutout_geometry.extract_cutout_contours(region_mask, 256) == expected
|
||||
selection_mask = SelectionMask(np.ones((2, 2), dtype=np.float32), (0, 0, 2, 2))
|
||||
with patch.object(cutout_geometry, "extract_selection_contours", return_value=expected) as trace:
|
||||
assert cutout_geometry.extract_cutout_contours(selection_mask, 32) == expected
|
||||
assert cutout_geometry.extract_cutout_contours(selection_mask, 256) == expected
|
||||
|
||||
assert trace.call_args_list[0].kwargs["max_resolution"] == 512
|
||||
assert trace.call_args_list[1].kwargs["max_resolution"] == 1024
|
||||
|
||||
|
||||
def test_cutout_point_spacing_uses_the_complete_source_plane():
|
||||
def test_cutout_sampling_spacing_uses_the_complete_source_plane():
|
||||
source = SimpleNamespace(matrix_world=object())
|
||||
with (
|
||||
patch.object(cutout_geometry, "image_empty_bounds", return_value=(-2.0, 2.0, -1.0, 1.0)),
|
||||
@@ -346,7 +397,7 @@ def test_cutout_point_spacing_uses_the_complete_source_plane():
|
||||
return_value=(((1.0, 0.0), (0.0, 1.0)), 1.0),
|
||||
),
|
||||
):
|
||||
spacing = cutout_geometry.cutout_point_spacing(source, 200)
|
||||
spacing = cutout_geometry.cutout_sampling_spacing(source, 200)
|
||||
|
||||
assert spacing == 0.02
|
||||
|
||||
@@ -401,7 +452,7 @@ def test_poisson_balloon_profile_scales_in_world_units():
|
||||
assert np.isclose(scaled[4], original[4] * 2.0, atol=1e-6)
|
||||
|
||||
|
||||
def test_poisson_balloon_profile_makes_wide_regions_thicker():
|
||||
def test_poisson_balloon_profile_makes_wide_selections_thicker():
|
||||
faces = ((0, 1, 4), (1, 2, 4), (2, 3, 4), (3, 0, 4))
|
||||
boundary = np.asarray((True, True, True, True, False))
|
||||
wide = np.asarray(
|
||||
@@ -445,11 +496,11 @@ def test_teddy_balloon_profile_uses_chordal_axis_radius():
|
||||
assert np.isclose(profile[4], 0.45, atol=1e-6)
|
||||
|
||||
|
||||
def test_cutout_domain_adapts_boundary_spacing_for_a_small_valid_region():
|
||||
regions = ((((2046, 2046), (2050, 2046), (2050, 2050), (2046, 2050)),),)
|
||||
def test_cutout_domain_adapts_boundary_spacing_for_a_small_valid_selection():
|
||||
components = ((((2046, 2046), (2050, 2046), (2050, 2050), (2046, 2050)),),)
|
||||
|
||||
domain = cutout_geometry.cutout_domain(
|
||||
RegionContours(regions, (4096, 4096)),
|
||||
SelectionContours(components, (4096, 4096)),
|
||||
(-1.0, 1.0, -1.0, 1.0),
|
||||
0.5,
|
||||
((1.0, 0.0), (0.0, 1.0)),
|
||||
@@ -460,10 +511,10 @@ def test_cutout_domain_adapts_boundary_spacing_for_a_small_valid_region():
|
||||
|
||||
|
||||
def test_cutout_domain_builds_a_full_image_at_minimum_detail():
|
||||
regions = ((((0, 0), (4096, 0), (4096, 4096), (0, 4096)),),)
|
||||
components = ((((0, 0), (4096, 0), (4096, 4096), (0, 4096)),),)
|
||||
|
||||
domain = cutout_geometry.cutout_domain(
|
||||
RegionContours(regions, (4096, 4096)),
|
||||
SelectionContours(components, (4096, 4096)),
|
||||
(-1.0, 1.0, -1.0, 1.0),
|
||||
0.5,
|
||||
((1.0, 0.0), (0.0, 1.0)),
|
||||
@@ -474,10 +525,10 @@ def test_cutout_domain_builds_a_full_image_at_minimum_detail():
|
||||
|
||||
|
||||
def test_cutout_domain_centers_vertices_on_crop_bounds():
|
||||
regions = ((((0, 0), (100, 0), (100, 50), (0, 50)),),)
|
||||
components = ((((0, 0), (100, 0), (100, 50), (0, 50)),),)
|
||||
|
||||
vertices, _faces, uv, _boundary, _planar, _profile = cutout_geometry.cutout_domain(
|
||||
RegionContours(regions, (100, 50)),
|
||||
SelectionContours(components, (100, 50)),
|
||||
(2.0, 6.0, -3.0, 1.0),
|
||||
0.5,
|
||||
((1.0, 0.0), (0.0, 1.0)),
|
||||
@@ -489,8 +540,8 @@ def test_cutout_domain_centers_vertices_on_crop_bounds():
|
||||
assert np.allclose(uv, expected_uv, atol=1e-6)
|
||||
|
||||
|
||||
def test_cutout_domain_keeps_overlapping_regions_disconnected():
|
||||
regions = (
|
||||
def test_cutout_domain_keeps_overlapping_selection_components_disconnected():
|
||||
components = (
|
||||
(
|
||||
((20, 0), (100, 0), (100, 80), (20, 80)),
|
||||
((50, 20), (50, 60), (70, 60), (70, 20)),
|
||||
@@ -498,7 +549,7 @@ def test_cutout_domain_keeps_overlapping_regions_disconnected():
|
||||
(((0, 20), (40, 20), (40, 50), (0, 50)),),
|
||||
)
|
||||
domain = cutout_geometry.cutout_domain(
|
||||
RegionContours(regions, (120, 80)),
|
||||
SelectionContours(components, (120, 80)),
|
||||
(0.0, 1.0, 0.0, 1.0),
|
||||
0.025,
|
||||
((1.0, 0.0), (0.0, 1.0)),
|
||||
@@ -511,7 +562,7 @@ def test_cutout_domain_keeps_overlapping_regions_disconnected():
|
||||
& (centers[:, 1] > 0.35)
|
||||
& (centers[:, 1] < 0.65)
|
||||
)
|
||||
in_left_region = (
|
||||
in_left_component = (
|
||||
(centers[:, 0] > 0.02)
|
||||
& (centers[:, 0] < 0.12)
|
||||
& (centers[:, 1] > 0.4)
|
||||
@@ -519,17 +570,17 @@ def test_cutout_domain_keeps_overlapping_regions_disconnected():
|
||||
)
|
||||
|
||||
assert not in_hole.any()
|
||||
assert in_left_region.any()
|
||||
assert in_left_component.any()
|
||||
assert mesh_component_count(domain[1]) == 2
|
||||
|
||||
|
||||
def test_region_calibration_uses_the_final_region_mask():
|
||||
region_mask = RegionMask(
|
||||
def test_selection_calibration_uses_the_final_selection_mask():
|
||||
selection_mask = SelectionMask(
|
||||
np.asarray(((1.0, 1.0), (1.0, 0.0)), dtype=np.float32),
|
||||
(0, 0, 2, 2),
|
||||
)
|
||||
|
||||
valid, transparent_fraction = cutout_geometry.region_calibration(region_mask, (2, 2))
|
||||
valid, transparent_fraction = cutout_geometry.selection_calibration(selection_mask, (2, 2))
|
||||
|
||||
assert np.array_equal(valid, ((True, True), (True, False)))
|
||||
assert transparent_fraction == 0.25
|
||||
|
||||
@@ -7,8 +7,8 @@ import bpy
|
||||
SOURCE_ROOT = Path(__file__).parents[1] / "src"
|
||||
sys.path.insert(0, str(SOURCE_ROOT))
|
||||
|
||||
from anyimage.common.image_interaction import ImageRegionGesture
|
||||
from anyimage.common.image_region import RegionPath
|
||||
from anyimage.common.selection import SelectionPath
|
||||
from anyimage.common.viewport import ImageSelectionGesture
|
||||
from anyimage.operators.cutout_tool import interaction
|
||||
from anyimage.operators.cutout_tool import main as cutout_main
|
||||
|
||||
@@ -32,9 +32,9 @@ class WindowManager:
|
||||
draw(menu, None)
|
||||
|
||||
|
||||
def cutout_context(window_manager, point_spacing=32, generate_normal=False):
|
||||
def cutout_context(window_manager, mesh_detail="LOW", generate_normal=False):
|
||||
settings = SimpleNamespace(
|
||||
cutout_point_spacing=point_spacing,
|
||||
cutout_mesh_detail=mesh_detail,
|
||||
cutout_generate_normal=generate_normal,
|
||||
)
|
||||
return SimpleNamespace(
|
||||
@@ -57,8 +57,8 @@ def test_cutout_pie_uses_two_local_shapes_without_ai(monkeypatch):
|
||||
context,
|
||||
object(),
|
||||
source,
|
||||
RegionPath(points=((0, 0), (10, 0), (0, 10))),
|
||||
refine_region=True,
|
||||
SelectionPath(points=((0, 0), (10, 0), (0, 10))),
|
||||
refine_selection=True,
|
||||
)
|
||||
|
||||
assert [call[2].shape for call in window_manager.pie.calls] == [
|
||||
@@ -70,17 +70,17 @@ def test_cutout_pie_uses_two_local_shapes_without_ai(monkeypatch):
|
||||
"Balloon",
|
||||
]
|
||||
assert all(
|
||||
call[0] == "anyimage.cutout_region_to_shape"
|
||||
call[0] == "anyimage.cutout_selection_to_shape"
|
||||
for call in window_manager.pie.calls
|
||||
)
|
||||
assert not any(call[2].refine_region for call in window_manager.pie.calls)
|
||||
assert not any(call[2].refine_selection for call in window_manager.pie.calls)
|
||||
assert not any(call[2].generate_normal for call in window_manager.pie.calls)
|
||||
assert all(call[2].mesh_resolution == 64 for call in window_manager.pie.calls)
|
||||
|
||||
|
||||
def test_cutout_pie_passes_image_lasso_options_to_all_ai_shapes(monkeypatch):
|
||||
window_manager = WindowManager()
|
||||
context = cutout_context(window_manager, point_spacing=16, generate_normal=True)
|
||||
context = cutout_context(window_manager, mesh_detail="MEDIUM", generate_normal=True)
|
||||
source = cutout_source(size=(4096, 2048))
|
||||
monkeypatch.setattr(interaction, "ai_ready", lambda: True)
|
||||
|
||||
@@ -88,11 +88,11 @@ def test_cutout_pie_passes_image_lasso_options_to_all_ai_shapes(monkeypatch):
|
||||
context,
|
||||
object(),
|
||||
source,
|
||||
RegionPath(
|
||||
SelectionPath(
|
||||
points=((0, 0), (10, 0), (0, 10)),
|
||||
invert=True,
|
||||
),
|
||||
refine_region=True,
|
||||
refine_selection=True,
|
||||
)
|
||||
|
||||
assert [call[2].shape for call in window_manager.pie.calls] == [
|
||||
@@ -101,10 +101,10 @@ def test_cutout_pie_passes_image_lasso_options_to_all_ai_shapes(monkeypatch):
|
||||
"DEPTH_BALLOON",
|
||||
"DEPTH_SURFACE",
|
||||
]
|
||||
assert all(call[2].refine_region for call in window_manager.pie.calls)
|
||||
assert all(call[2].refine_selection for call in window_manager.pie.calls)
|
||||
assert all(call[2].generate_normal for call in window_manager.pie.calls)
|
||||
assert all(
|
||||
RegionPath.from_json(call[2].region_path_json).invert
|
||||
SelectionPath.from_json(call[2].selection_path_json).invert
|
||||
for call in window_manager.pie.calls
|
||||
)
|
||||
assert all(call[2].mesh_resolution == 256 for call in window_manager.pie.calls)
|
||||
@@ -124,6 +124,13 @@ def test_cutout_mesh_detail_tracks_the_source_long_edge_and_clamps():
|
||||
assert interaction.cutout_mesh_resolution((65536, 32768), 4) == 1024
|
||||
|
||||
|
||||
def test_cutout_mesh_detail_maps_to_denser_sampling():
|
||||
assert [
|
||||
interaction.cutout_mesh_detail_spacing(detail)
|
||||
for detail in ("LOW", "MEDIUM", "HIGH", "ULTRA")
|
||||
] == [32, 16, 8, 4]
|
||||
|
||||
|
||||
def test_cutout_reuses_image_lasso_gesture_and_double_click_selects_all(monkeypatch):
|
||||
bpy.ops.wm.read_factory_settings(use_empty=True)
|
||||
image = bpy.data.images.new("Source", width=320, height=180)
|
||||
@@ -138,23 +145,23 @@ def test_cutout_reuses_image_lasso_gesture_and_double_click_selects_all(monkeypa
|
||||
cutout_main, "resolve_image_edit_click", lambda *_args: (source, True)
|
||||
)
|
||||
monkeypatch.setattr(cutout_main, "video_total_frames", lambda _image: 1)
|
||||
monkeypatch.setattr(cutout_main, "apply_region_settings", lambda *_args: None)
|
||||
monkeypatch.setattr(cutout_main, "validate_cutout_region", lambda *_args: None)
|
||||
monkeypatch.setattr(cutout_main, "apply_selection_settings", lambda *_args: None)
|
||||
monkeypatch.setattr(cutout_main, "validate_cutout_selection", lambda *_args: None)
|
||||
|
||||
operator = SimpleNamespace(
|
||||
full_image=True,
|
||||
refine_region=False,
|
||||
refine_selection=False,
|
||||
report=lambda *_args: None,
|
||||
)
|
||||
operator.execute = MethodType(cutout_main.SelectCutoutRegion.execute, operator)
|
||||
result = cutout_main.SelectCutoutRegion.invoke(
|
||||
operator.execute = MethodType(cutout_main.SelectCutoutSelection.execute, operator)
|
||||
result = cutout_main.SelectCutoutSelection.invoke(
|
||||
operator, context, SimpleNamespace(value="DOUBLE_CLICK")
|
||||
)
|
||||
|
||||
assert issubclass(cutout_main.SelectCutoutRegion, ImageRegionGesture)
|
||||
assert issubclass(cutout_main.SelectCutoutSelection, ImageSelectionGesture)
|
||||
assert result == {"FINISHED"}
|
||||
region_path = RegionPath.from_json(window_manager.pie.calls[0][2].region_path_json)
|
||||
assert region_path.points == ((0, 0), (320, 0), (320, 180), (0, 180))
|
||||
selection_path = SelectionPath.from_json(window_manager.pie.calls[0][2].selection_path_json)
|
||||
assert selection_path.points == ((0, 0), (320, 0), (320, 180), (0, 180))
|
||||
assert window_manager.pie.calls[0][2].mesh_resolution == 10
|
||||
assert not hasattr(cutout_main, "pending_selection")
|
||||
|
||||
|
||||
@@ -10,21 +10,26 @@ from mathutils import Vector
|
||||
SOURCE_ROOT = Path(__file__).parents[1] / "src"
|
||||
sys.path.insert(0, str(SOURCE_ROOT))
|
||||
|
||||
from anyimage.common import image_data
|
||||
from anyimage.common import depth_data
|
||||
from anyimage.common import depth as depth_data
|
||||
from anyimage.common import image as image_data
|
||||
from anyimage.common import material as material_data
|
||||
from anyimage.common import node as node_asset
|
||||
from anyimage.common import object as object_data
|
||||
from anyimage.common.image_region import (
|
||||
RegionMask,
|
||||
RegionPath,
|
||||
rasterize_region_path,
|
||||
from anyimage.common.selection import (
|
||||
SelectionMask,
|
||||
SelectionPath,
|
||||
rasterize_selection_path,
|
||||
)
|
||||
from anyimage.operators.convert_to_plane import object as plane_object
|
||||
from anyimage.operators.cutout_tool import geometry as cutout_geometry
|
||||
from anyimage.operators.cutout_tool import main as cutout_main
|
||||
from anyimage.operators.cutout_tool import object as cutout_object
|
||||
from anyimage.operators.cutout_tool import shape as cutout_shape
|
||||
from anyimage.server.geometry.depth_texture import write_vector_depth_texture
|
||||
from anyimage.server.geometry.depth_texture import (
|
||||
write_vector_depth_texture,
|
||||
write_z_depth_texture,
|
||||
)
|
||||
from anyimage.server.models.geometry import GeometryFrame
|
||||
|
||||
|
||||
def _inputs(group):
|
||||
@@ -51,26 +56,26 @@ def _create_cutout_shape(
|
||||
bounds=None,
|
||||
depth_image=None,
|
||||
depth_metadata=None,
|
||||
region_values=None,
|
||||
selection_values=None,
|
||||
):
|
||||
if bounds is None:
|
||||
bounds = (0, 0, *tuple(int(value) for value in source.data.size))
|
||||
left, top, right, bottom = bounds
|
||||
if region_values is None:
|
||||
region_values = np.ones((bottom - top, right - left), dtype=np.float32)
|
||||
region_mask = RegionMask(region_values, bounds)
|
||||
if selection_values is None:
|
||||
selection_values = np.ones((bottom - top, right - left), dtype=np.float32)
|
||||
selection_mask = SelectionMask(selection_values, bounds)
|
||||
base_shape = cutout_geometry.build_base_shape(
|
||||
bpy.context,
|
||||
source,
|
||||
detail,
|
||||
region_mask,
|
||||
selection_mask,
|
||||
)
|
||||
return cutout_object.create_shape_object(
|
||||
bpy.context,
|
||||
source,
|
||||
shape,
|
||||
detail,
|
||||
region_mask,
|
||||
selection_mask,
|
||||
base_shape,
|
||||
depth_image=depth_image,
|
||||
depth_metadata=depth_metadata,
|
||||
@@ -82,7 +87,7 @@ def _depth_image(points, valid, name="Depth"):
|
||||
valid = np.asarray(valid, dtype=bool)
|
||||
height, width = valid.shape
|
||||
pixels = np.zeros((height, width, 4), dtype=np.float32)
|
||||
pixels[valid, :3] = points[valid]
|
||||
pixels[..., :3] = points
|
||||
pixels[..., 3] = valid
|
||||
image = bpy.data.images.new(
|
||||
name,
|
||||
@@ -112,9 +117,15 @@ def test_depth_result_image_loads_server_vector_exr_and_keeps_data_name(tmp_path
|
||||
source = bpy.data.objects.new("Source", None)
|
||||
source.data = source_image
|
||||
points = np.arange(12, dtype=np.float32).reshape((2, 2, 3))
|
||||
valid = np.asarray(((True, False), (True, True)))
|
||||
validity = np.asarray(((1.0, 0.0), (0.5, 1.0)), dtype=np.float32)
|
||||
frame = GeometryFrame(
|
||||
depth=points[..., 2].copy(),
|
||||
validity=validity,
|
||||
intrinsics=np.eye(3, dtype=np.float32),
|
||||
points=points,
|
||||
)
|
||||
path = tmp_path / "vector-depth.exr"
|
||||
write_vector_depth_texture(points, valid, path)
|
||||
write_vector_depth_texture(frame, path)
|
||||
metadata = _depth_metadata(np.eye(3), (2, 2), 1.0)
|
||||
|
||||
image = depth_data.load_depth_result_image(path, source, metadata)
|
||||
@@ -122,12 +133,38 @@ def test_depth_result_image_loads_server_vector_exr_and_keeps_data_name(tmp_path
|
||||
packed = np.empty(16, dtype=np.float32)
|
||||
image.pixels.foreach_get(packed)
|
||||
packed = np.flipud(packed.reshape((2, 2, 4)))
|
||||
assert np.array_equal(packed[valid, :3], points[valid])
|
||||
assert np.array_equal(packed[..., 3] > 0.5, valid)
|
||||
assert np.array_equal(packed[..., :3], points)
|
||||
assert np.array_equal(packed[..., 3], validity)
|
||||
assert image.alpha_mode == "CHANNEL_PACKED"
|
||||
assert image.name == "Source_depth.exr"
|
||||
|
||||
|
||||
def test_z_depth_texture_keeps_depth_outside_positive_validity(tmp_path):
|
||||
bpy.ops.wm.read_factory_settings(use_empty=True)
|
||||
source_image = bpy.data.images.new("Source.png", width=2, height=1)
|
||||
source = bpy.data.objects.new("Source", None)
|
||||
source.data = source_image
|
||||
frame = GeometryFrame(
|
||||
depth=np.asarray(((2.0, 7.0),), dtype=np.float32),
|
||||
validity=np.asarray(((1.0, 0.0),), dtype=np.float32),
|
||||
intrinsics=np.eye(3, dtype=np.float32),
|
||||
)
|
||||
path = tmp_path / "z-depth.exr"
|
||||
write_z_depth_texture(frame, path)
|
||||
|
||||
image = depth_data.load_depth_result_image(
|
||||
path,
|
||||
source,
|
||||
_depth_metadata(np.eye(3), (2, 1), 2.0),
|
||||
)
|
||||
|
||||
packed = np.empty(8, dtype=np.float32)
|
||||
image.pixels.foreach_get(packed)
|
||||
packed = packed.reshape((1, 2, 4))
|
||||
assert np.array_equal(packed[..., :3], np.repeat(frame.depth[..., None], 3, axis=2))
|
||||
assert np.array_equal(packed[..., 3], frame.validity)
|
||||
|
||||
|
||||
def test_cutout_modifier_wraps_four_menu_controlled_shape_groups():
|
||||
bpy.ops.wm.read_factory_settings(use_empty=True)
|
||||
|
||||
@@ -370,8 +407,8 @@ def test_cutout_modifier_wraps_four_menu_controlled_shape_groups():
|
||||
def test_image_layer_switches_between_tangent_and_object_space_normal():
|
||||
bpy.ops.wm.read_factory_settings(use_empty=True)
|
||||
|
||||
assert object_data.IMAGE_MATERIAL_NODE_GROUP_NAME == "O Image Layer"
|
||||
group = object_data.material_node_group()
|
||||
assert material_data.IMAGE_MATERIAL_NODE_GROUP_NAME == "O Image Layer"
|
||||
group = material_data.material_node_group()
|
||||
normal = next(
|
||||
item
|
||||
for item in group.interface.items_tree
|
||||
@@ -416,7 +453,7 @@ def test_image_layer_switches_between_tangent_and_object_space_normal():
|
||||
|
||||
color = bpy.data.images.new("Color", width=2, height=2)
|
||||
normal_image = bpy.data.images.new("Normal", width=2, height=2)
|
||||
object_material = object_data.create_image_material(
|
||||
object_material = material_data.create_image_material(
|
||||
SimpleNamespace(data=SimpleNamespace(name="Object Normal")),
|
||||
color,
|
||||
1.0,
|
||||
@@ -426,7 +463,7 @@ def test_image_layer_switches_between_tangent_and_object_space_normal():
|
||||
layer = _group_node(group, object_material.node_tree)
|
||||
assert layer.inputs["Object Space"].default_value is True
|
||||
|
||||
tangent_material = object_data.create_image_material(
|
||||
tangent_material = material_data.create_image_material(
|
||||
SimpleNamespace(data=SimpleNamespace(name="Tangent Normal")),
|
||||
color,
|
||||
1.0,
|
||||
@@ -434,7 +471,7 @@ def test_image_layer_switches_between_tangent_and_object_space_normal():
|
||||
)
|
||||
layer = _group_node(group, tangent_material.node_tree)
|
||||
assert layer.inputs["Object Space"].default_value is False
|
||||
reused = object_data.material_node_group()
|
||||
reused = material_data.material_node_group()
|
||||
assert {
|
||||
node.space for node in reused.nodes if node.bl_idname == "ShaderNodeNormalMap"
|
||||
} == {"TANGENT", "OBJECT"}
|
||||
@@ -503,7 +540,7 @@ def test_balloon_build_starts_from_the_unified_single_sheet():
|
||||
)
|
||||
assert normal_scale_attribute.attribute_type == "GEOMETRY"
|
||||
assert normal_scale_attribute.attribute_name == cutout_shape.BALLOON_ATTRIBUTE_NAME
|
||||
image_layer = _group_node(object_data.material_node_group(), material_tree)
|
||||
image_layer = _group_node(material_data.material_node_group(), material_tree)
|
||||
normal_scale_link = next(
|
||||
link
|
||||
for link in material_tree.links
|
||||
@@ -572,7 +609,7 @@ def test_cutout_material_uses_flat_tangent_normal():
|
||||
color = bpy.data.images.new("Color", width=2, height=2)
|
||||
normal = bpy.data.images.new("Normal", width=1, height=1)
|
||||
|
||||
result = object_data.create_image_material(
|
||||
result = material_data.create_image_material(
|
||||
source,
|
||||
color,
|
||||
0.0,
|
||||
@@ -580,7 +617,7 @@ def test_cutout_material_uses_flat_tangent_normal():
|
||||
scene=bpy.context.scene,
|
||||
)
|
||||
|
||||
layer = _group_node(object_data.material_node_group(), result.node_tree)
|
||||
layer = _group_node(material_data.material_node_group(), result.node_tree)
|
||||
assert layer.inputs["Object Space"].default_value is False
|
||||
|
||||
|
||||
@@ -624,7 +661,7 @@ def test_depth_surface_protected_smooth_evaluates():
|
||||
== cutout_shape.CUTOUT_SHAPE_INDICES["DEPTH_SURFACE"]
|
||||
)
|
||||
expected_cleanup = cutout_geometry.cutout_cleanup_value(
|
||||
cutout_geometry.cutout_point_spacing(source, 8),
|
||||
cutout_geometry.cutout_sampling_spacing(source, 8),
|
||||
cutout_geometry.world_plane_metric(source.matrix_world)[0],
|
||||
)
|
||||
assert np.isclose(
|
||||
@@ -703,6 +740,44 @@ def test_depth_surface_protected_smooth_evaluates():
|
||||
assert np.isfinite(internal_positions).all()
|
||||
|
||||
|
||||
def test_depth_surface_keeps_continuous_points_across_zero_validity_edge():
|
||||
bpy.ops.wm.read_factory_settings(use_empty=True)
|
||||
image = bpy.data.images.new("Surface", width=64, height=64)
|
||||
source = bpy.data.objects.new("Surface", None)
|
||||
source.empty_display_type = "IMAGE"
|
||||
source.data = image
|
||||
source.empty_display_size = 2.0
|
||||
bpy.context.collection.objects.link(source)
|
||||
size = 8
|
||||
depth = np.ones((size, size), dtype=np.float32)
|
||||
x, y = np.meshgrid(np.arange(size), np.arange(size))
|
||||
intrinsics = np.asarray(
|
||||
((8.0, 0.0, 3.5), (0.0, 8.0, 3.5), (0.0, 0.0, 1.0)),
|
||||
dtype=np.float32,
|
||||
)
|
||||
points = np.empty((size, size, 3), dtype=np.float32)
|
||||
points[..., 0] = (x - intrinsics[0, 2]) / intrinsics[0, 0] * depth
|
||||
points[..., 1] = (y - intrinsics[1, 2]) / intrinsics[1, 1] * depth
|
||||
points[..., 2] = depth
|
||||
validity = np.ones((size, size), dtype=bool)
|
||||
validity[[0, -1], :] = False
|
||||
validity[:, [0, -1]] = False
|
||||
|
||||
result = _create_cutout_shape(
|
||||
source,
|
||||
"DEPTH_SURFACE",
|
||||
8,
|
||||
depth_image=_depth_image(points, validity),
|
||||
depth_metadata=_depth_metadata(intrinsics, (size, size), 1.0),
|
||||
)
|
||||
positions = _evaluated_positions(result)
|
||||
|
||||
assert np.isfinite(positions).all()
|
||||
assert np.ptp(positions[:, 0]) < 1e-4
|
||||
assert np.ptp(positions[:, 1]) > 1.5
|
||||
assert np.ptp(positions[:, 2]) > 1.5
|
||||
|
||||
|
||||
def _evaluated_positions(obj):
|
||||
evaluated = obj.evaluated_get(bpy.context.evaluated_depsgraph_get())
|
||||
result = evaluated.to_mesh()
|
||||
@@ -951,8 +1026,8 @@ def test_depth_balloon_uses_metric_plane_inputs_without_controller():
|
||||
depth_image = _depth_image(z_depth, valid)
|
||||
depth_metadata = _depth_metadata(intrinsics, (41, 61), 2.0)
|
||||
|
||||
region_values = np.ones((60, 40), dtype=np.float32)
|
||||
region_values[:, :2] = 0.0
|
||||
selection_values = np.ones((60, 40), dtype=np.float32)
|
||||
selection_values[:, :2] = 0.0
|
||||
result = _create_cutout_shape(
|
||||
source,
|
||||
"DEPTH_BALLOON",
|
||||
@@ -960,7 +1035,7 @@ def test_depth_balloon_uses_metric_plane_inputs_without_controller():
|
||||
depth_image=depth_image,
|
||||
depth_metadata=depth_metadata,
|
||||
bounds=crop_bounds,
|
||||
region_values=region_values,
|
||||
selection_values=selection_values,
|
||||
)
|
||||
|
||||
assert all(abs(vertex.co.x) < 1e-8 for vertex in result.data.vertices)
|
||||
@@ -1135,16 +1210,16 @@ def test_local_cutout_selection_matches_uint8_tenth_alpha_boundary():
|
||||
top_down = np.ones((4, 4, 4), dtype=np.float32)
|
||||
top_down[1, 1, 3] = 25.0 / 255.0
|
||||
top_down[1, 2, 3] = 26.0 / 255.0
|
||||
region_mask = rasterize_region_path(
|
||||
selection_mask = rasterize_selection_path(
|
||||
(4, 4),
|
||||
RegionPath(
|
||||
SelectionPath(
|
||||
points=((0, 0), (4, 0), (4, 4), (0, 4)),
|
||||
fill_rule="NONZERO",
|
||||
),
|
||||
source_alpha=top_down[:, :, 3],
|
||||
alpha_threshold=0.1,
|
||||
)
|
||||
selection = region_mask.full_values((4, 4))
|
||||
selection = selection_mask.full_values((4, 4))
|
||||
|
||||
assert selection[1, 1] == 0.0
|
||||
assert selection[1, 2] > 0.1
|
||||
@@ -1162,7 +1237,7 @@ def test_local_cutout_builds_from_blender_image_pixels_without_a_job():
|
||||
bpy.context.collection.objects.link(source)
|
||||
source.select_set(True)
|
||||
bpy.context.view_layer.objects.active = source
|
||||
region_path = RegionPath(
|
||||
selection_path = SelectionPath(
|
||||
points=((-8, 8), (72, 8), (72, 40), (-8, 40)),
|
||||
fill_rule="NONZERO",
|
||||
)
|
||||
@@ -1174,9 +1249,9 @@ def test_local_cutout_builds_from_blender_image_pixels_without_a_job():
|
||||
wraps=image_data.image_rgba,
|
||||
) as read_rgba:
|
||||
rgba = image_data.image_rgba(image)
|
||||
region_mask = rasterize_region_path(
|
||||
selection_mask = rasterize_selection_path(
|
||||
tuple(image.size),
|
||||
region_path,
|
||||
selection_path,
|
||||
source_alpha=rgba[:, :, 3],
|
||||
alpha_threshold=cutout_geometry.CUTOUT_ALPHA_THRESHOLD,
|
||||
antialias=True,
|
||||
@@ -1186,7 +1261,7 @@ def test_local_cutout_builds_from_blender_image_pixels_without_a_job():
|
||||
source,
|
||||
"SURFACE",
|
||||
16,
|
||||
region_mask,
|
||||
selection_mask,
|
||||
source_rgba=rgba,
|
||||
)
|
||||
|
||||
@@ -1203,7 +1278,7 @@ def test_local_cutout_builds_from_blender_image_pixels_without_a_job():
|
||||
== cutout_shape.CUTOUT_SHAPE_INDICES["SURFACE"]
|
||||
)
|
||||
expected_cleanup = cutout_geometry.cutout_cleanup_value(
|
||||
cutout_geometry.cutout_point_spacing(source, 16),
|
||||
cutout_geometry.cutout_sampling_spacing(source, 16),
|
||||
cutout_geometry.world_plane_metric(source.matrix_world)[0],
|
||||
)
|
||||
assert np.isclose(
|
||||
@@ -1245,7 +1320,7 @@ def test_local_cutout_builds_from_blender_image_pixels_without_a_job():
|
||||
color_texture.image.pixels.foreach_get(copied)
|
||||
copied = copied.reshape((34, 64, 4))
|
||||
assert np.allclose(copied[:, :, :3], 1.0)
|
||||
left, top, right, bottom = region_mask.bounds
|
||||
left, top, right, bottom = selection_mask.bounds
|
||||
assert np.allclose(
|
||||
copied[:, :, 3], np.flipud(rgba[top:bottom, left:right, 3])
|
||||
)
|
||||
@@ -1257,7 +1332,7 @@ def test_balloon_normal_scale_uses_existing_image_layer_input():
|
||||
{"BALLOON", "DEPTH_BALLOON"}
|
||||
)
|
||||
group = bpy.data.node_groups.new(
|
||||
object_data.IMAGE_MATERIAL_NODE_GROUP_NAME,
|
||||
material_data.IMAGE_MATERIAL_NODE_GROUP_NAME,
|
||||
"ShaderNodeTree",
|
||||
)
|
||||
group.interface.new_socket(
|
||||
|
||||
@@ -59,12 +59,14 @@ class ExtensionPackagingTest(unittest.TestCase):
|
||||
self.assertNotIn("assets/O_AnyImage.blend1", names)
|
||||
for path in (
|
||||
"common/__init__.py",
|
||||
"common/ai_job.py",
|
||||
"common/image_data.py",
|
||||
"common/image_interaction.py",
|
||||
"common/image_region.py",
|
||||
"common/ai.py",
|
||||
"common/depth.py",
|
||||
"common/image.py",
|
||||
"common/material.py",
|
||||
"common/node.py",
|
||||
"common/object.py",
|
||||
"common/selection.py",
|
||||
"common/viewport.py",
|
||||
"operators/cutout_tool/__init__.py",
|
||||
"operators/cutout_tool/main.py",
|
||||
"operators/cutout_tool/interaction.py",
|
||||
@@ -98,12 +100,12 @@ class ExtensionPackagingTest(unittest.TestCase):
|
||||
names,
|
||||
)
|
||||
self.assertIn("server/media/input.py", names)
|
||||
self.assertIn("common/depth_data.py", names)
|
||||
self.assertIn("common/depth.py", names)
|
||||
self.assertIn("server/geometry/projection.py", names)
|
||||
self.assertIn("server/models/geometry.py", names)
|
||||
self.assertIn("server/jobs/remove_background.py", names)
|
||||
self.assertIn("server/jobs/refine_image_region.py", names)
|
||||
self.assertIn("server/region/refine_region.py", names)
|
||||
self.assertIn("server/jobs/refine_image_selection.py", names)
|
||||
self.assertIn("server/selection/refine_selection.py", names)
|
||||
self.assertIn("server/jobs/cutout.py", names)
|
||||
self.assertIn("server/jobs/depth_plane.py", names)
|
||||
self.assertIn("server/geometry/depth_texture.py", names)
|
||||
|
||||
@@ -1,3 +1,5 @@
|
||||
import ast
|
||||
import importlib
|
||||
import importlib.util
|
||||
import sys
|
||||
import tempfile
|
||||
@@ -7,6 +9,47 @@ from types import ModuleType, SimpleNamespace
|
||||
from unittest.mock import patch
|
||||
|
||||
PROJECT_ROOT = Path(__file__).resolve().parents[1]
|
||||
SOURCE_ROOT = PROJECT_ROOT / "src"
|
||||
sys.path.insert(0, str(SOURCE_ROOT))
|
||||
COMMON_ROOT = SOURCE_ROOT / "anyimage" / "common"
|
||||
COMMON_MODULES = {
|
||||
"ai",
|
||||
"depth",
|
||||
"image",
|
||||
"material",
|
||||
"node",
|
||||
"object",
|
||||
"selection",
|
||||
"viewport",
|
||||
}
|
||||
|
||||
|
||||
def test_common_modules_follow_business_boundaries():
|
||||
module_names = {
|
||||
path.stem
|
||||
for path in COMMON_ROOT.glob("*.py")
|
||||
if path.name != "__init__.py"
|
||||
}
|
||||
assert module_names == COMMON_MODULES
|
||||
|
||||
for module_name in COMMON_MODULES:
|
||||
importlib.import_module(f"anyimage.common.{module_name}")
|
||||
|
||||
ai_ui_definitions = {
|
||||
path.name
|
||||
for path in COMMON_ROOT.glob("*.py")
|
||||
if "def draw_ai_setup(" in path.read_text(encoding="utf-8")
|
||||
or "def draw_ai_property(" in path.read_text(encoding="utf-8")
|
||||
}
|
||||
assert ai_ui_definitions == {"ai.py"}
|
||||
|
||||
init_tree = ast.parse(
|
||||
(COMMON_ROOT / "__init__.py").read_text(encoding="utf-8")
|
||||
)
|
||||
assert not any(
|
||||
isinstance(node, (ast.Import, ast.ImportFrom))
|
||||
for node in init_tree.body
|
||||
)
|
||||
|
||||
|
||||
def load_image_modules():
|
||||
@@ -24,12 +67,12 @@ def load_image_modules():
|
||||
|
||||
module_paths = (
|
||||
(
|
||||
"anyimage.common.image_data",
|
||||
"anyimage.common.image",
|
||||
PROJECT_ROOT
|
||||
/ "src"
|
||||
/ "anyimage"
|
||||
/ "common"
|
||||
/ "image_data.py",
|
||||
/ "image.py",
|
||||
),
|
||||
(
|
||||
"anyimage.common.node",
|
||||
@@ -39,6 +82,14 @@ def load_image_modules():
|
||||
/ "common"
|
||||
/ "node.py",
|
||||
),
|
||||
(
|
||||
"anyimage.common.material",
|
||||
PROJECT_ROOT
|
||||
/ "src"
|
||||
/ "anyimage"
|
||||
/ "common"
|
||||
/ "material.py",
|
||||
),
|
||||
(
|
||||
"anyimage.common.object",
|
||||
PROJECT_ROOT
|
||||
|
||||
@@ -15,8 +15,8 @@ class NodeAssetsTest(unittest.TestCase):
|
||||
self.assertEqual(offenders, [])
|
||||
|
||||
def test_runtime_node_group_entries_load_assets(self):
|
||||
object_source = (
|
||||
SOURCE_ROOT / "common" / "object.py"
|
||||
material_source = (
|
||||
SOURCE_ROOT / "common" / "material.py"
|
||||
).read_text(encoding="utf-8")
|
||||
mesh_plane_source = (
|
||||
SOURCE_ROOT / "operators" / "convert_to_plane" / "mesh_plane.py"
|
||||
@@ -24,7 +24,7 @@ class NodeAssetsTest(unittest.TestCase):
|
||||
depth_plane_source = (
|
||||
SOURCE_ROOT / "operators" / "convert_to_plane" / "object.py"
|
||||
).read_text(encoding="utf-8")
|
||||
self.assertIn("return load_node_group(group_name)", object_source)
|
||||
self.assertIn("return load_node_group(group_name)", material_source)
|
||||
self.assertIn(
|
||||
"load_node_group(MESH_PLANE_NODE_GROUP_NAME)",
|
||||
mesh_plane_source,
|
||||
|
||||
@@ -259,7 +259,7 @@ class ServerRuntimeTest(unittest.TestCase):
|
||||
{
|
||||
"download-model",
|
||||
"download-required-models",
|
||||
"refine-image-region",
|
||||
"refine-image-selection",
|
||||
"remove-background",
|
||||
"upscale-image",
|
||||
"generate-depth-plane-geometry",
|
||||
@@ -391,19 +391,27 @@ class ServerRuntimeTest(unittest.TestCase):
|
||||
import numpy as np
|
||||
|
||||
depth = importlib.import_module("server.geometry.depth_texture")
|
||||
GeometryFrame = importlib.import_module(
|
||||
"server.models.geometry"
|
||||
).GeometryFrame
|
||||
texture_path = self.storage_root / "z-depth.exr"
|
||||
metadata_path = self.storage_root / "depth.json"
|
||||
valid = np.asarray(((True, True, False), (True, True, True)))
|
||||
calibration = np.asarray(((False, True, False), (False, False, True)))
|
||||
frame = SimpleNamespace(
|
||||
depth=np.asarray(((1.0, 2.0, 3.0), (4.0, 5.0, 6.0))),
|
||||
validity = np.asarray(
|
||||
((1.0, 1.0, 0.0), (1.0, 1.0, 1.0)),
|
||||
dtype=np.float32,
|
||||
)
|
||||
calibration = np.asarray(((False, True, True), (False, False, True)))
|
||||
frame = GeometryFrame(
|
||||
depth=np.asarray(
|
||||
((1.0, 2.0, 3.0), (4.0, 5.0, 6.0)),
|
||||
dtype=np.float32,
|
||||
),
|
||||
validity=validity,
|
||||
points=np.ones((2, 3, 3), dtype=np.float32),
|
||||
intrinsics=np.eye(3, dtype=np.float32),
|
||||
valid_mask=valid,
|
||||
model_mask=None,
|
||||
)
|
||||
|
||||
depth.write_z_depth_texture(frame.depth, frame.valid_mask, texture_path)
|
||||
depth.write_z_depth_texture(frame, texture_path)
|
||||
depth.write_depth_metadata(frame, calibration, metadata_path)
|
||||
|
||||
self.assertTrue(texture_path.is_file())
|
||||
@@ -418,6 +426,9 @@ class ServerRuntimeTest(unittest.TestCase):
|
||||
from PIL import Image
|
||||
|
||||
normal_texture = importlib.import_module("server.geometry.normal_texture")
|
||||
GeometryFrame = importlib.import_module(
|
||||
"server.models.geometry"
|
||||
).GeometryFrame
|
||||
output_path = self.storage_root / "object-normal.png"
|
||||
camera_normals = np.asarray(
|
||||
[[
|
||||
@@ -429,9 +440,15 @@ class ServerRuntimeTest(unittest.TestCase):
|
||||
dtype=np.float32,
|
||||
)
|
||||
valid = np.asarray([[True, True, True, False]], dtype=bool)
|
||||
frame = GeometryFrame(
|
||||
depth=np.ones((1, 4), dtype=np.float32),
|
||||
validity=np.ones((1, 4), dtype=np.float32),
|
||||
intrinsics=np.eye(3, dtype=np.float32),
|
||||
normal=camera_normals,
|
||||
)
|
||||
|
||||
normal_texture.write_camera_normal_texture(
|
||||
camera_normals,
|
||||
frame,
|
||||
valid,
|
||||
(0, 0),
|
||||
(4, 1),
|
||||
@@ -455,10 +472,153 @@ class ServerRuntimeTest(unittest.TestCase):
|
||||
),
|
||||
)
|
||||
|
||||
def test_moge_model_input_composites_rgba_over_black(self):
|
||||
import numpy as np
|
||||
from PIL import Image
|
||||
|
||||
artifacts = sys.modules["server.geometry.prediction_artifacts"]
|
||||
first_path = self.storage_root / "first.png"
|
||||
second_path = self.storage_root / "second.png"
|
||||
first = np.asarray(
|
||||
[[(240, 120, 60, 0), (200, 100, 50, 128), (9, 8, 7, 255)]],
|
||||
dtype=np.uint8,
|
||||
)
|
||||
second = first.copy()
|
||||
second[0, 0, :3] = (1, 2, 3)
|
||||
Image.fromarray(first, mode="RGBA").save(first_path)
|
||||
Image.fromarray(second, mode="RGBA").save(second_path)
|
||||
|
||||
first_output = artifacts._limited_model_input(
|
||||
first_path,
|
||||
self.storage_root / "first-model.png",
|
||||
2048,
|
||||
)
|
||||
second_output = artifacts._limited_model_input(
|
||||
second_path,
|
||||
self.storage_root / "second-model.png",
|
||||
2048,
|
||||
)
|
||||
|
||||
with Image.open(first_output) as opened:
|
||||
first_rgb = np.asarray(opened, dtype=np.uint8)
|
||||
with Image.open(second_output) as opened:
|
||||
second_rgb = np.asarray(opened, dtype=np.uint8)
|
||||
np.testing.assert_array_equal(first_rgb, second_rgb)
|
||||
np.testing.assert_array_equal(
|
||||
first_rgb,
|
||||
np.asarray([[(0, 0, 0), (100, 50, 25), (9, 8, 7)]], dtype=np.uint8),
|
||||
)
|
||||
|
||||
def test_geometry_writers_reject_missing_or_non_finite_fields(self):
|
||||
import numpy as np
|
||||
|
||||
depth = importlib.import_module("server.geometry.depth_texture")
|
||||
normal = importlib.import_module("server.geometry.normal_texture")
|
||||
GeometryFrame = importlib.import_module(
|
||||
"server.models.geometry"
|
||||
).GeometryFrame
|
||||
base = {
|
||||
"depth": np.ones((1, 2), dtype=np.float32),
|
||||
"validity": np.ones((1, 2), dtype=np.float32),
|
||||
"intrinsics": np.eye(3, dtype=np.float32),
|
||||
}
|
||||
with self.assertRaisesRegex(ValueError, "point field"):
|
||||
depth.write_vector_depth_texture(
|
||||
GeometryFrame(**base),
|
||||
self.storage_root / "missing-points.exr",
|
||||
)
|
||||
point_frame = GeometryFrame(
|
||||
**base,
|
||||
points=np.asarray([[(0.0, 0.0, 1.0), (np.nan, 0.0, 1.0)]], dtype=np.float32),
|
||||
)
|
||||
with self.assertRaisesRegex(ValueError, "Point field"):
|
||||
depth.write_vector_depth_texture(
|
||||
point_frame,
|
||||
self.storage_root / "nan-points.exr",
|
||||
)
|
||||
depth_frame = GeometryFrame(
|
||||
**{**base, "depth": np.asarray(((1.0, np.inf),), dtype=np.float32)}
|
||||
)
|
||||
with self.assertRaisesRegex(ValueError, "Depth field"):
|
||||
depth.write_z_depth_texture(
|
||||
depth_frame,
|
||||
self.storage_root / "inf-depth.exr",
|
||||
)
|
||||
with self.assertRaisesRegex(ValueError, "normal field"):
|
||||
normal.write_camera_normal_texture(
|
||||
GeometryFrame(**base),
|
||||
np.ones((1, 2), dtype=bool),
|
||||
(0, 0),
|
||||
(2, 1),
|
||||
(2, 1),
|
||||
self.storage_root / "missing-normal.png",
|
||||
)
|
||||
normal_frame = GeometryFrame(
|
||||
**base,
|
||||
normal=np.asarray(
|
||||
[[(0.0, 0.0, 1.0), (0.0, np.nan, 1.0)]],
|
||||
dtype=np.float32,
|
||||
),
|
||||
)
|
||||
with self.assertRaisesRegex(ValueError, "Normal field"):
|
||||
normal.write_camera_normal_texture(
|
||||
normal_frame,
|
||||
np.ones((1, 2), dtype=bool),
|
||||
(0, 0),
|
||||
(2, 1),
|
||||
(2, 1),
|
||||
self.storage_root / "nan-normal.png",
|
||||
)
|
||||
self.assertFalse(any(self.storage_root.glob("*.exr")))
|
||||
self.assertFalse(any(self.storage_root.glob("*-normal.png")))
|
||||
|
||||
def test_moge_artifact_orchestration_uses_one_geometry_frame(self):
|
||||
import numpy as np
|
||||
from PIL import Image
|
||||
|
||||
artifacts = sys.modules["server.geometry.prediction_artifacts"]
|
||||
GeometryFrame = importlib.import_module(
|
||||
"server.models.geometry"
|
||||
).GeometryFrame
|
||||
input_path = self.storage_root / "selection.png"
|
||||
Image.new("RGBA", (2, 2), (255, 128, 64, 255)).save(input_path)
|
||||
output_directory = self.storage_root / "job"
|
||||
output_directory.mkdir()
|
||||
context = SimpleNamespace(
|
||||
directory=output_directory,
|
||||
resource=lambda _name: object(),
|
||||
progress=lambda *_args: None,
|
||||
check_cancelled=lambda: None,
|
||||
)
|
||||
frame = GeometryFrame(
|
||||
depth=np.ones((2, 2), dtype=np.float32),
|
||||
validity=np.ones((2, 2), dtype=np.float32),
|
||||
intrinsics=np.eye(3, dtype=np.float32),
|
||||
normal=np.zeros((2, 2, 3), dtype=np.float32),
|
||||
points=np.ones((2, 2, 3), dtype=np.float32),
|
||||
)
|
||||
|
||||
with patch.object(artifacts.moge2, "infer", return_value=[frame]) as infer:
|
||||
result = artifacts.generate_moge2_artifacts(
|
||||
context,
|
||||
{"max_input_size": 2048},
|
||||
input_path,
|
||||
depth_mode="VECTOR",
|
||||
normal_mode="OBJECT",
|
||||
)
|
||||
|
||||
self.assertEqual(
|
||||
set(result),
|
||||
{"vector_depth", "depth_metadata", "object_normal"},
|
||||
)
|
||||
self.assertFalse((output_directory / "model-input.png").exists())
|
||||
self.assertTrue(all(path.is_file() for path in result.values()))
|
||||
self.assertTrue(infer.call_args.kwargs["include_points"])
|
||||
|
||||
def test_generate_cutout_artifacts_only_returns_requested_outputs(self):
|
||||
job = sys.modules["server.jobs.cutout"]
|
||||
input_path = self.storage_root / "input.png"
|
||||
input_path.write_bytes(b"region")
|
||||
input_path.write_bytes(b"selection")
|
||||
context = SimpleNamespace(
|
||||
directory=self.storage_root / "job",
|
||||
progress=lambda *_args: None,
|
||||
@@ -505,10 +665,10 @@ class ServerRuntimeTest(unittest.TestCase):
|
||||
normal_mode="OBJECT",
|
||||
)
|
||||
|
||||
def test_generate_cutout_artifacts_refine_an_already_rasterized_region(self):
|
||||
def test_generate_cutout_artifacts_refine_an_already_rasterized_selection(self):
|
||||
job = sys.modules["server.jobs.cutout"]
|
||||
input_path = self.storage_root / "input.png"
|
||||
input_path.write_bytes(b"region")
|
||||
input_path.write_bytes(b"selection")
|
||||
directory = self.storage_root / "job"
|
||||
context = SimpleNamespace(
|
||||
directory=directory,
|
||||
@@ -520,12 +680,12 @@ class ServerRuntimeTest(unittest.TestCase):
|
||||
output.write_bytes(b"refined")
|
||||
return output, (2, 3, 8, 9)
|
||||
|
||||
with patch.object(job, "refine_region", side_effect=generate):
|
||||
with patch.object(job, "refine_selection", side_effect=generate):
|
||||
result = job.run(
|
||||
context,
|
||||
{
|
||||
"input": str(input_path),
|
||||
"refine_region": True,
|
||||
"refine_selection": True,
|
||||
"refine_model": "BEN2_BASE",
|
||||
"device": "cpu",
|
||||
},
|
||||
@@ -533,9 +693,12 @@ class ServerRuntimeTest(unittest.TestCase):
|
||||
|
||||
self.assertEqual(
|
||||
result,
|
||||
{"region_image": "region.png", "region_bounds": "region_bounds.json"},
|
||||
{
|
||||
"selection_image": "selection.png",
|
||||
"selection_bounds": "selection_bounds.json",
|
||||
},
|
||||
)
|
||||
self.assertEqual(
|
||||
(directory / "region_bounds.json").read_text(encoding="utf-8"),
|
||||
(directory / "selection_bounds.json").read_text(encoding="utf-8"),
|
||||
"[2,3,8,9]",
|
||||
)
|
||||
|
||||
Reference in New Issue
Block a user