icrdr/AnyImage
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat(node_assets): add image plane geometry nodes and validation scripts

Browse Source 
- Implemented `image_plane.py` to build the Image Plane Geometry Nodes group with customizable parameters for geometry, subdivide, and thickness.
- Created `image_plane_validation.py` to validate the functionality of the image plane, ensuring correct geometry generation and material assignment.
- Developed `material_layers.py` to construct shader node groups for image materials, including depth and normal handling.
- Authored `plane_layouts.py` to define node arrangements for image geometry assets.
- Introduced `shadeless.py` to create a shadeless shader node group that displays color without scene lighting while preserving shadows and reflections.
This commit is contained in:
马南
2026-09-08 23:24:27 +08:00
parent 196b964efc
commit 4c902ed7db
155 changed files with 9700 additions and 10393 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
AGENTS.md
View File

@@ -96,4 +96,4 @@ uv run sync_models
- 修改器 Geometry Nodes 的输入 socket 默认使用 `SINGLE`
- socket `subtype` 按业务语义设置：长度用 `DISTANCE`，`0–1` 比例用 `FACTOR`，`0–100` 用 `PERCENTAGE`；无对应范围语义时保留默认
- 接口或布局变更须验证 socket 顺序与结构、未使用输出隐藏及几何求值；纯布局变更比较运算、默认值和实际连接，确认功能不变
- 在 Blender 节点编辑器按实际绘制尺寸检查总览和局部步骤，确认主轴、支线长度、节点间距及配对区域清楚；坐标检查不能代替视觉检查
- 在 Blender 节点编辑器按实际绘制尺寸检查总览和局部步骤，确认主轴、支线长度、节点间距及配对区域清楚；

79
docs/internals/architecture.md
View File

@@ -1,68 +1,43 @@
# 程序架构
AnyImage 由 Blender 交互层、BlendJob Runtime 和独立 Job Server 三部分组成。AI 依赖运行在 Server Environment 中，Blender 数据在主线程创建和修改。
## 对应文件
```text
src/anyimage/
├── __init__.py # 类型、工具、菜单与 Runtime 注册
├── menu.py # Image Empty 上下文入口
├── panel.py # Server 与 Debug 面板
├── preferences.py # Storage Root、Device 与功能默认值
├── properties.py # Scene 状态、模型状态与动态枚举
├── runtime.py # Environment 和 JobRuntime
├── common/ # Blender 侧共享原子能力
├── operators/ # Blender 侧业务入口
└── server/ # 独立计算端
```
## 职责
| 层次 | 职责 |
| 模块 | 职责 |
| --- | --- |
| Blender 交互层 | UI、手势、RNA 状态、输入准备、结果导入和 Undo |
| BlendJob Runtime | Environment、Server 生命周期、队列、进度、取消和 Job 目录 |
| Job Server | 媒体解码、ONNX 推理、数组派生和文件输出 |
| `__init__.py` | 注册类型、工具、菜单、快捷键与 Runtime |
| `menu.py`、`panel.py` | Image Empty 上下文菜单与 Job Server 面板 |
| `preferences.py` | 扩展设置及读取入口 |
| `properties.py` | Scene 手势状态、视频帧数、模型状态与枚举 |
| `common/` | Blender 侧公共能力 |
| `operators/` | 交互、任务参数、结果导入和 Undo |
| `server/` | 独立环境内的媒体处理、推理和产物写出 |
Operator 向 Server 发送普通字典。Server 返回文件名和 Job 目录；Operator 的 `response()` 再读取这些文件，并在 Blender 主线程中创建 Image、Mesh、Material 和 Geometry Nodes Modifier。
## 注册与状态
## 注册与配置
```mermaid
flowchart TD
Register["anyimage.register()"] --> Types["注册 Preferences / PropertyGroup / Operator / Panel"]
Types --> Tools["注册 Image Edit 与 Cutout WorkspaceTool"]
Tools --> RNA["挂载 Scene 与 Image Property"]
RNA --> Menu["挂载上下文菜单和剪贴板快捷键"]
Menu --> Runtime["注册 JobRuntime"]
A[register] --> B[按 CLASSES 顺序注册类型]
B --> C[注册 Frame、Mask、Rectify、Cutout 工具]
C --> D[挂载 Scene.anyimage_settings]
D --> E[挂载 View3D 与 Outliner 菜单及剪贴板快捷键]
E --> F[注册 JobRuntime]
```
`__init__.py` 按依赖顺序注册，并在注销时逆序清理。扩展级设置统一由 `preferences.addon_preferences()` 读取；`runtime.py` 只消费 `configured_storage_root` 回调，不持有 Blender 偏好设置的定位逻辑。交互中的临时状态和 Debug 参数保存在 `Scene.anyimage_settings`。`max_ai_input_size`、`max_frame_resolution` 与 `max_cutout_mesh_resolution` 都在 `preferences.py` 定义与读取：它们分别限制 AI 分析图、Frame 输出和 Cutout 几何采样的最长边。Upscale 还用 AI 输入上限判断当前 Image 是否允许执行。
注销先关闭 Runtime 和交互入口，再移除 Scene 属性，按 `CLASSES` 逆序注销类型。设备、存储目录、模型、材质显示适配和功能上限统一通过 `preferences.py` 读取。
## 同步与异步路径
**Maximum AI Input Size** 限制模型分析图和 Upscale 可提交源图；**Maximum Frame Resolution** 限制 Frame 输出；**Maximum Cutout Mesh Resolution** 限制轮廓网格采样。三者各自作用于对应数据阶段。
无需 AI 或媒体处理的功能直接在 Blender 中完成，例如 Plane、Frame、Mask、Rectify，以及不生成 Normal Map 的 Surface 和 Balloon。需要模型或独立文件处理时，功能 Operator 派生自 `JobOperatorBase`：
## 执行边界
```mermaid
sequenceDiagram
participant User as 用户
participant Op as Blender Operator
participant RT as JobRuntime
participant Job as Server Job
User->>Op: 执行功能
Op->>Op: 准备文件与普通字典
Op->>RT: 提交 Job
RT->>Job: 排队执行
Job-->>RT: 写入产物并返回索引
RT-->>Op: 主线程 response
Op->>Op: 导入数据并提交 Undo
flowchart TD
A[Operator 在主线程读取 Blender 数据] --> B[准备输入文件]
B --> C[request 返回普通字典]
C --> D[BlendJob 队列执行 Server Job]
D --> E[模型与媒体处理写入 Job 目录]
E --> F[主线程 response 加载结果]
F --> G[提交 Image 或 Object 修改及 Undo]
G --> H[清理临时输入]
```
`server/` 保持纯 Python 实现；Operator 在 Blender 主线程读取产物并创建 RNA 数据。
Plane、Frame、Mask、Rectify 和无需 AI 的 Cutout 在 Blender 内执行。Server 的 `jobs/` 编排任务，`models/` 负责模型适配，`geometry/` 处理深度与法线产物。Server 模块通过内部相对导入共享能力，Blender 侧通过 `common/` 复用。
几何模型的原始输出先由 `server/models/` 适配为单帧 `GeometryFrame`。它统一保存 Depth、Validity、Intrinsics，以及可选的 Normal 和 Points；Depth、Normal 与 Metadata Writer 只消费该结构。MoGe-2 的 Mask 转为 `0/1` Validity，DA3 的 Confidence 作为连续 Validity。
## 共享能力
`common/` 按 AI、Image、Selection、Viewport、Depth、Material、Object 和 Node 八个业务模块提供 Blender 侧复用能力。公开函数见[Common 公共函数](common.md)。
MoGe-2 输出统一为 `GeometryFrame`：包含 Depth、Validity、Intrinsics，以及按需提供的 Normal 和 Points。产物写入器消费同一个 Frame，保持深度、法线和相机数据一致。参见 [Runtime](runtime.md) 与 [产物协议](outputs.md)。

154
docs/internals/common.md
View File

@@ -1,126 +1,54 @@
# Common 公共函数
# Blender 公共能力
`src/anyimage/common/` 按 AI、Image、Selection、Viewport、Depth、Material、Object 和 Node 八个业务模块提供 Blender 侧复用能力。调用方直接从对应模块导入函数。
`src/anyimage/common/` 按业务职责提供公共函数，调用方从对应模块直接导入。
## Image 数据块
`image.py` 处理 Blender `Image` 数据块及其在 Image Empty 上的完整生命周期，包括像素读取、动画帧、Job 输入、结果 Image 创建和 Empty 引用替换。
| 函数 | 职责 |
| 模块 | 职责与主要入口 |
| --- | --- |
| `image.is_image_empty()` | 判断对象是否为带有效 Image 数据的 Empty。 |
| `image.require_image_empty()` | 按名称取得 Image Empty；对象已失效时抛错。 |
| `image.image_pixels()` | 以 Blender 原始顺序读取一维 float32 像素。 |
| `image.image_rgba()` | 读取为 Top-down RGBA float32 数组。 |
| `image.homography_from_points()` | 求解四点之间的 Homography。 |
| `image.projective_output_size()` | 按源像素密度、目标宽高比和资源上限计算投影输出尺寸。 |
| `image.premultiplied_rgba()` | 将 Blender 顺序像素转换为 Top-down 预乘 Alpha RGBA。 |
| `image.sample_premultiplied_rgba()` | 在图片边界坐标中双线性采样预乘 Alpha RGBA，范围外返回透明。 |
| `image.straight_rgba()` | 转换为 straight RGBA，并把相邻可见 RGB 扩展到一圈透明边缘。 |
| `image.warp_projective_pixels()` | 将源 Quad 重采样为矩形 Blender Image 像素。 |
| `image.video_total_frames()` | 返回 Movie 或 Sequence 总帧数，静态图返回 1。 |
| `image.image_user_settings()` | 读取 Image Empty 的帧起点、偏移、时长和循环设置。 |
| `image.default_extract_frames()` | 取得当前 Empty 默认应处理的帧数。 |
| `image.source_start_frame()` | 从源 Image Empty 取得 Job 应处理的起始帧。 |
| `image.prepare_image_input()` | 将 Packed、Dirty、Sequence 等输入准备为 Job 可读路径，并返回临时标记。 |
| `image.prepare_image_bounds_input()` | 复用调用方已读取的 RGBA，按几何 bounds 写出保留原始 Alpha 的矩形 Job 输入。 |
| `image.cleanup_image_input()` | 清理由图片输入 helper 创建的临时文件或目录。 |
| `image.image_base_name()` | 去除图片或视频扩展名和 Blender 数字后缀，得到数据块基础名。 |
| `image.depth_data_name()` | 生成 Depth 结果数据块文件名。 |
| `image.color_data_name()` | 生成 Color 结果数据块文件名。 |
| `image.load_image_edit_result()` | 加载、Pack 并配置直接编辑源 Image 的 Job 结果，以源 Image 完整名称请求新数据块名称。 |
| `image.load_color_result_image()` | 加载、命名、Pack 并配置材质 Color 或 Sequence 结果；不修改 Empty。 |
| `image.load_normal_result_image()` | 加载、命名、设为 Non-Color 并 Pack Normal Map。 |
| `image.load_packed_rgba_image()` | 加载已有 RGBA 文件，继承源图颜色空间和 Alpha mode，命名并 Pack。 |
| `image.create_image_edit_result()` | 从本地编辑后的 RGBA 像素创建 Packed Image，以源 Image 完整名称请求新数据块名称；不修改 Empty。 |
| `image.replace_empty_image()` | 执行 Image Empty 结果替换、共享引用分流、旧数据清理、帧设置继承和可选画布对齐。 |
| `image.image_empty_bounds()` | 计算 Image Empty 在对象局部坐标中的实际显示边界。 |
| `ai.py` | 环境和模型就绪检查、输入路径校验、`production_device()`、`moge2_parameters()` 与安装 UI |
| `image.py` | `image_rgba()`、动画帧设置、Job 输入准备、像素采样、结果 Image 和 `replace_empty_image()` |
| `selection.py` | `SelectionPath`、`SelectionMask`、non-zero winding 栅格化与 Alpha 修整 |
| `viewport.py` | 原生点选、屏幕投影、`ImageGesture`、Lasso / Brush / Polyline 与 Overlay |
| `depth.py` | 深度图片和元数据加载、相机坐标与显示范围的尺度换算 |
| `material.py` | Image Layer、Normal、Alpha、Shadeless 和动画材质 |
| `object.py` | Modifier 输入设置与结果对象激活、源对象替换 |
| `node.py` | `node_asset_path()` 与 `load_node_group()` |
Remove Background、Upscale、Frame、Mask 与 Rectify 都属于直接 Image 编辑，结果保留 active 或源 Image 数据块的完整名称，包括名称中的 `.png` 等格式文字。源数据仍被其他对象引用时，重名结果由 Blender 自动追加数字后缀；源数据可清理时，替换入口让结果恢复准确原名。`color_data_name()`、`depth_data_name()`、`normal_data_name()` 仅用于 Plane 与 Cutout 等材质派生贴图。
## 图像生命周期
## Image Selection 与视图交互
```mermaid
flowchart TD
A[Image 与 Image Empty] --> B[image_rgba 读取业务像素]
B --> C{处理位置}
C -- 本地 --> D[像素算法]
C -- Server --> E[prepare_image_input 或 prepare_image_bounds_input]
E --> F[Job 输出文件]
F --> G[加载并 Pack]
D --> G
G --> H[replace_empty_image]
H --> I[保留身份、共享引用、帧设置和画框]
```
这组公共能力使用固定术语：
业务像素采用 Top-down RGBA，`image_pixels()` 使用 Blender 像素顺序。预乘采样用于投影插值；文件或 Packed 图像的业务读取保留隐藏 RGB。直接编辑结果通过统一入口处理共享 Image、失败恢复和源数据清理。
- Selection 是用户在图片空间圈定的业务范围；`SelectionPath` 只保存路径点，并提供 JSON 编解码。
- `SelectionMask` 是 Selection 的纯几何栅格化结果，只包含局部像素权重 `values` 和源图几何 `bounds`。
- Mask 表示二维像素权重或 Alpha 运算。
`prepare_image_input()` 返回路径及临时标记；`cleanup_image_input()` 与创建方配对。Cutout 的 `prepare_image_bounds_input()` 复用已读取的源 RGBA，输出指定几何 bounds 的矩形图片。
`viewport.py` 处理 View3D 中的点击、坐标投影、`ImageGesture` 和 Overlay 绘制；它为 Lasso、Brush 与 Polyline 提供对应状态机，Brush 拖动阶段只维护简化后的屏幕路径及 Overlay、提交共用的圆形印记和连接条。`selection.py` 负责图片空间 Selection 的值、栅格化和 Alpha 修整；`SelectionPath → SelectionMask` 只在图片编辑或 Shape 生成阶段执行。Frame 在自身业务模块中组合冻结的 Viewport 投影矩阵与对象世界矩阵，把屏幕像素映射到 Image Empty 局部平面；正交视图接受任意有限深度，透视视图只采样观察方向前方的有效交点。
## 手势与 Selection
Viewport 对象命中继续由 Blender 原生点选决定。Mask、Rectify 与 Cutout 点到另一对象时只保留原生选择结果；没有命中对象时保留当前 active Image Empty，并把透明区域或 Viewport 空白作为图片操作起点。Frame 的普通空白点击继续清空选择。手势是否与图片范围或可见内容有效相交，由各业务在提交阶段判断。
```mermaid
flowchart TD
A[原生对象点选] --> B[ImageGesture 冻结源图与交互状态]
B --> C[屏幕路径与 Overlay]
C --> D[提交时投影到图片空间]
D --> E[SelectionPath]
E --> F[局部 non-zero winding 栅格化]
F --> G[SelectionMask values 与 bounds]
G --> H[Mask Alpha 运算或 Cutout 内容轮廓]
```
所有 Path 使用 non-zero winding 填充，同方向重复绕行的重叠区域保持选中。栅格器按路径边与像素中心扫描行的交点累计 winding，只建立 Path Bounds 的局部画布，抗锯齿时增加 2 px 边缘。栅格阶段不接收图片 Alpha 或颜色，Mask 与 Cutout 在各自业务层组合几何 Mask 和图片 Alpha。
Brush 的圆形印记和连接条由 `brush_footprint_polygons()` 生成，供预览与提交共用。拖动时维护屏幕路径与局部 Overlay，提交时生成覆盖值。Mask、Rectify 与 Cutout 的空命中保留 active Image Empty；点中另一对象时完成原生选择。
### Viewport 交互
## 材质、节点与对象
| 函数 | 职责 |
| --- | --- |
| `viewport.report_image_edit_exception()` | 将可预期的 Selection 警告和异常映射为 Operator Report。 |
| `viewport.image_edit_poll()` | 判断当前上下文是否允许启动图片编辑工具。 |
| `viewport.resolve_image_edit_click()` | 执行原生点选；点到另一对象时只保留选择结果，空命中时保留 active Image Empty 并开始点选型编辑。 |
| `viewport.active_image_edit_source()` | 要求并取得当前 Selection 内的 active Image Empty。 |
| `viewport.image_edit_selection_keymap()` | 创建普通点击和 Shift 点击的对象选择 keymap，并配置普通点击的空命中策略。 |
| `viewport.image_edit_drag_keymap()` | 创建拖动图片手势及对象点选 keymap。 |
| `viewport.image_edit_point_keymap()` | 创建点选型图片手势及 Shift 追加选择 keymap。 |
| `viewport.active_view3d_tool_id()` | 取得当前 View3D Workspace Tool ID。 |
| `viewport.draw_dashed_line()` | 绘制手势或透视框虚线。 |
| `viewport.draw_polygon_fill()` | 绘制手势或透视框填充预览。 |
| `viewport.ImageGesture` | 复用 Lasso、Brush 与 Polyline 的 Modal 生命周期、源图解析和 Overlay 绘制。 |
| `viewport.brush_footprint_polygons()` | 生成 Brush Overlay 与释放提交共用的圆形印记和连接条。 |
| `viewport.preview_union_geometry()` | 在 screen-space scanline bands 中融合 Brush Primitive，并返回一次填充与外轮廓虚线。 |
| `viewport.preview_fill_geometry()` | 三角化 Lasso、Polyline 的 non-zero winding 填充与外轮廓。 |
| `viewport.rectangle_screen_path()` | 将两个屏幕点规范为 Frame 使用的矩形四角。 |
| `viewport.screen_path_to_image_pixels()` | 将屏幕路径投影为 Image Empty 像素坐标。 |
| `viewport.serialize_matrix()` | 将 4×4 Matrix 序列化为 Operator 可保存的字符串。 |
| `viewport.deserialize_matrix()` | 恢复已序列化的 4×4 Matrix。 |
| `viewport.drawing_in_region()` | 判断 Modal 绘制是否仍位于启动时的 Area 和 Region。 |
`create_image_material()` 连接颜色、法线、Alpha 和动画帧设置；材质显示适配读取当前 Preferences。`set_modifier_input()` 按实际 RNA 输入形式设置值，`finalize_object_result()` 完成结果对象激活和源对象处理。
### Selection 栅格化
| 函数 | 职责 |
| --- | --- |
| `selection.selection_path_bounds()` | 校验 Selection Path 并返回整数像素包围框。 |
| `selection.rasterize_selection_path()` | 以局部 non-zero scanline 把 `SelectionPath` 栅格化为纯几何 `SelectionMask`。 |
| `selection.trim_pixels_to_alpha()` | 按 Alpha 非零区域修整 RGBA 像素。 |
## Depth 产物
`depth.py` 负责 Blender 侧共享的 Depth 文件加载和相机尺度换算，直接加载 Server 写出的 EXR。
| 函数 | 职责 |
| --- | --- |
| `depth.load_depth_metadata()` | 读取并校验 `depth.json` 中的图片尺寸、相机内参与参考深度。 |
| `depth.load_depth_result_image()` | 直接加载、命名、设为 Non-Color 并 Pack `vector-depth.exr` 或 `z-depth.exr`。 |
| `depth.camera_depth_values()` | 从已加载 Depth Image 读取用于本地标定的 Z 与有效区。 |
| `depth.depth_uniform_scale()` | 根据相机内参、参考深度和 Image Empty 范围计算统一尺度。 |
## Object 构建
`material.py` 构建 Image 材质；`object.py` 处理 Object 与 Geometry Nodes Modifier；`node.py` 加载节点资产。
| 函数 | 职责 |
| --- | --- |
| `node.load_node_group()` | 从随扩展分发的 Blend 资产加载指定 Node Group。 |
| `material.create_image_material()` | 从 Image Empty 创建包含颜色、位移、Normal Map、Alpha 和动画设置的完整材质；非 Shadeless 材质由 Color 直接驱动 Principled IOR Level。 |
| `object.modifier_input_identifier()` | 按接口名称解析 Geometry Nodes Modifier 输入 Identifier。 |
| `object.set_modifier_input()` | 跨受支持 Blender RNA 形式设置 Modifier 输入。 |
| `object.finalize_object_result()` | 激活结果对象，并按需替换或保留源 Empty。 |
## AI 准备
`ai.py` 统一处理 AI 功能的 Environment、模型、Device、输入校验、Job 参数和公共 UI。
| 函数 | 职责 |
| --- | --- |
| `ai.draw_ai_setup()` | 绘制 AI 未就绪时的安装入口，并返回当前状态。 |
| `ai.draw_ai_property()` | 按 AI 就绪状态绘制并启用业务属性。 |
| `ai.require_input_path()` | 解析 Blender 路径并要求其为现存文件或目录。 |
| `ai.require_background_input_path()` | 在通用路径校验之外限制 BEN2 支持的媒体类型。 |
| `ai.production_device()` | 读取正式任务使用的计算设备。 |
| `ai.require_environment()` | 要求 AnyImage AI Environment 已就绪。 |
| `ai.invoke_ai_setup_if_needed()` | Environment 未就绪时启动安装交互。 |
| `ai.require_model()` | 校验模型标识和下载状态，并返回模型记录。 |
| `ai.da3_parameters()` | 组装 `debug-da3` 参数。 |
| `ai.moge2_parameters()` | 组装 MoGe-2 正式任务与 Debug 共用参数。 |
Geometry Nodes 从 `src/anyimage/assets/O_AnyImage.blend` 按公开名称加载。构建与布局职责见[节点资产](node-assets.md)，深度字段见[Job 产物](outputs.md)。

100
docs/internals/development.md
View File

@@ -1,92 +1,48 @@
# 开发与发布
AnyImage 的仓库环境用于测试 Blender 侧代码、Server 纯 Python 模块、Manifest 和打包逻辑。AI Runtime 使用独立的 Python 3.12 Environment。
## 对应文件
```text
.
├── pyproject.toml
├── uv.lock
├── .python-version
├── tools/
│ ├── pack.py
│ ├── model_exports.py
│ ├── sync_models.py
│ ├── build_node.py
│ └── node_assets/
├── src/anyimage/blender_manifest.toml
├── tests/
└── .github/workflows/
├── test.yml
├── build.yml
└── docs.yml
```
## 开发环境
仓库固定使用 CPython 3.11，因为开发依赖 `bpy==4.5.3` 对应这一解释器版本。项目根目录执行：
仓库使用 CPython 3.11 与 `bpy==4.5.3` 测试 Blender 代码；AI Server 使用 `runtime.py` 声明的独立 Python 3.12 Environment。
```bash
uv sync
uv run pytest
```
快速检查注册、Manifest 和打包时使用：
## 测试组织
```bash
uv run pytest tests/test_addon.py tests/test_manifest.py tests/test_pack.py -v
```
| 测试范围 | 文件 |
| --- | --- |
| 扩展注册、设置、安装和 Job 交互 | `test_addon_registration.py`、`test_preferences.py`、`test_ai_setup.py`、`test_job_operator.py` |
| Frame、Mask、Rectify、Plane、Cutout | 对应 `test_*_tool.py`、`test_*_operators.py` 与对象、投影、网格测试 |
| 图像身份、Alpha、材质与 Undo | 图像编辑集成、共享替换、材质适配与 Undo 测试 |
| Server | `test_job_server.py`、`test_server_jobs.py`、`test_media_input.py`、`test_geometry_artifacts.py` |
| 模型 | `test_model_manager.py`、目录、下载、推理与各 ONNX 模型测试 |
| 节点资产 | 布局检查、Depth Plane 模式、深度分离与 Cutout 体积求值 |
| 工具与包协议 | 扩展 Manifest、打包、模型同步、节点启动器和文档命令测试 |
测试按边界组织：`test_addon.py` 以 Blender Stub 验证注册和 Operator 流程；`test_mesh.py`、`test_image.py` 等验证可脱离 Blender 的算法；`test_server.py`、`test_models.py` 和各 ONNX 测试覆盖计算端；`test_pack.py` 与 `test_manifest.py` 固定发布包结构。
Blender Stub 共用 `blender_test_support.py`；Server 和实际 Blender 几何 fixture 按使用范围存放在对应 support 模块。测试命名描述功能、触发条件与可观察结果。
## 扩展打包
## 发布命令
```bash
uv run pack
```
| 命令 | 职责 |
| --- | --- |
| `uv run pack` | 生成 Windows、macOS arm64、Linux 扩展包 |
| `uv run pack --prepare` | 为本机同步扩展 wheel 与 Manifest |
| `uv run build_node` | 构建、验证节点资产并运行测试 |
| `uv run sync_models` | 准备、校验并上传模型声明的文件 |
| `uv run docs -dev` | 本地预览文档 |
| `uv run docs` | 通过 Mike 发布项目版本并更新 latest |
`tools/pack.py` 为 `windows-x64`、`macos-arm64` 和 `linux-x64` 分别解析依赖 wheel，生成平台专属 Manifest，再把 `src/anyimage/` 与 wheel 写入 `dist/`：
`tools/pack.py` 从 `pyproject.toml` 读取依赖，按平台与 Blender Python 版本解析 wheel，使用 Blender 提供的 NumPy。产物命名为 `AnyImage.v<version>.<platform>.zip`，写入 `dist/`。本地 BlendJob wheel 内容变化后运行 `uv lock --refresh`，并核对锁文件、Manifest 与打包测试。
```text
dist/
├── AnyImage.v<version>.windows-x64.zip
├── AnyImage.v<version>.macos-arm64.zip
└── AnyImage.v<version>.linux-x64.zip
```
`tools/sync_models.py` 根据模型目录选择上游 ONNX 或本地导出流程；`model_exports.py` 处理对应 PyTorch 权重导出。文件校验依据声明大小和 SHA-256，上传使用逐文件复制。`--prepare-only` 用于本地准备。
本地 `src/anyimage/wheels/` 是 wheel 解析的优先来源，用于携带尚未发布的 BlendJob 构建。依赖、wheel 或 Manifest 变化后，需要同时检查 `pyproject.toml`、`uv.lock`、`blender_manifest.toml` 和打包测试；本地 wheel 内容变化后运行 `uv lock --refresh`。
## 模型发布
```bash
uv run sync_models
```
`tools/sync_models.py` 从公开原址下载已有 ONNX；Real-ESRGAN General WDN x4v3 与 SAFMN Real x4 则从官方 PyTorch 权重调用 `tools/model_exports.py` 构建。所有制品通过固定大小、SHA-256、ONNX Checker 和 PyTorch/ONNX Runtime 数值对比后，逐文件上传到 R2。添加 `--prepare-only` 可仅准备和验证制品。
## 文档
```bash
uv run mkdocs serve
uv run mkdocs build --strict
```
站点导航由 `mkdocs.yml` 显式维护，因此移动页面时必须同时更新导航和相对链接。本地版本化发布使用 Mike；CI 构建严格模式站点并部署到 Cloudflare Worker。
## CI
## CI 与文档导航
```mermaid
flowchart TD
Main["push main / Pull Request"] --> Test["uv run --frozen pytest"]
Tag["push v* tag"] --> Release["测试 → tools/pack.py → Release ZIP"]
Docs["main 上文档变更"] --> Site["mkdocs --strict → Cloudflare"]
A[main push、Pull Request 或手动触发] --> B[test.yml 运行测试]
C[v 前缀 Tag] --> D[build.yml 测试、打包并发布 ZIP]
E[main 文档路径变化或手动触发] --> F[docs.yml 构建站点并部署 Cloudflare]
```
`test.yml` 在 main push、Pull Request 与手动触发时运行完整测试。`build.yml` 只响应 `v*` tag，测试通过后构建并上传三个平台包。`docs.yml` 只在文档相关路径变化或手动触发时部署站点。
## 许可证
AnyImage Extension 代码采用 GPL-3.0-or-later，完整文本见仓库 `LICENSE`。模型权重保持各自上游许可证；模型目录声明下载来源、文件校验和许可信息。
Geometry Nodes 资产必须由 Blender 构建和验证，见[节点资产](node-assets.md)。
站点导航由 `mkdocs.yml` 显式维护，页面移动时同步导航和相对链接。节点资产的脚本职责及检查入口见[节点资产](node-assets.md)。

55
docs/internals/index.md
View File

@@ -1,44 +1,27 @@
# 内部实现总览
Blender 进程负责交互与数据创建，独立 Job Server 负责模型推理和文件处理。两侧通过 BlendJob 传递可序列化参数与 Job 目录中的产物。
AnyImage 在 Blender 主线程处理交互、像素编辑和数据创建，通过 BlendJob 把模型推理交给独立 Job Server。
```mermaid
flowchart TD
Input["Image Empty / 手势"] --> Operator["Blender Operator"]
Operator --> Choice{"需要独立计算？"}
Choice -- "否" --> Build["Blender 主线程构建"]
Choice -- "是" --> Runtime["BlendJob Runtime"]
Runtime --> Server["Job Server"]
Server --> Files["Job 产物"]
Files --> Build
Build --> Result["Image / Mesh / Material / Modifier"]
A[Image Empty 或剪贴板图片] --> B[Operator 与手势工具]
B --> C{需要 AI？}
C -- 是 --> D[准备文件与普通字典参数]
D --> E[JobRuntime → Job Server]
E --> F[图片与相机元数据]
F --> G[Blender 主线程导入]
C -- 否 --> G
G --> H[Image 编辑或 Mesh 与材质创建]
```
## 项目主干
| 主题 | 对应职责 |
| --- | --- |
| [程序架构](architecture.md) | 进程、注册、配置与主线程边界 |
| [Common](common.md) | Blender 侧图像、手势、几何与 AI 公共能力 |
| [Runtime 与 Server](runtime.md) | Environment、Job、模型下载与 Session 生命周期 |
| [Job 产物](outputs.md) | 文件协议、相机坐标和 Blender 消费方式 |
| [Operator 索引](operators/index.md) | 按实际用户功能查找实现 |
| [节点资产](node-assets.md) | 资产、源码职责、构建和验证 |
| [开发与发布](development.md) | 测试组织、工具命令与发布流程 |
```text
.
├── src/anyimage/
│ ├── __init__.py # Blender 注册入口
│ ├── menu.py # Image Empty 右键菜单
│ ├── preferences.py # 扩展级配置
│ ├── properties.py # Scene 状态与模型枚举
│ ├── runtime.py # BlendJob Runtime 配置
│ ├── common/ # Blender 侧共享原子能力
│ ├── operators/ # 交互、任务提交与结果导入
│ ├── server/ # 无 Blender 依赖的计算端
│ ├── assets/ # 随扩展分发的节点资产
│ └── wheels/ # 随扩展分发的 BlendJob wheel
├── tools/ # 打包、模型同步与 Blender 节点资产
└── tests/ # Python 自动化测试
```
## 文档分层
- [程序架构](architecture.md)：注册、配置、线程和模块边界。
- [Common 公共函数](common.md)：仅列生产功能从 `common/` 外部调用的函数。
- [Job Runtime 与 Server](runtime.md)：任务生命周期、媒体输入和模型 Session。
- [Operator 实现索引](operators/index.md)：按 `src/anyimage/operators/` 中的完整功能拆分实现页。
- [Job 产物](outputs.md)：文件用途和 Blender 导入关系。
- [开发与发布](development.md)：本地环境、测试、打包和 CI。
- [节点资产](node-assets.md)：Geometry Nodes 的构建、加载与验证。
源码位于 `src/anyimage/`，开发工具位于 `tools/`，自动化测试位于 `tests/`。扩展级设置由 `preferences.py` 定义，运行时资源由 `runtime.py` 接入 BlendJob。

55
docs/internals/node-assets.md
View File

@@ -29,15 +29,36 @@ uv run build_node --skip-tests
| 节点组 | 构建模块 |
| --- | --- |
| `O Mesh Plane` | `build_mesh_plane.py` |
| `O Image Depth Plane` | `build_depth_plane.py` |
| `O Image Layer`、`O Image Depth Layer` | `build_material_layers.py` |
| `O Shadeless` | `build_shadeless.py` |
| `O Image Cutout`、`O Image Depth Cutout`、`O Image Depth Balloon` | `build_cutout_modifiers.py` |
| `O Image Plane` | `image_plane.py` |
| `O Image Depth Plane` | `depth_plane.py` |
| `O Image Layer`、`O Image Depth Layer` | `material_layers.py` |
| `O Shadeless` | `shadeless.py` |
| `O Image Cutout`、`O Image Depth Cutout`、`O Image Depth Balloon` | `cutout.py` |
构建从空 Factory 文件生成全部节点组，关闭 `.blend1` 备份，并覆盖唯一发布资产。
Depth Cutout 的布局由 `cutout_depth_surface_layout.py` 编排：Split 属性采样位于主链上方，原始几何旁路沿下方的 Reroute 通道汇入 Switch；长距离投影引用共用分段干线，并在 Uniform 厚度与零厚度输出附近分叉。几何统计支线就近放置在其使用区域。
## 源码职责
`tools/node_assets/` 的模块按资产、几何计算、布局和验证组织：
| 模块 | 职责 |
| --- | --- |
| `asset_library.py` | 构建全部节点组并保存唯一资产 |
| `image_plane.py`、`depth_plane.py` | 普通平面与 Relief / Camera 深度平面 |
| `material_layers.py`、`shadeless.py` | 图像材质层与无光照着色 |
| `cutout.py`、`cutout_shapes.py` | Cutout 修改器接口与各预设几何 |
| `depth_plane_camera.py`、`depth_surface.py` | Camera 平面、共享投影、深度分离与厚度 |
| `cutout_volume.py`、`cutout_rounding.py` | 平滑中心曲面与轮廓圆润 |
| `cutout_inflation.py`、`cutout_shell.py` | 正面膨胀、法线和正背面位置 |
| `plane_layouts.py`、`depth_plane_layout.py` | 平面资产与 Camera 分支布局 |
| `cutout_layout.py`、`cutout_shape_layouts.py`、`depth_cutout_layout.py` | Cutout 阶段布局、坐标与跨区域几何引用 |
| `node_graph.py`、`node_layout.py` | 共用字段节点、平滑构建、布局与边界检查 |
| `asset_validation.py` | 资产清单、接口、结构与布局检查 |
| `image_plane_validation.py`、`cutout_validation.py` | 平面和 Cutout 几何求值验证 |
`tools/build_node.py` 依次启动资产构建与三个验证脚本。运行时加载入口位于 `common/node.py`。
Depth Cutout 的布局由 `depth_cutout_layout.py` 编排：Split 属性采样位于主链上方，原始几何旁路沿下方的 Reroute 通道汇入 Switch；长距离投影引用共用分段干线，并在 Uniform 厚度与零厚度输出附近分叉。几何统计支线就近放置在其使用区域。
三个 Cutout 节点组直接作为修改器使用，统一接收局部 XY 平面，以 `-Z` 表示向图片内部的方向。每组在塑形与平滑后处理 **Inward Axis**：默认 `-Z` 直接输出原生几何，`+X` 分支执行一次 Z 到 X 的轴转换；运行时对象矩阵使用逆变换补偿，保证两种模式落在相同世界空间位置。
@@ -89,19 +110,35 @@ flowchart TD
未关联选中切边的顶点使用原始 XYZ。切口顶点以 `原 XYZ × 所属侧 Z / 原 Z` 保持原相机射线，随后执行既有 Uniform Scale、Reference Depth 和 Depth Scale 投影。
厚度前固定顶点数据，使正背面和侧壁使用相同位置。连接时以原顶点编号和正背面身份构造临时整数坐标，合并同一身份的副本后恢复场景坐标，保持近距离分离区域各自独立。实现位于 `cutout_depth_surface.py`，由 `build_cutout_modifiers.py` 构建进共享节点资产。
厚度前固定顶点数据，使正背面和侧壁使用相同位置。连接时以原顶点编号和正背面身份构造临时整数坐标，合并同一身份的副本后恢复场景坐标，保持近距离分离区域各自独立。实现位于 `depth_surface.py`，由 `cutout.py` 构建进共享节点资产。
暂存值通过 Store Named Attribute 写入保留前缀 `o_depth_`：面中心与相机采样使用 FACE，所属侧相机采样与切口标记使用 CORNER，最终相机坐标、表面位置与源顶点编号使用 POINT。计算区域就近使用 Named Attribute，末尾用一个 Wildcard 模式的 Remove Named Attribute 匹配 `o_depth_*`，输出保留 UVMap 与用户属性。
### 整图 Depth Plane
`O Image Depth Plane` 提供默认 **Relief** 与 **Camera** 模式，共用完整矩形网格和 `depth.exr`。RGB 保存相机 XYZ，Alpha 保存有效性。Relief 与 Depth Balloon 读取 B/Z，Camera 与 Depth Cutout 读取 XYZ。用户控制为 **Mode**、**Subdivide**、**Thickness**、**Depth Scale** 和 Options 中的 **Reference Depth**、**Normal Smooth**；**Depth Image** 与 **Uniform Scale** 为隐藏数据输入。
```mermaid
flowchart TD
A[O Image Plane 零厚度矩形细分] --> B[按 Depth Scale 插值相机 XY 与相对 Z]
B --> C{Thickness}
C -- 0 --> D[单层曲面]
C -- 大于 0 --> E[沿平滑法线向内生成背面并连接边界]
D --> F[清理临时属性并输出]
E --> F
```
两个独立输入均显示 Thickness，默认均为 0 m：第一个控制 Relief 的固定底面基础厚度，第二个控制 Camera 的向内壳厚度。上图描述 Camera 分支：Depth Scale 为 0 时恢复平面，1 时使用完整相机投影；正面 POINT 法线按 Normal Smooth 平滑并归一化，再按 Thickness 向内生成背面。Normal Smooth 范围 0–50、默认 50，仅影响 Camera。模式切换保留各自厚度值并复用图片。Camera 分支位于 `depth_plane_camera.py`，投影与厚度共用 `depth_surface.py`，布局由 `depth_plane_layout.py` 编排。
### 几何节点布局
全部五个几何节点组使用连续横排的几何主轴。主轴可适当宽松，支线内部紧凑，同一目的的计算聚在同一侧，内部间距明显小于与其他部分的距离；独立用途可在两侧错落展开。长算式横排，末端就近汇入主轴。Join Geometry、Menu Switch、Combine XYZ 的同级独立分支末端排成一列并对齐输出端，菜单分支按输入顺序上下排列，共享上游留在公共位置。必要的向左引用直接连接。Group Input 与 Named Attribute 就近放置，全接线的简单运算可折叠，常数和域设置保持可见。
`geometry_layouts.py` 组织平面资产布局，`cutout_shape_layouts.py` 和 `cutout_depth_surface_layout.py` 组织 Cutout 的局部计算阶段；`cutout_layout.py` 将阶段布局放入三个修改器，`node_layout.py` 验证节点边界与同级输入对齐。具体规则见 [AGENTS.md](../../AGENTS.md)。布局验收包含实际节点编辑器预览及几何求值。
`plane_layouts.py` 组织平面资产布局，`cutout_shape_layouts.py` 和 `depth_cutout_layout.py` 组织 Cutout 的局部计算阶段；`cutout_layout.py` 将阶段布局放入三个修改器，`node_layout.py` 验证节点边界与同级输入对齐。布局验收包含实际节点编辑器预览及几何求值。
## 修改节点组
1. 修改 `tools/node_assets/build_*.py`；
1. 修改 `tools/node_assets/` 中对应资产、计算或布局模块；
2. 更新对应验证；
3. 运行一键构建脚本；
4. 搜索并清理旧名称、接口和路径；

2
docs/internals/operators/clipboard-image.md
View File

@@ -38,4 +38,4 @@ flowchart TD
落点仍在 `actions.py` 中完成：3D View 创建 Reference 或 Plane，Node Editor 创建与当前节点树匹配的 Image 节点，绘制模式把 Image 配置为当前画笔的 Texture。鼠标位置和视图位置也在这里换算。
Plane 路径调用 `convert_to_plane/mesh_plane.py` 创建中心原点基础网格，并从节点资产加载 `O Mesh Plane`。`actions.py` 创建图像材质并放入 Mesh 第 `0` 个材质槽，节点组只生成几何。
Plane 路径调用 `convert_to_plane/image_plane.py` 创建中心原点基础网格，并从节点资产加载 `O Image Plane`。`actions.py` 创建图像材质并放入 Mesh 第 `0` 个材质槽，节点组只生成几何。

61
docs/internals/operators/convert-to-plane.md
View File

@@ -1,56 +1,23 @@
# Plane 转换
Plane 转换从当前 Image Empty 创建普通 Plane 或 Depth Plane：普通 Plane 同步完成，Depth Plane 先取得 MoGe-2 Z Depth 与 Tangent Normal 文件，再回到 Blender 创建对象。
## 执行过程
`operators/convert_to_plane/conversion.py` 提供 `ConvertToPlane`、`ConvertToDepthPlane` 与 `GenerateDepthPlane`。对象和材质导入位于 `object.py`，基础矩形网格位于 `image_plane.py`。
```mermaid
flowchart TD
Empty["Image Empty"] --> Kind{"转换类型"}
Kind -- "Plane" --> Color["复制 Color Image"]
Color --> Plane["创建 Plane Object"]
Kind -- "Depth Plane" --> Job["generate-depth-plane-geometry"]
Job --> Result["Color Image + z-depth.exr + depth.json + tangent-normal.png"]
Result --> Import["加载并 Pack Depth / Normal Image"]
Import --> Depth["创建 Depth Plane Object"]
A[Image Empty] --> B{转换类型}
B -- Plane --> C[复制 Color 与动画设置]
C --> D[中心原点四边形与 O Image Plane]
B -- Depth Plane --> E[检查静态图与 AI 就绪状态]
E --> F[generate-depth-plane-geometry]
F --> G[Color、depth.exr、depth.json、Tangent Normal]
G --> H[加载并 Pack 数据]
H --> I[O Image Depth Plane 与材质]
D --> J[替换源 Empty 并保留世界位置]
I --> J
```
两条路径共用 Image Empty 的显示范围、颜色材质和中心原点 Plane。Depth Plane 通过一次 MoGe-2 Server Job 同时生成 Z Depth 与 Tangent Normal；节点根据相机元数据生成单面地形体块，Normal Map 则进入材质着色。
普通 Plane 支持图片、视频和序列，材质保留 Image User 帧设置。基础四边形以局部中心为原点，对象矩阵补偿原 Image Empty 显示范围；节点组生成细分与厚度，材质位于 Mesh 第 0 个槽。
## 节点说明
Depth Plane 从 Preferences 取得 MoGe-2 模型、分辨率级别与 AI 输入上限，一次推理生成相机 XYZ 深度和切线法线。`create_depth_plane_from_result()` 创建颜色材质与深度修改器，并在失败时清理未使用的数据块。
### 选择 Plane 或 Depth Plane
`operators/convert_to_plane/__init__.py` 导出注册类型，实际入口在 `conversion.py`。`ConvertToPlane` 进入本地路径，`ConvertToDepthPlane` 检查静态图片和 AI 环境后启动 `GenerateDepthPlane`。
### 创建普通 Plane
`conversion.py` 先调用 `object.create_color_image()` 复制当前 Image，再由 `object.create_plane_object()` 创建对象。`mesh_plane.py` 把基础四边形顶点放在局部原点两侧，并把 Image Empty 显示范围中心补入对象矩阵，因此世界空间位置不变而对象原点位于面中心。随后从节点资产加载 `O Mesh Plane` 修改器。
材质存入 Mesh 第 `0` 个材质槽。`O Mesh Plane` 将带匿名标记的输入几何与生成几何临时合并，统一设置材质索引 `0` 后再删除输入几何，从而让纯生成面保留原 Mesh 的材质集合。Movie 或 Sequence 的 Image User 设置会保留。转换完成后，新 Plane 替换源 Empty，并沿用原对象名称。
### 生成 Depth 与 Normal
Depth 路径中的 `GenerateDepthPlane` 位于 `conversion.py`。它从 Preferences 读取 **Depth & Normal Model**、**MoGe Resolution Level** 和 **Maximum AI Input Size**，提交 `generate-depth-plane-geometry` Job。
`server/jobs/depth_plane.py` 编排完整图片任务，`server/geometry/prediction_artifacts.py` 把图片合成到黑底后调用一次 MoGe-2，并从同一个 `GeometryFrame` 写出 `z-depth.exr`、`depth.json` 与 `tangent-normal.png`。该 Artifact 生成层也由 Cutout Job 复用；两种顶层 Job 继续各自负责完整图片与 Selection 的输入、结果协议。
EXR 的 RGB 重复保存完整相机 Z，Alpha 保存模型 Validity；JSON 保存图片尺寸、相机内参与有效区域的参考深度。Tangent Normal 使用 PNG 保存，只参与材质着色。AI 输入上限只影响模型分析图，颜色副本仍保留原始分辨率。
### 创建 Depth Plane
Job 返回后，`object.create_depth_plane_from_result()` 加载并 Pack `z-depth.exr` 与 `tangent-normal.png`。Depth 数据块仍由 `depth_data_name()` 命名；Normal 图片设为 Non-Color，并以 Tangent Space 接入 `O Image Layer`。结果材质与 Cutout 一样使用 Principled IOR `1.2`，并由 `O Image Layer` Color 直接驱动 IOR Level。`depth.json` 的参考深度乘以统一尺度后初始化 **Reference Depth**，使它和采样深度处于同一局部距离单位。
`O Image Depth Plane` 把 `O Mesh Plane` 的固定底面放在原图片平面，`Thickness` 只沿正法线移动可变面与中间环，再按各层位置应用深度位移：
```mermaid
flowchart TD
Depth["Z Depth × Uniform Scale"] --> Offset["(Reference Depth - Depth) × Depth Scale"]
Offset --> Clamp["负向位移限制在固定底面之前"]
Block["O Mesh Plane 厚度体块"] --> Weight["固定底面 0 → 中间环 → 可变面 1"]
Clamp --> Apply["位移 × 层权重"]
Weight --> Apply
Apply --> Terrain["单面地形体块"]
```
固定底面位于原图片平面，未置换可变面位于正法线方向的 **Thickness** 距离处。**Depth Scale** 为 `0` 时成品厚度等于 **Thickness**；正向位移可以超出基础厚度，负向位移不会穿过固定底面。`conversion.py` 最后用一个 Undo 步骤替换源 Empty。
`O Image Depth Plane` 提供默认 Relief 和 Camera：Relief 读取深度 Z 形成固定底面的地形；Camera 使用相机 XYZ 投影并沿平滑法线形成壳。两个模式各自保存 Thickness。完整接口见[节点资产](../node-assets.md)，文件协议见[Job 产物](../outputs.md)。

121
docs/internals/operators/cutout-tool.md
View File

@@ -1,108 +1,39 @@
# Cutout Tool
# Cutout
Cutout 从图片上的一个 Selection 开始，可按需生成 Depth 或 Normal Map，最后在 Blender 中创建带材质与 Modifier 的 Mesh Object。
## 执行过程
Cutout 在静态 Image Empty 上圈定 Selection，按轮廓创建 Flat、Solid、Depth Solid 或 Depth Balloon。入口位于 `operators/cutout_tool/main.py`，菜单与采样级别映射位于 `interaction.py`，标识和资产对应关系位于 `shape.py`。
```mermaid
flowchart TD
Select["圈定 SelectionPath"]
Menu["打开 Shape Pie Menu"]
Shape["选择 Shape"]
Raster["SelectionPath → 几何 SelectionMask"]
Route{"需要后台生成数据？"}
Local["读取局部 RGBA<br/>源 Alpha × Selection 值"]
Bounds["写出几何 bounds 矩形 RGBA"]
Job["生成 Depth / Normal 产物"]
Encoded["加载并 Pack 输入 RGBA"]
Ready["贴图 Alpha × Selection Mask"]
Build["构建 BaseShape"]
Create["创建 Blender 对象"]
Select --> Menu --> Shape --> Raster --> Route
Route -- "否" --> Local --> Ready
Route -- "是" --> Bounds --> Job --> Encoded --> Ready
Ready --> Build --> Create
A[Lasso 拖动并提交 SelectionPath] --> B[选择 Shape]
B --> C[栅格化 SelectionMask 并读取源 RGBA]
C --> D{需要 Depth 或 Normal？}
D -- 是 --> E[准备 bounds 矩形 RGBA]
E --> F[generate-cutout-artifacts]
F --> G[加载 Depth、Metadata 与 Normal]
G --> H[源 Alpha 与 SelectionMask 组合]
D -- 否 --> H
H --> I[轮廓、采样、约束三角化与 o_balloon]
I --> J[BaseShape]
J --> K[创建材质、对象与对应节点修改器]
```
Surface 与 Balloon 在本地组合源 Alpha 与几何 Selection 后构建 `BaseShape`；Depth Shape 和 Normal Map 先通过同一个矩形图片协议生成所需产物。
## 节点说明
### 圈定 Selection，选择 Shape
Cutout 工具由 `__init__.py` 注册，交互从 `main.py` 中的 `SelectCutoutSelection` 开始。它接收 Lasso 或双击整图手势，只构建内存中的 `SelectionPath`，然后调用 `interaction.py` 打开 Shape 饼菜单。此时不读取源图像素、不栅格化 Mask，也不扫描可见 Alpha。
对象命中由 Blender 原生点选决定。普通点击另一对象时只切换 active；空白单击保留当前 active Image Empty。Lasso 可从透明区域或 Viewport 空白开始拖动，双击也可从空命中位置打开整图 Shape 菜单。起始点不判断图片范围或 Alpha，完整 Selection 的几何与内容有效性在后续栅格化和 Shape 构建阶段判断。
`interaction.py` 读取工具选项，并在构建菜单按钮时把只含 Points 的完整 `SelectionPath` 编码一次，同时把当前 **Inward Axis** 固定到各 `CutoutSelectionToShape`。这次编码只用于跨 Blender Operator 传值，不属于 Selection Mask 构建。**Mesh Detail** 的 Low、Medium、High、Ultra 分别携带 32、16、8、4 px 的源图采样间距；档位越高，采样越密。菜单中的 Surface、Balloon、Depth Balloon 和 Depth Surface，以及它们对应的节点分支索引，统一来自 `shape.py`。
### 栅格化 Selection
用户选定 Shape 后，`main.py` 中的 `CutoutSelectionToShape` 接过主线。它从自身 Property 恢复 `SelectionPath`，再使用 `common/selection.py` 以 non-zero winding 在 Path Bounds 与 2 px 抗锯齿边缘内栅格化为一次纯几何 `SelectionMask`。`values` 保存局部几何权重，`bounds` 保存它在源图中的位置；栅格阶段不读取、接收或合并源 Alpha。
实际 Cutout 业务随后读取一次源 RGBA。无需 AI 时，它在几何 bounds 内显式计算 `source_alpha × selection_values`，由该内容 Mask 判断空内容并约束轮廓；原始 `SelectionMask` 不被修改。此时按 `Selection bounds 长边 ÷ Mesh Detail 像素间距` 计算实际采样长边；只有它超过 **Maximum Cutout Mesh Resolution** 时才等比增大有效间距。bounds 的原始像素长边本身不会触发降密。
### 需要后台生成数据？
选择 Depth Balloon、Depth Surface 或开启 **Normal Map** 时需要提交 Job。
Surface 和 Balloon 的本地路径直接使用本地内容 Mask。
### 生成 Depth / Normal 产物
`main.py` 先通过 `common/image.py` 把 Selection 几何 bounds 内的原始 RGBA 写成矩形输入。该文件保留源 Alpha，不预乘 Lasso 值；文件、Packed 或内存图片都使用同一个入口。随后把请求提交给 `server/jobs/cutout.py`，生成本次 Shape 需要的产物：
- 需要 Vector Depth 时写出相机空间 XYZ 组成的 `vector-depth.exr`，需要 Z Depth 时写出 `z-depth.exr`，两者都配套 `depth.json`；
- **Normal Map** 根据既定空间写出 `tangent-normal.png` 或 `object-normal.png`。
Depth 与 Normal Map 共享同一个 MoGe-2 `GeometryFrame`。模型输入先按 Alpha 合成到黑底；Vector/Z Depth 的 RGB 保留完整连续 Field，Alpha 保存模型 Validity。Depth Surface 的 Normal Map 编码为 Cutout 原生 XY / Z 对象空间，正面中性法线为局部 `+Z`；其余 Shape 使用 Tangent Space。`main.normal_mode_for_shape()` 根据 Shape 确定 Normal Space。
**Maximum AI Input Size** 约束 MoGe-2 的分析图长边；网格密度使用 **Maximum Cutout Mesh Resolution**，最终颜色图使用 Selection bounds 分辨率。
Color 使用 Selection bounds 内的源 RGBA；Normal 的可见区域、MoGe-2 黑底输入和 Depth Metadata 的参考区域使用源 Alpha。Depth EXR 的 RGB 保存连续 Field，Alpha 保存模型 Validity。Selection Mask 只在 Blender 构建网格时与贴图 Alpha 相乘。
### 构建 BaseShape
Job 完成后，`main.py` 在 Blender 主线程加载并 Pack 可选的 Depth 与 Normal 图片，并复用提交前的矩形 RGBA 输入作为颜色图。它从同一 Image 读取 Alpha，与已栅格化的 Selection Mask 相乘后构建网格内容 Mask；颜色及 AI 贴图保持原 Alpha。`main.create_cutout_shape()` 调用 `geometry.py` 中的 `build_base_shape()`：
1. 从内容 Alpha 值提取外轮廓和孔洞；
2. 按 Mesh Detail 对应的采样间距生成轮廓与内部采样点；
3. 进行约束三角化；
4. 计算四种 Shape 共用的 `o_balloon`；
5. 汇总顶点、三角面、UV 和 Balloon 权重，得到内存中的 `BaseShape`。
`geometry.py` 还提供 Depth 标定、采样和网格尺度计算，供后面的对象创建阶段使用。
### 创建 Blender 对象
`main.create_cutout_shape()` 把 `BaseShape`、颜色 Image 和可选产物交给 `object.py` 的 `create_shape_object()`。这里创建 Material、Mesh Object，并挂上统一的 `O Image Cutout` Modifier。材质通过 `common/material.py` 创建，Cutout 使用 `1.2` 的 IOR，并由 `O Image Layer` Color 直接驱动 Principled BSDF 的 IOR Level。本地 Cutout 创建一次几何 bounds 颜色图；AI Cutout 直接复用已编码并 Pack 的矩形输入。
| Shape | Modifier 如何形成结果 | 必需后台产物 |
| 预设 | 节点组 | AI 产物 |
| --- | --- | --- |
| Surface | 保持基础平面轮廓，可增加真实厚度 | 无 |
| Balloon | 使用 `o_balloon` 从边缘向内部鼓起 | 无 |
| Depth Balloon | 在 Balloon 上叠加相对基准面的深度 | `z-depth.exr`、`depth.json` |
| Depth Surface | 使用相机点 XYZ 重建透视表面 | `vector-depth.exr`、`depth.json` |
| Flat | `O Image Cutout`，Thickness 0 | 可选 Tangent Normal |
| Solid | `O Image Cutout`，Thickness 1 | 可选 Tangent Normal |
| Depth Solid | `O Image Depth Cutout` | Depth 与 Object Normal |
| Depth Balloon | `O Image Depth Balloon` | Depth 与 Tangent Normal |
Depth Shape 的 Reference Depth，以及 Depth Balloon 的 Direction，都由 `object.py` 调用 `geometry.py` 的标定函数，根据最终网格内容 Mask 计算。透明像素达到 `5%` 时使用有效内容区域自适应标定，否则使用稳定的固定基准。
## 轮廓与网格
Normal Map 关闭时材质使用几何法线；开启时，`object.py` 把生成的图片和已经确定的 Normal Space 交给材质节点。最后 `main.py` 提交 Undo，整条 Cutout 流程结束。
`geometry.py` 从内容 Alpha 提取连通轮廓和孔洞，按 Mesh Detail 采样并生成 `BaseShape`。Low、Medium、High、Ultra 分别以 32、16、8、4 个源像素作为基础间距，并受 Maximum Cutout Mesh Resolution 限制。
### Inward Axis
`fine_mesh.py` 负责 Fine Outline 的细节保留与约束网格，`polygon.py` 提供轮廓运算。Balloon Profile 按 Preferences 选择 Poisson 或 Teddy，生成所有预设共用的 POINT 属性 `o_balloon`。
```mermaid
flowchart TD
Base["BaseShape 与四种 Shape<br/>原生 XY / Z"] --> Switch{"Inward Axis"}
Switch -- "-Z（默认）" --> Native["节点几何直接输出<br/>对象沿用源 Empty local transform"]
Switch -- "+X" --> Orient["O Image Cutout 执行 Z → X<br/>对象矩阵执行 X → Z"]
Native --> World["相同世界空间结果"]
Orient --> World
Normal["Depth Surface<br/>Object Space Normal"] --> Mode{"Inward Axis"}
Mode -- "-Z" --> Keep["保持原生 Z 轴通道"]
Mode -- "+X" --> Swizzle["按 Z → X 转换一次"]
```
## AI 与对象创建
BaseShape 与 Shape 子节点始终在局部 XY 平面建模，并沿 `-Z` 向图片内部生成厚度和深度。统一的 `O Image Cutout` Modifier 只在选择 `+X` 时转换最终几何；`object.py` 同步使用逆矩阵，因此默认 `-Z` 与源 Image Empty 的 local transform 一致，`+X` 模式仍保持相同世界空间形状。Depth Surface 的 Object Space Normal 默认直接使用原生 Z 轴 Artifact，仅在 `+X` 模式转换通道；Tangent Space Normal 不变。
Server 接收 Selection bounds 内保留源 Alpha 的 RGBA，一次 MoGe-2 推理按 `generate_depth` 与 `normal_mode` 返回产物。深度边缘延展使用所选 Alpha Threshold，Normal 使用原预测；AI 输入上限只影响分析分辨率。
Balloon 与 Depth Balloon 的材质会创建 Geometry Attribute 节点，把 `o_balloon` 的 Float 输出直接连接到 `O Image Layer` 已有的 **Normal Scale**；Surface 与 Depth Surface 不建立这条连接。
`object.py` 的 `create_shape_object()` 消费 BaseShape、Color、Depth 和 Normal，加载对应节点组。基础网格位于局部 XY，向图片内部为 -Z；选择 +X 时节点与对象矩阵配对变换，保持世界几何一致。Object Normal 同步按方向转换。
材质使用 IOR 1.2，颜色进入 Image Layer，`o_normal_scale` 控制法线影响。Depth Solid 使用正面 FACE 标记，其他适用预设使用轮廓权重。节点计算见[节点资产](../node-assets.md)。

32
docs/internals/operators/debug.md
View File

@@ -1,32 +0,0 @@
# Debug Operator
Debug Operator 从面板中的模型参数发起一次原始推理，把结果文件写入 Job 目录并打开目录供开发者检查。
## 执行过程
```mermaid
flowchart TD
Panel["Debug 面板"] --> Operator["选择模型 Debug Operator"]
Operator --> Request["读取 Scene 参数"]
Request --> Job["提交对应 Debug Job"]
Job --> Output["写出原始预测产物"]
Output --> Directory["打开 Job 目录"]
```
DA3、MoGe-2、BEN2 和 Upscale 都走这条调试路径，用于验证模型、设备、请求参数和输出格式。
## 节点说明
### 从 Debug 面板进入
入口显示在 `src/anyimage/panel.py`，四个 Operator 定义在 `operators/debug.py`：`DebugDa3`、`DebugMoge2`、`DebugBen2` 和 `DebugUpscale`。它们读取 `Scene.anyimage_settings` 中的输入路径、设备、模型和分辨率参数，并检查 Environment 与模型是否可用。
### 提交原始推理
`debug.py` 分别提交 `debug-da3`、`debug-moge2`、`debug-ben2` 和 `debug-upscale` Job。Server 侧由 `server/jobs/debug_predictions.py` 接收请求，再调用相应模型适配器。
DA3 和 MoGe-2 的可视化与数组输出由 `server/outputs/debug_da3.py`、`server/outputs/debug_moge2.py` 写出；BEN2 与 Upscale 保存模型返回的图片结果。
### 查看结果
任务完成后，`debug.py` 的 `_open_job_directory()` 调用 Blender 的路径打开操作，直接显示该 Job 的目录。

21
docs/internals/operators/frame.md Normal file
View File

@@ -0,0 +1,21 @@
# Frame 多图合成
`operators/image_edit_tool/frame.py` 定义 `FrameImages`、`FrameTool` 和工具激活入口。Frame 将选中的静态 Image Empty 按当前视图合成为一张图片。
```mermaid
flowchart TD
A[拖出屏幕矩形] --> B[冻结视图矩阵并收集选中静态图片]
B --> C[检查矩形与 active 图片的有效投射交集]
C --> D[按 Region 与 Maximum Frame Resolution 计算输出尺寸]
D --> E[观察射线投影到各图片平面]
E --> F[按样本深度从远到近合成，等深度 active 优先]
F --> G[边界覆盖采样与独立 Alpha]
G --> H[创建 Packed Image 并复用 active Object]
H --> I[删除其他参与图片对象并提交 Undo]
```
正交视图接受有限深度，透视视图采样观察方向前方的交点。交叠检查采用连续投影范围；输出尺寸受 Region、最长边上限和总像素上限约束。
RGB 与 Alpha 分别累计，保留被 Mask 隐藏的有效颜色。边界使用覆盖采样，内部复用直接采样，避免黑边和不必要的模糊。结果平面朝向观察者，位置由 active 图片的有效深度确定。
Frame 保留 active 对象身份、名称和 Collection；图像替换复用 [Common](../common.md) 的共享数据与失败恢复逻辑。

57
docs/internals/operators/image-edit-tool.md
View File

@@ -1,57 +0,0 @@
# Image Edit Tool
工具栏包含 **Frame**、**Mask**、**Rectify** 三合一 Image Edit 组，Cutout 作为紧接其后的独立工具。
## 工具设置
Frame 和 Rectify 没有业务选项。Frame 让 active Image Empty 承载合成结果，并删除其他参与合并的选中 Image Empty；Rectify 直接替换 active Image Empty 的图片。
Mask 保存 **Gesture**、**Mode** 和 **Radius**。Gesture 包含 Lasso、Brush 与 Polyline；Mode 包含 Set、Add 与 Subtract，默认使用 Subtract；Radius 默认 25 px，仅在 Brush 时显示和生效。
## 执行过程
```mermaid
flowchart TD
Group["Frame / Mask / Rectify"]
Group --> Kind{"子工具"}
Kind -- "Frame" --> Sources["选中 still Image Empty<br/>active 为对象和深度基准"]
Sources --> Project["Frame 像素生成观察射线<br/>与图片平面求交"]
Project --> Composite["逐像素深度与 Alpha 合成"]
Composite --> Canvas["完整 Frame Canvas<br/>范围外透明"]
Canvas --> Facing["active 朝向当前视角<br/>删除其他参与对象"]
Kind -- "Mask" --> Gesture{"Lasso / Brush / Polyline"}
Gesture --> Mask["栅格化 SelectionMask"]
Mask --> Mode{"Set / Add / Subtract"}
Mode --> Alpha["只合成当前 Alpha<br/>保持完整 Canvas"]
Kind -- "Rectify" --> Quad["四点 Quad 与 Homography"]
Alpha --> Replace["替换 active Empty 的 Image"]
Quad --> Replace
```
## Frame
`frame.py` 读取所有选中的 still Image Empty，并要求 active object 是 Selection 内的 still Image Empty。仅有 active 状态但未选中的图片不构成输入；active 是非图片对象时，即使 Selection 中存在图片也以 warning 取消。Frame 可以从 Viewport 任意位置开始，但结果矩形必须与 active 图片投射相交。选中的非 Image Empty 被忽略；任一选中 Image Empty 是视频或序列时整体取消。
每个参与图片使用冻结的 Viewport 投影矩阵与对象世界矩阵建立屏幕到图片平面的 Projective Transform。Frame 输出像素中心沿当前观察射线映射到图片局部平面，再换算为源像素坐标；有限且位于 Image Empty 显示边界内的交点参与采样。正交视图接受任意有限视图深度；透视视图只接受观察方向前方的交点。图片部分跨越透视观察原点时，前方区域继续采样，后方、平行射线和图片范围外透明。
输出像素按观察射线上的几何深度从远到近排序，再以预乘 Alpha 合成；同深度时 active 位于最上层。Frame 与 active 的交叠通过连续有效投射区域和 Frame 矩形的正面积交集判断，不依赖源 Alpha 或离散输出像素中心。输出分辨率使用 Frame 在当前 Region 中的屏幕像素宽高，不超过 Region 尺寸，并由 Preferences 的 **Maximum Frame Resolution** 等比限制最长边；总像素不超过 16,777,216。透明边缘保留在完整 Canvas 中；紧邻内容的一圈透明像素保留相邻 RGB，避免线性插值产生黑边。
Frame 与 Cutout Lasso 只在 `CLICK_DRAG` 后进入手势。Mask 与 Rectify 使用 `PRESS` 启动。三个绘制型工具都由 Blender 原生点选决定命中对象：点到另一对象时只切换 active，不开始编辑；没有命中对象时保留当前 active Image Empty，因此透明区域和 Viewport 空白都可作为起点。Mask 使用 `PAINT_CROSS` cursor；Brush 单击生成一个圆形印记。Polyline 通过逐点单击建立路径，至少三个点后在起点周围显示随鼠标接近而放大的闭合圈，并以点击 15 px UI Scale 命中范围、双击或 Enter 完成；Backspace 删除最后一点。手势与图片范围、可见内容的有效性在提交阶段判断。Frame 的普通空白点击继续清空选择，Shift 点击由显式 Viewport 对象选择 keymap 处理。
结果平面本地 XY 平行视平面且正面朝向观察者。正交视图与 active 原点位于透视观察方向前方时，结果沿用 active 对象原点的视图深度；透视 active 原点位于视后方但图片仍有有效前方交叠时，结果使用有效交叠区域屏幕中心与 active 图片平面的交点深度。active Blender Object 保留名称、Collection 和身份并承载 packed 合成 Image，其他参与 Image Empty 随后删除；整个操作属于一个 Undo 步骤。
## Mask
`mask.py` 定义一个 WorkspaceTool 和一个 `EditImageAlpha` Operator。Operator 启动时冻结 Scene 中保存的 Gesture、Mode 和 Radius；Lasso 把自由手绘闭合路径转换为图片像素，Polyline 把逐点闭合路径送入同一个 SelectionMask 与 Alpha 合成入口。
Brush 拖动阶段只维护经过距离与近共线简化的屏幕轨迹，并为每个采样点生成圆、为每对相邻点生成等宽连接条；不读取源 RGBA，不创建临时 Image，也不投影、栅格化或更新 Pixel Buffer。Overlay 在局部 screen-space scanline bands 中合并这些 Primitive，生成无内部接缝、尖角或三角孔的一次填充与闭合外轮廓；Subtract 为红色，Set/Add 为灰色。释放后把同一组 Primitive 投影到图片，以逐像素最大值合入单个覆盖画布，再通过公共 Alpha 合成与 Image 替换入口提交 Packed 结果。取消或切换工具只结束 Overlay，源 Image 与共享引用始终不变。
设当前 Alpha 为 `A`、手势权重为 `M`：Set 计算 `A × M`，Add 计算 `max(A, M)`，Subtract 计算 `A × (1 - M)`。RGB 原样保留。结果始终使用源图片宽高和原有画框，因此 Image Empty 的 identity、matrix、display size 和 offset 保持不变；全透明结果同样有效。
## Rectify
`rectify.py` 接受任意顺序的四个角点并规范为凸 Quad，创建 512 × 512 预览，随后按目标宽高比执行分块 Homography。输出最长边不超过 8192 px，总像素不超过 16,777,216；插值使用预乘 Alpha，结果按可见 Alpha trim 后映射回源图片平面。
Mask 与 Rectify 最终都以 packed Image 调用 `common/image.py` 的 `replace_empty_image()`。Mask 不提供 placement bounds；Rectify 使用校正区域的 placement bounds 调整图片画框。Frame 复用同模块的 Image 创建、Homography、预乘采样和单对象替换函数，并在自身业务边界完成多图合成与对象删除。
三个工具的结果都以源 Image 的完整名称请求命名，不移除 `.png` 等名称内容，也不添加材质贴图后缀。源数据仍被其他对象引用时，Blender 为结果追加数字后缀；源数据可清理时，替换入口让结果恢复准确原名。

26
docs/internals/operators/index.md
View File

@@ -1,17 +1,15 @@
# Operator 实现索引
`src/anyimage/operators/` 负责 Blender 侧交互、Job 请求组装和结果导入。
| 源码（相对 `src/anyimage/`） | 功能 |
| --- | --- |
| `operators/ai_setup.py` | [AI 环境管理](ai-setup.md) |
| `operators/clipboard_image/` | [剪贴板图片](clipboard-image.md) |
| `operators/convert_to_plane/` | [Plane 与 Depth Plane](convert-to-plane.md) |
| `operators/cutout_tool/` | [Cutout](cutout-tool.md) |
| `operators/image_edit_tool/frame.py` | [Frame 多图合成](frame.md) |
| `operators/image_edit_tool/mask.py` | [Mask Alpha 编辑](mask.md) |
| `operators/image_edit_tool/rectify.py` | [Rectify 透视校正](rectify.md) |
| `operators/remove_background.py` | [Remove Background](remove-background.md) |
| `operators/upscale.py` | [Upscale](upscale.md) |
| Operator 路径 | 实现页 | 主要职责 |
| --- | --- | --- |
| `operators/__init__.py` | 本页 | 汇总 Blender 类型注册顺序 |
| `operators/ai_setup.py` | [AI 环境管理](ai-setup.md) | 环境安装、模型下载与缓存释放 |
| `operators/clipboard_image/` | [Clipboard Image](clipboard-image.md) | 系统剪贴板读取、粘贴与快捷键 |
| `operators/convert_to_plane/` | [Convert to Plane](convert-to-plane.md) | Mesh Plane 与 Depth Plane 创建 |
| `operators/cutout_tool/` | [Cutout Tool](cutout-tool.md) | Selection、基础 Shape 构建与 Blender 对象创建 |
| `operators/image_edit_tool/` | [Image Edit Tool](image-edit-tool.md) | Frame 合成、Mask Alpha 编辑与 Rectify 透视校正 |
| `operators/debug.py` | [Debug Operator](debug.md) | 模型接口与原始产物验证 |
| `operators/remove_background.py` | [Remove Background](remove-background.md) | BEN2 去背与结果替换 |
| `operators/upscale.py` | [Upscale](upscale.md) | 当前 Image 的 4× 高清与输入长边检查 |
`operators/__init__.py` 聚合注册类，扩展入口按顺序注册并逆序注销。
类型在 `operators/__init__.py` 和扩展 `__init__.py` 汇总注册。AI Operator 使用普通字典提交任务，并在主线程响应中读取文件和修改 Blender 数据。

23
docs/internals/operators/mask.md Normal file
View File

@@ -0,0 +1,23 @@
# Mask Alpha 编辑
`operators/image_edit_tool/mask.py` 定义 `EditImageAlpha` 和 `MaskTool`，使用 `common/viewport.py` 的 `ImageGesture` 管理 Lasso、Brush 与 Polyline。
```mermaid
flowchart TD
A[冻结 Scene 的 Gesture、Mode、Radius] --> B[屏幕手势与 Overlay]
B --> C[完成后投影并生成覆盖 M]
C --> D[读取源 RGBA]
D --> E[按模式计算 Alpha]
E --> F[保留 RGB 和源画布尺寸]
F --> G[替换 Packed Image 并提交一个 Undo]
```
| 模式 | 源 Alpha A 与覆盖 M 的结果 |
| --- | --- |
| Set | `A × M` |
| Add | `max(A, M)` |
| Subtract | `A × (1 − M)` |
Brush 拖动时维护简化屏幕路径、圆形印记与连接条的并集预览；释放时一次生成图片覆盖。Polyline 使用逐点路径，至少三个点后通过闭合圈、双击或 Enter 完成，Backspace 删除末点。
结果保留 Empty 的身份、矩阵、显示尺寸和偏移。全透明结果有效，取消或切换工具清理未提交的 Overlay；共享 Image 通过统一替换入口分流。

17
docs/internals/operators/rectify.md Normal file
View File

@@ -0,0 +1,17 @@
# Rectify 透视校正
`operators/image_edit_tool/rectify.py` 定义 `RectifyImagePerspective` 与 `RectifyTool`，将图片内的四角区域校正为矩形。
```mermaid
flowchart TD
A[点选四角] --> B[规范为凸 Quad 并检查图片交叠]
B --> C[512×512 预览与宽高比调整]
C --> D[按像素密度和输出上限确定尺寸]
D --> E[分块 Homography 与预乘 Alpha 插值]
E --> F[按可见 Alpha 裁剪]
F --> G[映射 placement bounds 并替换 Packed Image]
```
四角允许任意点击顺序，退化或非凸输入在提交前校验。预览根据源颜色空间配置显示，像素重采样复用 `common/image.py`。输出最长边上限 8192，总像素上限 16,777,216。
`replace_empty_image()` 根据校正区域的 placement bounds 调整画框，保留源 Image 名称、对象身份和共享引用语义。

49
docs/internals/operators/upscale.md
View File

@@ -1,44 +1,21 @@
# Upscale
Upscale 从当前 Image Empty 读取单图或多帧媒体，在输入长边允许时生成 4× RGBA 结果并替换原图。
## 执行过程
`operators/upscale.py` 从当前 Image Empty 生成 2× 结果并替换原图，支持单图、视频与序列帧。
```mermaid
flowchart TD
Empty["Image Empty"] --> Gate{"长边小于输入上限？"}
Gate -- "否" --> Disabled["禁用或报告错误"]
Gate -- "是" --> Prepare["准备媒体输入"]
Prepare --> Job["upscale-image Job"]
Job --> RGB["分块执行 4× RGB 推理"]
Job --> Alpha["独立缩放 Alpha"]
RGB --> Merge["合并 RGBA"]
Alpha --> Merge
Merge --> Replace["导入并替换当前 Image"]
A[检查源图长边小于 AI 输入上限] --> B[冻结 Preferences 模型与尺寸设置]
B --> C[准备媒体输入]
C --> D[upscale-image 逐帧处理]
D --> E{所选模型}
E -- Real-ESRGAN 或 SAFMN --> F[256×256 tile 与边缘上下文执行 4× RGB 推理]
E -- PiSA-SR --> G[分阶段生成推理，细节强度 0.7]
F --> H[统一缩放到源图 2×]
G --> H
H --> I[原 Alpha 独立 Lanczos 缩放并回填]
I --> J[写出 upscale 序列并替换 Image]
```
模型只处理 RGB，Alpha 单独放大后再合并。Blender 侧在提交前检查 **Maximum AI Input Size**，Server 模型层也应用同一个输入上限。
`server/models/upscale.py` 统一处理输入上限、模型分派和最终 RGB/Alpha 尺寸。`onnx_upscale.py` 处理常规 4× 模型，`onnx_pisa_sr.py` 处理 PiSA-SR。多帧与多 tile 只在最后一次运行请求内存收缩。
## 节点说明
### 检查尺寸并准备输入
入口位于 `operators/upscale.py`。`UpscaleImage.poll()` 读取 `preferences.py` 中的 **Maximum AI Input Size**；当前 Image 长边必须严格小于该值。菜单显示的预计尺寸来自同一文件的 `upscale_output_size()`。
执行时，`upscale.py` 冻结当前模型和输入上限，并通过 `common/image.py` 准备单图、视频或序列输入。Packed Image 与内存修改结果会导出临时文件，多帧输入保留帧起点和抽帧设置。
### 分块放大 RGB
`UpscaleImage` 提交 `upscale-image` Job。`server/jobs/upscale.py` 展开媒体帧并逐帧调用模型；`server/models/upscale.py` 负责输入约束、RGB/Alpha 分离和结果合并，`server/models/onnx_upscale.py` 以 256 × 256 分块执行 4× ONNX 推理并带入相邻上下文。
`server/media/resolution.py` 统一解释输入长边上限。Blender 侧要求源图长边小于该值，结果长边小于该值的四倍。
### 合并 Alpha
`server/models/upscale.py` 使用 Lanczos 将 Alpha 放大到目标尺寸，再与 4× RGB 合并为 RGBA。Provider 资源不足时，Server 释放当前 Upscale Session，并返回降低输入上限或改用 CPU 的提示。
### 导入并替换结果
响应阶段回到 `operators/upscale.py`，通过 `common/image.py` 读取 `upscale.*` 结果序列并替换当前 Empty。
后续 Upscale 和 Depth Plane 都以当前结果 Image 为输入。
Provider 资源错误会释放当前 Upscale Session，并返回降低输入上限或切换 CPU 的提示。响应通过 `common/image.py` 加载、Pack、保留图像共享与帧设置，并提交 Undo。

72
docs/internals/outputs.md
View File

@@ -1,51 +1,43 @@
# Job 产物
Server 在 Job 目录写入图片、JSON 或 Debug NPZ；Blender 根据 Job 返回值读取文件并创建 Image、Mesh、Material 与 Object。
Server 把结果写入 Job 目录，返回以相对文件名为值的字典。Blender 主线程通过 JobResult 读取文件，再创建或更新 Image、Mesh、Material 和 Object。
## 正式 Job
| Job | 文件 | 消费方 |
| --- | --- | --- |
| `remove-background` | `image.0001.png` 起 | Image Empty 替换 |
| `upscale-image` | `upscale.0001.png` 起 | 2× Image Empty 替换 |
| `generate-depth-plane-geometry` | `image.0001.*`、`depth.exr`、`depth.json`、`tangent-normal.png` | Plane 对象与材质 |
| `generate-cutout-artifacts` | 按请求提供深度、元数据和法线 | Cutout 对象与材质 |
| Job | 主要文件 | Blender 消费方式 |
| | | |
| `remove-background` | `image.0001.png` 起 | 替换 Image Empty 的颜色数据 |
| `upscale-image` | `upscale.0001.png` 起 | 替换或新建高清颜色数据 |
| `generate-depth-plane-geometry` | `image.0001.*`、`z-depth.exr`、`depth.json` | 创建 Depth Plane |
| `generate-cutout-artifacts` | 可选 Depth 与 Normal 文件 | 创建 Cutout Shape 与材质 |
## 深度与法线协议
Depth Job 保存可由 Blender 直接加载的模型空间 EXR；Blender 主线程负责轮廓提取、三角化、Depth 标定和数据创建。
| 返回字段 | 文件 | 内容 |
| --- | --- | --- |
| `depth` | `depth.exr` | float32 RGB 保存相机空间 XYZ，Alpha 保存 Validity |
| `depth_metadata` | `depth.json` | `image_size`、3×3 `intrinsics`、`reference_depth` |
| `tangent_normal` | `tangent-normal.png` | 切线空间法线 |
| `object_normal` | `object-normal.png` | Cutout 原生 XY / Z 坐标的对象空间法线 |
## Depth 与 Normal 文件
```mermaid
flowchart TD
A[矩形源 RGBA] --> B[限尺寸并合成黑底模型输入]
B --> C[MoGe-2 GeometryFrame]
C --> D[原预测生成 Normal]
C --> E{Cutout 深度？}
E -- 是 --> F[按源 Alpha 延展轮廓深度]
E -- 否 --> G[完整相机 Points]
F --> G
G --> H[depth.exr 与 depth.json]
D --> I[Blender 加载并 Pack]
H --> I
```
MoGe-2 的原始结果由模型适配器转换为单帧 `GeometryFrame`。它包含 Depth、Validity、Intrinsics，以及可选的 Normal 和 Points；`geometry/prediction_artifacts.py` 把同一 Frame 交给各 Writer，并按请求写出以下文件：
`depth_texture.py` 保存完整有限 Points；模型 Mask 转为 `0/1` Validity。参考深度取有效、有限、正深度且满足源 Alpha 条件的样本中位数：整图使用 Alpha ≥ 0.1，Cutout 使用所选 Alpha Threshold 并要求 Alpha > 0。非法字段和空参考区域抛出错误。
| 文件 | 内容 |
| --- | --- |
| `vector-depth.exr` | RGB 保存完整相机空间 XYZ，Alpha 保存 Validity |
| `z-depth.exr` | RGB 重复保存完整相机 Z，Alpha 保存 Validity |
| `depth.json` | 保存 `image_size`、3×3 `intrinsics` 与有效区域的 `reference_depth` |
| `tangent-normal.png` | Tangent Space Normal Map |
| `object-normal.png` | Cutout 原生 XY / Z 空间的 Object Space Normal Map |
`common/depth.py` 加载深度并配置 Non-Color、CHANNEL_PACKED 与 Pack。Relief 和 Depth Balloon 读取 B/Z，Camera 与 Depth Solid 读取 XYZ，共用同一 EXR。
MoGe-2 的布尔 Mask 以 `0/1` 写入 Validity。Mask 外的有限 Depth/XYZ 仍保留在 RGB，使 Geometry Nodes 的线性采样维持连续；输入 Alpha、正深度和有限性只参与 `depth.json` 的参考样本筛选。任一待写 Field 包含 NaN/Inf 时不生成 EXR。
## Selection 与颜色
Blender 通过 `common/depth.py` 加载 EXR，设为 Non-Color 并 Pack。Depth 与 Normal 数据块分别由 `depth_data_name()` 和 `normal_data_name()` 命名。Geometry Nodes 采样已加载的 EXR。
Cutout 提交几何 bounds 内的原始 RGBA。Blender 在创建网格时组合源 Alpha 和 `SelectionMask`；颜色图保留矩形源 RGBA，法线可见区域取源 Alpha。AI 分析尺寸与颜色分辨率分别处理。
## Cutout Selection 产物
Blender 先把 `SelectionPath` 栅格化为 `SelectionMask`，再把局部 RGBA 作为 Server 输入。MoGe-2 输入统一按 `RGB × Alpha` 合成到黑底；完全透明像素下的隐藏 RGB 不影响预测。Server 不接收 Path 或 Mask 对象协议：
- 需要 Vector Depth 时请求 `vector-depth.exr` 与 `depth.json`；
- 需要 Z Depth 时请求 `z-depth.exr` 与 `depth.json`；
- Normal Map 按空间请求 `tangent-normal.png` 或 `object-normal.png`。
Cutout 的 Color、Normal 与 Depth 使用同一图片 Alpha 内容范围；Blender 再将该 Alpha 与 `SelectionMask` 相乘构建 Shape。Surface 与 Balloon 的本地路径直接组合源 Alpha 和 `SelectionMask`。
## Debug 产物
| Job | 文件 |
| | |
| `debug-da3` | `prediction.npz` |
| `debug-moge2` | `prediction*.npz` |
| `debug-ben2` | `foreground*.png` |
| `debug-upscale` | `prediction*.png` |
Debug Job 保存模型适配层的直接输出。
Remove Background、Upscale、Frame、Mask、Rectify 通过 `common/image.py` 的编辑结果入口保留源 Image 名称和共享引用语义。Plane 与 Cutout 的材质贴图使用 Color、Depth、Normal 命名函数。

102
docs/internals/runtime.md
View File

@@ -1,93 +1,53 @@
# Job Runtime 与 Server
AnyImage 使用 BlendJob 管理独立 Python Environment、Server 进程与单 Worker 队列。Blender 侧准备输入、提交普通字典参数并导入结果。
## 目录职责
```text
src/anyimage/
├── runtime.py # Environment 与 JobRuntime 配置
└── server/
├── app.py # 公开 Job 注册
├── model_catalog.py # 模型声明
├── model_download.py # 下载与校验
├── model_manager.py # 模型目录与 Session 生命周期
├── jobs/ # 公开协议与流程编排
├── media/ # 图片、视频、序列帧和尺寸处理
├── geometry/ # Depth / Normal 文件产物与相机元数据
├── models/ # 模型适配与 ONNX 推理
└── outputs/ # Debug Prediction 序列化
```
`jobs/` 编排任务，领域目录提供媒体、Geometry、模型和输出能力。
## 生命周期
`runtime.py` 配置 BlendJob `JobRuntime` 和独立 Python 3.12 Environment。`server/app.py` 注册 Job，`model_manager.py` 作为 `model_manager` Resource 管理下载文件与缓存 Session。
```mermaid
flowchart TD
Request["Operator request()"] --> Runtime["JobRuntime"]
Runtime --> Queue["单 Worker 队列"]
Queue --> Job["app.py 注册的 Job"]
Job --> Resource["ModelManager / Media"]
Resource --> Directory["jobs/<job-id>/"]
Directory --> Response["Operator response()"]
A[Operator request] --> B[JobRuntime]
B --> C[Server 单 Worker 队列]
C --> D[Job 调用媒体与模型模块]
D --> E[写入 jobs/job-id 目录]
E --> F[主线程 response]
```
Job 在阶段边界调用 `context.check_cancelled()`，并把稳定产物写入当前 Job 目录。Blender 响应完成后，再由输入所有者清理临时 Packed Image 文件。
Job 在阶段和帧边界检查取消，进度通过 `context.progress()` 返回。视频临时帧由 `media/input.py` 的上下文管理器清理，Blender 导出的临时输入由对应 Operator 清理。
## 公开 Job
## Job 与数据模块
| Job | 主要用途 |
| Job | 实现与用途 |
| --- | --- |
| `download-model`、`download-required-models` | 准备模型文件 |
| `remove-background` | 生成 BEN2 透明前景 |
| `upscale-image` | 4× 图片或帧序列 |
| `generate-depth-plane-geometry` | 为 Depth Plane 复制颜色并生成 Z Depth 与相机元数据 |
| `generate-cutout-artifacts` | 为 Cutout 生成 Depth 或 Normal 文件产物 |
| `debug-*` | 保存模型适配层的直接输出 |
| `download-model`、`download-required-models` | `app.py` 调用 ModelManager 下载单个或缺失模型 |
| `remove-background` | `jobs/remove_background.py`，生成 BEN2 前景 |
| `upscale-image` | `jobs/upscale.py`，生成 2× 图像或序列 |
| `generate-depth-plane-geometry` | `jobs/depth_plane.py`，复制整图颜色并生成深度、元数据与切线法线 |
| `generate-cutout-artifacts` | `jobs/cutout.py`，按请求生成 Cutout 深度和法线 |
`generate-depth-plane-geometry` 对应 `jobs/depth_plane.py`；`generate-cutout-artifacts` 对应 `jobs/cutout.py`。共享的 MoGe-2 推理编排位于 `geometry/prediction_artifacts.py`，它把 Selection RGBA 合成到黑底后执行推理，并把单帧 `GeometryFrame` 交给 Depth、Normal 与 Metadata Writer。Depth EXR 与元数据写入位于 `geometry/depth_texture.py`，Normal 编码位于 `geometry/normal_texture.py`。Selection Mask 由 Blender 在最终图片或网格阶段组合。
`media/` 处理单图、序列、视频与输入尺寸。Depth Plane 和 Cutout 按单张静态图片工作；Remove Background 与 Upscale 逐帧处理动画输入。
## 尺寸约束
`geometry/prediction_artifacts.py` 将限尺寸后的 RGBA 合成到黑底，运行一次 MoGe-2，再调用深度、法线和元数据写入器。Cutout 的深度分支通过 `edge_depth.py` 延展轮廓附近的深度；法线使用原预测。文件字段见 [Job 产物](outputs.md)。
`server/media/resolution.py` 提供 **Maximum AI Input Size**，默认 `2048 px`。它限制 Upscale 的可提交源图长边，以及 Depth、Normal Map 的模型分析图长边；材质颜色图使用原始分辨率。
## 模型与缓存
Blender 侧的 **Maximum Cutout Mesh Resolution** 统一限制所有 Cutout Shape 实际几何范围的最长网格采样边，不限制 Selection bounds 的原始像素尺寸。
| 用途 | 模型 | Session 分类 |
| --- | --- | --- |
| Depth 与 Normal | MoGe-2 ViT-S / ViT-B / ViT-L Normal | geometry |
| Remove Background | BEN2 | ben2 |
| Upscale | Real-ESRGAN WDN、SAFMN、Real-ESRGAN x4plus、PiSA-SR | upscale |
`generate-depth-plane-geometry` 与 `generate-cutout-artifacts` 只接受一帧，因此 Depth Plane 与 Cutout 是静态图工作流。Remove Background 与 Upscale 继续通过媒体入口处理单图、视频和序列帧。
## 模型
Depth Plane、Cutout Depth 与 Normal Map 使用 MoGe-2，Remove Background 使用 BEN2，Upscale 使用所选高清模型。MoGe-2 生产推理直接返回 `GeometryFrame`；DA3 调试推理保留模型原始 Prediction，并可按帧适配到相同结构。
`ModelManager` 按模型族管理下载目录与 ONNX Session，同一族只保留最近使用的模型和 Device，避免同类模型重复占用资源。
BEN2、DA3、MoGe-2 与 Upscale 共用 ONNX Runtime 推理入口。CUDA Session 使用可收缩的 memory arena；单次图片、完整帧集合或完整 Upscale tile 集合只在最后一次模型运行结束时收缩空闲 arena。Session 持有的模型权重与执行计划继续由 `ModelManager` 缓存，后续相同模型与 Device 直接复用。CPU、DirectML 与 CoreML 按各自 Provider 的普通运行方式执行。macOS arm64 默认使用 CPU；用户可选择 `coreml`，由 ONNX Runtime CoreML Execution Provider 在 Apple CPU、GPU 与 Neural Engine 之间调度，不使用 PyTorch MPS backend。MoGe-2 的动态 ONNX 图只把静态输入 shape 分区交给 CoreML，其余节点由 CPU Provider 执行；其他模型不使用该限制。
`model_catalog.py` 声明文件来源、许可、大小和 SHA-256。`model_download.py` 下载文件并校验；ModelManager 按模型路径和设备缓存各类最近使用的 Session，切换模型或设备时释放对应类别。
```mermaid
flowchart TD
Request["模型请求"] --> Cache["ModelManager 获取缓存 Session"]
Cache --> Runs["执行单次或批量模型运行"]
Runs --> Final["最后一次模型运行"]
Final --> Shrink["CUDA 收缩空闲 memory arena"]
Shrink --> Reuse["Session 留在 ModelManager 中复用"]
A[取得缓存 Session] --> B[运行单图、多帧或 tile]
B --> C[最后一次推理]
C --> D[CUDA 收缩空闲 arena]
D --> E[保留 Session 供后续复用]
E --> F[Clear Models 或关闭 Server 时释放]
```
## Environment 与 Storage Root
Windows 默认 DirectML，macOS arm64 默认 CPU 并支持选择 CoreML。MoGe-2 为 CoreML 配置静态 shape 分区，其余计算由 CPU 执行。PiSA-SR 在 CoreML 上返回明确的支持限制；其分阶段 Session 在切换阶段、取消和清理时关闭。
`runtime.py` 声明独立 Python 3.12 环境和固定依赖。首次安装会准备默认 MoGe-2、BEN2 与 Real-ESRGAN WDN 模型。Storage Root 默认是 `~/.anyimage`：
## Environment 与存储
```text
~/.anyimage/
├── .venv/
├── models/
├── jobs/
├── environment.json
├── manifest.json
├── install.log
└── server.log
```
切换 Storage Root 等同于切换整套 Environment、模型与任务数据，不自动迁移旧目录。
各 Job 的稳定产物见 [Job 产物](outputs.md)。
首次安装通过 `post_install()` 下载默认 MoGe-2、BEN2 与 Real-ESRGAN WDN。Storage Root 默认 `~/.anyimage`，其中保存 `.venv/`、`models/`、`jobs/` 及环境状态和日志。设置入口与安装流程见 [AI 环境管理](operators/ai-setup.md)。

6
mkdocs.yml
View File

@@ -24,9 +24,9 @@ nav:
- Clipboard Image: internals/operators/clipboard-image.md
- Convert to Plane: internals/operators/convert-to-plane.md
- Cutout Tool: internals/operators/cutout-tool.md
- Debug Operator: internals/operators/debug.md
- Image Tool: internals/operators/image-tool.md
- Perspective Tool: internals/operators/perspective-tool.md
- Frame: internals/operators/frame.md
- Mask: internals/operators/mask.md
- Rectify: internals/operators/rectify.md
- Remove Background: internals/operators/remove-background.md
- Upscale: internals/operators/upscale.md
- 开发维护:

9
openspec/changes/add-projected-depth-plane/design.md
View File

@@ -6,7 +6,7 @@ Cutout 的双击入口集中在 `SelectCutoutSelection.invoke()`、`full_image`
## Goals / Non-Goals
本次目标是建立独立的整图 Depth Surface 转换，并让 Cutout 通过套索提交区域。Depth Surface 的用户控制固定为 Subdivide、Thickness、Depth Scale、Reference Depth。
本次目标是建立独立的整图 Depth Surface 转换，并让 Cutout 通过套索提交区域。Depth Surface 的用户控制固定为 Subdivide、Thickness、Depth Scale、Reference Depth、Normal Smooth。
## Decisions
@@ -42,6 +42,7 @@ Depth Surface 使用 `create_mesh_plane_mesh()` 和 `build_mesh_plane_matrix()`
| Subdivide | 6，范围 0–10 | 沿用 O Mesh Plane 的细分 |
| Thickness | 0，最小 0 | 均匀曲面壳厚度，DISTANCE |
| Depth Scale | 1，最小 0 | 平面到相机曲面的缩放，可大于 1 |
| Normal Smooth | 50，范围 0–50 | 厚度方向的法线平滑次数，放在 Options |
| Reference Depth | Metadata reference_depth × Uniform Scale | 局部深度基准，DISTANCE，放在 Options |
Geometry 为结构输入；Depth Image 与 Uniform Scale 为隐藏数据输入。所有修改器输入使用 SINGLE。
@@ -56,16 +57,16 @@ flowchart TD
Texture[VECTOR Depth × Uniform Scale] --> Projection
Projection --> Thickness{Thickness}
Thickness -- 0 --> Surface[单层曲面]
Thickness -- 大于 0 --> Shell[沿曲面法线生成均匀壳并连接外边界]
Thickness -- 大于 0 --> Shell[沿平滑法线向内生成均匀壳并连接外边界]
Surface --> Output[保留 UV 与材质，清理临时属性]
Shell --> Output
```
提取现有曲面分支中可复用的投影、均匀厚度节点构建函数，按业务职责集中在共用构建模块；新组只组合自身需要的步骤。零厚度保持单层矩形拓扑，正厚度保留投影正面、沿内侧扩展背面并连接四周。厚度方向和法线处理与现有 Uniform Depth Surface 一致。
提取现有曲面分支中可复用的投影、均匀厚度节点构建函数，按业务职责集中在共用构建模块；新组只组合自身需要的步骤。零厚度保持单层矩形拓扑，正厚度保留投影正面、沿内侧扩展背面并连接四周。厚度采用正面 POINT 法线，按 Normal Smooth 迭代次数进行邻域平滑后归一化，使对应正背面顶点之间的距离等于 Thickness。
### 验证边界
以合成相机 XYZ 纹理验证投影和坐标方向，用相同 Subdivide 的普通 Plane 对照基础拓扑与 UV；使用非方图、偏移 Image Empty、旋转缩放矩阵验证定位。检查四项可见参数、隐藏数据输入、SINGLE、材质槽和临时属性清理。
以合成相机 XYZ 纹理验证投影和坐标方向，用相同 Subdivide 的普通 Plane 对照基础拓扑与 UV；使用非方图、偏移 Image Empty、旋转缩放矩阵验证定位。检查五项可见参数、隐藏数据输入、SINGLE、材质槽和临时属性清理。
节点布局按实际 Blender 绘制尺寸检查总览和局部，几何主链连续、计算支线就近接入。实现阶段按项目总则只运行测试；资产构建、资产视觉验收和节点资产文档同步作为后续发布准备，当前提案不执行构建。

10
openspec/changes/add-projected-depth-plane/specs/projected-depth-surface/spec.md
View File

@@ -32,12 +32,12 @@ Depth Surface SHALL 使用普通 Convert to Plane 的基础 Mesh 创建、中心
- **WHEN** 图片包含透明孔洞或纹理深度跳变
- **THEN** 基础网格仍为完整矩形细分，Alpha 由材质表达
### Requirement: Surface controls are limited to four parameters
`O Image Depth Surface` SHALL 只公开 Subdivide、Thickness、Depth Scale、Reference Depth 四项用户控制。默认值 SHALL 分别为 6、0、1 和 Metadata reference_depth 乘以 Uniform Scale。Geometry SHALL 为结构输入，Depth Image 与 Uniform Scale SHALL 为隐藏数据输入。输入 MUST 使用 SINGLE；Thickness、Reference Depth MUST 使用 DISTANCE。
### Requirement: Surface controls include normal smoothing
`O Image Depth Surface` SHALL 只公开 Subdivide、Thickness、Depth Scale、Reference Depth 以及 Normal Smooth 五项用户控制。默认值 SHALL 分别为 6、0、1 和 Metadata reference_depth 乘以 Uniform Scale。Geometry SHALL 为结构输入，Depth Image 与 Uniform Scale SHALL 为隐藏数据输入。输入 MUST 使用 SINGLE；Thickness、Reference Depth MUST 使用 DISTANCE。
#### Scenario: Inspect a new modifier
- **WHEN** 创建 Depth Surface 修改器
- **THEN** 用户看到规定的四项控制，Reference Depth 位于 Options
- **THEN** 用户看到规定的五项控制，Reference Depth 与 Normal Smooth 位于 Options；Normal Smooth 默认 50，范围 0–50
- **AND** 不出现 Cutout 的 Mode、Split、Balloon、Inflation、轮廓或轴向控制
### Requirement: Geometry follows camera projection
@@ -56,7 +56,7 @@ Depth Surface SHALL 按现有 Depth Cutout 的相机 XYZ 坐标约定投影。
- **THEN** 曲面 Z 增加 d × s，投影 XY 保持不变
### Requirement: Thickness forms a uniform surface shell
Thickness 为 0 时系统 SHALL 输出单层曲面；Thickness 大于 0 时 SHALL 保留投影正面，按现有 Uniform Depth Surface 的法线厚度语义向内生成背面并连接矩形边界。
Thickness 为 0 时系统 SHALL 输出单层曲面；Thickness 大于 0 时 SHALL 保留投影正面，按正面 POINT 法线按 Normal Smooth 进行邻域平滑并归一化后的方向向内生成背面并连接矩形边界。
#### Scenario: Zero thickness
- **WHEN** Thickness 为 0
@@ -64,7 +64,7 @@ Thickness 为 0 时系统 SHALL 输出单层曲面；Thickness 大于 0 时 SHAL
#### Scenario: Positive thickness
- **WHEN** 对平坦深度输入设置正 Thickness
- **THEN** 正背面沿法线距离等于 Thickness，边界闭合且法线朝向一致
- **THEN** 对应正背面顶点之间的距离等于 Thickness，边界闭合且法线朝向一致
### Requirement: One prediction produces complete image artifacts
整图 Depth Surface SHALL 通过共用 Artifact 层执行一次 MoGe-2 推理，生成 VECTOR Depth、Metadata 与 Tangent Normal，并保留完整 Color。Blender SHALL 在主线程加载、Pack 并设置图片色彩空间，将 Normal 用于材质，将 VECTOR Depth 用于几何。

33
openspec/changes/add-projected-depth-plane/tasks.md
View File

@@ -1,24 +1,31 @@
## 1. 删除 Cutout 双击整图入口
- [ ] 1.1 删除 Cutout 的 DOUBLE_CLICK 整图绑定、full_image Property、整图 Selection 分支及失去用途的 import，保留共享套索交互。
- [ ] 1.2 调整交互和注册测试，验证图片内外双击均不打开 Shape 菜单或提交任务，有效套索继续生成 Shape。
- [x] 1.1 删除 Cutout 的 DOUBLE_CLICK 整图绑定、full_image Property、整图 Selection 分支及失去用途的 import，保留共享套索交互。
- [x] 1.2 调整交互和注册测试，验证图片内外双击均不打开 Shape 菜单或提交任务，有效套索继续生成 Shape。
## 2. 新增整图 Depth Surface 转换
- [ ] 2.1 增加 ConvertToDepthSurface、菜单入口和正逆序注册，验证静态图片、AI 环境及忙碌状态检查。
- [ ] 2.2 扩展共用整图 Job 的 Z/VECTOR 产物选择及 Blender 响应分派，保持普通 Depth Plane 的 Z 路径；验证 VECTOR Depth 和 Tangent Normal 来自一次推理。
- [ ] 2.3 复用 common 的图片、深度、材质及对象收尾函数创建 Depth Surface；共用 Plane 的基础 Mesh 和中心矩阵，初始化四项参数及隐藏深度数据。
- [ ] 2.4 验证完整颜色、材质槽、深度与法线 Pack、源对象替换、失败清理和单步 Undo。
- [x] 2.1 增加 ConvertToDepthSurface、菜单入口和正逆序注册，验证静态图片、AI 环境及忙碌状态检查。
- [x] 2.2 扩展共用整图 Job 的 Z/VECTOR 产物选择及 Blender 响应分派，保持普通 Depth Plane 的 Z 路径；验证 VECTOR Depth 和 Tangent Normal 来自一次推理。
- [x] 2.3 复用 common 的图片、深度、材质及对象收尾函数创建 Depth Surface；共用 Plane 的基础 Mesh 和中心矩阵，初始化四项参数及隐藏深度数据。
- [x] 2.4 验证完整颜色、材质槽、深度与法线 Pack、源对象替换、失败清理和单步 Undo。
## 3. 构建定义与几何验证
- [ ] 3.1 按业务职责提取共用相机投影和均匀厚度构建步骤，新增 O Image Depth Surface 构建定义，并接入节点组构建和验证索引。
- [ ] 3.2 以 O Mesh Plane 的零厚度输出生成完整矩形基础网格，设置 Subdivide、Thickness、Depth Scale、Reference Depth 以及隐藏数据接口，检查 SINGLE 与 socket subtype。
- [ ] 3.3 增加合成纹理几何测试：同级 Plane 拓扑与 UV 对照、Depth Scale 0/0.5/1/>1、Reference Depth、相机 Y 方向和非中心主点。
- [ ] 3.4 增加零厚度单层、正厚度闭合、透明区域完整矩形、深度跳变连接、材质和临时属性清理验证。
- [ ] 3.5 编排新节点组布局与验证入口，覆盖未用输出隐藏、主链及支线间距；记录发布准备时所需的实际绘制尺寸视觉验收项目。
- [x] 3.1 按业务职责提取共用相机投影和均匀厚度构建步骤，新增 O Image Depth Surface 构建定义，并接入节点组构建和验证索引。
- [x] 3.2 以 O Mesh Plane 的零厚度输出生成完整矩形基础网格，设置 Subdivide、Thickness、Depth Scale、Reference Depth 以及隐藏数据接口，检查 SINGLE 与 socket subtype。
- [x] 3.3 增加合成纹理几何测试：同级 Plane 拓扑与 UV 对照、Depth Scale 0/0.5/1/>1、Reference Depth、相机 Y 方向和非中心主点。
- [x] 3.4 增加零厚度单层、正厚度闭合、透明区域完整矩形、深度跳变连接、材质和临时属性清理验证。
- [x] 3.5 编排新节点组布局与验证入口，覆盖未用输出隐藏、主链及支线间距；记录发布准备时所需的实际绘制尺寸视觉验收项目。
## 4. 回归与交付检查
- [ ] 4.1 运行相关转换、交互、注册、Server Job 和节点构建定义测试，修复回归；检查现有 Depth Plane 和圈选 Cutout 的行为。
- [ ] 4.2 运行 uv run pytest，记录测试结果及需要 Blender 或已构建新资产才能完成的验收项。按项目总则，代码实施只运行测试，资产构建、二进制产物和节点资产文档同步留给发布准备。
- [x] 4.1 运行相关转换、交互、注册、Server Job 和节点构建定义测试，修复回归；检查现有 Depth Plane 和圈选 Cutout 的行为。
- [x] 4.2 运行 uv run pytest，记录测试结果及需要 Blender 或已构建新资产才能完成的验收项。按项目总则，代码实施只运行测试，资产构建、二进制产物和节点资产文档同步留给发布准备。
发布准备验收：构建节点资产后，在 Blender 节点编辑器按实际绘制尺寸检查新组总览、投影支线、厚度分支及旁路；检查修改器四项参数的实际显示，并对已发布资产运行几何验证。本次不生成资产或文档产物。
测试记录：全量测试两次均为 723 passed、2 xfailed、62 subtests passed；显式调用 pytest.main 返回 0。两次完整进程均在 pytest 返回后以退出码 1 结束，退出阶段异常尚未定位。新曲面几何、接口、对象定位及失败清理测试全部通过。OpenSpec strict 校验通过。
资产更新记录：已按用户要求运行 build_node --skip-tests，Blender 4.5.10 LTS 构建及三项资产验证均成功。更新后资产与转换回归 53 passed；构建定义和磁盘发布资产使用相同几何用例验证，31 passed，命令退出码均为 0。已同步节点资产说明。Computer Use 应用授权超时，实际节点编辑器视觉验收仍待完成。

2
openspec/changes/remove-debug-workflows/.openspec.yaml Normal file
View File

@@ -0,0 +1,2 @@
schema: spec-driven
created: 2026-09-08

69
openspec/changes/remove-debug-workflows/design.md Normal file
View File

@@ -0,0 +1,69 @@
## Context
Debug 调用链跨越 Blender UI、Scene 参数、Operator、Server Job 和原始预测写出。`server/outputs/` 当前专用于 Debug；正式产物由 geometry 等模块生成。删除依据 `proposal.md` 的完整工作流边界进行，同时全面审查 tests、node_assets 和内部文档。
## Goals / Non-Goals
目标是清除 Debug 入口及其专用依赖，精简有效测试，使节点资产模块和内部文档的名称对应实际职责。实施范围包括代码、测试和用户明确要求整理的内部文档；节点图行为与接口保持稳定，按用户补充要求将 `O Mesh Plane` 统一为 `O Image Plane`。
## Decisions
### 按调用链删除
```mermaid
flowchart TD
A[删除 Debug 主面板与四个子面板] --> B[删除四个 Debug Operator 与注册]
B --> C[删除四个 Debug Job 与注册]
C --> D[删除 Debug 序列化与帧复制]
D --> E[按剩余引用清理属性与辅助函数]
E --> F[验证正式功能与注册生命周期]
```
直接删除实现与注册，使源码和可调用接口保持一致。清理 `server/outputs/` 整个专用包、`copy_frames` 实现和 `media/__init__.py` 导出，避免留下失去调用方的模块。
### 共用代码按引用决定归属
| 对象 | 处理 |
| --- | --- |
| `DebugPanel` 与四个模型子面板、`draw_da3_input_settings` | 删除并清理 `panel.py` 导入 |
| `DebugDa3`、`DebugMoge2`、`DebugBen2`、`DebugUpscale` | 删除模块及所有注册项 |
| Scene 的 `input_path`、`background_input_path`、`model`、`device`、`process_res`、`process_res_method`、`moge_resolution_level`、`moge_model`、`ref_view_strategy` | 核对剩余调用后删除 Debug 专用声明 |
| Scene 的 `video_frames`、Cutout 和 Mask 参数 | 保留正式调用需要的属性 |
| `da3_parameters`、`model_enum_items`、`moge2_model_enum_items` | 核对引用后删除专用函数及孤立导入 |
| `moge2_parameters`、输入校验函数、偏好设置中的设备选择 | 保留正式调用使用的实现 |
| DA3 模型适配器、ONNX 推理、目录及缓存 | 当前调用方仅为 Debug，整条删除 |
| MoGe-2、BEN2、Upscale 模型适配器及模型目录 | 保留正式推理所需能力，Upscale 统一保留源 Alpha，删除专用开关 |
属性按所属类型检查；Operator 上同名的正式属性独立保留。MoGe-2 原始预测结构仍用于正式几何转换，保留其适配和数据校验。
### 测试随接口收缩
删除专用导出与 Debug 推理测试，将原本仅通过 Debug 覆盖的公共模型行为转到正式入口。更新 Blender 注册及逆序注销、Server Job 清单与打包清单测试，明确 Debug 接口和文件缺席；正式流程继续覆盖设备、模型选择、视频多帧及推理后的内存释放。
### 全面审查测试价值与归属
先逐文件记录实际验证的行为，再对每个用例决定保留、合并、重写或删除。保留能检测输出错误、协议破坏、数据丢失、取消与资源清理失败等实际问题的测试。删除失效功能测试、相同风险的重复断言，以及只复述实现细节且无法发现行为回归的测试；mock 和源码检查按其验证价值判断，不机械删去。
文件按稳定功能组织，类名和用例名描述实际行为及关键条件。已发现 `test_server_jobs.py` 混合媒体输入、下载、模型生命周期与 Job 输出，`test_blender_addon.py` 混合设备、运行时和 Operator 行为，应按职责拆分。`test_docs.py` 实际验证文档命令，应命名为对应命令职责。其他文件同样逐项审查，避免只处理这几个示例。共享 fixture 仅抽取确有复用的部分，测试通过情况不能代替覆盖价值判断。
### 节点资产按功能组织源码
梳理全部模块的构建对象、几何处理、布局和验证职责，形成现名到职责的映射，再确定重命名、合并或拆分。`build_depth_plane.py`、`build_cutout_modifiers.py` 等动作式文件名改为资产或业务分类命名，函数继续使用 `build_*` 表达构建动作。公共深度表面与布局模块依据实际调用范围命名，`node_graph.py` 和 `node_layout.py` 依据实际公共能力保留或细化。
一次更新模块导入、`tools/build_node.py` 的脚本列表、测试动态加载路径与文档路径，删除旧入口。将 `O Mesh Plane`、相关模块及函数统一命名为 `O Image Plane` / `image_plane`。使用项目配置的 Blender 修改现有 `.blend` 资产名称，比较保存前后全部节点组的接口、默认值、运算、连接和布局，验证功能一致。其他节点组名称保持稳定。
### 内部文档按最终实现重整
逐页核对总览、架构、Common、Runtime、产物、开发、节点资产和全部 Operator 说明。以当前源码中的注册、调用链、参数和产物协议为依据，合并重复说明，删除过期内容，按实际主题调整文件名与层次，并同步所有内部链接与 MkDocs 导航。
删除 Debug 专页及引用；节点资产页使用最终模块名及职责。实现流程用从上到下的 Mermaid 图，正文以中文正面说明当前职责。通过路径、导航、链接和源码交叉核对验收，不运行文档构建。
## Risks / Trade-offs
- [误删同名正式参数或公共模型函数] → 逐项核查调用方，并运行正式工作流回归测试。
- [注册表或包导出残留导致启动失败] → 验证扩展导入、注册与逆序注销、Server Job 清单和打包文件选择。
- [并行变更修改相同文件] → 实施时基于最新工作区逐项修改，保留其他变更的逻辑与测试。
- [旧脚本调用已删除 Debug 接口] → 调用方改用对应正式 Operator 或 Job；删除直接生效，不保留转发接口。
- [精简测试掩盖回归] → 按行为和风险核对精简前后覆盖，保留独有场景，最终运行完整测试。
- [节点模块重命名遗漏动态路径] → 核查构建启动器、脚本加载、测试和文档引用，并测试入口解析。
- [文档沿用旧描述导致再次偏离] → 每页对照最终源码核验，尤其是节点组名、socket、Job 参数和输出协议。

35
openspec/changes/remove-debug-workflows/proposal.md Normal file
View File

@@ -0,0 +1,35 @@
## Why
模型调试入口已完成阶段性用途，继续保留会增加 UI、参数、Job 和测试的维护成本。结合删除工作，全面整理测试、节点资产源码和内部文档，让名称、职责、验证与实际功能保持一致。
## What Changes
- **BREAKING**：移除 Debug 主面板、DA3 / MoGe-2 / BEN2 / Upscale 子面板及四个 `anyimage.debug_*` Operator。
- **BREAKING**：移除 `debug-da3`、`debug-moge2`、`debug-ben2`、`debug-upscale` Job 和原始预测导出实现。
- 删除 Debug 专用 Scene 属性、UI 辅助函数、参数组装、序列化与帧复制代码，清理注册、导入和导出。
- 删除当前仅由 Debug 调用的 DA3 模型适配器、ONNX 推理、目录声明及缓存分支；Upscale 统一保留源 Alpha。
- 根据正式功能的实际引用保留共用模型、输入处理、设备设置、视频帧数和推理能力。
- 调整注册、Job、模型及打包测试，覆盖删除后的边界和正式功能回归。
- 全面审查 `tests/` 的文件、测试类、用例和辅助代码，仅保留能发现实际行为回归或约束关键协议的测试，按验证功能整理名称和归属。
- 全面梳理 `tools/node_assets/` 的构建、几何、布局及验证职责，重命名或拆并职责不符的模块，同步所有导入与脚本引用。
- 全面重整 `docs/internals/`，逐页对照最终源码与功能逻辑修订内容、文件名和索引，清理 Debug 说明并同步 `mkdocs.yml` 导航。
## Capabilities
### New Capabilities
- `production-only-ai-workflows`：约束扩展仅暴露正式 AI 工作流，规定 Debug 入口及专用实现的移除和正式功能的可用性。
- `maintainable-project-structure`：规定测试价值与命名、节点资产模块职责和内部文档与实现的一致性。
### Modified Capabilities
无。当前 `openspec/specs/` 尚无已归档规范。
## Impact
- Blender：`panel.py`、`properties.py`、扩展与 Operator 注册、`operators/debug.py`、`common/ai.py`。
- Server：`app.py`、`jobs/debug_predictions.py`、`outputs/`、`media/` 及仅被 Debug 引用的模型适配辅助逻辑。
- 测试：整个 `tests/`，包含测试归属、名称、公共 fixture 和重复覆盖的整理。
- 节点资产：整个 `tools/node_assets/`，以及 `tools/build_node.py`、相关测试和调用引用。
- 文档：整个 `docs/internals/` 及 `mkdocs.yml` 导航；按用户明确要求同步实现说明。
- 本次整理运行测试，不构建文档、扩展包或节点资产；节点资产整理保持现有节点图与资产接口行为。

37
openspec/changes/remove-debug-workflows/specs/maintainable-project-structure/spec.md Normal file
View File

@@ -0,0 +1,37 @@
## ADDED Requirements
### Requirement: Tests protect meaningful behavior
维护者 MUST 审查全部测试文件、用例和辅助代码，按实际回归风险保留、合并、重写或删除。保留的测试 SHALL 验证可观察行为或关键协议，每项独有且仍适用的风险覆盖 MUST 保留。
#### Scenario: Audit completed
- **WHEN** 全部测试整理完成
- **THEN** 失效功能测试和无独立价值的重复测试已删除，保留用例能说明所保护的行为，完整测试通过
### Requirement: Test names reflect tested functionality
测试文件、类和函数 MUST 与其验证的功能、行为和关键条件相符；跨独立职责的测试集合 SHALL 按稳定功能拆分。
#### Scenario: Locate a behavior test
- **WHEN** 按测试路径、类名和函数名查找某项功能
- **THEN** 名称与测试主体的实际断言对应，共用辅助代码拥有明确的复用职责
### Requirement: Node asset modules reflect responsibilities
`tools/node_assets/` 的全部模块 MUST 按实际资产或业务职责审查命名与边界，所有重命名 MUST 同步导入、构建与验证入口及测试路径，并删除旧路径和转发层。
#### Scenario: Load reorganized asset tools
- **WHEN** 启动器解析脚本并加载节点构建与验证模块
- **THEN** 所有路径指向最终模块，文件名与构建对象、几何处理、布局或验证职责对应
#### Scenario: Preserve node behavior
- **WHEN** 对重组后的节点构建函数运行针对性测试
- **THEN** 普通图像平面统一使用 `O Image Plane` 名称，源码使用 `image_plane`；资产重命名前后接口、默认值、运算、连接及布局一致
### Requirement: Internal documentation matches implementation
`docs/internals/` 全部页面 MUST 与最终源码中的功能、职责、参数、调用链和产物对应，页面名称和目录索引 MUST 与主题对应。实现流程 SHALL 使用从上到下的 Mermaid 图。
#### Scenario: Read final implementation documentation
- **WHEN** 按总览或 MkDocs 导航访问任一内部主题
- **THEN** 链接有效，内容描述当前实现，源码路径和功能名称可核对，Debug 专页和过期 Debug 说明已清理

45
openspec/changes/remove-debug-workflows/specs/production-only-ai-workflows/spec.md Normal file
View File

@@ -0,0 +1,45 @@
## ADDED Requirements
### Requirement: Blender exposes production AI workflows
扩展 MUST 移除 Debug 主面板、DA3 / MoGe-2 / BEN2 / Upscale 子面板及四个 `anyimage.debug_*` Operator，并保持正式类型注册和逆序注销完整。
#### Scenario: Extension registration lifecycle
- **WHEN** 扩展注册、注销并重新注册
- **THEN** Debug 面板和 Operator 均不在注册清单中，正式面板与 Operator 正常可用
### Requirement: Server registers production jobs only
Server MUST 移除 `debug-da3`、`debug-moge2`、`debug-ben2` 和 `debug-upscale` Job 及其处理实现，不提供兼容入口。
#### Scenario: Registered jobs
- **WHEN** Server 加载 Job 注册清单
- **THEN** 四个 Debug Job 均缺席，正式 Job 保持已注册且可执行
### Requirement: Debug implementation is removed completely
源码和扩展文件选择 MUST 移除 Debug 专用模块、Scene 参数、UI 辅助函数、参数组装、原始预测序列化、帧复制及相应导入和导出。
#### Scenario: Source and packaging boundaries
- **WHEN** 检查模块引用与扩展打包文件清单
- **THEN** `operators/debug.py`、`server/jobs/debug_predictions.py`、`server/outputs/` 和 Debug 专用辅助实现均缺席，保留模块可正常导入
#### Scenario: Scene settings
- **WHEN** 读取 `AnyImageSettings` 的属性声明
- **THEN** Debug 专用属性已删除，正式使用的 `video_frames`、Cutout 和 Mask 属性保持可用
#### Scenario: Debug-only model dependencies
- **WHEN** 加载模型目录与缓存 Resource
- **THEN** DA3 模型声明、推理模块与缓存分支均已移除，正式 MoGe-2、BEN2 和 Upscale 模型保持可用
### Requirement: Shared production inference remains available
系统 SHALL 保持正式深度处理、Cutout、Remove Background 和 Upscale 所需的模型、输入校验、设备选择、多帧处理及内存释放行为。
#### Scenario: Production model requests
- **WHEN** 正式 Operator 组装并提交 AI 请求
- **THEN** 请求使用正式模型参数及 Add-on Preferences 设备设置，输入校验和视频帧数传递保持有效
#### Scenario: Production output and memory lifecycle
- **WHEN** 正式 Job 完成单帧或多帧模型推理
- **THEN** 生成对应正式产物，并按既有最终推理边界请求内存释放

53
openspec/changes/remove-debug-workflows/tasks.md Normal file
View File

@@ -0,0 +1,53 @@
## 1. Blender 入口与设置
- [x] 1.1 删除 Debug 主面板、四个子面板和专用绘制函数，清理 `panel.py` 导入及扩展注册项。
- [x] 1.2 删除 `operators/debug.py` 及 Operator 注册项，验证正式类型注册与逆序注销。
- [x] 1.3 按引用删除 Debug 专用 Scene 属性、枚举回调和 `da3_parameters`，保留正式输入校验、`moge2_parameters`、`video_frames` 及偏好设备设置。
## 2. Server 实现与依赖
- [x] 2.1 删除四个 Debug Job 的注册和 `jobs/debug_predictions.py`。
- [x] 2.2 删除 `server/outputs/` 专用包、`copy_frames` 及媒体模块导出，清理孤立导入。
- [x] 2.3 沿删除后的调用链检查模型适配与公共模块，移除仅服务 Debug 的分支，确认正式预测数据转换和产物写出仍完整。
## 3. 测试与验收
- [x] 3.1 删除 Debug 专用测试及测试模块加载，更新 Blender 注册与 Scene 属性断言。
- [x] 3.2 更新 Server Job 注册和打包文件清单测试，验证 Debug Job、模块及导出缺席；使用文件选择检查，避免生成扩展包。
- [x] 3.3 将 Debug 测试中仍适用的公共模型断言移到正式入口，验证模型选择、设备、输入校验、多帧、产物及最终推理内存释放。
- [x] 3.4 运行 Debug 删除相关测试，复查 `src`、`tests`、`tools` 及运行配置中的 Debug 引用，确认无残留实现或失效导入。
## 4. 全部测试整理
- [x] 4.1 逐文件审查全部测试类、用例和辅助代码，列出保护的行为与风险，决定保留、合并、重写或删除。
- [x] 4.2 删除失效、无独立价值的重复用例和单纯复述实现的断言，保留所有仍适用的独有行为覆盖。
- [x] 4.3 按实际职责整理全部测试文件、类和函数名，拆分跨功能集合，包括 Blender 注册、设备、Runtime、媒体输入、模型下载与 Job 输出。
- [x] 4.4 清理 fixture、动态导入和辅助函数，仅提取确有复用的代码，核对整理前后关键风险覆盖。
## 5. 节点资产源码整理
- [x] 5.1 遍历 `tools/node_assets/` 全部模块，核对文件名与构建对象、几何处理、布局及验证职责，确定目标命名和拆并方案。
- [x] 5.2 按稳定功能重命名或拆并模块，同步全部导入、`tools/build_node.py` 脚本列表和测试加载路径，移除旧路径与转发代码。
- [x] 5.3 通过针对性测试核对 socket 顺序、默认值、运算、连接和布局保持一致，将源码、测试、文档和正式资产中的 `O Mesh Plane` 统一为 `O Image Plane`。
## 6. 内部文档整理
- [x] 6.1 对照最终源码审查 `docs/internals/` 全部页面，核对功能名、路径、参数、调用链、产物和实际职责。
- [x] 6.2 重整总览、架构、Common、Runtime、产物和开发说明，按主题调整文件名与层次，合并重复内容并删除过期说明。
- [x] 6.3 重整全部 Operator 页面和索引，删除 Debug 专页与引用，以 Mermaid 流程说明正式功能主链。
- [x] 6.4 更新节点资产文档，使模块名、节点组、接口、构建与验证流程对应实际实现。
- [x] 6.5 同步 `mkdocs.yml` 导航和全部受影响的内部链接，检查链接与源码引用有效性。
## 7. 最终验收
- [x] 7.1 运行 `uv run pytest`，处理行为回归与测试收集问题，核对关键功能覆盖。
- [x] 7.2 复查全部改名的旧路径、失效导入和文档引用，确认 tests、node_assets、docs/internals 的名称与内容边界一致。
- [x] 7.3 核查最终差异并汇报主要删除、合并、重命名与验证结果；确认未构建文档、扩展包或重建节点资产；已有资产仅修改图像平面组名并验证内容一致。
## 验证结果
- `uv run pytest -q`：724 passed、2 xfailed、48 subtests passed，退出码 0。
- 模拟 Blender 测试仅拦截 Python 导入，保留真实 `bpy` 模块身份，并恢复原有 AnyImage 模块，避免原生退出崩溃。
- 23 个节点源码模块经 AST 比对，变化仅限命名与导入。
- 使用项目配置的 Blender 对原有资产重命名；保存前后节点接口、属性、默认值、连接和布局一致。
- 内部文档链接、Python 语法和 OpenSpec 严格校验通过。

2
openspec/changes/unify-depth-plane-and-artifact/.openspec.yaml Normal file
View File

@@ -0,0 +1,2 @@
schema: spec-driven
created: 2026-09-08

75
openspec/changes/unify-depth-plane-and-artifact/design.md Normal file
View File

@@ -0,0 +1,75 @@
## Context
`build_depth_plane.py` 当前读取纹理 R 通道并生成固定底面浮雕；`build_depth_surface.py` 使用 XYZ 投影并沿平滑法线生成壳。Cutout 的 Depth Solid 请求 VECTOR，Depth Balloon 请求 Z，但 Balloon 节点和 `common/depth.py` 已读取 B/Z。Server 将两类产物写成相同规格的 RGBA float32 EXR。
本设计以当前工作区已实现的 `add-projected-depth-plane` 为基线，覆盖其中关于独立 Surface 入口与 Z/VECTOR 双产物的决定。
## Goals / Non-Goals
统一整图深度几何的交互入口和深度产物契约，保留两种形态及 Cutout 的现有几何效果。删除范围限定于独立 Z 文件输出及失去用途的分支；内部标量深度、标定元数据、Cutout 的形状参数继续承担原有职责。
## Decisions
### 一个入口与模式接口
保留 `ConvertToDepthPlane`、`anyimage.convert_to_depth_plane`、Convert to Depth Plane 和 `O Image Depth Plane`。删除独立 Surface 注册、菜单、类、结果类型映射和公开资产组。模式在修改器中选择，默认 Relief；模式切换只触发节点求值。
| 接口顺序 | 默认值与语义 |
| --- | --- |
| Geometry | 结构输入 |
| Mode | Relief / Camera，默认 Relief |
| Subdivide | 6，范围 0–10 |
| Thickness（Relief） | 0，最小 0，DISTANCE |
| Thickness（Camera） | 0，最小 0，DISTANCE |
| Depth Scale | 1，最小 0 |
| Options / Reference Depth | 元数据 reference_depth × Uniform Scale，DISTANCE |
| Options / Normal Smooth | 50，范围 0–50，仅影响 Camera 厚度方向 |
| Data / Uniform Scale | 隐藏，默认 1 |
| Data / Depth Image | 隐藏 |
输入统一 SINGLE。两个独立输入均显示 Thickness，依次控制 Relief 的基础厚度和 Camera 的向内壳厚度，默认均为 0 m；按不同 socket identifier 保存，切换模式保留各自数值。Normal Smooth 保留在 Options，并通过说明明确适用模式。
### 两条几何分支共用数据
```mermaid
flowchart TD
Input[图片矩形 Mesh 与 depth.exr] --> Mode{Mode}
Mode --> Relief[Relief：固定 XY，读取 B 深度，固定底面]
Mode --> Camera[Camera：相机 XYZ 投影，沿平滑法线生成壳]
Relief --> Output[保留 UV 材质与用户属性，清理临时属性]
Camera --> Output
```
Relief 沿用现有厚度、底面权重、侧面插值和负位移限制，只将标量采样改为 B/Z。Camera 沿用现有完整矩形细分与投影公式：令采样相机 XYZ 乘以 Uniform Scale 得到 C，局部 XY 从基础 XY 按 Depth Scale 插值到 `(C.x, -C.y)`，局部 Z 为 `(Reference Depth - C.z) × Depth Scale`。正 Thickness 保留正面，背面沿正面 POINT 法线按 Normal Smooth 迭代平滑并归一化后向内偏移。
共用投影和壳构建函数，两个分支由一个公开节点组组织。构建模块以职责命名；删除仅服务旧公开 Surface 组的构建和布局入口。节点资产始终按名称加载。
### 统一深度产物与请求
```mermaid
flowchart TD
Request[整图转换或 Cutout 深度请求] --> Infer[一次 MoGe 推理，保留 points]
Infer --> Correct[按现有调用条件修正边缘深度]
Correct --> Artifact[depth.exr：RGB 相机 XYZ，A 有效性]
Correct --> Metadata[depth.json：尺寸 内参 基准深度]
Artifact --> Load[结果键 depth，主线程加载并 Pack]
Load --> XYZ[Camera 与 Depth Cutout：读取 XYZ]
Load --> Z[Relief 与 Depth Balloon：读取 B/Z]
```
将深度产物选择收敛为布尔 `generate_depth`，默认 false；整图 Job 固定请求 true，Cutout 根据 Depth Solid / Depth Balloon 判断。替换 `depth_mode` 参数与 `depth_mode_for_shape`，生成法线的参数保持独立。只生成法线时仍可不保留 points。
写出函数统一为 `write_depth_texture`，沿用当前 vector 写出的 RGBA float32 数值、有效性和非有限值处理；文件名 `depth.exr`，返回键 `depth`。删除 Z 写出函数与旧结果键，直接更新全部调用和测试。所有需要深度的推理保留 XYZ，内部 `GeometryFrame.depth` 继续用于标量运算；元数据仍通过 `depth_metadata` 返回 `depth.json`。深度与法线继续来自同一次预测，保持各自现有边缘修正语义。
### 交付与验证
客户端、Server、节点构建及测试作为一次契约变更交付，不增加旧键、旧名称转发。已有 blend 内嵌旧组不主动迁移；需要新模式接口时重新转换源图片。归档时协调前序未归档规格，确保最终规格只保留统一入口和产物契约。
实现阶段更新脚本、验证、测试与节点资产文档并运行测试。资产更新阶段构建并验证 `O_AnyImage.blend`，对定义与保存资产使用同组几何用例，最后在 Blender 检查总览和局部实际绘制布局；视觉验收单独记录，不能由坐标检查代替。
## Risks / Trade-offs
- R 与 Z 在旧标量 EXR 中相同，普通灰度测试会掩盖读错通道 → 使用 X、Y、Z 明显不同的合成纹理覆盖全部消费者。
- Balloon 从仅保留 depth 改为保留 points，增加中间内存 → 验证单次推理、释放路径和正常生成；文件格式与通道数保持一致。
- 两种模式中 Thickness 的几何含义不同 → 明确接口说明，并测试切换保留值及两种模式各自的厚度行为。
- 新代码与旧节点资产混用可能造成接口错误 → 资产更新时同时检查菜单、注册、公开组清单及实际资产求值。

26
openspec/changes/unify-depth-plane-and-artifact/proposal.md Normal file
View File

@@ -0,0 +1,26 @@
## Why
整图 Depth Plane 与 Depth Surface 共用图片、网格和深度预测，只在几何模式上不同，独立入口使选择与切换复杂。相机 XYZ 已包含标量 Z 深度，统一产物可以消除生成和读取中的格式分支。
## What Changes
- 合并为一个 Convert to Depth Plane 入口与 `O Image Depth Plane` 节点组，提供 Relief（固定底面浮雕）和 Camera（相机投影曲面）模式，默认 Relief；切换模式复用已加载深度。
- Camera 保留沿平滑法线的 Thickness 与 Normal Smooth（默认 50，范围 0–50），整图保持连续矩形网格。
- **BREAKING**：删除独立 Depth Surface Operator、菜单项和公开资产组，直接使用合并接口。
- **BREAKING**：删除 `z-depth.exr` 输出，将 `vector-depth.exr` 更名为 `depth.exr`，结果键统一为 `depth`。RGB 为相机 XYZ，Alpha 为有效性。
- 所有整图和 Cutout 深度消费者统一读取该产物；标量深度读取 B/Z 通道。内部 `GeometryFrame.depth` 与 `depth.json` 继续承担计算和标定职责。
## Capabilities
### New Capabilities
- `unified-depth-plane`: 一个整图转换入口与节点组中的 Relief / Camera 模式、接口及几何行为。
- `depth-artifact-contract`: 统一深度文件、结果键、通道约定及全部消费者的读取契约。
### Modified Capabilities
无。当前 `openspec/specs/` 为空；本变更承接已实现但未归档的 `add-projected-depth-plane`，以本次契约定义合并后的行为。
## Impact
涉及整图转换 Operator、菜单与注册、对象创建与结果加载、Cutout 请求与响应、Server Job 与预测产物层、深度纹理写出、节点构建与验证及相关测试。节点资产文档随节点接口同步；本次只创建提案，资产构建和 Blender 视觉验收在实现后的资产更新阶段执行。无需新增依赖。

38
openspec/changes/unify-depth-plane-and-artifact/specs/depth-artifact-contract/spec.md Normal file
View File

@@ -0,0 +1,38 @@
## ADDED Requirements
### Requirement: Depth has one file contract
系统 SHALL 将所有深度图片写为 `depth.exr`，通过结果键 `depth` 返回。文件 MUST 使用 RGBA float32，RGB 为相机 XYZ，Alpha 为有效性，并沿用相机坐标、像素方向和非有限值处理约定。内部标量深度和 `depth.json` SHALL 保留。
#### Scenario: Generate depth artifacts
- **WHEN** 整图转换、Depth Solid 或 Depth Balloon 请求深度
- **THEN** 结果包含 `depth` 与 `depth_metadata`，对应 `depth.exr` 和 `depth.json`
- **AND** 不生成 `z-depth.exr`、`vector-depth.exr` 或旧结果键
#### Scenario: Preserve independent channels
- **WHEN** 输出合成数据中 X、Y、Z 不同的深度图
- **THEN** 解码后的 RGB 分别保持 XYZ 数值和有效性，不将 Z 复制到 RGB
### Requirement: Depth generation is a boolean request
共享产物生成 SHALL 使用布尔 `generate_depth` 决定是否生成深度，法线请求保持独立；深度消费者 SHALL 请求保留 XYZ 的单次推理，并沿用现有边缘深度修正条件。
#### Scenario: Generate depth and normals together
- **WHEN** 同时请求深度和法线
- **THEN** 只运行一次预测，生成统一深度、元数据和请求的法线产物
#### Scenario: Generate normals alone
- **WHEN** generate_depth 为 false 且请求法线
- **THEN** 生成法线，不写出深度图片或深度元数据
### Requirement: All consumers use the unified channels
整图与 Cutout 结果加载器 SHALL 从 `depth` 加载深度并沿用 Pack、数据色彩空间及尺度标定。Camera 与 Depth Cutout SHALL 读取 XYZ；Relief、Depth Balloon 和公共标量深度读取函数 SHALL 读取 B/Z。
#### Scenario: Scalar depth is distinct from X
- **WHEN** X 与 Z 明显不同的纹理用于 Relief 或 Depth Balloon
- **THEN** 几何位移由 Z 决定，效果与使用相同标量 Z 的现有结果一致
#### Scenario: Projected geometry keeps camera coordinates
- **WHEN** 统一深度纹理用于 Camera 或 Depth Cutout
- **THEN** 投影位置、有效性和尺度与更名前的 XYZ 输入一致

58
openspec/changes/unify-depth-plane-and-artifact/specs/unified-depth-plane/spec.md Normal file
View File

@@ -0,0 +1,58 @@
## ADDED Requirements
### Requirement: One conversion exposes two geometry modes
系统 SHALL 仅提供 Convert to Depth Plane 整图深度入口，使用 `anyimage.convert_to_depth_plane` 与资产组 `O Image Depth Plane`。Mode SHALL 提供 Relief 与 Camera，默认 Relief。独立 Depth Surface 入口、注册类和公开资产组 SHALL 删除。
#### Scenario: Convert and switch modes
- **WHEN** 静态 Image Empty 完成转换后，用户在修改器切换 Relief / Camera
- **THEN** 同一对象和深度图片产生对应几何，不重新推理、不创建新深度图片
- **AND** 两种模式共享并保留已有控制值
#### Scenario: Preserve conversion lifecycle
- **WHEN** 转换成功或失败
- **THEN** 成功时沿用源名称、替换源对象且一次 Undo 可恢复；失败时清理部分结果并保留源对象
- **AND** AI 环境检查和静态媒体限制继续生效
### Requirement: Unified modifier has a stable interface
接口 SHALL 依次为 Geometry、Mode、Subdivide、两个独立的 Thickness、Depth Scale、Options 内 Reference Depth 与 Normal Smooth、Data 内 Uniform Scale 与 Depth Image。两个 Thickness SHALL 同名显示，依次控制 Relief 与 Camera，默认均为 0 m、最小 0，使用不同 identifier 独立保存。Subdivide SHALL 默认为 6、范围 0–10；Depth Scale 默认为 1、最小 0；Normal Smooth 默认为 50、范围 0–50。Reference Depth SHALL 由元数据基准深度乘以 Uniform Scale 初始化。Data 输入 SHALL 隐藏；所有输入 SHALL 使用 SINGLE，长度使用 DISTANCE。
#### Scenario: Inspect mode controls
- **WHEN** 创建或切换修改器模式
- **THEN** 接口顺序和默认值符合契约，Normal Smooth 只影响 Camera 的厚度法线
- **AND** 整图模式不提供 Split 或 Cutout 轮廓控制
#### Scenario: Keep mode thickness independent
- **WHEN** 在一种模式中修改另一模式对应的 Thickness，并切换模式后返回
- **THEN** 当前模式几何保持不变，两个厚度值分别保留
### Requirement: Relief preserves the fixed base
Relief SHALL 读取 B/Z 深度，保持基础 XY、固定底面、现有侧壁插值和负位移限制。Thickness SHALL 表示浮雕的基础厚度。
#### Scenario: Evaluate relief
- **WHEN** 用与旧标量输入相同的 Z、不同的 X/Y 求值 Relief
- **THEN** 顶点位置、面连接、UV 与材质和原 Depth Plane 结果一致
### Requirement: Camera preserves the projected shell
Camera SHALL 使用完整矩形细分；相机 XYZ 乘以 Uniform Scale 后得到 C，XY SHALL 按 Depth Scale 从基础 XY 插值到 `(C.x, -C.y)`，Z SHALL 为 `(Reference Depth - C.z) × Depth Scale`。零 Thickness SHALL 输出单层；正 Thickness SHALL 沿平滑归一化后的正面 POINT 法线向内生成背面并闭合边界。
#### Scenario: Match the ordinary plane
- **WHEN** Camera 的 Thickness 和 Depth Scale 均为 0
- **THEN** 相同 Subdivide 下几何与普通 Plane 一致，包括非中心显示偏移和对象变换后的世界位置
#### Scenario: Project and thicken
- **WHEN** 使用非中心主点合成数据，Depth Scale 为 0、0.5、1 或 2，Normal Smooth 为 0、2 或 50
- **THEN** 投影符合公式；正 Thickness 的对应正背面距离等于 Thickness，0 次平滑使用原始法线
- **AND** 透明区域与深度跳变区域保持完整矩形连接
### Requirement: Asset evaluation preserves geometry data
统一组 SHALL 从资产按名称加载，保留 UV、材质和用户属性，并清理自身临时属性。定义和保存资产 SHALL 满足相同接口与几何契约，布局 SHALL 在 Blender 中按实际绘制尺寸检查。
#### Scenario: Validate the delivered asset
- **WHEN** 加载更新后的节点资产
- **THEN** 公开组清单包含统一 Depth Plane 且没有独立 Depth Surface，两种模式通过几何求值
- **AND** 总览和局部步骤没有节点遮挡，未用输出隐藏，主轴与支线清楚

30
openspec/changes/unify-depth-plane-and-artifact/tasks.md Normal file
View File

@@ -0,0 +1,30 @@
## 1. 统一深度输出与读取
- [x] 1.1 将深度写出统一为 write_depth_texture、depth.exr 和结果键 depth，删除 Z 写出分支；保留内部 depth 与 depth.json，验证 RGBA 数值和有效性。
- [x] 1.2 将共享生成层、整图 Job 和 Cutout 请求改为 generate_depth 布尔值，验证深度保留 points、法线共用单次推理及边缘修正路径。
- [x] 1.3 更新整图与 Cutout 结果加载、测试夹具和公共读取调用，验证 Pack、色彩空间、标定和非有限值处理；删除旧参数和结果键。
## 2. 合并整图节点与入口
- [x] 2.1 将 Relief 与 Camera 组合为 O Image Depth Plane，增加 Mode 与统一接口；Relief 改读 B/Z，Camera 复用投影及平滑法线壳。
- [x] 2.2 重排合并组的布局，更新构建入口、公开组清单与验证；删除独立 Surface 构建和布局入口，同步节点资产文档。
- [x] 2.3 保留单一 ConvertToDepthPlane，删除独立 Surface 类、菜单、注册和对象类型分支，统一结果对象与默认值。
## 3. 回归验证
- [x] 3.1 使用 X/Y/Z 不同的纹理覆盖 Relief、Camera、Depth Cutout、Depth Balloon 和公共标量读取，验证各自位置、有效性及尺度。
- [x] 3.2 覆盖模式切换无需推理且保留参数、Relief 固定底面、Camera 投影与法线厚度、矩形拓扑、UV、材质及临时属性清理。
- [x] 3.3 更新注册与逆序注销、静态媒体和 AI 检查、成功替换与单次 Undo、失败清理测试，运行相关测试及必要的完整回归。
- [x] 3.4 搜索生产代码中的旧文件名、结果键、模式分支和独立 Surface 入口，确认无兼容转发或残留消费路径。
## 4. 资产更新验收
- [x] 4.1 在资产更新阶段构建并验证 O_AnyImage.blend，以相同用例检查保存资产和构建定义的接口与两种模式求值。
- [x] 4.2 在 Blender 节点编辑器检查总览及局部实际布局，确认主轴、分支顺序、间距、旁路和未用输出隐藏，记录视觉验收结果。
## 验证记录
- 已构建 O_AnyImage.blend，Blender 4.5.10 LTS 的资产、Mesh Plane、Cutout 验证通过。
- 完整回归报告 746 passed、2 xfailed、60 subtests passed；pytest 完成后进程退出码为 1，与本变更前已记录的退出现象一致。随后清理了 3 个合并入口造成的重复测试。
- 清理重复测试后的受影响范围回归：349 passed、30 subtests passed，退出码 0；覆盖保存资产与构建定义。
- 视觉验收待完成：Computer Use 能捕获 Blender 启动画面，但激活、点击与 Escape 后界面均未变化，尚未进入节点编辑器。坐标与几何校验已通过，不能替代实际视觉验收。

14
src/anyimage/__init__.py
View File

@@ -10,14 +10,7 @@ from .operators.image_edit_tool import (
MaskTool,
RectifyTool,
)
from .panel import (
Ben2RemovalDebugPanel,
Da3OutputDebugPanel,
DebugPanel,
Moge2ApiDebugPanel,
ServerPanel,
UpscaleDebugPanel,
)
from .panel import ServerPanel
from .preferences import AnyImagePreferences
from .properties import AnyImageSettings
from .runtime import runtime
@@ -30,11 +23,6 @@ CLASSES = (
*OPERATOR_CLASSES,
AnyImageImageMenu,
ServerPanel,
DebugPanel,
Moge2ApiDebugPanel,
Da3OutputDebugPanel,
Ben2RemovalDebugPanel,
UpscaleDebugPanel,
)

BIN
src/anyimage/assets/O_AnyImage.blend LFS
View File

Binary file not shown.

13
src/anyimage/common/ai.py
View File

@@ -79,19 +79,6 @@ def require_model(model_key):
return model
def da3_parameters(settings, path, model, process_res, start_frame):
return {
"input": str(path),
"model": model,
"process_res": process_res,
"process_res_method": "upper_bound_resize",
"video_frames": settings.video_frames,
"start_frame": start_frame,
"device": production_device().lower(),
"ref_view_strategy": settings.ref_view_strategy,
}
def moge2_parameters(settings, path, model, resolution_level, start_frame):
return {
"input": str(path),

2
src/anyimage/common/material.py
View File

@@ -281,5 +281,3 @@ def set_displacement_only(material):
if owner is not None and hasattr(owner, property_name):
setattr(owner, property_name, "DISPLACEMENT")
return

29
src/anyimage/licenses/scikit-image.txt
View File

@@ -1,29 +0,0 @@
approximate_polygon from scikit-image 0.25.2, skimage/measure/_polygon.py
Copyright: 2009-2022 the scikit-image team
License: BSD-3-Clause
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. Neither the name of the University nor the names of its contributors
may be used to endorse or promote products derived from this software
without specific prior written permission.
.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
``AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE HOLDERS OR
CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF
LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING
NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

2
src/anyimage/menu.py
View File

@@ -16,7 +16,7 @@ class AnyImageImageMenu(bpy.types.Menu):
layout.operator(
"anyimage.convert_to_plane",
text="Convert to Plane",
icon="MESH_PLANE",
icon="IMAGE_PLANE",
)
layout.operator(
"anyimage.convert_to_depth_plane",

5
src/anyimage/operators/__init__.py
View File

@@ -3,7 +3,6 @@ from .cutout_tool import (
CutoutSelectionToShape,
SelectCutoutSelection,
)
from .debug import DebugBen2, DebugDa3, DebugMoge2, DebugUpscale
from .image_edit_tool import CLASSES as IMAGE_EDIT_TOOL_CLASSES
from .convert_to_plane import (
ConvertToDepthPlane,
@@ -25,10 +24,6 @@ CLASSES = (
OpenAIEnvironmentSettings,
DownloadModel,
DownloadRequiredModels,
DebugDa3,
DebugMoge2,
DebugBen2,
DebugUpscale,
GenerateDepthPlane,
CutoutSelectionToShape,
SelectCutoutSelection,

4
src/anyimage/operators/clipboard_image/actions.py
View File

@@ -147,11 +147,11 @@ def add_image_plane(
if width <= 0 or height <= 0:
raise PasteTargetError("The clipboard image has invalid dimensions")
from ..convert_to_plane.mesh_plane import create_mesh_plane_object
from ..convert_to_plane.image_plane import create_image_plane_object
half_width, half_height = _plane_half_dimensions(width, height)
material = _create_image_material(image, shadeless, scene=context.scene)
plane = create_mesh_plane_object(
plane = create_image_plane_object(
context,
image.name,
(-half_width, half_width, -half_height, half_height),

2
src/anyimage/operators/convert_to_plane/__init__.py
View File

@@ -1,4 +1,4 @@
"""Mesh Plane and Depth Plane conversion operators."""
"""Rectangular Plane and Depth Plane conversion operators."""
from .conversion import (
ConvertToDepthPlane,

6
src/anyimage/operators/convert_to_plane/conversion.py
View File

@@ -46,7 +46,7 @@ class GenerateDepthPlane(JobOperatorBase, bpy.types.Operator):
bl_idname = "anyimage.generate_depth_plane"
bl_label = "Generate Depth Plane"
job_type = "generate-depth-plane-geometry"
starting_message = "Preparing Depth Plane"
starting_message = "Preparing depth conversion"
input_path: bpy.props.StringProperty(options=_HIDDEN)
source_object_name: bpy.props.StringProperty(options=_HIDDEN)
@@ -94,7 +94,7 @@ class GenerateDepthPlane(JobOperatorBase, bpy.types.Operator):
def start_depth_plane_job(operator, context):
source_object = context.object
if video_total_frames(source_object.data) != 1:
operator.report({"ERROR"}, "Depth Plane currently supports still images")
operator.report({"ERROR"}, "Depth conversion currently supports still images")
return {"CANCELLED"}
setup_result = invoke_ai_setup_if_needed()
if setup_result is not None:
@@ -157,7 +157,7 @@ class ConvertToPlane(bpy.types.Operator):
class ConvertToDepthPlane(bpy.types.Operator):
bl_idname = "anyimage.convert_to_depth_plane"
bl_label = "Convert to Depth Plane"
bl_description = "Convert the complete Image Empty into a front-projected relief"
bl_description = "Convert the complete Image Empty into a Relief or Camera depth plane"
bl_options = {"REGISTER", "UNDO"}
mesh_detail: bpy.props.IntProperty(name="Subdivide", default=6, min=0, max=10)

22
src/anyimage/operators/convert_to_plane/mesh_plane.py → src/anyimage/operators/convert_to_plane/image_plane.py
View File

@@ -1,4 +1,4 @@
"""Create Mesh Plane objects used by conversion and clipboard paste."""
"""Create Image Plane objects used by conversion and clipboard paste."""
import bpy
@@ -6,10 +6,10 @@ from ...common.object import modifier_input_identifier, set_modifier_input
from ...common.node import load_node_group
MESH_PLANE_NODE_GROUP_NAME = "O Mesh Plane"
IMAGE_PLANE_NODE_GROUP_NAME = "O Image Plane"
def build_mesh_plane_matrix(matrix_world, bounds):
def build_image_plane_matrix(matrix_world, bounds):
from mathutils import Matrix
x_min, x_max, y_min, y_max = bounds
@@ -17,7 +17,7 @@ def build_mesh_plane_matrix(matrix_world, bounds):
return matrix_world @ Matrix.Translation(center)
def create_mesh_plane_mesh(name, bounds):
def create_image_plane_mesh(name, bounds):
x_min, x_max, y_min, y_max = bounds
if x_max <= x_min or y_max <= y_min:
raise ValueError("The plane has invalid dimensions")
@@ -46,14 +46,14 @@ def create_mesh_plane_mesh(name, bounds):
return mesh
def add_mesh_plane_processing(
def add_image_plane_processing(
plane_object,
subdivisions=0,
thickness=0.0,
):
node_group = load_node_group(MESH_PLANE_NODE_GROUP_NAME)
node_group = load_node_group(IMAGE_PLANE_NODE_GROUP_NAME)
modifier = plane_object.modifiers.new(
name=MESH_PLANE_NODE_GROUP_NAME,
name=IMAGE_PLANE_NODE_GROUP_NAME,
type="NODES",
)
modifier.node_group = node_group
@@ -68,7 +68,7 @@ def add_mesh_plane_processing(
return modifier
def create_mesh_plane_object(
def create_image_plane_object(
context,
name,
bounds,
@@ -77,7 +77,7 @@ def create_mesh_plane_object(
subdivisions=0,
thickness=0.0,
):
mesh = create_mesh_plane_mesh(name, bounds)
mesh = create_image_plane_mesh(name, bounds)
mesh.materials.append(material)
plane = bpy.data.objects.new(name, mesh)
context.collection.objects.link(plane)
@@ -85,8 +85,8 @@ def create_mesh_plane_object(
from mathutils import Matrix
matrix_world = Matrix.Identity(4)
plane.matrix_world = build_mesh_plane_matrix(matrix_world, bounds)
add_mesh_plane_processing(
plane.matrix_world = build_image_plane_matrix(matrix_world, bounds)
add_image_plane_processing(
plane,
subdivisions=subdivisions,
thickness=thickness,

81
src/anyimage/operators/convert_to_plane/object.py
View File

@@ -23,16 +23,13 @@ from ...common.object import (
set_modifier_input,
)
from ...common.node import load_node_group
from .mesh_plane import (
build_mesh_plane_matrix,
create_mesh_plane_mesh,
create_mesh_plane_object,
from .image_plane import (
build_image_plane_matrix,
create_image_plane_mesh,
create_image_plane_object,
)
DEPTH_PLANE_NODE_GROUP_NAME = "O Image Depth Plane"
def create_color_image(source_object):
source = source_object.data
image = source.copy()
@@ -49,7 +46,7 @@ def create_color_image(source_object):
def create_plane_object(context, source_object, subdivisions, material):
source_name = source_object.name
plane = create_mesh_plane_object(
plane = create_image_plane_object(
context,
source_name,
image_empty_bounds(source_object),
@@ -71,35 +68,42 @@ def create_depth_plane_object(
depth_image,
depth_metadata,
):
node_group = load_node_group("O Image Depth Plane")
source_name = source_object.name
bounds = image_empty_bounds(source_object)
mesh = create_mesh_plane_mesh(source_name, bounds)
for polygon in mesh.polygons:
polygon.use_smooth = True
mesh.materials.append(material)
plane = bpy.data.objects.new(source_name, mesh)
context.collection.objects.link(plane)
plane.matrix_world = build_mesh_plane_matrix(source_object.matrix_world, bounds)
plane["anyimage_mesh_shape"] = "DEPTH_PLANE"
mesh = create_image_plane_mesh(source_name, bounds)
plane = None
try:
for polygon in mesh.polygons:
polygon.use_smooth = True
mesh.materials.append(material)
plane = bpy.data.objects.new(source_name, mesh)
context.collection.objects.link(plane)
plane.matrix_world = build_image_plane_matrix(source_object.matrix_world, bounds)
plane["anyimage_mesh_shape"] = "DEPTH_PLANE"
node_group = load_node_group(DEPTH_PLANE_NODE_GROUP_NAME)
modifier = plane.modifiers.new(name=node_group.name, type="NODES")
modifier.node_group = node_group
modifier.show_group_selector = False
uniform_scale = depth_uniform_scale(source_object, depth_metadata)
for name, value in (
("Subdivide", max(0, min(10, int(subdivisions)))),
("Thickness", 0.1),
("Depth Scale", 1.0),
("Reference Depth", depth_metadata["reference_depth"] * uniform_scale),
("Uniform Scale", uniform_scale),
("Depth Image", depth_image),
):
set_modifier_input(
modifier,
modifier_input_identifier(node_group, name),
value,
)
modifier = plane.modifiers.new(name=node_group.name, type="NODES")
modifier.node_group = node_group
modifier.show_group_selector = False
uniform_scale = depth_uniform_scale(source_object, depth_metadata)
for name, value in (
("Subdivide", max(0, min(10, int(subdivisions)))),
("Depth Scale", 1.0),
("Reference Depth", depth_metadata["reference_depth"] * uniform_scale),
("Uniform Scale", uniform_scale),
("Depth Image", depth_image),
):
set_modifier_input(
modifier,
modifier_input_identifier(node_group, name),
value,
)
except Exception:
if plane is not None:
bpy.data.objects.remove(plane, do_unlink=True)
if mesh.users == 0:
bpy.data.meshes.remove(mesh)
raise
return finalize_object_result(context, source_object, plane)
@@ -113,7 +117,7 @@ def create_depth_plane_from_result(context, result, operator):
try:
depth_metadata = load_depth_metadata(result.file("depth_metadata"))
depth_image = load_depth_result_image(
result.file("z_depth"),
result.file("depth"),
source_object,
depth_metadata,
)
@@ -141,9 +145,10 @@ def create_depth_plane_from_result(context, result, operator):
except Exception:
if material is not None and material.users == 0:
bpy.data.materials.remove(material, do_unlink=True)
for image in (depth_image, normal_image):
for image in (color_image, depth_image, normal_image):
if image is not None and image.users == 0:
bpy.data.images.remove(image, do_unlink=True)
raise
if "FINISHED" not in bpy.ops.ed.undo_push(message="Convert to Depth Plane"):
raise RuntimeError("Unable to create the Convert to Depth Plane Undo step")
label = "Convert to Depth Plane"
if "FINISHED" not in bpy.ops.ed.undo_push(message=label):
raise RuntimeError(f"Unable to create the {label} Undo step")

57
src/anyimage/operators/cutout_tool/main.py
View File

@@ -46,8 +46,6 @@ from ...common.viewport import (
ImageGesture,
image_edit_drag_keymap,
report_image_edit_exception,
resolve_image_edit_click,
serialize_matrix,
)
from ...common.selection import (
ImageEditWarning,
@@ -82,12 +80,8 @@ def normal_mode_for_shape(shape, generate_normal):
return NormalMode.TANGENT
def depth_mode_for_shape(shape):
if shape == "DEPTH_SOLID":
return "VECTOR"
if shape == "DEPTH_BALLOON":
return "Z"
return "NONE"
def shape_uses_depth(shape):
return shape in {"DEPTH_SOLID", "DEPTH_BALLOON"}
def create_cutout_shape(
@@ -322,19 +316,19 @@ class CutoutSelectionToShape(JobOperatorBase, bpy.types.Operator):
def request(self, context):
require_environment()
source_path = require_input_path(self.input_path)
depth_mode = depth_mode_for_shape(self.shape)
generate_depth = shape_uses_depth(self.shape)
normal_mode = normal_mode_for_shape(self.shape, self.generate_normal)
if depth_mode != "NONE" or normal_mode != NormalMode.NONE:
if generate_depth or normal_mode != NormalMode.NONE:
require_model(self.model)
parameters = {
"input": str(source_path),
"device": production_device().lower(),
"depth_mode": depth_mode,
"generate_depth": generate_depth,
"normal_mode": normal_mode.value,
"max_input_size": int(self.max_ai_input_size),
"alpha_threshold": float(self.alpha_threshold),
}
if depth_mode != "NONE" or normal_mode != NormalMode.NONE:
if generate_depth or normal_mode != NormalMode.NONE:
start_frame = source_start_frame(self.source_object_name)
parameters.update(
model=self.model,
@@ -388,11 +382,10 @@ class CutoutSelectionToShape(JobOperatorBase, bpy.types.Operator):
if normal_mode == NormalMode.OBJECT:
orient_object_normal_image(normal_image, inward_axis)
depth_metadata = None
depth_mode = depth_mode_for_shape(self.shape)
if depth_mode != "NONE":
if shape_uses_depth(self.shape):
depth_metadata = load_depth_metadata(result.file("depth_metadata"))
depth_image = load_depth_result_image(
result.file(f"{depth_mode.lower()}_depth"),
result.file("depth"),
source_object,
depth_metadata,
)
@@ -437,7 +430,6 @@ class SelectCutoutSelection(ImageGesture, bpy.types.Operator):
source_object_name: bpy.props.StringProperty(options={"HIDDEN", "SKIP_SAVE"})
source_matrix_data: bpy.props.StringProperty(options={"HIDDEN", "SKIP_SAVE"})
full_image: bpy.props.BoolProperty(default=False, options={"HIDDEN", "SKIP_SAVE"})
_pie_event = None
@@ -448,25 +440,7 @@ class SelectCutoutSelection(ImageGesture, bpy.types.Operator):
def invoke(self, context, event):
self._pie_event = event
if not self.full_image and event.value != "DOUBLE_CLICK":
return super().invoke(context, event)
source_object, should_edit = resolve_image_edit_click(
context, event, self.report
)
if not should_edit:
return {"FINISHED"} if source_object is not None else {"CANCELLED"}
if source_object is None:
return {"CANCELLED"}
if video_total_frames(source_object.data) > 1:
self.report({"WARNING"}, "Cutout only supports a still image")
return {"CANCELLED"}
self.source_object_name = source_object.name
self.source_matrix_data = serialize_matrix(source_object.matrix_world)
width, height = (int(value) for value in source_object.data.size)
selection_path = SelectionPath(
points=((0, 0), (width, 0), (width, height), (0, height)),
)
return self._submit_selection_path(context, selection_path)
return super().invoke(context, event)
def modal(self, context, event):
self._pie_event = event
@@ -495,16 +469,9 @@ class CutoutTool(bpy.types.WorkSpaceTool):
bl_description = "Lasso an Image Empty and choose a mesh shape"
bl_icon = "ops.mesh.primitive_sphere_add_gizmo"
bl_operator = SelectCutoutSelection.bl_idname
bl_keymap = (
(
SelectCutoutSelection.bl_idname,
{"type": "LEFTMOUSE", "value": "DOUBLE_CLICK"},
{"properties": [("full_image", True)]},
),
*image_edit_drag_keymap(
SelectCutoutSelection.bl_idname,
deselect_on_empty=False,
),
bl_keymap = image_edit_drag_keymap(
SelectCutoutSelection.bl_idname,
deselect_on_empty=False,
)
def draw_settings(context, layout, _tool):

129
src/anyimage/operators/debug.py
View File

@@ -1,129 +0,0 @@
from pathlib import Path
import bpy
from ..runtime import JobOperatorBase
from ..properties import BEN2_MODEL_KEY, REALESRGAN_GENERAL_WDN_MODEL_KEY
from ..common.ai import (
require_background_input_path,
da3_parameters,
require_input_path,
moge2_parameters,
require_environment,
require_model,
)
def _open_job_directory(directory):
directory = Path(directory)
if not directory.is_dir():
raise RuntimeError(f"Job directory does not exist: {directory}")
result = bpy.ops.wm.path_open(filepath=str(directory))
if "FINISHED" not in result:
raise RuntimeError(f"Unable to open job directory: {directory}")
class DebugDa3(JobOperatorBase, bpy.types.Operator):
bl_idname = "anyimage.debug_da3"
bl_label = "Debug DA3"
job_type = "debug-da3"
starting_message = "Starting DA3 API inference"
def request(self, context):
require_environment()
settings = context.scene.anyimage_settings
model = settings.model
require_model(model)
return {
**da3_parameters(
settings,
require_input_path(settings.input_path),
model,
settings.process_res,
1,
),
"device": settings.device.lower(),
"inference_mode": "joint",
"remove_background_model": "",
}
def response(self, _context, result):
_open_job_directory(result.directory)
return f"Saved DA3 API prediction to {result.directory}"
class DebugMoge2(JobOperatorBase, bpy.types.Operator):
bl_idname = "anyimage.debug_moge2"
bl_label = "Debug MoGe-2"
job_type = "debug-moge2"
starting_message = "Starting MoGe-2 inference"
def request(self, context):
require_environment()
settings = context.scene.anyimage_settings
require_model(settings.moge_model)
return {
**moge2_parameters(
settings,
require_input_path(settings.input_path),
settings.moge_model,
settings.moge_resolution_level,
1,
),
"device": settings.device.lower(),
}
def response(self, _context, result):
_open_job_directory(result.directory)
return f"Exported MoGe-2 data to {result.directory}"
class DebugBen2(JobOperatorBase, bpy.types.Operator):
bl_idname = "anyimage.debug_ben2"
bl_label = "Debug BEN2"
job_type = "debug-ben2"
starting_message = "Starting BEN2 background removal"
def request(self, context):
require_environment()
settings = context.scene.anyimage_settings
require_model(BEN2_MODEL_KEY)
return {
"input": str(require_background_input_path(settings.background_input_path)),
"model": BEN2_MODEL_KEY,
"device": settings.device.lower(),
"video_frames": settings.video_frames,
"start_frame": 1,
}
def response(self, _context, result):
_open_job_directory(result.directory)
return f"Removed background to {result.directory}"
class DebugUpscale(JobOperatorBase, bpy.types.Operator):
bl_idname = "anyimage.debug_upscale"
bl_label = "Debug Upscale"
job_type = "debug-upscale"
starting_message = "Starting Upscale inference"
model: bpy.props.StringProperty(
default=REALESRGAN_GENERAL_WDN_MODEL_KEY,
options={"HIDDEN", "SKIP_SAVE"},
)
def request(self, context):
require_environment()
settings = context.scene.anyimage_settings
require_model(self.model)
return {
"input": str(require_input_path(settings.input_path)),
"model": self.model,
"device": settings.device.lower(),
"video_frames": settings.video_frames,
"start_frame": 1,
}
def response(self, _context, result):
_open_job_directory(result.directory)
return f"Saved upscale API prediction to {result.directory}"

156
src/anyimage/panel.py
View File

@@ -1,11 +1,6 @@
import bpy
from .runtime import runtime
from .properties import (
BEN2_MODEL_KEY,
model_ready,
)
from .server.model_catalog import UPSCALE_MODELS
class ServerPanel(bpy.types.Panel):
bl_label = "Job Server"
@@ -110,154 +105,3 @@ class ServerPanel(bpy.types.Panel):
text="",
icon="TRASH",
)
class DebugPanel(bpy.types.Panel):
bl_label = "Debug"
bl_idname = "ANYIMAGE_PT_debug"
bl_space_type = "VIEW_3D"
bl_region_type = "UI"
bl_category = "AnyImage"
bl_options = {"DEFAULT_CLOSED"}
bl_order = 2
def draw(self, context):
layout = self.layout
settings = context.scene.anyimage_settings
layout.prop(settings, "device")
def draw_da3_input_settings(layout, settings, model_property):
layout.prop(settings, model_property)
layout.prop(settings, "input_path")
input_path = bpy.path.abspath(settings.input_path)
if input_path.lower().endswith(
(
".mp4",
".mov",
".avi",
".mkv",
".webm",
".m4v",
".mts",
".mpg",
".mpeg",
)
):
layout.prop(settings, "video_frames")
layout.prop(settings, "process_res")
layout.prop(settings, "process_res_method")
layout.prop(settings, "ref_view_strategy")
class Da3OutputDebugPanel(bpy.types.Panel):
bl_label = "DA3"
bl_idname = "ANYIMAGE_PT_debug_da3_output"
bl_parent_id = DebugPanel.bl_idname
bl_space_type = "VIEW_3D"
bl_region_type = "UI"
bl_category = "AnyImage"
bl_options = {"DEFAULT_CLOSED"}
bl_order = 1
def draw(self, context):
layout = self.layout
settings = context.scene.anyimage_settings
draw_da3_input_settings(layout, settings, "model")
export_row = layout.row()
export_row.enabled = (
not runtime.server_busy()
and model_ready(settings.model)
)
export_row.operator(
"anyimage.debug_da3",
text="Run DA3 API",
icon="EXPORT",
)
class Moge2ApiDebugPanel(bpy.types.Panel):
bl_label = "MoGe-2"
bl_idname = "ANYIMAGE_PT_debug_moge2_api"
bl_parent_id = DebugPanel.bl_idname
bl_space_type = "VIEW_3D"
bl_region_type = "UI"
bl_category = "AnyImage"
bl_options = {"DEFAULT_CLOSED"}
bl_order = 0
def draw(self, context):
layout = self.layout
settings = context.scene.anyimage_settings
layout.prop(settings, "input_path")
layout.prop(settings, "moge_model")
layout.prop(settings, "moge_resolution_level")
run_row = layout.row()
run_row.enabled = (
not runtime.server_busy()
and model_ready(settings.moge_model)
)
run_row.operator(
"anyimage.debug_moge2",
text="Run MoGe-2 API",
icon="EXPORT",
)
class Ben2RemovalDebugPanel(bpy.types.Panel):
bl_label = "BEN2"
bl_idname = "ANYIMAGE_PT_debug_ben2_removal"
bl_parent_id = DebugPanel.bl_idname
bl_space_type = "VIEW_3D"
bl_region_type = "UI"
bl_category = "AnyImage"
bl_options = {"DEFAULT_CLOSED"}
bl_order = 2
def draw(self, context):
layout = self.layout
settings = context.scene.anyimage_settings
layout.prop(settings, "background_input_path")
background_row = layout.row()
background_row.enabled = (
not runtime.server_busy()
and model_ready(BEN2_MODEL_KEY)
)
background_row.operator(
"anyimage.debug_ben2",
text="Remove Background",
icon="IMAGE_ALPHA",
)
class UpscaleDebugPanel(bpy.types.Panel):
bl_label = "Upscale"
bl_idname = "ANYIMAGE_PT_debug_upscale"
bl_parent_id = DebugPanel.bl_idname
bl_space_type = "VIEW_3D"
bl_region_type = "UI"
bl_category = "AnyImage"
bl_options = {"DEFAULT_CLOSED"}
bl_order = 3
def draw(self, context):
layout = self.layout
settings = context.scene.anyimage_settings
layout.prop(settings, "input_path")
input_path = bpy.path.abspath(settings.input_path)
if input_path.lower().endswith(
(".mp4", ".mov", ".avi", ".mkv", ".webm", ".m4v", ".mts", ".mpg", ".mpeg")
):
layout.prop(settings, "video_frames")
for model in UPSCALE_MODELS.values():
row = layout.row()
row.enabled = not runtime.server_busy() and model_ready(model.key)
operator = row.operator(
"anyimage.debug_upscale",
text=f"Run {model.label} API",
icon="EXPORT",
)
operator.model = model.key

106
src/anyimage/properties.py
View File

@@ -11,7 +11,6 @@ BEN2_MODEL_KEY = "BEN2_BASE"
REALESRGAN_GENERAL_WDN_MODEL_KEY = shared_model_catalog.DEFAULT_UPSCALE_MODEL_KEY
SAFMN_MODEL_KEY = shared_model_catalog.SAFMN_MODEL.key
REALESRGAN_X4PLUS_MODEL_KEY = shared_model_catalog.REALESRGAN_X4PLUS.key
DEFAULT_DA3_MODEL_KEY = "BASE"
DEFAULT_MOGE2_MODEL_KEY = "MOGE2_VITS_NORMAL"
REQUIRED_MODEL_KEYS = (
DEFAULT_MOGE2_MODEL_KEY,
@@ -145,33 +144,8 @@ def _model_items(models):
]
def model_enum_items(_owner, _context):
return _model_items(
model
for model in model_catalog()
if model.get("family") == "da3" and model.get("enabled")
)
def moge2_model_enum_items(_owner, _context):
return _model_items(
model for model in model_catalog() if model.get("family") == "moge2"
)
def depth_plane_model_enum_items(_owner, _context):
models = [model for model in model_catalog() if model.get("family") == "moge2"]
models.sort(key=lambda model: model.get("family") != "moge2")
items = _model_items(models)
items.append(
("ORIGINAL", "Original", "Use the original image as depth", len(items))
)
return items
def geometry_model_enum_items(_owner, _context):
models = [model for model in model_catalog() if model.get("family") == "moge2"]
models.sort(key=lambda model: model.get("family") != "moge2")
return _model_items(models)
@@ -302,16 +276,6 @@ class AnyImageSettings(bpy.types.PropertyGroup):
mask_gesture: mask_gesture_property()
mask_mode: mask_mode_property()
mask_radius: mask_radius_property()
input_path: bpy.props.StringProperty(
name="Image or Video",
description="A single image or video file used for reconstruction",
subtype="FILE_PATH",
)
background_input_path: bpy.props.StringProperty(
name="Image",
description="A single image used to test BEN2 background removal",
subtype="FILE_PATH",
)
video_frames: bpy.props.IntProperty(
name="Extract Frames",
description="Maximum number of consecutive frames extracted from the start of a video or frame sequence",
@@ -319,73 +283,3 @@ class AnyImageSettings(bpy.types.PropertyGroup):
min=1,
soft_max=64,
)
model: bpy.props.EnumProperty(
name="Model",
description="Depth Anything 3 model",
items=model_enum_items,
)
device: bpy.props.EnumProperty(
name="Debug Device",
description=(
"Inference device used by Debug tasks only; production tasks use "
"the Device setting in the Add-on Preferences"
),
items=device_enum_items,
default=0,
)
process_res: bpy.props.IntProperty(
name="Process Res",
description="DA3 base processing resolution",
default=504,
min=1,
soft_max=2048,
)
process_res_method: bpy.props.EnumProperty(
name="Process Res Method",
description="DA3 image resize method",
items=(
(
"upper_bound_resize",
"Upper Bound Resize",
"Make Process Res the longer side",
),
(
"lower_bound_resize",
"Lower Bound Resize",
"Make Process Res the shorter side",
),
),
default="upper_bound_resize",
)
moge_resolution_level: bpy.props.IntProperty(
name="Resolution Level",
description="MoGe-2 inference resolution level; higher is finer and slower",
default=5,
min=0,
max=9,
)
moge_model: bpy.props.EnumProperty(
name="Model",
description="MoGe-2 model used for depth estimation",
items=moge2_model_enum_items,
default=0,
)
ref_view_strategy: bpy.props.EnumProperty(
name="Ref View Strategy",
description="DA3 reference view selection strategy",
items=(
("first", "First", "Always use the first view"),
("middle", "Middle", "Use the middle view"),
(
"saddle_balanced",
"Saddle Balanced",
"Select a view with balanced feature metrics",
),
(
"saddle_sim_range",
"Saddle Sim Range",
"Select the view with the largest similarity range",
),
),
default="saddle_balanced",
)

32
src/anyimage/server/app.py
View File

@@ -2,18 +2,6 @@
from blendjob import JobServer
from .jobs.debug_predictions import (
run_ben2 as _debug_ben2,
)
from .jobs.debug_predictions import (
run_da3 as _debug_da3,
)
from .jobs.debug_predictions import (
run_moge2 as _debug_moge2,
)
from .jobs.debug_predictions import (
run_upscale as _debug_upscale,
)
from .jobs.cutout import run as _generate_cutout_artifacts
from .jobs.depth_plane import run as _generate_depth_plane_geometry
from .jobs.remove_background import run as _remove_background
@@ -58,23 +46,3 @@ def generate_depth_plane_geometry(context, parameters):
@server.job("generate-cutout-artifacts")
def generate_cutout_artifacts(context, parameters):
return _generate_cutout_artifacts(context, parameters)
@server.job("debug-da3")
def debug_da3(context, parameters):
return _debug_da3(context, parameters)
@server.job("debug-moge2")
def debug_moge2(context, parameters):
return _debug_moge2(context, parameters)
@server.job("debug-ben2")
def debug_ben2(context, parameters):
return _debug_ben2(context, parameters)
@server.job("debug-upscale")
def debug_upscale(context, parameters):
return _debug_upscale(context, parameters)

18
src/anyimage/server/geometry/depth_texture.py
View File

@@ -71,12 +71,12 @@ def _write_float_exr(rgba, output_path):
return output_path
def write_vector_depth_texture(frame, output_path):
def write_depth_texture(frame, output_path):
"""Write a frame's camera-space XYZ and validity as float RGBA."""
import numpy as np
if frame.points is None:
raise ValueError("Vector depth requires a point field")
raise ValueError("Depth requires a point field")
points = np.asarray(frame.points, dtype=np.float32)
validity = np.asarray(frame.validity, dtype=np.float32)
if not np.isfinite(points).all():
@@ -87,20 +87,6 @@ def write_vector_depth_texture(frame, output_path):
return _write_float_exr(rgba, output_path)
def write_z_depth_texture(frame, output_path):
"""Write a frame's camera Z and validity as float RGBA."""
import numpy as np
depth = np.asarray(frame.depth, dtype=np.float32)
validity = np.asarray(frame.validity, dtype=np.float32)
if not np.isfinite(depth).all():
raise ValueError("Depth field contains non-finite values")
rgba = np.empty((*depth.shape, 4), dtype=np.float32)
rgba[..., :3] = depth[..., None]
rgba[..., 3] = validity
return _write_float_exr(rgba, output_path)
def write_depth_metadata(frame, reference_valid, output_path):
"""Write camera values needed to scale a model-space depth texture."""
import numpy as np

2
src/anyimage/server/geometry/normal_texture.py
View File

@@ -65,5 +65,3 @@ def write_camera_normal_texture(
output_path = Path(output_path)
full.save(output_path)
return output_path

27
src/anyimage/server/geometry/prediction_artifacts.py
View File

@@ -4,8 +4,7 @@ from ..media import DEFAULT_MAX_AI_INPUT_SIZE, limit_image
from ..models import moge2
from .depth_texture import (
write_depth_metadata,
write_vector_depth_texture,
write_z_depth_texture,
write_depth_texture,
)
from .normal_texture import write_camera_normal_texture
from .edge_depth import extend_edge_depth
@@ -57,16 +56,13 @@ def generate_moge2_artifacts(
parameters,
input_path,
*,
depth_mode="NONE",
generate_depth=False,
normal_mode="NONE",
correct_edge_depth=False,
alpha_threshold=0.9,
):
"""Run MoGe-2 once and write only the requested file artifacts."""
depth_mode = str(depth_mode).upper()
normal_mode = str(normal_mode).upper()
if depth_mode not in {"NONE", "VECTOR", "Z"}:
raise ValueError(f"Unsupported depth mode: {depth_mode}")
if normal_mode not in {"NONE", "TANGENT", "OBJECT"}:
raise ValueError(f"Unsupported normal mode: {normal_mode}")
model_path = _limited_model_input(
@@ -79,32 +75,23 @@ def generate_moge2_artifacts(
context.resource("model_manager"),
parameters,
[model_path],
include_points=depth_mode == "VECTOR",
include_points=generate_depth,
cancel_check=context.check_cancelled,
)
if len(frames) != 1:
raise ValueError("Artifact generation requires one geometry frame")
frame = frames[0]
result = {}
if depth_mode != "NONE":
if generate_depth:
context.progress(0.45, "Generating depth texture")
alpha = _image_alpha(input_path, frame.depth.shape)
depth_frame = (
extend_edge_depth(frame, alpha, alpha_threshold)
if correct_edge_depth else frame
)
if depth_mode == "VECTOR":
depth_path = write_vector_depth_texture(
depth_frame,
context.directory / "vector-depth.exr",
)
result["vector_depth"] = depth_path
else:
depth_path = write_z_depth_texture(
depth_frame,
context.directory / "z-depth.exr",
)
result["z_depth"] = depth_path
result["depth"] = write_depth_texture(
depth_frame, context.directory / "depth.exr",
)
result["depth_metadata"] = write_depth_metadata(
depth_frame,
(alpha > 0.0) & (alpha >= alpha_threshold)

8
src/anyimage/server/jobs/cutout.py
View File

@@ -10,16 +10,16 @@ def run(context, parameters):
context.directory.mkdir(parents=True, exist_ok=True)
input_path = Path(parameters["input"])
result = {}
depth_mode = str(parameters.get("depth_mode", "NONE"))
generate_depth = bool(parameters.get("generate_depth", False))
normal_mode = str(parameters.get("normal_mode", "NONE"))
if depth_mode != "NONE" or normal_mode != "NONE":
if generate_depth or normal_mode != "NONE":
artifacts = generate_moge2_artifacts(
context,
parameters,
input_path,
depth_mode=depth_mode,
generate_depth=generate_depth,
normal_mode=normal_mode,
correct_edge_depth=depth_mode.upper() != "NONE",
correct_edge_depth=generate_depth,
alpha_threshold=float(parameters.get("alpha_threshold", 0.9)),
)
result.update({key: path.name for key, path in artifacts.items()})

96
src/anyimage/server/jobs/debug_predictions.py
View File

@@ -1,96 +0,0 @@
from ..media import copy_frames, input_frames
from ..models import da3
from ..models.ben2 import infer_foreground
from ..models.moge2 import infer_one as infer_moge2
from ..models.upscale import infer_one as infer_upscale
from ..outputs.debug_da3 import write_prediction as write_da3_prediction
from ..outputs.debug_moge2 import write_prediction as write_moge2_prediction
def run_da3(context, parameters):
context.progress(0.02, "Preparing DA3 API input")
context.directory.mkdir(parents=True, exist_ok=True)
with input_frames(context, parameters) as images:
prediction = da3.infer(
context.resource("model_manager"),
parameters,
images,
cancel_check=context.check_cancelled,
)
write_da3_prediction(prediction, context.directory / "prediction.npz")
copy_frames(images, context.directory)
context.check_cancelled()
context.progress(1.0, "Saved raw DA3 API prediction")
return {"directory": "."}
def run_moge2(context, parameters):
context.progress(0.02, "Preparing MoGe-2 API input")
context.directory.mkdir(parents=True, exist_ok=True)
with input_frames(context, parameters) as images:
for index, image in enumerate(images):
context.check_cancelled()
prediction = infer_moge2(
context.resource("model_manager"),
parameters,
image,
release_memory=index == len(images) - 1,
)
suffix = "" if len(images) == 1 else f".{index + 1:04d}"
write_moge2_prediction(
prediction,
context.directory / f"prediction{suffix}.npz",
)
copy_frames(images, context.directory)
context.check_cancelled()
context.progress(1.0, "Saved raw MoGe-2 API prediction")
return {"directory": "."}
def run_ben2(context, parameters):
context.progress(0.02, "Preparing BEN2 API input")
context.directory.mkdir(parents=True, exist_ok=True)
model_manager = context.resource("model_manager")
model_directory = model_manager.directory(parameters["model"])
with input_frames(context, parameters) as images:
for index, image in enumerate(images):
context.check_cancelled()
foreground, _load_ms, _inference_ms = infer_foreground(
image,
model_directory,
parameters["device"],
model_cache=model_manager,
cancel_check=context.check_cancelled,
release_memory=index == len(images) - 1,
)
suffix = "" if len(images) == 1 else f".{index + 1:04d}"
foreground.save(context.directory / f"foreground{suffix}.png")
copy_frames(images, context.directory)
context.check_cancelled()
context.progress(1.0, "Saved raw BEN2 API foreground")
return {"directory": "."}
def run_upscale(context, parameters):
context.progress(0.02, "Preparing Real-ESRGAN API input")
context.directory.mkdir(parents=True, exist_ok=True)
with input_frames(context, parameters) as images:
for index, image in enumerate(images):
context.check_cancelled()
prediction = infer_upscale(
context.resource("model_manager"),
parameters,
image,
cancel_check=context.check_cancelled,
release_memory=index == len(images) - 1,
)
suffix = "" if len(images) == 1 else f".{index + 1:04d}"
prediction.save(
context.directory / f"prediction{suffix}.png",
format="PNG",
compress_level=1,
)
copy_frames(images, context.directory)
context.check_cancelled()
context.progress(1.0, "Saved raw Real-ESRGAN API prediction")
return {"directory": "."}

10
src/anyimage/server/jobs/depth_plane.py
View File

@@ -1,23 +1,23 @@
"""Generate model-space Z Depth and preserve color for one complete image."""
"""Generate model-space depth and preserve color for one complete image."""
from ..geometry.prediction_artifacts import generate_moge2_artifacts
from ..media import copy_color_images, input_frames
def run(context, parameters):
context.progress(0.05, "Preparing Depth Plane")
context.progress(0.05, "Preparing depth conversion")
context.directory.mkdir(parents=True, exist_ok=True)
with input_frames(context, parameters) as images:
if len(images) != 1:
raise ValueError("Depth Plane currently supports one image")
raise ValueError("Depth conversion currently supports one image")
copy_color_images(images, context.directory)
artifacts = generate_moge2_artifacts(
context,
parameters,
images[0],
depth_mode="Z",
generate_depth=True,
normal_mode="TANGENT",
)
context.check_cancelled()
context.progress(1.0, "Depth Plane data is ready")
context.progress(1.0, "Depth conversion data is ready")
return {key: path.name for key, path in artifacts.items()}

1
src/anyimage/server/jobs/upscale.py
View File

@@ -29,7 +29,6 @@ def run(context, parameters):
parameters,
image,
cancel_check=context.check_cancelled,
preserve_alpha=True,
release_memory=index == total - 1,
)
prediction.save(

2
src/anyimage/server/media/__init__.py
View File

@@ -6,7 +6,6 @@ from .input import (
check_cancelled,
collect_images,
copy_color_images,
copy_frames,
extract_video_frames,
first_frames,
input_frames,
@@ -31,7 +30,6 @@ __all__ = (
"check_cancelled",
"collect_images",
"copy_color_images",
"copy_frames",
"extract_video_frames",
"first_frames",
"input_frames",

13
src/anyimage/server/media/input.py
View File

@@ -157,16 +157,3 @@ def copy_color_images(images, output_directory):
if Path(image_path).resolve() == destination.resolve():
continue
shutil.copy2(image_path, destination)
def copy_frames(images, output_directory):
"""Copy multiple input images as numbered raw Debug frame files."""
if len(images) <= 1:
return
for index, image_path in enumerate(images):
suffix = Path(image_path).suffix.lower() or ".jpg"
destination = Path(output_directory) / f"frame.{index + 1:04d}{suffix}"
if Path(image_path).resolve() == destination.resolve():
continue
shutil.copy2(image_path, destination)

80
src/anyimage/server/model_catalog.py
View File

@@ -9,70 +9,13 @@ class ModelSpec:
repository: str
license_name: str
description: str
supports_gs: bool
huggingface_repository: str = ""
download_patterns: tuple[str, ...] = ()
ready_patterns: tuple[str, ...] = ("config.json", "*.safetensors")
enabled: bool = True
r2_directory: str = ""
r2_files: tuple[tuple[str, int, str], ...] = ()
MODELS = {
"SMALL": ModelSpec(
key="SMALL",
label="DA3 Small",
directory_name="DA3-SMALL",
repository="depth-anything/DA3-SMALL",
license_name="Apache 2.0",
huggingface_repository="depth-anything/DA3-SMALL",
description="Fastest any-view model using ONNX Runtime",
supports_gs=False,
download_patterns=("model.onnx",),
ready_patterns=("model.onnx",),
r2_directory="DA3-SMALL",
r2_files=(
("model.onnx", 105484029, "7a9d81475b3e2334ace0d6d2c2a816b3c93081437e1866cc434ce1789125f631"),
),
),
"BASE": ModelSpec(
key="BASE",
label="DA3 Base",
directory_name="DA3-BASE",
repository="depth-anything/DA3-BASE",
license_name="Apache 2.0",
huggingface_repository="depth-anything/DA3-BASE",
description="Balanced any-view model using ONNX Runtime",
supports_gs=False,
download_patterns=("model.onnx",),
ready_patterns=("model.onnx",),
r2_directory="DA3-BASE",
r2_files=(
("model.onnx", 412856697, "31f38fc4990539d7902ba267924b9731db1acb00d840c248a60b1865810ff9e6"),
),
),
"GIANT_1_1": ModelSpec(
key="GIANT_1_1",
label="DA3 Giant 1.1",
directory_name="da3_giant_1_1",
repository="depth-anything/DA3-GIANT-1.1",
license_name="CC BY-NC 4.0",
description="Non-commercial Giant model with Gaussian head",
supports_gs=True,
enabled=False,
),
"NESTED_GIANT_LARGE_1_1": ModelSpec(
key="NESTED_GIANT_LARGE_1_1",
label="DA3 Nested Giant Large 1.1",
directory_name="da3_nested_giant_large_1_1",
repository="depth-anything/DA3NESTED-GIANT-LARGE-1.1",
license_name="CC BY-NC 4.0",
description="Non-commercial nested model with metric depth and Gaussian head",
supports_gs=True,
enabled=False,
),
}
BEN2_MODEL = ModelSpec(
key="BEN2_BASE",
label="BEN2 Base",
@@ -81,7 +24,6 @@ BEN2_MODEL = ModelSpec(
license_name="Apache 2.0",
huggingface_repository="onnx-community/BEN2-ONNX",
description="Fast high-quality FP16 ONNX background removal model",
supports_gs=False,
download_patterns=("onnx/model_fp16.onnx",),
ready_patterns=("onnx/model_fp16.onnx",),
r2_directory="BEN2-ONNX",
@@ -100,7 +42,6 @@ REALESRGAN_GENERAL_WDN_X4V3 = ModelSpec(
repository="xinntao/Real-ESRGAN",
license_name="BSD 3-Clause",
description="Fast and conservative 2x output from a 4x ONNX model",
supports_gs=False,
download_patterns=("realesr-general-wdn-x4v3.onnx",),
ready_patterns=("realesr-general-wdn-x4v3.onnx",),
r2_directory="realesr-general-wdn-x4v3",
@@ -118,7 +59,6 @@ SAFMN_MODEL = ModelSpec(
repository="sunny2109/SAFMN",
license_name="Apache 2.0",
description="Balanced 2x output from a real-world 4x ONNX model",
supports_gs=False,
download_patterns=("safmn-real-x4.onnx",),
ready_patterns=("safmn-real-x4.onnx",),
r2_directory="safmn-real-x4",
@@ -137,7 +77,6 @@ REALESRGAN_X4PLUS = ModelSpec(
license_name="BSD 3-Clause",
huggingface_repository="jonathanst29/tinier-upscale-models",
description="Detailed 2x output from a general-purpose 4x ONNX model",
supports_gs=False,
download_patterns=("realesrgan-x4plus.onnx",),
ready_patterns=("realesrgan-x4plus.onnx",),
r2_directory="realesrgan-x4plus",
@@ -156,7 +95,6 @@ PISA_SR = ModelSpec(
repository="csslc/PiSA-SR",
license_name="Apache 2.0 / CreativeML Open RAIL++-M",
description="Generative 2x upscale with fixed detail strength 0.7",
supports_gs=False,
download_patterns=("encoder.onnx", "decoder.onnx", "unet-pixel.onnx", "unet-full.onnx", "noise.npy", "LICENSE-APACHE.txt", "LICENSE-MODEL.txt", "NOTICE.txt"),
ready_patterns=("encoder.onnx", "decoder.onnx", "unet-pixel.onnx", "unet-full.onnx", "noise.npy", "LICENSE-APACHE.txt", "LICENSE-MODEL.txt", "NOTICE.txt"),
r2_directory="pisa-sr",
@@ -196,7 +134,6 @@ MOGE2_MODELS = {
license_name="MIT",
huggingface_repository="Ruicheng/moge-2-vits-normal-onnx",
description="Fastest MoGe-2 model using ONNX Runtime",
supports_gs=False,
download_patterns=("model.onnx",),
ready_patterns=("model.onnx",),
r2_directory="moge-2-vits-normal-onnx",
@@ -214,7 +151,6 @@ MOGE2_MODELS = {
license_name="MIT",
huggingface_repository="Ruicheng/moge-2-vitb-normal-onnx",
description="Balanced MoGe-2 model using ONNX Runtime",
supports_gs=False,
download_patterns=("model.onnx",),
ready_patterns=("model.onnx",),
r2_directory="moge-2-vitb-normal-onnx",
@@ -232,7 +168,6 @@ MOGE2_MODELS = {
license_name="MIT",
huggingface_repository="Ruicheng/moge-2-vitl-normal-onnx",
description="Highest-quality MoGe-2 model using ONNX Runtime",
supports_gs=False,
download_patterns=("model.onnx",),
ready_patterns=("model.onnx",),
r2_directory="moge-2-vitl-normal-onnx",
@@ -245,21 +180,10 @@ MOGE2_MODELS = {
}
MOGE2_MODEL = MOGE2_MODELS["MOGE2_VITS_NORMAL"]
DOWNLOADABLE_MODELS = {
**MODELS,
BEN2_MODEL.key: BEN2_MODEL,
**UPSCALE_MODELS,
**MOGE2_MODELS,
}
MODEL_NUMBERS = {
model_key: number for number, model_key in enumerate(MODELS)
}
def get_model(model_key):
try:
return MODELS[model_key]
except KeyError as error:
raise ValueError(f"Unknown DA3 model: {model_key}") from error
def get_downloadable_model(model_key):
@@ -281,16 +205,12 @@ def model_record(model, ready=False):
"label": model.label,
"description": model.description,
"license": model.license_name,
"supports_gs": model.supports_gs,
"enabled": model.enabled,
"family": (
"moge2"
if model.key.startswith("MOGE2_")
else "ben2"
if model.key == BEN2_MODEL.key
else "upscale"
if model.key in UPSCALE_MODELS
else "da3"
),
"ready": bool(ready),
}

25
src/anyimage/server/model_manager.py
View File

@@ -14,8 +14,6 @@ class ModelSessionCache:
def __init__(self):
self.ben2_model = None
self.ben2_key = None
self.da3_model = None
self.da3_key = None
self.moge_model = None
self.moge_key = None
self.upscale_model = None
@@ -39,21 +37,6 @@ class ModelSessionCache:
elapsed_ms = (time.perf_counter() - started) * 1000.0
return model, elapsed_ms
def get_da3(self, model_dir, device):
requested_device = getattr(device, "type", device)
key = (str(Path(model_dir).resolve()), requested_device, "onnx")
if self.da3_model is not None and self.da3_key == key:
return self.da3_model
self.release_geometry()
return self._load_da3(model_dir, requested_device, key)
def _load_da3(self, model_dir, requested_device, key):
from .models.onnx_da3 import create_session
model = create_session(model_dir, requested_device)
self.da3_model = model
self.da3_key = key
return model
def get_moge2(self, model_dir, requested_device):
from .models.onnx_moge2 import create_session
@@ -88,8 +71,6 @@ class ModelSessionCache:
self._release_runtime_cache()
def release_geometry(self):
self.da3_model = None
self.da3_key = None
self.moge_model = None
self.moge_key = None
self._release_runtime_cache()
@@ -107,17 +88,15 @@ class ModelSessionCache:
if close is not None:
close()
self.ben2_model = None
self.da3_model = None
self.moge_model = None
self.upscale_model = None
self.ben2_key = None
self.da3_key = None
self.moge_key = None
self.upscale_key = None
self._release_runtime_cache()
def snapshot(self):
geometry_key = self.da3_key or self.moge_key
geometry_key = self.moge_key
return {
"ben2": Path(self.ben2_key[0]).name if self.ben2_key else "",
"geometry": Path(geometry_key[0]).name if geometry_key else "",
@@ -169,8 +148,6 @@ class ModelManager:
def get_ben2(self, model_dir, device):
return self.sessions.get_ben2(model_dir, device)
def get_da3(self, model_dir, device):
return self.sessions.get_da3(model_dir, device)
def get_moge2(self, model_dir, device):
return self.sessions.get_moge2(model_dir, device)

3
src/anyimage/server/models/__init__.py
View File

@@ -1,8 +1,7 @@
from . import ben2, da3, moge2, upscale
from . import ben2, moge2, upscale
__all__ = (
"ben2",
"da3",
"moge2",
"upscale",
)

109
src/anyimage/server/models/da3.py
View File

@@ -1,109 +0,0 @@
from ..geometry import unproject_camera_depth
from .geometry import GeometryFrame
def merge_frame_predictions(predictions):
import numpy as np
if not predictions:
raise ValueError("DA3 returned no frame predictions")
merged = predictions[0]
for name in (
"depth",
"conf",
"extrinsics",
"intrinsics",
"processed_images",
):
setattr(
merged,
name,
np.concatenate(
[getattr(prediction, name) for prediction in predictions],
axis=0,
),
)
return merged
def infer(
model_manager,
parameters,
images,
*,
progress_callback=None,
cancel_check=None,
):
from .onnx_da3 import infer as infer_onnx
from .onnx_runtime import provider_label
model_directory = model_manager.directory(parameters["model"])
device = parameters["device"]
model = model_manager.get_da3(model_directory, device)
options = {
"process_res": parameters["process_res"],
"process_res_method": parameters["process_res_method"],
"ref_view_strategy": parameters["ref_view_strategy"],
}
def run(paths, *, release_memory):
try:
return infer_onnx(
model,
paths,
**options,
release_memory=release_memory,
)
except UnicodeDecodeError as error:
if provider_label(model) == "DirectML":
raise RuntimeError(
"DA3 inference failed on DirectML. Switch the device to "
"CPU in AnyImage Preferences and retry. "
f"Original error: {error}"
) from error
raise
if parameters.get("inference_mode", "joint") != "per_frame":
return run(
[str(path) for path in images],
release_memory=True,
)
predictions = []
for index, image in enumerate(images):
if cancel_check is not None:
cancel_check()
if progress_callback is not None:
progress_callback(index, len(images))
predictions.append(
run(
[str(image)],
release_memory=index == len(images) - 1,
)
)
return merge_frame_predictions(predictions)
def geometry_frames(prediction):
import numpy as np
depth = np.asarray(prediction.depth, dtype=np.float32)
validity = np.asarray(prediction.conf, dtype=np.float32)
intrinsics = np.asarray(prediction.intrinsics, dtype=np.float32)
if validity.shape != depth.shape or intrinsics.shape != (len(depth), 3, 3):
raise ValueError("DA3 prediction fields do not share one frame layout")
return [
GeometryFrame(
depth=frame_depth,
validity=frame_validity,
intrinsics=frame_intrinsics,
points=unproject_camera_depth(frame_depth, frame_intrinsics).astype(
np.float32
),
)
for frame_depth, frame_validity, frame_intrinsics in zip(
depth,
validity,
intrinsics,
)
]

1
src/anyimage/server/models/onnx_ben2.py
View File

@@ -55,4 +55,3 @@ def infer(session, image, *, release_memory=True):
foreground = image.convert("RGBA")
foreground.putalpha(mask)
return foreground

246
src/anyimage/server/models/onnx_da3.py
View File

@@ -1,246 +0,0 @@
"""Depth Anything 3 inference with ONNX Runtime and NumPy."""
from __future__ import annotations
from dataclasses import dataclass
from pathlib import Path
import numpy as np
try:
from .onnx_runtime import create_session as create_onnx_session
from .onnx_runtime import run_session
except ImportError:
from onnx_runtime import create_session as create_onnx_session
from onnx_runtime import run_session
ONNX_PATH = "model.onnx"
PATCH_SIZE = 14
# DA3 cross-view attention is quadratic in (views x patches); the peak
# attention tensor alone can exhaust memory long before ONNX Runtime reports
# a generic allocation failure.
ATTENTION_BUDGET_BYTES = 4 * 1024 * 1024 * 1024
STRATEGY_CODES = {
"first": 0,
"middle": 1,
"saddle_balanced": 2,
"saddle_sim_range": 3,
}
IMAGE_MEAN = np.asarray((0.485, 0.456, 0.406), dtype=np.float32)
IMAGE_STD = np.asarray((0.229, 0.224, 0.225), dtype=np.float32)
@dataclass
class Prediction:
depth: np.ndarray
conf: np.ndarray
extrinsics: np.ndarray
intrinsics: np.ndarray
processed_images: np.ndarray
is_metric: int = 0
sky: np.ndarray | None = None
gaussians: object | None = None
aux: dict | None = None
scale_factor: float | None = None
def create_session(model_dir, requested_device="auto"):
path = Path(model_dir) / ONNX_PATH
return create_onnx_session(path, requested_device)
def nearest_multiple(value, multiple=PATCH_SIZE):
lower = value // multiple * multiple
upper = lower + multiple
return upper if abs(upper - value) <= abs(value - lower) else lower
def target_size(size, process_res, method):
width, height = size
boundary = max(width, height)
if method.startswith("lower_bound"):
boundary = min(width, height)
scale = float(process_res) / boundary
width = max(1, int(round(width * scale)))
height = max(1, int(round(height * scale)))
if method.endswith("resize"):
return (
max(PATCH_SIZE, nearest_multiple(width)),
max(PATCH_SIZE, nearest_multiple(height)),
)
if method.endswith("crop"):
return (
max(PATCH_SIZE, width // PATCH_SIZE * PATCH_SIZE),
max(PATCH_SIZE, height // PATCH_SIZE * PATCH_SIZE),
)
raise ValueError(f"Unsupported process_res_method: {method}")
def resize_and_crop(image, process_res, method):
from PIL import Image
width, height = image.size
boundary = max(width, height)
if method.startswith("lower_bound"):
boundary = min(width, height)
scale = float(process_res) / boundary
resized_size = (
max(1, int(round(width * scale))),
max(1, int(round(height * scale))),
)
resampling = (
Image.Resampling.BICUBIC
if scale > 1.0
else Image.Resampling.BOX
)
image = image.resize(resized_size, resample=resampling)
final_size = target_size((width, height), process_res, method)
if method.endswith("resize"):
if image.size != final_size:
upscale = final_size[0] > image.width or final_size[1] > image.height
resampling = (
Image.Resampling.BICUBIC
if upscale
else Image.Resampling.BOX
)
image = image.resize(final_size, resample=resampling)
return image
left = (image.width - final_size[0]) // 2
top = (image.height - final_size[1]) // 2
return image.crop((left, top, left + final_size[0], top + final_size[1]))
def preprocess(image_paths, process_res=504, process_res_method="upper_bound_resize"):
from PIL import Image
processed = []
for path in image_paths:
with Image.open(path) as source:
image = resize_and_crop(
source.convert("RGB"),
process_res,
process_res_method,
)
processed.append(np.asarray(image, dtype=np.uint8))
min_height = min(image.shape[0] for image in processed)
min_width = min(image.shape[1] for image in processed)
cropped = []
for image in processed:
top = (image.shape[0] - min_height) // 2
left = (image.shape[1] - min_width) // 2
cropped.append(
image[top : top + min_height, left : left + min_width]
)
images = np.stack(cropped)
normalized = images.astype(np.float32) / 255.0
normalized = (normalized - IMAGE_MEAN) / IMAGE_STD
model_input = normalized.transpose(0, 3, 1, 2)[None]
return np.ascontiguousarray(model_input), images
def quaternion_to_matrix(quaternion):
i, j, k, real = np.moveaxis(quaternion, -1, 0)
two_scale = 2.0 / np.sum(quaternion * quaternion, axis=-1)
values = (
1 - two_scale * (j * j + k * k),
two_scale * (i * j - k * real),
two_scale * (i * k + j * real),
two_scale * (i * j + k * real),
1 - two_scale * (i * i + k * k),
two_scale * (j * k - i * real),
two_scale * (i * k - j * real),
two_scale * (j * k + i * real),
1 - two_scale * (i * i + j * j),
)
return np.stack(values, axis=-1).reshape(quaternion.shape[:-1] + (3, 3))
def camera_from_pose(pose_encoding, height, width):
translation = pose_encoding[..., :3]
rotation = quaternion_to_matrix(pose_encoding[..., 3:7])
rotation_inverse = np.swapaxes(rotation, -1, -2)
translation_inverse = -rotation_inverse @ translation[..., None]
extrinsics = np.concatenate(
(rotation_inverse, translation_inverse),
axis=-1,
).astype(np.float32)
field_of_view_height = pose_encoding[..., 7]
field_of_view_width = pose_encoding[..., 8]
focal_y = (height / 2.0) / np.maximum(
np.tan(field_of_view_height / 2.0),
1e-6,
)
focal_x = (width / 2.0) / np.maximum(
np.tan(field_of_view_width / 2.0),
1e-6,
)
intrinsics = np.zeros(pose_encoding.shape[:-1] + (3, 3), dtype=np.float32)
intrinsics[..., 0, 0] = focal_x
intrinsics[..., 1, 1] = focal_y
intrinsics[..., 0, 2] = width / 2.0
intrinsics[..., 1, 2] = height / 2.0
intrinsics[..., 2, 2] = 1.0
return extrinsics, intrinsics
def estimate_attention_bytes(views, height, width):
"""Estimate the peak cross-view attention tensor size in bytes."""
patches = (height // PATCH_SIZE) * (width // PATCH_SIZE)
tokens = views * patches
# 12 attention heads in DA3 Base; float32 activations.
return tokens * tokens * 12 * 4
def infer(
session,
image_paths,
process_res=504,
process_res_method="upper_bound_resize",
ref_view_strategy="saddle_balanced",
*,
release_memory=True,
):
inputs = session.get_inputs()
model_input, processed_images = preprocess(
image_paths,
process_res,
process_res_method,
)
_, views, _, height, width = model_input.shape
attention_bytes = estimate_attention_bytes(views, height, width)
if attention_bytes > ATTENTION_BUDGET_BYTES:
raise RuntimeError(
f"DA3 cannot process {views} frame(s) at {width}x{height}: "
"the cross-view attention would need about "
f"{attention_bytes // 2**30} GB. Reduce the frame count or "
"the Resolution Scale."
)
strategy_code = np.asarray(
[STRATEGY_CODES.get(ref_view_strategy, 2)],
dtype=np.int64,
)
input_names = [value.name for value in inputs]
outputs = run_session(
session,
None,
{
input_names[0]: model_input,
input_names[1]: strategy_code,
},
release_memory=release_memory,
)
depth, confidence, pose_encoding = outputs[:3]
height, width = depth.shape[-2:]
extrinsics, intrinsics = camera_from_pose(
pose_encoding[0],
height,
width,
)
return Prediction(
depth=np.asarray(depth[0], dtype=np.float32),
conf=np.asarray(confidence[0], dtype=np.float32),
extrinsics=extrinsics,
intrinsics=intrinsics,
processed_images=processed_images,
aux={},
)

2
src/anyimage/server/models/onnx_upscale.py
View File

@@ -153,5 +153,3 @@ def infer(session, image, cancel_check=None, *, release_memory=True):
raise UpscaleResourceError(
"the complete upscale result exceeded available system memory"
) from error

3
src/anyimage/server/models/upscale.py
View File

@@ -12,7 +12,6 @@ def infer_one(
parameters,
image_path,
cancel_check=None,
preserve_alpha=False,
release_memory=True,
):
from PIL import Image
@@ -33,7 +32,7 @@ def infer_one(
opened,
parameters.get("max_input_size", DEFAULT_MAX_AI_INPUT_SIZE),
)
source_alpha = _source_alpha(source) if preserve_alpha else None
source_alpha = _source_alpha(source)
prediction = infer(
model,
source,

1
src/anyimage/server/outputs/__init__.py
View File

@@ -1 +0,0 @@
"""Debug output serializers used by Job Server workflows."""

24
src/anyimage/server/outputs/debug_da3.py
View File

@@ -1,24 +0,0 @@
from pathlib import Path
def write_prediction(prediction, output_path):
import numpy as np
values = {}
for name in (
"depth",
"conf",
"extrinsics",
"intrinsics",
"processed_images",
"is_metric",
"sky",
"scale_factor",
):
value = getattr(prediction, name, None)
if value is not None:
values[name] = np.asarray(value)
output_path = Path(output_path)
output_path.parent.mkdir(parents=True, exist_ok=True)
np.savez(output_path, **values)
return output_path

13
src/anyimage/server/outputs/debug_moge2.py
View File

@@ -1,13 +0,0 @@
from pathlib import Path
def write_prediction(prediction, output_path):
import numpy as np
output_path = Path(output_path)
output_path.parent.mkdir(parents=True, exist_ok=True)
np.savez(
output_path,
**{name: np.asarray(value) for name, value in prediction.items()},
)
return output_path

187
tests/blender_test_support.py Normal file
View File

@@ -0,0 +1,187 @@
import importlib
import builtins
import sys
import unittest
from pathlib import Path
from types import ModuleType, SimpleNamespace
from unittest.mock import patch
PROJECT_ROOT = Path(__file__).resolve().parents[1]
SOURCE_ROOT = PROJECT_ROOT / "src"
def mock_bpy_imports(fake_bpy):
"""Stub Python imports while preserving Blender's native module identity."""
original_import = builtins.__import__
def import_with_fake_bpy(name, globals=None, locals=None, fromlist=(), level=0):
if name == "bpy" and level == 0:
return fake_bpy
return original_import(name, globals, locals, fromlist, level)
return patch("builtins.__import__", import_with_fake_bpy)
class BlenderTestCase(unittest.TestCase):
@classmethod
def setUpClass(cls):
cls.previous_addon_modules = {
name: module for name, module in sys.modules.items()
if name == "anyimage" or name.startswith("anyimage.")
}
cls.remove_modules()
def draw_status_bar(_owner, _context):
pass
draw_status_bar._draw_funcs = [draw_status_bar]
def append_status_bar(draw):
draw._owner = "anyimage"
draw_status_bar._draw_funcs.append(draw)
cls.status_bar_draws.append(draw)
def remove_status_bar(draw):
draw_status_bar._draw_funcs.remove(draw)
cls.status_bar_draws.remove(draw)
fake_bpy = ModuleType("bpy")
fake_bpy.props = SimpleNamespace(
BoolProperty=lambda **options: options,
EnumProperty=lambda **options: options,
FloatProperty=lambda **options: options,
FloatVectorProperty=lambda **options: options,
IntProperty=lambda **options: options,
PointerProperty=lambda **options: options,
StringProperty=lambda **options: options,
)
fake_bpy.types = SimpleNamespace(
AddonPreferences=object,
Menu=object,
Operator=object,
Panel=object,
PropertyGroup=object,
WorkSpaceTool=object,
Image=type("Image", (), {}),
Scene=type("Scene", (), {}),
STATUSBAR_HT_header=SimpleNamespace(
draw=draw_status_bar,
append=append_status_bar,
remove=remove_status_bar,
),
VIEW3D_MT_object_context_menu=SimpleNamespace(
append=lambda draw: cls.context_menu_draws.append(draw),
prepend=lambda draw: cls.context_menu_draws.insert(0, draw),
remove=lambda draw: cls.context_menu_draws.remove(draw),
),
OUTLINER_MT_object=SimpleNamespace(
append=lambda draw: cls.outliner_menu_draws.append(draw),
prepend=lambda draw: cls.outliner_menu_draws.insert(0, draw),
remove=lambda draw: cls.outliner_menu_draws.remove(draw),
),
)
cls.registered = []
cls.registered_tools = []
cls.status_bar_draws = []
cls.context_menu_draws = []
cls.outliner_menu_draws = []
fake_bpy.utils = SimpleNamespace(
register_class=cls.registered.append,
unregister_class=cls.registered.remove,
register_tool=lambda tool, **options: cls.registered_tools.append(
(tool, options)
),
unregister_tool=lambda tool: cls.registered_tools.remove(
next(item for item in cls.registered_tools if item[0] is tool)
),
)
fake_bpy.context = SimpleNamespace(
preferences=SimpleNamespace(addons={}),
)
fake_bpy.app = SimpleNamespace(online_access=True)
fake_bpy.path = SimpleNamespace(abspath=lambda path: path)
fake_bpy.data = SimpleNamespace(
objects=SimpleNamespace(get=lambda _name: None),
)
cls.bpy_import_patch = mock_bpy_imports(fake_bpy)
cls.bpy_import_patch.start()
cls.addClassCleanup(cls.bpy_import_patch.stop)
sys.path.insert(0, str(SOURCE_ROOT))
cls.paths_before_import = tuple(sys.path)
cls.anyimage = importlib.import_module("anyimage")
cls.runtime = importlib.import_module("anyimage.runtime")
cls.ai_setup = importlib.import_module("anyimage.operators.ai_setup")
cls.ai = importlib.import_module("anyimage.common.ai")
cls.image_data = importlib.import_module("anyimage.common.image")
cls.convert_to_plane = importlib.import_module(
"anyimage.operators.convert_to_plane.conversion"
)
cls.convert_to_plane_object = importlib.import_module(
"anyimage.operators.convert_to_plane.object"
)
cls.cutout_main = importlib.import_module(
"anyimage.operators.cutout_tool.main"
)
cls.remove_background = importlib.import_module(
"anyimage.operators.remove_background"
)
cls.upscale = importlib.import_module("anyimage.operators.upscale")
cls.image_interaction = importlib.import_module("anyimage.common.viewport")
cls.edit_mask = importlib.import_module(
"anyimage.operators.image_edit_tool.mask"
)
cls.image_selection = importlib.import_module("anyimage.common.selection")
cls.image_frame = importlib.import_module(
"anyimage.operators.image_edit_tool.frame"
)
cls.rectify = importlib.import_module(
"anyimage.operators.image_edit_tool.rectify"
)
cls.cutout_geometry = importlib.import_module(
"anyimage.operators.cutout_tool.geometry"
)
cls.paths_after_import = tuple(sys.path)
cls.fake_bpy = fake_bpy
@classmethod
def tearDownClass(cls):
sys.path.remove(str(SOURCE_ROOT))
cls.remove_modules()
sys.modules.update(cls.previous_addon_modules)
@staticmethod
def remove_modules():
for module_name in tuple(sys.modules):
if module_name == "anyimage" or module_name.startswith("anyimage."):
sys.modules.pop(module_name)
def setUp(self):
preferences = SimpleNamespace(
storage_root="",
device="CUDA",
upscale_model="REALESRGAN_GENERAL_WDN_X4V3",
max_ai_input_size=2048,
)
self.fake_bpy.context.preferences.addons = {
"anyimage": SimpleNamespace(preferences=preferences)
}
def tearDown(self):
self.anyimage.runtime.close_active()
self.registered.clear()
self.registered_tools.clear()
self.status_bar_draws.clear()
self.context_menu_draws.clear()
self.outliner_menu_draws.clear()
self.fake_bpy.data = SimpleNamespace(
objects=SimpleNamespace(get=lambda _name: None),
)
if hasattr(self.fake_bpy.types.Scene, "anyimage_settings"):
del self.fake_bpy.types.Scene.anyimage_settings
if hasattr(self.fake_bpy.types.Image, "anyimage_upscale_source"):
del self.fake_bpy.types.Image.anyimage_upscale_source

88
tests/common_test_support.py Normal file
View File

@@ -0,0 +1,88 @@
import importlib
import importlib.util
import sys
from pathlib import Path
from types import ModuleType, SimpleNamespace
from unittest.mock import patch
from blender_test_support import mock_bpy_imports
PROJECT_ROOT = Path(__file__).resolve().parents[1]
SOURCE_ROOT = PROJECT_ROOT / "src"
sys.path.insert(0, str(SOURCE_ROOT))
def load_common_modules():
fake_bpy = ModuleType("bpy")
fake_bpy.props = SimpleNamespace(
BoolProperty=lambda **options: options,
EnumProperty=lambda **options: options,
FloatProperty=lambda **options: options,
IntProperty=lambda **options: options,
StringProperty=lambda **options: options,
)
fake_bpy.types = SimpleNamespace(Menu=object, Operator=object)
fake_bpy.path = SimpleNamespace(abspath=lambda path: path)
fake_bpy.context = SimpleNamespace(scene=None)
module_paths = (
(
"anyimage.common.image",
PROJECT_ROOT
/ "src"
/ "anyimage"
/ "common"
/ "image.py",
),
(
"anyimage.common.node",
PROJECT_ROOT
/ "src"
/ "anyimage"
/ "common"
/ "node.py",
),
(
"anyimage.common.material",
PROJECT_ROOT
/ "src"
/ "anyimage"
/ "common"
/ "material.py",
),
(
"anyimage.common.object",
PROJECT_ROOT
/ "src"
/ "anyimage"
/ "common"
/ "object.py",
),
)
packages = {
name: ModuleType(name)
for name in (
"anyimage",
"anyimage.operators",
"anyimage.common",
"anyimage.operators.convert_to_plane",
)
}
for package in packages.values():
package.__path__ = []
loaded = []
preferences = ModuleType("anyimage.preferences")
preferences.configured_material_view_adaptation = lambda: True
with mock_bpy_imports(fake_bpy), patch.dict(sys.modules, {"anyimage.preferences": preferences, **packages}):
for module_name, module_path in module_paths:
specification = importlib.util.spec_from_file_location(
module_name,
module_path,
)
module = importlib.util.module_from_spec(specification)
sys.modules[module_name] = module
specification.loader.exec_module(module)
loaded.append(module)
return SimpleNamespace(image=loaded[0], node=loaded[1], material=loaded[2], object=loaded[3])

121
tests/depth_surface_test_support.py Normal file
View File

@@ -0,0 +1,121 @@
"""Behavior checks for face separation and connected Depth Surface shells."""
import sys
from pathlib import Path
from collections import Counter
import bpy
import numpy as np
ROOT = Path(__file__).parents[1]
sys.path.insert(0, str(ROOT / "src"))
sys.path.insert(0, str(ROOT / "tools" / "node_assets"))
from cutout import build_cutout_group
def surface(*, step=6.0, size=1.0, resolution=2, wrapper=False):
bpy.ops.wm.read_factory_settings(use_empty=True)
group = build_cutout_group("DEPTH_SOLID")
# Two columns with a complete depth jump along their common boundary.
points = [
(x * size / resolution, y * size / resolution, 0)
for y in range(resolution + 1)
for x in range(2 * resolution + 1)
]
stride = 2 * resolution + 1
faces = [
(
y * stride + x,
y * stride + x + 1,
(y + 1) * stride + x + 1,
(y + 1) * stride + x,
)
for y in range(resolution)
for x in range(2 * resolution)
]
mesh = bpy.data.meshes.new("Step")
mesh.from_pydata(points, [], faces)
uv = mesh.uv_layers.new(name="UVMap")
for loop in mesh.loops:
x, y, _ = points[loop.vertex_index]
uv.data[loop.index].uv = (x / (2 * size), y / size)
obj = bpy.data.objects.new("Step", mesh)
bpy.context.collection.objects.link(obj)
image = bpy.data.images.new("Camera", width=64, height=32, float_buffer=True)
image.colorspace_settings.name = "Non-Color"
image.alpha_mode = "CHANNEL_PACKED"
yy, xx = np.mgrid[:32, :64]
pixels = np.ones((32, 64, 4), np.float32)
pixels[..., 0] = (xx + 0.5) / 32
pixels[..., 1] = -(yy + 0.5) / 32
pixels[..., 2] = 1 + (xx >= 32) * step
image.pixels.foreach_set(pixels.ravel())
modifier = obj.modifiers.new("Depth Surface", "NODES")
modifier.node_group = group
inputs = {
s.name: s.identifier
for s in group.interface.items_tree
if s.item_type == "SOCKET" and s.in_out == "INPUT"
}
def set_value(name, value):
modifier[inputs[name]] = value
obj.update_tag(refresh={"DATA"})
bpy.context.view_layer.update()
for name, value in (
("Depth Image", image),
("Uniform Scale", size),
("Reference Depth", size),
("Depth Scale", 1.0),
("Thickness", 0.0),
("Front Inflation", 0.0),
("Split Threshold", 0.0),
("Mode", 1),
("Smooth", 0),
):
set_value(name, value)
if wrapper:
set_value("Cleanup Threshold", 0.0)
set_value("Inward Axis", 0)
set_value("Smooth", 0)
return obj, set_value
def evaluated(obj):
result = obj.evaluated_get(bpy.context.evaluated_depsgraph_get())
mesh = result.to_mesh()
try:
return np.array([v.co[:] for v in mesh.vertices]), [
tuple(p.vertices) for p in mesh.polygons
]
finally:
result.to_mesh_clear()
def assert_closed(faces):
edges = Counter(
tuple(sorted((a, b))) for f in faces for a, b in zip(f, (*f[1:], f[0]))
)
assert edges and set(edges.values()) == {2}, Counter(edges.values())
def component_count(faces):
adjacency = {}
for face in faces:
for a, b in zip(face, (*face[1:], face[0])):
adjacency.setdefault(a, set()).add(b)
adjacency.setdefault(b, set()).add(a)
remaining = set(adjacency)
count = 0
while remaining:
count += 1
pending = [remaining.pop()]
while pending:
for neighbor in adjacency[pending.pop()] & remaining:
remaining.remove(neighbor)
pending.append(neighbor)
return count

214
tests/image_object_test_support.py Normal file
View File

@@ -0,0 +1,214 @@
import sys
from pathlib import Path
import bpy
import numpy as np
SOURCE_ROOT = Path(__file__).parents[1] / "src"
sys.path.insert(0, str(SOURCE_ROOT))
from anyimage.common import image as image_data
from anyimage.common import object as object_data
from anyimage.common.selection import SelectionMask
from anyimage.operators.cutout_tool import geometry as cutout_geometry
from anyimage.operators.cutout_tool import object as cutout_object
def _inputs(group):
return [
item.name
for item in group.interface.items_tree
if item.item_type == "SOCKET" and item.in_out == "INPUT"
]
def _group_node(group, node_tree):
return next(
node
for node in node_tree.nodes
if node.bl_idname == "ShaderNodeGroup" and node.node_tree == group
)
def _create_cutout_shape(
source,
shape,
detail,
*,
bounds=None,
depth_image=None,
depth_metadata=None,
selection_values=None,
inward_axis=cutout_object.CutoutInwardAxis.NEGATIVE_Z,
):
if bounds is None:
bounds = (0, 0, *tuple(int(value) for value in source.data.size))
left, top, right, bottom = bounds
if selection_values is None:
selection_values = np.ones((bottom - top, right - left), dtype=np.float32)
selection_mask = SelectionMask(selection_values, bounds)
source_rgba = image_data.image_rgba(source.data)
color_image = cutout_object.create_cropped_color_image(
source,
selection_mask.bounds,
source_rgba,
)
base_shape = cutout_geometry.build_base_shape(
bpy.context,
source,
detail,
selection_mask.values,
selection_mask.bounds,
)
return cutout_object.create_shape_object(
bpy.context,
source,
shape,
detail,
selection_mask.values,
selection_mask.bounds,
base_shape,
depth_image=depth_image,
depth_metadata=depth_metadata,
inward_axis=inward_axis,
color_image=color_image,
)
def _depth_image(points, valid, name="Depth"):
points = np.asarray(points, dtype=np.float32)
valid = np.asarray(valid, dtype=bool)
height, width = valid.shape
pixels = np.zeros((height, width, 4), dtype=np.float32)
pixels[..., :3] = points
pixels[..., 3] = valid
image = bpy.data.images.new(
name,
width=width,
height=height,
alpha=True,
float_buffer=True,
)
image.colorspace_settings.name = "Non-Color"
image.alpha_mode = "CHANNEL_PACKED"
image.pixels.foreach_set(np.flipud(pixels).ravel())
image.pack()
return image
def _depth_metadata(intrinsics, image_size, reference_depth):
return {
"intrinsics": tuple(tuple(float(value) for value in row) for row in intrinsics),
"image_size": tuple(int(value) for value in image_size),
"reference_depth": float(reference_depth),
}
def _evaluated_positions(obj):
evaluated = obj.evaluated_get(bpy.context.evaluated_depsgraph_get())
result = evaluated.to_mesh()
try:
return np.asarray([tuple(vertex.co) for vertex in result.vertices])
finally:
evaluated.to_mesh_clear()
def _add_cutout_modifier(obj, shape):
group = cutout_object.load_cutout_node_group(shape)
modifier = obj.modifiers.new(group.name, "NODES")
modifier.node_group = group
if shape in {"FLAT", "DEPTH_SOLID"}:
object_data.set_modifier_input(modifier, object_data.modifier_input_identifier(group, "Mode"), 1)
object_data.set_modifier_input(
modifier,
object_data.modifier_input_identifier(group, "Inward Axis"),
0,
)
object_data.set_modifier_input(
modifier,
object_data.modifier_input_identifier(group, "Cleanup Threshold"),
0.0,
)
object_data.set_modifier_input(
modifier,
object_data.modifier_input_identifier(group, "Smooth Weight"),
0.0,
)
object_data.set_modifier_input(
modifier,
object_data.modifier_input_identifier(group, "Smooth"),
1,
)
return modifier
def _depth_surface_step_object():
vertices = [(float(x), float(y), 0.0) for y in range(3) for x in range(3)]
faces = []
for y in range(2):
for x in range(2):
a = y * 3 + x
faces.extend(((a, a + 1, a + 4), (a, a + 4, a + 3)))
data = bpy.data.meshes.new("Depth Surface Step")
data.from_pydata(vertices, (), faces)
uv_layer = data.uv_layers.new(name="UVMap")
for loop in data.loops:
coordinate = data.vertices[loop.vertex_index].co
uv_layer.data[loop.index].uv = (coordinate.x * 0.5, coordinate.y * 0.5)
obj = bpy.data.objects.new("Depth Surface Step", data)
bpy.context.collection.objects.link(obj)
modifier = _add_cutout_modifier(obj, "DEPTH_SOLID")
points = np.empty((3, 3, 3), dtype=np.float32)
for y in range(3):
for x in range(3):
points[y, x] = (float(x), -float(y), 1.0)
points[0, 0, 2] = 6.0
depth_image = _depth_image(points, np.ones((3, 3), dtype=bool))
for name, value in (
("Depth Image", depth_image),
("Depth Scale", 1.0),
("Uniform Scale", 1.0),
("Reference Depth", 1.0),
("Thickness", 0.0),
("Smooth", 0),
):
object_data.set_modifier_input(
modifier,
object_data.modifier_input_identifier(modifier.node_group, name),
value,
)
return obj, modifier
def _evaluated_topology(obj):
evaluated = obj.evaluated_get(bpy.context.evaluated_depsgraph_get())
result = evaluated.to_mesh()
try:
face_edges = {
tuple(sorted((start, end)))
for polygon in result.polygons
for start, end in zip(
polygon.vertices,
(*polygon.vertices[1:], polygon.vertices[0]),
)
}
mesh_edges = {
tuple(sorted(edge.vertices))
for edge in result.edges
}
face_vertices = {
vertex
for polygon in result.polygons
for vertex in polygon.vertices
}
return (
len(result.vertices),
len(result.polygons),
mesh_edges - face_edges,
set(range(len(result.vertices))) - face_vertices,
)
finally:
evaluated.to_mesh_clear()

66
tests/server_runtime_test_support.py Normal file
View File

@@ -0,0 +1,66 @@
import importlib.util
import sys
import tempfile
import time
import unittest
from pathlib import Path
PROJECT_ROOT = Path(__file__).resolve().parents[1]
def load_server_module():
server_directory = PROJECT_ROOT / "src" / "anyimage" / "server"
specification = importlib.util.spec_from_file_location(
"server",
server_directory / "__init__.py",
submodule_search_locations=[str(server_directory)],
)
module = importlib.util.module_from_spec(specification)
sys.modules["server"] = module
specification.loader.exec_module(module)
return module
class ServerTestCase(unittest.TestCase):
@classmethod
def setUpClass(cls):
cls.server_module = load_server_module()
importlib.import_module("server.geometry.prediction_artifacts")
importlib.import_module("server.jobs.cutout")
from blendjob import JobServer
from server.model_manager import ModelManager, ModelSessionCache
from server.models import onnx_runtime
cls.JobServer = JobServer
cls.ModelManager, cls.ModelSessionCache = ModelManager, ModelSessionCache
cls.onnx_runtime = onnx_runtime
cls.server = cls.server_module.server
cls.app_module = sys.modules["server.app"]
def setUp(self):
self.jobs_temp = tempfile.TemporaryDirectory()
self.addCleanup(self.jobs_temp.cleanup)
self.storage_root = Path(self.jobs_temp.name)
def create_cache(self):
return self.ModelSessionCache()
def create_server(self):
source = self.server_module.server
server = self.JobServer("AnyImage Job Server", storage_root=self.storage_root)
server.handlers.update(source.handlers)
server.resource_factories.update(source.resource_factories)
server.bind(self.storage_root)
return server
def wait_for_job(self, context, timeout=2.0):
deadline = time.monotonic() + timeout
while time.monotonic() < deadline:
if context._snapshot()["state"] in {"succeeded", "failed", "cancelled"}:
return context._snapshot()
time.sleep(0.01)
self.fail(f"Job did not finish: {context._snapshot()}")

102
tests/server_test_support.py Normal file
View File

@@ -0,0 +1,102 @@
import sys
from pathlib import Path
from types import SimpleNamespace
PROJECT_ROOT = Path(__file__).resolve().parents[1]
def load_server_module(name):
addon_directory = PROJECT_ROOT / "src" / "anyimage"
sys.path.insert(0, str(addon_directory))
try:
return __import__(f"server.{name}", fromlist=[name])
finally:
sys.path.remove(str(addon_directory))
def _make_fake_av(frame_count):
"""Build a fake av module yielding frame_count integer-valued frames."""
class FakeImage:
def __init__(self, value):
self.value = value
def save(self, path, format=None):
Path(path).write_text(str(self.value), encoding="utf-8")
class FakeFrame:
def __init__(self, value):
self.value = value
def to_image(self):
return FakeImage(self.value)
class FakeContainer:
def __init__(self):
self.position = 0
self.closed = False
self.streams = SimpleNamespace(video=[object()])
def decode(self, _stream):
while self.position < frame_count:
frame = FakeFrame(self.position)
self.position += 1
yield frame
def close(self):
self.closed = True
class FakeAv:
def __init__(self):
self.containers = []
def open(self, _path):
container = FakeContainer()
self.containers.append(container)
return container
return FakeAv()
class FakeModels:
def __init__(self, directory="model"):
self.model_directory = Path(directory)
def directory(self, _model):
return self.model_directory
def get_ben2(self, _model_directory, _device):
from server.models.onnx_ben2 import create_session
return create_session(self.model_directory, "cpu"), 0.0
def get_upscale(self, _model_directory, _device):
return object()
class FakeJobContext:
def __init__(self, directory, job_type, models=None):
self.directory = Path(directory)
self.job_type = job_type
self.models = models or FakeModels()
self.status = {}
self.logs = []
def resource(self, name):
if name != "model_manager":
raise KeyError(name)
return self.models
def progress(self, progress, message, **details):
self.status = {
"progress": progress,
"message": message,
**details,
}
def log(self, message, *, level="INFO"):
self.logs.append((level, message))
def check_cancelled(self):
return None

368
tests/test_addon_registration.py Normal file
View File

@@ -0,0 +1,368 @@
import importlib
import sys
from types import SimpleNamespace
from unittest.mock import patch
from blender_test_support import BlenderTestCase
class AddonRegistrationTest(BlenderTestCase):
def test_import_does_not_add_addon_directories_to_sys_path(self):
self.assertEqual(self.paths_after_import, self.paths_before_import)
def test_operator_properties_use_plain_names(self):
operators = self.anyimage.operators
for operator in operators.CLASSES:
for name in getattr(operator, "__annotations__", {}):
self.assertFalse(name.startswith("anyimage_"), (operator, name))
def test_semantic_mesh_operators_are_registered_without_dialog_options(self):
operator = self.convert_to_plane.ConvertToPlane
self.assertIn(operator, self.anyimage.CLASSES)
self.assertIn(self.convert_to_plane.ConvertToDepthPlane, self.anyimage.CLASSES)
self.assertFalse(hasattr(self.convert_to_plane, "ConvertToDepthSurface"))
self.assertIn(self.cutout_main.CutoutSelectionToShape, self.anyimage.CLASSES)
self.assertEqual(operator.bl_idname, "anyimage.convert_to_plane")
self.assertNotIn("shadeless", operator.__annotations__)
self.assertNotIn("invoke", operator.__dict__)
self.assertNotIn("draw", operator.__dict__)
self.assertIn("REGISTER", operator.bl_options)
self.assertIn("UNDO", operator.bl_options)
self.assertEqual(operator.__annotations__["mesh_detail"]["default"], 0)
self.assertEqual(
self.convert_to_plane.ConvertToDepthPlane.__annotations__["mesh_detail"][
"default"
],
6,
)
self.assertEqual(
self.convert_to_plane.GenerateDepthPlane.__annotations__["mesh_detail"][
"default"
],
6,
)
image_empty = SimpleNamespace(
type="EMPTY",
empty_display_type="IMAGE",
data=object(),
)
image_plane = SimpleNamespace(
type="MESH",
modifiers=[
SimpleNamespace(node_group=SimpleNamespace(name="O Image Plane"))
],
)
plain_mesh = SimpleNamespace(type="MESH", modifiers=[])
self.assertTrue(operator.poll(SimpleNamespace(object=image_empty)))
self.assertFalse(operator.poll(SimpleNamespace(object=image_plane)))
self.assertFalse(operator.poll(SimpleNamespace(object=plain_mesh)))
def test_image_edit_tools_and_operators_are_registered(self):
operator = self.edit_mask.EditImageAlpha
frame_operator = self.image_frame.FrameImages
self.assertIn(operator, self.anyimage.CLASSES)
self.assertIn(frame_operator, self.anyimage.CLASSES)
self.assertIn(
self.image_frame.ActivateImageEditTool,
self.anyimage.CLASSES,
)
self.assertIn(
self.rectify.RectifyImagePerspective,
self.anyimage.CLASSES,
)
self.assertNotIn("selection_path_json", operator.__annotations__)
mask_tool = self.edit_mask.MaskTool
self.assertNotIn(mask_tool, self.anyimage.CLASSES)
self.assertEqual(mask_tool.bl_operator, operator.bl_idname)
self.assertEqual(operator.bl_options, {"UNDO"})
self.assertNotIn("draw", operator.__dict__)
self.assertEqual(mask_tool.bl_icon, "ops.sculpt.lasso_mask")
self.assertNotIn("bl_options", mask_tool.__dict__)
self.assertEqual(mask_tool.bl_cursor, "PAINT_CROSS")
self.assertEqual(
mask_tool.bl_keymap[0][1]["value"],
"PRESS",
)
self.assertEqual(
mask_tool.bl_keymap[1][1],
{"type": "LEFTMOUSE", "value": "CLICK", "shift": True},
)
frame_tool = self.image_frame.FrameTool
self.assertNotIn(frame_tool, self.anyimage.CLASSES)
self.assertEqual(frame_tool.bl_operator, frame_operator.bl_idname)
self.assertEqual(frame_tool.bl_icon, "ops.sculpt.border_mask")
self.assertEqual(frame_operator.bl_options, {"UNDO"})
self.assertNotIn("bl_options", frame_tool.__dict__)
self.assertEqual(frame_tool.bl_keymap[0][1]["value"], "CLICK_DRAG")
self.assertIsNone(frame_tool.bl_keymap[0][2])
self.assertEqual(
[entry[0] for entry in frame_tool.bl_keymap[1:]],
["view3d.select", "view3d.select"],
)
self.assertEqual(
frame_tool.bl_keymap[1][2],
{"properties": [("deselect_all", True)]},
)
self.assertEqual(
frame_tool.bl_keymap[2][2],
{"properties": [("toggle", True)]},
)
self.assertNotIn("draw_settings", frame_tool.__dict__)
self.assertIn("draw_settings", mask_tool.__dict__)
settings_value = SimpleNamespace(
mask_gesture="LASSO",
mask_mode="SET",
mask_radius=25,
)
context = SimpleNamespace(
scene=SimpleNamespace(anyimage_settings=settings_value)
)
drawn_settings = []
layout = SimpleNamespace(
prop=lambda _settings, name, **_options: drawn_settings.append(name)
)
mask_tool.draw_settings(context, layout, None)
self.assertEqual(drawn_settings, ["mask_gesture", "mask_mode"])
settings_value.mask_gesture = "BRUSH"
drawn_settings.clear()
mask_tool.draw_settings(context, layout, None)
self.assertEqual(
drawn_settings,
["mask_gesture", "mask_mode", "mask_radius"],
)
settings_value.mask_gesture = "POLYLINE"
drawn_settings.clear()
mask_tool.draw_settings(context, layout, None)
self.assertEqual(drawn_settings, ["mask_gesture", "mask_mode"])
cutout_tool = importlib.import_module(
"anyimage.operators.cutout_tool"
).CutoutTool
self.assertEqual(
cutout_tool.bl_icon,
"ops.mesh.primitive_sphere_add_gizmo",
)
self.assertNotIn("bl_options", cutout_tool.__dict__)
self.assertEqual(cutout_tool.bl_keymap[0][1]["value"], "CLICK_DRAG")
self.assertEqual(
[entry[0] for entry in cutout_tool.bl_keymap[1:]],
["view3d.select", "view3d.select"],
)
self.assertEqual(
cutout_tool.bl_keymap[1][2],
{"properties": [("deselect_all", False)]},
)
self.assertEqual(
cutout_tool.bl_keymap[2][2],
{"properties": [("toggle", True)]},
)
cutout_module = sys.modules[cutout_tool.__module__]
self.assertNotIn(
"selection_path_json",
cutout_module.SelectCutoutSelection.__annotations__,
)
self.assertIn(
"selection_path_json",
cutout_module.CutoutSelectionToShape.__annotations__,
)
drawn_settings.clear()
with (
patch.object(
cutout_module,
"draw_ai_setup",
return_value={"ready": True},
) as setup,
patch.object(
cutout_module,
"draw_ai_property",
side_effect=lambda _layout, _settings, name, _status: (
drawn_settings.append(name)
),
) as draw,
):
cutout_tool.draw_settings(context, layout, None)
setup.assert_called_once_with(layout)
self.assertEqual(draw.call_count, 1)
self.assertEqual(
drawn_settings,
[
"cutout_generate_normal",
"cutout_mesh_detail",
"cutout_fine_outline",
"cutout_alpha_threshold",
"cutout_inward_axis",
],
)
self.assertEqual(operator.__annotations__["gesture"]["default"], "LASSO")
self.assertEqual(
[item[:2] for item in operator.__annotations__["gesture"]["items"]],
[
("LASSO", "Lasso"),
("BRUSH", "Brush"),
("POLYLINE", "Polyline"),
],
)
self.assertEqual(operator.__annotations__["mode"]["default"], "SUBTRACT")
self.assertEqual(operator.__annotations__["radius"]["default"], 25)
settings = self.anyimage.properties.AnyImageSettings.__annotations__
self.assertEqual(settings["mask_gesture"]["default"], "LASSO")
self.assertEqual(settings["mask_mode"]["default"], "SUBTRACT")
self.assertEqual(settings["mask_radius"]["default"], 25)
self.assertEqual(
{name for name in settings if name.startswith("mask_")},
{"mask_gesture", "mask_mode", "mask_radius"},
)
self.assertEqual(
set(operator.__annotations__),
{
"source_object_name",
"source_matrix_data",
"gesture",
"mode",
"radius",
},
)
self.assertEqual(settings["cutout_mesh_detail"]["default"], "LOW")
self.assertFalse(settings["cutout_fine_outline"]["default"])
self.assertFalse(cutout_module.CutoutSelectionToShape.__annotations__["fine_outline"]["default"])
self.assertEqual(settings["cutout_alpha_threshold"]["default"], 0.9)
self.assertEqual(settings["cutout_alpha_threshold"]["min"], 0.0)
self.assertEqual(settings["cutout_alpha_threshold"]["max"], 1.0)
self.assertEqual(settings["cutout_alpha_threshold"]["soft_min"], 0.1)
self.assertEqual(settings["cutout_alpha_threshold"]["soft_max"], 0.9)
self.assertEqual(settings["cutout_alpha_threshold"]["subtype"], "FACTOR")
self.assertEqual(
[item[:2] for item in settings["cutout_mesh_detail"]["items"]],
[
("LOW", "Low"),
("MEDIUM", "Medium"),
("HIGH", "High"),
("ULTRA", "Ultra"),
],
)
self.assertFalse(settings["cutout_generate_normal"]["default"])
self.assertEqual(settings["cutout_inward_axis"]["default"], "NEGATIVE_Z")
self.assertFalse(any("crop" in name for name in settings))
self.assertEqual(
settings["cutout_mesh_detail"]["name"], "Mesh Detail"
)
self.assertEqual(
settings["cutout_generate_normal"]["name"],
"Normal Map",
)
self.assertNotIn("image_extract_continuous", settings)
self.assertNotIn("image_extract_padding", settings)
self.assertNotIn("image_extract_remove_background", settings)
def test_rectify_tool_registration_and_properties(self):
operator = self.rectify.RectifyImagePerspective
tool = self.rectify.RectifyTool
settings = self.anyimage.properties.AnyImageSettings.__annotations__
self.assertIn(operator, self.anyimage.CLASSES)
self.assertNotIn(tool, self.anyimage.CLASSES)
self.assertEqual(tool.bl_operator, operator.bl_idname)
self.assertEqual(tool.bl_icon, "ops.sculpt.line_project")
self.assertNotIn("bl_options", tool.__dict__)
self.assertEqual(tool.bl_keymap[0][1]["value"], "PRESS")
self.assertEqual(
tool.bl_keymap[1][1],
{"type": "LEFTMOUSE", "value": "CLICK", "shift": True},
)
self.assertNotIn("bl_cursor", tool.__dict__)
self.assertNotIn("draw_settings", tool.__dict__)
self.assertNotIn("rectify_aspect_mode", settings)
self.assertNotIn("rectify_aspect_ratio", settings)
self.assertNotIn("aspect_mode", operator.__annotations__)
self.assertIn("aspect_ratio", operator.__annotations__)
self.assertNotIn("continuous", operator.__annotations__)
self.assertNotIn("padding", operator.__annotations__)
def test_registration_and_reverse_cleanup_preserve_other_addons(self):
self.assertIn(
self.anyimage.clipboard_image.PasteClipboardImage,
self.anyimage.CLASSES,
)
self.assertIn(
self.anyimage.clipboard_image.TrackNativeCopy,
self.anyimage.CLASSES,
)
existing_context_draw = object()
existing_outliner_draw = object()
self.context_menu_draws.append(existing_context_draw)
self.outliner_menu_draws.append(existing_outliner_draw)
self.assertFalse(any("debug" in cls.bl_idname.lower() for cls in self.anyimage.CLASSES if hasattr(cls, "bl_idname")))
settings = self.anyimage.properties.AnyImageSettings.__annotations__
self.assertEqual(set(settings), {
"cutout_alpha_threshold", "cutout_mesh_detail", "cutout_fine_outline",
"cutout_generate_normal", "cutout_inward_axis", "mask_gesture",
"mask_mode", "mask_radius", "video_frames",
})
self.anyimage.register()
system_classes = self.anyimage.runtime.operator_classes()
self.assertTrue(set(system_classes) <= set(self.registered))
self.assertEqual(
self.registered_tools,
[
(
self.image_frame.FrameTool,
{
"group": True,
"separator": True,
},
),
(
self.edit_mask.MaskTool,
{
"after": {self.image_frame.FrameTool.bl_idname},
"separator": False,
},
),
(
self.rectify.RectifyTool,
{
"after": {self.edit_mask.MaskTool.bl_idname},
"separator": False,
},
),
(
self.anyimage.CutoutTool,
{},
),
],
)
self.assertEqual(len(self.status_bar_draws), 1)
self.assertEqual(self.status_bar_draws[0]._owner, "anyimage")
self.assertIs(
self.context_menu_draws[0],
self.anyimage.draw_image_context_menu,
)
self.assertIs(
self.outliner_menu_draws[0],
self.anyimage.draw_image_context_menu,
)
self.assertFalse(hasattr(self.fake_bpy.types.Image, "anyimage_upscale_source"))
unregister_order = []
original_unregister = self.fake_bpy.utils.unregister_class
def unregister(cls):
unregister_order.append(cls)
original_unregister(cls)
with patch.object(self.fake_bpy.utils, "unregister_class", side_effect=unregister):
self.anyimage.unregister()
self.assertEqual(
[cls for cls in unregister_order if cls in self.anyimage.CLASSES],
list(reversed(self.anyimage.CLASSES)),
)
self.assertEqual(self.registered, [])
self.assertEqual(self.registered_tools, [])
self.assertEqual(self.status_bar_draws, [])
self.assertEqual(self.context_menu_draws, [existing_context_draw])
self.assertEqual(self.outliner_menu_draws, [existing_outliner_draw])
self.assertFalse(hasattr(self.fake_bpy.types.Image, "anyimage_upscale_source"))

151
tests/test_ai_image_operators.py Normal file
View File

@@ -0,0 +1,151 @@
from pathlib import Path
from types import SimpleNamespace
from unittest.mock import Mock, patch
from blender_test_support import BlenderTestCase
class AiImageOperatorsTest(BlenderTestCase):
def test_upscale_operator_defaults_to_wdn_and_sends_selected_model(self):
operator_type = self.upscale.UpscaleImage
self.assertIn(operator_type, self.anyimage.CLASSES)
self.assertEqual(
operator_type.__annotations__["model"]["default"],
"REALESRGAN_GENERAL_WDN_X4V3",
)
self.assertEqual(
operator_type.__annotations__["max_input_size"]["default"],
2048,
)
self.assertIn(self.runtime.JobOperatorBase, operator_type.__mro__)
self.assertEqual(operator_type.bl_options, {"UNDO"})
operator = operator_type()
operator.input_path = "input.png"
operator.source_object_name = "Source"
operator.model = "SAFMN_REAL_X4"
operator.max_input_size = 1024
context = SimpleNamespace(
scene=SimpleNamespace(anyimage_settings=SimpleNamespace(video_frames=7))
)
with (
patch.object(self.upscale, "require_environment"),
patch.object(self.upscale, "require_model") as require_model,
patch.object(
self.upscale, "require_input_path", return_value=Path("input.png")
),
patch.object(self.upscale, "production_device", return_value="CUDA"),
patch.object(self.upscale, "source_start_frame", return_value=4),
):
request = operator.request(context)
require_model.assert_called_once_with("SAFMN_REAL_X4")
self.assertEqual(
request,
{
"input": "input.png",
"model": "SAFMN_REAL_X4",
"device": "cuda",
"video_frames": 7,
"start_frame": 4,
"max_input_size": 1024,
},
)
def test_upscale_operator_opens_setup_when_required_ai_is_missing(self):
expected = {"CANCELLED"}
with patch.object(
self.upscale,
"invoke_ai_setup_if_needed",
return_value=expected,
) as setup:
result = self.upscale.UpscaleImage().invoke(None, None)
setup.assert_called_once_with()
self.assertIs(result, expected)
def test_upscale_operator_runs_immediately_when_ai_is_ready(self):
operator = self.upscale.UpscaleImage()
context = object()
expected = {"FINISHED"}
with (
patch.object(
self.upscale,
"invoke_ai_setup_if_needed",
return_value=None,
),
patch.object(
operator,
"execute",
return_value=expected,
) as execute,
):
result = operator.invoke(context, None)
execute.assert_called_once_with(context)
self.assertIs(result, expected)
self.assertNotIn("REGISTER", operator.bl_options)
def test_remove_background_loads_a_plain_image_edit_result(self):
source = SimpleNamespace()
result_image = SimpleNamespace()
result = SimpleNamespace(directory=Path("result"))
ops = SimpleNamespace(
ed=SimpleNamespace(undo_push=Mock(return_value={"FINISHED"}))
)
with (
patch.object(self.image_data, "require_image_empty", return_value=source),
patch.object(
self.image_data,
"load_image_edit_result",
return_value=result_image,
) as load,
patch.object(self.image_data, "replace_empty_image") as replace,
patch.object(self.remove_background.bpy, "ops", ops, create=True),
):
self.remove_background.apply_background_response(result, "Source")
load.assert_called_once_with(source, Path("result"))
replace.assert_called_once_with(source, result_image)
def test_upscale_loads_a_plain_image_edit_result(self):
source_image = SimpleNamespace()
source = SimpleNamespace(data=source_image)
result_image = SimpleNamespace()
result = SimpleNamespace(directory=Path("result"))
operator = self.upscale.UpscaleImage()
operator.source_object_name = "Source"
operator.source_image_name = "Poster.png"
operator.model = "SAFMN_REAL_X4"
images = SimpleNamespace(get=lambda name: source_image if name == "Poster.png" else None)
with (
patch.object(self.upscale, "require_image_empty", return_value=source),
patch.object(
self.upscale,
"load_image_edit_result",
return_value=result_image,
) as load,
patch.object(self.upscale, "replace_empty_image") as replace,
patch.object(
self.upscale,
"upscale_model_label",
return_value="SAFMN Real x4",
),
patch.object(
self.upscale.bpy,
"data",
SimpleNamespace(images=images),
),
):
message = operator.response(None, result)
load.assert_called_once_with(
source,
Path("result"),
frame_prefix="upscale.",
)
replace.assert_called_once_with(source, result_image)
self.assertEqual(message, "Image is upscaled with SAFMN Real x4")

209
tests/test_ai_setup.py Normal file
View File

@@ -0,0 +1,209 @@
import importlib
import tempfile
from pathlib import Path
from types import SimpleNamespace
from unittest.mock import patch
from blender_test_support import BlenderTestCase
class AiSetupTest(BlenderTestCase):
def test_runtime_exposes_declared_environment_and_post_install(self):
runtime = self.anyimage.runtime
self.assertEqual(runtime.environment["python"], "3.12")
self.assertIn("numpy==2.4.2", runtime.environment["packages"])
self.assertIn("scipy==1.15.3", runtime.environment["packages"])
self.assertFalse(
any(
"blendjob" in package.lower()
for package in runtime.environment["packages"]
)
)
self.assertTrue(callable(runtime.post_install))
def test_anyimage_post_install_is_ordinary_code_using_runtime_request(self):
calls = []
fake_runtime = SimpleNamespace(
request=lambda job_type, parameters: calls.append((job_type, parameters))
)
self.runtime.post_install(fake_runtime)
self.assertEqual(
calls,
[
("download-model", {"model": "MOGE2_VITS_NORMAL"}),
("download-model", {"model": "BEN2_BASE"}),
(
"download-model",
{"model": "REALESRGAN_GENERAL_WDN_X4V3"},
),
],
)
def test_anyimage_post_install_names_each_failed_model(self):
calls = []
def request(_job_type, parameters):
calls.append(parameters["model"])
raise RuntimeError("network unavailable")
with self.assertRaisesRegex(
RuntimeError,
(
"MoGe-2 ViT-S Normal: network unavailable; "
"BEN2 Base: network unavailable; "
"Real-ESRGAN General WDN x4v3: network unavailable"
),
):
self.runtime.post_install(SimpleNamespace(request=request))
self.assertEqual(
calls,
[
"MOGE2_VITS_NORMAL",
"BEN2_BASE",
"REALESRGAN_GENERAL_WDN_X4V3",
],
)
def test_required_model_operator_requests_all_default_models(self):
operator = self.anyimage.operators.DownloadRequiredModels()
with patch.object(self.ai_setup, "require_environment"):
parameters = operator.request(None)
self.assertEqual(
parameters,
{
"models": [
"MOGE2_VITS_NORMAL",
"BEN2_BASE",
"REALESRGAN_GENERAL_WDN_X4V3",
]
},
)
def test_ai_status_checks_environment_and_required_model_files(self):
properties = self.anyimage.properties
with (
patch.object(self.anyimage.runtime, "environment_ready", return_value=True),
patch.object(
properties,
"model_catalog",
return_value=(
{"key": "MOGE2_VITS_NORMAL", "ready": True},
{"key": "BEN2_BASE", "ready": False},
{
"key": "REALESRGAN_GENERAL_WDN_X4V3",
"ready": True,
},
),
),
):
status = properties.ai_status()
self.assertTrue(status["environment_ready"])
self.assertEqual(status["missing_models"], ("BEN2_BASE",))
self.assertFalse(status["ready"])
self.assertEqual(properties.ai_setup_label(status), "Download Required Models…")
with (
patch.object(self.anyimage.runtime, "environment_ready", return_value=False),
patch.object(
properties,
"model_catalog",
return_value=tuple(
{"key": key, "ready": True}
for key in properties.REQUIRED_MODEL_KEYS
),
),
):
status = properties.ai_status()
self.assertFalse(status["ready"])
self.assertEqual(
properties.ai_setup_label(status), "Set Up AI Server…"
)
def test_setup_ai_environment_selects_the_needed_install_step(self):
calls = []
ops = SimpleNamespace(
anyimage=SimpleNamespace(
install_environment=lambda *_args: (
calls.append("official") or {"FINISHED"}
),
install_environment_mirror=lambda *_args: (
calls.append("mirror") or {"FINISHED"}
),
download_required_models=lambda *_args: (
calls.append("models") or {"FINISHED"}
),
)
)
operator = self.ai_setup.SetupAIEnvironment()
with patch.object(self.fake_bpy, "ops", ops, create=True):
operator.use_mirror = True
with patch.object(
self.ai_setup,
"ai_status",
return_value={
"environment_ready": False,
"missing_models": ("MOGE2_VITS_NORMAL", "BEN2_BASE"),
"ready": False,
},
):
self.assertEqual(operator.execute(None), {"FINISHED"})
with patch.object(
self.ai_setup,
"ai_status",
return_value={
"environment_ready": True,
"missing_models": ("BEN2_BASE",),
"ready": False,
},
):
self.assertEqual(operator.execute(None), {"FINISHED"})
self.assertEqual(calls, ["mirror", "models"])
def test_ai_environment_ui_types_are_registered(self):
self.assertIn(self.ai_setup.SetupAIEnvironment, self.anyimage.CLASSES)
self.assertIn(
self.ai_setup.OpenAIEnvironmentSettings,
self.anyimage.CLASSES,
)
self.assertEqual(self.ai_setup.SetupAIEnvironment.bl_label, "Set Up AI Server")
self.assertEqual(
self.ai_setup.OpenAIEnvironmentSettings.bl_label, "Settings"
)
def test_ui_reads_required_model_files_without_starting_server(self):
properties = importlib.import_module("anyimage.properties")
with tempfile.TemporaryDirectory() as directory:
root = Path(directory)
moge = root / "models" / "moge-2-vits-normal-onnx"
ben2 = root / "models" / "BEN2-ONNX" / "onnx"
wdn = root / "models" / "realesr-general-wdn-x4v3"
moge.mkdir(parents=True)
ben2.mkdir(parents=True)
wdn.mkdir(parents=True)
(moge / "model.onnx").touch()
(ben2 / "model_fp16.onnx").touch()
(wdn / "realesr-general-wdn-x4v3.onnx").touch()
with (
patch.object(
self.anyimage.runtime,
"storage_root",
return_value=root,
),
patch.object(
self.anyimage.runtime,
"resource",
side_effect=AssertionError("UI must not start the Server"),
),
):
self.assertTrue(properties.required_models_ready())

1
tests/test_alpha_presentation.py
View File

@@ -2,7 +2,6 @@ import sys
from pathlib import Path
from types import SimpleNamespace
from unittest.mock import patch
import bpy
import numpy as np
import pytest

86
tests/test_balloon_profiles.py Normal file
View File

@@ -0,0 +1,86 @@
import sys
from pathlib import Path
from unittest.mock import patch
import numpy as np
SOURCE_ROOT = Path(__file__).parents[1] / "src"
sys.path.insert(0, str(SOURCE_ROOT))
from anyimage.operators.cutout_tool import geometry as cutout_geometry
from anyimage.operators.cutout_tool.geometry import poisson_balloon_profile
def test_poisson_balloon_profile_is_zero_on_boundary_and_positive_inside():
points = np.asarray(
((0, 0), (2, 0), (2, 2), (0, 2), (1, 1)),
dtype=float,
)
faces = ((0, 1, 4), (1, 2, 4), (2, 3, 4), (3, 0, 4))
boundary = np.asarray((True, True, True, True, False))
profile = poisson_balloon_profile(points, faces, boundary)
assert np.all(profile[:4] == 0.0)
assert profile[4] > 0.0
def test_poisson_balloon_profile_scales_in_world_units():
points = np.asarray(
((0, 0), (2, 0), (2, 2), (0, 2), (1, 1)),
dtype=float,
)
faces = ((0, 1, 4), (1, 2, 4), (2, 3, 4), (3, 0, 4))
boundary = np.asarray((True, True, True, True, False))
original = poisson_balloon_profile(points, faces, boundary)
scaled = poisson_balloon_profile(points * 2.0, faces, boundary)
assert np.isclose(scaled[4], original[4] * 2.0, atol=1e-6)
def test_poisson_balloon_profile_makes_wide_selections_thicker():
faces = ((0, 1, 4), (1, 2, 4), (2, 3, 4), (3, 0, 4))
boundary = np.asarray((True, True, True, True, False))
wide = np.asarray(
((0, 0), (4, 0), (4, 2), (0, 2), (2, 1)),
dtype=float,
)
narrow = np.asarray(
((0, 0), (4, 0), (4, 0.6), (0, 0.6), (2, 0.3)),
dtype=float,
)
wide_profile = poisson_balloon_profile(wide, faces, boundary)
narrow_profile = poisson_balloon_profile(narrow, faces, boundary)
assert wide_profile[4] > narrow_profile[4]
def test_teddy_balloon_profile_uses_chordal_axis_radius():
points = np.asarray(
((0, 0), (4, 0), (4, 2), (0, 2), (2, 1)),
dtype=float,
)
faces = ((0, 1, 4), (1, 2, 4), (2, 3, 4), (3, 0, 4))
boundary = np.asarray((True, True, True, True, False))
nodes = np.asarray(((1, 1), (3, 1)), dtype=float)
with patch.object(
cutout_geometry,
"chordal_axis",
return_value=(nodes, [(0, 1)], np.asarray((1.0, 1.0))),
):
profile = cutout_geometry.teddy_balloon_heights(
points,
faces,
points[:4],
boundary,
0.45,
)
assert np.all(profile[:4] == 0.0)
assert np.isclose(profile[4], 0.45, atol=1e-6)

4856
tests/test_blender_addon.py
View File

File diff suppressed because it is too large Load Diff

1
tests/test_brush_strokes.py
View File

@@ -2,7 +2,6 @@ import sys
from pathlib import Path
from types import SimpleNamespace
from unittest.mock import Mock, patch
import numpy as np
import pytest

11
tests/test_clipboard_image.py
View File

@@ -6,6 +6,7 @@ import unittest
from pathlib import Path
from types import ModuleType, SimpleNamespace
from unittest.mock import Mock, patch
from blender_test_support import mock_bpy_imports
PROJECT_ROOT = Path(__file__).resolve().parents[1]
@@ -15,6 +16,10 @@ sys.path.insert(0, str(PROJECT_ROOT / "src"))
class ClipboardImageTest(unittest.TestCase):
@classmethod
def setUpClass(cls):
cls.previous_addon_modules = {
name: module for name, module in sys.modules.items()
if name == "anyimage" or name.startswith("anyimage.")
}
cls.temporary_directory = tempfile.TemporaryDirectory()
cls.remove_anyimage_modules()
fake_bpy = ModuleType("bpy")
@@ -47,7 +52,9 @@ class ClipboardImageTest(unittest.TestCase):
)
)
fake_bpy.data = SimpleNamespace()
sys.modules["bpy"] = fake_bpy
cls.bpy_import_patch = mock_bpy_imports(fake_bpy)
cls.bpy_import_patch.start()
cls.addClassCleanup(cls.bpy_import_patch.stop)
cls.fake_bpy = fake_bpy
cls.addon = ModuleType("anyimage")
@@ -64,9 +71,9 @@ class ClipboardImageTest(unittest.TestCase):
@classmethod
def tearDownClass(cls):
sys.modules.pop("bpy", None)
cls.remove_anyimage_modules()
cls.temporary_directory.cleanup()
sys.modules.update(cls.previous_addon_modules)
def test_operators_use_anyimage_prefix_and_plain_properties(self):
clipboard_image = self.addon.clipboard_image

1
tests/test_cutout_fine_mesh.py
View File

@@ -1,6 +1,5 @@
import sys
from pathlib import Path
import numpy as np
import pytest
from scipy.sparse import coo_matrix

400
tests/test_cutout_geometry.py
View File

@@ -1,149 +1,21 @@
import json
import builtins
import unittest
import sys
from pathlib import Path
from types import SimpleNamespace
from unittest.mock import patch
import numpy as np
SOURCE_ROOT = Path(__file__).parents[1] / "src"
sys.path.insert(0, str(SOURCE_ROOT))
from anyimage.common.selection import (
SelectionMask,
SelectionPath,
rasterize_selection_path,
)
from anyimage.common.depth import (
depth_uniform_scale,
load_depth_metadata,
)
from anyimage.common.selection import SelectionPath, rasterize_selection_path
from anyimage.operators.cutout_tool import geometry as cutout_geometry
from anyimage.operators.cutout_tool import object as cutout_object
from anyimage.operators.cutout_tool import shape as cutout_shape
from anyimage.operators.cutout_tool.geometry import (
SelectionContours,
extract_selection_contours,
poisson_balloon_profile,
)
from anyimage.server.models import moge2 as moge2_model
from anyimage.server.models import da3 as da3_model
from anyimage.server.models.geometry import GeometryFrame
def test_cutout_mesh_limit_caps_actual_sampling_resolution():
assert cutout_geometry.limit_cutout_point_spacing(32, (4096, 1024), 1024) == 32
assert cutout_geometry.limit_cutout_point_spacing(32, (4096, 1024), 64) == 64
assert cutout_geometry.limit_cutout_point_spacing(4, (8192, 4096), 1024) == 8
def test_depth_metadata_loader_uses_one_schema_for_plane_and_cutout(tmp_path):
path = tmp_path / "depth.json"
path.write_text(
json.dumps(
{
"image_size": [3, 2],
"intrinsics": np.eye(3).tolist(),
"reference_depth": 1.0,
}
),
encoding="utf-8",
)
metadata = load_depth_metadata(path)
assert metadata["image_size"] == (3, 2)
assert metadata["intrinsics"] == ((1.0, 0.0, 0.0), (0.0, 1.0, 0.0), (0.0, 0.0, 1.0))
assert metadata["reference_depth"] == 1.0
def test_cutout_node_groups_are_loaded_from_the_asset():
loaded = object()
with patch.object(
cutout_object, "load_node_group", return_value=loaded
) as load:
for shape in cutout_shape.CUTOUT_NODE_GROUP_NAMES:
assert cutout_object.load_cutout_node_group(shape) is loaded
load.assert_called_with(cutout_shape.CUTOUT_NODE_GROUP_NAMES[shape], asset_revision=11)
assert load.call_count == 4
def test_base_plane_fit_is_stable_for_a_slender_diagonal_domain():
along = np.linspace(-1.0, 1.0, 101)
across = np.resize(np.asarray((-0.01, 0.0, 0.01)), len(along))
coordinates = np.column_stack(
(
along - across,
along + across,
)
)
values = 0.4 * along + np.sign(across) * 0.05
plane = cutout_geometry._stable_plane(coordinates, values)
selected_plane = coordinates @ plane[:2] + plane[2]
corners = np.asarray(((-1.0, -1.0), (-1.0, 1.0), (1.0, -1.0), (1.0, 1.0)))
corner_plane = corners @ plane[:2] + plane[2]
assert np.ptp(corner_plane) <= np.ptp(selected_plane) * 1.25
assert np.corrcoef(selected_plane, along)[0, 1] > 0.99
def test_depth_balloon_plane_calibration_matches_cutout_axes():
vertices = []
uv = []
camera_depth = np.empty((3, 3), dtype=np.float32)
for row in range(3):
local_y = 1.0 - row
for column in range(3):
local_x = column - 1.0
vertices.append((local_x, local_y, 0.0))
uv.append((column * 0.5, 1.0 - row * 0.5))
camera_depth[row, column] = 2.0 + 0.2 * local_x + 0.1 * local_y
reference, direction = cutout_geometry.depth_balloon_projection_parameters(
vertices,
uv,
camera_depth,
np.ones((3, 3), dtype=bool),
0.5,
)
expected_direction = np.asarray((0.1, 0.05, 1.0), dtype=np.float64)
expected_direction /= np.linalg.norm(expected_direction)
assert np.isclose(
reference,
np.percentile(camera_depth, 95.0) * 0.5,
atol=1e-6,
)
assert np.allclose(direction, expected_direction, atol=1e-6)
def test_depth_balloon_reference_uses_the_shared_upper_depth_percentile():
size = 10
row, column = np.indices((size, size))
camera_depth = np.full((size, size), 2.0, dtype=np.float32)
camera_depth[(row + column) % 2 == 0] = 2.2
vertices = []
uv = []
for y in range(size):
for x in range(size):
vertices.append((float(x), float(size - 1 - y), 0.0))
uv.append((x / (size - 1), 1.0 - y / (size - 1)))
reference, direction = cutout_geometry.depth_balloon_projection_parameters(
vertices,
uv,
camera_depth,
np.ones((size, size), dtype=bool),
1.0,
)
assert reference > float(np.median(camera_depth))
assert np.isclose(reference, 2.2, atol=1e-6)
assert np.allclose(direction, (0.0, 0.0, 1.0), atol=1e-6)
from anyimage.operators.cutout_tool.geometry import SelectionContours, extract_selection_contours
def mesh_component_count(faces):
@@ -167,113 +39,21 @@ def mesh_component_count(faces):
return count
def test_depth_surface_projection_uses_one_scale_for_all_camera_axes():
points = np.zeros((51, 101, 3), dtype=np.float32)
points[..., 2] = np.linspace(1.0, 3.0, 51 * 101).reshape(51, 101)
metadata = {
"intrinsics": (
(100.0, 0.0, 50.0),
(0.0, 100.0, 25.0),
(0.0, 0.0, 1.0),
),
"image_size": (101, 51),
"reference_depth": 2.0,
}
uniform_scale = depth_uniform_scale(
SimpleNamespace(),
metadata,
bounds=(-2.0, 2.0, -1.0, 1.0),
)
assert uniform_scale == 2.0
reference_depth = cutout_geometry.metric_depth_reference(
points[..., 2],
np.ones((51, 101), dtype=bool),
uniform_scale,
)
assert np.isclose(reference_depth, 5.8, atol=1e-6)
def test_cutout_mesh_limit_caps_actual_sampling_resolution():
assert cutout_geometry.limit_cutout_point_spacing(32, (4096, 1024), 1024) == 32
assert cutout_geometry.limit_cutout_point_spacing(32, (4096, 1024), 64) == 64
assert cutout_geometry.limit_cutout_point_spacing(4, (8192, 4096), 1024) == 8
def test_geometry_frame_requires_matching_float32_fields():
with np.testing.assert_raises_regex(ValueError, "validity"):
GeometryFrame(
depth=np.ones((2, 2), dtype=np.float32),
validity=np.ones((1, 2), dtype=np.float32),
intrinsics=np.eye(3, dtype=np.float32),
)
with np.testing.assert_raises_regex(ValueError, "normal"):
GeometryFrame(
depth=np.ones((2, 2), dtype=np.float32),
validity=np.ones((2, 2), dtype=np.float32),
intrinsics=np.eye(3, dtype=np.float32),
normal=np.ones((2, 2, 2), dtype=np.float32),
)
def test_moge_infer_returns_geometry_frames_with_model_validity():
mask = np.asarray(((True, False), (True, True)))
prediction = {
"depth": np.ones((2, 2), dtype=np.float32),
"normal": np.zeros((2, 2, 3), dtype=np.float32),
"mask": mask,
"intrinsics": np.asarray(
((1.0, 0.0, 0.5), (0.0, 1.0, 0.5), (0.0, 0.0, 1.0)),
dtype=np.float32,
),
"points": np.ones((2, 2, 3), dtype=np.float32),
}
with patch.object(moge2_model, "infer_one", return_value=prediction):
frames = moge2_model.infer(None, {}, ["image"], include_points=True)
assert len(frames) == 1
frame = frames[0]
assert np.array_equal(frame.validity, mask.astype(np.float32))
assert np.array_equal(frame.normal, prediction["normal"])
assert np.array_equal(frame.points, prediction["points"])
assert np.array_equal(
frame.intrinsics,
np.asarray(((2.0, 0.0, 1.0), (0.0, 2.0, 1.0), (0.0, 0.0, 1.0))),
)
def test_moge_z_depth_frame_does_not_require_points():
prediction = {
"depth": np.ones((2, 2), dtype=np.float32),
"normal": np.zeros((2, 2, 3), dtype=np.float32),
"mask": np.ones((2, 2), dtype=bool),
"intrinsics": np.eye(3, dtype=np.float32),
"points": np.ones((2, 2, 3), dtype=np.float32),
}
with patch.object(moge2_model, "infer_one", return_value=prediction):
frame = moge2_model.infer(None, {}, ["image"], include_points=False)[0]
assert frame.points is None
assert frame.validity.all()
def test_da3_geometry_frames_use_confidence_as_validity_and_derive_points():
prediction = SimpleNamespace(
depth=np.ones((2, 2, 3), dtype=np.float32),
conf=np.asarray(
(
((0.25, 0.5, 0.75), (1.0, 0.5, 0.25)),
((0.1, 0.2, 0.3), (0.4, 0.5, 0.6)),
),
dtype=np.float32,
),
intrinsics=np.repeat(np.eye(3, dtype=np.float32)[None], 2, axis=0),
)
frames = da3_model.geometry_frames(prediction)
assert len(frames) == 2
assert frames[0].normal is None
assert np.array_equal(frames[1].validity, prediction.conf[1])
assert frames[0].points.shape == (2, 3, 3)
assert frames[0].points.dtype == np.float32
def test_cutout_node_groups_are_loaded_from_the_asset():
loaded = object()
with patch.object(
cutout_object, "load_node_group", return_value=loaded
) as load:
for shape in cutout_shape.CUTOUT_NODE_GROUP_NAMES:
assert cutout_object.load_cutout_node_group(shape) is loaded
load.assert_called_with(cutout_shape.CUTOUT_NODE_GROUP_NAMES[shape], asset_revision=11)
assert load.call_count == 4
def test_selection_contours_group_holes_with_their_connected_component():
@@ -347,28 +127,6 @@ def test_depth_surface_cutout_uses_default_alpha_boundary():
np.testing.assert_allclose(contour.max(axis=0), (11.0, 8.0))
def test_cutout_threshold_controls_contour_and_calibration():
alpha = np.zeros((12, 16), dtype=np.float32)
alpha[2:10, 2:14] = 0.2
alpha[4:8, 5:11] = 1.0
for threshold, lower, upper in [
(0.0, (2, 2), (14, 10)),
(0.1, (2, 2), (14, 10)),
(0.5, (5, 4), (11, 8)),
(1.0, (5, 4), (11, 8)),
]:
contours = cutout_geometry.extract_cutout_contours(
alpha, 64, threshold=threshold
)
contour = np.asarray(contours.components[0][0])
np.testing.assert_allclose(contour.min(axis=0), lower)
np.testing.assert_allclose(contour.max(axis=0), upper)
valid, _ = cutout_geometry.selection_calibration(
alpha, alpha.shape, threshold=threshold
)
np.testing.assert_array_equal(valid, (alpha > 0) & (alpha >= threshold))
def test_blender_cutout_alpha_pipeline_does_not_import_pillow():
original_import = builtins.__import__
@@ -466,83 +224,6 @@ def test_cutout_cleanup_value_scales_the_ui_without_changing_node_area():
)
def test_cutout_depth_calibration_uses_five_percent_transparency_threshold():
assert not cutout_geometry.use_adaptive_depth_calibration(0.0499)
assert cutout_geometry.use_adaptive_depth_calibration(0.05)
def test_poisson_balloon_profile_is_zero_on_boundary_and_positive_inside():
points = np.asarray(
((0, 0), (2, 0), (2, 2), (0, 2), (1, 1)),
dtype=float,
)
faces = ((0, 1, 4), (1, 2, 4), (2, 3, 4), (3, 0, 4))
boundary = np.asarray((True, True, True, True, False))
profile = poisson_balloon_profile(points, faces, boundary)
assert np.all(profile[:4] == 0.0)
assert profile[4] > 0.0
def test_poisson_balloon_profile_scales_in_world_units():
points = np.asarray(
((0, 0), (2, 0), (2, 2), (0, 2), (1, 1)),
dtype=float,
)
faces = ((0, 1, 4), (1, 2, 4), (2, 3, 4), (3, 0, 4))
boundary = np.asarray((True, True, True, True, False))
original = poisson_balloon_profile(points, faces, boundary)
scaled = poisson_balloon_profile(points * 2.0, faces, boundary)
assert np.isclose(scaled[4], original[4] * 2.0, atol=1e-6)
def test_poisson_balloon_profile_makes_wide_selections_thicker():
faces = ((0, 1, 4), (1, 2, 4), (2, 3, 4), (3, 0, 4))
boundary = np.asarray((True, True, True, True, False))
wide = np.asarray(
((0, 0), (4, 0), (4, 2), (0, 2), (2, 1)),
dtype=float,
)
narrow = np.asarray(
((0, 0), (4, 0), (4, 0.6), (0, 0.6), (2, 0.3)),
dtype=float,
)
wide_profile = poisson_balloon_profile(wide, faces, boundary)
narrow_profile = poisson_balloon_profile(narrow, faces, boundary)
assert wide_profile[4] > narrow_profile[4]
def test_teddy_balloon_profile_uses_chordal_axis_radius():
points = np.asarray(
((0, 0), (4, 0), (4, 2), (0, 2), (2, 1)),
dtype=float,
)
faces = ((0, 1, 4), (1, 2, 4), (2, 3, 4), (3, 0, 4))
boundary = np.asarray((True, True, True, True, False))
nodes = np.asarray(((1, 1), (3, 1)), dtype=float)
with patch.object(
cutout_geometry,
"chordal_axis",
return_value=(nodes, [(0, 1)], np.asarray((1.0, 1.0))),
):
profile = cutout_geometry.teddy_balloon_heights(
points,
faces,
points[:4],
boundary,
0.45,
)
assert np.all(profile[:4] == 0.0)
assert np.isclose(profile[4], 0.45, atol=1e-6)
def test_cutout_domain_adapts_boundary_spacing_for_a_small_valid_selection():
components = ((((2046, 2046), (2050, 2046), (2050, 2050), (2046, 2050)),),)
@@ -621,16 +302,39 @@ def test_cutout_domain_keeps_overlapping_selection_components_disconnected():
assert mesh_component_count(domain[1]) == 2
def test_selection_calibration_uses_the_final_selection_mask():
selection_mask = SelectionMask(
np.asarray(((1.0, 1.0), (1.0, 0.0)), dtype=np.float32),
(0, 0, 2, 2),
)
class CutoutContourTest(unittest.TestCase):
def test_cutout_contour_is_smooth_and_evenly_resampled(self):
import numpy as np
valid, transparent_fraction = cutout_geometry.selection_calibration(
selection_mask.values,
(2, 2),
)
contour = cutout_geometry.smooth_selection_contour(
((0, 0), (3, 0), (4, 0), (4, 2), (0, 2)),
0.25,
)
closed = np.vstack((contour, contour[0]))
lengths = np.linalg.norm(np.diff(closed, axis=0), axis=1)
assert np.array_equal(valid, ((True, True), (True, False)))
assert transparent_fraction == 0.25
self.assertGreater(len(contour), 20)
self.assertGreater(cutout_geometry.contour_area(contour), 0.0)
self.assertLess(float(lengths.std() / lengths.mean()), 0.15)
def test_cutout_contour_smooths_pixel_stairs_before_coarse_resampling(self):
import numpy as np
staircase = [(0.0, 0.0)]
for value in range(16):
staircase.append((value + 1.0, float(value)))
staircase.append((value + 1.0, value + 1.0))
staircase.append((0.0, 16.0))
smooth = cutout_geometry.smooth_selection_contour(staircase, 2.0)
def diagonal_noise(points):
points = np.asarray(points)
selected = (
(points[:, 0] > 2.0)
& (points[:, 0] < 14.0)
& (np.abs(points[:, 1] - points[:, 0]) < 2.0)
)
return float(np.std(points[selected, 1] - points[selected, 0]))
self.assertLess(diagonal_noise(smooth), 0.05)

75
tests/test_cutout_interaction.py
View File

@@ -1,14 +1,13 @@
import json
import sys
from pathlib import Path
from types import MethodType, SimpleNamespace
from types import SimpleNamespace
SOURCE_ROOT = Path(__file__).parents[1] / "src"
sys.path.insert(0, str(SOURCE_ROOT))
from anyimage.common.selection import SelectionPath
from anyimage.common import viewport
from anyimage.common.viewport import ImageGesture
from anyimage.operators.cutout_tool import interaction
from anyimage.operators.cutout_tool import main as cutout_main
@@ -173,69 +172,27 @@ def test_cutout_mesh_detail_maps_to_denser_sampling():
] == [32, 16, 8, 4]
def test_cutout_reuses_image_lasso_gesture_and_double_click_selects_all(monkeypatch):
source = cutout_source(size=(65536, 32768))
def test_cutout_lasso_submission_passes_only_the_selected_path(monkeypatch):
source = cutout_source()
window_manager = WindowManager()
context = cutout_context(window_manager)
monkeypatch.setattr(
cutout_main, "resolve_image_edit_click", lambda *_args: (source, True)
)
monkeypatch.setattr(cutout_main, "video_total_frames", lambda _image: 1)
monkeypatch.setattr(cutout_main, "require_image_empty", lambda _name: source)
monkeypatch.setattr(cutout_main, "serialize_matrix", lambda _matrix: "matrix")
monkeypatch.setattr(
cutout_main,
"image_rgba",
lambda *_args: (_ for _ in ()).throw(AssertionError("unexpected pixel read")),
)
monkeypatch.setattr(
cutout_main,
"rasterize_selection_path",
lambda *_args, **_kwargs: (_ for _ in ()).throw(
AssertionError("unexpected Selection rasterization")
),
)
operator = SimpleNamespace(
full_image=True,
report=lambda *_args: None,
)
source.matrix_world = object()
operator._submit_selection_path = MethodType(
cutout_main.SelectCutoutSelection._submit_selection_path,
operator,
)
result = cutout_main.SelectCutoutSelection.invoke(
operator, context, SimpleNamespace(value="DOUBLE_CLICK")
)
assert issubclass(cutout_main.SelectCutoutSelection, ImageGesture)
monkeypatch.setattr(interaction, "ai_ready", lambda: False)
path = SelectionPath(points=((0, 0), (10, 0), (0, 10)))
operator = SimpleNamespace(source_object_name=source.name, _pie_event=object())
result = cutout_main.SelectCutoutSelection._submit_selection_path(operator, context, path)
assert result == {"FINISHED"}
selection_path = SelectionPath.from_json(window_manager.pie.calls[0][2].selection_path_json)
assert selection_path.points == (
(0, 0),
(65536, 0),
(65536, 32768),
(0, 32768),
)
assert window_manager.pie.calls[0][2].point_spacing == 32
assert not hasattr(cutout_main, "pending_selection")
assert SelectionPath.from_json(window_manager.pie.calls[0][2].selection_path_json) == path
def test_cutout_tool_prioritizes_double_click_before_lasso_drag():
assert cutout_main.CutoutTool.bl_keymap[0][1]["value"] == "DOUBLE_CLICK"
assert cutout_main.CutoutTool.bl_keymap[1][1]["value"] == "CLICK_DRAG"
assert [entry[0] for entry in cutout_main.CutoutTool.bl_keymap[2:]] == [
"view3d.select",
"view3d.select",
]
assert cutout_main.CutoutTool.bl_keymap[2][2] == {
"properties": [("deselect_all", False)]
}
assert cutout_main.CutoutTool.bl_keymap[3][2] == {
"properties": [("toggle", True)]
}
def test_cutout_tool_only_binds_lasso_drag_and_native_selection():
keymap = cutout_main.CutoutTool.bl_keymap
assert keymap[0][1]["value"] == "CLICK_DRAG"
assert [entry[0] for entry in keymap[1:]] == ["view3d.select", "view3d.select"]
assert all(entry[1]["value"] != "DOUBLE_CLICK" for entry in keymap)
assert "full_image" not in cutout_main.SelectCutoutSelection.__annotations__
assert keymap[1][2] == {"properties": [("deselect_all", False)]}
assert keymap[2][2] == {"properties": [("toggle", True)]}
def test_cutout_f_has_no_option_shortcut(monkeypatch):

930
tests/test_cutout_objects.py
View File

File diff suppressed because it is too large Load Diff

412
tests/test_cutout_operators.py Normal file
View File

@@ -0,0 +1,412 @@
import importlib
import json
from pathlib import Path
from types import SimpleNamespace
from unittest.mock import patch
from blender_test_support import BlenderTestCase
class CutoutOperatorsTest(BlenderTestCase):
def test_flat_and_solid_without_ai_build_locally(self):
import numpy as np
source = SimpleNamespace(
data=SimpleNamespace(size=(10, 10)), select_set=lambda _value: None
)
context = SimpleNamespace(
view_layer=SimpleNamespace(objects=SimpleNamespace(active=None))
)
selection_mask = self.image_selection.SelectionMask(
np.ones((8, 8), dtype=np.float32), (1, 1, 9, 9)
)
source_rgba = np.ones((10, 10, 4), dtype=np.float32)
for shape in ("FLAT", "SOLID"):
with self.subTest(shape=shape):
operator = SimpleNamespace(
source_object_name="Source",
selection_path_json=json.dumps(
{"points": [[1, 1], [9, 1], [9, 9], [1, 9]]}
),
shape=shape,
point_spacing=32,
alpha_threshold=0.25,
fine_outline=False,
generate_normal=False,
report=lambda *_args: None,
_needs_generation=lambda: False,
)
with (
patch.object(
self.cutout_main,
"require_image_empty",
return_value=source,
),
patch.object(
self.cutout_main, "video_total_frames", return_value=1
),
patch.object(
self.cutout_main, "image_rgba", return_value=source_rgba
) as read_rgba,
patch.object(
self.cutout_main,
"rasterize_selection_path",
return_value=selection_mask,
) as rasterize,
patch.object(
self.cutout_main,
"configured_max_ai_input_size",
return_value=2048,
),
patch.object(
self.cutout_main,
"configured_max_cutout_mesh_resolution",
return_value=1024,
),
patch.object(
self.cutout_main,
"_cutout_content_values",
return_value=selection_mask.values,
),
patch.object(
self.cutout_main,
"create_cropped_color_image",
return_value="Color",
),
patch.object(
self.cutout_main, "create_cutout_shape"
) as convert,
):
result = self.cutout_main.CutoutSelectionToShape.execute(
operator, context
)
self.assertEqual(result, {"FINISHED"})
read_rgba.assert_called_once_with(source.data)
rasterize.assert_called_once()
convert.assert_called_once_with(
context,
source,
shape,
32.0,
selection_mask.values,
selection_mask.bounds,
inward_axis="NEGATIVE_Z",
alpha_threshold=0.25,
fine_outline=False,
color_image="Color",
)
def test_ai_cutout_writes_the_selection_bounds_image(self):
import numpy as np
source = SimpleNamespace(
data=SimpleNamespace(size=(4096, 2048)),
select_set=lambda _value: None,
)
context = SimpleNamespace(
view_layer=SimpleNamespace(objects=SimpleNamespace(active=None))
)
source_rgba = np.ones((4, 8, 4), dtype=np.float32)
selection_mask = self.image_selection.SelectionMask(
np.ones((2, 6), dtype=np.float32),
(1, 1, 7, 3),
)
operator = self.cutout_main.CutoutSelectionToShape()
operator.alpha_threshold = 0.25
operator.fine_outline = False
operator.source_object_name = "Source"
operator.selection_path_json = json.dumps(
{"points": [[1, 1], [7, 1], [7, 3], [1, 3]]}
)
operator.shape = "DEPTH_SOLID"
operator.point_spacing = 32
operator.generate_normal = False
operator.report = lambda *_args: None
with (
patch.object(
self.cutout_main,
"require_image_empty",
return_value=source,
),
patch.object(self.cutout_main, "video_total_frames", return_value=1),
patch.object(
self.cutout_main,
"image_rgba",
return_value=source_rgba,
) as read_rgba,
patch.object(
self.cutout_main,
"rasterize_selection_path",
return_value=selection_mask,
) as rasterize,
patch.object(
self.cutout_main,
"configured_max_ai_input_size",
return_value=2048,
),
patch.object(
self.cutout_main,
"configured_max_cutout_mesh_resolution",
return_value=1024,
),
patch.object(
self.cutout_main,
"invoke_ai_setup_if_needed",
return_value=None,
),
patch.object(
self.cutout_main,
"cutout_geometry_settings",
return_value=("MODEL", 5),
),
patch.object(
self.cutout_main,
"prepare_image_bounds_input",
return_value=Path(__file__),
) as prepare_input,
patch.object(
self.cutout_main.JobOperatorBase,
"execute",
return_value={"FINISHED"},
),
):
result = operator.execute(context)
self.assertEqual(result, {"FINISHED"})
read_rgba.assert_called_once_with(source.data)
rasterize.assert_called_once()
prepare_input.assert_called_once_with(
source.data,
source_rgba,
selection_mask.bounds,
)
def test_empty_cutout_selection_is_rejected_after_shape_choice(self):
source = SimpleNamespace(
data=SimpleNamespace(size=(4096, 2048)),
select_set=lambda _value: None,
)
context = SimpleNamespace(
view_layer=SimpleNamespace(objects=SimpleNamespace(active=None))
)
reports = []
operator = SimpleNamespace(
source_object_name="Source",
selection_path_json=json.dumps(
{"points": [[1, 1], [7, 1], [7, 3], [1, 3]]}
),
shape="FLAT",
point_spacing=32,
generate_normal=False,
report=lambda level, message: reports.append((level, message)),
)
with (
patch.object(
self.cutout_main,
"require_image_empty",
return_value=source,
),
patch.object(self.cutout_main, "video_total_frames", return_value=1),
patch.object(self.cutout_main, "image_rgba"),
patch.object(
self.cutout_main,
"rasterize_selection_path",
side_effect=self.image_selection.ImageEditWarning(
"The selection contains no visible image pixels"
),
),
patch.object(self.cutout_main, "create_cutout_shape") as create_shape,
):
result = self.cutout_main.CutoutSelectionToShape.execute(
operator,
context,
)
self.assertEqual(result, {"CANCELLED"})
create_shape.assert_not_called()
self.assertEqual(
reports,
[
(
{"WARNING"},
"The selection contains no visible image pixels",
)
],
)
def test_cutout_request_describes_ai_capabilities(self):
operator = self.cutout_main.CutoutSelectionToShape()
operator.alpha_threshold = 0.25
operator.fine_outline = False
operator.input_path = __file__
operator.source_object_name = "Source"
operator.shape = "DEPTH_SOLID"
operator.model = self.anyimage.properties.DEFAULT_MOGE2_MODEL_KEY
operator.resolution_level = 5
operator.max_ai_input_size = 2048
operator.generate_normal = True
context = SimpleNamespace(
scene=SimpleNamespace(anyimage_settings=SimpleNamespace())
)
with (
patch.object(self.cutout_main, "require_environment"),
patch.object(self.cutout_main, "require_model") as require_model,
patch.object(
self.cutout_main, "production_device", return_value="CPU"
),
patch.object(self.cutout_main, "source_start_frame", return_value=1),
patch.object(
self.cutout_main,
"moge2_parameters",
return_value={"model": operator.model, "resolution_level": 5},
),
):
parameters = operator.request(context)
self.assertEqual(
[call.args[0] for call in require_model.call_args_list],
[operator.model],
)
self.assertTrue(parameters["generate_depth"])
self.assertEqual(parameters["alpha_threshold"], 0.25)
self.assertEqual(parameters["normal_mode"], "OBJECT")
self.assertNotIn("polygon", parameters)
self.assertNotIn("shape", parameters)
self.assertNotIn("normal_analysis", parameters)
def test_cutout_geometry_settings_use_configured_moge(self):
preferences = SimpleNamespace(
geometry_model="MOGE2_VITB_NORMAL",
geometry_resolution_level=7,
)
with patch.object(
self.cutout_main,
"addon_preferences",
return_value=preferences,
):
settings = self.cutout_main.cutout_geometry_settings(None)
self.assertEqual(
settings,
("MOGE2_VITB_NORMAL", 7),
)
def test_cutout_tool_settings_gate_normal_generation(self):
class Row:
def __init__(self, events):
self.events = events
self.enabled = True
def prop(self, _owner, name, **_options):
self.events.append(("prop", name, self.enabled))
class Layout:
def __init__(self):
self.events = []
def operator(self, identifier, **_options):
self.events.append(("operator", identifier, True))
def row(self):
return Row(self.events)
def prop(self, _owner, name, **_options):
self.events.append(("prop", name, True))
context = SimpleNamespace(
scene=SimpleNamespace(anyimage_settings=SimpleNamespace())
)
missing = {
"environment_ready": False,
"missing_models": (),
"ready": False,
}
setup = ("operator", "anyimage.setup_ai_environment", True)
with patch.object(self.ai, "ai_status", return_value=missing):
cutout_tool = importlib.import_module(
"anyimage.operators.cutout_tool"
).CutoutTool
layout = Layout()
cutout_tool.draw_settings(context, layout, None)
self.assertEqual(
layout.events,
[
setup,
("prop", "cutout_generate_normal", False),
("prop", "cutout_mesh_detail", True),
("prop", "cutout_fine_outline", True),
("prop", "cutout_alpha_threshold", True),
("prop", "cutout_inward_axis", True),
],
)
ready = {"environment_ready": True, "missing_models": (), "ready": True}
with patch.object(self.ai, "ai_status", return_value=ready):
layout = Layout()
cutout_tool.draw_settings(context, layout, None)
self.assertNotIn(setup, layout.events)
self.assertTrue(all(event[2] for event in layout.events))
def test_cutout_ai_response_reuses_bounds_color_and_lasso(self):
import numpy as np
source = SimpleNamespace(data=SimpleNamespace())
color_image = SimpleNamespace(users=0)
pure_values = np.ones((3, 4), dtype=np.float32)
pure_values[:, 0] = 0.0
selection_mask = self.image_selection.SelectionMask(
pure_values,
(10, 20, 14, 23),
)
operator = self.cutout_main.CutoutSelectionToShape()
operator.alpha_threshold = 0.25
operator.fine_outline = False
operator.source_object_name = "Source"
operator.selection_bounds_json = "[10,20,14,23]"
operator.input_path = __file__
operator.shape = "FLAT"
operator.point_spacing = 32
operator.generate_normal = True
operator._selection_mask = selection_mask
result = SimpleNamespace(value={}, file=lambda _key: Path(__file__))
with (
patch.object(
self.cutout_main,
"require_image_empty",
return_value=source,
),
patch.object(
self.cutout_main,
"_generated_color_result",
return_value=(
color_image,
np.ones((3, 4), dtype=np.float32),
selection_mask.bounds,
),
) as color_result,
patch.object(self.cutout_main, "create_cutout_shape") as create_shape,
):
message = operator.response(SimpleNamespace(), result)
color_result.assert_called_once_with(
__file__,
source,
selection_mask.bounds,
)
self.assertEqual(message, "Flat is ready")
content_values = create_shape.call_args.args[4]
np.testing.assert_array_equal(content_values, pure_values)
self.assertEqual(create_shape.call_args.args[5], selection_mask.bounds)
self.assertIs(create_shape.call_args.kwargs["color_image"], color_image)
self.assertEqual(create_shape.call_args.kwargs["alpha_threshold"], 0.25)
self.assertIs(create_shape.call_args.kwargs["fine_outline"], False)

15
tests/test_cutout_volume.py
View File

@@ -1,7 +1,6 @@
"""Contracts for smooth depth volume, editable presets and normal masks."""
from pathlib import Path
import sys
import bpy
import numpy as np
import pytest
@@ -9,7 +8,7 @@ import pytest
ROOT = Path(__file__).resolve().parents[1]
sys.path.insert(0, str(ROOT / "src"))
sys.path.insert(0, str(ROOT / "tools/node_assets"))
from build_cutout_modifiers import build_cutout_group
from cutout import build_cutout_group
from cutout_volume import build_volume_fields
from cutout_rounding import build_volume_rounding
from anyimage.operators.cutout_tool.object import load_cutout_node_group
@@ -66,7 +65,7 @@ def test_depth_rounding_options_are_consolidated():
@pytest.mark.parametrize("iterations", [0, 50])
def test_uniform_thickness_follows_the_tilted_front_normal(iterations):
from scipy.spatial import cKDTree
from test_depth_surface_nodes import surface, evaluated, assert_closed
from depth_surface_test_support import surface, evaluated, assert_closed
obj, set_value = surface(step=0, resolution=4)
image = bpy.data.images["Camera"]
pixels = np.array(image.pixels[:], dtype=np.float32).reshape(32, 64, 4)
@@ -85,7 +84,7 @@ def test_uniform_thickness_follows_the_tilted_front_normal(iterations):
def test_balloon_shell_converges_to_front_with_thickness():
from scipy.spatial import cKDTree
from test_depth_surface_nodes import surface, evaluated, assert_closed
from depth_surface_test_support import surface, evaluated, assert_closed
obj, set_value = surface(step=1, resolution=10)
xy = np.array([v.co[:2] for v in obj.data.vertices])
profile = 0.3 * np.sin(np.pi * xy[:, 0] / 2) * np.sin(np.pi * xy[:, 1])
@@ -198,7 +197,7 @@ def test_smooth_back_rejects_front_noise_and_preserves_center_depth():
def test_depth_cutout_normal_map_only_affects_front_faces():
from test_depth_surface_nodes import surface
from depth_surface_test_support import surface
obj, set_value = surface()
set_value("Thickness", 0.3)
bpy.context.view_layer.update()
@@ -217,7 +216,7 @@ def test_depth_cutout_normal_map_only_affects_front_faces():
@pytest.mark.parametrize("structure", [0, 1, 4])
@pytest.mark.parametrize("thickness", [0.0, 0.02, 0.1])
def test_uniform_front_inflation_scales_with_distance(thickness, structure):
from test_depth_surface_nodes import surface, evaluated, assert_closed
from depth_surface_test_support import surface, evaluated, assert_closed
obj, set_value = surface(step=0, resolution=10)
xy = np.array([v.co[:2] for v in obj.data.vertices])
profile = 0.3 * np.sin(np.pi * xy[:, 0] / 2) * np.sin(np.pi * xy[:, 1])
@@ -245,7 +244,7 @@ def test_uniform_front_inflation_scales_with_distance(thickness, structure):
def test_normal_smooth_controls_front_and_rear_directions():
from test_depth_surface_nodes import surface, evaluated
from depth_surface_test_support import surface, evaluated
obj, set_value = surface(step=1, resolution=10)
set_value("Front Inflation", 1.0)
xy = np.array([v.co[:2] for v in obj.data.vertices])
@@ -274,7 +273,7 @@ def test_normal_smooth_controls_front_and_rear_directions():
def test_balloon_suppression_is_stronger_without_a_startup_threshold():
from test_depth_surface_nodes import surface, evaluated
from depth_surface_test_support import surface, evaluated
obj, set_value = surface(step=0, resolution=10)
xy = np.array([v.co[:2] for v in obj.data.vertices])
profile = 0.3 * np.sin(np.pi * xy[:, 0] / 2) * np.sin(np.pi * xy[:, 1])

181
tests/test_depth_calibration.py Normal file
View File

@@ -0,0 +1,181 @@
import json
import sys
from pathlib import Path
from types import SimpleNamespace
import numpy as np
SOURCE_ROOT = Path(__file__).parents[1] / "src"
sys.path.insert(0, str(SOURCE_ROOT))
from anyimage.common.selection import SelectionMask
from anyimage.common.depth import (
depth_uniform_scale,
load_depth_metadata,
)
from anyimage.operators.cutout_tool import geometry as cutout_geometry
def test_depth_metadata_loader_uses_one_schema_for_plane_and_cutout(tmp_path):
path = tmp_path / "depth.json"
path.write_text(
json.dumps(
{
"image_size": [3, 2],
"intrinsics": np.eye(3).tolist(),
"reference_depth": 1.0,
}
),
encoding="utf-8",
)
metadata = load_depth_metadata(path)
assert metadata["image_size"] == (3, 2)
assert metadata["intrinsics"] == ((1.0, 0.0, 0.0), (0.0, 1.0, 0.0), (0.0, 0.0, 1.0))
assert metadata["reference_depth"] == 1.0
def test_base_plane_fit_is_stable_for_a_slender_diagonal_domain():
along = np.linspace(-1.0, 1.0, 101)
across = np.resize(np.asarray((-0.01, 0.0, 0.01)), len(along))
coordinates = np.column_stack(
(
along - across,
along + across,
)
)
values = 0.4 * along + np.sign(across) * 0.05
plane = cutout_geometry._stable_plane(coordinates, values)
selected_plane = coordinates @ plane[:2] + plane[2]
corners = np.asarray(((-1.0, -1.0), (-1.0, 1.0), (1.0, -1.0), (1.0, 1.0)))
corner_plane = corners @ plane[:2] + plane[2]
assert np.ptp(corner_plane) <= np.ptp(selected_plane) * 1.25
assert np.corrcoef(selected_plane, along)[0, 1] > 0.99
def test_depth_balloon_plane_calibration_matches_cutout_axes():
vertices = []
uv = []
camera_depth = np.empty((3, 3), dtype=np.float32)
for row in range(3):
local_y = 1.0 - row
for column in range(3):
local_x = column - 1.0
vertices.append((local_x, local_y, 0.0))
uv.append((column * 0.5, 1.0 - row * 0.5))
camera_depth[row, column] = 2.0 + 0.2 * local_x + 0.1 * local_y
reference, direction = cutout_geometry.depth_balloon_projection_parameters(
vertices,
uv,
camera_depth,
np.ones((3, 3), dtype=bool),
0.5,
)
expected_direction = np.asarray((0.1, 0.05, 1.0), dtype=np.float64)
expected_direction /= np.linalg.norm(expected_direction)
assert np.isclose(
reference,
np.percentile(camera_depth, 95.0) * 0.5,
atol=1e-6,
)
assert np.allclose(direction, expected_direction, atol=1e-6)
def test_depth_balloon_reference_uses_the_shared_upper_depth_percentile():
size = 10
row, column = np.indices((size, size))
camera_depth = np.full((size, size), 2.0, dtype=np.float32)
camera_depth[(row + column) % 2 == 0] = 2.2
vertices = []
uv = []
for y in range(size):
for x in range(size):
vertices.append((float(x), float(size - 1 - y), 0.0))
uv.append((x / (size - 1), 1.0 - y / (size - 1)))
reference, direction = cutout_geometry.depth_balloon_projection_parameters(
vertices,
uv,
camera_depth,
np.ones((size, size), dtype=bool),
1.0,
)
assert reference > float(np.median(camera_depth))
assert np.isclose(reference, 2.2, atol=1e-6)
assert np.allclose(direction, (0.0, 0.0, 1.0), atol=1e-6)
def test_depth_surface_projection_uses_one_scale_for_all_camera_axes():
points = np.zeros((51, 101, 3), dtype=np.float32)
points[..., 2] = np.linspace(1.0, 3.0, 51 * 101).reshape(51, 101)
metadata = {
"intrinsics": (
(100.0, 0.0, 50.0),
(0.0, 100.0, 25.0),
(0.0, 0.0, 1.0),
),
"image_size": (101, 51),
"reference_depth": 2.0,
}
uniform_scale = depth_uniform_scale(
SimpleNamespace(),
metadata,
bounds=(-2.0, 2.0, -1.0, 1.0),
)
assert uniform_scale == 2.0
reference_depth = cutout_geometry.metric_depth_reference(
points[..., 2],
np.ones((51, 101), dtype=bool),
uniform_scale,
)
assert np.isclose(reference_depth, 5.8, atol=1e-6)
def test_cutout_threshold_controls_contour_and_calibration():
alpha = np.zeros((12, 16), dtype=np.float32)
alpha[2:10, 2:14] = 0.2
alpha[4:8, 5:11] = 1.0
for threshold, lower, upper in [
(0.0, (2, 2), (14, 10)),
(0.1, (2, 2), (14, 10)),
(0.5, (5, 4), (11, 8)),
(1.0, (5, 4), (11, 8)),
]:
contours = cutout_geometry.extract_cutout_contours(
alpha, 64, threshold=threshold
)
contour = np.asarray(contours.components[0][0])
np.testing.assert_allclose(contour.min(axis=0), lower)
np.testing.assert_allclose(contour.max(axis=0), upper)
valid, _ = cutout_geometry.selection_calibration(
alpha, alpha.shape, threshold=threshold
)
np.testing.assert_array_equal(valid, (alpha > 0) & (alpha >= threshold))
def test_cutout_depth_calibration_uses_five_percent_transparency_threshold():
assert not cutout_geometry.use_adaptive_depth_calibration(0.0499)
assert cutout_geometry.use_adaptive_depth_calibration(0.05)
def test_selection_calibration_uses_the_final_selection_mask():
selection_mask = SelectionMask(
np.asarray(((1.0, 1.0), (1.0, 0.0)), dtype=np.float32),
(0, 0, 2, 2),
)
valid, transparent_fraction = cutout_geometry.selection_calibration(
selection_mask.values,
(2, 2),
)
assert np.array_equal(valid, ((True, True), (True, False)))
assert transparent_fraction == 0.25

337
tests/test_depth_plane_modes.py Normal file
View File

@@ -0,0 +1,337 @@
"""Evaluate complete-image projection and rectangular surface topology."""
import sys
from collections import Counter
from pathlib import Path
import bpy
import numpy as np
import pytest
ROOT = Path(__file__).parents[1]
sys.path[:0] = [str(ROOT / "src"), str(ROOT / "tools" / "node_assets")]
from image_plane import build_image_plane_group
from depth_plane import build_depth_plane_group
from node_layout import validate_node_layout
@pytest.fixture(params=("definition", "asset"))
def surface(request):
bpy.ops.wm.read_factory_settings(use_empty=True)
if request.param == "asset":
with bpy.data.libraries.load(str(ROOT / "src/anyimage/assets/O_AnyImage.blend")) as (_, target):
target.node_groups = ["O Image Depth Plane"]
group = target.node_groups[0]
plane = group.nodes["Group"].node_tree
else:
plane = build_image_plane_group()
group = build_depth_plane_group(plane)
mesh = bpy.data.meshes.new("Rectangle")
mesh.from_pydata([(-2, -1, 0), (2, -1, 0), (2, 1, 0), (-2, 1, 0)], [], [(0, 1, 2, 3)])
material = bpy.data.materials.new("Image")
mesh.materials.append(material)
obj = bpy.data.objects.new("Surface", mesh)
bpy.context.collection.objects.link(obj)
modifier = obj.modifiers.new("Surface", "NODES")
modifier.node_group = group
image = bpy.data.images.new("Camera", width=16, height=8, float_buffer=True)
image.colorspace_settings.name = "Non-Color"
image.alpha_mode = "CHANNEL_PACKED"
yy, xx = np.mgrid[:8, :16]
# Camera principal point is deliberately off center; Y points down.
pixels = np.ones((8, 16, 4), dtype=np.float32)
pixels[..., 0] = ((xx + 0.5) - 5) / 4
pixels[..., 1] = (3 - (yy + 0.5)) / 4
pixels[..., 2] = 2 + (xx + 0.5) / 16
pixels[..., 3] = xx >= 8
image.pixels.foreach_set(pixels.ravel())
inputs = {item.name: item for item in group.interface.items_tree
if item.item_type == "SOCKET" and item.in_out == "INPUT"}
thickness_inputs = [item for item in group.interface.items_tree
if item.item_type == "SOCKET" and item.in_out == "INPUT"
and item.name == "Thickness"]
inputs["Relief Thickness"] = thickness_inputs[0]
def set_value(name, value):
modifier[inputs[name].identifier] = value
obj.update_tag(refresh={"DATA"})
bpy.context.view_layer.update()
for name, value in (("Mode", 1), ("Subdivide", 2), ("Depth Image", image),
("Uniform Scale", 0.5), ("Reference Depth", 1.2),
("Depth Scale", 0.0), ("Thickness", 0.0)):
set_value(name, value)
return obj, set_value, plane, inputs, image
def evaluated(obj):
result = obj.evaluated_get(bpy.context.evaluated_depsgraph_get())
mesh = result.to_mesh()
try:
return (
np.array([v.co[:] for v in mesh.vertices]),
[tuple(face.vertices) for face in mesh.polygons],
np.array([value.uv[:] for value in mesh.uv_layers["UVMap"].data]),
[material.name if material else None for material in mesh.materials],
[face.material_index for face in mesh.polygons],
[attr.name for attr in mesh.attributes],
)
finally:
result.to_mesh_clear()
@pytest.mark.parametrize("subdivide", [0, 1, 3])
def test_zero_depth_matches_plane_geometry_and_uv(surface, subdivide):
obj, set_value, plane, inputs, _ = surface
set_value("Subdivide", subdivide)
projected = evaluated(obj)
modifier = obj.modifiers[0]
modifier.node_group = plane
for item in plane.interface.items_tree:
if item.item_type == "SOCKET" and item.in_out == "INPUT":
if item.name == "Subdivide":
modifier[item.identifier] = subdivide
elif item.name == "Thickness":
modifier[item.identifier] = 0.0
obj.update_tag(refresh={"DATA"})
bpy.context.view_layer.update()
regular = evaluated(obj)
np.testing.assert_allclose(projected[0], regular[0], atol=1e-6)
assert projected[1] == regular[1]
np.testing.assert_allclose(projected[2], regular[2], atol=1e-6)
assert projected[3:5] == regular[3:5]
@pytest.mark.parametrize("strength", [0, 0.5, 1, 2])
def test_camera_projection_and_reference_depth(surface, strength):
obj, set_value, _, _, _ = surface
base = evaluated(obj)
uv = (base[0][:, :2] + (2, 1)) / (4, 2)
texel = np.clip(uv * (16, 8), (0.5, 0.5), (15.5, 7.5))
camera = np.column_stack(((texel[:, 0] - 5) / 8,
(texel[:, 1] - 3) / 8,
1.2 - (2 + texel[:, 0] / 16) * 0.5))
expected = base[0] * (1 - strength) + camera * strength
set_value("Depth Scale", float(strength))
projected = evaluated(obj)
np.testing.assert_allclose(projected[0], expected, atol=1e-6)
assert projected[1] == base[1]
np.testing.assert_allclose(projected[2], base[2])
set_value("Reference Depth", 1.7)
shifted = evaluated(obj)[0]
np.testing.assert_allclose(shifted - projected[0],
np.tile((0, 0, 0.5 * strength), (len(shifted), 1)), atol=1e-6)
@pytest.mark.parametrize("thickness", [0.000001, 0.2, 2.0])
def test_shell_keeps_front_and_closes_rectangular_boundary(surface, thickness):
obj, set_value, _, _, _ = surface
set_value("Depth Scale", 1.0)
front = evaluated(obj)[0]
set_value("Thickness", thickness)
points, faces, uv, materials, indices, attributes = evaluated(obj)
assert len(points) == 2 * len(front)
edges = Counter(tuple(sorted((a, b))) for f in faces for a, b in zip(f, (*f[1:], f[0])))
assert set(edges.values()) == {2}
normal = np.array((0.25, 0, 1.0))
normal /= np.linalg.norm(normal)
expected = np.vstack((front, front - normal * thickness))
from scipy.spatial import cKDTree
assert cKDTree(points).query(expected)[0].max() < 1e-6
assert np.isfinite(points).all() and np.isfinite(uv).all()
assert materials[0] == "Image" and set(indices) == {0}
assert not any(name.startswith("o_depth_") for name in attributes)
def test_depth_jump_keeps_rectangular_faces(surface):
obj, set_value, _, _, image = surface
base = evaluated(obj)
pixels = np.ones((8, 16, 4), dtype=np.float32)
pixels[:, 8:, 2] = 10
image.pixels.foreach_set(pixels.ravel())
image.update()
set_value("Depth Scale", 1.0)
result = evaluated(obj)
assert result[1] == base[1]
assert len(result[0]) == len(base[0])
assert np.ptp(result[0][:, 2]) > 4
@pytest.mark.parametrize("thickness", [0.0, 0.4])
def test_relief_reads_z_and_keeps_the_base(surface, thickness):
obj, set_value, _, inputs, image = surface
set_value("Mode", 0)
set_value("Relief Thickness", thickness)
set_value("Depth Scale", 1.0)
set_value("Reference Depth", 2.0)
pixels = np.ones((8, 16, 4), dtype=np.float32)
pixels[..., 0], pixels[..., 1], pixels[..., 2] = -9, 13, 2
image.pixels.foreach_set(pixels.ravel())
image.update()
set_value("Uniform Scale", 0.5)
first = evaluated(obj)
assert first[0][:, 2].max() == pytest.approx(thickness + 1)
assert first[0][:, 2].min() == pytest.approx(0 if thickness else 1)
pixels[..., 0], pixels[..., 1] = 31, -22
image.pixels.foreach_set(pixels.ravel())
image.update()
set_value("Normal Smooth", 0)
np.testing.assert_allclose(evaluated(obj)[0], first[0])
original_images = set(bpy.data.images)
set_value("Mode", 1)
camera_points = evaluated(obj)[0]
assert camera_points.shape != first[0].shape or not np.allclose(camera_points, first[0])
set_value("Mode", 0)
np.testing.assert_allclose(evaluated(obj)[0], first[0])
assert set(bpy.data.images) == original_images
assert obj.modifiers[0][inputs["Relief Thickness"].identifier] == pytest.approx(thickness)
@pytest.mark.parametrize("normal_smooth", [0, 2, 50])
def test_curved_shell_smooths_normals_without_shrinking_thickness(surface, normal_smooth):
from scipy.spatial import cKDTree
obj, set_value, _, _, image = surface
yy, xx = np.mgrid[:8, :16]
pixels = np.ones((8, 16, 4), dtype=np.float32)
pixels[..., 0] = (xx + 0.5 - 8) / 4
pixels[..., 1] = (4 - yy - 0.5) / 4
pixels[..., 2] = 2 + ((xx + 0.5 - 8) / 8) ** 2 + 0.12 * (xx % 2)
image.pixels.foreach_set(pixels.ravel())
image.update()
set_value("Subdivide", 3)
set_value("Depth Scale", 1.0)
front, faces, *_ = evaluated(obj)
result = obj.evaluated_get(bpy.context.evaluated_depsgraph_get())
mesh = result.to_mesh()
try:
raw_normals = np.array([v.normal[:] for v in mesh.vertices])
finally:
result.to_mesh_clear()
set_value("Normal Smooth", normal_smooth)
thickness = 0.001
set_value("Thickness", thickness)
points = evaluated(obj)[0]
distance, source = cKDTree(front).query(points)
rear = distance > thickness * 0.5
assert rear.sum() == len(front)
assert len(set(source[rear])) == len(front)
np.testing.assert_allclose(distance[rear], thickness, atol=1e-6)
normals = np.empty_like(front)
normals[source[rear]] = (front[source[rear]] - points[rear]) / thickness
assert np.all(np.sum(normals * raw_normals, axis=1) > 0)
edges = np.array(list({tuple(sorted((a, b))) for f in faces
for a, b in zip(f, (*f[1:], f[0]))}))
roughness = lambda n: np.mean(np.linalg.norm(n[edges[:, 0]] - n[edges[:, 1]], axis=1))
if normal_smooth == 0:
np.testing.assert_allclose(normals, raw_normals, atol=2e-4)
else:
assert roughness(normals) < roughness(raw_normals)
@pytest.mark.parametrize("mode,active,inactive", [
(0, "Relief Thickness", "Thickness"),
(1, "Thickness", "Relief Thickness"),
])
def test_modes_keep_independent_thickness_values(surface, mode, active, inactive):
obj, set_value, _, inputs, _ = surface
set_value("Depth Scale", 1.0)
set_value("Mode", mode)
set_value(active, 0.2)
before = evaluated(obj)
set_value(inactive, 0.7)
np.testing.assert_allclose(evaluated(obj)[0], before[0])
assert evaluated(obj)[1] == before[1]
set_value("Mode", 1 - mode)
set_value("Mode", mode)
np.testing.assert_allclose(evaluated(obj)[0], before[0])
assert obj.modifiers[0][inputs[active].identifier] == pytest.approx(0.2)
assert obj.modifiers[0][inputs[inactive].identifier] == pytest.approx(0.7)
def test_surface_interface_and_authored_layout(surface):
obj, _, _, inputs, _ = surface
assert list(inputs) == ["Geometry", "Mode", "Subdivide", "Thickness", "Depth Scale",
"Reference Depth", "Normal Smooth", "Uniform Scale", "Depth Image", "Relief Thickness"]
assert [name for name, item in inputs.items() if not item.hide_in_modifier][1:] == [
"Mode", "Subdivide", "Thickness", "Depth Scale", "Reference Depth", "Normal Smooth", "Relief Thickness"]
assert inputs["Normal Smooth"].default_value == 50
assert inputs["Normal Smooth"].min_value == 0
assert inputs["Normal Smooth"].max_value == 50
assert inputs["Normal Smooth"].parent.name == "Options"
assert inputs["Thickness"].default_value == inputs["Relief Thickness"].default_value == 0
assert inputs["Thickness"].identifier != inputs["Relief Thickness"].identifier
actual_inputs = [item.name for item in obj.modifiers[0].node_group.interface.items_tree
if item.item_type == "SOCKET" and item.in_out == "INPUT"]
assert actual_inputs == ["Geometry", "Mode", "Subdivide", "Thickness", "Thickness",
"Depth Scale", "Reference Depth", "Normal Smooth", "Uniform Scale", "Depth Image"]
assert inputs["Mode"].default_value == "Relief"
assert inputs["Subdivide"].default_value == 6
assert inputs["Depth Scale"].default_value == 1
assert all(item.structure_type == "SINGLE" for item in inputs.values())
assert inputs["Thickness"].subtype == inputs["Reference Depth"].subtype == "DISTANCE"
validate_node_layout(obj.modifiers[0].node_group)
def test_conversion_preserves_offset_world_placement_and_initializes_inputs(surface, monkeypatch):
from anyimage.operators.convert_to_plane import object as conversion
from anyimage.common.image import image_empty_bounds
from mathutils import Matrix, Vector
obj, _, _, inputs, image = surface
group = obj.modifiers[0].node_group
monkeypatch.setattr(conversion, "load_node_group", lambda name: group)
source = bpy.data.objects.new("Offset Image", None)
source.empty_display_type = "IMAGE"
source.data = image
source.empty_display_size = 4
source.empty_image_offset = (-0.2, -0.8)
source.matrix_world = Matrix.Translation((3, -2, 5)) @ Matrix.Rotation(0.7, 4, "Y") @ Matrix.Diagonal((2, 3, 1, 1))
bpy.context.collection.objects.link(source)
bounds = image_empty_bounds(source)
matrix = source.matrix_world.copy()
material = bpy.data.materials.new("Conversion")
metadata = {"image_size": (16, 8), "reference_depth": 2.0,
"intrinsics": ((4, 0, 5), (0, 4, 3), (0, 0, 1))}
result = conversion.create_depth_plane_object(
bpy.context, source, 2, material, image, metadata)
assert result.name == "Offset Image"
assert result["anyimage_mesh_shape"] == "DEPTH_PLANE"
assert bpy.context.view_layer.objects.active == result
expected = sorted(tuple(matrix @ Vector((x, y, 0)))
for x in bounds[:2] for y in bounds[2:])
actual = sorted(tuple(result.matrix_world @ v.co) for v in result.data.vertices)
np.testing.assert_allclose(actual, expected, atol=1e-6)
center = matrix @ Vector(((bounds[0] + bounds[1]) / 2, (bounds[2] + bounds[3]) / 2, 0))
np.testing.assert_allclose(result.matrix_world.translation, center, atol=1e-6)
modifier = result.modifiers[0]
assert modifier[inputs["Thickness"].identifier] == 0
assert modifier[inputs["Relief Thickness"].identifier] == 0
assert modifier[inputs["Depth Image"].identifier] == image
assert modifier[inputs["Reference Depth"].identifier] == pytest.approx(
2 * modifier[inputs["Uniform Scale"].identifier])
assert result.data.materials[0] == material
def test_failed_modifier_initialization_removes_partial_object(surface, monkeypatch):
from anyimage.operators.convert_to_plane import object as conversion
obj, _, _, _, image = surface
monkeypatch.setattr(conversion, "load_node_group", lambda name: obj.modifiers[0].node_group)
source = bpy.data.objects.new("Source", None)
source.empty_display_type = "IMAGE"
source.data = image
bpy.context.collection.objects.link(source)
material = bpy.data.materials.new("Unused")
before_objects, before_meshes = set(bpy.data.objects), set(bpy.data.meshes)
def fail(*args):
raise RuntimeError("input failed")
monkeypatch.setattr(conversion, "set_modifier_input", fail)
metadata = {"image_size": (16, 8), "reference_depth": 2,
"intrinsics": ((4, 0, 5), (0, 4, 3), (0, 0, 1))}
with pytest.raises(RuntimeError, match="input failed"):
conversion.create_depth_plane_object(bpy.context, source, 2, material, image, metadata)
assert set(bpy.data.objects) == before_objects
assert set(bpy.data.meshes) == before_meshes
assert material.users == 0

110
tests/test_depth_surface_nodes.py → tests/test_depth_surface_split.py
View File

@@ -3,85 +3,19 @@
import sys
from pathlib import Path
from collections import Counter
import bpy
import bmesh
import numpy as np
import pytest
ROOT = Path(__file__).parents[1]
sys.path.insert(0, str(ROOT / "src"))
sys.path.insert(0, str(ROOT / "tools" / "node_assets"))
from build_cutout_modifiers import build_cutout_group
def surface(*, step=6.0, size=1.0, resolution=2, wrapper=False):
bpy.ops.wm.read_factory_settings(use_empty=True)
group = build_cutout_group("DEPTH_SOLID")
# Two columns with a complete depth jump along their common boundary.
points = [
(x * size / resolution, y * size / resolution, 0)
for y in range(resolution + 1)
for x in range(2 * resolution + 1)
]
stride = 2 * resolution + 1
faces = [
(
y * stride + x,
y * stride + x + 1,
(y + 1) * stride + x + 1,
(y + 1) * stride + x,
)
for y in range(resolution)
for x in range(2 * resolution)
]
mesh = bpy.data.meshes.new("Step")
mesh.from_pydata(points, [], faces)
uv = mesh.uv_layers.new(name="UVMap")
for loop in mesh.loops:
x, y, _ = points[loop.vertex_index]
uv.data[loop.index].uv = (x / (2 * size), y / size)
obj = bpy.data.objects.new("Step", mesh)
bpy.context.collection.objects.link(obj)
image = bpy.data.images.new("Camera", width=64, height=32, float_buffer=True)
image.colorspace_settings.name = "Non-Color"
image.alpha_mode = "CHANNEL_PACKED"
yy, xx = np.mgrid[:32, :64]
pixels = np.ones((32, 64, 4), np.float32)
pixels[..., 0] = (xx + 0.5) / 32
pixels[..., 1] = -(yy + 0.5) / 32
pixels[..., 2] = 1 + (xx >= 32) * step
image.pixels.foreach_set(pixels.ravel())
modifier = obj.modifiers.new("Depth Surface", "NODES")
modifier.node_group = group
inputs = {
s.name: s.identifier
for s in group.interface.items_tree
if s.item_type == "SOCKET" and s.in_out == "INPUT"
}
def set_value(name, value):
modifier[inputs[name]] = value
obj.update_tag(refresh={"DATA"})
bpy.context.view_layer.update()
for name, value in (
("Depth Image", image),
("Uniform Scale", size),
("Reference Depth", size),
("Depth Scale", 1.0),
("Thickness", 0.0),
("Front Inflation", 0.0),
("Split Threshold", 0.0),
("Mode", 1),
("Smooth", 0),
):
set_value(name, value)
if wrapper:
set_value("Cleanup Threshold", 0.0)
set_value("Inward Axis", 0)
set_value("Smooth", 0)
return obj, set_value
from cutout import build_cutout_group
from depth_surface_test_support import assert_closed, component_count, evaluated, surface
@pytest.mark.parametrize("mode", [0, 1])
@@ -112,24 +46,6 @@ def test_shell_walls_directly_connect_caps(mode, split):
assert_closed(shell_faces)
def evaluated(obj):
result = obj.evaluated_get(bpy.context.evaluated_depsgraph_get())
mesh = result.to_mesh()
try:
return np.array([v.co[:] for v in mesh.vertices]), [
tuple(p.vertices) for p in mesh.polygons
]
finally:
result.to_mesh_clear()
def assert_closed(faces):
edges = Counter(
tuple(sorted((a, b))) for f in faces for a, b in zip(f, (*f[1:], f[0]))
)
assert edges and set(edges.values()) == {2}, Counter(edges.values())
@pytest.mark.parametrize("thickness", [0.1, 1.4])
@pytest.mark.parametrize("split", [0.0, 0.5])
def test_surface_caps_and_segmented_walls_are_connected(thickness, split):
@@ -214,24 +130,6 @@ def test_full_wrapper_smooth_keeps_closed_shell():
assert_closed(faces)
def component_count(faces):
adjacency = {}
for face in faces:
for a, b in zip(face, (*face[1:], face[0])):
adjacency.setdefault(a, set()).add(b)
adjacency.setdefault(b, set()).add(a)
remaining = set(adjacency)
count = 0
while remaining:
count += 1
pending = [remaining.pop()]
while pending:
for neighbor in adjacency[pending.pop()] & remaining:
remaining.remove(neighbor)
pending.append(neighbor)
return count
def test_coincident_independent_sheets_do_not_weld_together():
obj, set_value = surface(step=0.0)
mesh = obj.data

0
tests/test_docs.py → tests/test_docs_command.py
View File

36
tests/test_edge_depth.py
View File

@@ -2,7 +2,6 @@ import sys
from pathlib import Path
from types import SimpleNamespace
from unittest.mock import patch
import numpy as np
import pytest
from PIL import Image
@@ -160,21 +159,18 @@ def test_alpha_resize_retains_threshold_values(tmp_path):
@pytest.mark.parametrize(
"depth_mode,normal_mode,correct",
"generate_depth,normal_mode,correct",
[
("VECTOR", "OBJECT", True),
("VECTOR", "TANGENT", True),
("Z", "TANGENT", True),
("Z", "TANGENT", False),
("NONE", "OBJECT", True),
(True, "OBJECT", True),
(True, "TANGENT", True),
(True, "TANGENT", False),
(False, "OBJECT", True),
],
)
def test_artifacts_use_corrected_depth_and_original_normals(
tmp_path, depth_mode, normal_mode, correct
tmp_path, generate_depth, normal_mode, correct
):
frame, alpha = silhouette()
if depth_mode == "Z":
frame.points = None
rgba = np.full((*alpha.shape, 4), 255, np.uint8)
rgba[..., 3] = np.round(alpha * 255).astype(np.uint8)
path = tmp_path / "source.png"
@@ -188,12 +184,9 @@ def test_artifacts_use_corrected_depth_and_original_normals(
with (
patch.object(artifacts.moge2, "infer", return_value=[frame]),
patch.object(
artifacts, "write_vector_depth_texture",
return_value=tmp_path / "vector.exr",
) as vector,
patch.object(
artifacts, "write_z_depth_texture", return_value=tmp_path / "z.exr"
) as z,
artifacts, "write_depth_texture",
return_value=tmp_path / "depth.exr",
) as depth,
patch.object(
artifacts, "write_depth_metadata", return_value=tmp_path / "depth.json"
) as metadata,
@@ -203,13 +196,13 @@ def test_artifacts_use_corrected_depth_and_original_normals(
patch.object(artifacts, "extend_edge_depth", wraps=extend_edge_depth) as repair,
):
artifacts.generate_moge2_artifacts(
context, {}, path, depth_mode=depth_mode, normal_mode=normal_mode,
context, {}, path, generate_depth=generate_depth, normal_mode=normal_mode,
correct_edge_depth=correct,
)
assert normal.call_args.args[0] is frame
assert repair.call_count == int(correct and depth_mode != "NONE")
if depth_mode != "NONE":
written_frame = (vector if depth_mode == "VECTOR" else z).call_args.args[0]
assert repair.call_count == int(correct and generate_depth)
if generate_depth:
written_frame = depth.call_args.args[0]
assert metadata.call_args.args[0] is written_frame
if correct:
assert written_frame.depth[12, 24] == 2
@@ -217,5 +210,4 @@ def test_artifacts_use_corrected_depth_and_original_normals(
assert written_frame is frame
else:
metadata.assert_not_called()
vector.assert_not_called()
z.assert_not_called()
depth.assert_not_called()

1
tests/test_empty_image_edits.py
View File

@@ -1,7 +1,6 @@
import sys
from pathlib import Path
from unittest.mock import patch
import bpy
import numpy as np
import pytest

63
tests/test_extension_packaging.py
View File

@@ -4,7 +4,6 @@ import tomllib
import unittest
from pathlib import Path
from unittest.mock import patch
from zipfile import ZipFile
PROJECT_ROOT = Path(__file__).resolve().parents[1]
@@ -19,9 +18,34 @@ def load_pack():
return module
class ArchiveContents:
def __init__(self, path, records):
self.files = {}
records[path] = self
def __enter__(self):
return self
def __exit__(self, *args):
pass
def write(self, source, target):
self.files[Path(target).as_posix()] = None
def writestr(self, target, value):
self.files[target] = value.encode()
def namelist(self):
return list(self.files)
def read(self, name):
return self.files[name]
class ExtensionPackagingTest(unittest.TestCase):
def test_builds_platform_specific_extension_zips(self):
def test_selects_platform_archive_contents_and_manifest(self):
pack = load_pack()
records = {}
with tempfile.TemporaryDirectory() as directory:
directory_path = Path(directory)
wheel_path = directory_path / "blendjob-0.1.16-py3-none-any.whl"
@@ -29,10 +53,9 @@ class ExtensionPackagingTest(unittest.TestCase):
original_dist = pack.DIST_DIR
pack.DIST_DIR = directory_path / "dist"
try:
with patch.object(
pack,
"download_wheels",
return_value=[wheel_path],
with (
patch.object(pack, "download_wheels", return_value=[wheel_path]),
patch.object(pack, "ZipFile", side_effect=lambda path, *args: ArchiveContents(path, records)),
):
output_paths = pack.build_all()
finally:
@@ -47,7 +70,7 @@ class ExtensionPackagingTest(unittest.TestCase):
)
for output_path in output_paths:
platform = output_path.stem.split(".")[-1]
with ZipFile(output_path) as archive:
with records[output_path] as archive:
names = set(archive.namelist())
manifest = archive.read("blender_manifest.toml").decode("utf-8")
@@ -77,7 +100,7 @@ class ExtensionPackagingTest(unittest.TestCase):
"operators/convert_to_plane/__init__.py",
"operators/convert_to_plane/object.py",
"operators/convert_to_plane/conversion.py",
"operators/convert_to_plane/mesh_plane.py",
"operators/convert_to_plane/image_plane.py",
"operators/image_edit_tool/__init__.py",
"operators/image_edit_tool/frame.py",
"operators/image_edit_tool/mask.py",
@@ -116,13 +139,15 @@ class ExtensionPackagingTest(unittest.TestCase):
self.assertNotIn("server/jobs/prepare_geometry_data.py", names)
self.assertNotIn("server/geometry/cutout.py", names)
self.assertIn("server/models/ben2.py", names)
self.assertIn("server/models/da3.py", names)
self.assertNotIn("server/models/da3.py", names)
self.assertIn("server/models/moge2.py", names)
self.assertIn("server/jobs/debug_predictions.py", names)
self.assertNotIn("server/jobs/debug_predictions.py", names)
self.assertNotIn("operators/debug.py", names)
self.assertFalse(any(name.startswith("server/outputs/") for name in names))
self.assertIn("server/model_catalog.py", names)
self.assertIn("server/model_download.py", names)
self.assertIn("server/models/onnx_ben2.py", names)
self.assertIn("server/models/onnx_da3.py", names)
self.assertNotIn("server/models/onnx_da3.py", names)
self.assertIn("server/models/onnx_moge2.py", names)
self.assertIn("server/models/onnx_upscale.py", names)
self.assertIn("server/models/upscale.py", names)
@@ -176,6 +201,7 @@ class ExtensionPackagingTest(unittest.TestCase):
def test_downloads_both_python_versions_for_each_platform(self):
pack = load_pack()
records = {}
for platform, config in pack.PLATFORM_CONFIGS.items():
with self.subTest(platform=platform), tempfile.TemporaryDirectory() as directory:
destination = Path(directory)
@@ -222,9 +248,12 @@ class ExtensionPackagingTest(unittest.TestCase):
[wheel.name for wheel in wheels], sorted({wheel.name for wheel in wheels})
)
self.assertFalse(any(wheel.name.startswith("numpy-") for wheel in wheels))
with patch.object(pack, "DIST_DIR", destination / "dist"):
with (
patch.object(pack, "DIST_DIR", destination / "dist"),
patch.object(pack, "ZipFile", side_effect=lambda path, *args: ArchiveContents(path, records)),
):
output = pack._build_extension(platform, wheels)
with ZipFile(output) as archive:
with records[output] as archive:
manifest = tomllib.loads(archive.read("blender_manifest.toml").decode())
expected = {f"wheels/{wheel.name}" for wheel in wheels}
self.assertEqual({name[2:] for name in manifest["wheels"]}, expected)
@@ -321,13 +350,17 @@ class ExtensionPackagingTest(unittest.TestCase):
def test_archive_excludes_blender_numpy(self):
pack = load_pack()
records = {}
with tempfile.TemporaryDirectory() as directory:
root = Path(directory)
wheel = root / "numpy-1.26.4-cp311-cp311-win_amd64.whl"
wheel.write_bytes(b"numpy")
with patch.object(pack, "DIST_DIR", root / "dist"):
with (
patch.object(pack, "DIST_DIR", root / "dist"),
patch.object(pack, "ZipFile", side_effect=lambda path, *args: ArchiveContents(path, records)),
):
output = pack._build_extension("windows-x64", [wheel])
with ZipFile(output) as archive:
with records[output] as archive:
self.assertFalse(any("numpy-" in name for name in archive.namelist()))
self.assertEqual(
tomllib.loads(archive.read("blender_manifest.toml").decode())[

1
tests/test_frame_antialiasing.py
View File

@@ -1,7 +1,6 @@
import sys
from pathlib import Path
from unittest.mock import patch
import numpy as np
import pytest

1
tests/test_frame_projection.py
View File

@@ -1,6 +1,5 @@
import sys
from pathlib import Path
import numpy as np
import pytest

Some files were not shown because too many files have changed in this diff Show More