icrdr/AnyImage
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Add new functionality for image rectification and depth metadata handling

Browse Source 
- Implemented a preview tool for rectifying images with interactive aspect controls in `preview.py`.
- Added tests for depth metadata loading to ensure consistent schema usage in `test_depth_metadata.py`.
- Developed comprehensive tests for cutout calibration processes in `test_cutout_calibration.py`, including stability checks and depth calibration.
- Created tests for frame compositing operations in `test_frame_compositing.py`, ensuring correct handling of depth and transparency.
- Introduced material slot inheritance functionality in `material_slots.py` to manage geometry material assignments.
- Built shared shader nodes for material processing in `shader.py`, including roughness and bump mapping setups.
This commit is contained in:
马南
2026-09-12 19:20:06 +08:00
parent e8268504c2
commit 938d95db0f
175 changed files with 3812 additions and 6864 deletions
Download Patch File Download Diff File Expand all files Collapse all files

3
.env.example Normal file
View File

@@ -0,0 +1,3 @@
# 复制为 .env，并填写本机 Blender 可执行文件路径。
# Windows 路径可使用正斜杠。
BLENDER_BIN="C:/Program Files/Blender Foundation/Blender 4.5/blender.exe"

4
AGENTS.md
View File

@@ -7,7 +7,7 @@
- 说明性内容以中文为主，专有名词和约定俗成的表达可使用英文 - 说明性内容以中文为主，专有名词和约定俗成的表达可使用英文
- 修改代码、依赖或执行方式后， - 修改代码、依赖或执行方式后，
- 不保留旧路径、转发函数、兼容层或胶水代码 - 不保留旧路径、转发函数、兼容层或胶水代码
- 更新并运行相关测试，但不主动同步说明文档、构建文档或打包扩展 - 仅更新并运行相关测试，但不主动同步说明文档、构建文档或打包扩展
- 优先选择依赖少、直接、易验证的实现 - 优先选择依赖少、直接、易验证的实现
- 复用来自 `common` 的函数，避免重复造轮子 - 复用来自 `common` 的函数，避免重复造轮子
- 确保文件名/文件夹名和内容边界一致 - 确保文件名/文件夹名和内容边界一致
@@ -75,7 +75,7 @@ uv run sync-models
- `README.md` 面向用户，不在其中堆叠内部细节。实现原理放入 `docs/internals` 对应主题 - `README.md` 面向用户，不在其中堆叠内部细节。实现原理放入 `docs/internals` 对应主题
- 文档只正面描述当前主题的职责与实现，不记录无关模块、未采用路径、历史差异、边界辩解、“不做什么”的排除性信息。 - 文档只正面描述当前主题的职责与实现，不记录无关模块、未采用路径、历史差异、边界辩解、“不做什么”的排除性信息。
- 实现原理用流程图替代文字，Mermaid 默认使用从上到下 - 根据内容选择最清晰的表达方式，优先简洁文字；复杂流程或关系在有助于理解时使用图表。Mermaid 流程图默认从上到下。
## Blender 开发规范 ## Blender 开发规范

13
docs/internals/architecture.md
View File

@@ -3,7 +3,7 @@
| 模块 | 职责 | | 模块 | 职责 |
| --- | --- | | --- | --- |
| `__init__.py` | 注册类型、工具、菜单、快捷键与 Runtime | | `__init__.py` | 注册类型、工具、菜单、快捷键与 Runtime |
| `menu.py`、`panel.py` | Image Empty 上下文菜单与 Job Server 面板 | | `menu.py`、`panel.py` | Image Empty 与 Shader Image Texture 上下文菜单与 Job Server 面板 |
| `preferences.py` | 扩展设置及读取入口 | | `preferences.py` | 扩展设置及读取入口 |
| `properties.py` | Scene 手势状态、视频帧数、模型状态与枚举 | | `properties.py` | Scene 手势状态、视频帧数、模型状态与枚举 |
| `common/` | Blender 侧公共能力 | | `common/` | Blender 侧公共能力 |
@@ -12,15 +12,6 @@
## 注册与配置 ## 注册与配置
```mermaid
flowchart TD
A[register] --> B[按 CLASSES 顺序注册类型]
B --> C[注册 Frame、Mask、Rectify、Cutout 工具]
C --> D[挂载 Scene.anyimage_settings]
D --> E[挂载 View3D 与 Outliner 菜单及剪贴板快捷键]
E --> F[注册 JobRuntime]
```
注销先关闭 Runtime 和交互入口，再移除 Scene 属性，按 `CLASSES` 逆序注销类型。设备、存储目录、模型、材质显示适配和功能上限统一通过 `preferences.py` 读取。 注销先关闭 Runtime 和交互入口，再移除 Scene 属性，按 `CLASSES` 逆序注销类型。设备、存储目录、模型、材质显示适配和功能上限统一通过 `preferences.py` 读取。
**Maximum AI Input Size** 限制模型分析图和 Upscale 可提交源图；**Maximum Frame Resolution** 限制 Frame 输出；**Maximum Cutout Mesh Resolution** 限制轮廓网格采样。三者各自作用于对应数据阶段。 **Maximum AI Input Size** 限制模型分析图和 Upscale 可提交源图；**Maximum Frame Resolution** 限制 Frame 输出；**Maximum Cutout Mesh Resolution** 限制轮廓网格采样。三者各自作用于对应数据阶段。
@@ -40,4 +31,4 @@ flowchart TD
Plane、Frame、Mask、Rectify 和无需 AI 的 Cutout 在 Blender 内执行。Server 的 `jobs/` 编排任务，`models/` 负责模型适配，`geometry/` 处理深度与法线产物。Server 模块通过内部相对导入共享能力，Blender 侧通过 `common/` 复用。 Plane、Frame、Mask、Rectify 和无需 AI 的 Cutout 在 Blender 内执行。Server 的 `jobs/` 编排任务，`models/` 负责模型适配，`geometry/` 处理深度与法线产物。Server 模块通过内部相对导入共享能力，Blender 侧通过 `common/` 复用。
MoGe-2 输出统一为 `GeometryFrame`：包含 Depth、Validity、Intrinsics，以及按需提供的 Normal 和 Points。产物写入器消费同一个 Frame，保持深度、法线和相机数据一致。参见 [Runtime](runtime.md) 与 [产物协议](outputs.md)。 MoGe-2 与 MoGe-3 输出统一为 `GeometryFrame`：包含 Depth、Validity、Intrinsics，以及按需提供的 Normal 和 Points。产物写入器消费同一个 Frame，保持深度、法线和相机数据一致。参见 [Runtime](runtime.md) 与 [产物协议](outputs.md)。

18
docs/internals/common.md
View File

@@ -6,6 +6,9 @@
| --- | --- | | --- | --- |
| `ai.py` | 环境和模型就绪检查、输入路径校验、`production_device()`、`moge2_parameters()` 与安装 UI | | `ai.py` | 环境和模型就绪检查、输入路径校验、`production_device()`、`moge2_parameters()` 与安装 UI |
| `image.py` | `image_rgba()`、动画帧设置、Job 输入准备、像素采样、结果 Image 和 `replace_empty_image()` | | `image.py` | `image_rgba()`、动画帧设置、Job 输入准备、像素采样、结果 Image 和 `replace_empty_image()` |
| `color_image.py` | 材质颜色与 AI 分析输入的准备和清理 |
| `hdr_image.py` | HDR 去背景分析输入及浮点颜色保留 |
| `image_target.py` | Image Empty 与 Shader Image Texture 编辑目标 |
| `selection.py` | `SelectionPath`、`SelectionMask`、non-zero winding 栅格化与 Alpha 修整 | | `selection.py` | `SelectionPath`、`SelectionMask`、non-zero winding 栅格化与 Alpha 修整 |
| `viewport.py` | 原生点选、屏幕投影、`ImageGesture`、Lasso / Brush / Polyline 与 Overlay | | `viewport.py` | 原生点选、屏幕投影、`ImageGesture`、Lasso / Brush / Polyline 与 Overlay |
| `depth.py` | 深度图片和元数据加载、相机坐标与显示范围的尺度换算 | | `depth.py` | 深度图片和元数据加载、相机坐标与显示范围的尺度换算 |
@@ -15,22 +18,9 @@
## 图像生命周期 ## 图像生命周期
```mermaid
flowchart TD
A[Image 与 Image Empty] --> B[image_rgba 读取业务像素]
B --> C{处理位置}
C -- 本地 --> D[像素算法]
C -- Server --> E[prepare_image_input 或 prepare_image_bounds_input]
E --> F[Job 输出文件]
F --> G[加载并 Pack]
D --> G
G --> H[replace_empty_image]
H --> I[保留身份、共享引用、帧设置和画框]
```
业务像素采用 Top-down RGBA，`image_pixels()` 使用 Blender 像素顺序。预乘采样用于投影插值；文件或 Packed 图像的业务读取保留隐藏 RGB。直接编辑结果通过统一入口处理共享 Image、失败恢复和源数据清理。 业务像素采用 Top-down RGBA，`image_pixels()` 使用 Blender 像素顺序。预乘采样用于投影插值；文件或 Packed 图像的业务读取保留隐藏 RGB。直接编辑结果通过统一入口处理共享 Image、失败恢复和源数据清理。
`prepare_image_input()` 返回路径及临时标记；`cleanup_image_input()` 与创建方配对。Cutout 的 `prepare_image_bounds_input()` 复用已读取的源 RGBA，输出指定几何 bounds 的矩形图片。 `prepare_image_input()` 返回路径及临时标记；`cleanup_image_input()` 与创建方配对。`color_image.py` 的 `prepare_material_color_input()` 准备材质颜色，`material_analysis_input()` 提供分析输入；Cutout 按几何 bounds 裁切。
## 手势与 Selection ## 手势与 Selection

14
docs/internals/development.md
View File

@@ -15,11 +15,11 @@ uv run pytest
| Frame、Mask、Rectify、Plane、Cutout | 对应 `test_*_tool.py`、`test_*_operators.py` 与对象、投影、网格测试 | | Frame、Mask、Rectify、Plane、Cutout | 对应 `test_*_tool.py`、`test_*_operators.py` 与对象、投影、网格测试 |
| 图像身份、Alpha、材质与 Undo | 图像编辑集成、共享替换、材质适配与 Undo 测试 | | 图像身份、Alpha、材质与 Undo | 图像编辑集成、共享替换、材质适配与 Undo 测试 |
| Server | `test_job_server.py`、`test_server_jobs.py`、`test_media_input.py`、`test_geometry_artifacts.py` | | Server | `test_job_server.py`、`test_server_jobs.py`、`test_media_input.py`、`test_geometry_artifacts.py` |
| 模型 | `test_model_manager.py`、目录、下载、推理与各 ONNX 模型测试 | | 模型 | `test_model_manager.py`、下载、推理与各 ONNX 模型测试 |
| 节点资产 | 布局检查、Depth Plane 模式、深度分离与 Cutout 体积求值 | | 节点资产 | 布局前后功能等价、深度分离与 Cutout 体积求值 |
| 工具与包协议 | 扩展 Manifest、打包、模型同步、节点启动器和文档命令测试 | | 工具与包协议 | 扩展 Manifest、打包、模型同步与节点启动器 |
Blender Stub 共用 `blender_test_support.py`；Server 和实际 Blender 几何 fixture 按使用范围存放在对应 support 模块。测试命名描述功能、触发条件与可观察结果。 Blender Stub 共用 `tests/support/blender.py`；Server 和实际 Blender 几何 fixture 按使用范围存放在对应 support 模块。测试命名描述功能、触发条件与可观察结果。
## 发布命令 ## 发布命令
@@ -27,14 +27,14 @@ Blender Stub 共用 `blender_test_support.py`；Server 和实际 Blender 几何
| --- | --- | | --- | --- |
| `uv run pack` | 生成 Windows、macOS arm64、Linux 扩展包 | | `uv run pack` | 生成 Windows、macOS arm64、Linux 扩展包 |
| `uv run pack --prepare` | 为本机同步扩展 wheel 与 Manifest | | `uv run pack --prepare` | 为本机同步扩展 wheel 与 Manifest |
| `uv run build-nodes` | 构建、验证节点资产并运行测试 | | `uv run node-group build` | 构建、验证节点资产并运行测试 |
| `uv run sync-models` | 准备、校验并上传模型声明的文件 | | `uv run model sync` | 准备、校验并上传模型声明的文件 |
| `uv run docs -dev` | 本地预览文档 | | `uv run docs -dev` | 本地预览文档 |
| `uv run docs` | 通过 Mike 发布项目版本并更新 latest | | `uv run docs` | 通过 Mike 发布项目版本并更新 latest |
`tools/pack.py` 从 `pyproject.toml` 读取依赖，按平台与 Blender Python 版本解析 wheel，使用 Blender 提供的 NumPy。产物命名为 `AnyImage.v<version>.<platform>.zip`，写入 `dist/`。本地 BlendJob wheel 内容变化后运行 `uv lock --refresh`，并核对锁文件、Manifest 与打包测试。 `tools/pack.py` 从 `pyproject.toml` 读取依赖，按平台与 Blender Python 版本解析 wheel，使用 Blender 提供的 NumPy。产物命名为 `AnyImage.v<version>.<platform>.zip`，写入 `dist/`。本地 BlendJob wheel 内容变化后运行 `uv lock --refresh`，并核对锁文件、Manifest 与打包测试。
`tools/models/cli.py` 根据模型目录选择上游 ONNX 或本地导出流程；`tools/models/exports.py` 处理对应 PyTorch 权重导出。文件校验依据声明大小和 SHA-256，上传使用逐文件复制。`--prepare-only` 用于本地准备。 `tools/models/cli.py` 根据模型目录选择上游 ONNX 或本地导出流程；`tools/models/exports.py` 处理对应 PyTorch 权重导出。文件校验依据声明大小和 SHA-256，上传使用逐文件复制。`uv run model prepare` 用于本地准备。
## CI 与文档导航 ## CI 与文档导航

10
docs/internals/index.md
View File

@@ -2,16 +2,6 @@
AnyImage 在 Blender 主线程处理交互、像素编辑和数据创建，通过 BlendJob 把模型推理交给独立 Job Server。 AnyImage 在 Blender 主线程处理交互、像素编辑和数据创建，通过 BlendJob 把模型推理交给独立 Job Server。
```mermaid
flowchart TD
A[Image Empty 或剪贴板图片] --> B[Operator 与手势工具]
B --> C{需要 AI？}
C -- 是 --> D[准备文件与普通字典参数]
D --> E[JobRuntime → Job Server]
E --> F[图片与相机元数据]
F --> G[Blender 主线程导入]
C -- 否 --> G
G --> H[Image 编辑或 Mesh 与材质创建]
``` ```
| 主题 | 对应职责 | | 主题 | 对应职责 |

64
docs/internals/node-assets.md
View File

@@ -1,11 +1,26 @@
# 节点资产 # 节点资产
## 资产清单
节点组文件在 `tools/nodes/groups`
| 节点组 | 构建模块 |
| ------------------------ | ------------------------- |
| `O Image Plane` | `image_plane.py` |
| `O Image Depth Plane` | `image_depth_plane.py` |
| `O Image Relief Plane` | `image_relief_plane.py` |
| `O Image Layer` | `image_layer.py` |
| `O Image Depth Layer` | `image_depth_layer.py` |
| `O Shadeless` | `shadeless.py` |
| `O Image Cutout` | `image_cutout.py` |
| `O Image Depth Cutout` | `image_depth_cutout.py` |
| `O Image Depth Balloon` | `image_depth_balloon.py` |
| `O Image Depth Panorama` | `image_depth_panorama.py` |
## 构建节点资产 ## 构建节点资产
生产节点组由 `uv run node-group build` 构建到 `src/anyimage/assets/O_AnyImage.blend` 生产节点组由 `uv run node-group build` 构建到 `src/anyimage/assets/O_AnyImage.blend`
节点组文件在 `tools/nodes/groups`
节点组规则： 节点组规则：
- 字段默认直接连接，多个消费者可复用同一输出；按消费位置的几何、域、类型和输入属性版本验证求值等价性 - 字段默认直接连接，多个消费者可复用同一输出；按消费位置的几何、域、类型和输入属性版本验证求值等价性
@@ -20,52 +35,9 @@
- socket `subtype` 按业务语义设置：长度用 `DISTANCE`，`0–1` 比例用 `FACTOR`，`0–100` 用 `PERCENTAGE`；无对应范围语义时保留默认 - socket `subtype` 按业务语义设置：长度用 `DISTANCE`，`0–1` 比例用 `FACTOR`，`0–100` 用 `PERCENTAGE`；无对应范围语义时保留默认
- 接口或布局变更须验证 socket 顺序与结构、未使用输出隐藏及几何求值；纯布局变更比较运算、默认值和实际连接，确认功能不变 - 接口或布局变更须验证 socket 顺序与结构、未使用输出隐藏及几何求值；纯布局变更比较运算、默认值和实际连接，确认功能不变
```mermaid
flowchart TD
A[字段生产与消费] --> B{需要按名称访问}
B -->|是| C[命名属性]
B -->|否| D{目标上下文重新求值是否等价}
D -->|是| E[字段直连并显式转换域]
D -->|否| F[调整消费顺序或在源几何采样]
F --> G{仍需保存源数据}
G -->|否| E
G -->|是| H[仅存储必要命名数据]
H --> I[最后消费者后精确清理]
```
## 字段与临时属性
平滑循环按固定源位置查询原边界索引，并在当前边界网格上消费位置；拓扑不变的边界与影响权重直接求值。Depth Plane 由 UV 矩形判断原边界，Panorama 的闭合源球面以 False 表示原边界。
厚度构建使用源顶点索引排列构造网格，完成挤出与合并后按索引从源表面采样位置、厚度和法线。Depth Cutout 的中心、圆化、结构及膨胀字段在投影源表面求值。
| 属性 | 域 / 类型 | 写入与最后消费 |
| --- | --- | --- |
| `.o_anyimage_depth_cut` | CORNER / BOOLEAN | 三个深度组在 Split Edges 前保存切分标记，供投影调整射线，投影后精确清理 |
| `.o_anyimage_original_boundary` | POINT / BOOLEAN | Depth Cutout 在切分前保存任意输入轮廓，供切口平滑保护原边界，平滑后精确清理 |
| `UVMap` | CORNER / FLOAT2 | Image Plane 与 Panorama 的输出 UV，供材质读取 |
| `o_normal_scale` | POINT 或 FACE / FLOAT | Cutout 输出法线掩码，Depth Cutout 合并背面与侧壁后统一写入，供形状和材质消费 |
## 资产清单
| 节点组 | 构建模块 |
| --- | --- |
| `O Image Plane` | `image_plane.py` |
| `O Image Depth Plane` | `image_depth_plane.py` |
| `O Image Relief Plane` | `image_relief_plane.py` |
| `O Image Layer` | `image_layer.py` |
| `O Image Depth Layer` | `image_depth_layer.py` |
| `O Shadeless` | `shadeless.py` |
| `O Image Cutout` | `image_cutout.py` |
| `O Image Depth Cutout` | `image_depth_cutout.py` |
| `O Image Depth Balloon` | `image_depth_balloon.py` |
| `O Image Depth Panorama` | `image_depth_panorama.py` |
## 构建资产脚本 ## 构建资产脚本
`tools/nodes/build.py` 按依赖顺序构建资产，调用 `arrangement/` 统一排列几何节点组后保存。`common/` 提供共用构建函数，`preview/` 导出交互预览。 `tools/nodes/build.py` 按依赖顺序构建资产，调用 `arrangement/` 统一排列几何节点组后保存。`common/` 提供共用构建函数，其中 `material_slots.py` 继承几何材质槽，`shader.py` 构建材质着色节点，`preview/` 导出交互预览。
保存后，`check.py` 在独立 Blender 进程中重新加载资产并做基本几何求值。接口、默认值和详细几何行为由 `tests/` 检查。 保存后，`check.py` 在独立 Blender 进程中重新加载资产并做基本几何求值。接口、默认值和详细几何行为由 `tests/` 检查。

6
docs/internals/operators/ai-setup.md
View File

@@ -6,7 +6,7 @@ AI 环境管理从当前安装状态出发，补齐独立运行环境和必需
```mermaid ```mermaid
flowchart TD flowchart TD
Entry["Set Up AI Environment"] --> Status["读取环境与必需模型状态"] Entry["Set Up AI Server"] --> Status["读取环境与必需模型状态"]
Status --> Ready{"是否全部就绪？"} Status --> Ready{"是否全部就绪？"}
Ready -- "是" --> Done["结束"] Ready -- "是" --> Done["结束"]
Ready -- "否" --> Environment{"运行环境是否就绪？"} Ready -- "否" --> Environment{"运行环境是否就绪？"}
@@ -29,7 +29,7 @@ flowchart TD
环境尚未就绪时，`SetupAIEnvironment` 调用 Runtime 注册的安装 Operator。`use_mirror` 决定 uv、Python 和 Python 包的下载源，模型使用模型目录中声明的来源。 环境尚未就绪时，`SetupAIEnvironment` 调用 Runtime 注册的安装 Operator。`use_mirror` 决定 uv、Python 和 Python 包的下载源，模型使用模型目录中声明的来源。
安装完成后继续进入模型下载，不在 Blender Python 中加载 ONNX Runtime 或模型依赖。 安装完成后通过 `post_install()` 下载默认模型，模型依赖加载于独立 Server 环境。
### 下载模型 ### 下载模型
@@ -39,4 +39,4 @@ Job 在 `server/app.py` 注册；`server/model_catalog.py` 保存模型名称、
### 释放模型缓存 ### 释放模型缓存
`ai_setup.py` 中的 `ClearModels` 调用 `runtime.clear_resource("model_manager")`，释放 Server 中已加载的模型 Session。它不会删除磁盘上的模型文件，下次推理仍可从现有文件重新加载。 `ai_setup.py` 中的 `ClearModels` 调用 `runtime.clear_resource("model_manager")`，释放 Server 中已加载的模型 Session。下次推理从模型目录重新加载 Session。

14
docs/internals/operators/convert-to-panorama.md Normal file
View File

@@ -0,0 +1,14 @@
# Panorama 转换
`operators/convert_to_panorama/` 将静态 2:1 等距柱状 Image Empty 转换为径向深度表面。`operators.py` 提交任务，`object.py` 导入对象，`__init__.py` 汇总导出与注册清单。
```mermaid
flowchart TD
A[准备颜色与分析图] --> B[generate-panorama-geometry]
B --> C[采样 12 个透视视图并推理]
C --> D[融合径向距离与有效性]
D --> E[写出 depth.exr 与 depth.json]
E --> F[创建材质与 O Image Depth Panorama]
```
任务使用 Preferences 选择的几何模型。源颜色由 Blender 侧保存，Server 的 `geometry/panorama.py` 负责视图采样与距离融合，`jobs/panorama.py` 编排推理和产物写入。

26
docs/internals/operators/convert-to-plane.md
View File

@@ -1,23 +1,21 @@
# Plane 转换 # Plane 转换
`operators/convert_to_plane/operators.py` 提供 `ConvertToPlane`、`ConvertToDepthPlane` 与 `GenerateDepthPlane`。对象和材质导入位于 `object.py`，基础矩形网格位于 `image_plane.py`。 `operators/convert_to_plane/operators.py` 提供 `ConvertToPlane`、`ConvertToDepthPlane`、`ConvertToReliefPlane` 与 `GenerateDepthPlane`。`object.py` 导入对象和材质，`image_plane.py` 构建基础矩形网格，`__init__.py` 汇总导出与注册清单。
```mermaid ```mermaid
flowchart TD flowchart TD
A[Image Empty] --> B{转换类型} A[Image Empty] --> B[准备材质颜色]
B -- Plane --> C[复制 Color 与动画设置] B --> C{转换类型}
C --> D[中心原点四边形与 O Image Plane] C -- Plane --> D[四边形与 O Image Plane]
B -- Depth Plane --> E[检查静态图与 AI 就绪状态] C -- Depth 或 Relief --> E[静态图提交几何 Job]
E --> F[generate-depth-plane-geometry] E --> F[加载深度、元数据与法线]
F --> G[Color、depth.exr、depth.json、Tangent Normal] F --> G[对应节点修改器与材质]
G --> H[加载并 Pack 数据] D --> H[创建对象并替换源 Empty]
H --> I[O Image Depth Plane 与材质] G --> H
D --> J[替换源 Empty 并保留世界位置]
I --> J
``` ```
普通 Plane 支持图片、视频和序列，材质保留 Image User 帧设置。基础四边形以局部中心为原点，对象矩阵补偿原 Image Empty 显示范围；节点组生成细分与厚度，材质位于 Mesh 第 0 个槽。 普通 Plane 支持图片、视频和序列，材质保留 Image User 帧设置。基础网格以局部中心为原点，对象矩阵补偿源 Empty 的显示范围。
Depth Plane 从 Preferences 取得 MoGe-2 模型、分辨率级别与 AI 输入上限，一次推理生成相机 XYZ 深度和切线法线。`create_depth_plane_from_result()` 创建颜色材质与深度修改器，并在失败时清理未使用的数据块。 Depth 与 Relief 从 Preferences 读取几何模型、分辨率级别和 AI 输入上限。`create_depth_plane_from_result()` 导入产物并创建修改器。`O Image Depth Plane` 使用相机 XYZ 与对象空间法线构建深度表面；`O Image Relief Plane` 使用深度 Z 与切线空间法线构建浮雕。
`O Image Depth Plane` 提供默认 Relief 和 Camera：Relief 读取深度 Z 形成固定底面的地形；Camera 使用相机 XYZ 投影并沿平滑法线形成壳。两个模式各自保存 Thickness。完整接口见[节点资产](../node-assets.md)，文件协议见[Job 产物](../outputs.md)。 节点构建见[节点资产](../node-assets.md)，文件协议见[Job 产物](../outputs.md)。

8
docs/internals/operators/cutout-tool.md
View File

@@ -4,7 +4,7 @@ Cutout 在静态 Image Empty 上圈定 Selection，按轮廓创建 Flat、Solid
```mermaid ```mermaid
flowchart TD flowchart TD
A[Lasso 拖动并提交 SelectionPath] --> B[选择 Shape] A[Lasso 或 Polyline 提交 SelectionPath] --> B[选择 Shape]
B --> C[栅格化 SelectionMask 并读取源 RGBA] B --> C[栅格化 SelectionMask 并读取源 RGBA]
C --> D{需要 Depth 或 Normal？} C --> D{需要 Depth 或 Normal？}
D -- 是 --> E[准备 bounds 矩形 RGBA] D -- 是 --> E[准备 bounds 矩形 RGBA]
@@ -28,11 +28,13 @@ flowchart TD
`geometry.py` 从内容 Alpha 提取连通轮廓和孔洞，按 Mesh Detail 采样并生成 `BaseShape`。Low、Medium、High、Ultra 分别以 32、16、8、4 个源像素作为基础间距，并受 Maximum Cutout Mesh Resolution 限制。 `geometry.py` 从内容 Alpha 提取连通轮廓和孔洞，按 Mesh Detail 采样并生成 `BaseShape`。Low、Medium、High、Ultra 分别以 32、16、8、4 个源像素作为基础间距，并受 Maximum Cutout Mesh Resolution 限制。
`fine_mesh.py` 负责 Fine Outline 的细节保留与约束网格，`polygon.py` 提供轮廓运算。Balloon Profile 按 Preferences 选择 Poisson 或 Teddy，生成所有预设共用的 POINT 属性 `o_balloon`。 `fine_mesh.py` 负责 Fine Outline 的细节保留与约束网格，`polygon.py` 提供轮廓运算。`balloon.py` 负责 Balloon Profile，按 Preferences 选择 Poisson 或 Teddy，生成所有预设共用的 POINT 属性 `o_balloon`。
`depth_calibration.py` 负责深度参考平面、尺度与方向校准；`geometry.py` 保留轮廓和网格生成。
## AI 与对象创建 ## AI 与对象创建
Server 接收 Selection bounds 内保留源 Alpha 的 RGBA，一次 MoGe-2 推理按 `generate_depth` 与 `normal_mode` 返回产物。深度边缘延展使用所选 Alpha Threshold，Normal 使用原预测；AI 输入上限只影响分析分辨率。 Server 接收 Selection bounds 内保留源 Alpha 的 RGBA，一次 MoGe 推理按 `generate_depth` 与 `normal_mode` 返回产物。深度参考区域使用所选 Alpha Threshold，Normal 使用原预测；AI 输入上限只影响分析分辨率。
`object.py` 的 `create_shape_object()` 消费 BaseShape、Color、Depth 和 Normal，加载对应节点组。基础网格位于局部 XY，向图片内部为 -Z；选择 +X 时节点与对象矩阵配对变换，保持世界几何一致。Object Normal 同步按方向转换。 `object.py` 的 `create_shape_object()` 消费 BaseShape、Color、Depth 和 Normal，加载对应节点组。基础网格位于局部 XY，向图片内部为 -Z；选择 +X 时节点与对象矩阵配对变换，保持世界几何一致。Object Normal 同步按方向转换。

2
docs/internals/operators/frame.md
View File

@@ -1,6 +1,6 @@
# Frame 多图合成 # Frame 多图合成
`operators/frame_tool.py` 定义 `FrameImages` 和 `FrameTool`。Frame 将选中的静态 Image Empty 按当前视图合成为一张图片。 `operators/frame_tool/operators.py` 定义 `FrameImages` 和 `FrameTool`。`projection.py` 负责视图投影、交叠和结果平面定位，`compositing.py` 负责像素深度排序与边界覆盖采样，`__init__.py` 导出类型及注册清单。Frame 将选中的静态 Image Empty 按当前视图合成为一张图片。
```mermaid ```mermaid
flowchart TD flowchart TD

7
docs/internals/operators/index.md
View File

@@ -5,11 +5,12 @@
| `operators/ai_setup.py` | [AI 环境管理](ai-setup.md) | | `operators/ai_setup.py` | [AI 环境管理](ai-setup.md) |
| `operators/clipboard_image/` | [剪贴板图片](clipboard-image.md) | | `operators/clipboard_image/` | [剪贴板图片](clipboard-image.md) |
| `operators/convert_to_plane/` | [Plane 与 Depth Plane](convert-to-plane.md) | | `operators/convert_to_plane/` | [Plane 与 Depth Plane](convert-to-plane.md) |
| `operators/convert_to_panorama/` | [Panorama](convert-to-panorama.md) |
| `operators/cutout_tool/` | [Cutout](cutout-tool.md) | | `operators/cutout_tool/` | [Cutout](cutout-tool.md) |
| `operators/frame_tool.py` | [Frame 多图合成](frame.md) | | `operators/frame_tool/operators.py` | [Frame 多图合成](frame.md) |
| `operators/mask_tool.py` | [Mask Alpha 编辑](mask.md) | | `operators/mask_tool.py` | [Mask Alpha 编辑](mask.md) |
| `operators/rectify_tool.py` | [Rectify 透视校正](rectify.md) | | `operators/rectify_tool/operators.py` | [Rectify 透视校正](rectify.md) |
| `operators/remove_background.py` | [Remove Background](remove-background.md) | | `operators/remove_background.py` | [Remove Background](remove-background.md) |
| `operators/upscale.py` | [Upscale](upscale.md) | | `operators/upscale.py` | [Upscale](upscale.md) |
单文件功能按业务命名；复杂功能包在 `operators.py` 定义 Operator、对应 WorkSpaceTool 和入口协调逻辑，其余模块承载具体实现。包的 `__init__.py` 汇总导出，类型在 `operators/__init__.py` 和扩展 `__init__.py` 汇总注册。AI Operator 使用普通字典提交任务，并在主线程响应中读取文件和修改 Blender 数据。 单文件功能按业务命名；复杂功能包在 `operators.py` 定义 Operator、对应 WorkSpaceTool 和入口协调逻辑，其余模块承载具体实现。包的 `__init__.py` 汇总导出与 `CLASSES` 注册清单，类型在 `operators/__init__.py` 和扩展 `__init__.py` 汇总注册。AI Operator 使用普通字典提交任务，并在主线程响应中读取文件和修改 Blender 数据。

2
docs/internals/operators/rectify.md
View File

@@ -1,6 +1,6 @@
# Rectify 透视校正 # Rectify 透视校正
`operators/rectify_tool.py` 定义 `RectifyImagePerspective` 与 `RectifyTool`，将图片内的四角区域校正为矩形。 `operators/rectify_tool/operators.py` 定义 `RectifyImagePerspective` 与 `RectifyTool`，将图片内的四角区域校正为矩形。`geometry.py` 负责四角验证与像素校正，`preview.py` 负责预览图、Overlay 和宽高比控制，`__init__.py` 导出类型及注册清单。
```mermaid ```mermaid
flowchart TD flowchart TD

40
docs/internals/operators/remove-background.md
View File

@@ -1,37 +1,19 @@
# Remove Background # Remove Background
Remove Background 从当前 Image Empty 准备单图或多帧输入，运行 BEN2 生成前景 Alpha，再用结果 Image 替换原图。 `operators/remove_background.py` 从 Image Empty 或当前 Shader Image Texture 准备输入。`RemoveImageBackground` 协调设置与输入，`RunBackgroundRemoval` 提交 Job 并应用结果。
## 执行过程
```mermaid ```mermaid
flowchart TD flowchart TD
Empty["Image Empty"] --> Prepare["准备图片、视频或序列输入"] A[ImageEditTarget 取得编辑目标] --> B[准备图像输入]
Prepare --> Job["remove-background Job"] B --> C[remove-background Job]
Job --> Frames["逐帧运行 BEN2"] C --> D[所选背景模型生成前景 Alpha]
Frames --> Result["写出 PNG 结果序列"] D --> E{输入类型}
Result --> Import["创建结果 Image"] E -- 普通媒体 --> F[PNG 结果序列]
Import --> Replace["替换 Empty.data"] E -- HDR --> G[alpha.npy]
F --> H[应用结果与 Undo]
G --> H
``` ```
Packed 图片和内存中修改过的单图会先导出临时文件；文件图片、视频和序列沿用原路径。 `server/jobs/remove_background.py` 展开媒体并调用 `server/models/background.py`，支持 BiRefNet 和 BEN2。模型前景 Alpha 与源 Alpha 合成。
## 节点说明 普通结果通过 `ImageEditTarget` 保留共享图像引用与帧设置。HDR 使用 `common/hdr_image.py` 的 `HdrBackgroundInput` 准备分析输入，并将返回 Alpha 应用于原浮点 RGB。Operator 负责临时输入清理。
### 准备媒体输入
入口是 `operators/remove_background.py` 中的 `RemoveImageBackground`。它检查当前对象、AI Environment 和帧数，再调用 `common/image.py` 的 `prepare_image_input()`。多帧输入会先写入 Scene 中的抽帧设置。
准备完成后，内部 Operator `RunBackgroundRemoval` 保存输入路径、源对象名和临时文件清理标记。
### 运行 BEN2
`RunBackgroundRemoval` 提交 `remove-background` Job。`server/jobs/remove_background.py` 使用公共媒体输入层展开单图、视频或序列，并逐帧调用 `server/models/ben2.py`。
BEN2 推理前先用源 Alpha 屏蔽透明区域，模型输出的前景 Alpha 再与源 Alpha 合成，RGB 保持源图内容。多帧结果按连续文件名写入 Job 目录。
### 导入并替换结果
Job 返回后，`remove_background.py` 的 `apply_background_response()` 调用 `common/image.py` 创建单图或序列 Image，并恢复源 Image User 的帧起点、偏移和循环设置。
`replace_empty_image()` 随后替换 `Empty.data`，提交一个 Remove Background Undo 步骤，并清理本次创建的临时输入。

22
docs/internals/operators/upscale.md
View File

@@ -1,21 +1,15 @@
# Upscale # Upscale
`operators/upscale.py` 从当前 Image Empty 生成 2× 结果并替换原图，支持单图、视频与序列帧。 `operators/upscale.py` 从 Image Empty 或当前 Shader Image Texture 生成 2× 结果。普通媒体支持单图、视频和序列帧。
```mermaid ```mermaid
flowchart TD flowchart TD
A[检查源图长边小于 AI 输入上限] --> B[冻结 Preferences 模型与尺寸设置] A[检查源图尺寸并冻结模型设置] --> B[准备媒体输入]
B --> C[准备媒体输入] B --> C[upscale-image 逐帧处理]
C --> D[upscale-image 逐帧处理] C --> D[分块执行 4× RGB 推理]
D --> E{所选模型} D --> E[统一缩放为源图 2×]
E -- Real-ESRGAN 或 SAFMN --> F[256×256 tile 与边缘上下文执行 4× RGB 推理] E --> F[原 Alpha 独立 Lanczos 缩放]
E -- PiSA-SR --> G[分阶段生成推理，细节强度 0.7] F --> G[ImageEditTarget 应用结果并提交 Undo]
F --> H[统一缩放到源图 2×]
G --> H
H --> I[原 Alpha 独立 Lanczos 缩放并回填]
I --> J[写出 upscale 序列并替换 Image]
``` ```
`server/models/upscale.py` 统一处理输入上限、模型分派和最终 RGB/Alpha 尺寸。`onnx_upscale.py` 处理常规 4× 模型，`onnx_pisa_sr.py` 处理 PiSA-SR。多帧与多 tile 只在最后一次运行请求内存收缩。 `server/models/upscale.py` 处理输入上限和最终尺寸，`onnx_upscale.py` 执行 Real-ESRGAN 与 HAT 模型。ModelManager 管理 Session；Provider 资源错误会释放当前 Session 并返回处理建议。
Provider 资源错误会释放当前 Upscale Session，并返回降低输入上限或切换 CPU 的提示。响应通过 `common/image.py` 加载、Pack、保留图像共享与帧设置，并提交 Undo。

49
docs/internals/outputs.md
View File

@@ -1,43 +1,40 @@
# Job 产物 # Job 产物
Server 把结果写入 Job 目录，返回以相对文件名为值的字典。Blender 主线程通过 JobResult 读取文件，再创建或更新 Image、Mesh、Material 和 Object。 Server 将结果写入 Job 目录，返回以相对文件名为值的字典。Blender 主线程通过 JobResult 读取文件，创建或更新 Image、Mesh、Material 和 Object。
| Job | 文件 | 消费方 | | Job | 文件 | 消费方 |
| --- | --- | --- | | --- | --- | --- |
| `remove-background` | `image.0001.png` 起 | Image Empty 替换 | | `remove-background` | PNG 序列或 `alpha.npy` | ImageEditTarget 或 HDR Alpha 编辑 |
| `upscale-image` | `upscale.0001.png` 起 | 2× Image Empty 替换 | | `upscale-image` | `upscale.0001.png` 起 | 图像替换 |
| `generate-depth-plane-geometry` | `image.0001.*`、`depth.exr`、`depth.json`、`tangent-normal.png` | Plane 对象与材质 | | `generate-depth-plane-geometry` | `depth.exr`、`depth.json` 和法线 | Depth / Relief Plane |
| `generate-cutout-artifacts` | 按请求提供深度、元数据和法线 | Cutout 对象与材质 | | `generate-cutout-artifacts` | 按请求提供深度、元数据和法线 | Cutout |
| `generate-panorama-geometry` | `depth.exr`、`depth.json` | Panorama |
## 深度与法线协议 ## 深度与法线协议
| 返回字段 | 文件 | 内容 | | 返回字段 | 内容 |
| --- | --- | --- | | --- | --- |
| `depth` | `depth.exr` | float32 RGB 保存相机空间 XYZ，Alpha 保存 Validity | | `depth` | float32 EXR，透视产物 RGB 保存相机空间 XYZ，Alpha 保存源 Alpha × 模型有效性 |
| `depth_metadata` | `depth.json` | `image_size`、3×3 `intrinsics`、`reference_depth` | | `depth_metadata` | JSON，透视产物包含 `image_size`、3×3 `intrinsics`、`reference_depth` |
| `tangent_normal` | `tangent-normal.png` | 切线空间法线 | | `tangent_normal` | `tangent-normal.png`，切线空间法线 |
| `object_normal` | `object-normal.png` | Cutout 原生 XY / Z 坐标的对象空间法线 | | `object_normal` | `object-normal.png`，对象空间法线 |
```mermaid ```mermaid
flowchart TD flowchart TD
A[矩形源 RGBA] --> B[限尺寸并合成黑底模型输入] A[源 RGB 与 Alpha] --> B[限尺寸 RGB 推理]
B --> C[MoGe-2 GeometryFrame] B --> C[GeometryFrame]
C --> D[原预测生成 Normal] C --> D[写出深度与法线]
C --> E{Cutout 深度？} A --> E[Alpha 有效性与参考区域]
E -- 是 --> F[按源 Alpha 延展轮廓深度] E --> D
E -- 否 --> G[完整相机 Points] D --> F[Blender 加载并 Pack]
F --> G
G --> H[depth.exr 与 depth.json]
D --> I[Blender 加载并 Pack]
H --> I
``` ```
`depth_texture.py` 保存完整有限 Points；模型 Mask 转为 `0/1` Validity。参考深度取有效、有限、正深度且满足源 Alpha 条件的样本中位数：整图使用 Alpha ≥ 0.1，Cutout 使用所选 Alpha Threshold 并要求 Alpha > 0。非法字段和空参考区域抛出错误。 参考深度取满足模型有效性、有限正深度与 Alpha 阈值的样本中位数。`common/depth.py` 加载 EXR，设置 Non-Color 和 CHANNEL_PACKED。Depth Plane 使用对象空间法线，Relief Plane 使用切线空间法线。
`common/depth.py` 加载深度并配置 Non-Color、CHANNEL_PACKED 与 Pack。Relief 和 Depth Balloon 读取 B/Z，Camera 与 Depth Solid 读取 XYZ，共用同一 EXR。 Panorama 的 EXR RGB 保存径向距离，Alpha 保存源 Alpha 与融合有效性；元数据包含 `projection: equirectangular` 和 `image_size`。
## Selection 与颜色 ## 颜色与图像编辑
Cutout 提交几何 bounds 内的原始 RGBA。Blender 在创建网格时组合源 Alpha 和 `SelectionMask`；颜色图保留矩形源 RGBA，法线可见区域取源 Alpha。AI 分析尺寸与颜色分辨率分别处理。 `common/color_image.py` 为 Plane、Panorama 和 Cutout 准备独立的材质颜色与 AI 分析输入。Cutout 使用 Selection bounds 内的源图；网格创建时组合源 Alpha 与 SelectionMask。颜色分辨率与分析尺寸分别处理。
Remove Background、Upscale、Frame、Mask、Rectify 通过 `common/image.py` 的编辑结果入口保留源 Image 名称和共享引用语义。Plane 与 Cutout 的材质贴图使用 Color、Depth、Normal 命名函数。 `ImageEditTarget` 统一 Image Empty 和 Shader Image Texture 的编辑目标及共享引用。HDR 去背景通过 `HdrBackgroundInput` 保留 Blender 浮点 RGB，并应用 Server 返回的连续 Alpha。

53
docs/internals/runtime.md
View File

@@ -1,53 +1,36 @@
# Job Runtime 与 Server # Job Runtime 与 Server
`runtime.py` 配置 BlendJob `JobRuntime` 和独立 Python 3.12 Environment。`server/app.py` 注册 Job，`model_manager.py` 作为 `model_manager` Resource 管理下载文件与缓存 Session。 `runtime.py` 配置 BlendJob `JobRuntime` 和独立 Python 3.12 Environment。`server/app.py` 注册 Job，`model_manager.py` 管理模型文件与缓存 Session。
```mermaid ```mermaid
flowchart TD flowchart TD
A[Operator request] --> B[JobRuntime] A[Operator request] --> B[JobRuntime 队列]
B --> C[Server 单 Worker 队列] B --> C[Server Job 调用媒体与模型模块]
C --> D[Job 调用媒体与模型模块] C --> D[写入 Job 目录]
D --> E[写入 jobs/job-id 目录] D --> E[主线程 response 导入结果]
E --> F[主线程 response]
``` ```
Job 在阶段和帧边界检查取消，进度通过 `context.progress()` 返回。视频临时帧由 `media/input.py` 的上下文管理器清理，Blender 导出的临时输入由对应 Operator 清理。
## Job 与数据模块
| Job | 实现与用途 | | Job | 实现与用途 |
| --- | --- | | --- | --- |
| `download-model`、`download-required-models` | `app.py` 调用 ModelManager 下载单个或缺失模型 | | `download-model`、`download-required-models` | `app.py` 下载单个或缺失模型 |
| `remove-background` | `jobs/remove_background.py`，生成 BEN2 前景 | | `remove-background` | `jobs/remove_background.py` 生成前景 Alpha |
| `upscale-image` | `jobs/upscale.py`，生成 2× 图像或序列 | | `upscale-image` | `jobs/upscale.py` 生成 2× 图像或序列 |
| `generate-depth-plane-geometry` | `jobs/depth_plane.py`，复制整图颜色并生成深度、元数据与切线法线 | | `generate-depth-plane-geometry` | `jobs/depth_plane.py` 生成深度、元数据和法线 |
| `generate-cutout-artifacts` | `jobs/cutout.py`，按请求生成 Cutout 深度和法线 | | `generate-cutout-artifacts` | `jobs/cutout.py` 按请求生成深度和法线 |
| `generate-panorama-geometry` | `jobs/panorama.py` 生成全景径向深度 |
`media/` 处理单图、序列、视频与输入尺寸。Depth Plane 和 Cutout 按单张静态图片工作；Remove Background 与 Upscale 逐帧处理动画输入。 `media/` 展开单图、序列与视频，并限制分析尺寸。Job 在阶段与帧边界检查取消，通过 `context.progress()` 返回进度。媒体上下文清理抽取的临时帧，Operator 清理 Blender 导出的临时输入。
`geometry/prediction_artifacts.py` 将限尺寸后的 RGBA 合成到黑底，运行一次 MoGe-2，再调用深度、法线和元数据写入器。Cutout 的深度分支通过 `edge_depth.py` 延展轮廓附近的深度；法线使用原预测。文件字段见 [Job 产物](outputs.md)。 `geometry/prediction_artifacts.py` 缩放源 RGB 后运行一次 MoGe，将同一预测交给深度、法线和元数据写入器。源 Alpha 用于产物有效性和参考深度采样。文件协议见 [Job 产物](outputs.md)。
## 模型与缓存 ## 模型与缓存
| 用途 | 模型 | Session 分类 | | 用途 | 模型 | Session 分类 |
| --- | --- | --- | | --- | --- | --- |
| Depth 与 Normal | MoGe-2 ViT-S / ViT-B / ViT-L Normal | geometry | | Depth 与 Normal | MoGe-2 ViT-S / ViT-B Normal、MoGe-3 ViT-L | geometry |
| Remove Background | BEN2 | ben2 | | Remove Background | BiRefNet Lite、BEN2、BiRefNet HR Matting | background |
| Upscale | Real-ESRGAN WDN、SAFMN、Real-ESRGAN x4plus、PiSA-SR | upscale | | Upscale | Real-ESRGAN WDN、Real-ESRGAN x4plus、HAT GAN x4 Sharper | upscale |
`model_catalog.py` 声明文件来源、许可、大小和 SHA-256。`model_download.py` 下载文件并校验；ModelManager 按模型路径和设备缓存各类最近使用的 Session，切换模型或设备时释放对应类别。 `model_catalog.py` 声明文件来源、许可、大小和 SHA-256；`model_download.py` 下载并校验。ModelManager 按模型路径和设备缓存 Session，切换模型或设备时释放对应类别。Clear Models 或关闭 Server 时释放已加载资源。
```mermaid 首次安装通过 `post_install()` 下载默认 MoGe-2、BiRefNet Lite 与 Real-ESRGAN WDN。Storage Root 默认 `~/.anyimage`，保存环境、模型、Job 产物和日志。设置入口见 [AI 环境管理](operators/ai-setup.md)。
flowchart TD
A[取得缓存 Session] --> B[运行单图、多帧或 tile]
B --> C[最后一次推理]
C --> D[CUDA 收缩空闲 arena]
D --> E[保留 Session 供后续复用]
E --> F[Clear Models 或关闭 Server 时释放]
```
Windows 默认 DirectML，macOS arm64 默认 CPU 并支持选择 CoreML。MoGe-2 为 CoreML 配置静态 shape 分区，其余计算由 CPU 执行。PiSA-SR 在 CoreML 上返回明确的支持限制；其分阶段 Session 在切换阶段、取消和清理时关闭。
## Environment 与存储
首次安装通过 `post_install()` 下载默认 MoGe-2、BEN2 与 Real-ESRGAN WDN。Storage Root 默认 `~/.anyimage`，其中保存 `.venv/`、`models/`、`jobs/` 及环境状态和日志。设置入口与安装流程见 [AI 环境管理](operators/ai-setup.md)。

1
mkdocs.yml
View File

@@ -23,6 +23,7 @@ nav:
- AI 环境管理: internals/operators/ai-setup.md - AI 环境管理: internals/operators/ai-setup.md
- Clipboard Image: internals/operators/clipboard-image.md - Clipboard Image: internals/operators/clipboard-image.md
- Convert to Plane: internals/operators/convert-to-plane.md - Convert to Plane: internals/operators/convert-to-plane.md
- Convert to Panorama: internals/operators/convert-to-panorama.md
- Cutout Tool: internals/operators/cutout-tool.md - Cutout Tool: internals/operators/cutout-tool.md
- Frame: internals/operators/frame.md - Frame: internals/operators/frame.md
- Mask: internals/operators/mask.md - Mask: internals/operators/mask.md

2
openspec/changes/add-cutout-polyline/.openspec.yaml Normal file
View File

@@ -0,0 +1,2 @@
schema: spec-driven
created: 2026-09-12

49
openspec/changes/add-cutout-polyline/design.md Normal file
View File

@@ -0,0 +1,49 @@
## Context
Cutout 的 `SelectCutoutSelection` 已继承 `common.viewport.ImageGesture`，目前固定为 Lasso 并通过拖动键位启动。Mask 的 Polyline 已由该共用类实现。Cutout 完成选区后打开 Shape 饼菜单，Depth Solid 经异步生成后使用 `O Image Depth Cutout` 创建对象。
节点构建内部的 Uniform Thickness 输入在保存前显示名改为 Thickness；节点组同时存在 Balloon 使用的 Thickness 输入，初始化必须区分 socket 标识。
## Goals
支持 Cutout 逐点剪出及 Uniform 初始化；统一 Mask 的 Extend 命名和 Set 默认值。
## Decisions
### 共用手势与独立设置
在 `properties.py` 定义 Cutout 的 Lasso / Polyline 属性，由 Scene 保存，默认 Lasso。Cutout 在启动时读取手势，沿用 `ImageGesture` 的逐点输入、实时预览、闭合提示和取消清理。使用共用按下启动键位支持首点输入，并验证 Lasso 拖动完成以及空白处源对象保持行为。共用逻辑留在 `common`，Cutout 负责自己的状态文本和选区提交。
### 创建时应用 Uniform
```mermaid
flowchart TD
A[读取 Cutout Gesture] --> B[ImageGesture 收集并确认选区]
B --> C[Shape 饼菜单携带本次 Gesture 快照]
C --> D[CutoutSelectionToShape 保存 Gesture]
D --> E[生成所需图像并在主线程创建对象]
E --> F{Polyline 且 Depth Solid}
F -->|是| G[设置新修改器 Mode 为 Uniform]
F -->|否| H[按 Shape 初始化修改器]
```
手势作为普通字符串从选择 Operator 显式传到饼菜单、转换 Operator 和对象创建链路，异步响应使用该快照，避免用户中途切换 Scene 设置影响结果。服务器生成协议继续处理深度和法线产物；修改器初始化由主线程完成。Uniform 自动设置针对本次新建的 Depth Solid 对象，之后用户可手动修改 Mode。
### 节点厚度默认值
将构建内部 Uniform Thickness 默认值设为 0.5，并验证最终资产对应 socket 的默认值和新修改器实际值。检查现有通用 Thickness 赋值只初始化其目标输入，避免同名 socket 覆盖 Uniform 默认值。按现有资产修订机制更新 Depth Solid 版本并运行 `uv run node-group build`。
### Mask 模式
统一为 `SET` / Set、`EXTEND` / Extend、`SUBTRACT` / Subtract，保留枚举数值 0、1、2。Scene 和 Operator 均通过共用属性默认 Set。Extend 保持 `max(alpha, mask)`，Set 保持 `alpha * mask`，Subtract 保持 `alpha * (1-mask)`；状态文本和测试同步命名。
## Risks / Trade-offs
- 按下启动可能改变 Cutout 的源对象选择时机 → 覆盖 Lasso、Polyline、空白点击和取消行为。
- 异步生成期间手势被切换 → 在提交时保存本次手势，并覆盖设置改变后的响应。
- 同名 Thickness socket 容易导致写入错误 → 验证具体 socket identifier、资产默认值及实际修改器值。
- 旧脚本传入 `ADD` 将失效 → 调用方统一使用 `EXTEND`，保持原枚举数值且不增加别名。
## Migration Plan
实现代码与测试后重建节点资产，执行相关测试及全量测试。回退时将本次代码、测试和资产作为同一变更回退。

27
openspec/changes/add-cutout-polyline/proposal.md Normal file
View File

@@ -0,0 +1,27 @@
## Why
Cutout 目前只能自由绘制 Lasso，沿直线轮廓剪出时需要逐点定义边界。复用 Mask 的 Polyline 交互，并为这种剪出方式设置 Uniform 厚度，可减少手动调整；Mask 使用 Extend 和默认 Set，使名称与常用操作一致。
## What Changes
- Cutout 增加 Gesture 选项 Lasso / Polyline，默认 Lasso，复用共用逐点交互和选区提交链路。
- Polyline 剪出创建使用 `O Image Depth Cutout` 的 Depth Solid 对象时，自动设置修改器 Mode 为 Uniform。
- `O Image Depth Cutout` 的 Uniform Thickness 接口默认值改为 0.5（资产中显示为 Thickness）。
- **BREAKING** Mask 的 `ADD` / Add 统一重命名为 `EXTEND` / Extend，保留扩展 Alpha 的运算语义；默认模式从 Subtract 改为 Set。
## Capabilities
### New Capabilities
- `cutout-polyline`: Cutout 手势选择、共用 Polyline 交互，以及创建结果的 Uniform 初始化。
- `mask-mode-options`: Mask 的 Set / Extend / Subtract 名称、运算和默认值。
### Modified Capabilities
无。当前 `openspec/specs` 尚无已归档的对应规范。
## Impact
- `properties.py`、`common/viewport.py`、`operators/cutout_tool`、`operators/mask_tool.py` 和相关注册、交互、对象创建测试。
- `tools/nodes/groups/image_depth_cutout.py`、节点资产 `O_AnyImage.blend` 及节点测试；实现阶段需运行节点构建与验证。
- 无新增依赖。已有 change 中的 Mask、Polyline 和 Cutout 提案作为实现背景，本 change 定义本次行为。

57
openspec/changes/add-cutout-polyline/specs/cutout-polyline/spec.md Normal file
View File

@@ -0,0 +1,57 @@
## ADDED Requirements
### Requirement: Cutout offers persistent gesture selection
Cutout SHALL 提供 Lasso 和 Polyline 手势，默认 Lasso，并保存到 Scene 的独立 Cutout 设置。
#### Scenario: User selects Polyline
- **WHEN** 用户将 Cutout Gesture 切换为 Polyline 并开始剪出
- **THEN** 工具以首次左键按下的位置为起点，随后通过点击添加顶点
#### Scenario: User returns to Cutout
- **WHEN** 用户选择 Polyline 后切换到其他工具再返回
- **THEN** Cutout 继续使用 Polyline，Mask Gesture 保持自己的设置
#### Scenario: Default Lasso interaction
- **WHEN** 新 Scene 中用户拖动 Cutout 并释放左键
- **THEN** 工具以 Lasso 收集轮廓并在有效选区完成后打开 Shape 饼菜单
### Requirement: Cutout reuses shared Polyline interaction
Cutout MUST 复用 Mask 所用的共用 Polyline 手势，提供轮廓预览、起点闭合提示、左键添加顶点、Backspace 删除末点、点击起点或双击或 Enter 完成，以及 RMB、Esc 和切换工具取消。
#### Scenario: User completes a polygon
- **WHEN** 至少三个顶点后，用户点击起点闭合范围内、双击或按 Enter / Numpad Enter
- **THEN** 工具提交已确认的闭合 SelectionPath 并打开 Shape 饼菜单，闭合点击和双击第二击不添加重复顶点
#### Scenario: Too few points
- **WHEN** 少于三个顶点时用户尝试完成
- **THEN** 工具继续收集顶点，不打开 Shape 菜单
#### Scenario: User edits or cancels a polygon
- **WHEN** 用户按 Backspace，或通过 RMB、Esc、切换工具取消
- **THEN** Backspace 删除末点并至少保留起点；取消清理预览且不创建结果对象
#### Scenario: Empty click preserves the source
- **WHEN** 用户在空白处启动 Cutout 手势
- **THEN** 工具保持现有源对象选择规则，不因空白点击清空源对象
### Requirement: Polyline initializes Depth Solid with Uniform mode
通过 Polyline 创建 Depth Solid 时，系统 MUST 将新对象的 `O Image Depth Cutout` 修改器 Mode 初始化为 Uniform。手势 MUST 使用本次提交快照。
#### Scenario: Polyline depth result completes asynchronously
- **WHEN** 用户提交 Polyline 并选择 Depth Solid，在生成结束前将 Cutout Gesture 改为 Lasso
- **THEN** 本次结果的 Mode 仍为 Uniform，用户可在创建后手动调整
#### Scenario: Lasso or another shape is used
- **WHEN** 用户通过 Lasso 创建 Depth Solid，或通过 Polyline 创建 Flat、Solid、Depth Balloon
- **THEN** 对象沿用该 Shape 的现有 Mode 初始化规则
### Requirement: Uniform thickness defaults to half a unit
`O Image Depth Cutout` 的 Uniform Thickness socket MUST 默认 0.5，保存资产中对应显示为 Thickness 的 socket 和新建修改器 MUST 使用该值。
#### Scenario: Node asset is built and loaded
- **WHEN** 完成节点资产构建并从资产加载节点组创建 Polyline Depth Solid
- **THEN** Mode 为 Uniform，Uniform 厚度值为 0.5，通用 Thickness 初始化不会覆盖该值

25
openspec/changes/add-cutout-polyline/specs/mask-mode-options/spec.md Normal file
View File

@@ -0,0 +1,25 @@
## ADDED Requirements
### Requirement: Mask exposes Set Extend and Subtract
Mask MUST 按顺序提供 `SET` / Set、`EXTEND` / Extend、`SUBTRACT` / Subtract，枚举数值分别为 0、1、2；界面和状态文本 MUST 使用 Extend。
#### Scenario: User extends Alpha
- **WHEN** 用户以 Extend 提交值为 mask 的选区到现有 alpha
- **THEN** 结果 Alpha 为 `max(alpha, mask)`，RGB 保持不变
#### Scenario: User uses Set or Subtract
- **WHEN** 用户使用 Set 或 Subtract
- **THEN** 结果 Alpha 分别为 `alpha * mask` 或 `alpha * (1-mask)`，RGB 保持不变
### Requirement: Mask defaults to Set
Scene 的 Mask Mode 和 Mask Operator 的模式属性 MUST 默认 Set，并通过同一属性定义保持一致。
#### Scenario: New scene and operator defaults
- **WHEN** 创建新 Scene 或未指定模式的 Mask Operator
- **THEN** 模式默认值为 `SET`
#### Scenario: User chooses another mode
- **WHEN** 用户主动设置 Extend 或 Subtract 后启动 Mask
- **THEN** 本次操作使用用户保存的模式

21
openspec/changes/add-cutout-polyline/tasks.md Normal file
View File

@@ -0,0 +1,21 @@
## 1. Cutout 手势
- [x] 1.1 在 properties.py 添加默认 Lasso 的 Cutout Gesture 属性并接入 Scene、Operator 和工具设置，检查注册与逆序注销测试。
- [x] 1.2 将 Cutout 接入共用 Polyline 启动、预览与完成逻辑，更新状态文本；覆盖逐点、闭合、双击、Enter、Backspace、取消、Lasso 回归和空白处源对象保持行为。
## 2. Uniform 初始化与资产
- [x] 2.1 将本次 Gesture 快照从选区提交经 Shape 菜单传入转换 Operator 和对象创建链路；覆盖异步期间切换设置的行为。
- [x] 2.2 为 Polyline 创建的 Depth Solid 设置 Mode 为 Uniform，验证 Lasso 与其他 Shape 的初始化行为。
- [x] 2.3 将节点内部 Uniform Thickness 默认值改为 0.5，检查同名 Thickness socket 的赋值目标并更新资产修订号；补充资产默认值与实际修改器值测试。
- [x] 2.4 运行 `uv run node-group build`，更新并验证 O_AnyImage.blend，确认源码与保存资产的 Capture Attribute 数量为零。
## 3. Mask 模式
- [x] 3.1 将 Mask 的 ADD / Add 统一改为 EXTEND / Extend，保持枚举数值与 Alpha 运算语义，将共用模式属性默认值改为 SET。
- [x] 3.2 更新模式、状态文本、设置持久化和注册测试，验证三种模式的 Alpha 结果及 RGB 保持行为。
## 4. 验证
- [x] 4.1 运行相关测试及 `uv run pytest`，确认正常退出。
- [x] 4.2 检查运行时代码和测试中的旧 Mask 模式引用及最终差异，确认源码、测试和节点资产一致。

4
pyproject.toml
View File

@@ -1,6 +1,6 @@
[project] [project]
name = "anyimage" name = "anyimage"
version = "0.13.2" version = "0.13.3"
description = "Fast image-guided structure building tools for Blender" description = "Fast image-guided structure building tools for Blender"
readme = "README.md" readme = "README.md"
requires-python = ">=3.11,<3.12" requires-python = ">=3.11,<3.12"
@@ -9,7 +9,7 @@ dependencies = ["blendjob==0.1.17", "numpy>=1.26,<2.0", "scipy==1.15.3"]
[dependency-groups] [dependency-groups]
dev = [ dev = [
"av==17.0.0",
"bpy==4.5.3", "bpy==4.5.3",
"huggingface-hub==1.4.1", "huggingface-hub==1.4.1",
"mike>=2.1.3,<3", "mike>=2.1.3,<3",

BIN
src/anyimage/assets/O_AnyImage.blend LFS
View File

Binary file not shown.

2
src/anyimage/blender_manifest.toml
View File

@@ -1,7 +1,7 @@
schema_version = "1.0.0" schema_version = "1.0.0"
id = "anyimage" id = "anyimage"
version = "0.13.2" version = "0.13.3"
name = "AnyImage" name = "AnyImage"
tagline = "Build spatial structures from images, fast" tagline = "Build spatial structures from images, fast"
maintainer = "OmooLab" maintainer = "OmooLab"

4
src/anyimage/common/ai.py
View File

@@ -69,12 +69,10 @@ def require_model(model_key):
return model return model
def moge2_parameters(settings, path, model, resolution_level, start_frame): def moge2_parameters(path, model, resolution_level):
return { return {
"input": str(path), "input": str(path),
"model": model, "model": model,
"resolution_level": int(resolution_level), "resolution_level": int(resolution_level),
"video_frames": settings.video_frames,
"start_frame": start_frame,
"device": production_device().lower(), "device": production_device().lower(),
} }

4
src/anyimage/common/color_image.py
View File

@@ -7,12 +7,12 @@ from pathlib import Path
import bpy import bpy
from .image import cleanup_image_input, image_base_name, image_is_packed, image_rgba, is_animated_image, video_total_frames from .image import cleanup_image_input, image_base_name, image_is_packed, image_rgba, is_animated_image
def require_static_color_image(image): def require_static_color_image(image):
"""Reject animated inputs before preparing material resources.""" """Reject animated inputs before preparing material resources."""
if is_animated_image(image) or video_total_frames(image) != 1: if is_animated_image(image):
raise ValueError("Material conversion supports static images only") raise ValueError("Material conversion supports static images only")

284
src/anyimage/common/image.py
View File

@@ -1,14 +1,12 @@
"""Shared Blender Image and Image Empty lifecycle helpers.""" """Shared Blender Image and Image Empty lifecycle helpers."""
import math import math
import re
import shutil import shutil
import tempfile import tempfile
from pathlib import Path from pathlib import Path
import bpy import bpy
from ..server.media.input import validate_input_path
VIEW_TRANSFORM_COLOR_SPACES = ( VIEW_TRANSFORM_COLOR_SPACES = (
("ACES 1.3", "ACES 1.3 sRGB"), ("ACES 1.3", "ACES 1.3 sRGB"),
@@ -36,19 +34,6 @@ MAX_PROJECTIVE_OUTPUT_PIXELS = 16_777_216
PROJECTIVE_CHUNK_ROWS = 256 PROJECTIVE_CHUNK_ROWS = 256
VIDEO_NAME_SUFFIXES = (
".mp4",
".mov",
".avi",
".mkv",
".webm",
".m4v",
".mts",
".mpg",
".mpeg",
)
def is_image_empty(candidate): def is_image_empty(candidate):
return bool( return bool(
candidate candidate
@@ -290,21 +275,10 @@ def warp_projective_pixels(source_pixels, source_size, quad, output_size):
return np.flipud(straight_rgba(output)).ravel() return np.flipud(straight_rgba(output)).ravel()
def is_animated_image(image): def is_animated_image(image):
return getattr(image, "source", None) in {"MOVIE", "SEQUENCE"} return getattr(image, "source", None) in {"MOVIE", "SEQUENCE"}
def video_total_frames(image):
"""Return the total animated frame count, or 1 for a still image."""
source = getattr(image, "source", None)
if source == "MOVIE":
return int(getattr(image, "frame_duration", 1) or 1)
if source == "SEQUENCE":
return len(collect_sequence_frames(image)) or 1
return 1
def image_user_settings(source_object): def image_user_settings(source_object):
"""Return an image owner's playback settings as a four-item tuple.""" """Return an image owner's playback settings as a four-item tuple."""
image_user = getattr(source_object, "image_user", None) image_user = getattr(source_object, "image_user", None)
@@ -318,28 +292,11 @@ def image_user_settings(source_object):
) )
def extract_frame_range(image, image_user):
"""Return the clamped (start_frame, frame_count) range extracted from the source."""
total_frames = video_total_frames(image)
if total_frames <= 1:
return 1, total_frames
frame_offset = int(getattr(image_user, "frame_offset", 0) or 0)
image_frames = int(getattr(image_user, "frame_duration", 0) or 0)
frame_count = (
min(image_frames, total_frames)
if image_frames >= 1
else total_frames
)
start_frame = min(1 + frame_offset, total_frames - frame_count + 1)
start_frame = max(start_frame, 1)
return start_frame, frame_count
def image_path(image): def image_path(image):
filepath_from_user = getattr(image, "filepath_from_user", None) filepath_from_user = getattr(image, "filepath_from_user", None)
filepath = ( filepath = (
filepath_from_user() filepath_from_user()
if callable(filepath_from_user) and getattr(image, "source", None) != "SEQUENCE" if callable(filepath_from_user)
else getattr(image, "filepath", "") else getattr(image, "filepath", "")
) )
if not filepath: if not filepath:
@@ -371,7 +328,7 @@ def cleanup_image_input(path, temporary):
def image_base_name(image): def image_base_name(image):
name = image.name name = image.name
lower_name = name.lower() lower_name = name.lower()
for suffix in IMAGE_NAME_SUFFIXES + VIDEO_NAME_SUFFIXES: for suffix in IMAGE_NAME_SUFFIXES:
suffix_index = lower_name.rfind(suffix) suffix_index = lower_name.rfind(suffix)
if suffix_index <= 0: if suffix_index <= 0:
continue continue
@@ -409,137 +366,35 @@ def save_packed_image(image, _scene):
return output_path return output_path
def collect_sequence_frames(image):
"""Return the numbered frame files of a SEQUENCE image, or [] when absent."""
source_path = image_path(image)
if not is_animated_image(image) or not source_path.is_file():
return []
match = re.match(
r"^(?P<prefix>.*?)(?P<digits>\d+)(?P<suffix>\.[^.]+)$",
source_path.name,
)
if match is None:
return []
prefix = match.group("prefix")
suffix = match.group("suffix").lower()
frames = []
for path in source_path.parent.iterdir():
frame_match = re.match(
r"^(?P<prefix>.*?)(?P<digits>\d+)(?P<suffix>\.[^.]+)$",
path.name,
)
if (
frame_match is not None
and frame_match.group("prefix") == prefix
and frame_match.group("suffix").lower() == suffix
):
frames.append((int(frame_match.group("digits")), path))
return [path for _digits, path in sorted(frames)]
def prepare_image_input(image, scene): def prepare_image_input(image, scene):
source_path = image_path(image) """Prepare one still image for a Server job."""
if is_animated_image(image):
raise RuntimeError("Only still images are supported")
if image_is_packed(image): if image_is_packed(image):
return save_packed_image(image, scene), True return save_packed_image(image, scene), True
if getattr(image, "source", None) == "MOVIE": source_path = image_path(image)
if not source_path.is_file():
raise RuntimeError(
f"Source video file does not exist: {source_path}"
)
return source_path, False
if getattr(image, "source", None) == "SEQUENCE":
validate_input_path(source_path, images_only=True)
sequence_frames = collect_sequence_frames(image)
if sequence_frames:
temporary_directory = Path(tempfile.mkdtemp(prefix="anyimage-input-"))
try:
for index, frame_path in enumerate(sequence_frames):
suffix = frame_path.suffix.lower() or ".jpg"
shutil.copy2(
frame_path,
temporary_directory / f"frame.{index + 1:04d}{suffix}",
)
except Exception:
shutil.rmtree(temporary_directory, ignore_errors=True)
raise
return temporary_directory, True
if not source_path.is_file(): if not source_path.is_file():
raise RuntimeError( raise RuntimeError(f"Source image file does not exist: {source_path}")
f"Source image file does not exist: {source_path}"
)
return source_path, False return source_path, False
def configure_image_user( def load_image_edit_result(source_image, output_path):
image_user, """Load and pack the image file returned by a Job."""
frame_count=None, output_path = Path(output_path)
frame_start=1, if not output_path.is_file():
frame_offset=0, raise RuntimeError(f"Color result image does not exist: {output_path}")
use_cyclic=None, result_image = None
):
"""Map the image to scene frames and enable auto refresh."""
if image_user is None:
return
if hasattr(image_user, "frame_start"):
image_user.frame_start = frame_start
if hasattr(image_user, "frame_offset"):
image_user.frame_offset = frame_offset
if hasattr(image_user, "use_auto_refresh"):
image_user.use_auto_refresh = True
if use_cyclic is not None and hasattr(image_user, "use_cyclic"):
image_user.use_cyclic = use_cyclic
if frame_count is not None and hasattr(image_user, "frame_duration"):
image_user.frame_duration = frame_count
def pack_sequence_if_possible(image, expected_frames):
"""Pack a frame sequence when Blender supports it; otherwise keep external files."""
if expected_frames <= 1 or not image.has_data:
return False
try: try:
image.pack() result_image = bpy.data.images.load(str(output_path), check_existing=False)
except (OSError, RuntimeError): result_image.colorspace_settings.name = source_image.colorspace_settings.name
return False result_image.alpha_mode = source_image.alpha_mode
if len(getattr(image, "packed_files", ())) >= expected_frames: result_image.pack()
return True result_image.name = source_image.name
try: except (OSError, RuntimeError, TypeError, ValueError) as error:
image.unpack(method="REMOVE") if result_image is not None:
except (OSError, RuntimeError): bpy.data.images.remove(result_image, do_unlink=True)
pass raise RuntimeError("Unable to load the color result") from error
return False return result_image
def sorted_frame_paths(directory, prefix=None):
"""Return image files in a directory sorted by their trailing frame number."""
def frame_key(path):
match = re.search(r"(\d+)(?=\.[^.]+$)", path.name)
return (int(match.group(1)) if match is not None else 0, path.name)
return sorted(
(
path
for path in Path(directory).iterdir()
if path.is_file()
and path.suffix.lower() in IMAGE_NAME_SUFFIXES
and (prefix is None or path.stem.startswith(prefix))
),
key=frame_key,
)
def force_sequence_source(image):
"""Ensure a loaded image is a SEQUENCE datablock for its numbered frames."""
if getattr(image, "source", None) == "SEQUENCE":
return image
try:
image.source = "SEQUENCE"
except (OSError, RuntimeError, TypeError) as error:
raise RuntimeError(
"Blender cannot load the frame sequence"
) from error
if getattr(image, "source", None) != "SEQUENCE":
raise RuntimeError("Blender cannot load the frame sequence")
return image
def load_normal_result_image(path, source_object): def load_normal_result_image(path, source_object):
@@ -556,22 +411,6 @@ def color_alpha_mode(color_space):
return "STRAIGHT" if color_space in dict(VIEW_TRANSFORM_COLOR_SPACES).values() else "PREMUL" return "STRAIGHT" if color_space in dict(VIEW_TRANSFORM_COLOR_SPACES).values() else "PREMUL"
def configure_color_alpha(image):
"""Apply the presentation mode to an owned Color image."""
alpha_mode = color_alpha_mode(image.colorspace_settings.name)
if image.alpha_mode != alpha_mode:
pixels = (
image_pixels(image)
if getattr(image, "source", None) == "GENERATED"
or getattr(image, "is_dirty", False)
else None
)
image.alpha_mode = alpha_mode
if pixels is not None:
image.pixels.foreach_set(pixels)
image.update()
def create_image_edit_result(source_image, pixels, output_size): def create_image_edit_result(source_image, pixels, output_size):
"""Create a packed Image datablock from edited RGBA pixels.""" """Create a packed Image datablock from edited RGBA pixels."""
output_width, output_height = (int(value) for value in output_size) output_width, output_height = (int(value) for value in output_size)
@@ -680,11 +519,6 @@ def replace_empty_image(source_object, result_image, *, placement_bounds=None,
image_user = getattr(source_object, "image_user", None) image_user = getattr(source_object, "image_user", None)
original_timing = image_user_settings(source_object) if image_user is not None else None original_timing = image_user_settings(source_object) if image_user is not None else None
original_auto_refresh = getattr(image_user, "use_auto_refresh", False) original_auto_refresh = getattr(image_user, "use_auto_refresh", False)
frame_count = (
video_total_frames(result_image)
if is_animated_image(result_image)
else None
)
alignment = ( alignment = (
empty_display_alignment( empty_display_alignment(
source_object, source_object,
@@ -695,13 +529,6 @@ def replace_empty_image(source_object, result_image, *, placement_bounds=None,
if placement_bounds is not None if placement_bounds is not None
else None else None
) )
if frame_count is not None:
(
original_frame_start,
_original_frame_offset,
_original_frame_duration,
original_use_cyclic,
) = image_user_settings(source_object)
source_image.use_fake_user = True source_image.use_fake_user = True
original_alignment = ( original_alignment = (
@@ -720,13 +547,6 @@ def replace_empty_image(source_object, result_image, *, placement_bounds=None,
final_image = source_image final_image = source_image
source_object.data = final_image source_object.data = final_image
source_object.use_empty_image_alpha = True source_object.use_empty_image_alpha = True
if frame_count is not None:
configure_image_user(
image_user,
frame_count,
frame_start=original_frame_start,
use_cyclic=original_use_cyclic,
)
if alignment is not None: if alignment is not None:
source_object.empty_display_size = alignment[0] source_object.empty_display_size = alignment[0]
source_object.empty_image_offset = alignment[1] source_object.empty_image_offset = alignment[1]
@@ -751,64 +571,6 @@ def replace_empty_image(source_object, result_image, *, placement_bounds=None,
bpy.data.images.remove(result_image) bpy.data.images.remove(result_image)
def _load_packed_result_image(
source_image,
output_directory,
*,
frame_prefix,
result_name,
alpha_mode,
):
output_directory = Path(output_directory)
color_paths = sorted_frame_paths(output_directory, frame_prefix)
if not color_paths:
raise RuntimeError(
f"Color result has no images: {output_directory}"
)
source_color_space = source_image.colorspace_settings.name
animated = is_animated_image(source_image)
try:
result_image = bpy.data.images.load(
str(color_paths[0]),
check_existing=False,
)
result_image.colorspace_settings.name = source_color_space
result_image.alpha_mode = alpha_mode
if animated:
force_sequence_source(result_image)
except (OSError, RuntimeError, TypeError, ValueError) as error:
if "result_image" in locals():
bpy.data.images.remove(result_image, do_unlink=True)
raise RuntimeError("Unable to load the color result") from error
try:
if animated:
pack_sequence_if_possible(result_image, len(color_paths))
else:
result_image.pack()
result_image.name = result_name
except (OSError, RuntimeError, TypeError, ValueError) as error:
bpy.data.images.remove(result_image, do_unlink=True)
raise RuntimeError("Unable to prepare the color result") from error
return result_image
def load_image_edit_result(
source_image,
output_directory,
*,
frame_prefix="image.",
):
"""Load one packed Job result that directly edits the source Image."""
return _load_packed_result_image(
source_image,
output_directory,
frame_prefix=frame_prefix,
result_name=source_image.name,
alpha_mode=source_image.alpha_mode,
)
def image_empty_bounds(source_object): def image_empty_bounds(source_object):
image_width, image_height = source_object.data.size image_width, image_height = source_object.data.size
if image_width <= 0 or image_height <= 0: if image_width <= 0 or image_height <= 0:

24
src/anyimage/common/image_target.py
View File

@@ -6,8 +6,6 @@ from uuid import uuid4
import bpy import bpy
from .image import ( from .image import (
configure_image_user,
extract_frame_range,
image_content_state, image_content_state,
image_user_settings, image_user_settings,
is_animated_image, is_animated_image,
@@ -16,7 +14,6 @@ from .image import (
prepare_image_input, prepare_image_input,
replace_empty_image, replace_empty_image,
restore_image_content, restore_image_content,
video_total_frames,
) )
@@ -84,7 +81,6 @@ def replace_texture_image(node, result_image):
original_timing = image_user_settings(node) original_timing = image_user_settings(node)
original_auto_refresh = node.image_user.use_auto_refresh original_auto_refresh = node.image_user.use_auto_refresh
original_content = None original_content = None
frame_count = video_total_frames(result_image) if is_animated_image(result_image) else None
try: try:
if has_other_image_user(node): if has_other_image_user(node):
final_image = result_image final_image = result_image
@@ -94,13 +90,6 @@ def replace_texture_image(node, result_image):
restore_image_content(source_image, result_content) restore_image_content(source_image, result_content)
final_image = source_image final_image = source_image
node.image = final_image node.image = final_image
if frame_count is not None:
configure_image_user(
node.image_user,
frame_count,
frame_start=original_timing[0],
use_cyclic=original_timing[3],
)
except Exception: except Exception:
if original_content is not None: if original_content is not None:
restore_image_content(source_image, original_content) restore_image_content(source_image, original_content)
@@ -122,8 +111,6 @@ class ImageEditTarget:
owner: object owner: object
image: object image: object
tree: object = None tree: object = None
start_frame: int = 1
frame_count: int = 1
node_identity: str = "" node_identity: str = ""
@classmethod @classmethod
@@ -132,7 +119,8 @@ class ImageEditTarget:
if owner is None: if owner is None:
raise RuntimeError("Select an Image Empty or a material Image Texture") raise RuntimeError("Select an Image Empty or a material Image Texture")
image = owner_image(owner) image = owner_image(owner)
start_frame, frame_count = extract_frame_range(image, owner.image_user) if is_animated_image(image):
raise RuntimeError("Only still images are supported")
tree = None if is_image_empty(owner) else owner.id_data tree = None if is_image_empty(owner) else owner.id_data
identity = "" identity = ""
if tree is not None: if tree is not None:
@@ -141,7 +129,7 @@ class ImageEditTarget:
identity = uuid4().hex identity = uuid4().hex
owner["anyimage_identity"] = identity owner["anyimage_identity"] = identity
return cls( return cls(
owner, image, tree, start_frame, frame_count, identity, owner, image, tree, identity,
) )
def validate(self): def validate(self):
@@ -165,13 +153,11 @@ class ImageEditTarget:
def prepare(self, context): def prepare(self, context):
self.validate() self.validate()
if video_total_frames(self.image) > 1:
context.scene.anyimage_settings.video_frames = self.frame_count
return prepare_image_input(self.image, context.scene) return prepare_image_input(self.image, context.scene)
def apply(self, directory, *, frame_prefix="image."): def apply(self, output_path):
self.validate() self.validate()
result_image = load_image_edit_result(self.image, directory, frame_prefix=frame_prefix) result_image = load_image_edit_result(self.image, output_path)
return self.commit(result_image) return self.commit(result_image)
def commit(self, result_image, *, isolate_shared=False): def commit(self, result_image, *, isolate_shared=False):

4
src/anyimage/common/viewport.py
View File

@@ -10,7 +10,7 @@ from .image import (
image_empty_bounds, image_empty_bounds,
is_image_empty, is_image_empty,
require_image_empty, require_image_empty,
video_total_frames, is_animated_image,
) )
from .selection import ImageEditWarning, SelectionPath from .selection import ImageEditWarning, SelectionPath
@@ -1019,7 +1019,7 @@ class ImageGesture:
return {"CANCELLED"} return {"CANCELLED"}
if source_object is None: if source_object is None:
return {"CANCELLED"} return {"CANCELLED"}
if video_total_frames(source_object.data) > 1: if is_animated_image(source_object.data):
self.report({"WARNING"}, "Image gestures only support a still image") self.report({"WARNING"}, "Image gestures only support a still image")
return {"CANCELLED"} return {"CANCELLED"}
self.source_object_name = source_object.name self.source_object_name = source_object.name

28
src/anyimage/operators/__init__.py
View File

@@ -1,17 +1,9 @@
from .remove_background import RemoveImageBackground, RunBackgroundRemoval from .remove_background import RemoveImageBackground, RunBackgroundRemoval
from .cutout_tool import ( from .cutout_tool import CLASSES as CUTOUT_CLASSES
CutoutSelectionToShape, from .frame_tool import CLASSES as FRAME_CLASSES
SelectCutoutSelection,
)
from .frame_tool import FrameImages
from .mask_tool import EditImageAlpha from .mask_tool import EditImageAlpha
from .rectify_tool import RectifyImagePerspective from .rectify_tool import CLASSES as RECTIFY_CLASSES
from .convert_to_plane import ( from .convert_to_plane import CLASSES as PLANE_CLASSES
ConvertToDepthPlane,
ConvertToReliefPlane,
ConvertToPlane,
GenerateDepthPlane,
)
from .ai_setup import ( from .ai_setup import (
ClearModels, ClearModels,
DownloadModel, DownloadModel,
@@ -25,21 +17,17 @@ from .convert_to_panorama import CLASSES as PANORAMA_CLASSES
CLASSES = ( CLASSES = (
*PANORAMA_CLASSES, *PANORAMA_CLASSES,
*PLANE_CLASSES,
*CUTOUT_CLASSES,
SetupAIEnvironment, SetupAIEnvironment,
OpenAIEnvironmentSettings, OpenAIEnvironmentSettings,
DownloadModel, DownloadModel,
DownloadRequiredModels, DownloadRequiredModels,
GenerateDepthPlane,
CutoutSelectionToShape,
SelectCutoutSelection,
RunBackgroundRemoval, RunBackgroundRemoval,
ClearModels, ClearModels,
FrameImages, *FRAME_CLASSES,
EditImageAlpha, EditImageAlpha,
RectifyImagePerspective, *RECTIFY_CLASSES,
RemoveImageBackground, RemoveImageBackground,
UpscaleImage, UpscaleImage,
ConvertToPlane,
ConvertToDepthPlane,
ConvertToReliefPlane,
) )

47
src/anyimage/operators/ai_setup.py
View File

@@ -21,11 +21,8 @@ class SetupAIEnvironment(bpy.types.Operator):
bl_description = "Install the environment and required AI models" bl_description = "Install the environment and required AI models"
use_mirror: bpy.props.BoolProperty( use_mirror: bpy.props.BoolProperty(
name="Use Mirror", name="Use China Mirror",
description=( description="Download Python and packages from China mirrors",
"Use the China mirrors for uv, Python and Python packages; "
"model download sources are unchanged"
),
default=False, default=False,
) )
@@ -34,36 +31,24 @@ class SetupAIEnvironment(bpy.types.Operator):
return not runtime.server_busy() return not runtime.server_busy()
def invoke(self, context, _event): def invoke(self, context, _event):
return context.window_manager.invoke_props_dialog(self, width=440) return context.window_manager.invoke_props_dialog(
self, width=440, title="Set Up AI", confirm_text="Install"
)
def draw(self, _context): def draw(self, _context):
layout = self.layout layout = self.layout
status = ai_status() status = ai_status()
if not status["environment_ready"]: if not status["environment_ready"]:
required_size = sum( required_size = sum(
model_download_size(model_key) model_download_size(model_key) for model_key in DEFAULT_MODEL_KEYS
for model_key in DEFAULT_MODEL_KEYS
) )
layout.label( layout.label(
text="AnyImage AI features run in an isolated Python environment." text="Set up AI features for depth, background removal and upscaling."
)
layout.label(
text="This installs Python, ONNX Runtime and supporting packages."
) )
layout.label( layout.label(
text=( text=(
"It also downloads MoGe-2 ViT-S Normal, BiRefNet Lite and " "Downloads the runtime and required models "
"Real-ESRGAN General WDN x4v3 " f"({format_download_size(required_size)} for models)."
f"({format_download_size(required_size)})."
)
)
layout.label(
text="Allow about 600 MB of downloads and 1 GB of disk space."
)
layout.label(
text=(
"Installation usually takes 5–20 minutes, "
"depending on network speed."
) )
) )
layout.separator(type="LINE") layout.separator(type="LINE")
@@ -71,7 +56,7 @@ class SetupAIEnvironment(bpy.types.Operator):
return return
if status["ready"]: if status["ready"]:
layout.label(text="The AI Environment is ready.", icon="CHECKMARK") layout.label(text="AI Server is ready.", icon="CHECKMARK")
return return
labels = [] labels = []
@@ -83,12 +68,13 @@ class SetupAIEnvironment(bpy.types.Operator):
for model_key in status["missing_models"] for model_key in status["missing_models"]
) )
layout.label( layout.label(
text="The AI Environment is already installed.", text="AI environment installed.",
icon="CHECKMARK", icon="CHECKMARK",
) )
layout.label(text=f"Required models to download: {', '.join(labels)}") layout.label(text=f"Download: {', '.join(labels)}")
layout.label(text=f"Download size: {format_download_size(missing_size)}") layout.label(
layout.label(text="The download usually takes 2–10 minutes.") text=f"{format_download_size(missing_size)} · Usually 2–10 minutes."
)
def execute(self, _context): def execute(self, _context):
status = ai_status() status = ai_status()
@@ -159,7 +145,8 @@ class DownloadRequiredModels(JobOperatorBase, bpy.types.Operator):
class ClearModels(bpy.types.Operator): class ClearModels(bpy.types.Operator):
bl_idname = "anyimage.clear_models" bl_idname = "anyimage.clear_models"
bl_label = "Clear Models" bl_label = "Unload Models"
bl_description = "Unload AI models to free memory. Downloaded model files are kept."
def execute(self, _context): def execute(self, _context):
try: try:

3
src/anyimage/operators/clipboard_image/__init__.py
View File

@@ -1,11 +1,12 @@
from .operators import ( from .operators import (
CLASSES,
PasteClipboardImage, PasteClipboardImage,
TrackNativeCopy, TrackNativeCopy,
register_keymaps, register_keymaps,
unregister_keymaps, unregister_keymaps,
) )
CLASSES = (TrackNativeCopy, PasteClipboardImage)
__all__ = ( __all__ = (
"CLASSES", "CLASSES",
"PasteClipboardImage", "PasteClipboardImage",

3
src/anyimage/operators/clipboard_image/operators.py
View File

@@ -148,9 +148,6 @@ class PasteClipboardImage(bpy.types.Operator):
return {"FINISHED"} return {"FINISHED"}
CLASSES = (TrackNativeCopy, PasteClipboardImage)
def unregister_keymaps(): def unregister_keymaps():
global _native_copy_token global _native_copy_token
for keymap, keymap_item in _keymap_items: for keymap, keymap_item in _keymap_items:

12
src/anyimage/operators/convert_to_panorama/operators.py
View File

@@ -3,7 +3,7 @@
import bpy import bpy
from ...common.ai import invoke_ai_setup_if_needed, production_device, report_ai_error, require_environment, require_input_path, require_model from ...common.ai import invoke_ai_setup_if_needed, production_device, report_ai_error, require_environment, require_input_path, require_model
from ...common.image import is_image_empty, video_total_frames from ...common.image import is_image_empty, is_animated_image
from ...common.color_image import prepare_material_color_input, material_analysis_input, cleanup_material_color_input from ...common.color_image import prepare_material_color_input, material_analysis_input, cleanup_material_color_input
from ...preferences import addon_preferences, configured_max_ai_input_size from ...preferences import addon_preferences, configured_max_ai_input_size
from ...preferences import configured_geometry_model from ...preferences import configured_geometry_model
@@ -26,7 +26,7 @@ class GeneratePanorama(JobOperatorBase, bpy.types.Operator):
source_object_name: bpy.props.StringProperty(options=_HIDDEN) source_object_name: bpy.props.StringProperty(options=_HIDDEN)
source_identity: bpy.props.StringProperty(options=_HIDDEN) source_identity: bpy.props.StringProperty(options=_HIDDEN)
image_identity: bpy.props.StringProperty(options=_HIDDEN) image_identity: bpy.props.StringProperty(options=_HIDDEN)
mesh_detail: bpy.props.IntProperty(default=7, min=0, max=8, options=_HIDDEN) mesh_detail: bpy.props.IntProperty(default=8, min=0, max=8, options=_HIDDEN)
model: bpy.props.StringProperty(default=DEFAULT_GEOMETRY_MODEL_KEY, options=_HIDDEN) model: bpy.props.StringProperty(default=DEFAULT_GEOMETRY_MODEL_KEY, options=_HIDDEN)
resolution_level: bpy.props.IntProperty(default=5, min=0, max=9, options=_HIDDEN) resolution_level: bpy.props.IntProperty(default=5, min=0, max=9, options=_HIDDEN)
max_input_size: bpy.props.IntProperty(default=DEFAULT_MAX_AI_INPUT_SIZE, max_input_size: bpy.props.IntProperty(default=DEFAULT_MAX_AI_INPUT_SIZE,
@@ -39,7 +39,7 @@ class GeneratePanorama(JobOperatorBase, bpy.types.Operator):
"input": str(require_input_path(self.input_path)), "input": str(require_input_path(self.input_path)),
"device": production_device().lower(), "model": self.model, "device": production_device().lower(), "model": self.model,
"resolution_level": int(self.resolution_level), "resolution_level": int(self.resolution_level),
"max_input_size": int(self.max_input_size), "video_frames": 1, "start_frame": 1, "max_input_size": int(self.max_input_size),
} }
def response(self, context, result): def response(self, context, result):
@@ -53,10 +53,10 @@ class GeneratePanorama(JobOperatorBase, bpy.types.Operator):
class ConvertToPanorama(bpy.types.Operator): class ConvertToPanorama(bpy.types.Operator):
bl_idname = "anyimage.convert_to_panorama" bl_idname = "anyimage.convert_to_panorama"
bl_label = "Convert to Panorama" bl_label = "Convert to Panorama"
bl_description = "Convert a static 2:1 equirectangular Image Empty into a radial depth surface" bl_description = "Turn a still 2:1 panoramic image into a surrounding 3D surface using AI-estimated depth."
bl_options = {"REGISTER", "UNDO"} bl_options = {"REGISTER", "UNDO"}
mesh_detail: bpy.props.IntProperty(name="Subdivide", default=7, min=0, max=8) mesh_detail: bpy.props.IntProperty(name="Subdivide", default=8, min=0, max=8)
@classmethod @classmethod
def poll(cls, context): def poll(cls, context):
@@ -68,7 +68,7 @@ class ConvertToPanorama(bpy.types.Operator):
source = context.object source = context.object
image = source.data image = source.data
width, height = image.size width, height = image.size
if image.source in {"MOVIE", "SEQUENCE"} or video_total_frames(image) != 1 or height <= 0 or width != height * 2: if is_animated_image(image) or height <= 0 or width != height * 2:
self.report({"ERROR"}, "Panorama conversion requires a static 2:1 equirectangular image") self.report({"ERROR"}, "Panorama conversion requires a static 2:1 equirectangular image")
return {"CANCELLED"} return {"CANCELLED"}
setup = invoke_ai_setup_if_needed(self) setup = invoke_ai_setup_if_needed(self)

3
src/anyimage/operators/convert_to_plane/__init__.py
View File

@@ -7,7 +7,10 @@ from .operators import (
GenerateDepthPlane, GenerateDepthPlane,
) )
CLASSES = (GenerateDepthPlane, ConvertToPlane, ConvertToDepthPlane, ConvertToReliefPlane)
__all__ = ( __all__ = (
"CLASSES",
"ConvertToDepthPlane", "ConvertToDepthPlane",
"ConvertToReliefPlane", "ConvertToReliefPlane",
"ConvertToPlane", "ConvertToPlane",

11
src/anyimage/operators/convert_to_plane/operators.py
View File

@@ -75,18 +75,14 @@ class GenerateDepthPlane(JobOperatorBase, bpy.types.Operator):
parameters = { parameters = {
"input": str(input_path), "input": str(input_path),
"device": production_device().lower(), "device": production_device().lower(),
"video_frames": 1,
"start_frame": 1,
"max_input_size": int(self.max_input_size), "max_input_size": int(self.max_input_size),
"plane_type": self.plane_type, "plane_type": self.plane_type,
} }
parameters.update( parameters.update(
moge2_parameters( moge2_parameters(
context.scene.anyimage_settings,
input_path, input_path,
self.model, self.model,
self.resolution_level, self.resolution_level,
1,
) )
) )
return parameters return parameters
@@ -175,7 +171,10 @@ class ConvertToPlane(bpy.types.Operator):
class ConvertToDepthPlane(bpy.types.Operator): class ConvertToDepthPlane(bpy.types.Operator):
bl_idname = "anyimage.convert_to_depth_plane" bl_idname = "anyimage.convert_to_depth_plane"
bl_label = "Convert to Depth Plane" bl_label = "Convert to Depth Plane"
bl_description = "Convert the complete Image Empty into a camera-projected depth plane" bl_description = (
"Use AI-estimated depth to turn the image into a 3D surface "
"that follows the source image's perspective."
)
bl_options = {"REGISTER", "UNDO"} bl_options = {"REGISTER", "UNDO"}
mesh_detail: bpy.props.IntProperty(name="Subdivide", default=6, min=0, max=10) mesh_detail: bpy.props.IntProperty(name="Subdivide", default=6, min=0, max=10)
@@ -193,7 +192,7 @@ class ConvertToDepthPlane(bpy.types.Operator):
class ConvertToReliefPlane(bpy.types.Operator): class ConvertToReliefPlane(bpy.types.Operator):
bl_idname = "anyimage.convert_to_relief_plane" bl_idname = "anyimage.convert_to_relief_plane"
bl_label = "Convert to Relief Plane" bl_label = "Convert to Relief Plane"
bl_description = "Convert the complete Image Empty into a fixed-base relief plane" bl_description = "Use AI-estimated depth to create a raised relief on a flat base."
bl_options = {"REGISTER", "UNDO"} bl_options = {"REGISTER", "UNDO"}
mesh_detail: bpy.props.IntProperty(name="Subdivide", default=6, min=0, max=10) mesh_detail: bpy.props.IntProperty(name="Subdivide", default=6, min=0, max=10)

3
src/anyimage/operators/cutout_tool/__init__.py
View File

@@ -1,6 +1,9 @@
from .operators import CutoutSelectionToShape, CutoutTool, SelectCutoutSelection from .operators import CutoutSelectionToShape, CutoutTool, SelectCutoutSelection
CLASSES = (CutoutSelectionToShape, SelectCutoutSelection)
__all__ = ( __all__ = (
"CLASSES",
"CutoutSelectionToShape", "CutoutSelectionToShape",
"CutoutTool", "CutoutTool",
"SelectCutoutSelection", "SelectCutoutSelection",

396
src/anyimage/operators/cutout_tool/balloon.py Normal file
View File

@@ -0,0 +1,396 @@
"""Compute Poisson and Teddy height profiles for Cutout meshes."""
import math
DEFAULT_INFLATION = 0.45
def distances_to_contour(points, contour):
import numpy as np
points = np.asarray(points, dtype=np.float64)
contour = np.asarray(contour, dtype=np.float64)
squared = np.full(len(points), np.inf, dtype=np.float64)
for start, end in zip(contour, np.roll(contour, -1, axis=0)):
segment = end - start
length_squared = float(segment.dot(segment))
if length_squared <= 1e-16:
continue
factor = np.clip(
(points - start) @ segment / length_squared,
0.0,
1.0,
)
nearest = start + factor[:, None] * segment
squared = np.minimum(
squared,
np.sum((points - nearest) ** 2, axis=1),
)
return np.sqrt(squared)
def chordal_axis(contour, epsilon=1e-9):
import numpy as np
from mathutils import Vector
from mathutils.geometry import delaunay_2d_cdt
contour = np.asarray(contour, dtype=np.float64)
boundary_count = len(contour)
boundary_edges = [
(index, (index + 1) % boundary_count) for index in range(boundary_count)
]
result = delaunay_2d_cdt(
[Vector(point) for point in contour],
boundary_edges,
[list(range(boundary_count))],
1,
epsilon,
)
points = np.asarray(
[tuple(point) for point in result[0]],
dtype=np.float64,
)
triangles = [tuple(face) for face in result[2] if len(face) == 3]
boundary = {tuple(sorted(edge)) for edge in boundary_edges}
node_indices = {}
nodes = []
segments = set()
def edge_node(edge):
key = ("edge", *tuple(sorted(edge)))
if key not in node_indices:
node_indices[key] = len(nodes)
nodes.append((points[edge[0]] + points[edge[1]]) * 0.5)
return node_indices[key]
for face_index, face in enumerate(triangles):
edges = [
(face[0], face[1]),
(face[1], face[2]),
(face[2], face[0]),
]
internal = [edge for edge in edges if tuple(sorted(edge)) not in boundary]
internal_nodes = [edge_node(edge) for edge in internal]
if len(internal_nodes) == 2:
segments.add(tuple(sorted(internal_nodes)))
elif len(internal_nodes) == 3:
key = ("face", face_index)
center_index = len(nodes)
node_indices[key] = center_index
nodes.append(points[list(face)].mean(axis=0))
for midpoint_index in internal_nodes:
segments.add(tuple(sorted((center_index, midpoint_index))))
if not nodes:
center = contour.mean(axis=0)
return (
np.asarray((center,), dtype=np.float64),
[],
np.asarray(
(distances_to_contour((center,), contour)[0],),
dtype=np.float64,
),
)
nodes = np.asarray(nodes, dtype=np.float64)
radii = distances_to_contour(nodes, contour)
active = set(segments)
while len(active) > 1:
degree = [0] * len(nodes)
for start, end in active:
degree[start] += 1
degree[end] += 1
removable = []
for edge in active:
start, end = edge
leaf = start if degree[start] == 1 else end if degree[end] == 1 else None
if leaf is None:
continue
length = float(np.linalg.norm(nodes[start] - nodes[end]))
if length < max(radii[leaf] * 0.5, epsilon):
removable.append(edge)
if not removable:
break
active.difference_update(removable)
if active:
used = sorted({index for edge in active for index in edge})
remap = {old: new for new, old in enumerate(used)}
nodes = nodes[used]
radii = radii[used]
active = {tuple(sorted((remap[start], remap[end]))) for start, end in active}
else:
widest = int(np.argmax(radii))
nodes = nodes[[widest]]
radii = radii[[widest]]
return nodes, sorted(active), radii
def smooth_interior_heights(
heights,
planar_faces,
boundary_mask,
iterations=2,
factor=0.3,
):
import numpy as np
values = np.asarray(heights, dtype=np.float64).copy()
boundary_mask = np.asarray(boundary_mask, dtype=bool)
neighbors = [set() for _index in range(len(values))]
for face in planar_faces:
for start, end in zip(face, face[1:] + face[:1]):
neighbors[start].add(end)
neighbors[end].add(start)
for _iteration in range(iterations):
updated = values.copy()
for index in np.flatnonzero(~boundary_mask):
adjacent = [
neighbor for neighbor in neighbors[index] if not boundary_mask[neighbor]
]
if adjacent:
average = float(values[adjacent].mean())
updated[index] = values[index] * (1.0 - factor) + average * factor
updated[boundary_mask] = 0.0
values = updated
return values
def teddy_balloon_heights(
planar_points,
planar_faces,
contour,
boundary_mask,
inflation,
):
import numpy as np
points = np.asarray(planar_points, dtype=np.float64)
boundary_mask = np.asarray(boundary_mask, dtype=bool)
nodes, segments, radii = chordal_axis(contour)
squared_heights = np.zeros(len(points), dtype=np.float64)
for start, end in segments:
axis_start = nodes[start]
axis_end = nodes[end]
axis = axis_end - axis_start
length_squared = float(axis.dot(axis))
if length_squared <= 1e-16:
continue
factors = np.clip(
(points - axis_start) @ axis / length_squared,
0.0,
1.0,
)
centers = axis_start + factors[:, None] * axis
local_radii = radii[start] + factors * (radii[end] - radii[start])
distance_squared = np.sum((points - centers) ** 2, axis=1)
squared_heights = np.maximum(
squared_heights,
local_radii * local_radii - distance_squared,
)
degree = np.zeros(len(nodes), dtype=np.int64)
for start, end in segments:
degree[start] += 1
degree[end] += 1
for index in np.flatnonzero(degree != 2):
distance_squared = np.sum((points - nodes[index]) ** 2, axis=1)
squared_heights = np.maximum(
squared_heights,
radii[index] * radii[index] - distance_squared,
)
missing = (~boundary_mask) & (squared_heights <= 0.0)
if np.any(missing):
fallback = distances_to_contour(points[missing], contour)
squared_heights[missing] = fallback * fallback
heights = inflation * np.sqrt(np.maximum(squared_heights, 0.0))
heights[boundary_mask] = 0.0
return smooth_interior_heights(
heights,
planar_faces,
boundary_mask,
)
def cotangent_poisson_field(
planar_points,
planar_faces,
boundary_mask,
tolerance=1e-8,
):
import numpy as np
points = np.asarray(planar_points, dtype=np.float64)
faces = np.asarray(planar_faces, dtype=np.int64)
if faces.size == 0:
faces = np.empty((0, 3), dtype=np.int64)
if faces.ndim != 2 or faces.shape[1] != 3:
raise ValueError("Balloon profile faces must be triangles")
boundary_mask = np.asarray(boundary_mask, dtype=bool)
triangles = points[faces]
a, b, c = (triangles[:, index] for index in range(3))
cross = np.abs(
(b[:, 0] - a[:, 0]) * (c[:, 1] - a[:, 1])
- (b[:, 1] - a[:, 1]) * (c[:, 0] - a[:, 0])
)
usable = cross > 1e-14
faces = faces[usable]
a, b, c = a[usable], b[usable], c[usable]
cross = cross[usable]
loads = np.bincount(
faces.ravel(),
weights=np.repeat(cross / 6.0, 3),
minlength=len(points),
)
edges = np.concatenate(
(
faces[:, (1, 2)],
faces[:, (0, 2)],
faces[:, (0, 1)],
),
axis=0,
)
edge_weights = np.concatenate(
(
np.sum((b - a) * (c - a), axis=1),
np.sum((a - b) * (c - b), axis=1),
np.sum((a - c) * (b - c), axis=1),
)
) / np.tile(cross * 2.0, 3)
edges.sort(axis=1)
edges, inverse = np.unique(edges, axis=0, return_inverse=True)
edge_weights = np.bincount(
inverse,
weights=edge_weights,
minlength=len(edges),
)
interior = np.flatnonzero(~boundary_mask)
field = np.zeros(len(points), dtype=np.float64)
if not len(interior):
return field
compact = np.full(len(points), -1, dtype=np.int64)
compact[interior] = np.arange(len(interior), dtype=np.int64)
# Constrained triangulation can create negative cotangent weights.
# Clamping them keeps the Dirichlet system positive definite.
edge_weights = np.maximum(edge_weights, 0.0)
usable = edge_weights > 1e-14
compact_edges = compact[edges[usable]]
edge_weights = edge_weights[usable]
start_compact = compact_edges[:, 0]
end_compact = compact_edges[:, 1]
start_usable = start_compact >= 0
end_usable = end_compact >= 0
diagonal = np.zeros(len(interior), dtype=np.float64)
diagonal += np.bincount(
start_compact[start_usable],
weights=edge_weights[start_usable],
minlength=len(interior),
)
diagonal += np.bincount(
end_compact[end_usable],
weights=edge_weights[end_usable],
minlength=len(interior),
)
if np.any(diagonal <= 1e-14):
raise ValueError("Unable to build a stable Balloon profile")
interior_edges = start_usable & end_usable
edge_starts = start_compact[interior_edges]
edge_ends = end_compact[interior_edges]
weights = edge_weights[interior_edges]
directed_starts = np.concatenate((edge_starts, edge_ends))
directed_ends = np.concatenate((edge_ends, edge_starts))
directed_weights = np.concatenate((weights, weights))
def multiply(values):
result = diagonal * values
if len(directed_weights):
result -= np.bincount(
directed_starts,
weights=directed_weights * values[directed_ends],
minlength=len(values),
)
return result
right_hand_side = loads[interior]
solution = np.zeros(len(interior), dtype=np.float64)
residual = right_hand_side.copy()
preconditioned = residual / diagonal
direction = preconditioned.copy()
residual_product = float(residual.dot(preconditioned))
target = max(float(np.linalg.norm(right_hand_side)) * tolerance, 1e-12)
max_iterations = max(64, min(len(interior) * 4, 4000))
for _iteration in range(max_iterations):
if float(np.linalg.norm(residual)) <= target:
break
multiplied = multiply(direction)
denominator = float(direction.dot(multiplied))
if denominator <= 0.0 or not math.isfinite(denominator):
raise ValueError("Unable to solve the Balloon profile")
step = residual_product / denominator
solution += step * direction
residual -= step * multiplied
preconditioned = residual / diagonal
next_product = float(residual.dot(preconditioned))
if not math.isfinite(next_product):
raise ValueError("Unable to solve the Balloon profile")
if abs(residual_product) <= 1e-30:
break
direction = preconditioned + next_product / residual_product * direction
residual_product = next_product
field[interior] = np.maximum(solution, 0.0)
return field
def poisson_balloon_profile(
planar_points,
planar_faces,
boundary_mask,
inflation=DEFAULT_INFLATION,
):
import numpy as np
field = cotangent_poisson_field(
planar_points,
planar_faces,
boundary_mask,
)
return 2.0 * float(inflation) * np.sqrt(field)
def balloon_profile(
algorithm,
planar_points,
planar_faces,
boundary_mask,
components,
inflation=DEFAULT_INFLATION,
):
import numpy as np
algorithm = str(algorithm).upper()
if algorithm == "POISSON":
return poisson_balloon_profile(
planar_points,
planar_faces,
boundary_mask,
inflation,
)
if algorithm != "TEDDY":
raise ValueError(f"Unknown Balloon profile algorithm: {algorithm}")
result = np.zeros(len(planar_points), dtype=np.float64)
for start, end, component_faces, outer_contour in components:
result[start:end] = teddy_balloon_heights(
np.asarray(planar_points)[start:end],
component_faces,
outer_contour,
np.asarray(boundary_mask)[start:end],
inflation,
)
return result

169
src/anyimage/operators/cutout_tool/depth_calibration.py Normal file
View File

@@ -0,0 +1,169 @@
"""Calibrate metric reference depth and the Depth Balloon base plane."""
PLANE_VARIANCE_FLOOR_RATIO = 0.025
DEPTH_REFERENCE_PERCENTILE = 95.0
ADAPTIVE_DEPTH_TRANSPARENCY_RATIO = 0.05
DEFAULT_DEPTH_BASELINE = 8.0
DEFAULT_DEPTH_DIRECTION = (0.0, 0.0, 1.0)
def use_adaptive_depth_calibration(selection_transparent_fraction):
"""Return whether the Selection input has enough transparent context."""
return float(selection_transparent_fraction) >= ADAPTIVE_DEPTH_TRANSPARENCY_RATIO
def metric_depth_reference(camera_depth, valid, uniform_scale):
"""Return the shared upper-percentile scene depth for both Depth shapes."""
import numpy as np
camera_depth = np.asarray(camera_depth, dtype=np.float32)
valid = np.asarray(valid, dtype=bool)
if valid.shape != camera_depth.shape:
raise ValueError("Depth values and validity mask dimensions do not match")
usable = valid & np.isfinite(camera_depth) & (camera_depth > 0.0)
if not usable.any():
raise ValueError("The depth field contains no valid metric depth")
if not uniform_scale > 0.0:
raise ValueError("Depth Uniform Scale must be positive")
reference = np.percentile(
camera_depth[usable],
DEPTH_REFERENCE_PERCENTILE,
)
return float(reference) * float(uniform_scale)
def _sample_valid_bilinear(values, valid, x, y):
import numpy as np
height, width = values.shape
x = np.clip(x, 0.0, max(width - 1, 0))
y = np.clip(y, 0.0, max(height - 1, 0))
x0 = np.floor(x).astype(np.int64)
y0 = np.floor(y).astype(np.int64)
x1 = np.minimum(x0 + 1, width - 1)
y1 = np.minimum(y0 + 1, height - 1)
fx = x - x0
fy = y - y0
samples = np.stack(
(
values[y0, x0],
values[y0, x1],
values[y1, x0],
values[y1, x1],
),
axis=1,
)
weights = np.stack(
(
(1.0 - fx) * (1.0 - fy),
fx * (1.0 - fy),
(1.0 - fx) * fy,
fx * fy,
),
axis=1,
)
sample_valid = np.stack(
(
valid[y0, x0],
valid[y0, x1],
valid[y1, x0],
valid[y1, x1],
),
axis=1,
)
weights *= sample_valid
totals = weights.sum(axis=1)
result = np.zeros(len(x), dtype=np.float32)
usable = totals > 1e-8
result[usable] = (samples[usable] * weights[usable]).sum(axis=1) / totals[usable]
return result, usable
def _weighted_plane(coordinates, values, weights):
import numpy as np
total = float(np.sum(weights))
if total <= 1e-12:
return np.zeros(3, dtype=np.float32)
center = np.sum(coordinates * weights[:, None], axis=0) / total
value_center = float(np.sum(values * weights) / total)
centered = coordinates - center
covariance = (centered * weights[:, None]).T @ centered / total
cross = centered.T @ (weights * (values - value_center)) / total
eigenvalues, eigenvectors = np.linalg.eigh(covariance)
largest = max(float(eigenvalues[-1]), 0.0)
if largest <= 1e-12:
slopes = np.zeros(2, dtype=np.float64)
else:
supported = np.maximum(eigenvalues, largest * PLANE_VARIANCE_FLOOR_RATIO)
slopes = eigenvectors @ ((eigenvectors.T @ cross) / supported)
intercept = value_center - float(center @ slopes)
return np.asarray((*slopes, intercept), dtype=np.float32)
def _stable_plane(coordinates, values):
import numpy as np
coordinates = np.asarray(coordinates, dtype=np.float64)
values = np.asarray(values, dtype=np.float64)
if len(coordinates) < 2:
return np.asarray((0.0, 0.0, np.median(values)), dtype=np.float32)
weights = np.ones(len(values), dtype=np.float64)
plane = np.zeros(3, dtype=np.float32)
for _iteration in range(4):
plane = _weighted_plane(coordinates, values, weights)
residual = values - coordinates @ plane[:2] - plane[2]
centered = residual - np.median(residual)
scale = max(float(np.median(np.abs(centered))) * 1.4826, 1e-6)
weights = np.minimum(
1.0,
(2.5 * scale) / np.maximum(np.abs(centered), 1e-12),
)
return plane
def depth_balloon_projection_parameters(
vertices,
vertex_uv,
camera_depth,
valid,
uniform_scale,
):
"""Fit a metric base plane and return its center depth and direction."""
import numpy as np
vertices = np.asarray(vertices, dtype=np.float32)
uv = np.asarray(vertex_uv, dtype=np.float32)
if vertices.ndim != 2 or vertices.shape[1] != 3:
raise ValueError("Depth Balloon vertices must have three components")
if uv.shape != (len(vertices), 2):
raise ValueError("Depth Balloon UV coordinates do not match its vertices")
camera_depth = np.asarray(camera_depth, dtype=np.float32)
valid = np.asarray(valid, dtype=bool)
if valid.shape != camera_depth.shape:
raise ValueError("Depth Balloon depth and mask dimensions do not match")
if not uniform_scale > 0.0:
raise ValueError("Depth Balloon Uniform Scale must be positive")
height, width = camera_depth.shape
sample_x = uv[:, 0] * max(width - 1, 0)
sample_y = (1.0 - uv[:, 1]) * max(height - 1, 0)
calibration_valid = valid & np.isfinite(camera_depth) & (camera_depth > 0.0)
sampled, usable = _sample_valid_bilinear(
camera_depth,
calibration_valid,
sample_x,
sample_y,
)
if not usable.any():
raise ValueError("MoGe-2 found no valid Depth Balloon samples")
local_coordinates = vertices[:, :2]
metric_depth = sampled[usable] * float(uniform_scale)
plane = _stable_plane(local_coordinates[usable], metric_depth)
reference = metric_depth_reference(
camera_depth,
calibration_valid,
uniform_scale,
)
direction = np.asarray((plane[0], plane[1], 1.0), dtype=np.float64)
direction /= np.linalg.norm(direction)
return reference, tuple(float(value) for value in direction)

565
src/anyimage/operators/cutout_tool/geometry.py
View File

@@ -5,6 +5,7 @@ from dataclasses import dataclass
from ...preferences import addon_preferences from ...preferences import addon_preferences
from ...common.image import image_empty_bounds from ...common.image import image_empty_bounds
from . import balloon
@dataclass(frozen=True) @dataclass(frozen=True)
@@ -661,568 +662,6 @@ def triangulate_selection_contours(
return (*result, components) if return_components else result return (*result, components) if return_components else result
DEFAULT_INFLATION = 0.45
def distances_to_contour(points, contour):
import numpy as np
points = np.asarray(points, dtype=np.float64)
contour = np.asarray(contour, dtype=np.float64)
squared = np.full(len(points), np.inf, dtype=np.float64)
for start, end in zip(contour, np.roll(contour, -1, axis=0)):
segment = end - start
length_squared = float(segment.dot(segment))
if length_squared <= 1e-16:
continue
factor = np.clip(
(points - start) @ segment / length_squared,
0.0,
1.0,
)
nearest = start + factor[:, None] * segment
squared = np.minimum(
squared,
np.sum((points - nearest) ** 2, axis=1),
)
return np.sqrt(squared)
def chordal_axis(contour, epsilon=1e-9):
import numpy as np
from mathutils import Vector
from mathutils.geometry import delaunay_2d_cdt
contour = np.asarray(contour, dtype=np.float64)
boundary_count = len(contour)
boundary_edges = [
(index, (index + 1) % boundary_count) for index in range(boundary_count)
]
result = delaunay_2d_cdt(
[Vector(point) for point in contour],
boundary_edges,
[list(range(boundary_count))],
1,
epsilon,
)
points = np.asarray(
[tuple(point) for point in result[0]],
dtype=np.float64,
)
triangles = [tuple(face) for face in result[2] if len(face) == 3]
boundary = {tuple(sorted(edge)) for edge in boundary_edges}
node_indices = {}
nodes = []
segments = set()
def edge_node(edge):
key = ("edge", *tuple(sorted(edge)))
if key not in node_indices:
node_indices[key] = len(nodes)
nodes.append((points[edge[0]] + points[edge[1]]) * 0.5)
return node_indices[key]
for face_index, face in enumerate(triangles):
edges = [
(face[0], face[1]),
(face[1], face[2]),
(face[2], face[0]),
]
internal = [edge for edge in edges if tuple(sorted(edge)) not in boundary]
internal_nodes = [edge_node(edge) for edge in internal]
if len(internal_nodes) == 2:
segments.add(tuple(sorted(internal_nodes)))
elif len(internal_nodes) == 3:
key = ("face", face_index)
center_index = len(nodes)
node_indices[key] = center_index
nodes.append(points[list(face)].mean(axis=0))
for midpoint_index in internal_nodes:
segments.add(tuple(sorted((center_index, midpoint_index))))
if not nodes:
center = contour.mean(axis=0)
return (
np.asarray((center,), dtype=np.float64),
[],
np.asarray(
(distances_to_contour((center,), contour)[0],),
dtype=np.float64,
),
)
nodes = np.asarray(nodes, dtype=np.float64)
radii = distances_to_contour(nodes, contour)
active = set(segments)
while len(active) > 1:
degree = [0] * len(nodes)
for start, end in active:
degree[start] += 1
degree[end] += 1
removable = []
for edge in active:
start, end = edge
leaf = start if degree[start] == 1 else end if degree[end] == 1 else None
if leaf is None:
continue
length = float(np.linalg.norm(nodes[start] - nodes[end]))
if length < max(radii[leaf] * 0.5, epsilon):
removable.append(edge)
if not removable:
break
active.difference_update(removable)
if active:
used = sorted({index for edge in active for index in edge})
remap = {old: new for new, old in enumerate(used)}
nodes = nodes[used]
radii = radii[used]
active = {tuple(sorted((remap[start], remap[end]))) for start, end in active}
else:
widest = int(np.argmax(radii))
nodes = nodes[[widest]]
radii = radii[[widest]]
return nodes, sorted(active), radii
def smooth_interior_heights(
heights,
planar_faces,
boundary_mask,
iterations=2,
factor=0.3,
):
import numpy as np
values = np.asarray(heights, dtype=np.float64).copy()
boundary_mask = np.asarray(boundary_mask, dtype=bool)
neighbors = [set() for _index in range(len(values))]
for face in planar_faces:
for start, end in zip(face, face[1:] + face[:1]):
neighbors[start].add(end)
neighbors[end].add(start)
for _iteration in range(iterations):
updated = values.copy()
for index in np.flatnonzero(~boundary_mask):
adjacent = [
neighbor for neighbor in neighbors[index] if not boundary_mask[neighbor]
]
if adjacent:
average = float(values[adjacent].mean())
updated[index] = values[index] * (1.0 - factor) + average * factor
updated[boundary_mask] = 0.0
values = updated
return values
def teddy_balloon_heights(
planar_points,
planar_faces,
contour,
boundary_mask,
inflation,
):
import numpy as np
points = np.asarray(planar_points, dtype=np.float64)
boundary_mask = np.asarray(boundary_mask, dtype=bool)
nodes, segments, radii = chordal_axis(contour)
squared_heights = np.zeros(len(points), dtype=np.float64)
for start, end in segments:
axis_start = nodes[start]
axis_end = nodes[end]
axis = axis_end - axis_start
length_squared = float(axis.dot(axis))
if length_squared <= 1e-16:
continue
factors = np.clip(
(points - axis_start) @ axis / length_squared,
0.0,
1.0,
)
centers = axis_start + factors[:, None] * axis
local_radii = radii[start] + factors * (radii[end] - radii[start])
distance_squared = np.sum((points - centers) ** 2, axis=1)
squared_heights = np.maximum(
squared_heights,
local_radii * local_radii - distance_squared,
)
degree = np.zeros(len(nodes), dtype=np.int64)
for start, end in segments:
degree[start] += 1
degree[end] += 1
for index in np.flatnonzero(degree != 2):
distance_squared = np.sum((points - nodes[index]) ** 2, axis=1)
squared_heights = np.maximum(
squared_heights,
radii[index] * radii[index] - distance_squared,
)
missing = (~boundary_mask) & (squared_heights <= 0.0)
if np.any(missing):
fallback = distances_to_contour(points[missing], contour)
squared_heights[missing] = fallback * fallback
heights = inflation * np.sqrt(np.maximum(squared_heights, 0.0))
heights[boundary_mask] = 0.0
return smooth_interior_heights(
heights,
planar_faces,
boundary_mask,
)
def cotangent_poisson_field(
planar_points,
planar_faces,
boundary_mask,
tolerance=1e-8,
):
import numpy as np
points = np.asarray(planar_points, dtype=np.float64)
faces = np.asarray(planar_faces, dtype=np.int64)
if faces.size == 0:
faces = np.empty((0, 3), dtype=np.int64)
if faces.ndim != 2 or faces.shape[1] != 3:
raise ValueError("Balloon profile faces must be triangles")
boundary_mask = np.asarray(boundary_mask, dtype=bool)
triangles = points[faces]
a, b, c = (triangles[:, index] for index in range(3))
cross = np.abs(
(b[:, 0] - a[:, 0]) * (c[:, 1] - a[:, 1])
- (b[:, 1] - a[:, 1]) * (c[:, 0] - a[:, 0])
)
usable = cross > 1e-14
faces = faces[usable]
a, b, c = a[usable], b[usable], c[usable]
cross = cross[usable]
loads = np.bincount(
faces.ravel(),
weights=np.repeat(cross / 6.0, 3),
minlength=len(points),
)
edges = np.concatenate(
(
faces[:, (1, 2)],
faces[:, (0, 2)],
faces[:, (0, 1)],
),
axis=0,
)
edge_weights = np.concatenate(
(
np.sum((b - a) * (c - a), axis=1),
np.sum((a - b) * (c - b), axis=1),
np.sum((a - c) * (b - c), axis=1),
)
) / np.tile(cross * 2.0, 3)
edges.sort(axis=1)
edges, inverse = np.unique(edges, axis=0, return_inverse=True)
edge_weights = np.bincount(
inverse,
weights=edge_weights,
minlength=len(edges),
)
interior = np.flatnonzero(~boundary_mask)
field = np.zeros(len(points), dtype=np.float64)
if not len(interior):
return field
compact = np.full(len(points), -1, dtype=np.int64)
compact[interior] = np.arange(len(interior), dtype=np.int64)
# Constrained triangulation can create negative cotangent weights.
# Clamping them keeps the Dirichlet system positive definite.
edge_weights = np.maximum(edge_weights, 0.0)
usable = edge_weights > 1e-14
compact_edges = compact[edges[usable]]
edge_weights = edge_weights[usable]
start_compact = compact_edges[:, 0]
end_compact = compact_edges[:, 1]
start_usable = start_compact >= 0
end_usable = end_compact >= 0
diagonal = np.zeros(len(interior), dtype=np.float64)
diagonal += np.bincount(
start_compact[start_usable],
weights=edge_weights[start_usable],
minlength=len(interior),
)
diagonal += np.bincount(
end_compact[end_usable],
weights=edge_weights[end_usable],
minlength=len(interior),
)
if np.any(diagonal <= 1e-14):
raise ValueError("Unable to build a stable Balloon profile")
interior_edges = start_usable & end_usable
edge_starts = start_compact[interior_edges]
edge_ends = end_compact[interior_edges]
weights = edge_weights[interior_edges]
directed_starts = np.concatenate((edge_starts, edge_ends))
directed_ends = np.concatenate((edge_ends, edge_starts))
directed_weights = np.concatenate((weights, weights))
def multiply(values):
result = diagonal * values
if len(directed_weights):
result -= np.bincount(
directed_starts,
weights=directed_weights * values[directed_ends],
minlength=len(values),
)
return result
right_hand_side = loads[interior]
solution = np.zeros(len(interior), dtype=np.float64)
residual = right_hand_side.copy()
preconditioned = residual / diagonal
direction = preconditioned.copy()
residual_product = float(residual.dot(preconditioned))
target = max(float(np.linalg.norm(right_hand_side)) * tolerance, 1e-12)
max_iterations = max(64, min(len(interior) * 4, 4000))
for _iteration in range(max_iterations):
if float(np.linalg.norm(residual)) <= target:
break
multiplied = multiply(direction)
denominator = float(direction.dot(multiplied))
if denominator <= 0.0 or not math.isfinite(denominator):
raise ValueError("Unable to solve the Balloon profile")
step = residual_product / denominator
solution += step * direction
residual -= step * multiplied
preconditioned = residual / diagonal
next_product = float(residual.dot(preconditioned))
if not math.isfinite(next_product):
raise ValueError("Unable to solve the Balloon profile")
if abs(residual_product) <= 1e-30:
break
direction = preconditioned + next_product / residual_product * direction
residual_product = next_product
field[interior] = np.maximum(solution, 0.0)
return field
def poisson_balloon_profile(
planar_points,
planar_faces,
boundary_mask,
inflation=DEFAULT_INFLATION,
):
import numpy as np
field = cotangent_poisson_field(
planar_points,
planar_faces,
boundary_mask,
)
return 2.0 * float(inflation) * np.sqrt(field)
def balloon_profile(
algorithm,
planar_points,
planar_faces,
boundary_mask,
components,
inflation=DEFAULT_INFLATION,
):
import numpy as np
algorithm = str(algorithm).upper()
if algorithm == "POISSON":
return poisson_balloon_profile(
planar_points,
planar_faces,
boundary_mask,
inflation,
)
if algorithm != "TEDDY":
raise ValueError(f"Unknown Balloon profile algorithm: {algorithm}")
result = np.zeros(len(planar_points), dtype=np.float64)
for start, end, component_faces, outer_contour in components:
result[start:end] = teddy_balloon_heights(
np.asarray(planar_points)[start:end],
component_faces,
outer_contour,
np.asarray(boundary_mask)[start:end],
inflation,
)
return result
PLANE_VARIANCE_FLOOR_RATIO = 0.025
DEPTH_REFERENCE_PERCENTILE = 95.0
ADAPTIVE_DEPTH_TRANSPARENCY_RATIO = 0.05
DEFAULT_DEPTH_BASELINE = 8.0
DEFAULT_DEPTH_DIRECTION = (0.0, 0.0, 1.0)
def use_adaptive_depth_calibration(selection_transparent_fraction):
"""Return whether the Selection input has enough transparent context."""
return float(selection_transparent_fraction) >= ADAPTIVE_DEPTH_TRANSPARENCY_RATIO
def metric_depth_reference(camera_depth, valid, uniform_scale):
"""Return the shared upper-percentile scene depth for both Depth shapes."""
import numpy as np
camera_depth = np.asarray(camera_depth, dtype=np.float32)
valid = np.asarray(valid, dtype=bool)
if valid.shape != camera_depth.shape:
raise ValueError("Depth values and validity mask dimensions do not match")
usable = valid & np.isfinite(camera_depth) & (camera_depth > 0.0)
if not usable.any():
raise ValueError("The depth field contains no valid metric depth")
if not uniform_scale > 0.0:
raise ValueError("Depth Uniform Scale must be positive")
reference = np.percentile(
camera_depth[usable],
DEPTH_REFERENCE_PERCENTILE,
)
return float(reference) * float(uniform_scale)
def _sample_valid_bilinear(values, valid, x, y):
import numpy as np
height, width = values.shape
x = np.clip(x, 0.0, max(width - 1, 0))
y = np.clip(y, 0.0, max(height - 1, 0))
x0 = np.floor(x).astype(np.int64)
y0 = np.floor(y).astype(np.int64)
x1 = np.minimum(x0 + 1, width - 1)
y1 = np.minimum(y0 + 1, height - 1)
fx = x - x0
fy = y - y0
samples = np.stack(
(
values[y0, x0],
values[y0, x1],
values[y1, x0],
values[y1, x1],
),
axis=1,
)
weights = np.stack(
(
(1.0 - fx) * (1.0 - fy),
fx * (1.0 - fy),
(1.0 - fx) * fy,
fx * fy,
),
axis=1,
)
sample_valid = np.stack(
(
valid[y0, x0],
valid[y0, x1],
valid[y1, x0],
valid[y1, x1],
),
axis=1,
)
weights *= sample_valid
totals = weights.sum(axis=1)
result = np.zeros(len(x), dtype=np.float32)
usable = totals > 1e-8
result[usable] = (samples[usable] * weights[usable]).sum(axis=1) / totals[usable]
return result, usable
def _weighted_plane(coordinates, values, weights):
import numpy as np
total = float(np.sum(weights))
if total <= 1e-12:
return np.zeros(3, dtype=np.float32)
center = np.sum(coordinates * weights[:, None], axis=0) / total
value_center = float(np.sum(values * weights) / total)
centered = coordinates - center
covariance = (centered * weights[:, None]).T @ centered / total
cross = centered.T @ (weights * (values - value_center)) / total
eigenvalues, eigenvectors = np.linalg.eigh(covariance)
largest = max(float(eigenvalues[-1]), 0.0)
if largest <= 1e-12:
slopes = np.zeros(2, dtype=np.float64)
else:
supported = np.maximum(eigenvalues, largest * PLANE_VARIANCE_FLOOR_RATIO)
slopes = eigenvectors @ ((eigenvectors.T @ cross) / supported)
intercept = value_center - float(center @ slopes)
return np.asarray((*slopes, intercept), dtype=np.float32)
def _stable_plane(coordinates, values):
import numpy as np
coordinates = np.asarray(coordinates, dtype=np.float64)
values = np.asarray(values, dtype=np.float64)
if len(coordinates) < 2:
return np.asarray((0.0, 0.0, np.median(values)), dtype=np.float32)
weights = np.ones(len(values), dtype=np.float64)
plane = np.zeros(3, dtype=np.float32)
for _iteration in range(4):
plane = _weighted_plane(coordinates, values, weights)
residual = values - coordinates @ plane[:2] - plane[2]
centered = residual - np.median(residual)
scale = max(float(np.median(np.abs(centered))) * 1.4826, 1e-6)
weights = np.minimum(
1.0,
(2.5 * scale) / np.maximum(np.abs(centered), 1e-12),
)
return plane
def depth_balloon_projection_parameters(
vertices,
vertex_uv,
camera_depth,
valid,
uniform_scale,
):
"""Fit a metric base plane and return its center depth and direction."""
import numpy as np
vertices = np.asarray(vertices, dtype=np.float32)
uv = np.asarray(vertex_uv, dtype=np.float32)
if vertices.ndim != 2 or vertices.shape[1] != 3:
raise ValueError("Depth Balloon vertices must have three components")
if uv.shape != (len(vertices), 2):
raise ValueError("Depth Balloon UV coordinates do not match its vertices")
camera_depth = np.asarray(camera_depth, dtype=np.float32)
valid = np.asarray(valid, dtype=bool)
if valid.shape != camera_depth.shape:
raise ValueError("Depth Balloon depth and mask dimensions do not match")
if not uniform_scale > 0.0:
raise ValueError("Depth Balloon Uniform Scale must be positive")
height, width = camera_depth.shape
sample_x = uv[:, 0] * max(width - 1, 0)
sample_y = (1.0 - uv[:, 1]) * max(height - 1, 0)
calibration_valid = valid & np.isfinite(camera_depth) & (camera_depth > 0.0)
sampled, usable = _sample_valid_bilinear(
camera_depth,
calibration_valid,
sample_x,
sample_y,
)
if not usable.any():
raise ValueError("MoGe-2 found no valid Depth Balloon samples")
local_coordinates = vertices[:, :2]
metric_depth = sampled[usable] * float(uniform_scale)
plane = _stable_plane(local_coordinates[usable], metric_depth)
reference = metric_depth_reference(
camera_depth,
calibration_valid,
uniform_scale,
)
direction = np.asarray((plane[0], plane[1], 1.0), dtype=np.float64)
direction /= np.linalg.norm(direction)
return reference, tuple(float(value) for value in direction)
CUTOUT_ALPHA_THRESHOLD = 0.9 CUTOUT_ALPHA_THRESHOLD = 0.9
CUTOUT_CLEANUP_UI_SCALE = 100.0 CUTOUT_CLEANUP_UI_SCALE = 100.0
@@ -1313,7 +752,7 @@ def cutout_domain(
planar = np.concatenate(refined_points) planar = np.concatenate(refined_points)
planar_faces, components = refined_faces, refined_components planar_faces, components = refined_faces, refined_components
boundary = boundary_vertices_from_faces(len(planar), planar_faces) boundary = boundary_vertices_from_faces(len(planar), planar_faces)
profile = balloon_profile( profile = balloon.balloon_profile(
balloon_algorithm, planar, planar_faces, boundary, components balloon_algorithm, planar, planar_faces, boundary, components
) )
faces = np.asarray(planar_faces, dtype=np.int32) faces = np.asarray(planar_faces, dtype=np.int32)

9
src/anyimage/operators/cutout_tool/interaction.py
View File

@@ -31,9 +31,9 @@ def _pie_button(
selection_path_json, selection_path_json,
generate_normal, generate_normal,
point_spacing, point_spacing,
inward_axis,
alpha_threshold, alpha_threshold,
fine_outline, fine_outline,
gesture,
): ):
operator = pie.operator( operator = pie.operator(
"anyimage.cutout_selection_to_shape", "anyimage.cutout_selection_to_shape",
@@ -45,9 +45,9 @@ def _pie_button(
operator.shape = shape operator.shape = shape
operator.generate_normal = generate_normal operator.generate_normal = generate_normal
operator.point_spacing = point_spacing operator.point_spacing = point_spacing
operator.inward_axis = inward_axis
operator.alpha_threshold = alpha_threshold operator.alpha_threshold = alpha_threshold
operator.fine_outline = fine_outline operator.fine_outline = fine_outline
operator.gesture = gesture
def open_shape_pie( def open_shape_pie(
@@ -55,6 +55,8 @@ def open_shape_pie(
event, event,
source_object, source_object,
selection_path, selection_path,
*,
gesture="LASSO",
): ):
source_name = source_object.name source_name = source_object.name
selection_path_json = selection_path.to_json() selection_path_json = selection_path.to_json()
@@ -64,7 +66,6 @@ def open_shape_pie(
generate_normal = ready and bool( generate_normal = ready and bool(
getattr(settings, "cutout_generate_normal", False) getattr(settings, "cutout_generate_normal", False)
) )
inward_axis = str(getattr(settings, "cutout_inward_axis", "NEGATIVE_Z"))
alpha_threshold = settings.cutout_alpha_threshold alpha_threshold = settings.cutout_alpha_threshold
fine_outline = settings.cutout_fine_outline fine_outline = settings.cutout_fine_outline
@@ -81,9 +82,9 @@ def open_shape_pie(
selection_path_json, selection_path_json,
generate_normal, generate_normal,
point_spacing, point_spacing,
inward_axis,
alpha_threshold, alpha_threshold,
fine_outline, fine_outline,
gesture,
) )
context.window_manager.popup_menu_pie( context.window_manager.popup_menu_pie(

71
src/anyimage/operators/cutout_tool/object.py
View File

@@ -15,23 +15,24 @@ from ...common.object import (
set_modifier_input, set_modifier_input,
) )
from ...common.node import load_node_group from ...common.node import load_node_group
from .geometry import ( from .depth_calibration import (
CUTOUT_ALPHA_THRESHOLD,
DEFAULT_DEPTH_BASELINE, DEFAULT_DEPTH_BASELINE,
DEFAULT_DEPTH_DIRECTION, DEFAULT_DEPTH_DIRECTION,
depth_balloon_projection_parameters,
metric_depth_reference,
use_adaptive_depth_calibration,
)
from .geometry import (
CUTOUT_ALPHA_THRESHOLD,
crop_plane_bounds, crop_plane_bounds,
cutout_cleanup_value, cutout_cleanup_value,
cutout_sampling_spacing, cutout_sampling_spacing,
depth_balloon_projection_parameters,
metric_depth_reference,
selection_calibration, selection_calibration,
use_adaptive_depth_calibration,
world_plane_metric, world_plane_metric,
) )
from .shape import ( from .shape import (
BALLOON_ATTRIBUTE_NAME, BALLOON_ATTRIBUTE_NAME,
NORMAL_SCALE_ATTRIBUTE_NAME, NORMAL_SCALE_ATTRIBUTE_NAME,
CUTOUT_X_TO_Z,
CUTOUT_NODE_GROUP_NAMES, CUTOUT_NODE_GROUP_NAMES,
DEPTH_CUTOUT_SHAPES, DEPTH_CUTOUT_SHAPES,
) )
@@ -43,38 +44,11 @@ class NormalMode(StrEnum):
OBJECT = "OBJECT" OBJECT = "OBJECT"
class CutoutInwardAxis(StrEnum): def _cutout_world_matrix(source_matrix, crop_center):
NEGATIVE_Z = "NEGATIVE_Z"
POSITIVE_X = "POSITIVE_X"
def _cutout_world_matrix(source_matrix, crop_center, inward_axis):
from mathutils import Matrix from mathutils import Matrix
center_x, center_y = (float(value) for value in crop_center) center_x, center_y = (float(value) for value in crop_center)
matrix = source_matrix @ Matrix.Translation((center_x, center_y, 0.0)) return source_matrix @ Matrix.Translation((center_x, center_y, 0.0))
if CutoutInwardAxis(inward_axis) == CutoutInwardAxis.POSITIVE_X:
matrix @= Matrix(CUTOUT_X_TO_Z)
return matrix
def orient_object_normal_image(image, inward_axis):
"""Convert native Z-axis Cutout normals to the selected object axes."""
if CutoutInwardAxis(inward_axis) == CutoutInwardAxis.NEGATIVE_Z:
return image
import numpy as np
pixels = np.empty(int(image.size[0]) * int(image.size[1]) * 4, dtype=np.float32)
image.pixels.foreach_get(pixels)
rgba = pixels.reshape((-1, 4))
normal = rgba[:, :3] * 2.0 - 1.0
rgba[:, :3] = np.column_stack(
(-normal[:, 2], -normal[:, 0], normal[:, 1])
) * 0.5 + 0.5
image.pixels.foreach_set(pixels)
image.update()
image.pack()
return image
def _assign_uv(mesh, uv): def _assign_uv(mesh, uv):
@@ -99,7 +73,7 @@ def _assign_float_point_attribute(mesh, name, values):
def load_cutout_node_group(shape): def load_cutout_node_group(shape):
return load_node_group(CUTOUT_NODE_GROUP_NAMES[shape], asset_revision=23 if shape == "DEPTH_SOLID" else 12 if shape == "DEPTH_BALLOON" else 11) return load_node_group(CUTOUT_NODE_GROUP_NAMES[shape], asset_revision=26 if shape == "DEPTH_SOLID" else 14 if shape == "DEPTH_BALLOON" else 13)
def _connect_cutout_normal_scale(material): def _connect_cutout_normal_scale(material):
@@ -134,13 +108,12 @@ def create_shape_object(
depth_metadata=None, depth_metadata=None,
normal_image=None, normal_image=None,
normal_mode=NormalMode.NONE, normal_mode=NormalMode.NONE,
inward_axis=CutoutInwardAxis.NEGATIVE_Z,
color_image, color_image,
gesture="LASSO",
): ):
import numpy as np import numpy as np
normal_mode = NormalMode(normal_mode) normal_mode = NormalMode(normal_mode)
inward_axis = CutoutInwardAxis(inward_axis)
bounds = crop_plane_bounds( bounds = crop_plane_bounds(
source_object, source_object,
content_bounds, content_bounds,
@@ -229,7 +202,6 @@ def create_shape_object(
mesh_object.matrix_world = _cutout_world_matrix( mesh_object.matrix_world = _cutout_world_matrix(
source_object.matrix_world, source_object.matrix_world,
crop_center, crop_center,
inward_axis,
) )
mesh_object["anyimage_mesh_shape"] = shape mesh_object["anyimage_mesh_shape"] = shape
node_group = load_cutout_node_group(shape) node_group = load_cutout_node_group(shape)
@@ -237,10 +209,6 @@ def create_shape_object(
modifier.node_group = node_group modifier.node_group = node_group
modifier.show_group_selector = False modifier.show_group_selector = False
for name, value in ( for name, value in (
(
"Inward Axis",
0 if inward_axis == CutoutInwardAxis.NEGATIVE_Z else 1,
),
("Cleanup Threshold", cleanup_value), ("Cleanup Threshold", cleanup_value),
( (
"Thickness", "Thickness",
@@ -249,9 +217,24 @@ def create_shape_object(
): ):
set_modifier_input( set_modifier_input(
modifier, modifier,
modifier_input_identifier(node_group, name), modifier_input_identifier(
node_group, name,
subtype="NONE" if name == "Thickness" and shape == "DEPTH_SOLID" else None,
),
value, value,
) )
if shape in {"FLAT", "SOLID", "DEPTH_SOLID"} and gesture == "POLYLINE":
set_modifier_input(
modifier,
modifier_input_identifier(node_group, "Mode"),
1,
)
if shape == "SOLID":
set_modifier_input(
modifier,
modifier_input_identifier(node_group, "Thickness", subtype="DISTANCE"),
0.5,
)
if shape in DEPTH_CUTOUT_SHAPES: if shape in DEPTH_CUTOUT_SHAPES:
for name, value in ( for name, value in (
("Depth Scale", 1.0), ("Depth Scale", 1.0),

79
src/anyimage/operators/cutout_tool/operators.py
View File

@@ -13,6 +13,7 @@ from ...preferences import (
from ...properties import ( from ...properties import (
cutout_alpha_threshold_property, cutout_alpha_threshold_property,
cutout_fine_outline_property, cutout_fine_outline_property,
cutout_gesture_property,
) )
from ...runtime import JobOperatorBase from ...runtime import JobOperatorBase
from ...server.model_catalog import DEFAULT_GEOMETRY_MODEL_KEY from ...server.model_catalog import DEFAULT_GEOMETRY_MODEL_KEY
@@ -35,7 +36,7 @@ from ...common.image import (
image_rgba, image_rgba,
load_normal_result_image, load_normal_result_image,
require_image_empty, require_image_empty,
video_total_frames, is_animated_image,
) )
from ...common.depth import load_depth_metadata, load_depth_result_image from ...common.depth import load_depth_metadata, load_depth_result_image
from ...common.color_image import ( from ...common.color_image import (
@@ -47,7 +48,7 @@ from ...common.color_image import (
) )
from ...common.viewport import ( from ...common.viewport import (
ImageGesture, ImageGesture,
image_edit_drag_keymap, image_edit_point_keymap,
report_image_edit_exception, report_image_edit_exception,
) )
from ...common.selection import ( from ...common.selection import (
@@ -62,10 +63,8 @@ from .geometry import (
) )
from .interaction import open_shape_pie from .interaction import open_shape_pie
from .object import ( from .object import (
CutoutInwardAxis,
NormalMode, NormalMode,
create_shape_object, create_shape_object,
orient_object_normal_image,
) )
from .shape import CUTOUT_SHAPES, DEPTH_CUTOUT_SHAPES from .shape import CUTOUT_SHAPES, DEPTH_CUTOUT_SHAPES
@@ -100,8 +99,8 @@ def create_cutout_shape(
depth_metadata=None, depth_metadata=None,
normal_image=None, normal_image=None,
normal_mode=NormalMode.NONE, normal_mode=NormalMode.NONE,
inward_axis=CutoutInwardAxis.NEGATIVE_Z,
color_image, color_image,
gesture="LASSO",
): ):
base_shape = build_base_shape( base_shape = build_base_shape(
context, context,
@@ -125,8 +124,8 @@ def create_cutout_shape(
depth_metadata=depth_metadata, depth_metadata=depth_metadata,
normal_image=normal_image, normal_image=normal_image,
normal_mode=normal_mode, normal_mode=normal_mode,
inward_axis=inward_axis,
color_image=color_image, color_image=color_image,
gesture=gesture,
) )
undo_message = f"Convert to {shape.replace('_', ' ').title()}" undo_message = f"Convert to {shape.replace('_', ' ').title()}"
if "FINISHED" not in bpy.ops.ed.undo_push(message=undo_message): if "FINISHED" not in bpy.ops.ed.undo_push(message=undo_message):
@@ -191,6 +190,7 @@ class CutoutSelectionToShape(JobOperatorBase, bpy.types.Operator):
selection_path_json: bpy.props.StringProperty(options=_HIDDEN) selection_path_json: bpy.props.StringProperty(options=_HIDDEN)
selection_bounds_json: bpy.props.StringProperty(options=_HIDDEN) selection_bounds_json: bpy.props.StringProperty(options=_HIDDEN)
shape: bpy.props.StringProperty(default="SOLID", options=_HIDDEN) shape: bpy.props.StringProperty(default="SOLID", options=_HIDDEN)
gesture: bpy.props.StringProperty(default="LASSO", options=_HIDDEN)
point_spacing: bpy.props.FloatProperty( point_spacing: bpy.props.FloatProperty(
default=32.0, default=32.0,
min=0.01, min=0.01,
@@ -215,23 +215,13 @@ class CutoutSelectionToShape(JobOperatorBase, bpy.types.Operator):
generate_normal: bpy.props.BoolProperty(default=False, options=_HIDDEN) generate_normal: bpy.props.BoolProperty(default=False, options=_HIDDEN)
alpha_threshold: cutout_alpha_threshold_property() alpha_threshold: cutout_alpha_threshold_property()
fine_outline: cutout_fine_outline_property() fine_outline: cutout_fine_outline_property()
inward_axis: bpy.props.StringProperty(
default=CutoutInwardAxis.NEGATIVE_Z.value,
options=_HIDDEN,
)
def _needs_generation(self): def _needs_generation(self):
return self.shape in DEPTH_CUTOUT_SHAPES or self.generate_normal return self.shape in DEPTH_CUTOUT_SHAPES or self.generate_normal
def execute(self, context): def execute(self, context):
try: try:
self.inward_axis = getattr(
self,
"inward_axis",
CutoutInwardAxis.NEGATIVE_Z.value,
)
source_object = require_image_empty(self.source_object_name) source_object = require_image_empty(self.source_object_name)
if video_total_frames(source_object.data) != 1: if is_animated_image(source_object.data):
raise ValueError("Cutout currently supports still images") raise ValueError("Cutout currently supports still images")
if self.shape not in CUTOUT_SHAPES: if self.shape not in CUTOUT_SHAPES:
raise ValueError(f"Unsupported Cutout shape: {self.shape}") raise ValueError(f"Unsupported Cutout shape: {self.shape}")
@@ -263,8 +253,9 @@ class CutoutSelectionToShape(JobOperatorBase, bpy.types.Operator):
create_cutout_shape( create_cutout_shape(
context, source_object, self.shape, self.point_spacing, context, source_object, self.shape, self.point_spacing,
content_values, selection_mask.bounds, content_values, selection_mask.bounds,
inward_axis=self.inward_axis, alpha_threshold=self.alpha_threshold, alpha_threshold=self.alpha_threshold,
fine_outline=self.fine_outline, color_image=color_image, fine_outline=self.fine_outline, color_image=color_image,
gesture=self.gesture,
) )
return {"FINISHED"} return {"FINISHED"}
setup_result = invoke_ai_setup_if_needed(self) setup_result = invoke_ai_setup_if_needed(self)
@@ -317,21 +308,14 @@ class CutoutSelectionToShape(JobOperatorBase, bpy.types.Operator):
) )
parameters.update( parameters.update(
moge2_parameters( moge2_parameters(
context.scene.anyimage_settings,
source_path, source_path,
self.model, self.model,
self.resolution_level, self.resolution_level,
1,
) )
) )
return parameters return parameters
def response(self, context, result): def response(self, context, result):
inward_axis = getattr(
self,
"inward_axis",
CutoutInwardAxis.NEGATIVE_Z.value,
)
source_object = require_image_empty(self.source_object_name) source_object = require_image_empty(self.source_object_name)
source_bounds = tuple(json.loads(self.selection_bounds_json)) source_bounds = tuple(json.loads(self.selection_bounds_json))
color_image, source_alpha, color_bounds = _generated_color_result( color_image, source_alpha, color_bounds = _generated_color_result(
@@ -358,8 +342,6 @@ class CutoutSelectionToShape(JobOperatorBase, bpy.types.Operator):
result.file(normal_key), result.file(normal_key),
source_object, source_object,
) )
if normal_mode == NormalMode.OBJECT:
orient_object_normal_image(normal_image, inward_axis)
depth_metadata = None depth_metadata = None
if shape_uses_depth(self.shape): if shape_uses_depth(self.shape):
depth_metadata = load_depth_metadata(result.file("depth_metadata")) depth_metadata = load_depth_metadata(result.file("depth_metadata"))
@@ -379,10 +361,10 @@ class CutoutSelectionToShape(JobOperatorBase, bpy.types.Operator):
depth_metadata=depth_metadata, depth_metadata=depth_metadata,
normal_image=normal_image, normal_image=normal_image,
normal_mode=normal_mode, normal_mode=normal_mode,
inward_axis=inward_axis,
alpha_threshold=self.alpha_threshold, alpha_threshold=self.alpha_threshold,
fine_outline=self.fine_outline, fine_outline=self.fine_outline,
color_image=color_image, color_image=color_image,
gesture=self.gesture,
) )
except Exception: except Exception:
if normal_image is not None and normal_image.users == 0: if normal_image is not None and normal_image.users == 0:
@@ -401,22 +383,32 @@ class CutoutSelectionToShape(JobOperatorBase, bpy.types.Operator):
class SelectCutoutSelection(ImageGesture, bpy.types.Operator): class SelectCutoutSelection(ImageGesture, bpy.types.Operator):
bl_idname = "anyimage.select_cutout_selection" bl_idname = "anyimage.select_cutout_selection"
bl_label = "Start Cutout Selection" bl_label = "Start Cutout Selection"
bl_description = "Lasso an Image Empty selection, then choose its mesh shape" bl_description = "Select an Image Empty area with a lasso or polyline, then choose its mesh shape"
bl_options = {"INTERNAL"} bl_options = {"INTERNAL"}
active_tool_id = "anyimage.cutout_lasso" active_tool_id = "anyimage.cutout_lasso"
gesture = "LASSO" resolve_on_press = True
gesture: cutout_gesture_property()
source_object_name: bpy.props.StringProperty(options={"HIDDEN", "SKIP_SAVE"}) source_object_name: bpy.props.StringProperty(options={"HIDDEN", "SKIP_SAVE"})
source_matrix_data: bpy.props.StringProperty(options={"HIDDEN", "SKIP_SAVE"}) source_matrix_data: bpy.props.StringProperty(options={"HIDDEN", "SKIP_SAVE"})
_pie_event = None _pie_event = None
_polyline_confirm_type = None
def _read_gesture_settings(self, context):
self.gesture = context.scene.anyimage_settings.cutout_gesture
def _set_status(self, context): def _set_status(self, context):
action = (
"click points; click start, double-click, or Enter for shapes; Backspace removes"
if self.gesture == "POLYLINE" else "drag and release for shapes"
)
context.workspace.status_text_set( context.workspace.status_text_set(
"Cutout Lasso: drag and release for shapes; RMB or Esc cancels" f"Cutout {self.gesture.title()}: {action}; RMB or Esc cancels"
) )
def invoke(self, context, event): def invoke(self, context, event):
self._polyline_confirm_type = None
self._pie_event = event self._pie_event = event
return super().invoke(context, event) return super().invoke(context, event)
@@ -424,6 +416,19 @@ class SelectCutoutSelection(ImageGesture, bpy.types.Operator):
self._pie_event = event self._pie_event = event
return ImageGesture.modal(self, context, event) return ImageGesture.modal(self, context, event)
def _confirm_polyline(self, context):
if len(self._path) >= 3:
self._polyline_confirm_type = self._pie_event.type
return {"RUNNING_MODAL"}
def _modal_polyline(self, context, event):
if self._polyline_confirm_type is not None:
if event.type == self._polyline_confirm_type and event.value == "RELEASE":
self._polyline_confirm_type = None
return ImageGesture._confirm_polyline(self, context)
return {"RUNNING_MODAL"}
return ImageGesture._modal_polyline(self, context, event)
def _submit_selection_path(self, context, selection_path): def _submit_selection_path(self, context, selection_path):
try: try:
source_object = require_image_empty(self.source_object_name) source_object = require_image_empty(self.source_object_name)
@@ -435,6 +440,7 @@ class SelectCutoutSelection(ImageGesture, bpy.types.Operator):
self._pie_event, self._pie_event,
source_object, source_object,
selection_path, selection_path,
gesture=self.gesture,
) )
return {"FINISHED"} return {"FINISHED"}
@@ -444,19 +450,18 @@ class CutoutTool(bpy.types.WorkSpaceTool):
bl_context_mode = "OBJECT" bl_context_mode = "OBJECT"
bl_idname = "anyimage.cutout_lasso" bl_idname = "anyimage.cutout_lasso"
bl_label = "Cutout" bl_label = "Cutout"
bl_description = "Lasso an Image Empty and choose a mesh shape" bl_description = "Select an Image Empty area with a lasso or polyline and choose a mesh shape"
bl_icon = "ops.mesh.primitive_sphere_add_gizmo" bl_icon = "ops.mesh.primitive_sphere_add_gizmo"
bl_operator = SelectCutoutSelection.bl_idname bl_operator = SelectCutoutSelection.bl_idname
bl_keymap = image_edit_drag_keymap( bl_keymap = image_edit_point_keymap(
SelectCutoutSelection.bl_idname, SelectCutoutSelection.bl_idname,
deselect_on_empty=False,
) )
def draw_settings(context, layout, _tool): def draw_settings(context, layout, _tool):
settings = context.scene.anyimage_settings settings = context.scene.anyimage_settings
status = draw_ai_setup(layout) layout.prop(settings, "cutout_gesture")
draw_ai_property(layout, settings, "cutout_generate_normal", status)
layout.prop(settings, "cutout_mesh_detail") layout.prop(settings, "cutout_mesh_detail")
layout.prop(settings, "cutout_fine_outline") layout.prop(settings, "cutout_fine_outline")
layout.prop(settings, "cutout_alpha_threshold") layout.prop(settings, "cutout_alpha_threshold")
layout.prop(settings, "cutout_inward_axis") status = draw_ai_setup(layout)
draw_ai_property(layout, settings, "cutout_generate_normal", status)

6
src/anyimage/operators/cutout_tool/shape.py
View File

@@ -2,12 +2,6 @@
BALLOON_ATTRIBUTE_NAME = "o_balloon" BALLOON_ATTRIBUTE_NAME = "o_balloon"
NORMAL_SCALE_ATTRIBUTE_NAME = "o_normal_scale" NORMAL_SCALE_ATTRIBUTE_NAME = "o_normal_scale"
CUTOUT_X_TO_Z = (
(0.0, -1.0, 0.0, 0.0),
(0.0, 0.0, 1.0, 0.0),
(-1.0, 0.0, 0.0, 0.0),
(0.0, 0.0, 0.0, 1.0),
)
CUTOUT_Z_TO_X = ( CUTOUT_Z_TO_X = (
(0.0, 0.0, -1.0, 0.0), (0.0, 0.0, -1.0, 0.0),
(-1.0, 0.0, 0.0, 0.0), (-1.0, 0.0, 0.0, 0.0),

648
src/anyimage/operators/frame_tool.py
View File

@@ -1,648 +0,0 @@
import math
import bpy
from ..common.image import (
MAX_PROJECTIVE_OUTPUT_PIXELS,
PROJECTIVE_CHUNK_ROWS,
create_image_edit_result,
homography_from_points,
image_empty_bounds,
image_pixels,
is_image_empty,
premultiplied_rgba,
replace_empty_image,
sample_rgba,
transform_homography,
video_total_frames,
)
from ..preferences import configured_max_frame_resolution
from ..common.viewport import (
active_image_edit_source,
active_view3d_tool_id,
draw_dashed_line,
draw_polygon_fill,
drawing_in_region,
image_edit_drag_keymap,
image_edit_poll,
rectangle_screen_path,
report_image_edit_exception,
)
from ..common.selection import ImageEditWarning, clip_polygon_halfplanes
FRAME_TOOL_ID = "anyimage.frame"
PROJECTIVE_RELATIVE_EPSILON = 1e-12
def _matrix_array(matrix):
import numpy as np
return np.asarray(
tuple(tuple(float(value) for value in row) for row in matrix),
dtype=np.float64,
)
def frame_source_projection(
projection_matrix,
view_matrix,
matrix_world,
bounds,
region_size,
*,
perspective,
):
"""Return normalized image-plane transforms for one frozen view."""
import numpy as np
x_min, x_max, y_min, y_max = (float(value) for value in bounds)
region_width, region_height = (float(value) for value in region_size)
if min(x_max - x_min, y_max - y_min, region_width, region_height) <= 0.0:
raise ValueError("The Image Empty projection is invalid")
source_plane = np.asarray(
(
(x_max - x_min, 0.0, x_min),
(0.0, y_min - y_max, y_max),
(0.0, 0.0, 0.0),
(0.0, 0.0, 1.0),
),
dtype=np.float64,
)
world = _matrix_array(matrix_world)
source_to_world = world @ source_plane
source_to_view = _matrix_array(view_matrix) @ source_to_world
source_to_clip = _matrix_array(projection_matrix) @ source_to_world
source_to_screen = np.stack(
(
0.5 * (source_to_clip[0] + source_to_clip[3]),
0.5 * (source_to_clip[1] + source_to_clip[3]),
source_to_clip[3],
)
)
scale = float(np.max(np.abs(source_to_screen)))
if not math.isfinite(scale) or scale <= 0.0:
raise ValueError("The Image Empty projection is invalid")
source_to_screen /= scale
if not np.isfinite(source_to_screen).all():
raise ValueError("The Image Empty projection is invalid")
condition = float(np.linalg.cond(source_to_screen))
if not math.isfinite(condition) or condition > 1.0 / np.finfo(np.float64).eps:
raise ValueError("The Image Empty is edge-on to the current view")
return {
"bounds": (x_min, x_max, y_min, y_max),
"origin_depth": -float((_matrix_array(view_matrix) @ world)[2, 3]),
"perspective": bool(perspective),
"region_size": (region_width, region_height),
"screen_to_source": np.linalg.inv(source_to_screen),
"source_to_screen": source_to_screen,
"source_to_view": source_to_view,
"source_to_world": source_to_world,
}
def _projective_points(points, matrix):
"""Map points and return finite results plus per-point validity."""
import numpy as np
points = np.asarray(points, dtype=np.float64)
flat = points.reshape((-1, 2))
homogeneous = np.column_stack((flat, np.ones(len(flat), dtype=np.float64)))
mapped = homogeneous @ np.asarray(matrix, dtype=np.float64).T
magnitude = np.max(np.abs(mapped), axis=1)
valid = (
np.isfinite(mapped).all(axis=1)
& (np.abs(mapped[:, 2]) > PROJECTIVE_RELATIVE_EPSILON * magnitude)
)
result = np.full((len(flat), 2), np.nan, dtype=np.float64)
result[valid] = mapped[valid, :2] / mapped[valid, 2, None]
return result.reshape(points.shape), valid.reshape(points.shape[:-1])
def frame_source_coordinates(source, screen_points):
"""Return source pixels, view depth, validity, and normalized coordinates."""
import numpy as np
screen_points = np.asarray(screen_points, dtype=np.float64)
region_width, region_height = source["region_size"]
normalized_screen = screen_points / (region_width, region_height)
normalized_source, valid = _projective_points(
normalized_screen,
source["screen_to_source"],
)
flat_source = normalized_source.reshape((-1, 2))
homogeneous = np.column_stack(
(flat_source, np.ones(len(flat_source), dtype=np.float64))
)
depth = -(homogeneous @ source["source_to_view"][2])
depth = depth.reshape(valid.shape)
valid &= np.isfinite(depth)
if source["perspective"]:
valid &= depth > 0.0
valid &= (
(normalized_source[..., 0] >= 0.0)
& (normalized_source[..., 0] <= 1.0)
& (normalized_source[..., 1] >= 0.0)
& (normalized_source[..., 1] <= 1.0)
)
image_width, image_height = source["image_size"]
source_points = normalized_source * (image_width, image_height)
source_points[~valid] = -1.0
return source_points, depth, valid, normalized_source
def frame_source_overlap(source, frame_quad):
"""Clip the source plane before dividing its homogeneous projection."""
import numpy as np
frame = np.asarray(frame_quad, dtype=np.float64) / source["region_size"]
left, bottom = frame.min(axis=0)
right, top = frame.max(axis=0)
x, y, w = source["source_to_screen"]
boundaries = [x - left * w, right * w - x, y - bottom * w, top * w - y]
if source["perspective"]:
boundaries.insert(0, -source["source_to_view"][2])
polygon = clip_polygon_halfplanes(
((0.0, 0.0), (1.0, 0.0), (1.0, 1.0), (0.0, 1.0)), boundaries
)
if len(polygon) < 3:
return ()
screen, valid = _projective_points(polygon, source["source_to_screen"])
if not np.all(valid):
return ()
screen *= source["region_size"]
# Center the coordinates to avoid cancellation in a distant small polygon.
centered = screen - screen[0]
area = abs(np.sum(centered[:, 0] * np.roll(centered[:, 1], -1)
- centered[:, 1] * np.roll(centered[:, 0], -1))) * 0.5
return tuple(map(tuple, screen)) if area > 1e-8 else ()
def polygon_centroid(polygon):
"""Return the area centroid of one non-degenerate screen polygon."""
cross_sum = 0.0
x_sum = 0.0
y_sum = 0.0
for first, second in zip(polygon, (*polygon[1:], polygon[0])):
cross = first[0] * second[1] - second[0] * first[1]
cross_sum += cross
x_sum += (first[0] + second[0]) * cross
y_sum += (first[1] + second[1]) * cross
if abs(cross_sum) <= 1e-12:
raise ValueError("Frame does not overlap the active Image Empty")
return x_sum / (3.0 * cross_sum), y_sum / (3.0 * cross_sum)
def _pixel_quad(image_size):
width, height = (float(value) for value in image_size)
if min(width, height) <= 0.0:
raise ValueError("The source image has no readable dimensions")
return ((0.0, 0.0), (width, 0.0), (width, height), (0.0, height))
def frame_output_size(frame_quad, region_size, max_resolution):
"""Choose output resolution from Frame pixels within the current Region."""
frame_width = abs(float(frame_quad[1][0] - frame_quad[0][0]))
frame_height = abs(float(frame_quad[0][1] - frame_quad[3][1]))
region_width, region_height = (float(value) for value in region_size)
maximum = float(max_resolution)
if min(frame_width, frame_height, region_width, region_height, maximum) <= 0.0:
raise ValueError("The Frame output dimensions must be positive")
scale = min(
1.0,
region_width / frame_width,
region_height / frame_height,
maximum / max(frame_width, frame_height),
math.sqrt(
MAX_PROJECTIVE_OUTPUT_PIXELS / max(frame_width * frame_height, 1.0)
),
)
return (
max(2, int(round(frame_width * scale))),
max(2, int(round(frame_height * scale))),
)
def _frame_source(source_object, region, view3d, view_matrix, *, active):
import numpy as np
image_size = tuple(source_object.data.size)
pixels = image_pixels(source_object.data)
width, height = image_size
bounds = image_empty_bounds(source_object)
source = frame_source_projection(
view3d.perspective_matrix,
view_matrix,
source_object.matrix_world,
bounds,
(region.width, region.height),
perspective=view3d.is_perspective,
)
source.update(
{
"active": bool(active),
"image_size": image_size,
"name": source_object.name,
"object": source_object,
"pixels": premultiplied_rgba(pixels, image_size),
"rgba": np.flipud(pixels.reshape((height, width, 4))),
}
)
return source
def _composite_frame_samples(sources, screen_points):
"""Resolve independently sorted samples with an unweighted bottom RGB base."""
import numpy as np
samples, depths, coordinates = [], [], []
for source in sources:
points, depth, valid, _ = frame_source_coordinates(source, screen_points)
sample = sample_rgba(source["pixels"], points)
sample[~valid] = 0
depths.append(np.where(valid, depth, -math.inf))
samples.append(sample)
coordinates.append(points)
samples = np.stack(samples)
depths = np.stack(depths)
order = np.argsort(-depths, axis=0, kind="stable")
bottom = order[0]
covered = np.isfinite(np.take_along_axis(depths, bottom[None], axis=0)[0])
result = np.zeros_like(samples[0])
for index, source in enumerate(sources):
selected = covered & (bottom == index)
if np.any(selected):
result[..., :3][selected] = sample_rgba(
source["rgba"], coordinates[index][selected]
)[..., :3]
for index in range(len(sources)):
layer = np.take_along_axis(samples, order[index][None, ..., None], axis=0)[0]
alpha = layer[..., 3:4]
result[..., 3:4] = alpha + result[..., 3:4] * (1 - alpha)
if index:
result[..., :3] = layer[..., :3] + result[..., :3] * (1 - alpha)
return result, covered
def _frame_boundary_pixels(sources, points, output_to_screen):
"""Conservatively detect projected contour and depth-order crossings."""
import numpy as np
boundary = np.zeros(points.shape[:-1], dtype=bool)
for source in sources:
width, height = source["region_size"]
mapping = source["screen_to_source"] @ np.diag((1 / width, 1 / height, 1)) @ output_to_screen
u, v, w = mapping
lines = [u, v, u - w, v - w, w]
positive = np.ones_like(boundary)
negative = np.ones_like(boundary)
for line in (u, v, w - u, w - v, w):
radius = 0.5 * (abs(line[0]) + abs(line[1]))
value = points[..., 0] * line[0] + points[..., 1] * line[1] + line[2]
positive &= value + radius >= 0
negative &= value - radius <= 0
source_boundary = np.zeros_like(boundary)
if source["perspective"]:
lines.append(source["source_to_view"][2] @ mapping)
for line in lines:
radius = 0.5 * (abs(line[0]) + abs(line[1]))
value = points[..., 0] * line[0] + points[..., 1] * line[1] + line[2]
source_boundary |= np.abs(value) < radius - 1e-10 * max(radius, 1e-20)
boundary |= source_boundary & (positive | negative)
if len(sources) > 1:
first_order = None
for offset in ((-0.499999, -0.499999), (0.499999, -0.499999),
(-0.499999, 0.499999), (0.499999, 0.499999)):
screen = transform_homography(points + offset, output_to_screen)
depths = []
for source in sources:
_, depth, valid, _ = frame_source_coordinates(source, screen)
depths.append(np.where(valid, depth, -math.inf))
order = np.argsort(-np.stack(depths), axis=0, kind="stable")
if first_order is None:
first_order = order
else:
boundary |= np.any(order != first_order, axis=0)
return boundary
def composite_frame_pixels(sources, frame_quad, output_size):
"""Composite projected Image Empty sources into one Blender-order RGBA image."""
import numpy as np
output_width, output_height = (int(value) for value in output_size)
if min(output_width, output_height) <= 0:
raise ValueError("The Frame output dimensions must be positive")
sources = tuple(
sorted(sources, key=lambda source: (source["active"], source["name"]))
)
if not sources or not sources[-1]["active"]:
raise ValueError("Frame requires one active Image Empty")
output_quad = _pixel_quad(output_size)
output_to_screen = homography_from_points(output_quad, frame_quad)
output = np.zeros((output_height, output_width, 4), dtype=np.float32)
covered = np.zeros((output_height, output_width), dtype=bool)
target_x = np.arange(output_width, dtype=np.float64) + 0.5
for row_start in range(0, output_height, PROJECTIVE_CHUNK_ROWS):
row_end = min(row_start + PROJECTIVE_CHUNK_ROWS, output_height)
target_y = np.arange(row_start, row_end, dtype=np.float64) + 0.5
x, y = np.meshgrid(target_x, target_y)
points = np.stack((x, y), axis=-1)
composite, coverage = _composite_frame_samples(
sources, transform_homography(points, output_to_screen)
)
boundary = _frame_boundary_pixels(sources, points, output_to_screen)
if np.any(boundary):
edge_points = points[boundary]
accumulated = np.zeros((len(edge_points), 4), dtype=np.float32)
counts = np.zeros(len(edge_points), dtype=np.int32)
for dy in (-0.375, -0.125, 0.125, 0.375):
for dx in (-0.375, -0.125, 0.125, 0.375):
sample, hit = _composite_frame_samples(
sources,
transform_homography(edge_points + (dx, dy), output_to_screen),
)
accumulated += sample
counts += hit
accumulated[..., 3] /= 16
accumulated[..., :3] /= np.maximum(counts, 1)[..., None]
composite[boundary] = accumulated
coverage[boundary] = counts > 0
covered[row_start:row_end] = coverage
output[row_start:row_end] = composite
output[..., 3][output[..., 3] <= 1e-8] = 0.0
output[~covered] = 0.0
return np.flipud(output).ravel()
def view_facing_frame_geometry(corners, output_size):
"""Return a world matrix array and display size for frame-plane corners."""
import numpy as np
corners = np.asarray(corners, dtype=np.float64)
if corners.shape != (4, 3) or not np.isfinite(corners).all():
raise ValueError("The Frame view plane is invalid")
top_left, top_right, _bottom_right, bottom_left = corners
right = top_right - top_left
up = top_left - bottom_left
width = float(np.linalg.norm(right))
height = float(np.linalg.norm(up))
if min(width, height) <= 1e-10:
raise ValueError("Draw a larger Frame")
right /= width
up /= height
forward = np.cross(right, up)
forward_length = float(np.linalg.norm(forward))
if forward_length <= 1e-10:
raise ValueError("The Frame view plane is invalid")
forward /= forward_length
matrix = np.identity(4, dtype=np.float64)
matrix[:3, 0] = right
matrix[:3, 1] = up
matrix[:3, 2] = forward
matrix[:3, 3] = corners.mean(axis=0)
display_size = width if output_size[0] >= output_size[1] else height
return matrix, display_size
def frame_result_transform(frame_quad, source_origin, region, view3d, output_size):
"""Return the view-facing object matrix and Image Empty display size."""
from mathutils import Matrix
from bpy_extras import view3d_utils
corners = tuple(
tuple(
float(value)
for value in view3d_utils.region_2d_to_location_3d(
region,
view3d,
point,
source_origin,
)
)
for point in frame_quad
)
matrix, display_size = view_facing_frame_geometry(corners, output_size)
return Matrix(tuple(tuple(row) for row in matrix)), display_size
def frame_depth_location(active_source, overlap_polygon):
"""Return the world point whose view depth places the Frame result."""
import numpy as np
source_object = active_source["object"]
if not active_source["perspective"] or active_source["origin_depth"] > 0.0:
return source_object.matrix_world.translation.copy()
centroid = np.asarray((polygon_centroid(overlap_polygon),), dtype=np.float64)
_points, depth, valid, normalized_source = frame_source_coordinates(
active_source,
centroid,
)
if not bool(valid[0]) or depth[0] <= 0.0:
raise ValueError("The Frame result depth is invalid")
source_point = np.asarray(
(normalized_source[0, 0], normalized_source[0, 1], 1.0),
dtype=np.float64,
)
world = active_source["source_to_world"] @ source_point
if not np.isfinite(world).all() or abs(world[3]) <= 1e-12:
raise ValueError("The Frame result depth is invalid")
return tuple(float(value) for value in world[:3] / world[3])
def selected_frame_sources(context):
"""Return selected still Image Empties with the active object last."""
active = active_image_edit_source(context)
sources = [
source for source in context.selected_objects if is_image_empty(source)
]
for source in sources:
if video_total_frames(source.data) > 1:
raise ValueError("Frame only supports still images")
return tuple(
sorted(sources, key=lambda source: (source is active, source.name))
)
def apply_frame(context, source_objects, frame_quad):
"""Bake selected projections into the active Image Empty."""
active = context.active_object
if active not in source_objects:
raise ValueError("The active Image Empty is not a Frame source")
view3d = context.region_data
sources = tuple(
_frame_source(
source,
context.region,
view3d,
view3d.view_matrix,
active=source is active,
)
for source in source_objects
)
active_source = next(source for source in sources if source["active"])
active_overlap = frame_source_overlap(active_source, frame_quad)
if not active_overlap:
raise ImageEditWarning("Frame does not overlap the active Image Empty")
output_size = frame_output_size(
frame_quad,
(context.region.width, context.region.height),
configured_max_frame_resolution(context),
)
pixels = composite_frame_pixels(sources, frame_quad, output_size)
depth_location = frame_depth_location(active_source, active_overlap)
matrix, display_size = frame_result_transform(
frame_quad,
depth_location,
context.region,
view3d,
output_size,
)
result_image = create_image_edit_result(active.data, pixels, output_size)
old_matrix = active.matrix_world.copy()
old_display_size = active.empty_display_size
old_offset = tuple(active.empty_image_offset)
source_images = []
for source in source_objects:
if all(image is not source.data for image in source_images):
source_images.append(source.data)
try:
active.matrix_world = matrix
active.empty_display_size = display_size
active.empty_image_offset = (-0.5, -0.5)
replace_empty_image(
active, result_image,
removed_objects=tuple(source for source in source_objects if source != active),
)
except Exception:
active.matrix_world = old_matrix
active.empty_display_size = old_display_size
active.empty_image_offset = old_offset
raise
for source in source_objects:
if source is not active:
bpy.data.objects.remove(source, do_unlink=True)
for image in source_images:
try:
if image.users == 0 and not image.use_fake_user:
bpy.data.images.remove(image, do_unlink=True)
except (ReferenceError, RuntimeError):
pass
return active
class FrameImages(bpy.types.Operator):
bl_idname = "anyimage.frame_images"
bl_label = "Frame Images"
bl_description = "Bake selected Image Empties into a view-facing frame"
bl_options = {"UNDO"}
_active_name = ""
_source_names = ()
_start = None
_cursor = None
_handle = None
_area_pointer = 0
_region_pointer = 0
@classmethod
def poll(cls, context):
return image_edit_poll(context)
def _draw_overlay(self):
if not drawing_in_region(self._area_pointer, self._region_pointer):
return
frame = rectangle_screen_path(self._start, self._cursor)
if not frame:
return
draw_polygon_fill(frame)
draw_dashed_line((*frame, frame[0]))
def _finish(self, context):
if self._handle is not None:
bpy.types.SpaceView3D.draw_handler_remove(self._handle, "WINDOW")
self._handle = None
context.workspace.status_text_set(None)
if context.area is not None:
context.area.tag_redraw()
def cancel(self, context):
self._finish(context)
def invoke(self, context, event):
try:
sources = selected_frame_sources(context)
except (TypeError, ValueError) as error:
report_image_edit_exception(self, error)
return {"CANCELLED"}
self._active_name = context.active_object.name
self._source_names = tuple(source.name for source in sources)
point = (float(event.mouse_region_x), float(event.mouse_region_y))
self._start = point
self._cursor = point
self._area_pointer = context.area.as_pointer()
self._region_pointer = context.region.as_pointer()
self._handle = bpy.types.SpaceView3D.draw_handler_add(
self._draw_overlay, (), "WINDOW", "POST_PIXEL"
)
context.workspace.status_text_set(
"Frame: drag a rectangle to merge selected Image Empties; "
"RMB or Esc cancels"
)
context.window_manager.modal_handler_add(self)
context.area.tag_redraw()
return {"RUNNING_MODAL"}
def modal(self, context, event):
active_tool = active_view3d_tool_id(context)
if active_tool is not None and active_tool != FRAME_TOOL_ID:
self._finish(context)
return {"CANCELLED"}
if event.type in {"ESC", "RIGHTMOUSE"} and event.value == "PRESS":
self._finish(context)
return {"CANCELLED"}
if event.type == "MOUSEMOVE":
self._cursor = (
float(event.mouse_region_x),
float(event.mouse_region_y),
)
context.area.tag_redraw()
return {"RUNNING_MODAL"}
if event.type == "LEFTMOUSE" and event.value == "RELEASE":
self._cursor = (
float(event.mouse_region_x),
float(event.mouse_region_y),
)
frame = rectangle_screen_path(self._start, self._cursor)
if not frame:
self._finish(context)
return {"CANCELLED"}
try:
active = bpy.data.objects.get(self._active_name)
sources = tuple(
bpy.data.objects.get(name) for name in self._source_names
)
if (
not is_image_empty(active)
or any(not is_image_empty(source) for source in sources)
or context.active_object is not active
):
raise ValueError("The selected Image Empties changed during Frame")
apply_frame(context, sources, frame)
except (RuntimeError, TypeError, ValueError) as error:
report_image_edit_exception(self, error)
self._finish(context)
return {"CANCELLED"}
self._finish(context)
return {"FINISHED"}
return {"PASS_THROUGH"}
class FrameTool(bpy.types.WorkSpaceTool):
bl_space_type = "VIEW_3D"
bl_context_mode = "OBJECT"
bl_idname = FRAME_TOOL_ID
bl_label = "Frame"
bl_description = "Bake selected Image Empties into a view-facing frame"
bl_icon = "ops.sculpt.border_mask"
bl_operator = FrameImages.bl_idname
bl_keymap = image_edit_drag_keymap(FrameImages.bl_idname)

7
src/anyimage/operators/frame_tool/__init__.py Normal file
View File

@@ -0,0 +1,7 @@
"""Frame conversion operators and workspace tool."""
from .operators import FrameImages, FrameTool
CLASSES = (FrameImages,)
__all__ = ("CLASSES", "FrameImages", "FrameTool")

141
src/anyimage/operators/frame_tool/compositing.py Normal file
View File

@@ -0,0 +1,141 @@
"""Composite projected image samples with independent RGB and Alpha coverage."""
import math
from ...common.image import (
PROJECTIVE_CHUNK_ROWS,
homography_from_points,
sample_rgba,
transform_homography,
)
from .projection import frame_source_coordinates
def _pixel_quad(image_size):
width, height = (float(value) for value in image_size)
if min(width, height) <= 0.0:
raise ValueError("The source image has no readable dimensions")
return ((0.0, 0.0), (width, 0.0), (width, height), (0.0, height))
def _composite_frame_samples(sources, screen_points):
"""Resolve independently sorted samples with an unweighted bottom RGB base."""
import numpy as np
samples, depths, coordinates = [], [], []
for source in sources:
points, depth, valid, _ = frame_source_coordinates(source, screen_points)
sample = sample_rgba(source["pixels"], points)
sample[~valid] = 0
depths.append(np.where(valid, depth, -math.inf))
samples.append(sample)
coordinates.append(points)
samples = np.stack(samples)
depths = np.stack(depths)
order = np.argsort(-depths, axis=0, kind="stable")
bottom = order[0]
covered = np.isfinite(np.take_along_axis(depths, bottom[None], axis=0)[0])
result = np.zeros_like(samples[0])
for index, source in enumerate(sources):
selected = covered & (bottom == index)
if np.any(selected):
result[..., :3][selected] = sample_rgba(
source["rgba"], coordinates[index][selected]
)[..., :3]
for index in range(len(sources)):
layer = np.take_along_axis(samples, order[index][None, ..., None], axis=0)[0]
alpha = layer[..., 3:4]
result[..., 3:4] = alpha + result[..., 3:4] * (1 - alpha)
if index:
result[..., :3] = layer[..., :3] + result[..., :3] * (1 - alpha)
return result, covered
def _frame_boundary_pixels(sources, points, output_to_screen):
"""Conservatively detect projected contour and depth-order crossings."""
import numpy as np
boundary = np.zeros(points.shape[:-1], dtype=bool)
for source in sources:
width, height = source["region_size"]
mapping = source["screen_to_source"] @ np.diag((1 / width, 1 / height, 1)) @ output_to_screen
u, v, w = mapping
lines = [u, v, u - w, v - w, w]
positive = np.ones_like(boundary)
negative = np.ones_like(boundary)
for line in (u, v, w - u, w - v, w):
radius = 0.5 * (abs(line[0]) + abs(line[1]))
value = points[..., 0] * line[0] + points[..., 1] * line[1] + line[2]
positive &= value + radius >= 0
negative &= value - radius <= 0
source_boundary = np.zeros_like(boundary)
if source["perspective"]:
lines.append(source["source_to_view"][2] @ mapping)
for line in lines:
radius = 0.5 * (abs(line[0]) + abs(line[1]))
value = points[..., 0] * line[0] + points[..., 1] * line[1] + line[2]
source_boundary |= np.abs(value) < radius - 1e-10 * max(radius, 1e-20)
boundary |= source_boundary & (positive | negative)
if len(sources) > 1:
first_order = None
for offset in ((-0.499999, -0.499999), (0.499999, -0.499999),
(-0.499999, 0.499999), (0.499999, 0.499999)):
screen = transform_homography(points + offset, output_to_screen)
depths = []
for source in sources:
_, depth, valid, _ = frame_source_coordinates(source, screen)
depths.append(np.where(valid, depth, -math.inf))
order = np.argsort(-np.stack(depths), axis=0, kind="stable")
if first_order is None:
first_order = order
else:
boundary |= np.any(order != first_order, axis=0)
return boundary
def composite_frame_pixels(sources, frame_quad, output_size):
"""Composite projected Image Empty sources into one Blender-order RGBA image."""
import numpy as np
output_width, output_height = (int(value) for value in output_size)
if min(output_width, output_height) <= 0:
raise ValueError("The Frame output dimensions must be positive")
sources = tuple(
sorted(sources, key=lambda source: (source["active"], source["name"]))
)
if not sources or not sources[-1]["active"]:
raise ValueError("Frame requires one active Image Empty")
output_quad = _pixel_quad(output_size)
output_to_screen = homography_from_points(output_quad, frame_quad)
output = np.zeros((output_height, output_width, 4), dtype=np.float32)
covered = np.zeros((output_height, output_width), dtype=bool)
target_x = np.arange(output_width, dtype=np.float64) + 0.5
for row_start in range(0, output_height, PROJECTIVE_CHUNK_ROWS):
row_end = min(row_start + PROJECTIVE_CHUNK_ROWS, output_height)
target_y = np.arange(row_start, row_end, dtype=np.float64) + 0.5
x, y = np.meshgrid(target_x, target_y)
points = np.stack((x, y), axis=-1)
composite, coverage = _composite_frame_samples(
sources, transform_homography(points, output_to_screen)
)
boundary = _frame_boundary_pixels(sources, points, output_to_screen)
if np.any(boundary):
edge_points = points[boundary]
accumulated = np.zeros((len(edge_points), 4), dtype=np.float32)
counts = np.zeros(len(edge_points), dtype=np.int32)
for dy in (-0.375, -0.125, 0.125, 0.375):
for dx in (-0.375, -0.125, 0.125, 0.375):
sample, hit = _composite_frame_samples(
sources,
transform_homography(edge_points + (dx, dy), output_to_screen),
)
accumulated += sample
counts += hit
accumulated[..., 3] /= 16
accumulated[..., :3] /= np.maximum(counts, 1)[..., None]
composite[boundary] = accumulated
coverage[boundary] = counts > 0
covered[row_start:row_end] = coverage
output[row_start:row_end] = composite
output[..., 3][output[..., 3] <= 1e-8] = 0.0
output[~covered] = 0.0
return np.flipud(output).ravel()

264
src/anyimage/operators/frame_tool/operators.py Normal file
View File

@@ -0,0 +1,264 @@
"""Frame interaction, source preparation, and Blender image replacement."""
import bpy
from ...common.image import (
create_image_edit_result,
image_empty_bounds,
image_pixels,
is_image_empty,
premultiplied_rgba,
replace_empty_image,
is_animated_image,
)
from ...preferences import configured_max_frame_resolution
from ...common.viewport import (
active_image_edit_source,
active_view3d_tool_id,
draw_dashed_line,
draw_polygon_fill,
drawing_in_region,
image_edit_drag_keymap,
image_edit_poll,
rectangle_screen_path,
report_image_edit_exception,
)
from ...common.selection import ImageEditWarning
from .projection import (
frame_depth_location,
frame_output_size,
frame_result_transform,
frame_source_overlap,
frame_source_projection,
)
from .compositing import composite_frame_pixels
FRAME_TOOL_ID = "anyimage.frame"
def _frame_source(source_object, region, view3d, view_matrix, *, active):
import numpy as np
image_size = tuple(source_object.data.size)
pixels = image_pixels(source_object.data)
width, height = image_size
bounds = image_empty_bounds(source_object)
source = frame_source_projection(
view3d.perspective_matrix,
view_matrix,
source_object.matrix_world,
bounds,
(region.width, region.height),
perspective=view3d.is_perspective,
)
source.update(
{
"active": bool(active),
"image_size": image_size,
"name": source_object.name,
"object": source_object,
"pixels": premultiplied_rgba(pixels, image_size),
"rgba": np.flipud(pixels.reshape((height, width, 4))),
}
)
return source
def selected_frame_sources(context):
"""Return selected still Image Empties with the active object last."""
active = active_image_edit_source(context)
sources = [
source for source in context.selected_objects if is_image_empty(source)
]
for source in sources:
if is_animated_image(source.data):
raise ValueError("Frame only supports still images")
return tuple(
sorted(sources, key=lambda source: (source is active, source.name))
)
def apply_frame(context, source_objects, frame_quad):
"""Bake selected projections into the active Image Empty."""
active = context.active_object
if active not in source_objects:
raise ValueError("The active Image Empty is not a Frame source")
view3d = context.region_data
sources = tuple(
_frame_source(
source,
context.region,
view3d,
view3d.view_matrix,
active=source is active,
)
for source in source_objects
)
active_source = next(source for source in sources if source["active"])
active_overlap = frame_source_overlap(active_source, frame_quad)
if not active_overlap:
raise ImageEditWarning("Frame does not overlap the active Image Empty")
output_size = frame_output_size(
frame_quad,
(context.region.width, context.region.height),
configured_max_frame_resolution(context),
)
pixels = composite_frame_pixels(sources, frame_quad, output_size)
depth_location = frame_depth_location(active_source, active_overlap)
matrix, display_size = frame_result_transform(
frame_quad,
depth_location,
context.region,
view3d,
output_size,
)
result_image = create_image_edit_result(active.data, pixels, output_size)
old_matrix = active.matrix_world.copy()
old_display_size = active.empty_display_size
old_offset = tuple(active.empty_image_offset)
source_images = []
for source in source_objects:
if all(image is not source.data for image in source_images):
source_images.append(source.data)
try:
active.matrix_world = matrix
active.empty_display_size = display_size
active.empty_image_offset = (-0.5, -0.5)
replace_empty_image(
active, result_image,
removed_objects=tuple(source for source in source_objects if source != active),
)
except Exception:
active.matrix_world = old_matrix
active.empty_display_size = old_display_size
active.empty_image_offset = old_offset
raise
for source in source_objects:
if source is not active:
bpy.data.objects.remove(source, do_unlink=True)
for image in source_images:
try:
if image.users == 0 and not image.use_fake_user:
bpy.data.images.remove(image, do_unlink=True)
except (ReferenceError, RuntimeError):
pass
return active
class FrameImages(bpy.types.Operator):
bl_idname = "anyimage.frame_images"
bl_label = "Frame Images"
bl_description = "Draw a rectangle to combine selected images into one image from the current view."
bl_options = {"UNDO"}
_active_name = ""
_source_names = ()
_start = None
_cursor = None
_handle = None
_area_pointer = 0
_region_pointer = 0
@classmethod
def poll(cls, context):
return image_edit_poll(context)
def _draw_overlay(self):
if not drawing_in_region(self._area_pointer, self._region_pointer):
return
frame = rectangle_screen_path(self._start, self._cursor)
if not frame:
return
draw_polygon_fill(frame)
draw_dashed_line((*frame, frame[0]))
def _finish(self, context):
if self._handle is not None:
bpy.types.SpaceView3D.draw_handler_remove(self._handle, "WINDOW")
self._handle = None
context.workspace.status_text_set(None)
if context.area is not None:
context.area.tag_redraw()
def cancel(self, context):
self._finish(context)
def invoke(self, context, event):
try:
sources = selected_frame_sources(context)
except (TypeError, ValueError) as error:
report_image_edit_exception(self, error)
return {"CANCELLED"}
self._active_name = context.active_object.name
self._source_names = tuple(source.name for source in sources)
point = (float(event.mouse_region_x), float(event.mouse_region_y))
self._start = point
self._cursor = point
self._area_pointer = context.area.as_pointer()
self._region_pointer = context.region.as_pointer()
self._handle = bpy.types.SpaceView3D.draw_handler_add(
self._draw_overlay, (), "WINDOW", "POST_PIXEL"
)
context.workspace.status_text_set(
"Frame: drag a rectangle to merge selected Image Empties; "
"RMB or Esc cancels"
)
context.window_manager.modal_handler_add(self)
context.area.tag_redraw()
return {"RUNNING_MODAL"}
def modal(self, context, event):
active_tool = active_view3d_tool_id(context)
if active_tool is not None and active_tool != FRAME_TOOL_ID:
self._finish(context)
return {"CANCELLED"}
if event.type in {"ESC", "RIGHTMOUSE"} and event.value == "PRESS":
self._finish(context)
return {"CANCELLED"}
if event.type == "MOUSEMOVE":
self._cursor = (
float(event.mouse_region_x),
float(event.mouse_region_y),
)
context.area.tag_redraw()
return {"RUNNING_MODAL"}
if event.type == "LEFTMOUSE" and event.value == "RELEASE":
self._cursor = (
float(event.mouse_region_x),
float(event.mouse_region_y),
)
frame = rectangle_screen_path(self._start, self._cursor)
if not frame:
self._finish(context)
return {"CANCELLED"}
try:
active = bpy.data.objects.get(self._active_name)
sources = tuple(
bpy.data.objects.get(name) for name in self._source_names
)
if (
not is_image_empty(active)
or any(not is_image_empty(source) for source in sources)
or context.active_object is not active
):
raise ValueError("The selected Image Empties changed during Frame")
apply_frame(context, sources, frame)
except (RuntimeError, TypeError, ValueError) as error:
report_image_edit_exception(self, error)
self._finish(context)
return {"CANCELLED"}
self._finish(context)
return {"FINISHED"}
return {"PASS_THROUGH"}
class FrameTool(bpy.types.WorkSpaceTool):
bl_space_type = "VIEW_3D"
bl_context_mode = "OBJECT"
bl_idname = FRAME_TOOL_ID
bl_label = "Frame"
bl_description = "Draw a rectangle to combine selected images into one image from the current view."
bl_icon = "ops.sculpt.border_mask"
bl_operator = FrameImages.bl_idname
bl_keymap = image_edit_drag_keymap(FrameImages.bl_idname)

264
src/anyimage/operators/frame_tool/projection.py Normal file
View File

@@ -0,0 +1,264 @@
"""Project image planes and place the view-facing Frame result."""
import math
from ...common.image import MAX_PROJECTIVE_OUTPUT_PIXELS
from ...common.selection import clip_polygon_halfplanes
PROJECTIVE_RELATIVE_EPSILON = 1e-12
def _matrix_array(matrix):
import numpy as np
return np.asarray(
tuple(tuple(float(value) for value in row) for row in matrix),
dtype=np.float64,
)
def frame_source_projection(
projection_matrix,
view_matrix,
matrix_world,
bounds,
region_size,
*,
perspective,
):
"""Return normalized image-plane transforms for one frozen view."""
import numpy as np
x_min, x_max, y_min, y_max = (float(value) for value in bounds)
region_width, region_height = (float(value) for value in region_size)
if min(x_max - x_min, y_max - y_min, region_width, region_height) <= 0.0:
raise ValueError("The Image Empty projection is invalid")
source_plane = np.asarray(
(
(x_max - x_min, 0.0, x_min),
(0.0, y_min - y_max, y_max),
(0.0, 0.0, 0.0),
(0.0, 0.0, 1.0),
),
dtype=np.float64,
)
world = _matrix_array(matrix_world)
source_to_world = world @ source_plane
source_to_view = _matrix_array(view_matrix) @ source_to_world
source_to_clip = _matrix_array(projection_matrix) @ source_to_world
source_to_screen = np.stack(
(
0.5 * (source_to_clip[0] + source_to_clip[3]),
0.5 * (source_to_clip[1] + source_to_clip[3]),
source_to_clip[3],
)
)
scale = float(np.max(np.abs(source_to_screen)))
if not math.isfinite(scale) or scale <= 0.0:
raise ValueError("The Image Empty projection is invalid")
source_to_screen /= scale
if not np.isfinite(source_to_screen).all():
raise ValueError("The Image Empty projection is invalid")
condition = float(np.linalg.cond(source_to_screen))
if not math.isfinite(condition) or condition > 1.0 / np.finfo(np.float64).eps:
raise ValueError("The Image Empty is edge-on to the current view")
return {
"bounds": (x_min, x_max, y_min, y_max),
"origin_depth": -float((_matrix_array(view_matrix) @ world)[2, 3]),
"perspective": bool(perspective),
"region_size": (region_width, region_height),
"screen_to_source": np.linalg.inv(source_to_screen),
"source_to_screen": source_to_screen,
"source_to_view": source_to_view,
"source_to_world": source_to_world,
}
def _projective_points(points, matrix):
"""Map points and return finite results plus per-point validity."""
import numpy as np
points = np.asarray(points, dtype=np.float64)
flat = points.reshape((-1, 2))
homogeneous = np.column_stack((flat, np.ones(len(flat), dtype=np.float64)))
mapped = homogeneous @ np.asarray(matrix, dtype=np.float64).T
magnitude = np.max(np.abs(mapped), axis=1)
valid = (
np.isfinite(mapped).all(axis=1)
& (np.abs(mapped[:, 2]) > PROJECTIVE_RELATIVE_EPSILON * magnitude)
)
result = np.full((len(flat), 2), np.nan, dtype=np.float64)
result[valid] = mapped[valid, :2] / mapped[valid, 2, None]
return result.reshape(points.shape), valid.reshape(points.shape[:-1])
def frame_source_coordinates(source, screen_points):
"""Return source pixels, view depth, validity, and normalized coordinates."""
import numpy as np
screen_points = np.asarray(screen_points, dtype=np.float64)
region_width, region_height = source["region_size"]
normalized_screen = screen_points / (region_width, region_height)
normalized_source, valid = _projective_points(
normalized_screen,
source["screen_to_source"],
)
flat_source = normalized_source.reshape((-1, 2))
homogeneous = np.column_stack(
(flat_source, np.ones(len(flat_source), dtype=np.float64))
)
depth = -(homogeneous @ source["source_to_view"][2])
depth = depth.reshape(valid.shape)
valid &= np.isfinite(depth)
if source["perspective"]:
valid &= depth > 0.0
valid &= (
(normalized_source[..., 0] >= 0.0)
& (normalized_source[..., 0] <= 1.0)
& (normalized_source[..., 1] >= 0.0)
& (normalized_source[..., 1] <= 1.0)
)
image_width, image_height = source["image_size"]
source_points = normalized_source * (image_width, image_height)
source_points[~valid] = -1.0
return source_points, depth, valid, normalized_source
def frame_source_overlap(source, frame_quad):
"""Clip the source plane before dividing its homogeneous projection."""
import numpy as np
frame = np.asarray(frame_quad, dtype=np.float64) / source["region_size"]
left, bottom = frame.min(axis=0)
right, top = frame.max(axis=0)
x, y, w = source["source_to_screen"]
boundaries = [x - left * w, right * w - x, y - bottom * w, top * w - y]
if source["perspective"]:
boundaries.insert(0, -source["source_to_view"][2])
polygon = clip_polygon_halfplanes(
((0.0, 0.0), (1.0, 0.0), (1.0, 1.0), (0.0, 1.0)), boundaries
)
if len(polygon) < 3:
return ()
screen, valid = _projective_points(polygon, source["source_to_screen"])
if not np.all(valid):
return ()
screen *= source["region_size"]
# Center the coordinates to avoid cancellation in a distant small polygon.
centered = screen - screen[0]
area = abs(np.sum(centered[:, 0] * np.roll(centered[:, 1], -1)
- centered[:, 1] * np.roll(centered[:, 0], -1))) * 0.5
return tuple(map(tuple, screen)) if area > 1e-8 else ()
def polygon_centroid(polygon):
"""Return the area centroid of one non-degenerate screen polygon."""
cross_sum = 0.0
x_sum = 0.0
y_sum = 0.0
for first, second in zip(polygon, (*polygon[1:], polygon[0])):
cross = first[0] * second[1] - second[0] * first[1]
cross_sum += cross
x_sum += (first[0] + second[0]) * cross
y_sum += (first[1] + second[1]) * cross
if abs(cross_sum) <= 1e-12:
raise ValueError("Frame does not overlap the active Image Empty")
return x_sum / (3.0 * cross_sum), y_sum / (3.0 * cross_sum)
def frame_output_size(frame_quad, region_size, max_resolution):
"""Choose output resolution from Frame pixels within the current Region."""
frame_width = abs(float(frame_quad[1][0] - frame_quad[0][0]))
frame_height = abs(float(frame_quad[0][1] - frame_quad[3][1]))
region_width, region_height = (float(value) for value in region_size)
maximum = float(max_resolution)
if min(frame_width, frame_height, region_width, region_height, maximum) <= 0.0:
raise ValueError("The Frame output dimensions must be positive")
scale = min(
1.0,
region_width / frame_width,
region_height / frame_height,
maximum / max(frame_width, frame_height),
math.sqrt(
MAX_PROJECTIVE_OUTPUT_PIXELS / max(frame_width * frame_height, 1.0)
),
)
return (
max(2, int(round(frame_width * scale))),
max(2, int(round(frame_height * scale))),
)
def view_facing_frame_geometry(corners, output_size):
"""Return a world matrix array and display size for frame-plane corners."""
import numpy as np
corners = np.asarray(corners, dtype=np.float64)
if corners.shape != (4, 3) or not np.isfinite(corners).all():
raise ValueError("The Frame view plane is invalid")
top_left, top_right, _bottom_right, bottom_left = corners
right = top_right - top_left
up = top_left - bottom_left
width = float(np.linalg.norm(right))
height = float(np.linalg.norm(up))
if min(width, height) <= 1e-10:
raise ValueError("Draw a larger Frame")
right /= width
up /= height
forward = np.cross(right, up)
forward_length = float(np.linalg.norm(forward))
if forward_length <= 1e-10:
raise ValueError("The Frame view plane is invalid")
forward /= forward_length
matrix = np.identity(4, dtype=np.float64)
matrix[:3, 0] = right
matrix[:3, 1] = up
matrix[:3, 2] = forward
matrix[:3, 3] = corners.mean(axis=0)
display_size = width if output_size[0] >= output_size[1] else height
return matrix, display_size
def frame_result_transform(frame_quad, source_origin, region, view3d, output_size):
"""Return the view-facing object matrix and Image Empty display size."""
from mathutils import Matrix
from bpy_extras import view3d_utils
corners = tuple(
tuple(
float(value)
for value in view3d_utils.region_2d_to_location_3d(
region,
view3d,
point,
source_origin,
)
)
for point in frame_quad
)
matrix, display_size = view_facing_frame_geometry(corners, output_size)
return Matrix(tuple(tuple(row) for row in matrix)), display_size
def frame_depth_location(active_source, overlap_polygon):
"""Return the world point whose view depth places the Frame result."""
import numpy as np
source_object = active_source["object"]
if not active_source["perspective"] or active_source["origin_depth"] > 0.0:
return source_object.matrix_world.translation.copy()
centroid = np.asarray((polygon_centroid(overlap_polygon),), dtype=np.float64)
_points, depth, valid, normalized_source = frame_source_coordinates(
active_source,
centroid,
)
if not bool(valid[0]) or depth[0] <= 0.0:
raise ValueError("The Frame result depth is invalid")
source_point = np.asarray(
(normalized_source[0, 0], normalized_source[0, 1], 1.0),
dtype=np.float64,
)
world = active_source["source_to_world"] @ source_point
if not np.isfinite(world).all() or abs(world[3]) <= 1e-12:
raise ValueError("The Frame result depth is invalid")
return tuple(float(value) for value in world[:3] / world[3])

6
src/anyimage/operators/mask_tool.py
View File

@@ -39,7 +39,7 @@ def compose_alpha(source_alpha, mask_values, mode):
mode = str(mode) mode = str(mode)
if mode == "SET": if mode == "SET":
result = alpha * mask result = alpha * mask
elif mode == "ADD": elif mode == "EXTEND":
result = np.maximum(alpha, mask) result = np.maximum(alpha, mask)
elif mode == "SUBTRACT": elif mode == "SUBTRACT":
result = alpha * (1.0 - mask) result = alpha * (1.0 - mask)
@@ -72,7 +72,7 @@ def rasterize_brush_path(image_size, path, radius, project_path):
class EditImageAlpha(ImageGesture, bpy.types.Operator): class EditImageAlpha(ImageGesture, bpy.types.Operator):
bl_idname = "anyimage.edit_image_alpha" bl_idname = "anyimage.edit_image_alpha"
bl_label = "Edit Image Alpha" bl_label = "Edit Image Alpha"
bl_description = "Edit Image Empty Alpha with a lasso, brush, or polyline" bl_description = "Hide or reveal image areas with a lasso, brush, or polyline."
bl_options = {"UNDO"} bl_options = {"UNDO"}
active_tool_id = MASK_TOOL_ID active_tool_id = MASK_TOOL_ID
resolve_on_press = True resolve_on_press = True
@@ -157,7 +157,7 @@ class MaskTool(bpy.types.WorkSpaceTool):
bl_context_mode = "OBJECT" bl_context_mode = "OBJECT"
bl_idname = MASK_TOOL_ID bl_idname = MASK_TOOL_ID
bl_label = "Mask" bl_label = "Mask"
bl_description = "Edit Image Empty Alpha with a lasso, brush, or polyline" bl_description = "Hide or reveal image areas with a lasso, brush, or polyline."
bl_icon = "ops.sculpt.lasso_mask" bl_icon = "ops.sculpt.lasso_mask"
bl_operator = EditImageAlpha.bl_idname bl_operator = EditImageAlpha.bl_idname
bl_cursor = "PAINT_CROSS" bl_cursor = "PAINT_CROSS"

7
src/anyimage/operators/rectify_tool/__init__.py Normal file
View File

@@ -0,0 +1,7 @@
"""Perspective correction operators and workspace tool."""
from .operators import RectifyImagePerspective, RectifyTool
CLASSES = (RectifyImagePerspective,)
__all__ = ("CLASSES", "RectifyImagePerspective", "RectifyTool")

144
src/anyimage/operators/rectify_tool/geometry.py Normal file
View File

@@ -0,0 +1,144 @@
"""Validate perspective quads and resample the selected image region."""
import math
from ...common.selection import (
clip_polygon_halfplanes,
trim_pixels_to_alpha,
)
from ...common.image import (
projective_output_size,
warp_projective_pixels,
)
MIN_INTERACTIVE_ASPECT = 0.05
MAX_INTERACTIVE_ASPECT = 20.0
def validate_perspective_quad(points, min_area=1e-6):
"""Return four convex corners without changing their correspondence."""
import numpy as np
quad = np.asarray(points, dtype=np.float64)
if quad.shape != (4, 2) or not np.isfinite(quad).all():
raise ValueError("Rectify requires four valid image corners")
edges = np.roll(quad, -1, axis=0) - quad
if np.any(np.linalg.norm(edges, axis=1) < 2.0):
raise ValueError("Rectify corners are too close together")
crosses = np.cross(edges, np.roll(edges, -1, axis=0))
if np.any(np.abs(crosses) <= 1e-8) or not (
np.all(crosses > 0.0) or np.all(crosses < 0.0)
):
raise ValueError("Rectify corners must define one convex quadrilateral")
area = 0.5 * abs(
float(
np.dot(quad[:, 0], np.roll(quad[:, 1], -1))
- np.dot(quad[:, 1], np.roll(quad[:, 0], -1))
)
)
if area < min_area:
raise ValueError("Draw a larger Rectify quad")
return quad
def canonical_perspective_quad(points, min_area=1e-6):
"""Order four unordered points as source-oriented top-left through bottom-left."""
import numpy as np
points = np.asarray(points, dtype=np.float64)
if points.shape != (4, 2) or not np.isfinite(points).all():
raise ValueError("Rectify requires four valid image corners")
distances = np.linalg.norm(points[:, None] - points[None, :], axis=2)
distances += np.eye(4) * 1e9
if float(distances.min()) < 2.0:
raise ValueError("Rectify corners are too close together")
center = points.mean(axis=0)
ring = points[np.argsort(np.arctan2(points[:, 1] - center[1], points[:, 0] - center[0]))]
ring = validate_perspective_quad(ring, min_area)
candidates = []
for winding in (ring, ring[::-1]):
for start in range(4):
candidate = np.roll(winding, -start, axis=0)
top = candidate[1] - candidate[0]
left = candidate[3] - candidate[0]
score = top[0] / np.linalg.norm(top) + left[1] / np.linalg.norm(left)
candidates.append((float(score), -candidate[0, 1], -candidate[0, 0], candidate))
return max(candidates, key=lambda item: item[:3])[3]
def perspective_quad_overlaps_image(points, image_size):
import numpy as np
width, height = image_size
if min(width, height) <= 0:
raise ValueError("The source image has no readable dimensions")
clipped = np.asarray(clip_polygon_halfplanes(
points, ((1, 0, 0), (-1, 0, width), (0, 1, 0), (0, -1, height))
))
if clipped.ndim != 2 or len(clipped) < 3:
return False
area = 0.5 * abs(
float(
np.dot(clipped[:, 0], np.roll(clipped[:, 1], -1))
- np.dot(clipped[:, 1], np.roll(clipped[:, 0], -1))
)
)
return area > 1e-6
def perspective_quad_aspect(quad):
"""Estimate an initial output aspect from the visible quad edge lengths."""
import numpy as np
quad = validate_perspective_quad(quad)
width = 0.5 * (
np.linalg.norm(quad[1] - quad[0]) + np.linalg.norm(quad[2] - quad[3])
)
height = 0.5 * (
np.linalg.norm(quad[2] - quad[1]) + np.linalg.norm(quad[3] - quad[0])
)
return min(
max(float(width / max(height, 1e-8)), MIN_INTERACTIVE_ASPECT),
MAX_INTERACTIVE_ASPECT,
)
def perspective_placement_bounds(quad, output_size, trim_bounds):
"""Map a rectified Alpha trim back to an approximate source-image rectangle."""
quad = validate_perspective_quad(quad)
source_left = math.floor(float(quad[:, 0].min()))
source_top = math.floor(float(quad[:, 1].min()))
source_right = math.ceil(float(quad[:, 0].max()))
source_bottom = math.ceil(float(quad[:, 1].max()))
output_width, output_height = (float(value) for value in output_size)
trim_left, trim_top, trim_right, trim_bottom = trim_bounds
source_width = source_right - source_left
source_height = source_bottom - source_top
return (
source_left + trim_left * source_width / output_width,
source_top + trim_top * source_height / output_height,
source_left + trim_right * source_width / output_width,
source_top + trim_bottom * source_height / output_height,
)
def extract_perspective_pixels(
source_pixels,
source_size,
quad,
aspect_ratio,
):
output_size = projective_output_size(quad, aspect_ratio)
pixels = warp_projective_pixels(
source_pixels, source_size, quad, output_size
)
pixels, trim_bounds = trim_pixels_to_alpha(pixels, (0, 0, *output_size))
trimmed_size = (
trim_bounds[2] - trim_bounds[0],
trim_bounds[3] - trim_bounds[1],
)
placement_bounds = perspective_placement_bounds(
quad, output_size, trim_bounds
)
return pixels, trimmed_size, placement_bounds

326
src/anyimage/operators/rectify_tool.py → src/anyimage/operators/rectify_tool/operators.py
View File

@@ -1,9 +1,11 @@
"""Rectify interaction and Blender image replacement."""
import json import json
import math import math
import bpy import bpy
from ..common.viewport import ( from ...common.viewport import (
active_view3d_tool_id, active_view3d_tool_id,
deserialize_matrix, deserialize_matrix,
draw_dashed_line, draw_dashed_line,
@@ -16,311 +18,45 @@ from ..common.viewport import (
screen_path_to_image_pixels, screen_path_to_image_pixels,
serialize_matrix, serialize_matrix,
) )
from ..common.selection import clip_polygon_halfplanes, trim_pixels_to_alpha from ...common.image import (
from ..common.image import (
create_image_edit_result, create_image_edit_result,
image_base_name,
image_pixels, image_pixels,
image_empty_bounds, image_empty_bounds,
projective_output_size,
replace_empty_image, replace_empty_image,
require_image_empty, require_image_empty,
video_total_frames, is_animated_image,
warp_projective_pixels, warp_projective_pixels,
) )
MIN_QUAD_AREA = 16.0 from .geometry import (
RECTIFY_TOOL_ID = "anyimage.rectify" MAX_INTERACTIVE_ASPECT,
PREVIEW_TEXTURE_SIZE = 512 MIN_INTERACTIVE_ASPECT,
ASPECT_PIXELS_PER_DOUBLE = 240.0 canonical_perspective_quad,
MIN_INTERACTIVE_ASPECT = 0.05 extract_perspective_pixels,
MAX_INTERACTIVE_ASPECT = 20.0 perspective_quad_aspect,
ASPECT_PRESETS = ( perspective_quad_overlaps_image,
("9 : 16", 9.0 / 16.0), validate_perspective_quad,
("3 : 4", 3.0 / 4.0), )
("1 : 1", 1.0), from .preview import (
("4 : 3", 4.0 / 3.0), PREVIEW_TEXTURE_SIZE,
("16 : 9", 16.0 / 9.0), aspect_ratio_from_mouse,
create_perspective_preview_image,
draw_aspect_hud,
draw_preview_frame,
interactive_preview_polygon,
preview_draw_bounds,
preview_texture_draw_options,
snapped_aspect_ratio,
) )
def validate_perspective_quad(points, min_area=1e-6): MIN_QUAD_AREA = 16.0
"""Return four convex corners without changing their correspondence.""" RECTIFY_TOOL_ID = "anyimage.rectify"
import numpy as np
quad = np.asarray(points, dtype=np.float64)
if quad.shape != (4, 2) or not np.isfinite(quad).all():
raise ValueError("Rectify requires four valid image corners")
edges = np.roll(quad, -1, axis=0) - quad
if np.any(np.linalg.norm(edges, axis=1) < 2.0):
raise ValueError("Rectify corners are too close together")
crosses = np.cross(edges, np.roll(edges, -1, axis=0))
if np.any(np.abs(crosses) <= 1e-8) or not (
np.all(crosses > 0.0) or np.all(crosses < 0.0)
):
raise ValueError("Rectify corners must define one convex quadrilateral")
area = 0.5 * abs(
float(
np.dot(quad[:, 0], np.roll(quad[:, 1], -1))
- np.dot(quad[:, 1], np.roll(quad[:, 0], -1))
)
)
if area < min_area:
raise ValueError("Draw a larger Rectify quad")
return quad
def canonical_perspective_quad(points, min_area=1e-6):
"""Order four unordered points as source-oriented top-left through bottom-left."""
import numpy as np
points = np.asarray(points, dtype=np.float64)
if points.shape != (4, 2) or not np.isfinite(points).all():
raise ValueError("Rectify requires four valid image corners")
distances = np.linalg.norm(points[:, None] - points[None, :], axis=2)
distances += np.eye(4) * 1e9
if float(distances.min()) < 2.0:
raise ValueError("Rectify corners are too close together")
center = points.mean(axis=0)
ring = points[np.argsort(np.arctan2(points[:, 1] - center[1], points[:, 0] - center[0]))]
ring = validate_perspective_quad(ring, min_area)
candidates = []
for winding in (ring, ring[::-1]):
for start in range(4):
candidate = np.roll(winding, -start, axis=0)
top = candidate[1] - candidate[0]
left = candidate[3] - candidate[0]
score = top[0] / np.linalg.norm(top) + left[1] / np.linalg.norm(left)
candidates.append((float(score), -candidate[0, 1], -candidate[0, 0], candidate))
return max(candidates, key=lambda item: item[:3])[3]
def preview_polygon(points):
"""Return a non-crossing preview hull for the current unordered clicks."""
import numpy as np
points = np.asarray(points, dtype=np.float64)
if len(points) < 3:
return points
center = points.mean(axis=0)
return points[np.argsort(np.arctan2(points[:, 1] - center[1], points[:, 0] - center[0]))]
def interactive_preview_polygon(points, cursor):
"""Return the clicked points plus one distinct moving cursor point."""
vertices = list(points)
if (
cursor is not None
and vertices
and len(vertices) < 4
and math.dist(vertices[-1], cursor) >= 2.0
):
vertices.append(cursor)
return preview_polygon(vertices)
def perspective_quad_overlaps_image(points, image_size):
import numpy as np
width, height = image_size
if min(width, height) <= 0:
raise ValueError("The source image has no readable dimensions")
clipped = np.asarray(clip_polygon_halfplanes(
points, ((1, 0, 0), (-1, 0, width), (0, 1, 0), (0, -1, height))
))
if clipped.ndim != 2 or len(clipped) < 3:
return False
area = 0.5 * abs(
float(
np.dot(clipped[:, 0], np.roll(clipped[:, 1], -1))
- np.dot(clipped[:, 1], np.roll(clipped[:, 0], -1))
)
)
return area > 1e-6
def aspect_ratio_from_mouse(initial_ratio, horizontal_delta):
ratio = initial_ratio * math.pow(
2.0, float(horizontal_delta) / ASPECT_PIXELS_PER_DOUBLE
)
return min(max(ratio, MIN_INTERACTIVE_ASPECT), MAX_INTERACTIVE_ASPECT)
def snapped_aspect_ratio(aspect_ratio):
label, ratio = min(
ASPECT_PRESETS,
key=lambda preset: abs(math.log(aspect_ratio / preset[1])),
)
return ratio, label
def aspect_ratio_label(aspect_ratio, preset_label=None):
if preset_label is not None:
return preset_label
if aspect_ratio >= 1.0:
return f"{aspect_ratio:.2f} : 1"
return f"1 : {1.0 / aspect_ratio:.2f}"
def preview_draw_bounds(region_size, aspect_ratio):
region_width, region_height = (float(value) for value in region_size)
available_width = min(region_width * 0.65, 640.0)
available_height = min(region_height * 0.65, 640.0)
if aspect_ratio >= available_width / available_height:
width = available_width
height = width / aspect_ratio
else:
height = available_height
width = height * aspect_ratio
return (
(region_width - width) * 0.5,
(region_height - height) * 0.5,
width,
height,
)
def draw_preview_frame(bounds):
import gpu
from gpu_extras.batch import batch_for_shader
left, bottom, width, height = bounds
vertices = (
(left, bottom),
(left + width, bottom),
(left + width, bottom + height),
(left, bottom + height),
(left, bottom),
)
shader = gpu.shader.from_builtin("UNIFORM_COLOR")
batch = batch_for_shader(shader, "LINE_STRIP", {"pos": vertices})
gpu.state.blend_set("ALPHA")
gpu.state.line_width_set(2.0)
shader.bind()
shader.uniform_float("color", (0.12, 0.52, 1.0, 1.0))
batch.draw(shader)
gpu.state.line_width_set(1.0)
gpu.state.blend_set("NONE")
def draw_centered_text(text, center_x, baseline_y, size, color):
import blf
font_id = 0
blf.size(font_id, size)
width, _height = blf.dimensions(font_id, text)
x = center_x - width * 0.5
blf.position(font_id, x + 1.0, baseline_y - 1.0, 0.0)
blf.color(font_id, 0.0, 0.0, 0.0, 0.9)
blf.draw(font_id, text)
blf.position(font_id, x, baseline_y, 0.0)
blf.color(font_id, *color)
blf.draw(font_id, text)
def draw_aspect_hud(bounds, aspect_ratio, preset_label=None):
left, bottom, width, height = bounds
center_x = left + width * 0.5
draw_centered_text(
aspect_ratio_label(aspect_ratio, preset_label),
center_x,
bottom + height * 0.5 - 10.0,
22,
(0.25, 0.65, 1.0, 1.0),
)
draw_centered_text(
"Move horizontally • Ctrl snap • LMB confirm • Backspace edit",
center_x,
bottom - 26.0,
13,
(0.85, 0.9, 1.0, 1.0),
)
def create_perspective_preview_image(source_image, pixels):
image = bpy.data.images.new(
f"{image_base_name(source_image)}.rectify-preview",
width=PREVIEW_TEXTURE_SIZE,
height=PREVIEW_TEXTURE_SIZE,
alpha=True,
)
try:
image.colorspace_settings.name = "sRGB"
image.alpha_mode = "STRAIGHT"
image.pixels.foreach_set(pixels)
image.update()
except Exception:
bpy.data.images.remove(image, do_unlink=True)
raise
return image
def preview_texture_draw_options(blender_version):
if tuple(blender_version) >= (5, 0, 0):
return {"is_scene_linear_with_rec709_srgb_target": True}
return {}
def perspective_quad_aspect(quad):
"""Estimate an initial output aspect from the visible quad edge lengths."""
import numpy as np
quad = validate_perspective_quad(quad)
width = 0.5 * (
np.linalg.norm(quad[1] - quad[0]) + np.linalg.norm(quad[2] - quad[3])
)
height = 0.5 * (
np.linalg.norm(quad[2] - quad[1]) + np.linalg.norm(quad[3] - quad[0])
)
return min(
max(float(width / max(height, 1e-8)), MIN_INTERACTIVE_ASPECT),
MAX_INTERACTIVE_ASPECT,
)
def perspective_placement_bounds(quad, output_size, trim_bounds):
"""Map a rectified Alpha trim back to an approximate source-image rectangle."""
quad = validate_perspective_quad(quad)
source_left = math.floor(float(quad[:, 0].min()))
source_top = math.floor(float(quad[:, 1].min()))
source_right = math.ceil(float(quad[:, 0].max()))
source_bottom = math.ceil(float(quad[:, 1].max()))
output_width, output_height = (float(value) for value in output_size)
trim_left, trim_top, trim_right, trim_bottom = trim_bounds
source_width = source_right - source_left
source_height = source_bottom - source_top
return (
source_left + trim_left * source_width / output_width,
source_top + trim_top * source_height / output_height,
source_left + trim_right * source_width / output_width,
source_top + trim_bottom * source_height / output_height,
)
def extract_perspective_pixels(
source_pixels,
source_size,
quad,
aspect_ratio,
):
output_size = projective_output_size(quad, aspect_ratio)
pixels = warp_projective_pixels(
source_pixels, source_size, quad, output_size
)
pixels, trim_bounds = trim_pixels_to_alpha(pixels, (0, 0, *output_size))
trimmed_size = (
trim_bounds[2] - trim_bounds[0],
trim_bounds[3] - trim_bounds[1],
)
placement_bounds = perspective_placement_bounds(
quad, output_size, trim_bounds
)
return pixels, trimmed_size, placement_bounds
class RectifyImagePerspective(bpy.types.Operator): class RectifyImagePerspective(bpy.types.Operator):
bl_idname = "anyimage.rectify_image_perspective" bl_idname = "anyimage.rectify_image_perspective"
bl_label = "Rectify Image Perspective" bl_label = "Rectify Image Perspective"
bl_description = "Rectify four image corners into a rectangular image" bl_description = "Select four corners of an image area to correct its perspective into a rectangle."
bl_options = {"REGISTER", "UNDO"} bl_options = {"REGISTER", "UNDO"}
source_object_name: bpy.props.StringProperty(options={"HIDDEN", "SKIP_SAVE"}) source_object_name: bpy.props.StringProperty(options={"HIDDEN", "SKIP_SAVE"})
@@ -456,7 +192,7 @@ class RectifyImagePerspective(bpy.types.Operator):
return {"FINISHED"} if source_object is not None else {"CANCELLED"} return {"FINISHED"} if source_object is not None else {"CANCELLED"}
if source_object is None: if source_object is None:
return {"CANCELLED"} return {"CANCELLED"}
if video_total_frames(source_object.data) > 1: if is_animated_image(source_object.data):
self.report( self.report(
{"WARNING"}, {"WARNING"},
"Rectify only supports a still image", "Rectify only supports a still image",
@@ -621,7 +357,7 @@ class RectifyImagePerspective(bpy.types.Operator):
self.report({"ERROR"}, str(error)) self.report({"ERROR"}, str(error))
return {"CANCELLED"} return {"CANCELLED"}
try: try:
if video_total_frames(source_object.data) > 1: if is_animated_image(source_object.data):
raise RuntimeError("Rectify only supports a still image") raise RuntimeError("Rectify only supports a still image")
quad = validate_perspective_quad(json.loads(self.quad_json)) quad = validate_perspective_quad(json.loads(self.quad_json))
if not perspective_quad_overlaps_image( if not perspective_quad_overlaps_image(
@@ -632,8 +368,6 @@ class RectifyImagePerspective(bpy.types.Operator):
"Rectify quad does not overlap the source image", "Rectify quad does not overlap the source image",
) )
return {"CANCELLED"} return {"CANCELLED"}
import numpy as np
source_pixels = self._source_pixels source_pixels = self._source_pixels
if source_pixels is None or len(source_pixels) != len( if source_pixels is None or len(source_pixels) != len(
source_object.data.pixels source_object.data.pixels
@@ -672,7 +406,7 @@ class RectifyTool(bpy.types.WorkSpaceTool):
bl_context_mode = "OBJECT" bl_context_mode = "OBJECT"
bl_idname = RECTIFY_TOOL_ID bl_idname = RECTIFY_TOOL_ID
bl_label = "Rectify" bl_label = "Rectify"
bl_description = "Rectify four image corners into a rectangular image" bl_description = "Select four corners of an image area to correct its perspective into a rectangle."
bl_icon = "ops.sculpt.line_project" bl_icon = "ops.sculpt.line_project"
bl_operator = RectifyImagePerspective.bl_idname bl_operator = RectifyImagePerspective.bl_idname
bl_keymap = image_edit_point_keymap(RectifyImagePerspective.bl_idname) bl_keymap = image_edit_point_keymap(RectifyImagePerspective.bl_idname)

168
src/anyimage/operators/rectify_tool/preview.py Normal file
View File

@@ -0,0 +1,168 @@
"""Rectify preview images, overlays, and interactive aspect controls."""
import math
import bpy
from ...common.image import image_base_name
from .geometry import (
MAX_INTERACTIVE_ASPECT,
MIN_INTERACTIVE_ASPECT,
)
PREVIEW_TEXTURE_SIZE = 512
ASPECT_PIXELS_PER_DOUBLE = 240.0
ASPECT_PRESETS = (
("9 : 16", 9.0 / 16.0),
("3 : 4", 3.0 / 4.0),
("1 : 1", 1.0),
("4 : 3", 4.0 / 3.0),
("16 : 9", 16.0 / 9.0),
)
def preview_polygon(points):
"""Return a non-crossing preview hull for the current unordered clicks."""
import numpy as np
points = np.asarray(points, dtype=np.float64)
if len(points) < 3:
return points
center = points.mean(axis=0)
return points[np.argsort(np.arctan2(points[:, 1] - center[1], points[:, 0] - center[0]))]
def interactive_preview_polygon(points, cursor):
"""Return the clicked points plus one distinct moving cursor point."""
vertices = list(points)
if (
cursor is not None
and vertices
and len(vertices) < 4
and math.dist(vertices[-1], cursor) >= 2.0
):
vertices.append(cursor)
return preview_polygon(vertices)
def aspect_ratio_from_mouse(initial_ratio, horizontal_delta):
ratio = initial_ratio * math.pow(
2.0, float(horizontal_delta) / ASPECT_PIXELS_PER_DOUBLE
)
return min(max(ratio, MIN_INTERACTIVE_ASPECT), MAX_INTERACTIVE_ASPECT)
def snapped_aspect_ratio(aspect_ratio):
label, ratio = min(
ASPECT_PRESETS,
key=lambda preset: abs(math.log(aspect_ratio / preset[1])),
)
return ratio, label
def aspect_ratio_label(aspect_ratio, preset_label=None):
if preset_label is not None:
return preset_label
if aspect_ratio >= 1.0:
return f"{aspect_ratio:.2f} : 1"
return f"1 : {1.0 / aspect_ratio:.2f}"
def preview_draw_bounds(region_size, aspect_ratio):
region_width, region_height = (float(value) for value in region_size)
available_width = min(region_width * 0.65, 640.0)
available_height = min(region_height * 0.65, 640.0)
if aspect_ratio >= available_width / available_height:
width = available_width
height = width / aspect_ratio
else:
height = available_height
width = height * aspect_ratio
return (
(region_width - width) * 0.5,
(region_height - height) * 0.5,
width,
height,
)
def draw_preview_frame(bounds):
import gpu
from gpu_extras.batch import batch_for_shader
left, bottom, width, height = bounds
vertices = (
(left, bottom),
(left + width, bottom),
(left + width, bottom + height),
(left, bottom + height),
(left, bottom),
)
shader = gpu.shader.from_builtin("UNIFORM_COLOR")
batch = batch_for_shader(shader, "LINE_STRIP", {"pos": vertices})
gpu.state.blend_set("ALPHA")
gpu.state.line_width_set(2.0)
shader.bind()
shader.uniform_float("color", (0.12, 0.52, 1.0, 1.0))
batch.draw(shader)
gpu.state.line_width_set(1.0)
gpu.state.blend_set("NONE")
def draw_centered_text(text, center_x, baseline_y, size, color):
import blf
font_id = 0
blf.size(font_id, size)
width, _height = blf.dimensions(font_id, text)
x = center_x - width * 0.5
blf.position(font_id, x + 1.0, baseline_y - 1.0, 0.0)
blf.color(font_id, 0.0, 0.0, 0.0, 0.9)
blf.draw(font_id, text)
blf.position(font_id, x, baseline_y, 0.0)
blf.color(font_id, *color)
blf.draw(font_id, text)
def draw_aspect_hud(bounds, aspect_ratio, preset_label=None):
left, bottom, width, height = bounds
center_x = left + width * 0.5
draw_centered_text(
aspect_ratio_label(aspect_ratio, preset_label),
center_x,
bottom + height * 0.5 - 10.0,
22,
(0.25, 0.65, 1.0, 1.0),
)
draw_centered_text(
"Move horizontally • Ctrl snap • LMB confirm • Backspace edit",
center_x,
bottom - 26.0,
13,
(0.85, 0.9, 1.0, 1.0),
)
def create_perspective_preview_image(source_image, pixels):
image = bpy.data.images.new(
f"{image_base_name(source_image)}.rectify-preview",
width=PREVIEW_TEXTURE_SIZE,
height=PREVIEW_TEXTURE_SIZE,
alpha=True,
)
try:
image.colorspace_settings.name = "sRGB"
image.alpha_mode = "STRAIGHT"
image.pixels.foreach_set(pixels)
image.update()
except Exception:
bpy.data.images.remove(image, do_unlink=True)
raise
return image
def preview_texture_draw_options(blender_version):
if tuple(blender_version) >= (5, 0, 0):
return {"is_scene_linear_with_rec709_srgb_target": True}
return {}

7
src/anyimage/operators/remove_background.py
View File

@@ -86,23 +86,20 @@ class RunBackgroundRemoval(JobOperatorBase, bpy.types.Operator):
require_environment() require_environment()
model = configured_background_model(context) model = configured_background_model(context)
require_model(model) require_model(model)
settings = context.scene.anyimage_settings
parameters = { parameters = {
"input": str(require_input_path(self.input_path)), "input": str(require_input_path(self.input_path)),
"model": model, "model": model,
"device": production_device().lower(), "device": production_device().lower(),
"video_frames": settings.video_frames,
"start_frame": self._image_target.start_frame,
} }
if getattr(self, "_hdr_input", None) is not None: if getattr(self, "_hdr_input", None) is not None:
parameters.update(output_kind="alpha", video_frames=1, start_frame=1) parameters["output_kind"] = "alpha"
return parameters return parameters
def response(self, _context, result): def response(self, _context, result):
if getattr(self, "_hdr_input", None) is not None: if getattr(self, "_hdr_input", None) is not None:
self._hdr_input.apply(self._image_target, result.file("alpha"), prepare_undo=True) self._hdr_input.apply(self._image_target, result.file("alpha"), prepare_undo=True)
else: else:
self._image_target.apply(result.directory) self._image_target.apply(result.file("foreground"))
if "FINISHED" not in bpy.ops.ed.undo_push(message="Remove Background"): if "FINISHED" not in bpy.ops.ed.undo_push(message="Remove Background"):
raise RuntimeError("Unable to create the Remove Background Undo step") raise RuntimeError("Unable to create the Remove Background Undo step")
return "Image background is removed" return "Image background is removed"

5
src/anyimage/operators/upscale.py
View File

@@ -115,18 +115,15 @@ class UpscaleImage(JobOperatorBase, bpy.types.Operator):
def request(self, context): def request(self, context):
require_environment() require_environment()
require_model(self.model) require_model(self.model)
settings = context.scene.anyimage_settings
return { return {
"input": str(require_input_path(self.input_path)), "input": str(require_input_path(self.input_path)),
"model": self.model, "model": self.model,
"device": production_device().lower(), "device": production_device().lower(),
"video_frames": settings.video_frames,
"start_frame": self._image_target.start_frame,
"max_input_size": int(self.max_input_size), "max_input_size": int(self.max_input_size),
} }
def response(self, _context, result): def response(self, _context, result):
self._image_target.apply(result.directory, frame_prefix="upscale.") self._image_target.apply(result.file("upscale"))
model_label = upscale_model_label(self.model) model_label = upscale_model_label(self.model)
return f"Image is upscaled with {model_label}" return f"Image is upscaled with {model_label}"

6
src/anyimage/panel.py
View File

@@ -87,15 +87,15 @@ class ServerPanel(bpy.types.Panel):
cached_row = layout.row(align=True) cached_row = layout.row(align=True)
if len(cached_names) > 1: if len(cached_names) > 1:
cached_list = cached_row.column(align=True) cached_list = cached_row.column(align=True)
cached_list.label(text="Cached:") cached_list.label(text="Loaded Models:")
for name in cached_names: for name in cached_names:
cached_list.label(text=name) cached_list.label(text=name)
else: else:
cached_row.label( cached_row.label(
text=( text=(
f"Cached: {cached_names[0]}" f"Loaded Models: {cached_names[0]}"
if cached_names if cached_names
else "Cached: None" else "No Models Loaded"
) )
) )
clear = cached_row.row(align=True) clear = cached_row.row(align=True)

42
src/anyimage/preferences.py
View File

@@ -39,9 +39,9 @@ BALLOON_PROFILE_ALGORITHM_ITEMS = (
( (
"POISSON", "POISSON",
"Poisson", "Poisson",
"Smooth cotangent-Laplacian Balloon profile", "Create a smooth, rounded bulge within the Cutout outline.",
), ),
("TEDDY", "Teddy", "Chordal-axis Balloon profile"), ("TEDDY", "Teddy", "Create a bulge shaped by the Cutout's central branches."),
) )
@@ -136,27 +136,25 @@ class AnyImagePreferences(bpy.types.AddonPreferences):
bl_idname = ADDON_PACKAGE bl_idname = ADDON_PACKAGE
adapt_material_to_view_transform: bpy.props.BoolProperty( adapt_material_to_view_transform: bpy.props.BoolProperty(
name="Adapt to Scene View Transform", name="Preserve Image Colors",
description=( description=(
"Use the scene view transform's inverse color space with Straight alpha " "Compensate for the scene's view transform to keep image colors "
"for new material color textures; otherwise use sRGB with Premultiplied alpha" "close to the original when creating materials."
), ),
default=True, default=True,
) )
storage_root: bpy.props.StringProperty( storage_root: bpy.props.StringProperty(
name="Storage Root", name="Storage Folder",
description=( description=(
"Shared user-writable storage for the environment, models and jobs; " "Choose where AnyImage stores its AI environment, downloaded models, and task files."
"all Blender versions can use the same directory"
), ),
subtype="DIR_PATH", subtype="DIR_PATH",
default=str(DEFAULT_STORAGE_ROOT), default=str(DEFAULT_STORAGE_ROOT),
) )
device: bpy.props.EnumProperty( device: bpy.props.EnumProperty(
name="Device", name="AI Processing Device",
description=( description=(
"Inference device used by production tasks; only devices supported " "Choose the CPU or GPU used to run AI models."
"on this machine are listed"
), ),
items=device_items, items=device_items,
default=0, default=0,
@@ -170,8 +168,8 @@ class AnyImagePreferences(bpy.types.AddonPreferences):
max_ai_input_size: bpy.props.IntProperty( max_ai_input_size: bpy.props.IntProperty(
name="Maximum AI Input Size", name="Maximum AI Input Size",
description=( description=(
"Maximum source long edge eligible for Upscale and maximum long edge " "Limit the longest image edge used for depth, normal, and selection refinement. "
"used by Depth, Normal and Selection refinement" "Upscale requires a source image smaller than this limit."
), ),
default=DEFAULT_MAX_AI_INPUT_SIZE, default=DEFAULT_MAX_AI_INPUT_SIZE,
min=MIN_MAX_AI_INPUT_SIZE, min=MIN_MAX_AI_INPUT_SIZE,
@@ -181,7 +179,10 @@ class AnyImagePreferences(bpy.types.AddonPreferences):
) )
max_cutout_mesh_resolution: bpy.props.IntProperty( max_cutout_mesh_resolution: bpy.props.IntProperty(
name="Maximum Cutout Mesh Resolution", name="Maximum Cutout Mesh Resolution",
description="Maximum sampling resolution along the longest Cutout edge", description=(
"Limit the longest image edge used to sample the Cutout mesh. "
"Higher values allow finer outlines and denser meshes."
),
default=DEFAULT_MAX_CUTOUT_MESH_RESOLUTION, default=DEFAULT_MAX_CUTOUT_MESH_RESOLUTION,
min=MIN_CUTOUT_MESH_RESOLUTION, min=MIN_CUTOUT_MESH_RESOLUTION,
max=MAX_CUTOUT_MESH_RESOLUTION, max=MAX_CUTOUT_MESH_RESOLUTION,
@@ -190,7 +191,7 @@ class AnyImagePreferences(bpy.types.AddonPreferences):
) )
max_frame_resolution: bpy.props.IntProperty( max_frame_resolution: bpy.props.IntProperty(
name="Maximum Frame Resolution", name="Maximum Frame Resolution",
description="Maximum output resolution along the longest Frame edge", description="Set the maximum length, in pixels, of the combined image's longest edge.",
default=DEFAULT_MAX_FRAME_RESOLUTION, default=DEFAULT_MAX_FRAME_RESOLUTION,
min=MIN_FRAME_RESOLUTION, min=MIN_FRAME_RESOLUTION,
max=MAX_FRAME_RESOLUTION, max=MAX_FRAME_RESOLUTION,
@@ -199,7 +200,7 @@ class AnyImagePreferences(bpy.types.AddonPreferences):
) )
balloon_profile_algorithm: bpy.props.EnumProperty( balloon_profile_algorithm: bpy.props.EnumProperty(
name="Balloon Profile", name="Balloon Profile",
description="Algorithm used to generate o_balloon for every Cutout shape", description="Control how the Cutout surface bulges outward from its outline.",
items=BALLOON_PROFILE_ALGORITHM_ITEMS, items=BALLOON_PROFILE_ALGORITHM_ITEMS,
default="POISSON", default="POISSON",
) )
@@ -210,14 +211,17 @@ class AnyImagePreferences(bpy.types.AddonPreferences):
default=DEFAULT_BACKGROUND_MODEL_KEY, default=DEFAULT_BACKGROUND_MODEL_KEY,
) )
geometry_model: bpy.props.EnumProperty( geometry_model: bpy.props.EnumProperty(
name="Generate Depth Map Model", name="Depth & Normal Model",
description="Model used to generate depth maps", description="Choose the AI model used to generate depth and surface normals.",
items=geometry_model_items, items=geometry_model_items,
default=0, default=0,
) )
geometry_resolution_level: bpy.props.IntProperty( geometry_resolution_level: bpy.props.IntProperty(
name="MoGe Resolution Level", name="MoGe Resolution Level",
description="MoGe-2 inference resolution used for geometry and normal generation", description=(
"Set the processing resolution for depth and normal generation. "
"Higher levels can capture finer details but take more time and memory."
),
default=5, default=5,
min=0, min=0,
max=9, max=9,

66
src/anyimage/properties.py
View File

@@ -140,6 +140,18 @@ def upscale_model_label(model_key):
return model.label if model is not None else model_key return model.label if model is not None else model_key
def cutout_gesture_property():
return bpy.props.EnumProperty(
name="Gesture",
description="Shape used to select the Cutout area",
items=(
("LASSO", "Lasso", "Draw a closed freehand area"),
("POLYLINE", "Polyline", "Click points to define a closed area"),
),
default="LASSO",
)
def mask_gesture_property(): def mask_gesture_property():
return bpy.props.EnumProperty( return bpy.props.EnumProperty(
name="Gesture", name="Gesture",
@@ -156,19 +168,31 @@ def mask_gesture_property():
def mask_mode_property(): def mask_mode_property():
return bpy.props.EnumProperty( return bpy.props.EnumProperty(
name="Mode", name="Mode",
description="How the gesture changes image Alpha", description="Choose how the selected or painted area changes image visibility.",
items=( items=(
("SET", "Set", "Keep current Alpha inside the gesture", "SELECT_SET", 0), (
("ADD", "Add", "Extend Alpha with the gesture", "SELECT_EXTEND", 1), "SET",
"Set",
"Keep visible content inside the selected or painted area and hide everything outside.",
"SELECT_SET",
0,
),
(
"EXTEND",
"Extend",
"Reveal the selected or painted area while keeping other areas unchanged.",
"SELECT_EXTEND",
1,
),
( (
"SUBTRACT", "SUBTRACT",
"Subtract", "Subtract",
"Remove Alpha inside the gesture", "Hide the selected or painted area while keeping other areas unchanged.",
"SELECT_SUBTRACT", "SELECT_SUBTRACT",
2, 2,
), ),
), ),
default="SUBTRACT", default="SET",
) )
@@ -183,24 +207,12 @@ def mask_radius_property():
) )
def cutout_inward_axis_property():
return bpy.props.EnumProperty(
name="Inward Axis",
description="Local axis pointing inward from the image sheet",
items=(
("NEGATIVE_Z", "-Z", "Match the source Image Empty local axes"),
("POSITIVE_X", "+X", "Use the original Cutout local axes"),
),
default="NEGATIVE_Z",
)
def cutout_generate_normal_property(): def cutout_generate_normal_property():
return bpy.props.BoolProperty( return bpy.props.BoolProperty(
name="Normal Map", name="Normal Map",
description=( description=(
"Generate a normal map for Flat and Solid; " "Generate a normal map to add surface detail under lighting for Flat and Solid shapes. "
"Depth Solid and Depth Balloon always generate normal maps" "Depth Solid and Depth Balloon always generate normal maps."
), ),
default=False, default=False,
) )
@@ -209,7 +221,10 @@ def cutout_generate_normal_property():
def cutout_alpha_threshold_property(): def cutout_alpha_threshold_property():
return bpy.props.FloatProperty( return bpy.props.FloatProperty(
name="Alpha Threshold", name="Alpha Threshold",
description="Minimum Alpha kept in the Cutout outline", description=(
"Exclude areas below this opacity from the Cutout outline. "
"Higher values exclude more semi-transparent edges."
),
default=0.9, default=0.9,
min=0.0, min=0.0,
max=1.0, max=1.0,
@@ -230,7 +245,7 @@ def cutout_fine_outline_property():
def cutout_mesh_detail_property(): def cutout_mesh_detail_property():
return bpy.props.EnumProperty( return bpy.props.EnumProperty(
name="Mesh Detail", name="Mesh Detail",
description="Mesh sampling density", description="Control mesh density. Higher levels use more vertices to represent finer shapes.",
items=( items=(
("LOW", "Low", "32 source image pixels between neighboring mesh points"), ("LOW", "Low", "32 source image pixels between neighboring mesh points"),
("MEDIUM", "Medium", "16 source image pixels between neighboring mesh points"), ("MEDIUM", "Medium", "16 source image pixels between neighboring mesh points"),
@@ -242,18 +257,11 @@ def cutout_mesh_detail_property():
class AnyImageSettings(bpy.types.PropertyGroup): class AnyImageSettings(bpy.types.PropertyGroup):
cutout_gesture: cutout_gesture_property()
cutout_alpha_threshold: cutout_alpha_threshold_property() cutout_alpha_threshold: cutout_alpha_threshold_property()
cutout_mesh_detail: cutout_mesh_detail_property() cutout_mesh_detail: cutout_mesh_detail_property()
cutout_fine_outline: cutout_fine_outline_property() cutout_fine_outline: cutout_fine_outline_property()
cutout_generate_normal: cutout_generate_normal_property() cutout_generate_normal: cutout_generate_normal_property()
cutout_inward_axis: cutout_inward_axis_property()
mask_gesture: mask_gesture_property() mask_gesture: mask_gesture_property()
mask_mode: mask_mode_property() mask_mode: mask_mode_property()
mask_radius: mask_radius_property() mask_radius: mask_radius_property()
video_frames: bpy.props.IntProperty(
name="Extract Frames",
description="Maximum number of consecutive frames extracted from the start of a video or frame sequence",
default=16,
min=1,
soft_max=64,
)

2
src/anyimage/runtime.py
View File

@@ -12,7 +12,7 @@ ENVIRONMENT = {
"numpy==2.4.2", "numpy==2.4.2",
"scipy==1.15.3", "scipy==1.15.3",
"pillow==12.1.1", "pillow==12.1.1",
"av==17.0.0",
"trimesh==4.11.2", "trimesh==4.11.2",
"huggingface-hub==1.4.1", "huggingface-hub==1.4.1",
], ],

7
src/anyimage/server/geometry/prediction_artifacts.py
View File

@@ -64,16 +64,13 @@ def generate_moge_artifacts(
parameters.get("max_input_size", DEFAULT_MAX_AI_INPUT_SIZE), parameters.get("max_input_size", DEFAULT_MAX_AI_INPUT_SIZE),
) )
try: try:
frames = moge.infer( frame = moge.infer(
context.resource("model_manager"), context.resource("model_manager"),
parameters, parameters,
[model_path], model_path,
include_points=generate_depth, include_points=generate_depth,
cancel_check=context.check_cancelled, cancel_check=context.check_cancelled,
) )
if len(frames) != 1:
raise ValueError("Artifact generation requires one geometry frame")
frame = frames[0]
result = {} result = {}
if generate_depth: if generate_depth:
context.progress(0.45, "Generating depth texture") context.progress(0.45, "Generating depth texture")

4
src/anyimage/server/jobs/cutout.py
View File

@@ -1,6 +1,6 @@
"""Generate optional AI files from one rectangular Cutout image input.""" """Generate optional AI files from one rectangular Cutout image input."""
from pathlib import Path from ..media.input import validate_input_path
from ..geometry.prediction_artifacts import generate_moge_artifacts from ..geometry.prediction_artifacts import generate_moge_artifacts
@@ -8,7 +8,7 @@ from ..geometry.prediction_artifacts import generate_moge_artifacts
def run(context, parameters): def run(context, parameters):
context.progress(0.05, "Preparing Cutout Selection") context.progress(0.05, "Preparing Cutout Selection")
context.directory.mkdir(parents=True, exist_ok=True) context.directory.mkdir(parents=True, exist_ok=True)
input_path = Path(parameters["input"]) input_path = validate_input_path(parameters["input"])
result = {} result = {}
generate_depth = bool(parameters.get("generate_depth", False)) generate_depth = bool(parameters.get("generate_depth", False))
normal_mode = str(parameters.get("normal_mode", "NONE")) normal_mode = str(parameters.get("normal_mode", "NONE"))

20
src/anyimage/server/jobs/depth_plane.py
View File

@@ -1,7 +1,7 @@
"""Generate model-space depth and normals for one complete image.""" """Generate model-space depth and normals for one complete image."""
from ..geometry.prediction_artifacts import generate_moge_artifacts from ..geometry.prediction_artifacts import generate_moge_artifacts
from ..media import input_frames from ..media.input import validate_input_path
def run(context, parameters): def run(context, parameters):
@@ -10,16 +10,14 @@ def run(context, parameters):
raise ValueError(f"Unsupported plane type: {plane_type}") raise ValueError(f"Unsupported plane type: {plane_type}")
context.progress(0.05, "Preparing depth conversion") context.progress(0.05, "Preparing depth conversion")
context.directory.mkdir(parents=True, exist_ok=True) context.directory.mkdir(parents=True, exist_ok=True)
with input_frames(context, parameters) as images: input_path = validate_input_path(parameters["input"])
if len(images) != 1: artifacts = generate_moge_artifacts(
raise ValueError("Depth conversion currently supports one image") context,
artifacts = generate_moge_artifacts( parameters,
context, input_path,
parameters, generate_depth=True,
images[0], normal_mode="OBJECT" if plane_type == "DEPTH" else "TANGENT",
generate_depth=True, )
normal_mode="OBJECT" if plane_type == "DEPTH" else "TANGENT",
)
context.check_cancelled() context.check_cancelled()
context.progress(1.0, "Depth conversion data is ready") context.progress(1.0, "Depth conversion data is ready")
return {key: path.name for key, path in artifacts.items()} return {key: path.name for key, path in artifacts.items()}

103
src/anyimage/server/jobs/panorama.py
View File

@@ -1,12 +1,10 @@
"""Generate a radial depth texture from the current RGBA panorama.""" """Generate a radial depth texture from the current RGBA panorama."""
import json import json
from pathlib import Path
from ..geometry.depth_texture import write_float_exr from ..geometry.depth_texture import write_float_exr
from ..geometry.panorama import camera_bases, merge_distances, sample_panorama, view_rays from ..geometry.panorama import camera_bases, merge_distances, sample_panorama, view_rays
from ..media import VIDEO_SUFFIXES, input_frames from ..media.input import open_image, validate_input_path
from ..media.input import open_image
from ..media.resolution import DEFAULT_MAX_AI_INPUT_SIZE, normalized_max_input_size from ..media.resolution import DEFAULT_MAX_AI_INPUT_SIZE, normalized_max_input_size
from ..models.moge import infer_image as infer from ..models.moge import infer_image as infer
@@ -15,63 +13,58 @@ def run(context, parameters):
import numpy as np import numpy as np
context.check_cancelled() context.check_cancelled()
input_path = Path(parameters["input"]) input_path = validate_input_path(parameters["input"])
if input_path.is_dir() or input_path.suffix.lower() in VIDEO_SUFFIXES:
raise ValueError("Panorama conversion requires one static 2:1 image")
maximum = normalized_max_input_size(parameters.get("max_input_size", DEFAULT_MAX_AI_INPUT_SIZE)) maximum = normalized_max_input_size(parameters.get("max_input_size", DEFAULT_MAX_AI_INPUT_SIZE))
context.directory.mkdir(parents=True, exist_ok=True) context.directory.mkdir(parents=True, exist_ok=True)
outputs = [context.directory / name for name in ("depth.exr", "depth.json")] outputs = [context.directory / name for name in ("depth.exr", "depth.json")]
try: try:
with input_frames(context, parameters) as images: with open_image(input_path) as opened:
if len(images) != 1: if opened.width != opened.height * 2:
raise ValueError("Panorama conversion requires one static 2:1 image") raise ValueError("Panorama conversion requires one static 2:1 image")
with open_image(images[0]) as opened: source = opened.convert("RGBA")
if getattr(opened, "n_frames", 1) != 1 or opened.width != opened.height * 2: width, height = source.size
raise ValueError("Panorama conversion requires one static 2:1 image") size = min(maximum, max(512, width // 4))
source = opened.convert("RGBA") image = np.asarray(source, dtype=np.float32) / 255.0
width, height = source.size manager = context.resource("model_manager")
size = min(maximum, max(512, width // 4)) bases = camera_bases()
image = np.asarray(source, dtype=np.float32) / 255.0 rays = view_rays(size)
manager = context.resource("model_manager") distances, masks = [], []
bases = camera_bases() session = None
rays = view_rays(size) for index, basis in enumerate(bases):
distances, masks = [], [] context.check_cancelled()
session = None context.progress(0.05 + 0.65 * index / len(bases), f"Generating panorama view {index + 1}/12")
for index, basis in enumerate(bases): directions = rays @ basis
context.check_cancelled() alpha = sample_panorama(image[..., 3], directions, order=0)
context.progress(0.05 + 0.65 * index / len(bases), f"Generating panorama view {index + 1}/12") visible = alpha > 0
directions = rays @ basis if not visible.any():
alpha = sample_panorama(image[..., 3], directions, order=0) distances.append(np.ones((size, size), dtype=np.float32))
visible = alpha > 0 masks.append(visible)
if not visible.any(): continue
distances.append(np.ones((size, size), dtype=np.float32)) if session is None:
masks.append(visible) session = manager.get_moge(manager.directory(parameters["model"]), parameters["device"])
continue color = sample_panorama(image[..., :3], directions) * alpha[..., None]
if session is None: prediction = infer(session, color, parameters["resolution_level"],
session = manager.get_moge(manager.directory(parameters["model"]), parameters["device"]) fov_x=90.0, source_valid=visible)
color = sample_panorama(image[..., :3], directions) * alpha[..., None] distances.append(np.linalg.norm(prediction["points"], axis=-1))
prediction = infer(session, color, parameters["resolution_level"], masks.append((prediction["mask"] > 0.5) & visible)
fov_x=90.0, source_valid=visible) del prediction
distances.append(np.linalg.norm(prediction["points"], axis=-1)) context.progress(0.75, "Merging panoramic distance")
masks.append((prediction["mask"] > 0.5) & visible) distance, valid = merge_distances(distances, masks, bases, width, height,
del prediction cancel_check=context.check_cancelled)
context.progress(0.75, "Merging panoramic distance") alpha = np.asarray(source.getchannel("A"),
distance, valid = merge_distances(distances, masks, bases, width, height, dtype=np.float32) / 255.0
cancel_check=context.check_cancelled) alpha *= valid
alpha = np.asarray(source.getchannel("A"), valid_depth = (alpha >= 0.9) & np.isfinite(distance) & (distance > 0)
dtype=np.float32) / 255.0 if not valid_depth.any():
alpha *= valid raise ValueError("The panorama contains no valid depth pixels")
valid_depth = (alpha >= 0.9) & np.isfinite(distance) & (distance > 0) if not np.isfinite(distance).all() or (distance <= 0).any():
if not valid_depth.any(): raise ValueError("Panorama fusion produced invalid distances")
raise ValueError("The panorama contains no valid depth pixels") rgba = np.empty((height, width, 4), dtype=np.float32)
if not np.isfinite(distance).all() or (distance <= 0).any(): rgba[..., :3], rgba[..., 3] = distance[..., None], alpha
raise ValueError("Panorama fusion produced invalid distances") write_float_exr(rgba, outputs[0])
rgba = np.empty((height, width, 4), dtype=np.float32) outputs[1].write_text(json.dumps({
rgba[..., :3], rgba[..., 3] = distance[..., None], alpha "projection": "equirectangular", "image_size": [width, height],
write_float_exr(rgba, outputs[0]) }, separators=(",", ":")), encoding="utf-8")
outputs[1].write_text(json.dumps({
"projection": "equirectangular", "image_size": [width, height],
}, separators=(",", ":")), encoding="utf-8")
context.check_cancelled() context.check_cancelled()
context.progress(1.0, "Panorama conversion data is ready") context.progress(1.0, "Panorama conversion data is ready")
return {"depth": outputs[0].name, "depth_metadata": outputs[1].name} return {"depth": outputs[0].name, "depth_metadata": outputs[1].name}

84
src/anyimage/server/jobs/remove_background.py
View File

@@ -1,9 +1,8 @@
import time import time
from ..media import input_frames from ..media.input import open_image, validate_input_path
from ..media.input import open_image
from ..model_catalog import BACKGROUND_MODELS from ..model_catalog import BACKGROUND_MODELS
from ..models.background import infer_alpha, infer_foreground, infer_foreground_frames, inference_device from ..models.background import infer_alpha, infer_foreground, inference_device
def run(context, parameters): def run(context, parameters):
@@ -24,62 +23,45 @@ def run(context, parameters):
if output_kind not in {"foreground", "alpha"}: if output_kind not in {"foreground", "alpha"}:
raise ValueError(f"Unknown background removal output: {output_kind}") raise ValueError(f"Unknown background removal output: {output_kind}")
with input_frames(context, parameters) as images: input_path = validate_input_path(parameters["input"])
context.progress(0.12, f"Loading {parameters['model']} for {device.upper()}") context.progress(0.12, f"Loading {parameters['model']} for {device.upper()}")
if output_kind == "alpha": if output_kind == "alpha":
import numpy as np import numpy as np
if len(images) != 1: with open_image(input_path) as source:
raise ValueError("Alpha output requires one still image") alpha, load_ms, inference_ms = infer_alpha(
with open_image(images[0]) as source: source, model_directory, device, model_cache=model_manager,
alpha, load_ms, inference_ms = infer_alpha(
source, model_directory, device, model_cache=model_manager,
cancel_check=context.check_cancelled,
)
context.check_cancelled()
np.save(output_directory / "alpha.npy", alpha, allow_pickle=False)
elif len(images) == 1:
context.progress(0.35, "Removing background")
foreground, load_ms, inference_ms = infer_foreground(
images[0],
model_directory,
device,
model_cache=model_manager,
cancel_check=context.check_cancelled, cancel_check=context.check_cancelled,
preserve_input_alpha=True,
)
context.check_cancelled()
context.progress(0.92, "Writing transparent PNG")
output_path = output_directory / "image.0001.png"
foreground.convert("RGBA").save(
output_path,
format="PNG",
compress_level=1,
)
if not output_path.is_file():
raise RuntimeError("Background removal did not write a foreground image")
else:
load_ms, inference_ms = infer_foreground_frames(
images,
model_directory,
device,
output_directory,
context.progress,
0.12,
0.7,
model_cache=model_manager,
cancel_check=context.check_cancelled,
output_prefix="image",
preserve_input_alpha=True,
) )
context.check_cancelled()
np.save(output_directory / "alpha.npy", alpha, allow_pickle=False)
else:
context.progress(0.35, "Removing background")
foreground, load_ms, inference_ms = infer_foreground(
input_path,
model_directory,
device,
model_cache=model_manager,
cancel_check=context.check_cancelled,
preserve_input_alpha=True,
)
context.check_cancelled()
context.progress(0.92, "Writing transparent PNG")
output_path = output_directory / "foreground.png"
foreground.convert("RGBA").save(
output_path,
format="PNG",
compress_level=1,
)
if not output_path.is_file():
raise RuntimeError("Background removal did not write a foreground image")
context.check_cancelled() context.check_cancelled()
total_ms = (time.perf_counter() - total_started) * 1000.0 total_ms = (time.perf_counter() - total_started) * 1000.0
frame_text = "" if len(images) == 1 else f" from {len(images)} frame(s)"
message = ( message = (
f"Background removed{frame_text} in {inference_ms:.0f} ms " f"Background removed in {inference_ms:.0f} ms "
f"(model load {load_ms:.0f} ms, total {total_ms:.0f} ms)" f"(model load {load_ms:.0f} ms, total {total_ms:.0f} ms)"
) )
context.log(message) context.log(message)
context.progress(1.0, message) context.progress(1.0, message)
return {"alpha": "alpha.npy"} if output_kind == "alpha" else {"image": "."} return {"alpha": "alpha.npy"} if output_kind == "alpha" else {"foreground": "foreground.png"}

43
src/anyimage/server/jobs/upscale.py
View File

@@ -1,13 +1,8 @@
from ..media import DEFAULT_MAX_AI_INPUT_SIZE, input_frames from ..media import DEFAULT_MAX_AI_INPUT_SIZE
from ..media.input import validate_input_path
from ..models.upscale import infer_one from ..models.upscale import infer_one
def frame_progress_message(message, index, total):
if total > 1:
return f"{message} ({index + 1}/{total})"
return message
def run(context, parameters): def run(context, parameters):
context.log( context.log(
f"Upscale model: {parameters['model']}; " f"Upscale model: {parameters['model']}; "
@@ -15,27 +10,17 @@ def run(context, parameters):
f"{parameters.get('max_input_size', DEFAULT_MAX_AI_INPUT_SIZE)} px" f"{parameters.get('max_input_size', DEFAULT_MAX_AI_INPUT_SIZE)} px"
) )
context.progress(0.02, "Preparing upscale input") context.progress(0.02, "Preparing upscale input")
context.directory.mkdir(parents=True, exist_ok=True)
with input_frames(context, parameters) as images:
total = len(images)
for index, image in enumerate(images):
context.check_cancelled()
context.progress(
0.08 + index / total * 0.84,
frame_progress_message("Upscaling image", index, total),
)
prediction = infer_one(
context.resource("model_manager"),
parameters,
image,
cancel_check=context.check_cancelled,
release_memory=index == total - 1,
)
prediction.save(
context.directory / f"upscale.{index + 1:04d}.png",
format="PNG",
compress_level=1,
)
context.check_cancelled() context.check_cancelled()
input_path = validate_input_path(parameters["input"])
context.directory.mkdir(parents=True, exist_ok=True)
context.progress(0.08, "Upscaling image")
prediction = infer_one(
context.resource("model_manager"),
parameters,
input_path,
cancel_check=context.check_cancelled,
)
context.check_cancelled()
prediction.save(context.directory / "upscale.png", format="PNG", compress_level=1)
context.progress(1.0, "Image upscale complete") context.progress(1.0, "Image upscale complete")
return {"upscale": "."} return {"upscale": "upscale.png"}

14
src/anyimage/server/media/__init__.py
View File

@@ -1,14 +1,7 @@
from .input import ( from .input import (
IMAGE_SUFFIXES, IMAGE_SUFFIXES,
VIDEO_SUFFIXES,
TaskCancelled, TaskCancelled,
analysis_frames,
check_cancelled, check_cancelled,
collect_images,
extract_video_frames,
first_frames,
input_frames,
prepare_input,
) )
@@ -23,15 +16,8 @@ from .resolution import (
__all__ = ( __all__ = (
"IMAGE_SUFFIXES", "IMAGE_SUFFIXES",
"VIDEO_SUFFIXES",
"TaskCancelled", "TaskCancelled",
"analysis_frames",
"check_cancelled", "check_cancelled",
"collect_images",
"extract_video_frames",
"first_frames",
"input_frames",
"prepare_input",
"DEFAULT_MAX_AI_INPUT_SIZE", "DEFAULT_MAX_AI_INPUT_SIZE",
"MAX_MAX_AI_INPUT_SIZE", "MAX_MAX_AI_INPUT_SIZE",
"MIN_MAX_AI_INPUT_SIZE", "MIN_MAX_AI_INPUT_SIZE",

168
src/anyimage/server/media/input.py
View File

@@ -1,46 +1,21 @@
import shutil
import tempfile
from contextlib import contextmanager from contextlib import contextmanager
from pathlib import Path from pathlib import Path
IMAGE_SUFFIXES = {".jpg", ".jpeg", ".png", ".webp"} IMAGE_SUFFIXES = {".jpg", ".jpeg", ".png", ".webp"}
VIDEO_SUFFIXES = {
".mp4",
".mov",
".avi",
".mkv",
".webm",
".m4v",
".mts",
".mpg",
".mpeg",
}
def validate_input_path(value, *, images_only=False): def validate_input_path(value):
"""Validate supported media without importing image or video libraries.""" """Validate one supported image file without importing image libraries."""
path = Path(value) path = Path(value)
if path.is_dir() and not images_only:
if not collect_images(path):
supported = ", ".join(sorted(IMAGE_SUFFIXES))
raise RuntimeError(
f"No supported image frames found in: {path}. Supported images: {supported}."
)
return path
if not path.is_file(): if not path.is_file():
kind = "video" if path.suffix.lower() in VIDEO_SUFFIXES else "image" raise RuntimeError(f"Source image file does not exist: {path}")
raise RuntimeError(f"Source {kind} file does not exist: {path}")
suffix = path.suffix.lower() suffix = path.suffix.lower()
supported = IMAGE_SUFFIXES if images_only else IMAGE_SUFFIXES | VIDEO_SUFFIXES if suffix not in IMAGE_SUFFIXES:
if suffix not in supported: raise RuntimeError(
message = (
f"Unsupported input format: {suffix or '(no extension)'}. " f"Unsupported input format: {suffix or '(no extension)'}. "
f"Supported images: {', '.join(sorted(IMAGE_SUFFIXES))}." f"Supported images: {', '.join(sorted(IMAGE_SUFFIXES))}."
) )
if not images_only:
message += f" Supported videos: {', '.join(sorted(VIDEO_SUFFIXES))}."
raise RuntimeError(message)
return path return path
@@ -54,6 +29,8 @@ def open_image(path):
try: try:
opened = Image.open(path) opened = Image.open(path)
opened.load() opened.load()
if getattr(opened, "n_frames", 1) != 1:
raise RuntimeError("Only still images are supported")
except (OSError, ValueError, SyntaxError) as error: except (OSError, ValueError, SyntaxError) as error:
raise RuntimeError(f"Unable to read image: {path}") from error raise RuntimeError(f"Unable to read image: {path}") from error
yield opened yield opened
@@ -69,134 +46,3 @@ class TaskCancelled(RuntimeError):
def check_cancelled(cancel_check): def check_cancelled(cancel_check):
if cancel_check is not None and cancel_check(): if cancel_check is not None and cancel_check():
raise TaskCancelled("Task cancelled") raise TaskCancelled("Task cancelled")
def collect_images(directory):
directory = Path(directory)
return sorted(
path
for path in directory.iterdir()
if path.is_file() and path.suffix.lower() in IMAGE_SUFFIXES
)
def first_frames(images, maximum_frames, start_frame=1):
"""Return up to maximum_frames image paths in order from start_frame."""
if maximum_frames <= 0:
return images
start_index = max(start_frame - 1, 0)
return images[start_index:start_index + maximum_frames]
def extract_video_frames(
video_path,
output_directory,
maximum_frames,
av,
start_frame=1,
cancel_check=None,
):
output_directory = Path(output_directory)
output_directory.mkdir(parents=True, exist_ok=True)
try:
container = av.open(str(video_path))
except Exception as error:
raise RuntimeError(f"Unable to read video: {video_path}") from error
image_paths = []
skipped = 0
try:
try:
stream = container.streams.video[0]
frames = iter(container.decode(stream))
except Exception as error:
raise RuntimeError(f"Unable to read video: {video_path}") from error
while True:
try:
frame = next(frames)
except StopIteration:
break
except Exception as error:
raise RuntimeError(f"Unable to read video: {video_path}") from error
check_cancelled(cancel_check)
if skipped < start_frame - 1:
skipped += 1
continue
image_path = output_directory / f"frame.{len(image_paths) + 1:04d}.jpg"
try:
decoded = frame.to_image()
except Exception as error:
raise RuntimeError(f"Unable to read video: {video_path}") from error
decoded.save(image_path, format="JPEG")
image_paths.append(image_path)
if len(image_paths) >= maximum_frames:
break
finally:
container.close()
if not image_paths:
raise RuntimeError(f"Unable to read video: {video_path}")
return image_paths
def prepare_input(
input_path,
video_frames,
temporary_root,
start_frame=1,
cancel_check=None,
):
check_cancelled(cancel_check)
input_path = validate_input_path(input_path)
if input_path.is_dir():
images = collect_images(input_path)
return first_frames(images, video_frames, start_frame), None
suffix = input_path.suffix.lower()
if suffix in IMAGE_SUFFIXES:
return [input_path], None
import av
temporary_directory = Path(
tempfile.mkdtemp(prefix="anyimage-video-", dir=temporary_root)
)
try:
images = extract_video_frames(
input_path,
temporary_directory,
video_frames,
av,
start_frame=start_frame,
cancel_check=cancel_check,
)
except Exception:
shutil.rmtree(temporary_directory, ignore_errors=True)
raise
return images, temporary_directory
@contextmanager
def input_frames(context, parameters):
"""Yield prepared input frames and clean up extracted video frames."""
images, temporary_directory = prepare_input(
parameters["input"],
parameters.get("video_frames", 16),
context.directory,
start_frame=parameters.get("start_frame", 1),
cancel_check=context.check_cancelled,
)
try:
yield images
finally:
if temporary_directory is not None:
shutil.rmtree(temporary_directory, ignore_errors=True)
@contextmanager
def analysis_frames(context, parameters, fallback_images):
"""Yield model-analysis frames, falling back to the color input."""
analysis_input = parameters.get("analysis_input")
if not analysis_input or analysis_input == parameters["input"]:
yield fallback_images
return
with input_frames(context, dict(parameters, input=analysis_input)) as images:
yield images

24
src/anyimage/server/model_catalog.py
View File

@@ -22,7 +22,7 @@ BIREFNET_LITE = ModelSpec(
directory_name="birefnet-lite", directory_name="birefnet-lite",
repository="ZhengPeng7/BiRefNet_lite", repository="ZhengPeng7/BiRefNet_lite",
license_name="MIT", license_name="MIT",
description="Compact 1024px background removal with continuous alpha", description="Remove backgrounds at 1024 px processing resolution while preserving soft, semi-transparent edges.",
download_patterns=("model.onnx", "LICENSE.txt"), download_patterns=("model.onnx", "LICENSE.txt"),
ready_patterns=("model.onnx", "LICENSE.txt"), ready_patterns=("model.onnx", "LICENSE.txt"),
r2_directory="birefnet-lite", r2_directory="birefnet-lite",
@@ -40,7 +40,7 @@ BEN2_MODEL = ModelSpec(
repository="onnx-community/BEN2-ONNX", repository="onnx-community/BEN2-ONNX",
license_name="Apache 2.0", license_name="Apache 2.0",
huggingface_repository="onnx-community/BEN2-ONNX", huggingface_repository="onnx-community/BEN2-ONNX",
description="General-purpose FP16 ONNX background removal model", description="Remove backgrounds from general images while preserving foreground transparency.",
download_patterns=("onnx/model_fp16.onnx",), download_patterns=("onnx/model_fp16.onnx",),
ready_patterns=("onnx/model_fp16.onnx",), ready_patterns=("onnx/model_fp16.onnx",),
r2_directory="BEN2-ONNX", r2_directory="BEN2-ONNX",
@@ -58,7 +58,7 @@ REALESRGAN_GENERAL_WDN_X4V3 = ModelSpec(
directory_name="realesr-general-wdn-x4v3", directory_name="realesr-general-wdn-x4v3",
repository="xinntao/Real-ESRGAN", repository="xinntao/Real-ESRGAN",
license_name="BSD 3-Clause", license_name="BSD 3-Clause",
description="Fast and conservative 2x output from a 4x ONNX model", description="Enlarge images to twice their width and height with fast, restrained detail enhancement.",
download_patterns=("realesr-general-wdn-x4v3.onnx",), download_patterns=("realesr-general-wdn-x4v3.onnx",),
ready_patterns=("realesr-general-wdn-x4v3.onnx",), ready_patterns=("realesr-general-wdn-x4v3.onnx",),
r2_directory="realesr-general-wdn-x4v3", r2_directory="realesr-general-wdn-x4v3",
@@ -77,7 +77,7 @@ REALESRGAN_X4PLUS = ModelSpec(
repository="jonathanst29/tinier-upscale-models", repository="jonathanst29/tinier-upscale-models",
license_name="BSD 3-Clause", license_name="BSD 3-Clause",
huggingface_repository="jonathanst29/tinier-upscale-models", huggingface_repository="jonathanst29/tinier-upscale-models",
description="Detailed 2x output from a general-purpose 4x ONNX model", description="Enlarge images to twice their width and height with stronger detail enhancement.",
download_patterns=("realesrgan-x4plus.onnx",), download_patterns=("realesrgan-x4plus.onnx",),
ready_patterns=("realesrgan-x4plus.onnx",), ready_patterns=("realesrgan-x4plus.onnx",),
r2_directory="realesrgan-x4plus", r2_directory="realesrgan-x4plus",
@@ -97,7 +97,10 @@ HAT_GAN_X4_SHARPER = ModelSpec(
directory_name="hat-gan-x4-sharper", directory_name="hat-gan-x4-sharper",
repository="XPixelGroup/HAT", repository="XPixelGroup/HAT",
license_name="Apache 2.0", license_name="Apache 2.0",
description="Detailed texture enhancement with 2x output; higher processing cost", description=(
"Enlarge images to twice their width and height with detailed texture enhancement. "
"Requires more processing time."
),
download_patterns=("model.onnx", "LICENSE.txt"), download_patterns=("model.onnx", "LICENSE.txt"),
ready_patterns=("model.onnx", "LICENSE.txt"), ready_patterns=("model.onnx", "LICENSE.txt"),
r2_directory="hat-gan-x4-sharper", r2_directory="hat-gan-x4-sharper",
@@ -129,7 +132,7 @@ MOGE2_MODELS = {
repository="Ruicheng/moge-2-vits-normal-onnx", repository="Ruicheng/moge-2-vits-normal-onnx",
license_name="MIT", license_name="MIT",
huggingface_repository="Ruicheng/moge-2-vits-normal-onnx", huggingface_repository="Ruicheng/moge-2-vits-normal-onnx",
description="Fastest MoGe-2 model using ONNX Runtime", description="Generate depth and surface normals with the fastest MoGe-2 option.",
download_patterns=("model.onnx",), download_patterns=("model.onnx",),
ready_patterns=("model.onnx",), ready_patterns=("model.onnx",),
r2_directory="moge-2-vits-normal-onnx", r2_directory="moge-2-vits-normal-onnx",
@@ -146,7 +149,7 @@ MOGE2_MODELS = {
repository="Ruicheng/moge-2-vitb-normal-onnx", repository="Ruicheng/moge-2-vitb-normal-onnx",
license_name="MIT", license_name="MIT",
huggingface_repository="Ruicheng/moge-2-vitb-normal-onnx", huggingface_repository="Ruicheng/moge-2-vitb-normal-onnx",
description="Balanced MoGe-2 model using ONNX Runtime", description="Generate depth and surface normals with a balance of processing speed and detail.",
download_patterns=("model.onnx",), download_patterns=("model.onnx",),
ready_patterns=("model.onnx",), ready_patterns=("model.onnx",),
r2_directory="moge-2-vitb-normal-onnx", r2_directory="moge-2-vitb-normal-onnx",
@@ -163,7 +166,7 @@ MOGE3_VITL = ModelSpec(
directory_name="moge-3-vitl-onnx", directory_name="moge-3-vitl-onnx",
repository="Ruicheng/moge-3-vitl", repository="Ruicheng/moge-3-vitl",
license_name="MIT", license_name="MIT",
description="MoGe-3 geometry with three refinement steps using ONNX Runtime", description="Generate depth and surface normals with MoGe-3, refining the prediction in three steps.",
download_patterns=("backbone.onnx", "refiner.onnx"), download_patterns=("backbone.onnx", "refiner.onnx"),
ready_patterns=("backbone.onnx", "refiner.onnx"), ready_patterns=("backbone.onnx", "refiner.onnx"),
r2_directory="moge-3-vitl-onnx", r2_directory="moge-3-vitl-onnx",
@@ -180,7 +183,10 @@ BIREFNET_HR_MATTING = ModelSpec(
directory_name="birefnet-hr-matting", directory_name="birefnet-hr-matting",
repository="ZhengPeng7/BiRefNet_HR-matting", repository="ZhengPeng7/BiRefNet_HR-matting",
license_name="MIT", license_name="MIT",
description="2048px matting with continuous alpha; CPU only, high memory and processing cost", description=(
"Remove backgrounds at 2048 px processing resolution while preserving soft, semi-transparent edges. "
"Runs on CPU and requires more time and memory."
),
download_patterns=("model.onnx", "LICENSE.txt"), download_patterns=("model.onnx", "LICENSE.txt"),
ready_patterns=("model.onnx", "LICENSE.txt"), ready_patterns=("model.onnx", "LICENSE.txt"),
r2_directory="birefnet-hr-matting", r2_directory="birefnet-hr-matting",

46
src/anyimage/server/models/background.py
View File

@@ -75,7 +75,7 @@ def infer_foreground(
from PIL import Image, ImageChops from PIL import Image, ImageChops
check_cancelled(cancel_check) check_cancelled(cancel_check)
input_path = validate_input_path(Path(input_path), images_only=True) input_path = validate_input_path(Path(input_path))
with open_image(input_path) as opened_image: with open_image(input_path) as opened_image:
source = opened_image.convert("RGBA") source = opened_image.convert("RGBA")
alpha, load_ms, inference_ms = infer_alpha( alpha, load_ms, inference_ms = infer_alpha(
@@ -91,47 +91,3 @@ def infer_foreground(
foreground.paste(source.convert("RGB"), (0, 0), source.getchannel("A")) foreground.paste(source.convert("RGB"), (0, 0), source.getchannel("A"))
foreground.putalpha(mask) foreground.putalpha(mask)
return foreground, load_ms, inference_ms return foreground, load_ms, inference_ms
def infer_foreground_frames(
images,
model_dir,
device,
output_directory,
progress_callback,
start_progress,
progress_span,
model_cache=None,
cancel_check=None,
output_prefix="foreground",
preserve_input_alpha=False,
):
"""Remove each frame's background and write numbered transparent PNG files."""
total_load_ms = 0.0
total_inference_ms = 0.0
for index, image_path in enumerate(images):
check_cancelled(cancel_check)
progress_callback(
start_progress + progress_span * index / len(images),
f"Removing background ({index + 1}/{len(images)})",
)
foreground, load_ms, inference_ms = infer_foreground(
image_path,
model_dir,
device,
model_cache=model_cache,
cancel_check=cancel_check,
preserve_input_alpha=preserve_input_alpha,
release_memory=index == len(images) - 1,
)
output_path = output_directory / f"{output_prefix}.{index + 1:04d}.png"
foreground.convert("RGBA").save(
output_path,
format="PNG",
compress_level=1,
)
if not output_path.is_file():
raise RuntimeError(f"Background removal did not write: {output_path}")
total_load_ms += load_ms
total_inference_ms += inference_ms
return total_load_ms, total_inference_ms

26
src/anyimage/server/models/moge.py
View File

@@ -38,30 +38,16 @@ def _geometry_frame(prediction, include_points):
def infer( def infer(
model_manager, model_manager,
parameters, parameters,
images, image_path,
*, *,
include_points, include_points,
progress_callback=None,
cancel_check=None, cancel_check=None,
): ):
frames = [] """Infer one image and return its geometry fields."""
for index, image_path in enumerate(images): if cancel_check is not None:
if cancel_check is not None: cancel_check()
cancel_check() prediction = infer_one(model_manager, parameters, image_path)
if progress_callback is not None: return _geometry_frame(prediction, include_points)
progress_callback(index, len(images))
frames.append(
_geometry_frame(
infer_one(
model_manager,
parameters,
image_path,
release_memory=index == len(images) - 1,
),
include_points,
)
)
return frames
def infer_one( def infer_one(

12
tests/anyimage/common/support.py
View File

@@ -9,14 +9,6 @@ from tests.support.blender import mock_bpy_imports
def load_common_modules(): def load_common_modules():
fake_bpy = ModuleType("bpy") fake_bpy = ModuleType("bpy")
fake_bpy.props = SimpleNamespace(
BoolProperty=lambda **options: options,
EnumProperty=lambda **options: options,
FloatProperty=lambda **options: options,
IntProperty=lambda **options: options,
StringProperty=lambda **options: options,
)
fake_bpy.types = SimpleNamespace(Menu=object, Operator=object)
fake_bpy.path = SimpleNamespace(abspath=lambda path: path) fake_bpy.path = SimpleNamespace(abspath=lambda path: path)
fake_bpy.context = SimpleNamespace(scene=None) fake_bpy.context = SimpleNamespace(scene=None)
@@ -58,9 +50,7 @@ def load_common_modules():
name: ModuleType(name) name: ModuleType(name)
for name in ( for name in (
"anyimage", "anyimage",
"anyimage.operators",
"anyimage.common", "anyimage.common",
"anyimage.operators.convert_to_plane",
) )
} }
for package in packages.values(): for package in packages.values():
@@ -80,4 +70,4 @@ def load_common_modules():
specification.loader.exec_module(module) specification.loader.exec_module(module)
loaded.append(module) loaded.append(module)
return SimpleNamespace(image=loaded[0], node=loaded[1], material=loaded[2], object=loaded[3]) return SimpleNamespace(image=loaded[0], material=loaded[2], object=loaded[3])

52
tests/anyimage/common/test_alpha_presentation.py
View File

@@ -35,9 +35,9 @@ def test_owned_color_images_preserve_source_and_use_premul(floating):
def test_job_loader_preserves_edit_alpha_policy(tmp_path): def test_job_loader_preserves_edit_alpha_policy(tmp_path):
rgba = np.array([[[51, 102, 204, 64]]], dtype=np.uint8) rgba = np.array([[[51, 102, 204, 64]]], dtype=np.uint8)
Image.fromarray(rgba).save(tmp_path / "image.0001.png") Image.fromarray(rgba).save(tmp_path / "foreground.png")
original = bpy.data.images.new("Job source", width=1, height=1, alpha=True) original = bpy.data.images.new("Job source", width=1, height=1, alpha=True)
result = images.load_image_edit_result(original, tmp_path) result = images.load_image_edit_result(original, tmp_path / "foreground.png")
try: try:
assert result.alpha_mode == "STRAIGHT" assert result.alpha_mode == "STRAIGHT"
assert result.packed_file is not None assert result.packed_file is not None
@@ -47,27 +47,6 @@ def test_job_loader_preserves_edit_alpha_policy(tmp_path):
bpy.data.images.remove(original) bpy.data.images.remove(original)
def test_failed_empty_replacement_restores_blending_and_timing():
original = SimpleNamespace(name="Original", use_fake_user=False, users=1)
timing = SimpleNamespace(frame_start=3, frame_offset=2, frame_duration=7, use_cyclic=True)
source = SimpleNamespace(data=original, use_empty_image_alpha=False, image_user=timing)
result = SimpleNamespace(source="SEQUENCE", users=1)
def fail_configuration(*args, **kwargs):
timing.frame_start = 99
raise RuntimeError("Failed configuration")
with patch.object(images, "has_other_image_empty", return_value=True), patch.object(images, "video_total_frames", return_value=3), patch.object(
images, "configure_image_user", side_effect=fail_configuration
):
with pytest.raises(RuntimeError, match="Failed configuration"):
images.replace_empty_image(source, result)
assert source.data is original
assert not source.use_empty_image_alpha
assert not original.use_fake_user
assert images.image_user_settings(source) == (3, 2, 7, True)
def test_business_pixels_ignore_premul_read_interpretation(): def test_business_pixels_ignore_premul_read_interpretation():
original = bpy.data.images.new("Premul read source", width=1, height=1, alpha=True) original = bpy.data.images.new("Premul read source", width=1, height=1, alpha=True)
original.alpha_mode = "PREMUL" original.alpha_mode = "PREMUL"
@@ -188,10 +167,11 @@ def test_isolated_decode_failure_releases_copy():
assert source.alpha_mode == "PREMUL" assert source.alpha_mode == "PREMUL"
def test_mask_pack_reopen_add_and_repeat_frame_preserve_hidden_color(tmp_path): def test_mask_pack_reopen_extend_and_repeat_frame_preserve_hidden_color(tmp_path):
from anyimage.common.selection import SelectionMask from anyimage.common.selection import SelectionMask
from anyimage.operators.mask_tool import apply_alpha_mask from anyimage.operators.mask_tool import apply_alpha_mask
from anyimage.operators import frame_tool as frame from anyimage.operators.frame_tool.projection import frame_source_projection
from anyimage.operators.frame_tool.compositing import composite_frame_pixels
source = bpy.data.images.new("Roundtrip source", width=2, height=2, alpha=True) source = bpy.data.images.new("Roundtrip source", width=2, height=2, alpha=True)
rgba = np.tile(np.array([0.2, 0.4, 0.8, 1], dtype=np.float32), (2, 2, 1)) rgba = np.tile(np.array([0.2, 0.4, 0.8, 1], dtype=np.float32), (2, 2, 1))
@@ -208,15 +188,15 @@ def test_mask_pack_reopen_add_and_repeat_frame_preserve_hidden_color(tmp_path):
restored = images.image_rgba(result) restored = images.image_rgba(result)
np.testing.assert_allclose(restored[..., :3], rgba[..., :3], atol=1 / 255) np.testing.assert_allclose(restored[..., :3], rgba[..., :3], atol=1 / 255)
np.testing.assert_array_equal(restored[..., 3], 0) np.testing.assert_array_equal(restored[..., 3], 0)
added = images.create_image_edit_result(result, apply_alpha_mask(restored, full, "ADD"), (2, 2)) added = images.create_image_edit_result(result, apply_alpha_mask(restored, full, "EXTEND"), (2, 2))
bpy.data.images.remove(result) bpy.data.images.remove(result)
result = added result = added
for _ in range(2): for _ in range(2):
source = frame.frame_source_projection(np.eye(4), np.eye(4), np.eye(4), (-1, 1, -1, 1), (2, 2), perspective=False) source = frame_source_projection(np.eye(4), np.eye(4), np.eye(4), (-1, 1, -1, 1), (2, 2), perspective=False)
rgba_read = images.image_rgba(result) rgba_read = images.image_rgba(result)
source.update(rgba=rgba_read, pixels=images.premultiplied_rgba(images.image_pixels(result), (2, 2)), source.update(rgba=rgba_read, pixels=images.premultiplied_rgba(images.image_pixels(result), (2, 2)),
image_size=(2, 2), active=True, name=result.name) image_size=(2, 2), active=True, name=result.name)
pixels = frame.composite_frame_pixels((source,), ((0, 2), (2, 2), (2, 0), (0, 0)), (2, 2)) pixels = composite_frame_pixels((source,), ((0, 2), (2, 2), (2, 0), (0, 0)), (2, 2))
next_result = images.create_image_edit_result(result, pixels, (2, 2)) next_result = images.create_image_edit_result(result, pixels, (2, 2))
bpy.data.images.remove(result) bpy.data.images.remove(result)
result = next_result result = next_result
@@ -225,3 +205,19 @@ def test_mask_pack_reopen_add_and_repeat_frame_preserve_hidden_color(tmp_path):
np.testing.assert_allclose(images.image_rgba(result), rgba, atol=1 / 255) np.testing.assert_allclose(images.image_rgba(result), rgba, atol=1 / 255)
finally: finally:
bpy.data.images.remove(result) bpy.data.images.remove(result)
def test_job_loader_reads_only_the_returned_file(tmp_path):
Image.new("RGBA", (2, 2), (255, 0, 0, 255)).save(tmp_path / "foreground.png")
Image.new("RGBA", (4, 4), (0, 0, 255, 255)).save(tmp_path / "upscale.png")
original = bpy.data.images.new("Source", width=1, height=1, alpha=True)
result = images.load_image_edit_result(original, tmp_path / "upscale.png")
try:
assert tuple(result.size) == (4, 4)
assert result.source == "FILE"
assert result.packed_file is not None
with pytest.raises(RuntimeError, match="Color result image does not exist"):
images.load_image_edit_result(original, tmp_path / "missing.png")
finally:
bpy.data.images.remove(result)
bpy.data.images.remove(original)

6
tests/anyimage/common/test_color_image.py
View File

@@ -171,8 +171,10 @@ def test_hdr_analysis_preview_is_separate_from_full_precision_color(source):
assert not preview.exists() assert not preview.exists()
@pytest.mark.parametrize("alpha_mode", ["STRAIGHT", "PREMUL", "CHANNEL_PACKED", "NONE"]) @pytest.mark.parametrize("alpha_mode,source_kind", [
@pytest.mark.parametrize("source_kind", ["generated", "file", "dirty", "packed"]) ("STRAIGHT", "generated"), ("PREMUL", "file"),
("CHANNEL_PACKED", "dirty"), ("NONE", "packed"),
])
def test_float_color_conversion_preserves_rendered_color_and_opacity(source, tmp_path, alpha_mode, source_kind): def test_float_color_conversion_preserves_rendered_color_and_opacity(source, tmp_path, alpha_mode, source_kind):
from anyimage.common.hdr_image import save_hdr_result from anyimage.common.hdr_image import save_hdr_result

25
tests/anyimage/common/test_depth_metadata.py Normal file
View File

@@ -0,0 +1,25 @@
import json
import numpy as np
from anyimage.common.depth import load_depth_metadata
def test_depth_metadata_loader_uses_one_schema_for_plane_and_cutout(tmp_path):
path = tmp_path / "depth.json"
path.write_text(
json.dumps(
{
"image_size": [3, 2],
"intrinsics": np.eye(3).tolist(),
"reference_depth": 1.0,
}
),
encoding="utf-8",
)
metadata = load_depth_metadata(path)
assert metadata["image_size"] == (3, 2)
assert metadata["intrinsics"] == ((1.0, 0.0, 0.0), (0.0, 1.0, 0.0), (0.0, 0.0, 1.0))
assert metadata["reference_depth"] == 1.0

93
tests/anyimage/common/test_empty_image_edits.py
View File

@@ -33,9 +33,12 @@ def edit_result(source):
) )
@pytest.mark.parametrize("alpha_mode", ["STRAIGHT", "PREMUL", "CHANNEL_PACKED", "NONE"]) @pytest.mark.parametrize("alpha_mode,shared,entry", [
@pytest.mark.parametrize("shared", [False, True]) ("STRAIGHT", False, "pixels"),
@pytest.mark.parametrize("entry", ["pixels", "image.", "upscale."]) ("PREMUL", True, "foreground.png"),
("CHANNEL_PACKED", False, "upscale.png"),
("NONE", True, "pixels"),
])
def test_image_edit_replacement_preserves_alpha_mode(source, tmp_path, alpha_mode, shared, entry): def test_image_edit_replacement_preserves_alpha_mode(source, tmp_path, alpha_mode, shared, entry):
original = source.data original = source.data
original.alpha_mode = alpha_mode original.alpha_mode = alpha_mode
@@ -44,8 +47,8 @@ def test_image_edit_replacement_preserves_alpha_mode(source, tmp_path, alpha_mod
if entry == "pixels": if entry == "pixels":
result = edit_result(source) result = edit_result(source)
else: else:
Image.new("RGBA", (4, 3), (100, 150, 200, 128)).save(tmp_path / f"{entry}0001.png") Image.new("RGBA", (4, 3), (100, 150, 200, 128)).save(tmp_path / entry)
result = images.load_image_edit_result(original, tmp_path, frame_prefix=entry) result = images.load_image_edit_result(original, tmp_path / entry)
assert result.alpha_mode == alpha_mode assert result.alpha_mode == alpha_mode
images.replace_empty_image(source, result) images.replace_empty_image(source, result)
assert source.data.alpha_mode == alpha_mode assert source.data.alpha_mode == alpha_mode
@@ -131,45 +134,6 @@ def test_frame_sharing_uses_surviving_objects(source, outside):
assert tuple(original.size) == (2, 2) assert tuple(original.size) == (2, 2)
@pytest.mark.parametrize("shared", [False, True])
def test_failed_display_configuration_restores_content(source, tmp_path, shared):
original = source.data
if shared:
create_empty("Other", original)
# Preserve unsaved pixels as well as the previous packed representation.
original.pixels.foreach_set(np.tile([0.1, 0.9, 0.2, 0.5], 4).astype(np.float32))
packed = bytes(original.packed_file.data)
pixels = images.image_pixels(original)
alpha = original.alpha_mode
source.image_user.frame_start = 5
source.image_user.use_auto_refresh = False
for index in (1, 2):
Image.new("RGBA", (4, 3), (100, 150, 200, 255)).save(tmp_path / f"image.{index:04d}.png")
result = bpy.data.images.load(str(tmp_path / "image.0001.png"))
result.source = "SEQUENCE"
count = len(bpy.data.images) - 1
def fail_configuration(user, *args, **kwargs):
user.frame_start = 99
user.use_auto_refresh = True
raise RuntimeError("Display failed")
with patch.object(images, "configure_image_user", side_effect=fail_configuration):
with pytest.raises(RuntimeError, match="Display failed"):
images.replace_empty_image(source, result)
assert source.data == original
assert tuple(original.size) == (2, 2)
assert original.alpha_mode == alpha
assert original.is_dirty
assert bytes(original.packed_file.data) == packed
np.testing.assert_array_equal(images.image_pixels(original), pixels)
assert source.image_user.frame_start == 5
assert not source.image_user.use_auto_refresh
assert not source.use_empty_image_alpha
assert not original.use_fake_user
assert len(bpy.data.images) == count
def test_failed_content_write_restores_source(source): def test_failed_content_write_restores_source(source):
original = source.data original = source.data
packed = bytes(original.packed_file.data) packed = bytes(original.packed_file.data)
@@ -193,47 +157,6 @@ def test_failed_content_write_restores_source(source):
assert len(bpy.data.images) == 1 assert len(bpy.data.images) == 1
def test_animated_result_preserves_id_and_reopens(source, tmp_path):
original = source.data
source.image_user.frame_start = 4
source.image_user.use_cyclic = True
for index in (1, 2):
Image.new("RGBA", (4, 3), (index * 80, 100, 50, 255)).save(tmp_path / f"image.{index:04d}.png")
original.source = "SEQUENCE"
result = images.load_image_edit_result(source.data, tmp_path)
expected_path = result.filepath_raw
expected_packed = bytes(result.packed_file.data) if result.packed_file else None
final = images.replace_empty_image(source, result)
assert final == original
assert final.source == "SEQUENCE"
assert final.filepath_raw == expected_path
assert (bytes(final.packed_file.data) if final.packed_file else None) == expected_packed
assert len(images.collect_sequence_frames(final)) == 2
assert source.image_user.frame_start == 4
assert source.image_user.frame_duration == 2
assert source.image_user.use_cyclic
assert source.image_user.use_auto_refresh
path = str(tmp_path / "animation.blend")
bpy.ops.wm.save_as_mainfile(filepath=path)
bpy.ops.wm.open_mainfile(filepath=path)
reopened = bpy.data.objects["Source"].data
assert reopened.name == "Source.png"
assert reopened.source == "SEQUENCE"
frames = images.collect_sequence_frames(reopened)
assert len(frames) == 2
for index, frame in enumerate(frames, 1):
loaded = bpy.data.images.load(str(frame), check_existing=False)
try:
assert tuple(loaded.size) == (4, 3)
np.testing.assert_allclose(
images.image_pixels(loaded)[:4],
np.array([index * 80, 100, 50, 255]) / 255,
atol=1 / 255,
)
finally:
bpy.data.images.remove(loaded)
@pytest.mark.parametrize("floating", [False, True]) @pytest.mark.parametrize("floating", [False, True])
def test_failed_generated_commit_restores_unsaved_pixels(source, floating): def test_failed_generated_commit_restores_unsaved_pixels(source, floating):
original = bpy.data.images.new("Generated", width=3, height=2, alpha=True, float_buffer=floating) original = bpy.data.images.new("Generated", width=3, height=2, alpha=True, float_buffer=floating)

241
tests/anyimage/common/test_image_data.py
View File

@@ -1,6 +1,4 @@
import tempfile
import unittest import unittest
from pathlib import Path
from types import SimpleNamespace from types import SimpleNamespace
from tests.anyimage.common.support import load_common_modules from tests.anyimage.common.support import load_common_modules
@@ -21,165 +19,6 @@ class ImageDataTest(unittest.TestCase):
) )
def test_video_total_frames_reads_movie_duration(self):
image = SimpleNamespace(source="MOVIE", frame_duration=97)
self.assertEqual(self.modules.image.video_total_frames(image), 97)
def test_video_total_frames_counts_sequence_files(self):
with tempfile.TemporaryDirectory() as directory:
root = Path(directory)
(root / "foreground-0001.png").touch()
(root / "foreground-0002.png").touch()
image = SimpleNamespace(
source="SEQUENCE",
filepath=str(root / "foreground-0001.png"),
)
self.assertEqual(self.modules.image.video_total_frames(image), 2)
def test_video_total_frames_returns_one_for_still_image(self):
image = SimpleNamespace(source="FILE", frame_duration=1)
self.assertEqual(self.modules.image.video_total_frames(image), 1)
def test_extract_frame_count_uses_empty_image_frames(self):
source_object = SimpleNamespace(
data=SimpleNamespace(source="MOVIE", frame_duration=97),
image_user=SimpleNamespace(frame_duration=24),
)
self.assertEqual(self.modules.image.extract_frame_range(source_object.data, getattr(source_object, "image_user", None))[1], 24)
def test_extract_frame_count_caps_at_video_total(self):
source_object = SimpleNamespace(
data=SimpleNamespace(source="MOVIE", frame_duration=97),
image_user=SimpleNamespace(frame_duration=200),
)
self.assertEqual(self.modules.image.extract_frame_range(source_object.data, getattr(source_object, "image_user", None))[1], 97)
def test_extract_frame_count_falls_back_to_video_total(self):
source_object = SimpleNamespace(
data=SimpleNamespace(source="MOVIE", frame_duration=97),
)
self.assertEqual(self.modules.image.extract_frame_range(source_object.data, getattr(source_object, "image_user", None))[1], 97)
def test_extract_frame_range_honors_offset_within_clip(self):
source_object = SimpleNamespace(
data=SimpleNamespace(source="MOVIE", frame_duration=97),
image_user=SimpleNamespace(
frame_start=4,
frame_offset=70,
frame_duration=24,
),
)
self.assertEqual(
self.modules.image.extract_frame_range(source_object.data, source_object.image_user),
(71, 24),
)
def test_extract_frame_range_clamps_large_offset_to_clip_tail(self):
source_object = SimpleNamespace(
data=SimpleNamespace(source="MOVIE", frame_duration=97),
image_user=SimpleNamespace(
frame_start=1,
frame_offset=90,
frame_duration=24,
),
)
self.assertEqual(
self.modules.image.extract_frame_range(source_object.data, source_object.image_user),
(74, 24),
)
def test_extract_frame_range_clamps_negative_offset_to_start(self):
source_object = SimpleNamespace(
data=SimpleNamespace(source="MOVIE", frame_duration=97),
image_user=SimpleNamespace(
frame_start=1,
frame_offset=-5,
frame_duration=24,
),
)
self.assertEqual(
self.modules.image.extract_frame_range(source_object.data, source_object.image_user),
(1, 24),
)
def test_extract_frame_range_returns_full_clip_for_still(self):
source_object = SimpleNamespace(
data=SimpleNamespace(source="FILE", frame_duration=1),
image_user=SimpleNamespace(
frame_start=1,
frame_offset=50,
frame_duration=24,
),
)
self.assertEqual(
self.modules.image.extract_frame_range(source_object.data, source_object.image_user),
(1, 1),
)
def test_extract_frame_range_honors_single_frame_setting(self):
source_object = SimpleNamespace(
data=SimpleNamespace(source="MOVIE", frame_duration=97),
image_user=SimpleNamespace(
frame_start=1,
frame_offset=70,
frame_duration=1,
),
)
self.assertEqual(
self.modules.image.extract_frame_range(source_object.data, source_object.image_user),
(71, 1),
)
def test_configure_image_user_preserves_cyclic_setting(self):
image_user = SimpleNamespace(
frame_start=0,
frame_offset=5,
use_auto_refresh=False,
use_cyclic=False,
frame_duration=1,
)
self.modules.image.configure_image_user(
image_user,
97,
frame_start=4,
frame_offset=70,
)
self.assertFalse(image_user.use_cyclic)
self.assertTrue(image_user.use_auto_refresh)
self.assertEqual(image_user.frame_start, 4)
self.assertEqual(image_user.frame_offset, 70)
self.assertEqual(image_user.frame_duration, 97)
self.modules.image.configure_image_user(
image_user,
use_cyclic=True,
)
self.assertTrue(image_user.use_cyclic)
def test_image_user_settings_reads_empty_values(self): def test_image_user_settings_reads_empty_values(self):
source_object = SimpleNamespace( source_object = SimpleNamespace(
image_user=SimpleNamespace( image_user=SimpleNamespace(
@@ -208,84 +47,6 @@ class ImageDataTest(unittest.TestCase):
) )
def test_image_base_name_strips_video_extension(self):
image = SimpleNamespace(name="微信视频2026-07-23_192756_414.mp4")
self.assertEqual(
self.modules.image.image_base_name(image),
"微信视频2026-07-23_192756_414",
)
def test_force_sequence_source_keeps_loaded_sequence(self):
image = SimpleNamespace(source="SEQUENCE")
result = self.modules.image.force_sequence_source(image)
self.assertIs(result, image)
self.assertEqual(image.source, "SEQUENCE")
def test_force_sequence_source_switches_single_image(self):
image = SimpleNamespace(source="FILE")
result = self.modules.image.force_sequence_source(image)
self.assertIs(result, image)
self.assertEqual(image.source, "SEQUENCE")
def test_force_sequence_source_reports_unloadable_sequence(self):
class BlockedImage:
@property
def source(self):
return "FILE"
@source.setter
def source(self, value):
raise RuntimeError("blocked")
with self.assertRaisesRegex(RuntimeError, "cannot load"):
self.modules.image.force_sequence_source(BlockedImage())
def test_collect_sequence_frames_returns_numbered_files_in_order(self):
with tempfile.TemporaryDirectory() as directory:
root = Path(directory)
(root / "frame-0001.jpg").touch()
(root / "frame-0002.jpg").touch()
(root / "frame-0003.jpg").touch()
(root / "notes.txt").touch()
image = SimpleNamespace(
source="SEQUENCE",
filepath=str(root / "frame-0002.jpg"),
)
frames = self.modules.image.collect_sequence_frames(image)
self.assertEqual(
[path.name for path in frames],
["frame-0001.jpg", "frame-0002.jpg", "frame-0003.jpg"],
)
def test_sorted_frame_paths_uses_numeric_frame_order(self):
with tempfile.TemporaryDirectory() as directory:
root = Path(directory)
(root / "frame-2.jpg").touch()
(root / "frame-10.jpg").touch()
(root / "frame-1.jpg").touch()
frames = self.modules.image.sorted_frame_paths(root)
self.assertEqual(
[path.name for path in frames],
["frame-1.jpg", "frame-2.jpg", "frame-10.jpg"],
)
def test_depth_data_uses_source_image_name(self): def test_depth_data_uses_source_image_name(self):
source_object = SimpleNamespace( source_object = SimpleNamespace(
name="obj_img", name="obj_img",
@@ -315,8 +76,6 @@ class ImageDataTest(unittest.TestCase):
) )
def test_image_empty_bounds_keep_image_aspect_and_empty_offset(self): def test_image_empty_bounds_keep_image_aspect_and_empty_offset(self):
source = SimpleNamespace( source = SimpleNamespace(
data=SimpleNamespace(size=(800, 400)), data=SimpleNamespace(size=(800, 400)),

1
tests/anyimage/common/test_image_materials.py
View File

@@ -5,7 +5,6 @@ import numpy as np
from anyimage.common import image as image_data from anyimage.common import image as image_data
from anyimage.common import material as material_data from anyimage.common import material as material_data
from anyimage.operators.convert_to_plane import object as plane_object
from anyimage.operators.cutout_tool import object as cutout_object from anyimage.operators.cutout_tool import object as cutout_object
from anyimage.operators.cutout_tool import shape as cutout_shape from anyimage.operators.cutout_tool import shape as cutout_shape
from tests.support.image_objects import _create_cutout_shape, _group_node, _inputs from tests.support.image_objects import _create_cutout_shape, _group_node, _inputs

99
tests/anyimage/common/test_image_target.py
View File

@@ -18,21 +18,19 @@ def test_target_uses_edit_tree_and_survives_renaming(texture, tmp_path):
texture.id_data.nodes.active = other texture.id_data.nodes.active = other
context.space_data.edit_tree = None context.space_data.edit_tree = None
write_result(tmp_path) write_result(tmp_path)
target.apply(tmp_path) target.apply(tmp_path / "foreground.png")
assert tuple(texture.image.size) == (8, 6) assert tuple(texture.image.size) == (8, 6)
assert other.image is None assert other.image is None
@pytest.mark.parametrize("color_space,alpha_mode", [ @pytest.mark.parametrize("color_space,alpha_mode,filename,shared", [
("sRGB", "PREMUL"), ("sRGB", "PREMUL", "foreground.png", True),
("AgX Base sRGB", "STRAIGHT"), ("AgX Base sRGB", "STRAIGHT", "upscale.png", False),
("Filmic sRGB", "STRAIGHT"), ("Filmic sRGB", "STRAIGHT", "foreground.png", False),
("sRGB", "STRAIGHT"), ("sRGB", "STRAIGHT", "upscale.png", True),
("AgX Base sRGB", "PREMUL"), ("AgX Base sRGB", "PREMUL", "foreground.png", True),
]) ])
@pytest.mark.parametrize("prefix", ["image.", "upscale."]) def test_image_actions_preserve_source_color_space_and_alpha(texture, tmp_path, color_space, alpha_mode, filename, shared):
@pytest.mark.parametrize("shared", [False, True])
def test_image_actions_preserve_source_color_space_and_alpha(texture, tmp_path, color_space, alpha_mode, prefix, shared):
texture.image.colorspace_settings.name = color_space texture.image.colorspace_settings.name = color_space
texture.image.alpha_mode = alpha_mode texture.image.alpha_mode = alpha_mode
original = texture.image original = texture.image
@@ -41,8 +39,8 @@ def test_image_actions_preserve_source_color_space_and_alpha(texture, tmp_path,
other = texture.id_data.nodes.new("ShaderNodeTexImage") other = texture.id_data.nodes.new("ShaderNodeTexImage")
other.image = original other.image = original
target = targets.ImageEditTarget.capture(texture_context(texture)) target = targets.ImageEditTarget.capture(texture_context(texture))
write_result(tmp_path, prefix) write_result(tmp_path, filename)
target.apply(tmp_path, frame_prefix=prefix) target.apply(tmp_path / filename)
result = texture.image result = texture.image
assert result.colorspace_settings.name == color_space assert result.colorspace_settings.name == color_space
assert result.alpha_mode == alpha_mode assert result.alpha_mode == alpha_mode
@@ -102,7 +100,7 @@ def test_target_rejects_changed_identity_before_loading_result(texture, tmp_path
bpy.data.images.remove(texture.image) bpy.data.images.remove(texture.image)
count = len(bpy.data.images) count = len(bpy.data.images)
with pytest.raises(RuntimeError, match="no longer available"): with pytest.raises(RuntimeError, match="no longer available"):
target.apply(tmp_path) target.apply(tmp_path / "foreground.png")
assert len(bpy.data.images) == count assert len(bpy.data.images) == count
@@ -137,7 +135,7 @@ def test_texture_commit_preserves_structure_and_isolates_users(texture, tmp_path
shared = reference in {"same_tree", "other_material", "empty", "brush"} shared = reference in {"same_tree", "other_material", "empty", "brush"}
links = [(link.from_socket, link.to_socket) for link in texture.id_data.links] links = [(link.from_socket, link.to_socket) for link in texture.id_data.links]
write_result(tmp_path) write_result(tmp_path)
target.apply(tmp_path) target.apply(tmp_path / "foreground.png")
assert (texture.image == original) == (not shared) assert (texture.image == original) == (not shared)
assert tuple(texture.image.size) == (8, 6) assert tuple(texture.image.size) == (8, 6)
assert texture.image.colorspace_settings.name == "Non-Color" assert texture.image.colorspace_settings.name == "Non-Color"
@@ -163,63 +161,46 @@ def test_texture_commit_preserves_structure_and_isolates_users(texture, tmp_path
assert len(reopened.id_data.links) == len(links) assert len(reopened.id_data.links) == len(links)
@pytest.mark.parametrize("shared", [False, True])
def test_animated_texture_preserves_playback_and_reopens(texture, tmp_path, shared):
write_result(tmp_path, count=4) @pytest.mark.parametrize("source", ["MOVIE", "SEQUENCE"])
texture.image.source = "SEQUENCE" def test_target_rejects_animated_image(texture, source):
texture.image.filepath_raw = str(tmp_path / "image.0001.png") texture.image.source = source
texture.image_user.frame_start = 9 with pytest.raises(RuntimeError, match="Only still images"):
texture.image_user.frame_offset = 1 targets.ImageEditTarget.capture(texture_context(texture))
texture.image_user.frame_duration = 2
texture.image_user.use_cyclic = True
context = texture_context(texture)
target = targets.ImageEditTarget.capture(context)
assert (target.start_frame, target.frame_count) == (2, 2)
if shared:
other = texture.id_data.nodes.new("ShaderNodeTexImage")
other.image = texture.image
with patch.object(targets, "prepare_image_input", return_value=(tmp_path, False)):
target.prepare(context)
assert context.scene.anyimage_settings.video_frames == 2
target.apply(tmp_path)
assert images.image_user_settings(texture) == (9, 0, 4, True)
assert texture.image_user.use_auto_refresh
assert texture.image.source == "SEQUENCE"
path = str(tmp_path / "animated.blend")
bpy.ops.wm.save_as_mainfile(filepath=path)
bpy.ops.wm.open_mainfile(filepath=path)
reopened = bpy.data.materials["Texture material"].node_tree.nodes["Source texture"]
assert images.image_user_settings(reopened) == (9, 0, 4, True)
assert len(images.collect_sequence_frames(reopened.image)) == 4
@pytest.mark.parametrize("shared", [False, True]) @pytest.mark.parametrize("shared", [False, True])
def test_texture_failed_commit_restores_dirty_pixels_and_playback(texture, tmp_path, shared): def test_failed_texture_assignment_restores_source(texture, tmp_path, shared):
original = texture.image original = texture.image
if shared:
other = texture.id_data.nodes.new("ShaderNodeTexImage")
other.image = original
original.pixels.foreach_set(np.tile([0.1, 0.3, 0.7, 0.5], 12).astype(np.float32)) original.pixels.foreach_set(np.tile([0.1, 0.3, 0.7, 0.5], 12).astype(np.float32))
packed = bytes(original.packed_file.data) packed = bytes(original.packed_file.data)
pixels = images.image_pixels(original) pixels = images.image_pixels(original)
timing = images.image_user_settings(texture) result = images.load_image_edit_result(original, write_result(tmp_path))
write_result(tmp_path, count=2)
result = bpy.data.images.load(str(tmp_path / "image.0001.png"))
result.source = "SEQUENCE"
count = len(bpy.data.images) - 1 count = len(bpy.data.images) - 1
def fail_configuration(user, *args, **kwargs): class FailingTarget:
user.frame_start = 80 failed = False
user.use_auto_refresh = True
raise RuntimeError("Playback failed")
with patch.object(targets, "configure_image_user", side_effect=fail_configuration): def __getattr__(self, name):
with pytest.raises(RuntimeError, match="Playback failed"): return getattr(texture, name)
targets.replace_texture_image(texture, result)
@property
def image(self):
return texture.image
@image.setter
def image(self, value):
texture.image = value
if not self.failed:
self.failed = True
raise RuntimeError("Assignment failed")
with patch.object(targets, "has_other_image_user", return_value=shared):
with pytest.raises(RuntimeError, match="Assignment failed"):
targets.replace_texture_image(FailingTarget(), result)
assert texture.image == original assert texture.image == original
assert tuple(original.size) == (4, 3) assert tuple(original.size) == (4, 3)
assert bytes(original.packed_file.data) == packed assert bytes(original.packed_file.data) == packed
np.testing.assert_array_equal(images.image_pixels(original), pixels) np.testing.assert_array_equal(images.image_pixels(original), pixels)
assert images.image_user_settings(texture) == timing
assert not texture.image_user.use_auto_refresh
assert len(bpy.data.images) == count assert len(bpy.data.images) == count

40
tests/anyimage/common/test_input_preparation.py
View File

@@ -7,20 +7,6 @@ from tests.support.blender import BlenderTestCase
class InputPreparationTest(BlenderTestCase): class InputPreparationTest(BlenderTestCase):
def test_sequence_format_is_checked_before_copy(self):
with tempfile.TemporaryDirectory() as directory:
path = Path(directory) / "frame.0001.exr"
path.touch()
image = SimpleNamespace(source="SEQUENCE")
with (
patch.object(self.image_data, "image_path", return_value=path),
patch.object(self.image_data, "image_is_packed", return_value=False),
patch.object(self.image_data.tempfile, "mkdtemp") as create,
):
with self.assertRaisesRegex(RuntimeError, "Unsupported input format: .exr"):
self.image_data.prepare_image_input(image, None)
create.assert_not_called()
self.assertTrue(path.exists())
def test_packed_export_failure_removes_temporary_directory(self): def test_packed_export_failure_removes_temporary_directory(self):
for failure in (OSError("disk full"), None): for failure in (OSError("disk full"), None):
@@ -34,21 +20,11 @@ class InputPreparationTest(BlenderTestCase):
self.assertEqual(image.file_format, "OPEN_EXR") self.assertEqual(image.file_format, "OPEN_EXR")
self.assertFalse(output.exists()) self.assertFalse(output.exists())
def test_sequence_copy_failure_preserves_originals(self):
with tempfile.TemporaryDirectory() as directory: def test_animated_inputs_are_rejected_before_export(self):
root = Path(directory) for source in ("MOVIE", "SEQUENCE"):
source = root / "frame.0001.png" with self.subTest(source=source), patch.object(self.image_data, "save_packed_image") as export:
source.write_bytes(b"source") image = SimpleNamespace(source=source, packed_file=object())
output = root / "prepared" with self.assertRaisesRegex(RuntimeError, "Only still images"):
output.mkdir() self.image_data.prepare_image_input(image, None)
with ( export.assert_not_called()
patch.object(self.image_data, "image_path", return_value=source),
patch.object(self.image_data, "image_is_packed", return_value=False),
patch.object(self.image_data, "collect_sequence_frames", return_value=[source]),
patch.object(self.image_data.tempfile, "mkdtemp", return_value=str(output)),
patch.object(self.image_data.shutil, "copy2", side_effect=OSError("disk full")),
):
with self.assertRaises(OSError):
self.image_data.prepare_image_input(SimpleNamespace(source="SEQUENCE"), None)
self.assertFalse(output.exists())
self.assertEqual(source.read_bytes(), b"source")

1
tests/anyimage/common/test_material_alpha.py
View File

@@ -19,7 +19,6 @@ def test_material_alpha_strength_follows_preference(image_layer, monkeypatch, en
assert shader.node_tree.name == "O Shadeless" assert shader.node_tree.name == "O Shadeless"
assert shader.inputs["Color"].links[0].from_node.type == "TEX_IMAGE" assert shader.inputs["Color"].links[0].from_node.type == "TEX_IMAGE"
assert shader.inputs["Alpha"].links[0].from_socket.name == "Alpha" assert shader.inputs["Alpha"].links[0].from_socket.name == "Alpha"
assert not any(node.type in {"MIX_SHADER", "BSDF_TRANSPARENT"} for node in result.node_tree.nodes)
assert len([node for node in result.node_tree.nodes if node.type == "GROUP"]) == 1 assert len([node for node in result.node_tree.nodes if node.type == "GROUP"]) == 1
return return
layer = next(node for node in result.node_tree.nodes if node.type == "GROUP" and node.node_tree == image_layer) layer = next(node for node in result.node_tree.nodes if node.type == "GROUP" and node.node_tree == image_layer)

30
tests/anyimage/common/test_material_settings.py
View File

@@ -29,36 +29,6 @@ class MaterialSettingsTest(unittest.TestCase):
) )
def test_material_color_space_falls_back_when_unsupported(self):
class ColorSpaceSettings:
def __init__(self):
self._name = "sRGB"
@property
def name(self):
return self._name
@name.setter
def name(self, value):
if value != "sRGB":
raise TypeError("unsupported color space")
self._name = value
image = SimpleNamespace(
colorspace_settings=ColorSpaceSettings(),
users=0, use_fake_user=False, alpha_mode="STRAIGHT",
)
scene = SimpleNamespace(
view_settings=SimpleNamespace(view_transform="ACES 2.0")
)
result = self.modules.material.configure_material_color_image(image, scene)
self.assertIs(result, image)
self.assertEqual(image.alpha_mode, "PREMUL")
self.assertEqual(image.colorspace_settings.name, "sRGB")
def test_surface_displacement_is_set_to_displacement_only(self): def test_surface_displacement_is_set_to_displacement_only(self):
class Material: class Material:
surface_displacement_method = "BUMP" surface_displacement_method = "BUMP"

56
tests/anyimage/common/test_material_view_adaptation.py
View File

@@ -35,18 +35,6 @@ def color_texture(result):
return next(node.image for node in result.node_tree.nodes if node.type == "TEX_IMAGE") return next(node.image for node in result.node_tree.nodes if node.type == "TEX_IMAGE")
def test_adaptation_preference_defaults_and_reads_current_value():
with patch.object(preferences, "addon_preferences", return_value=None):
assert preferences.configured_material_view_adaptation()
with patch.object(preferences, "addon_preferences", return_value=SimpleNamespace()):
assert preferences.configured_material_view_adaptation()
settings = SimpleNamespace(adapt_material_to_view_transform=False)
with patch.object(preferences, "addon_preferences", return_value=settings):
assert not preferences.configured_material_view_adaptation()
settings.adapt_material_to_view_transform = True
assert preferences.configured_material_view_adaptation()
@pytest.mark.parametrize("enabled", [False, True]) @pytest.mark.parametrize("enabled", [False, True])
@pytest.mark.parametrize("view,space", [*material.VIEW_TRANSFORM_COLOR_SPACES, ("Standard", "sRGB"), ("Unknown", "sRGB")]) @pytest.mark.parametrize("view,space", [*material.VIEW_TRANSFORM_COLOR_SPACES, ("Standard", "sRGB"), ("Unknown", "sRGB")])
def test_color_space_and_alpha_follow_preference(color_data, enabled, view, space): def test_color_space_and_alpha_follow_preference(color_data, enabled, view, space):
@@ -85,11 +73,11 @@ def test_material_pixels_survive_pack_and_library_reload(color_data, tmp_path, e
source.pixels.foreach_set((expected.astype(np.float32) / 255).ravel()) source.pixels.foreach_set((expected.astype(np.float32) / 255).ravel())
else: else:
source = bpy.data.images.new("Material pixels", width=2, height=1, alpha=True, float_buffer=kind == "float") source = bpy.data.images.new("Material pixels", width=2, height=1, alpha=True, float_buffer=kind == "float")
source.alpha_mode = "PREMUL"
source.pixels.foreach_set((expected.astype(np.float32) / 255).ravel()) source.pixels.foreach_set((expected.astype(np.float32) / 255).ravel())
if kind == "packed": if kind == "packed":
source.pack() source.pack()
source.reload() source.reload()
images.configure_color_alpha(source)
before = images.image_pixels(source) before = images.image_pixels(source)
owner = bpy.data.objects.new("Color source owner", None) owner = bpy.data.objects.new("Color source owner", None)
owner.empty_display_type = "IMAGE" owner.empty_display_type = "IMAGE"
@@ -175,51 +163,13 @@ def test_material_configures_the_image_selected_by_its_caller(color_data, entry,
def test_edit_results_preserve_source_alpha_with_adaptation_enabled(color_data, tmp_path): def test_edit_results_preserve_source_alpha_with_adaptation_enabled(color_data, tmp_path):
source = bpy.data.images.new("Editing source", width=1, height=1, alpha=True) source = bpy.data.images.new("Editing source", width=1, height=1, alpha=True)
rgba = np.array([0.2, 0.4, 0.8, 0.25], dtype=np.float32) rgba = np.array([0.2, 0.4, 0.8, 0.25], dtype=np.float32)
Image.fromarray(np.array([[[51, 102, 204, 64]]], dtype=np.uint8)).save(tmp_path / "image.0001.png") Image.fromarray(np.array([[[51, 102, 204, 64]]], dtype=np.uint8)).save(tmp_path / "foreground.png")
with patch.object(preferences, "addon_preferences", return_value=SimpleNamespace(adapt_material_to_view_transform=True)): with patch.object(preferences, "addon_preferences", return_value=SimpleNamespace(adapt_material_to_view_transform=True)):
edited = images.create_image_edit_result(source, rgba, (1, 1)) edited = images.create_image_edit_result(source, rgba, (1, 1))
loaded = images.load_image_edit_result(source, tmp_path) loaded = images.load_image_edit_result(source, tmp_path / "foreground.png")
for result in (edited, loaded): for result in (edited, loaded):
assert result.colorspace_settings.name == "sRGB" assert result.colorspace_settings.name == "sRGB"
assert result.alpha_mode == source.alpha_mode assert result.alpha_mode == source.alpha_mode
np.testing.assert_allclose(images.image_pixels(result), rgba, atol=1 / 255) np.testing.assert_allclose(images.image_pixels(result), rgba, atol=1 / 255)
assert source.colorspace_settings.name == "sRGB" assert source.colorspace_settings.name == "sRGB"
assert source.alpha_mode == "STRAIGHT" assert source.alpha_mode == "STRAIGHT"
@pytest.mark.parametrize("supported", [{"sRGB", "Non-Color"}, {"Non-Color"}])
def test_unavailable_color_space_falls_back_or_preserves_original(supported):
class ColorSpace:
value = "Non-Color"
@property
def name(self):
return self.value
@name.setter
def name(self, value):
if value not in supported:
raise TypeError("Unsupported color space")
self.value = value
image = SimpleNamespace(colorspace_settings=ColorSpace(), users=0, use_fake_user=False, alpha_mode="STRAIGHT")
with patch.object(material, "configured_material_view_adaptation", return_value=True):
if "sRGB" in supported:
result = material.configure_material_color_image(image, view_scene("AgX"))
assert result.colorspace_settings.name == "sRGB"
assert result.alpha_mode == "PREMUL"
else:
with pytest.raises(RuntimeError, match="material color space"):
material.configure_material_color_image(image, view_scene("AgX"))
assert image.colorspace_settings.name == "Non-Color"
assert image.alpha_mode == "STRAIGHT"
def test_missing_scene_uses_srgb(color_data):
image = bpy.data.images.new("Missing scene", width=1, height=1)
with patch.object(material, "bpy", SimpleNamespace(context=SimpleNamespace(scene=None))), patch.object(
material, "configured_material_view_adaptation", return_value=True
):
result = material.configure_material_color_image(image)
assert result.colorspace_settings.name == "sRGB"
assert result.alpha_mode == "PREMUL"

18
tests/anyimage/common/test_modifier_inputs.py
View File

@@ -9,8 +9,8 @@ class ModifierInputsTest(unittest.TestCase):
def test_runtime_modifier_enum_input_uses_string_identifier(self): def test_runtime_modifier_enum_input_uses_string_identifier(self):
enum_items = ( enum_items = (
SimpleNamespace(identifier="SURFACE"), SimpleNamespace(identifier="Balloon"),
SimpleNamespace(identifier="BALLOON"), SimpleNamespace(identifier="Shell"),
) )
value_property = SimpleNamespace(type="ENUM", enum_items=enum_items) value_property = SimpleNamespace(type="ENUM", enum_items=enum_items)
input_slot = SimpleNamespace( input_slot = SimpleNamespace(
@@ -19,13 +19,13 @@ class ModifierInputsTest(unittest.TestCase):
) )
modifier = SimpleNamespace( modifier = SimpleNamespace(
properties=SimpleNamespace( properties=SimpleNamespace(
inputs=SimpleNamespace(Shape=input_slot), inputs=SimpleNamespace(Mode=input_slot),
) )
) )
self.modules.object.set_modifier_input(modifier, "Shape", 1) self.modules.object.set_modifier_input(modifier, "Mode", 1)
self.assertEqual(input_slot.value, "BALLOON") self.assertEqual(input_slot.value, "Shell")
def test_modifier_inputs_support_rna_and_dictionary_values(self): def test_modifier_inputs_support_rna_and_dictionary_values(self):
@@ -44,9 +44,9 @@ class ModifierInputsTest(unittest.TestCase):
self.assertEqual(input_slot.value, 2.0) self.assertEqual(input_slot.value, 2.0)
class LegacyModifier(dict): class DictionaryModifier(dict):
properties = None properties = None
legacy_modifier = LegacyModifier() dictionary_modifier = DictionaryModifier()
self.modules.object.set_modifier_input(legacy_modifier, "Shape", 1) self.modules.object.set_modifier_input(dictionary_modifier, "Mode", 1)
self.assertEqual(legacy_modifier["Shape"], 1) self.assertEqual(dictionary_modifier["Mode"], 1)

14
tests/anyimage/common/test_node_loading.py
View File

@@ -1,14 +0,0 @@
from tests.support.paths import PROJECT_ROOT
import unittest
from tests.anyimage.common.support import load_common_modules
class NodeLoadingTest(unittest.TestCase):
@classmethod
def setUpClass(cls):
cls.modules = load_common_modules()
def test_node_asset_path_resolves_bundled_library(self):
self.assertEqual(
self.modules.node.node_asset_path(),
PROJECT_ROOT / "src" / "anyimage" / "assets" / "O_AnyImage.blend",
)

15
tests/anyimage/operators/test_ai_image_operators.py
View File

@@ -21,11 +21,11 @@ class AiImageOperatorsTest(BlenderTestCase):
self.assertEqual(operator_type.bl_options, {"UNDO"}) self.assertEqual(operator_type.bl_options, {"UNDO"})
operator = operator_type() operator = operator_type()
operator.input_path = "input.png" operator.input_path = "input.png"
operator._image_target = SimpleNamespace(start_frame=4) operator._image_target = SimpleNamespace()
operator.model = "HAT_GAN_X4_SHARPER" operator.model = "HAT_GAN_X4_SHARPER"
operator.max_input_size = 1024 operator.max_input_size = 1024
context = SimpleNamespace( context = SimpleNamespace(
scene=SimpleNamespace(anyimage_settings=SimpleNamespace(video_frames=7)) scene=SimpleNamespace(anyimage_settings=SimpleNamespace())
) )
with ( with (
patch.object(self.upscale, "require_environment"), patch.object(self.upscale, "require_environment"),
@@ -44,8 +44,6 @@ class AiImageOperatorsTest(BlenderTestCase):
"input": "input.png", "input": "input.png",
"model": "HAT_GAN_X4_SHARPER", "model": "HAT_GAN_X4_SHARPER",
"device": "cuda", "device": "cuda",
"video_frames": 7,
"start_frame": 4,
"max_input_size": 1024, "max_input_size": 1024,
}, },
) )
@@ -89,7 +87,7 @@ class AiImageOperatorsTest(BlenderTestCase):
def test_remove_background_loads_a_plain_image_edit_result(self): def test_remove_background_loads_a_plain_image_edit_result(self):
result = SimpleNamespace(directory=Path("result")) result = SimpleNamespace(file=lambda key: Path("result") / f"{key}.png")
operator = self.remove_background.RunBackgroundRemoval() operator = self.remove_background.RunBackgroundRemoval()
operator._image_target = Mock() operator._image_target = Mock()
ops = SimpleNamespace( ops = SimpleNamespace(
@@ -100,12 +98,12 @@ class AiImageOperatorsTest(BlenderTestCase):
): ):
operator.response(None, result) operator.response(None, result)
operator._image_target.apply.assert_called_once_with(Path("result")) operator._image_target.apply.assert_called_once_with(Path("result/foreground.png"))
ops.ed.undo_push.assert_called_once_with(message="Remove Background") ops.ed.undo_push.assert_called_once_with(message="Remove Background")
def test_upscale_loads_a_plain_image_edit_result(self): def test_upscale_loads_a_plain_image_edit_result(self):
result = SimpleNamespace(directory=Path("result")) result = SimpleNamespace(file=lambda key: Path("result") / f"{key}.png")
operator = self.upscale.UpscaleImage() operator = self.upscale.UpscaleImage()
operator._image_target = Mock() operator._image_target = Mock()
operator.model = "HAT_GAN_X4_SHARPER" operator.model = "HAT_GAN_X4_SHARPER"
@@ -119,7 +117,6 @@ class AiImageOperatorsTest(BlenderTestCase):
message = operator.response(None, result) message = operator.response(None, result)
operator._image_target.apply.assert_called_once_with( operator._image_target.apply.assert_called_once_with(
Path("result"), Path("result/upscale.png"),
frame_prefix="upscale.",
) )
self.assertEqual(message, "Image is upscaled with HAT Sharper") self.assertEqual(message, "Image is upscaled with HAT Sharper")

4
tests/anyimage/operators/test_ai_input_failures.py
View File

@@ -19,10 +19,10 @@ class AiInputFailuresTest(BlenderTestCase):
): ):
operator = operator_type() operator = operator_type()
operator.report = Mock() operator.report = Mock()
target = Mock(image=SimpleNamespace(size=(16, 16)), start_frame=1) target = Mock(image=SimpleNamespace(size=(16, 16)))
target.prepare.return_value = (path, False) target.prepare.return_value = (path, False)
context = SimpleNamespace(scene=SimpleNamespace( context = SimpleNamespace(scene=SimpleNamespace(
anyimage_settings=SimpleNamespace(video_frames=1))) anyimage_settings=SimpleNamespace()))
with ExitStack() as stack: with ExitStack() as stack:
stack.enter_context(patch.object(module.ImageEditTarget, "capture", return_value=target)) stack.enter_context(patch.object(module.ImageEditTarget, "capture", return_value=target))
stack.enter_context(patch.object(module, "require_environment")) stack.enter_context(patch.object(module, "require_environment"))

34
tests/anyimage/operators/test_ai_setup.py
View File

@@ -2,11 +2,43 @@ import importlib
import tempfile import tempfile
from pathlib import Path from pathlib import Path
from types import SimpleNamespace from types import SimpleNamespace
from unittest.mock import patch from unittest.mock import Mock, patch
from tests.support.blender import BlenderTestCase from tests.support.blender import BlenderTestCase
class AiSetupTest(BlenderTestCase): class AiSetupTest(BlenderTestCase):
def test_setup_dialog_uses_install_title_and_button(self):
show_dialog = Mock(return_value={"RUNNING_MODAL"})
context = SimpleNamespace(
window_manager=SimpleNamespace(invoke_props_dialog=show_dialog)
)
operator = self.ai_setup.SetupAIEnvironment()
self.assertEqual(operator.invoke(context, None), {"RUNNING_MODAL"})
show_dialog.assert_called_once_with(
operator, width=440, title="Set Up AI", confirm_text="Install"
)
def test_setup_dialog_shows_install_summary_and_mirror_option(self):
labels = []
properties = []
operator = self.ai_setup.SetupAIEnvironment()
operator.layout = SimpleNamespace(
label=lambda **kwargs: labels.append(kwargs["text"]),
separator=lambda **_kwargs: None,
prop=lambda _owner, name: properties.append(name),
)
with patch.object(
self.ai_setup, "ai_status", return_value={"environment_ready": False}
):
operator.draw(None)
self.assertEqual(labels, [
"Set up AI features for depth, background removal and upscaling.",
"Downloads the runtime and required models (259 MB for models).",
])
self.assertEqual(properties, ["use_mirror"])
def test_runtime_exposes_declared_environment_and_post_install(self): def test_runtime_exposes_declared_environment_and_post_install(self):
runtime = self.anyimage.runtime runtime = self.anyimage.runtime
self.assertEqual(runtime.environment["python"], "3.12") self.assertEqual(runtime.environment["python"], "3.12")

8
tests/anyimage/operators/test_balloon_profiles.py
View File

@@ -2,8 +2,8 @@ from unittest.mock import patch
import numpy as np import numpy as np
from anyimage.operators.cutout_tool import geometry as cutout_geometry from anyimage.operators.cutout_tool import balloon
from anyimage.operators.cutout_tool.geometry import poisson_balloon_profile from anyimage.operators.cutout_tool.balloon import poisson_balloon_profile
def test_poisson_balloon_profile_is_zero_on_boundary_and_positive_inside(): def test_poisson_balloon_profile_is_zero_on_boundary_and_positive_inside():
@@ -62,11 +62,11 @@ def test_teddy_balloon_profile_uses_chordal_axis_radius():
nodes = np.asarray(((1, 1), (3, 1)), dtype=float) nodes = np.asarray(((1, 1), (3, 1)), dtype=float)
with patch.object( with patch.object(
cutout_geometry, balloon,
"chordal_axis", "chordal_axis",
return_value=(nodes, [(0, 1)], np.asarray((1.0, 1.0))), return_value=(nodes, [(0, 1)], np.asarray((1.0, 1.0))),
): ):
profile = cutout_geometry.teddy_balloon_heights( profile = balloon.teddy_balloon_heights(
points, points,
faces, faces,
points[:4], points[:4],

4
tests/anyimage/operators/test_clipboard_cache.py
View File

@@ -23,7 +23,7 @@ def clipboard():
bpy.ops.wm.read_factory_settings(use_empty=True) bpy.ops.wm.read_factory_settings(use_empty=True)
@pytest.mark.parametrize("edit", ["paint", "packed", "hidden_rgb", "size", "space", "alpha", "legacy", "unreadable"]) @pytest.mark.parametrize("edit", ["paint", "packed", "hidden_rgb", "size", "space", "alpha", "unreadable"])
def test_changed_cache_is_skipped_without_modifying_candidate(clipboard, edit, monkeypatch): def test_changed_cache_is_skipped_without_modifying_candidate(clipboard, edit, monkeypatch):
original, reused = actions.acquire_packed_image(clipboard, ".png") original, reused = actions.acquire_packed_image(clipboard, ".png")
expected = images.image_pixels(original) expected = images.image_pixels(original)
@@ -41,8 +41,6 @@ def test_changed_cache_is_skipped_without_modifying_candidate(clipboard, edit, m
original.colorspace_settings.name = "Non-Color" original.colorspace_settings.name = "Non-Color"
elif edit == "alpha": elif edit == "alpha":
original.alpha_mode = "PREMUL" original.alpha_mode = "PREMUL"
elif edit == "legacy":
del original[actions.CLIPBOARD_CONTENT_PROPERTY]
else: else:
read = actions._clipboard_content_hash read = actions._clipboard_content_hash
def fail_candidate(image): def fail_candidate(image):

5
tests/anyimage/operators/test_clipboard_image.py
View File

@@ -84,7 +84,6 @@ class ClipboardImageTest(unittest.TestCase):
) )
for name in clipboard_image.PasteClipboardImage.__annotations__: for name in clipboard_image.PasteClipboardImage.__annotations__:
self.assertFalse(name.startswith("anyimage_"), name) self.assertFalse(name.startswith("anyimage_"), name)
self.assertFalse(name.startswith("o_"), name)
def test_registers_and_removes_clipboard_shortcuts(self): def test_registers_and_removes_clipboard_shortcuts(self):
class KeyMapItems: class KeyMapItems:
@@ -345,10 +344,6 @@ class ClipboardImageTest(unittest.TestCase):
) )
self.assertFalse(options["default"]) self.assertFalse(options["default"])
annotations = self.addon.clipboard_image.PasteClipboardImage.__annotations__
self.assertNotIn("subdivisions", annotations)
self.assertNotIn("thickness", annotations)
self.assertNotIn("unshaded", annotations)
def test_native_copy_takes_priority_while_clipboard_is_unchanged(self): def test_native_copy_takes_priority_while_clipboard_is_unchanged(self):
clipboard_image = self.addon.clipboard_image clipboard_image = self.addon.clipboard_image

41
tests/anyimage/common/test_depth_calibration.py → tests/anyimage/operators/test_cutout_calibration.py
View File

@@ -1,34 +1,11 @@
import json
from types import SimpleNamespace from types import SimpleNamespace
import numpy as np import numpy as np
from anyimage.common.selection import SelectionMask from anyimage.common.selection import SelectionMask
from anyimage.common.depth import ( from anyimage.common.depth import depth_uniform_scale
depth_uniform_scale,
load_depth_metadata,
)
from anyimage.operators.cutout_tool import geometry as cutout_geometry from anyimage.operators.cutout_tool import geometry as cutout_geometry
from anyimage.operators.cutout_tool import depth_calibration
def test_depth_metadata_loader_uses_one_schema_for_plane_and_cutout(tmp_path):
path = tmp_path / "depth.json"
path.write_text(
json.dumps(
{
"image_size": [3, 2],
"intrinsics": np.eye(3).tolist(),
"reference_depth": 1.0,
}
),
encoding="utf-8",
)
metadata = load_depth_metadata(path)
assert metadata["image_size"] == (3, 2)
assert metadata["intrinsics"] == ((1.0, 0.0, 0.0), (0.0, 1.0, 0.0), (0.0, 0.0, 1.0))
assert metadata["reference_depth"] == 1.0
def test_base_plane_fit_is_stable_for_a_slender_diagonal_domain(): def test_base_plane_fit_is_stable_for_a_slender_diagonal_domain():
@@ -42,7 +19,7 @@ def test_base_plane_fit_is_stable_for_a_slender_diagonal_domain():
) )
values = 0.4 * along + np.sign(across) * 0.05 values = 0.4 * along + np.sign(across) * 0.05
plane = cutout_geometry._stable_plane(coordinates, values) plane = depth_calibration._stable_plane(coordinates, values)
selected_plane = coordinates @ plane[:2] + plane[2] selected_plane = coordinates @ plane[:2] + plane[2]
corners = np.asarray(((-1.0, -1.0), (-1.0, 1.0), (1.0, -1.0), (1.0, 1.0))) corners = np.asarray(((-1.0, -1.0), (-1.0, 1.0), (1.0, -1.0), (1.0, 1.0)))
corner_plane = corners @ plane[:2] + plane[2] corner_plane = corners @ plane[:2] + plane[2]
@@ -63,7 +40,7 @@ def test_depth_balloon_plane_calibration_matches_cutout_axes():
uv.append((column * 0.5, 1.0 - row * 0.5)) uv.append((column * 0.5, 1.0 - row * 0.5))
camera_depth[row, column] = 2.0 + 0.2 * local_x + 0.1 * local_y camera_depth[row, column] = 2.0 + 0.2 * local_x + 0.1 * local_y
reference, direction = cutout_geometry.depth_balloon_projection_parameters( reference, direction = depth_calibration.depth_balloon_projection_parameters(
vertices, vertices,
uv, uv,
camera_depth, camera_depth,
@@ -93,7 +70,7 @@ def test_depth_balloon_reference_uses_the_shared_upper_depth_percentile():
vertices.append((float(x), float(size - 1 - y), 0.0)) vertices.append((float(x), float(size - 1 - y), 0.0))
uv.append((x / (size - 1), 1.0 - y / (size - 1))) uv.append((x / (size - 1), 1.0 - y / (size - 1)))
reference, direction = cutout_geometry.depth_balloon_projection_parameters( reference, direction = depth_calibration.depth_balloon_projection_parameters(
vertices, vertices,
uv, uv,
camera_depth, camera_depth,
@@ -126,7 +103,7 @@ def test_depth_surface_projection_uses_one_scale_for_all_camera_axes():
) )
assert uniform_scale == 2.0 assert uniform_scale == 2.0
reference_depth = cutout_geometry.metric_depth_reference( reference_depth = depth_calibration.metric_depth_reference(
points[..., 2], points[..., 2],
np.ones((51, 101), dtype=bool), np.ones((51, 101), dtype=bool),
uniform_scale, uniform_scale,
@@ -157,8 +134,8 @@ def test_cutout_threshold_controls_contour_and_calibration():
def test_cutout_depth_calibration_uses_five_percent_transparency_threshold(): def test_cutout_depth_calibration_uses_five_percent_transparency_threshold():
assert not cutout_geometry.use_adaptive_depth_calibration(0.0499) assert not depth_calibration.use_adaptive_depth_calibration(0.0499)
assert cutout_geometry.use_adaptive_depth_calibration(0.05) assert depth_calibration.use_adaptive_depth_calibration(0.05)
def test_selection_calibration_uses_the_final_selection_mask(): def test_selection_calibration_uses_the_final_selection_mask():

9
tests/anyimage/operators/test_cutout_depth.py
View File

@@ -3,7 +3,8 @@ import numpy as np
import pytest import pytest
from anyimage.operators.cutout_tool.object import load_cutout_node_group from anyimage.operators.cutout_tool.object import load_cutout_node_group
from anyimage.operators.cutout_tool.operators import normal_mode_for_shape from anyimage.operators.cutout_tool.operators import normal_mode_for_shape
from anyimage.operators.cutout_tool.geometry import refine_balloon_triangles, poisson_balloon_profile from anyimage.operators.cutout_tool.geometry import refine_balloon_triangles
from anyimage.operators.cutout_tool.balloon import poisson_balloon_profile
@pytest.mark.parametrize("enabled", [False, True]) @pytest.mark.parametrize("enabled", [False, True])
@@ -20,18 +21,18 @@ def test_new_cutout_does_not_reuse_a_saved_wrapper():
saved.interface.new_socket(name="Shape", in_out="INPUT", socket_type="NodeSocketMenu") saved.interface.new_socket(name="Shape", in_out="INPUT", socket_type="NodeSocketMenu")
saved.use_fake_user = True saved.use_fake_user = True
current = load_cutout_node_group("SOLID") current = load_cutout_node_group("SOLID")
assert current != saved and current.get("anyimage_asset_revision") == 11 assert current != saved and current.get("anyimage_asset_revision") == 13
assert saved in bpy.data.node_groups.values() assert saved in bpy.data.node_groups.values()
assert current == load_cutout_node_group("FLAT") assert current == load_cutout_node_group("FLAT")
assert any(item.name == "Shape" for item in saved.interface.items_tree) assert any(item.name == "Shape" for item in saved.interface.items_tree)
def test_depth_cutout_loads_boundary_blur_revision_and_preserves_existing_group(): def test_depth_cutout_loads_uniform_thickness_revision_and_preserves_existing_group():
bpy.ops.wm.read_factory_settings(use_empty=True) bpy.ops.wm.read_factory_settings(use_empty=True)
saved = bpy.data.node_groups.new("O Image Depth Cutout", "GeometryNodeTree") saved = bpy.data.node_groups.new("O Image Depth Cutout", "GeometryNodeTree")
saved["anyimage_asset_revision"] = 11 saved["anyimage_asset_revision"] = 11
current = load_cutout_node_group("DEPTH_SOLID") current = load_cutout_node_group("DEPTH_SOLID")
assert current != saved and current["anyimage_asset_revision"] == 23 assert current != saved and current["anyimage_asset_revision"] == 26
assert saved in bpy.data.node_groups.values() assert saved in bpy.data.node_groups.values()
assert saved["anyimage_asset_revision"] == 11 assert saved["anyimage_asset_revision"] == 11
assert load_cutout_node_group("DEPTH_SOLID") == current assert load_cutout_node_group("DEPTH_SOLID") == current

3
tests/anyimage/operators/test_cutout_fine_mesh.py
View File

@@ -5,6 +5,7 @@ from scipy.sparse.csgraph import connected_components
from scipy.spatial import cKDTree from scipy.spatial import cKDTree
from anyimage.operators.cutout_tool import geometry as g from anyimage.operators.cutout_tool import geometry as g
from anyimage.operators.cutout_tool.balloon import poisson_balloon_profile
from anyimage.operators.cutout_tool import fine_mesh as fine from anyimage.operators.cutout_tool import fine_mesh as fine
@@ -42,7 +43,7 @@ def assert_supported(points, faces, boundary, components):
).tocsr() ).tocsr()
count, _labels = connected_components(graph[interior][:, interior], directed=False) count, _labels = connected_components(graph[interior][:, interior], directed=False)
assert count == len(components) assert count == len(components)
profile = g.poisson_balloon_profile(points, faces, boundary) profile = poisson_balloon_profile(points, faces, boundary)
assert np.isfinite(profile).all() assert np.isfinite(profile).all()
assert np.all(profile[interior] > 0) assert np.all(profile[interior] > 0)

2
tests/anyimage/operators/test_cutout_geometry.py
View File

@@ -47,7 +47,7 @@ def test_cutout_node_groups_are_loaded_from_the_asset():
for shape in cutout_shape.CUTOUT_NODE_GROUP_NAMES: for shape in cutout_shape.CUTOUT_NODE_GROUP_NAMES:
assert cutout_object.load_cutout_node_group(shape) is loaded assert cutout_object.load_cutout_node_group(shape) is loaded
load.assert_called_with(cutout_shape.CUTOUT_NODE_GROUP_NAMES[shape], load.assert_called_with(cutout_shape.CUTOUT_NODE_GROUP_NAMES[shape],
asset_revision=23 if shape == "DEPTH_SOLID" else 12 if shape == "DEPTH_BALLOON" else 11) asset_revision=26 if shape == "DEPTH_SOLID" else 14 if shape == "DEPTH_BALLOON" else 13)
assert load.call_count == 4 assert load.call_count == 4

120
tests/anyimage/operators/test_cutout_interaction.py
View File

@@ -1,5 +1,9 @@
import json import json
from types import SimpleNamespace from types import SimpleNamespace
from types import MethodType
from unittest.mock import Mock
import pytest
from anyimage.common.selection import SelectionPath from anyimage.common.selection import SelectionPath
@@ -21,8 +25,10 @@ class Pie:
class WindowManager: class WindowManager:
def __init__(self): def __init__(self):
self.pie = Pie() self.pie = Pie()
self.events = []
def popup_menu_pie(self, _event, draw, **_options): def popup_menu_pie(self, _event, draw, **_options):
self.events.append(_event)
menu = SimpleNamespace(layout=SimpleNamespace(menu_pie=lambda: self.pie)) menu = SimpleNamespace(layout=SimpleNamespace(menu_pie=lambda: self.pie))
draw(menu, None) draw(menu, None)
@@ -31,14 +37,12 @@ def cutout_context(
window_manager, window_manager,
mesh_detail="LOW", mesh_detail="LOW",
generate_normal=False, generate_normal=False,
inward_axis="NEGATIVE_Z",
alpha_threshold=0.9, alpha_threshold=0.9,
fine_outline=True, fine_outline=True,
): ):
settings = SimpleNamespace( settings = SimpleNamespace(
cutout_mesh_detail=mesh_detail, cutout_mesh_detail=mesh_detail,
cutout_generate_normal=generate_normal, cutout_generate_normal=generate_normal,
cutout_inward_axis=inward_axis,
cutout_alpha_threshold=alpha_threshold, cutout_alpha_threshold=alpha_threshold,
cutout_fine_outline=fine_outline, cutout_fine_outline=fine_outline,
) )
@@ -94,7 +98,6 @@ def test_cutout_pie_uses_two_local_shapes_without_ai(monkeypatch):
) )
assert not any(call[2].generate_normal for call in window_manager.pie.calls) assert not any(call[2].generate_normal for call in window_manager.pie.calls)
assert all(call[2].point_spacing == 32 for call in window_manager.pie.calls) assert all(call[2].point_spacing == 32 for call in window_manager.pie.calls)
assert all(call[2].inward_axis == "NEGATIVE_Z" for call in window_manager.pie.calls)
def test_cutout_pie_passes_image_lasso_options_to_all_ai_shapes(monkeypatch): def test_cutout_pie_passes_image_lasso_options_to_all_ai_shapes(monkeypatch):
@@ -103,7 +106,6 @@ def test_cutout_pie_passes_image_lasso_options_to_all_ai_shapes(monkeypatch):
window_manager, window_manager,
mesh_detail="MEDIUM", mesh_detail="MEDIUM",
generate_normal=True, generate_normal=True,
inward_axis="POSITIVE_X",
alpha_threshold=0.1, alpha_threshold=0.1,
fine_outline=False, fine_outline=False,
) )
@@ -149,7 +151,6 @@ def test_cutout_pie_passes_image_lasso_options_to_all_ai_shapes(monkeypatch):
for call in window_manager.pie.calls for call in window_manager.pie.calls
) )
assert all(call[2].point_spacing == 16 for call in window_manager.pie.calls) assert all(call[2].point_spacing == 16 for call in window_manager.pie.calls)
assert all(call[2].inward_axis == "POSITIVE_X" for call in window_manager.pie.calls)
def test_cutout_normal_space_is_derived_from_shape(): def test_cutout_normal_space_is_derived_from_shape():
@@ -175,20 +176,19 @@ def test_cutout_lasso_submission_passes_only_the_selected_path(monkeypatch):
monkeypatch.setattr(cutout_main, "require_image_empty", lambda _name: source) monkeypatch.setattr(cutout_main, "require_image_empty", lambda _name: source)
monkeypatch.setattr(interaction, "ai_ready", lambda: False) monkeypatch.setattr(interaction, "ai_ready", lambda: False)
path = SelectionPath(points=((0, 0), (10, 0), (0, 10))) path = SelectionPath(points=((0, 0), (10, 0), (0, 10)))
operator = SimpleNamespace(source_object_name=source.name, _pie_event=object()) operator = SimpleNamespace(source_object_name=source.name, _pie_event=object(), gesture="LASSO")
result = cutout_main.SelectCutoutSelection._submit_selection_path(operator, context, path) result = cutout_main.SelectCutoutSelection._submit_selection_path(operator, context, path)
assert result == {"FINISHED"} assert result == {"FINISHED"}
assert SelectionPath.from_json(window_manager.pie.calls[0][2].selection_path_json) == path assert SelectionPath.from_json(window_manager.pie.calls[0][2].selection_path_json) == path
def test_cutout_tool_only_binds_lasso_drag_and_native_selection(): def test_cutout_tool_starts_on_press_and_keeps_shift_selection():
keymap = cutout_main.CutoutTool.bl_keymap keymap = cutout_main.CutoutTool.bl_keymap
assert keymap[0][1]["value"] == "CLICK_DRAG" assert keymap[0][1]["value"] == "PRESS"
assert [entry[0] for entry in keymap[1:]] == ["view3d.select", "view3d.select"] assert len(keymap) == 2
assert all(entry[1]["value"] != "DOUBLE_CLICK" for entry in keymap) assert keymap[1][0] == "view3d.select"
assert "full_image" not in cutout_main.SelectCutoutSelection.__annotations__ assert keymap[1][2] == {"properties": [("toggle", True)]}
assert keymap[1][2] == {"properties": [("deselect_all", False)]} assert cutout_main.SelectCutoutSelection.resolve_on_press
assert keymap[2][2] == {"properties": [("toggle", True)]}
def test_cutout_f_has_no_option_shortcut(monkeypatch): def test_cutout_f_has_no_option_shortcut(monkeypatch):
@@ -209,7 +209,7 @@ def test_cutout_f_has_no_option_shortcut(monkeypatch):
def test_cutout_status_has_no_letter_shortcut(): def test_cutout_status_has_no_letter_shortcut():
status = [] status = []
operator = SimpleNamespace() operator = SimpleNamespace(gesture="LASSO")
context = SimpleNamespace( context = SimpleNamespace(
workspace=SimpleNamespace(status_text_set=status.append) workspace=SimpleNamespace(status_text_set=status.append)
) )
@@ -220,3 +220,95 @@ def test_cutout_status_has_no_letter_shortcut():
"Cutout Lasso: drag and release for shapes; RMB or Esc cancels" "Cutout Lasso: drag and release for shapes; RMB or Esc cancels"
] ]
assert "F" not in status[0] assert "F" not in status[0]
def test_cutout_menu_keeps_submitted_gesture_during_redraw(monkeypatch):
window_manager = WindowManager()
context = cutout_context(window_manager)
monkeypatch.setattr(interaction, "ai_ready", lambda: True)
def delayed_popup(_event, draw, **_options):
context.scene.anyimage_settings.cutout_gesture = "LASSO"
draw(SimpleNamespace(layout=SimpleNamespace(menu_pie=lambda: window_manager.pie)), None)
window_manager.popup_menu_pie = delayed_popup
interaction.open_shape_pie(
context, object(), cutout_source(),
SelectionPath(points=((0, 0), (10, 0), (0, 10))), gesture="POLYLINE",
)
assert all(call[2].gesture == "POLYLINE" for call in window_manager.pie.calls)
@pytest.mark.parametrize("finish", ("START", "DOUBLE_CLICK", "RET", "NUMPAD_ENTER"))
def test_cutout_polyline_edits_and_submits_through_shared_gesture(monkeypatch, finish):
window_manager = WindowManager()
context = cutout_context(window_manager)
context.area = SimpleNamespace(tag_redraw=Mock())
context.region = context.region_data = object()
context.scene.anyimage_settings.cutout_gesture = "POLYLINE"
source = cutout_source()
monkeypatch.setattr(viewport, "active_view3d_tool_id", lambda _context: None)
monkeypatch.setattr(viewport, "require_image_empty", lambda _name: source)
monkeypatch.setattr(cutout_main, "require_image_empty", lambda _name: source)
monkeypatch.setattr(viewport, "deserialize_matrix", lambda _data: None)
monkeypatch.setattr(viewport, "image_empty_bounds", lambda _source: None)
monkeypatch.setattr(viewport, "screen_path_to_image_pixels", lambda path, *_args: path)
monkeypatch.setattr(interaction, "ai_ready", lambda: False)
operator = SimpleNamespace(
source_object_name=source.name, source_matrix_data="matrix",
active_tool_id="anyimage.cutout_lasso", _path=[(0, 0)],
_polyline_confirm_type=None,
_finish=Mock(),
)
for name in ("_modal_polyline", "_confirm_polyline", "_complete", "_submit_selection_path"):
setattr(operator, name, MethodType(getattr(cutout_main.SelectCutoutSelection, name), operator))
cutout_main.SelectCutoutSelection._read_gesture_settings(operator, context)
def event(kind, value="PRESS", point=(100, 100)):
return cutout_main.SelectCutoutSelection.modal(operator, context, SimpleNamespace(
type=kind, value=value, mouse_region_x=point[0], mouse_region_y=point[1],
))
assert event("RET") == {"RUNNING_MODAL"}
event("LEFTMOUSE", point=(100, 0))
event("LEFTMOUSE", point=(100, 100))
event("BACK_SPACE")
assert operator._path == [(0, 0), (100, 0)]
event("LEFTMOUSE", point=(100, 100))
if finish == "START":
result = event("LEFTMOUSE", point=(0, 0))
elif finish == "DOUBLE_CLICK":
result = event("LEFTMOUSE", "DOUBLE_CLICK")
else:
result = event(finish)
assert result == {"RUNNING_MODAL"}
assert window_manager.events == []
operator._finish.assert_not_called()
points = tuple(operator._path)
assert event("MOUSEMOVE", point=(200, 200)) == {"RUNNING_MODAL"}
assert event("BACK_SPACE") == {"RUNNING_MODAL"}
assert tuple(operator._path) == points
release_type = "LEFTMOUSE" if finish in {"START", "DOUBLE_CLICK"} else finish
assert event("SPACE", "RELEASE") == {"RUNNING_MODAL"}
result = event(release_type, "RELEASE")
assert result == {"FINISHED"}
assert len(window_manager.events) == 1
assert window_manager.events[0].value == "RELEASE"
operator._finish.assert_called_once_with(context)
submitted = window_manager.pie.calls[0][2]
assert submitted.gesture == "POLYLINE"
assert SelectionPath.from_json(submitted.selection_path_json).points == ((0, 0), (100, 0), (100, 100))
@pytest.mark.parametrize("reason", ("ESC", "RIGHTMOUSE", "TOOL_SWITCH"))
def test_cutout_polyline_cancel_cleans_preview(monkeypatch, reason):
context = SimpleNamespace()
operator = SimpleNamespace(
gesture="POLYLINE", active_tool_id="anyimage.cutout_lasso", _finish=Mock(),
_polyline_confirm_type="LEFTMOUSE",
)
operator._cancel_and_finish = MethodType(viewport.ImageGesture._cancel_and_finish, operator)
monkeypatch.setattr(viewport, "active_view3d_tool_id", lambda _context: "other" if reason == "TOOL_SWITCH" else None)
result = cutout_main.SelectCutoutSelection.modal(operator, context, SimpleNamespace(type=reason, value="PRESS"))
assert result == {"CANCELLED"}
operator._finish.assert_called_once_with(context)

Some files were not shown because too many files have changed in this diff Show More