icrdr/AnyImage
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat: Implement cutout tool and depth generation features

Browse Source 
- Added a new cutout tool for selecting regions in images, allowing users to choose mesh shapes based on selected areas.
- Introduced depth generation capabilities for rasterized images, including normal data extraction.
- Created jobs for generating depth planes and region data, enhancing the AI data generation workflow.
- Developed a selection refinement process for cutout regions, improving accuracy in generated outputs.
- Established a new region module for handling rasterized image regions and their associated AI generation tasks.
- Enhanced node graph builders for asset generation, facilitating smoother integration of new features.
This commit is contained in:
马南
2026-08-30 23:09:13 +08:00
parent 32bcd45d7b
commit 08309f719d
88 changed files with 4145 additions and 5639 deletions
Download Patch File Download Diff File Expand all files Collapse all files

1
AGENTS.md
View File

@@ -54,6 +54,7 @@ uv lock --refresh
## 命名规范 ## 命名规范
- 变量、函数、Docstring 使用英文；同一概念在代码和文档中使用统一表达 - 变量、函数、Docstring 使用英文；同一概念在代码和文档中使用统一表达
- 功能代码的动作前缀应表达产物边界：`generate_*` 表示 AI 生成，`build_*` 只在内存中构建值或几何，`create_*` 表示创建 Blender Image、Mesh、Material、Object 等真实数据
- 名称应说明业务职责，并与用户概念、公开协议和界面术语保持一致；不要使用脱离上下文后无法判断用途的宽泛名称 - 名称应说明业务职责，并与用户概念、公开协议和界面术语保持一致；不要使用脱离上下文后无法判断用途的宽泛名称
- 文件和目录名应反映职责层级：集合层使用复数，单个功能或动作使用单数或动宾结构；同一层级保持一致 - 文件和目录名应反映职责层级：集合层使用复数，单个功能或动作使用单数或动宾结构；同一层级保持一致
- Python 类使用大驼峰，不添加项目名前缀。Operator 使用动宾结构，如 `ConvertToDepthPlane`、`RemoveImageBackground`；Menu、Panel、AddonPreferences、PropertyGroup 分别使用实际类型后缀，如 `AnyImageImageMenu`、`ServerPanel`、`AnyImagePreferences`、`AnyImageSettings` - Python 类使用大驼峰，不添加项目名前缀。Operator 使用动宾结构，如 `ConvertToDepthPlane`、`RemoveImageBackground`；Menu、Panel、AddonPreferences、PropertyGroup 分别使用实际类型后缀，如 `AnyImageImageMenu`、`ServerPanel`、`AnyImagePreferences`、`AnyImageSettings`

14
README.md
View File

@@ -9,14 +9,14 @@ AnyImage 是一款 Blender Extension，用于把图片带入可编辑的 2.5D
- 在 3D View、Node Editor 与绘画模式中用 `Ctrl+V`（macOS 为 `Cmd+V`）粘贴并 Pack 剪贴板图片 - 在 3D View、Node Editor 与绘画模式中用 `Ctrl+V`（macOS 为 `Cmd+V`）粘贴并 Pack 剪贴板图片
- 将 Image Empty 转换为带 `O Mesh Plane` Geometry Nodes 的普通 Plane - 将 Image Empty 转换为带 `O Mesh Plane` Geometry Nodes 的普通 Plane
- 使用 MoGe-2 或 DA3 将静态图转换为可调 Depth Plane - 使用 MoGe-2 或 DA3 将静态图转换为可调 Depth Plane
- 用 Cutout 套索在图片可见 Alpha 内生成 Surface、Balloon、Depth Balloon 或 Depth Surface，并通过统一的 `O Image Cutout` 修改器切换 Shape、按创建分辨率清理碎岛和调整表面 - 用 Cutout 套索在图片可见 Alpha 内生成 Surface、Balloon、Depth Balloon 或 Depth Surface，并通过统一的 `O Image Cutout` 修改器切换 Shape、按创建分辨率清理碎岛、根据透明选区校准浮雕基准和调整表面
- 用 Image Box、Lasso、Line、Polyline 和 Perspective 清理或矫正静态图片 - 用 Image Box、Lasso、Line、Polyline 和 Perspective 清理或矫正静态图片
- 用 BEN2 为单图、视频或序列帧去除背景 - 用 BEN2 为单图、视频或序列帧去除背景
- 用三档 ONNX 模型将单图、视频或序列帧放大 4× - 用三档 ONNX 模型将单图、视频或序列帧放大 4×，并独立重采样 Alpha
AI 推理运行在 Blender 之外的独立 Python 3.12 Environment 中。Blender 进程只负责 UI、任务提交和结果导入，不加载 ONNX Runtime 或模型权重。 AI 推理运行在 Blender 之外的独立 Python 3.12 Environment 中。Blender 进程只负责 UI、任务提交和结果导入，不加载 ONNX Runtime 或模型权重。
全局 **Maximum AI Input Size** 默认限制 Upscale、Depth 与 Normal 分析的长边为 2048 px；超过时等比缩放分析输入，但不会降低颜色贴图分辨率。Cutout 在套索裁切后应用该限制，并以同一分析尺度计算网格密度。 全局 **Maximum AI Input Size** 默认设为 2048 px。当前图片长边达到该值时 Upscale 不可用；Depth 与 Normal 的分析输入超过该值时等比缩小，但不会降低颜色贴图分辨率。Cutout 在套索裁切后应用该限制，并以同一分析尺度计算网格密度。
## 安装 ## 安装
@@ -28,7 +28,7 @@ AI 推理运行在 Blender 之外的独立 Python 3.12 Environment 中。Blender
3. 选中 Image Empty，在右键 **AnyImage** 菜单中点击 **Set Up AI Environment…**。 3. 选中 Image Empty，在右键 **AnyImage** 菜单中点击 **Set Up AI Environment…**。
4. 安装完成后，在扩展 Preferences 中确认默认 MoGe-2、BEN2 与 Real-ESRGAN WDN 模型显示 Ready。 4. 安装完成后，在扩展 Preferences 中确认默认 MoGe-2、BEN2 与 Real-ESRGAN WDN 模型显示 Ready。
普通 Plane、基础 Image 编辑，以及不带 Refine / Normal 的 Cutout Surface 与 Balloon 无需 AI Environment。Cutout 的 Depth 与 Normal 统一使用 MoGe-2；两者同时启用时共享同一次 Prediction，不会重复运行模型。 普通 Plane、基础 Image 编辑，以及不带 Refine / Normal 的 Cutout Surface 与 Balloon 无需 AI Environment。Cutout 的 Depth 与 Normal 统一使用 MoGe-2；两者同时启用时共享同一次 Prediction，不会重复运行模型。Cutout Normal 可选择 None、基于完整图片平面的 Tangent Space 浮雕，或直接编码到对象局部轴的 Object Space。
## 快速使用 ## 快速使用
@@ -37,9 +37,9 @@ AI 推理运行在 Blender 之外的独立 Python 3.12 Environment 中。Blender
- **Convert to Plane**：同步创建普通图片网格； - **Convert to Plane**：同步创建普通图片网格；
- **Convert to Depth Plane**：使用 Preferences 中选择的深度模型处理静态图； - **Convert to Depth Plane**：使用 Preferences 中选择的深度模型处理静态图；
- **Remove Background**：生成透明图片或序列； - **Remove Background**：生成透明图片或序列；
- **Upscale**：使用默认高清模型生成 4× 结果。 - **Upscale**：使用默认高清模型把当前图片编辑为 4× 结果。
3D Viewport 工具栏提供 Image Mask、Perspective 与 Cutout。工作流对比和当前输入范围见[选择工作流](docs/guides/workflows.md)。 3D Viewport 工具栏提供 Image Mask、Perspective 与 Cutout。Remove Background、Upscale、Image Mask 和 Perspective 都只更新当前 Empty；共享 Image 会保留原数据，并为当前 Empty 创建独立结果。工作流对比和当前输入范围见[选择工作流](docs/guides/workflows.md)。
## 源码结构 ## 源码结构
@@ -52,7 +52,7 @@ src/anyimage/
└── __init__.py # Blender 注册入口 └── __init__.py # Blender 注册入口
``` ```
程序边界、功能文件结构和调用流程见[内部实现总览](docs/internals/index.md)。文档按开始使用、功能指南、参考和内部实现分层，不在 README 重复内部算法细节。 程序边界、功能文件结构和调用流程见[内部实现总览](docs/internals/index.md)；开发新功能前先查看[Common 公共函数](docs/internals/common.md)，复用已经存在的 Image、ImageGeometry、Material 和对象替换入口。文档按开始使用、功能指南、参考和内部实现分层，不在 README 重复内部算法细节。
## 开发 ## 开发

10
docs/guides/cutout.md
View File

@@ -21,7 +21,7 @@ Cutout 用套索从一张 Image Empty 中选择区域，再生成 Surface、Ball
四种结果都保留源 Image Empty，并在同一位置创建新网格。它们共享选区轮廓、UV、材质和基础三角网格，因此可以从同一张图多次生成不同 Shape 进行比较。 四种结果都保留源 Image Empty，并在同一位置创建新网格。它们共享选区轮廓、UV、材质和基础三角网格，因此可以从同一张图多次生成不同 Shape 进行比较。
无论是否启用 Refine Selection，套索超出图片的部分都不会生成网格，源图中 `Alpha <= 0.1` 的像素也始终从几何中排除。双击选择完整画面时同样只为 `Alpha > 0.1` 的区域生成网格。 无论是否启用 Refine Selection，套索超出图片的部分都不会生成网格，源图中 `Alpha <= 0.1` 的像素也始终从 Region 中排除。双击使用完整画面时同样只为 `Alpha > 0.1` 的 Region 生成网格。
## 工具设置 ## 工具设置
@@ -38,19 +38,19 @@ Point Spacing 先按完整源图换算基础密度；选区裁切后的长边若
四个 Shape 都使用同一个 `O Image Cutout` Geometry Nodes 修改器。创建时 Shape 菜单已经设为饼菜单所选类型；切换 Shape 后，Blender 只显示该类型实际使用的输入。常用参数包括： 四个 Shape 都使用同一个 `O Image Cutout` Geometry Nodes 修改器。创建时 Shape 菜单已经设为饼菜单所选类型；切换 Shape 后，Blender 只显示该类型实际使用的输入。常用参数包括：
- **Thickness**：Surface / Depth Surface 使用场景距离；Balloon / Depth Balloon 使用鼓起高度倍率。 - **Thickness**：Surface / Depth Surface 使用场景距离；Balloon / Depth Balloon 使用鼓起高度倍率。
- **Cleanup**：绝对面积阈值；创建时根据 Point Spacing 和图片平面尺寸自动写入，面积小于该值的 Mesh Island 会在生成任意 Shape 前移除。 - **Cleanup**：碎岛面积阈值；创建时根据 Point Spacing 和图片平面尺寸自动写入。为便于在修改器中调节，显示值放大了 `100` 倍，节点内部会还原为实际面积，因此这次显示缩放不会改变默认清理效果。
- **Smooth Iterations**：位于外层，控制平滑次数，默认 `0`；**Smooth** 位于 Options 中，控制每次平滑的混合强度。四种 Shape 都可使用。 - **Smooth Iterations**：位于外层，控制平滑次数，默认 `0`；**Smooth** 位于 Options 中，控制每次平滑的混合强度。四种 Shape 都可使用。
- **Depth Scale**：Depth Shape 的深度强度；`0` 回到基础平面或 Balloon。 - **Depth Scale**：Depth Shape 的深度强度；`0` 回到基础平面或 Balloon。
两个 Depth Shape 共用 **Reference Depth**；Depth Balloon 还提供 **Depth Direction**。插件自动写入的 Depth Image 与 Uniform Scale 位于折叠的 Data 面板，一般无需手动修改。 两个 Depth Shape 共用 **Base Plane Depth**；Depth Balloon 还提供 **Depth Direction**。生成前会检查最终 Region 输入的 Alpha：`Alpha < 0.1` 的像素达到 `5%` 时，根据有效 Region 自适应 Base Plane Depth，并为 Depth Balloon 拟合 Direction；不足 `5%` 时使用稳定的固定基准。这个判断不依赖双击全图或 Refine Selection。插件自动写入的 Depth Image 与 Uniform Scale 位于折叠的 Data 面板，一般无需手动修改。
Balloon 的轮廓高度算法由扩展 Preferences 中的 **Balloon Profile** 控制：Poisson 更平滑，Teddy 更强调轮廓骨架。修改默认值只影响之后创建的 Shape。 Balloon 的轮廓高度算法由扩展 Preferences 中的 **Balloon Profile** 控制：Poisson 更平滑，Teddy 更强调轮廓骨架。修改默认值只影响之后创建的 Shape。
## Refine 与 Normal ## Refine 与 Normal
Refine Selection 只在“套索与源图可见 Alpha 的交集”内收紧用于建网格的最终选择，不把 AI 生成的颜色写回原图。材质继续使用当前 Image Empty 的源颜色和原 Alpha。 Refine Selection 只在“套索与源图可见 Alpha 的交集”内收紧最终 Region，不把 AI 生成的颜色写回原图。材质继续使用当前 Image Empty 的源颜色和原 Alpha。
Normal 关闭时使用几何法线；开启后，Surface、Balloon 与 Depth Balloon 使用适配成品网格的 Tangent Space 法线，Depth Surface 使用 Object Space 法线。Cutout 始终使用 MoGe-2；Depth 与 Normal 同时需要时共享同一次 Prediction，不会重复运行模型。 Normal 设为 **None** 时使用几何法线；**Tangent** 把 MoGe-2 结果作为基于完整图片平面的浮雕细节，**Object** 则直接编码到 Cutout 对象局部轴。两种模式都不根据 Geometry Nodes 的成品网格再次烘焙。Normal 保持模型分析分辨率，Color Image 仍保留原始裁切分辨率；Depth 与 Normal 同时需要时共享同一次 Prediction，不会重复运行模型。
## 当前范围 ## 当前范围

2
docs/guides/models-and-storage.md
View File

@@ -20,7 +20,7 @@ AnyImage 把运行环境、模型和任务结果放在 Blender 之外，多个 B
## 全局 AI 输入尺寸 ## 全局 AI 输入尺寸
Environment 区的 **Maximum AI Input Size** 控制 Upscale、Depth 与 Normal 分析输入的最大长边，默认 `2048 px`，可在 `256–8192 px` 之间调整。超过上限的输入会保持宽高比缩小；颜色贴图仍使用原始分辨率。Cutout 在完成套索裁切后再应用上限，因此整张原图很大但选区较小时不会被无谓缩小。 Environment 区的 **Maximum AI Input Size** 默认 `2048 px`，可在 `256–8192 px` 之间调整。它既是允许执行 Upscale 的当前图片长边上限，也是 Depth 与 Normal 分析输入的最大长边；Upscale 在达到上限时置灰，Depth 与 Normal 则保持宽高比缩小分析输入。颜色贴图仍使用当前图片的原始分辨率。Cutout 在完成套索裁切后再应用上限，因此整张图片很大但选区较小时不会被无谓缩小。
## Device ## Device
扩展偏好设置中的 **Device** 控制 Depth Plane、Cutout 的 AI 增强、Remove Background 和 Upscale 等正式任务。 扩展偏好设置中的 **Device** 控制 Depth Plane、Cutout 的 AI 增强、Remove Background 和 Upscale 等正式任务。

10
docs/guides/upscale.md
View File

@@ -1,6 +1,6 @@
# 高清放大 # 高清放大
Upscale 用 ONNX 模型把 Image Empty 的图片放大 4 倍，不创建网格。它支持单图、Movie 和 Image Sequence，并保留源 Alpha。 Upscale 用 ONNX 模型把 Image Empty 的 RGB 放大 4 倍，不创建网格。它支持单图、Movie 和 Image Sequence，并以 Lanczos 独立放大源 Alpha 后重新合并。
## 使用 ## 使用
@@ -8,7 +8,7 @@ Upscale 用 ONNX 模型把 Image Empty 的图片放大 4 倍，不创建网格
2. 按需要设置 Environment 区的 **Maximum AI Input Size**；默认 `2048 px`，最高 `8192 px`。 2. 按需要设置 Environment 区的 **Maximum AI Input Size**；默认 `2048 px`，最高 `8192 px`。
3. 选中 Image Empty，在右键 **AnyImage** 菜单中点击 **Upscale**。 3. 选中 Image Empty，在右键 **AnyImage** 菜单中点击 **Upscale**。
输入长边超过全局上限时，会先保持宽高比缩小，再执行 4× 放大。因此默认设置下结果长边最多为 `8192 px`。完成后，当前 Empty 会改用名称以 `_upscale.png` 结尾的结果图片。 菜单项只显示本次操作预计生成的 4× 尺寸。当前图片长边必须小于全局上限；达到上限后 Upscale 会置灰，因此默认设置下单次结果长边小于 `8192 px`。完成后，当前 Empty 会直接使用放大结果；原 Image 没有其他引用时沿用原名并移除旧数据，存在其他引用时则为当前 Empty 保留独立结果。
## 模型选择 ## 模型选择
@@ -18,10 +18,10 @@ Upscale 用 ONNX 模型把 Image Empty 的图片放大 4 倍，不创建网格
后两项可在扩展偏好设置中按需下载。 后两项可在扩展偏好设置中按需下载。
对高清结果再次执行 Upscale 时，AnyImage 会从最初的原图重新生成并替换当前结果，不会连续放大已有高清图。 AnyImage 不追踪高清结果的原图引用，Upscale 始终处理当前 Image。只要当前长边仍小于全局上限，就可以再次放大；达到上限后不能继续执行。
把高清结果继续转换为 Depth Plane 时，颜色材质可以继续使用当前高清图，分析输入则遵守全局上限；Cutout 同样只缩放裁切后的分析输入，不会把限制后的临时图用作 Color。 把高清结果继续转换为 Depth Plane 时，颜色材质和分析都以当前 Image 为来源，分析输入仍遵守全局上限；Cutout 同样只缩放裁切后的分析输入，不会把限制后的临时图用作 Color。
如果显存或内存不足，请降低 **Maximum AI Input Size**，或改用 CPU。 如果显存或内存不足，请降低 **Maximum AI Input Size**，或改用 CPU。
分块推理、Alpha 合并与原图引用见 [Upscale 实现原理](../internals/operators/upscale.md)。 分块推理、Alpha 合并与输入限制见 [Upscale 实现原理](../internals/operators/upscale.md)。

10
docs/internals/architecture.md
View File

@@ -37,7 +37,7 @@ flowchart TD
Menu --> Runtime["注册 JobRuntime"] Menu --> Runtime["注册 JobRuntime"]
``` ```
`__init__.py` 按依赖顺序注册，并在注销时逆序清理。扩展级设置统一由 `preferences.addon_preferences()` 读取；`runtime.py` 只消费 `configured_storage_root` 回调，不持有 Blender 偏好设置的定位逻辑。交互中的临时状态和 Debug 参数保存在 `Scene.anyimage_settings`，Upscale 的原图引用保存在 `Image.anyimage_upscale_source`。 全局 `max_ai_input_size` 也在 `preferences.py` 定义与读取，由 Operator 冻结到每次 Job 请求。 `__init__.py` 按依赖顺序注册，并在注销时逆序清理。扩展级设置统一由 `preferences.addon_preferences()` 读取；`runtime.py` 只消费 `configured_storage_root` 回调，不持有 Blender 偏好设置的定位逻辑。交互中的临时状态和 Debug 参数保存在 `Scene.anyimage_settings`。全局 `max_ai_input_size` 在 `preferences.py` 定义与读取，由 Operator 冻结到每次 Job 请求；Upscale 还用它判断当前 Image 是否允许执行。
## 同步与异步路径 ## 同步与异步路径
@@ -60,6 +60,10 @@ sequenceDiagram
这条边界也决定了模块依赖方向：`server/` 不导入 `bpy` 或 `operators/`；Blender 侧可以读取 Server 产物，但不会复用 Server 中的 Blender 无关算法来创建 RNA 数据。 这条边界也决定了模块依赖方向：`server/` 不导入 `bpy` 或 `operators/`；Blender 侧可以读取 Server 产物，但不会复用 Server 中的 Blender 无关算法来创建 RNA 数据。
## 共享能力 ## 共享能力 { #shared-capabilities }
`operators/common/` 保存多个功能真实复用的能力：图片输入与命名、像素写回、手势状态、Mask 运算、Image Empty 坐标、材质、节点资产、法线烘焙和 Job 参数辅助。共享模块只承载跨功能概念；完整工作流仍留在各自功能包中，以便从用户入口追踪到实现。 `operators/common/` 只保存多个生产功能真实复用的稳定原子能力；完整工作流、功能参数和专用算法留在所属模块。公共层不保留旧路径转发函数，也不为了测试或资产工具建立入口。
全部可由功能模块调用的函数见[Common 公共函数](common.md)。该索引只列 `common/` 外部的生产调用，不重复列出公共模块内部辅助函数；创建新功能前先核对该页，避免重新实现 Image 引用分流、像素方向、Job 输入生命周期、数据块命名、节点资产加载和对象激活逻辑。
新增公共函数前应确认至少有两个真实消费者，或它是两个公共流程共同依赖的原子能力。只有单一功能使用的逻辑应留在所属模块；删除或移动能力时同步清理注册、测试、工具脚本和文档，不保留兼容胶水。

102
docs/internals/common.md Normal file
View File

@@ -0,0 +1,102 @@
# Common 公共函数
`src/anyimage/operators/common/` 集中已经被多个功能复用的 Blender 侧原子能力，目的是让开发新功能时能够先查找并复用现有实现，避免重复造轮子。
本页只列功能模块直接调用的公共函数，不列 `common/` 内部辅助函数、测试或资产工具专用函数，也不列公共类型。
## Image 数据块
`image_data.py` 处理 Blender `Image` 数据块及其在 Image Empty 上的使用方式，包括像素读取、动画帧、Job 输入、结果 Image 创建和 Empty 引用替换。它不处理 Viewport 手势或选区算法。
| 函数 | 职责 |
| --- | --- |
| `image_data.is_image_empty()` | 判断对象是否为带有效 Image 数据的 Empty。 |
| `image_data.require_image_empty()` | 按名称取得 Image Empty；对象已失效时抛错。 |
| `image_data.image_pixels()` | 以 Blender 原始顺序读取一维 float32 像素。 |
| `image_data.image_rgba()` | 读取为 Top-down RGBA float32 数组。 |
| `image_data.video_total_frames()` | 返回 Movie 或 Sequence 总帧数，静态图返回 1。 |
| `image_data.image_user_settings()` | 读取 Image Empty 的帧起点、偏移、时长和循环设置。 |
| `image_data.default_extract_frames()` | 取得当前 Empty 默认应处理的帧数。 |
| `image_data.source_start_frame()` | 从源 Image Empty 取得 Job 应处理的起始帧。 |
| `image_data.prepare_image_input()` | 将 Packed、Dirty、Sequence 等输入准备为 Job 可读路径，并返回临时标记。 |
| `image_data.prepare_region_input()` | 将 `RegionMask` 裁切为带 Mask Alpha 的临时 RGBA Job 输入。 |
| `image_data.cleanup_image_input()` | 按临时标记清理由 `prepare_image_input()` 创建的文件或目录。 |
| `image_data.image_base_name()` | 去除图片或视频扩展名和 Blender 数字后缀，得到数据块基础名。 |
| `image_data.depth_data_name()` | 生成 Depth 结果数据块文件名。 |
| `image_data.color_data_name()` | 生成 Color 结果数据块文件名。 |
| `image_data.load_color_result_image()` | 加载、命名、Pack 并配置 Job 的 Color 或 Sequence 结果；不修改 Empty。 |
| `image_data.load_normal_result_image()` | 加载、命名、设为 Non-Color 并 Pack Normal 结果。 |
| `image_data.create_pixel_result_image()` | 从本地编辑后的 RGBA 像素创建 Packed Image；不修改 Empty。 |
| `image_data.replace_empty_image()` | 执行 Image Empty 结果替换、共享引用分流、旧数据清理、帧设置继承和可选画布对齐。 |
| `image_data.image_empty_bounds()` | 计算 Image Empty 在对象局部坐标中的实际显示边界。 |
`load_*_result_image()` 和 `create_pixel_result_image()` 只创建结果 Image；`replace_empty_image()` 负责把结果提交给当前 Empty。两者分别表达“创建产物”和“替换引用”。
## Image Region 与视图交互
这组公共能力使用固定术语：
- Region 是用户在图片空间圈定的业务范围；`RegionPath` 保存路径点、反选状态和填充规则。
- `RegionMask` 是 Region 的栅格化结果，只包含局部像素权重 `values` 和源图 `bounds`。
- Mask 只表示二维像素权重或 Alpha 运算，不作为手势、协议或业务范围的别名。
- Selection 只用于 Blender 对象选择和界面动作 Refine Selection，不表示中间数据。
`image_interaction.py` 处理 3D Viewport 中的点击、坐标投影、`ImageRegionGesture`、工具设置和预览绘制；`image_region.py` 负责图片空间 Region 的值、栅格化和 Alpha 裁切。前者负责用户如何圈定 Region，后者负责 Region 如何变成 Mask。
### Viewport 交互
| 函数 | 职责 |
| --- | --- |
| `image_interaction.report_image_edit_exception()` | 将可预期的图片编辑警告和异常映射为 Operator Report。 |
| `image_interaction.image_edit_poll()` | 判断当前上下文是否允许启动 Image 编辑工具。 |
| `image_interaction.resolve_image_edit_click()` | 解析点击到 Image Empty，并保持或恢复选择状态。 |
| `image_interaction.active_view3d_tool_id()` | 取得当前 View3D Workspace Tool ID。 |
| `image_interaction.apply_region_settings()` | 将当前 Region 工具的 Refine 设置同步到手势 Operator。 |
| `image_interaction.draw_region_settings()` | 绘制 Region 工具的公共 Refine 设置。 |
| `image_interaction.draw_dashed_line()` | 绘制手势或透视框虚线。 |
| `image_interaction.draw_polygon_fill()` | 绘制手势或透视框填充预览。 |
| `image_interaction.gesture_property()` | 声明统一的 Box、Lasso、Line 和 Polyline RNA 属性。 |
| `image_interaction.screen_path_to_image_pixels()` | 将屏幕路径投影为 Image Empty 像素坐标。 |
| `image_interaction.serialize_matrix()` | 将 4×4 Matrix 序列化为 Operator 可保存的字符串。 |
| `image_interaction.deserialize_matrix()` | 恢复已序列化的 4×4 Matrix。 |
| `image_interaction.drawing_in_region()` | 判断 Modal 绘制是否仍位于启动时的 Area 和 Region。 |
### Region 栅格化
| 函数 | 职责 |
| --- | --- |
| `image_region.region_path_bounds()` | 校验 Region Path 并返回整数像素包围框。 |
| `image_region.rasterize_region_path()` | 把 `RegionPath` 栅格化为 `RegionMask`，并可与源 Alpha 相交。 |
| `image_region.crop_pixels_to_alpha()` | 按 Alpha 非零区域紧裁切 RGBA 像素。 |
## Mesh 结果构建
`mesh_result.py` 集中 Convert to Plane 与 Cutout 共同使用的结果构建能力：读取图片几何产物、创建材质、加载节点资产、设置修改器输入，并完成结果对象替换。
`ImageGeometry` 表示按图片像素组织的三维点、相机内参、有效区域、参考深度和选区统计；`load_image_geometry()` 从 `image_geometry.npz` 读取并校验这组数据。
| 函数 | 职责 |
| --- | --- |
| `mesh_result.load_image_geometry()` | 读取并校验 `image_geometry.npz`，返回用于构建 Mesh 的 `ImageGeometry`。 |
| `mesh_result.create_image_material()` | 从 Image Empty 创建包含颜色、位移、Normal、Alpha 和动画设置的完整材质。 |
| `mesh_result.load_asset_node_group()` | 从随扩展分发的 Blend 资产加载指定 Node Group。 |
| `mesh_result.modifier_input_identifier()` | 按接口名称解析 Geometry Nodes Modifier 输入 Identifier。 |
| `mesh_result.set_modifier_input()` | 跨受支持 Blender RNA 形式设置 Modifier 输入。 |
| `mesh_result.finalize_object_result()` | 激活结果对象，并按需替换或保留源 Empty。 |
节点资产构建和验证专用函数位于 `tools/node_assets/`，不属于运行时公共能力。
## AI 任务准备
`ai_job.py` 处理多个 AI 功能共同使用的任务准备，包括输入路径校验、Environment 与模型检查、Device 选择，以及 DA3 和 MoGe-2 请求参数组装。它不包含业务 Operator、结果导入或 Undo 操作。
| 函数 | 职责 |
| --- | --- |
| `ai_job.require_input_path()` | 解析 Blender 路径并要求其为现存文件或目录。 |
| `ai_job.require_background_input_path()` | 在通用路径校验之外限制 BEN2 支持的媒体类型。 |
| `ai_job.production_device()` | 读取正式任务使用的计算设备。 |
| `ai_job.require_environment()` | 要求 AnyImage AI Environment 已就绪。 |
| `ai_job.invoke_ai_setup_if_needed()` | Environment 未就绪时启动安装交互。 |
| `ai_job.require_model()` | 校验模型标识和下载状态，并返回模型记录。 |
| `ai_job.da3_parameters()` | 组装 DA3 正式任务与 Debug 共用参数。 |
| `ai_job.moge2_parameters()` | 组装 MoGe-2 正式任务与 Debug 共用参数。 |

3
docs/internals/index.md
View File

@@ -39,10 +39,11 @@ flowchart TD
## 文档分层 ## 文档分层
- [程序架构](architecture.md)：注册、配置、线程和模块边界。 - [程序架构](architecture.md)：注册、配置、线程和模块边界。
- [Common 公共函数](common.md)：仅列生产功能从 `operators/common/` 外部调用的函数。
- [Job Runtime 与 Server](runtime.md)：任务生命周期、媒体输入和模型 Session。 - [Job Runtime 与 Server](runtime.md)：任务生命周期、媒体输入和模型 Session。
- [Operator 实现索引](operators/index.md)：按 `src/anyimage/operators/` 中的完整功能拆分实现页。 - [Operator 实现索引](operators/index.md)：按 `src/anyimage/operators/` 中的完整功能拆分实现页。
- [Job 产物](outputs.md)：文件用途和 Blender 导入关系。 - [Job 产物](outputs.md)：文件用途和 Blender 导入关系。
- [开发与发布](development.md)：本地环境、测试、打包和 CI。 - [开发与发布](development.md)：本地环境、测试、打包和 CI。
- [节点资产](node-assets.md)：Geometry Nodes 的构建、加载与验证。 - [节点资产](node-assets.md)：Geometry Nodes 的构建、加载与验证。
内部实现文档描述模块之间的职责和主要数据流。函数级分支、参数校验与异常处理以源码和测试为准，避免文档重复实现细节。 内部实现文档描述模块之间的职责和主要数据流。[Common 公共函数](common.md)按真实外部调用列出开发新功能时应优先复用的稳定入口；函数级分支、参数校验与异常处理仍以源码和测试为准。

8
docs/internals/node-assets.md
View File

@@ -7,12 +7,12 @@ AnyImage 将稳定、可复用的 Geometry Nodes 和 Shader 节点组构建到 `
```text ```text
src/anyimage/ src/anyimage/
├── assets/O_AnyImage.blend ├── assets/O_AnyImage.blend
└── operators/common/ ├── operators/common/mesh_result.py # 运行时资产加载与 Modifier 输入
├── node_asset.py # 资产路径、加载与 Modifier 输入 └── operators/convert_to_plane/nodes.py # Depth Plane 运行时节点图
└── node_graph.py # 共享节点图构造函数
tools/node_assets/ tools/node_assets/
├── build.py # 统一构建入口 ├── build.py # 统一构建入口
├── node_graph.py # Cutout 资产构建专用节点图函数
├── build_mesh_plane.py ├── build_mesh_plane.py
├── build_shadeless.py ├── build_shadeless.py
├── build_cutout.py ├── build_cutout.py
@@ -27,7 +27,7 @@ tools/node_assets/
flowchart TD flowchart TD
Scripts["tools/node_assets/build_*.py"] --> Build["Blender 后台构建"] Scripts["tools/node_assets/build_*.py"] --> Build["Blender 后台构建"]
Build --> Blend["assets/O_AnyImage.blend"] Build --> Blend["assets/O_AnyImage.blend"]
Blend --> Loader["operators/common/node_asset.py"] Blend --> Loader["operators/common/mesh_result.py"]
Loader --> Modifier["运行时 Node Group / Modifier"] Loader --> Modifier["运行时 Node Group / Modifier"]
``` ```

13
docs/internals/operators/convert-to-plane.md
View File

@@ -1,6 +1,6 @@
# Plane 转换 # Plane 转换
Plane 转换包含两条路径：普通 Plane 在 Blender 内同步完成；Depth Plane 先通过 `prepare-geometry-data` Job 取得深度场，再回到 Blender 构建面片。两者共享图片命名、材质与基础四边形能力，但使用不同的 Geometry Nodes。 Plane 转换包含两条路径：普通 Plane 在 Blender 内同步完成；Depth Plane 先通过 `generate-depth-plane` Job 取得深度场，再回到 Blender 构建面片。两者共享图片命名、材质与基础四边形能力，但使用不同的 Geometry Nodes。
## 对应文件 ## 对应文件
@@ -13,7 +13,8 @@ operators/convert_to_plane/
└── nodes.py # 运行时 O Image Depth Plane 节点图 └── nodes.py # 运行时 O Image Depth Plane 节点图
server/ server/
├── jobs/prepare_geometry_data.py ├── jobs/generate_depth_plane.py
├── geometry/generate_depth.py
├── models/da3.py ├── models/da3.py
├── models/moge2.py ├── models/moge2.py
└── geometry/projection.py └── geometry/projection.py
@@ -27,8 +28,8 @@ flowchart TD
Kind -- "Plane" --> Copy["复制 Image 与材质"] Kind -- "Plane" --> Copy["复制 Image 与材质"]
Copy --> Quad["创建四边形 + O Mesh Plane"] Copy --> Quad["创建四边形 + O Mesh Plane"]
Kind -- "Depth Plane" --> Input["准备颜色与分析输入"] Kind -- "Depth Plane" --> Input["准备颜色与分析输入"]
Input --> Job["prepare-geometry-data"] Input --> Job["generate-depth-plane"]
Job --> Field["image.0001 + field.npz"] Job --> Field["image.0001 + image_geometry.npz"]
Field --> Anchor["按 Empty 范围锚定相机点"] Field --> Anchor["按 Empty 范围锚定相机点"]
Anchor --> Depth["创建深度 Image + 材质 + 节点修改器"] Anchor --> Depth["创建深度 Image + 材质 + 节点修改器"]
Quad --> Replace["替换源 Empty"] Quad --> Replace["替换源 Empty"]
@@ -45,7 +46,7 @@ flowchart TD
`ConvertToDepthPlane` 当前只接受静态图片。深度模型和分辨率来自扩展 Preferences：MoGe-2 使用 Resolution Level，DA3 使用按原图尺寸换算并对齐到模型 Patch 的处理分辨率。 `ConvertToDepthPlane` 当前只接受静态图片。深度模型和分辨率来自扩展 Preferences：MoGe-2 使用 Resolution Level，DA3 使用按原图尺寸换算并对齐到模型 Patch 的处理分辨率。
`GenerateDepthPlane` 把颜色输入、分析输入和模型参数提交给 `prepare-geometry-data`。Server 将模型结果统一为 `GeometryFrame`，写出包含相机点、深度、内参、有效区域和参考深度的 `field.npz`。这一步不创建网格。 `max_input_size` 同时传入 Job；Server 通过 `media/resolution.py` 在模型推理前限制分析图长边，默认 `2048`。颜色输入仍由未缩放的原图复制，受限分析图不会成为材质 Color。 `GenerateDepthPlane` 把当前 Image 随模型参数提交给 `generate-depth-plane`。`jobs/generate_depth_plane.py` 负责单图协议，`geometry/generate_depth.py` 负责 AI 深度生成；Server 将模型结果统一为 `GeometryFrame`，写出包含相机点、深度、内参、有效区域和参考深度的 `image_geometry.npz`。Blender 侧通过公共 `load_image_geometry()` 读取为 `ImageGeometry`，与 Cutout 使用同一协议校验。这一步不创建网格。`max_input_size` 同时传入 Job；Server 通过 `media/resolution.py` 在模型推理前限制分析图长边，默认 `2048`。颜色输入仍按当前 Image 的未缩放分辨率复制，受限分析图不会成为材质 Color。
Blender 响应阶段执行三件事： Blender 响应阶段执行三件事：
@@ -56,3 +57,5 @@ Blender 响应阶段执行三件事：
Depth 节点组先复用 `O Mesh Plane` 完成细分与材质，再生成背面厚度、限制正面向内穿透，最后在 Thickness 为零时删除背面和侧面。`Depth Amount` 控制位移强度；`Depth Image` 与 `Material` 是由插件填入的内部输入。 Depth 节点组先复用 `O Mesh Plane` 完成细分与材质，再生成背面厚度、限制正面向内穿透，最后在 Thickness 为零时删除背面和侧面。`Depth Amount` 控制位移强度；`Depth Image` 与 `Material` 是由插件填入的内部输入。
与 Cutout 的 Depth Surface 不同，Depth Plane 只使用锚定后的 Z 高度，不在 Geometry Nodes 中重建完整相机空间 XYZ。它强调一张图上的可编辑浮雕，而不是局部透视表面。 与 Cutout 的 Depth Surface 不同，Depth Plane 只使用锚定后的 Z 高度，不在 Geometry Nodes 中重建完整相机空间 XYZ。它强调一张图上的可编辑浮雕，而不是局部透视表面。
颜色材质、Normal 导入和结果对象激活分别复用 `create_image_material()`、`load_normal_result_image()` 与 `finalize_object_result()`；普通 Plane 和 Depth Plane 不再各自维护对象替换步骤。

108
docs/internals/operators/cutout-tool.md
View File

@@ -1,80 +1,76 @@
# Cutout # Cutout Tool
Cutout 从一条图片空间套索生成规则三角基础面，再交给统一的 `O Image Cutout` Geometry Nodes 修改器形成 Surface、Balloon、Depth Balloon 或 Depth Surface。公开节点组用 Shape 菜单路由到四个原有内部节点组；轮廓和 Mesh 始终由 Blender 主线程创建，Server 只在需要时准备 Refine、深度或法线数据。 Cutout 由两个核心 Operator 组成：
## 对应文件 - `SelectCutoutRegion`（`select_region.py`）把 Viewport 手势转换为图片空间 Region，并弹出 Shape 菜单。
- `CutoutRegionToShape`（`region_to_shape.py`）准备所需数据，再构建并创建 Shape。
```text 路径约定：Blender 侧文件相对 `src/anyimage/operators/cutout_tool/`，Server 侧文件相对 `src/anyimage/server/`。
operators/cutout_tool/
├── selection_tool.py # 套索交互、设置与 Shape 饼菜单
├── shape_conversion.py # 本地 / Job 路径选择和任务参数
├── build_shapes.py # 裁切贴图、Mesh、材质与修改器
├── shape_types.py # Shape 与节点组映射
├── mask_regions.py # Alpha 外轮廓与孔洞
├── planar_mesh.py # 规则采样与约束三角化
├── balloon_profile.py # Poisson / Teddy profile
├── depth_calibration.py # Reference 与 Direction
└── node_assets.py # Cutout 节点资产加载
server/ ## 执行路径
├── jobs/prepare_geometry_data.py # Cutout / Depth Plane 数据编排
├── selection/
│ ├── polygon_mask.py # Polygon 栅格化与选区裁切
│ └── cutout_images.py # Refine 与 Cutout 局部图片
└── geometry/normal_texture.py # 相机轴法线编码
```
## 路径选择
```mermaid ```mermaid
flowchart TD flowchart TD
Lasso["套索 + Shape"] --> Refine{"Refine Selection？"} Select["圈定 Cutout Region<br/>选择 Shape"]
Refine -- "是" --> Ben2["BEN2 × 1"] Raster["栅格化 Region"]
Refine -- "否" --> Selection["套索 Selection"] Route{"需要 AI 数据？"}
Ben2 --> Selection Local["使用本地 Region 数据"]
Selection --> NeedMoge{"需要 Depth 或 Normal？"} AI["生成 Refine / Depth / Normal"]
NeedMoge -- "否" --> Mesh["轮廓 → 规则点 → 约束三角化"] Ready["汇总 Shape 输入"]
NeedMoge -- "是" --> Moge["MoGe-2 × 1\\n同一 Prediction"] Build["构建基础 Shape"]
Moge --> Data["Depth Field / Camera Normal"] Create["创建 Blender 对象"]
Data --> Mesh
Mesh --> Profile["计算 o_balloon"] Select --> Raster
Profile --> Nodes["加载 O Image Cutout 并设置 Shape"] Raster --> Route
Nodes --> Bake["按需烘焙最终 Tangent Normal"] Route -- "否" --> Local --> Ready
Route -- "是" --> AI --> Ready
Ready --> Build --> Create
``` ```
Cutout 只接受静态图片。Refine Selection 是网格生成前唯一可选的选区分析：开启时执行一次 BEN2，关闭时直接使用套索 Selection。Surface 与 Balloon 在不需要 Normal 时不运行 MoGe-2；两个 Depth Shape 和任意开启 Normal 的 Shape 都统一使用 MoGe-2。Depth 与 Normal 同时启用时只执行一次推理，同一个 Prediction 同时提供相机点、深度和图片空间法线。 ## 节点说明
Cutout 不使用 DA3。若全局 Depth Model 当前选择 DA3，Cutout 会回退到默认 MoGe-2；选择 MoGe-2 档位时则沿用该档位和 Resolution Level。DA3 仍可用于独立的 Depth Plane 流程。 ### 圈定 Cutout Region
## 选择与局部数据 `SelectCutoutRegion` 将 Lasso 路径投射到图片像素坐标，形成 `RegionPath`，然后把 Region 和用户选择的 Shape 交给 `CutoutRegionToShape`。双击图片时，Region 使用完整图片范围。
Cutout 复用 `ImageMaskGesture` 的采点和屏幕预览，并复用 `operators/common/image_mask.py` 的可见选区语义：最终几何范围始终是套索、图片边界与源图 `Alpha > 0.1` 区域的交集。套索采用 NONZERO 填充规则，使反向内环可以表达孔洞。`Point Spacing` 先根据完整源图最长边换算为基础 `mesh_detail`；获得裁切 Selection 后，再按 `min(1, max_input_size / crop_long_edge)` 缩放有效 `mesh_detail`。因此只有裁切图超过全局上限时网格才会变稀，Color Image 仍使用原始裁切尺寸。 `RegionPath` 定义在 `operators/common/image_region.py`，包含图片空间路径点、反选状态和填充规则。它既供 Cutout 使用，也供 Image Tool 使用。
Job 路径将数据拆成三层：源颜色、最终选择和只供模型分析的 RGBA。写出最终选择前会再次以 `Alpha > 0.1` 与源图求交，因此未 Refine、Refine 和双击全图遵循相同边界。`bounds.json` 把可见选区的局部矩形映射回源图，Blender 据此恢复局部平面范围和 UV。未走 Job 时，`build_shapes.py` 在内存中完成相同的求交、裁切与坐标换算。 ### 栅格化 Region
## 基础网格 `CutoutRegionToShape` 把 `RegionPath` 栅格化为 `RegionMask`。`RegionMask` 只包含局部像素权重 `values` 和它在源图中的 `bounds`，后续几何、颜色裁切和 AI 输入都以它为范围。
最终 Alpha 先转为带包含关系的外轮廓与孔洞。`planar_mesh.py` 在内部生成近似六角排列的规则点，并用轮廓作为约束进行 Delaunay 三角化。边界按目标间距重采样，短轮廓会提高边界采样密度，内部点仍保持用户设置的间距。 Surface 或 Balloon 在未启用 Refine 与 Normal 时直接使用本地 Region 数据；选择 Depth Shape，或启用 Refine、Normal 时进入 AI 路径。两条路径最终都调用 `region_to_shape()`。
四种 Shape 共享同一张单面基础网格、UV 和 `o_balloon` 点属性。创建阶段以目标 Point Spacing 的平方作为一个采样单元面积，再除以 Image Empty 图片平面从局部到世界的面积缩放，得到修改器局部坐标中的 Cleanup 阈值。这个值只取决于工具分辨率和图片尺寸/变换，不受碎片数量或碎面面积分布影响。Balloon Profile 由 Preferences 选择 Poisson 或 Teddy；节点组只消费计算后的属性，不再重复推导轮廓。 ### 生成 AI 数据
## 统一修改器与四种 Shape `generate-region-data` Job（`jobs/generate_region_data.py`）根据请求生成所需数据：
创建结果时始终添加 `O Image Cutout`，并按饼菜单选择写入 Shape。节点组先统一执行 Cleanup，再由 Menu Switch 仅求值当前 Shape 的内部节点组，最后统一执行 Smooth；因此四种 Shape 都可使用 Cleanup、Smooth 和 Smooth Iterations。各分支独有输入直接连接到对应内部组，Blender 4.5 会在修改器中隐藏当前 Shape 不使用的输入。Cleanup 直接比较 Mesh Island 总面积与创建时写入的绝对面积阈值，不再统计当前网格面的中位面积。 - `generate_selection()`（`region/generate_selection.py`）使用 BEN2 收紧 Region；
- `generate_depth(..., normal_mode=...)`（`geometry/generate_depth.py`）使用 MoGe-2 生成 Depth，并可从同一次 Prediction 派生 Tangent 或 Object Normal。
| Shape | 内部节点组 | 主要几何来源 | Blender 传入的已是带 Region Alpha 的局部 RGBA。Depth 与 Normal 同时启用时只运行一次 MoGe-2，颜色贴图仍使用 Region 的原始裁切分辨率。
### 构建基础 Shape
`region_to_shape()` 汇总 Region、Depth 和 Normal 输入，再依次执行几何构建与 Blender 数据创建。
`build_base_shape()` 从 `RegionMask` 构建纯内存 `BaseShape`：先由 `region_contours.py` 提取包含外轮廓与孔洞的 `RegionContours`，再由 `triangulation.py` 按 Point Spacing 采样并约束三角化，最后使用 `balloon.py` 计算 `o_balloon`。`depth_calibration.py` 负责把 AI Depth 标定为 Blender 中使用的深度基准与方向。四种 Shape 共用这张单面基础网格。
### 创建 Blender 对象
`create_shape_object()` 创建裁切后的 Color Image、Material、Mesh Object 和统一的 `O Image Cutout` Modifier。Shape 的差异由 Modifier 中对应的几何分支表达：
| Shape | 形成方式 | Shape 自身所需 AI 数据 |
| --- | --- | --- | | --- | --- | --- |
| Surface | `O Image Surface` | 基础薄片与可调 Thickness | | Surface | 保持基础 Shape 的平面轮廓，并可增加真实厚度 | 无 |
| Balloon | `O Image Balloon` | `o_balloon` 鼓起高度 | | Balloon | 使用 `o_balloon` 从轮廓向内部平滑鼓起 | 无 |
| Depth Balloon | `O Image Depth Balloon` | Balloon profile + 相对基准面的非负深度高度 | | Depth Balloon | 在 Balloon 上叠加相对基准面的图片深度，并沿 Depth Direction 形成浮雕 | MoGe-2 Depth |
| Depth Surface | `O Image Depth Surface` | MoGe-2 相机点的完整投影 | | Depth Surface | 使用完整相机点 XYZ 重建带透视位移的表面，并可增加厚度 | MoGe-2 Camera Points |
四个内部节点组不再标记为 Modifier，也不再各自执行 Cleanup。原本只在 `O Image Depth Surface` 内的 Smooth 后处理已移到公开节点组；统一节点组的 Smooth 默认值为 `1`；外层 `Smooth Iterations` 默认值为 `0`，只有用户增加迭代次数后才执行平滑。两个 Depth 内部节点组也统一使用 `Reference Depth`，总节点组只暴露一个共用输入。 四种 Shape 共用 Region、基础三角面、UV、材质、Cleanup 和 Smooth。Cleanup 的 UI 数值放大 `100` 倍，节点内部乘以 `0.01` 还原，因此显示尺度不会改变实际清理阈值。
两个 Depth 形状把相机点 RGB 写入 Packed 浮点 `_depth.exr`，并以隐藏的 `Uniform Scale` 匹配 Image Empty 尺寸。未启用 Refine 时 Reference 使用固定值，Depth Balloon Direction 使用默认轴向；启用 Refine 后，`calibration_valid` 限制在模型有效区域与最终选择的交集，Reference 使用其深度第 95 百分位，Depth Balloon 还拟合局部基准方向。 ### Depth 与 Normal
Depth Balloon 保留图片平面 XY，只使用相机 Z 形成向外高度；Depth Surface 将完整 XYZ 映射到 Cutout 局部轴，因此能表达透视位移。两者都只采样深度 Image 的 RGB，Alpha 仅保存有效性和深度跳变信息。 Depth Shape 根据最终 Region Alpha 选择基准：透明像素比例达到 `5%` 时从有效内容自适应计算 Base Plane Depth；不足 `5%` 时使用固定基准。Depth Balloon 还会在自适应路径中拟合 Depth Direction。
## 法线空间 Normal 是材质层的浮雕细节，与四种 Shape 都可组合。None 使用几何法线；Tangent 以完整图片平面为切线基准；Object 直接编码到 Cutout 对象局部轴。Geometry Nodes 负责大轮廓和深度，Normal 负责表面细节。
MoGe-2 在 Job 阶段输出图片空间的相机轴法线预测；这一步不依赖 Blender 网格。Depth Surface 在导入时将它重排到 Cutout 对象轴并以 Object Space 使用；Surface、Balloon 与 Depth Balloon 则先创建基础网格并完成 Geometry Nodes 求值，再根据成品网格 UV、法线和切线基底烘成 Tangent Space。也就是说，网格前产生的是模型 Prediction，最终法线贴图的空间转换仍发生在网格生成之后。

24
docs/internals/operators/image-tool.md
View File

@@ -1,6 +1,6 @@
# Image Tool # Image Tool
`operators/image_tool/` 对应工具栏中的 Box、Lasso、Line 与 Polyline。四个 WorkspaceTool 共用一个 `EditImageMask` Operator，把屏幕手势转换为图片 Alpha；只有 Box 与 Lasso 可以用 BEN2 细化选择。 `operators/image_tool/` 对应工具栏中的 Box、Lasso、Line 与 Polyline。四个 WorkspaceTool 共用一个 `EditImageMask` Operator：手势先形成图片空间 Region，Region 栅格化为 Mask 后才修改图片 Alpha；只有 Box 与 Lasso 可以用 BEN2 执行 Refine Selection。
## 对应文件 ## 对应文件
@@ -20,20 +20,22 @@ operators/image_tool/
```mermaid ```mermaid
flowchart TD flowchart TD
Tool["Box / Lasso / Line / Polyline"] --> Gesture["ImageMaskGesture"] Tool["Box / Lasso / Line / Polyline"] --> Gesture["ImageRegionGesture"]
Gesture --> Polygon["图片像素 Polygon"] Gesture --> Region["RegionPath"]
Polygon --> Refine{"Refine Selection?"} Region --> Raster["栅格化 Region"]
Refine -- "否" --> Mask["本地几何 Mask"] Raster --> Data["RegionMask"]
Refine -- "是" --> Job["refine-image-mask"] Data --> Refine{"Refine Selection?"}
Refine -- "否" --> Mask["Region Mask"]
Refine -- "是" --> Input["带 Region Alpha 的裁切 RGBA"]
Input --> Job["refine-image-mask"]
Job --> Mask Job --> Mask
Mask --> Invert["可选反选"] Mask --> Alpha["源 Alpha × Mask"]
Invert --> Alpha["源 Alpha × 最终 Mask"]
Alpha --> Crop["紧裁切并写回 Empty"] Alpha --> Crop["紧裁切并写回 Empty"]
``` ```
`EditImageMask` 只接受静态图片。未启用 Refine 时，Polygon 在 Blender 侧直接栅格化；启用后，Operator 准备单图输入并提交 Job，响应阶段读取局部 Alpha 与源图 `bounds`，再映射回完整画布。反选始终在最终 Mask 上执行。 `EditImageMask` 只接受静态图片。`ImageRegionGesture` 把屏幕路径投射为 `RegionPath`；`rasterize_region_path()` 再生成 `RegionMask`。未启用 Refine 时直接把 Region Mask 乘入源 Alpha；启用后，Operator 将 `RegionMask` 写成局部 RGBA 并提交 Job，响应阶段读取细化后的 Alpha 与 `bounds`，再映射回完整画布。
Box 和 Lasso 分别保存自己的 Refine 状态。Line 与 Polyline 强制关闭 Refine，只使用几何 Mask；Line 将有向线扩展为半平面，Polyline 显式闭合用户确认的顶点。 Box 和 Lasso 分别保存自己的 Refine Selection 状态。Line 与 Polyline 强制关闭 Refine，只使用本地 Region Mask；Line 将有向线扩展为半平面 Path，Polyline 显式闭合用户确认的 Path。
## 手势与预览 ## 手势与预览
@@ -43,4 +45,4 @@ Box 和 Lasso 分别保存自己的 Refine 状态。Line 与 Polyline 强制关
## 写回结果 ## 写回结果
最终像素保留源 RGB，只修改 Alpha，再按非零 Alpha 紧裁切。公共写回函数只替换当前 Empty 的 Image，并更新 `empty_display_size` 与 `empty_image_offset`，因此剩余像素继续对齐原图片平面，对象 `matrix_world` 不变。 最终像素通过公共 `image_pixels()` / `image_rgba()` 读取，保留源 RGB，只修改 Alpha，再按非零 Alpha 紧裁切。公共像素写回函数创建结果后，通过统一的 Image Empty 替换入口处理共享引用，并更新 `empty_display_size` 与 `empty_image_offset`，因此剩余像素继续对齐原图片平面，对象 `matrix_world` 不变。

6
docs/internals/operators/index.md
View File

@@ -1,6 +1,6 @@
# Operator 实现索引 # Operator 实现索引
`src/anyimage/operators/` 负责 Blender 侧交互、Job 请求组装和结果导入。内部实现按完整功能包或顶层业务模块拆页；只提供共享能力的 `common/` 不单独成页，其职责统一放在[程序架构](../architecture.md)中说明。 `src/anyimage/operators/` 负责 Blender 侧交互、Job 请求组装和结果导入。内部实现按完整功能包或顶层业务模块拆页；`common/` 的边界见[程序架构](../architecture.md)，实际公开入口见[Common 公共函数](../common.md)。
| Operator 路径 | 实现页 | 主要职责 | | Operator 路径 | 实现页 | 主要职责 |
| --- | --- | --- | | --- | --- | --- |
@@ -8,11 +8,11 @@
| `operators/ai_setup.py` | [AI 环境管理](ai-setup.md) | 环境安装、模型下载与缓存释放 | | `operators/ai_setup.py` | [AI 环境管理](ai-setup.md) | 环境安装、模型下载与缓存释放 |
| `operators/clipboard_image/` | [Clipboard Image](clipboard-image.md) | 系统剪贴板读取、粘贴与快捷键 | | `operators/clipboard_image/` | [Clipboard Image](clipboard-image.md) | 系统剪贴板读取、粘贴与快捷键 |
| `operators/convert_to_plane/` | [Convert to Plane](convert-to-plane.md) | Mesh Plane 与 Depth Plane 创建 | | `operators/convert_to_plane/` | [Convert to Plane](convert-to-plane.md) | Mesh Plane 与 Depth Plane 创建 |
| `operators/cutout_tool/` | [Cutout Tool](cutout-tool.md) | 区域选择、轮廓生成与 Shape 构建 | | `operators/cutout_tool/` | [Cutout Tool](cutout-tool.md) | Region 选择、基础 Shape 构建与 Blender 对象创建 |
| `operators/debug.py` | [Debug Operator](debug.md) | 模型接口与原始产物验证 | | `operators/debug.py` | [Debug Operator](debug.md) | 模型接口与原始产物验证 |
| `operators/image_tool/` | [Image Tool](image-tool.md) | Box、Lasso、Line 与 Polyline 编辑 | | `operators/image_tool/` | [Image Tool](image-tool.md) | Box、Lasso、Line 与 Polyline 编辑 |
| `operators/perspective_tool/` | [Perspective Tool](perspective-tool.md) | 四点透视框与透视校正 | | `operators/perspective_tool/` | [Perspective Tool](perspective-tool.md) | 四点透视框与透视校正 |
| `operators/remove_background.py` | [Remove Background](remove-background.md) | BEN2 去背与结果替换 | | `operators/remove_background.py` | [Remove Background](remove-background.md) | BEN2 去背与结果替换 |
| `operators/upscale.py` | [Upscale](upscale.md) | 4× 高清与原图引用维护 | | `operators/upscale.py` | [Upscale](upscale.md) | 当前 Image 的 4× 高清与输入长边检查 |
`operators/__init__.py` 只聚合注册类，不承载独立业务流程。具体类的注册和逆序注销由扩展入口统一管理，整体边界见[程序架构](../architecture.md)。 `operators/__init__.py` 只聚合注册类，不承载独立业务流程。具体类的注册和逆序注销由扩展入口统一管理，整体边界见[程序架构](../architecture.md)。

4
docs/internals/operators/perspective-tool.md
View File

@@ -10,7 +10,7 @@ operators/perspective_tool/
└── edit_perspective.py # 四点交互、预览、Homography 与写回 └── edit_perspective.py # 四点交互、预览、Homography 与写回
``` ```
图片选择、裁切和 Empty 写回复用 `operators/common/` 的基础能力，不与 Image Mask Operator 共享业务状态。 图片坐标投射、Alpha 裁切和 Empty 写回复用 `operators/common/` 的基础能力；四点数据始终称为 Quad，不与 Image Region 或 Image Mask Operator 共享业务状态。
## 矫正流程 ## 矫正流程
@@ -29,4 +29,4 @@ flowchart TD
Homography 使用预乘 Alpha 插值，避免透明边缘混入黑色。输出最长边限制为 8192 px，总像素限制为 16,777,216，并按 256 行分块计算，控制中间数组大小。 Homography 使用预乘 Alpha 插值，避免透明边缘混入黑色。输出最长边限制为 8192 px，总像素限制为 16,777,216，并按 256 行分块计算，控制中间数组大小。
Perspective 是 deform 工具，不提供 Refine 或反选。结果比例可能不同于原四边形的轴对齐范围，因此 Empty 写回保持中心和覆盖尺度近似一致，而不是假设当前 3D View 相机等于图片拍摄相机。 Perspective 是 deform 工具，不提供 Refine 或反选。它通过公共 `image_pixels()` 读取供预览与最终 Warp 共用的源像素，并与 Image Mask 共用像素结果创建和 Image Empty 替换入口。结果比例可能不同于原四边形的轴对齐范围，因此 Empty 写回保持中心和覆盖尺度近似一致，而不是假设当前 3D View 相机等于图片拍摄相机。

4
docs/internals/operators/remove-background.md
View File

@@ -8,7 +8,7 @@
src/anyimage/ src/anyimage/
├── operators/ ├── operators/
│ ├── remove_background.py # Blender 入口、任务参数与结果替换 │ ├── remove_background.py # Blender 入口、任务参数与结果替换
│ └── common/image.py # 媒体输入准备与结果 Image 创建 │ └── common/image_data.py # 媒体输入准备与结果 Image 创建
└── server/ └── server/
├── jobs/remove_background.py # 单图与多帧任务编排 ├── jobs/remove_background.py # 单图与多帧任务编排
└── models/ben2.py # BEN2 前景适配 └── models/ben2.py # BEN2 前景适配
@@ -28,4 +28,4 @@ flowchart TD
Packed 图片或内存中修改过的单图会先导出为临时输入；文件图片、视频和序列则沿用可读取的源路径。多帧输入由统一媒体层展开，结果保持连续帧命名，导入后继续沿用源 Image User 的帧起点、偏移与循环设置。 Packed 图片或内存中修改过的单图会先导出为临时输入；文件图片、视频和序列则沿用可读取的源路径。多帧输入由统一媒体层展开，结果保持连续帧命名，导入后继续沿用源 Image User 的帧起点、偏移与循环设置。
BEN2 推理前使用输入 Alpha 屏蔽透明区域，模型输出的前景 Alpha 再与源 Alpha 合成，RGB 保持为源图内容。响应阶段只替换当前 Empty 的 Image 数据；当旧 Image 没有其他用户和 Fake User 时，旧数据块会被清理并由结果继承名称。 BEN2 推理前使用输入 Alpha 屏蔽透明区域，模型输出的前景 Alpha 再与源 Alpha 合成，RGB 保持为源图内容。响应阶段通过公共 Image Empty 替换入口导入结果；当旧 Image 没有其他用户和 Fake User 时，旧数据块会被清理并由结果继承名称。

19
docs/internals/operators/upscale.md
View File

@@ -1,6 +1,6 @@
# Upscale # Upscale
本页对应 `src/anyimage/operators/upscale.py`。该模块读取扩展首选项中的模型与全局 AI 输入长边上限，提交 4× 高清任务，并维护高清结果到原始 Image 的引用。 本页对应 `src/anyimage/operators/upscale.py`。该模块读取扩展首选项中的模型与全局 AI 输入长边上限，检查当前 Image 是否可处理，再提交 4× 高清任务。
## 对应文件 ## 对应文件
@@ -8,7 +8,7 @@
src/anyimage/ src/anyimage/
├── operators/ ├── operators/
│ ├── upscale.py # Blender 入口、任务参数与结果替换 │ ├── upscale.py # Blender 入口、任务参数与结果替换
│ └── common/image.py # 媒体输入准备与结果 Image 创建 │ └── common/image_data.py # 媒体输入准备与结果 Image 创建
└── server/ └── server/
├── jobs/upscale.py # 单图与多帧任务编排 ├── jobs/upscale.py # 单图与多帧任务编排
├── media/resolution.py # 共用长边限制与等比缩放 ├── media/resolution.py # 共用长边限制与等比缩放
@@ -20,17 +20,18 @@ src/anyimage/
```mermaid ```mermaid
flowchart TD flowchart TD
Empty["Image Empty"] --> Source["解析原始 Image"] Empty["Image Empty"] --> Source["读取当前 Image"]
Source --> Prepare["准备媒体输入"] Source --> Gate{"当前长边小于全局上限？"}
Gate -- "否" --> Disabled["Upscale 置灰"]
Gate -- "是" --> Prepare["准备媒体输入"]
Prepare --> Job["upscale-image Job"] Prepare --> Job["upscale-image Job"]
Job --> Limit["按全局输入长边上限等比缩放"] Job --> RGB["分块执行 4× RGB 推理"]
Limit --> RGB["分块执行 4× RGB 推理"] Job --> Alpha["独立缩放 Alpha"]
Limit --> Alpha["独立缩放 Alpha"]
RGB --> Merge["合并 RGBA 结果"] RGB --> Merge["合并 RGBA 结果"]
Alpha --> Merge Alpha --> Merge
Merge --> Replace["导入并替换当前 Image"] Merge --> Replace["导入并替换当前 Image"]
``` ```
模型以 256×256 分块处理 RGB，并带入相邻上下文以减弱块边界；Alpha 按同一受限输入尺寸单独缩放，最后合并为 RGBA。`max_input_size` 来自全局 **Maximum AI Input Size**，默认 `2048`，只约束送入模型的长边；4× 输出长边因此默认为最多 `8192`。输入尺寸不超过上限时不会发生预缩放。 模型以 256×256 分块处理 RGB，并带入相邻上下文以减弱块边界；Alpha 不进入模型，使用 Lanczos 独立放大后合并为 RGBA。`max_input_size` 来自全局 **Maximum AI Input Size**，默认 `2048`。Operator 只允许当前 Image 长边小于该值时执行，因此正式流程不会预缩放输入，4× 输出长边小于上限的四倍。Server 仍保留相同上限作为任务边界。
结果以 `_upscale.png` 结尾，并通过 `Image.anyimage_upscale_source` 保存原图引用。再次对高清结果执行时，Operator 会从原图重新生成，而不是继续放大上一次输出。模型与全局输入上限在每次执行时从 Preferences 读取，再写入可序列化的隐藏属性交给 Job。Provider 资源错误会释放当前 Upscale Session，并提示降低全局输入上限或改用 CPU。 菜单根据当前尺寸只显示预计的 4× 结果尺寸。结果通过与 Remove Background、Image Mask 和 Perspective 相同的 Image Empty 替换入口处理：原 Image 没有其他用户且未启用 Fake User 时，移除旧数据并让结果沿用原名；存在其他用户时保留原 Image，并只让当前 Empty 使用独立结果。结果不增加 Upscale 专属后缀，也不保存来源引用。后续 Upscale 与 Depth Plane 都读取当前 Image。模型与全局输入上限在每次执行时从 Preferences 读取，再写入可序列化的隐藏属性交给 Job。Provider 资源错误会释放当前 Upscale Session，并提示降低全局输入上限或改用 CPU。

28
docs/internals/outputs.md
View File

@@ -21,16 +21,16 @@ flowchart TD
| Job | 主要文件 | 消费方 | | Job | 主要文件 | 消费方 |
| --- | --- | --- | | --- | --- | --- |
| `remove-background` | `image.0001.png` 起 | 替换 Image Empty 的颜色数据 | | `remove-background` | `image.0001.png` 起 | 替换 Image Empty 的颜色数据 |
| `upscale-image` | `upscale.0001.png` 起 | 创建 `_upscale.png` Image 并记录原图引用 | | `upscale-image` | `upscale.0001.png` 起 | 按共享引用情况替换当前 Image Empty 的颜色数据，并沿用原 Image 命名语义 |
| `refine-image-mask` | `selection-input.png`、`image.png`、`bounds.json` | Image Box / Lasso 读取 Alpha 与原图坐标 | | `refine-image-mask` | `image.png`、`bounds.json` | Image Box / Lasso 读取细化 Alpha 与原图坐标 |
| `prepare-geometry-data`：Depth Plane | `image.0001.*`、`field.npz` | 创建颜色图、深度图、材质和 Plane | | `generate-depth-plane` | `image.0001.*`、`image_geometry.npz` | 创建颜色图、深度图、材质和 Plane |
| `prepare-geometry-data`：Cutout | `image.0001.png`、`selection.png`、`analysis.png`、`bounds.json`，以及按形状生成的 `field.npz` / `normal.png` | Blender 主线程构建局部网格与贴图 | | `generate-region-data` | 可选的 `region.png`、`region_bounds.json`、`image_geometry.npz`、`normal.png` | Blender 主线程构建 Cutout Shape 与贴图 |
`prepare-geometry-data` 不输出 Blender 网格。它只准备颜色、选区和模型数组；轮廓提取、约束三角化、Geometry Nodes Modifier 与法线烘焙都在 Blender 响应阶段完成。 两个几何 Job 都不输出 Blender 网格。`generate-depth-plane` 生成完整图片的颜色副本与深度场；`generate-region-data` 只生成请求的 AI 产物。轮廓提取、约束三角化以及 Mesh、Material 和 Geometry Nodes 创建都在 Blender 主线程完成。
## 深度场 ## 深度场
`field.npz` 是 Depth Plane、Depth Balloon 与 Depth Surface 共用的模型中间结果： `image_geometry.npz` 是 Depth Plane、Depth Balloon 与 Depth Surface 共用的模型中间结果。Blender 侧统一通过 `operators/common/mesh_result.py` 的 `load_image_geometry()` 读取为 `ImageGeometry`，功能模块不分别解释字段：
| 字段 | 含义 | | 字段 | 含义 |
| --- | --- | | --- | --- |
@@ -38,21 +38,21 @@ flowchart TD
| `depth` | 相机深度 | | `depth` | 相机深度 |
| `intrinsics` | 像素坐标下的 3×3 相机内参 | | `intrinsics` | 像素坐标下的 3×3 相机内参 |
| `valid` | 模型与数值检查后的有效区域 | | `valid` | 模型与数值检查后的有效区域 |
| `calibration_valid` | 用于结果定位或深度基准的区域；Cutout 时与选择相交 | | `calibration_valid` | 用于结果定位或深度基准的区域；Cutout 时与 Region 相交 |
| `image_size` | 深度场宽高 | | `image_size` | 深度场宽高 |
| `reference_depth` | 校准区域中的参考深度 | | `reference_depth` | 校准区域中的参考深度 |
Server 保留模型空间数据，不提前换算 Blender 单位。Depth Plane 在导入时根据 Image Empty 平面范围和相机内参锚定点场；Cutout 深度形状则计算 `Uniform Scale` 和局部坐标，再把相机点写入 Packed 浮点 Image 供 Geometry Nodes 采样。 Server 保留模型空间数据，不提前换算 Blender 单位。Depth Plane 在导入时根据 Image Empty 平面范围和相机内参锚定点场；Cutout 深度形状则计算 `Uniform Scale` 和局部坐标，再把相机点写入 Packed 浮点 Image 供 Geometry Nodes 采样。
## Cutout 三类图片 ## Cutout Region 产物
Cutout 把“显示内容”“几何范围”和“模型分析”分开： Blender 侧先把 `RegionPath` 栅格化为 `RegionMask`，再将 Region 裁切为带有效 Alpha 的临时 RGBA 输入。Server 不接收路径或 Mask 协议，只按需产生 AI 数据：
- `image.0001.png` 保存裁切矩形中的源颜色与原 Alpha，用作材质。 - 启用 Refine 时，`region.png` 保存细化后的局部 RGBA，`region_bounds.json` 保存它在输入裁切图中的范围；
- `selection.png` 保存最终选择 Alpha，用于轮廓与网格。 - Depth Shape 使用 `image_geometry.npz`；
- `analysis.png` 只保留选择内的有效 RGBA，供深度或法线模型分析。 - Tangent 或 Object Normal 使用 `normal.png`。
`bounds.json` 把裁切矩形映射回源图像素坐标。这样 Server 可以只处理局部数据，Blender 仍能恢复局部平面范围、UV 和世界位置。 Blender 将局部范围与原始 `RegionMask.bounds` 合成回源图坐标，并直接从源 Image 创建最终 Color Image。未启用 Refine、Depth 或 Normal 时不提交 Job。
## Debug 产物 ## Debug 产物
@@ -65,7 +65,7 @@ Debug Job 保留模型适配层的直接输出，不执行正式功能的 Blende
| `debug-ben2` | `foreground*.png` | | `debug-ben2` | `foreground*.png` |
| `debug-upscale` | `prediction*.png` | | `debug-upscale` | `prediction*.png` |
这些文件用于验证模型、Device 和适配层。正式 Job 会继续派生选择、Alpha、深度场或可导入图片，因此两者的文件结构有意不同。 这些文件用于验证模型、Device 和适配层。正式 Job 会继续派生 Region、Alpha、深度场或可导入图片，因此两者的文件结构有意不同。
## 单图与序列 ## 单图与序列

13
docs/internals/runtime.md
View File

@@ -14,7 +14,7 @@ src/anyimage/
├── model_manager.py # 模型目录与 Session 缓存 ├── model_manager.py # 模型目录与 Session 缓存
├── media/input.py # 图片、视频和序列输入 ├── media/input.py # 图片、视频和序列输入
├── jobs/ # app.py 可提交的公开任务 ├── jobs/ # app.py 可提交的公开任务
├── selection/ # 选区栅格化与 Cutout 图片准备 ├── region/ # BEN2 Region 细化
├── models/ # 模型适配与 ONNX 推理 ├── models/ # 模型适配与 ONNX 推理
├── geometry/ # 深度反投影与法线纹理编码 ├── geometry/ # 深度反投影与法线纹理编码
└── outputs/ # Debug Prediction 序列化 └── outputs/ # Debug Prediction 序列化
@@ -41,17 +41,18 @@ Runtime 在首次任务或显式启动时拉起 Server。任务在队列中串
| `download-model`、`download-required-models` | 准备模型文件 | | `download-model`、`download-required-models` | 准备模型文件 |
| `remove-background` | BEN2 透明前景 | | `remove-background` | BEN2 透明前景 |
| `upscale-image` | 4× 图片或帧序列 | | `upscale-image` | 4× 图片或帧序列 |
| `refine-image-mask` | 为 Image Box / Lasso 细化选区 | | `refine-image-mask` | 为 Image Box / Lasso 细化 Region |
| `prepare-geometry-data` | 为 Depth Plane 或 Cutout 准备颜色、选区、深度和法线数据 | | `generate-depth-plane` | 为 Depth Plane 生成颜色副本与深度场 |
| `generate-region-data` | 按需为 Cutout Region 生成 Refine、Depth 或 Normal |
| `debug-*` | 保存模型适配层的直接输出 | | `debug-*` | 保存模型适配层的直接输出 |
## 媒体输入 ## 媒体输入
`server/media/input.py` 把单图、视频和图片目录统一为有序帧路径，并用上下文管理器负责临时帧目录的生命周期。它还支持将颜色输入与分析输入分开：例如 Upscale 结果可以作为颜色来源，同时使用它保存的原图引用进行 Depth Plane 分析。帧展开是媒体输入的内部能力，不再暴露没有业务消费者的独立 Job。 `server/media/resolution.py` 提供默认 `2048` 的共用输入长边限制；Upscale 直接限制每帧，`prepare-geometry-data` 则在 Cutout 裁切和分析输入选择完成后限制 Depth / Normal 模型输入。该临时缩放图不参与颜色产物写入。 `server/media/input.py` 把单图、视频和图片目录统一为有序帧路径，并用上下文管理器负责临时帧目录的生命周期。帧展开是媒体输入的内部能力，不再暴露没有业务消费者的独立 Job。`server/media/resolution.py` 提供默认 `2048` 的共用输入长边限制；Upscale Operator 只允许长边小于该值的当前 Image 进入正式任务，Depth Plane 与 Cutout 则只限制模型分析输入，不降低颜色贴图分辨率。
当前 `prepare-geometry-data` 只接受一帧，因此 Depth Plane 与 Cutout 都是静态图工作流。Remove Background 与 Upscale 通过同一媒体入口处理单图、视频和序列帧。 `generate-depth-plane` 与 `generate-region-data` 只接受一帧，因此 Depth Plane 与 Cutout 都是静态图工作流。Remove Background 与 Upscale 通过同一媒体入口处理单图、视频和序列帧。
公开 Job 不与 Operator 一一对应：Depth Plane 和 Cutout 共享 `prepare-geometry-data`，四个 Debug Job 共享 `jobs/debug_predictions.py`。Job 只负责协议和编排；可复用的输入、选区、模型与几何数据能力分别位于 `media/`、`selection/`、`models/` 和 `geometry/`，Job 之间不互相导入实现。 公开 Job 按稳定的计算边界划分，不按界面入口机械映射。Depth Plane 使用 `jobs/generate_depth_plane.py`，Cutout 使用 `jobs/generate_region_data.py`，四个 Debug Job 共享 `jobs/debug_predictions.py`。Job 只负责协议和编排；可复用的输入、AI Region、模型与几何数据能力分别位于 `media/`、`region/`、`models/` 和 `geometry/`，Job 之间不互相导入实现。
## 模型层 ## 模型层

4
docs/reference/features.md
View File

@@ -34,11 +34,11 @@ Cutout 提供四种结果：
| Depth Balloon | `O Image Depth Balloon` | MoGe-2 高度 | | Depth Balloon | `O Image Depth Balloon` | MoGe-2 高度 |
| Depth Surface | `O Image Depth Surface` | MoGe-2 相机点 | | Depth Surface | `O Image Depth Surface` | MoGe-2 相机点 |
四种 Shape 共用 `O Image Cutout` 修改器、规则三角基础面、UV 和 `o_balloon` 属性。Shape 菜单选择内部几何分支，Cleanup 在分支前执行，Smooth 在分支后执行；Point Spacing 控制网格密度，Normal 与 Refine Selection 按需启用 MoGe-2 和 BEN2。 四种 Shape 共用 `O Image Cutout` 修改器、规则三角基础面、UV 和 `o_balloon` 属性。Shape 菜单选择内部几何分支，Cleanup 在分支前执行且使用便于调节的百倍显示值，Smooth 在分支后执行；Depth Shape 根据 Region 透明比例选择固定或自适应 Base Plane Depth，Normal 可选 None、Tangent 或 Object，Refine Selection 按需启用 BEN2。
### 图片结果 ### 图片结果
Image Mask 和 Perspective 直接更新当前 Empty 的图片并 Pack。Remove Background 输出透明颜色图或序列。Upscale 通过三档 ONNX 模型输出 4× RGBA，并保存原图引用，重复执行时从原图重新生成。 Image Mask 和 Perspective 直接更新当前 Empty 的图片并 Pack。Remove Background 输出透明颜色图或序列。Upscale 通过三档 ONNX 模型把当前 Image 输出为 4× RGBA；当前长边达到全局输入上限时不可执行。
## 模型 ## 模型

1
mkdocs.yml
View File

@@ -29,6 +29,7 @@ nav:
- 内部实现: - 内部实现:
- 总览: internals/index.md - 总览: internals/index.md
- 程序架构: internals/architecture.md - 程序架构: internals/architecture.md
- Common 公共函数: internals/common.md
- 基础设施: - 基础设施:
- Job Runtime 与 Server: internals/runtime.md - Job Runtime 与 Server: internals/runtime.md
- Job 产物: internals/outputs.md - Job 产物: internals/outputs.md

5
src/anyimage/__init__.py
View File

@@ -66,9 +66,6 @@ def register():
separator=True, separator=True,
) )
bpy.types.Scene.anyimage_settings = bpy.props.PointerProperty(type=AnyImageSettings) bpy.types.Scene.anyimage_settings = bpy.props.PointerProperty(type=AnyImageSettings)
bpy.types.Image.anyimage_upscale_source = bpy.props.PointerProperty(
type=bpy.types.Image
)
bpy.types.VIEW3D_MT_object_context_menu.prepend(draw_image_context_menu) bpy.types.VIEW3D_MT_object_context_menu.prepend(draw_image_context_menu)
bpy.types.OUTLINER_MT_object.prepend(draw_image_context_menu) bpy.types.OUTLINER_MT_object.prepend(draw_image_context_menu)
clipboard_image.register_keymaps() clipboard_image.register_keymaps()
@@ -88,7 +85,5 @@ def unregister():
bpy.types.OUTLINER_MT_object.remove(draw_image_context_menu) bpy.types.OUTLINER_MT_object.remove(draw_image_context_menu)
if hasattr(bpy.types.Scene, "anyimage_settings"): if hasattr(bpy.types.Scene, "anyimage_settings"):
del bpy.types.Scene.anyimage_settings del bpy.types.Scene.anyimage_settings
if hasattr(bpy.types.Image, "anyimage_upscale_source"):
del bpy.types.Image.anyimage_upscale_source
for class_type in reversed(CLASSES): for class_type in reversed(CLASSES):
bpy.utils.unregister_class(class_type) bpy.utils.unregister_class(class_type)

BIN
src/anyimage/assets/O_AnyImage.blend LFS
View File

Binary file not shown.

10
src/anyimage/menu.py
View File

@@ -1,6 +1,7 @@
import bpy import bpy
from .operators.common.image import is_image_empty from .operators.common.image_data import is_image_empty
from .operators.upscale import upscale_output_size
class AnyImageImageMenu(bpy.types.Menu): class AnyImageImageMenu(bpy.types.Menu):
@@ -53,11 +54,10 @@ def upscale_menu_label(context):
source_object = getattr(context, "object", None) source_object = getattr(context, "object", None)
if not is_image_empty(source_object): if not is_image_empty(source_object):
return "Upscale" return "Upscale"
width, height = ( output_size = upscale_output_size(source_object.data)
int(value) for value in getattr(source_object.data, "size", (0, 0)) if output_size is None:
)
if width <= 0 or height <= 0:
return "Upscale" return "Upscale"
width, height = output_size
return f"Upscale ({width} × {height})" return f"Upscale ({width} × {height})"

8
src/anyimage/operators/__init__.py
View File

@@ -1,8 +1,6 @@
from .remove_background import RemoveImageBackground, RunBackgroundRemoval from .remove_background import RemoveImageBackground, RunBackgroundRemoval
from .cutout_tool import ( from .cutout_tool import (
ActivateCutoutTool, CutoutRegionToShape,
ConvertCutoutShape,
GenerateCutoutShape,
SelectCutoutRegion, SelectCutoutRegion,
) )
from .debug import DebugBen2, DebugDa3, DebugMoge2, DebugUpscale from .debug import DebugBen2, DebugDa3, DebugMoge2, DebugUpscale
@@ -33,10 +31,8 @@ CLASSES = (
DebugBen2, DebugBen2,
DebugUpscale, DebugUpscale,
GenerateDepthPlane, GenerateDepthPlane,
GenerateCutoutShape, CutoutRegionToShape,
SelectCutoutRegion, SelectCutoutRegion,
ConvertCutoutShape,
ActivateCutoutTool,
RunBackgroundRemoval, RunBackgroundRemoval,
ClearModels, ClearModels,
*IMAGE_TOOL_CLASSES, *IMAGE_TOOL_CLASSES,

4
src/anyimage/operators/ai_setup.py
View File

@@ -8,7 +8,7 @@ from ..properties import (
model_download_size, model_download_size,
model_record, model_record,
) )
from .common.operator import HIDDEN, require_environment from .common.ai_job import require_environment
def format_download_size(size): def format_download_size(size):
@@ -128,7 +128,7 @@ class DownloadModel(JobOperatorBase, bpy.types.Operator):
bl_label = "Download Model" bl_label = "Download Model"
job_type = "download-model" job_type = "download-model"
model: bpy.props.StringProperty(options=HIDDEN) model: bpy.props.StringProperty(options={"HIDDEN", "SKIP_SAVE"})
def request(self, _context): def request(self, _context):
require_environment() require_environment()

2
src/anyimage/operators/common/__init__.py
View File

@@ -1 +1 @@
"""Shared Blender-side image, material, normal, and operator helpers.""" """Shared Blender-side image data, interaction, mesh result, and AI Job helpers."""

43
src/anyimage/operators/common/operator.py → src/anyimage/operators/common/ai_job.py
View File

@@ -1,4 +1,3 @@
import shutil
from pathlib import Path from pathlib import Path
import bpy import bpy
@@ -6,24 +5,22 @@ import bpy
from ...preferences import addon_preferences from ...preferences import addon_preferences
from ...properties import ai_ready, model_ready, model_record from ...properties import ai_ready, model_ready, model_record
from ...runtime import runtime from ...runtime import runtime
from .image import extract_frame_range
BEN2_IMAGE_SUFFIXES = {".jpg", ".jpeg", ".png", ".webp"} BEN2_IMAGE_SUFFIXES = {".jpg", ".jpeg", ".png", ".webp"}
BEN2_VIDEO_SUFFIXES = { BEN2_VIDEO_SUFFIXES = {
".mp4", ".mov", ".avi", ".mkv", ".webm", ".m4v", ".mts", ".mpg", ".mpeg" ".mp4", ".mov", ".avi", ".mkv", ".webm", ".m4v", ".mts", ".mpg", ".mpeg"
} }
HIDDEN = {"HIDDEN", "SKIP_SAVE"}
def input_path(value): def require_input_path(value):
path = Path(bpy.path.abspath(value)) path = Path(bpy.path.abspath(value))
if path.is_file() or path.is_dir(): if path.is_file() or path.is_dir():
return path return path
raise ValueError("Select a valid image, video, or frame directory") raise ValueError("Select a valid image, video, or frame directory")
def background_input_path(value): def require_background_input_path(value):
path = input_path(value) path = require_input_path(value)
if path.is_dir() or path.suffix.lower() in ( if path.is_dir() or path.suffix.lower() in (
BEN2_IMAGE_SUFFIXES | BEN2_VIDEO_SUFFIXES BEN2_IMAGE_SUFFIXES | BEN2_VIDEO_SUFFIXES
): ):
@@ -33,11 +30,6 @@ def background_input_path(value):
) )
def source_start_frame(source_object_name):
source_object = bpy.data.objects.get(source_object_name)
return extract_frame_range(source_object)[0] if source_object else 1
def production_device(): def production_device():
return getattr(addon_preferences(), "device", "AUTO") return getattr(addon_preferences(), "device", "AUTO")
@@ -62,35 +54,6 @@ def require_model(model_key):
return model return model
def cleanup_temporary_input(path, delete_input):
if not delete_input or not path:
return
temporary_path = Path(path)
try:
if temporary_path.is_dir():
shutil.rmtree(temporary_path, ignore_errors=True)
return
temporary_path.unlink(missing_ok=True)
temporary_path.parent.rmdir()
except OSError:
pass
def open_job_directory(directory):
directory = Path(directory)
if not directory.is_dir():
raise RuntimeError(f"Job directory does not exist: {directory}")
result = bpy.ops.wm.path_open(filepath=str(directory))
if "FINISHED" not in result:
raise RuntimeError(f"Unable to open job directory: {directory}")
def push_undo(message):
result = bpy.ops.ed.undo_push(message=message)
if "FINISHED" not in result:
raise RuntimeError(f"Unable to create the {message} Undo step")
def da3_parameters(settings, path, model, process_res, start_frame): def da3_parameters(settings, path, model, process_res, start_frame):
return { return {
"input": str(path), "input": str(path),

334
src/anyimage/operators/common/image.py → src/anyimage/operators/common/image_data.py
View File

@@ -5,9 +5,6 @@ from pathlib import Path
import bpy import bpy
DA3_PATCH_SIZE = 14
DA3_MIN_PROCESS_RES = 252
DA3_MAX_PROCESS_RES = 1008
IMAGE_NAME_SUFFIXES = ( IMAGE_NAME_SUFFIXES = (
".jpeg", ".jpeg",
".jpg", ".jpg",
@@ -44,6 +41,33 @@ def is_image_empty(candidate):
) )
def require_image_empty(object_name):
"""Return a named Image Empty or raise when it is no longer available."""
source_object = bpy.data.objects.get(object_name)
if not is_image_empty(source_object):
raise RuntimeError("The source Image Empty is no longer available")
return source_object
def image_pixels(image):
"""Read one Blender Image into a flat float32 pixel array."""
import numpy as np
pixels = np.empty(len(image.pixels), dtype=np.float32)
image.pixels.foreach_get(pixels)
return pixels
def image_rgba(image):
"""Read one Blender Image into a top-down RGBA float32 array."""
import numpy as np
width, height = (int(value) for value in image.size)
if width <= 0 or height <= 0:
raise ValueError("The source image has no readable dimensions")
return np.flipud(image_pixels(image).reshape((height, width, 4)))
def is_animated_image(image): def is_animated_image(image):
return getattr(image, "source", None) in {"MOVIE", "SEQUENCE"} return getattr(image, "source", None) in {"MOVIE", "SEQUENCE"}
@@ -95,6 +119,11 @@ def default_extract_frames(source_object):
return extract_frame_range(source_object)[1] return extract_frame_range(source_object)[1]
def source_start_frame(source_object_name):
source_object = bpy.data.objects.get(source_object_name)
return extract_frame_range(source_object)[0] if source_object else 1
def image_path(image): def image_path(image):
filepath_from_user = getattr(image, "filepath_from_user", None) filepath_from_user = getattr(image, "filepath_from_user", None)
filepath = ( filepath = (
@@ -113,14 +142,17 @@ def image_is_packed(image):
return bool(getattr(image, "packed_files", ())) return bool(getattr(image, "packed_files", ()))
def remove_temporary_input(path): def cleanup_image_input(path, temporary):
"""Remove a temporary input file or frame directory.""" """Remove a prepared input and its temporary container when requested."""
if not temporary or not path:
return
path = Path(path) path = Path(path)
try: try:
if path.is_dir(): if path.is_dir():
shutil.rmtree(path, ignore_errors=True) shutil.rmtree(path, ignore_errors=True)
else: else:
path.unlink(missing_ok=True) path.unlink(missing_ok=True)
path.parent.rmdir()
except OSError: except OSError:
pass pass
@@ -149,62 +181,10 @@ def normal_data_name(source_object):
return f"{image_base_name(source_object.data)}_normal.png" return f"{image_base_name(source_object.data)}_normal.png"
def copy_depth_image(source_object):
"""Return a renamed copy of the source image for use as the depth map."""
depth_image = source_object.data.copy()
depth_image.name = depth_data_name(source_object)
return depth_image
def color_data_name(source_object): def color_data_name(source_object):
return f"{image_base_name(source_object.data)}_color.png" return f"{image_base_name(source_object.data)}_color.png"
def material_data_name(source_object):
return image_base_name(source_object.data)
def scaled_process_resolution(image, scale_percent):
"""Return the scaled long edge clamped and aligned for DA3 inference."""
width, height = image.size
if width <= 0 or height <= 0:
raise ValueError("The source image has no readable dimensions")
if scale_percent < 1.0 or scale_percent > 100.0:
raise ValueError("Resolution Scale must be between 1% and 100%")
process_res = max(width, height) * scale_percent / 100.0
process_res = max(DA3_MIN_PROCESS_RES, process_res)
process_res = min(DA3_MAX_PROCESS_RES, process_res)
return int(process_res / DA3_PATCH_SIZE + 0.5) * DA3_PATCH_SIZE
def process_resolution_scale_property():
return bpy.props.FloatProperty(
name="Resolution Scale",
description=(
"Scale the source image long edge for DA3 processing; the result "
"is limited to 252-1008 px and aligned to a multiple of 14"
),
default=100.0,
min=1.0,
max=100.0,
step=100,
precision=0,
subtype="PERCENTAGE",
)
def draw_process_resolution(layout, owner, image):
process_res = scaled_process_resolution(
image,
owner.process_res_factor,
)
layout.prop(
owner,
"process_res_factor",
text=f"Resolution Scale ({process_res} px)",
)
def save_packed_image(image, _scene): def save_packed_image(image, _scene):
output_path = Path(tempfile.mkdtemp(prefix="anyimage-input-")) / "input.png" output_path = Path(tempfile.mkdtemp(prefix="anyimage-input-")) / "input.png"
original_file_format = image.file_format original_file_format = image.file_format
@@ -280,40 +260,42 @@ def prepare_image_input(image, scene):
return source_path, False return source_path, False
def analysis_source_image(image): def prepare_region_input(image, region_mask):
"""Return the oldest available source behind an upscaled image.""" """Write one cropped RGBA Region input owned by its temporary directory."""
current = image import numpy as np
visited = set()
while current is not None and id(current) not in visited:
visited.add(id(current))
source = getattr(current, "anyimage_upscale_source", None)
if source is None or source is current:
break
current = source
return current
left, top, right, bottom = region_mask.bounds
def prepare_image_analysis_inputs(image, scene): rgba = image_rgba(image)
"""Prepare display and analysis inputs, reusing one path when possible.""" cropped = rgba[top:bottom, left:right].copy()
source_path, delete_input = prepare_image_input(image, scene) if cropped.shape[:2] != region_mask.values.shape:
analysis_image = analysis_source_image(image) raise ValueError("The Region does not match the source image")
if analysis_image is image: cropped[:, :, 3] *= region_mask.values
return source_path, delete_input, source_path, False cropped[cropped[:, :, 3] <= 0.0, :3] = 0.0
try: output_directory = Path(tempfile.mkdtemp(prefix="anyimage-region-"))
analysis_path, delete_analysis_input = prepare_image_input( output_path = output_directory / "input.png"
analysis_image, temporary_image = bpy.data.images.new(
scene, "AnyImage Region Input",
) width=right - left,
except Exception: height=bottom - top,
if delete_input: alpha=True,
remove_temporary_input(source_path)
raise
return (
source_path,
delete_input,
analysis_path,
delete_analysis_input,
) )
original_format = temporary_image.file_format
try:
temporary_image.colorspace_settings.name = image.colorspace_settings.name
temporary_image.alpha_mode = "STRAIGHT"
temporary_image.pixels.foreach_set(np.flipud(cropped).ravel())
temporary_image.file_format = "PNG"
temporary_image.save(filepath=str(output_path), save_copy=True)
except (OSError, RuntimeError):
cleanup_image_input(output_path, True)
raise
finally:
temporary_image.file_format = original_format
bpy.data.images.remove(temporary_image, do_unlink=True)
if not output_path.is_file():
cleanup_image_input(output_path, True)
raise RuntimeError("Blender did not export the Region input")
return output_path
def configure_image_user( def configure_image_user(
@@ -388,31 +370,98 @@ def force_sequence_source(image):
return image return image
def create_sequence_image(name, frame_directory, frame_prefix="frame."): def load_normal_result_image(path, source_object):
"""Load a numbered frame directory as a SEQUENCE image datablock.""" """Load, name, configure, and pack a tangent normal image."""
frame_paths = sorted_frame_paths(frame_directory, frame_prefix) image = bpy.data.images.load(str(path), check_existing=False)
if not frame_paths: image.name = normal_data_name(source_object)
raise RuntimeError(f"No image frames found in: {frame_directory}") image.colorspace_settings.name = "Non-Color"
try: image.pack()
image = bpy.data.images.load(
str(frame_paths[0]),
check_existing=False,
)
force_sequence_source(image)
except (OSError, RuntimeError) as error:
if "image" in locals():
bpy.data.images.remove(image, do_unlink=True)
raise RuntimeError("Unable to load the frame sequence") from error
image.name = name
pack_sequence_if_possible(image, len(frame_paths))
return image return image
def create_pixel_result_image(source_image, pixels, output_size, suffix=""):
"""Create a packed Image datablock from edited RGBA pixels."""
output_width, output_height = (int(value) for value in output_size)
if output_width <= 0 or output_height <= 0:
raise ValueError("The edited image dimensions must be positive")
if len(pixels) != output_width * output_height * 4:
raise ValueError("The edited pixels do not match the image dimensions")
image = bpy.data.images.new(
f"{image_base_name(source_image)}{suffix}",
width=output_width,
height=output_height,
alpha=True,
)
try:
image.colorspace_settings.name = source_image.colorspace_settings.name
image.alpha_mode = "STRAIGHT"
image.pixels.foreach_set(pixels)
image.pack()
image.update()
except Exception:
bpy.data.images.remove(image, do_unlink=True)
raise
return image
def create_color_image(
def replace_empty_image(source_object, result_image, *, placement_bounds=None):
"""Replace one Empty image, retaining shared source images."""
source_image = source_object.data
source_name = source_image.name
original_fake_user = source_image.use_fake_user
image_user = getattr(source_object, "image_user", None)
frame_count = (
video_total_frames(result_image)
if is_animated_image(result_image)
else None
)
alignment = (
empty_display_alignment(
source_object,
tuple(source_image.size),
tuple(result_image.size),
placement_bounds,
)
if placement_bounds is not None
else None
)
if frame_count is not None:
(
original_frame_start,
_original_frame_offset,
_original_frame_duration,
original_use_cyclic,
) = image_user_settings(source_object)
source_image.use_fake_user = True
try:
source_object.data = result_image
if frame_count is not None:
configure_image_user(
image_user,
frame_count,
frame_start=original_frame_start,
use_cyclic=original_use_cyclic,
)
if alignment is not None:
source_object.empty_display_size = alignment[0]
source_object.empty_image_offset = alignment[1]
except Exception:
source_object.data = source_image
source_image.use_fake_user = original_fake_user
raise
source_image.use_fake_user = original_fake_user
if not original_fake_user and int(source_image.users) == 0:
bpy.data.images.remove(source_image, do_unlink=True)
result_image.name = source_name
return result_image
def load_color_result_image(
source_object, source_object,
output_directory, output_directory,
assign_to_source=True,
*, *,
frame_prefix="image.", frame_prefix="image.",
): ):
@@ -439,39 +488,20 @@ def create_color_image(
bpy.data.images.remove(result_image, do_unlink=True) bpy.data.images.remove(result_image, do_unlink=True)
raise RuntimeError("Unable to load the color result") from error raise RuntimeError("Unable to load the color result") from error
original_fake_user = source_image.use_fake_user
try: try:
result_name = color_data_name(source_object) result_name = color_data_name(source_object)
if animated: if animated:
pack_sequence_if_possible(result_image, len(color_paths)) pack_sequence_if_possible(result_image, len(color_paths))
else: else:
result_image.pack() result_image.pack()
if assign_to_source: result_image.name = result_name
image_user = getattr(source_object, "image_user", None) except (OSError, RuntimeError, TypeError, ValueError) as error:
(
original_frame_start,
_original_frame_offset,
_original_frame_duration,
original_use_cyclic,
) = image_user_settings(source_object)
source_image.use_fake_user = True
source_object.data = result_image
if animated:
configure_image_user(
image_user,
len(color_paths),
frame_start=original_frame_start,
use_cyclic=original_use_cyclic,
)
except (OSError, RuntimeError) as error:
source_image.use_fake_user = original_fake_user
bpy.data.images.remove(result_image, do_unlink=True) bpy.data.images.remove(result_image, do_unlink=True)
raise RuntimeError("Unable to prepare the color result") from error raise RuntimeError("Unable to prepare the color result") from error
result_image.name = result_name
return result_image return result_image
def plane_bounds(source_object): def image_empty_bounds(source_object):
image_width, image_height = source_object.data.size image_width, image_height = source_object.data.size
if image_width <= 0 or image_height <= 0: if image_width <= 0 or image_height <= 0:
raise ValueError("The source image has no readable dimensions") raise ValueError("The source image has no readable dimensions")
@@ -494,3 +524,47 @@ def plane_bounds(source_object):
y_min, y_min,
y_min + height, y_min + height,
) )
def empty_display_alignment(
source_object,
source_size,
output_size,
placement_bounds,
):
"""Return the display size and offset for an edited Image Empty."""
source_width, source_height = source_size
output_width, output_height = output_size
left, top, right, bottom = placement_bounds
if (
source_width <= 0
or source_height <= 0
or output_width <= 0
or output_height <= 0
or right <= left
or bottom <= top
):
raise ValueError("The edited image placement is invalid")
x_min, x_max, y_min, y_max = image_empty_bounds(source_object)
local_width = x_max - x_min
local_height = y_max - y_min
desired_x_min = x_min + left * local_width / source_width
desired_x_max = x_min + right * local_width / source_width
desired_y_max = y_max - top * local_height / source_height
desired_y_min = y_max - bottom * local_height / source_height
desired_width = desired_x_max - desired_x_min
desired_height = desired_y_max - desired_y_min
display_size = max(desired_width, desired_height)
aspect = output_width / output_height
if aspect >= 1.0:
display_width = display_size
display_height = display_size / aspect
else:
display_width = display_size * aspect
display_height = display_size
center_x = (desired_x_min + desired_x_max) * 0.5
center_y = (desired_y_min + desired_y_max) * 0.5
return display_size, (
center_x / display_width - 0.5,
center_y / display_height - 0.5,
)

567
src/anyimage/operators/common/image_edit.py
View File

@@ -1,567 +0,0 @@
import json
import math
import bpy
from ...properties import (
BEN2_MODEL_KEY,
ai_ready,
ai_setup_label,
ai_status,
)
from .operator import production_device, require_environment, require_model
from .image import image_base_name, is_image_empty, plane_bounds
class ImageEditWarning(ValueError):
pass
def report_image_edit_exception(operator, error):
level = "WARNING" if isinstance(error, ImageEditWarning) else "ERROR"
operator.report({level}, str(error))
def image_edit_poll(context):
return context.area is not None and context.area.type == "VIEW_3D"
def _restore_selection(context, active_object, selected_objects):
bpy.ops.object.select_all(action="DESELECT")
for selected in selected_objects:
if bpy.data.objects.get(selected.name) is not None:
selected.select_set(True)
if (
active_object is not None
and bpy.data.objects.get(active_object.name) is not None
):
context.view_layer.objects.active = active_object
def resolve_image_edit_click(context, event, report):
"""Resolve the first click into source selection or image editing."""
active_object = context.active_object
selected_objects = tuple(context.selected_objects)
try:
bpy.ops.view3d.select(
location=(event.mouse_region_x, event.mouse_region_y),
extend=False,
deselect_all=False,
)
except RuntimeError:
pass
clicked_object = context.active_object
if is_image_empty(clicked_object) and clicked_object != active_object:
return clicked_object, False
if is_image_empty(clicked_object):
return clicked_object, True
if is_image_empty(active_object):
_restore_selection(context, active_object, selected_objects)
return active_object, True
report({"WARNING"}, "Click an Image Empty to use an Image tool")
return None, False
def active_view3d_tool_id(context):
tools = getattr(getattr(context, "workspace", None), "tools", None)
from_mode = getattr(tools, "from_space_view3d_mode", None)
if not callable(from_mode):
return None
try:
tool = from_mode(getattr(context, "mode", "OBJECT"), create=False)
except (RuntimeError, TypeError):
return None
return getattr(tool, "idname", None)
def polygon_bounds(polygon):
import numpy as np
points = np.asarray(polygon, dtype=np.float64)
if points.ndim != 2 or points.shape[0] < 3 or points.shape[1] != 2:
raise ValueError("The selection does not define a valid image region")
if not np.isfinite(points).all():
raise ValueError("The selection contains invalid image coordinates")
left = int(math.floor(float(points[:, 0].min())))
top = int(math.floor(float(points[:, 1].min())))
right = int(math.ceil(float(points[:, 0].max())))
bottom = int(math.ceil(float(points[:, 1].max())))
if right <= left or bottom <= top:
raise ValueError("The selection does not define a valid image region")
return left, top, right, bottom
def crop_pixels_to_alpha(pixels, bounds):
"""Crop Blender-order RGBA pixels to their nonzero Alpha bounds."""
import numpy as np
left, top, right, bottom = bounds
width = right - left
height = bottom - top
rgba = np.flipud(
np.asarray(pixels, dtype=np.float32).reshape((height, width, 4))
)
visible_y, visible_x = np.nonzero(rgba[:, :, 3] > 0.0)
if not len(visible_x):
raise ImageEditWarning("The edit contains no visible image pixels")
content_left = int(visible_x.min())
content_top = int(visible_y.min())
content_right = int(visible_x.max()) + 1
content_bottom = int(visible_y.max()) + 1
output = rgba[content_top:content_bottom, content_left:content_right].copy()
return np.flipud(output).ravel(), (
left + content_left,
top + content_top,
left + content_right,
top + content_bottom,
)
def empty_display_alignment(
source_object, source_size, output_size, placement_bounds
):
"""Calculate image canvas placement without changing the Empty transform."""
source_width, source_height = (float(value) for value in source_size)
output_width, output_height = (float(value) for value in output_size)
left, top, right, bottom = (float(value) for value in placement_bounds)
x_min, x_max, y_min, y_max = plane_bounds(source_object)
local_width = x_max - x_min
local_height = y_max - y_min
desired_x_min = x_min + left * local_width / source_width
desired_x_max = x_min + right * local_width / source_width
desired_y_max = y_max - top * local_height / source_height
desired_y_min = y_max - bottom * local_height / source_height
desired_width = desired_x_max - desired_x_min
desired_height = desired_y_max - desired_y_min
display_size = max(desired_width, desired_height)
aspect = output_width / output_height
if aspect >= 1.0:
display_width = display_size
display_height = display_size / aspect
else:
display_width = display_size * aspect
display_height = display_size
center_x = (desired_x_min + desired_x_max) * 0.5
center_y = (desired_y_min + desired_y_max) * 0.5
return display_size, (
center_x / display_width - 0.5,
center_y / display_height - 0.5,
)
def write_image_pixels(image, pixels):
image.alpha_mode = "STRAIGHT"
image.pixels.foreach_set(pixels)
image.pack()
image.update()
def replace_empty_image_pixels(
source_object, pixels, output_size, placement_bounds, suffix=""
):
source_image = source_object.data
source_size = tuple(source_image.size)
output_width, output_height = (int(value) for value in output_size)
if output_width <= 0 or output_height <= 0:
raise ValueError("The edited image dimensions must be positive")
if len(pixels) != output_width * output_height * 4:
raise ValueError("The edited pixels do not match the image dimensions")
display_size, image_offset = empty_display_alignment(
source_object,
source_size,
(output_width, output_height),
placement_bounds,
)
image = bpy.data.images.new(
f"{image_base_name(source_image)}{suffix}",
width=output_width,
height=output_height,
alpha=True,
)
try:
image.colorspace_settings.name = source_image.colorspace_settings.name
write_image_pixels(image, pixels)
except Exception:
bpy.data.images.remove(image, do_unlink=True)
raise
source_name = source_image.name
source_object.data = image
if not source_image.use_fake_user and int(source_image.users) == 0:
bpy.data.images.remove(source_image)
image.name = source_name
source_object.empty_display_size = display_size
source_object.empty_image_offset = image_offset
return image
def image_mask_refine_request(input_path, selection_data):
require_environment()
require_model(BEN2_MODEL_KEY)
polygon = json.loads(selection_data)
polygon_bounds(polygon)
return {
"input": str(input_path),
"polygon": polygon,
"model": BEN2_MODEL_KEY,
"device": production_device().lower(),
}
def refine_selection_setting_name(operator):
if getattr(operator, "active_tool_id", None) == "anyimage.cutout_lasso":
return "cutout_refine_selection"
return {
"BOX": "image_box_refine_selection",
"LASSO": "image_lasso_refine_selection",
}.get(operator.gesture)
def apply_image_mask_settings(operator, context):
settings = context.scene.anyimage_settings
setting_name = refine_selection_setting_name(operator)
operator.refine_selection = (
setting_name is not None
and ai_ready()
and getattr(settings, setting_name)
)
def draw_image_mask_operator_settings(layout, operator):
if operator.gesture not in {"BOX", "LASSO"}:
layout.prop(operator, "invert")
return
status = ai_status()
row = layout.row()
row.enabled = status["ready"]
row.prop(operator, "refine_selection")
layout.prop(operator, "invert")
if not status["ready"]:
layout.operator(
"anyimage.setup_ai_environment",
text=ai_setup_label(status),
icon="IMPORT",
)
def draw_image_mask_settings(context, layout, setting_name):
settings = context.scene.anyimage_settings
status = ai_status()
row = layout.row()
row.enabled = status["ready"]
row.prop(settings, setting_name)
if not status["ready"]:
layout.operator(
"anyimage.setup_ai_environment",
text=ai_setup_label(status),
icon="IMPORT",
)
def dashed_line_segments(vertices, dash_length=4.0, gap_length=4.0):
if dash_length <= 0.0 or gap_length < 0.0:
raise ValueError("Dash length must be positive and gap length cannot be negative")
pattern_length = dash_length + gap_length
segments = []
distance_offset = 0.0
for start, end in zip(vertices, vertices[1:]):
start_x, start_y = (float(value) for value in start)
end_x, end_y = (float(value) for value in end)
segment_length = math.hypot(end_x - start_x, end_y - start_y)
if segment_length <= 0.0:
continue
direction_x = (end_x - start_x) / segment_length
direction_y = (end_y - start_y) / segment_length
position = 0.0
while position < segment_length:
phase = (distance_offset + position) % pattern_length
if phase < dash_length:
run_length = min(dash_length - phase, segment_length - position)
segments.extend(
(
(
start_x + direction_x * position,
start_y + direction_y * position,
),
(
start_x + direction_x * (position + run_length),
start_y + direction_y * (position + run_length),
),
)
)
else:
run_length = min(pattern_length - phase, segment_length - position)
position += run_length
distance_offset += segment_length
return tuple(segments)
def draw_dashed_line(
vertices,
color=(1.0, 1.0, 1.0, 0.9),
segments=None,
):
import gpu
from gpu_extras.batch import batch_for_shader
if segments is None:
segments = dashed_line_segments(vertices)
if not segments:
return
shader = gpu.shader.from_builtin("UNIFORM_COLOR")
batch = batch_for_shader(shader, "LINES", {"pos": segments})
gpu.state.blend_set("ALPHA")
gpu.state.line_width_set(1.0)
shader.bind()
shader.uniform_float("color", color)
batch.draw(shader)
gpu.state.blend_set("NONE")
def _signed_area(vertices):
return sum(
start[0] * end[1] - end[0] * start[1]
for start, end in zip(vertices, (*vertices[1:], vertices[0]))
) * 0.5
def _triangle_cross(start, middle, end):
return (middle[0] - start[0]) * (end[1] - start[1]) - (
middle[1] - start[1]
) * (end[0] - start[0])
def _point_in_triangle(point, start, middle, end, orientation, epsilon):
return all(
orientation * cross >= -epsilon
for cross in (
_triangle_cross(start, middle, point),
_triangle_cross(middle, end, point),
_triangle_cross(end, start, point),
)
)
def _triangulated_polygon_vertices_python(vertices):
vertices = tuple(tuple(float(value) for value in vertex) for vertex in vertices)
if len(vertices) < 3:
return ()
area = _signed_area(vertices)
if abs(area) <= 1e-8:
return ()
orientation = 1.0 if area > 0.0 else -1.0
remaining = list(range(len(vertices)))
triangles = []
epsilon = 1e-8
while len(remaining) > 3:
for position, current in enumerate(remaining):
previous = remaining[position - 1]
following = remaining[(position + 1) % len(remaining)]
start = vertices[previous]
middle = vertices[current]
end = vertices[following]
if orientation * _triangle_cross(start, middle, end) <= epsilon:
continue
if any(
_point_in_triangle(
vertices[index],
start,
middle,
end,
orientation,
epsilon,
)
for index in remaining
if index not in {previous, current, following}
):
continue
triangles.extend((start, middle, end))
del remaining[position]
break
else:
return ()
triangles.extend(vertices[index] for index in remaining)
return tuple(triangles)
def triangulated_polygon_vertices(vertices):
vertices = tuple(tuple(float(value) for value in vertex) for vertex in vertices)
try:
import mathutils
points = [
mathutils.Vector((vertex[0], vertex[1], 0.0)) for vertex in vertices
]
triangles = mathutils.geometry.tessellate_polygon([points])
return tuple(
(float(point[0]), float(point[1]))
for triangle in triangles
for point in triangle
)
except (AttributeError, ImportError, RuntimeError, TypeError, ValueError):
return _triangulated_polygon_vertices_python(vertices)
def scanline_fill_triangles(vertices, fill_rule='EVEN_ODD', step=2.0):
vertices = tuple(tuple(float(value) for value in vertex) for vertex in vertices)
if len(vertices) < 3:
return ()
if fill_rule not in {'EVEN_ODD', 'NONZERO'}:
raise ValueError(f'Unknown polygon fill rule: {fill_rule}')
minimum_y = min(vertex[1] for vertex in vertices)
maximum_y = max(vertex[1] for vertex in vertices)
band_bottom = minimum_y
triangles = []
edges = tuple(zip(vertices, (*vertices[1:], vertices[0])))
while band_bottom < maximum_y:
band_top = min(band_bottom + step, maximum_y)
sample_y = (band_bottom + band_top) * 0.5
events = []
for start, end in edges:
if (start[1] <= sample_y < end[1]) or (
end[1] <= sample_y < start[1]
):
factor = (sample_y - start[1]) / (end[1] - start[1])
events.append(
(
start[0] + (end[0] - start[0]) * factor,
1 if end[1] > start[1] else -1,
)
)
events.sort(key=lambda event: event[0])
intervals = []
winding = 0
interval_start = None
index = 0
while index < len(events):
x = events[index][0]
delta = 0
count = 0
while index < len(events) and abs(events[index][0] - x) <= 1e-8:
delta += events[index][1]
count += 1
index += 1
was_inside = winding != 0
if fill_rule == 'EVEN_ODD':
if count % 2:
winding = 0 if was_inside else 1
else:
winding += delta
is_inside = winding != 0
if not was_inside and is_inside:
interval_start = x
elif was_inside and not is_inside and interval_start is not None:
intervals.append((interval_start, x))
interval_start = None
for left, right in intervals:
if right - left <= 1e-8:
continue
triangles.extend(
(
(left, band_bottom),
(right, band_bottom),
(right, band_top),
(left, band_bottom),
(right, band_top),
(left, band_top),
)
)
band_bottom = band_top
return tuple(triangles)
def _point_on_segment(point, start, end, epsilon=1e-8):
return (
min(start[0], end[0]) - epsilon
<= point[0]
<= max(start[0], end[0]) + epsilon
and min(start[1], end[1]) - epsilon
<= point[1]
<= max(start[1], end[1]) + epsilon
)
def _segments_intersect(first_start, first_end, second_start, second_end):
crosses = (
_triangle_cross(first_start, first_end, second_start),
_triangle_cross(first_start, first_end, second_end),
_triangle_cross(second_start, second_end, first_start),
_triangle_cross(second_start, second_end, first_end),
)
if (crosses[0] > 1e-8) != (crosses[1] > 1e-8) and (
crosses[2] > 1e-8
) != (crosses[3] > 1e-8):
return True
return any(
abs(cross) <= 1e-8 and _point_on_segment(point, start, end)
for cross, point, start, end in (
(crosses[0], second_start, first_start, first_end),
(crosses[1], second_end, first_start, first_end),
(crosses[2], first_start, second_start, second_end),
(crosses[3], first_end, second_start, second_end),
)
)
def polygon_self_intersects(vertices, cell_size=32.0):
vertices = tuple(tuple(float(value) for value in vertex) for vertex in vertices)
if len(vertices) < 4:
return False
edges = tuple(zip(vertices, (*vertices[1:], vertices[0])))
cells = {}
last_edge = len(edges) - 1
for edge_index, (start, end) in enumerate(edges):
left = math.floor(min(start[0], end[0]) / cell_size)
right = math.floor(max(start[0], end[0]) / cell_size)
bottom = math.floor(min(start[1], end[1]) / cell_size)
top = math.floor(max(start[1], end[1]) / cell_size)
edge_cells = tuple(
(x, y)
for x in range(left, right + 1)
for y in range(bottom, top + 1)
)
candidates = {
candidate
for cell in edge_cells
for candidate in cells.get(cell, ())
}
for candidate in candidates:
if abs(edge_index - candidate) <= 1 or {
edge_index,
candidate,
} == {0, last_edge}:
continue
if _segments_intersect(start, end, *edges[candidate]):
return True
for cell in edge_cells:
cells.setdefault(cell, []).append(edge_index)
return False
def preview_fill_triangles(vertices, fill_rule='EVEN_ODD'):
if not polygon_self_intersects(vertices):
triangles = triangulated_polygon_vertices(vertices)
if triangles:
return triangles
return scanline_fill_triangles(vertices, fill_rule)
def draw_polygon_fill(
vertices,
triangles=None,
color=(1.0, 1.0, 1.0, 0.16),
):
import gpu
from gpu_extras.batch import batch_for_shader
if triangles is None:
triangles = triangulated_polygon_vertices(vertices)
if not triangles:
return
shader = gpu.shader.from_builtin("UNIFORM_COLOR")
batch = batch_for_shader(shader, "TRIS", {"pos": triangles})
gpu.state.blend_set("ALPHA")
shader.bind()
shader.uniform_float("color", color)
batch.draw(shader)
gpu.state.blend_set("NONE")

420
src/anyimage/operators/common/image_gesture.py
View File

@@ -1,420 +0,0 @@
import json
import math
import bpy
from .image import is_image_empty, plane_bounds, video_total_frames
from .image_edit import (
active_view3d_tool_id,
apply_image_mask_settings,
dashed_line_segments,
draw_dashed_line,
draw_image_mask_operator_settings,
draw_polygon_fill,
image_edit_poll,
polygon_bounds,
preview_fill_triangles,
report_image_edit_exception,
resolve_image_edit_click,
)
from .image_interaction import (
deserialize_matrix,
drawing_in_region,
screen_path_to_image_pixels,
serialize_matrix,
)
GESTURE_TOOL_IDS = {
"BOX": "anyimage.image_box",
"LASSO": "anyimage.image_lasso",
"LINE": "anyimage.image_line",
"POLYLINE": "anyimage.image_polyline",
}
LINE_SNAP_RADIANS = math.radians(5.0)
POLYLINE_CLOSE_RADIUS = 8.0
LASSO_POINT_DISTANCE = 2.0
LASSO_SIMPLIFY_TOLERANCE = 0.75
def append_lasso_point(path, point):
point = (float(point[0]), float(point[1]))
if not path:
path.append(point)
return True
if math.dist(point, path[-1]) < LASSO_POINT_DISTANCE:
return False
if len(path) >= 2:
anchor = path[-2]
previous = path[-1]
segment_x = point[0] - anchor[0]
segment_y = point[1] - anchor[1]
segment_length_squared = segment_x * segment_x + segment_y * segment_y
if segment_length_squared > 0.0:
factor = (
(previous[0] - anchor[0]) * segment_x
+ (previous[1] - anchor[1]) * segment_y
) / segment_length_squared
projected = (
anchor[0] + segment_x * factor,
anchor[1] + segment_y * factor,
)
forward = (
(previous[0] - anchor[0]) * (point[0] - previous[0])
+ (previous[1] - anchor[1]) * (point[1] - previous[1])
)
if (
0.0 <= factor <= 1.0
and forward >= 0.0
and math.dist(previous, projected) <= LASSO_SIMPLIFY_TOLERANCE
):
path[-1] = point
return True
path.append(point)
return True
def gesture_property():
return bpy.props.EnumProperty(
items=(
("BOX", "Box", "Edit with a rectangular gesture"),
("LASSO", "Lasso", "Edit with a freehand gesture"),
("LINE", "Line", "Edit one side of a line"),
("POLYLINE", "Polyline", "Edit with a clicked polygon"),
),
default="BOX",
options={"HIDDEN", "SKIP_SAVE"},
)
def box_screen_path(start, end):
x0, y0 = start
x1, y1 = end
if abs(x1 - x0) < 2.0 or abs(y1 - y0) < 2.0:
return ()
return ((x0, y0), (x1, y0), (x1, y1), (x0, y1))
def snapped_line_endpoint(start, end, snap):
if not snap:
return tuple(float(value) for value in end)
delta_x = float(end[0]) - float(start[0])
delta_y = float(end[1]) - float(start[1])
length = math.hypot(delta_x, delta_y)
if length <= 0.0:
return tuple(float(value) for value in end)
angle = round(math.atan2(delta_y, delta_x) / LINE_SNAP_RADIANS)
angle *= LINE_SNAP_RADIANS
return (
float(start[0]) + math.cos(angle) * length,
float(start[1]) + math.sin(angle) * length,
)
def line_half_plane_polygon(start, end, rectangle_size):
"""Clip a rectangle to the left side of one directed line."""
width, height = (float(value) for value in rectangle_size)
if math.dist(start, end) < 2.0:
return ()
vertices = [(0.0, 0.0), (width, 0.0), (width, height), (0.0, height)]
def side(point):
return (end[0] - start[0]) * (point[1] - start[1]) - (end[1] - start[1]) * (
point[0] - start[0]
)
result = []
previous = vertices[-1]
previous_side = side(previous)
for current in vertices:
current_side = side(current)
if (current_side >= 0.0) != (previous_side >= 0.0):
factor = previous_side / (previous_side - current_side)
result.append(
(
previous[0] + (current[0] - previous[0]) * factor,
previous[1] + (current[1] - previous[1]) * factor,
)
)
if current_side >= 0.0:
result.append(current)
previous = current
previous_side = current_side
return tuple(result)
def image_line_half_plane_polygon(line, image_size):
"""Keep the image-space half-plane shown by the screen-space preview."""
if len(line) != 2:
raise ValueError("Drag a longer line across the image plane")
return line_half_plane_polygon(line[1], line[0], image_size)
class ImageMaskGesture:
active_tool_id = None
preview_fill_rule = 'EVEN_ODD'
_path = None
_cursor = None
_handle = None
_fill_polygon = None
_fill_triangles = ()
_outline_polygon = None
_outline_segments = ()
_area_pointer = 0
_region_pointer = 0
_region_size = (0, 0)
def draw(self, _context):
draw_image_mask_operator_settings(self.layout, self)
@classmethod
def poll(cls, context):
return image_edit_poll(context)
def _screen_polygon(self):
if self.gesture == "BOX":
return box_screen_path(self._path[0], self._cursor)
if self.gesture == "LASSO":
return tuple(self._path)
if self.gesture == "LINE":
return line_half_plane_polygon(
self._path[0], self._cursor, self._region_size
)
vertices = list(self._path)
if self._cursor is not None:
vertices.append(self._cursor)
return tuple(vertices)
def _draw_selection(self):
if not self._path or not drawing_in_region(
self._area_pointer, self._region_pointer
):
return
polygon = self._screen_polygon()
if polygon != self._outline_polygon:
self._outline_polygon = polygon
if self.gesture == 'LINE':
outline = (self._path[0], self._cursor)
else:
outline = (*polygon, polygon[0]) if len(polygon) > 1 else ()
self._outline_segments = dashed_line_segments(outline)
if len(polygon) >= 3:
if polygon != self._fill_polygon:
self._fill_polygon = polygon
self._fill_triangles = preview_fill_triangles(
polygon, self.preview_fill_rule
)
draw_polygon_fill(
polygon,
self._fill_triangles,
(1.0, 0.25, 0.2, 0.16) if self.invert else (1.0, 1.0, 1.0, 0.16),
)
if self.gesture == "LINE":
draw_dashed_line((), segments=self._outline_segments)
return
if len(polygon) > 1:
draw_dashed_line((), segments=self._outline_segments)
if (
self.gesture == "POLYLINE"
and len(self._path) >= 3
and self._cursor is not None
and math.dist(self._cursor, self._path[0]) <= POLYLINE_CLOSE_RADIUS
):
self._draw_polyline_close_indicator()
def _draw_polyline_close_indicator(self):
import gpu
from gpu_extras.batch import batch_for_shader
center_x, center_y = self._path[0]
vertices = tuple(
(
center_x + math.cos(index * math.tau / 24.0) * POLYLINE_CLOSE_RADIUS,
center_y + math.sin(index * math.tau / 24.0) * POLYLINE_CLOSE_RADIUS,
)
for index in range(25)
)
shader = gpu.shader.from_builtin("UNIFORM_COLOR")
batch = batch_for_shader(shader, "LINE_STRIP", {"pos": vertices})
gpu.state.blend_set("ALPHA")
gpu.state.line_width_set(2.0)
shader.bind()
shader.uniform_float("color", (1.0, 1.0, 1.0, 0.95))
batch.draw(shader)
gpu.state.line_width_set(1.0)
gpu.state.blend_set("NONE")
def _set_status(self, context):
invert = "On" if self.invert else "Off"
if self.gesture == "POLYLINE":
action = "click points; click start or Enter to finish; Backspace removes"
elif self.gesture == "LINE":
action = "drag line; Ctrl snaps 5°; release to edit"
else:
action = "drag and release to edit"
context.workspace.status_text_set(
f"Image {self.gesture.title()}: {action}; F toggles Invert ({invert}); "
"RMB or Esc cancels"
)
def _finish(self, context):
if self._handle is not None:
bpy.types.SpaceView3D.draw_handler_remove(self._handle, "WINDOW")
self._handle = None
context.workspace.status_text_set(None)
if context.area is not None:
context.area.tag_redraw()
def cancel(self, context):
self._finish(context)
def invoke(self, context, event):
source_object, should_edit = resolve_image_edit_click(
context, event, self.report
)
if not should_edit:
return {"FINISHED"} if source_object is not None else {"CANCELLED"}
if source_object is None:
return {"CANCELLED"}
if video_total_frames(source_object.data) > 1:
self.report({"WARNING"}, "Image tools only support a still image")
return {"CANCELLED"}
apply_image_mask_settings(self, context)
self.invert = False
self.source_object_name = source_object.name
self.source_matrix_data = serialize_matrix(source_object.matrix_world)
self.selection_data = ""
point = (float(event.mouse_region_x), float(event.mouse_region_y))
self._path = [point]
self._cursor = point
self._fill_polygon = None
self._fill_triangles = ()
self._outline_polygon = None
self._outline_segments = ()
self._area_pointer = context.area.as_pointer()
self._region_pointer = context.region.as_pointer()
self._region_size = (context.region.width, context.region.height)
self._handle = bpy.types.SpaceView3D.draw_handler_add(
self._draw_selection, (), "WINDOW", "POST_PIXEL"
)
self._set_status(context)
context.window_manager.modal_handler_add(self)
context.area.tag_redraw()
return {"RUNNING_MODAL"}
def _complete(self, context):
source_object = bpy.data.objects.get(self.source_object_name)
if not is_image_empty(source_object):
raise RuntimeError("The source Image Empty is no longer available")
matrix = deserialize_matrix(self.source_matrix_data)
if self.gesture == "LINE":
line = screen_path_to_image_pixels(
(self._path[0], self._cursor),
context.region,
context.region_data,
matrix,
plane_bounds(source_object),
tuple(source_object.data.size),
)
polygon = image_line_half_plane_polygon(
line, tuple(source_object.data.size)
)
else:
path = self._screen_polygon()
if len(path) < 3:
self._finish(context)
return {"CANCELLED"}
polygon = screen_path_to_image_pixels(
path,
context.region,
context.region_data,
matrix,
plane_bounds(source_object),
tuple(source_object.data.size),
)
polygon_bounds(polygon)
self.selection_data = json.dumps(polygon, separators=(",", ":"))
self._finish(context)
return self.execute(context)
def modal(self, context, event):
active_tool = active_view3d_tool_id(context)
expected_tool = self.active_tool_id or GESTURE_TOOL_IDS[self.gesture]
if active_tool is not None and active_tool != expected_tool:
self._finish(context)
return {"CANCELLED"}
if event.type in {"ESC", "RIGHTMOUSE"} and event.value == "PRESS":
self._finish(context)
return {"CANCELLED"}
if event.type == "F" and event.value == "PRESS":
self.invert = not self.invert
self._set_status(context)
context.area.tag_redraw()
return {"RUNNING_MODAL"}
if self.gesture == "POLYLINE":
return self._modal_polyline(context, event)
if event.type == "MOUSEMOVE":
point = (float(event.mouse_region_x), float(event.mouse_region_y))
if self.gesture == "LASSO":
append_lasso_point(self._path, point)
self._cursor = point
elif self.gesture == "LINE":
self._cursor = snapped_line_endpoint(self._path[0], point, event.ctrl)
else:
self._cursor = point
context.area.tag_redraw()
return {"RUNNING_MODAL"}
if event.type == "LEFTMOUSE" and event.value == "RELEASE":
point = (float(event.mouse_region_x), float(event.mouse_region_y))
if self.gesture == "LASSO":
append_lasso_point(self._path, point)
self._cursor = point
elif self.gesture == "LINE":
self._cursor = snapped_line_endpoint(self._path[0], point, event.ctrl)
if math.dist(self._path[0], self._cursor) < 2.0:
self._finish(context)
return {"CANCELLED"}
else:
self._cursor = point
try:
return self._complete(context)
except (RuntimeError, TypeError, ValueError) as error:
report_image_edit_exception(self, error)
self._finish(context)
return {"CANCELLED"}
return {"PASS_THROUGH"}
def _modal_polyline(self, context, event):
if event.type == "BACK_SPACE" and event.value == "PRESS":
if len(self._path) > 1:
self._path.pop()
context.area.tag_redraw()
return {"RUNNING_MODAL"}
if event.type in {"RET", "NUMPAD_ENTER"} and event.value == "PRESS":
return self._confirm_polyline(context)
if event.type == "MOUSEMOVE":
self._cursor = (
float(event.mouse_region_x),
float(event.mouse_region_y),
)
context.area.tag_redraw()
return {"RUNNING_MODAL"}
if event.type == "LEFTMOUSE" and event.value == "PRESS":
point = (float(event.mouse_region_x), float(event.mouse_region_y))
if (
len(self._path) >= 3
and math.dist(point, self._path[0]) <= POLYLINE_CLOSE_RADIUS
):
return self._confirm_polyline(context)
if math.dist(point, self._path[-1]) >= 2.0:
self._path.append(point)
context.area.tag_redraw()
return {"RUNNING_MODAL"}
return {"PASS_THROUGH"}
def _confirm_polyline(self, context):
self._cursor = self._path[0]
try:
return self._complete(context)
except (RuntimeError, TypeError, ValueError) as error:
report_image_edit_exception(self, error)
self._finish(context)
return {"CANCELLED"}

838
src/anyimage/operators/common/image_interaction.py
View File

@@ -3,6 +3,14 @@ import math
import bpy import bpy
from ...properties import ai_ready, ai_setup_label, ai_status
from .image_data import (
image_empty_bounds,
is_image_empty,
require_image_empty,
video_total_frames,
)
from .image_region import ImageEditWarning, RegionPath
MIN_PATH_DISTANCE = 2.0 MIN_PATH_DISTANCE = 2.0
@@ -18,8 +26,8 @@ def simplify_screen_path(path):
def screen_path_to_image_pixels( def screen_path_to_image_pixels(
path, path,
region, view_region,
region_data, view3d,
matrix_world, matrix_world,
local_bounds, local_bounds,
image_size, image_size,
@@ -34,13 +42,13 @@ def screen_path_to_image_pixels(
pixels = [] pixels = []
for point in simplify_screen_path(path): for point in simplify_screen_path(path):
ray_origin = inverse @ view3d_utils.region_2d_to_origin_3d( ray_origin = inverse @ view3d_utils.region_2d_to_origin_3d(
region, view_region,
region_data, view3d,
point, point,
) )
ray_direction = inverse.to_3x3() @ view3d_utils.region_2d_to_vector_3d( ray_direction = inverse.to_3x3() @ view3d_utils.region_2d_to_vector_3d(
region, view_region,
region_data, view3d,
point, point,
) )
if abs(ray_direction.z) <= 1e-10: if abs(ray_direction.z) <= 1e-10:
@@ -72,10 +80,6 @@ def deserialize_matrix(data):
return Matrix(tuple(tuple(values[row * 4 : row * 4 + 4]) for row in range(4))) return Matrix(tuple(tuple(values[row * 4 : row * 4 + 4]) for row in range(4)))
def hidden_string_property():
return bpy.props.StringProperty(options={"HIDDEN", "SKIP_SAVE"})
def drawing_in_region(area_pointer, region_pointer): def drawing_in_region(area_pointer, region_pointer):
area = getattr(bpy.context, "area", None) area = getattr(bpy.context, "area", None)
region = getattr(bpy.context, "region", None) region = getattr(bpy.context, "region", None)
@@ -85,3 +89,817 @@ def drawing_in_region(area_pointer, region_pointer):
and area.as_pointer() == area_pointer and area.as_pointer() == area_pointer
and region.as_pointer() == region_pointer and region.as_pointer() == region_pointer
) )
def report_image_edit_exception(operator, error):
level = "WARNING" if isinstance(error, ImageEditWarning) else "ERROR"
operator.report({level}, str(error))
def image_edit_poll(context):
return context.area is not None and context.area.type == "VIEW_3D"
def _restore_selection(context, active_object, selected_objects):
bpy.ops.object.select_all(action="DESELECT")
for selected in selected_objects:
if bpy.data.objects.get(selected.name) is not None:
selected.select_set(True)
if (
active_object is not None
and bpy.data.objects.get(active_object.name) is not None
):
context.view_layer.objects.active = active_object
def resolve_image_edit_click(context, event, report):
"""Resolve the first click into source selection or image editing."""
active_object = context.active_object
selected_objects = tuple(context.selected_objects)
try:
bpy.ops.view3d.select(
location=(event.mouse_region_x, event.mouse_region_y),
extend=False,
deselect_all=False,
)
except RuntimeError:
pass
clicked_object = context.active_object
if is_image_empty(clicked_object) and clicked_object != active_object:
return clicked_object, False
if is_image_empty(clicked_object):
return clicked_object, True
if is_image_empty(active_object):
_restore_selection(context, active_object, selected_objects)
return active_object, True
report({"WARNING"}, "Click an Image Empty to use an Image tool")
return None, False
def active_view3d_tool_id(context):
tools = getattr(getattr(context, "workspace", None), "tools", None)
from_mode = getattr(tools, "from_space_view3d_mode", None)
if not callable(from_mode):
return None
try:
tool = from_mode(getattr(context, "mode", "OBJECT"), create=False)
except (RuntimeError, TypeError):
return None
return getattr(tool, "idname", None)
def refine_selection_setting_name(operator):
if getattr(operator, "active_tool_id", None) == "anyimage.cutout_lasso":
return "cutout_refine_selection"
return {
"BOX": "image_box_refine_selection",
"LASSO": "image_lasso_refine_selection",
}.get(operator.gesture)
def apply_region_settings(operator, context):
settings = context.scene.anyimage_settings
setting_name = refine_selection_setting_name(operator)
operator.refine_selection = (
setting_name is not None
and ai_ready()
and getattr(settings, setting_name)
)
def draw_region_operator_settings(layout, operator):
if operator.gesture not in {"BOX", "LASSO"}:
layout.prop(operator, "invert")
return
status = ai_status()
row = layout.row()
row.enabled = status["ready"]
row.prop(operator, "refine_selection")
layout.prop(operator, "invert")
if not status["ready"]:
layout.operator(
"anyimage.setup_ai_environment",
text=ai_setup_label(status),
icon="IMPORT",
)
def draw_region_settings(context, layout, setting_name):
settings = context.scene.anyimage_settings
status = ai_status()
row = layout.row()
row.enabled = status["ready"]
row.prop(settings, setting_name)
if not status["ready"]:
layout.operator(
"anyimage.setup_ai_environment",
text=ai_setup_label(status),
icon="IMPORT",
)
def dashed_line_segments(vertices, dash_length=4.0, gap_length=4.0):
if dash_length <= 0.0 or gap_length < 0.0:
raise ValueError("Dash length must be positive and gap length cannot be negative")
pattern_length = dash_length + gap_length
segments = []
distance_offset = 0.0
for start, end in zip(vertices, vertices[1:]):
start_x, start_y = (float(value) for value in start)
end_x, end_y = (float(value) for value in end)
segment_length = math.hypot(end_x - start_x, end_y - start_y)
if segment_length <= 0.0:
continue
direction_x = (end_x - start_x) / segment_length
direction_y = (end_y - start_y) / segment_length
position = 0.0
while position < segment_length:
phase = (distance_offset + position) % pattern_length
if phase < dash_length:
run_length = min(dash_length - phase, segment_length - position)
segments.extend(
(
(
start_x + direction_x * position,
start_y + direction_y * position,
),
(
start_x + direction_x * (position + run_length),
start_y + direction_y * (position + run_length),
),
)
)
else:
run_length = min(pattern_length - phase, segment_length - position)
position += run_length
distance_offset += segment_length
return tuple(segments)
def draw_dashed_line(
vertices,
color=(1.0, 1.0, 1.0, 0.9),
segments=None,
):
import gpu
from gpu_extras.batch import batch_for_shader
if segments is None:
segments = dashed_line_segments(vertices)
if not segments:
return
shader = gpu.shader.from_builtin("UNIFORM_COLOR")
batch = batch_for_shader(shader, "LINES", {"pos": segments})
gpu.state.blend_set("ALPHA")
gpu.state.line_width_set(1.0)
shader.bind()
shader.uniform_float("color", color)
batch.draw(shader)
gpu.state.blend_set("NONE")
def _signed_area(vertices):
return sum(
start[0] * end[1] - end[0] * start[1]
for start, end in zip(vertices, (*vertices[1:], vertices[0]))
) * 0.5
def _triangle_cross(start, middle, end):
return (middle[0] - start[0]) * (end[1] - start[1]) - (
middle[1] - start[1]
) * (end[0] - start[0])
def _point_in_triangle(point, start, middle, end, orientation, epsilon):
return all(
orientation * cross >= -epsilon
for cross in (
_triangle_cross(start, middle, point),
_triangle_cross(middle, end, point),
_triangle_cross(end, start, point),
)
)
def _triangulated_polygon_vertices_python(vertices):
vertices = tuple(tuple(float(value) for value in vertex) for vertex in vertices)
if len(vertices) < 3:
return ()
area = _signed_area(vertices)
if abs(area) <= 1e-8:
return ()
orientation = 1.0 if area > 0.0 else -1.0
remaining = list(range(len(vertices)))
triangles = []
epsilon = 1e-8
while len(remaining) > 3:
for position, current in enumerate(remaining):
previous = remaining[position - 1]
following = remaining[(position + 1) % len(remaining)]
start = vertices[previous]
middle = vertices[current]
end = vertices[following]
if orientation * _triangle_cross(start, middle, end) <= epsilon:
continue
if any(
_point_in_triangle(
vertices[index],
start,
middle,
end,
orientation,
epsilon,
)
for index in remaining
if index not in {previous, current, following}
):
continue
triangles.extend((start, middle, end))
del remaining[position]
break
else:
return ()
triangles.extend(vertices[index] for index in remaining)
return tuple(triangles)
def triangulated_polygon_vertices(vertices):
vertices = tuple(tuple(float(value) for value in vertex) for vertex in vertices)
try:
import mathutils
points = [
mathutils.Vector((vertex[0], vertex[1], 0.0)) for vertex in vertices
]
triangles = mathutils.geometry.tessellate_polygon([points])
return tuple(
(float(point[0]), float(point[1]))
for triangle in triangles
for point in triangle
)
except (AttributeError, ImportError, RuntimeError, TypeError, ValueError):
return _triangulated_polygon_vertices_python(vertices)
def scanline_fill_triangles(vertices, fill_rule='EVEN_ODD', step=2.0):
vertices = tuple(tuple(float(value) for value in vertex) for vertex in vertices)
if len(vertices) < 3:
return ()
if fill_rule not in {'EVEN_ODD', 'NONZERO'}:
raise ValueError(f'Unknown polygon fill rule: {fill_rule}')
minimum_y = min(vertex[1] for vertex in vertices)
maximum_y = max(vertex[1] for vertex in vertices)
band_bottom = minimum_y
triangles = []
edges = tuple(zip(vertices, (*vertices[1:], vertices[0])))
while band_bottom < maximum_y:
band_top = min(band_bottom + step, maximum_y)
sample_y = (band_bottom + band_top) * 0.5
events = []
for start, end in edges:
if (start[1] <= sample_y < end[1]) or (
end[1] <= sample_y < start[1]
):
factor = (sample_y - start[1]) / (end[1] - start[1])
events.append(
(
start[0] + (end[0] - start[0]) * factor,
1 if end[1] > start[1] else -1,
)
)
events.sort(key=lambda event: event[0])
intervals = []
winding = 0
interval_start = None
index = 0
while index < len(events):
x = events[index][0]
delta = 0
count = 0
while index < len(events) and abs(events[index][0] - x) <= 1e-8:
delta += events[index][1]
count += 1
index += 1
was_inside = winding != 0
if fill_rule == 'EVEN_ODD':
if count % 2:
winding = 0 if was_inside else 1
else:
winding += delta
is_inside = winding != 0
if not was_inside and is_inside:
interval_start = x
elif was_inside and not is_inside and interval_start is not None:
intervals.append((interval_start, x))
interval_start = None
for left, right in intervals:
if right - left <= 1e-8:
continue
triangles.extend(
(
(left, band_bottom),
(right, band_bottom),
(right, band_top),
(left, band_bottom),
(right, band_top),
(left, band_top),
)
)
band_bottom = band_top
return tuple(triangles)
def _point_on_segment(point, start, end, epsilon=1e-8):
return (
min(start[0], end[0]) - epsilon
<= point[0]
<= max(start[0], end[0]) + epsilon
and min(start[1], end[1]) - epsilon
<= point[1]
<= max(start[1], end[1]) + epsilon
)
def _segments_intersect(first_start, first_end, second_start, second_end):
crosses = (
_triangle_cross(first_start, first_end, second_start),
_triangle_cross(first_start, first_end, second_end),
_triangle_cross(second_start, second_end, first_start),
_triangle_cross(second_start, second_end, first_end),
)
if (crosses[0] > 1e-8) != (crosses[1] > 1e-8) and (
crosses[2] > 1e-8
) != (crosses[3] > 1e-8):
return True
return any(
abs(cross) <= 1e-8 and _point_on_segment(point, start, end)
for cross, point, start, end in (
(crosses[0], second_start, first_start, first_end),
(crosses[1], second_end, first_start, first_end),
(crosses[2], first_start, second_start, second_end),
(crosses[3], first_end, second_start, second_end),
)
)
def polygon_self_intersects(vertices, cell_size=32.0):
vertices = tuple(tuple(float(value) for value in vertex) for vertex in vertices)
if len(vertices) < 4:
return False
edges = tuple(zip(vertices, (*vertices[1:], vertices[0])))
cells = {}
last_edge = len(edges) - 1
for edge_index, (start, end) in enumerate(edges):
left = math.floor(min(start[0], end[0]) / cell_size)
right = math.floor(max(start[0], end[0]) / cell_size)
bottom = math.floor(min(start[1], end[1]) / cell_size)
top = math.floor(max(start[1], end[1]) / cell_size)
edge_cells = tuple(
(x, y)
for x in range(left, right + 1)
for y in range(bottom, top + 1)
)
candidates = {
candidate
for cell in edge_cells
for candidate in cells.get(cell, ())
}
for candidate in candidates:
if abs(edge_index - candidate) <= 1 or {
edge_index,
candidate,
} == {0, last_edge}:
continue
if _segments_intersect(start, end, *edges[candidate]):
return True
for cell in edge_cells:
cells.setdefault(cell, []).append(edge_index)
return False
def preview_fill_triangles(vertices, fill_rule='EVEN_ODD'):
if not polygon_self_intersects(vertices):
triangles = triangulated_polygon_vertices(vertices)
if triangles:
return triangles
return scanline_fill_triangles(vertices, fill_rule)
def draw_polygon_fill(
vertices,
triangles=None,
color=(1.0, 1.0, 1.0, 0.16),
):
import gpu
from gpu_extras.batch import batch_for_shader
if triangles is None:
triangles = triangulated_polygon_vertices(vertices)
if not triangles:
return
shader = gpu.shader.from_builtin("UNIFORM_COLOR")
batch = batch_for_shader(shader, "TRIS", {"pos": triangles})
gpu.state.blend_set("ALPHA")
shader.bind()
shader.uniform_float("color", color)
batch.draw(shader)
gpu.state.blend_set("NONE")
GESTURE_TOOL_IDS = {
"BOX": "anyimage.image_box",
"LASSO": "anyimage.image_lasso",
"LINE": "anyimage.image_line",
"POLYLINE": "anyimage.image_polyline",
}
LINE_SNAP_RADIANS = math.radians(5.0)
POLYLINE_CLOSE_RADIUS = 8.0
LASSO_POINT_DISTANCE = 2.0
LASSO_SIMPLIFY_TOLERANCE = 0.75
def append_lasso_point(path, point):
point = (float(point[0]), float(point[1]))
if not path:
path.append(point)
return True
if math.dist(point, path[-1]) < LASSO_POINT_DISTANCE:
return False
if len(path) >= 2:
anchor = path[-2]
previous = path[-1]
segment_x = point[0] - anchor[0]
segment_y = point[1] - anchor[1]
segment_length_squared = segment_x * segment_x + segment_y * segment_y
if segment_length_squared > 0.0:
factor = (
(previous[0] - anchor[0]) * segment_x
+ (previous[1] - anchor[1]) * segment_y
) / segment_length_squared
projected = (
anchor[0] + segment_x * factor,
anchor[1] + segment_y * factor,
)
forward = (
(previous[0] - anchor[0]) * (point[0] - previous[0])
+ (previous[1] - anchor[1]) * (point[1] - previous[1])
)
if (
0.0 <= factor <= 1.0
and forward >= 0.0
and math.dist(previous, projected) <= LASSO_SIMPLIFY_TOLERANCE
):
path[-1] = point
return True
path.append(point)
return True
def gesture_property():
return bpy.props.EnumProperty(
items=(
("BOX", "Box", "Edit with a rectangular gesture"),
("LASSO", "Lasso", "Edit with a freehand gesture"),
("LINE", "Line", "Edit one side of a line"),
("POLYLINE", "Polyline", "Edit with a clicked polygon"),
),
default="BOX",
options={"HIDDEN", "SKIP_SAVE"},
)
def box_screen_path(start, end):
x0, y0 = start
x1, y1 = end
if abs(x1 - x0) < 2.0 or abs(y1 - y0) < 2.0:
return ()
return ((x0, y0), (x1, y0), (x1, y1), (x0, y1))
def snapped_line_endpoint(start, end, snap):
if not snap:
return tuple(float(value) for value in end)
delta_x = float(end[0]) - float(start[0])
delta_y = float(end[1]) - float(start[1])
length = math.hypot(delta_x, delta_y)
if length <= 0.0:
return tuple(float(value) for value in end)
angle = round(math.atan2(delta_y, delta_x) / LINE_SNAP_RADIANS)
angle *= LINE_SNAP_RADIANS
return (
float(start[0]) + math.cos(angle) * length,
float(start[1]) + math.sin(angle) * length,
)
def line_half_plane_polygon(start, end, rectangle_size):
"""Clip a rectangle to the left side of one directed line."""
width, height = (float(value) for value in rectangle_size)
if math.dist(start, end) < 2.0:
return ()
vertices = [(0.0, 0.0), (width, 0.0), (width, height), (0.0, height)]
def side(point):
return (end[0] - start[0]) * (point[1] - start[1]) - (end[1] - start[1]) * (
point[0] - start[0]
)
result = []
previous = vertices[-1]
previous_side = side(previous)
for current in vertices:
current_side = side(current)
if (current_side >= 0.0) != (previous_side >= 0.0):
factor = previous_side / (previous_side - current_side)
result.append(
(
previous[0] + (current[0] - previous[0]) * factor,
previous[1] + (current[1] - previous[1]) * factor,
)
)
if current_side >= 0.0:
result.append(current)
previous = current
previous_side = current_side
return tuple(result)
def image_line_half_plane_polygon(line, image_size):
"""Keep the image-space half-plane shown by the screen-space preview."""
if len(line) != 2:
raise ValueError("Drag a longer line across the image plane")
return line_half_plane_polygon(line[1], line[0], image_size)
class ImageRegionGesture:
active_tool_id = None
preview_fill_rule = 'EVEN_ODD'
_path = None
_cursor = None
_handle = None
_fill_polygon = None
_fill_triangles = ()
_outline_polygon = None
_outline_segments = ()
_area_pointer = 0
_region_pointer = 0
_region_size = (0, 0)
def draw(self, _context):
draw_region_operator_settings(self.layout, self)
@classmethod
def poll(cls, context):
return image_edit_poll(context)
def _screen_path(self):
if self.gesture == "BOX":
return box_screen_path(self._path[0], self._cursor)
if self.gesture == "LASSO":
return tuple(self._path)
if self.gesture == "LINE":
return line_half_plane_polygon(
self._path[0], self._cursor, self._region_size
)
vertices = list(self._path)
if self._cursor is not None:
vertices.append(self._cursor)
return tuple(vertices)
def _draw_region(self):
if not self._path or not drawing_in_region(
self._area_pointer, self._region_pointer
):
return
polygon = self._screen_path()
if polygon != self._outline_polygon:
self._outline_polygon = polygon
if self.gesture == 'LINE':
outline = (self._path[0], self._cursor)
else:
outline = (*polygon, polygon[0]) if len(polygon) > 1 else ()
self._outline_segments = dashed_line_segments(outline)
if len(polygon) >= 3:
if polygon != self._fill_polygon:
self._fill_polygon = polygon
self._fill_triangles = preview_fill_triangles(
polygon, self.preview_fill_rule
)
draw_polygon_fill(
polygon,
self._fill_triangles,
(1.0, 0.25, 0.2, 0.16) if self.invert else (1.0, 1.0, 1.0, 0.16),
)
if self.gesture == "LINE":
draw_dashed_line((), segments=self._outline_segments)
return
if len(polygon) > 1:
draw_dashed_line((), segments=self._outline_segments)
if (
self.gesture == "POLYLINE"
and len(self._path) >= 3
and self._cursor is not None
and math.dist(self._cursor, self._path[0]) <= POLYLINE_CLOSE_RADIUS
):
self._draw_polyline_close_indicator()
def _draw_polyline_close_indicator(self):
import gpu
from gpu_extras.batch import batch_for_shader
center_x, center_y = self._path[0]
vertices = tuple(
(
center_x + math.cos(index * math.tau / 24.0) * POLYLINE_CLOSE_RADIUS,
center_y + math.sin(index * math.tau / 24.0) * POLYLINE_CLOSE_RADIUS,
)
for index in range(25)
)
shader = gpu.shader.from_builtin("UNIFORM_COLOR")
batch = batch_for_shader(shader, "LINE_STRIP", {"pos": vertices})
gpu.state.blend_set("ALPHA")
gpu.state.line_width_set(2.0)
shader.bind()
shader.uniform_float("color", (1.0, 1.0, 1.0, 0.95))
batch.draw(shader)
gpu.state.line_width_set(1.0)
gpu.state.blend_set("NONE")
def _set_status(self, context):
invert = "On" if self.invert else "Off"
if self.gesture == "POLYLINE":
action = "click points; click start or Enter to finish; Backspace removes"
elif self.gesture == "LINE":
action = "drag line; Ctrl snaps 5°; release to edit"
else:
action = "drag and release to edit"
context.workspace.status_text_set(
f"Image {self.gesture.title()}: {action}; F toggles Invert ({invert}); "
"RMB or Esc cancels"
)
def _finish(self, context):
if self._handle is not None:
bpy.types.SpaceView3D.draw_handler_remove(self._handle, "WINDOW")
self._handle = None
context.workspace.status_text_set(None)
if context.area is not None:
context.area.tag_redraw()
def cancel(self, context):
self._finish(context)
def invoke(self, context, event):
source_object, should_edit = resolve_image_edit_click(
context, event, self.report
)
if not should_edit:
return {"FINISHED"} if source_object is not None else {"CANCELLED"}
if source_object is None:
return {"CANCELLED"}
if video_total_frames(source_object.data) > 1:
self.report({"WARNING"}, "Image tools only support a still image")
return {"CANCELLED"}
apply_region_settings(self, context)
self.invert = False
self.source_object_name = source_object.name
self.source_matrix_data = serialize_matrix(source_object.matrix_world)
self.region_path_json = ""
point = (float(event.mouse_region_x), float(event.mouse_region_y))
self._path = [point]
self._cursor = point
self._fill_polygon = None
self._fill_triangles = ()
self._outline_polygon = None
self._outline_segments = ()
self._area_pointer = context.area.as_pointer()
self._region_pointer = context.region.as_pointer()
self._region_size = (context.region.width, context.region.height)
self._handle = bpy.types.SpaceView3D.draw_handler_add(
self._draw_region, (), "WINDOW", "POST_PIXEL"
)
self._set_status(context)
context.window_manager.modal_handler_add(self)
context.area.tag_redraw()
return {"RUNNING_MODAL"}
def _complete(self, context):
source_object = require_image_empty(self.source_object_name)
matrix = deserialize_matrix(self.source_matrix_data)
if self.gesture == "LINE":
line = screen_path_to_image_pixels(
(self._path[0], self._cursor),
context.region,
context.region_data,
matrix,
image_empty_bounds(source_object),
tuple(source_object.data.size),
)
region_path = image_line_half_plane_polygon(
line, tuple(source_object.data.size)
)
else:
path = self._screen_path()
if len(path) < 3:
self._finish(context)
return {"CANCELLED"}
region_path = screen_path_to_image_pixels(
path,
context.region,
context.region_data,
matrix,
image_empty_bounds(source_object),
tuple(source_object.data.size),
)
self.region_path_json = RegionPath(
points=region_path,
invert=self.invert,
fill_rule=self.preview_fill_rule,
).to_json()
self._finish(context)
return self.execute(context)
def modal(self, context, event):
active_tool = active_view3d_tool_id(context)
expected_tool = self.active_tool_id or GESTURE_TOOL_IDS[self.gesture]
if active_tool is not None and active_tool != expected_tool:
self._finish(context)
return {"CANCELLED"}
if event.type in {"ESC", "RIGHTMOUSE"} and event.value == "PRESS":
self._finish(context)
return {"CANCELLED"}
if event.type == "F" and event.value == "PRESS":
self.invert = not self.invert
self._set_status(context)
context.area.tag_redraw()
return {"RUNNING_MODAL"}
if self.gesture == "POLYLINE":
return self._modal_polyline(context, event)
if event.type == "MOUSEMOVE":
point = (float(event.mouse_region_x), float(event.mouse_region_y))
if self.gesture == "LASSO":
append_lasso_point(self._path, point)
self._cursor = point
elif self.gesture == "LINE":
self._cursor = snapped_line_endpoint(self._path[0], point, event.ctrl)
else:
self._cursor = point
context.area.tag_redraw()
return {"RUNNING_MODAL"}
if event.type == "LEFTMOUSE" and event.value == "RELEASE":
point = (float(event.mouse_region_x), float(event.mouse_region_y))
if self.gesture == "LASSO":
append_lasso_point(self._path, point)
self._cursor = point
elif self.gesture == "LINE":
self._cursor = snapped_line_endpoint(self._path[0], point, event.ctrl)
if math.dist(self._path[0], self._cursor) < 2.0:
self._finish(context)
return {"CANCELLED"}
else:
self._cursor = point
try:
return self._complete(context)
except (RuntimeError, TypeError, ValueError) as error:
report_image_edit_exception(self, error)
self._finish(context)
return {"CANCELLED"}
return {"PASS_THROUGH"}
def _modal_polyline(self, context, event):
if event.type == "BACK_SPACE" and event.value == "PRESS":
if len(self._path) > 1:
self._path.pop()
context.area.tag_redraw()
return {"RUNNING_MODAL"}
if event.type in {"RET", "NUMPAD_ENTER"} and event.value == "PRESS":
return self._confirm_polyline(context)
if event.type == "MOUSEMOVE":
self._cursor = (
float(event.mouse_region_x),
float(event.mouse_region_y),
)
context.area.tag_redraw()
return {"RUNNING_MODAL"}
if event.type == "LEFTMOUSE" and event.value == "PRESS":
point = (float(event.mouse_region_x), float(event.mouse_region_y))
if (
len(self._path) >= 3
and math.dist(point, self._path[0]) <= POLYLINE_CLOSE_RADIUS
):
return self._confirm_polyline(context)
if math.dist(point, self._path[-1]) >= 2.0:
self._path.append(point)
context.area.tag_redraw()
return {"RUNNING_MODAL"}
return {"PASS_THROUGH"}
def _confirm_polyline(self, context):
self._cursor = self._path[0]
try:
return self._complete(context)
except (RuntimeError, TypeError, ValueError) as error:
report_image_edit_exception(self, error)
self._finish(context)
return {"CANCELLED"}

141
src/anyimage/operators/common/image_mask.py
View File

@@ -1,141 +0,0 @@
from .image_edit import (
ImageEditWarning,
crop_pixels_to_alpha,
polygon_bounds,
replace_empty_image_pixels,
)
def polygon_mask(image_size, polygon, fill_rule="EVEN_ODD"):
import numpy as np
width, height = (int(value) for value in image_size)
if width <= 0 or height <= 0:
raise ValueError("The source image has no readable dimensions")
if fill_rule not in {"EVEN_ODD", "NONZERO"}:
raise ValueError(f"Unknown polygon fill rule: {fill_rule}")
left, top, right, bottom = polygon_bounds(polygon)
left = max(left, 0)
top = max(top, 0)
right = min(right, width)
bottom = min(bottom, height)
mask = np.zeros((height, width), dtype=bool)
if right <= left or bottom <= top:
return mask
points = np.asarray(polygon, dtype=np.float64)
x = np.arange(left, right, dtype=np.float64) + 0.5
for row_start in range(top, bottom, 128):
row_end = min(row_start + 128, bottom)
y = np.arange(row_start, row_end, dtype=np.float64) + 0.5
state = np.zeros(
(len(y), len(x)),
dtype=bool if fill_rule == "EVEN_ODD" else np.int16,
)
previous = points[-1]
for current in points:
crosses = (current[1] > y) != (previous[1] > y)
edge_x = (previous[0] - current[0]) * (y - current[1]) / (
previous[1] - current[1] + 1e-300
) + current[0]
selected = crosses[:, None] & (x[None, :] < edge_x[:, None])
if fill_rule == "EVEN_ODD":
state ^= selected
else:
winding = 1 if current[1] > previous[1] else -1
state += selected * winding
previous = current
mask[row_start:row_end, left:right] = state != 0
return mask
def refined_mask_from_pixels(mask_pixels, mask_size, bounds, source_size):
import numpy as np
mask_width, mask_height = (int(value) for value in mask_size)
source_width, source_height = (int(value) for value in source_size)
local = np.flipud(
np.asarray(mask_pixels, dtype=np.float32).reshape((mask_height, mask_width, 4))
)[:, :, 3]
result = np.zeros((source_height, source_width), dtype=np.float32)
left, top, right, bottom = (int(value) for value in bounds)
source_left = max(left, 0)
source_top = max(top, 0)
source_right = min(right, source_width)
source_bottom = min(bottom, source_height)
if source_right <= source_left or source_bottom <= source_top:
return result
result[source_top:source_bottom, source_left:source_right] = local[
source_top - top : source_bottom - top,
source_left - left : source_right - left,
]
return result
def visible_selection_mask(
source_alpha, selection_mask, invert=False, alpha_threshold=0.0
):
"""Restrict a selection to source pixels above an Alpha threshold."""
import numpy as np
alpha = np.asarray(source_alpha)
mask = np.asarray(selection_mask, dtype=np.float32)
if mask.shape != alpha.shape:
raise ValueError("The selection does not match the source image")
mask = np.clip(mask, 0.0, 1.0)
if invert:
mask = 1.0 - mask
return np.where(alpha > float(alpha_threshold), mask, 0.0)
def edit_image_mask_pixels(source_pixels, source_size, selection_mask, invert=False):
"""Apply a mask while preserving source RGB, then tightly crop the canvas."""
import numpy as np
width, height = (int(value) for value in source_size)
rgba = np.flipud(
np.asarray(source_pixels, dtype=np.float32).reshape((height, width, 4))
)
mask = visible_selection_mask(rgba[:, :, 3], selection_mask, invert)
rgba[:, :, 3] *= mask
return crop_pixels_to_alpha(np.flipud(rgba).ravel(), (0, 0, width, height))
def edit_image_object(source_object, selection_mask, invert=False):
import numpy as np
image = source_object.data
source_size = tuple(image.size)
source_pixels = np.empty(len(image.pixels), dtype=np.float32)
image.pixels.foreach_get(source_pixels)
pixels, crop_bounds = edit_image_mask_pixels(
source_pixels, source_size, selection_mask, invert
)
left, top, right, bottom = crop_bounds
return replace_empty_image_pixels(
source_object,
pixels,
(right - left, bottom - top),
crop_bounds,
)
def selected_image_rgba(
source_object, polygon, invert=False, fill_rule="EVEN_ODD"
):
import numpy as np
image = source_object.data
width, height = (int(value) for value in image.size)
pixels = np.empty(len(image.pixels), dtype=np.float32)
image.pixels.foreach_get(pixels)
rgba = np.flipud(pixels.reshape((height, width, 4)))
selection = visible_selection_mask(
rgba[:, :, 3],
polygon_mask((width, height), polygon, fill_rule),
invert,
)
rgba[:, :, 3] *= selection
if not np.any(rgba[:, :, 3] > 0.0):
raise ImageEditWarning("The selection contains no visible image pixels")
return rgba

230
src/anyimage/operators/common/image_region.py Normal file
View File

@@ -0,0 +1,230 @@
import json
import math
from dataclasses import dataclass
class ImageEditWarning(ValueError):
pass
@dataclass(frozen=True)
class RegionPath:
points: tuple[tuple[float, float], ...]
invert: bool = False
fill_rule: str = "EVEN_ODD"
def __post_init__(self):
points = tuple(
(float(point[0]), float(point[1]))
for point in self.points
)
region_path_bounds(points)
if self.fill_rule not in {"EVEN_ODD", "NONZERO"}:
raise ValueError(f"Unknown Region Path fill rule: {self.fill_rule}")
object.__setattr__(self, "points", points)
object.__setattr__(self, "invert", bool(self.invert))
def to_json(self):
return json.dumps(
{
"points": self.points,
"invert": self.invert,
"fill_rule": self.fill_rule,
},
separators=(",", ":"),
)
@classmethod
def from_json(cls, value):
data = json.loads(value)
if not isinstance(data, dict):
raise ValueError("The Region Path data is invalid")
return cls(
points=tuple(data.get("points", ())),
invert=bool(data.get("invert", False)),
fill_rule=str(data.get("fill_rule", "EVEN_ODD")),
)
@dataclass(frozen=True)
class RegionMask:
values: object
bounds: tuple[int, int, int, int]
def __post_init__(self):
import numpy as np
values = np.asarray(self.values, dtype=np.float32)
bounds = tuple(int(value) for value in self.bounds)
if values.ndim != 2:
raise ValueError("The region mask must be two-dimensional")
if len(bounds) != 4:
raise ValueError("The region bounds are invalid")
left, top, right, bottom = bounds
if right <= left or bottom <= top:
raise ValueError("The region bounds are empty")
if values.shape != (bottom - top, right - left):
raise ValueError("The region mask does not match its bounds")
if not np.isfinite(values).all():
raise ValueError("The region mask contains invalid values")
object.__setattr__(self, "values", np.clip(values, 0.0, 1.0))
object.__setattr__(self, "bounds", bounds)
def full_values(self, image_size):
import numpy as np
width, height = (int(value) for value in image_size)
result = np.zeros((height, width), dtype=np.float32)
left, top, right, bottom = self.bounds
source_left = max(left, 0)
source_top = max(top, 0)
source_right = min(right, width)
source_bottom = min(bottom, height)
if source_right <= source_left or source_bottom <= source_top:
return result
result[source_top:source_bottom, source_left:source_right] = self.values[
source_top - top : source_bottom - top,
source_left - left : source_right - left,
]
return result
def region_path_bounds(points):
import numpy as np
points = np.asarray(points, dtype=np.float64)
if points.ndim != 2 or points.shape[0] < 3 or points.shape[1] != 2:
raise ValueError("The region does not define a valid image path")
if not np.isfinite(points).all():
raise ValueError("The region contains invalid image coordinates")
left = int(math.floor(float(points[:, 0].min())))
top = int(math.floor(float(points[:, 1].min())))
right = int(math.ceil(float(points[:, 0].max())))
bottom = int(math.ceil(float(points[:, 1].max())))
if right <= left or bottom <= top:
raise ValueError("The region does not define a valid image area")
return left, top, right, bottom
def _rasterize_path(image_size, points, fill_rule):
import numpy as np
width, height = (int(value) for value in image_size)
if width <= 0 or height <= 0:
raise ValueError("The source image has no readable dimensions")
left, top, right, bottom = region_path_bounds(points)
left = max(left, 0)
top = max(top, 0)
right = min(right, width)
bottom = min(bottom, height)
mask = np.zeros((height, width), dtype=bool)
if right <= left or bottom <= top:
return mask
points = np.asarray(points, dtype=np.float64)
x = np.arange(left, right, dtype=np.float64) + 0.5
for row_start in range(top, bottom, 128):
row_end = min(row_start + 128, bottom)
y = np.arange(row_start, row_end, dtype=np.float64) + 0.5
state = np.zeros(
(len(y), len(x)),
dtype=bool if fill_rule == "EVEN_ODD" else np.int16,
)
previous = points[-1]
for current in points:
crosses = (current[1] > y) != (previous[1] > y)
edge_x = (previous[0] - current[0]) * (y - current[1]) / (
previous[1] - current[1] + 1e-300
) + current[0]
selected = crosses[:, None] & (x[None, :] < edge_x[:, None])
if fill_rule == "EVEN_ODD":
state ^= selected
else:
winding = 1 if current[1] > previous[1] else -1
state += selected * winding
previous = current
mask[row_start:row_end, left:right] = state != 0
return mask
def _antialias_mask(mask):
import numpy as np
values = np.asarray(mask, dtype=np.float32)
kernel = np.asarray((0.029, 0.235, 0.472, 0.235, 0.029), dtype=np.float32)
radius = len(kernel) // 2
horizontal = np.zeros_like(values)
padded = np.pad(values, ((0, 0), (radius, radius)), mode="edge")
for offset, weight in enumerate(kernel):
horizontal += padded[:, offset : offset + values.shape[1]] * weight
result = np.zeros_like(values)
padded = np.pad(horizontal, ((radius, radius), (0, 0)), mode="edge")
for offset, weight in enumerate(kernel):
result += padded[offset : offset + values.shape[0]] * weight
return result
def rasterize_region_path(
image_size,
region_path,
*,
source_alpha=None,
alpha_threshold=0.0,
antialias=False,
):
import numpy as np
width, height = (int(value) for value in image_size)
mask = _rasterize_path(
(width, height),
region_path.points,
region_path.fill_rule,
)
if antialias:
mask = _antialias_mask(mask)
else:
mask = mask.astype(np.float32)
if region_path.invert:
mask = 1.0 - mask
if source_alpha is not None:
alpha = np.asarray(source_alpha)
if alpha.shape != (height, width):
raise ValueError("The source Alpha does not match the image region")
mask = np.where(alpha > float(alpha_threshold), mask, 0.0)
visible_y, visible_x = np.nonzero(mask > float(alpha_threshold))
if not len(visible_x):
raise ImageEditWarning("The region contains no visible image pixels")
bounds = (
int(visible_x.min()),
int(visible_y.min()),
int(visible_x.max()) + 1,
int(visible_y.max()) + 1,
)
left, top, right, bottom = bounds
return RegionMask(mask[top:bottom, left:right], bounds)
def crop_pixels_to_alpha(pixels, bounds):
"""Crop Blender-order RGBA pixels to their nonzero Alpha bounds."""
import numpy as np
left, top, right, bottom = bounds
width = right - left
height = bottom - top
rgba = np.flipud(
np.asarray(pixels, dtype=np.float32).reshape((height, width, 4))
)
visible_y, visible_x = np.nonzero(rgba[:, :, 3] > 0.0)
if not len(visible_x):
raise ImageEditWarning("The edit contains no visible image pixels")
content_left = int(visible_x.min())
content_top = int(visible_y.min())
content_right = int(visible_x.max()) + 1
content_bottom = int(visible_y.max()) + 1
output = rgba[content_top:content_bottom, content_left:content_right].copy()
return np.flipud(output).ravel(), (
left + content_left,
top + content_top,
left + content_right,
top + content_bottom,
)

160
src/anyimage/operators/common/material.py → src/anyimage/operators/common/mesh_result.py
View File

@@ -1,7 +1,130 @@
from dataclasses import dataclass
from pathlib import Path
import bpy import bpy
from .image import configure_image_user, is_animated_image from .image_data import (
from .node_asset import load_shadeless_node_group configure_image_user,
image_base_name,
image_pixels,
is_animated_image,
)
@dataclass(frozen=True)
class ImageGeometry:
points: object
intrinsics: object
valid: object
calibration_valid: object
image_size: tuple
reference_depth: float
region_transparent_fraction: float
def load_image_geometry(path):
"""Load and validate the shared image geometry Job artifact."""
import numpy as np
with np.load(path, allow_pickle=False) as archive:
geometry = ImageGeometry(
points=np.asarray(archive["points"], dtype=np.float32),
intrinsics=np.asarray(archive["intrinsics"], dtype=np.float32),
valid=np.asarray(archive["valid"], dtype=bool),
calibration_valid=np.asarray(archive["calibration_valid"], dtype=bool),
image_size=tuple(int(value) for value in archive["image_size"]),
reference_depth=float(archive["reference_depth"]),
region_transparent_fraction=float(
archive["region_transparent_fraction"]
),
)
if (
geometry.points.ndim != 3
or geometry.points.shape[2] != 3
or geometry.valid.shape != geometry.points.shape[:2]
or geometry.calibration_valid.shape != geometry.points.shape[:2]
):
raise ValueError("The mesh point field is invalid")
if (
geometry.intrinsics.shape != (3, 3)
or not geometry.reference_depth > 0.0
or not 0.0 <= geometry.region_transparent_fraction <= 1.0
):
raise ValueError("The mesh camera field is invalid")
return geometry
NODE_ASSET_NAME = "O_AnyImage.blend"
def node_asset_path():
return Path(__file__).resolve().parents[2] / "assets" / NODE_ASSET_NAME
def load_asset_node_group(group_name):
existing = bpy.data.node_groups.get(group_name)
if existing is not None:
return existing
asset_path = node_asset_path()
if not asset_path.is_file():
raise RuntimeError(f"Missing node asset: {asset_path}")
if bpy.app.version < (5, 0, 0):
directory = str(asset_path) + "/NodeTree/"
result = bpy.ops.wm.append(
filepath=directory + group_name,
directory=directory,
filename=group_name,
link=False,
do_reuse_local_id=True,
)
if "FINISHED" not in result:
raise RuntimeError("Unable to append the node asset")
node_group = bpy.data.node_groups.get(group_name)
if node_group is None:
raise RuntimeError("Appended node asset is unavailable")
node_group.use_fake_user = True
return node_group
with bpy.data.libraries.load(str(asset_path), link=True) as (source, target):
if group_name not in source.node_groups:
raise RuntimeError(f"Node asset is missing {group_name}")
target.node_groups = [group_name]
linked_group = target.node_groups[0]
node_group = bpy.data.pack_linked_ids_hierarchy(root_id=linked_group)
if linked_group != node_group and linked_group.users == 0:
bpy.data.node_groups.remove(linked_group)
node_group.use_fake_user = True
return node_group
def modifier_input_identifier(node_group, name):
for item in node_group.interface.items_tree:
if item.item_type != "SOCKET" or item.in_out != "INPUT":
continue
if item.name == name:
return item.identifier
raise RuntimeError(f"Geometry Nodes input is missing: {name}")
def _modifier_input_slots(modifier):
"""Return the runtime input collection, or None for the legacy API."""
properties = getattr(modifier, "properties", None)
if properties is None:
return None
return getattr(properties, "inputs", None)
def set_modifier_input(modifier, identifier, value):
"""Assign a value to a Geometry Nodes modifier input socket."""
inputs = _modifier_input_slots(modifier)
if inputs is not None:
# Blender 5.2+ exposes inputs as runtime RNA keyed by socket identifier.
getattr(inputs, identifier).value = value
else:
# Blender 4.x exposes inputs as modifier ID properties.
modifier[identifier] = value
DISPLACEMENT_STRENGTH = 1.0 DISPLACEMENT_STRENGTH = 1.0
DISPLACEMENT_MIDLEVEL = 0.0 DISPLACEMENT_MIDLEVEL = 0.0
@@ -11,6 +134,7 @@ BACKGROUND_DEPTH_ATTRIBUTE = "o_background_depth"
DEPTH_SCALE_ATTRIBUTE = "o_depth_scale" DEPTH_SCALE_ATTRIBUTE = "o_depth_scale"
IMAGE_MATERIAL_NODE_GROUP_NAME = "O Image Layer" IMAGE_MATERIAL_NODE_GROUP_NAME = "O Image Layer"
DEPTH_MATERIAL_NODE_GROUP_NAME = "O Image Depth Layer" DEPTH_MATERIAL_NODE_GROUP_NAME = "O Image Depth Layer"
SHADELESS_NODE_GROUP_NAME = "O Shadeless"
VIEW_TRANSFORM_COLOR_SPACES = ( VIEW_TRANSFORM_COLOR_SPACES = (
("ACES 1.3", "ACES 1.3 sRGB"), ("ACES 1.3", "ACES 1.3 sRGB"),
("ACES 2.0", "ACES 2.0 sRGB"), ("ACES 2.0", "ACES 2.0 sRGB"),
@@ -44,10 +168,7 @@ def set_material_color_space(image, scene=None):
or getattr(image, "is_dirty", False) or getattr(image, "is_dirty", False)
) )
): ):
import numpy as np pixels = image_pixels(image)
pixels = np.empty(len(image.pixels), dtype=np.float32)
image.pixels.foreach_get(pixels)
for color_space in dict.fromkeys((preferred, "sRGB", original)): for color_space in dict.fromkeys((preferred, "sRGB", original)):
if not color_space: if not color_space:
continue continue
@@ -251,8 +372,8 @@ def material_node_group(depth_plane=False):
return group return group
def create_material( def create_image_material(
name, source_object,
color_image, color_image,
depth_scale, depth_scale,
displacement_image=None, displacement_image=None,
@@ -270,7 +391,7 @@ def create_material(
normal_space="TANGENT", normal_space="TANGENT",
scene=None, scene=None,
): ):
material = bpy.data.materials.new(name) material = bpy.data.materials.new(image_base_name(source_object.data))
material.use_nodes = True material.use_nodes = True
if hasattr(material, "surface_render_method"): if hasattr(material, "surface_render_method"):
material.surface_render_method = "DITHERED" material.surface_render_method = "DITHERED"
@@ -332,7 +453,7 @@ def create_material(
links.new(normal_texture.outputs["Color"], material_inputs.inputs["Normal"]) links.new(normal_texture.outputs["Color"], material_inputs.inputs["Normal"])
if shadeless: if shadeless:
shader = nodes.new("ShaderNodeGroup") shader = nodes.new("ShaderNodeGroup")
shader.node_tree = load_shadeless_node_group() shader.node_tree = load_asset_node_group(SHADELESS_NODE_GROUP_NAME)
shader.name = SHADELESS_NODE_GROUP_NAME shader.name = SHADELESS_NODE_GROUP_NAME
transparent = nodes.new("ShaderNodeBsdfTransparent") transparent = nodes.new("ShaderNodeBsdfTransparent")
mix_shader = nodes.new("ShaderNodeMixShader") mix_shader = nodes.new("ShaderNodeMixShader")
@@ -434,3 +555,22 @@ def set_displacement_only(material):
if owner is not None and hasattr(owner, property_name): if owner is not None and hasattr(owner, property_name):
setattr(owner, property_name, "DISPLACEMENT") setattr(owner, property_name, "DISPLACEMENT")
return return
def finalize_object_result(
context,
source_object,
result_object,
*,
keep_source=False,
):
"""Activate a result object and optionally replace its source object."""
source_name = source_object.name
for selected in context.selected_objects:
selected.select_set(False)
result_object.select_set(True)
context.view_layer.objects.active = result_object
if not keep_source:
bpy.data.objects.remove(source_object, do_unlink=True)
result_object.name = source_name
return result_object

119
src/anyimage/operators/common/node_asset.py
View File

@@ -1,119 +0,0 @@
from pathlib import Path
import bpy
NODE_ASSET_NAME = "O_AnyImage.blend"
MESH_PLANE_NODE_GROUP_NAME = "O Mesh Plane"
SHADELESS_NODE_GROUP_NAME = "O Shadeless"
def node_asset_path():
return Path(__file__).resolve().parents[2] / "assets" / NODE_ASSET_NAME
def load_asset_node_group(group_name):
existing = bpy.data.node_groups.get(group_name)
if existing is not None:
for item in existing.interface.items_tree:
if (
item.item_type == "SOCKET"
and item.in_out == "INPUT"
and item.name == "Normal"
):
item.default_value = (0.5, 0.5, 1.0, 1.0)
for node in existing.nodes:
if node.bl_idname == "ShaderNodeNormalMap":
node.space = "TANGENT"
return existing
asset_path = node_asset_path()
if not asset_path.is_file():
raise RuntimeError(f"Missing node asset: {asset_path}")
if bpy.app.version < (5, 0, 0):
directory = str(asset_path) + "/NodeTree/"
result = bpy.ops.wm.append(
filepath=directory + group_name,
directory=directory,
filename=group_name,
link=False,
do_reuse_local_id=True,
)
if "FINISHED" not in result:
raise RuntimeError("Unable to append the node asset")
node_group = bpy.data.node_groups.get(group_name)
if node_group is None:
raise RuntimeError("Appended node asset is unavailable")
node_group.use_fake_user = True
return node_group
with bpy.data.libraries.load(str(asset_path), link=True) as (source, target):
if group_name not in source.node_groups:
raise RuntimeError(f"Node asset is missing {group_name}")
target.node_groups = [group_name]
linked_group = target.node_groups[0]
node_group = bpy.data.pack_linked_ids_hierarchy(root_id=linked_group)
if linked_group != node_group and linked_group.users == 0:
bpy.data.node_groups.remove(linked_group)
node_group.use_fake_user = True
return node_group
def load_shadeless_node_group():
return load_asset_node_group(SHADELESS_NODE_GROUP_NAME)
def load_mesh_plane_node_group():
return load_asset_node_group(MESH_PLANE_NODE_GROUP_NAME)
def modifier_input_identifier(node_group, name):
for item in node_group.interface.items_tree:
if item.item_type != "SOCKET" or item.in_out != "INPUT":
continue
if item.name == name:
return item.identifier
raise RuntimeError(f"Geometry Nodes input is missing: {name}")
def modifier_input_slots(modifier):
"""Return the runtime input collection, or None for the legacy API."""
properties = getattr(modifier, "properties", None)
if properties is None:
return None
return getattr(properties, "inputs", None)
def set_modifier_input(modifier, identifier, value):
"""Assign a value to a Geometry Nodes modifier input socket."""
inputs = modifier_input_slots(modifier)
if inputs is not None:
# Blender 5.2+ exposes inputs as runtime RNA keyed by socket identifier.
getattr(inputs, identifier).value = value
else:
# Blender 4.x exposes inputs as modifier ID properties.
modifier[identifier] = value
def get_modifier_input(modifier, identifier):
"""Read a Geometry Nodes modifier input socket value."""
inputs = modifier_input_slots(modifier)
if inputs is not None:
return getattr(inputs, identifier).value
return modifier[identifier]
def modifier_input_driver_path(modifier, identifier):
"""Return the driver data path for a modifier input socket."""
if modifier_input_slots(modifier) is not None:
return f'modifiers["{modifier.name}"].properties.inputs.{identifier}.value'
return f'modifiers["{modifier.name}"]["{identifier}"]'
def drive_modifier_input(owner_object, modifier, identifier, expression="frame"):
"""Add a scripted driver to a modifier input so it follows the scene frame."""
path = modifier_input_driver_path(modifier, identifier)
driver = owner_object.driver_add(path)
driver.driver.type = "SCRIPTED"
driver.driver.expression = expression
return driver

111
src/anyimage/operators/convert_to_plane/build.py
View File

@@ -1,19 +1,20 @@
import bpy import bpy
from ..common.image import ( from ..common.mesh_result import load_image_geometry
from ..common.image_data import (
color_data_name, color_data_name,
create_color_image,
depth_data_name, depth_data_name,
is_image_empty, image_pixels,
material_data_name, load_color_result_image,
normal_data_name, load_normal_result_image,
plane_bounds, image_empty_bounds,
require_image_empty,
) )
from ..common.material import create_material from ..common.mesh_result import create_image_material
from ..common.node_asset import modifier_input_identifier, set_modifier_input from ..common.mesh_result import modifier_input_identifier, set_modifier_input
from ..common.normal_map import bake_tangent_normal_image from ..common.mesh_result import finalize_object_result
from ..common.operator import push_undo
from .mesh_plane import create_mesh_plane_mesh, create_mesh_plane_object from .mesh_plane import create_mesh_plane_mesh, create_mesh_plane_object
from .normal_map import bake_tangent_normal_image
from .nodes import depth_plane_node_group from .nodes import depth_plane_node_group
@@ -23,10 +24,7 @@ def copy_color_image(source_object):
if getattr(source, "source", None) == "GENERATED" or getattr( if getattr(source, "source", None) == "GENERATED" or getattr(
source, "is_dirty", False source, "is_dirty", False
): ):
import numpy as np pixels = image_pixels(source)
pixels = np.empty(len(source.pixels), dtype=np.float32)
source.pixels.foreach_get(pixels)
image.pixels.foreach_set(pixels) image.pixels.foreach_set(pixels)
image.update() image.update()
image.name = color_data_name(source_object) image.name = color_data_name(source_object)
@@ -38,7 +36,7 @@ def build_mesh_plane(context, source_object, subdivisions, material):
plane = create_mesh_plane_object( plane = create_mesh_plane_object(
context, context,
source_name, source_name,
plane_bounds(source_object), image_empty_bounds(source_object),
material, material,
matrix_world=source_object.matrix_world, matrix_world=source_object.matrix_world,
subdivisions=subdivisions, subdivisions=subdivisions,
@@ -46,58 +44,32 @@ def build_mesh_plane(context, source_object, subdivisions, material):
for polygon in plane.data.polygons: for polygon in plane.data.polygons:
polygon.use_smooth = True polygon.use_smooth = True
plane["anyimage_mesh_shape"] = "PLANE" plane["anyimage_mesh_shape"] = "PLANE"
for selected in context.selected_objects: return finalize_object_result(context, source_object, plane)
selected.select_set(False)
plane.select_set(True)
context.view_layer.objects.active = plane
bpy.data.objects.remove(source_object, do_unlink=True)
plane.name = source_name
return plane
def load_depth_field(path):
import numpy as np
with np.load(path, allow_pickle=False) as archive:
points = np.asarray(archive["points"], dtype=np.float32)
intrinsics = np.asarray(archive["intrinsics"], dtype=np.float32)
valid = np.asarray(archive["valid"], dtype=bool)
calibration_valid = np.asarray(archive["calibration_valid"], dtype=bool)
image_size = tuple(int(value) for value in archive["image_size"])
reference_depth = float(archive["reference_depth"])
if (
points.ndim != 3
or points.shape[2] != 3
or valid.shape != points.shape[:2]
or calibration_valid.shape != points.shape[:2]
):
raise ValueError("The mesh point field is invalid")
if intrinsics.shape != (3, 3) or not reference_depth > 0.0:
raise ValueError("The mesh camera field is invalid")
return points, intrinsics, valid, calibration_valid, image_size, reference_depth
def anchored_point_field(source_object, field): def anchored_point_field(source_object, field):
import numpy as np import numpy as np
points, intrinsics, valid, _calibration, image_size, reference_depth = field points = field.points
width, height = image_size intrinsics = field.intrinsics
x_min, x_max, y_min, y_max = plane_bounds(source_object) valid = field.valid
width, height = field.image_size
x_min, x_max, y_min, y_max = image_empty_bounds(source_object)
plane_width = x_max - x_min plane_width = x_max - x_min
plane_height = y_max - y_min plane_height = y_max - y_min
fx, fy = float(intrinsics[0, 0]), float(intrinsics[1, 1]) fx, fy = float(intrinsics[0, 0]), float(intrinsics[1, 1])
cx, cy = float(intrinsics[0, 2]), float(intrinsics[1, 2]) cx, cy = float(intrinsics[0, 2]), float(intrinsics[1, 2])
if fx <= 0.0 or fy <= 0.0: if fx <= 0.0 or fy <= 0.0:
raise ValueError("Camera intrinsics contain an invalid focal length") raise ValueError("Camera intrinsics contain an invalid focal length")
scale_x = plane_width * fx / (max(width - 1, 1) * reference_depth) scale_x = plane_width * fx / (max(width - 1, 1) * field.reference_depth)
scale_y = plane_height * fy / (max(height - 1, 1) * reference_depth) scale_y = plane_height * fy / (max(height - 1, 1) * field.reference_depth)
scale_z = float(np.sqrt(scale_x * scale_y)) scale_z = float(np.sqrt(scale_x * scale_y))
camera_x = x_min + plane_width * cx / max(width - 1, 1) camera_x = x_min + plane_width * cx / max(width - 1, 1)
camera_y = y_max - plane_height * cy / max(height - 1, 1) camera_y = y_max - plane_height * cy / max(height - 1, 1)
anchored = np.empty_like(points, dtype=np.float32) anchored = np.empty_like(points, dtype=np.float32)
anchored[..., 0] = camera_x + points[..., 0] * scale_x anchored[..., 0] = camera_x + points[..., 0] * scale_x
anchored[..., 1] = camera_y - points[..., 1] * scale_y anchored[..., 1] = camera_y - points[..., 1] * scale_y
anchored[..., 2] = (reference_depth - points[..., 2]) * scale_z anchored[..., 2] = (field.reference_depth - points[..., 2]) * scale_z
return anchored, valid, points[..., 2] return anchored, valid, points[..., 2]
@@ -122,17 +94,9 @@ def create_depth_image(source_object, anchored_points):
return image return image
def load_normal_image(path, source_object):
image = bpy.data.images.load(str(path), check_existing=False)
image.name = normal_data_name(source_object)
image.colorspace_settings.name = "Non-Color"
image.pack()
return image
def build_depth_plane(context, source_object, subdivisions, material, field): def build_depth_plane(context, source_object, subdivisions, material, field):
source_name = source_object.name source_name = source_object.name
mesh = create_mesh_plane_mesh(source_name, plane_bounds(source_object)) mesh = create_mesh_plane_mesh(source_name, image_empty_bounds(source_object))
for polygon in mesh.polygons: for polygon in mesh.polygons:
polygon.use_smooth = True polygon.use_smooth = True
mesh.materials.append(material) mesh.materials.append(material)
@@ -160,33 +124,21 @@ def build_depth_plane(context, source_object, subdivisions, material, field):
modifier_input_identifier(node_group, name), modifier_input_identifier(node_group, name),
value, value,
) )
for selected in context.selected_objects: return finalize_object_result(context, source_object, plane)
selected.select_set(False)
plane.select_set(True)
context.view_layer.objects.active = plane
bpy.data.objects.remove(source_object, do_unlink=True)
plane.name = source_name
return plane
def build_depth_plane_response(context, result, operator): def build_depth_plane_response(context, result, operator):
source_object = bpy.data.objects.get(operator.source_object_name) source_object = require_image_empty(operator.source_object_name)
if not is_image_empty(source_object): color_image = load_color_result_image(source_object, result.directory)
raise RuntimeError("The source Image Empty no longer exists")
color_image = create_color_image(
source_object,
result.directory,
assign_to_source=False,
)
color_image.name = color_data_name(source_object) color_image.name = color_data_name(source_object)
normal_path = result.directory / "normal.png" normal_path = result.directory / "normal.png"
normal_image = ( normal_image = (
load_normal_image(normal_path, source_object) load_normal_result_image(normal_path, source_object)
if normal_path.is_file() if normal_path.is_file()
else None else None
) )
material = create_material( material = create_image_material(
material_data_name(source_object), source_object,
color_image, color_image,
0.0, 0.0,
normal_image=normal_image, normal_image=normal_image,
@@ -197,7 +149,7 @@ def build_depth_plane_response(context, result, operator):
source_object, source_object,
operator.mesh_detail, operator.mesh_detail,
material, material,
load_depth_field(result.file("field")), load_image_geometry(result.file("image_geometry")),
) )
if normal_image is not None: if normal_image is not None:
bake_tangent_normal_image( bake_tangent_normal_image(
@@ -206,4 +158,5 @@ def build_depth_plane_response(context, result, operator):
plane, plane,
normal_image, normal_image,
) )
push_undo("Convert to Depth Plane") if "FINISHED" not in bpy.ops.ed.undo_push(message="Convert to Depth Plane"):
raise RuntimeError("Unable to create the Convert to Depth Plane Undo step")

81
src/anyimage/operators/convert_to_plane/conversion.py
View File

@@ -8,30 +8,29 @@ from ...server.media.resolution import (
MAX_MAX_AI_INPUT_SIZE, MAX_MAX_AI_INPUT_SIZE,
MIN_MAX_AI_INPUT_SIZE, MIN_MAX_AI_INPUT_SIZE,
) )
from ..common.image import ( from ..common.image_data import (
analysis_source_image, cleanup_image_input,
image_user_settings, image_user_settings,
is_image_empty, is_image_empty,
material_data_name, prepare_image_input,
prepare_image_analysis_inputs, source_start_frame,
remove_temporary_input,
scaled_process_resolution,
video_total_frames, video_total_frames,
) )
from ..common.material import create_material from ..common.mesh_result import create_image_material
from ..common.operator import ( from ..common.ai_job import (
HIDDEN,
cleanup_temporary_input,
da3_parameters, da3_parameters,
input_path, require_input_path,
invoke_ai_setup_if_needed, invoke_ai_setup_if_needed,
moge2_parameters, moge2_parameters,
production_device, production_device,
require_environment, require_environment,
require_model, require_model,
source_start_frame,
) )
from .build import build_depth_plane_response, build_mesh_plane, copy_color_image from .build import build_depth_plane_response, build_mesh_plane, copy_color_image
from .resolution import scaled_process_resolution
_HIDDEN = {"HIDDEN", "SKIP_SAVE"}
def depth_plane_settings(context): def depth_plane_settings(context):
@@ -46,48 +45,39 @@ def depth_plane_settings(context):
class GenerateDepthPlane(JobOperatorBase, bpy.types.Operator): class GenerateDepthPlane(JobOperatorBase, bpy.types.Operator):
bl_idname = "anyimage.generate_depth_plane" bl_idname = "anyimage.generate_depth_plane"
bl_label = "Generate Depth Plane" bl_label = "Generate Depth Plane"
job_type = "prepare-geometry-data" job_type = "generate-depth-plane"
starting_message = "Preparing Depth Plane" starting_message = "Preparing Depth Plane"
input_path: bpy.props.StringProperty(options=HIDDEN) input_path: bpy.props.StringProperty(options=_HIDDEN)
analysis_input_path: bpy.props.StringProperty(options=HIDDEN) source_object_name: bpy.props.StringProperty(options=_HIDDEN)
source_object_name: bpy.props.StringProperty(options=HIDDEN) mesh_detail: bpy.props.IntProperty(default=0, min=0, max=10, options=_HIDDEN)
mesh_detail: bpy.props.IntProperty(default=0, min=0, max=10, options=HIDDEN) model: bpy.props.StringProperty(default=DEFAULT_MOGE2_MODEL_KEY, options=_HIDDEN)
model: bpy.props.StringProperty(default=DEFAULT_MOGE2_MODEL_KEY, options=HIDDEN) process_res: bpy.props.IntProperty(default=1008, min=1, options=_HIDDEN)
process_res: bpy.props.IntProperty(default=1008, min=1, options=HIDDEN) resolution_level: bpy.props.IntProperty(default=5, min=0, max=9, options=_HIDDEN)
resolution_level: bpy.props.IntProperty(default=5, min=0, max=9, options=HIDDEN)
max_input_size: bpy.props.IntProperty( max_input_size: bpy.props.IntProperty(
default=DEFAULT_MAX_AI_INPUT_SIZE, default=DEFAULT_MAX_AI_INPUT_SIZE,
min=MIN_MAX_AI_INPUT_SIZE, min=MIN_MAX_AI_INPUT_SIZE,
max=MAX_MAX_AI_INPUT_SIZE, max=MAX_MAX_AI_INPUT_SIZE,
options=HIDDEN, options=_HIDDEN,
) )
delete_input: bpy.props.BoolProperty(default=False, options=HIDDEN) delete_input: bpy.props.BoolProperty(default=False, options=_HIDDEN)
delete_analysis_input: bpy.props.BoolProperty(default=False, options=HIDDEN)
def request(self, context): def request(self, context):
require_environment() require_environment()
require_model(self.model) require_model(self.model)
parameters = { parameters = {
"input": str(input_path(self.input_path)), "input": str(require_input_path(self.input_path)),
"analysis_input": str(input_path(self.analysis_input_path)),
"shape": "DEPTH_PLANE",
"polygon": None,
"mesh_detail": self.mesh_detail,
"device": production_device().lower(), "device": production_device().lower(),
"video_frames": 1, "video_frames": 1,
"start_frame": source_start_frame(self.source_object_name), "start_frame": source_start_frame(self.source_object_name),
"max_input_size": int(self.max_input_size), "max_input_size": int(self.max_input_size),
"refine_selection": False,
"normal_analysis": False,
"invert": False,
} }
settings = context.scene.anyimage_settings settings = context.scene.anyimage_settings
if self.model.startswith("MOGE2_"): if self.model.startswith("MOGE2_"):
parameters.update( parameters.update(
moge2_parameters( moge2_parameters(
settings, settings,
input_path(self.input_path), require_input_path(self.input_path),
self.model, self.model,
self.resolution_level, self.resolution_level,
source_start_frame(self.source_object_name), source_start_frame(self.source_object_name),
@@ -97,13 +87,12 @@ class GenerateDepthPlane(JobOperatorBase, bpy.types.Operator):
parameters.update( parameters.update(
da3_parameters( da3_parameters(
settings, settings,
input_path(self.input_path), require_input_path(self.input_path),
self.model, self.model,
self.process_res, self.process_res,
source_start_frame(self.source_object_name), source_start_frame(self.source_object_name),
) )
) )
parameters.update(shape="DEPTH_PLANE", polygon=None)
return parameters return parameters
def response(self, context, result): def response(self, context, result):
@@ -111,11 +100,7 @@ class GenerateDepthPlane(JobOperatorBase, bpy.types.Operator):
return "Depth Plane is ready" return "Depth Plane is ready"
def cleanup(self): def cleanup(self):
cleanup_temporary_input(self.input_path, self.delete_input) cleanup_image_input(self.input_path, self.delete_input)
cleanup_temporary_input(
self.analysis_input_path,
self.delete_analysis_input,
)
def start_depth_plane_job(operator, context): def start_depth_plane_job(operator, context):
@@ -127,13 +112,13 @@ def start_depth_plane_job(operator, context):
if setup_result is not None: if setup_result is not None:
return setup_result return setup_result
try: try:
source_path, delete_input, analysis_path, delete_analysis_input = ( source_path, delete_input = prepare_image_input(
prepare_image_analysis_inputs(source_object.data, context.scene) source_object.data,
context.scene,
) )
analysis_image = analysis_source_image(source_object.data)
model, process_factor, resolution_level = depth_plane_settings(context) model, process_factor, resolution_level = depth_plane_settings(context)
process_res = ( process_res = (
scaled_process_resolution(analysis_image, process_factor) scaled_process_resolution(source_object.data, process_factor)
if not model.startswith("MOGE2_") if not model.startswith("MOGE2_")
else 1 else 1
) )
@@ -142,7 +127,6 @@ def start_depth_plane_job(operator, context):
return {"CANCELLED"} return {"CANCELLED"}
result = bpy.ops.anyimage.generate_depth_plane( result = bpy.ops.anyimage.generate_depth_plane(
input_path=str(source_path), input_path=str(source_path),
analysis_input_path=str(analysis_path),
source_object_name=source_object.name, source_object_name=source_object.name,
mesh_detail=operator.mesh_detail, mesh_detail=operator.mesh_detail,
model=model, model=model,
@@ -150,13 +134,10 @@ def start_depth_plane_job(operator, context):
resolution_level=resolution_level, resolution_level=resolution_level,
max_input_size=configured_max_ai_input_size(context), max_input_size=configured_max_ai_input_size(context),
delete_input=delete_input, delete_input=delete_input,
delete_analysis_input=delete_analysis_input,
) )
if "CANCELLED" in result: if "CANCELLED" in result:
if delete_input: if delete_input:
remove_temporary_input(source_path) cleanup_image_input(source_path, True)
if delete_analysis_input:
remove_temporary_input(analysis_path)
return {"CANCELLED"} if "CANCELLED" in result else {"FINISHED"} return {"CANCELLED"} if "CANCELLED" in result else {"FINISHED"}
@@ -178,8 +159,8 @@ class ConvertToPlane(bpy.types.Operator):
frame_start, frame_offset, frame_duration, use_cyclic = image_user_settings( frame_start, frame_offset, frame_duration, use_cyclic = image_user_settings(
source_object source_object
) )
material = create_material( material = create_image_material(
material_data_name(source_object), source_object,
color_image, color_image,
1.0, 1.0,
frame_start=frame_start, frame_start=frame_start,

9
src/anyimage/operators/convert_to_plane/mesh_plane.py
View File

@@ -2,11 +2,8 @@
import bpy import bpy
from ..common.node_asset import ( from ..common.mesh_result import modifier_input_identifier, set_modifier_input
load_mesh_plane_node_group, from .nodes import mesh_plane_node_group
modifier_input_identifier,
set_modifier_input,
)
MESH_PLANE_NODE_GROUP_NAME = "O Mesh Plane" MESH_PLANE_NODE_GROUP_NAME = "O Mesh Plane"
@@ -44,7 +41,7 @@ def add_mesh_plane_processing(
subdivisions=0, subdivisions=0,
thickness=0.0, thickness=0.0,
): ):
node_group = load_mesh_plane_node_group() node_group = mesh_plane_node_group()
modifier = plane_object.modifiers.new( modifier = plane_object.modifiers.new(
name=MESH_PLANE_NODE_GROUP_NAME, name=MESH_PLANE_NODE_GROUP_NAME,
type="NODES", type="NODES",

55
src/anyimage/operators/convert_to_plane/nodes.py
View File

@@ -2,8 +2,33 @@
import bpy import bpy
from ..common.node_asset import load_mesh_plane_node_group from ..common.mesh_result import load_asset_node_group
from ..common.node_graph import group_input, interface_socket
MESH_PLANE_NODE_GROUP_NAME = "O Mesh Plane"
def mesh_plane_node_group():
return load_asset_node_group(MESH_PLANE_NODE_GROUP_NAME)
def _interface_socket(group, name, in_out, socket_type):
socket = group.interface.new_socket(
name=name,
in_out=in_out,
socket_type=socket_type,
)
if in_out == "INPUT" and hasattr(socket, "structure_type"):
socket.structure_type = "SINGLE"
return socket
def _group_input(nodes, location, visible_names):
node = nodes.new("NodeGroupInput")
node.location = location
visible = set(visible_names)
for output in node.outputs:
output.hide = output.name not in visible
return node
def depth_plane_node_group(): def depth_plane_node_group():
group_name = "O Image Depth Plane" group_name = "O Image Depth Plane"
@@ -16,42 +41,42 @@ def depth_plane_node_group():
group.nodes.clear() group.nodes.clear()
group.interface.clear() group.interface.clear()
group.is_modifier = True group.is_modifier = True
interface_socket(group, "Geometry", "INPUT", "NodeSocketGeometry") _interface_socket(group, "Geometry", "INPUT", "NodeSocketGeometry")
subdivide = interface_socket(group, "Subdivide", "INPUT", "NodeSocketInt") subdivide = _interface_socket(group, "Subdivide", "INPUT", "NodeSocketInt")
subdivide.default_value = 7 subdivide.default_value = 7
subdivide.min_value = 0 subdivide.min_value = 0
subdivide.max_value = 10 subdivide.max_value = 10
thickness = interface_socket(group, "Thickness", "INPUT", "NodeSocketFloat") thickness = _interface_socket(group, "Thickness", "INPUT", "NodeSocketFloat")
thickness.default_value = 0.1 thickness.default_value = 0.1
thickness.min_value = 0.0 thickness.min_value = 0.0
thickness.subtype = "DISTANCE" thickness.subtype = "DISTANCE"
depth_amount = interface_socket(group, "Depth Amount", "INPUT", "NodeSocketFloat") depth_amount = _interface_socket(group, "Depth Amount", "INPUT", "NodeSocketFloat")
depth_amount.default_value = 1.0 depth_amount.default_value = 1.0
depth_amount.min_value = 0.0 depth_amount.min_value = 0.0
depth_image = interface_socket(group, "Depth Image", "INPUT", "NodeSocketImage") depth_image = _interface_socket(group, "Depth Image", "INPUT", "NodeSocketImage")
depth_image.hide_in_modifier = True depth_image.hide_in_modifier = True
material = interface_socket(group, "Material", "INPUT", "NodeSocketMaterial") material = _interface_socket(group, "Material", "INPUT", "NodeSocketMaterial")
material.hide_in_modifier = True material.hide_in_modifier = True
interface_socket(group, "Geometry", "OUTPUT", "NodeSocketGeometry") _interface_socket(group, "Geometry", "OUTPUT", "NodeSocketGeometry")
nodes = group.nodes nodes = group.nodes
links = group.links links = group.links
geometry_input = group_input( geometry_input = _group_input(
nodes, nodes,
(-1850.0, 450.0), (-1850.0, 450.0),
{"Geometry", "Subdivide", "Material"}, {"Geometry", "Subdivide", "Material"},
) )
shape_input = group_input( shape_input = _group_input(
nodes, nodes,
(-1250.0, 850.0), (-1250.0, 850.0),
{"Thickness", "Depth Amount"}, {"Thickness", "Depth Amount"},
) )
image_input = group_input(nodes, (-1050.0, 50.0), {"Depth Image"}) image_input = _group_input(nodes, (-1050.0, 50.0), {"Depth Image"})
cleanup_input = group_input(nodes, (650.0, -200.0), {"Thickness"}) cleanup_input = _group_input(nodes, (650.0, -200.0), {"Thickness"})
group_output = nodes.new("NodeGroupOutput") group_output = nodes.new("NodeGroupOutput")
group_output.location = (1600.0, 450.0) group_output.location = (1600.0, 450.0)
mesh_plane = nodes.new("GeometryNodeGroup") mesh_plane = nodes.new("GeometryNodeGroup")
mesh_plane.node_tree = load_mesh_plane_node_group() mesh_plane.node_tree = mesh_plane_node_group()
mesh_plane.inputs["Thickness"].default_value = 0.0 mesh_plane.inputs["Thickness"].default_value = 0.0
mesh_plane.location = (-1550.0, 450.0) mesh_plane.location = (-1550.0, 450.0)
links.new(geometry_input.outputs["Geometry"], mesh_plane.inputs["Geometry"]) links.new(geometry_input.outputs["Geometry"], mesh_plane.inputs["Geometry"])
@@ -150,7 +175,7 @@ def depth_plane_node_group():
def plane_node_group(shape): def plane_node_group(shape):
if shape == "PLANE": if shape == "PLANE":
return load_mesh_plane_node_group() return mesh_plane_node_group()
if shape == "DEPTH_PLANE": if shape == "DEPTH_PLANE":
return depth_plane_node_group() return depth_plane_node_group()
raise ValueError(f"Unsupported Plane shape: {shape}") raise ValueError(f"Unsupported Plane shape: {shape}")

7
src/anyimage/operators/common/normal_map.py → src/anyimage/operators/convert_to_plane/normal_map.py
View File

@@ -1,3 +1,6 @@
from ..common.image_data import image_pixels
def _normalized(values, fallback): def _normalized(values, fallback):
import numpy as np import numpy as np
@@ -91,9 +94,7 @@ def bake_tangent_normal_image(context, source_object, target_object, image):
import numpy as np import numpy as np
width, height = (int(value) for value in image.size) width, height = (int(value) for value in image.size)
rgba = np.empty(width * height * 4, dtype=np.float32) rgba = image_pixels(image).reshape((height, width, 4))
image.pixels.foreach_get(rgba)
rgba = rgba.reshape((height, width, 4))
source_normal = rgba[..., :3].astype(np.float64) * 2.0 - 1.0 source_normal = rgba[..., :3].astype(np.float64) * 2.0 - 1.0
source_to_target = ( source_to_target = (

16
src/anyimage/operators/convert_to_plane/resolution.py Normal file
View File

@@ -0,0 +1,16 @@
DA3_PATCH_SIZE = 14
DA3_MIN_PROCESS_RES = 252
DA3_MAX_PROCESS_RES = 1008
def scaled_process_resolution(image, scale_percent):
"""Return the scaled long edge clamped and aligned for DA3 inference."""
width, height = image.size
if width <= 0 or height <= 0:
raise ValueError("The source image has no readable dimensions")
if scale_percent < 1.0 or scale_percent > 100.0:
raise ValueError("Resolution Scale must be between 1% and 100%")
process_res = max(width, height) * scale_percent / 100.0
process_res = max(DA3_MIN_PROCESS_RES, process_res)
process_res = min(DA3_MAX_PROCESS_RES, process_res)
return int(process_res / DA3_PATCH_SIZE + 0.5) * DA3_PATCH_SIZE

10
src/anyimage/operators/cutout_tool/__init__.py
View File

@@ -1,12 +1,8 @@
"""Cutout selection, shape conversion, and geometry algorithms.""" from .region_to_shape import CutoutRegionToShape
from .select_region import CutoutTool, SelectCutoutRegion
from .selection_tool import ActivateCutoutTool, CutoutTool, SelectCutoutRegion
from .shape_conversion import ConvertCutoutShape, GenerateCutoutShape
__all__ = ( __all__ = (
"ActivateCutoutTool", "CutoutRegionToShape",
"ConvertCutoutShape",
"CutoutTool", "CutoutTool",
"GenerateCutoutShape",
"SelectCutoutRegion", "SelectCutoutRegion",
) )

814
src/anyimage/operators/cutout_tool/build_shapes.py
View File

@@ -1,814 +0,0 @@
import json
import bpy
from ...preferences import addon_preferences, configured_max_ai_input_size
from ...server.media.resolution import limited_image_size
from ..common.image import (
color_data_name,
create_color_image,
depth_data_name,
is_image_empty,
material_data_name,
normal_data_name,
plane_bounds,
video_total_frames,
)
from ..common.material import create_material
from ..common.node_asset import modifier_input_identifier, set_modifier_input
from ..common.normal_map import bake_tangent_normal_image
from ..common.operator import (
push_undo,
)
from .depth_calibration import (
DEFAULT_DEPTH_DIRECTION,
UNREFINED_DEPTH_REFERENCE,
depth_balloon_projection_parameters,
metric_depth_reference,
)
from .planar_mesh import (
metric_transform,
prepare_planar_regions,
triangulate_planar_regions,
)
from .balloon_profile import balloon_profile as calculate_balloon_profile
from .mask_regions import alpha_regions
from .node_assets import cutout_node_group
from ..common.image_edit import (
ImageEditWarning,
polygon_bounds,
)
from ..common.image_mask import polygon_mask, visible_selection_mask
from .shape_types import (
BALLOON_ATTRIBUTE_NAME,
CUTOUT_SHAPE_INDICES,
)
CUTOUT_ALPHA_THRESHOLD = 0.1
CUTOUT_AXIS_TO_SOURCE = (
(0.0, -1.0, 0.0, 0.0),
(0.0, 0.0, 1.0, 0.0),
(-1.0, 0.0, 0.0, 0.0),
(0.0, 0.0, 0.0, 1.0),
)
def _surface_depth_image(source_object, points, valid, camera_depth):
import numpy as np
points = np.asarray(points, dtype=np.float32)
valid = np.asarray(valid, dtype=bool)
height, width = valid.shape
if points.shape != (height, width, 3):
raise ValueError("Depth Surface point dimensions do not match")
discontinuity = _depth_discontinuity_map(camera_depth, valid)
discontinuity[~valid] = -1.0
pixels = np.empty((height, width, 4), dtype=np.float32)
pixels[:, :, :3] = points
pixels[:, :, 3] = discontinuity
image = bpy.data.images.new(
depth_data_name(source_object),
width=width,
height=height,
alpha=True,
float_buffer=True,
)
image.colorspace_settings.name = "Non-Color"
image.alpha_mode = "CHANNEL_PACKED"
image.pixels.foreach_set(np.flipud(pixels).ravel())
image.pack()
return image
def _cutout_world_matrix(source_matrix, crop_center):
from mathutils import Matrix
center_x, center_y = (float(value) for value in crop_center)
return (
source_matrix
@ Matrix.Translation((center_x, center_y, 0.0))
@ Matrix(CUTOUT_AXIS_TO_SOURCE)
)
def _orient_cutout_vertices(vertices):
import numpy as np
vertices = np.asarray(vertices, dtype=np.float64)
return np.column_stack(
(
-vertices[:, 2],
-vertices[:, 0],
vertices[:, 1],
)
)
def _cutout_domain(
regions_pixels,
image_size,
bounds,
spacing,
world_metric,
balloon_algorithm="POISSON",
):
import numpy as np
x_min, x_max, y_min, y_max = bounds
regions, spacing = prepare_planar_regions(
regions_pixels,
image_size,
bounds,
spacing=spacing,
world_metric=world_metric,
)
planar, planar_faces, boundary, components = triangulate_planar_regions(
regions,
spacing,
return_components=True,
)
profile = calculate_balloon_profile(
balloon_algorithm, planar, planar_faces, boundary, components
)
faces = np.asarray(planar_faces, dtype=np.int32)
local = planar @ np.linalg.inv(metric_transform(world_metric))
uv = np.column_stack(
(
(local[:, 0] - x_min) / (x_max - x_min),
(local[:, 1] - y_min) / (y_max - y_min),
)
).astype(np.float32)
crop_center = np.asarray(
((x_min + x_max) * 0.5, (y_min + y_max) * 0.5),
dtype=np.float64,
)
local -= crop_center
vertices = np.column_stack((local, np.zeros(len(local), dtype=np.float64)))
return vertices, faces, uv, boundary, planar, profile
def _world_plane_metric(matrix_world):
from mathutils import Vector
linear = matrix_world.to_3x3()
x = linear @ Vector((1.0, 0.0, 0.0))
y = linear @ Vector((0.0, 1.0, 0.0))
z = linear @ Vector((0.0, 0.0, 1.0))
return ((x.dot(x), x.dot(y)), (x.dot(y), y.dot(y))), z.length
def _cutout_cleanup_area(spacing, world_metric):
import numpy as np
if spacing <= 0.0:
raise ValueError("Cutout point spacing must be greater than zero")
area_scale = abs(float(np.linalg.det(metric_transform(world_metric))))
if area_scale <= 1e-12:
raise ValueError("The Image Empty has a degenerate world transform")
return float(spacing) ** 2 / area_scale
def _load_field(path):
import numpy as np
with np.load(path, allow_pickle=False) as archive:
points = np.asarray(archive["points"], dtype=np.float32)
intrinsics = np.asarray(archive["intrinsics"], dtype=np.float32)
valid = np.asarray(archive["valid"], dtype=bool)
calibration_valid = np.asarray(archive["calibration_valid"], dtype=bool)
image_size = tuple(int(value) for value in archive["image_size"])
reference_depth = float(archive["reference_depth"])
if (
points.ndim != 3
or points.shape[2] != 3
or valid.shape != points.shape[:2]
or calibration_valid.shape != points.shape[:2]
):
raise ValueError("The mesh point field is invalid")
if intrinsics.shape != (3, 3) or not reference_depth > 0.0:
raise ValueError("The mesh camera field is invalid")
return points, intrinsics, valid, calibration_valid, image_size, reference_depth
def _surface_uniform_scale(
source_object,
field,
bounds=None,
):
import numpy as np
(
_points,
intrinsics,
_valid,
_calibration_valid,
image_size,
reference_depth,
) = field
width, height = image_size
x_min, x_max, y_min, y_max = bounds or plane_bounds(source_object)
plane_width = x_max - x_min
plane_height = y_max - y_min
fx, fy = float(intrinsics[0, 0]), float(intrinsics[1, 1])
if fx <= 0.0 or fy <= 0.0:
raise ValueError("Camera intrinsics contain an invalid focal length")
scale_x = plane_width * fx / (max(width - 1, 1) * reference_depth)
scale_y = plane_height * fy / (max(height - 1, 1) * reference_depth)
return float(np.sqrt(scale_x * scale_y))
def _depth_discontinuity_map(depth, valid):
import numpy as np
depth = np.asarray(depth, dtype=np.float32)
valid = np.asarray(valid, dtype=bool) & np.isfinite(depth) & (depth > 0.0)
padded_depth = np.pad(depth, 1, mode="edge")
padded_valid = np.pad(valid, 1, mode="edge")
windows = np.lib.stride_tricks.sliding_window_view(padded_depth, (3, 3))
usable = np.lib.stride_tricks.sliding_window_view(padded_valid, (3, 3))
minimum = np.min(np.where(usable, windows, np.inf), axis=(-2, -1))
maximum = np.max(np.where(usable, windows, -np.inf), axis=(-2, -1))
result = (maximum - minimum) / np.maximum(np.abs(depth), 1e-6)
result[~usable.all(axis=(-2, -1))] = 1.0
return result.astype(np.float32)
def _counter_clockwise_faces(points, faces):
import numpy as np
points = np.asarray(points, dtype=np.float64)
result = np.asarray(faces, dtype=np.int32).copy()
polygons = points[result]
following = np.roll(polygons, -1, axis=1)
signed_area = np.sum(
polygons[..., 0] * following[..., 1]
- polygons[..., 1] * following[..., 0],
axis=1,
)
result[signed_area < 0.0] = result[signed_area < 0.0, ::-1]
return result
def _assign_uv(mesh, uv):
import numpy as np
for layer in list(mesh.uv_layers):
mesh.uv_layers.remove(layer)
uv_layer = mesh.uv_layers.new(name="UVMap")
loop_vertices = np.empty(len(mesh.loops), dtype=np.int32)
mesh.loops.foreach_get("vertex_index", loop_vertices)
loop_uv = np.asarray(uv, dtype=np.float32)[loop_vertices]
uv_layer.data.foreach_set("uv", loop_uv.ravel())
def _assign_float_point_attribute(mesh, name, values):
import numpy as np
attribute = mesh.attributes.new(name=name, type="FLOAT", domain="POINT")
attribute.data.foreach_set("value", np.asarray(values, dtype=np.float32))
def _image_rgba(image):
import numpy as np
width, height = (int(value) for value in image.size)
pixels = np.empty(len(image.pixels), dtype=np.float32)
image.pixels.foreach_get(pixels)
return np.flipud(pixels.reshape((height, width, 4)))
def _mask_crop_bounds(alpha):
import numpy as np
values = np.asarray(alpha)
visible_y, visible_x = np.nonzero(values > CUTOUT_ALPHA_THRESHOLD)
if not len(visible_x):
raise ValueError("The Cutout selection contains no visible pixels")
return (
int(visible_x.min()),
int(visible_y.min()),
int(visible_x.max()) + 1,
int(visible_y.max()) + 1,
)
def _crop_plane_bounds(source_object, crop_bounds, source_size):
source_width, source_height = (float(value) for value in source_size)
left, top, right, bottom = (float(value) for value in crop_bounds)
x_min, x_max, y_min, y_max = plane_bounds(source_object)
width = x_max - x_min
height = y_max - y_min
return (
x_min + left * width / source_width,
x_min + right * width / source_width,
y_max - bottom * height / source_height,
y_max - top * height / source_height,
)
def _cropped_color_image(source_object, crop_bounds):
import numpy as np
source = source_object.data
rgba = _image_rgba(source)
left, top, right, bottom = crop_bounds
width = right - left
height = bottom - top
if width <= 0 or height <= 0:
raise ValueError("The Cutout crop dimensions must be positive")
cropped = np.zeros((height, width, 4), dtype=np.float32)
source_height, source_width = rgba.shape[:2]
source_left = max(0, left)
source_top = max(0, top)
source_right = min(source_width, right)
source_bottom = min(source_height, bottom)
if source_right > source_left and source_bottom > source_top:
cropped[
source_top - top : source_bottom - top,
source_left - left : source_right - left,
] = rgba[source_top:source_bottom, source_left:source_right]
image = bpy.data.images.new(
color_data_name(source_object),
width=width,
height=height,
alpha=True,
)
try:
image.colorspace_settings.name = source.colorspace_settings.name
image.alpha_mode = source.alpha_mode
image.pixels.foreach_set(np.flipud(cropped).ravel())
image.pack()
image.update()
except Exception:
bpy.data.images.remove(image, do_unlink=True)
raise
return image
def _cutout_regions(
alpha, mesh_detail, threshold=CUTOUT_ALPHA_THRESHOLD
):
contour_resolution = max(512, min(int(mesh_detail) * 8, 1024))
return alpha_regions(
alpha,
max_resolution=contour_resolution,
threshold=threshold,
)
def _antialias_selection(selection):
import numpy as np
values = np.asarray(selection, dtype=np.float32)
kernel = np.asarray((0.029, 0.235, 0.472, 0.235, 0.029), dtype=np.float32)
radius = len(kernel) // 2
horizontal = np.zeros_like(values)
padded = np.pad(values, ((0, 0), (radius, radius)), mode="edge")
for offset, weight in enumerate(kernel):
horizontal += padded[:, offset : offset + values.shape[1]] * weight
result = np.zeros_like(values)
padded = np.pad(horizontal, ((radius, radius), (0, 0)), mode="edge")
for offset, weight in enumerate(kernel):
result += padded[offset : offset + values.shape[0]] * weight
return result
def _geometry_cutout_regions(
alpha,
polygon,
mesh_detail,
invert=False,
refine_selection=False,
selection=None,
):
import numpy as np
values = np.asarray(alpha, dtype=np.float32)
if values.ndim != 2:
raise ValueError("Cutout Alpha must be two-dimensional")
source_height, source_width = values.shape
source_size = (source_width, source_height)
if refine_selection:
geometry_alpha = np.where(values > CUTOUT_ALPHA_THRESHOLD, values, 0.0)
regions, _image_size = _cutout_regions(geometry_alpha, mesh_detail)
return regions, source_size
geometry_alpha = selection
if geometry_alpha is None:
geometry_alpha = _antialias_selection(
polygon_mask(source_size, polygon, "NONZERO")
)
geometry_alpha = visible_selection_mask(
values,
geometry_alpha,
invert,
alpha_threshold=CUTOUT_ALPHA_THRESHOLD,
)
regions, _image_size = _cutout_regions(geometry_alpha, mesh_detail)
return regions, source_size
def _local_cutout_rgba(source_object, polygon, invert=False):
import numpy as np
rgba = _image_rgba(source_object.data).copy()
height, width = rgba.shape[:2]
selection = _antialias_selection(
polygon_mask((width, height), polygon, "NONZERO")
)
selection = visible_selection_mask(
rgba[:, :, 3],
selection,
invert,
alpha_threshold=CUTOUT_ALPHA_THRESHOLD,
)
rgba[:, :, 3] *= selection
rgba[rgba[:, :, 3] <= 0.0, :3] = 0.0
if not np.any(rgba[:, :, 3] > 0.0):
raise ImageEditWarning("The selection contains no visible image pixels")
return rgba, selection
def _new_material(
context,
source_object,
color_image,
normal_image=None,
normal_space="TANGENT",
):
return create_material(
material_data_name(source_object),
color_image,
0.0,
normal_image=normal_image,
normal_space=normal_space,
scene=context.scene,
)
def build_cutout_object(
context,
source_object,
shape,
detail,
material,
field=None,
regions=None,
image_size=None,
bounds=None,
keep_original=False,
refine_selection=False,
):
import numpy as np
bounds = bounds or plane_bounds(source_object)
crop_center = (
(bounds[0] + bounds[1]) * 0.5,
(bounds[2] + bounds[3]) * 0.5,
)
world_metric, world_z_scale = _world_plane_metric(source_object.matrix_world)
depth_image = None
depth_reference = None
depth_uniform_scale = None
depth_direction = None
preferences = addon_preferences(context)
balloon_algorithm = getattr(
preferences, "balloon_profile_algorithm", "POISSON"
)
point_spacing = _cutout_point_spacing(source_object, detail)
cleanup_area = _cutout_cleanup_area(point_spacing, world_metric)
domain = _cutout_domain(
regions,
image_size,
bounds,
point_spacing,
world_metric,
balloon_algorithm,
)
source_vertices, faces, uv, boundary, planar, balloon_profile = domain
faces = _counter_clockwise_faces(planar, faces)
vertices = _orient_cutout_vertices(source_vertices)
if world_z_scale <= 1e-10:
raise ValueError("The Image Empty has a degenerate world transform")
balloon_profile /= world_z_scale
if shape in {"DEPTH_BALLOON", "DEPTH_SURFACE"}:
(
points,
_intrinsics,
valid,
calibration_valid,
_image_size,
_reference,
) = field
depth_image = _surface_depth_image(
source_object, points, valid, points[..., 2]
)
depth_uniform_scale = _surface_uniform_scale(
source_object,
field,
bounds,
)
if not refine_selection:
depth_reference = UNREFINED_DEPTH_REFERENCE
if shape == "DEPTH_BALLOON":
depth_direction = DEFAULT_DEPTH_DIRECTION
elif shape == "DEPTH_SURFACE":
depth_reference = metric_depth_reference(
points[..., 2],
calibration_valid,
depth_uniform_scale,
)
else:
depth_reference, depth_direction = depth_balloon_projection_parameters(
vertices,
uv,
points[..., 2],
calibration_valid,
depth_uniform_scale,
)
source_name = source_object.name
mesh = bpy.data.meshes.new(source_name)
mesh.from_pydata(np.asarray(vertices).tolist(), [], faces)
for polygon in mesh.polygons:
polygon.use_smooth = True
_assign_uv(mesh, uv)
_assign_float_point_attribute(mesh, BALLOON_ATTRIBUTE_NAME, balloon_profile)
mesh.materials.append(material)
mesh.update()
mesh_object = bpy.data.objects.new(source_name, mesh)
context.collection.objects.link(mesh_object)
mesh_object.matrix_world = _cutout_world_matrix(
source_object.matrix_world, crop_center
)
mesh_object["anyimage_mesh_shape"] = shape
node_group = cutout_node_group(shape)
modifier = mesh_object.modifiers.new(name=node_group.name, type="NODES")
modifier.node_group = node_group
modifier.show_group_selector = False
set_modifier_input(
modifier,
modifier_input_identifier(node_group, "Shape"),
CUTOUT_SHAPE_INDICES[shape],
)
set_modifier_input(
modifier,
modifier_input_identifier(node_group, "Cleanup"),
cleanup_area,
)
set_modifier_input(
modifier,
modifier_input_identifier(node_group, "Smooth Iterations"),
0,
)
set_modifier_input(
modifier,
modifier_input_identifier(node_group, "Smooth"),
1.0,
)
thickness = 1.0 if shape in {"BALLOON", "DEPTH_BALLOON"} else 0.0
set_modifier_input(
modifier,
modifier_input_identifier(node_group, "Thickness"),
thickness,
)
if shape in {"DEPTH_BALLOON", "DEPTH_SURFACE"}:
set_modifier_input(
modifier,
modifier_input_identifier(
node_group,
"Depth Scale",
),
1.0,
)
if shape == "DEPTH_SURFACE":
set_modifier_input(
modifier,
modifier_input_identifier(node_group, "Uniform Scale"),
depth_uniform_scale,
)
set_modifier_input(
modifier,
modifier_input_identifier(node_group, "Reference Depth"),
depth_reference,
)
if shape == "DEPTH_BALLOON":
set_modifier_input(
modifier,
modifier_input_identifier(node_group, "Reference Depth"),
depth_reference,
)
set_modifier_input(
modifier,
modifier_input_identifier(node_group, "Depth Direction"),
depth_direction,
)
set_modifier_input(
modifier,
modifier_input_identifier(node_group, "Uniform Scale"),
depth_uniform_scale,
)
if depth_image is not None:
set_modifier_input(
modifier,
modifier_input_identifier(node_group, "Depth Image"),
depth_image,
)
for selected in context.selected_objects:
selected.select_set(False)
mesh_object.select_set(True)
context.view_layer.objects.active = mesh_object
if not keep_original:
bpy.data.objects.remove(source_object, do_unlink=True)
mesh_object.name = source_name
return mesh_object
def _load_normal_image(path, source_object):
image = bpy.data.images.load(str(path), check_existing=False)
image.name = normal_data_name(source_object)
image.colorspace_settings.name = "Non-Color"
image.pack()
return image
def _orient_cutout_object_normal(image):
import numpy as np
width, height = (int(value) for value in image.size)
pixels = np.empty(width * height * 4, dtype=np.float32)
image.pixels.foreach_get(pixels)
rgba = pixels.reshape((height, width, 4))
source = rgba[:, :, :3] * 2.0 - 1.0
oriented = np.empty_like(source)
oriented[:, :, 0] = -source[:, :, 2]
oriented[:, :, 1] = -source[:, :, 0]
oriented[:, :, 2] = source[:, :, 1]
rgba[:, :, :3] = np.clip(oriented * 0.5 + 0.5, 0.0, 1.0)
image.pixels.foreach_set(rgba.ravel())
image.update()
def limited_cutout_mesh_detail(mesh_detail, crop_size, max_input_size):
crop_width, crop_height = (int(value) for value in crop_size)
limited_width, limited_height = limited_image_size(
(crop_width, crop_height),
max_input_size,
)
scale = max(limited_width, limited_height) / max(crop_width, crop_height)
return max(4, min(1024, round(int(mesh_detail) * scale)))
def _cutout_point_spacing(source_object, detail):
import math
x_min, x_max, y_min, y_max = plane_bounds(source_object)
metric, _world_z_scale = _world_plane_metric(source_object.matrix_world)
width = (x_max - x_min) * math.sqrt(max(metric[0][0], 0.0))
height = (y_max - y_min) * math.sqrt(max(metric[1][1], 0.0))
return max(width, height) / max(int(detail), 8)
def build_local_cutout(
operator,
context,
source_object,
shape,
selection_data,
invert=False,
):
if video_total_frames(source_object.data) != 1:
raise ValueError("Mesh conversion currently supports still images")
polygon = json.loads(selection_data) if selection_data else None
if not isinstance(polygon, list) or len(polygon) < 3:
raise ValueError("The Cutout selection is invalid")
_rgba, selection = _local_cutout_rgba(source_object, polygon, invert)
source_size = tuple(source_object.data.size)
crop_bounds = _mask_crop_bounds(selection)
left, top, right, bottom = crop_bounds
cropped_selection = selection[top:bottom, left:right]
mesh_detail = limited_cutout_mesh_detail(
operator.mesh_detail,
(right - left, bottom - top),
configured_max_ai_input_size(context),
)
regions, image_size = _cutout_regions(
cropped_selection,
mesh_detail,
threshold=CUTOUT_ALPHA_THRESHOLD,
)
local_bounds = _crop_plane_bounds(
source_object,
crop_bounds,
source_size,
)
color_image = None
material = None
try:
color_image = _cropped_color_image(source_object, crop_bounds)
material = _new_material(context, source_object, color_image)
build_cutout_object(
context,
source_object,
shape,
mesh_detail,
material,
regions=regions,
image_size=image_size,
bounds=local_bounds,
keep_original=True,
)
except Exception:
if material is not None and material.users == 0:
bpy.data.materials.remove(material)
if color_image is not None and color_image.users == 0:
bpy.data.images.remove(color_image)
raise
push_undo(f"Convert to {shape.replace('_', ' ').title()}")
def build_cutout_response(context, result, operator):
source_object = bpy.data.objects.get(operator.source_object_name)
if not is_image_empty(source_object):
raise RuntimeError("The source Image Empty no longer exists")
source_size = tuple(source_object.data.size)
crop_bounds = tuple(
int(value)
for value in json.loads(
result.file("bounds").read_text(encoding="utf-8")
)
)
local_bounds = _crop_plane_bounds(
source_object,
crop_bounds,
source_size,
)
selection_image = bpy.data.images.load(
str(result.file("selection")),
check_existing=False,
)
try:
selection_alpha = _image_rgba(selection_image)[:, :, 3]
mesh_detail = limited_cutout_mesh_detail(
operator.mesh_detail,
selection_image.size,
operator.max_input_size,
)
regions, image_size = _cutout_regions(
selection_alpha,
mesh_detail,
threshold=CUTOUT_ALPHA_THRESHOLD,
)
finally:
bpy.data.images.remove(selection_image)
color_image = create_color_image(
source_object,
result.directory,
assign_to_source=False,
)
normal_path = result.directory / "normal.png"
normal_image = (
_load_normal_image(normal_path, source_object)
if normal_path.is_file()
else None
)
if normal_image is not None and operator.shape == "DEPTH_SURFACE":
_orient_cutout_object_normal(normal_image)
material = _new_material(
context,
source_object,
color_image,
normal_image,
normal_space=(
"OBJECT"
if operator.shape == "DEPTH_SURFACE" and normal_image is not None
else "TANGENT"
),
)
field = (
_load_field(result.file("field"))
if operator.shape in {"DEPTH_BALLOON", "DEPTH_SURFACE"}
else None
)
mesh_object = build_cutout_object(
context,
source_object,
operator.shape,
mesh_detail,
material,
field=field,
regions=regions,
image_size=image_size,
bounds=local_bounds,
keep_original=True,
refine_selection=operator.refine_selection,
)
if normal_image is not None and operator.shape != "DEPTH_SURFACE":
bake_tangent_normal_image(
context,
source_object,
mesh_object,
normal_image,
)
push_undo(f"Convert to {operator.shape.replace('_', ' ').title()}")

1
src/anyimage/operators/cutout_tool/geometry/__init__.py Normal file
View File

@@ -0,0 +1 @@
"""Cutout 的内存几何算法。"""

120
src/anyimage/operators/cutout_tool/balloon_profile.py → src/anyimage/operators/cutout_tool/geometry/balloon.py
View File

@@ -231,31 +231,49 @@ def cotangent_poisson_field(
import numpy as np import numpy as np
points = np.asarray(planar_points, dtype=np.float64) points = np.asarray(planar_points, dtype=np.float64)
faces = np.asarray(planar_faces, dtype=np.int64)
if faces.size == 0:
faces = np.empty((0, 3), dtype=np.int64)
if faces.ndim != 2 or faces.shape[1] != 3:
raise ValueError("Balloon profile faces must be triangles")
boundary_mask = np.asarray(boundary_mask, dtype=bool) boundary_mask = np.asarray(boundary_mask, dtype=bool)
loads = np.zeros(len(points), dtype=np.float64) triangles = points[faces]
edge_weights = {} a, b, c = (triangles[:, index] for index in range(3))
cross = np.abs(
def add_edge(start, end, weight): (b[:, 0] - a[:, 0]) * (c[:, 1] - a[:, 1])
edge = tuple(sorted((int(start), int(end)))) - (b[:, 1] - a[:, 1]) * (c[:, 0] - a[:, 0])
edge_weights[edge] = edge_weights.get(edge, 0.0) + weight )
usable = cross > 1e-14
for face in planar_faces: faces = faces[usable]
first, second, third = (int(index) for index in face) a, b, c = a[usable], b[usable], c[usable]
a, b, c = points[[first, second, third]] cross = cross[usable]
cross = abs( loads = np.bincount(
(b[0] - a[0]) * (c[1] - a[1]) faces.ravel(),
- (b[1] - a[1]) * (c[0] - a[0]) weights=np.repeat(cross / 6.0, 3),
minlength=len(points),
)
edges = np.concatenate(
(
faces[:, (1, 2)],
faces[:, (0, 2)],
faces[:, (0, 1)],
),
axis=0,
)
edge_weights = np.concatenate(
(
np.sum((b - a) * (c - a), axis=1),
np.sum((a - b) * (c - b), axis=1),
np.sum((a - c) * (b - c), axis=1),
) )
if cross <= 1e-14: ) / np.tile(cross * 2.0, 3)
continue edges.sort(axis=1)
area = cross * 0.5 edges, inverse = np.unique(edges, axis=0, return_inverse=True)
loads[[first, second, third]] += area / 3.0 edge_weights = np.bincount(
cotangent_a = float((b - a).dot(c - a) / cross) inverse,
cotangent_b = float((a - b).dot(c - b) / cross) weights=edge_weights,
cotangent_c = float((a - c).dot(b - c) / cross) minlength=len(edges),
add_edge(second, third, cotangent_a * 0.5) )
add_edge(first, third, cotangent_b * 0.5)
add_edge(first, second, cotangent_c * 0.5)
interior = np.flatnonzero(~boundary_mask) interior = np.flatnonzero(~boundary_mask)
field = np.zeros(len(points), dtype=np.float64) field = np.zeros(len(points), dtype=np.float64)
@@ -263,38 +281,46 @@ def cotangent_poisson_field(
return field return field
compact = np.full(len(points), -1, dtype=np.int64) compact = np.full(len(points), -1, dtype=np.int64)
compact[interior] = np.arange(len(interior), dtype=np.int64) compact[interior] = np.arange(len(interior), dtype=np.int64)
diagonal = np.zeros(len(interior), dtype=np.float64)
edge_starts = []
edge_ends = []
weights = []
# Constrained triangulation can create negative cotangent weights. # Constrained triangulation can create negative cotangent weights.
# Clamping them keeps the Dirichlet system positive definite. # Clamping them keeps the Dirichlet system positive definite.
for (start, end), weight in edge_weights.items(): edge_weights = np.maximum(edge_weights, 0.0)
weight = max(float(weight), 0.0) usable = edge_weights > 1e-14
if weight <= 1e-14: compact_edges = compact[edges[usable]]
continue edge_weights = edge_weights[usable]
start_compact = compact[start] start_compact = compact_edges[:, 0]
end_compact = compact[end] end_compact = compact_edges[:, 1]
if start_compact >= 0: start_usable = start_compact >= 0
diagonal[start_compact] += weight end_usable = end_compact >= 0
if end_compact >= 0: diagonal = np.zeros(len(interior), dtype=np.float64)
diagonal[end_compact] += weight diagonal += np.bincount(
if start_compact >= 0 and end_compact >= 0: start_compact[start_usable],
edge_starts.append(start_compact) weights=edge_weights[start_usable],
edge_ends.append(end_compact) minlength=len(interior),
weights.append(weight) )
diagonal += np.bincount(
end_compact[end_usable],
weights=edge_weights[end_usable],
minlength=len(interior),
)
if np.any(diagonal <= 1e-14): if np.any(diagonal <= 1e-14):
raise ValueError("Unable to build a stable Balloon profile") raise ValueError("Unable to build a stable Balloon profile")
edge_starts = np.asarray(edge_starts, dtype=np.int64) interior_edges = start_usable & end_usable
edge_ends = np.asarray(edge_ends, dtype=np.int64) edge_starts = start_compact[interior_edges]
weights = np.asarray(weights, dtype=np.float64) edge_ends = end_compact[interior_edges]
weights = edge_weights[interior_edges]
directed_starts = np.concatenate((edge_starts, edge_ends))
directed_ends = np.concatenate((edge_ends, edge_starts))
directed_weights = np.concatenate((weights, weights))
def multiply(values): def multiply(values):
result = diagonal * values result = diagonal * values
if len(weights): if len(directed_weights):
np.add.at(result, edge_starts, -weights * values[edge_ends]) result -= np.bincount(
np.add.at(result, edge_ends, -weights * values[edge_starts]) directed_starts,
weights=directed_weights * values[directed_ends],
minlength=len(values),
)
return result return result
right_hand_side = loads[interior] right_hand_side = loads[interior]

11
src/anyimage/operators/cutout_tool/depth_calibration.py → src/anyimage/operators/cutout_tool/geometry/depth_calibration.py
View File

@@ -3,10 +3,19 @@
PLANE_VARIANCE_FLOOR_RATIO = 0.025 PLANE_VARIANCE_FLOOR_RATIO = 0.025
DEPTH_REFERENCE_PERCENTILE = 95.0 DEPTH_REFERENCE_PERCENTILE = 95.0
UNREFINED_DEPTH_REFERENCE = 8.0 ADAPTIVE_DEPTH_TRANSPARENCY_RATIO = 0.05
DEFAULT_DEPTH_BASELINE = 8.0
DEFAULT_DEPTH_DIRECTION = (0.0, 0.0, 1.0) DEFAULT_DEPTH_DIRECTION = (0.0, 0.0, 1.0)
def use_adaptive_depth_calibration(region_transparent_fraction):
"""Return whether the Region input has enough transparent context."""
return (
float(region_transparent_fraction)
>= ADAPTIVE_DEPTH_TRANSPARENCY_RATIO
)
def metric_depth_reference(camera_depth, valid, uniform_scale): def metric_depth_reference(camera_depth, valid, uniform_scale):
"""Return the shared upper-percentile scene depth for both Depth shapes.""" """Return the shared upper-percentile scene depth for both Depth shapes."""
import numpy as np import numpy as np

81
src/anyimage/operators/cutout_tool/mask_regions.py → src/anyimage/operators/cutout_tool/geometry/region_contours.py
View File

@@ -1,4 +1,34 @@
"""Convert a Cutout Alpha mask into outer contours and holes.""" """Extract Cutout Region contours from a Region Mask."""
from dataclasses import dataclass
@dataclass(frozen=True)
class RegionContours:
components: tuple
image_size: tuple[int, int]
def __post_init__(self):
components = tuple(
tuple(
tuple((float(x), float(y)) for x, y in contour)
for contour in component
)
for component in self.components
)
image_size = tuple(int(value) for value in self.image_size)
if not components or any(not component for component in components):
raise ValueError("The Region has no usable contour components")
if any(
len(contour) < 3
for component in components
for contour in component
):
raise ValueError("A Region contour has fewer than three points")
if len(image_size) != 2 or min(image_size) <= 0:
raise ValueError("The Region contour image size is invalid")
object.__setattr__(self, "components", components)
object.__setattr__(self, "image_size", image_size)
def _choose_boundary_edge(start, direction, candidates): def _choose_boundary_edge(start, direction, candidates):
@@ -27,7 +57,7 @@ def _simplify_axis_aligned_contour(points):
return simplified return simplified
def trace_mask_contours(mask): def _trace_region_contours(mask):
import numpy as np import numpy as np
mask = np.asarray(mask, dtype=bool) mask = np.asarray(mask, dtype=bool)
@@ -113,8 +143,8 @@ def _point_inside_contour(point, contour):
return inside return inside
def mask_regions(mask): def _group_region_contours(mask):
contours = trace_mask_contours(mask) contours = _trace_region_contours(mask)
outer = [contour for contour in contours if _signed_contour_area(contour) > 0.0] outer = [contour for contour in contours if _signed_contour_area(contour) > 0.0]
holes = [contour for contour in contours if _signed_contour_area(contour) < 0.0] holes = [contour for contour in contours if _signed_contour_area(contour) < 0.0]
if not outer: if not outer:
@@ -143,7 +173,7 @@ def mask_regions(mask):
return regions return regions
def _resize_alpha(values, size): def _resize_values(values, size):
import numpy as np import numpy as np
target_width, target_height = (int(value) for value in size) target_width, target_height = (int(value) for value in size)
@@ -174,10 +204,10 @@ def _resize_alpha(values, size):
) )
def alpha_regions(alpha, max_resolution=512, threshold=0.5): def extract_region_contours(region_mask, max_resolution=512, threshold=0.5):
import numpy as np import numpy as np
values = np.asarray(alpha) values = np.asarray(region_mask.values)
if values.ndim != 2: if values.ndim != 2:
raise ValueError("Cutout Alpha must be two-dimensional") raise ValueError("Cutout Alpha must be two-dimensional")
if np.issubdtype(values.dtype, np.floating): if np.issubdtype(values.dtype, np.floating):
@@ -188,33 +218,24 @@ def alpha_regions(alpha, max_resolution=512, threshold=0.5):
scale = min(1.0, float(max_resolution) / max(source_size)) scale = min(1.0, float(max_resolution) / max(source_size))
sampled_size = tuple(max(1, round(value * scale)) for value in source_size) sampled_size = tuple(max(1, round(value * scale)) for value in source_size)
if sampled_size != source_size: if sampled_size != source_size:
values = _resize_alpha(values, sampled_size) values = _resize_values(values, sampled_size)
mask = values >= 255.0 * threshold mask = values >= 255.0 * threshold
regions = mask_regions(mask) components = _group_region_contours(mask)
scale_x = source_size[0] / sampled_size[0] scale_x = source_size[0] / sampled_size[0]
scale_y = source_size[1] / sampled_size[1] scale_y = source_size[1] / sampled_size[1]
return ( return RegionContours(
[ components=tuple(
[ tuple(
[ tuple(
[ (
min(max(float(x) * scale_x, 0.0), source_size[0]), min(max(float(x) * scale_x, 0.0), source_size[0]),
min(max(float(y) * scale_y, 0.0), source_size[1]), min(max(float(y) * scale_y, 0.0), source_size[1]),
] )
for x, y in contour for x, y in contour
] )
for contour in region for contour in component
] )
for region in regions for component in components
], ),
source_size, image_size=source_size,
) )
def image_alpha_regions(path, max_resolution=512, threshold=0.5):
import numpy as np
from PIL import Image
with Image.open(path) as source:
alpha = np.asarray(source.convert("RGBA").getchannel("A"), dtype=np.uint8)
return alpha_regions(alpha, max_resolution=max_resolution, threshold=threshold)

94
src/anyimage/operators/cutout_tool/planar_mesh.py → src/anyimage/operators/cutout_tool/geometry/triangulation.py
View File

@@ -3,7 +3,7 @@ import math
POINT_SPACING = 0.1 POINT_SPACING = 0.1
def polygon_area(points): def contour_area(points):
import numpy as np import numpy as np
points = np.asarray(points, dtype=np.float64) points = np.asarray(points, dtype=np.float64)
@@ -16,10 +16,10 @@ def polygon_area(points):
) )
def resample_closed_polygon(polygon, spacing): def resample_closed_contour(contour, spacing):
import numpy as np import numpy as np
points = np.asarray(polygon, dtype=np.float64) points = np.asarray(contour, dtype=np.float64)
if len(points) < 3: if len(points) < 3:
raise ValueError("The lasso does not define a valid image region") raise ValueError("The lasso does not define a valid image region")
segments = np.roll(points, -1, axis=0) - points segments = np.roll(points, -1, axis=0) - points
@@ -40,9 +40,9 @@ def resample_closed_polygon(polygon, spacing):
return points[indices] + segments[indices] * factors[:, None] return points[indices] + segments[indices] * factors[:, None]
def smooth_lasso_contour(polygon, spacing, iterations=16): def smooth_region_contour(contour, spacing, iterations=16):
contour = resample_closed_polygon(polygon, spacing * 0.25) contour = resample_closed_contour(contour, spacing * 0.25)
original_area = abs(polygon_area(contour)) original_area = abs(contour_area(contour))
for _index in range(iterations): for _index in range(iterations):
import numpy as np import numpy as np
@@ -51,14 +51,14 @@ def smooth_lasso_contour(polygon, spacing, iterations=16):
+ contour * 0.75 + contour * 0.75
+ np.roll(contour, -1, axis=0) * 0.125 + np.roll(contour, -1, axis=0) * 0.125
) )
smooth_area = abs(polygon_area(contour)) smooth_area = abs(contour_area(contour))
if original_area > 0.0 and smooth_area > 0.0: if original_area > 0.0 and smooth_area > 0.0:
center = contour.mean(axis=0) center = contour.mean(axis=0)
contour = center + (contour - center) * math.sqrt( contour = center + (contour - center) * math.sqrt(
original_area / smooth_area original_area / smooth_area
) )
contour = resample_closed_polygon(contour, spacing) contour = resample_closed_contour(contour, spacing)
if polygon_area(contour) < 0.0: if contour_area(contour) < 0.0:
contour = contour[::-1].copy() contour = contour[::-1].copy()
return contour return contour
@@ -98,10 +98,6 @@ def _edge_bucket_indices(starts, ends, minimum_y, bucket_height, bucket_count):
return buckets return buckets
def points_inside_polygon(points, polygon):
return points_inside_contours(points, (polygon,))
def points_inside_contours(points, contours): def points_inside_contours(points, contours):
import numpy as np import numpy as np
@@ -236,10 +232,6 @@ def uniform_interior_points_for_contours(contours, spacing):
) )
def uniform_interior_points(contour, spacing):
return uniform_interior_points_for_contours((contour,), spacing)
def metric_transform(world_metric=None): def metric_transform(world_metric=None):
import numpy as np import numpy as np
@@ -259,7 +251,7 @@ def metric_transform(world_metric=None):
def prepare_planar_contours( def prepare_planar_contours(
polygons, contours,
image_size, image_size,
local_bounds, local_bounds,
spacing=POINT_SPACING, spacing=POINT_SPACING,
@@ -274,9 +266,9 @@ def prepare_planar_contours(
local_width = x_max - x_min local_width = x_max - x_min
local_height = y_max - y_min local_height = y_max - y_min
transform = metric_transform(world_metric) transform = metric_transform(world_metric)
contours = [] prepared = []
for polygon in polygons: for contour in contours:
pixels = np.asarray(polygon, dtype=np.float64) pixels = np.asarray(contour, dtype=np.float64)
if pixels.ndim != 2 or pixels.shape[0] < 3 or pixels.shape[1] != 2: if pixels.ndim != 2 or pixels.shape[0] < 3 or pixels.shape[1] != 2:
continue continue
local = np.empty_like(pixels) local = np.empty_like(pixels)
@@ -289,30 +281,29 @@ def prepare_planar_contours(
if perimeter <= 1e-12: if perimeter <= 1e-12:
continue continue
contour_spacing = min(spacing, perimeter / 8.0) contour_spacing = min(spacing, perimeter / 8.0)
smoothed = smooth_lasso_contour(planar, contour_spacing) smoothed = smooth_region_contour(planar, contour_spacing)
smoothed_local = smoothed @ np.linalg.inv(transform) smoothed_local = smoothed @ np.linalg.inv(transform)
smoothed_local[:, 0] = np.clip(smoothed_local[:, 0], x_min, x_max) smoothed_local[:, 0] = np.clip(smoothed_local[:, 0], x_min, x_max)
smoothed_local[:, 1] = np.clip(smoothed_local[:, 1], y_min, y_max) smoothed_local[:, 1] = np.clip(smoothed_local[:, 1], y_min, y_max)
contours.append(smoothed_local @ transform) prepared.append(smoothed_local @ transform)
if not contours: if not prepared:
raise ValueError("The Cutout has no region large enough for this resolution") raise ValueError("The Cutout has no region large enough for this resolution")
interior = uniform_interior_points_for_contours(contours, spacing) interior = uniform_interior_points_for_contours(prepared, spacing)
return contours, interior, spacing return prepared, interior, spacing
def prepare_planar_regions( def _prepare_region_contours(
regions, region_contours,
image_size,
local_bounds, local_bounds,
spacing=POINT_SPACING, spacing=POINT_SPACING,
world_metric=None, world_metric=None,
): ):
prepared = [] prepared = []
errors = [] errors = []
for region in regions: for component in region_contours.components:
try: try:
contours, interior, _spacing = prepare_planar_contours( contours, interior, _spacing = prepare_planar_contours(
region, component,
image_size, region_contours.image_size,
local_bounds, local_bounds,
spacing=spacing, spacing=spacing,
world_metric=world_metric, world_metric=world_metric,
@@ -328,25 +319,7 @@ def prepare_planar_regions(
return prepared, spacing return prepared, spacing
def prepare_planar_points( def _triangulate_contour_component(contours, interior, spacing):
polygon,
image_size,
local_bounds,
spacing=POINT_SPACING,
world_metric=None,
):
contours, interior, spacing = prepare_planar_contours(
(polygon,),
image_size,
local_bounds,
spacing=spacing,
world_metric=world_metric,
)
contour = contours[0]
return contour, interior, spacing
def triangulate_planar_contours(contours, interior, spacing):
import numpy as np import numpy as np
from mathutils import Vector from mathutils import Vector
from mathutils.geometry import delaunay_2d_cdt from mathutils.geometry import delaunay_2d_cdt
@@ -413,16 +386,28 @@ def boundary_vertices_from_faces(vertex_count, faces):
return boundary return boundary
def triangulate_planar_regions(regions, spacing, return_components=False): def triangulate_region_contours(
region_contours,
local_bounds,
spacing=POINT_SPACING,
world_metric=None,
return_components=False,
):
import numpy as np import numpy as np
prepared, spacing = _prepare_region_contours(
region_contours,
local_bounds,
spacing=spacing,
world_metric=world_metric,
)
points = [] points = []
faces = [] faces = []
boundaries = [] boundaries = []
components = [] components = []
offset = 0 offset = 0
for contours, interior in regions: for contours, interior in prepared:
region_points, region_faces, region_boundary = triangulate_planar_contours( region_points, region_faces, region_boundary = _triangulate_contour_component(
contours, contours,
interior, interior,
spacing, spacing,
@@ -449,4 +434,3 @@ def triangulate_planar_regions(regions, spacing, return_components=False):
np.concatenate(boundaries), np.concatenate(boundaries),
) )
return (*result, components) if return_components else result return (*result, components) if return_components else result

10
src/anyimage/operators/cutout_tool/node_assets.py
View File

@@ -1,10 +0,0 @@
"""Load Geometry Nodes assets used by Cutout shapes."""
from ..common.node_asset import load_asset_node_group
from .shape_types import CUTOUT_NODE_GROUP_NAME, CUTOUT_SHAPES
def cutout_node_group(shape):
if shape not in CUTOUT_SHAPES:
raise ValueError(f"Unsupported Cutout shape: {shape}")
return load_asset_node_group(CUTOUT_NODE_GROUP_NAME)

842
src/anyimage/operators/cutout_tool/region_to_shape.py Normal file
View File

@@ -0,0 +1,842 @@
import json
from dataclasses import dataclass
from enum import StrEnum
import bpy
from ...preferences import addon_preferences, configured_max_ai_input_size
from ...properties import BEN2_MODEL_KEY, DEFAULT_MOGE2_MODEL_KEY
from ...runtime import JobOperatorBase
from ...server.media.resolution import limited_image_size
from ...server.media.resolution import (
DEFAULT_MAX_AI_INPUT_SIZE,
MAX_MAX_AI_INPUT_SIZE,
MIN_MAX_AI_INPUT_SIZE,
)
from ..common.image_data import (
cleanup_image_input,
color_data_name,
depth_data_name,
image_rgba,
load_normal_result_image,
image_empty_bounds,
prepare_region_input,
require_image_empty,
source_start_frame,
video_total_frames,
)
from ..common.image_interaction import report_image_edit_exception
from ..common.image_region import (
ImageEditWarning,
RegionMask,
RegionPath,
rasterize_region_path,
)
from ..common.mesh_result import (
create_image_material,
finalize_object_result,
load_asset_node_group,
load_image_geometry,
modifier_input_identifier,
set_modifier_input,
)
from ..common.ai_job import (
invoke_ai_setup_if_needed,
moge2_parameters,
production_device,
require_environment,
require_input_path,
require_model,
)
from .geometry.depth_calibration import (
DEFAULT_DEPTH_BASELINE,
DEFAULT_DEPTH_DIRECTION,
depth_balloon_projection_parameters,
metric_depth_reference,
use_adaptive_depth_calibration,
)
from .geometry.triangulation import (
metric_transform,
triangulate_region_contours,
)
from .geometry.balloon import balloon_profile as calculate_balloon_profile
from .geometry.region_contours import extract_region_contours
CUTOUT_ALPHA_THRESHOLD = 0.1
CUTOUT_CLEANUP_UI_SCALE = 100.0
BALLOON_ATTRIBUTE_NAME = "o_balloon"
CUTOUT_NODE_GROUP_NAME = "O Image Cutout"
CUTOUT_NODE_GROUP_NAMES = {
"SURFACE": "O Image Surface",
"BALLOON": "O Image Balloon",
"DEPTH_BALLOON": "O Image Depth Balloon",
"DEPTH_SURFACE": "O Image Depth Surface",
}
CUTOUT_SHAPE_INDICES = {
shape: index for index, shape in enumerate(CUTOUT_NODE_GROUP_NAMES)
}
CUTOUT_SHAPES = frozenset(CUTOUT_NODE_GROUP_NAMES)
DEPTH_CUTOUT_SHAPES = frozenset({"DEPTH_BALLOON", "DEPTH_SURFACE"})
_HIDDEN = {"HIDDEN", "SKIP_SAVE"}
class NormalMode(StrEnum):
NONE = "NONE"
TANGENT = "TANGENT"
OBJECT = "OBJECT"
@dataclass(frozen=True)
class BaseShape:
vertices: object
faces: object
uv: object
balloon: object
CUTOUT_AXIS_TO_SOURCE = (
(0.0, -1.0, 0.0, 0.0),
(0.0, 0.0, 1.0, 0.0),
(-1.0, 0.0, 0.0, 0.0),
(0.0, 0.0, 0.0, 1.0),
)
def _cutout_depth_image(source_object, points, valid, camera_depth):
import numpy as np
points = np.asarray(points, dtype=np.float32)
valid = np.asarray(valid, dtype=bool)
height, width = valid.shape
if points.shape != (height, width, 3):
raise ValueError("Depth Surface point dimensions do not match")
discontinuity = _depth_discontinuity_map(camera_depth, valid)
discontinuity[~valid] = -1.0
pixels = np.empty((height, width, 4), dtype=np.float32)
pixels[:, :, :3] = points
pixels[:, :, 3] = discontinuity
image = bpy.data.images.new(
depth_data_name(source_object),
width=width,
height=height,
alpha=True,
float_buffer=True,
)
image.colorspace_settings.name = "Non-Color"
image.alpha_mode = "CHANNEL_PACKED"
image.pixels.foreach_set(np.flipud(pixels).ravel())
image.pack()
return image
def _cutout_world_matrix(source_matrix, crop_center):
from mathutils import Matrix
center_x, center_y = (float(value) for value in crop_center)
return (
source_matrix
@ Matrix.Translation((center_x, center_y, 0.0))
@ Matrix(CUTOUT_AXIS_TO_SOURCE)
)
def _orient_cutout_vertices(vertices):
import numpy as np
vertices = np.asarray(vertices, dtype=np.float64)
return np.column_stack(
(
-vertices[:, 2],
-vertices[:, 0],
vertices[:, 1],
)
)
def _cutout_domain(
region_contours,
bounds,
spacing,
world_metric,
balloon_algorithm="POISSON",
):
import numpy as np
x_min, x_max, y_min, y_max = bounds
planar, planar_faces, boundary, components = triangulate_region_contours(
region_contours,
bounds,
spacing=spacing,
world_metric=world_metric,
return_components=True,
)
profile = calculate_balloon_profile(
balloon_algorithm, planar, planar_faces, boundary, components
)
faces = np.asarray(planar_faces, dtype=np.int32)
local = planar @ np.linalg.inv(metric_transform(world_metric))
uv = np.column_stack(
(
(local[:, 0] - x_min) / (x_max - x_min),
(local[:, 1] - y_min) / (y_max - y_min),
)
).astype(np.float32)
crop_center = np.asarray(
((x_min + x_max) * 0.5, (y_min + y_max) * 0.5),
dtype=np.float64,
)
local -= crop_center
vertices = np.column_stack((local, np.zeros(len(local), dtype=np.float64)))
return vertices, faces, uv, boundary, planar, profile
def _world_plane_metric(matrix_world):
from mathutils import Vector
linear = matrix_world.to_3x3()
x = linear @ Vector((1.0, 0.0, 0.0))
y = linear @ Vector((0.0, 1.0, 0.0))
z = linear @ Vector((0.0, 0.0, 1.0))
return ((x.dot(x), x.dot(y)), (x.dot(y), y.dot(y))), z.length
def _cutout_cleanup_value(spacing, world_metric):
import numpy as np
if spacing <= 0.0:
raise ValueError("Cutout point spacing must be greater than zero")
area_scale = abs(float(np.linalg.det(metric_transform(world_metric))))
if area_scale <= 1e-12:
raise ValueError("The Image Empty has a degenerate world transform")
return CUTOUT_CLEANUP_UI_SCALE * float(spacing) ** 2 / area_scale
def _depth_uniform_scale(
source_object,
field,
bounds=None,
):
import numpy as np
intrinsics = field.intrinsics
width, height = field.image_size
x_min, x_max, y_min, y_max = bounds or image_empty_bounds(source_object)
plane_width = x_max - x_min
plane_height = y_max - y_min
fx, fy = float(intrinsics[0, 0]), float(intrinsics[1, 1])
if fx <= 0.0 or fy <= 0.0:
raise ValueError("Camera intrinsics contain an invalid focal length")
scale_x = plane_width * fx / (max(width - 1, 1) * field.reference_depth)
scale_y = plane_height * fy / (max(height - 1, 1) * field.reference_depth)
return float(np.sqrt(scale_x * scale_y))
def _depth_discontinuity_map(depth, valid):
import numpy as np
depth = np.asarray(depth, dtype=np.float32)
valid = np.asarray(valid, dtype=bool) & np.isfinite(depth) & (depth > 0.0)
padded_depth = np.pad(depth, 1, mode="edge")
padded_valid = np.pad(valid, 1, mode="edge")
windows = np.lib.stride_tricks.sliding_window_view(padded_depth, (3, 3))
usable = np.lib.stride_tricks.sliding_window_view(padded_valid, (3, 3))
minimum = np.min(np.where(usable, windows, np.inf), axis=(-2, -1))
maximum = np.max(np.where(usable, windows, -np.inf), axis=(-2, -1))
result = (maximum - minimum) / np.maximum(np.abs(depth), 1e-6)
result[~usable.all(axis=(-2, -1))] = 1.0
return result.astype(np.float32)
def _counter_clockwise_faces(points, faces):
import numpy as np
points = np.asarray(points, dtype=np.float64)
result = np.asarray(faces, dtype=np.int32).copy()
polygons = points[result]
following = np.roll(polygons, -1, axis=1)
signed_area = np.sum(
polygons[..., 0] * following[..., 1]
- polygons[..., 1] * following[..., 0],
axis=1,
)
result[signed_area < 0.0] = result[signed_area < 0.0, ::-1]
return result
def _assign_uv(mesh, uv):
import numpy as np
for layer in list(mesh.uv_layers):
mesh.uv_layers.remove(layer)
uv_layer = mesh.uv_layers.new(name="UVMap")
loop_vertices = np.empty(len(mesh.loops), dtype=np.int32)
mesh.loops.foreach_get("vertex_index", loop_vertices)
loop_uv = np.asarray(uv, dtype=np.float32)[loop_vertices]
uv_layer.data.foreach_set("uv", loop_uv.ravel())
def _assign_float_point_attribute(mesh, name, values):
import numpy as np
attribute = mesh.attributes.new(name=name, type="FLOAT", domain="POINT")
attribute.data.foreach_set("value", np.asarray(values, dtype=np.float32))
def _crop_plane_bounds(source_object, crop_bounds, source_size):
source_width, source_height = (float(value) for value in source_size)
left, top, right, bottom = (float(value) for value in crop_bounds)
x_min, x_max, y_min, y_max = image_empty_bounds(source_object)
width = x_max - x_min
height = y_max - y_min
return (
x_min + left * width / source_width,
x_min + right * width / source_width,
y_max - bottom * height / source_height,
y_max - top * height / source_height,
)
def _cropped_color_image(source_object, crop_bounds, source_rgba):
import numpy as np
source = source_object.data
rgba = np.asarray(source_rgba)
left, top, right, bottom = crop_bounds
width = right - left
height = bottom - top
if width <= 0 or height <= 0:
raise ValueError("The Cutout crop dimensions must be positive")
cropped = np.zeros((height, width, 4), dtype=np.float32)
source_height, source_width = rgba.shape[:2]
source_left = max(0, left)
source_top = max(0, top)
source_right = min(source_width, right)
source_bottom = min(source_height, bottom)
if source_right > source_left and source_bottom > source_top:
cropped[
source_top - top : source_bottom - top,
source_left - left : source_right - left,
] = rgba[source_top:source_bottom, source_left:source_right]
image = bpy.data.images.new(
color_data_name(source_object),
width=width,
height=height,
alpha=True,
)
try:
image.colorspace_settings.name = source.colorspace_settings.name
image.alpha_mode = source.alpha_mode
image.pixels.foreach_set(np.flipud(cropped).ravel())
image.pack()
image.update()
except Exception:
bpy.data.images.remove(image, do_unlink=True)
raise
return image
def _extract_cutout_contours(
region_mask,
mesh_detail,
threshold=CUTOUT_ALPHA_THRESHOLD,
):
contour_resolution = max(512, min(int(mesh_detail) * 8, 1024))
return extract_region_contours(
region_mask,
max_resolution=contour_resolution,
threshold=threshold,
)
def cutout_node_group(shape):
if shape not in CUTOUT_SHAPES:
raise ValueError(f"Unsupported Cutout shape: {shape}")
return load_asset_node_group(CUTOUT_NODE_GROUP_NAME)
def limited_cutout_mesh_detail(mesh_detail, crop_size, max_input_size):
crop_width, crop_height = (int(value) for value in crop_size)
limited_width, limited_height = limited_image_size(
(crop_width, crop_height),
max_input_size,
)
scale = max(limited_width, limited_height) / max(crop_width, crop_height)
return max(4, min(1024, round(int(mesh_detail) * scale)))
def _cutout_point_spacing(source_object, detail):
import math
x_min, x_max, y_min, y_max = image_empty_bounds(source_object)
metric, _world_z_scale = _world_plane_metric(source_object.matrix_world)
width = (x_max - x_min) * math.sqrt(max(metric[0][0], 0.0))
height = (y_max - y_min) * math.sqrt(max(metric[1][1], 0.0))
return max(width, height) / max(int(detail), 8)
def build_base_shape(context, source_object, detail, region_mask):
import numpy as np
bounds = _crop_plane_bounds(
source_object,
region_mask.bounds,
tuple(source_object.data.size),
)
region_contours = _extract_cutout_contours(
region_mask,
detail,
threshold=CUTOUT_ALPHA_THRESHOLD,
)
world_metric, world_z_scale = _world_plane_metric(source_object.matrix_world)
if world_z_scale <= 1e-10:
raise ValueError("The Image Empty has a degenerate world transform")
preferences = addon_preferences(context)
balloon_algorithm = getattr(
preferences, "balloon_profile_algorithm", "POISSON"
)
point_spacing = _cutout_point_spacing(source_object, detail)
domain = _cutout_domain(
region_contours,
bounds,
point_spacing,
world_metric,
balloon_algorithm,
)
source_vertices, faces, uv, _boundary, planar, balloon = domain
return BaseShape(
vertices=_orient_cutout_vertices(source_vertices),
faces=_counter_clockwise_faces(planar, faces),
uv=np.asarray(uv, dtype=np.float32),
balloon=np.asarray(balloon, dtype=np.float64) / world_z_scale,
)
def create_shape_object(
context,
source_object,
shape,
detail,
region_mask,
base_shape,
*,
field=None,
normal_image=None,
normal_mode=NormalMode.NONE,
source_rgba=None,
):
import numpy as np
normal_mode = NormalMode(normal_mode)
bounds = _crop_plane_bounds(
source_object,
region_mask.bounds,
tuple(source_object.data.size),
)
crop_center = (
(bounds[0] + bounds[1]) * 0.5,
(bounds[2] + bounds[3]) * 0.5,
)
world_metric, _world_z_scale = _world_plane_metric(source_object.matrix_world)
cleanup_value = _cutout_cleanup_value(
_cutout_point_spacing(source_object, detail),
world_metric,
)
color_image = _cropped_color_image(
source_object,
region_mask.bounds,
image_rgba(source_object.data) if source_rgba is None else source_rgba,
)
material = None
depth_image = None
mesh = None
mesh_object = None
try:
material = create_image_material(
source_object,
color_image,
0.0,
normal_image=normal_image,
normal_space=(
NormalMode.TANGENT.value
if normal_mode == NormalMode.NONE
else normal_mode.value
),
scene=context.scene,
)
depth_reference = None
depth_uniform_scale = None
depth_direction = None
if shape in DEPTH_CUTOUT_SHAPES:
points = field.points
valid = field.valid
calibration_valid = field.calibration_valid
depth_image = _cutout_depth_image(
source_object, points, valid, points[..., 2]
)
depth_uniform_scale = _depth_uniform_scale(
source_object,
field,
bounds,
)
if not use_adaptive_depth_calibration(
field.region_transparent_fraction
):
depth_reference = DEFAULT_DEPTH_BASELINE
if shape == "DEPTH_BALLOON":
depth_direction = DEFAULT_DEPTH_DIRECTION
elif shape == "DEPTH_SURFACE":
depth_reference = metric_depth_reference(
points[..., 2],
calibration_valid,
depth_uniform_scale,
)
else:
depth_reference, depth_direction = (
depth_balloon_projection_parameters(
base_shape.vertices,
base_shape.uv,
points[..., 2],
calibration_valid,
depth_uniform_scale,
)
)
source_name = source_object.name
mesh = bpy.data.meshes.new(source_name)
mesh.from_pydata(
np.asarray(base_shape.vertices).tolist(),
[],
base_shape.faces,
)
for face in mesh.polygons:
face.use_smooth = True
_assign_uv(mesh, base_shape.uv)
_assign_float_point_attribute(
mesh,
BALLOON_ATTRIBUTE_NAME,
base_shape.balloon,
)
mesh.materials.append(material)
mesh.update()
mesh_object = bpy.data.objects.new(source_name, mesh)
context.collection.objects.link(mesh_object)
mesh_object.matrix_world = _cutout_world_matrix(
source_object.matrix_world,
crop_center,
)
mesh_object["anyimage_mesh_shape"] = shape
node_group = cutout_node_group(shape)
modifier = mesh_object.modifiers.new(name=node_group.name, type="NODES")
modifier.node_group = node_group
modifier.show_group_selector = False
for name, value in (
("Shape", CUTOUT_SHAPE_INDICES[shape]),
("Cleanup", cleanup_value),
("Smooth Iterations", 0),
("Smooth", 1.0),
(
"Thickness",
1.0 if shape in {"BALLOON", "DEPTH_BALLOON"} else 0.0,
),
):
set_modifier_input(
modifier,
modifier_input_identifier(node_group, name),
value,
)
if shape in DEPTH_CUTOUT_SHAPES:
for name, value in (
("Depth Scale", 1.0),
("Uniform Scale", depth_uniform_scale),
("Base Plane Depth", depth_reference),
("Depth Image", depth_image),
):
set_modifier_input(
modifier,
modifier_input_identifier(node_group, name),
value,
)
if shape == "DEPTH_BALLOON":
set_modifier_input(
modifier,
modifier_input_identifier(node_group, "Depth Direction"),
depth_direction,
)
return finalize_object_result(
context,
source_object,
mesh_object,
keep_source=True,
)
except Exception:
if mesh_object is not None:
bpy.data.objects.remove(mesh_object, do_unlink=True)
if mesh is not None and mesh.users == 0:
bpy.data.meshes.remove(mesh)
if material is not None and material.users == 0:
bpy.data.materials.remove(material)
if color_image.users == 0:
bpy.data.images.remove(color_image, do_unlink=True)
if normal_image is not None and normal_image.users == 0:
bpy.data.images.remove(normal_image, do_unlink=True)
if depth_image is not None and depth_image.users == 0:
bpy.data.images.remove(depth_image, do_unlink=True)
raise
def region_to_shape(
context,
source_object,
shape,
detail,
region_mask,
*,
field=None,
normal_image=None,
normal_mode=NormalMode.NONE,
source_rgba=None,
):
crop_width = region_mask.bounds[2] - region_mask.bounds[0]
crop_height = region_mask.bounds[3] - region_mask.bounds[1]
detail = limited_cutout_mesh_detail(
detail,
(crop_width, crop_height),
configured_max_ai_input_size(context),
)
base_shape = build_base_shape(
context,
source_object,
detail,
region_mask,
)
result = create_shape_object(
context,
source_object,
shape,
detail,
region_mask,
base_shape,
field=field,
normal_image=normal_image,
normal_mode=normal_mode,
source_rgba=source_rgba,
)
undo_message = f"Convert to {shape.replace('_', ' ').title()}"
if "FINISHED" not in bpy.ops.ed.undo_push(message=undo_message):
raise RuntimeError(f"Unable to create the {undo_message} Undo step")
return result
def _region_mask_from_image(path, bounds):
image = bpy.data.images.load(str(path), check_existing=False)
try:
return RegionMask(
image_rgba(image)[:, :, 3],
tuple(int(value) for value in bounds),
)
finally:
bpy.data.images.remove(image, do_unlink=True)
def _generated_region_mask(result, input_path, source_bounds):
if not isinstance(result.value, dict) or "region" not in result.value:
return _region_mask_from_image(input_path, source_bounds)
local_bounds = tuple(
int(value)
for value in json.loads(
result.file("region_bounds").read_text(encoding="utf-8")
)
)
bounds = (
source_bounds[0] + local_bounds[0],
source_bounds[1] + local_bounds[1],
source_bounds[0] + local_bounds[2],
source_bounds[1] + local_bounds[3],
)
return _region_mask_from_image(result.file("region"), bounds)
def cutout_moge_settings(context):
preferences = addon_preferences(context)
model = getattr(preferences, "mesh_depth_model", DEFAULT_MOGE2_MODEL_KEY)
if not model.startswith("MOGE2_"):
model = DEFAULT_MOGE2_MODEL_KEY
return (
model,
int(getattr(preferences, "mesh_resolution_level", 5)),
)
class CutoutRegionToShape(JobOperatorBase, bpy.types.Operator):
bl_idname = "anyimage.cutout_region_to_shape"
bl_label = "Cutout Region to Shape"
bl_description = "Build the selected image Region as a Cutout shape"
bl_options = {"INTERNAL"}
job_type = "generate-region-data"
starting_message = "Generating Cutout Region data"
input_path: bpy.props.StringProperty(options=_HIDDEN)
source_object_name: bpy.props.StringProperty(options=_HIDDEN)
region_path_json: bpy.props.StringProperty(options=_HIDDEN)
region_bounds_json: bpy.props.StringProperty(options=_HIDDEN)
shape: bpy.props.StringProperty(options=_HIDDEN)
mesh_detail: bpy.props.IntProperty(
default=128,
min=4,
max=1024,
options=_HIDDEN,
)
model: bpy.props.StringProperty(
default=DEFAULT_MOGE2_MODEL_KEY,
options=_HIDDEN,
)
resolution_level: bpy.props.IntProperty(
default=5,
min=0,
max=9,
options=_HIDDEN,
)
max_input_size: bpy.props.IntProperty(
default=DEFAULT_MAX_AI_INPUT_SIZE,
min=MIN_MAX_AI_INPUT_SIZE,
max=MAX_MAX_AI_INPUT_SIZE,
options=_HIDDEN,
)
normal_mode: bpy.props.StringProperty(
default=NormalMode.NONE.value,
options=_HIDDEN,
)
refine_selection: bpy.props.BoolProperty(default=False, options=_HIDDEN)
def _needs_generation(self):
return (
self.refine_selection
or self.shape in DEPTH_CUTOUT_SHAPES
or NormalMode(self.normal_mode) != NormalMode.NONE
)
def execute(self, context):
try:
source_object = require_image_empty(self.source_object_name)
if video_total_frames(source_object.data) != 1:
raise ValueError("Cutout currently supports still images")
if self.shape not in CUTOUT_SHAPES:
raise ValueError(f"Unsupported Cutout shape: {self.shape}")
self.normal_mode = NormalMode(self.normal_mode).value
region_path = RegionPath.from_json(self.region_path_json)
source_rgba = image_rgba(source_object.data)
region_mask = rasterize_region_path(
tuple(source_object.data.size),
region_path,
source_alpha=source_rgba[:, :, 3],
alpha_threshold=CUTOUT_ALPHA_THRESHOLD,
antialias=True,
)
context.view_layer.objects.active = source_object
source_object.select_set(True)
self.max_input_size = configured_max_ai_input_size(context)
if not self._needs_generation():
region_to_shape(
context,
source_object,
self.shape,
self.mesh_detail,
region_mask,
source_rgba=source_rgba,
)
return {"FINISHED"}
setup_result = invoke_ai_setup_if_needed()
if setup_result is not None:
return setup_result
self.model, self.resolution_level = cutout_moge_settings(context)
self.region_bounds_json = json.dumps(
region_mask.bounds,
separators=(",", ":"),
)
self.input_path = str(
prepare_region_input(source_object.data, region_mask)
)
except (
ImageEditWarning,
OSError,
RuntimeError,
TypeError,
ValueError,
) as error:
report_image_edit_exception(self, error)
return {"CANCELLED"}
result = super().execute(context)
if "CANCELLED" in result:
cleanup_image_input(self.input_path, True)
return result
def request(self, context):
require_environment()
source_path = require_input_path(self.input_path)
output_depth = self.shape in DEPTH_CUTOUT_SHAPES
normal_mode = NormalMode(self.normal_mode)
if self.refine_selection:
require_model(BEN2_MODEL_KEY)
if output_depth or normal_mode != NormalMode.NONE:
require_model(self.model)
parameters = {
"input": str(source_path),
"device": production_device().lower(),
"refine_region": self.refine_selection,
"output_depth": output_depth,
"normal_mode": normal_mode.value,
"max_input_size": int(self.max_input_size),
}
if self.refine_selection:
parameters["refine_model"] = BEN2_MODEL_KEY
if output_depth or normal_mode != NormalMode.NONE:
start_frame = source_start_frame(self.source_object_name)
parameters.update(
model=self.model,
resolution_level=self.resolution_level,
)
parameters.update(
moge2_parameters(
context.scene.anyimage_settings,
source_path,
self.model,
self.resolution_level,
start_frame,
)
)
return parameters
def response(self, context, result):
source_object = require_image_empty(self.source_object_name)
source_bounds = tuple(json.loads(self.region_bounds_json))
region_mask = _generated_region_mask(
result,
self.input_path,
source_bounds,
)
value = result.value if isinstance(result.value, dict) else {}
normal_image = (
load_normal_result_image(result.file("normal"), source_object)
if "normal" in value
else None
)
field = (
load_image_geometry(result.file("image_geometry"))
if self.shape in DEPTH_CUTOUT_SHAPES
else None
)
region_to_shape(
context,
source_object,
self.shape,
self.mesh_detail,
region_mask,
field=field,
normal_image=normal_image,
normal_mode=NormalMode(self.normal_mode),
)
return f"{self.shape.replace('_', ' ').title()} is ready"
def cleanup(self):
cleanup_image_input(self.input_path, True)

136
src/anyimage/operators/cutout_tool/selection_tool.py → src/anyimage/operators/cutout_tool/select_region.py
View File

@@ -1,27 +1,42 @@
import json
import bpy import bpy
from ...properties import ( from ...properties import (
refine_selection_property, refine_selection_property,
) )
from ..common.image import is_image_empty, video_total_frames from ..common.image_data import image_rgba, require_image_empty, video_total_frames
from ..common.image_edit import ( from ..common.image_interaction import (
ImageEditWarning, apply_region_settings,
apply_image_mask_settings, draw_region_settings,
draw_image_mask_settings,
polygon_bounds,
report_image_edit_exception, report_image_edit_exception,
resolve_image_edit_click, resolve_image_edit_click,
) )
from ..common.image_interaction import hidden_string_property, serialize_matrix from ..common.image_interaction import serialize_matrix
from ..common.image_gesture import ImageMaskGesture, gesture_property from ..common.image_interaction import ImageRegionGesture, gesture_property
from ..common.image_mask import selected_image_rgba from ..common.image_region import (
ImageEditWarning,
RegionPath,
rasterize_region_path,
)
CUTOUT_DETAIL_MIN = 4 CUTOUT_DETAIL_MIN = 4
CUTOUT_DETAIL_MAX = 1024 CUTOUT_DETAIL_MAX = 1024
def _validate_cutout_region(
source_object,
region_path,
):
image = source_object.data
rgba = image_rgba(image)
rasterize_region_path(
tuple(image.size),
region_path,
source_alpha=rgba[:, :, 3],
alpha_threshold=0.1,
antialias=True,
)
def cutout_mesh_detail(image_size, point_spacing): def cutout_mesh_detail(image_size, point_spacing):
longest_edge = max(int(value) for value in image_size) longest_edge = max(int(value) for value in image_size)
detail = round(longest_edge / max(int(point_spacing), 1)) detail = round(longest_edge / max(int(point_spacing), 1))
@@ -34,23 +49,21 @@ def _pie_button(
label, label,
icon, icon,
source_name, source_name,
selection_data, region_path_json,
refine_selection, refine_selection,
normal_analysis, normal_mode,
invert,
mesh_detail, mesh_detail,
): ):
operator = pie.operator( operator = pie.operator(
"anyimage.convert_cutout_shape", "anyimage.cutout_region_to_shape",
text=label, text=label,
icon=icon, icon=icon,
) )
operator.source_object_name = source_name operator.source_object_name = source_name
operator.selection_data = selection_data operator.region_path_json = region_path_json
operator.shape = shape operator.shape = shape
operator.refine_selection = refine_selection operator.refine_selection = refine_selection
operator.normal_analysis = normal_analysis operator.normal_mode = normal_mode
operator.invert = invert
operator.mesh_detail = mesh_detail operator.mesh_detail = mesh_detail
@@ -58,29 +71,27 @@ def _open_shape_pie(
context, context,
event, event,
source_object, source_object,
polygon, region_path,
refine_selection=False, refine_selection=False,
invert=False,
): ):
source_name = source_object.name source_name = source_object.name
selection_data = json.dumps(polygon, separators=(",", ":")) region_path_json = region_path.to_json()
settings = context.scene.anyimage_settings settings = context.scene.anyimage_settings
point_spacing = settings.cutout_point_spacing point_spacing = settings.cutout_point_spacing
normal_analysis = bool(getattr(settings, "cutout_normal_analysis", False)) normal_mode = str(getattr(settings, "cutout_normal_mode", "NONE"))
mesh_detail = cutout_mesh_detail(source_object.data.size, point_spacing) mesh_detail = cutout_mesh_detail(source_object.data.size, point_spacing)
def draw(menu, _context): def draw(menu, _context):
pie = menu.layout.menu_pie() pie = menu.layout.menu_pie()
_pie_button( _pie_button(
pie, pie,
"CUTOUT", "SURFACE",
"Surface", "Surface",
"MESH_GRID", "MESH_GRID",
source_name, source_name,
selection_data, region_path_json,
refine_selection, refine_selection,
normal_analysis, normal_mode,
invert,
mesh_detail, mesh_detail,
) )
_pie_button( _pie_button(
@@ -89,10 +100,9 @@ def _open_shape_pie(
"Balloon", "Balloon",
"MESH_UVSPHERE", "MESH_UVSPHERE",
source_name, source_name,
selection_data, region_path_json,
refine_selection, refine_selection,
normal_analysis, normal_mode,
invert,
mesh_detail, mesh_detail,
) )
_pie_button( _pie_button(
@@ -101,10 +111,9 @@ def _open_shape_pie(
"Depth Balloon", "Depth Balloon",
"META_BALL", "META_BALL",
source_name, source_name,
selection_data, region_path_json,
refine_selection, refine_selection,
normal_analysis, normal_mode,
invert,
mesh_detail, mesh_detail,
) )
_pie_button( _pie_button(
@@ -113,10 +122,9 @@ def _open_shape_pie(
"Depth Surface", "Depth Surface",
"SURFACE_NCURVE", "SURFACE_NCURVE",
source_name, source_name,
selection_data, region_path_json,
refine_selection, refine_selection,
normal_analysis, normal_mode,
invert,
mesh_detail, mesh_detail,
) )
@@ -128,16 +136,16 @@ def _open_shape_pie(
) )
class SelectCutoutRegion(ImageMaskGesture, bpy.types.Operator): class SelectCutoutRegion(ImageRegionGesture, bpy.types.Operator):
bl_idname = "anyimage.select_cutout_region" bl_idname = "anyimage.select_cutout_region"
bl_label = "Select Cutout Region" bl_label = "Select Cutout Region"
bl_description = "Lasso an Image Empty region, then choose its mesh shape" bl_description = "Lasso an Image Empty region, then choose its mesh shape"
bl_options = {"INTERNAL"} bl_options = {"INTERNAL"}
active_tool_id = "anyimage.cutout_lasso" active_tool_id = "anyimage.cutout_lasso"
source_object_name: hidden_string_property() source_object_name: bpy.props.StringProperty(options={"HIDDEN", "SKIP_SAVE"})
source_matrix_data: hidden_string_property() source_matrix_data: bpy.props.StringProperty(options={"HIDDEN", "SKIP_SAVE"})
selection_data: hidden_string_property() region_path_json: bpy.props.StringProperty(options={"HIDDEN", "SKIP_SAVE"})
refine_selection: refine_selection_property() refine_selection: refine_selection_property()
invert: bpy.props.BoolProperty(default=False, options={"SKIP_SAVE"}) invert: bpy.props.BoolProperty(default=False, options={"SKIP_SAVE"})
gesture: gesture_property() gesture: gesture_property()
@@ -167,15 +175,15 @@ class SelectCutoutRegion(ImageMaskGesture, bpy.types.Operator):
if video_total_frames(source_object.data) > 1: if video_total_frames(source_object.data) > 1:
self.report({"WARNING"}, "Image tools only support a still image") self.report({"WARNING"}, "Image tools only support a still image")
return {"CANCELLED"} return {"CANCELLED"}
apply_image_mask_settings(self, context) apply_region_settings(self, context)
self.invert = False self.invert = False
self.source_object_name = source_object.name self.source_object_name = source_object.name
self.source_matrix_data = serialize_matrix(source_object.matrix_world) self.source_matrix_data = serialize_matrix(source_object.matrix_world)
width, height = (int(value) for value in source_object.data.size) width, height = (int(value) for value in source_object.data.size)
self.selection_data = json.dumps( self.region_path_json = RegionPath(
((0, 0), (width, 0), (width, height), (0, height)), points=((0, 0), (width, 0), (width, height), (0, height)),
separators=(",", ":"), fill_rule="NONZERO",
) ).to_json()
return self.execute(context) return self.execute(context)
def modal(self, context, event): def modal(self, context, event):
@@ -183,47 +191,27 @@ class SelectCutoutRegion(ImageMaskGesture, bpy.types.Operator):
return super().modal(context, event) return super().modal(context, event)
def execute(self, context): def execute(self, context):
source_object = bpy.data.objects.get(self.source_object_name) try:
if not is_image_empty(source_object): source_object = require_image_empty(self.source_object_name)
self.report({"ERROR"}, "The source Image Empty is no longer available") except RuntimeError as error:
self.report({"ERROR"}, str(error))
return {"CANCELLED"} return {"CANCELLED"}
try: try:
polygon = json.loads(self.selection_data) region_path = RegionPath.from_json(self.region_path_json)
polygon_bounds(polygon) _validate_cutout_region(source_object, region_path)
selected_image_rgba( except (ImageEditWarning, TypeError, ValueError) as error:
source_object, polygon, self.invert, "NONZERO"
)
except (ImageEditWarning, json.JSONDecodeError, TypeError, ValueError) as error:
report_image_edit_exception(self, error) report_image_edit_exception(self, error)
return {"CANCELLED"} return {"CANCELLED"}
_open_shape_pie( _open_shape_pie(
context, context,
self._pie_event, self._pie_event,
source_object, source_object,
polygon, region_path,
self.refine_selection, self.refine_selection,
self.invert,
) )
return {"FINISHED"} return {"FINISHED"}
class ActivateCutoutTool(bpy.types.Operator):
bl_idname = "anyimage.activate_cutout_tool"
bl_label = "Cutout Tool"
bl_description = "Activate the Cutout Lasso tool"
bl_options = {"INTERNAL"}
@classmethod
def poll(cls, context):
return context.area is not None and context.area.type == "VIEW_3D"
def execute(self, _context):
return bpy.ops.wm.tool_set_by_id(
name=CutoutTool.bl_idname,
space_type="VIEW_3D",
)
class CutoutTool(bpy.types.WorkSpaceTool): class CutoutTool(bpy.types.WorkSpaceTool):
bl_space_type = "VIEW_3D" bl_space_type = "VIEW_3D"
bl_context_mode = "OBJECT" bl_context_mode = "OBJECT"
@@ -248,5 +236,5 @@ class CutoutTool(bpy.types.WorkSpaceTool):
def draw_settings(context, layout, _tool): def draw_settings(context, layout, _tool):
settings = context.scene.anyimage_settings settings = context.scene.anyimage_settings
layout.prop(settings, "cutout_point_spacing") layout.prop(settings, "cutout_point_spacing")
layout.prop(settings, "cutout_normal_analysis") layout.prop(settings, "cutout_normal_mode")
draw_image_mask_settings(context, layout, "cutout_refine_selection") draw_region_settings(context, layout, "cutout_refine_selection")

214
src/anyimage/operators/cutout_tool/shape_conversion.py
View File

@@ -1,214 +0,0 @@
import json
import bpy
from ...preferences import addon_preferences, configured_max_ai_input_size
from ...properties import BEN2_MODEL_KEY, DEFAULT_MOGE2_MODEL_KEY
from ...runtime import JobOperatorBase
from ...server.media.resolution import (
DEFAULT_MAX_AI_INPUT_SIZE,
MAX_MAX_AI_INPUT_SIZE,
MIN_MAX_AI_INPUT_SIZE,
)
from ..common.image import (
is_image_empty,
prepare_image_input,
remove_temporary_input,
video_total_frames,
)
from ..common.image_edit import report_image_edit_exception
from ..common.operator import (
HIDDEN,
cleanup_temporary_input,
input_path,
invoke_ai_setup_if_needed,
moge2_parameters,
production_device,
require_environment,
require_model,
source_start_frame,
)
from .build_shapes import build_cutout_response, build_local_cutout
from .shape_types import DEPTH_CUTOUT_SHAPES, LOCAL_CUTOUT_SHAPES
def cutout_moge_settings(context):
preferences = addon_preferences(context)
model = getattr(preferences, "mesh_depth_model", DEFAULT_MOGE2_MODEL_KEY)
if not model.startswith("MOGE2_"):
model = DEFAULT_MOGE2_MODEL_KEY
return (
model,
int(getattr(preferences, "mesh_resolution_level", 5)),
)
class GenerateCutoutShape(JobOperatorBase, bpy.types.Operator):
bl_idname = "anyimage.generate_cutout_shape"
bl_label = "Generate Cutout Shape"
job_type = "prepare-geometry-data"
starting_message = "Preparing Cutout shape"
input_path: bpy.props.StringProperty(options=HIDDEN)
source_object_name: bpy.props.StringProperty(options=HIDDEN)
shape: bpy.props.StringProperty(options=HIDDEN)
selection_data: bpy.props.StringProperty(options=HIDDEN)
mesh_detail: bpy.props.IntProperty(default=128, min=4, max=1024, options=HIDDEN)
model: bpy.props.StringProperty(default=DEFAULT_MOGE2_MODEL_KEY, options=HIDDEN)
resolution_level: bpy.props.IntProperty(default=5, min=0, max=9, options=HIDDEN)
max_input_size: bpy.props.IntProperty(
default=DEFAULT_MAX_AI_INPUT_SIZE,
min=MIN_MAX_AI_INPUT_SIZE,
max=MAX_MAX_AI_INPUT_SIZE,
options=HIDDEN,
)
normal_analysis: bpy.props.BoolProperty(default=False, options=HIDDEN)
delete_input: bpy.props.BoolProperty(default=False, options=HIDDEN)
refine_selection: bpy.props.BoolProperty(default=False, options=HIDDEN)
invert: bpy.props.BoolProperty(default=False, options=HIDDEN)
def request(self, context):
require_environment()
if self.shape in DEPTH_CUTOUT_SHAPES or self.normal_analysis:
require_model(self.model)
if self.refine_selection:
require_model(BEN2_MODEL_KEY)
polygon = json.loads(self.selection_data) if self.selection_data else None
if polygon is None or len(polygon) < 3:
raise ValueError("The Cutout selection is invalid")
parameters = {
"input": str(input_path(self.input_path)),
"analysis_input": str(input_path(self.input_path)),
"shape": self.shape,
"polygon": polygon,
"mesh_detail": self.mesh_detail,
"device": production_device().lower(),
"video_frames": 1,
"start_frame": source_start_frame(self.source_object_name),
"max_input_size": int(self.max_input_size),
"refine_selection": self.refine_selection,
"normal_analysis": self.normal_analysis,
"invert": self.invert,
}
if self.shape in DEPTH_CUTOUT_SHAPES or self.normal_analysis:
parameters.update(
model=self.model,
resolution_level=self.resolution_level,
)
if self.refine_selection:
parameters["refine_model"] = BEN2_MODEL_KEY
if self.shape not in DEPTH_CUTOUT_SHAPES:
return parameters
settings = context.scene.anyimage_settings
parameters.update(
moge2_parameters(
settings,
input_path(self.input_path),
self.model,
self.resolution_level,
source_start_frame(self.source_object_name),
)
)
parameters.update(shape=self.shape, polygon=polygon)
return parameters
def response(self, context, result):
build_cutout_response(context, result, self)
return f"{self.shape.replace('_', ' ').title()} is ready"
def cleanup(self):
cleanup_temporary_input(self.input_path, self.delete_input)
def start_cutout_job(
operator,
context,
shape,
selection_data,
refine_selection=False,
normal_analysis=False,
invert=False,
):
source_object = context.object
if video_total_frames(source_object.data) != 1:
operator.report({"ERROR"}, "Cutout currently supports still images")
return {"CANCELLED"}
setup_result = invoke_ai_setup_if_needed()
if setup_result is not None:
return setup_result
try:
source_path, delete_input = prepare_image_input(
source_object.data,
context.scene,
)
model, resolution_level = cutout_moge_settings(context)
except (RuntimeError, ValueError) as error:
operator.report({"ERROR"}, str(error))
return {"CANCELLED"}
result = bpy.ops.anyimage.generate_cutout_shape(
input_path=str(source_path),
source_object_name=source_object.name,
shape=shape,
selection_data=selection_data,
mesh_detail=operator.mesh_detail,
model=model,
resolution_level=resolution_level,
max_input_size=configured_max_ai_input_size(context),
normal_analysis=normal_analysis,
delete_input=delete_input,
refine_selection=refine_selection,
invert=invert,
)
if "CANCELLED" in result and delete_input:
remove_temporary_input(source_path)
return {"CANCELLED"} if "CANCELLED" in result else {"FINISHED"}
class ConvertCutoutShape(bpy.types.Operator):
bl_idname = "anyimage.convert_cutout_shape"
bl_label = "Convert Cutout Shape"
bl_description = "Convert the saved Cutout selection into the chosen shape"
bl_options = {"UNDO"}
source_object_name: bpy.props.StringProperty(options=HIDDEN)
selection_data: bpy.props.StringProperty(options=HIDDEN)
shape: bpy.props.StringProperty(options=HIDDEN)
mesh_detail: bpy.props.IntProperty(default=128, min=4, max=1024, options=HIDDEN)
refine_selection: bpy.props.BoolProperty(default=False, options=HIDDEN)
normal_analysis: bpy.props.BoolProperty(default=False, options=HIDDEN)
invert: bpy.props.BoolProperty(default=False, options=HIDDEN)
def execute(self, context):
source_object = bpy.data.objects.get(self.source_object_name)
if not is_image_empty(source_object):
self.report({"ERROR"}, "The source Image Empty is no longer available")
return {"CANCELLED"}
context.view_layer.objects.active = source_object
source_object.select_set(True)
if (
self.shape in LOCAL_CUTOUT_SHAPES
and not self.refine_selection
and not self.normal_analysis
):
try:
build_local_cutout(
self,
context,
source_object,
self.shape,
self.selection_data,
self.invert,
)
except Exception as error:
report_image_edit_exception(self, error)
return {"CANCELLED"}
return {"FINISHED"}
return start_cutout_job(
self,
context,
self.shape,
self.selection_data,
self.refine_selection,
self.normal_analysis,
self.invert,
)

23
src/anyimage/operators/cutout_tool/shape_types.py
View File

@@ -1,23 +0,0 @@
"""Cutout shape identifiers and Geometry Nodes asset names."""
CUTOUT_NODE_GROUP_NAME = "O Image Cutout"
CUTOUT_NODE_GROUP_NAMES = {
"CUTOUT": "O Image Surface",
"BALLOON": "O Image Balloon",
"DEPTH_BALLOON": "O Image Depth Balloon",
"DEPTH_SURFACE": "O Image Depth Surface",
}
CUTOUT_SHAPE_LABELS = {
"CUTOUT": "Surface",
"BALLOON": "Balloon",
"DEPTH_BALLOON": "Depth Balloon",
"DEPTH_SURFACE": "Depth Surface",
}
CUTOUT_SHAPE_INDICES = {
shape: index for index, shape in enumerate(CUTOUT_NODE_GROUP_NAMES)
}
CUTOUT_SHAPES = frozenset(CUTOUT_NODE_GROUP_NAMES)
LOCAL_CUTOUT_SHAPES = frozenset({"CUTOUT", "BALLOON"})
DEPTH_CUTOUT_SHAPES = frozenset({"DEPTH_BALLOON", "DEPTH_SURFACE"})
BALLOON_ATTRIBUTE_NAME = "o_balloon"

34
src/anyimage/operators/debug.py
View File

@@ -1,18 +1,28 @@
from pathlib import Path
import bpy import bpy
from ..runtime import JobOperatorBase from ..runtime import JobOperatorBase
from ..properties import BEN2_MODEL_KEY, REALESRGAN_GENERAL_WDN_MODEL_KEY from ..properties import BEN2_MODEL_KEY, REALESRGAN_GENERAL_WDN_MODEL_KEY
from .common.operator import ( from .common.ai_job import (
background_input_path, require_background_input_path,
da3_parameters, da3_parameters,
input_path, require_input_path,
moge2_parameters, moge2_parameters,
open_job_directory,
require_environment, require_environment,
require_model, require_model,
) )
def _open_job_directory(directory):
directory = Path(directory)
if not directory.is_dir():
raise RuntimeError(f"Job directory does not exist: {directory}")
result = bpy.ops.wm.path_open(filepath=str(directory))
if "FINISHED" not in result:
raise RuntimeError(f"Unable to open job directory: {directory}")
class DebugDa3(JobOperatorBase, bpy.types.Operator): class DebugDa3(JobOperatorBase, bpy.types.Operator):
bl_idname = "anyimage.debug_da3" bl_idname = "anyimage.debug_da3"
bl_label = "Debug DA3" bl_label = "Debug DA3"
@@ -27,7 +37,7 @@ class DebugDa3(JobOperatorBase, bpy.types.Operator):
return { return {
**da3_parameters( **da3_parameters(
settings, settings,
input_path(settings.input_path), require_input_path(settings.input_path),
model, model,
settings.process_res, settings.process_res,
1, 1,
@@ -38,7 +48,7 @@ class DebugDa3(JobOperatorBase, bpy.types.Operator):
} }
def response(self, _context, result): def response(self, _context, result):
open_job_directory(result.directory) _open_job_directory(result.directory)
return f"Saved DA3 API prediction to {result.directory}" return f"Saved DA3 API prediction to {result.directory}"
@@ -55,7 +65,7 @@ class DebugMoge2(JobOperatorBase, bpy.types.Operator):
return { return {
**moge2_parameters( **moge2_parameters(
settings, settings,
input_path(settings.input_path), require_input_path(settings.input_path),
settings.moge_model, settings.moge_model,
settings.moge_resolution_level, settings.moge_resolution_level,
1, 1,
@@ -64,7 +74,7 @@ class DebugMoge2(JobOperatorBase, bpy.types.Operator):
} }
def response(self, _context, result): def response(self, _context, result):
open_job_directory(result.directory) _open_job_directory(result.directory)
return f"Exported MoGe-2 data to {result.directory}" return f"Exported MoGe-2 data to {result.directory}"
@@ -79,7 +89,7 @@ class DebugBen2(JobOperatorBase, bpy.types.Operator):
settings = context.scene.anyimage_settings settings = context.scene.anyimage_settings
require_model(BEN2_MODEL_KEY) require_model(BEN2_MODEL_KEY)
return { return {
"input": str(background_input_path(settings.background_input_path)), "input": str(require_background_input_path(settings.background_input_path)),
"model": BEN2_MODEL_KEY, "model": BEN2_MODEL_KEY,
"device": settings.device.lower(), "device": settings.device.lower(),
"video_frames": settings.video_frames, "video_frames": settings.video_frames,
@@ -87,7 +97,7 @@ class DebugBen2(JobOperatorBase, bpy.types.Operator):
} }
def response(self, _context, result): def response(self, _context, result):
open_job_directory(result.directory) _open_job_directory(result.directory)
return f"Removed background to {result.directory}" return f"Removed background to {result.directory}"
@@ -107,7 +117,7 @@ class DebugUpscale(JobOperatorBase, bpy.types.Operator):
settings = context.scene.anyimage_settings settings = context.scene.anyimage_settings
require_model(self.model) require_model(self.model)
return { return {
"input": str(input_path(settings.input_path)), "input": str(require_input_path(settings.input_path)),
"model": self.model, "model": self.model,
"device": settings.device.lower(), "device": settings.device.lower(),
"video_frames": settings.video_frames, "video_frames": settings.video_frames,
@@ -115,5 +125,5 @@ class DebugUpscale(JobOperatorBase, bpy.types.Operator):
} }
def response(self, _context, result): def response(self, _context, result):
open_job_directory(result.directory) _open_job_directory(result.directory)
return f"Saved upscale API prediction to {result.directory}" return f"Saved upscale API prediction to {result.directory}"

4
src/anyimage/operators/image_tool/box.py
View File

@@ -1,6 +1,6 @@
import bpy import bpy
from ..common.image_edit import draw_image_mask_settings from ..common.image_interaction import draw_region_settings
from .edit_mask import EditImageMask from .edit_mask import EditImageMask
@@ -21,4 +21,4 @@ class ImageBoxTool(bpy.types.WorkSpaceTool):
) )
def draw_settings(context, layout, _tool): def draw_settings(context, layout, _tool):
draw_image_mask_settings(context, layout, "image_box_refine_selection") draw_region_settings(context, layout, "image_box_refine_selection")

196
src/anyimage/operators/image_tool/edit_mask.py
View File

@@ -2,38 +2,119 @@ import json
import bpy import bpy
from ...properties import refine_selection_property from ...properties import BEN2_MODEL_KEY, refine_selection_property
from ...runtime import JobOperatorBase from ...runtime import JobOperatorBase
from ..common.image import ( from ..common.image_data import (
is_image_empty, cleanup_image_input,
prepare_image_input, create_pixel_result_image,
remove_temporary_input, image_rgba,
image_pixels,
prepare_region_input,
replace_empty_image,
require_image_empty,
video_total_frames, video_total_frames,
) )
from ..common.image_edit import ( from ..common.image_interaction import (
ImageEditWarning, image_edit_poll,
image_mask_refine_request,
polygon_bounds,
report_image_edit_exception, report_image_edit_exception,
) )
from ..common.image_gesture import ImageMaskGesture, gesture_property from ..common.image_interaction import ImageRegionGesture, gesture_property
from ..common.image_interaction import hidden_string_property from ..common.image_region import (
from ..common.image_mask import ( ImageEditWarning,
edit_image_object, RegionPath,
polygon_mask, crop_pixels_to_alpha,
refined_mask_from_pixels, rasterize_region_path,
)
from ..common.ai_job import (
production_device,
require_environment,
require_model,
) )
from ..common.operator import HIDDEN, cleanup_temporary_input, push_undo
class EditImageMask(ImageMaskGesture, bpy.types.Operator):
_HIDDEN = {"HIDDEN", "SKIP_SAVE"}
def _refined_mask_from_pixels(mask_pixels, mask_size, bounds, source_size):
import numpy as np
mask_width, mask_height = (int(value) for value in mask_size)
source_width, source_height = (int(value) for value in source_size)
local = np.flipud(
np.asarray(mask_pixels, dtype=np.float32).reshape(
(mask_height, mask_width, 4)
)
)[:, :, 3]
result = np.zeros((source_height, source_width), dtype=np.float32)
left, top, right, bottom = (int(value) for value in bounds)
source_left = max(left, 0)
source_top = max(top, 0)
source_right = min(right, source_width)
source_bottom = min(bottom, source_height)
if source_right <= source_left or source_bottom <= source_top:
return result
result[source_top:source_bottom, source_left:source_right] = local[
source_top - top : source_bottom - top,
source_left - left : source_right - left,
]
return result
def _edit_image_mask_pixels(
source_pixels,
source_size,
region_mask,
invert=False,
):
"""Apply a mask while preserving source RGB, then tightly crop the canvas."""
import numpy as np
width, height = (int(value) for value in source_size)
rgba = np.flipud(
np.asarray(source_pixels, dtype=np.float32).reshape((height, width, 4))
)
mask = np.asarray(region_mask, dtype=np.float32)
if invert:
mask = 1.0 - mask
mask = np.where(rgba[:, :, 3] > 0.0, mask, 0.0)
rgba[:, :, 3] *= mask
return crop_pixels_to_alpha(
np.flipud(rgba).ravel(),
(0, 0, width, height),
)
def _edit_image_object(source_object, region_mask, invert=False):
image = source_object.data
source_size = tuple(image.size)
pixels, crop_bounds = _edit_image_mask_pixels(
image_pixels(image),
source_size,
region_mask,
invert,
)
left, top, right, bottom = crop_bounds
result_image = create_pixel_result_image(
image,
pixels,
(right - left, bottom - top),
)
return replace_empty_image(
source_object,
result_image,
placement_bounds=crop_bounds,
)
class EditImageMask(ImageRegionGesture, bpy.types.Operator):
bl_idname = "anyimage.edit_image_mask" bl_idname = "anyimage.edit_image_mask"
bl_label = "Edit Image Mask" bl_label = "Edit Image Mask"
bl_description = "Edit and tightly crop an Image Empty with a gesture" bl_description = "Edit and tightly crop an Image Empty with a gesture"
bl_options = {"REGISTER", "UNDO"} bl_options = {"REGISTER", "UNDO"}
source_object_name: hidden_string_property() source_object_name: bpy.props.StringProperty(options={"HIDDEN", "SKIP_SAVE"})
source_matrix_data: hidden_string_property() source_matrix_data: bpy.props.StringProperty(options={"HIDDEN", "SKIP_SAVE"})
selection_data: hidden_string_property() region_path_json: bpy.props.StringProperty(options={"HIDDEN", "SKIP_SAVE"})
refine_selection: refine_selection_property() refine_selection: refine_selection_property()
invert: bpy.props.BoolProperty( invert: bpy.props.BoolProperty(
name="Invert", name="Invert",
@@ -45,35 +126,37 @@ class EditImageMask(ImageMaskGesture, bpy.types.Operator):
def execute(self, context): def execute(self, context):
source_object = bpy.data.objects.get(self.source_object_name) try:
if not is_image_empty(source_object): source_object = require_image_empty(self.source_object_name)
self.report({"ERROR"}, "The source Image Empty is no longer available") except RuntimeError as error:
self.report({"ERROR"}, str(error))
return {"CANCELLED"} return {"CANCELLED"}
try: try:
polygon = json.loads(self.selection_data) region_path = RegionPath.from_json(self.region_path_json)
polygon_bounds(polygon)
if video_total_frames(source_object.data) > 1: if video_total_frames(source_object.data) > 1:
raise RuntimeError("Image tools only support a still image") raise RuntimeError("Image tools only support a still image")
region_mask = rasterize_region_path(
tuple(source_object.data.size),
region_path,
source_alpha=image_rgba(source_object.data)[:, :, 3],
)
if self.refine_selection: if self.refine_selection:
source_path, delete_input = prepare_image_input( source_path = prepare_region_input(
source_object.data, context.scene source_object.data,
region_mask,
) )
result = bpy.ops.anyimage.refine_image_mask( result = bpy.ops.anyimage.refine_image_mask(
input_path=str(source_path), input_path=str(source_path),
delete_input=delete_input,
source_object_name=self.source_object_name, source_object_name=self.source_object_name,
selection_data=self.selection_data, region_bounds_json=json.dumps(region_mask.bounds),
invert=self.invert,
) )
if "CANCELLED" in result: if "CANCELLED" in result:
if delete_input: cleanup_image_input(source_path, True)
remove_temporary_input(source_path)
return {"CANCELLED"} return {"CANCELLED"}
return {"FINISHED"} return {"FINISHED"}
edit_image_object( _edit_image_object(
source_object, source_object,
polygon_mask(tuple(source_object.data.size), polygon), region_mask.full_values(tuple(source_object.data.size)),
self.invert,
) )
except (json.JSONDecodeError, RuntimeError, TypeError, ValueError) as error: except (json.JSONDecodeError, RuntimeError, TypeError, ValueError) as error:
report_image_edit_exception(self, error) report_image_edit_exception(self, error)
@@ -87,29 +170,39 @@ class RefineImageMask(JobOperatorBase, bpy.types.Operator):
job_type = "refine-image-mask" job_type = "refine-image-mask"
starting_message = "Refining image selection" starting_message = "Refining image selection"
input_path: bpy.props.StringProperty(options=HIDDEN) input_path: bpy.props.StringProperty(options=_HIDDEN)
delete_input: bpy.props.BoolProperty(default=False, options=HIDDEN) source_object_name: bpy.props.StringProperty(options={"HIDDEN", "SKIP_SAVE"})
source_object_name: hidden_string_property() region_bounds_json: bpy.props.StringProperty(options={"HIDDEN", "SKIP_SAVE"})
selection_data: hidden_string_property()
invert: bpy.props.BoolProperty(default=False, options=HIDDEN)
def request(self, _context): def request(self, _context):
return image_mask_refine_request(self.input_path, self.selection_data) require_environment()
require_model(BEN2_MODEL_KEY)
return {
"input": str(self.input_path),
"model": BEN2_MODEL_KEY,
"device": production_device().lower(),
}
def response(self, _context, result): def response(self, _context, result):
source_object = bpy.data.objects.get(self.source_object_name) source_object = require_image_empty(self.source_object_name)
if not is_image_empty(source_object):
raise RuntimeError("The source Image Empty is no longer available")
import numpy as np import numpy as np
bounds = tuple(json.loads(result.file("bounds").read_text(encoding="utf-8"))) source_bounds = tuple(json.loads(self.region_bounds_json))
local_bounds = tuple(
json.loads(result.file("bounds").read_text(encoding="utf-8"))
)
bounds = (
source_bounds[0] + local_bounds[0],
source_bounds[1] + local_bounds[1],
source_bounds[0] + local_bounds[2],
source_bounds[1] + local_bounds[3],
)
refined_image = bpy.data.images.load( refined_image = bpy.data.images.load(
str(result.file("image")), check_existing=False str(result.file("image")), check_existing=False
) )
try: try:
refined_pixels = np.empty(len(refined_image.pixels), dtype=np.float32) refined_pixels = image_pixels(refined_image)
refined_image.pixels.foreach_get(refined_pixels) mask = _refined_mask_from_pixels(
mask = refined_mask_from_pixels(
refined_pixels, refined_pixels,
tuple(refined_image.size), tuple(refined_image.size),
bounds, bounds,
@@ -118,15 +211,16 @@ class RefineImageMask(JobOperatorBase, bpy.types.Operator):
finally: finally:
bpy.data.images.remove(refined_image, do_unlink=True) bpy.data.images.remove(refined_image, do_unlink=True)
try: try:
edit_image_object(source_object, mask, self.invert) _edit_image_object(source_object, mask)
except ImageEditWarning as error: except ImageEditWarning as error:
self.report({"WARNING"}, str(error)) self.report({"WARNING"}, str(error))
return str(error) return str(error)
push_undo("Edit Image Mask") if "FINISHED" not in bpy.ops.ed.undo_push(message="Edit Image Mask"):
raise RuntimeError("Unable to create the Edit Image Mask Undo step")
return "Image selection refined" return "Image selection refined"
def cleanup(self): def cleanup(self):
cleanup_temporary_input(self.input_path, self.delete_input) cleanup_image_input(self.input_path, True)
class ActivateImageTool(bpy.types.Operator): class ActivateImageTool(bpy.types.Operator):

4
src/anyimage/operators/image_tool/lasso.py
View File

@@ -1,6 +1,6 @@
import bpy import bpy
from ..common.image_edit import draw_image_mask_settings from ..common.image_interaction import draw_region_settings
from .edit_mask import EditImageMask from .edit_mask import EditImageMask
@@ -21,4 +21,4 @@ class ImageLassoTool(bpy.types.WorkSpaceTool):
) )
def draw_settings(context, layout, _tool): def draw_settings(context, layout, _tool):
draw_image_mask_settings(context, layout, "image_lasso_refine_selection") draw_region_settings(context, layout, "image_lasso_refine_selection")

120
src/anyimage/operators/perspective_tool/edit_perspective.py
View File

@@ -6,24 +6,24 @@ import bpy
from ..common.image_interaction import ( from ..common.image_interaction import (
deserialize_matrix, deserialize_matrix,
drawing_in_region, drawing_in_region,
hidden_string_property,
screen_path_to_image_pixels, screen_path_to_image_pixels,
serialize_matrix, serialize_matrix,
) )
from ..common.image import ( from ..common.image_region import crop_pixels_to_alpha
from ..common.image_data import (
create_pixel_result_image,
image_base_name, image_base_name,
is_image_empty, image_pixels,
plane_bounds, image_empty_bounds,
replace_empty_image,
require_image_empty,
video_total_frames, video_total_frames,
) )
from ..common.image_edit import ( from ..common.image_interaction import (
active_view3d_tool_id, active_view3d_tool_id,
crop_pixels_to_alpha,
draw_dashed_line, draw_dashed_line,
draw_polygon_fill, draw_polygon_fill,
image_edit_poll, image_edit_poll,
polygon_bounds,
replace_empty_image_pixels,
report_image_edit_exception, report_image_edit_exception,
resolve_image_edit_click, resolve_image_edit_click,
) )
@@ -69,7 +69,7 @@ def validate_perspective_quad(points, min_area=1e-6):
) )
) )
if area < min_area: if area < min_area:
raise ValueError("Draw a larger perspective region") raise ValueError("Draw a larger perspective quad")
return quad return quad
@@ -313,7 +313,7 @@ def homography_from_points(source, target):
values.extend((u, v)) values.extend((u, v))
matrix = np.asarray(rows, dtype=np.float64) matrix = np.asarray(rows, dtype=np.float64)
if np.linalg.cond(matrix) > 1e12: if np.linalg.cond(matrix) > 1e12:
raise ValueError("The perspective region is too degenerate to transform") raise ValueError("The perspective quad is too degenerate to transform")
coefficients = np.linalg.solve(matrix, np.asarray(values, dtype=np.float64)) coefficients = np.linalg.solve(matrix, np.asarray(values, dtype=np.float64))
return np.append(coefficients, 1.0).reshape((3, 3)) return np.append(coefficients, 1.0).reshape((3, 3))
@@ -438,9 +438,13 @@ def perspective_quad_aspect(quad):
) )
def perspective_placement_bounds(polygon, output_size, crop_bounds): def perspective_placement_bounds(quad, output_size, crop_bounds):
"""Map a rectified Alpha crop back to an approximate source-image rectangle.""" """Map a rectified Alpha crop back to an approximate source-image rectangle."""
source_left, source_top, source_right, source_bottom = polygon_bounds(polygon) quad = validate_perspective_quad(quad)
source_left = math.floor(float(quad[:, 0].min()))
source_top = math.floor(float(quad[:, 1].min()))
source_right = math.ceil(float(quad[:, 0].max()))
source_bottom = math.ceil(float(quad[:, 1].max()))
output_width, output_height = (float(value) for value in output_size) output_width, output_height = (float(value) for value in output_size)
crop_left, crop_top, crop_right, crop_bottom = crop_bounds crop_left, crop_top, crop_right, crop_bottom = crop_bounds
source_width = source_right - source_left source_width = source_right - source_left
@@ -456,12 +460,12 @@ def perspective_placement_bounds(polygon, output_size, crop_bounds):
def extract_perspective_pixels( def extract_perspective_pixels(
source_pixels, source_pixels,
source_size, source_size,
polygon, quad,
aspect_ratio, aspect_ratio,
): ):
output_size = perspective_output_size(polygon, aspect_ratio) output_size = perspective_output_size(quad, aspect_ratio)
pixels = warp_perspective_pixels( pixels = warp_perspective_pixels(
source_pixels, source_size, polygon, output_size source_pixels, source_size, quad, output_size
) )
pixels, crop_bounds = crop_pixels_to_alpha(pixels, (0, 0, *output_size)) pixels, crop_bounds = crop_pixels_to_alpha(pixels, (0, 0, *output_size))
cropped_size = ( cropped_size = (
@@ -469,34 +473,20 @@ def extract_perspective_pixels(
crop_bounds[3] - crop_bounds[1], crop_bounds[3] - crop_bounds[1],
) )
placement_bounds = perspective_placement_bounds( placement_bounds = perspective_placement_bounds(
polygon, output_size, crop_bounds quad, output_size, crop_bounds
) )
return pixels, cropped_size, placement_bounds return pixels, cropped_size, placement_bounds
def create_perspective_result(
source_object,
pixels,
output_size,
placement_bounds,
):
return replace_empty_image_pixels(
source_object,
pixels,
output_size,
placement_bounds,
)
class EditImagePerspective(bpy.types.Operator): class EditImagePerspective(bpy.types.Operator):
bl_idname = "anyimage.edit_image_perspective" bl_idname = "anyimage.edit_image_perspective"
bl_label = "Edit Image Perspective" bl_label = "Edit Image Perspective"
bl_description = "Deform four image corners into a rectangular image" bl_description = "Deform four image corners into a rectangular image"
bl_options = {"REGISTER", "UNDO"} bl_options = {"REGISTER", "UNDO"}
source_object_name: hidden_string_property() source_object_name: bpy.props.StringProperty(options={"HIDDEN", "SKIP_SAVE"})
source_matrix_data: hidden_string_property() source_matrix_data: bpy.props.StringProperty(options={"HIDDEN", "SKIP_SAVE"})
selection_data: hidden_string_property() quad_json: bpy.props.StringProperty(options={"HIDDEN", "SKIP_SAVE"})
aspect_ratio: bpy.props.FloatProperty( aspect_ratio: bpy.props.FloatProperty(
default=1.0, default=1.0,
min=MIN_INTERACTIVE_ASPECT, min=MIN_INTERACTIVE_ASPECT,
@@ -632,7 +622,7 @@ class EditImagePerspective(bpy.types.Operator):
return {"CANCELLED"} return {"CANCELLED"}
self.source_object_name = source_object.name self.source_object_name = source_object.name
self.source_matrix_data = serialize_matrix(source_object.matrix_world) self.source_matrix_data = serialize_matrix(source_object.matrix_world)
self.selection_data = "" self.quad_json = ""
self._phase = "POINTS" self._phase = "POINTS"
self._preview_image = None self._preview_image = None
self._source_pixels = None self._source_pixels = None
@@ -704,36 +694,37 @@ class EditImagePerspective(bpy.types.Operator):
self._set_point_status(context) self._set_point_status(context)
context.area.tag_redraw() context.area.tag_redraw()
return {"RUNNING_MODAL"} return {"RUNNING_MODAL"}
source_object = bpy.data.objects.get(self.source_object_name) try:
if not is_image_empty(source_object): source_object = require_image_empty(self.source_object_name)
self.report({"ERROR"}, "The source Image Empty is no longer available") except RuntimeError as error:
self.report({"ERROR"}, str(error))
self._finish(context) self._finish(context)
return {"CANCELLED"} return {"CANCELLED"}
try: try:
canonical_perspective_quad(self._points, MIN_QUAD_AREA) canonical_perspective_quad(self._points, MIN_QUAD_AREA)
matrix_world = deserialize_matrix(self.source_matrix_data) matrix_world = deserialize_matrix(self.source_matrix_data)
polygon = screen_path_to_image_pixels( quad = screen_path_to_image_pixels(
self._points, self._points,
context.region, context.region,
context.region_data, context.region_data,
matrix_world, matrix_world,
plane_bounds(source_object), image_empty_bounds(source_object),
tuple(source_object.data.size), tuple(source_object.data.size),
) )
polygon = canonical_perspective_quad(polygon) quad = canonical_perspective_quad(quad)
if not perspective_quad_overlaps_image( if not perspective_quad_overlaps_image(
polygon, tuple(source_object.data.size) quad, tuple(source_object.data.size)
): ):
self.report( self.report(
{"WARNING"}, {"WARNING"},
"Perspective region does not overlap the source image", "Perspective quad does not overlap the source image",
) )
self._finish(context) self._finish(context)
return {"CANCELLED"} return {"CANCELLED"}
self.selection_data = json.dumps( self.quad_json = json.dumps(
polygon.tolist(), separators=(",", ":") quad.tolist(), separators=(",", ":")
) )
self.aspect_ratio = perspective_quad_aspect(polygon) self.aspect_ratio = perspective_quad_aspect(quad)
self._initial_aspect_ratio = self.aspect_ratio self._initial_aspect_ratio = self.aspect_ratio
self._raw_aspect_ratio = self.aspect_ratio self._raw_aspect_ratio = self.aspect_ratio
self._aspect_preset_label = None self._aspect_preset_label = None
@@ -751,14 +742,11 @@ class EditImagePerspective(bpy.types.Operator):
self._aspect_anchor_x = center_x self._aspect_anchor_x = center_x
import numpy as np import numpy as np
self._source_pixels = np.empty( self._source_pixels = image_pixels(source_object.data)
len(source_object.data.pixels), dtype=np.float32
)
source_object.data.pixels.foreach_get(self._source_pixels)
preview_pixels = warp_perspective_pixels( preview_pixels = warp_perspective_pixels(
self._source_pixels, self._source_pixels,
tuple(source_object.data.size), tuple(source_object.data.size),
polygon, quad,
(PREVIEW_TEXTURE_SIZE, PREVIEW_TEXTURE_SIZE), (PREVIEW_TEXTURE_SIZE, PREVIEW_TEXTURE_SIZE),
) )
if not np.any( if not np.any(
@@ -766,7 +754,7 @@ class EditImagePerspective(bpy.types.Operator):
): ):
self.report( self.report(
{"WARNING"}, {"WARNING"},
"Perspective region contains no visible image pixels", "Perspective quad contains no visible image pixels",
) )
self._finish(context) self._finish(context)
return {"CANCELLED"} return {"CANCELLED"}
@@ -785,20 +773,21 @@ class EditImagePerspective(bpy.types.Operator):
return {"PASS_THROUGH"} return {"PASS_THROUGH"}
def execute(self, context): def execute(self, context):
source_object = bpy.data.objects.get(self.source_object_name) try:
if not is_image_empty(source_object): source_object = require_image_empty(self.source_object_name)
self.report({"ERROR"}, "The source Image Empty is no longer available") except RuntimeError as error:
self.report({"ERROR"}, str(error))
return {"CANCELLED"} return {"CANCELLED"}
try: try:
if video_total_frames(source_object.data) > 1: if video_total_frames(source_object.data) > 1:
raise RuntimeError("Image Perspective only supports a still image") raise RuntimeError("Image Perspective only supports a still image")
polygon = validate_perspective_quad(json.loads(self.selection_data)) quad = validate_perspective_quad(json.loads(self.quad_json))
if not perspective_quad_overlaps_image( if not perspective_quad_overlaps_image(
polygon, tuple(source_object.data.size) quad, tuple(source_object.data.size)
): ):
self.report( self.report(
{"WARNING"}, {"WARNING"},
"Perspective region does not overlap the source image", "Perspective quad does not overlap the source image",
) )
return {"CANCELLED"} return {"CANCELLED"}
import numpy as np import numpy as np
@@ -807,21 +796,22 @@ class EditImagePerspective(bpy.types.Operator):
if source_pixels is None or len(source_pixels) != len( if source_pixels is None or len(source_pixels) != len(
source_object.data.pixels source_object.data.pixels
): ):
source_pixels = np.empty( source_pixels = image_pixels(source_object.data)
len(source_object.data.pixels), dtype=np.float32
)
source_object.data.pixels.foreach_get(source_pixels)
pixels, output_size, placement_bounds = extract_perspective_pixels( pixels, output_size, placement_bounds = extract_perspective_pixels(
source_pixels, source_pixels,
tuple(source_object.data.size), tuple(source_object.data.size),
polygon, quad,
self.aspect_ratio, self.aspect_ratio,
) )
create_perspective_result( result_image = create_pixel_result_image(
source_object, source_object.data,
pixels, pixels,
output_size, output_size,
placement_bounds, )
replace_empty_image(
source_object,
result_image,
placement_bounds=placement_bounds,
) )
except ( except (
json.JSONDecodeError, json.JSONDecodeError,

55
src/anyimage/operators/remove_background.py
View File

@@ -2,27 +2,27 @@ import bpy
from ..properties import BEN2_MODEL_KEY from ..properties import BEN2_MODEL_KEY
from ..runtime import JobOperatorBase from ..runtime import JobOperatorBase
from .common.operator import ( from .common.ai_job import (
HIDDEN, require_background_input_path,
background_input_path,
cleanup_temporary_input,
invoke_ai_setup_if_needed, invoke_ai_setup_if_needed,
production_device, production_device,
push_undo,
require_environment, require_environment,
require_model, require_model,
source_start_frame,
) )
from .common.image import ( from .common.image_data import (
cleanup_image_input,
default_extract_frames, default_extract_frames,
is_image_empty, is_image_empty,
prepare_image_input, prepare_image_input,
remove_temporary_input, source_start_frame,
video_total_frames, video_total_frames,
) )
_HIDDEN = {"HIDDEN", "SKIP_SAVE"}
class RemoveImageBackground(bpy.types.Operator): class RemoveImageBackground(bpy.types.Operator):
bl_idname = "anyimage.remove_image_background" bl_idname = "anyimage.remove_image_background"
bl_label = "Remove Background" bl_label = "Remove Background"
@@ -65,29 +65,22 @@ class RemoveImageBackground(bpy.types.Operator):
if "CANCELLED" not in result: if "CANCELLED" not in result:
return {"FINISHED"} return {"FINISHED"}
if delete_input: if delete_input:
remove_temporary_input(source_path) cleanup_image_input(source_path, True)
return {"CANCELLED"} return {"CANCELLED"}
def apply_background_response(result, source_object_name): def apply_background_response(result, source_object_name):
from .common.image import create_color_image, is_image_empty from .common.image_data import (
load_color_result_image,
replace_empty_image,
require_image_empty,
)
source_object = bpy.data.objects.get(source_object_name) source_object = require_image_empty(source_object_name)
if not is_image_empty(source_object): result_image = load_color_result_image(source_object, result.directory)
raise RuntimeError("The source image empty is no longer available") replace_empty_image(source_object, result_image)
source_image = source_object.data if "FINISHED" not in bpy.ops.ed.undo_push(message="Remove Background"):
source_name = source_image.name raise RuntimeError("Unable to create the Remove Background Undo step")
original_fake_user = source_image.use_fake_user
result_image = create_color_image(source_object, result.directory)
source_image.use_fake_user = original_fake_user
if not original_fake_user and int(source_image.users) == 0:
bpy.data.images.remove(source_image)
result_image.name = source_name
push_undo("Remove Background")
class RunBackgroundRemoval(JobOperatorBase, bpy.types.Operator): class RunBackgroundRemoval(JobOperatorBase, bpy.types.Operator):
@@ -96,16 +89,16 @@ class RunBackgroundRemoval(JobOperatorBase, bpy.types.Operator):
job_type = "remove-background" job_type = "remove-background"
starting_message = "Starting Image Empty background removal" starting_message = "Starting Image Empty background removal"
input_path: bpy.props.StringProperty(options=HIDDEN) input_path: bpy.props.StringProperty(options=_HIDDEN)
source_object_name: bpy.props.StringProperty(options=HIDDEN) source_object_name: bpy.props.StringProperty(options=_HIDDEN)
delete_input: bpy.props.BoolProperty(default=False, options=HIDDEN) delete_input: bpy.props.BoolProperty(default=False, options=_HIDDEN)
def request(self, context): def request(self, context):
require_environment() require_environment()
require_model(BEN2_MODEL_KEY) require_model(BEN2_MODEL_KEY)
settings = context.scene.anyimage_settings settings = context.scene.anyimage_settings
return { return {
"input": str(background_input_path(self.input_path)), "input": str(require_background_input_path(self.input_path)),
"model": BEN2_MODEL_KEY, "model": BEN2_MODEL_KEY,
"device": production_device().lower(), "device": production_device().lower(),
"video_frames": settings.video_frames, "video_frames": settings.video_frames,
@@ -117,4 +110,4 @@ class RunBackgroundRemoval(JobOperatorBase, bpy.types.Operator):
return "Image background is removed" return "Image background is removed"
def cleanup(self): def cleanup(self):
cleanup_temporary_input(self.input_path, self.delete_input) cleanup_image_input(self.input_path, self.delete_input)

102
src/anyimage/operators/upscale.py
View File

@@ -8,25 +8,43 @@ from ..server.media.resolution import (
MIN_MAX_AI_INPUT_SIZE, MIN_MAX_AI_INPUT_SIZE,
) )
from ..runtime import JobOperatorBase from ..runtime import JobOperatorBase
from .common.operator import ( from .common.ai_job import (
HIDDEN, require_input_path,
cleanup_temporary_input,
input_path,
invoke_ai_setup_if_needed, invoke_ai_setup_if_needed,
production_device, production_device,
require_environment, require_environment,
require_model, require_model,
source_start_frame,
) )
from .common.image import ( from .common.image_data import (
create_color_image, cleanup_image_input,
default_extract_frames, default_extract_frames,
image_base_name,
is_image_empty, is_image_empty,
load_color_result_image,
prepare_image_input, prepare_image_input,
source_start_frame,
replace_empty_image,
require_image_empty,
video_total_frames, video_total_frames,
) )
_HIDDEN = {"HIDDEN", "SKIP_SAVE"}
UPSCALE_FACTOR = 4
def upscale_output_size(image):
width, height = (int(value) for value in getattr(image, "size", (0, 0)))
if width <= 0 or height <= 0:
return None
return width * UPSCALE_FACTOR, height * UPSCALE_FACTOR
def upscale_input_size_allowed(image, max_input_size):
width, height = (int(value) for value in getattr(image, "size", (0, 0)))
if width <= 0 or height <= 0:
return True
return max(width, height) < int(max_input_size)
class UpscaleImage(JobOperatorBase, bpy.types.Operator): class UpscaleImage(JobOperatorBase, bpy.types.Operator):
bl_idname = "anyimage.upscale_image" bl_idname = "anyimage.upscale_image"
@@ -38,7 +56,7 @@ class UpscaleImage(JobOperatorBase, bpy.types.Operator):
model: bpy.props.StringProperty( model: bpy.props.StringProperty(
default="REALESRGAN_GENERAL_WDN_X4V3", default="REALESRGAN_GENERAL_WDN_X4V3",
options=HIDDEN, options=_HIDDEN,
) )
max_input_size: bpy.props.IntProperty( max_input_size: bpy.props.IntProperty(
name="Max AI Input Size", name="Max AI Input Size",
@@ -48,18 +66,26 @@ class UpscaleImage(JobOperatorBase, bpy.types.Operator):
max=MAX_MAX_AI_INPUT_SIZE, max=MAX_MAX_AI_INPUT_SIZE,
step=256, step=256,
subtype="PIXEL", subtype="PIXEL",
options=HIDDEN, options=_HIDDEN,
) )
input_path: bpy.props.StringProperty(options=HIDDEN) input_path: bpy.props.StringProperty(options=_HIDDEN)
source_object_name: bpy.props.StringProperty(options=HIDDEN) source_object_name: bpy.props.StringProperty(options=_HIDDEN)
source_image_name: bpy.props.StringProperty(options=HIDDEN) source_image_name: bpy.props.StringProperty(options=_HIDDEN)
delete_input: bpy.props.BoolProperty(default=False, options=HIDDEN) delete_input: bpy.props.BoolProperty(default=False, options=_HIDDEN)
@classmethod @classmethod
def poll(cls, context): def poll(cls, context):
from ..runtime import runtime from ..runtime import runtime
return is_image_empty(context.object) and not runtime.server_busy() source_object = getattr(context, "object", None)
return (
is_image_empty(source_object)
and upscale_input_size_allowed(
source_object.data,
configured_max_ai_input_size(context),
)
and not runtime.server_busy()
)
def invoke(self, context, _event): def invoke(self, context, _event):
setup_result = invoke_ai_setup_if_needed() setup_result = invoke_ai_setup_if_needed()
@@ -74,23 +100,15 @@ class UpscaleImage(JobOperatorBase, bpy.types.Operator):
if not is_image_empty(source_object): if not is_image_empty(source_object):
self.report({"ERROR"}, "Select an Image Empty to upscale") self.report({"ERROR"}, "Select an Image Empty to upscale")
return {"CANCELLED"} return {"CANCELLED"}
current_image = source_object.data source_image = source_object.data
try: if not upscale_input_size_allowed(source_image, self.max_input_size):
original_image = getattr(
current_image,
"anyimage_upscale_source",
None,
)
except ReferenceError:
original_image = None
source_image = original_image or current_image
self.source_image_name = source_image.name
self._rebuild_from_original = source_image is not current_image
if self._rebuild_from_original:
self.report( self.report(
{"WARNING"}, {"ERROR"},
"This image was already upscaled; rebuilding from the original", "Image long edge must be smaller than Maximum AI Input Size "
f"({self.max_input_size} px)",
) )
return {"CANCELLED"}
self.source_image_name = source_image.name
if video_total_frames(source_image) > 1: if video_total_frames(source_image) > 1:
context.scene.anyimage_settings.video_frames = ( context.scene.anyimage_settings.video_frames = (
default_extract_frames(source_object) default_extract_frames(source_object)
@@ -109,7 +127,7 @@ class UpscaleImage(JobOperatorBase, bpy.types.Operator):
self.delete_input = delete_input self.delete_input = delete_input
result = super().execute(context) result = super().execute(context)
if "CANCELLED" in result: if "CANCELLED" in result:
cleanup_temporary_input(self.input_path, self.delete_input) cleanup_image_input(self.input_path, self.delete_input)
return result return result
def request(self, context): def request(self, context):
@@ -117,7 +135,7 @@ class UpscaleImage(JobOperatorBase, bpy.types.Operator):
require_model(self.model) require_model(self.model)
settings = context.scene.anyimage_settings settings = context.scene.anyimage_settings
return { return {
"input": str(input_path(self.input_path)), "input": str(require_input_path(self.input_path)),
"model": self.model, "model": self.model,
"device": production_device().lower(), "device": production_device().lower(),
"video_frames": settings.video_frames, "video_frames": settings.video_frames,
@@ -126,28 +144,18 @@ class UpscaleImage(JobOperatorBase, bpy.types.Operator):
} }
def response(self, _context, result): def response(self, _context, result):
source_object = bpy.data.objects.get(self.source_object_name) source_object = require_image_empty(self.source_object_name)
if not is_image_empty(source_object):
raise RuntimeError("The source image empty is no longer available")
source_image = bpy.data.images.get(self.source_image_name) source_image = bpy.data.images.get(self.source_image_name)
if source_image is None: if source_image is None:
raise RuntimeError("The original source image is no longer available") raise RuntimeError("The source image is no longer available")
replaced_image = source_object.data result_image = load_color_result_image(
result_image = create_color_image(
source_object, source_object,
result.directory, result.directory,
frame_prefix="upscale.", frame_prefix="upscale.",
) )
if replaced_image is not source_image: replace_empty_image(source_object, result_image)
replaced_image.use_fake_user = False
if int(replaced_image.users) == 0:
bpy.data.images.remove(replaced_image, do_unlink=True)
result_image.name = f"{image_base_name(source_image)}_upscale.png"
result_image.anyimage_upscale_source = source_image
model_label = upscale_model_label(self.model) model_label = upscale_model_label(self.model)
if getattr(self, "_rebuild_from_original", False):
return f"Image is rebuilt from the original with {model_label}"
return f"Image is upscaled with {model_label}" return f"Image is upscaled with {model_label}"
def cleanup(self): def cleanup(self):
cleanup_temporary_input(self.input_path, self.delete_input) cleanup_image_input(self.input_path, self.delete_input)

4
src/anyimage/preferences.py
View File

@@ -96,8 +96,8 @@ class AnyImagePreferences(bpy.types.AddonPreferences):
max_ai_input_size: bpy.props.IntProperty( max_ai_input_size: bpy.props.IntProperty(
name="Maximum AI Input Size", name="Maximum AI Input Size",
description=( description=(
"Maximum long edge used by Upscale, Depth and Normal analysis; " "Maximum source long edge eligible for Upscale and maximum long edge "
"larger inputs are resized proportionally without changing color textures" "used by Depth and Normal analysis"
), ),
default=DEFAULT_MAX_AI_INPUT_SIZE, default=DEFAULT_MAX_AI_INPUT_SIZE,
min=MIN_MAX_AI_INPUT_SIZE, min=MIN_MAX_AI_INPUT_SIZE,

23
src/anyimage/properties.py
View File

@@ -232,11 +232,24 @@ def refine_selection_property():
) )
def cutout_normal_analysis_property(): def cutout_normal_mode_property():
return bpy.props.BoolProperty( return bpy.props.EnumProperty(
name="Normal", name="Normal",
description="Use MoGe-2 to add a normal texture for the selected shape", description="Normal texture generated with MoGe-2",
default=False, items=(
("NONE", "None", "Do not generate a normal texture"),
(
"TANGENT",
"Tangent",
"Generate a normal texture relative to the image plane",
),
(
"OBJECT",
"Object",
"Generate a normal texture in Cutout object-local axes",
),
),
default="NONE",
) )
@@ -255,7 +268,7 @@ def cutout_point_spacing_property():
class AnyImageSettings(bpy.types.PropertyGroup): class AnyImageSettings(bpy.types.PropertyGroup):
cutout_point_spacing: cutout_point_spacing_property() cutout_point_spacing: cutout_point_spacing_property()
cutout_normal_analysis: cutout_normal_analysis_property() cutout_normal_mode: cutout_normal_mode_property()
cutout_refine_selection: refine_selection_property() cutout_refine_selection: refine_selection_property()
image_box_refine_selection: refine_selection_property() image_box_refine_selection: refine_selection_property()
image_lasso_refine_selection: refine_selection_property() image_lasso_refine_selection: refine_selection_property()

14
src/anyimage/server/app.py
View File

@@ -10,7 +10,8 @@ from .jobs.debug_predictions import (
run_upscale as _debug_upscale, run_upscale as _debug_upscale,
) )
from .jobs.refine_image_mask import run as _refine_image_mask from .jobs.refine_image_mask import run as _refine_image_mask
from .jobs.prepare_geometry_data import run as _prepare_geometry_data from .jobs.generate_depth_plane import run as _generate_depth_plane
from .jobs.generate_region_data import run as _generate_region_data
from .jobs.upscale import run as _upscale_image from .jobs.upscale import run as _upscale_image
from .model_manager import ModelManager from .model_manager import ModelManager
@@ -50,9 +51,14 @@ def upscale_image(context, parameters):
return _upscale_image(context, parameters) return _upscale_image(context, parameters)
@server.job("prepare-geometry-data") @server.job("generate-depth-plane")
def prepare_geometry_data(context, parameters): def generate_depth_plane(context, parameters):
return _prepare_geometry_data(context, parameters) return _generate_depth_plane(context, parameters)
@server.job("generate-region-data")
def generate_region_data(context, parameters):
return _generate_region_data(context, parameters)
@server.job("debug-da3") @server.job("debug-da3")

181
src/anyimage/server/geometry/generate_depth.py Normal file
View File

@@ -0,0 +1,181 @@
"""Generate model-derived depth and normal data for one raster image."""
from ..media import DEFAULT_MAX_AI_INPUT_SIZE, limit_image
from ..models import geometry_adapter, moge2
from .normal_texture import write_camera_normal_texture
def _limited_model_input(source_path, output_path, max_input_size):
from PIL import Image
with Image.open(source_path) as opened:
limited = limit_image(opened, max_input_size)
if limited is opened:
return source_path
limited.save(output_path, format="PNG", compress_level=1)
return output_path
def _alpha_mask(image_path, shape):
import numpy as np
from PIL import Image
with Image.open(image_path) as opened:
alpha = opened.convert("RGBA").getchannel("A")
alpha = alpha.resize(
(int(shape[1]), int(shape[0])),
Image.Resampling.NEAREST,
)
return np.asarray(alpha, dtype=np.uint8) >= 26
def transparent_fraction(image_path, alpha_threshold=26):
import numpy as np
from PIL import Image
with Image.open(image_path) as opened:
alpha = np.asarray(
opened.convert("RGBA").getchannel("A"),
dtype=np.uint8,
)
return float(np.count_nonzero(alpha < alpha_threshold) / alpha.size)
def _write_image_geometry_archive(
frame,
output_path,
calibration_valid,
region_transparent_fraction,
):
import numpy as np
depth = np.asarray(frame.depth, dtype=np.float32)
points = np.asarray(frame.points, dtype=np.float32)
intrinsics = np.asarray(frame.intrinsics, dtype=np.float32)
valid = np.asarray(frame.valid_mask, dtype=bool)
if depth.ndim != 2 or points.shape != (*depth.shape, 3):
raise ValueError("The depth model returned an invalid point field")
valid &= np.isfinite(depth) & (depth > 0.0)
valid &= np.isfinite(points).all(axis=-1)
if not valid.any():
raise ValueError("The depth model returned no valid geometry")
calibration_valid = np.asarray(calibration_valid, dtype=bool) & valid
if not calibration_valid.any():
raise ValueError("The depth field contains no valid Region pixels")
reference_values = depth[calibration_valid]
if frame.model_mask is not None:
model_mask = np.asarray(frame.model_mask, dtype=bool)
masked = depth[calibration_valid & model_mask]
if len(masked):
reference_values = masked
np.savez_compressed(
output_path,
points=points,
depth=depth,
intrinsics=intrinsics,
valid=valid,
calibration_valid=calibration_valid,
image_size=np.asarray((depth.shape[1], depth.shape[0]), dtype=np.int32),
reference_depth=np.asarray(np.median(reference_values), dtype=np.float32),
region_transparent_fraction=np.asarray(
region_transparent_fraction,
dtype=np.float32,
),
)
def _write_normal(
prediction,
color_path,
output_path,
normal_mode,
):
import numpy as np
from PIL import Image
normal = np.asarray(prediction.normal[0], dtype=np.float32)
valid = _alpha_mask(color_path, normal.shape[:2])
with Image.open(color_path) as source:
output_size = source.size
return write_camera_normal_texture(
normal,
valid,
(0, 0),
output_size,
output_size,
output_path,
normal_mode=normal_mode,
)
def generate_depth(
context,
parameters,
input_path,
*,
output_depth,
normal_mode="NONE",
):
"""Generate requested AI data from an already prepared image Region."""
normal_mode = str(normal_mode).upper()
if normal_mode not in {"NONE", "TANGENT", "OBJECT"}:
raise ValueError(f"Unsupported normal mode: {normal_mode}")
model_path = _limited_model_input(
input_path,
context.directory / "model-input.png",
parameters.get("max_input_size", DEFAULT_MAX_AI_INPUT_SIZE),
)
try:
model_manager = context.resource("model_manager")
adapter = geometry_adapter(parameters["model"])
prediction = None
if output_depth:
context.progress(0.45, "Generating depth")
if adapter is moge2:
prediction = adapter.infer(
model_manager,
parameters,
[model_path],
include_points=True,
cancel_check=context.check_cancelled,
)
else:
prediction = adapter.infer(
model_manager,
parameters,
[model_path],
cancel_check=context.check_cancelled,
)
frame = adapter.geometry_frame(prediction)
calibration_valid = _alpha_mask(input_path, frame.valid_mask.shape)
archive_path = context.directory / "image_geometry.npz"
_write_image_geometry_archive(
frame,
archive_path,
calibration_valid,
transparent_fraction(input_path),
)
else:
archive_path = None
normal_path = None
if normal_mode != "NONE":
context.progress(0.72, "Generating surface normal")
if prediction is None or adapter is not moge2:
prediction = moge2.infer(
model_manager,
parameters,
[model_path],
include_points=False,
cancel_check=context.check_cancelled,
)
normal_path = _write_normal(
prediction,
input_path,
context.directory / "normal.png",
normal_mode,
)
return archive_path, normal_path
finally:
if model_path != input_path:
model_path.unlink(missing_ok=True)

33
src/anyimage/server/geometry/normal_texture.py
View File

@@ -3,14 +3,14 @@
from pathlib import Path from pathlib import Path
def resized_selection_mask(selection_mask, shape): def resized_valid_mask(valid_mask, shape):
import numpy as np import numpy as np
from PIL import Image from PIL import Image
selection_mask = np.asarray(selection_mask, dtype=bool) valid_mask = np.asarray(valid_mask, dtype=bool)
if selection_mask.shape == tuple(shape): if valid_mask.shape == tuple(shape):
return selection_mask return valid_mask
resized = Image.fromarray(selection_mask.astype(np.uint8) * 255, mode="L") resized = Image.fromarray(valid_mask.astype(np.uint8) * 255, mode="L")
resized = resized.resize( resized = resized.resize(
(int(shape[1]), int(shape[0])), (int(shape[1]), int(shape[0])),
Image.Resampling.NEAREST, Image.Resampling.NEAREST,
@@ -25,8 +25,9 @@ def write_camera_normal_texture(
crop_size, crop_size,
source_size, source_size,
output_path, output_path,
normal_mode="TANGENT",
): ):
"""Encode MoGe camera-axis normals into a full-image RGB texture.""" """Encode MoGe camera-axis normals in Blender tangent or object space."""
import numpy as np import numpy as np
from PIL import Image from PIL import Image
@@ -35,14 +36,26 @@ def write_camera_normal_texture(
if normal.ndim != 3 or normal.shape[2] != 3: if normal.ndim != 3 or normal.shape[2] != 3:
raise ValueError("Depth Balloon normal data must have three components") raise ValueError("Depth Balloon normal data must have three components")
if valid.shape != normal.shape[:2]: if valid.shape != normal.shape[:2]:
valid = resized_selection_mask(valid, normal.shape[:2]) valid = resized_valid_mask(valid, normal.shape[:2])
normal_mode = str(normal_mode).upper()
if normal_mode == "TANGENT":
converted = normal * np.asarray((1.0, -1.0, -1.0), dtype=np.float32)
empty_normal = (128, 128, 255)
elif normal_mode == "OBJECT":
converted = normal[..., (2, 0, 1)] * np.asarray(
(-1.0, -1.0, -1.0),
dtype=np.float32,
)
empty_normal = (255, 128, 128)
else:
raise ValueError(f"Unsupported normal mode: {normal_mode}")
encoded = np.clip( encoded = np.clip(
(normal * np.asarray((1.0, -1.0, -1.0), dtype=np.float32) + 1.0) (converted + 1.0)
* 127.5, * 127.5,
0.0, 0.0,
255.0, 255.0,
).astype(np.uint8) ).astype(np.uint8)
encoded[~valid] = (128, 128, 255) encoded[~valid] = empty_normal
crop_width, crop_height = (int(value) for value in crop_size) crop_width, crop_height = (int(value) for value in crop_size)
crop = Image.fromarray(encoded, mode="RGB") crop = Image.fromarray(encoded, mode="RGB")
if crop.size != (crop_width, crop_height): if crop.size != (crop_width, crop_height):
@@ -51,7 +64,7 @@ def write_camera_normal_texture(
Image.Resampling.BILINEAR, Image.Resampling.BILINEAR,
) )
source_width, source_height = (int(value) for value in source_size) source_width, source_height = (int(value) for value in source_size)
full = Image.new("RGB", (source_width, source_height), (128, 128, 255)) full = Image.new("RGB", (source_width, source_height), empty_normal)
full.paste(crop, tuple(int(value) for value in origin)) full.paste(crop, tuple(int(value) for value in origin))
output_path = Path(output_path) output_path = Path(output_path)
full.save(output_path) full.save(output_path)

22
src/anyimage/server/jobs/generate_depth_plane.py Normal file
View File

@@ -0,0 +1,22 @@
"""Generate depth data and preserve color for one complete image."""
from ..geometry.generate_depth import generate_depth
from ..media import copy_color_images, input_frames
def run(context, parameters):
context.progress(0.05, "Preparing Depth Plane")
context.directory.mkdir(parents=True, exist_ok=True)
with input_frames(context, parameters) as images:
if len(images) != 1:
raise ValueError("Depth Plane currently supports one image")
copy_color_images(images, context.directory)
archive_path, _normal_path = generate_depth(
context,
parameters,
images[0],
output_depth=True,
)
context.check_cancelled()
context.progress(1.0, "Depth Plane data is ready")
return {"image_geometry": archive_path.name}

51
src/anyimage/server/jobs/generate_region_data.py Normal file
View File

@@ -0,0 +1,51 @@
"""Generate optional AI data for one Blender-rasterized Cutout Region."""
import json
from pathlib import Path
from ..geometry.generate_depth import generate_depth
from ..region import generate_selection
def run(context, parameters):
context.progress(0.05, "Preparing Cutout Region")
context.directory.mkdir(parents=True, exist_ok=True)
input_path = Path(parameters["input"])
region_path = input_path
result = {}
if parameters.get("refine_region"):
context.progress(0.2, "Generating refined Region")
region_path, bounds = generate_selection(
context,
input_path,
parameters["refine_model"],
parameters["device"],
context.directory / "region.png",
)
bounds_path = context.directory / "region_bounds.json"
bounds_path.write_text(
json.dumps(bounds, separators=(",", ":")),
encoding="utf-8",
)
result.update(
region=region_path.name,
region_bounds=bounds_path.name,
)
output_depth = bool(parameters.get("output_depth"))
normal_mode = str(parameters.get("normal_mode", "NONE"))
if output_depth or normal_mode != "NONE":
archive_path, normal_path = generate_depth(
context,
parameters,
region_path,
output_depth=output_depth,
normal_mode=normal_mode,
)
if archive_path is not None:
result["image_geometry"] = archive_path.name
if normal_path is not None:
result["normal"] = normal_path.name
context.check_cancelled()
context.progress(1.0, "Cutout Region data is ready")
return result

255
src/anyimage/server/jobs/prepare_geometry_data.py
View File

@@ -1,255 +0,0 @@
from ..geometry.normal_texture import write_camera_normal_texture
from ..media import (
DEFAULT_MAX_AI_INPUT_SIZE,
analysis_frames,
copy_color_images,
input_frames,
limit_image,
)
from ..models import geometry_adapter, moge2
from ..selection.cutout_images import refine_selection, write_images, write_selection
def _limited_model_input(source_path, output_path, max_input_size):
from PIL import Image
with Image.open(source_path) as opened:
limited = limit_image(opened, max_input_size)
if limited is opened:
return source_path
limited.save(output_path, format="PNG", compress_level=1)
return output_path
def _write_field(frame, output_path, calibration_valid=None):
import numpy as np
depth = np.asarray(frame.depth, dtype=np.float32)
points = np.asarray(frame.points, dtype=np.float32)
intrinsics = np.asarray(frame.intrinsics, dtype=np.float32)
valid = np.asarray(frame.valid_mask, dtype=bool)
if depth.ndim != 2 or points.shape != (*depth.shape, 3):
raise ValueError("The depth model returned an invalid point field")
valid &= np.isfinite(depth) & (depth > 0.0)
valid &= np.isfinite(points).all(axis=-1)
if not valid.any():
raise ValueError("The depth model returned no valid geometry")
if calibration_valid is None:
calibration_valid = valid.copy()
else:
calibration_valid = np.asarray(calibration_valid, dtype=bool)
if calibration_valid.shape != valid.shape:
raise ValueError("The depth calibration mask dimensions do not match")
calibration_valid &= valid
if not calibration_valid.any():
raise ValueError("The depth field contains no valid calibration pixels")
reference_values = depth[calibration_valid]
if frame.model_mask is not None:
model_mask = np.asarray(frame.model_mask, dtype=bool)
masked = depth[calibration_valid & model_mask]
if len(masked):
reference_values = masked
np.savez_compressed(
output_path,
points=points,
depth=depth,
intrinsics=intrinsics,
valid=valid,
calibration_valid=calibration_valid,
image_size=np.asarray((depth.shape[1], depth.shape[0]), dtype=np.int32),
reference_depth=np.asarray(np.median(reference_values), dtype=np.float32),
)
def _selection_calibration_mask(frame, selection_path):
import numpy as np
from PIL import Image
valid = np.asarray(frame.valid_mask, dtype=bool)
with Image.open(selection_path) as opened:
selection = opened.convert("RGBA").getchannel("A")
selection = selection.resize(
(valid.shape[1], valid.shape[0]),
Image.Resampling.NEAREST,
)
selected = np.asarray(selection, dtype=np.uint8) >= 26
calibration_valid = valid & selected
if not calibration_valid.any():
raise ValueError("The depth field contains no valid selected pixels")
return calibration_valid
def _write_camera_normal_from_prediction(prediction, color_path, output_path):
import numpy as np
from PIL import Image
normal = np.asarray(prediction.normal[0], dtype=np.float32)
with Image.open(color_path) as source:
alpha = source.convert("RGBA").getchannel("A")
source_size = source.size
valid = (
np.asarray(
alpha.resize((normal.shape[1], normal.shape[0]), Image.Resampling.BILINEAR)
)
> 127
)
return write_camera_normal_texture(
normal,
valid,
(0, 0),
source_size,
source_size,
output_path,
)
def _write_camera_normal(context, parameters, model_path, color_path=None):
prediction = moge2.infer(
context.resource("model_manager"),
parameters,
[model_path],
include_points=False,
cancel_check=context.check_cancelled,
)
return _write_camera_normal_from_prediction(
prediction,
color_path or model_path,
context.directory / "normal.png",
)
def _write_depth_field(
context,
parameters,
model_path,
normal_color_path=None,
include_normal=False,
selection_path=None,
):
model_manager = context.resource("model_manager")
adapter = geometry_adapter(parameters["model"])
if adapter is moge2:
prediction = adapter.infer(
model_manager,
parameters,
[model_path],
include_points=True,
cancel_check=context.check_cancelled,
)
else:
prediction = adapter.infer(
model_manager,
parameters,
[model_path],
cancel_check=context.check_cancelled,
)
context.check_cancelled()
field_path = context.directory / "field.npz"
frame = adapter.geometry_frame(prediction)
calibration_valid = frame.valid_mask
if selection_path is not None:
calibration_valid = _selection_calibration_mask(frame, selection_path)
_write_field(frame, field_path, calibration_valid)
normal_path = None
if include_normal and adapter is moge2:
normal_path = _write_camera_normal_from_prediction(
prediction,
normal_color_path or model_path,
context.directory / "normal.png",
)
return field_path, normal_path
def run(context, parameters):
context.progress(0.02, "Preparing geometry input")
context.directory.mkdir(parents=True, exist_ok=True)
with input_frames(context, parameters) as images:
if len(images) != 1:
raise ValueError("Empty Image to Mesh currently supports one image")
shape = parameters["shape"]
polygon = parameters.get("polygon")
analysis_path = None
selection_path = None
result = {}
if polygon:
selection_bounds = None
full_selection_path = context.directory / "selection-full.png"
if parameters.get("refine_selection"):
refine_selection(
context,
parameters,
images[0],
polygon,
full_selection_path,
)
else:
_selection_path, selection_bounds = write_selection(
images[0],
polygon,
full_selection_path,
parameters.get("invert", False),
)
color_path, selection_path, analysis_path, bounds_path = (
write_images(
images[0],
full_selection_path,
context.directory,
selection_bounds,
)
)
result.update(
selection=selection_path.name,
bounds=bounds_path.name,
)
else:
copy_color_images(images, context.directory)
normal_analysis = bool(parameters.get("normal_analysis"))
with analysis_frames(context, parameters, images) as model_images:
if len(model_images) != 1:
raise ValueError("Empty Image to Mesh currently supports one image")
model_path = analysis_path or model_images[0]
limited_model_path = model_path
if normal_analysis or shape in {
"DEPTH_PLANE",
"DEPTH_BALLOON",
"DEPTH_SURFACE",
}:
limited_model_path = _limited_model_input(
model_path,
context.directory / "model-input.png",
parameters.get("max_input_size", DEFAULT_MAX_AI_INPUT_SIZE),
)
if shape in {"CUTOUT", "BALLOON"} and normal_analysis:
context.progress(0.4, "Estimating surface normal")
result["normal"] = _write_camera_normal(
context, parameters, limited_model_path, model_path
).name
elif shape in {"DEPTH_PLANE", "DEPTH_BALLOON", "DEPTH_SURFACE"}:
context.progress(0.4, "Estimating image depth")
field_path, normal_path = _write_depth_field(
context,
parameters,
limited_model_path,
normal_color_path=model_path,
include_normal=normal_analysis,
selection_path=selection_path,
)
result["field"] = field_path.name
if (
normal_analysis
and normal_path is None
and shape == "DEPTH_SURFACE"
):
normal_path = _write_camera_normal(
context, parameters, limited_model_path, model_path
)
if normal_path is not None:
result["normal"] = normal_path.name
if limited_model_path != model_path:
limited_model_path.unlink(missing_ok=True)
context.check_cancelled()
context.progress(1.0, "Geometry data is ready")
return result

31
src/anyimage/server/jobs/refine_image_mask.py
View File

@@ -1,12 +1,7 @@
import json import json
import time import time
from ..models.ben2 import infer_foreground from ..region import generate_selection
from ..selection.polygon_mask import (
apply_selection_mask,
create_selection_canvas,
crop_to_alpha,
)
def run(context, parameters): def run(context, parameters):
@@ -16,28 +11,14 @@ def run(context, parameters):
context.check_cancelled() context.check_cancelled()
output_directory = context.directory output_directory = context.directory
output_directory.mkdir(parents=True, exist_ok=True) output_directory.mkdir(parents=True, exist_ok=True)
canvas, selection_mask, bounds = create_selection_canvas(
parameters["input"], parameters["polygon"]
)
selection_path = output_directory / "selection-input.png"
canvas.save(selection_path, format="PNG", compress_level=1)
model_manager = context.resource("model_manager")
context.progress(0.2, f"Loading BEN2 for {parameters['device'].upper()}") context.progress(0.2, f"Loading BEN2 for {parameters['device'].upper()}")
foreground, _load_ms, _inference_ms = infer_foreground( output_path, bounds = generate_selection(
selection_path, context,
model_manager.directory(parameters["model"]), parameters["input"],
parameters["model"],
parameters["device"], parameters["device"],
model_cache=model_manager, output_directory / "image.png",
cancel_check=context.check_cancelled,
preserve_input_alpha=True,
) )
context.check_cancelled()
context.progress(0.9, "Applying selection mask")
output, bounds = crop_to_alpha(
apply_selection_mask(foreground, selection_mask), bounds
)
output_path = output_directory / "image.png"
output.save(output_path, format="PNG", compress_level=1)
bounds_path = output_directory / "bounds.json" bounds_path = output_directory / "bounds.json"
bounds_path.write_text(json.dumps(bounds, separators=(",", ":")), encoding="utf-8") bounds_path.write_text(json.dumps(bounds, separators=(",", ":")), encoding="utf-8")
elapsed_ms = (time.perf_counter() - started) * 1000.0 elapsed_ms = (time.perf_counter() - started) * 1000.0

5
src/anyimage/server/region/__init__.py Normal file
View File

@@ -0,0 +1,5 @@
"""AI generation for rasterized image Regions."""
from .generate_selection import generate_selection
__all__ = ("generate_selection",)

40
src/anyimage/server/region/generate_selection.py Normal file
View File

@@ -0,0 +1,40 @@
from ..models.ben2 import infer_foreground
def _crop_to_alpha(image):
import numpy as np
pixels = np.asarray(image.convert("RGBA"), dtype=np.uint8)
visible_y, visible_x = np.nonzero(pixels[:, :, 3] > 0)
if not len(visible_x):
raise ValueError("The generated Region contains no visible pixels")
bounds = (
int(visible_x.min()),
int(visible_y.min()),
int(visible_x.max()) + 1,
int(visible_y.max()) + 1,
)
return image.crop(bounds), bounds
def generate_selection(
context,
input_path,
model,
device,
output_path,
):
"""Generate a refined selection from an already rasterized Region."""
model_manager = context.resource("model_manager")
foreground, _load_ms, _inference_ms = infer_foreground(
input_path,
model_manager.directory(model),
device,
model_cache=model_manager,
cancel_check=context.check_cancelled,
preserve_input_alpha=True,
)
context.check_cancelled()
output, bounds = _crop_to_alpha(foreground)
output.save(output_path, format="PNG", compress_level=1)
return output_path, bounds

22
src/anyimage/server/selection/__init__.py
View File

@@ -1,22 +0,0 @@
"""Image selection rasterization and Cutout image preparation."""
from .cutout_images import refine_selection, write_images, write_selection
from .polygon_mask import (
apply_selection_mask,
create_selection_canvas,
crop_to_alpha,
polygon_bounds,
polygon_selection_mask,
)
__all__ = (
"apply_selection_mask",
"create_selection_canvas",
"crop_to_alpha",
"polygon_bounds",
"polygon_selection_mask",
"refine_selection",
"write_images",
"write_selection",
)

162
src/anyimage/server/selection/cutout_images.py
View File

@@ -1,162 +0,0 @@
"""Prepare Cutout selection images inside the Job Server."""
import json
from ..models.ben2 import infer_foreground
from .polygon_mask import (
apply_selection_mask,
create_selection_canvas,
polygon_selection_mask,
)
def _write_masked_color(input_path, polygon, output_path, invert=False):
import numpy as np
from PIL import Image, ImageFilter, ImageOps
with Image.open(input_path) as opened:
source = opened.convert("RGBA")
selection = Image.fromarray(
polygon_selection_mask(source.size, polygon, "NONZERO").astype(np.uint8) * 255,
mode="L",
).filter(ImageFilter.GaussianBlur(radius=0.75))
if invert:
selection = ImageOps.invert(selection)
pixels = np.asarray(source, dtype=np.uint8).copy()
source_alpha = pixels[:, :, 3].copy()
pixels[:, :, 3] = np.asarray(selection, dtype=np.uint8)
pixels[source_alpha < 26] = 0
pixels[pixels[:, :, 3] == 0, :3] = 0
Image.fromarray(pixels, mode="RGBA").save(
output_path,
format="PNG",
compress_level=1,
)
return output_path
def write_selection(input_path, polygon, output_path, invert=False):
import numpy as np
from PIL import Image, ImageFilter
if invert:
_write_masked_color(input_path, polygon, output_path, invert=True)
with Image.open(input_path) as source:
return output_path, (0, 0, source.width, source.height)
canvas, selection_mask, bounds = create_selection_canvas(
input_path, polygon, "NONZERO"
)
selection = Image.fromarray(
selection_mask.astype(np.uint8) * 255,
mode="L",
).filter(ImageFilter.GaussianBlur(radius=0.75))
pixels = np.asarray(canvas, dtype=np.uint8).copy()
source_alpha = pixels[:, :, 3].copy()
pixels[:, :, 3] = np.asarray(selection, dtype=np.uint8)
pixels[source_alpha < 26] = 0
pixels[pixels[:, :, 3] == 0, :3] = 0
Image.fromarray(pixels, mode="RGBA").save(
output_path,
format="PNG",
compress_level=1,
)
return output_path, bounds
def refine_selection(context, parameters, input_path, polygon, output_path):
from PIL import Image, ImageChops
canvas, selection_mask, bounds = create_selection_canvas(
input_path, polygon, "NONZERO"
)
selection_path = context.directory / "selection-input.png"
canvas.save(selection_path, format="PNG", compress_level=1)
model_manager = context.resource("model_manager")
foreground, _load_ms, _inference_ms = infer_foreground(
selection_path,
model_manager.directory(parameters["refine_model"]),
parameters["device"],
model_cache=model_manager,
cancel_check=context.check_cancelled,
preserve_input_alpha=True,
)
refined = apply_selection_mask(foreground, selection_mask)
with Image.open(input_path) as opened:
source = opened.convert("RGBA")
if parameters.get("invert"):
mask = Image.new("L", source.size, 0)
mask.paste(refined.getchannel("A"), bounds[:2])
source.putalpha(
ImageChops.multiply(source.getchannel("A"), ImageChops.invert(mask))
)
output = source
else:
output = Image.new("RGBA", source.size, (0, 0, 0, 0))
output.paste(refined, bounds[:2])
output.save(output_path, format="PNG", compress_level=1)
return output_path
def write_images(input_path, selection_path, output_directory, selection_bounds=None):
import numpy as np
from PIL import Image
with Image.open(input_path) as opened:
source = opened.convert("RGBA")
with Image.open(selection_path) as opened:
selection = opened.convert("RGBA")
if selection_bounds is None and selection.size != source.size:
raise ValueError("The Cutout selection does not match the source image")
if selection_bounds is None:
selection_bounds = (0, 0, source.width, source.height)
color_canvas = Image.new("RGBA", selection.size, (0, 0, 0, 0))
color_canvas.paste(source, (-selection_bounds[0], -selection_bounds[1]))
source_alpha = np.asarray(color_canvas, dtype=np.uint8)[:, :, 3]
selection_pixels = np.asarray(selection, dtype=np.uint8).copy()
alpha = selection_pixels[:, :, 3]
alpha[alpha <= 1] = 0
alpha[source_alpha < 26] = 0
selection_pixels[alpha == 0, :3] = 0
selection = Image.fromarray(selection_pixels, mode="RGBA")
visible_y, visible_x = np.nonzero(alpha >= 26)
if not len(visible_x):
raise ValueError("The Cutout selection contains no visible pixels")
content_bounds = (
int(visible_x.min()),
int(visible_y.min()),
int(visible_x.max()) + 1,
int(visible_y.max()) + 1,
)
selection_left, selection_top, _selection_right, _selection_bottom = (
selection_bounds
)
bounds = (
selection_left + content_bounds[0],
selection_top + content_bounds[1],
selection_left + content_bounds[2],
selection_top + content_bounds[3],
)
color = color_canvas.crop(content_bounds)
color_path = output_directory / "image.0001.png"
color.save(color_path, format="PNG", compress_level=1)
selection_crop = selection.crop(content_bounds)
selection_path = output_directory / "selection.png"
selection_crop.save(selection_path, format="PNG", compress_level=1)
analysis_pixels = np.asarray(color, dtype=np.uint8).copy()
mask_alpha = np.asarray(selection_crop, dtype=np.uint8)[:, :, 3]
analysis_pixels[:, :, 3] = np.minimum(analysis_pixels[:, :, 3], mask_alpha)
analysis_pixels[mask_alpha < 26] = 0
analysis_path = output_directory / "analysis.png"
Image.fromarray(analysis_pixels, mode="RGBA").save(
analysis_path,
format="PNG",
compress_level=1,
)
bounds_path = output_directory / "bounds.json"
bounds_path.write_text(
json.dumps(bounds, separators=(",", ":")),
encoding="utf-8",
)
return color_path, selection_path, analysis_path, bounds_path

151
src/anyimage/server/selection/polygon_mask.py
View File

@@ -1,151 +0,0 @@
import math
def polygon_bounds(polygon):
import numpy as np
points = np.asarray(polygon, dtype=np.float64)
if points.ndim != 2 or points.shape[0] < 3 or points.shape[1] != 2:
raise ValueError("The selection does not define a valid image region")
if not np.isfinite(points).all():
raise ValueError("The selection contains invalid image coordinates")
left = int(math.floor(float(points[:, 0].min())))
top = int(math.floor(float(points[:, 1].min())))
right = int(math.ceil(float(points[:, 0].max())))
bottom = int(math.ceil(float(points[:, 1].max())))
if right <= left or bottom <= top:
raise ValueError("The selection does not define a valid image region")
return left, top, right, bottom
def polygon_selection_mask(image_size, polygon, fill_rule="EVEN_ODD"):
"""Rasterize a polygon with an explicit fill rule."""
import numpy as np
width, height = (int(value) for value in image_size)
points = np.asarray(polygon, dtype=np.float64)
if width <= 0 or height <= 0:
raise ValueError("The selection canvas has no readable dimensions")
if points.ndim != 2 or points.shape[0] < 3 or points.shape[1] != 2:
raise ValueError("The selection does not define a valid image region")
if not np.isfinite(points).all():
raise ValueError("The selection contains invalid image coordinates")
if fill_rule not in {"EVEN_ODD", "NONZERO"}:
raise ValueError(f"Unknown polygon fill rule: {fill_rule}")
scanlines = np.arange(height, dtype=np.float64) + 0.5
crossings = [[] for _index in range(height)]
for start, end in zip(points, np.roll(points, -1, axis=0)):
selected = np.flatnonzero(
(start[1] > scanlines) != (end[1] > scanlines)
)
if not len(selected):
continue
crossing_x = start[0] + (
(scanlines[selected] - start[1])
* (end[0] - start[0])
/ (end[1] - start[1])
)
winding = 1 if end[1] > start[1] else -1
for row, value in zip(selected, crossing_x):
crossings[int(row)].append((float(value), winding))
mask = np.zeros((height, width), dtype=bool)
for row, values in enumerate(crossings):
values.sort()
state = 0
index = 0
while index < len(values):
crossing_x = values[index][0]
winding = 0
count = 0
while index < len(values) and values[index][0] == crossing_x:
winding += values[index][1]
count += 1
index += 1
state = (
state + winding
if fill_rule == "NONZERO"
else state ^ (count % 2)
)
if index >= len(values) or state == 0:
continue
following_x = values[index][0]
first = max(0, int(math.ceil(crossing_x - 0.5)))
last = min(width, int(math.floor(following_x - 0.5)) + 1)
mask[row, first:last] = True
return mask
def create_selection_canvas(input_path, polygon, fill_rule="EVEN_ODD"):
"""Create an unbounded selection canvas with transparent black outside."""
import numpy as np
from PIL import Image
with Image.open(input_path) as opened_image:
source = opened_image.convert("RGBA")
left, top, right, bottom = polygon_bounds(polygon)
if right <= 0 or bottom <= 0 or left >= source.width or top >= source.height:
raise ValueError("The selection does not overlap the source image")
canvas = Image.new("RGBA", (right - left, bottom - top), (0, 0, 0, 0))
canvas.paste(source, (-left, -top))
relative = [(float(x) - left, float(y) - top) for x, y in polygon]
selection_mask = polygon_selection_mask(canvas.size, relative, fill_rule)
pixels = np.asarray(canvas, dtype=np.uint8).copy()
pixels[~selection_mask] = 0
return Image.fromarray(pixels, mode="RGBA"), selection_mask, (
left,
top,
right,
bottom,
)
def apply_selection_mask(foreground, selection_mask):
"""Clear pixels outside the source selection after background removal."""
import numpy as np
from PIL import Image
pixels = np.asarray(foreground.convert("RGBA"), dtype=np.uint8).copy()
if pixels.shape[:2] != selection_mask.shape:
pixels = np.asarray(
foreground.convert("RGBA").resize(
(selection_mask.shape[1], selection_mask.shape[0]),
Image.Resampling.BILINEAR,
),
dtype=np.uint8,
)
pixels[~selection_mask] = 0
pixels[pixels[:, :, 3] == 0, :3] = 0
return Image.fromarray(pixels, mode="RGBA")
def crop_to_alpha(image, bounds):
"""Crop an RGBA mask image to its nonzero Alpha bounds."""
import numpy as np
from PIL import Image
pixels = np.asarray(image.convert("RGBA"), dtype=np.uint8)
visible_y, visible_x = np.nonzero(pixels[:, :, 3] > 0)
if not len(visible_x):
raise ValueError("The selection contains no visible image pixels")
content_left = int(visible_x.min())
content_top = int(visible_y.min())
content_right = int(visible_x.max()) + 1
content_bottom = int(visible_y.max()) + 1
output = Image.new(
"RGBA",
(
content_right - content_left,
content_bottom - content_top,
),
(0, 0, 0, 0),
)
output.paste(image, (-content_left, -content_top))
left, top, _right, _bottom = bounds
return output, (
left + content_left,
top + content_top,
left + content_right,
top + content_bottom,
)

860
tests/test_addon.py
View File

File diff suppressed because it is too large Load Diff

51
tests/test_cutout.py
View File

@@ -8,8 +8,9 @@ import bpy
SOURCE_ROOT = Path(__file__).parents[1] / "src" SOURCE_ROOT = Path(__file__).parents[1] / "src"
sys.path.insert(0, str(SOURCE_ROOT)) sys.path.insert(0, str(SOURCE_ROOT))
from anyimage.operators.common.image_gesture import ImageMaskGesture from anyimage.operators.common.image_interaction import ImageRegionGesture
from anyimage.operators.cutout_tool import selection_tool as cutout from anyimage.operators.cutout_tool import select_region as cutout
from anyimage.operators.common.image_region import RegionPath
class Pie: class Pie:
@@ -31,10 +32,10 @@ class WindowManager:
draw(menu, None) draw(menu, None)
def cutout_context(window_manager, point_spacing=32, normal_analysis=False): def cutout_context(window_manager, point_spacing=32, normal_mode="NONE"):
settings = SimpleNamespace( settings = SimpleNamespace(
cutout_point_spacing=point_spacing, cutout_point_spacing=point_spacing,
cutout_normal_analysis=normal_analysis, cutout_normal_mode=normal_mode,
) )
return SimpleNamespace( return SimpleNamespace(
window_manager=window_manager, window_manager=window_manager,
@@ -51,10 +52,15 @@ def test_cutout_pie_uses_four_semantic_shapes_in_cardinal_order():
context = cutout_context(window_manager) context = cutout_context(window_manager)
source = cutout_source() source = cutout_source()
cutout._open_shape_pie(context, object(), source, ((0, 0), (10, 0), (0, 10))) cutout._open_shape_pie(
context,
object(),
source,
RegionPath(points=((0, 0), (10, 0), (0, 10))),
)
assert [call[2].shape for call in window_manager.pie.calls] == [ assert [call[2].shape for call in window_manager.pie.calls] == [
"CUTOUT", "SURFACE",
"BALLOON", "BALLOON",
"DEPTH_BALLOON", "DEPTH_BALLOON",
"DEPTH_SURFACE", "DEPTH_SURFACE",
@@ -66,18 +72,18 @@ def test_cutout_pie_uses_four_semantic_shapes_in_cardinal_order():
"Depth Surface", "Depth Surface",
] ]
assert all( assert all(
call[0] == "anyimage.convert_cutout_shape" for call in window_manager.pie.calls call[0] == "anyimage.cutout_region_to_shape"
for call in window_manager.pie.calls
) )
assert not any(call[2].refine_selection for call in window_manager.pie.calls) assert not any(call[2].refine_selection for call in window_manager.pie.calls)
assert not any(call[2].normal_analysis for call in window_manager.pie.calls) assert all(call[2].normal_mode == "NONE" for call in window_manager.pie.calls)
assert not any(call[2].invert for call in window_manager.pie.calls)
assert all(call[2].mesh_detail == 64 for call in window_manager.pie.calls) assert all(call[2].mesh_detail == 64 for call in window_manager.pie.calls)
def test_cutout_pie_passes_image_lasso_options_to_every_shape(): def test_cutout_pie_passes_image_lasso_options_to_every_shape():
window_manager = WindowManager() window_manager = WindowManager()
context = cutout_context( context = cutout_context(
window_manager, point_spacing=16, normal_analysis=True window_manager, point_spacing=16, normal_mode="OBJECT"
) )
source = cutout_source(size=(4096, 2048)) source = cutout_source(size=(4096, 2048))
@@ -85,14 +91,19 @@ def test_cutout_pie_passes_image_lasso_options_to_every_shape():
context, context,
object(), object(),
source, source,
((0, 0), (10, 0), (0, 10)), RegionPath(
points=((0, 0), (10, 0), (0, 10)),
invert=True,
),
refine_selection=True, refine_selection=True,
invert=True,
) )
assert all(call[2].refine_selection for call in window_manager.pie.calls) assert all(call[2].refine_selection for call in window_manager.pie.calls)
assert all(call[2].normal_analysis for call in window_manager.pie.calls) assert all(call[2].normal_mode == "OBJECT" for call in window_manager.pie.calls)
assert all(call[2].invert for call in window_manager.pie.calls) assert all(
RegionPath.from_json(call[2].region_path_json).invert
for call in window_manager.pie.calls
)
assert all(call[2].mesh_detail == 256 for call in window_manager.pie.calls) assert all(call[2].mesh_detail == 256 for call in window_manager.pie.calls)
@@ -116,8 +127,8 @@ def test_cutout_reuses_image_lasso_gesture_and_double_click_selects_all(monkeypa
cutout, "resolve_image_edit_click", lambda *_args: (source, True) cutout, "resolve_image_edit_click", lambda *_args: (source, True)
) )
monkeypatch.setattr(cutout, "video_total_frames", lambda _image: 1) monkeypatch.setattr(cutout, "video_total_frames", lambda _image: 1)
monkeypatch.setattr(cutout, "apply_image_mask_settings", lambda *_args: None) monkeypatch.setattr(cutout, "apply_region_settings", lambda *_args: None)
monkeypatch.setattr(cutout, "selected_image_rgba", lambda *_args: object()) monkeypatch.setattr(cutout, "_validate_cutout_region", lambda *_args: None)
operator = SimpleNamespace( operator = SimpleNamespace(
full_image=True, full_image=True,
@@ -129,10 +140,12 @@ def test_cutout_reuses_image_lasso_gesture_and_double_click_selects_all(monkeypa
operator, context, SimpleNamespace(value="DOUBLE_CLICK") operator, context, SimpleNamespace(value="DOUBLE_CLICK")
) )
assert issubclass(cutout.SelectCutoutRegion, ImageMaskGesture) assert issubclass(cutout.SelectCutoutRegion, ImageRegionGesture)
assert result == {"FINISHED"} assert result == {"FINISHED"}
selection = json.loads(window_manager.pie.calls[0][2].selection_data) region_path = RegionPath.from_json(
assert selection == [[0, 0], [320, 0], [320, 180], [0, 180]] window_manager.pie.calls[0][2].region_path_json
)
assert region_path.points == ((0, 0), (320, 0), (320, 180), (0, 180))
assert window_manager.pie.calls[0][2].mesh_detail == 10 assert window_manager.pie.calls[0][2].mesh_detail == 10
assert not hasattr(cutout, "pending_selection") assert not hasattr(cutout, "pending_selection")

55
tests/test_image.py
View File

@@ -27,21 +27,31 @@ def load_image_modules():
module_paths = ( module_paths = (
( (
"anyimage.operators.common.image", "anyimage.operators.common.image_data",
PROJECT_ROOT / "src" / "anyimage" / "operators" / "common" / "image.py",
),
(
"anyimage.operators.common.node_asset",
PROJECT_ROOT PROJECT_ROOT
/ "src" / "src"
/ "anyimage" / "anyimage"
/ "operators" / "operators"
/ "common" / "common"
/ "node_asset.py", / "image_data.py",
), ),
( (
"anyimage.operators.common.material", "anyimage.operators.common.mesh_result",
PROJECT_ROOT / "src" / "anyimage" / "operators" / "common" / "material.py", PROJECT_ROOT
/ "src"
/ "anyimage"
/ "operators"
/ "common"
/ "mesh_result.py",
),
(
"anyimage.operators.convert_to_plane.resolution",
PROJECT_ROOT
/ "src"
/ "anyimage"
/ "operators"
/ "convert_to_plane"
/ "resolution.py",
), ),
) )
packages = { packages = {
@@ -50,6 +60,7 @@ def load_image_modules():
"anyimage", "anyimage",
"anyimage.operators", "anyimage.operators",
"anyimage.operators.common", "anyimage.operators.common",
"anyimage.operators.convert_to_plane",
) )
} }
for package in packages.values(): for package in packages.values():
@@ -452,21 +463,7 @@ class ImageTest(unittest.TestCase):
"img_normal.png", "img_normal.png",
) )
def test_copy_depth_image_renames_still_source(self): def test_color_name_uses_source_image_base_name(self):
class FakeImage:
def __init__(self, name):
self.name = name
def copy(self):
return FakeImage("img.png.001")
source_object = SimpleNamespace(data=FakeImage("img.png"))
depth_image = self.image_depth.copy_depth_image(source_object)
self.assertEqual(depth_image.name, "img_depth.exr")
def test_color_and_material_names_use_source_image_base_name(self):
source_object = SimpleNamespace( source_object = SimpleNamespace(
data=SimpleNamespace(name="img.png"), data=SimpleNamespace(name="img.png"),
) )
@@ -475,19 +472,11 @@ class ImageTest(unittest.TestCase):
self.image_depth.color_data_name(source_object), self.image_depth.color_data_name(source_object),
"img_color.png", "img_color.png",
) )
self.assertEqual(
self.image_depth.material_data_name(source_object),
"img",
)
source_object.data.name = "img.png.001" source_object.data.name = "img.png.001"
self.assertEqual( self.assertEqual(
self.image_depth.color_data_name(source_object), self.image_depth.color_data_name(source_object),
"img.001_color.png", "img.001_color.png",
) )
self.assertEqual(
self.image_depth.material_data_name(source_object),
"img.001",
)
def test_surface_displacement_is_set_to_displacement_only(self): def test_surface_displacement_is_set_to_displacement_only(self):
class Material: class Material:
@@ -501,14 +490,14 @@ class ImageTest(unittest.TestCase):
"DISPLACEMENT", "DISPLACEMENT",
) )
def test_plane_bounds_keep_image_aspect_and_empty_offset(self): def test_image_empty_bounds_keep_image_aspect_and_empty_offset(self):
source = SimpleNamespace( source = SimpleNamespace(
data=SimpleNamespace(size=(800, 400)), data=SimpleNamespace(size=(800, 400)),
empty_display_size=2.0, empty_display_size=2.0,
empty_image_offset=(-0.5, -0.5), empty_image_offset=(-0.5, -0.5),
) )
bounds = self.image_depth.plane_bounds(source) bounds = self.image_depth.image_empty_bounds(source)
self.assertEqual(bounds, (-1.0, 1.0, -0.5, 0.5)) self.assertEqual(bounds, (-1.0, 1.0, -0.5, 0.5))

251
tests/test_mesh.py
View File

@@ -1,24 +1,31 @@
import builtins import builtins
import sys import sys
import tempfile
from pathlib import Path from pathlib import Path
from types import SimpleNamespace from types import SimpleNamespace
from unittest.mock import patch from unittest.mock import patch
import numpy as np import numpy as np
from PIL import Image
SOURCE_ROOT = Path(__file__).parents[1] / "src" SOURCE_ROOT = Path(__file__).parents[1] / "src"
sys.path.insert(0, str(SOURCE_ROOT)) sys.path.insert(0, str(SOURCE_ROOT))
from anyimage.operators.cutout_tool import balloon_profile as cutout_profile from anyimage.operators.cutout_tool import region_to_shape as mesh
from anyimage.operators.cutout_tool import build_shapes as mesh from anyimage.operators.cutout_tool import region_to_shape as cutout_nodes
from anyimage.operators.cutout_tool import depth_calibration as cutout_depth from anyimage.operators.cutout_tool import region_to_shape as shapes
from anyimage.operators.cutout_tool import node_assets as cutout_nodes from anyimage.operators.cutout_tool.geometry import balloon as cutout_profile
from anyimage.operators.cutout_tool import shape_types as shapes from anyimage.operators.cutout_tool.geometry import depth_calibration as cutout_depth
from anyimage.operators.cutout_tool.balloon_profile import poisson_balloon_profile from anyimage.operators.cutout_tool.geometry.balloon import poisson_balloon_profile
from anyimage.operators.common.normal_map import rasterize_tangent_normal from anyimage.operators.common.mesh_result import ImageGeometry, load_image_geometry
from anyimage.operators.cutout_tool.mask_regions import alpha_regions, image_alpha_regions from anyimage.operators.common.image_region import (
RegionMask,
RegionPath,
rasterize_region_path,
)
from anyimage.operators.convert_to_plane.normal_map import rasterize_tangent_normal
from anyimage.operators.cutout_tool.geometry.region_contours import (
RegionContours,
extract_region_contours,
)
from anyimage.server.models import moge2 as moge2_model from anyimage.server.models import moge2 as moge2_model
@@ -27,6 +34,29 @@ def test_cutout_mesh_detail_uses_the_cropped_analysis_scale():
assert mesh.limited_cutout_mesh_detail(256, (1024, 512), 2048) == 256 assert mesh.limited_cutout_mesh_detail(256, (1024, 512), 2048) == 256
assert mesh.limited_cutout_mesh_detail(8, (8192, 4096), 2048) == 4 assert mesh.limited_cutout_mesh_detail(8, (8192, 4096), 2048) == 4
def test_image_geometry_loader_uses_one_schema_for_plane_and_cutout(tmp_path):
path = tmp_path / "image_geometry.npz"
points = np.ones((2, 3, 3), dtype=np.float32)
valid = np.ones((2, 3), dtype=bool)
np.savez_compressed(
path,
points=points,
intrinsics=np.eye(3, dtype=np.float32),
valid=valid,
calibration_valid=valid,
image_size=np.asarray((3, 2), dtype=np.int32),
reference_depth=np.asarray(1.0, dtype=np.float32),
region_transparent_fraction=np.asarray(0.25, dtype=np.float32),
)
field = load_image_geometry(path)
assert field.points.shape == (2, 3, 3)
assert field.image_size == (3, 2)
assert field.reference_depth == 1.0
assert field.region_transparent_fraction == 0.25
def test_cutout_node_groups_are_loaded_from_the_asset(): def test_cutout_node_groups_are_loaded_from_the_asset():
loaded = object() loaded = object()
with patch.object( with patch.object(
@@ -171,16 +201,19 @@ def test_depth_surface_discontinuity_map_uses_local_three_by_three_range():
def test_depth_surface_projection_uses_one_scale_for_all_camera_axes(): def test_depth_surface_projection_uses_one_scale_for_all_camera_axes():
points = np.zeros((51, 101, 3), dtype=np.float32) points = np.zeros((51, 101, 3), dtype=np.float32)
points[..., 2] = np.linspace(1.0, 3.0, 51 * 101).reshape(51, 101) points[..., 2] = np.linspace(1.0, 3.0, 51 * 101).reshape(51, 101)
field = ( field = ImageGeometry(
points, points=points,
np.asarray(((100.0, 0.0, 50.0), (0.0, 100.0, 25.0), (0.0, 0.0, 1.0))), intrinsics=np.asarray(
np.ones((51, 101), dtype=bool), ((100.0, 0.0, 50.0), (0.0, 100.0, 25.0), (0.0, 0.0, 1.0))
np.ones((51, 101), dtype=bool), ),
(101, 51), valid=np.ones((51, 101), dtype=bool),
2.0, calibration_valid=np.ones((51, 101), dtype=bool),
image_size=(101, 51),
reference_depth=2.0,
region_transparent_fraction=0.0,
) )
uniform_scale = mesh._surface_uniform_scale( uniform_scale = mesh._depth_uniform_scale(
SimpleNamespace(), SimpleNamespace(),
field, field,
bounds=(-2.0, 2.0, -1.0, 1.0), bounds=(-2.0, 2.0, -1.0, 1.0),
@@ -189,7 +222,7 @@ def test_depth_surface_projection_uses_one_scale_for_all_camera_axes():
assert uniform_scale == 2.0 assert uniform_scale == 2.0
reference_depth = cutout_depth.metric_depth_reference( reference_depth = cutout_depth.metric_depth_reference(
points[..., 2], points[..., 2],
field[3], field.calibration_valid,
uniform_scale, uniform_scale,
) )
assert np.isclose(reference_depth, 5.8, atol=1e-6) assert np.isclose(reference_depth, 5.8, atol=1e-6)
@@ -212,20 +245,24 @@ def test_moge_surface_field_uses_the_model_validity_mask():
assert np.array_equal(frame.valid_mask, prediction.mask[0]) assert np.array_equal(frame.valid_mask, prediction.mask[0])
def test_cutout_regions_group_holes_with_their_connected_component(): def test_region_contours_group_holes_with_their_connected_component():
with tempfile.TemporaryDirectory() as directory: alpha = np.zeros((120, 200), dtype=np.uint8)
path = Path(directory) / "cutout.png" alpha[20:100, 50:160] = 255
pixels = np.zeros((120, 200, 4), dtype=np.uint8) alpha[45:75, 80:120] = 0
pixels[20:100, 50:160] = (20, 40, 60, 255) alpha[5:15, 5:15] = 255
pixels[45:75, 80:120] = 0
pixels[5:15, 5:15] = (20, 40, 60, 255)
Image.fromarray(pixels, mode="RGBA").save(path)
regions, size = image_alpha_regions(path, max_resolution=200) region_contours = extract_region_contours(
RegionMask(alpha, (0, 0, 200, 120)),
max_resolution=200,
)
assert size == (200, 120) assert region_contours.image_size == (200, 120)
assert sorted(len(region) for region in regions) == [1, 2] assert sorted(len(component) for component in region_contours.components) == [1, 2]
contours = [contour for region in regions for contour in region] contours = [
contour
for component in region_contours.components
for contour in component
]
bounds = [ bounds = [
(*np.min(contour, axis=0), *np.max(contour, axis=0)) (*np.min(contour, axis=0), *np.max(contour, axis=0))
for contour in contours for contour in contours
@@ -235,16 +272,19 @@ def test_cutout_regions_group_holes_with_their_connected_component():
assert (5.0, 5.0, 15.0, 15.0) in bounds assert (5.0, 5.0, 15.0, 15.0) in bounds
def test_alpha_regions_accepts_blender_float_alpha_with_a_hole(): def test_region_contours_accept_blender_float_mask_with_a_hole():
alpha = np.zeros((120, 200), dtype=np.float32) alpha = np.zeros((120, 200), dtype=np.float32)
alpha[20:100, 50:160] = 1.0 alpha[20:100, 50:160] = 1.0
alpha[45:75, 80:120] = 0.0 alpha[45:75, 80:120] = 0.0
regions, size = alpha_regions(alpha, max_resolution=200) region_contours = extract_region_contours(
RegionMask(alpha, (0, 0, 200, 120)),
max_resolution=200,
)
assert size == (200, 120) assert region_contours.image_size == (200, 120)
assert len(regions) == 1 assert len(region_contours.components) == 1
assert len(regions[0]) == 2 assert len(region_contours.components[0]) == 2
def test_cutout_crop_bounds_map_to_the_source_plane(): def test_cutout_crop_bounds_map_to_the_source_plane():
@@ -268,10 +308,14 @@ def test_depth_surface_cutout_uses_half_alpha_boundary():
alpha[2:10, 2:14] = 0.49 alpha[2:10, 2:14] = 0.49
alpha[4:8, 5:11] = 0.51 alpha[4:8, 5:11] = 0.51
regions, size = mesh._cutout_regions(alpha, 64, threshold=0.5) region_contours = mesh._extract_cutout_contours(
contour = np.asarray(regions[0][0]) RegionMask(alpha, (0, 0, 16, 12)),
64,
threshold=0.5,
)
contour = np.asarray(region_contours.components[0][0])
assert size == (16, 12) assert region_contours.image_size == (16, 12)
np.testing.assert_allclose(contour.min(axis=0), (5.0, 4.0)) np.testing.assert_allclose(contour.min(axis=0), (5.0, 4.0))
np.testing.assert_allclose(contour.max(axis=0), (11.0, 8.0)) np.testing.assert_allclose(contour.max(axis=0), (11.0, 8.0))
@@ -283,91 +327,31 @@ def test_blender_cutout_alpha_pipeline_does_not_import_pillow():
raise ModuleNotFoundError("Pillow is unavailable in Blender") raise ModuleNotFoundError("Pillow is unavailable in Blender")
return original_import(name, *args, **kwargs) return original_import(name, *args, **kwargs)
selection = np.zeros((32, 48), dtype=bool)
selection[4:28, 6:42] = True
with patch("builtins.__import__", side_effect=without_pillow): with patch("builtins.__import__", side_effect=without_pillow):
antialiased = mesh._antialias_selection(selection) region_mask = rasterize_region_path(
regions, size = alpha_regions(antialiased, max_resolution=24) (48, 32),
RegionPath(points=((6, 4), (42, 4), (42, 28), (6, 28))),
antialias=True,
)
antialiased = region_mask.full_values((48, 32))
region_contours = extract_region_contours(
RegionMask(antialiased, (0, 0, 48, 32)),
max_resolution=24,
)
assert size == (48, 32) assert region_contours.image_size == (48, 32)
assert regions assert region_contours.components
assert np.any((antialiased > 0.0) & (antialiased < 1.0)) assert np.any((antialiased > 0.0) & (antialiased < 1.0))
def test_unrefined_cutout_geometry_uses_strict_tenth_source_alpha_threshold():
alpha = np.full((40, 60), 0.1, dtype=np.float32)
alpha[10:30, 20:40] = 0.11
polygon = ((0, 0), (60, 0), (60, 40), (0, 40))
regions, size = mesh._geometry_cutout_regions(alpha, polygon, 64)
contour = np.asarray(regions[0][0])
assert size == (60, 40)
np.testing.assert_allclose(contour.min(axis=0), (20.0, 10.0))
np.testing.assert_allclose(contour.max(axis=0), (40.0, 30.0))
def test_cutout_geometry_excludes_zero_source_alpha_without_refine():
alpha = np.zeros((80, 120), dtype=np.float32)
alpha[10:70, 10:110] = 1.0
alpha[30:50, 45:75] = 0.0
polygon = ((10, 10), (110, 10), (110, 70), (10, 70))
regions, size = mesh._geometry_cutout_regions(alpha, polygon, 64)
assert size == (120, 80)
assert len(regions) == 1
assert len(regions[0]) == 2
def test_cutout_geometry_uses_final_alpha_with_refine():
alpha = np.zeros((80, 120), dtype=np.float32)
alpha[10:70, 10:110] = 1.0
alpha[30:50, 45:75] = 0.0
polygon = ((10, 10), (110, 10), (110, 70), (10, 70))
regions, size = mesh._geometry_cutout_regions(
alpha, polygon, 64, refine_selection=True
)
assert size == (120, 80)
assert len(regions) == 1
assert len(regions[0]) == 2
def test_unrefined_cutout_geometry_discards_out_of_frame_selection():
alpha = np.zeros((60, 100), dtype=np.float32)
alpha[10:50] = 1.0
polygon = ((-20, 10), (120, 10), (120, 50), (-20, 50))
regions, size = mesh._geometry_cutout_regions(alpha, polygon, 64)
points = np.concatenate(
[np.asarray(contour) for region in regions for contour in region]
)
assert size == (100, 60)
assert points[:, 0].min() >= 0.0
assert points[:, 0].max() <= 100.0
def test_refined_cutout_geometry_discards_out_of_frame_selection():
alpha = np.zeros((60, 100), dtype=np.float32)
alpha[10:50] = 1.0
polygon = ((-20, 10), (120, 10), (120, 50), (-20, 50))
regions, size = mesh._geometry_cutout_regions(
alpha, polygon, 64, refine_selection=True
)
points = np.concatenate(
[np.asarray(contour) for region in regions for contour in region]
)
assert size == (100, 60)
assert points[:, 0].min() >= 0.0
assert points[:, 0].max() <= 100.0
def test_blender_cutout_contour_resolution_tracks_mesh_detail(): def test_blender_cutout_contour_resolution_tracks_mesh_detail():
expected = ([(((0, 0), (1, 0), (0, 1)),)], (1, 1)) expected = RegionContours(
alpha = np.ones((2, 2), dtype=np.float32) components=(((((0, 0), (1, 0), (0, 1))),),),
with patch.object(mesh, "alpha_regions", return_value=expected) as trace: image_size=(1, 1),
assert mesh._cutout_regions(alpha, 32) == expected )
assert mesh._cutout_regions(alpha, 256) == expected region_mask = RegionMask(np.ones((2, 2), dtype=np.float32), (0, 0, 2, 2))
with patch.object(mesh, "extract_region_contours", return_value=expected) as trace:
assert mesh._extract_cutout_contours(region_mask, 32) == expected
assert mesh._extract_cutout_contours(region_mask, 256) == expected
assert trace.call_args_list[0].kwargs["max_resolution"] == 512 assert trace.call_args_list[0].kwargs["max_resolution"] == 512
assert trace.call_args_list[1].kwargs["max_resolution"] == 1024 assert trace.call_args_list[1].kwargs["max_resolution"] == 1024
@@ -375,7 +359,7 @@ def test_blender_cutout_contour_resolution_tracks_mesh_detail():
def test_cutout_point_spacing_uses_the_complete_source_plane(): def test_cutout_point_spacing_uses_the_complete_source_plane():
source = SimpleNamespace(matrix_world=object()) source = SimpleNamespace(matrix_world=object())
with ( with (
patch.object(mesh, "plane_bounds", return_value=(-2.0, 2.0, -1.0, 1.0)), patch.object(mesh, "image_empty_bounds", return_value=(-2.0, 2.0, -1.0, 1.0)),
patch.object( patch.object(
mesh, mesh,
"_world_plane_metric", "_world_plane_metric",
@@ -387,23 +371,28 @@ def test_cutout_point_spacing_uses_the_complete_source_plane():
assert spacing == 0.02 assert spacing == 0.02
def test_cutout_cleanup_area_uses_point_spacing_and_plane_area_scale(): def test_cutout_cleanup_value_scales_the_ui_without_changing_node_area():
assert np.isclose( assert np.isclose(
mesh._cutout_cleanup_area( mesh._cutout_cleanup_value(
0.25, 0.25,
((1.0, 0.0), (0.0, 1.0)), ((1.0, 0.0), (0.0, 1.0)),
), ),
0.0625, 6.25,
) )
assert np.isclose( assert np.isclose(
mesh._cutout_cleanup_area( mesh._cutout_cleanup_value(
0.25, 0.25,
((4.0, 0.0), (0.0, 9.0)), ((4.0, 0.0), (0.0, 9.0)),
), ),
0.0625 / 6.0, 6.25 / 6.0,
) )
def test_cutout_depth_calibration_uses_five_percent_transparency_threshold():
assert not cutout_depth.use_adaptive_depth_calibration(0.0499)
assert cutout_depth.use_adaptive_depth_calibration(0.05)
def test_poisson_balloon_profile_is_zero_on_boundary_and_positive_inside(): def test_poisson_balloon_profile_is_zero_on_boundary_and_positive_inside():
points = np.asarray( points = np.asarray(
((0, 0), (2, 0), (2, 2), (0, 2), (1, 1)), ((0, 0), (2, 0), (2, 2), (0, 2), (1, 1)),
@@ -482,8 +471,7 @@ def test_cutout_domain_adapts_boundary_spacing_for_a_small_valid_region():
) )
domain = mesh._cutout_domain( domain = mesh._cutout_domain(
regions, RegionContours(regions, (4096, 4096)),
(4096, 4096),
(-1.0, 1.0, -1.0, 1.0), (-1.0, 1.0, -1.0, 1.0),
0.5, 0.5,
((1.0, 0.0), (0.0, 1.0)), ((1.0, 0.0), (0.0, 1.0)),
@@ -497,8 +485,7 @@ def test_cutout_domain_builds_a_full_image_at_minimum_detail():
regions = ((((0, 0), (4096, 0), (4096, 4096), (0, 4096)),),) regions = ((((0, 0), (4096, 0), (4096, 4096), (0, 4096)),),)
domain = mesh._cutout_domain( domain = mesh._cutout_domain(
regions, RegionContours(regions, (4096, 4096)),
(4096, 4096),
(-1.0, 1.0, -1.0, 1.0), (-1.0, 1.0, -1.0, 1.0),
0.5, 0.5,
((1.0, 0.0), (0.0, 1.0)), ((1.0, 0.0), (0.0, 1.0)),
@@ -511,8 +498,7 @@ def test_cutout_domain_centers_vertices_on_crop_bounds():
regions = ((((0, 0), (100, 0), (100, 50), (0, 50)),),) regions = ((((0, 0), (100, 0), (100, 50), (0, 50)),),)
vertices, _faces, uv, _boundary, _planar, _profile = mesh._cutout_domain( vertices, _faces, uv, _boundary, _planar, _profile = mesh._cutout_domain(
regions, RegionContours(regions, (100, 50)),
(100, 50),
(2.0, 6.0, -3.0, 1.0), (2.0, 6.0, -3.0, 1.0),
0.5, 0.5,
((1.0, 0.0), (0.0, 1.0)), ((1.0, 0.0), (0.0, 1.0)),
@@ -535,8 +521,7 @@ def test_cutout_domain_keeps_overlapping_regions_disconnected():
), ),
) )
domain = mesh._cutout_domain( domain = mesh._cutout_domain(
regions, RegionContours(regions, (120, 80)),
(120, 80),
(0.0, 1.0, 0.0, 1.0), (0.0, 1.0, 0.0, 1.0),
0.025, 0.025,
((1.0, 0.0), (0.0, 1.0)), ((1.0, 0.0), (0.0, 1.0)),

318
tests/test_mesh_blender.py
View File

@@ -10,13 +10,22 @@ from mathutils import Matrix, Vector
SOURCE_ROOT = Path(__file__).parents[1] / "src" SOURCE_ROOT = Path(__file__).parents[1] / "src"
sys.path.insert(0, str(SOURCE_ROOT)) sys.path.insert(0, str(SOURCE_ROOT))
from anyimage.operators.common import material, node_graph from anyimage.operators.common import image_data as common_image
from anyimage.operators.cutout_tool import build_shapes as mesh from anyimage.operators.common import mesh_result as material
from anyimage.operators.cutout_tool import node_assets as cutout_nodes from anyimage.operators.common.mesh_result import ImageGeometry
from anyimage.operators.cutout_tool import shape_types as shapes from anyimage.operators.common.image_region import (
RegionMask,
RegionPath,
rasterize_region_path,
)
from anyimage.operators.cutout_tool import region_to_shape as mesh
from anyimage.operators.cutout_tool import region_to_shape as cutout_nodes
from anyimage.operators.cutout_tool import region_to_shape as shapes
from anyimage.operators.convert_to_plane import nodes as plane_nodes from anyimage.operators.convert_to_plane import nodes as plane_nodes
from anyimage.operators.convert_to_plane import build as plane_build from anyimage.operators.convert_to_plane import build as plane_build
from anyimage.operators.common.normal_map import bake_tangent_normal_image from anyimage.operators.convert_to_plane.normal_map import (
bake_tangent_normal_image,
)
def _inputs(group): def _inputs(group):
@@ -35,6 +44,31 @@ def _group_node(group, node_tree):
) )
def _create_cutout_shape(source, shape, detail, *, bounds=None, field=None):
if bounds is None:
bounds = (0, 0, *tuple(int(value) for value in source.data.size))
left, top, right, bottom = bounds
region_mask = RegionMask(
np.ones((bottom - top, right - left), dtype=np.float32),
bounds,
)
base_shape = mesh.build_base_shape(
bpy.context,
source,
detail,
region_mask,
)
return mesh.create_shape_object(
bpy.context,
source,
shape,
detail,
region_mask,
base_shape,
field=field,
)
def test_depth_surface_image_packs_metric_points_and_edge_validity(): def test_depth_surface_image_packs_metric_points_and_edge_validity():
bpy.ops.wm.read_factory_settings(use_empty=True) bpy.ops.wm.read_factory_settings(use_empty=True)
source_image = bpy.data.images.new("Source.png", width=2, height=2) source_image = bpy.data.images.new("Source.png", width=2, height=2)
@@ -44,7 +78,7 @@ def test_depth_surface_image_packs_metric_points_and_edge_validity():
depth = np.asarray(((1.0, 1.0), (1.0, 2.0)), dtype=np.float32) depth = np.asarray(((1.0, 1.0), (1.0, 2.0)), dtype=np.float32)
valid = np.asarray(((True, False), (True, True))) valid = np.asarray(((True, False), (True, True)))
image = mesh._surface_depth_image(source, points, valid, depth) image = mesh._cutout_depth_image(source, points, valid, depth)
packed = np.empty(16, dtype=np.float32) packed = np.empty(16, dtype=np.float32)
image.pixels.foreach_get(packed) image.pixels.foreach_get(packed)
@@ -92,7 +126,7 @@ def test_cutout_modifier_wraps_four_menu_controlled_shape_groups():
plane = plane_nodes.plane_node_group("PLANE") plane = plane_nodes.plane_node_group("PLANE")
depth_plane = plane_nodes.plane_node_group("DEPTH_PLANE") depth_plane = plane_nodes.plane_node_group("DEPTH_PLANE")
cutout = cutout_nodes.cutout_node_group("CUTOUT") cutout = cutout_nodes.cutout_node_group("SURFACE")
assert plane.name == "O Mesh Plane" assert plane.name == "O Mesh Plane"
assert cutout.name == shapes.CUTOUT_NODE_GROUP_NAME assert cutout.name == shapes.CUTOUT_NODE_GROUP_NAME
assert all( assert all(
@@ -117,7 +151,7 @@ def test_cutout_modifier_wraps_four_menu_controlled_shape_groups():
"Smooth Iterations", "Smooth Iterations",
"Cleanup", "Cleanup",
"Smooth", "Smooth",
"Reference Depth", "Base Plane Depth",
"Depth Direction", "Depth Direction",
"Uniform Scale", "Uniform Scale",
"Depth Image", "Depth Image",
@@ -129,7 +163,7 @@ def test_cutout_modifier_wraps_four_menu_controlled_shape_groups():
assert [item.name for item in panels[0].interface_items] == [ assert [item.name for item in panels[0].interface_items] == [
"Cleanup", "Cleanup",
"Smooth", "Smooth",
"Reference Depth", "Base Plane Depth",
"Depth Direction", "Depth Direction",
] ]
assert [item.name for item in panels[1].interface_items] == [ assert [item.name for item in panels[1].interface_items] == [
@@ -148,9 +182,11 @@ def test_cutout_modifier_wraps_four_menu_controlled_shape_groups():
menu_switch = next( menu_switch = next(
node for node in cutout.nodes if node.bl_idname == "GeometryNodeMenuSwitch" node for node in cutout.nodes if node.bl_idname == "GeometryNodeMenuSwitch"
) )
assert [item.name for item in menu_switch.enum_items] == list( labels = {
shapes.CUTOUT_SHAPE_LABELS.values() shape: shape.replace("_", " ").title()
) for shape in shapes.CUTOUT_NODE_GROUP_NAMES
}
assert [item.name for item in menu_switch.enum_items] == list(labels.values())
assert any( assert any(
link.from_socket.name == "Shape" link.from_socket.name == "Shape"
and link.to_node == menu_switch and link.to_node == menu_switch
@@ -171,7 +207,7 @@ def test_cutout_modifier_wraps_four_menu_controlled_shape_groups():
} }
assert set(child_nodes) == set(child_groups.values()) assert set(child_nodes) == set(child_groups.values())
for shape, group in child_groups.items(): for shape, group in child_groups.items():
label = shapes.CUTOUT_SHAPE_LABELS[shape] label = labels[shape]
assert any( assert any(
link.from_node == child_nodes[group] link.from_node == child_nodes[group]
and link.to_node == menu_switch and link.to_node == menu_switch
@@ -179,14 +215,14 @@ def test_cutout_modifier_wraps_four_menu_controlled_shape_groups():
for link in cutout.links for link in cutout.links
) )
assert _inputs(child_groups["CUTOUT"]) == ["Geometry", "Thickness"] assert _inputs(child_groups["SURFACE"]) == ["Geometry", "Thickness"]
assert _inputs(child_groups["BALLOON"]) == ["Geometry", "Thickness"] assert _inputs(child_groups["BALLOON"]) == ["Geometry", "Thickness"]
assert _inputs(child_groups["DEPTH_BALLOON"]) == [ assert _inputs(child_groups["DEPTH_BALLOON"]) == [
"Geometry", "Geometry",
"Thickness", "Thickness",
"Depth Scale", "Depth Scale",
"Double Sided", "Double Sided",
"Reference Depth", "Base Plane Depth",
"Depth Direction", "Depth Direction",
"Uniform Scale", "Uniform Scale",
"Depth Image", "Depth Image",
@@ -195,7 +231,7 @@ def test_cutout_modifier_wraps_four_menu_controlled_shape_groups():
"Geometry", "Geometry",
"Thickness", "Thickness",
"Depth Scale", "Depth Scale",
"Reference Depth", "Base Plane Depth",
"Uniform Scale", "Uniform Scale",
"Depth Image", "Depth Image",
] ]
@@ -216,7 +252,7 @@ def test_cutout_modifier_wraps_four_menu_controlled_shape_groups():
"Thickness": set(shapes.CUTOUT_SHAPES), "Thickness": set(shapes.CUTOUT_SHAPES),
"Depth Scale": {"DEPTH_BALLOON", "DEPTH_SURFACE"}, "Depth Scale": {"DEPTH_BALLOON", "DEPTH_SURFACE"},
"Double Sided": {"DEPTH_BALLOON"}, "Double Sided": {"DEPTH_BALLOON"},
"Reference Depth": {"DEPTH_BALLOON", "DEPTH_SURFACE"}, "Base Plane Depth": {"DEPTH_BALLOON", "DEPTH_SURFACE"},
"Depth Direction": {"DEPTH_BALLOON"}, "Depth Direction": {"DEPTH_BALLOON"},
"Uniform Scale": {"DEPTH_BALLOON", "DEPTH_SURFACE"}, "Uniform Scale": {"DEPTH_BALLOON", "DEPTH_SURFACE"},
"Depth Image": {"DEPTH_BALLOON", "DEPTH_SURFACE"}, "Depth Image": {"DEPTH_BALLOON", "DEPTH_SURFACE"},
@@ -252,7 +288,7 @@ def test_cutout_modifier_wraps_four_menu_controlled_shape_groups():
for shape, group in child_groups.items() for shape, group in child_groups.items()
} }
assert repeat_counts == { assert repeat_counts == {
"CUTOUT": 1, "SURFACE": 1,
"BALLOON": 0, "BALLOON": 0,
"DEPTH_BALLOON": 0, "DEPTH_BALLOON": 0,
"DEPTH_SURFACE": 1, "DEPTH_SURFACE": 1,
@@ -278,6 +314,15 @@ def test_cutout_modifier_wraps_four_menu_controlled_shape_groups():
) )
assert cleanup.default_value == 0.0 assert cleanup.default_value == 0.0
assert cleanup.subtype == "NONE" assert cleanup.subtype == "NONE"
cleanup_scale = next(
link.to_node
for link in cutout.links
if link.from_socket.name == "Cleanup"
and link.from_node.bl_idname == "NodeGroupInput"
)
assert cleanup_scale.bl_idname == "ShaderNodeMath"
assert cleanup_scale.operation == "MULTIPLY"
assert np.isclose(cleanup_scale.inputs[1].default_value, 0.01)
assert smooth.default_value == 1.0 assert smooth.default_value == 1.0
assert smooth.subtype == "FACTOR" assert smooth.subtype == "FACTOR"
assert iterations.default_value == 0 assert iterations.default_value == 0
@@ -304,7 +349,7 @@ def test_cutout_modifier_wraps_four_menu_controlled_shape_groups():
for node in extrudes for node in extrudes
) )
for group in ( for group in (
child_groups["CUTOUT"], child_groups["SURFACE"],
child_groups["DEPTH_SURFACE"], child_groups["DEPTH_SURFACE"],
depth_plane, depth_plane,
): ):
@@ -377,8 +422,8 @@ def test_image_layer_switches_between_tangent_and_object_space_normal():
color = bpy.data.images.new("Color", width=2, height=2) color = bpy.data.images.new("Color", width=2, height=2)
normal_image = bpy.data.images.new("Normal", width=2, height=2) normal_image = bpy.data.images.new("Normal", width=2, height=2)
object_material = material.create_material( object_material = material.create_image_material(
"Object Normal", SimpleNamespace(data=SimpleNamespace(name="Object Normal")),
color, color,
1.0, 1.0,
normal_image=normal_image, normal_image=normal_image,
@@ -387,8 +432,8 @@ def test_image_layer_switches_between_tangent_and_object_space_normal():
layer = _group_node(group, object_material.node_tree) layer = _group_node(group, object_material.node_tree)
assert layer.inputs["Object Space"].default_value is True assert layer.inputs["Object Space"].default_value is True
tangent_material = material.create_material( tangent_material = material.create_image_material(
"Tangent Normal", SimpleNamespace(data=SimpleNamespace(name="Tangent Normal")),
color, color,
1.0, 1.0,
normal_image=normal_image, normal_image=normal_image,
@@ -442,17 +487,10 @@ def test_balloon_build_starts_from_the_unified_single_sheet():
source.data = image source.data = image
source.empty_display_size = 2.0 source.empty_display_size = 2.0
bpy.context.collection.objects.link(source) bpy.context.collection.objects.link(source)
balloon_material = bpy.data.materials.new("Balloon Sheet.material") result = _create_cutout_shape(
result = mesh.build_cutout_object(
bpy.context,
source, source,
"BALLOON", "BALLOON",
8, 8,
balloon_material,
regions=((((0, 0), (64, 0), (64, 64), (0, 64)),),),
image_size=(64, 64),
keep_original=True,
) )
assert all(abs(vertex.co.x) < 1e-8 for vertex in result.data.vertices) assert all(abs(vertex.co.x) < 1e-8 for vertex in result.data.vertices)
@@ -478,22 +516,15 @@ def test_teddy_preference_writes_the_shared_balloon_attribute():
source.data = image source.data = image
source.empty_display_size = 2.0 source.empty_display_size = 2.0
bpy.context.collection.objects.link(source) bpy.context.collection.objects.link(source)
teddy_material = bpy.data.materials.new("Teddy Sheet.material")
with patch.object( with patch.object(
mesh, mesh,
"addon_preferences", "addon_preferences",
return_value=SimpleNamespace(balloon_profile_algorithm="TEDDY"), return_value=SimpleNamespace(balloon_profile_algorithm="TEDDY"),
): ):
result = mesh.build_cutout_object( result = _create_cutout_shape(
bpy.context,
source, source,
"CUTOUT", "SURFACE",
8, 8,
teddy_material,
regions=((((0, 0), (64, 0), (64, 64), (0, 64)),),),
image_size=(64, 64),
keep_original=True,
) )
attribute = result.data.attributes[shapes.BALLOON_ATTRIBUTE_NAME] attribute = result.data.attributes[shapes.BALLOON_ATTRIBUTE_NAME]
@@ -502,22 +533,24 @@ def test_teddy_preference_writes_the_shared_balloon_attribute():
assert profile.min() == 0.0 and profile.max() > 0.0 assert profile.min() == 0.0 and profile.max() > 0.0
def test_depth_surface_object_normal_is_remapped_to_cutout_axes(): def test_cutout_material_uses_flat_tangent_normal():
bpy.ops.wm.read_factory_settings(use_empty=True) bpy.ops.wm.read_factory_settings(use_empty=True)
image = bpy.data.images.new( source = bpy.data.objects.new("Source", None)
"Surface Normal", source.empty_display_type = "IMAGE"
width=1, source.data = bpy.data.images.new("Source", width=2, height=2)
height=1, color = bpy.data.images.new("Color", width=2, height=2)
alpha=True, normal = bpy.data.images.new("Normal", width=1, height=1)
float_buffer=True,
result = material.create_image_material(
source,
color,
0.0,
normal_image=normal,
scene=bpy.context.scene,
) )
image.pixels.foreach_set((0.5, 0.5, 1.0, 1.0))
mesh._orient_cutout_object_normal(image) layer = _group_node(material.material_node_group(), result.node_tree)
assert layer.inputs["Object Space"].default_value is False
pixels = np.empty(4, dtype=np.float32)
image.pixels.foreach_get(pixels)
assert np.allclose(pixels, (0.0, 0.5, 0.5, 1.0), atol=1e-6)
def test_internal_depth_plane_image_uses_standard_data_name(): def test_internal_depth_plane_image_uses_standard_data_name():
@@ -540,8 +573,6 @@ def test_depth_surface_protected_smooth_evaluates():
source.data = image source.data = image
source.empty_display_size = 2.0 source.empty_display_size = 2.0
bpy.context.collection.objects.link(source) bpy.context.collection.objects.link(source)
surface_material = bpy.data.materials.new("Surface.material")
size = 8 size = 8
depth = np.ones((size, size), dtype=np.float32) depth = np.ones((size, size), dtype=np.float32)
x, y = np.meshgrid(np.arange(size), np.arange(size)) x, y = np.meshgrid(np.arange(size), np.arange(size))
@@ -553,25 +584,21 @@ def test_depth_surface_protected_smooth_evaluates():
points[..., 0] = (x - intrinsics[0, 2]) / intrinsics[0, 0] * depth points[..., 0] = (x - intrinsics[0, 2]) / intrinsics[0, 0] * depth
points[..., 1] = (y - intrinsics[1, 2]) / intrinsics[1, 1] * depth points[..., 1] = (y - intrinsics[1, 2]) / intrinsics[1, 1] * depth
points[..., 2] = depth points[..., 2] = depth
field = ( field = ImageGeometry(
points, points=points,
intrinsics, intrinsics=intrinsics,
np.ones((size, size), dtype=bool), valid=np.ones((size, size), dtype=bool),
np.ones((size, size), dtype=bool), calibration_valid=np.ones((size, size), dtype=bool),
(size, size), image_size=(size, size),
1.0, reference_depth=1.0,
region_transparent_fraction=0.0,
) )
result = mesh.build_cutout_object( result = _create_cutout_shape(
bpy.context,
source, source,
"DEPTH_SURFACE", "DEPTH_SURFACE",
8, 8,
surface_material,
field=field, field=field,
regions=((((0, 0), (64, 0), (64, 64), (0, 64)),),),
image_size=(64, 64),
keep_original=True,
) )
assert all(abs(vertex.co.x) < 1e-8 for vertex in result.data.vertices) assert all(abs(vertex.co.x) < 1e-8 for vertex in result.data.vertices)
@@ -581,7 +608,7 @@ def test_depth_surface_protected_smooth_evaluates():
assert modifier[ assert modifier[
mesh.modifier_input_identifier(modifier.node_group, "Shape") mesh.modifier_input_identifier(modifier.node_group, "Shape")
] == shapes.CUTOUT_SHAPE_INDICES["DEPTH_SURFACE"] ] == shapes.CUTOUT_SHAPE_INDICES["DEPTH_SURFACE"]
expected_cleanup = mesh._cutout_cleanup_area( expected_cleanup = mesh._cutout_cleanup_value(
mesh._cutout_point_spacing(source, 8), mesh._cutout_point_spacing(source, 8),
mesh._world_plane_metric(source.matrix_world)[0], mesh._world_plane_metric(source.matrix_world)[0],
) )
@@ -599,7 +626,7 @@ def test_depth_surface_protected_smooth_evaluates():
] == 0 ] == 0
reference_id = mesh.modifier_input_identifier( reference_id = mesh.modifier_input_identifier(
modifier.node_group, modifier.node_group,
"Reference Depth", "Base Plane Depth",
) )
assert modifier[reference_id] == 8.0 assert modifier[reference_id] == 8.0
mesh.set_modifier_input( mesh.set_modifier_input(
@@ -656,52 +683,9 @@ def test_depth_surface_protected_smooth_evaluates():
finally: finally:
evaluated.to_mesh_clear() evaluated.to_mesh_clear()
mesh.set_modifier_input( assert internal_positions.ndim == 2
modifier, assert internal_positions.shape[1] == 3
mesh.modifier_input_identifier( assert np.isfinite(internal_positions).all()
modifier.node_group, "Smooth Iterations"
),
0,
)
external_group = bpy.data.node_groups.new("External O Smooth", "GeometryNodeTree")
external_group.is_modifier = True
node_graph.interface_socket(
external_group, "Geometry", "OUTPUT", "NodeSocketGeometry"
)
node_graph.interface_socket(
external_group, "Geometry", "INPUT", "NodeSocketGeometry"
)
external_iterations = node_graph.interface_socket(
external_group, "Iterations", "INPUT", "NodeSocketInt"
)
external_iterations.default_value = 4
external_iterations.min_value = 0
external_smooth = node_graph.interface_socket(
external_group, "Smooth", "INPUT", "NodeSocketFloat"
)
external_smooth.default_value = 1.0
external_input = external_group.nodes.new("NodeGroupInput")
external_output = external_group.nodes.new("NodeGroupOutput")
external_geometry = node_graph.expand_smooth(
external_group, external_input.outputs["Geometry"]
)
external_group.links.new(
external_geometry, external_output.inputs["Geometry"]
)
external_modifier = result.modifiers.new("External O Smooth", "NODES")
external_modifier.node_group = external_group
result.update_tag(refresh={"DATA"})
bpy.context.view_layer.update()
evaluated = result.evaluated_get(bpy.context.evaluated_depsgraph_get())
external_mesh = evaluated.to_mesh()
try:
external_positions = np.asarray(
[tuple(vertex.co) for vertex in external_mesh.vertices]
)
finally:
evaluated.to_mesh_clear()
assert internal_positions.shape == external_positions.shape
assert np.allclose(internal_positions, external_positions, atol=1e-6)
@@ -756,7 +740,7 @@ def test_cutout_thickness_keeps_one_connected_shell():
data.from_pydata(vertices, (), faces) data.from_pydata(vertices, (), faces)
obj = bpy.data.objects.new("Connected Cutout", data) obj = bpy.data.objects.new("Connected Cutout", data)
bpy.context.collection.objects.link(obj) bpy.context.collection.objects.link(obj)
modifier = _add_cutout_modifier(obj, "CUTOUT") modifier = _add_cutout_modifier(obj, "SURFACE")
mesh.set_modifier_input( mesh.set_modifier_input(
modifier, modifier,
@@ -923,7 +907,7 @@ def test_cutout_cleanup_uses_absolute_area_with_many_small_islands():
data.from_pydata(vertices, (), faces) data.from_pydata(vertices, (), faces)
obj = bpy.data.objects.new("Cleanup", data) obj = bpy.data.objects.new("Cleanup", data)
bpy.context.collection.objects.link(obj) bpy.context.collection.objects.link(obj)
modifier = _add_cutout_modifier(obj, "CUTOUT") modifier = _add_cutout_modifier(obj, "SURFACE")
cleanup_id = mesh.modifier_input_identifier(modifier.node_group, "Cleanup") cleanup_id = mesh.modifier_input_identifier(modifier.node_group, "Cleanup")
@@ -931,7 +915,7 @@ def test_cutout_cleanup_uses_absolute_area_with_many_small_islands():
bpy.context.view_layer.update() bpy.context.view_layer.update()
assert len(_evaluated_positions(obj)) == 45 assert len(_evaluated_positions(obj)) == 45
mesh.set_modifier_input(modifier, cleanup_id, 0.001) mesh.set_modifier_input(modifier, cleanup_id, 0.1)
obj.update_tag(refresh={"DATA"}) obj.update_tag(refresh={"DATA"})
bpy.context.view_layer.update() bpy.context.view_layer.update()
assert len(_evaluated_positions(obj)) == 9 assert len(_evaluated_positions(obj)) == 9
@@ -945,7 +929,6 @@ def test_depth_balloon_uses_metric_plane_inputs_without_controller():
source.data = image source.data = image
source.empty_display_size = 2.0 source.empty_display_size = 2.0
bpy.context.collection.objects.link(source) bpy.context.collection.objects.link(source)
balloon_material = bpy.data.materials.new("Depth Balloon.material")
crop_bounds = (50, 20, 90, 80) crop_bounds = (50, 20, 90, 80)
crop = ((0, 0), (40, 0), (40, 60), (0, 60)) crop = ((0, 0), (40, 0), (40, 60), (0, 60))
local_bounds = mesh._crop_plane_bounds( local_bounds = mesh._crop_plane_bounds(
@@ -965,26 +948,22 @@ def test_depth_balloon_uses_metric_plane_inputs_without_controller():
((50.0, 0.0, 20.0), (0.0, 50.0, 30.0), (0.0, 0.0, 1.0)), ((50.0, 0.0, 20.0), (0.0, 50.0, 30.0), (0.0, 0.0, 1.0)),
dtype=np.float32, dtype=np.float32,
) )
field = ( field = ImageGeometry(
points, points=points,
intrinsics, intrinsics=intrinsics,
np.ones((61, 41), dtype=bool), valid=np.ones((61, 41), dtype=bool),
np.ones((61, 41), dtype=bool), calibration_valid=np.ones((61, 41), dtype=bool),
(41, 61), image_size=(41, 61),
2.0, reference_depth=2.0,
region_transparent_fraction=0.05,
) )
result = mesh.build_cutout_object( result = _create_cutout_shape(
bpy.context,
source, source,
"DEPTH_BALLOON", "DEPTH_BALLOON",
8, 8,
balloon_material,
field=field, field=field,
regions=((crop,),), bounds=crop_bounds,
image_size=(40, 60),
bounds=local_bounds,
keep_original=True,
) )
assert all(abs(vertex.co.x) < 1e-8 for vertex in result.data.vertices) assert all(abs(vertex.co.x) < 1e-8 for vertex in result.data.vertices)
@@ -1019,15 +998,15 @@ def test_depth_balloon_uses_metric_plane_inputs_without_controller():
depth_reference_id = mesh.modifier_input_identifier( depth_reference_id = mesh.modifier_input_identifier(
modifier.node_group, modifier.node_group,
"Reference Depth", "Base Plane Depth",
) )
assert modifier[depth_reference_id] == 8.0 assert not np.isclose(modifier[depth_reference_id], 8.0)
depth_direction_id = mesh.modifier_input_identifier( depth_direction_id = mesh.modifier_input_identifier(
modifier.node_group, modifier.node_group,
"Depth Direction", "Depth Direction",
) )
direction = np.asarray(modifier[depth_direction_id], dtype=np.float64) direction = np.asarray(modifier[depth_direction_id], dtype=np.float64)
assert np.array_equal(direction, (0.0, 0.0, 1.0)) assert not np.allclose(direction, (0.0, 0.0, 1.0))
uniform_scale_id = mesh.modifier_input_identifier( uniform_scale_id = mesh.modifier_input_identifier(
modifier.node_group, modifier.node_group,
"Uniform Scale", "Uniform Scale",
@@ -1130,7 +1109,7 @@ def test_loaded_normal_image_uses_normal_texture_name(tmp_path):
Image.new("RGB", (1, 1), (128, 128, 255)).save(path) Image.new("RGB", (1, 1), (128, 128, 255)).save(path)
source = SimpleNamespace(data=SimpleNamespace(name="Poster.png")) source = SimpleNamespace(data=SimpleNamespace(name="Poster.png"))
image = mesh._load_normal_image(path, source) image = common_image.load_normal_result_image(path, source)
assert image.name == "Poster_normal.png" assert image.name == "Poster_normal.png"
assert image.colorspace_settings.name == "Non-Color" assert image.colorspace_settings.name == "Non-Color"
@@ -1155,20 +1134,19 @@ def test_converted_color_images_are_new_and_use_blender_duplicate_suffix():
def test_local_cutout_selection_matches_uint8_tenth_alpha_boundary(): def test_local_cutout_selection_matches_uint8_tenth_alpha_boundary():
bpy.ops.wm.read_factory_settings(use_empty=True)
image = bpy.data.images.new("Alpha Cutout", width=4, height=4, alpha=True)
top_down = np.ones((4, 4, 4), dtype=np.float32) top_down = np.ones((4, 4, 4), dtype=np.float32)
top_down[1, 1, 3] = 25.0 / 255.0 top_down[1, 1, 3] = 25.0 / 255.0
top_down[1, 2, 3] = 26.0 / 255.0 top_down[1, 2, 3] = 26.0 / 255.0
image.pixels.foreach_set(np.flipud(top_down).ravel()) region_mask = rasterize_region_path(
source = bpy.data.objects.new("Alpha Cutout", None) (4, 4),
source.empty_display_type = "IMAGE" RegionPath(
source.data = image points=((0, 0), (4, 0), (4, 4), (0, 4)),
fill_rule="NONZERO",
_rgba, selection = mesh._local_cutout_rgba( ),
source, source_alpha=top_down[:, :, 3],
((0, 0), (4, 0), (4, 4), (0, 4)), alpha_threshold=0.1,
) )
selection = region_mask.full_values((4, 4))
assert selection[1, 1] == 0.0 assert selection[1, 1] == 0.0
assert selection[1, 2] > 0.1 assert selection[1, 2] > 0.1
@@ -1184,28 +1162,47 @@ def test_local_cutout_builds_from_blender_image_pixels_without_a_job():
bpy.context.collection.objects.link(source) bpy.context.collection.objects.link(source)
source.select_set(True) source.select_set(True)
bpy.context.view_layer.objects.active = source bpy.context.view_layer.objects.active = source
selection = "[[-8,8],[72,8],[72,40],[-8,40]]" region_path = RegionPath(
source_bounds = mesh.plane_bounds(source) points=((-8, 8), (72, 8), (72, 40), (-8, 40)),
fill_rule="NONZERO",
)
source_bounds = mesh.image_empty_bounds(source)
with patch.object(mesh, "push_undo") as undo: with patch.object(
mesh.build_local_cutout( mesh,
SimpleNamespace(mesh_detail=16), "image_rgba",
bpy.context, wraps=mesh.image_rgba,
source, ) as read_rgba:
"CUTOUT", rgba = mesh.image_rgba(image)
selection, region_mask = rasterize_region_path(
tuple(image.size),
region_path,
source_alpha=rgba[:, :, 3],
alpha_threshold=mesh.CUTOUT_ALPHA_THRESHOLD,
antialias=True,
) )
with patch.object(mesh, "configured_max_ai_input_size", return_value=2048):
mesh.region_to_shape(
bpy.context,
source,
"SURFACE",
16,
region_mask,
source_rgba=rgba,
)
read_rgba.assert_called_once_with(image)
result = bpy.context.view_layer.objects.active result = bpy.context.view_layer.objects.active
assert result is not source assert result is not source
assert result.type == "MESH" assert result.type == "MESH"
assert result["anyimage_mesh_shape"] == "CUTOUT" assert result["anyimage_mesh_shape"] == "SURFACE"
modifier = result.modifiers[0] modifier = result.modifiers[0]
assert modifier.node_group.name == shapes.CUTOUT_NODE_GROUP_NAME assert modifier.node_group.name == shapes.CUTOUT_NODE_GROUP_NAME
assert modifier[ assert modifier[
mesh.modifier_input_identifier(modifier.node_group, "Shape") mesh.modifier_input_identifier(modifier.node_group, "Shape")
] == shapes.CUTOUT_SHAPE_INDICES["CUTOUT"] ] == shapes.CUTOUT_SHAPE_INDICES["SURFACE"]
expected_cleanup = mesh._cutout_cleanup_area( expected_cleanup = mesh._cutout_cleanup_value(
mesh._cutout_point_spacing(source, 16), mesh._cutout_point_spacing(source, 16),
mesh._world_plane_metric(source.matrix_world)[0], mesh._world_plane_metric(source.matrix_world)[0],
) )
@@ -1246,4 +1243,3 @@ def test_local_cutout_builds_from_blender_image_pixels_without_a_job():
color_texture.image.pixels.foreach_get(copied) color_texture.image.pixels.foreach_get(copied)
copied = copied.reshape((34, 64, 4)) copied = copied.reshape((34, 64, 4))
assert np.allclose(copied, 1.0) assert np.allclose(copied, 1.0)
undo.assert_called_once_with("Convert to Cutout")

42
tests/test_pack.py
View File

@@ -67,26 +67,21 @@ class PackTest(unittest.TestCase):
self.assertNotIn("assets/O_AnyImage.blend1", names) self.assertNotIn("assets/O_AnyImage.blend1", names)
for path in ( for path in (
"operators/common/__init__.py", "operators/common/__init__.py",
"operators/common/image.py", "operators/common/ai_job.py",
"operators/common/material.py", "operators/common/image_data.py",
"operators/common/node_asset.py",
"operators/common/normal_map.py",
"operators/common/operator.py",
"operators/common/image_edit.py",
"operators/common/image_gesture.py",
"operators/common/image_interaction.py", "operators/common/image_interaction.py",
"operators/common/image_mask.py", "operators/common/image_region.py",
"operators/common/node_graph.py", "operators/common/mesh_result.py",
"operators/convert_to_plane/normal_map.py",
"operators/convert_to_plane/resolution.py",
"operators/cutout_tool/__init__.py", "operators/cutout_tool/__init__.py",
"operators/cutout_tool/balloon_profile.py", "operators/cutout_tool/select_region.py",
"operators/cutout_tool/build_shapes.py", "operators/cutout_tool/region_to_shape.py",
"operators/cutout_tool/depth_calibration.py", "operators/cutout_tool/geometry/__init__.py",
"operators/cutout_tool/mask_regions.py", "operators/cutout_tool/geometry/balloon.py",
"operators/cutout_tool/node_assets.py", "operators/cutout_tool/geometry/depth_calibration.py",
"operators/cutout_tool/planar_mesh.py", "operators/cutout_tool/geometry/region_contours.py",
"operators/cutout_tool/selection_tool.py", "operators/cutout_tool/geometry/triangulation.py",
"operators/cutout_tool/shape_conversion.py",
"operators/cutout_tool/shape_types.py",
"operators/convert_to_plane/__init__.py", "operators/convert_to_plane/__init__.py",
"operators/convert_to_plane/build.py", "operators/convert_to_plane/build.py",
"operators/convert_to_plane/conversion.py", "operators/convert_to_plane/conversion.py",
@@ -118,10 +113,15 @@ class PackTest(unittest.TestCase):
self.assertIn("server/models/geometry.py", names) self.assertIn("server/models/geometry.py", names)
self.assertIn("server/jobs/remove_background.py", names) self.assertIn("server/jobs/remove_background.py", names)
self.assertIn("server/jobs/refine_image_mask.py", names) self.assertIn("server/jobs/refine_image_mask.py", names)
self.assertIn("server/selection/cutout_images.py", names) self.assertIn("server/region/generate_selection.py", names)
self.assertIn("server/selection/polygon_mask.py", names) self.assertIn("server/jobs/generate_region_data.py", names)
self.assertIn("server/jobs/prepare_geometry_data.py", names) self.assertIn("server/jobs/generate_depth_plane.py", names)
self.assertIn("server/geometry/generate_depth.py", names)
self.assertIn("server/geometry/normal_texture.py", names) self.assertIn("server/geometry/normal_texture.py", names)
self.assertNotIn("operators/common/image_selection.py", names)
self.assertNotIn("server/selection/cutout_images.py", names)
self.assertNotIn("server/selection/polygon_mask.py", names)
self.assertNotIn("server/jobs/prepare_geometry_data.py", names)
self.assertNotIn("server/geometry/cutout.py", names) self.assertNotIn("server/geometry/cutout.py", names)
self.assertIn("server/models/ben2.py", names) self.assertIn("server/models/ben2.py", names)
self.assertIn("server/models/da3.py", names) self.assertIn("server/models/da3.py", names)

701
tests/test_server.py
View File

@@ -41,8 +41,8 @@ class ServerTest(unittest.TestCase):
@classmethod @classmethod
def setUpClass(cls): def setUpClass(cls):
cls.server_module = load_server_module() cls.server_module = load_server_module()
importlib.import_module("server.selection.polygon_mask") importlib.import_module("server.geometry.generate_depth")
importlib.import_module("server.jobs.prepare_geometry_data") importlib.import_module("server.jobs.generate_region_data")
from blendjob import JobServer from blendjob import JobServer
from server.model_manager import ModelManager, ModelSessionCache from server.model_manager import ModelManager, ModelSessionCache
@@ -261,7 +261,8 @@ class ServerTest(unittest.TestCase):
"refine-image-mask", "refine-image-mask",
"remove-background", "remove-background",
"upscale-image", "upscale-image",
"prepare-geometry-data", "generate-depth-plane",
"generate-region-data",
"debug-da3", "debug-da3",
"debug-moge2", "debug-moge2",
"debug-ben2", "debug-ben2",
@@ -385,652 +386,116 @@ class ServerTest(unittest.TestCase):
) )
self.assertEqual(events, ["clear", "close"]) self.assertEqual(events, ["clear", "close"])
def test_polygon_selection_mask_uses_even_odd_fill_for_self_intersection(self): def test_region_transparent_fraction_uses_visible_alpha_threshold(self):
image_mask = sys.modules["server.selection.polygon_mask"]
polygon = (
(1, 1),
(9, 1),
(9, 9),
(1, 9),
(1, 1),
(3, 3),
(3, 7),
(7, 7),
(7, 3),
(3, 3),
(1, 1),
)
mask = image_mask.polygon_selection_mask((10, 10), polygon)
self.assertTrue(mask[2, 2])
self.assertFalse(mask[4, 4])
self.assertEqual(int(mask.sum()), 48)
def test_polygon_selection_mask_nonzero_keeps_same_direction_overlap(self):
image_mask = sys.modules["server.selection.polygon_mask"]
polygon = (
(1, 1),
(7, 1),
(7, 7),
(1, 7),
(1, 1),
(4, 1),
(10, 1),
(10, 7),
(4, 7),
(4, 1),
(1, 1),
)
even_odd = image_mask.polygon_selection_mask((12, 9), polygon)
nonzero = image_mask.polygon_selection_mask(
(12, 9), polygon, "NONZERO"
)
self.assertTrue(even_odd[3, 2])
self.assertFalse(even_odd[3, 5])
self.assertTrue(nonzero[3, 2])
self.assertTrue(nonzero[3, 5])
self.assertTrue(nonzero[3, 8])
def test_image_mask_canvas_clears_rgba_outside_the_selection(self):
import numpy as np import numpy as np
from PIL import Image from PIL import Image
image_mask = sys.modules["server.selection.polygon_mask"] depth = sys.modules["server.geometry.generate_depth"]
source_path = self.storage_root / "source.png" output_path = self.storage_root / "region.png"
pixels = np.full((2, 2, 4), (20, 40, 60, 255), dtype=np.uint8) pixels = np.full((10, 10, 4), 255, dtype=np.uint8)
Image.fromarray(pixels, mode="RGBA").save(source_path) pixels.reshape(-1, 4)[:5, 3] = 25
Image.fromarray(pixels, mode="RGBA").save(output_path)
canvas, _selection, bounds = image_mask.create_selection_canvas( self.assertEqual(depth.transparent_fraction(output_path), 0.05)
source_path,
((-1, -1), (2, -1), (-1, 2)),
)
result = np.asarray(canvas)
self.assertEqual(bounds, (-1, -1, 2, 2)) def test_image_geometry_keeps_model_and_region_validity_separate(self):
self.assertEqual(result.shape, (3, 3, 4))
self.assertEqual(tuple(result[2, 2]), (0, 0, 0, 0))
self.assertEqual(tuple(result[1, 1]), (20, 40, 60, 255))
def test_image_mask_crops_tightly_to_alpha(self):
import numpy as np
from PIL import Image
image_mask = sys.modules["server.selection.polygon_mask"]
pixels = np.zeros((4, 5, 4), dtype=np.uint8)
pixels[1:3, 2:4] = (20, 40, 60, 255)
result, bounds = image_mask.crop_to_alpha(
Image.fromarray(pixels, mode="RGBA"),
(10, 20, 15, 24),
)
result = np.asarray(result)
self.assertEqual(bounds, (12, 21, 14, 23))
self.assertEqual(result.shape, (2, 2, 4))
self.assertTrue(np.all(result[:, :, 3] == 255))
def test_mesh_refine_selection_keeps_source_size_and_selection_alpha(self):
cutout_job = importlib.import_module("server.selection.cutout_images")
import numpy as np
from PIL import Image
mesh_job = sys.modules["server.jobs.prepare_geometry_data"]
source_path = self.storage_root / "source.png"
pixels = np.full((6, 8, 4), (20, 40, 60, 255), dtype=np.uint8)
Image.fromarray(pixels, mode="RGBA").save(source_path)
job_directory = self.storage_root / "job"
job_directory.mkdir()
context = SimpleNamespace(
directory=job_directory,
resource=lambda _name: SimpleNamespace(
directory=lambda _model: Path("model")
),
check_cancelled=lambda: None,
)
def passthrough(path, *_args, **_kwargs):
return Image.open(path).convert("RGBA"), 0.0, 0.0
output_path = job_directory / "image.0001.png"
with patch.object(
cutout_job,
"infer_foreground",
side_effect=passthrough,
):
cutout_job.refine_selection(
context,
{"refine_model": "BEN2_BASE", "device": "cpu"},
source_path,
((1, 1), (6, 1), (1, 5)),
output_path,
)
result = np.asarray(Image.open(output_path).convert("RGBA"))
self.assertEqual(result.shape, (6, 8, 4))
self.assertEqual(tuple(result[1, 1]), (20, 40, 60, 255))
self.assertEqual(tuple(result[5, 7]), (0, 0, 0, 0))
def test_mesh_cutout_images_crop_color_and_preserve_source_alpha(self):
cutout_job = importlib.import_module("server.selection.cutout_images")
import json
import numpy as np
from PIL import Image
mesh_job = sys.modules["server.jobs.prepare_geometry_data"]
source_path = self.storage_root / "analysis-source.png"
selection_path = self.storage_root / "selection-source.png"
output_directory = self.storage_root / "cutout-images"
output_directory.mkdir()
source = np.full((6, 8, 4), (10, 80, 160, 255), dtype=np.uint8)
source[2, 3, 3] = 25
source[2, 4, 3] = 26
source[3, 5, 3] = 0
Image.fromarray(source, mode="RGBA").save(source_path)
selection = np.zeros((6, 8, 4), dtype=np.uint8)
selection[1:5, 2:7] = (200, 20, 40, 255)
selection[1, 2] = (200, 20, 40, 0)
Image.fromarray(selection, mode="RGBA").save(selection_path)
color_path, mask_path, analysis_path, bounds_path = (
cutout_job.write_images(
source_path,
selection_path,
output_directory,
)
)
color = np.asarray(Image.open(color_path).convert("RGBA"))
mask = np.asarray(Image.open(mask_path).convert("RGBA"))
analysis = np.asarray(Image.open(analysis_path).convert("RGBA"))
self.assertEqual(json.loads(bounds_path.read_text()), [2, 1, 7, 5])
self.assertEqual(color.shape, (4, 5, 4))
self.assertEqual(tuple(color[1, 1]), (10, 80, 160, 25))
self.assertEqual(int(mask[1, 1, 3]), 0)
self.assertEqual(tuple(analysis[1, 1]), (0, 0, 0, 0))
self.assertEqual(tuple(color[1, 2]), (10, 80, 160, 26))
self.assertGreater(int(mask[1, 2, 3]), 25)
self.assertEqual(int(mask[0, 0, 3]), 0)
self.assertEqual(tuple(color[0, 0]), (10, 80, 160, 255))
self.assertEqual(tuple(analysis[0, 0]), (0, 0, 0, 0))
self.assertEqual(tuple(analysis[1, 2]), (10, 80, 160, 26))
self.assertEqual(tuple(color[2, 3]), (10, 80, 160, 0))
self.assertEqual(int(mask[2, 3, 3]), 0)
self.assertEqual(tuple(analysis[2, 3]), (0, 0, 0, 0))
def test_mesh_unrefined_cutout_discards_selection_outside_the_source(self):
cutout_job = importlib.import_module("server.selection.cutout_images")
import json
import numpy as np
from PIL import Image
mesh_job = sys.modules["server.jobs.prepare_geometry_data"]
source_path = self.storage_root / 'outside-source.png'
full_selection_path = self.storage_root / 'outside-selection-full.png'
output_directory = self.storage_root / 'outside-cutout'
output_directory.mkdir()
Image.new('RGBA', (8, 6), (10, 80, 160, 255)).save(source_path)
polygon = ((-2, 1), (10, 1), (10, 5), (-2, 5))
_selection_path, selection_bounds = (
cutout_job.write_selection(
source_path,
polygon,
full_selection_path,
)
)
color_path, mask_path, analysis_path, bounds_path = (
cutout_job.write_images(
source_path,
full_selection_path,
output_directory,
selection_bounds,
)
)
color = np.asarray(Image.open(color_path).convert('RGBA'))
mask = np.asarray(Image.open(mask_path).convert('RGBA'))
analysis = np.asarray(Image.open(analysis_path).convert('RGBA'))
self.assertEqual(selection_bounds, (-2, 1, 10, 5))
self.assertEqual(json.loads(bounds_path.read_text()), [0, 1, 8, 5])
self.assertEqual(color.shape, (4, 8, 4))
self.assertEqual(tuple(color[1, 0]), (10, 80, 160, 255))
self.assertEqual(tuple(color[1, 7]), (10, 80, 160, 255))
self.assertGreater(int(mask[1, 0, 3]), 127)
self.assertEqual(tuple(analysis[1, 0]), (10, 80, 160, 255))
def test_mesh_job_discards_unrefined_outside_selection_with_normal(self):
import json
import numpy as np
from PIL import Image
mesh_job = sys.modules["server.jobs.prepare_geometry_data"]
source_path = self.storage_root / "mesh-source.png"
pixels = np.full((6, 8, 4), (20, 40, 60, 255), dtype=np.uint8)
Image.fromarray(pixels, mode="RGBA").save(source_path)
job_directory = self.storage_root / "mesh-job"
context = SimpleNamespace(
directory=job_directory,
progress=lambda *_args: None,
check_cancelled=lambda: None,
resource=lambda _name: object(),
)
normal = np.zeros((1, 4, 12, 3), dtype=np.float32)
normal[..., 0] = 1.0
prediction = SimpleNamespace(normal=normal)
received = []
received_images = []
def infer(_manager, parameters, images, **_kwargs):
received.append(parameters)
received_images.extend(images)
return prediction
with patch.object(mesh_job.moge2, "infer", side_effect=infer):
result = mesh_job.run(
context,
{
"input": str(source_path),
"shape": "BALLOON",
"polygon": ((-2, 1), (10, 1), (10, 5), (-2, 5)),
"mesh_detail": 64,
"invert": False,
"device": "cpu",
"model": "MOGE2_VITS_NORMAL",
"resolution_level": 5,
"normal_analysis": True,
},
)
color_path = job_directory / "image.0001.png"
color_pixels = np.asarray(Image.open(color_path).convert("RGBA"))
normal_pixels = np.asarray(
Image.open(job_directory / result["normal"]).convert("RGB")
)
self.assertEqual(
result,
{
"selection": "selection.png",
"bounds": "bounds.json",
"normal": "normal.png",
},
)
self.assertTrue(color_path.is_file())
self.assertFalse((job_directory / "contour.json").exists())
self.assertEqual(received[0]["model"], "MOGE2_VITS_NORMAL")
self.assertEqual(received_images, [job_directory / "analysis.png"])
analysis_pixels = np.asarray(
Image.open(received_images[0]).convert("RGBA")
)
self.assertEqual(
json.loads((job_directory / "bounds.json").read_text()),
[0, 1, 8, 5],
)
self.assertEqual(tuple(analysis_pixels[2, 2, :3]), (20, 40, 60))
self.assertGreater(int(analysis_pixels[2, 2, 3]), 240)
self.assertEqual(tuple(analysis_pixels[2, 0, :3]), (20, 40, 60))
self.assertEqual(tuple(color_pixels[2, 0, :3]), (20, 40, 60))
self.assertGreater(int(color_pixels[2, 0, 3]), 240)
self.assertEqual(tuple(color_pixels[2, 7, :3]), (20, 40, 60))
self.assertEqual(color_pixels.shape[:2], (4, 8))
self.assertEqual(analysis_pixels.shape[:2], (4, 8))
self.assertEqual(normal_pixels.shape[:2], (4, 8))
self.assertEqual(tuple(normal_pixels[0, 0]), (255, 127, 127))
self.assertEqual(tuple(normal_pixels[2, 0]), (255, 127, 127))
def test_cutout_limits_the_cropped_model_input_but_keeps_color_resolution(self):
import numpy as np
from PIL import Image
mesh_job = sys.modules["server.jobs.prepare_geometry_data"]
source_path = self.storage_root / "limited-cutout-source.png"
Image.new("RGBA", (20, 10), (20, 40, 60, 255)).save(source_path)
job_directory = self.storage_root / "limited-cutout-job"
context = SimpleNamespace(
directory=job_directory,
progress=lambda *_args: None,
check_cancelled=lambda: None,
resource=lambda _name: object(),
)
prediction = SimpleNamespace(
normal=np.zeros((1, 2, 8, 3), dtype=np.float32)
)
inferred_sizes = []
def infer(_manager, _parameters, images, **_kwargs):
with Image.open(images[0]) as model_input:
inferred_sizes.append(model_input.size)
return prediction
with patch.object(mesh_job.moge2, "infer", side_effect=infer):
result = mesh_job.run(
context,
{
"input": str(source_path),
"shape": "BALLOON",
"polygon": ((2, 3), (18, 3), (18, 7), (2, 7)),
"mesh_detail": 64,
"max_input_size": 8,
"invert": False,
"device": "cpu",
"model": "MOGE2_VITS_NORMAL",
"resolution_level": 5,
"normal_analysis": True,
},
)
with Image.open(job_directory / "image.0001.png") as color:
color_size = color.size
with Image.open(job_directory / result["normal"]) as normal:
normal_size = normal.size
self.assertEqual(inferred_sizes, [(8, 2)])
self.assertEqual(color_size, (16, 4))
self.assertEqual(normal_size, (16, 4))
self.assertFalse((job_directory / "model-input.png").exists())
def test_mesh_job_skips_moge_normal_when_option_is_disabled(self):
from PIL import Image
mesh_job = sys.modules["server.jobs.prepare_geometry_data"]
source_path = self.storage_root / "mesh-no-normal-source.png"
Image.new("RGBA", (8, 6), (20, 40, 60, 255)).save(source_path)
job_directory = self.storage_root / "mesh-no-normal-job"
context = SimpleNamespace(
directory=job_directory,
progress=lambda *_args: None,
check_cancelled=lambda: None,
resource=lambda _name: object(),
)
with patch.object(mesh_job, "_write_camera_normal") as write_normal:
result = mesh_job.run(
context,
{
"input": str(source_path),
"shape": "BALLOON",
"polygon": ((0, 0), (8, 0), (8, 6), (0, 6)),
"mesh_detail": 64,
"normal_analysis": False,
"invert": False,
"device": "cpu",
},
)
self.assertEqual(
result,
{"selection": "selection.png", "bounds": "bounds.json"},
)
write_normal.assert_not_called()
self.assertFalse((job_directory / "normal.png").exists())
def test_mesh_depth_balloon_uses_metric_point_field(self):
from PIL import Image
mesh_job = sys.modules["server.jobs.prepare_geometry_data"]
source_path = self.storage_root / "mesh-depth-balloon-source.png"
Image.new("RGBA", (6, 6), (20, 40, 60, 255)).save(source_path)
job_directory = self.storage_root / "mesh-depth-balloon-job"
context = SimpleNamespace(
directory=job_directory,
progress=lambda *_args: None,
check_cancelled=lambda: None,
)
field_path = job_directory / "field.npz"
with patch.object(
mesh_job,
"_write_depth_field",
return_value=(field_path, None),
) as write_surface:
result = mesh_job.run(
context,
{
"input": str(source_path),
"shape": "DEPTH_BALLOON",
"polygon": ((1, 1), (5, 1), (5, 5), (1, 5)),
"normal_analysis": False,
"video_frames": 1,
"invert": False,
},
)
self.assertEqual(
result,
{
"selection": "selection.png",
"bounds": "bounds.json",
"field": "field.npz",
},
)
self.assertTrue(write_surface.called)
self.assertEqual(
write_surface.call_args.kwargs["selection_path"],
job_directory / "selection.png",
)
def test_moge_depth_and_normal_share_one_prediction(self):
import numpy as np import numpy as np
mesh_job = sys.modules["server.jobs.prepare_geometry_data"] depth = sys.modules["server.geometry.generate_depth"]
job_directory = self.storage_root / "shared-moge-prediction" output_path = self.storage_root / "image_geometry.npz"
job_directory.mkdir() valid = np.asarray(((True, True, False), (True, True, True)))
color_path = job_directory / "analysis.png" calibration = np.asarray(((False, True, False), (False, False, True)))
normal_path = job_directory / "normal.png"
prediction = object()
valid = np.ones((2, 3), dtype=bool)
frame = SimpleNamespace( frame = SimpleNamespace(
depth=np.ones((2, 3), dtype=np.float32), depth=np.asarray(((1.0, 2.0, 3.0), (4.0, 5.0, 6.0))),
points=np.ones((2, 3, 3), dtype=np.float32), points=np.ones((2, 3, 3), dtype=np.float32),
intrinsics=np.eye(3, dtype=np.float32), intrinsics=np.eye(3, dtype=np.float32),
valid_mask=valid, valid_mask=valid,
model_mask=valid, model_mask=None,
)
context = SimpleNamespace(
directory=job_directory,
resource=lambda _name: object(),
check_cancelled=lambda: None,
)
parameters = {
"model": "MOGE2_VITS_NORMAL",
"device": "cpu",
"resolution_level": 5,
}
with (
patch.object(
mesh_job.moge2, "infer", return_value=prediction
) as infer,
patch.object(
mesh_job.moge2, "geometry_frame", return_value=frame
),
patch.object(mesh_job, "_write_field") as write_field,
patch.object(
mesh_job,
"_write_camera_normal_from_prediction",
return_value=normal_path,
) as write_normal,
):
field_path, written_normal_path = mesh_job._write_depth_field(
context,
parameters,
color_path,
include_normal=True,
)
infer.assert_called_once()
write_field.assert_called_once()
write_normal.assert_called_once_with(
prediction, color_path, normal_path
)
self.assertEqual(field_path, job_directory / "field.npz")
self.assertEqual(written_normal_path, normal_path)
def test_depth_calibration_mask_is_separate_from_model_validity(self):
import numpy as np
from PIL import Image
mesh_job = sys.modules["server.jobs.prepare_geometry_data"]
selection_path = self.storage_root / "depth-selection.png"
selection = np.zeros((2, 3, 4), dtype=np.uint8)
selection[0, 1, 3] = 255
selection[1, 2, 3] = 255
Image.fromarray(selection, mode="RGBA").save(selection_path)
frame = SimpleNamespace(
valid_mask=np.asarray(
((True, True, True), (True, True, False)),
dtype=bool,
)
) )
original_valid = frame.valid_mask.copy() depth._write_image_geometry_archive(frame, output_path, calibration, 0.05)
result = mesh_job._selection_calibration_mask(frame, selection_path)
np.testing.assert_array_equal( with np.load(output_path) as archive:
result,
((False, True, False), (False, False, False)),
)
np.testing.assert_array_equal(frame.valid_mask, original_valid)
def test_depth_field_writes_original_and_calibration_masks_separately(self):
import numpy as np
mesh_job = sys.modules["server.jobs.prepare_geometry_data"]
output_path = self.storage_root / "field.npz"
valid = np.asarray(((True, True), (True, False)), dtype=bool)
calibration = np.asarray(((False, True), (False, False)), dtype=bool)
depth = np.asarray(((2.0, 2.1), (2.2, 0.0)), dtype=np.float32)
points = np.zeros((2, 2, 3), dtype=np.float32)
points[..., 2] = depth
frame = SimpleNamespace(
depth=depth,
points=points,
intrinsics=np.eye(3, dtype=np.float32),
valid_mask=valid,
model_mask=valid,
)
mesh_job._write_field(frame, output_path, calibration)
with np.load(output_path, allow_pickle=False) as archive:
np.testing.assert_array_equal(archive["valid"], valid) np.testing.assert_array_equal(archive["valid"], valid)
np.testing.assert_array_equal( np.testing.assert_array_equal(archive["calibration_valid"], calibration)
archive["calibration_valid"], self.assertAlmostEqual(
calibration, float(archive["region_transparent_fraction"]),
0.05,
places=6,
) )
def test_mesh_job_uses_upscale_source_only_for_analysis(self): def test_generate_region_data_only_returns_requested_outputs(self):
from PIL import Image job = sys.modules["server.jobs.generate_region_data"]
input_path = self.storage_root / "input.png"
mesh_job = sys.modules["server.jobs.prepare_geometry_data"] input_path.write_bytes(b"region")
display_path = self.storage_root / "upscaled.png"
analysis_path = self.storage_root / "original.png"
Image.new("RGBA", (12, 8), (200, 20, 40, 255)).save(display_path)
Image.new("RGBA", (3, 2), (10, 80, 160, 255)).save(analysis_path)
job_directory = self.storage_root / "mesh-upscale-source-job"
context = SimpleNamespace( context = SimpleNamespace(
directory=job_directory, directory=self.storage_root / "job",
progress=lambda *_args: None, progress=lambda *_args: None,
check_cancelled=lambda: None, check_cancelled=lambda: None,
) )
field_path = job_directory / "field.npz" archive = context.directory / "image_geometry.npz"
normal = context.directory / "normal.png"
with patch.object( with patch.object(
mesh_job, job,
"_write_depth_field", "generate_depth",
return_value=(field_path, None), return_value=(archive, normal),
) as write_surface: ) as generate:
result = mesh_job.run( result = job.run(
context, context,
{ {
"input": str(display_path), "input": str(input_path),
"analysis_input": str(analysis_path), "output_depth": True,
"shape": "DEPTH_SURFACE", "normal_mode": "OBJECT",
"normal_analysis": False,
"video_frames": 1,
}, },
) )
self.assertEqual(result, {"field": "field.npz"}) self.assertEqual(
self.assertEqual(write_surface.call_args.args[2], analysis_path) result,
with Image.open(job_directory / "image.0001.png") as color: {"image_geometry": "image_geometry.npz", "normal": "normal.png"},
self.assertEqual(color.size, (12, 8)) )
generate.assert_called_once_with(
def test_mesh_masked_color_keeps_same_direction_overlap(self): context,
cutout_job = importlib.import_module("server.selection.cutout_images") {
import numpy as np "input": str(input_path),
from PIL import Image "output_depth": True,
"normal_mode": "OBJECT",
mesh_job = sys.modules["server.jobs.prepare_geometry_data"] },
source_path = self.storage_root / "opaque-overlap.png" input_path,
output_path = self.storage_root / "cutout-overlap.png" output_depth=True,
Image.new("RGBA", (12, 9), (20, 40, 60, 255)).save(source_path) normal_mode="OBJECT",
polygon = (
(1, 1),
(7, 1),
(7, 7),
(1, 7),
(1, 1),
(4, 1),
(10, 1),
(10, 7),
(4, 7),
(4, 1),
(1, 1),
) )
cutout_job._write_masked_color(source_path, polygon, output_path) def test_generate_region_data_refines_an_already_rasterized_region(self):
alpha = np.asarray(Image.open(output_path).convert("RGBA"))[:, :, 3] job = sys.modules["server.jobs.generate_region_data"]
input_path = self.storage_root / "input.png"
self.assertGreater(int(alpha[3, 2]), 200) input_path.write_bytes(b"region")
self.assertGreater(int(alpha[3, 5]), 200) directory = self.storage_root / "job"
self.assertGreater(int(alpha[3, 8]), 200) context = SimpleNamespace(
def test_mesh_masked_color_preserves_nonzero_winding_hole(self): directory=directory,
cutout_job = importlib.import_module("server.selection.cutout_images") progress=lambda *_args: None,
import numpy as np check_cancelled=lambda: None,
from PIL import Image
mesh_job = sys.modules["server.jobs.prepare_geometry_data"]
source_path = self.storage_root / "opaque-hole.png"
output_path = self.storage_root / "cutout-hole.png"
Image.new("RGBA", (10, 10), (20, 40, 60, 255)).save(source_path)
polygon = (
(1, 1),
(9, 1),
(9, 9),
(1, 9),
(1, 1),
(3, 3),
(3, 7),
(7, 7),
(7, 3),
(3, 3),
(1, 1),
) )
cutout_job._write_masked_color(source_path, polygon, output_path) def generate(_context, _input, _model, _device, output):
alpha = np.asarray(Image.open(output_path).convert("RGBA"))[:, :, 3] output.write_bytes(b"refined")
return output, (2, 3, 8, 9)
self.assertGreater(int(alpha[2, 2]), 200) with patch.object(job, "generate_selection", side_effect=generate):
self.assertLess(int(alpha[4, 4]), 32) result = job.run(
context,
{
"input": str(input_path),
"refine_region": True,
"refine_model": "BEN2_BASE",
"device": "cpu",
},
)
def test_mesh_masked_color_antialiases_the_cutout_edge(self): self.assertEqual(
cutout_job = importlib.import_module("server.selection.cutout_images") result,
import numpy as np {"region": "region.png", "region_bounds": "region_bounds.json"},
from PIL import Image )
self.assertEqual(
mesh_job = sys.modules["server.jobs.prepare_geometry_data"] (directory / "region_bounds.json").read_text(encoding="utf-8"),
source_path = self.storage_root / "opaque.png" "[2,3,8,9]",
output_path = self.storage_root / "cutout.png"
Image.new("RGBA", (16, 16), (20, 40, 60, 255)).save(source_path)
cutout_job._write_masked_color(
source_path,
((1, 1), (14, 1), (1, 14)),
output_path,
) )
alpha = np.asarray(Image.open(output_path).convert("RGBA"))[:, :, 3]
self.assertTrue(np.any((alpha > 0) & (alpha < 255)))
if __name__ == "__main__":
unittest.main()

7
tests/test_worker.py
View File

@@ -927,6 +927,13 @@ class WorkerTest(unittest.TestCase):
self.assertEqual(result.mode, "RGBA") self.assertEqual(result.mode, "RGBA")
self.assertEqual(result.size, (20, 12)) self.assertEqual(result.size, (20, 12))
alpha = np.asarray(result.getchannel("A")) alpha = np.asarray(result.getchannel("A"))
expected_alpha = np.asarray(
Image.fromarray(pixels[:, :, 3], mode="L").resize(
result.size,
Image.Resampling.LANCZOS,
)
)
np.testing.assert_array_equal(alpha, expected_alpha)
self.assertLess( self.assertLess(
int(alpha[:, 0].max()), int(alpha[:, 0].max()),
int(alpha[:, -1].min()), int(alpha[:, -1].min()),

5
tools/node_assets/build.py
View File

@@ -19,9 +19,6 @@ from build_cutout import (
) )
from build_mesh_plane import build_mesh_plane_group from build_mesh_plane import build_mesh_plane_group
from build_shadeless import build_shadeless_group from build_shadeless import build_shadeless_group
from anyimage.operators.common.node_graph import expand_smooth
OUTPUT_PATH = ( OUTPUT_PATH = (
PROJECT_ROOT PROJECT_ROOT
/ "src" / "src"
@@ -40,7 +37,7 @@ def main():
shape: build_node_group(shape) shape: build_node_group(shape)
for shape in CUTOUT_NODE_GROUP_NAMES for shape in CUTOUT_NODE_GROUP_NAMES
} }
build_cutout_modifier_group(shape_groups, expand_smooth) build_cutout_modifier_group(shape_groups)
OUTPUT_PATH.parent.mkdir(parents=True, exist_ok=True) OUTPUT_PATH.parent.mkdir(parents=True, exist_ok=True)
bpy.ops.wm.save_as_mainfile(filepath=str(OUTPUT_PATH), check_existing=False) bpy.ops.wm.save_as_mainfile(filepath=str(OUTPUT_PATH), check_existing=False)

25
tools/node_assets/build_cutout.py
View File

@@ -4,6 +4,8 @@ import math
import bpy import bpy
from node_graph import expand_smooth
from anyimage.operators.cutout_tool.shape_types import ( from anyimage.operators.cutout_tool.shape_types import (
BALLOON_ATTRIBUTE_NAME, BALLOON_ATTRIBUTE_NAME,
CUTOUT_NODE_GROUP_NAME, CUTOUT_NODE_GROUP_NAME,
@@ -727,7 +729,7 @@ def build_node_group(shape):
options = group.interface.new_panel(name="Options", default_closed=True) options = group.interface.new_panel(name="Options", default_closed=True)
reference_depth = _float_input( reference_depth = _float_input(
group, group,
"Reference Depth", "Base Plane Depth",
1.0, 1.0,
0.0, 0.0,
None, None,
@@ -751,7 +753,7 @@ def build_node_group(shape):
options = group.interface.new_panel(name="Options", default_closed=True) options = group.interface.new_panel(name="Options", default_closed=True)
reference_depth = _float_input( reference_depth = _float_input(
group, group,
"Reference Depth", "Base Plane Depth",
1.0, 1.0,
0.0, 0.0,
None, None,
@@ -788,7 +790,7 @@ def build_node_group(shape):
"Depth Image", "Depth Image",
"Depth Scale", "Depth Scale",
"Uniform Scale", "Uniform Scale",
"Reference Depth", "Base Plane Depth",
"Depth Direction", "Depth Direction",
}, },
) )
@@ -813,7 +815,7 @@ def build_node_group(shape):
controls.outputs["Uniform Scale"], controls.outputs["Uniform Scale"],
controls.outputs["Depth Direction"], controls.outputs["Depth Direction"],
controls.outputs["Depth Image"], controls.outputs["Depth Image"],
controls.outputs["Reference Depth"], controls.outputs["Base Plane Depth"],
) )
output_location = (5100.0, 600.0) output_location = (5100.0, 600.0)
else: else:
@@ -823,7 +825,7 @@ def build_node_group(shape):
controls.outputs["Thickness"], controls.outputs["Thickness"],
controls.outputs["Depth Scale"], controls.outputs["Depth Scale"],
controls.outputs["Uniform Scale"], controls.outputs["Uniform Scale"],
controls.outputs["Reference Depth"], controls.outputs["Base Plane Depth"],
controls.outputs["Depth Image"], controls.outputs["Depth Image"],
) )
output_location = (8200.0, 400.0) output_location = (8200.0, 400.0)
@@ -844,7 +846,7 @@ def build_node_group(shape):
return group return group
def build_cutout_modifier_group(shape_groups, expand_smooth): def build_cutout_modifier_group(shape_groups):
group = bpy.data.node_groups.get(CUTOUT_NODE_GROUP_NAME) group = bpy.data.node_groups.get(CUTOUT_NODE_GROUP_NAME)
if group is not None: if group is not None:
return group return group
@@ -896,16 +898,14 @@ def build_cutout_modifier_group(shape_groups, expand_smooth):
reference_depth = _float_input( reference_depth = _float_input(
group, group,
"Reference Depth", "Base Plane Depth",
1.0, 1.0,
0.0, 0.0,
None, None,
subtype="DISTANCE", subtype="DISTANCE",
parent=options, parent=options,
) )
reference_depth.description = ( reference_depth.description = "Depth of the flat base plane for both Depth shapes"
"Metric reference depth shared by both Depth shapes"
)
depth_direction = _vector_input( depth_direction = _vector_input(
group, group,
"Depth Direction", "Depth Direction",
@@ -936,10 +936,13 @@ def build_cutout_modifier_group(shape_groups, expand_smooth):
(-2700.0, 500.0), (-2700.0, 500.0),
{"Geometry", "Cleanup"}, {"Geometry", "Cleanup"},
) )
cleanup_area = _math(group.nodes, "MULTIPLY", (-2500.0, 750.0))
cleanup_area.inputs[1].default_value = 0.01
group.links.new(base_controls.outputs["Cleanup"], cleanup_area.inputs[0])
geometry = _island_cleanup( geometry = _island_cleanup(
group, group,
base_controls.outputs["Geometry"], base_controls.outputs["Geometry"],
base_controls.outputs["Cleanup"], cleanup_area.outputs[0],
(-2350.0, 250.0), (-2350.0, 250.0),
) )

13
src/anyimage/operators/common/node_graph.py → tools/node_assets/node_graph.py
View File

@@ -1,14 +1,4 @@
"""Shared Geometry Nodes construction helpers.""" """Node graph builders used only while generating bundled assets."""
def interface_socket(group, name, in_out, socket_type):
socket = group.interface.new_socket(
name=name,
in_out=in_out,
socket_type=socket_type,
)
if in_out == "INPUT" and hasattr(socket, "structure_type"):
socket.structure_type = "SINGLE"
return socket
def group_input(nodes, location, visible_names): def group_input(nodes, location, visible_names):
@@ -189,4 +179,3 @@ def expand_smooth(
node.location.x += x_offset node.location.x += x_offset
return repeat_output.outputs["Geometry"] return repeat_output.outputs["Geometry"]

8
tools/node_assets/validate.py
View File

@@ -11,10 +11,8 @@ sys.path.insert(
str(next((PROJECT_ROOT / "src" / "anyimage" / "wheels").glob("blendjob-*.whl"))), str(next((PROJECT_ROOT / "src" / "anyimage" / "wheels").glob("blendjob-*.whl"))),
) )
from anyimage.operators.common.node_asset import ( from anyimage.operators.common.mesh_result import (
load_asset_node_group, load_asset_node_group,
load_mesh_plane_node_group,
load_shadeless_node_group,
) )
from anyimage.operators.cutout_tool.shape_types import ( from anyimage.operators.cutout_tool.shape_types import (
CUTOUT_NODE_GROUP_NAME, CUTOUT_NODE_GROUP_NAME,
@@ -32,8 +30,8 @@ def input_names(group):
def main(): def main():
bpy.ops.wm.read_factory_settings(use_empty=True) bpy.ops.wm.read_factory_settings(use_empty=True)
mesh_plane = load_mesh_plane_node_group() mesh_plane = load_asset_node_group("O Mesh Plane")
shadeless = load_shadeless_node_group() shadeless = load_asset_node_group("O Shadeless")
cutout = load_asset_node_group(CUTOUT_NODE_GROUP_NAME) cutout = load_asset_node_group(CUTOUT_NODE_GROUP_NAME)
cutout_groups = { cutout_groups = {
name: bpy.data.node_groups[name] name: bpy.data.node_groups[name]

23
tools/node_assets/validate_cutout.py
View File

@@ -13,7 +13,7 @@ sys.path.insert(
from anyimage.operators.cutout_tool import node_assets as cutout_nodes from anyimage.operators.cutout_tool import node_assets as cutout_nodes
from anyimage.operators.cutout_tool import shape_types as shapes from anyimage.operators.cutout_tool import shape_types as shapes
from anyimage.operators.common.node_asset import ( from anyimage.operators.common.mesh_result import (
modifier_input_identifier, modifier_input_identifier,
set_modifier_input, set_modifier_input,
) )
@@ -125,7 +125,7 @@ def main():
"Smooth Iterations", "Smooth Iterations",
"Cleanup", "Cleanup",
"Smooth", "Smooth",
"Reference Depth", "Base Plane Depth",
"Depth Direction", "Depth Direction",
"Uniform Scale", "Uniform Scale",
"Depth Image", "Depth Image",
@@ -138,7 +138,7 @@ def main():
"Thickness", "Thickness",
"Depth Scale", "Depth Scale",
"Double Sided", "Double Sided",
"Reference Depth", "Base Plane Depth",
"Depth Direction", "Depth Direction",
"Uniform Scale", "Uniform Scale",
"Depth Image", "Depth Image",
@@ -147,7 +147,7 @@ def main():
"Geometry", "Geometry",
"Thickness", "Thickness",
"Depth Scale", "Depth Scale",
"Reference Depth", "Base Plane Depth",
"Uniform Scale", "Uniform Scale",
"Depth Image", "Depth Image",
], ],
@@ -171,7 +171,7 @@ def main():
"Thickness": set(shapes.CUTOUT_SHAPES), "Thickness": set(shapes.CUTOUT_SHAPES),
"Depth Scale": {"DEPTH_BALLOON", "DEPTH_SURFACE"}, "Depth Scale": {"DEPTH_BALLOON", "DEPTH_SURFACE"},
"Double Sided": {"DEPTH_BALLOON"}, "Double Sided": {"DEPTH_BALLOON"},
"Reference Depth": {"DEPTH_BALLOON", "DEPTH_SURFACE"}, "Base Plane Depth": {"DEPTH_BALLOON", "DEPTH_SURFACE"},
"Depth Direction": {"DEPTH_BALLOON"}, "Depth Direction": {"DEPTH_BALLOON"},
"Uniform Scale": {"DEPTH_BALLOON", "DEPTH_SURFACE"}, "Uniform Scale": {"DEPTH_BALLOON", "DEPTH_SURFACE"},
"Depth Image": {"DEPTH_BALLOON", "DEPTH_SURFACE"}, "Depth Image": {"DEPTH_BALLOON", "DEPTH_SURFACE"},
@@ -230,6 +230,15 @@ def main():
) )
assert cleanup.default_value == 0.0 assert cleanup.default_value == 0.0
assert cleanup.subtype == "NONE" assert cleanup.subtype == "NONE"
cleanup_scale = next(
link.to_node
for link in wrapper.links
if link.from_socket.name == "Cleanup"
and link.from_node.bl_idname == "NodeGroupInput"
)
assert cleanup_scale.bl_idname == "ShaderNodeMath"
assert cleanup_scale.operation == "MULTIPLY"
assert abs(cleanup_scale.inputs[1].default_value - 0.01) < 1e-7
assert smooth.default_value == 1.0 assert smooth.default_value == 1.0
assert iterations.default_value == 0 assert iterations.default_value == 0
assert iterations.min_value == 0 assert iterations.min_value == 0
@@ -341,7 +350,7 @@ def main():
("Depth Image", point_image), ("Depth Image", point_image),
("Depth Scale", 1.0), ("Depth Scale", 1.0),
("Uniform Scale", 1.0), ("Uniform Scale", 1.0),
("Reference Depth", 2.0), ("Base Plane Depth", 2.0),
("Depth Direction", (0.0, 0.0, 1.0)), ("Depth Direction", (0.0, 0.0, 1.0)),
): ):
set_modifier_input( set_modifier_input(
@@ -392,7 +401,7 @@ def main():
("Depth Image", point_image), ("Depth Image", point_image),
("Depth Scale", 1.0), ("Depth Scale", 1.0),
("Uniform Scale", 1.0), ("Uniform Scale", 1.0),
("Reference Depth", 2.0), ("Base Plane Depth", 2.0),
("Smooth Iterations", 0), ("Smooth Iterations", 0),
): ):
set_modifier_input( set_modifier_input(

12
tools/node_assets/validate_mesh_plane.py
View File

@@ -11,8 +11,7 @@ sys.path.insert(
str(next((PROJECT_ROOT / "src" / "anyimage" / "wheels").glob("blendjob-*.whl"))), str(next((PROJECT_ROOT / "src" / "anyimage" / "wheels").glob("blendjob-*.whl"))),
) )
from anyimage.operators.common.node_asset import ( from anyimage.operators.common.mesh_result import (
get_modifier_input,
modifier_input_identifier, modifier_input_identifier,
set_modifier_input, set_modifier_input,
) )
@@ -28,6 +27,13 @@ def evaluated_mesh(obj):
return evaluated, evaluated.to_mesh() return evaluated, evaluated.to_mesh()
def modifier_input_value(modifier, identifier):
inputs = getattr(getattr(modifier, "properties", None), "inputs", None)
if inputs is not None:
return getattr(inputs, identifier).value
return modifier[identifier]
def main(): def main():
bpy.ops.wm.read_factory_settings(use_empty=True) bpy.ops.wm.read_factory_settings(use_empty=True)
mesh = create_mesh_plane_mesh("Mesh Plane", (-1.0, 1.0, -0.5, 0.5)) mesh = create_mesh_plane_mesh("Mesh Plane", (-1.0, 1.0, -0.5, 0.5))
@@ -49,7 +55,7 @@ def main():
assert inputs["Subdivide"].max_value == 10 assert inputs["Subdivide"].max_value == 10
assert inputs["Thickness"].default_value == 0.0 assert inputs["Thickness"].default_value == 0.0
assert inputs["Material"].hide_in_modifier assert inputs["Material"].hide_in_modifier
assert get_modifier_input(modifier, inputs["Material"].identifier) == material assert modifier_input_value(modifier, inputs["Material"].identifier) == material
evaluated, result = evaluated_mesh(obj) evaluated, result = evaluated_mesh(obj)
try: try: