diff --git a/AGENTS.md b/AGENTS.md index 480acf2..0f5d147 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -54,6 +54,7 @@ uv lock --refresh ## 命名规范 - 变量、函数、Docstring 使用英文；同一概念在代码和文档中使用统一表达 +- 功能代码的动作前缀应表达产物边界：`generate_*` 表示 AI 生成，`build_*` 只在内存中构建值或几何，`create_*` 表示创建 Blender Image、Mesh、Material、Object 等真实数据 - 名称应说明业务职责，并与用户概念、公开协议和界面术语保持一致；不要使用脱离上下文后无法判断用途的宽泛名称 - 文件和目录名应反映职责层级：集合层使用复数，单个功能或动作使用单数或动宾结构；同一层级保持一致 - Python 类使用大驼峰，不添加项目名前缀。Operator 使用动宾结构，如 `ConvertToDepthPlane`、`RemoveImageBackground`；Menu、Panel、AddonPreferences、PropertyGroup 分别使用实际类型后缀，如 `AnyImageImageMenu`、`ServerPanel`、`AnyImagePreferences`、`AnyImageSettings` diff --git a/README.md b/README.md index ac29025..bd420aa 100644 --- a/README.md +++ b/README.md @@ -9,14 +9,14 @@ AnyImage 是一款 Blender Extension，用于把图片带入可编辑的 2.5D - 在 3D View、Node Editor 与绘画模式中用 `Ctrl+V`（macOS 为 `Cmd+V`）粘贴并 Pack 剪贴板图片 - 将 Image Empty 转换为带 `O Mesh Plane` Geometry Nodes 的普通 Plane - 使用 MoGe-2 或 DA3 将静态图转换为可调 Depth Plane -- 用 Cutout 套索在图片可见 Alpha 内生成 Surface、Balloon、Depth Balloon 或 Depth Surface，并通过统一的 `O Image Cutout` 修改器切换 Shape、按创建分辨率清理碎岛和调整表面 +- 用 Cutout 套索在图片可见 Alpha 内生成 Surface、Balloon、Depth Balloon 或 Depth Surface，并通过统一的 `O Image Cutout` 修改器切换 Shape、按创建分辨率清理碎岛、根据透明选区校准浮雕基准和调整表面 - 用 Image Box、Lasso、Line、Polyline 和 Perspective 清理或矫正静态图片 - 用 BEN2 为单图、视频或序列帧去除背景 -- 用三档 ONNX 模型将单图、视频或序列帧放大 4× +- 用三档 ONNX 模型将单图、视频或序列帧放大 4×，并独立重采样 Alpha AI 推理运行在 Blender 之外的独立 Python 3.12 Environment 中。Blender 进程只负责 UI、任务提交和结果导入，不加载 ONNX Runtime 或模型权重。 -全局 **Maximum AI Input Size** 默认限制 Upscale、Depth 与 Normal 分析的长边为 2048 px；超过时等比缩放分析输入，但不会降低颜色贴图分辨率。Cutout 在套索裁切后应用该限制，并以同一分析尺度计算网格密度。 +全局 **Maximum AI Input Size** 默认设为 2048 px。当前图片长边达到该值时 Upscale 不可用；Depth 与 Normal 的分析输入超过该值时等比缩小，但不会降低颜色贴图分辨率。Cutout 在套索裁切后应用该限制，并以同一分析尺度计算网格密度。 ## 安装 @@ -28,7 +28,7 @@ AI 推理运行在 Blender 之外的独立 Python 3.12 Environment 中。Blender 3. 选中 Image Empty，在右键 **AnyImage** 菜单中点击 **Set Up AI Environment…**。 4. 安装完成后，在扩展 Preferences 中确认默认 MoGe-2、BEN2 与 Real-ESRGAN WDN 模型显示 Ready。 -普通 Plane、基础 Image 编辑，以及不带 Refine / Normal 的 Cutout Surface 与 Balloon 无需 AI Environment。Cutout 的 Depth 与 Normal 统一使用 MoGe-2；两者同时启用时共享同一次 Prediction，不会重复运行模型。 +普通 Plane、基础 Image 编辑，以及不带 Refine / Normal 的 Cutout Surface 与 Balloon 无需 AI Environment。Cutout 的 Depth 与 Normal 统一使用 MoGe-2；两者同时启用时共享同一次 Prediction，不会重复运行模型。Cutout Normal 可选择 None、基于完整图片平面的 Tangent Space 浮雕，或直接编码到对象局部轴的 Object Space。 ## 快速使用 @@ -37,9 +37,9 @@ AI 推理运行在 Blender 之外的独立 Python 3.12 Environment 中。Blender - **Convert to Plane**：同步创建普通图片网格； - **Convert to Depth Plane**：使用 Preferences 中选择的深度模型处理静态图； - **Remove Background**：生成透明图片或序列； -- **Upscale**：使用默认高清模型生成 4× 结果。 +- **Upscale**：使用默认高清模型把当前图片编辑为 4× 结果。 -3D Viewport 工具栏提供 Image Mask、Perspective 与 Cutout。工作流对比和当前输入范围见[选择工作流](docs/guides/workflows.md)。 +3D Viewport 工具栏提供 Image Mask、Perspective 与 Cutout。Remove Background、Upscale、Image Mask 和 Perspective 都只更新当前 Empty；共享 Image 会保留原数据，并为当前 Empty 创建独立结果。工作流对比和当前输入范围见[选择工作流](docs/guides/workflows.md)。 ## 源码结构 @@ -52,7 +52,7 @@ src/anyimage/ └── __init__.py # Blender 注册入口 ``` -程序边界、功能文件结构和调用流程见[内部实现总览](docs/internals/index.md)。文档按开始使用、功能指南、参考和内部实现分层，不在 README 重复内部算法细节。 +程序边界、功能文件结构和调用流程见[内部实现总览](docs/internals/index.md)；开发新功能前先查看[Common 公共函数](docs/internals/common.md)，复用已经存在的 Image、ImageGeometry、Material 和对象替换入口。文档按开始使用、功能指南、参考和内部实现分层，不在 README 重复内部算法细节。 ## 开发 diff --git a/docs/guides/cutout.md b/docs/guides/cutout.md index e95c43f..6050517 100644 --- a/docs/guides/cutout.md +++ b/docs/guides/cutout.md @@ -21,7 +21,7 @@ Cutout 用套索从一张 Image Empty 中选择区域，再生成 Surface、Ball 四种结果都保留源 Image Empty，并在同一位置创建新网格。它们共享选区轮廓、UV、材质和基础三角网格，因此可以从同一张图多次生成不同 Shape 进行比较。 -无论是否启用 Refine Selection，套索超出图片的部分都不会生成网格，源图中 `Alpha <= 0.1` 的像素也始终从几何中排除。双击选择完整画面时同样只为 `Alpha > 0.1` 的区域生成网格。 +无论是否启用 Refine Selection，套索超出图片的部分都不会生成网格，源图中 `Alpha <= 0.1` 的像素也始终从 Region 中排除。双击使用完整画面时同样只为 `Alpha > 0.1` 的 Region 生成网格。 ## 工具设置 @@ -38,19 +38,19 @@ Point Spacing 先按完整源图换算基础密度；选区裁切后的长边若 四个 Shape 都使用同一个 `O Image Cutout` Geometry Nodes 修改器。创建时 Shape 菜单已经设为饼菜单所选类型；切换 Shape 后，Blender 只显示该类型实际使用的输入。常用参数包括： - **Thickness**：Surface / Depth Surface 使用场景距离；Balloon / Depth Balloon 使用鼓起高度倍率。 -- **Cleanup**：绝对面积阈值；创建时根据 Point Spacing 和图片平面尺寸自动写入，面积小于该值的 Mesh Island 会在生成任意 Shape 前移除。 +- **Cleanup**：碎岛面积阈值；创建时根据 Point Spacing 和图片平面尺寸自动写入。为便于在修改器中调节，显示值放大了 `100` 倍，节点内部会还原为实际面积，因此这次显示缩放不会改变默认清理效果。 - **Smooth Iterations**：位于外层，控制平滑次数，默认 `0`；**Smooth** 位于 Options 中，控制每次平滑的混合强度。四种 Shape 都可使用。 - **Depth Scale**：Depth Shape 的深度强度；`0` 回到基础平面或 Balloon。 -两个 Depth Shape 共用 **Reference Depth**；Depth Balloon 还提供 **Depth Direction**。插件自动写入的 Depth Image 与 Uniform Scale 位于折叠的 Data 面板，一般无需手动修改。 +两个 Depth Shape 共用 **Base Plane Depth**；Depth Balloon 还提供 **Depth Direction**。生成前会检查最终 Region 输入的 Alpha：`Alpha < 0.1` 的像素达到 `5%` 时，根据有效 Region 自适应 Base Plane Depth，并为 Depth Balloon 拟合 Direction；不足 `5%` 时使用稳定的固定基准。这个判断不依赖双击全图或 Refine Selection。插件自动写入的 Depth Image 与 Uniform Scale 位于折叠的 Data 面板，一般无需手动修改。 Balloon 的轮廓高度算法由扩展 Preferences 中的 **Balloon Profile** 控制：Poisson 更平滑，Teddy 更强调轮廓骨架。修改默认值只影响之后创建的 Shape。 ## Refine 与 Normal -Refine Selection 只在“套索与源图可见 Alpha 的交集”内收紧用于建网格的最终选择，不把 AI 生成的颜色写回原图。材质继续使用当前 Image Empty 的源颜色和原 Alpha。 +Refine Selection 只在“套索与源图可见 Alpha 的交集”内收紧最终 Region，不把 AI 生成的颜色写回原图。材质继续使用当前 Image Empty 的源颜色和原 Alpha。 -Normal 关闭时使用几何法线；开启后，Surface、Balloon 与 Depth Balloon 使用适配成品网格的 Tangent Space 法线，Depth Surface 使用 Object Space 法线。Cutout 始终使用 MoGe-2；Depth 与 Normal 同时需要时共享同一次 Prediction，不会重复运行模型。 +Normal 设为 **None** 时使用几何法线；**Tangent** 把 MoGe-2 结果作为基于完整图片平面的浮雕细节，**Object** 则直接编码到 Cutout 对象局部轴。两种模式都不根据 Geometry Nodes 的成品网格再次烘焙。Normal 保持模型分析分辨率，Color Image 仍保留原始裁切分辨率；Depth 与 Normal 同时需要时共享同一次 Prediction，不会重复运行模型。 ## 当前范围 diff --git a/docs/guides/models-and-storage.md b/docs/guides/models-and-storage.md index e927ce3..2360ac7 100644 --- a/docs/guides/models-and-storage.md +++ b/docs/guides/models-and-storage.md @@ -20,7 +20,7 @@ AnyImage 把运行环境、模型和任务结果放在 Blender 之外，多个 B ## 全局 AI 输入尺寸 -Environment 区的 **Maximum AI Input Size** 控制 Upscale、Depth 与 Normal 分析输入的最大长边，默认 `2048 px`，可在 `256–8192 px` 之间调整。超过上限的输入会保持宽高比缩小；颜色贴图仍使用原始分辨率。Cutout 在完成套索裁切后再应用上限，因此整张原图很大但选区较小时不会被无谓缩小。 +Environment 区的 **Maximum AI Input Size** 默认 `2048 px`，可在 `256–8192 px` 之间调整。它既是允许执行 Upscale 的当前图片长边上限，也是 Depth 与 Normal 分析输入的最大长边；Upscale 在达到上限时置灰，Depth 与 Normal 则保持宽高比缩小分析输入。颜色贴图仍使用当前图片的原始分辨率。Cutout 在完成套索裁切后再应用上限，因此整张图片很大但选区较小时不会被无谓缩小。 ## Device 扩展偏好设置中的 **Device** 控制 Depth Plane、Cutout 的 AI 增强、Remove Background 和 Upscale 等正式任务。 diff --git a/docs/guides/upscale.md b/docs/guides/upscale.md index d058951..eb75cb0 100644 --- a/docs/guides/upscale.md +++ b/docs/guides/upscale.md @@ -1,6 +1,6 @@ # 高清放大 -Upscale 用 ONNX 模型把 Image Empty 的图片放大 4 倍，不创建网格。它支持单图、Movie 和 Image Sequence，并保留源 Alpha。 +Upscale 用 ONNX 模型把 Image Empty 的 RGB 放大 4 倍，不创建网格。它支持单图、Movie 和 Image Sequence，并以 Lanczos 独立放大源 Alpha 后重新合并。 ## 使用 @@ -8,7 +8,7 @@ Upscale 用 ONNX 模型把 Image Empty 的图片放大 4 倍，不创建网格 2. 按需要设置 Environment 区的 **Maximum AI Input Size**；默认 `2048 px`，最高 `8192 px`。 3. 选中 Image Empty，在右键 **AnyImage** 菜单中点击 **Upscale**。 -输入长边超过全局上限时，会先保持宽高比缩小，再执行 4× 放大。因此默认设置下结果长边最多为 `8192 px`。完成后，当前 Empty 会改用名称以 `_upscale.png` 结尾的结果图片。 +菜单项只显示本次操作预计生成的 4× 尺寸。当前图片长边必须小于全局上限；达到上限后 Upscale 会置灰，因此默认设置下单次结果长边小于 `8192 px`。完成后，当前 Empty 会直接使用放大结果；原 Image 没有其他引用时沿用原名并移除旧数据，存在其他引用时则为当前 Empty 保留独立结果。 ## 模型选择 @@ -18,10 +18,10 @@ Upscale 用 ONNX 模型把 Image Empty 的图片放大 4 倍，不创建网格 后两项可在扩展偏好设置中按需下载。 -对高清结果再次执行 Upscale 时，AnyImage 会从最初的原图重新生成并替换当前结果，不会连续放大已有高清图。 +AnyImage 不追踪高清结果的原图引用，Upscale 始终处理当前 Image。只要当前长边仍小于全局上限，就可以再次放大；达到上限后不能继续执行。 -把高清结果继续转换为 Depth Plane 时，颜色材质可以继续使用当前高清图，分析输入则遵守全局上限；Cutout 同样只缩放裁切后的分析输入，不会把限制后的临时图用作 Color。 +把高清结果继续转换为 Depth Plane 时，颜色材质和分析都以当前 Image 为来源，分析输入仍遵守全局上限；Cutout 同样只缩放裁切后的分析输入，不会把限制后的临时图用作 Color。 如果显存或内存不足，请降低 **Maximum AI Input Size**，或改用 CPU。 -分块推理、Alpha 合并与原图引用见 [Upscale 实现原理](../internals/operators/upscale.md)。 +分块推理、Alpha 合并与输入限制见 [Upscale 实现原理](../internals/operators/upscale.md)。 diff --git a/docs/internals/architecture.md b/docs/internals/architecture.md index 453e4f6..747b7b1 100644 --- a/docs/internals/architecture.md +++ b/docs/internals/architecture.md @@ -37,7 +37,7 @@ flowchart TD Menu --> Runtime["注册 JobRuntime"] ``` -`__init__.py` 按依赖顺序注册，并在注销时逆序清理。扩展级设置统一由 `preferences.addon_preferences()` 读取；`runtime.py` 只消费 `configured_storage_root` 回调，不持有 Blender 偏好设置的定位逻辑。交互中的临时状态和 Debug 参数保存在 `Scene.anyimage_settings`，Upscale 的原图引用保存在 `Image.anyimage_upscale_source`。 全局 `max_ai_input_size` 也在 `preferences.py` 定义与读取，由 Operator 冻结到每次 Job 请求。 +`__init__.py` 按依赖顺序注册，并在注销时逆序清理。扩展级设置统一由 `preferences.addon_preferences()` 读取；`runtime.py` 只消费 `configured_storage_root` 回调，不持有 Blender 偏好设置的定位逻辑。交互中的临时状态和 Debug 参数保存在 `Scene.anyimage_settings`。全局 `max_ai_input_size` 在 `preferences.py` 定义与读取，由 Operator 冻结到每次 Job 请求；Upscale 还用它判断当前 Image 是否允许执行。 ## 同步与异步路径 @@ -60,6 +60,10 @@ sequenceDiagram 这条边界也决定了模块依赖方向：`server/` 不导入 `bpy` 或 `operators/`；Blender 侧可以读取 Server 产物，但不会复用 Server 中的 Blender 无关算法来创建 RNA 数据。 -## 共享能力 +## 共享能力 { #shared-capabilities } -`operators/common/` 保存多个功能真实复用的能力：图片输入与命名、像素写回、手势状态、Mask 运算、Image Empty 坐标、材质、节点资产、法线烘焙和 Job 参数辅助。共享模块只承载跨功能概念；完整工作流仍留在各自功能包中，以便从用户入口追踪到实现。 +`operators/common/` 只保存多个生产功能真实复用的稳定原子能力；完整工作流、功能参数和专用算法留在所属模块。公共层不保留旧路径转发函数，也不为了测试或资产工具建立入口。 + +全部可由功能模块调用的函数见[Common 公共函数](common.md)。该索引只列 `common/` 外部的生产调用，不重复列出公共模块内部辅助函数；创建新功能前先核对该页，避免重新实现 Image 引用分流、像素方向、Job 输入生命周期、数据块命名、节点资产加载和对象激活逻辑。 + +新增公共函数前应确认至少有两个真实消费者，或它是两个公共流程共同依赖的原子能力。只有单一功能使用的逻辑应留在所属模块；删除或移动能力时同步清理注册、测试、工具脚本和文档，不保留兼容胶水。 diff --git a/docs/internals/common.md b/docs/internals/common.md new file mode 100644 index 0000000..3a0217d --- /dev/null +++ b/docs/internals/common.md @@ -0,0 +1,102 @@ +# Common 公共函数 + +`src/anyimage/operators/common/` 集中已经被多个功能复用的 Blender 侧原子能力，目的是让开发新功能时能够先查找并复用现有实现，避免重复造轮子。 + +本页只列功能模块直接调用的公共函数，不列 `common/` 内部辅助函数、测试或资产工具专用函数，也不列公共类型。 + +## Image 数据块 + +`image_data.py` 处理 Blender `Image` 数据块及其在 Image Empty 上的使用方式，包括像素读取、动画帧、Job 输入、结果 Image 创建和 Empty 引用替换。它不处理 Viewport 手势或选区算法。 + +| 函数 | 职责 | +| --- | --- | +| `image_data.is_image_empty()` | 判断对象是否为带有效 Image 数据的 Empty。 | +| `image_data.require_image_empty()` | 按名称取得 Image Empty；对象已失效时抛错。 | +| `image_data.image_pixels()` | 以 Blender 原始顺序读取一维 float32 像素。 | +| `image_data.image_rgba()` | 读取为 Top-down RGBA float32 数组。 | +| `image_data.video_total_frames()` | 返回 Movie 或 Sequence 总帧数，静态图返回 1。 | +| `image_data.image_user_settings()` | 读取 Image Empty 的帧起点、偏移、时长和循环设置。 | +| `image_data.default_extract_frames()` | 取得当前 Empty 默认应处理的帧数。 | +| `image_data.source_start_frame()` | 从源 Image Empty 取得 Job 应处理的起始帧。 | +| `image_data.prepare_image_input()` | 将 Packed、Dirty、Sequence 等输入准备为 Job 可读路径，并返回临时标记。 | +| `image_data.prepare_region_input()` | 将 `RegionMask` 裁切为带 Mask Alpha 的临时 RGBA Job 输入。 | +| `image_data.cleanup_image_input()` | 按临时标记清理由 `prepare_image_input()` 创建的文件或目录。 | +| `image_data.image_base_name()` | 去除图片或视频扩展名和 Blender 数字后缀，得到数据块基础名。 | +| `image_data.depth_data_name()` | 生成 Depth 结果数据块文件名。 | +| `image_data.color_data_name()` | 生成 Color 结果数据块文件名。 | +| `image_data.load_color_result_image()` | 加载、命名、Pack 并配置 Job 的 Color 或 Sequence 结果；不修改 Empty。 | +| `image_data.load_normal_result_image()` | 加载、命名、设为 Non-Color 并 Pack Normal 结果。 | +| `image_data.create_pixel_result_image()` | 从本地编辑后的 RGBA 像素创建 Packed Image；不修改 Empty。 | +| `image_data.replace_empty_image()` | 执行 Image Empty 结果替换、共享引用分流、旧数据清理、帧设置继承和可选画布对齐。 | +| `image_data.image_empty_bounds()` | 计算 Image Empty 在对象局部坐标中的实际显示边界。 | + +`load_*_result_image()` 和 `create_pixel_result_image()` 只创建结果 Image；`replace_empty_image()` 负责把结果提交给当前 Empty。两者分别表达“创建产物”和“替换引用”。 + +## Image Region 与视图交互 + +这组公共能力使用固定术语： + +- Region 是用户在图片空间圈定的业务范围；`RegionPath` 保存路径点、反选状态和填充规则。 +- `RegionMask` 是 Region 的栅格化结果，只包含局部像素权重 `values` 和源图 `bounds`。 +- Mask 只表示二维像素权重或 Alpha 运算，不作为手势、协议或业务范围的别名。 +- Selection 只用于 Blender 对象选择和界面动作 Refine Selection，不表示中间数据。 + +`image_interaction.py` 处理 3D Viewport 中的点击、坐标投影、`ImageRegionGesture`、工具设置和预览绘制；`image_region.py` 负责图片空间 Region 的值、栅格化和 Alpha 裁切。前者负责用户如何圈定 Region，后者负责 Region 如何变成 Mask。 + +### Viewport 交互 + +| 函数 | 职责 | +| --- | --- | +| `image_interaction.report_image_edit_exception()` | 将可预期的图片编辑警告和异常映射为 Operator Report。 | +| `image_interaction.image_edit_poll()` | 判断当前上下文是否允许启动 Image 编辑工具。 | +| `image_interaction.resolve_image_edit_click()` | 解析点击到 Image Empty，并保持或恢复选择状态。 | +| `image_interaction.active_view3d_tool_id()` | 取得当前 View3D Workspace Tool ID。 | +| `image_interaction.apply_region_settings()` | 将当前 Region 工具的 Refine 设置同步到手势 Operator。 | +| `image_interaction.draw_region_settings()` | 绘制 Region 工具的公共 Refine 设置。 | +| `image_interaction.draw_dashed_line()` | 绘制手势或透视框虚线。 | +| `image_interaction.draw_polygon_fill()` | 绘制手势或透视框填充预览。 | +| `image_interaction.gesture_property()` | 声明统一的 Box、Lasso、Line 和 Polyline RNA 属性。 | +| `image_interaction.screen_path_to_image_pixels()` | 将屏幕路径投影为 Image Empty 像素坐标。 | +| `image_interaction.serialize_matrix()` | 将 4×4 Matrix 序列化为 Operator 可保存的字符串。 | +| `image_interaction.deserialize_matrix()` | 恢复已序列化的 4×4 Matrix。 | +| `image_interaction.drawing_in_region()` | 判断 Modal 绘制是否仍位于启动时的 Area 和 Region。 | + +### Region 栅格化 + +| 函数 | 职责 | +| --- | --- | +| `image_region.region_path_bounds()` | 校验 Region Path 并返回整数像素包围框。 | +| `image_region.rasterize_region_path()` | 把 `RegionPath` 栅格化为 `RegionMask`，并可与源 Alpha 相交。 | +| `image_region.crop_pixels_to_alpha()` | 按 Alpha 非零区域紧裁切 RGBA 像素。 | + +## Mesh 结果构建 + +`mesh_result.py` 集中 Convert to Plane 与 Cutout 共同使用的结果构建能力：读取图片几何产物、创建材质、加载节点资产、设置修改器输入，并完成结果对象替换。 + +`ImageGeometry` 表示按图片像素组织的三维点、相机内参、有效区域、参考深度和选区统计；`load_image_geometry()` 从 `image_geometry.npz` 读取并校验这组数据。 + +| 函数 | 职责 | +| --- | --- | +| `mesh_result.load_image_geometry()` | 读取并校验 `image_geometry.npz`，返回用于构建 Mesh 的 `ImageGeometry`。 | +| `mesh_result.create_image_material()` | 从 Image Empty 创建包含颜色、位移、Normal、Alpha 和动画设置的完整材质。 | +| `mesh_result.load_asset_node_group()` | 从随扩展分发的 Blend 资产加载指定 Node Group。 | +| `mesh_result.modifier_input_identifier()` | 按接口名称解析 Geometry Nodes Modifier 输入 Identifier。 | +| `mesh_result.set_modifier_input()` | 跨受支持 Blender RNA 形式设置 Modifier 输入。 | +| `mesh_result.finalize_object_result()` | 激活结果对象，并按需替换或保留源 Empty。 | + +节点资产构建和验证专用函数位于 `tools/node_assets/`，不属于运行时公共能力。 + +## AI 任务准备 + +`ai_job.py` 处理多个 AI 功能共同使用的任务准备，包括输入路径校验、Environment 与模型检查、Device 选择，以及 DA3 和 MoGe-2 请求参数组装。它不包含业务 Operator、结果导入或 Undo 操作。 + +| 函数 | 职责 | +| --- | --- | +| `ai_job.require_input_path()` | 解析 Blender 路径并要求其为现存文件或目录。 | +| `ai_job.require_background_input_path()` | 在通用路径校验之外限制 BEN2 支持的媒体类型。 | +| `ai_job.production_device()` | 读取正式任务使用的计算设备。 | +| `ai_job.require_environment()` | 要求 AnyImage AI Environment 已就绪。 | +| `ai_job.invoke_ai_setup_if_needed()` | Environment 未就绪时启动安装交互。 | +| `ai_job.require_model()` | 校验模型标识和下载状态，并返回模型记录。 | +| `ai_job.da3_parameters()` | 组装 DA3 正式任务与 Debug 共用参数。 | +| `ai_job.moge2_parameters()` | 组装 MoGe-2 正式任务与 Debug 共用参数。 | diff --git a/docs/internals/index.md b/docs/internals/index.md index a8633fd..20b592f 100644 --- a/docs/internals/index.md +++ b/docs/internals/index.md @@ -39,10 +39,11 @@ flowchart TD ## 文档分层 - [程序架构](architecture.md)：注册、配置、线程和模块边界。 +- [Common 公共函数](common.md)：仅列生产功能从 `operators/common/` 外部调用的函数。 - [Job Runtime 与 Server](runtime.md)：任务生命周期、媒体输入和模型 Session。 - [Operator 实现索引](operators/index.md)：按 `src/anyimage/operators/` 中的完整功能拆分实现页。 - [Job 产物](outputs.md)：文件用途和 Blender 导入关系。 - [开发与发布](development.md)：本地环境、测试、打包和 CI。 - [节点资产](node-assets.md)：Geometry Nodes 的构建、加载与验证。 -内部实现文档描述模块之间的职责和主要数据流。函数级分支、参数校验与异常处理以源码和测试为准，避免文档重复实现细节。 +内部实现文档描述模块之间的职责和主要数据流。[Common 公共函数](common.md)按真实外部调用列出开发新功能时应优先复用的稳定入口；函数级分支、参数校验与异常处理仍以源码和测试为准。 diff --git a/docs/internals/node-assets.md b/docs/internals/node-assets.md index 3bab46b..8b20663 100644 --- a/docs/internals/node-assets.md +++ b/docs/internals/node-assets.md @@ -7,12 +7,12 @@ AnyImage 将稳定、可复用的 Geometry Nodes 和 Shader 节点组构建到 ` ```text src/anyimage/ ├── assets/O_AnyImage.blend -└── operators/common/ - ├── node_asset.py # 资产路径、加载与 Modifier 输入 - └── node_graph.py # 共享节点图构造函数 +├── operators/common/mesh_result.py # 运行时资产加载与 Modifier 输入 +└── operators/convert_to_plane/nodes.py # Depth Plane 运行时节点图 tools/node_assets/ ├── build.py # 统一构建入口 +├── node_graph.py # Cutout 资产构建专用节点图函数 ├── build_mesh_plane.py ├── build_shadeless.py ├── build_cutout.py @@ -27,7 +27,7 @@ tools/node_assets/ flowchart TD Scripts["tools/node_assets/build_*.py"] --> Build["Blender 后台构建"] Build --> Blend["assets/O_AnyImage.blend"] - Blend --> Loader["operators/common/node_asset.py"] + Blend --> Loader["operators/common/mesh_result.py"] Loader --> Modifier["运行时 Node Group / Modifier"] ``` diff --git a/docs/internals/operators/convert-to-plane.md b/docs/internals/operators/convert-to-plane.md index 3c500d5..14f2f81 100644 --- a/docs/internals/operators/convert-to-plane.md +++ b/docs/internals/operators/convert-to-plane.md @@ -1,6 +1,6 @@ # Plane 转换 -Plane 转换包含两条路径：普通 Plane 在 Blender 内同步完成；Depth Plane 先通过 `prepare-geometry-data` Job 取得深度场，再回到 Blender 构建面片。两者共享图片命名、材质与基础四边形能力，但使用不同的 Geometry Nodes。 +Plane 转换包含两条路径：普通 Plane 在 Blender 内同步完成；Depth Plane 先通过 `generate-depth-plane` Job 取得深度场，再回到 Blender 构建面片。两者共享图片命名、材质与基础四边形能力，但使用不同的 Geometry Nodes。 ## 对应文件 @@ -13,7 +13,8 @@ operators/convert_to_plane/ └── nodes.py # 运行时 O Image Depth Plane 节点图 server/ -├── jobs/prepare_geometry_data.py +├── jobs/generate_depth_plane.py +├── geometry/generate_depth.py ├── models/da3.py ├── models/moge2.py └── geometry/projection.py @@ -27,8 +28,8 @@ flowchart TD Kind -- "Plane" --> Copy["复制 Image 与材质"] Copy --> Quad["创建四边形 + O Mesh Plane"] Kind -- "Depth Plane" --> Input["准备颜色与分析输入"] - Input --> Job["prepare-geometry-data"] - Job --> Field["image.0001 + field.npz"] + Input --> Job["generate-depth-plane"] + Job --> Field["image.0001 + image_geometry.npz"] Field --> Anchor["按 Empty 范围锚定相机点"] Anchor --> Depth["创建深度 Image + 材质 + 节点修改器"] Quad --> Replace["替换源 Empty"] @@ -45,7 +46,7 @@ flowchart TD `ConvertToDepthPlane` 当前只接受静态图片。深度模型和分辨率来自扩展 Preferences：MoGe-2 使用 Resolution Level，DA3 使用按原图尺寸换算并对齐到模型 Patch 的处理分辨率。 -`GenerateDepthPlane` 把颜色输入、分析输入和模型参数提交给 `prepare-geometry-data`。Server 将模型结果统一为 `GeometryFrame`，写出包含相机点、深度、内参、有效区域和参考深度的 `field.npz`。这一步不创建网格。 `max_input_size` 同时传入 Job；Server 通过 `media/resolution.py` 在模型推理前限制分析图长边，默认 `2048`。颜色输入仍由未缩放的原图复制，受限分析图不会成为材质 Color。 +`GenerateDepthPlane` 把当前 Image 随模型参数提交给 `generate-depth-plane`。`jobs/generate_depth_plane.py` 负责单图协议，`geometry/generate_depth.py` 负责 AI 深度生成；Server 将模型结果统一为 `GeometryFrame`，写出包含相机点、深度、内参、有效区域和参考深度的 `image_geometry.npz`。Blender 侧通过公共 `load_image_geometry()` 读取为 `ImageGeometry`，与 Cutout 使用同一协议校验。这一步不创建网格。`max_input_size` 同时传入 Job；Server 通过 `media/resolution.py` 在模型推理前限制分析图长边，默认 `2048`。颜色输入仍按当前 Image 的未缩放分辨率复制，受限分析图不会成为材质 Color。 Blender 响应阶段执行三件事： @@ -56,3 +57,5 @@ Blender 响应阶段执行三件事： Depth 节点组先复用 `O Mesh Plane` 完成细分与材质，再生成背面厚度、限制正面向内穿透，最后在 Thickness 为零时删除背面和侧面。`Depth Amount` 控制位移强度；`Depth Image` 与 `Material` 是由插件填入的内部输入。 与 Cutout 的 Depth Surface 不同，Depth Plane 只使用锚定后的 Z 高度，不在 Geometry Nodes 中重建完整相机空间 XYZ。它强调一张图上的可编辑浮雕，而不是局部透视表面。 + +颜色材质、Normal 导入和结果对象激活分别复用 `create_image_material()`、`load_normal_result_image()` 与 `finalize_object_result()`；普通 Plane 和 Depth Plane 不再各自维护对象替换步骤。 diff --git a/docs/internals/operators/cutout-tool.md b/docs/internals/operators/cutout-tool.md index ab6687b..053b7db 100644 --- a/docs/internals/operators/cutout-tool.md +++ b/docs/internals/operators/cutout-tool.md @@ -1,80 +1,76 @@ -# Cutout +# Cutout Tool -Cutout 从一条图片空间套索生成规则三角基础面，再交给统一的 `O Image Cutout` Geometry Nodes 修改器形成 Surface、Balloon、Depth Balloon 或 Depth Surface。公开节点组用 Shape 菜单路由到四个原有内部节点组；轮廓和 Mesh 始终由 Blender 主线程创建，Server 只在需要时准备 Refine、深度或法线数据。 +Cutout 由两个核心 Operator 组成： -## 对应文件 +- `SelectCutoutRegion`（`select_region.py`）把 Viewport 手势转换为图片空间 Region，并弹出 Shape 菜单。 +- `CutoutRegionToShape`（`region_to_shape.py`）准备所需数据，再构建并创建 Shape。 -```text -operators/cutout_tool/ -├── selection_tool.py # 套索交互、设置与 Shape 饼菜单 -├── shape_conversion.py # 本地 / Job 路径选择和任务参数 -├── build_shapes.py # 裁切贴图、Mesh、材质与修改器 -├── shape_types.py # Shape 与节点组映射 -├── mask_regions.py # Alpha 外轮廓与孔洞 -├── planar_mesh.py # 规则采样与约束三角化 -├── balloon_profile.py # Poisson / Teddy profile -├── depth_calibration.py # Reference 与 Direction -└── node_assets.py # Cutout 节点资产加载 +路径约定：Blender 侧文件相对 `src/anyimage/operators/cutout_tool/`，Server 侧文件相对 `src/anyimage/server/`。 -server/ -├── jobs/prepare_geometry_data.py # Cutout / Depth Plane 数据编排 -├── selection/ -│ ├── polygon_mask.py # Polygon 栅格化与选区裁切 -│ └── cutout_images.py # Refine 与 Cutout 局部图片 -└── geometry/normal_texture.py # 相机轴法线编码 -``` - -## 路径选择 +## 执行路径 ```mermaid flowchart TD - Lasso["套索 + Shape"] --> Refine{"Refine Selection？"} - Refine -- "是" --> Ben2["BEN2 × 1"] - Refine -- "否" --> Selection["套索 Selection"] - Ben2 --> Selection - Selection --> NeedMoge{"需要 Depth 或 Normal？"} - NeedMoge -- "否" --> Mesh["轮廓 → 规则点 → 约束三角化"] - NeedMoge -- "是" --> Moge["MoGe-2 × 1\\n同一 Prediction"] - Moge --> Data["Depth Field / Camera Normal"] - Data --> Mesh - Mesh --> Profile["计算 o_balloon"] - Profile --> Nodes["加载 O Image Cutout 并设置 Shape"] - Nodes --> Bake["按需烘焙最终 Tangent Normal"] + Select["圈定 Cutout Region
选择 Shape"] + Raster["栅格化 Region"] + Route{"需要 AI 数据？"} + Local["使用本地 Region 数据"] + AI["生成 Refine / Depth / Normal"] + Ready["汇总 Shape 输入"] + Build["构建基础 Shape"] + Create["创建 Blender 对象"] + + Select --> Raster + Raster --> Route + Route -- "否" --> Local --> Ready + Route -- "是" --> AI --> Ready + Ready --> Build --> Create ``` -Cutout 只接受静态图片。Refine Selection 是网格生成前唯一可选的选区分析：开启时执行一次 BEN2，关闭时直接使用套索 Selection。Surface 与 Balloon 在不需要 Normal 时不运行 MoGe-2；两个 Depth Shape 和任意开启 Normal 的 Shape 都统一使用 MoGe-2。Depth 与 Normal 同时启用时只执行一次推理，同一个 Prediction 同时提供相机点、深度和图片空间法线。 +## 节点说明 -Cutout 不使用 DA3。若全局 Depth Model 当前选择 DA3，Cutout 会回退到默认 MoGe-2；选择 MoGe-2 档位时则沿用该档位和 Resolution Level。DA3 仍可用于独立的 Depth Plane 流程。 +### 圈定 Cutout Region -## 选择与局部数据 +`SelectCutoutRegion` 将 Lasso 路径投射到图片像素坐标，形成 `RegionPath`，然后把 Region 和用户选择的 Shape 交给 `CutoutRegionToShape`。双击图片时，Region 使用完整图片范围。 -Cutout 复用 `ImageMaskGesture` 的采点和屏幕预览，并复用 `operators/common/image_mask.py` 的可见选区语义：最终几何范围始终是套索、图片边界与源图 `Alpha > 0.1` 区域的交集。套索采用 NONZERO 填充规则，使反向内环可以表达孔洞。`Point Spacing` 先根据完整源图最长边换算为基础 `mesh_detail`；获得裁切 Selection 后，再按 `min(1, max_input_size / crop_long_edge)` 缩放有效 `mesh_detail`。因此只有裁切图超过全局上限时网格才会变稀，Color Image 仍使用原始裁切尺寸。 +`RegionPath` 定义在 `operators/common/image_region.py`，包含图片空间路径点、反选状态和填充规则。它既供 Cutout 使用，也供 Image Tool 使用。 -Job 路径将数据拆成三层：源颜色、最终选择和只供模型分析的 RGBA。写出最终选择前会再次以 `Alpha > 0.1` 与源图求交，因此未 Refine、Refine 和双击全图遵循相同边界。`bounds.json` 把可见选区的局部矩形映射回源图，Blender 据此恢复局部平面范围和 UV。未走 Job 时，`build_shapes.py` 在内存中完成相同的求交、裁切与坐标换算。 +### 栅格化 Region -## 基础网格 +`CutoutRegionToShape` 把 `RegionPath` 栅格化为 `RegionMask`。`RegionMask` 只包含局部像素权重 `values` 和它在源图中的 `bounds`，后续几何、颜色裁切和 AI 输入都以它为范围。 -最终 Alpha 先转为带包含关系的外轮廓与孔洞。`planar_mesh.py` 在内部生成近似六角排列的规则点，并用轮廓作为约束进行 Delaunay 三角化。边界按目标间距重采样，短轮廓会提高边界采样密度，内部点仍保持用户设置的间距。 +Surface 或 Balloon 在未启用 Refine 与 Normal 时直接使用本地 Region 数据；选择 Depth Shape，或启用 Refine、Normal 时进入 AI 路径。两条路径最终都调用 `region_to_shape()`。 -四种 Shape 共享同一张单面基础网格、UV 和 `o_balloon` 点属性。创建阶段以目标 Point Spacing 的平方作为一个采样单元面积，再除以 Image Empty 图片平面从局部到世界的面积缩放，得到修改器局部坐标中的 Cleanup 阈值。这个值只取决于工具分辨率和图片尺寸/变换，不受碎片数量或碎面面积分布影响。Balloon Profile 由 Preferences 选择 Poisson 或 Teddy；节点组只消费计算后的属性，不再重复推导轮廓。 +### 生成 AI 数据 -## 统一修改器与四种 Shape +`generate-region-data` Job（`jobs/generate_region_data.py`）根据请求生成所需数据： -创建结果时始终添加 `O Image Cutout`，并按饼菜单选择写入 Shape。节点组先统一执行 Cleanup，再由 Menu Switch 仅求值当前 Shape 的内部节点组，最后统一执行 Smooth；因此四种 Shape 都可使用 Cleanup、Smooth 和 Smooth Iterations。各分支独有输入直接连接到对应内部组，Blender 4.5 会在修改器中隐藏当前 Shape 不使用的输入。Cleanup 直接比较 Mesh Island 总面积与创建时写入的绝对面积阈值，不再统计当前网格面的中位面积。 +- `generate_selection()`（`region/generate_selection.py`）使用 BEN2 收紧 Region； +- `generate_depth(..., normal_mode=...)`（`geometry/generate_depth.py`）使用 MoGe-2 生成 Depth，并可从同一次 Prediction 派生 Tangent 或 Object Normal。 -| Shape | 内部节点组 | 主要几何来源 | +Blender 传入的已是带 Region Alpha 的局部 RGBA。Depth 与 Normal 同时启用时只运行一次 MoGe-2，颜色贴图仍使用 Region 的原始裁切分辨率。 + +### 构建基础 Shape + +`region_to_shape()` 汇总 Region、Depth 和 Normal 输入，再依次执行几何构建与 Blender 数据创建。 + +`build_base_shape()` 从 `RegionMask` 构建纯内存 `BaseShape`：先由 `region_contours.py` 提取包含外轮廓与孔洞的 `RegionContours`，再由 `triangulation.py` 按 Point Spacing 采样并约束三角化，最后使用 `balloon.py` 计算 `o_balloon`。`depth_calibration.py` 负责把 AI Depth 标定为 Blender 中使用的深度基准与方向。四种 Shape 共用这张单面基础网格。 + +### 创建 Blender 对象 + +`create_shape_object()` 创建裁切后的 Color Image、Material、Mesh Object 和统一的 `O Image Cutout` Modifier。Shape 的差异由 Modifier 中对应的几何分支表达： + +| Shape | 形成方式 | Shape 自身所需 AI 数据 | | --- | --- | --- | -| Surface | `O Image Surface` | 基础薄片与可调 Thickness | -| Balloon | `O Image Balloon` | `o_balloon` 鼓起高度 | -| Depth Balloon | `O Image Depth Balloon` | Balloon profile + 相对基准面的非负深度高度 | -| Depth Surface | `O Image Depth Surface` | MoGe-2 相机点的完整投影 | +| Surface | 保持基础 Shape 的平面轮廓，并可增加真实厚度 | 无 | +| Balloon | 使用 `o_balloon` 从轮廓向内部平滑鼓起 | 无 | +| Depth Balloon | 在 Balloon 上叠加相对基准面的图片深度，并沿 Depth Direction 形成浮雕 | MoGe-2 Depth | +| Depth Surface | 使用完整相机点 XYZ 重建带透视位移的表面，并可增加厚度 | MoGe-2 Camera Points | -四个内部节点组不再标记为 Modifier，也不再各自执行 Cleanup。原本只在 `O Image Depth Surface` 内的 Smooth 后处理已移到公开节点组；统一节点组的 Smooth 默认值为 `1`；外层 `Smooth Iterations` 默认值为 `0`，只有用户增加迭代次数后才执行平滑。两个 Depth 内部节点组也统一使用 `Reference Depth`，总节点组只暴露一个共用输入。 +四种 Shape 共用 Region、基础三角面、UV、材质、Cleanup 和 Smooth。Cleanup 的 UI 数值放大 `100` 倍，节点内部乘以 `0.01` 还原，因此显示尺度不会改变实际清理阈值。 -两个 Depth 形状把相机点 RGB 写入 Packed 浮点 `_depth.exr`，并以隐藏的 `Uniform Scale` 匹配 Image Empty 尺寸。未启用 Refine 时 Reference 使用固定值，Depth Balloon Direction 使用默认轴向；启用 Refine 后，`calibration_valid` 限制在模型有效区域与最终选择的交集，Reference 使用其深度第 95 百分位，Depth Balloon 还拟合局部基准方向。 +### Depth 与 Normal -Depth Balloon 保留图片平面 XY，只使用相机 Z 形成向外高度；Depth Surface 将完整 XYZ 映射到 Cutout 局部轴，因此能表达透视位移。两者都只采样深度 Image 的 RGB，Alpha 仅保存有效性和深度跳变信息。 +Depth Shape 根据最终 Region Alpha 选择基准：透明像素比例达到 `5%` 时从有效内容自适应计算 Base Plane Depth；不足 `5%` 时使用固定基准。Depth Balloon 还会在自适应路径中拟合 Depth Direction。 -## 法线空间 - -MoGe-2 在 Job 阶段输出图片空间的相机轴法线预测；这一步不依赖 Blender 网格。Depth Surface 在导入时将它重排到 Cutout 对象轴并以 Object Space 使用；Surface、Balloon 与 Depth Balloon 则先创建基础网格并完成 Geometry Nodes 求值，再根据成品网格 UV、法线和切线基底烘成 Tangent Space。也就是说，网格前产生的是模型 Prediction，最终法线贴图的空间转换仍发生在网格生成之后。 +Normal 是材质层的浮雕细节，与四种 Shape 都可组合。None 使用几何法线；Tangent 以完整图片平面为切线基准；Object 直接编码到 Cutout 对象局部轴。Geometry Nodes 负责大轮廓和深度，Normal 负责表面细节。 diff --git a/docs/internals/operators/image-tool.md b/docs/internals/operators/image-tool.md index e765665..dc662d1 100644 --- a/docs/internals/operators/image-tool.md +++ b/docs/internals/operators/image-tool.md @@ -1,6 +1,6 @@ # Image Tool -`operators/image_tool/` 对应工具栏中的 Box、Lasso、Line 与 Polyline。四个 WorkspaceTool 共用一个 `EditImageMask` Operator，把屏幕手势转换为图片 Alpha；只有 Box 与 Lasso 可以用 BEN2 细化选择。 +`operators/image_tool/` 对应工具栏中的 Box、Lasso、Line 与 Polyline。四个 WorkspaceTool 共用一个 `EditImageMask` Operator：手势先形成图片空间 Region，Region 栅格化为 Mask 后才修改图片 Alpha；只有 Box 与 Lasso 可以用 BEN2 执行 Refine Selection。 ## 对应文件 @@ -20,20 +20,22 @@ operators/image_tool/ ```mermaid flowchart TD - Tool["Box / Lasso / Line / Polyline"] --> Gesture["ImageMaskGesture"] - Gesture --> Polygon["图片像素 Polygon"] - Polygon --> Refine{"Refine Selection?"} - Refine -- "否" --> Mask["本地几何 Mask"] - Refine -- "是" --> Job["refine-image-mask"] + Tool["Box / Lasso / Line / Polyline"] --> Gesture["ImageRegionGesture"] + Gesture --> Region["RegionPath"] + Region --> Raster["栅格化 Region"] + Raster --> Data["RegionMask"] + Data --> Refine{"Refine Selection?"} + Refine -- "否" --> Mask["Region Mask"] + Refine -- "是" --> Input["带 Region Alpha 的裁切 RGBA"] + Input --> Job["refine-image-mask"] Job --> Mask - Mask --> Invert["可选反选"] - Invert --> Alpha["源 Alpha × 最终 Mask"] + Mask --> Alpha["源 Alpha × Mask"] Alpha --> Crop["紧裁切并写回 Empty"] ``` -`EditImageMask` 只接受静态图片。未启用 Refine 时，Polygon 在 Blender 侧直接栅格化；启用后，Operator 准备单图输入并提交 Job，响应阶段读取局部 Alpha 与源图 `bounds`，再映射回完整画布。反选始终在最终 Mask 上执行。 +`EditImageMask` 只接受静态图片。`ImageRegionGesture` 把屏幕路径投射为 `RegionPath`；`rasterize_region_path()` 再生成 `RegionMask`。未启用 Refine 时直接把 Region Mask 乘入源 Alpha；启用后，Operator 将 `RegionMask` 写成局部 RGBA 并提交 Job，响应阶段读取细化后的 Alpha 与 `bounds`，再映射回完整画布。 -Box 和 Lasso 分别保存自己的 Refine 状态。Line 与 Polyline 强制关闭 Refine，只使用几何 Mask；Line 将有向线扩展为半平面，Polyline 显式闭合用户确认的顶点。 +Box 和 Lasso 分别保存自己的 Refine Selection 状态。Line 与 Polyline 强制关闭 Refine，只使用本地 Region Mask；Line 将有向线扩展为半平面 Path，Polyline 显式闭合用户确认的 Path。 ## 手势与预览 @@ -43,4 +45,4 @@ Box 和 Lasso 分别保存自己的 Refine 状态。Line 与 Polyline 强制关 ## 写回结果 -最终像素保留源 RGB，只修改 Alpha，再按非零 Alpha 紧裁切。公共写回函数只替换当前 Empty 的 Image，并更新 `empty_display_size` 与 `empty_image_offset`，因此剩余像素继续对齐原图片平面，对象 `matrix_world` 不变。 +最终像素通过公共 `image_pixels()` / `image_rgba()` 读取，保留源 RGB，只修改 Alpha，再按非零 Alpha 紧裁切。公共像素写回函数创建结果后，通过统一的 Image Empty 替换入口处理共享引用，并更新 `empty_display_size` 与 `empty_image_offset`，因此剩余像素继续对齐原图片平面，对象 `matrix_world` 不变。 diff --git a/docs/internals/operators/index.md b/docs/internals/operators/index.md index ce9bc0a..8b8d9d0 100644 --- a/docs/internals/operators/index.md +++ b/docs/internals/operators/index.md @@ -1,6 +1,6 @@ # Operator 实现索引 -`src/anyimage/operators/` 负责 Blender 侧交互、Job 请求组装和结果导入。内部实现按完整功能包或顶层业务模块拆页；只提供共享能力的 `common/` 不单独成页，其职责统一放在[程序架构](../architecture.md)中说明。 +`src/anyimage/operators/` 负责 Blender 侧交互、Job 请求组装和结果导入。内部实现按完整功能包或顶层业务模块拆页；`common/` 的边界见[程序架构](../architecture.md)，实际公开入口见[Common 公共函数](../common.md)。 | Operator 路径 | 实现页 | 主要职责 | | --- | --- | --- | @@ -8,11 +8,11 @@ | `operators/ai_setup.py` | [AI 环境管理](ai-setup.md) | 环境安装、模型下载与缓存释放 | | `operators/clipboard_image/` | [Clipboard Image](clipboard-image.md) | 系统剪贴板读取、粘贴与快捷键 | | `operators/convert_to_plane/` | [Convert to Plane](convert-to-plane.md) | Mesh Plane 与 Depth Plane 创建 | -| `operators/cutout_tool/` | [Cutout Tool](cutout-tool.md) | 区域选择、轮廓生成与 Shape 构建 | +| `operators/cutout_tool/` | [Cutout Tool](cutout-tool.md) | Region 选择、基础 Shape 构建与 Blender 对象创建 | | `operators/debug.py` | [Debug Operator](debug.md) | 模型接口与原始产物验证 | | `operators/image_tool/` | [Image Tool](image-tool.md) | Box、Lasso、Line 与 Polyline 编辑 | | `operators/perspective_tool/` | [Perspective Tool](perspective-tool.md) | 四点透视框与透视校正 | | `operators/remove_background.py` | [Remove Background](remove-background.md) | BEN2 去背与结果替换 | -| `operators/upscale.py` | [Upscale](upscale.md) | 4× 高清与原图引用维护 | +| `operators/upscale.py` | [Upscale](upscale.md) | 当前 Image 的 4× 高清与输入长边检查 | `operators/__init__.py` 只聚合注册类，不承载独立业务流程。具体类的注册和逆序注销由扩展入口统一管理，整体边界见[程序架构](../architecture.md)。 diff --git a/docs/internals/operators/perspective-tool.md b/docs/internals/operators/perspective-tool.md index 5316451..9f6b5fa 100644 --- a/docs/internals/operators/perspective-tool.md +++ b/docs/internals/operators/perspective-tool.md @@ -10,7 +10,7 @@ operators/perspective_tool/ └── edit_perspective.py # 四点交互、预览、Homography 与写回 ``` -图片选择、裁切和 Empty 写回复用 `operators/common/` 的基础能力，不与 Image Mask Operator 共享业务状态。 +图片坐标投射、Alpha 裁切和 Empty 写回复用 `operators/common/` 的基础能力；四点数据始终称为 Quad，不与 Image Region 或 Image Mask Operator 共享业务状态。 ## 矫正流程 @@ -29,4 +29,4 @@ flowchart TD Homography 使用预乘 Alpha 插值，避免透明边缘混入黑色。输出最长边限制为 8192 px，总像素限制为 16,777,216，并按 256 行分块计算，控制中间数组大小。 -Perspective 是 deform 工具，不提供 Refine 或反选。结果比例可能不同于原四边形的轴对齐范围，因此 Empty 写回保持中心和覆盖尺度近似一致，而不是假设当前 3D View 相机等于图片拍摄相机。 +Perspective 是 deform 工具，不提供 Refine 或反选。它通过公共 `image_pixels()` 读取供预览与最终 Warp 共用的源像素，并与 Image Mask 共用像素结果创建和 Image Empty 替换入口。结果比例可能不同于原四边形的轴对齐范围，因此 Empty 写回保持中心和覆盖尺度近似一致，而不是假设当前 3D View 相机等于图片拍摄相机。 diff --git a/docs/internals/operators/remove-background.md b/docs/internals/operators/remove-background.md index dee4864..9ac0a2d 100644 --- a/docs/internals/operators/remove-background.md +++ b/docs/internals/operators/remove-background.md @@ -8,7 +8,7 @@ src/anyimage/ ├── operators/ │ ├── remove_background.py # Blender 入口、任务参数与结果替换 -│ └── common/image.py # 媒体输入准备与结果 Image 创建 +│ └── common/image_data.py # 媒体输入准备与结果 Image 创建 └── server/ ├── jobs/remove_background.py # 单图与多帧任务编排 └── models/ben2.py # BEN2 前景适配 @@ -28,4 +28,4 @@ flowchart TD Packed 图片或内存中修改过的单图会先导出为临时输入；文件图片、视频和序列则沿用可读取的源路径。多帧输入由统一媒体层展开，结果保持连续帧命名，导入后继续沿用源 Image User 的帧起点、偏移与循环设置。 -BEN2 推理前使用输入 Alpha 屏蔽透明区域，模型输出的前景 Alpha 再与源 Alpha 合成，RGB 保持为源图内容。响应阶段只替换当前 Empty 的 Image 数据；当旧 Image 没有其他用户和 Fake User 时，旧数据块会被清理并由结果继承名称。 +BEN2 推理前使用输入 Alpha 屏蔽透明区域，模型输出的前景 Alpha 再与源 Alpha 合成，RGB 保持为源图内容。响应阶段通过公共 Image Empty 替换入口导入结果；当旧 Image 没有其他用户和 Fake User 时，旧数据块会被清理并由结果继承名称。 diff --git a/docs/internals/operators/upscale.md b/docs/internals/operators/upscale.md index dcffe00..1b3b731 100644 --- a/docs/internals/operators/upscale.md +++ b/docs/internals/operators/upscale.md @@ -1,6 +1,6 @@ # Upscale -本页对应 `src/anyimage/operators/upscale.py`。该模块读取扩展首选项中的模型与全局 AI 输入长边上限，提交 4× 高清任务，并维护高清结果到原始 Image 的引用。 +本页对应 `src/anyimage/operators/upscale.py`。该模块读取扩展首选项中的模型与全局 AI 输入长边上限，检查当前 Image 是否可处理，再提交 4× 高清任务。 ## 对应文件 @@ -8,7 +8,7 @@ src/anyimage/ ├── operators/ │ ├── upscale.py # Blender 入口、任务参数与结果替换 -│ └── common/image.py # 媒体输入准备与结果 Image 创建 +│ └── common/image_data.py # 媒体输入准备与结果 Image 创建 └── server/ ├── jobs/upscale.py # 单图与多帧任务编排 ├── media/resolution.py # 共用长边限制与等比缩放 @@ -20,17 +20,18 @@ src/anyimage/ ```mermaid flowchart TD - Empty["Image Empty"] --> Source["解析原始 Image"] - Source --> Prepare["准备媒体输入"] + Empty["Image Empty"] --> Source["读取当前 Image"] + Source --> Gate{"当前长边小于全局上限？"} + Gate -- "否" --> Disabled["Upscale 置灰"] + Gate -- "是" --> Prepare["准备媒体输入"] Prepare --> Job["upscale-image Job"] - Job --> Limit["按全局输入长边上限等比缩放"] - Limit --> RGB["分块执行 4× RGB 推理"] - Limit --> Alpha["独立缩放 Alpha"] + Job --> RGB["分块执行 4× RGB 推理"] + Job --> Alpha["独立缩放 Alpha"] RGB --> Merge["合并 RGBA 结果"] Alpha --> Merge Merge --> Replace["导入并替换当前 Image"] ``` -模型以 256×256 分块处理 RGB，并带入相邻上下文以减弱块边界；Alpha 按同一受限输入尺寸单独缩放，最后合并为 RGBA。`max_input_size` 来自全局 **Maximum AI Input Size**，默认 `2048`，只约束送入模型的长边；4× 输出长边因此默认为最多 `8192`。输入尺寸不超过上限时不会发生预缩放。 +模型以 256×256 分块处理 RGB，并带入相邻上下文以减弱块边界；Alpha 不进入模型，使用 Lanczos 独立放大后合并为 RGBA。`max_input_size` 来自全局 **Maximum AI Input Size**，默认 `2048`。Operator 只允许当前 Image 长边小于该值时执行，因此正式流程不会预缩放输入，4× 输出长边小于上限的四倍。Server 仍保留相同上限作为任务边界。 -结果以 `_upscale.png` 结尾，并通过 `Image.anyimage_upscale_source` 保存原图引用。再次对高清结果执行时，Operator 会从原图重新生成，而不是继续放大上一次输出。模型与全局输入上限在每次执行时从 Preferences 读取，再写入可序列化的隐藏属性交给 Job。Provider 资源错误会释放当前 Upscale Session，并提示降低全局输入上限或改用 CPU。 +菜单根据当前尺寸只显示预计的 4× 结果尺寸。结果通过与 Remove Background、Image Mask 和 Perspective 相同的 Image Empty 替换入口处理：原 Image 没有其他用户且未启用 Fake User 时，移除旧数据并让结果沿用原名；存在其他用户时保留原 Image，并只让当前 Empty 使用独立结果。结果不增加 Upscale 专属后缀，也不保存来源引用。后续 Upscale 与 Depth Plane 都读取当前 Image。模型与全局输入上限在每次执行时从 Preferences 读取，再写入可序列化的隐藏属性交给 Job。Provider 资源错误会释放当前 Upscale Session，并提示降低全局输入上限或改用 CPU。 diff --git a/docs/internals/outputs.md b/docs/internals/outputs.md index 7d6064b..4dead2c 100644 --- a/docs/internals/outputs.md +++ b/docs/internals/outputs.md @@ -21,16 +21,16 @@ flowchart TD | Job | 主要文件 | 消费方 | | --- | --- | --- | | `remove-background` | `image.0001.png` 起 | 替换 Image Empty 的颜色数据 | -| `upscale-image` | `upscale.0001.png` 起 | 创建 `_upscale.png` Image 并记录原图引用 | -| `refine-image-mask` | `selection-input.png`、`image.png`、`bounds.json` | Image Box / Lasso 读取 Alpha 与原图坐标 | -| `prepare-geometry-data`：Depth Plane | `image.0001.*`、`field.npz` | 创建颜色图、深度图、材质和 Plane | -| `prepare-geometry-data`：Cutout | `image.0001.png`、`selection.png`、`analysis.png`、`bounds.json`，以及按形状生成的 `field.npz` / `normal.png` | Blender 主线程构建局部网格与贴图 | +| `upscale-image` | `upscale.0001.png` 起 | 按共享引用情况替换当前 Image Empty 的颜色数据，并沿用原 Image 命名语义 | +| `refine-image-mask` | `image.png`、`bounds.json` | Image Box / Lasso 读取细化 Alpha 与原图坐标 | +| `generate-depth-plane` | `image.0001.*`、`image_geometry.npz` | 创建颜色图、深度图、材质和 Plane | +| `generate-region-data` | 可选的 `region.png`、`region_bounds.json`、`image_geometry.npz`、`normal.png` | Blender 主线程构建 Cutout Shape 与贴图 | -`prepare-geometry-data` 不输出 Blender 网格。它只准备颜色、选区和模型数组；轮廓提取、约束三角化、Geometry Nodes Modifier 与法线烘焙都在 Blender 响应阶段完成。 +两个几何 Job 都不输出 Blender 网格。`generate-depth-plane` 生成完整图片的颜色副本与深度场；`generate-region-data` 只生成请求的 AI 产物。轮廓提取、约束三角化以及 Mesh、Material 和 Geometry Nodes 创建都在 Blender 主线程完成。 ## 深度场 -`field.npz` 是 Depth Plane、Depth Balloon 与 Depth Surface 共用的模型中间结果： +`image_geometry.npz` 是 Depth Plane、Depth Balloon 与 Depth Surface 共用的模型中间结果。Blender 侧统一通过 `operators/common/mesh_result.py` 的 `load_image_geometry()` 读取为 `ImageGeometry`，功能模块不分别解释字段： | 字段 | 含义 | | --- | --- | @@ -38,21 +38,21 @@ flowchart TD | `depth` | 相机深度 | | `intrinsics` | 像素坐标下的 3×3 相机内参 | | `valid` | 模型与数值检查后的有效区域 | -| `calibration_valid` | 用于结果定位或深度基准的区域；Cutout 时与选择相交 | +| `calibration_valid` | 用于结果定位或深度基准的区域；Cutout 时与 Region 相交 | | `image_size` | 深度场宽高 | | `reference_depth` | 校准区域中的参考深度 | Server 保留模型空间数据，不提前换算 Blender 单位。Depth Plane 在导入时根据 Image Empty 平面范围和相机内参锚定点场；Cutout 深度形状则计算 `Uniform Scale` 和局部坐标，再把相机点写入 Packed 浮点 Image 供 Geometry Nodes 采样。 -## Cutout 三类图片 +## Cutout Region 产物 -Cutout 把“显示内容”“几何范围”和“模型分析”分开： +Blender 侧先把 `RegionPath` 栅格化为 `RegionMask`，再将 Region 裁切为带有效 Alpha 的临时 RGBA 输入。Server 不接收路径或 Mask 协议，只按需产生 AI 数据： -- `image.0001.png` 保存裁切矩形中的源颜色与原 Alpha，用作材质。 -- `selection.png` 保存最终选择 Alpha，用于轮廓与网格。 -- `analysis.png` 只保留选择内的有效 RGBA，供深度或法线模型分析。 +- 启用 Refine 时，`region.png` 保存细化后的局部 RGBA，`region_bounds.json` 保存它在输入裁切图中的范围； +- Depth Shape 使用 `image_geometry.npz`； +- Tangent 或 Object Normal 使用 `normal.png`。 -`bounds.json` 把裁切矩形映射回源图像素坐标。这样 Server 可以只处理局部数据，Blender 仍能恢复局部平面范围、UV 和世界位置。 +Blender 将局部范围与原始 `RegionMask.bounds` 合成回源图坐标，并直接从源 Image 创建最终 Color Image。未启用 Refine、Depth 或 Normal 时不提交 Job。 ## Debug 产物 @@ -65,7 +65,7 @@ Debug Job 保留模型适配层的直接输出，不执行正式功能的 Blende | `debug-ben2` | `foreground*.png` | | `debug-upscale` | `prediction*.png` | -这些文件用于验证模型、Device 和适配层。正式 Job 会继续派生选择、Alpha、深度场或可导入图片，因此两者的文件结构有意不同。 +这些文件用于验证模型、Device 和适配层。正式 Job 会继续派生 Region、Alpha、深度场或可导入图片，因此两者的文件结构有意不同。 ## 单图与序列 diff --git a/docs/internals/runtime.md b/docs/internals/runtime.md index 1d3d476..c54575d 100644 --- a/docs/internals/runtime.md +++ b/docs/internals/runtime.md @@ -14,7 +14,7 @@ src/anyimage/ ├── model_manager.py # 模型目录与 Session 缓存 ├── media/input.py # 图片、视频和序列输入 ├── jobs/ # app.py 可提交的公开任务 - ├── selection/ # 选区栅格化与 Cutout 图片准备 + ├── region/ # BEN2 Region 细化 ├── models/ # 模型适配与 ONNX 推理 ├── geometry/ # 深度反投影与法线纹理编码 └── outputs/ # Debug Prediction 序列化 @@ -41,17 +41,18 @@ Runtime 在首次任务或显式启动时拉起 Server。任务在队列中串 | `download-model`、`download-required-models` | 准备模型文件 | | `remove-background` | BEN2 透明前景 | | `upscale-image` | 4× 图片或帧序列 | -| `refine-image-mask` | 为 Image Box / Lasso 细化选区 | -| `prepare-geometry-data` | 为 Depth Plane 或 Cutout 准备颜色、选区、深度和法线数据 | +| `refine-image-mask` | 为 Image Box / Lasso 细化 Region | +| `generate-depth-plane` | 为 Depth Plane 生成颜色副本与深度场 | +| `generate-region-data` | 按需为 Cutout Region 生成 Refine、Depth 或 Normal | | `debug-*` | 保存模型适配层的直接输出 | ## 媒体输入 -`server/media/input.py` 把单图、视频和图片目录统一为有序帧路径，并用上下文管理器负责临时帧目录的生命周期。它还支持将颜色输入与分析输入分开：例如 Upscale 结果可以作为颜色来源，同时使用它保存的原图引用进行 Depth Plane 分析。帧展开是媒体输入的内部能力，不再暴露没有业务消费者的独立 Job。 `server/media/resolution.py` 提供默认 `2048` 的共用输入长边限制；Upscale 直接限制每帧，`prepare-geometry-data` 则在 Cutout 裁切和分析输入选择完成后限制 Depth / Normal 模型输入。该临时缩放图不参与颜色产物写入。 +`server/media/input.py` 把单图、视频和图片目录统一为有序帧路径，并用上下文管理器负责临时帧目录的生命周期。帧展开是媒体输入的内部能力，不再暴露没有业务消费者的独立 Job。`server/media/resolution.py` 提供默认 `2048` 的共用输入长边限制；Upscale Operator 只允许长边小于该值的当前 Image 进入正式任务，Depth Plane 与 Cutout 则只限制模型分析输入，不降低颜色贴图分辨率。 -当前 `prepare-geometry-data` 只接受一帧，因此 Depth Plane 与 Cutout 都是静态图工作流。Remove Background 与 Upscale 通过同一媒体入口处理单图、视频和序列帧。 +`generate-depth-plane` 与 `generate-region-data` 只接受一帧，因此 Depth Plane 与 Cutout 都是静态图工作流。Remove Background 与 Upscale 通过同一媒体入口处理单图、视频和序列帧。 -公开 Job 不与 Operator 一一对应：Depth Plane 和 Cutout 共享 `prepare-geometry-data`，四个 Debug Job 共享 `jobs/debug_predictions.py`。Job 只负责协议和编排；可复用的输入、选区、模型与几何数据能力分别位于 `media/`、`selection/`、`models/` 和 `geometry/`，Job 之间不互相导入实现。 +公开 Job 按稳定的计算边界划分，不按界面入口机械映射。Depth Plane 使用 `jobs/generate_depth_plane.py`，Cutout 使用 `jobs/generate_region_data.py`，四个 Debug Job 共享 `jobs/debug_predictions.py`。Job 只负责协议和编排；可复用的输入、AI Region、模型与几何数据能力分别位于 `media/`、`region/`、`models/` 和 `geometry/`，Job 之间不互相导入实现。 ## 模型层 diff --git a/docs/reference/features.md b/docs/reference/features.md index da46326..00a36da 100644 --- a/docs/reference/features.md +++ b/docs/reference/features.md @@ -34,11 +34,11 @@ Cutout 提供四种结果： | Depth Balloon | `O Image Depth Balloon` | MoGe-2 高度 | | Depth Surface | `O Image Depth Surface` | MoGe-2 相机点 | -四种 Shape 共用 `O Image Cutout` 修改器、规则三角基础面、UV 和 `o_balloon` 属性。Shape 菜单选择内部几何分支，Cleanup 在分支前执行，Smooth 在分支后执行；Point Spacing 控制网格密度，Normal 与 Refine Selection 按需启用 MoGe-2 和 BEN2。 +四种 Shape 共用 `O Image Cutout` 修改器、规则三角基础面、UV 和 `o_balloon` 属性。Shape 菜单选择内部几何分支，Cleanup 在分支前执行且使用便于调节的百倍显示值，Smooth 在分支后执行；Depth Shape 根据 Region 透明比例选择固定或自适应 Base Plane Depth，Normal 可选 None、Tangent 或 Object，Refine Selection 按需启用 BEN2。 ### 图片结果 -Image Mask 和 Perspective 直接更新当前 Empty 的图片并 Pack。Remove Background 输出透明颜色图或序列。Upscale 通过三档 ONNX 模型输出 4× RGBA，并保存原图引用，重复执行时从原图重新生成。 +Image Mask 和 Perspective 直接更新当前 Empty 的图片并 Pack。Remove Background 输出透明颜色图或序列。Upscale 通过三档 ONNX 模型把当前 Image 输出为 4× RGBA；当前长边达到全局输入上限时不可执行。 ## 模型 diff --git a/mkdocs.yml b/mkdocs.yml index 26b8783..3add937 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -29,6 +29,7 @@ nav: - 内部实现: - 总览: internals/index.md - 程序架构: internals/architecture.md + - Common 公共函数: internals/common.md - 基础设施: - Job Runtime 与 Server: internals/runtime.md - Job 产物: internals/outputs.md diff --git a/src/anyimage/__init__.py b/src/anyimage/__init__.py index 6895bab..c7b245a 100644 --- a/src/anyimage/__init__.py +++ b/src/anyimage/__init__.py @@ -66,9 +66,6 @@ def register(): separator=True, ) bpy.types.Scene.anyimage_settings = bpy.props.PointerProperty(type=AnyImageSettings) - bpy.types.Image.anyimage_upscale_source = bpy.props.PointerProperty( - type=bpy.types.Image - ) bpy.types.VIEW3D_MT_object_context_menu.prepend(draw_image_context_menu) bpy.types.OUTLINER_MT_object.prepend(draw_image_context_menu) clipboard_image.register_keymaps() @@ -88,7 +85,5 @@ def unregister(): bpy.types.OUTLINER_MT_object.remove(draw_image_context_menu) if hasattr(bpy.types.Scene, "anyimage_settings"): del bpy.types.Scene.anyimage_settings - if hasattr(bpy.types.Image, "anyimage_upscale_source"): - del bpy.types.Image.anyimage_upscale_source for class_type in reversed(CLASSES): bpy.utils.unregister_class(class_type) diff --git a/src/anyimage/assets/O_AnyImage.blend b/src/anyimage/assets/O_AnyImage.blend index a8523b9..105392e 100644 --- a/src/anyimage/assets/O_AnyImage.blend +++ b/src/anyimage/assets/O_AnyImage.blend @@ -1,3 +1,3 @@ version https://git-lfs.github.com/spec/v1 -oid sha256:c0340770d371f8e97896cd6905a7666b0faf18be09a9c2b58192c3582c9bb37c -size 1443288 +oid sha256:6d85f77daf41fe2b8d6b35d6597577d366bfda466a83ac818ca7c99b51bede0b +size 1446139 diff --git a/src/anyimage/menu.py b/src/anyimage/menu.py index 56543cc..c87a29c 100644 --- a/src/anyimage/menu.py +++ b/src/anyimage/menu.py @@ -1,6 +1,7 @@ import bpy -from .operators.common.image import is_image_empty +from .operators.common.image_data import is_image_empty +from .operators.upscale import upscale_output_size class AnyImageImageMenu(bpy.types.Menu): @@ -53,11 +54,10 @@ def upscale_menu_label(context): source_object = getattr(context, "object", None) if not is_image_empty(source_object): return "Upscale" - width, height = ( - int(value) for value in getattr(source_object.data, "size", (0, 0)) - ) - if width <= 0 or height <= 0: + output_size = upscale_output_size(source_object.data) + if output_size is None: return "Upscale" + width, height = output_size return f"Upscale ({width} × {height})" diff --git a/src/anyimage/operators/__init__.py b/src/anyimage/operators/__init__.py index 6185b2a..2aed2df 100644 --- a/src/anyimage/operators/__init__.py +++ b/src/anyimage/operators/__init__.py @@ -1,8 +1,6 @@ from .remove_background import RemoveImageBackground, RunBackgroundRemoval from .cutout_tool import ( - ActivateCutoutTool, - ConvertCutoutShape, - GenerateCutoutShape, + CutoutRegionToShape, SelectCutoutRegion, ) from .debug import DebugBen2, DebugDa3, DebugMoge2, DebugUpscale @@ -33,10 +31,8 @@ CLASSES = ( DebugBen2, DebugUpscale, GenerateDepthPlane, - GenerateCutoutShape, + CutoutRegionToShape, SelectCutoutRegion, - ConvertCutoutShape, - ActivateCutoutTool, RunBackgroundRemoval, ClearModels, *IMAGE_TOOL_CLASSES, diff --git a/src/anyimage/operators/ai_setup.py b/src/anyimage/operators/ai_setup.py index 2b51c75..98eb8a4 100644 --- a/src/anyimage/operators/ai_setup.py +++ b/src/anyimage/operators/ai_setup.py @@ -8,7 +8,7 @@ from ..properties import ( model_download_size, model_record, ) -from .common.operator import HIDDEN, require_environment +from .common.ai_job import require_environment def format_download_size(size): @@ -128,7 +128,7 @@ class DownloadModel(JobOperatorBase, bpy.types.Operator): bl_label = "Download Model" job_type = "download-model" - model: bpy.props.StringProperty(options=HIDDEN) + model: bpy.props.StringProperty(options={"HIDDEN", "SKIP_SAVE"}) def request(self, _context): require_environment() diff --git a/src/anyimage/operators/common/__init__.py b/src/anyimage/operators/common/__init__.py index 3f87aad..7c8a6d5 100644 --- a/src/anyimage/operators/common/__init__.py +++ b/src/anyimage/operators/common/__init__.py @@ -1 +1 @@ -"""Shared Blender-side image, material, normal, and operator helpers.""" +"""Shared Blender-side image data, interaction, mesh result, and AI Job helpers.""" diff --git a/src/anyimage/operators/common/operator.py b/src/anyimage/operators/common/ai_job.py similarity index 62% rename from src/anyimage/operators/common/operator.py rename to src/anyimage/operators/common/ai_job.py index 6a44752..1e4406f 100644 --- a/src/anyimage/operators/common/operator.py +++ b/src/anyimage/operators/common/ai_job.py @@ -1,4 +1,3 @@ -import shutil from pathlib import Path import bpy @@ -6,24 +5,22 @@ import bpy from ...preferences import addon_preferences from ...properties import ai_ready, model_ready, model_record from ...runtime import runtime -from .image import extract_frame_range BEN2_IMAGE_SUFFIXES = {".jpg", ".jpeg", ".png", ".webp"} BEN2_VIDEO_SUFFIXES = { ".mp4", ".mov", ".avi", ".mkv", ".webm", ".m4v", ".mts", ".mpg", ".mpeg" } -HIDDEN = {"HIDDEN", "SKIP_SAVE"} -def input_path(value): +def require_input_path(value): path = Path(bpy.path.abspath(value)) if path.is_file() or path.is_dir(): return path raise ValueError("Select a valid image, video, or frame directory") -def background_input_path(value): - path = input_path(value) +def require_background_input_path(value): + path = require_input_path(value) if path.is_dir() or path.suffix.lower() in ( BEN2_IMAGE_SUFFIXES | BEN2_VIDEO_SUFFIXES ): @@ -33,11 +30,6 @@ def background_input_path(value): ) -def source_start_frame(source_object_name): - source_object = bpy.data.objects.get(source_object_name) - return extract_frame_range(source_object)[0] if source_object else 1 - - def production_device(): return getattr(addon_preferences(), "device", "AUTO") @@ -62,35 +54,6 @@ def require_model(model_key): return model -def cleanup_temporary_input(path, delete_input): - if not delete_input or not path: - return - temporary_path = Path(path) - try: - if temporary_path.is_dir(): - shutil.rmtree(temporary_path, ignore_errors=True) - return - temporary_path.unlink(missing_ok=True) - temporary_path.parent.rmdir() - except OSError: - pass - - -def open_job_directory(directory): - directory = Path(directory) - if not directory.is_dir(): - raise RuntimeError(f"Job directory does not exist: {directory}") - result = bpy.ops.wm.path_open(filepath=str(directory)) - if "FINISHED" not in result: - raise RuntimeError(f"Unable to open job directory: {directory}") - - -def push_undo(message): - result = bpy.ops.ed.undo_push(message=message) - if "FINISHED" not in result: - raise RuntimeError(f"Unable to create the {message} Undo step") - - def da3_parameters(settings, path, model, process_res, start_frame): return { "input": str(path), diff --git a/src/anyimage/operators/common/image.py b/src/anyimage/operators/common/image_data.py similarity index 60% rename from src/anyimage/operators/common/image.py rename to src/anyimage/operators/common/image_data.py index 29f47a1..c37bc66 100644 --- a/src/anyimage/operators/common/image.py +++ b/src/anyimage/operators/common/image_data.py @@ -5,9 +5,6 @@ from pathlib import Path import bpy -DA3_PATCH_SIZE = 14 -DA3_MIN_PROCESS_RES = 252 -DA3_MAX_PROCESS_RES = 1008 IMAGE_NAME_SUFFIXES = ( ".jpeg", ".jpg", @@ -44,6 +41,33 @@ def is_image_empty(candidate): ) +def require_image_empty(object_name): + """Return a named Image Empty or raise when it is no longer available.""" + source_object = bpy.data.objects.get(object_name) + if not is_image_empty(source_object): + raise RuntimeError("The source Image Empty is no longer available") + return source_object + + +def image_pixels(image): + """Read one Blender Image into a flat float32 pixel array.""" + import numpy as np + + pixels = np.empty(len(image.pixels), dtype=np.float32) + image.pixels.foreach_get(pixels) + return pixels + + +def image_rgba(image): + """Read one Blender Image into a top-down RGBA float32 array.""" + import numpy as np + + width, height = (int(value) for value in image.size) + if width <= 0 or height <= 0: + raise ValueError("The source image has no readable dimensions") + return np.flipud(image_pixels(image).reshape((height, width, 4))) + + def is_animated_image(image): return getattr(image, "source", None) in {"MOVIE", "SEQUENCE"} @@ -95,6 +119,11 @@ def default_extract_frames(source_object): return extract_frame_range(source_object)[1] +def source_start_frame(source_object_name): + source_object = bpy.data.objects.get(source_object_name) + return extract_frame_range(source_object)[0] if source_object else 1 + + def image_path(image): filepath_from_user = getattr(image, "filepath_from_user", None) filepath = ( @@ -113,14 +142,17 @@ def image_is_packed(image): return bool(getattr(image, "packed_files", ())) -def remove_temporary_input(path): - """Remove a temporary input file or frame directory.""" +def cleanup_image_input(path, temporary): + """Remove a prepared input and its temporary container when requested.""" + if not temporary or not path: + return path = Path(path) try: if path.is_dir(): shutil.rmtree(path, ignore_errors=True) else: path.unlink(missing_ok=True) + path.parent.rmdir() except OSError: pass @@ -149,62 +181,10 @@ def normal_data_name(source_object): return f"{image_base_name(source_object.data)}_normal.png" -def copy_depth_image(source_object): - """Return a renamed copy of the source image for use as the depth map.""" - depth_image = source_object.data.copy() - depth_image.name = depth_data_name(source_object) - return depth_image - - def color_data_name(source_object): return f"{image_base_name(source_object.data)}_color.png" -def material_data_name(source_object): - return image_base_name(source_object.data) - - -def scaled_process_resolution(image, scale_percent): - """Return the scaled long edge clamped and aligned for DA3 inference.""" - width, height = image.size - if width <= 0 or height <= 0: - raise ValueError("The source image has no readable dimensions") - if scale_percent < 1.0 or scale_percent > 100.0: - raise ValueError("Resolution Scale must be between 1% and 100%") - process_res = max(width, height) * scale_percent / 100.0 - process_res = max(DA3_MIN_PROCESS_RES, process_res) - process_res = min(DA3_MAX_PROCESS_RES, process_res) - return int(process_res / DA3_PATCH_SIZE + 0.5) * DA3_PATCH_SIZE - - -def process_resolution_scale_property(): - return bpy.props.FloatProperty( - name="Resolution Scale", - description=( - "Scale the source image long edge for DA3 processing; the result " - "is limited to 252-1008 px and aligned to a multiple of 14" - ), - default=100.0, - min=1.0, - max=100.0, - step=100, - precision=0, - subtype="PERCENTAGE", - ) - - -def draw_process_resolution(layout, owner, image): - process_res = scaled_process_resolution( - image, - owner.process_res_factor, - ) - layout.prop( - owner, - "process_res_factor", - text=f"Resolution Scale ({process_res} px)", - ) - - def save_packed_image(image, _scene): output_path = Path(tempfile.mkdtemp(prefix="anyimage-input-")) / "input.png" original_file_format = image.file_format @@ -280,40 +260,42 @@ def prepare_image_input(image, scene): return source_path, False -def analysis_source_image(image): - """Return the oldest available source behind an upscaled image.""" - current = image - visited = set() - while current is not None and id(current) not in visited: - visited.add(id(current)) - source = getattr(current, "anyimage_upscale_source", None) - if source is None or source is current: - break - current = source - return current +def prepare_region_input(image, region_mask): + """Write one cropped RGBA Region input owned by its temporary directory.""" + import numpy as np - -def prepare_image_analysis_inputs(image, scene): - """Prepare display and analysis inputs, reusing one path when possible.""" - source_path, delete_input = prepare_image_input(image, scene) - analysis_image = analysis_source_image(image) - if analysis_image is image: - return source_path, delete_input, source_path, False - try: - analysis_path, delete_analysis_input = prepare_image_input( - analysis_image, - scene, - ) - except Exception: - if delete_input: - remove_temporary_input(source_path) - raise - return ( - source_path, - delete_input, - analysis_path, - delete_analysis_input, + left, top, right, bottom = region_mask.bounds + rgba = image_rgba(image) + cropped = rgba[top:bottom, left:right].copy() + if cropped.shape[:2] != region_mask.values.shape: + raise ValueError("The Region does not match the source image") + cropped[:, :, 3] *= region_mask.values + cropped[cropped[:, :, 3] <= 0.0, :3] = 0.0 + output_directory = Path(tempfile.mkdtemp(prefix="anyimage-region-")) + output_path = output_directory / "input.png" + temporary_image = bpy.data.images.new( + "AnyImage Region Input", + width=right - left, + height=bottom - top, + alpha=True, ) + original_format = temporary_image.file_format + try: + temporary_image.colorspace_settings.name = image.colorspace_settings.name + temporary_image.alpha_mode = "STRAIGHT" + temporary_image.pixels.foreach_set(np.flipud(cropped).ravel()) + temporary_image.file_format = "PNG" + temporary_image.save(filepath=str(output_path), save_copy=True) + except (OSError, RuntimeError): + cleanup_image_input(output_path, True) + raise + finally: + temporary_image.file_format = original_format + bpy.data.images.remove(temporary_image, do_unlink=True) + if not output_path.is_file(): + cleanup_image_input(output_path, True) + raise RuntimeError("Blender did not export the Region input") + return output_path def configure_image_user( @@ -388,31 +370,98 @@ def force_sequence_source(image): return image -def create_sequence_image(name, frame_directory, frame_prefix="frame."): - """Load a numbered frame directory as a SEQUENCE image datablock.""" - frame_paths = sorted_frame_paths(frame_directory, frame_prefix) - if not frame_paths: - raise RuntimeError(f"No image frames found in: {frame_directory}") - try: - image = bpy.data.images.load( - str(frame_paths[0]), - check_existing=False, - ) - force_sequence_source(image) - except (OSError, RuntimeError) as error: - if "image" in locals(): - bpy.data.images.remove(image, do_unlink=True) - raise RuntimeError("Unable to load the frame sequence") from error - image.name = name - pack_sequence_if_possible(image, len(frame_paths)) +def load_normal_result_image(path, source_object): + """Load, name, configure, and pack a tangent normal image.""" + image = bpy.data.images.load(str(path), check_existing=False) + image.name = normal_data_name(source_object) + image.colorspace_settings.name = "Non-Color" + image.pack() return image +def create_pixel_result_image(source_image, pixels, output_size, suffix=""): + """Create a packed Image datablock from edited RGBA pixels.""" + output_width, output_height = (int(value) for value in output_size) + if output_width <= 0 or output_height <= 0: + raise ValueError("The edited image dimensions must be positive") + if len(pixels) != output_width * output_height * 4: + raise ValueError("The edited pixels do not match the image dimensions") + image = bpy.data.images.new( + f"{image_base_name(source_image)}{suffix}", + width=output_width, + height=output_height, + alpha=True, + ) + try: + image.colorspace_settings.name = source_image.colorspace_settings.name + image.alpha_mode = "STRAIGHT" + image.pixels.foreach_set(pixels) + image.pack() + image.update() + except Exception: + bpy.data.images.remove(image, do_unlink=True) + raise + return image -def create_color_image( + +def replace_empty_image(source_object, result_image, *, placement_bounds=None): + """Replace one Empty image, retaining shared source images.""" + source_image = source_object.data + source_name = source_image.name + original_fake_user = source_image.use_fake_user + image_user = getattr(source_object, "image_user", None) + frame_count = ( + video_total_frames(result_image) + if is_animated_image(result_image) + else None + ) + alignment = ( + empty_display_alignment( + source_object, + tuple(source_image.size), + tuple(result_image.size), + placement_bounds, + ) + if placement_bounds is not None + else None + ) + if frame_count is not None: + ( + original_frame_start, + _original_frame_offset, + _original_frame_duration, + original_use_cyclic, + ) = image_user_settings(source_object) + + source_image.use_fake_user = True + try: + source_object.data = result_image + if frame_count is not None: + configure_image_user( + image_user, + frame_count, + frame_start=original_frame_start, + use_cyclic=original_use_cyclic, + ) + if alignment is not None: + source_object.empty_display_size = alignment[0] + source_object.empty_image_offset = alignment[1] + except Exception: + source_object.data = source_image + source_image.use_fake_user = original_fake_user + raise + + source_image.use_fake_user = original_fake_user + if not original_fake_user and int(source_image.users) == 0: + bpy.data.images.remove(source_image, do_unlink=True) + result_image.name = source_name + return result_image + + + +def load_color_result_image( source_object, output_directory, - assign_to_source=True, *, frame_prefix="image.", ): @@ -439,39 +488,20 @@ def create_color_image( bpy.data.images.remove(result_image, do_unlink=True) raise RuntimeError("Unable to load the color result") from error - original_fake_user = source_image.use_fake_user try: result_name = color_data_name(source_object) if animated: pack_sequence_if_possible(result_image, len(color_paths)) else: result_image.pack() - if assign_to_source: - image_user = getattr(source_object, "image_user", None) - ( - original_frame_start, - _original_frame_offset, - _original_frame_duration, - original_use_cyclic, - ) = image_user_settings(source_object) - source_image.use_fake_user = True - source_object.data = result_image - if animated: - configure_image_user( - image_user, - len(color_paths), - frame_start=original_frame_start, - use_cyclic=original_use_cyclic, - ) - except (OSError, RuntimeError) as error: - source_image.use_fake_user = original_fake_user + result_image.name = result_name + except (OSError, RuntimeError, TypeError, ValueError) as error: bpy.data.images.remove(result_image, do_unlink=True) raise RuntimeError("Unable to prepare the color result") from error - result_image.name = result_name return result_image -def plane_bounds(source_object): +def image_empty_bounds(source_object): image_width, image_height = source_object.data.size if image_width <= 0 or image_height <= 0: raise ValueError("The source image has no readable dimensions") @@ -494,3 +524,47 @@ def plane_bounds(source_object): y_min, y_min + height, ) + + +def empty_display_alignment( + source_object, + source_size, + output_size, + placement_bounds, +): + """Return the display size and offset for an edited Image Empty.""" + source_width, source_height = source_size + output_width, output_height = output_size + left, top, right, bottom = placement_bounds + if ( + source_width <= 0 + or source_height <= 0 + or output_width <= 0 + or output_height <= 0 + or right <= left + or bottom <= top + ): + raise ValueError("The edited image placement is invalid") + x_min, x_max, y_min, y_max = image_empty_bounds(source_object) + local_width = x_max - x_min + local_height = y_max - y_min + desired_x_min = x_min + left * local_width / source_width + desired_x_max = x_min + right * local_width / source_width + desired_y_max = y_max - top * local_height / source_height + desired_y_min = y_max - bottom * local_height / source_height + desired_width = desired_x_max - desired_x_min + desired_height = desired_y_max - desired_y_min + display_size = max(desired_width, desired_height) + aspect = output_width / output_height + if aspect >= 1.0: + display_width = display_size + display_height = display_size / aspect + else: + display_width = display_size * aspect + display_height = display_size + center_x = (desired_x_min + desired_x_max) * 0.5 + center_y = (desired_y_min + desired_y_max) * 0.5 + return display_size, ( + center_x / display_width - 0.5, + center_y / display_height - 0.5, + ) diff --git a/src/anyimage/operators/common/image_edit.py b/src/anyimage/operators/common/image_edit.py deleted file mode 100644 index d43d7d2..0000000 --- a/src/anyimage/operators/common/image_edit.py +++ /dev/null @@ -1,567 +0,0 @@ -import json -import math - -import bpy - -from ...properties import ( - BEN2_MODEL_KEY, - ai_ready, - ai_setup_label, - ai_status, -) -from .operator import production_device, require_environment, require_model -from .image import image_base_name, is_image_empty, plane_bounds - - -class ImageEditWarning(ValueError): - pass - - -def report_image_edit_exception(operator, error): - level = "WARNING" if isinstance(error, ImageEditWarning) else "ERROR" - operator.report({level}, str(error)) - - -def image_edit_poll(context): - return context.area is not None and context.area.type == "VIEW_3D" - - -def _restore_selection(context, active_object, selected_objects): - bpy.ops.object.select_all(action="DESELECT") - for selected in selected_objects: - if bpy.data.objects.get(selected.name) is not None: - selected.select_set(True) - if ( - active_object is not None - and bpy.data.objects.get(active_object.name) is not None - ): - context.view_layer.objects.active = active_object - - -def resolve_image_edit_click(context, event, report): - """Resolve the first click into source selection or image editing.""" - active_object = context.active_object - selected_objects = tuple(context.selected_objects) - try: - bpy.ops.view3d.select( - location=(event.mouse_region_x, event.mouse_region_y), - extend=False, - deselect_all=False, - ) - except RuntimeError: - pass - clicked_object = context.active_object - if is_image_empty(clicked_object) and clicked_object != active_object: - return clicked_object, False - if is_image_empty(clicked_object): - return clicked_object, True - if is_image_empty(active_object): - _restore_selection(context, active_object, selected_objects) - return active_object, True - report({"WARNING"}, "Click an Image Empty to use an Image tool") - return None, False - - -def active_view3d_tool_id(context): - tools = getattr(getattr(context, "workspace", None), "tools", None) - from_mode = getattr(tools, "from_space_view3d_mode", None) - if not callable(from_mode): - return None - try: - tool = from_mode(getattr(context, "mode", "OBJECT"), create=False) - except (RuntimeError, TypeError): - return None - return getattr(tool, "idname", None) - - -def polygon_bounds(polygon): - import numpy as np - - points = np.asarray(polygon, dtype=np.float64) - if points.ndim != 2 or points.shape[0] < 3 or points.shape[1] != 2: - raise ValueError("The selection does not define a valid image region") - if not np.isfinite(points).all(): - raise ValueError("The selection contains invalid image coordinates") - left = int(math.floor(float(points[:, 0].min()))) - top = int(math.floor(float(points[:, 1].min()))) - right = int(math.ceil(float(points[:, 0].max()))) - bottom = int(math.ceil(float(points[:, 1].max()))) - if right <= left or bottom <= top: - raise ValueError("The selection does not define a valid image region") - return left, top, right, bottom - - -def crop_pixels_to_alpha(pixels, bounds): - """Crop Blender-order RGBA pixels to their nonzero Alpha bounds.""" - import numpy as np - - left, top, right, bottom = bounds - width = right - left - height = bottom - top - rgba = np.flipud( - np.asarray(pixels, dtype=np.float32).reshape((height, width, 4)) - ) - visible_y, visible_x = np.nonzero(rgba[:, :, 3] > 0.0) - if not len(visible_x): - raise ImageEditWarning("The edit contains no visible image pixels") - content_left = int(visible_x.min()) - content_top = int(visible_y.min()) - content_right = int(visible_x.max()) + 1 - content_bottom = int(visible_y.max()) + 1 - output = rgba[content_top:content_bottom, content_left:content_right].copy() - return np.flipud(output).ravel(), ( - left + content_left, - top + content_top, - left + content_right, - top + content_bottom, - ) - - -def empty_display_alignment( - source_object, source_size, output_size, placement_bounds -): - """Calculate image canvas placement without changing the Empty transform.""" - source_width, source_height = (float(value) for value in source_size) - output_width, output_height = (float(value) for value in output_size) - left, top, right, bottom = (float(value) for value in placement_bounds) - x_min, x_max, y_min, y_max = plane_bounds(source_object) - local_width = x_max - x_min - local_height = y_max - y_min - desired_x_min = x_min + left * local_width / source_width - desired_x_max = x_min + right * local_width / source_width - desired_y_max = y_max - top * local_height / source_height - desired_y_min = y_max - bottom * local_height / source_height - desired_width = desired_x_max - desired_x_min - desired_height = desired_y_max - desired_y_min - display_size = max(desired_width, desired_height) - aspect = output_width / output_height - if aspect >= 1.0: - display_width = display_size - display_height = display_size / aspect - else: - display_width = display_size * aspect - display_height = display_size - center_x = (desired_x_min + desired_x_max) * 0.5 - center_y = (desired_y_min + desired_y_max) * 0.5 - return display_size, ( - center_x / display_width - 0.5, - center_y / display_height - 0.5, - ) - - - -def write_image_pixels(image, pixels): - image.alpha_mode = "STRAIGHT" - image.pixels.foreach_set(pixels) - image.pack() - image.update() - - -def replace_empty_image_pixels( - source_object, pixels, output_size, placement_bounds, suffix="" -): - source_image = source_object.data - source_size = tuple(source_image.size) - output_width, output_height = (int(value) for value in output_size) - if output_width <= 0 or output_height <= 0: - raise ValueError("The edited image dimensions must be positive") - if len(pixels) != output_width * output_height * 4: - raise ValueError("The edited pixels do not match the image dimensions") - display_size, image_offset = empty_display_alignment( - source_object, - source_size, - (output_width, output_height), - placement_bounds, - ) - image = bpy.data.images.new( - f"{image_base_name(source_image)}{suffix}", - width=output_width, - height=output_height, - alpha=True, - ) - try: - image.colorspace_settings.name = source_image.colorspace_settings.name - write_image_pixels(image, pixels) - except Exception: - bpy.data.images.remove(image, do_unlink=True) - raise - source_name = source_image.name - source_object.data = image - if not source_image.use_fake_user and int(source_image.users) == 0: - bpy.data.images.remove(source_image) - image.name = source_name - source_object.empty_display_size = display_size - source_object.empty_image_offset = image_offset - return image - - -def image_mask_refine_request(input_path, selection_data): - require_environment() - require_model(BEN2_MODEL_KEY) - polygon = json.loads(selection_data) - polygon_bounds(polygon) - return { - "input": str(input_path), - "polygon": polygon, - "model": BEN2_MODEL_KEY, - "device": production_device().lower(), - } - - -def refine_selection_setting_name(operator): - if getattr(operator, "active_tool_id", None) == "anyimage.cutout_lasso": - return "cutout_refine_selection" - return { - "BOX": "image_box_refine_selection", - "LASSO": "image_lasso_refine_selection", - }.get(operator.gesture) - - -def apply_image_mask_settings(operator, context): - settings = context.scene.anyimage_settings - setting_name = refine_selection_setting_name(operator) - operator.refine_selection = ( - setting_name is not None - and ai_ready() - and getattr(settings, setting_name) - ) - - -def draw_image_mask_operator_settings(layout, operator): - if operator.gesture not in {"BOX", "LASSO"}: - layout.prop(operator, "invert") - return - status = ai_status() - row = layout.row() - row.enabled = status["ready"] - row.prop(operator, "refine_selection") - layout.prop(operator, "invert") - if not status["ready"]: - layout.operator( - "anyimage.setup_ai_environment", - text=ai_setup_label(status), - icon="IMPORT", - ) - - -def draw_image_mask_settings(context, layout, setting_name): - settings = context.scene.anyimage_settings - status = ai_status() - row = layout.row() - row.enabled = status["ready"] - row.prop(settings, setting_name) - if not status["ready"]: - layout.operator( - "anyimage.setup_ai_environment", - text=ai_setup_label(status), - icon="IMPORT", - ) - - -def dashed_line_segments(vertices, dash_length=4.0, gap_length=4.0): - if dash_length <= 0.0 or gap_length < 0.0: - raise ValueError("Dash length must be positive and gap length cannot be negative") - pattern_length = dash_length + gap_length - segments = [] - distance_offset = 0.0 - for start, end in zip(vertices, vertices[1:]): - start_x, start_y = (float(value) for value in start) - end_x, end_y = (float(value) for value in end) - segment_length = math.hypot(end_x - start_x, end_y - start_y) - if segment_length <= 0.0: - continue - direction_x = (end_x - start_x) / segment_length - direction_y = (end_y - start_y) / segment_length - position = 0.0 - while position < segment_length: - phase = (distance_offset + position) % pattern_length - if phase < dash_length: - run_length = min(dash_length - phase, segment_length - position) - segments.extend( - ( - ( - start_x + direction_x * position, - start_y + direction_y * position, - ), - ( - start_x + direction_x * (position + run_length), - start_y + direction_y * (position + run_length), - ), - ) - ) - else: - run_length = min(pattern_length - phase, segment_length - position) - position += run_length - distance_offset += segment_length - return tuple(segments) - - -def draw_dashed_line( - vertices, - color=(1.0, 1.0, 1.0, 0.9), - segments=None, -): - import gpu - from gpu_extras.batch import batch_for_shader - - if segments is None: - segments = dashed_line_segments(vertices) - if not segments: - return - shader = gpu.shader.from_builtin("UNIFORM_COLOR") - batch = batch_for_shader(shader, "LINES", {"pos": segments}) - gpu.state.blend_set("ALPHA") - gpu.state.line_width_set(1.0) - shader.bind() - shader.uniform_float("color", color) - batch.draw(shader) - gpu.state.blend_set("NONE") - - -def _signed_area(vertices): - return sum( - start[0] * end[1] - end[0] * start[1] - for start, end in zip(vertices, (*vertices[1:], vertices[0])) - ) * 0.5 - - -def _triangle_cross(start, middle, end): - return (middle[0] - start[0]) * (end[1] - start[1]) - ( - middle[1] - start[1] - ) * (end[0] - start[0]) - - -def _point_in_triangle(point, start, middle, end, orientation, epsilon): - return all( - orientation * cross >= -epsilon - for cross in ( - _triangle_cross(start, middle, point), - _triangle_cross(middle, end, point), - _triangle_cross(end, start, point), - ) - ) - - -def _triangulated_polygon_vertices_python(vertices): - vertices = tuple(tuple(float(value) for value in vertex) for vertex in vertices) - if len(vertices) < 3: - return () - area = _signed_area(vertices) - if abs(area) <= 1e-8: - return () - orientation = 1.0 if area > 0.0 else -1.0 - remaining = list(range(len(vertices))) - triangles = [] - epsilon = 1e-8 - while len(remaining) > 3: - for position, current in enumerate(remaining): - previous = remaining[position - 1] - following = remaining[(position + 1) % len(remaining)] - start = vertices[previous] - middle = vertices[current] - end = vertices[following] - if orientation * _triangle_cross(start, middle, end) <= epsilon: - continue - if any( - _point_in_triangle( - vertices[index], - start, - middle, - end, - orientation, - epsilon, - ) - for index in remaining - if index not in {previous, current, following} - ): - continue - triangles.extend((start, middle, end)) - del remaining[position] - break - else: - return () - triangles.extend(vertices[index] for index in remaining) - return tuple(triangles) - - -def triangulated_polygon_vertices(vertices): - vertices = tuple(tuple(float(value) for value in vertex) for vertex in vertices) - try: - import mathutils - - points = [ - mathutils.Vector((vertex[0], vertex[1], 0.0)) for vertex in vertices - ] - triangles = mathutils.geometry.tessellate_polygon([points]) - return tuple( - (float(point[0]), float(point[1])) - for triangle in triangles - for point in triangle - ) - except (AttributeError, ImportError, RuntimeError, TypeError, ValueError): - return _triangulated_polygon_vertices_python(vertices) - - -def scanline_fill_triangles(vertices, fill_rule='EVEN_ODD', step=2.0): - vertices = tuple(tuple(float(value) for value in vertex) for vertex in vertices) - if len(vertices) < 3: - return () - if fill_rule not in {'EVEN_ODD', 'NONZERO'}: - raise ValueError(f'Unknown polygon fill rule: {fill_rule}') - minimum_y = min(vertex[1] for vertex in vertices) - maximum_y = max(vertex[1] for vertex in vertices) - band_bottom = minimum_y - triangles = [] - edges = tuple(zip(vertices, (*vertices[1:], vertices[0]))) - while band_bottom < maximum_y: - band_top = min(band_bottom + step, maximum_y) - sample_y = (band_bottom + band_top) * 0.5 - events = [] - for start, end in edges: - if (start[1] <= sample_y < end[1]) or ( - end[1] <= sample_y < start[1] - ): - factor = (sample_y - start[1]) / (end[1] - start[1]) - events.append( - ( - start[0] + (end[0] - start[0]) * factor, - 1 if end[1] > start[1] else -1, - ) - ) - events.sort(key=lambda event: event[0]) - intervals = [] - winding = 0 - interval_start = None - index = 0 - while index < len(events): - x = events[index][0] - delta = 0 - count = 0 - while index < len(events) and abs(events[index][0] - x) <= 1e-8: - delta += events[index][1] - count += 1 - index += 1 - was_inside = winding != 0 - if fill_rule == 'EVEN_ODD': - if count % 2: - winding = 0 if was_inside else 1 - else: - winding += delta - is_inside = winding != 0 - if not was_inside and is_inside: - interval_start = x - elif was_inside and not is_inside and interval_start is not None: - intervals.append((interval_start, x)) - interval_start = None - for left, right in intervals: - if right - left <= 1e-8: - continue - triangles.extend( - ( - (left, band_bottom), - (right, band_bottom), - (right, band_top), - (left, band_bottom), - (right, band_top), - (left, band_top), - ) - ) - band_bottom = band_top - return tuple(triangles) - - -def _point_on_segment(point, start, end, epsilon=1e-8): - return ( - min(start[0], end[0]) - epsilon - <= point[0] - <= max(start[0], end[0]) + epsilon - and min(start[1], end[1]) - epsilon - <= point[1] - <= max(start[1], end[1]) + epsilon - ) - - -def _segments_intersect(first_start, first_end, second_start, second_end): - crosses = ( - _triangle_cross(first_start, first_end, second_start), - _triangle_cross(first_start, first_end, second_end), - _triangle_cross(second_start, second_end, first_start), - _triangle_cross(second_start, second_end, first_end), - ) - if (crosses[0] > 1e-8) != (crosses[1] > 1e-8) and ( - crosses[2] > 1e-8 - ) != (crosses[3] > 1e-8): - return True - return any( - abs(cross) <= 1e-8 and _point_on_segment(point, start, end) - for cross, point, start, end in ( - (crosses[0], second_start, first_start, first_end), - (crosses[1], second_end, first_start, first_end), - (crosses[2], first_start, second_start, second_end), - (crosses[3], first_end, second_start, second_end), - ) - ) - - -def polygon_self_intersects(vertices, cell_size=32.0): - vertices = tuple(tuple(float(value) for value in vertex) for vertex in vertices) - if len(vertices) < 4: - return False - edges = tuple(zip(vertices, (*vertices[1:], vertices[0]))) - cells = {} - last_edge = len(edges) - 1 - for edge_index, (start, end) in enumerate(edges): - left = math.floor(min(start[0], end[0]) / cell_size) - right = math.floor(max(start[0], end[0]) / cell_size) - bottom = math.floor(min(start[1], end[1]) / cell_size) - top = math.floor(max(start[1], end[1]) / cell_size) - edge_cells = tuple( - (x, y) - for x in range(left, right + 1) - for y in range(bottom, top + 1) - ) - candidates = { - candidate - for cell in edge_cells - for candidate in cells.get(cell, ()) - } - for candidate in candidates: - if abs(edge_index - candidate) <= 1 or { - edge_index, - candidate, - } == {0, last_edge}: - continue - if _segments_intersect(start, end, *edges[candidate]): - return True - for cell in edge_cells: - cells.setdefault(cell, []).append(edge_index) - return False - - -def preview_fill_triangles(vertices, fill_rule='EVEN_ODD'): - if not polygon_self_intersects(vertices): - triangles = triangulated_polygon_vertices(vertices) - if triangles: - return triangles - return scanline_fill_triangles(vertices, fill_rule) - - -def draw_polygon_fill( - vertices, - triangles=None, - color=(1.0, 1.0, 1.0, 0.16), -): - import gpu - from gpu_extras.batch import batch_for_shader - - if triangles is None: - triangles = triangulated_polygon_vertices(vertices) - if not triangles: - return - shader = gpu.shader.from_builtin("UNIFORM_COLOR") - batch = batch_for_shader(shader, "TRIS", {"pos": triangles}) - gpu.state.blend_set("ALPHA") - shader.bind() - shader.uniform_float("color", color) - batch.draw(shader) - gpu.state.blend_set("NONE") diff --git a/src/anyimage/operators/common/image_gesture.py b/src/anyimage/operators/common/image_gesture.py deleted file mode 100644 index 1de148e..0000000 --- a/src/anyimage/operators/common/image_gesture.py +++ /dev/null @@ -1,420 +0,0 @@ -import json -import math - -import bpy - -from .image import is_image_empty, plane_bounds, video_total_frames -from .image_edit import ( - active_view3d_tool_id, - apply_image_mask_settings, - dashed_line_segments, - draw_dashed_line, - draw_image_mask_operator_settings, - draw_polygon_fill, - image_edit_poll, - polygon_bounds, - preview_fill_triangles, - report_image_edit_exception, - resolve_image_edit_click, -) -from .image_interaction import ( - deserialize_matrix, - drawing_in_region, - screen_path_to_image_pixels, - serialize_matrix, -) - -GESTURE_TOOL_IDS = { - "BOX": "anyimage.image_box", - "LASSO": "anyimage.image_lasso", - "LINE": "anyimage.image_line", - "POLYLINE": "anyimage.image_polyline", -} -LINE_SNAP_RADIANS = math.radians(5.0) -POLYLINE_CLOSE_RADIUS = 8.0 -LASSO_POINT_DISTANCE = 2.0 -LASSO_SIMPLIFY_TOLERANCE = 0.75 - - -def append_lasso_point(path, point): - point = (float(point[0]), float(point[1])) - if not path: - path.append(point) - return True - if math.dist(point, path[-1]) < LASSO_POINT_DISTANCE: - return False - if len(path) >= 2: - anchor = path[-2] - previous = path[-1] - segment_x = point[0] - anchor[0] - segment_y = point[1] - anchor[1] - segment_length_squared = segment_x * segment_x + segment_y * segment_y - if segment_length_squared > 0.0: - factor = ( - (previous[0] - anchor[0]) * segment_x - + (previous[1] - anchor[1]) * segment_y - ) / segment_length_squared - projected = ( - anchor[0] + segment_x * factor, - anchor[1] + segment_y * factor, - ) - forward = ( - (previous[0] - anchor[0]) * (point[0] - previous[0]) - + (previous[1] - anchor[1]) * (point[1] - previous[1]) - ) - if ( - 0.0 <= factor <= 1.0 - and forward >= 0.0 - and math.dist(previous, projected) <= LASSO_SIMPLIFY_TOLERANCE - ): - path[-1] = point - return True - path.append(point) - return True - -def gesture_property(): - return bpy.props.EnumProperty( - items=( - ("BOX", "Box", "Edit with a rectangular gesture"), - ("LASSO", "Lasso", "Edit with a freehand gesture"), - ("LINE", "Line", "Edit one side of a line"), - ("POLYLINE", "Polyline", "Edit with a clicked polygon"), - ), - default="BOX", - options={"HIDDEN", "SKIP_SAVE"}, - ) - - -def box_screen_path(start, end): - x0, y0 = start - x1, y1 = end - if abs(x1 - x0) < 2.0 or abs(y1 - y0) < 2.0: - return () - return ((x0, y0), (x1, y0), (x1, y1), (x0, y1)) - - -def snapped_line_endpoint(start, end, snap): - if not snap: - return tuple(float(value) for value in end) - delta_x = float(end[0]) - float(start[0]) - delta_y = float(end[1]) - float(start[1]) - length = math.hypot(delta_x, delta_y) - if length <= 0.0: - return tuple(float(value) for value in end) - angle = round(math.atan2(delta_y, delta_x) / LINE_SNAP_RADIANS) - angle *= LINE_SNAP_RADIANS - return ( - float(start[0]) + math.cos(angle) * length, - float(start[1]) + math.sin(angle) * length, - ) - - -def line_half_plane_polygon(start, end, rectangle_size): - """Clip a rectangle to the left side of one directed line.""" - width, height = (float(value) for value in rectangle_size) - if math.dist(start, end) < 2.0: - return () - vertices = [(0.0, 0.0), (width, 0.0), (width, height), (0.0, height)] - - def side(point): - return (end[0] - start[0]) * (point[1] - start[1]) - (end[1] - start[1]) * ( - point[0] - start[0] - ) - - result = [] - previous = vertices[-1] - previous_side = side(previous) - for current in vertices: - current_side = side(current) - if (current_side >= 0.0) != (previous_side >= 0.0): - factor = previous_side / (previous_side - current_side) - result.append( - ( - previous[0] + (current[0] - previous[0]) * factor, - previous[1] + (current[1] - previous[1]) * factor, - ) - ) - if current_side >= 0.0: - result.append(current) - previous = current - previous_side = current_side - return tuple(result) - - -def image_line_half_plane_polygon(line, image_size): - """Keep the image-space half-plane shown by the screen-space preview.""" - if len(line) != 2: - raise ValueError("Drag a longer line across the image plane") - return line_half_plane_polygon(line[1], line[0], image_size) - - -class ImageMaskGesture: - active_tool_id = None - preview_fill_rule = 'EVEN_ODD' - _path = None - _cursor = None - _handle = None - _fill_polygon = None - _fill_triangles = () - _outline_polygon = None - _outline_segments = () - _area_pointer = 0 - _region_pointer = 0 - _region_size = (0, 0) - - def draw(self, _context): - draw_image_mask_operator_settings(self.layout, self) - - @classmethod - def poll(cls, context): - return image_edit_poll(context) - - def _screen_polygon(self): - if self.gesture == "BOX": - return box_screen_path(self._path[0], self._cursor) - if self.gesture == "LASSO": - return tuple(self._path) - if self.gesture == "LINE": - return line_half_plane_polygon( - self._path[0], self._cursor, self._region_size - ) - vertices = list(self._path) - if self._cursor is not None: - vertices.append(self._cursor) - return tuple(vertices) - - def _draw_selection(self): - if not self._path or not drawing_in_region( - self._area_pointer, self._region_pointer - ): - return - polygon = self._screen_polygon() - if polygon != self._outline_polygon: - self._outline_polygon = polygon - if self.gesture == 'LINE': - outline = (self._path[0], self._cursor) - else: - outline = (*polygon, polygon[0]) if len(polygon) > 1 else () - self._outline_segments = dashed_line_segments(outline) - if len(polygon) >= 3: - if polygon != self._fill_polygon: - self._fill_polygon = polygon - self._fill_triangles = preview_fill_triangles( - polygon, self.preview_fill_rule - ) - draw_polygon_fill( - polygon, - self._fill_triangles, - (1.0, 0.25, 0.2, 0.16) if self.invert else (1.0, 1.0, 1.0, 0.16), - ) - if self.gesture == "LINE": - draw_dashed_line((), segments=self._outline_segments) - return - if len(polygon) > 1: - draw_dashed_line((), segments=self._outline_segments) - if ( - self.gesture == "POLYLINE" - and len(self._path) >= 3 - and self._cursor is not None - and math.dist(self._cursor, self._path[0]) <= POLYLINE_CLOSE_RADIUS - ): - self._draw_polyline_close_indicator() - - def _draw_polyline_close_indicator(self): - import gpu - from gpu_extras.batch import batch_for_shader - - center_x, center_y = self._path[0] - vertices = tuple( - ( - center_x + math.cos(index * math.tau / 24.0) * POLYLINE_CLOSE_RADIUS, - center_y + math.sin(index * math.tau / 24.0) * POLYLINE_CLOSE_RADIUS, - ) - for index in range(25) - ) - shader = gpu.shader.from_builtin("UNIFORM_COLOR") - batch = batch_for_shader(shader, "LINE_STRIP", {"pos": vertices}) - gpu.state.blend_set("ALPHA") - gpu.state.line_width_set(2.0) - shader.bind() - shader.uniform_float("color", (1.0, 1.0, 1.0, 0.95)) - batch.draw(shader) - gpu.state.line_width_set(1.0) - gpu.state.blend_set("NONE") - - def _set_status(self, context): - invert = "On" if self.invert else "Off" - if self.gesture == "POLYLINE": - action = "click points; click start or Enter to finish; Backspace removes" - elif self.gesture == "LINE": - action = "drag line; Ctrl snaps 5°; release to edit" - else: - action = "drag and release to edit" - context.workspace.status_text_set( - f"Image {self.gesture.title()}: {action}; F toggles Invert ({invert}); " - "RMB or Esc cancels" - ) - - def _finish(self, context): - if self._handle is not None: - bpy.types.SpaceView3D.draw_handler_remove(self._handle, "WINDOW") - self._handle = None - context.workspace.status_text_set(None) - if context.area is not None: - context.area.tag_redraw() - - def cancel(self, context): - self._finish(context) - - def invoke(self, context, event): - source_object, should_edit = resolve_image_edit_click( - context, event, self.report - ) - if not should_edit: - return {"FINISHED"} if source_object is not None else {"CANCELLED"} - if source_object is None: - return {"CANCELLED"} - if video_total_frames(source_object.data) > 1: - self.report({"WARNING"}, "Image tools only support a still image") - return {"CANCELLED"} - apply_image_mask_settings(self, context) - self.invert = False - self.source_object_name = source_object.name - self.source_matrix_data = serialize_matrix(source_object.matrix_world) - self.selection_data = "" - point = (float(event.mouse_region_x), float(event.mouse_region_y)) - self._path = [point] - self._cursor = point - self._fill_polygon = None - self._fill_triangles = () - self._outline_polygon = None - self._outline_segments = () - self._area_pointer = context.area.as_pointer() - self._region_pointer = context.region.as_pointer() - self._region_size = (context.region.width, context.region.height) - self._handle = bpy.types.SpaceView3D.draw_handler_add( - self._draw_selection, (), "WINDOW", "POST_PIXEL" - ) - self._set_status(context) - context.window_manager.modal_handler_add(self) - context.area.tag_redraw() - return {"RUNNING_MODAL"} - - def _complete(self, context): - source_object = bpy.data.objects.get(self.source_object_name) - if not is_image_empty(source_object): - raise RuntimeError("The source Image Empty is no longer available") - matrix = deserialize_matrix(self.source_matrix_data) - if self.gesture == "LINE": - line = screen_path_to_image_pixels( - (self._path[0], self._cursor), - context.region, - context.region_data, - matrix, - plane_bounds(source_object), - tuple(source_object.data.size), - ) - polygon = image_line_half_plane_polygon( - line, tuple(source_object.data.size) - ) - else: - path = self._screen_polygon() - if len(path) < 3: - self._finish(context) - return {"CANCELLED"} - polygon = screen_path_to_image_pixels( - path, - context.region, - context.region_data, - matrix, - plane_bounds(source_object), - tuple(source_object.data.size), - ) - polygon_bounds(polygon) - self.selection_data = json.dumps(polygon, separators=(",", ":")) - self._finish(context) - return self.execute(context) - - def modal(self, context, event): - active_tool = active_view3d_tool_id(context) - expected_tool = self.active_tool_id or GESTURE_TOOL_IDS[self.gesture] - if active_tool is not None and active_tool != expected_tool: - self._finish(context) - return {"CANCELLED"} - if event.type in {"ESC", "RIGHTMOUSE"} and event.value == "PRESS": - self._finish(context) - return {"CANCELLED"} - if event.type == "F" and event.value == "PRESS": - self.invert = not self.invert - self._set_status(context) - context.area.tag_redraw() - return {"RUNNING_MODAL"} - if self.gesture == "POLYLINE": - return self._modal_polyline(context, event) - if event.type == "MOUSEMOVE": - point = (float(event.mouse_region_x), float(event.mouse_region_y)) - if self.gesture == "LASSO": - append_lasso_point(self._path, point) - self._cursor = point - elif self.gesture == "LINE": - self._cursor = snapped_line_endpoint(self._path[0], point, event.ctrl) - else: - self._cursor = point - context.area.tag_redraw() - return {"RUNNING_MODAL"} - if event.type == "LEFTMOUSE" and event.value == "RELEASE": - point = (float(event.mouse_region_x), float(event.mouse_region_y)) - if self.gesture == "LASSO": - append_lasso_point(self._path, point) - self._cursor = point - elif self.gesture == "LINE": - self._cursor = snapped_line_endpoint(self._path[0], point, event.ctrl) - if math.dist(self._path[0], self._cursor) < 2.0: - self._finish(context) - return {"CANCELLED"} - else: - self._cursor = point - try: - return self._complete(context) - except (RuntimeError, TypeError, ValueError) as error: - report_image_edit_exception(self, error) - self._finish(context) - return {"CANCELLED"} - return {"PASS_THROUGH"} - - def _modal_polyline(self, context, event): - if event.type == "BACK_SPACE" and event.value == "PRESS": - if len(self._path) > 1: - self._path.pop() - context.area.tag_redraw() - return {"RUNNING_MODAL"} - if event.type in {"RET", "NUMPAD_ENTER"} and event.value == "PRESS": - return self._confirm_polyline(context) - if event.type == "MOUSEMOVE": - self._cursor = ( - float(event.mouse_region_x), - float(event.mouse_region_y), - ) - context.area.tag_redraw() - return {"RUNNING_MODAL"} - if event.type == "LEFTMOUSE" and event.value == "PRESS": - point = (float(event.mouse_region_x), float(event.mouse_region_y)) - if ( - len(self._path) >= 3 - and math.dist(point, self._path[0]) <= POLYLINE_CLOSE_RADIUS - ): - return self._confirm_polyline(context) - if math.dist(point, self._path[-1]) >= 2.0: - self._path.append(point) - context.area.tag_redraw() - return {"RUNNING_MODAL"} - return {"PASS_THROUGH"} - - def _confirm_polyline(self, context): - self._cursor = self._path[0] - try: - return self._complete(context) - except (RuntimeError, TypeError, ValueError) as error: - report_image_edit_exception(self, error) - self._finish(context) - return {"CANCELLED"} diff --git a/src/anyimage/operators/common/image_interaction.py b/src/anyimage/operators/common/image_interaction.py index c78691a..5f8deaa 100644 --- a/src/anyimage/operators/common/image_interaction.py +++ b/src/anyimage/operators/common/image_interaction.py @@ -3,6 +3,14 @@ import math import bpy +from ...properties import ai_ready, ai_setup_label, ai_status +from .image_data import ( + image_empty_bounds, + is_image_empty, + require_image_empty, + video_total_frames, +) +from .image_region import ImageEditWarning, RegionPath MIN_PATH_DISTANCE = 2.0 @@ -18,8 +26,8 @@ def simplify_screen_path(path): def screen_path_to_image_pixels( path, - region, - region_data, + view_region, + view3d, matrix_world, local_bounds, image_size, @@ -34,13 +42,13 @@ def screen_path_to_image_pixels( pixels = [] for point in simplify_screen_path(path): ray_origin = inverse @ view3d_utils.region_2d_to_origin_3d( - region, - region_data, + view_region, + view3d, point, ) ray_direction = inverse.to_3x3() @ view3d_utils.region_2d_to_vector_3d( - region, - region_data, + view_region, + view3d, point, ) if abs(ray_direction.z) <= 1e-10: @@ -72,10 +80,6 @@ def deserialize_matrix(data): return Matrix(tuple(tuple(values[row * 4 : row * 4 + 4]) for row in range(4))) -def hidden_string_property(): - return bpy.props.StringProperty(options={"HIDDEN", "SKIP_SAVE"}) - - def drawing_in_region(area_pointer, region_pointer): area = getattr(bpy.context, "area", None) region = getattr(bpy.context, "region", None) @@ -85,3 +89,817 @@ def drawing_in_region(area_pointer, region_pointer): and area.as_pointer() == area_pointer and region.as_pointer() == region_pointer ) + + +def report_image_edit_exception(operator, error): + level = "WARNING" if isinstance(error, ImageEditWarning) else "ERROR" + operator.report({level}, str(error)) + + +def image_edit_poll(context): + return context.area is not None and context.area.type == "VIEW_3D" + + +def _restore_selection(context, active_object, selected_objects): + bpy.ops.object.select_all(action="DESELECT") + for selected in selected_objects: + if bpy.data.objects.get(selected.name) is not None: + selected.select_set(True) + if ( + active_object is not None + and bpy.data.objects.get(active_object.name) is not None + ): + context.view_layer.objects.active = active_object + + +def resolve_image_edit_click(context, event, report): + """Resolve the first click into source selection or image editing.""" + active_object = context.active_object + selected_objects = tuple(context.selected_objects) + try: + bpy.ops.view3d.select( + location=(event.mouse_region_x, event.mouse_region_y), + extend=False, + deselect_all=False, + ) + except RuntimeError: + pass + clicked_object = context.active_object + if is_image_empty(clicked_object) and clicked_object != active_object: + return clicked_object, False + if is_image_empty(clicked_object): + return clicked_object, True + if is_image_empty(active_object): + _restore_selection(context, active_object, selected_objects) + return active_object, True + report({"WARNING"}, "Click an Image Empty to use an Image tool") + return None, False + + +def active_view3d_tool_id(context): + tools = getattr(getattr(context, "workspace", None), "tools", None) + from_mode = getattr(tools, "from_space_view3d_mode", None) + if not callable(from_mode): + return None + try: + tool = from_mode(getattr(context, "mode", "OBJECT"), create=False) + except (RuntimeError, TypeError): + return None + return getattr(tool, "idname", None) + + +def refine_selection_setting_name(operator): + if getattr(operator, "active_tool_id", None) == "anyimage.cutout_lasso": + return "cutout_refine_selection" + return { + "BOX": "image_box_refine_selection", + "LASSO": "image_lasso_refine_selection", + }.get(operator.gesture) + + +def apply_region_settings(operator, context): + settings = context.scene.anyimage_settings + setting_name = refine_selection_setting_name(operator) + operator.refine_selection = ( + setting_name is not None + and ai_ready() + and getattr(settings, setting_name) + ) + + +def draw_region_operator_settings(layout, operator): + if operator.gesture not in {"BOX", "LASSO"}: + layout.prop(operator, "invert") + return + status = ai_status() + row = layout.row() + row.enabled = status["ready"] + row.prop(operator, "refine_selection") + layout.prop(operator, "invert") + if not status["ready"]: + layout.operator( + "anyimage.setup_ai_environment", + text=ai_setup_label(status), + icon="IMPORT", + ) + + +def draw_region_settings(context, layout, setting_name): + settings = context.scene.anyimage_settings + status = ai_status() + row = layout.row() + row.enabled = status["ready"] + row.prop(settings, setting_name) + if not status["ready"]: + layout.operator( + "anyimage.setup_ai_environment", + text=ai_setup_label(status), + icon="IMPORT", + ) + + +def dashed_line_segments(vertices, dash_length=4.0, gap_length=4.0): + if dash_length <= 0.0 or gap_length < 0.0: + raise ValueError("Dash length must be positive and gap length cannot be negative") + pattern_length = dash_length + gap_length + segments = [] + distance_offset = 0.0 + for start, end in zip(vertices, vertices[1:]): + start_x, start_y = (float(value) for value in start) + end_x, end_y = (float(value) for value in end) + segment_length = math.hypot(end_x - start_x, end_y - start_y) + if segment_length <= 0.0: + continue + direction_x = (end_x - start_x) / segment_length + direction_y = (end_y - start_y) / segment_length + position = 0.0 + while position < segment_length: + phase = (distance_offset + position) % pattern_length + if phase < dash_length: + run_length = min(dash_length - phase, segment_length - position) + segments.extend( + ( + ( + start_x + direction_x * position, + start_y + direction_y * position, + ), + ( + start_x + direction_x * (position + run_length), + start_y + direction_y * (position + run_length), + ), + ) + ) + else: + run_length = min(pattern_length - phase, segment_length - position) + position += run_length + distance_offset += segment_length + return tuple(segments) + + +def draw_dashed_line( + vertices, + color=(1.0, 1.0, 1.0, 0.9), + segments=None, +): + import gpu + from gpu_extras.batch import batch_for_shader + + if segments is None: + segments = dashed_line_segments(vertices) + if not segments: + return + shader = gpu.shader.from_builtin("UNIFORM_COLOR") + batch = batch_for_shader(shader, "LINES", {"pos": segments}) + gpu.state.blend_set("ALPHA") + gpu.state.line_width_set(1.0) + shader.bind() + shader.uniform_float("color", color) + batch.draw(shader) + gpu.state.blend_set("NONE") + + +def _signed_area(vertices): + return sum( + start[0] * end[1] - end[0] * start[1] + for start, end in zip(vertices, (*vertices[1:], vertices[0])) + ) * 0.5 + + +def _triangle_cross(start, middle, end): + return (middle[0] - start[0]) * (end[1] - start[1]) - ( + middle[1] - start[1] + ) * (end[0] - start[0]) + + +def _point_in_triangle(point, start, middle, end, orientation, epsilon): + return all( + orientation * cross >= -epsilon + for cross in ( + _triangle_cross(start, middle, point), + _triangle_cross(middle, end, point), + _triangle_cross(end, start, point), + ) + ) + + +def _triangulated_polygon_vertices_python(vertices): + vertices = tuple(tuple(float(value) for value in vertex) for vertex in vertices) + if len(vertices) < 3: + return () + area = _signed_area(vertices) + if abs(area) <= 1e-8: + return () + orientation = 1.0 if area > 0.0 else -1.0 + remaining = list(range(len(vertices))) + triangles = [] + epsilon = 1e-8 + while len(remaining) > 3: + for position, current in enumerate(remaining): + previous = remaining[position - 1] + following = remaining[(position + 1) % len(remaining)] + start = vertices[previous] + middle = vertices[current] + end = vertices[following] + if orientation * _triangle_cross(start, middle, end) <= epsilon: + continue + if any( + _point_in_triangle( + vertices[index], + start, + middle, + end, + orientation, + epsilon, + ) + for index in remaining + if index not in {previous, current, following} + ): + continue + triangles.extend((start, middle, end)) + del remaining[position] + break + else: + return () + triangles.extend(vertices[index] for index in remaining) + return tuple(triangles) + + +def triangulated_polygon_vertices(vertices): + vertices = tuple(tuple(float(value) for value in vertex) for vertex in vertices) + try: + import mathutils + + points = [ + mathutils.Vector((vertex[0], vertex[1], 0.0)) for vertex in vertices + ] + triangles = mathutils.geometry.tessellate_polygon([points]) + return tuple( + (float(point[0]), float(point[1])) + for triangle in triangles + for point in triangle + ) + except (AttributeError, ImportError, RuntimeError, TypeError, ValueError): + return _triangulated_polygon_vertices_python(vertices) + + +def scanline_fill_triangles(vertices, fill_rule='EVEN_ODD', step=2.0): + vertices = tuple(tuple(float(value) for value in vertex) for vertex in vertices) + if len(vertices) < 3: + return () + if fill_rule not in {'EVEN_ODD', 'NONZERO'}: + raise ValueError(f'Unknown polygon fill rule: {fill_rule}') + minimum_y = min(vertex[1] for vertex in vertices) + maximum_y = max(vertex[1] for vertex in vertices) + band_bottom = minimum_y + triangles = [] + edges = tuple(zip(vertices, (*vertices[1:], vertices[0]))) + while band_bottom < maximum_y: + band_top = min(band_bottom + step, maximum_y) + sample_y = (band_bottom + band_top) * 0.5 + events = [] + for start, end in edges: + if (start[1] <= sample_y < end[1]) or ( + end[1] <= sample_y < start[1] + ): + factor = (sample_y - start[1]) / (end[1] - start[1]) + events.append( + ( + start[0] + (end[0] - start[0]) * factor, + 1 if end[1] > start[1] else -1, + ) + ) + events.sort(key=lambda event: event[0]) + intervals = [] + winding = 0 + interval_start = None + index = 0 + while index < len(events): + x = events[index][0] + delta = 0 + count = 0 + while index < len(events) and abs(events[index][0] - x) <= 1e-8: + delta += events[index][1] + count += 1 + index += 1 + was_inside = winding != 0 + if fill_rule == 'EVEN_ODD': + if count % 2: + winding = 0 if was_inside else 1 + else: + winding += delta + is_inside = winding != 0 + if not was_inside and is_inside: + interval_start = x + elif was_inside and not is_inside and interval_start is not None: + intervals.append((interval_start, x)) + interval_start = None + for left, right in intervals: + if right - left <= 1e-8: + continue + triangles.extend( + ( + (left, band_bottom), + (right, band_bottom), + (right, band_top), + (left, band_bottom), + (right, band_top), + (left, band_top), + ) + ) + band_bottom = band_top + return tuple(triangles) + + +def _point_on_segment(point, start, end, epsilon=1e-8): + return ( + min(start[0], end[0]) - epsilon + <= point[0] + <= max(start[0], end[0]) + epsilon + and min(start[1], end[1]) - epsilon + <= point[1] + <= max(start[1], end[1]) + epsilon + ) + + +def _segments_intersect(first_start, first_end, second_start, second_end): + crosses = ( + _triangle_cross(first_start, first_end, second_start), + _triangle_cross(first_start, first_end, second_end), + _triangle_cross(second_start, second_end, first_start), + _triangle_cross(second_start, second_end, first_end), + ) + if (crosses[0] > 1e-8) != (crosses[1] > 1e-8) and ( + crosses[2] > 1e-8 + ) != (crosses[3] > 1e-8): + return True + return any( + abs(cross) <= 1e-8 and _point_on_segment(point, start, end) + for cross, point, start, end in ( + (crosses[0], second_start, first_start, first_end), + (crosses[1], second_end, first_start, first_end), + (crosses[2], first_start, second_start, second_end), + (crosses[3], first_end, second_start, second_end), + ) + ) + + +def polygon_self_intersects(vertices, cell_size=32.0): + vertices = tuple(tuple(float(value) for value in vertex) for vertex in vertices) + if len(vertices) < 4: + return False + edges = tuple(zip(vertices, (*vertices[1:], vertices[0]))) + cells = {} + last_edge = len(edges) - 1 + for edge_index, (start, end) in enumerate(edges): + left = math.floor(min(start[0], end[0]) / cell_size) + right = math.floor(max(start[0], end[0]) / cell_size) + bottom = math.floor(min(start[1], end[1]) / cell_size) + top = math.floor(max(start[1], end[1]) / cell_size) + edge_cells = tuple( + (x, y) + for x in range(left, right + 1) + for y in range(bottom, top + 1) + ) + candidates = { + candidate + for cell in edge_cells + for candidate in cells.get(cell, ()) + } + for candidate in candidates: + if abs(edge_index - candidate) <= 1 or { + edge_index, + candidate, + } == {0, last_edge}: + continue + if _segments_intersect(start, end, *edges[candidate]): + return True + for cell in edge_cells: + cells.setdefault(cell, []).append(edge_index) + return False + + +def preview_fill_triangles(vertices, fill_rule='EVEN_ODD'): + if not polygon_self_intersects(vertices): + triangles = triangulated_polygon_vertices(vertices) + if triangles: + return triangles + return scanline_fill_triangles(vertices, fill_rule) + + +def draw_polygon_fill( + vertices, + triangles=None, + color=(1.0, 1.0, 1.0, 0.16), +): + import gpu + from gpu_extras.batch import batch_for_shader + + if triangles is None: + triangles = triangulated_polygon_vertices(vertices) + if not triangles: + return + shader = gpu.shader.from_builtin("UNIFORM_COLOR") + batch = batch_for_shader(shader, "TRIS", {"pos": triangles}) + gpu.state.blend_set("ALPHA") + shader.bind() + shader.uniform_float("color", color) + batch.draw(shader) + gpu.state.blend_set("NONE") + + +GESTURE_TOOL_IDS = { + "BOX": "anyimage.image_box", + "LASSO": "anyimage.image_lasso", + "LINE": "anyimage.image_line", + "POLYLINE": "anyimage.image_polyline", +} +LINE_SNAP_RADIANS = math.radians(5.0) +POLYLINE_CLOSE_RADIUS = 8.0 +LASSO_POINT_DISTANCE = 2.0 +LASSO_SIMPLIFY_TOLERANCE = 0.75 + + +def append_lasso_point(path, point): + point = (float(point[0]), float(point[1])) + if not path: + path.append(point) + return True + if math.dist(point, path[-1]) < LASSO_POINT_DISTANCE: + return False + if len(path) >= 2: + anchor = path[-2] + previous = path[-1] + segment_x = point[0] - anchor[0] + segment_y = point[1] - anchor[1] + segment_length_squared = segment_x * segment_x + segment_y * segment_y + if segment_length_squared > 0.0: + factor = ( + (previous[0] - anchor[0]) * segment_x + + (previous[1] - anchor[1]) * segment_y + ) / segment_length_squared + projected = ( + anchor[0] + segment_x * factor, + anchor[1] + segment_y * factor, + ) + forward = ( + (previous[0] - anchor[0]) * (point[0] - previous[0]) + + (previous[1] - anchor[1]) * (point[1] - previous[1]) + ) + if ( + 0.0 <= factor <= 1.0 + and forward >= 0.0 + and math.dist(previous, projected) <= LASSO_SIMPLIFY_TOLERANCE + ): + path[-1] = point + return True + path.append(point) + return True + + +def gesture_property(): + return bpy.props.EnumProperty( + items=( + ("BOX", "Box", "Edit with a rectangular gesture"), + ("LASSO", "Lasso", "Edit with a freehand gesture"), + ("LINE", "Line", "Edit one side of a line"), + ("POLYLINE", "Polyline", "Edit with a clicked polygon"), + ), + default="BOX", + options={"HIDDEN", "SKIP_SAVE"}, + ) + + +def box_screen_path(start, end): + x0, y0 = start + x1, y1 = end + if abs(x1 - x0) < 2.0 or abs(y1 - y0) < 2.0: + return () + return ((x0, y0), (x1, y0), (x1, y1), (x0, y1)) + + +def snapped_line_endpoint(start, end, snap): + if not snap: + return tuple(float(value) for value in end) + delta_x = float(end[0]) - float(start[0]) + delta_y = float(end[1]) - float(start[1]) + length = math.hypot(delta_x, delta_y) + if length <= 0.0: + return tuple(float(value) for value in end) + angle = round(math.atan2(delta_y, delta_x) / LINE_SNAP_RADIANS) + angle *= LINE_SNAP_RADIANS + return ( + float(start[0]) + math.cos(angle) * length, + float(start[1]) + math.sin(angle) * length, + ) + + +def line_half_plane_polygon(start, end, rectangle_size): + """Clip a rectangle to the left side of one directed line.""" + width, height = (float(value) for value in rectangle_size) + if math.dist(start, end) < 2.0: + return () + vertices = [(0.0, 0.0), (width, 0.0), (width, height), (0.0, height)] + + def side(point): + return (end[0] - start[0]) * (point[1] - start[1]) - (end[1] - start[1]) * ( + point[0] - start[0] + ) + + result = [] + previous = vertices[-1] + previous_side = side(previous) + for current in vertices: + current_side = side(current) + if (current_side >= 0.0) != (previous_side >= 0.0): + factor = previous_side / (previous_side - current_side) + result.append( + ( + previous[0] + (current[0] - previous[0]) * factor, + previous[1] + (current[1] - previous[1]) * factor, + ) + ) + if current_side >= 0.0: + result.append(current) + previous = current + previous_side = current_side + return tuple(result) + + +def image_line_half_plane_polygon(line, image_size): + """Keep the image-space half-plane shown by the screen-space preview.""" + if len(line) != 2: + raise ValueError("Drag a longer line across the image plane") + return line_half_plane_polygon(line[1], line[0], image_size) + + +class ImageRegionGesture: + active_tool_id = None + preview_fill_rule = 'EVEN_ODD' + _path = None + _cursor = None + _handle = None + _fill_polygon = None + _fill_triangles = () + _outline_polygon = None + _outline_segments = () + _area_pointer = 0 + _region_pointer = 0 + _region_size = (0, 0) + + def draw(self, _context): + draw_region_operator_settings(self.layout, self) + + @classmethod + def poll(cls, context): + return image_edit_poll(context) + + def _screen_path(self): + if self.gesture == "BOX": + return box_screen_path(self._path[0], self._cursor) + if self.gesture == "LASSO": + return tuple(self._path) + if self.gesture == "LINE": + return line_half_plane_polygon( + self._path[0], self._cursor, self._region_size + ) + vertices = list(self._path) + if self._cursor is not None: + vertices.append(self._cursor) + return tuple(vertices) + + def _draw_region(self): + if not self._path or not drawing_in_region( + self._area_pointer, self._region_pointer + ): + return + polygon = self._screen_path() + if polygon != self._outline_polygon: + self._outline_polygon = polygon + if self.gesture == 'LINE': + outline = (self._path[0], self._cursor) + else: + outline = (*polygon, polygon[0]) if len(polygon) > 1 else () + self._outline_segments = dashed_line_segments(outline) + if len(polygon) >= 3: + if polygon != self._fill_polygon: + self._fill_polygon = polygon + self._fill_triangles = preview_fill_triangles( + polygon, self.preview_fill_rule + ) + draw_polygon_fill( + polygon, + self._fill_triangles, + (1.0, 0.25, 0.2, 0.16) if self.invert else (1.0, 1.0, 1.0, 0.16), + ) + if self.gesture == "LINE": + draw_dashed_line((), segments=self._outline_segments) + return + if len(polygon) > 1: + draw_dashed_line((), segments=self._outline_segments) + if ( + self.gesture == "POLYLINE" + and len(self._path) >= 3 + and self._cursor is not None + and math.dist(self._cursor, self._path[0]) <= POLYLINE_CLOSE_RADIUS + ): + self._draw_polyline_close_indicator() + + def _draw_polyline_close_indicator(self): + import gpu + from gpu_extras.batch import batch_for_shader + + center_x, center_y = self._path[0] + vertices = tuple( + ( + center_x + math.cos(index * math.tau / 24.0) * POLYLINE_CLOSE_RADIUS, + center_y + math.sin(index * math.tau / 24.0) * POLYLINE_CLOSE_RADIUS, + ) + for index in range(25) + ) + shader = gpu.shader.from_builtin("UNIFORM_COLOR") + batch = batch_for_shader(shader, "LINE_STRIP", {"pos": vertices}) + gpu.state.blend_set("ALPHA") + gpu.state.line_width_set(2.0) + shader.bind() + shader.uniform_float("color", (1.0, 1.0, 1.0, 0.95)) + batch.draw(shader) + gpu.state.line_width_set(1.0) + gpu.state.blend_set("NONE") + + def _set_status(self, context): + invert = "On" if self.invert else "Off" + if self.gesture == "POLYLINE": + action = "click points; click start or Enter to finish; Backspace removes" + elif self.gesture == "LINE": + action = "drag line; Ctrl snaps 5°; release to edit" + else: + action = "drag and release to edit" + context.workspace.status_text_set( + f"Image {self.gesture.title()}: {action}; F toggles Invert ({invert}); " + "RMB or Esc cancels" + ) + + def _finish(self, context): + if self._handle is not None: + bpy.types.SpaceView3D.draw_handler_remove(self._handle, "WINDOW") + self._handle = None + context.workspace.status_text_set(None) + if context.area is not None: + context.area.tag_redraw() + + def cancel(self, context): + self._finish(context) + + def invoke(self, context, event): + source_object, should_edit = resolve_image_edit_click( + context, event, self.report + ) + if not should_edit: + return {"FINISHED"} if source_object is not None else {"CANCELLED"} + if source_object is None: + return {"CANCELLED"} + if video_total_frames(source_object.data) > 1: + self.report({"WARNING"}, "Image tools only support a still image") + return {"CANCELLED"} + apply_region_settings(self, context) + self.invert = False + self.source_object_name = source_object.name + self.source_matrix_data = serialize_matrix(source_object.matrix_world) + self.region_path_json = "" + point = (float(event.mouse_region_x), float(event.mouse_region_y)) + self._path = [point] + self._cursor = point + self._fill_polygon = None + self._fill_triangles = () + self._outline_polygon = None + self._outline_segments = () + self._area_pointer = context.area.as_pointer() + self._region_pointer = context.region.as_pointer() + self._region_size = (context.region.width, context.region.height) + self._handle = bpy.types.SpaceView3D.draw_handler_add( + self._draw_region, (), "WINDOW", "POST_PIXEL" + ) + self._set_status(context) + context.window_manager.modal_handler_add(self) + context.area.tag_redraw() + return {"RUNNING_MODAL"} + + def _complete(self, context): + source_object = require_image_empty(self.source_object_name) + matrix = deserialize_matrix(self.source_matrix_data) + if self.gesture == "LINE": + line = screen_path_to_image_pixels( + (self._path[0], self._cursor), + context.region, + context.region_data, + matrix, + image_empty_bounds(source_object), + tuple(source_object.data.size), + ) + region_path = image_line_half_plane_polygon( + line, tuple(source_object.data.size) + ) + else: + path = self._screen_path() + if len(path) < 3: + self._finish(context) + return {"CANCELLED"} + region_path = screen_path_to_image_pixels( + path, + context.region, + context.region_data, + matrix, + image_empty_bounds(source_object), + tuple(source_object.data.size), + ) + self.region_path_json = RegionPath( + points=region_path, + invert=self.invert, + fill_rule=self.preview_fill_rule, + ).to_json() + self._finish(context) + return self.execute(context) + + def modal(self, context, event): + active_tool = active_view3d_tool_id(context) + expected_tool = self.active_tool_id or GESTURE_TOOL_IDS[self.gesture] + if active_tool is not None and active_tool != expected_tool: + self._finish(context) + return {"CANCELLED"} + if event.type in {"ESC", "RIGHTMOUSE"} and event.value == "PRESS": + self._finish(context) + return {"CANCELLED"} + if event.type == "F" and event.value == "PRESS": + self.invert = not self.invert + self._set_status(context) + context.area.tag_redraw() + return {"RUNNING_MODAL"} + if self.gesture == "POLYLINE": + return self._modal_polyline(context, event) + if event.type == "MOUSEMOVE": + point = (float(event.mouse_region_x), float(event.mouse_region_y)) + if self.gesture == "LASSO": + append_lasso_point(self._path, point) + self._cursor = point + elif self.gesture == "LINE": + self._cursor = snapped_line_endpoint(self._path[0], point, event.ctrl) + else: + self._cursor = point + context.area.tag_redraw() + return {"RUNNING_MODAL"} + if event.type == "LEFTMOUSE" and event.value == "RELEASE": + point = (float(event.mouse_region_x), float(event.mouse_region_y)) + if self.gesture == "LASSO": + append_lasso_point(self._path, point) + self._cursor = point + elif self.gesture == "LINE": + self._cursor = snapped_line_endpoint(self._path[0], point, event.ctrl) + if math.dist(self._path[0], self._cursor) < 2.0: + self._finish(context) + return {"CANCELLED"} + else: + self._cursor = point + try: + return self._complete(context) + except (RuntimeError, TypeError, ValueError) as error: + report_image_edit_exception(self, error) + self._finish(context) + return {"CANCELLED"} + return {"PASS_THROUGH"} + + def _modal_polyline(self, context, event): + if event.type == "BACK_SPACE" and event.value == "PRESS": + if len(self._path) > 1: + self._path.pop() + context.area.tag_redraw() + return {"RUNNING_MODAL"} + if event.type in {"RET", "NUMPAD_ENTER"} and event.value == "PRESS": + return self._confirm_polyline(context) + if event.type == "MOUSEMOVE": + self._cursor = ( + float(event.mouse_region_x), + float(event.mouse_region_y), + ) + context.area.tag_redraw() + return {"RUNNING_MODAL"} + if event.type == "LEFTMOUSE" and event.value == "PRESS": + point = (float(event.mouse_region_x), float(event.mouse_region_y)) + if ( + len(self._path) >= 3 + and math.dist(point, self._path[0]) <= POLYLINE_CLOSE_RADIUS + ): + return self._confirm_polyline(context) + if math.dist(point, self._path[-1]) >= 2.0: + self._path.append(point) + context.area.tag_redraw() + return {"RUNNING_MODAL"} + return {"PASS_THROUGH"} + + def _confirm_polyline(self, context): + self._cursor = self._path[0] + try: + return self._complete(context) + except (RuntimeError, TypeError, ValueError) as error: + report_image_edit_exception(self, error) + self._finish(context) + return {"CANCELLED"} diff --git a/src/anyimage/operators/common/image_mask.py b/src/anyimage/operators/common/image_mask.py deleted file mode 100644 index 828d68b..0000000 --- a/src/anyimage/operators/common/image_mask.py +++ /dev/null @@ -1,141 +0,0 @@ -from .image_edit import ( - ImageEditWarning, - crop_pixels_to_alpha, - polygon_bounds, - replace_empty_image_pixels, -) - -def polygon_mask(image_size, polygon, fill_rule="EVEN_ODD"): - import numpy as np - - width, height = (int(value) for value in image_size) - if width <= 0 or height <= 0: - raise ValueError("The source image has no readable dimensions") - if fill_rule not in {"EVEN_ODD", "NONZERO"}: - raise ValueError(f"Unknown polygon fill rule: {fill_rule}") - left, top, right, bottom = polygon_bounds(polygon) - left = max(left, 0) - top = max(top, 0) - right = min(right, width) - bottom = min(bottom, height) - mask = np.zeros((height, width), dtype=bool) - if right <= left or bottom <= top: - return mask - - points = np.asarray(polygon, dtype=np.float64) - x = np.arange(left, right, dtype=np.float64) + 0.5 - for row_start in range(top, bottom, 128): - row_end = min(row_start + 128, bottom) - y = np.arange(row_start, row_end, dtype=np.float64) + 0.5 - state = np.zeros( - (len(y), len(x)), - dtype=bool if fill_rule == "EVEN_ODD" else np.int16, - ) - previous = points[-1] - for current in points: - crosses = (current[1] > y) != (previous[1] > y) - edge_x = (previous[0] - current[0]) * (y - current[1]) / ( - previous[1] - current[1] + 1e-300 - ) + current[0] - selected = crosses[:, None] & (x[None, :] < edge_x[:, None]) - if fill_rule == "EVEN_ODD": - state ^= selected - else: - winding = 1 if current[1] > previous[1] else -1 - state += selected * winding - previous = current - mask[row_start:row_end, left:right] = state != 0 - return mask - -def refined_mask_from_pixels(mask_pixels, mask_size, bounds, source_size): - import numpy as np - - mask_width, mask_height = (int(value) for value in mask_size) - source_width, source_height = (int(value) for value in source_size) - local = np.flipud( - np.asarray(mask_pixels, dtype=np.float32).reshape((mask_height, mask_width, 4)) - )[:, :, 3] - result = np.zeros((source_height, source_width), dtype=np.float32) - left, top, right, bottom = (int(value) for value in bounds) - source_left = max(left, 0) - source_top = max(top, 0) - source_right = min(right, source_width) - source_bottom = min(bottom, source_height) - if source_right <= source_left or source_bottom <= source_top: - return result - result[source_top:source_bottom, source_left:source_right] = local[ - source_top - top : source_bottom - top, - source_left - left : source_right - left, - ] - return result - - -def visible_selection_mask( - source_alpha, selection_mask, invert=False, alpha_threshold=0.0 -): - """Restrict a selection to source pixels above an Alpha threshold.""" - import numpy as np - - alpha = np.asarray(source_alpha) - mask = np.asarray(selection_mask, dtype=np.float32) - if mask.shape != alpha.shape: - raise ValueError("The selection does not match the source image") - mask = np.clip(mask, 0.0, 1.0) - if invert: - mask = 1.0 - mask - return np.where(alpha > float(alpha_threshold), mask, 0.0) - - -def edit_image_mask_pixels(source_pixels, source_size, selection_mask, invert=False): - """Apply a mask while preserving source RGB, then tightly crop the canvas.""" - import numpy as np - - width, height = (int(value) for value in source_size) - rgba = np.flipud( - np.asarray(source_pixels, dtype=np.float32).reshape((height, width, 4)) - ) - mask = visible_selection_mask(rgba[:, :, 3], selection_mask, invert) - rgba[:, :, 3] *= mask - return crop_pixels_to_alpha(np.flipud(rgba).ravel(), (0, 0, width, height)) - - -def edit_image_object(source_object, selection_mask, invert=False): - import numpy as np - - image = source_object.data - source_size = tuple(image.size) - source_pixels = np.empty(len(image.pixels), dtype=np.float32) - image.pixels.foreach_get(source_pixels) - pixels, crop_bounds = edit_image_mask_pixels( - source_pixels, source_size, selection_mask, invert - ) - left, top, right, bottom = crop_bounds - return replace_empty_image_pixels( - source_object, - pixels, - (right - left, bottom - top), - crop_bounds, - ) - - -def selected_image_rgba( - source_object, polygon, invert=False, fill_rule="EVEN_ODD" -): - import numpy as np - - image = source_object.data - width, height = (int(value) for value in image.size) - pixels = np.empty(len(image.pixels), dtype=np.float32) - image.pixels.foreach_get(pixels) - rgba = np.flipud(pixels.reshape((height, width, 4))) - selection = visible_selection_mask( - rgba[:, :, 3], - polygon_mask((width, height), polygon, fill_rule), - invert, - ) - rgba[:, :, 3] *= selection - if not np.any(rgba[:, :, 3] > 0.0): - raise ImageEditWarning("The selection contains no visible image pixels") - return rgba - - diff --git a/src/anyimage/operators/common/image_region.py b/src/anyimage/operators/common/image_region.py new file mode 100644 index 0000000..74132df --- /dev/null +++ b/src/anyimage/operators/common/image_region.py @@ -0,0 +1,230 @@ +import json +import math +from dataclasses import dataclass + + +class ImageEditWarning(ValueError): + pass + + +@dataclass(frozen=True) +class RegionPath: + points: tuple[tuple[float, float], ...] + invert: bool = False + fill_rule: str = "EVEN_ODD" + + def __post_init__(self): + points = tuple( + (float(point[0]), float(point[1])) + for point in self.points + ) + region_path_bounds(points) + if self.fill_rule not in {"EVEN_ODD", "NONZERO"}: + raise ValueError(f"Unknown Region Path fill rule: {self.fill_rule}") + object.__setattr__(self, "points", points) + object.__setattr__(self, "invert", bool(self.invert)) + + def to_json(self): + return json.dumps( + { + "points": self.points, + "invert": self.invert, + "fill_rule": self.fill_rule, + }, + separators=(",", ":"), + ) + + @classmethod + def from_json(cls, value): + data = json.loads(value) + if not isinstance(data, dict): + raise ValueError("The Region Path data is invalid") + return cls( + points=tuple(data.get("points", ())), + invert=bool(data.get("invert", False)), + fill_rule=str(data.get("fill_rule", "EVEN_ODD")), + ) + + +@dataclass(frozen=True) +class RegionMask: + values: object + bounds: tuple[int, int, int, int] + + def __post_init__(self): + import numpy as np + + values = np.asarray(self.values, dtype=np.float32) + bounds = tuple(int(value) for value in self.bounds) + if values.ndim != 2: + raise ValueError("The region mask must be two-dimensional") + if len(bounds) != 4: + raise ValueError("The region bounds are invalid") + left, top, right, bottom = bounds + if right <= left or bottom <= top: + raise ValueError("The region bounds are empty") + if values.shape != (bottom - top, right - left): + raise ValueError("The region mask does not match its bounds") + if not np.isfinite(values).all(): + raise ValueError("The region mask contains invalid values") + object.__setattr__(self, "values", np.clip(values, 0.0, 1.0)) + object.__setattr__(self, "bounds", bounds) + + def full_values(self, image_size): + import numpy as np + + width, height = (int(value) for value in image_size) + result = np.zeros((height, width), dtype=np.float32) + left, top, right, bottom = self.bounds + source_left = max(left, 0) + source_top = max(top, 0) + source_right = min(right, width) + source_bottom = min(bottom, height) + if source_right <= source_left or source_bottom <= source_top: + return result + result[source_top:source_bottom, source_left:source_right] = self.values[ + source_top - top : source_bottom - top, + source_left - left : source_right - left, + ] + return result + + +def region_path_bounds(points): + import numpy as np + + points = np.asarray(points, dtype=np.float64) + if points.ndim != 2 or points.shape[0] < 3 or points.shape[1] != 2: + raise ValueError("The region does not define a valid image path") + if not np.isfinite(points).all(): + raise ValueError("The region contains invalid image coordinates") + left = int(math.floor(float(points[:, 0].min()))) + top = int(math.floor(float(points[:, 1].min()))) + right = int(math.ceil(float(points[:, 0].max()))) + bottom = int(math.ceil(float(points[:, 1].max()))) + if right <= left or bottom <= top: + raise ValueError("The region does not define a valid image area") + return left, top, right, bottom + + +def _rasterize_path(image_size, points, fill_rule): + import numpy as np + + width, height = (int(value) for value in image_size) + if width <= 0 or height <= 0: + raise ValueError("The source image has no readable dimensions") + left, top, right, bottom = region_path_bounds(points) + left = max(left, 0) + top = max(top, 0) + right = min(right, width) + bottom = min(bottom, height) + mask = np.zeros((height, width), dtype=bool) + if right <= left or bottom <= top: + return mask + + points = np.asarray(points, dtype=np.float64) + x = np.arange(left, right, dtype=np.float64) + 0.5 + for row_start in range(top, bottom, 128): + row_end = min(row_start + 128, bottom) + y = np.arange(row_start, row_end, dtype=np.float64) + 0.5 + state = np.zeros( + (len(y), len(x)), + dtype=bool if fill_rule == "EVEN_ODD" else np.int16, + ) + previous = points[-1] + for current in points: + crosses = (current[1] > y) != (previous[1] > y) + edge_x = (previous[0] - current[0]) * (y - current[1]) / ( + previous[1] - current[1] + 1e-300 + ) + current[0] + selected = crosses[:, None] & (x[None, :] < edge_x[:, None]) + if fill_rule == "EVEN_ODD": + state ^= selected + else: + winding = 1 if current[1] > previous[1] else -1 + state += selected * winding + previous = current + mask[row_start:row_end, left:right] = state != 0 + return mask + + +def _antialias_mask(mask): + import numpy as np + + values = np.asarray(mask, dtype=np.float32) + kernel = np.asarray((0.029, 0.235, 0.472, 0.235, 0.029), dtype=np.float32) + radius = len(kernel) // 2 + horizontal = np.zeros_like(values) + padded = np.pad(values, ((0, 0), (radius, radius)), mode="edge") + for offset, weight in enumerate(kernel): + horizontal += padded[:, offset : offset + values.shape[1]] * weight + result = np.zeros_like(values) + padded = np.pad(horizontal, ((radius, radius), (0, 0)), mode="edge") + for offset, weight in enumerate(kernel): + result += padded[offset : offset + values.shape[0]] * weight + return result + + +def rasterize_region_path( + image_size, + region_path, + *, + source_alpha=None, + alpha_threshold=0.0, + antialias=False, +): + import numpy as np + + width, height = (int(value) for value in image_size) + mask = _rasterize_path( + (width, height), + region_path.points, + region_path.fill_rule, + ) + if antialias: + mask = _antialias_mask(mask) + else: + mask = mask.astype(np.float32) + if region_path.invert: + mask = 1.0 - mask + if source_alpha is not None: + alpha = np.asarray(source_alpha) + if alpha.shape != (height, width): + raise ValueError("The source Alpha does not match the image region") + mask = np.where(alpha > float(alpha_threshold), mask, 0.0) + visible_y, visible_x = np.nonzero(mask > float(alpha_threshold)) + if not len(visible_x): + raise ImageEditWarning("The region contains no visible image pixels") + bounds = ( + int(visible_x.min()), + int(visible_y.min()), + int(visible_x.max()) + 1, + int(visible_y.max()) + 1, + ) + left, top, right, bottom = bounds + return RegionMask(mask[top:bottom, left:right], bounds) + + +def crop_pixels_to_alpha(pixels, bounds): + """Crop Blender-order RGBA pixels to their nonzero Alpha bounds.""" + import numpy as np + + left, top, right, bottom = bounds + width = right - left + height = bottom - top + rgba = np.flipud( + np.asarray(pixels, dtype=np.float32).reshape((height, width, 4)) + ) + visible_y, visible_x = np.nonzero(rgba[:, :, 3] > 0.0) + if not len(visible_x): + raise ImageEditWarning("The edit contains no visible image pixels") + content_left = int(visible_x.min()) + content_top = int(visible_y.min()) + content_right = int(visible_x.max()) + 1 + content_bottom = int(visible_y.max()) + 1 + output = rgba[content_top:content_bottom, content_left:content_right].copy() + return np.flipud(output).ravel(), ( + left + content_left, + top + content_top, + left + content_right, + top + content_bottom, + ) diff --git a/src/anyimage/operators/common/material.py b/src/anyimage/operators/common/mesh_result.py similarity index 76% rename from src/anyimage/operators/common/material.py rename to src/anyimage/operators/common/mesh_result.py index 66433ce..6105538 100644 --- a/src/anyimage/operators/common/material.py +++ b/src/anyimage/operators/common/mesh_result.py @@ -1,7 +1,130 @@ +from dataclasses import dataclass +from pathlib import Path + import bpy -from .image import configure_image_user, is_animated_image -from .node_asset import load_shadeless_node_group +from .image_data import ( + configure_image_user, + image_base_name, + image_pixels, + is_animated_image, +) + + +@dataclass(frozen=True) +class ImageGeometry: + points: object + intrinsics: object + valid: object + calibration_valid: object + image_size: tuple + reference_depth: float + region_transparent_fraction: float + + +def load_image_geometry(path): + """Load and validate the shared image geometry Job artifact.""" + import numpy as np + + with np.load(path, allow_pickle=False) as archive: + geometry = ImageGeometry( + points=np.asarray(archive["points"], dtype=np.float32), + intrinsics=np.asarray(archive["intrinsics"], dtype=np.float32), + valid=np.asarray(archive["valid"], dtype=bool), + calibration_valid=np.asarray(archive["calibration_valid"], dtype=bool), + image_size=tuple(int(value) for value in archive["image_size"]), + reference_depth=float(archive["reference_depth"]), + region_transparent_fraction=float( + archive["region_transparent_fraction"] + ), + ) + if ( + geometry.points.ndim != 3 + or geometry.points.shape[2] != 3 + or geometry.valid.shape != geometry.points.shape[:2] + or geometry.calibration_valid.shape != geometry.points.shape[:2] + ): + raise ValueError("The mesh point field is invalid") + if ( + geometry.intrinsics.shape != (3, 3) + or not geometry.reference_depth > 0.0 + or not 0.0 <= geometry.region_transparent_fraction <= 1.0 + ): + raise ValueError("The mesh camera field is invalid") + return geometry + + +NODE_ASSET_NAME = "O_AnyImage.blend" + + +def node_asset_path(): + return Path(__file__).resolve().parents[2] / "assets" / NODE_ASSET_NAME + + +def load_asset_node_group(group_name): + existing = bpy.data.node_groups.get(group_name) + if existing is not None: + return existing + + asset_path = node_asset_path() + if not asset_path.is_file(): + raise RuntimeError(f"Missing node asset: {asset_path}") + if bpy.app.version < (5, 0, 0): + directory = str(asset_path) + "/NodeTree/" + result = bpy.ops.wm.append( + filepath=directory + group_name, + directory=directory, + filename=group_name, + link=False, + do_reuse_local_id=True, + ) + if "FINISHED" not in result: + raise RuntimeError("Unable to append the node asset") + node_group = bpy.data.node_groups.get(group_name) + if node_group is None: + raise RuntimeError("Appended node asset is unavailable") + node_group.use_fake_user = True + return node_group + + with bpy.data.libraries.load(str(asset_path), link=True) as (source, target): + if group_name not in source.node_groups: + raise RuntimeError(f"Node asset is missing {group_name}") + target.node_groups = [group_name] + linked_group = target.node_groups[0] + node_group = bpy.data.pack_linked_ids_hierarchy(root_id=linked_group) + if linked_group != node_group and linked_group.users == 0: + bpy.data.node_groups.remove(linked_group) + node_group.use_fake_user = True + return node_group + + +def modifier_input_identifier(node_group, name): + for item in node_group.interface.items_tree: + if item.item_type != "SOCKET" or item.in_out != "INPUT": + continue + if item.name == name: + return item.identifier + raise RuntimeError(f"Geometry Nodes input is missing: {name}") + + +def _modifier_input_slots(modifier): + """Return the runtime input collection, or None for the legacy API.""" + properties = getattr(modifier, "properties", None) + if properties is None: + return None + return getattr(properties, "inputs", None) + + +def set_modifier_input(modifier, identifier, value): + """Assign a value to a Geometry Nodes modifier input socket.""" + inputs = _modifier_input_slots(modifier) + if inputs is not None: + # Blender 5.2+ exposes inputs as runtime RNA keyed by socket identifier. + getattr(inputs, identifier).value = value + else: + # Blender 4.x exposes inputs as modifier ID properties. + modifier[identifier] = value + DISPLACEMENT_STRENGTH = 1.0 DISPLACEMENT_MIDLEVEL = 0.0 @@ -11,6 +134,7 @@ BACKGROUND_DEPTH_ATTRIBUTE = "o_background_depth" DEPTH_SCALE_ATTRIBUTE = "o_depth_scale" IMAGE_MATERIAL_NODE_GROUP_NAME = "O Image Layer" DEPTH_MATERIAL_NODE_GROUP_NAME = "O Image Depth Layer" +SHADELESS_NODE_GROUP_NAME = "O Shadeless" VIEW_TRANSFORM_COLOR_SPACES = ( ("ACES 1.3", "ACES 1.3 sRGB"), ("ACES 2.0", "ACES 2.0 sRGB"), @@ -44,10 +168,7 @@ def set_material_color_space(image, scene=None): or getattr(image, "is_dirty", False) ) ): - import numpy as np - - pixels = np.empty(len(image.pixels), dtype=np.float32) - image.pixels.foreach_get(pixels) + pixels = image_pixels(image) for color_space in dict.fromkeys((preferred, "sRGB", original)): if not color_space: continue @@ -251,8 +372,8 @@ def material_node_group(depth_plane=False): return group -def create_material( - name, +def create_image_material( + source_object, color_image, depth_scale, displacement_image=None, @@ -270,7 +391,7 @@ def create_material( normal_space="TANGENT", scene=None, ): - material = bpy.data.materials.new(name) + material = bpy.data.materials.new(image_base_name(source_object.data)) material.use_nodes = True if hasattr(material, "surface_render_method"): material.surface_render_method = "DITHERED" @@ -332,7 +453,7 @@ def create_material( links.new(normal_texture.outputs["Color"], material_inputs.inputs["Normal"]) if shadeless: shader = nodes.new("ShaderNodeGroup") - shader.node_tree = load_shadeless_node_group() + shader.node_tree = load_asset_node_group(SHADELESS_NODE_GROUP_NAME) shader.name = SHADELESS_NODE_GROUP_NAME transparent = nodes.new("ShaderNodeBsdfTransparent") mix_shader = nodes.new("ShaderNodeMixShader") @@ -434,3 +555,22 @@ def set_displacement_only(material): if owner is not None and hasattr(owner, property_name): setattr(owner, property_name, "DISPLACEMENT") return + + +def finalize_object_result( + context, + source_object, + result_object, + *, + keep_source=False, +): + """Activate a result object and optionally replace its source object.""" + source_name = source_object.name + for selected in context.selected_objects: + selected.select_set(False) + result_object.select_set(True) + context.view_layer.objects.active = result_object + if not keep_source: + bpy.data.objects.remove(source_object, do_unlink=True) + result_object.name = source_name + return result_object diff --git a/src/anyimage/operators/common/node_asset.py b/src/anyimage/operators/common/node_asset.py deleted file mode 100644 index 5823b11..0000000 --- a/src/anyimage/operators/common/node_asset.py +++ /dev/null @@ -1,119 +0,0 @@ -from pathlib import Path - -import bpy - - -NODE_ASSET_NAME = "O_AnyImage.blend" -MESH_PLANE_NODE_GROUP_NAME = "O Mesh Plane" -SHADELESS_NODE_GROUP_NAME = "O Shadeless" - - -def node_asset_path(): - return Path(__file__).resolve().parents[2] / "assets" / NODE_ASSET_NAME - - -def load_asset_node_group(group_name): - existing = bpy.data.node_groups.get(group_name) - if existing is not None: - for item in existing.interface.items_tree: - if ( - item.item_type == "SOCKET" - and item.in_out == "INPUT" - and item.name == "Normal" - ): - item.default_value = (0.5, 0.5, 1.0, 1.0) - for node in existing.nodes: - if node.bl_idname == "ShaderNodeNormalMap": - node.space = "TANGENT" - return existing - - asset_path = node_asset_path() - if not asset_path.is_file(): - raise RuntimeError(f"Missing node asset: {asset_path}") - if bpy.app.version < (5, 0, 0): - directory = str(asset_path) + "/NodeTree/" - result = bpy.ops.wm.append( - filepath=directory + group_name, - directory=directory, - filename=group_name, - link=False, - do_reuse_local_id=True, - ) - if "FINISHED" not in result: - raise RuntimeError("Unable to append the node asset") - node_group = bpy.data.node_groups.get(group_name) - if node_group is None: - raise RuntimeError("Appended node asset is unavailable") - node_group.use_fake_user = True - return node_group - - with bpy.data.libraries.load(str(asset_path), link=True) as (source, target): - if group_name not in source.node_groups: - raise RuntimeError(f"Node asset is missing {group_name}") - target.node_groups = [group_name] - linked_group = target.node_groups[0] - node_group = bpy.data.pack_linked_ids_hierarchy(root_id=linked_group) - if linked_group != node_group and linked_group.users == 0: - bpy.data.node_groups.remove(linked_group) - node_group.use_fake_user = True - return node_group - - -def load_shadeless_node_group(): - return load_asset_node_group(SHADELESS_NODE_GROUP_NAME) - - -def load_mesh_plane_node_group(): - return load_asset_node_group(MESH_PLANE_NODE_GROUP_NAME) - - -def modifier_input_identifier(node_group, name): - for item in node_group.interface.items_tree: - if item.item_type != "SOCKET" or item.in_out != "INPUT": - continue - if item.name == name: - return item.identifier - raise RuntimeError(f"Geometry Nodes input is missing: {name}") - - -def modifier_input_slots(modifier): - """Return the runtime input collection, or None for the legacy API.""" - properties = getattr(modifier, "properties", None) - if properties is None: - return None - return getattr(properties, "inputs", None) - - -def set_modifier_input(modifier, identifier, value): - """Assign a value to a Geometry Nodes modifier input socket.""" - inputs = modifier_input_slots(modifier) - if inputs is not None: - # Blender 5.2+ exposes inputs as runtime RNA keyed by socket identifier. - getattr(inputs, identifier).value = value - else: - # Blender 4.x exposes inputs as modifier ID properties. - modifier[identifier] = value - - -def get_modifier_input(modifier, identifier): - """Read a Geometry Nodes modifier input socket value.""" - inputs = modifier_input_slots(modifier) - if inputs is not None: - return getattr(inputs, identifier).value - return modifier[identifier] - - -def modifier_input_driver_path(modifier, identifier): - """Return the driver data path for a modifier input socket.""" - if modifier_input_slots(modifier) is not None: - return f'modifiers["{modifier.name}"].properties.inputs.{identifier}.value' - return f'modifiers["{modifier.name}"]["{identifier}"]' - - -def drive_modifier_input(owner_object, modifier, identifier, expression="frame"): - """Add a scripted driver to a modifier input so it follows the scene frame.""" - path = modifier_input_driver_path(modifier, identifier) - driver = owner_object.driver_add(path) - driver.driver.type = "SCRIPTED" - driver.driver.expression = expression - return driver diff --git a/src/anyimage/operators/convert_to_plane/build.py b/src/anyimage/operators/convert_to_plane/build.py index 7656a6f..1f2581b 100644 --- a/src/anyimage/operators/convert_to_plane/build.py +++ b/src/anyimage/operators/convert_to_plane/build.py @@ -1,19 +1,20 @@ import bpy -from ..common.image import ( +from ..common.mesh_result import load_image_geometry +from ..common.image_data import ( color_data_name, - create_color_image, depth_data_name, - is_image_empty, - material_data_name, - normal_data_name, - plane_bounds, + image_pixels, + load_color_result_image, + load_normal_result_image, + image_empty_bounds, + require_image_empty, ) -from ..common.material import create_material -from ..common.node_asset import modifier_input_identifier, set_modifier_input -from ..common.normal_map import bake_tangent_normal_image -from ..common.operator import push_undo +from ..common.mesh_result import create_image_material +from ..common.mesh_result import modifier_input_identifier, set_modifier_input +from ..common.mesh_result import finalize_object_result from .mesh_plane import create_mesh_plane_mesh, create_mesh_plane_object +from .normal_map import bake_tangent_normal_image from .nodes import depth_plane_node_group @@ -23,10 +24,7 @@ def copy_color_image(source_object): if getattr(source, "source", None) == "GENERATED" or getattr( source, "is_dirty", False ): - import numpy as np - - pixels = np.empty(len(source.pixels), dtype=np.float32) - source.pixels.foreach_get(pixels) + pixels = image_pixels(source) image.pixels.foreach_set(pixels) image.update() image.name = color_data_name(source_object) @@ -38,7 +36,7 @@ def build_mesh_plane(context, source_object, subdivisions, material): plane = create_mesh_plane_object( context, source_name, - plane_bounds(source_object), + image_empty_bounds(source_object), material, matrix_world=source_object.matrix_world, subdivisions=subdivisions, @@ -46,58 +44,32 @@ def build_mesh_plane(context, source_object, subdivisions, material): for polygon in plane.data.polygons: polygon.use_smooth = True plane["anyimage_mesh_shape"] = "PLANE" - for selected in context.selected_objects: - selected.select_set(False) - plane.select_set(True) - context.view_layer.objects.active = plane - bpy.data.objects.remove(source_object, do_unlink=True) - plane.name = source_name - return plane - - -def load_depth_field(path): - import numpy as np - - with np.load(path, allow_pickle=False) as archive: - points = np.asarray(archive["points"], dtype=np.float32) - intrinsics = np.asarray(archive["intrinsics"], dtype=np.float32) - valid = np.asarray(archive["valid"], dtype=bool) - calibration_valid = np.asarray(archive["calibration_valid"], dtype=bool) - image_size = tuple(int(value) for value in archive["image_size"]) - reference_depth = float(archive["reference_depth"]) - if ( - points.ndim != 3 - or points.shape[2] != 3 - or valid.shape != points.shape[:2] - or calibration_valid.shape != points.shape[:2] - ): - raise ValueError("The mesh point field is invalid") - if intrinsics.shape != (3, 3) or not reference_depth > 0.0: - raise ValueError("The mesh camera field is invalid") - return points, intrinsics, valid, calibration_valid, image_size, reference_depth + return finalize_object_result(context, source_object, plane) def anchored_point_field(source_object, field): import numpy as np - points, intrinsics, valid, _calibration, image_size, reference_depth = field - width, height = image_size - x_min, x_max, y_min, y_max = plane_bounds(source_object) + points = field.points + intrinsics = field.intrinsics + valid = field.valid + width, height = field.image_size + x_min, x_max, y_min, y_max = image_empty_bounds(source_object) plane_width = x_max - x_min plane_height = y_max - y_min fx, fy = float(intrinsics[0, 0]), float(intrinsics[1, 1]) cx, cy = float(intrinsics[0, 2]), float(intrinsics[1, 2]) if fx <= 0.0 or fy <= 0.0: raise ValueError("Camera intrinsics contain an invalid focal length") - scale_x = plane_width * fx / (max(width - 1, 1) * reference_depth) - scale_y = plane_height * fy / (max(height - 1, 1) * reference_depth) + scale_x = plane_width * fx / (max(width - 1, 1) * field.reference_depth) + scale_y = plane_height * fy / (max(height - 1, 1) * field.reference_depth) scale_z = float(np.sqrt(scale_x * scale_y)) camera_x = x_min + plane_width * cx / max(width - 1, 1) camera_y = y_max - plane_height * cy / max(height - 1, 1) anchored = np.empty_like(points, dtype=np.float32) anchored[..., 0] = camera_x + points[..., 0] * scale_x anchored[..., 1] = camera_y - points[..., 1] * scale_y - anchored[..., 2] = (reference_depth - points[..., 2]) * scale_z + anchored[..., 2] = (field.reference_depth - points[..., 2]) * scale_z return anchored, valid, points[..., 2] @@ -122,17 +94,9 @@ def create_depth_image(source_object, anchored_points): return image -def load_normal_image(path, source_object): - image = bpy.data.images.load(str(path), check_existing=False) - image.name = normal_data_name(source_object) - image.colorspace_settings.name = "Non-Color" - image.pack() - return image - - def build_depth_plane(context, source_object, subdivisions, material, field): source_name = source_object.name - mesh = create_mesh_plane_mesh(source_name, plane_bounds(source_object)) + mesh = create_mesh_plane_mesh(source_name, image_empty_bounds(source_object)) for polygon in mesh.polygons: polygon.use_smooth = True mesh.materials.append(material) @@ -160,33 +124,21 @@ def build_depth_plane(context, source_object, subdivisions, material, field): modifier_input_identifier(node_group, name), value, ) - for selected in context.selected_objects: - selected.select_set(False) - plane.select_set(True) - context.view_layer.objects.active = plane - bpy.data.objects.remove(source_object, do_unlink=True) - plane.name = source_name - return plane + return finalize_object_result(context, source_object, plane) def build_depth_plane_response(context, result, operator): - source_object = bpy.data.objects.get(operator.source_object_name) - if not is_image_empty(source_object): - raise RuntimeError("The source Image Empty no longer exists") - color_image = create_color_image( - source_object, - result.directory, - assign_to_source=False, - ) + source_object = require_image_empty(operator.source_object_name) + color_image = load_color_result_image(source_object, result.directory) color_image.name = color_data_name(source_object) normal_path = result.directory / "normal.png" normal_image = ( - load_normal_image(normal_path, source_object) + load_normal_result_image(normal_path, source_object) if normal_path.is_file() else None ) - material = create_material( - material_data_name(source_object), + material = create_image_material( + source_object, color_image, 0.0, normal_image=normal_image, @@ -197,7 +149,7 @@ def build_depth_plane_response(context, result, operator): source_object, operator.mesh_detail, material, - load_depth_field(result.file("field")), + load_image_geometry(result.file("image_geometry")), ) if normal_image is not None: bake_tangent_normal_image( @@ -206,4 +158,5 @@ def build_depth_plane_response(context, result, operator): plane, normal_image, ) - push_undo("Convert to Depth Plane") + if "FINISHED" not in bpy.ops.ed.undo_push(message="Convert to Depth Plane"): + raise RuntimeError("Unable to create the Convert to Depth Plane Undo step") diff --git a/src/anyimage/operators/convert_to_plane/conversion.py b/src/anyimage/operators/convert_to_plane/conversion.py index 34d88ce..1702d09 100644 --- a/src/anyimage/operators/convert_to_plane/conversion.py +++ b/src/anyimage/operators/convert_to_plane/conversion.py @@ -8,30 +8,29 @@ from ...server.media.resolution import ( MAX_MAX_AI_INPUT_SIZE, MIN_MAX_AI_INPUT_SIZE, ) -from ..common.image import ( - analysis_source_image, +from ..common.image_data import ( + cleanup_image_input, image_user_settings, is_image_empty, - material_data_name, - prepare_image_analysis_inputs, - remove_temporary_input, - scaled_process_resolution, + prepare_image_input, + source_start_frame, video_total_frames, ) -from ..common.material import create_material -from ..common.operator import ( - HIDDEN, - cleanup_temporary_input, +from ..common.mesh_result import create_image_material +from ..common.ai_job import ( da3_parameters, - input_path, + require_input_path, invoke_ai_setup_if_needed, moge2_parameters, production_device, require_environment, require_model, - source_start_frame, ) from .build import build_depth_plane_response, build_mesh_plane, copy_color_image +from .resolution import scaled_process_resolution + + +_HIDDEN = {"HIDDEN", "SKIP_SAVE"} def depth_plane_settings(context): @@ -46,48 +45,39 @@ def depth_plane_settings(context): class GenerateDepthPlane(JobOperatorBase, bpy.types.Operator): bl_idname = "anyimage.generate_depth_plane" bl_label = "Generate Depth Plane" - job_type = "prepare-geometry-data" + job_type = "generate-depth-plane" starting_message = "Preparing Depth Plane" - input_path: bpy.props.StringProperty(options=HIDDEN) - analysis_input_path: bpy.props.StringProperty(options=HIDDEN) - source_object_name: bpy.props.StringProperty(options=HIDDEN) - mesh_detail: bpy.props.IntProperty(default=0, min=0, max=10, options=HIDDEN) - model: bpy.props.StringProperty(default=DEFAULT_MOGE2_MODEL_KEY, options=HIDDEN) - process_res: bpy.props.IntProperty(default=1008, min=1, options=HIDDEN) - resolution_level: bpy.props.IntProperty(default=5, min=0, max=9, options=HIDDEN) + input_path: bpy.props.StringProperty(options=_HIDDEN) + source_object_name: bpy.props.StringProperty(options=_HIDDEN) + mesh_detail: bpy.props.IntProperty(default=0, min=0, max=10, options=_HIDDEN) + model: bpy.props.StringProperty(default=DEFAULT_MOGE2_MODEL_KEY, options=_HIDDEN) + process_res: bpy.props.IntProperty(default=1008, min=1, options=_HIDDEN) + resolution_level: bpy.props.IntProperty(default=5, min=0, max=9, options=_HIDDEN) max_input_size: bpy.props.IntProperty( default=DEFAULT_MAX_AI_INPUT_SIZE, min=MIN_MAX_AI_INPUT_SIZE, max=MAX_MAX_AI_INPUT_SIZE, - options=HIDDEN, + options=_HIDDEN, ) - delete_input: bpy.props.BoolProperty(default=False, options=HIDDEN) - delete_analysis_input: bpy.props.BoolProperty(default=False, options=HIDDEN) + delete_input: bpy.props.BoolProperty(default=False, options=_HIDDEN) def request(self, context): require_environment() require_model(self.model) parameters = { - "input": str(input_path(self.input_path)), - "analysis_input": str(input_path(self.analysis_input_path)), - "shape": "DEPTH_PLANE", - "polygon": None, - "mesh_detail": self.mesh_detail, + "input": str(require_input_path(self.input_path)), "device": production_device().lower(), "video_frames": 1, "start_frame": source_start_frame(self.source_object_name), "max_input_size": int(self.max_input_size), - "refine_selection": False, - "normal_analysis": False, - "invert": False, } settings = context.scene.anyimage_settings if self.model.startswith("MOGE2_"): parameters.update( moge2_parameters( settings, - input_path(self.input_path), + require_input_path(self.input_path), self.model, self.resolution_level, source_start_frame(self.source_object_name), @@ -97,13 +87,12 @@ class GenerateDepthPlane(JobOperatorBase, bpy.types.Operator): parameters.update( da3_parameters( settings, - input_path(self.input_path), + require_input_path(self.input_path), self.model, self.process_res, source_start_frame(self.source_object_name), ) ) - parameters.update(shape="DEPTH_PLANE", polygon=None) return parameters def response(self, context, result): @@ -111,11 +100,7 @@ class GenerateDepthPlane(JobOperatorBase, bpy.types.Operator): return "Depth Plane is ready" def cleanup(self): - cleanup_temporary_input(self.input_path, self.delete_input) - cleanup_temporary_input( - self.analysis_input_path, - self.delete_analysis_input, - ) + cleanup_image_input(self.input_path, self.delete_input) def start_depth_plane_job(operator, context): @@ -127,13 +112,13 @@ def start_depth_plane_job(operator, context): if setup_result is not None: return setup_result try: - source_path, delete_input, analysis_path, delete_analysis_input = ( - prepare_image_analysis_inputs(source_object.data, context.scene) + source_path, delete_input = prepare_image_input( + source_object.data, + context.scene, ) - analysis_image = analysis_source_image(source_object.data) model, process_factor, resolution_level = depth_plane_settings(context) process_res = ( - scaled_process_resolution(analysis_image, process_factor) + scaled_process_resolution(source_object.data, process_factor) if not model.startswith("MOGE2_") else 1 ) @@ -142,7 +127,6 @@ def start_depth_plane_job(operator, context): return {"CANCELLED"} result = bpy.ops.anyimage.generate_depth_plane( input_path=str(source_path), - analysis_input_path=str(analysis_path), source_object_name=source_object.name, mesh_detail=operator.mesh_detail, model=model, @@ -150,13 +134,10 @@ def start_depth_plane_job(operator, context): resolution_level=resolution_level, max_input_size=configured_max_ai_input_size(context), delete_input=delete_input, - delete_analysis_input=delete_analysis_input, ) if "CANCELLED" in result: if delete_input: - remove_temporary_input(source_path) - if delete_analysis_input: - remove_temporary_input(analysis_path) + cleanup_image_input(source_path, True) return {"CANCELLED"} if "CANCELLED" in result else {"FINISHED"} @@ -178,8 +159,8 @@ class ConvertToPlane(bpy.types.Operator): frame_start, frame_offset, frame_duration, use_cyclic = image_user_settings( source_object ) - material = create_material( - material_data_name(source_object), + material = create_image_material( + source_object, color_image, 1.0, frame_start=frame_start, diff --git a/src/anyimage/operators/convert_to_plane/mesh_plane.py b/src/anyimage/operators/convert_to_plane/mesh_plane.py index 9c942e9..7ca4d49 100644 --- a/src/anyimage/operators/convert_to_plane/mesh_plane.py +++ b/src/anyimage/operators/convert_to_plane/mesh_plane.py @@ -2,11 +2,8 @@ import bpy -from ..common.node_asset import ( - load_mesh_plane_node_group, - modifier_input_identifier, - set_modifier_input, -) +from ..common.mesh_result import modifier_input_identifier, set_modifier_input +from .nodes import mesh_plane_node_group MESH_PLANE_NODE_GROUP_NAME = "O Mesh Plane" @@ -44,7 +41,7 @@ def add_mesh_plane_processing( subdivisions=0, thickness=0.0, ): - node_group = load_mesh_plane_node_group() + node_group = mesh_plane_node_group() modifier = plane_object.modifiers.new( name=MESH_PLANE_NODE_GROUP_NAME, type="NODES", diff --git a/src/anyimage/operators/convert_to_plane/nodes.py b/src/anyimage/operators/convert_to_plane/nodes.py index 85ccbac..2451f99 100644 --- a/src/anyimage/operators/convert_to_plane/nodes.py +++ b/src/anyimage/operators/convert_to_plane/nodes.py @@ -2,8 +2,33 @@ import bpy -from ..common.node_asset import load_mesh_plane_node_group -from ..common.node_graph import group_input, interface_socket +from ..common.mesh_result import load_asset_node_group + +MESH_PLANE_NODE_GROUP_NAME = "O Mesh Plane" + + +def mesh_plane_node_group(): + return load_asset_node_group(MESH_PLANE_NODE_GROUP_NAME) + + +def _interface_socket(group, name, in_out, socket_type): + socket = group.interface.new_socket( + name=name, + in_out=in_out, + socket_type=socket_type, + ) + if in_out == "INPUT" and hasattr(socket, "structure_type"): + socket.structure_type = "SINGLE" + return socket + + +def _group_input(nodes, location, visible_names): + node = nodes.new("NodeGroupInput") + node.location = location + visible = set(visible_names) + for output in node.outputs: + output.hide = output.name not in visible + return node def depth_plane_node_group(): group_name = "O Image Depth Plane" @@ -16,42 +41,42 @@ def depth_plane_node_group(): group.nodes.clear() group.interface.clear() group.is_modifier = True - interface_socket(group, "Geometry", "INPUT", "NodeSocketGeometry") - subdivide = interface_socket(group, "Subdivide", "INPUT", "NodeSocketInt") + _interface_socket(group, "Geometry", "INPUT", "NodeSocketGeometry") + subdivide = _interface_socket(group, "Subdivide", "INPUT", "NodeSocketInt") subdivide.default_value = 7 subdivide.min_value = 0 subdivide.max_value = 10 - thickness = interface_socket(group, "Thickness", "INPUT", "NodeSocketFloat") + thickness = _interface_socket(group, "Thickness", "INPUT", "NodeSocketFloat") thickness.default_value = 0.1 thickness.min_value = 0.0 thickness.subtype = "DISTANCE" - depth_amount = interface_socket(group, "Depth Amount", "INPUT", "NodeSocketFloat") + depth_amount = _interface_socket(group, "Depth Amount", "INPUT", "NodeSocketFloat") depth_amount.default_value = 1.0 depth_amount.min_value = 0.0 - depth_image = interface_socket(group, "Depth Image", "INPUT", "NodeSocketImage") + depth_image = _interface_socket(group, "Depth Image", "INPUT", "NodeSocketImage") depth_image.hide_in_modifier = True - material = interface_socket(group, "Material", "INPUT", "NodeSocketMaterial") + material = _interface_socket(group, "Material", "INPUT", "NodeSocketMaterial") material.hide_in_modifier = True - interface_socket(group, "Geometry", "OUTPUT", "NodeSocketGeometry") + _interface_socket(group, "Geometry", "OUTPUT", "NodeSocketGeometry") nodes = group.nodes links = group.links - geometry_input = group_input( + geometry_input = _group_input( nodes, (-1850.0, 450.0), {"Geometry", "Subdivide", "Material"}, ) - shape_input = group_input( + shape_input = _group_input( nodes, (-1250.0, 850.0), {"Thickness", "Depth Amount"}, ) - image_input = group_input(nodes, (-1050.0, 50.0), {"Depth Image"}) - cleanup_input = group_input(nodes, (650.0, -200.0), {"Thickness"}) + image_input = _group_input(nodes, (-1050.0, 50.0), {"Depth Image"}) + cleanup_input = _group_input(nodes, (650.0, -200.0), {"Thickness"}) group_output = nodes.new("NodeGroupOutput") group_output.location = (1600.0, 450.0) mesh_plane = nodes.new("GeometryNodeGroup") - mesh_plane.node_tree = load_mesh_plane_node_group() + mesh_plane.node_tree = mesh_plane_node_group() mesh_plane.inputs["Thickness"].default_value = 0.0 mesh_plane.location = (-1550.0, 450.0) links.new(geometry_input.outputs["Geometry"], mesh_plane.inputs["Geometry"]) @@ -150,7 +175,7 @@ def depth_plane_node_group(): def plane_node_group(shape): if shape == "PLANE": - return load_mesh_plane_node_group() + return mesh_plane_node_group() if shape == "DEPTH_PLANE": return depth_plane_node_group() raise ValueError(f"Unsupported Plane shape: {shape}") diff --git a/src/anyimage/operators/common/normal_map.py b/src/anyimage/operators/convert_to_plane/normal_map.py similarity index 97% rename from src/anyimage/operators/common/normal_map.py rename to src/anyimage/operators/convert_to_plane/normal_map.py index 97a13de..7ab9701 100644 --- a/src/anyimage/operators/common/normal_map.py +++ b/src/anyimage/operators/convert_to_plane/normal_map.py @@ -1,3 +1,6 @@ +from ..common.image_data import image_pixels + + def _normalized(values, fallback): import numpy as np @@ -91,9 +94,7 @@ def bake_tangent_normal_image(context, source_object, target_object, image): import numpy as np width, height = (int(value) for value in image.size) - rgba = np.empty(width * height * 4, dtype=np.float32) - image.pixels.foreach_get(rgba) - rgba = rgba.reshape((height, width, 4)) + rgba = image_pixels(image).reshape((height, width, 4)) source_normal = rgba[..., :3].astype(np.float64) * 2.0 - 1.0 source_to_target = ( diff --git a/src/anyimage/operators/convert_to_plane/resolution.py b/src/anyimage/operators/convert_to_plane/resolution.py new file mode 100644 index 0000000..cf0093a --- /dev/null +++ b/src/anyimage/operators/convert_to_plane/resolution.py @@ -0,0 +1,16 @@ +DA3_PATCH_SIZE = 14 +DA3_MIN_PROCESS_RES = 252 +DA3_MAX_PROCESS_RES = 1008 + + +def scaled_process_resolution(image, scale_percent): + """Return the scaled long edge clamped and aligned for DA3 inference.""" + width, height = image.size + if width <= 0 or height <= 0: + raise ValueError("The source image has no readable dimensions") + if scale_percent < 1.0 or scale_percent > 100.0: + raise ValueError("Resolution Scale must be between 1% and 100%") + process_res = max(width, height) * scale_percent / 100.0 + process_res = max(DA3_MIN_PROCESS_RES, process_res) + process_res = min(DA3_MAX_PROCESS_RES, process_res) + return int(process_res / DA3_PATCH_SIZE + 0.5) * DA3_PATCH_SIZE diff --git a/src/anyimage/operators/cutout_tool/__init__.py b/src/anyimage/operators/cutout_tool/__init__.py index 0a38c38..1e61ccf 100644 --- a/src/anyimage/operators/cutout_tool/__init__.py +++ b/src/anyimage/operators/cutout_tool/__init__.py @@ -1,12 +1,8 @@ -"""Cutout selection, shape conversion, and geometry algorithms.""" - -from .selection_tool import ActivateCutoutTool, CutoutTool, SelectCutoutRegion -from .shape_conversion import ConvertCutoutShape, GenerateCutoutShape +from .region_to_shape import CutoutRegionToShape +from .select_region import CutoutTool, SelectCutoutRegion __all__ = ( - "ActivateCutoutTool", - "ConvertCutoutShape", + "CutoutRegionToShape", "CutoutTool", - "GenerateCutoutShape", "SelectCutoutRegion", ) diff --git a/src/anyimage/operators/cutout_tool/build_shapes.py b/src/anyimage/operators/cutout_tool/build_shapes.py deleted file mode 100644 index 48fc970..0000000 --- a/src/anyimage/operators/cutout_tool/build_shapes.py +++ /dev/null @@ -1,814 +0,0 @@ -import json - -import bpy - -from ...preferences import addon_preferences, configured_max_ai_input_size -from ...server.media.resolution import limited_image_size -from ..common.image import ( - color_data_name, - create_color_image, - depth_data_name, - is_image_empty, - material_data_name, - normal_data_name, - plane_bounds, - video_total_frames, -) -from ..common.material import create_material -from ..common.node_asset import modifier_input_identifier, set_modifier_input -from ..common.normal_map import bake_tangent_normal_image -from ..common.operator import ( - push_undo, -) -from .depth_calibration import ( - DEFAULT_DEPTH_DIRECTION, - UNREFINED_DEPTH_REFERENCE, - depth_balloon_projection_parameters, - metric_depth_reference, -) -from .planar_mesh import ( - metric_transform, - prepare_planar_regions, - triangulate_planar_regions, -) -from .balloon_profile import balloon_profile as calculate_balloon_profile -from .mask_regions import alpha_regions -from .node_assets import cutout_node_group -from ..common.image_edit import ( - ImageEditWarning, - polygon_bounds, -) -from ..common.image_mask import polygon_mask, visible_selection_mask -from .shape_types import ( - BALLOON_ATTRIBUTE_NAME, - CUTOUT_SHAPE_INDICES, -) - -CUTOUT_ALPHA_THRESHOLD = 0.1 - -CUTOUT_AXIS_TO_SOURCE = ( - (0.0, -1.0, 0.0, 0.0), - (0.0, 0.0, 1.0, 0.0), - (-1.0, 0.0, 0.0, 0.0), - (0.0, 0.0, 0.0, 1.0), -) - - -def _surface_depth_image(source_object, points, valid, camera_depth): - import numpy as np - - points = np.asarray(points, dtype=np.float32) - valid = np.asarray(valid, dtype=bool) - height, width = valid.shape - if points.shape != (height, width, 3): - raise ValueError("Depth Surface point dimensions do not match") - discontinuity = _depth_discontinuity_map(camera_depth, valid) - discontinuity[~valid] = -1.0 - pixels = np.empty((height, width, 4), dtype=np.float32) - pixels[:, :, :3] = points - pixels[:, :, 3] = discontinuity - image = bpy.data.images.new( - depth_data_name(source_object), - width=width, - height=height, - alpha=True, - float_buffer=True, - ) - image.colorspace_settings.name = "Non-Color" - image.alpha_mode = "CHANNEL_PACKED" - image.pixels.foreach_set(np.flipud(pixels).ravel()) - image.pack() - return image - - -def _cutout_world_matrix(source_matrix, crop_center): - from mathutils import Matrix - - center_x, center_y = (float(value) for value in crop_center) - return ( - source_matrix - @ Matrix.Translation((center_x, center_y, 0.0)) - @ Matrix(CUTOUT_AXIS_TO_SOURCE) - ) - - -def _orient_cutout_vertices(vertices): - import numpy as np - - vertices = np.asarray(vertices, dtype=np.float64) - return np.column_stack( - ( - -vertices[:, 2], - -vertices[:, 0], - vertices[:, 1], - ) - ) - - -def _cutout_domain( - regions_pixels, - image_size, - bounds, - spacing, - world_metric, - balloon_algorithm="POISSON", -): - import numpy as np - - x_min, x_max, y_min, y_max = bounds - - regions, spacing = prepare_planar_regions( - regions_pixels, - image_size, - bounds, - spacing=spacing, - world_metric=world_metric, - ) - planar, planar_faces, boundary, components = triangulate_planar_regions( - regions, - spacing, - return_components=True, - ) - profile = calculate_balloon_profile( - balloon_algorithm, planar, planar_faces, boundary, components - ) - faces = np.asarray(planar_faces, dtype=np.int32) - local = planar @ np.linalg.inv(metric_transform(world_metric)) - uv = np.column_stack( - ( - (local[:, 0] - x_min) / (x_max - x_min), - (local[:, 1] - y_min) / (y_max - y_min), - ) - ).astype(np.float32) - crop_center = np.asarray( - ((x_min + x_max) * 0.5, (y_min + y_max) * 0.5), - dtype=np.float64, - ) - local -= crop_center - vertices = np.column_stack((local, np.zeros(len(local), dtype=np.float64))) - return vertices, faces, uv, boundary, planar, profile - - -def _world_plane_metric(matrix_world): - from mathutils import Vector - - linear = matrix_world.to_3x3() - x = linear @ Vector((1.0, 0.0, 0.0)) - y = linear @ Vector((0.0, 1.0, 0.0)) - z = linear @ Vector((0.0, 0.0, 1.0)) - return ((x.dot(x), x.dot(y)), (x.dot(y), y.dot(y))), z.length - - -def _cutout_cleanup_area(spacing, world_metric): - import numpy as np - - if spacing <= 0.0: - raise ValueError("Cutout point spacing must be greater than zero") - area_scale = abs(float(np.linalg.det(metric_transform(world_metric)))) - if area_scale <= 1e-12: - raise ValueError("The Image Empty has a degenerate world transform") - return float(spacing) ** 2 / area_scale - - -def _load_field(path): - import numpy as np - - with np.load(path, allow_pickle=False) as archive: - points = np.asarray(archive["points"], dtype=np.float32) - intrinsics = np.asarray(archive["intrinsics"], dtype=np.float32) - valid = np.asarray(archive["valid"], dtype=bool) - calibration_valid = np.asarray(archive["calibration_valid"], dtype=bool) - image_size = tuple(int(value) for value in archive["image_size"]) - reference_depth = float(archive["reference_depth"]) - if ( - points.ndim != 3 - or points.shape[2] != 3 - or valid.shape != points.shape[:2] - or calibration_valid.shape != points.shape[:2] - ): - raise ValueError("The mesh point field is invalid") - if intrinsics.shape != (3, 3) or not reference_depth > 0.0: - raise ValueError("The mesh camera field is invalid") - return points, intrinsics, valid, calibration_valid, image_size, reference_depth - - -def _surface_uniform_scale( - source_object, - field, - bounds=None, -): - import numpy as np - - ( - _points, - intrinsics, - _valid, - _calibration_valid, - image_size, - reference_depth, - ) = field - width, height = image_size - x_min, x_max, y_min, y_max = bounds or plane_bounds(source_object) - plane_width = x_max - x_min - plane_height = y_max - y_min - fx, fy = float(intrinsics[0, 0]), float(intrinsics[1, 1]) - if fx <= 0.0 or fy <= 0.0: - raise ValueError("Camera intrinsics contain an invalid focal length") - scale_x = plane_width * fx / (max(width - 1, 1) * reference_depth) - scale_y = plane_height * fy / (max(height - 1, 1) * reference_depth) - return float(np.sqrt(scale_x * scale_y)) - - -def _depth_discontinuity_map(depth, valid): - import numpy as np - - depth = np.asarray(depth, dtype=np.float32) - valid = np.asarray(valid, dtype=bool) & np.isfinite(depth) & (depth > 0.0) - padded_depth = np.pad(depth, 1, mode="edge") - padded_valid = np.pad(valid, 1, mode="edge") - windows = np.lib.stride_tricks.sliding_window_view(padded_depth, (3, 3)) - usable = np.lib.stride_tricks.sliding_window_view(padded_valid, (3, 3)) - minimum = np.min(np.where(usable, windows, np.inf), axis=(-2, -1)) - maximum = np.max(np.where(usable, windows, -np.inf), axis=(-2, -1)) - result = (maximum - minimum) / np.maximum(np.abs(depth), 1e-6) - result[~usable.all(axis=(-2, -1))] = 1.0 - return result.astype(np.float32) - - -def _counter_clockwise_faces(points, faces): - import numpy as np - - points = np.asarray(points, dtype=np.float64) - result = np.asarray(faces, dtype=np.int32).copy() - polygons = points[result] - following = np.roll(polygons, -1, axis=1) - signed_area = np.sum( - polygons[..., 0] * following[..., 1] - - polygons[..., 1] * following[..., 0], - axis=1, - ) - result[signed_area < 0.0] = result[signed_area < 0.0, ::-1] - return result - - -def _assign_uv(mesh, uv): - import numpy as np - - for layer in list(mesh.uv_layers): - mesh.uv_layers.remove(layer) - uv_layer = mesh.uv_layers.new(name="UVMap") - loop_vertices = np.empty(len(mesh.loops), dtype=np.int32) - mesh.loops.foreach_get("vertex_index", loop_vertices) - loop_uv = np.asarray(uv, dtype=np.float32)[loop_vertices] - uv_layer.data.foreach_set("uv", loop_uv.ravel()) - - -def _assign_float_point_attribute(mesh, name, values): - import numpy as np - - attribute = mesh.attributes.new(name=name, type="FLOAT", domain="POINT") - attribute.data.foreach_set("value", np.asarray(values, dtype=np.float32)) - - -def _image_rgba(image): - import numpy as np - - width, height = (int(value) for value in image.size) - pixels = np.empty(len(image.pixels), dtype=np.float32) - image.pixels.foreach_get(pixels) - return np.flipud(pixels.reshape((height, width, 4))) - - -def _mask_crop_bounds(alpha): - import numpy as np - - values = np.asarray(alpha) - visible_y, visible_x = np.nonzero(values > CUTOUT_ALPHA_THRESHOLD) - if not len(visible_x): - raise ValueError("The Cutout selection contains no visible pixels") - return ( - int(visible_x.min()), - int(visible_y.min()), - int(visible_x.max()) + 1, - int(visible_y.max()) + 1, - ) - - -def _crop_plane_bounds(source_object, crop_bounds, source_size): - source_width, source_height = (float(value) for value in source_size) - left, top, right, bottom = (float(value) for value in crop_bounds) - x_min, x_max, y_min, y_max = plane_bounds(source_object) - width = x_max - x_min - height = y_max - y_min - return ( - x_min + left * width / source_width, - x_min + right * width / source_width, - y_max - bottom * height / source_height, - y_max - top * height / source_height, - ) - - -def _cropped_color_image(source_object, crop_bounds): - import numpy as np - - source = source_object.data - rgba = _image_rgba(source) - left, top, right, bottom = crop_bounds - width = right - left - height = bottom - top - if width <= 0 or height <= 0: - raise ValueError("The Cutout crop dimensions must be positive") - cropped = np.zeros((height, width, 4), dtype=np.float32) - source_height, source_width = rgba.shape[:2] - source_left = max(0, left) - source_top = max(0, top) - source_right = min(source_width, right) - source_bottom = min(source_height, bottom) - if source_right > source_left and source_bottom > source_top: - cropped[ - source_top - top : source_bottom - top, - source_left - left : source_right - left, - ] = rgba[source_top:source_bottom, source_left:source_right] - image = bpy.data.images.new( - color_data_name(source_object), - width=width, - height=height, - alpha=True, - ) - try: - image.colorspace_settings.name = source.colorspace_settings.name - image.alpha_mode = source.alpha_mode - image.pixels.foreach_set(np.flipud(cropped).ravel()) - image.pack() - image.update() - except Exception: - bpy.data.images.remove(image, do_unlink=True) - raise - return image - - -def _cutout_regions( - alpha, mesh_detail, threshold=CUTOUT_ALPHA_THRESHOLD -): - contour_resolution = max(512, min(int(mesh_detail) * 8, 1024)) - return alpha_regions( - alpha, - max_resolution=contour_resolution, - threshold=threshold, - ) - - -def _antialias_selection(selection): - import numpy as np - - values = np.asarray(selection, dtype=np.float32) - kernel = np.asarray((0.029, 0.235, 0.472, 0.235, 0.029), dtype=np.float32) - radius = len(kernel) // 2 - horizontal = np.zeros_like(values) - padded = np.pad(values, ((0, 0), (radius, radius)), mode="edge") - for offset, weight in enumerate(kernel): - horizontal += padded[:, offset : offset + values.shape[1]] * weight - result = np.zeros_like(values) - padded = np.pad(horizontal, ((radius, radius), (0, 0)), mode="edge") - for offset, weight in enumerate(kernel): - result += padded[offset : offset + values.shape[0]] * weight - return result - - -def _geometry_cutout_regions( - alpha, - polygon, - mesh_detail, - invert=False, - refine_selection=False, - selection=None, -): - import numpy as np - - values = np.asarray(alpha, dtype=np.float32) - if values.ndim != 2: - raise ValueError("Cutout Alpha must be two-dimensional") - source_height, source_width = values.shape - source_size = (source_width, source_height) - if refine_selection: - geometry_alpha = np.where(values > CUTOUT_ALPHA_THRESHOLD, values, 0.0) - regions, _image_size = _cutout_regions(geometry_alpha, mesh_detail) - return regions, source_size - geometry_alpha = selection - if geometry_alpha is None: - geometry_alpha = _antialias_selection( - polygon_mask(source_size, polygon, "NONZERO") - ) - geometry_alpha = visible_selection_mask( - values, - geometry_alpha, - invert, - alpha_threshold=CUTOUT_ALPHA_THRESHOLD, - ) - regions, _image_size = _cutout_regions(geometry_alpha, mesh_detail) - return regions, source_size - -def _local_cutout_rgba(source_object, polygon, invert=False): - import numpy as np - - rgba = _image_rgba(source_object.data).copy() - height, width = rgba.shape[:2] - selection = _antialias_selection( - polygon_mask((width, height), polygon, "NONZERO") - ) - selection = visible_selection_mask( - rgba[:, :, 3], - selection, - invert, - alpha_threshold=CUTOUT_ALPHA_THRESHOLD, - ) - rgba[:, :, 3] *= selection - rgba[rgba[:, :, 3] <= 0.0, :3] = 0.0 - if not np.any(rgba[:, :, 3] > 0.0): - raise ImageEditWarning("The selection contains no visible image pixels") - return rgba, selection - - -def _new_material( - context, - source_object, - color_image, - normal_image=None, - normal_space="TANGENT", -): - return create_material( - material_data_name(source_object), - color_image, - 0.0, - normal_image=normal_image, - normal_space=normal_space, - scene=context.scene, - ) - - -def build_cutout_object( - context, - source_object, - shape, - detail, - material, - field=None, - regions=None, - image_size=None, - bounds=None, - keep_original=False, - refine_selection=False, -): - import numpy as np - - bounds = bounds or plane_bounds(source_object) - crop_center = ( - (bounds[0] + bounds[1]) * 0.5, - (bounds[2] + bounds[3]) * 0.5, - ) - world_metric, world_z_scale = _world_plane_metric(source_object.matrix_world) - depth_image = None - depth_reference = None - depth_uniform_scale = None - depth_direction = None - preferences = addon_preferences(context) - balloon_algorithm = getattr( - preferences, "balloon_profile_algorithm", "POISSON" - ) - point_spacing = _cutout_point_spacing(source_object, detail) - cleanup_area = _cutout_cleanup_area(point_spacing, world_metric) - domain = _cutout_domain( - regions, - image_size, - bounds, - point_spacing, - world_metric, - balloon_algorithm, - ) - source_vertices, faces, uv, boundary, planar, balloon_profile = domain - faces = _counter_clockwise_faces(planar, faces) - vertices = _orient_cutout_vertices(source_vertices) - if world_z_scale <= 1e-10: - raise ValueError("The Image Empty has a degenerate world transform") - balloon_profile /= world_z_scale - if shape in {"DEPTH_BALLOON", "DEPTH_SURFACE"}: - ( - points, - _intrinsics, - valid, - calibration_valid, - _image_size, - _reference, - ) = field - depth_image = _surface_depth_image( - source_object, points, valid, points[..., 2] - ) - depth_uniform_scale = _surface_uniform_scale( - source_object, - field, - bounds, - ) - if not refine_selection: - depth_reference = UNREFINED_DEPTH_REFERENCE - if shape == "DEPTH_BALLOON": - depth_direction = DEFAULT_DEPTH_DIRECTION - elif shape == "DEPTH_SURFACE": - depth_reference = metric_depth_reference( - points[..., 2], - calibration_valid, - depth_uniform_scale, - ) - else: - depth_reference, depth_direction = depth_balloon_projection_parameters( - vertices, - uv, - points[..., 2], - calibration_valid, - depth_uniform_scale, - ) - - source_name = source_object.name - mesh = bpy.data.meshes.new(source_name) - mesh.from_pydata(np.asarray(vertices).tolist(), [], faces) - for polygon in mesh.polygons: - polygon.use_smooth = True - _assign_uv(mesh, uv) - _assign_float_point_attribute(mesh, BALLOON_ATTRIBUTE_NAME, balloon_profile) - mesh.materials.append(material) - mesh.update() - - mesh_object = bpy.data.objects.new(source_name, mesh) - context.collection.objects.link(mesh_object) - mesh_object.matrix_world = _cutout_world_matrix( - source_object.matrix_world, crop_center - ) - mesh_object["anyimage_mesh_shape"] = shape - node_group = cutout_node_group(shape) - modifier = mesh_object.modifiers.new(name=node_group.name, type="NODES") - modifier.node_group = node_group - modifier.show_group_selector = False - set_modifier_input( - modifier, - modifier_input_identifier(node_group, "Shape"), - CUTOUT_SHAPE_INDICES[shape], - ) - set_modifier_input( - modifier, - modifier_input_identifier(node_group, "Cleanup"), - cleanup_area, - ) - set_modifier_input( - modifier, - modifier_input_identifier(node_group, "Smooth Iterations"), - 0, - ) - set_modifier_input( - modifier, - modifier_input_identifier(node_group, "Smooth"), - 1.0, - ) - thickness = 1.0 if shape in {"BALLOON", "DEPTH_BALLOON"} else 0.0 - set_modifier_input( - modifier, - modifier_input_identifier(node_group, "Thickness"), - thickness, - ) - if shape in {"DEPTH_BALLOON", "DEPTH_SURFACE"}: - set_modifier_input( - modifier, - modifier_input_identifier( - node_group, - "Depth Scale", - ), - 1.0, - ) - if shape == "DEPTH_SURFACE": - set_modifier_input( - modifier, - modifier_input_identifier(node_group, "Uniform Scale"), - depth_uniform_scale, - ) - set_modifier_input( - modifier, - modifier_input_identifier(node_group, "Reference Depth"), - depth_reference, - ) - if shape == "DEPTH_BALLOON": - set_modifier_input( - modifier, - modifier_input_identifier(node_group, "Reference Depth"), - depth_reference, - ) - set_modifier_input( - modifier, - modifier_input_identifier(node_group, "Depth Direction"), - depth_direction, - ) - set_modifier_input( - modifier, - modifier_input_identifier(node_group, "Uniform Scale"), - depth_uniform_scale, - ) - if depth_image is not None: - set_modifier_input( - modifier, - modifier_input_identifier(node_group, "Depth Image"), - depth_image, - ) - for selected in context.selected_objects: - selected.select_set(False) - mesh_object.select_set(True) - context.view_layer.objects.active = mesh_object - if not keep_original: - bpy.data.objects.remove(source_object, do_unlink=True) - mesh_object.name = source_name - return mesh_object - - -def _load_normal_image(path, source_object): - image = bpy.data.images.load(str(path), check_existing=False) - image.name = normal_data_name(source_object) - image.colorspace_settings.name = "Non-Color" - image.pack() - return image - - -def _orient_cutout_object_normal(image): - import numpy as np - - width, height = (int(value) for value in image.size) - pixels = np.empty(width * height * 4, dtype=np.float32) - image.pixels.foreach_get(pixels) - rgba = pixels.reshape((height, width, 4)) - source = rgba[:, :, :3] * 2.0 - 1.0 - oriented = np.empty_like(source) - oriented[:, :, 0] = -source[:, :, 2] - oriented[:, :, 1] = -source[:, :, 0] - oriented[:, :, 2] = source[:, :, 1] - rgba[:, :, :3] = np.clip(oriented * 0.5 + 0.5, 0.0, 1.0) - image.pixels.foreach_set(rgba.ravel()) - image.update() - - -def limited_cutout_mesh_detail(mesh_detail, crop_size, max_input_size): - crop_width, crop_height = (int(value) for value in crop_size) - limited_width, limited_height = limited_image_size( - (crop_width, crop_height), - max_input_size, - ) - scale = max(limited_width, limited_height) / max(crop_width, crop_height) - return max(4, min(1024, round(int(mesh_detail) * scale))) - -def _cutout_point_spacing(source_object, detail): - import math - - x_min, x_max, y_min, y_max = plane_bounds(source_object) - metric, _world_z_scale = _world_plane_metric(source_object.matrix_world) - width = (x_max - x_min) * math.sqrt(max(metric[0][0], 0.0)) - height = (y_max - y_min) * math.sqrt(max(metric[1][1], 0.0)) - return max(width, height) / max(int(detail), 8) - - -def build_local_cutout( - operator, - context, - source_object, - shape, - selection_data, - invert=False, -): - if video_total_frames(source_object.data) != 1: - raise ValueError("Mesh conversion currently supports still images") - polygon = json.loads(selection_data) if selection_data else None - if not isinstance(polygon, list) or len(polygon) < 3: - raise ValueError("The Cutout selection is invalid") - _rgba, selection = _local_cutout_rgba(source_object, polygon, invert) - source_size = tuple(source_object.data.size) - crop_bounds = _mask_crop_bounds(selection) - left, top, right, bottom = crop_bounds - cropped_selection = selection[top:bottom, left:right] - mesh_detail = limited_cutout_mesh_detail( - operator.mesh_detail, - (right - left, bottom - top), - configured_max_ai_input_size(context), - ) - regions, image_size = _cutout_regions( - cropped_selection, - mesh_detail, - threshold=CUTOUT_ALPHA_THRESHOLD, - ) - local_bounds = _crop_plane_bounds( - source_object, - crop_bounds, - source_size, - ) - color_image = None - material = None - try: - color_image = _cropped_color_image(source_object, crop_bounds) - material = _new_material(context, source_object, color_image) - build_cutout_object( - context, - source_object, - shape, - mesh_detail, - material, - regions=regions, - image_size=image_size, - bounds=local_bounds, - keep_original=True, - ) - except Exception: - if material is not None and material.users == 0: - bpy.data.materials.remove(material) - if color_image is not None and color_image.users == 0: - bpy.data.images.remove(color_image) - raise - push_undo(f"Convert to {shape.replace('_', ' ').title()}") - - -def build_cutout_response(context, result, operator): - source_object = bpy.data.objects.get(operator.source_object_name) - if not is_image_empty(source_object): - raise RuntimeError("The source Image Empty no longer exists") - source_size = tuple(source_object.data.size) - crop_bounds = tuple( - int(value) - for value in json.loads( - result.file("bounds").read_text(encoding="utf-8") - ) - ) - local_bounds = _crop_plane_bounds( - source_object, - crop_bounds, - source_size, - ) - selection_image = bpy.data.images.load( - str(result.file("selection")), - check_existing=False, - ) - try: - selection_alpha = _image_rgba(selection_image)[:, :, 3] - mesh_detail = limited_cutout_mesh_detail( - operator.mesh_detail, - selection_image.size, - operator.max_input_size, - ) - regions, image_size = _cutout_regions( - selection_alpha, - mesh_detail, - threshold=CUTOUT_ALPHA_THRESHOLD, - ) - finally: - bpy.data.images.remove(selection_image) - color_image = create_color_image( - source_object, - result.directory, - assign_to_source=False, - ) - normal_path = result.directory / "normal.png" - - normal_image = ( - _load_normal_image(normal_path, source_object) - if normal_path.is_file() - else None - ) - if normal_image is not None and operator.shape == "DEPTH_SURFACE": - _orient_cutout_object_normal(normal_image) - material = _new_material( - context, - source_object, - color_image, - normal_image, - normal_space=( - "OBJECT" - if operator.shape == "DEPTH_SURFACE" and normal_image is not None - else "TANGENT" - ), - ) - field = ( - _load_field(result.file("field")) - if operator.shape in {"DEPTH_BALLOON", "DEPTH_SURFACE"} - else None - ) - mesh_object = build_cutout_object( - context, - source_object, - operator.shape, - mesh_detail, - material, - field=field, - regions=regions, - image_size=image_size, - bounds=local_bounds, - keep_original=True, - refine_selection=operator.refine_selection, - ) - if normal_image is not None and operator.shape != "DEPTH_SURFACE": - bake_tangent_normal_image( - context, - source_object, - mesh_object, - normal_image, - ) - push_undo(f"Convert to {operator.shape.replace('_', ' ').title()}") diff --git a/src/anyimage/operators/cutout_tool/geometry/__init__.py b/src/anyimage/operators/cutout_tool/geometry/__init__.py new file mode 100644 index 0000000..fe35ae7 --- /dev/null +++ b/src/anyimage/operators/cutout_tool/geometry/__init__.py @@ -0,0 +1 @@ +"""Cutout 的内存几何算法。""" diff --git a/src/anyimage/operators/cutout_tool/balloon_profile.py b/src/anyimage/operators/cutout_tool/geometry/balloon.py similarity index 80% rename from src/anyimage/operators/cutout_tool/balloon_profile.py rename to src/anyimage/operators/cutout_tool/geometry/balloon.py index 4c061bf..31e0eab 100644 --- a/src/anyimage/operators/cutout_tool/balloon_profile.py +++ b/src/anyimage/operators/cutout_tool/geometry/balloon.py @@ -231,31 +231,49 @@ def cotangent_poisson_field( import numpy as np points = np.asarray(planar_points, dtype=np.float64) + faces = np.asarray(planar_faces, dtype=np.int64) + if faces.size == 0: + faces = np.empty((0, 3), dtype=np.int64) + if faces.ndim != 2 or faces.shape[1] != 3: + raise ValueError("Balloon profile faces must be triangles") boundary_mask = np.asarray(boundary_mask, dtype=bool) - loads = np.zeros(len(points), dtype=np.float64) - edge_weights = {} - - def add_edge(start, end, weight): - edge = tuple(sorted((int(start), int(end)))) - edge_weights[edge] = edge_weights.get(edge, 0.0) + weight - - for face in planar_faces: - first, second, third = (int(index) for index in face) - a, b, c = points[[first, second, third]] - cross = abs( - (b[0] - a[0]) * (c[1] - a[1]) - - (b[1] - a[1]) * (c[0] - a[0]) + triangles = points[faces] + a, b, c = (triangles[:, index] for index in range(3)) + cross = np.abs( + (b[:, 0] - a[:, 0]) * (c[:, 1] - a[:, 1]) + - (b[:, 1] - a[:, 1]) * (c[:, 0] - a[:, 0]) + ) + usable = cross > 1e-14 + faces = faces[usable] + a, b, c = a[usable], b[usable], c[usable] + cross = cross[usable] + loads = np.bincount( + faces.ravel(), + weights=np.repeat(cross / 6.0, 3), + minlength=len(points), + ) + edges = np.concatenate( + ( + faces[:, (1, 2)], + faces[:, (0, 2)], + faces[:, (0, 1)], + ), + axis=0, + ) + edge_weights = np.concatenate( + ( + np.sum((b - a) * (c - a), axis=1), + np.sum((a - b) * (c - b), axis=1), + np.sum((a - c) * (b - c), axis=1), ) - if cross <= 1e-14: - continue - area = cross * 0.5 - loads[[first, second, third]] += area / 3.0 - cotangent_a = float((b - a).dot(c - a) / cross) - cotangent_b = float((a - b).dot(c - b) / cross) - cotangent_c = float((a - c).dot(b - c) / cross) - add_edge(second, third, cotangent_a * 0.5) - add_edge(first, third, cotangent_b * 0.5) - add_edge(first, second, cotangent_c * 0.5) + ) / np.tile(cross * 2.0, 3) + edges.sort(axis=1) + edges, inverse = np.unique(edges, axis=0, return_inverse=True) + edge_weights = np.bincount( + inverse, + weights=edge_weights, + minlength=len(edges), + ) interior = np.flatnonzero(~boundary_mask) field = np.zeros(len(points), dtype=np.float64) @@ -263,38 +281,46 @@ def cotangent_poisson_field( return field compact = np.full(len(points), -1, dtype=np.int64) compact[interior] = np.arange(len(interior), dtype=np.int64) - diagonal = np.zeros(len(interior), dtype=np.float64) - edge_starts = [] - edge_ends = [] - weights = [] # Constrained triangulation can create negative cotangent weights. # Clamping them keeps the Dirichlet system positive definite. - for (start, end), weight in edge_weights.items(): - weight = max(float(weight), 0.0) - if weight <= 1e-14: - continue - start_compact = compact[start] - end_compact = compact[end] - if start_compact >= 0: - diagonal[start_compact] += weight - if end_compact >= 0: - diagonal[end_compact] += weight - if start_compact >= 0 and end_compact >= 0: - edge_starts.append(start_compact) - edge_ends.append(end_compact) - weights.append(weight) + edge_weights = np.maximum(edge_weights, 0.0) + usable = edge_weights > 1e-14 + compact_edges = compact[edges[usable]] + edge_weights = edge_weights[usable] + start_compact = compact_edges[:, 0] + end_compact = compact_edges[:, 1] + start_usable = start_compact >= 0 + end_usable = end_compact >= 0 + diagonal = np.zeros(len(interior), dtype=np.float64) + diagonal += np.bincount( + start_compact[start_usable], + weights=edge_weights[start_usable], + minlength=len(interior), + ) + diagonal += np.bincount( + end_compact[end_usable], + weights=edge_weights[end_usable], + minlength=len(interior), + ) if np.any(diagonal <= 1e-14): raise ValueError("Unable to build a stable Balloon profile") - edge_starts = np.asarray(edge_starts, dtype=np.int64) - edge_ends = np.asarray(edge_ends, dtype=np.int64) - weights = np.asarray(weights, dtype=np.float64) + interior_edges = start_usable & end_usable + edge_starts = start_compact[interior_edges] + edge_ends = end_compact[interior_edges] + weights = edge_weights[interior_edges] + directed_starts = np.concatenate((edge_starts, edge_ends)) + directed_ends = np.concatenate((edge_ends, edge_starts)) + directed_weights = np.concatenate((weights, weights)) def multiply(values): result = diagonal * values - if len(weights): - np.add.at(result, edge_starts, -weights * values[edge_ends]) - np.add.at(result, edge_ends, -weights * values[edge_starts]) + if len(directed_weights): + result -= np.bincount( + directed_starts, + weights=directed_weights * values[directed_ends], + minlength=len(values), + ) return result right_hand_side = loads[interior] diff --git a/src/anyimage/operators/cutout_tool/depth_calibration.py b/src/anyimage/operators/cutout_tool/geometry/depth_calibration.py similarity index 94% rename from src/anyimage/operators/cutout_tool/depth_calibration.py rename to src/anyimage/operators/cutout_tool/geometry/depth_calibration.py index 531af07..2d35a47 100644 --- a/src/anyimage/operators/cutout_tool/depth_calibration.py +++ b/src/anyimage/operators/cutout_tool/geometry/depth_calibration.py @@ -3,10 +3,19 @@ PLANE_VARIANCE_FLOOR_RATIO = 0.025 DEPTH_REFERENCE_PERCENTILE = 95.0 -UNREFINED_DEPTH_REFERENCE = 8.0 +ADAPTIVE_DEPTH_TRANSPARENCY_RATIO = 0.05 +DEFAULT_DEPTH_BASELINE = 8.0 DEFAULT_DEPTH_DIRECTION = (0.0, 0.0, 1.0) +def use_adaptive_depth_calibration(region_transparent_fraction): + """Return whether the Region input has enough transparent context.""" + return ( + float(region_transparent_fraction) + >= ADAPTIVE_DEPTH_TRANSPARENCY_RATIO + ) + + def metric_depth_reference(camera_depth, valid, uniform_scale): """Return the shared upper-percentile scene depth for both Depth shapes.""" import numpy as np diff --git a/src/anyimage/operators/cutout_tool/mask_regions.py b/src/anyimage/operators/cutout_tool/geometry/region_contours.py similarity index 78% rename from src/anyimage/operators/cutout_tool/mask_regions.py rename to src/anyimage/operators/cutout_tool/geometry/region_contours.py index 07ada7d..55a097a 100644 --- a/src/anyimage/operators/cutout_tool/mask_regions.py +++ b/src/anyimage/operators/cutout_tool/geometry/region_contours.py @@ -1,4 +1,34 @@ -"""Convert a Cutout Alpha mask into outer contours and holes.""" +"""Extract Cutout Region contours from a Region Mask.""" + +from dataclasses import dataclass + + +@dataclass(frozen=True) +class RegionContours: + components: tuple + image_size: tuple[int, int] + + def __post_init__(self): + components = tuple( + tuple( + tuple((float(x), float(y)) for x, y in contour) + for contour in component + ) + for component in self.components + ) + image_size = tuple(int(value) for value in self.image_size) + if not components or any(not component for component in components): + raise ValueError("The Region has no usable contour components") + if any( + len(contour) < 3 + for component in components + for contour in component + ): + raise ValueError("A Region contour has fewer than three points") + if len(image_size) != 2 or min(image_size) <= 0: + raise ValueError("The Region contour image size is invalid") + object.__setattr__(self, "components", components) + object.__setattr__(self, "image_size", image_size) def _choose_boundary_edge(start, direction, candidates): @@ -27,7 +57,7 @@ def _simplify_axis_aligned_contour(points): return simplified -def trace_mask_contours(mask): +def _trace_region_contours(mask): import numpy as np mask = np.asarray(mask, dtype=bool) @@ -113,8 +143,8 @@ def _point_inside_contour(point, contour): return inside -def mask_regions(mask): - contours = trace_mask_contours(mask) +def _group_region_contours(mask): + contours = _trace_region_contours(mask) outer = [contour for contour in contours if _signed_contour_area(contour) > 0.0] holes = [contour for contour in contours if _signed_contour_area(contour) < 0.0] if not outer: @@ -143,7 +173,7 @@ def mask_regions(mask): return regions -def _resize_alpha(values, size): +def _resize_values(values, size): import numpy as np target_width, target_height = (int(value) for value in size) @@ -174,10 +204,10 @@ def _resize_alpha(values, size): ) -def alpha_regions(alpha, max_resolution=512, threshold=0.5): +def extract_region_contours(region_mask, max_resolution=512, threshold=0.5): import numpy as np - values = np.asarray(alpha) + values = np.asarray(region_mask.values) if values.ndim != 2: raise ValueError("Cutout Alpha must be two-dimensional") if np.issubdtype(values.dtype, np.floating): @@ -188,33 +218,24 @@ def alpha_regions(alpha, max_resolution=512, threshold=0.5): scale = min(1.0, float(max_resolution) / max(source_size)) sampled_size = tuple(max(1, round(value * scale)) for value in source_size) if sampled_size != source_size: - values = _resize_alpha(values, sampled_size) + values = _resize_values(values, sampled_size) mask = values >= 255.0 * threshold - regions = mask_regions(mask) + components = _group_region_contours(mask) scale_x = source_size[0] / sampled_size[0] scale_y = source_size[1] / sampled_size[1] - return ( - [ - [ - [ - [ + return RegionContours( + components=tuple( + tuple( + tuple( + ( min(max(float(x) * scale_x, 0.0), source_size[0]), min(max(float(y) * scale_y, 0.0), source_size[1]), - ] + ) for x, y in contour - ] - for contour in region - ] - for region in regions - ], - source_size, + ) + for contour in component + ) + for component in components + ), + image_size=source_size, ) - - -def image_alpha_regions(path, max_resolution=512, threshold=0.5): - import numpy as np - from PIL import Image - - with Image.open(path) as source: - alpha = np.asarray(source.convert("RGBA").getchannel("A"), dtype=np.uint8) - return alpha_regions(alpha, max_resolution=max_resolution, threshold=threshold) diff --git a/src/anyimage/operators/cutout_tool/planar_mesh.py b/src/anyimage/operators/cutout_tool/geometry/triangulation.py similarity index 89% rename from src/anyimage/operators/cutout_tool/planar_mesh.py rename to src/anyimage/operators/cutout_tool/geometry/triangulation.py index f561e19..2955287 100644 --- a/src/anyimage/operators/cutout_tool/planar_mesh.py +++ b/src/anyimage/operators/cutout_tool/geometry/triangulation.py @@ -3,7 +3,7 @@ import math POINT_SPACING = 0.1 -def polygon_area(points): +def contour_area(points): import numpy as np points = np.asarray(points, dtype=np.float64) @@ -16,10 +16,10 @@ def polygon_area(points): ) -def resample_closed_polygon(polygon, spacing): +def resample_closed_contour(contour, spacing): import numpy as np - points = np.asarray(polygon, dtype=np.float64) + points = np.asarray(contour, dtype=np.float64) if len(points) < 3: raise ValueError("The lasso does not define a valid image region") segments = np.roll(points, -1, axis=0) - points @@ -40,9 +40,9 @@ def resample_closed_polygon(polygon, spacing): return points[indices] + segments[indices] * factors[:, None] -def smooth_lasso_contour(polygon, spacing, iterations=16): - contour = resample_closed_polygon(polygon, spacing * 0.25) - original_area = abs(polygon_area(contour)) +def smooth_region_contour(contour, spacing, iterations=16): + contour = resample_closed_contour(contour, spacing * 0.25) + original_area = abs(contour_area(contour)) for _index in range(iterations): import numpy as np @@ -51,14 +51,14 @@ def smooth_lasso_contour(polygon, spacing, iterations=16): + contour * 0.75 + np.roll(contour, -1, axis=0) * 0.125 ) - smooth_area = abs(polygon_area(contour)) + smooth_area = abs(contour_area(contour)) if original_area > 0.0 and smooth_area > 0.0: center = contour.mean(axis=0) contour = center + (contour - center) * math.sqrt( original_area / smooth_area ) - contour = resample_closed_polygon(contour, spacing) - if polygon_area(contour) < 0.0: + contour = resample_closed_contour(contour, spacing) + if contour_area(contour) < 0.0: contour = contour[::-1].copy() return contour @@ -98,10 +98,6 @@ def _edge_bucket_indices(starts, ends, minimum_y, bucket_height, bucket_count): return buckets -def points_inside_polygon(points, polygon): - return points_inside_contours(points, (polygon,)) - - def points_inside_contours(points, contours): import numpy as np @@ -236,10 +232,6 @@ def uniform_interior_points_for_contours(contours, spacing): ) -def uniform_interior_points(contour, spacing): - return uniform_interior_points_for_contours((contour,), spacing) - - def metric_transform(world_metric=None): import numpy as np @@ -259,7 +251,7 @@ def metric_transform(world_metric=None): def prepare_planar_contours( - polygons, + contours, image_size, local_bounds, spacing=POINT_SPACING, @@ -274,9 +266,9 @@ def prepare_planar_contours( local_width = x_max - x_min local_height = y_max - y_min transform = metric_transform(world_metric) - contours = [] - for polygon in polygons: - pixels = np.asarray(polygon, dtype=np.float64) + prepared = [] + for contour in contours: + pixels = np.asarray(contour, dtype=np.float64) if pixels.ndim != 2 or pixels.shape[0] < 3 or pixels.shape[1] != 2: continue local = np.empty_like(pixels) @@ -289,30 +281,29 @@ def prepare_planar_contours( if perimeter <= 1e-12: continue contour_spacing = min(spacing, perimeter / 8.0) - smoothed = smooth_lasso_contour(planar, contour_spacing) + smoothed = smooth_region_contour(planar, contour_spacing) smoothed_local = smoothed @ np.linalg.inv(transform) smoothed_local[:, 0] = np.clip(smoothed_local[:, 0], x_min, x_max) smoothed_local[:, 1] = np.clip(smoothed_local[:, 1], y_min, y_max) - contours.append(smoothed_local @ transform) - if not contours: + prepared.append(smoothed_local @ transform) + if not prepared: raise ValueError("The Cutout has no region large enough for this resolution") - interior = uniform_interior_points_for_contours(contours, spacing) - return contours, interior, spacing + interior = uniform_interior_points_for_contours(prepared, spacing) + return prepared, interior, spacing -def prepare_planar_regions( - regions, - image_size, +def _prepare_region_contours( + region_contours, local_bounds, spacing=POINT_SPACING, world_metric=None, ): prepared = [] errors = [] - for region in regions: + for component in region_contours.components: try: contours, interior, _spacing = prepare_planar_contours( - region, - image_size, + component, + region_contours.image_size, local_bounds, spacing=spacing, world_metric=world_metric, @@ -328,25 +319,7 @@ def prepare_planar_regions( return prepared, spacing -def prepare_planar_points( - polygon, - image_size, - local_bounds, - spacing=POINT_SPACING, - world_metric=None, -): - contours, interior, spacing = prepare_planar_contours( - (polygon,), - image_size, - local_bounds, - spacing=spacing, - world_metric=world_metric, - ) - contour = contours[0] - return contour, interior, spacing - - -def triangulate_planar_contours(contours, interior, spacing): +def _triangulate_contour_component(contours, interior, spacing): import numpy as np from mathutils import Vector from mathutils.geometry import delaunay_2d_cdt @@ -413,16 +386,28 @@ def boundary_vertices_from_faces(vertex_count, faces): return boundary -def triangulate_planar_regions(regions, spacing, return_components=False): +def triangulate_region_contours( + region_contours, + local_bounds, + spacing=POINT_SPACING, + world_metric=None, + return_components=False, +): import numpy as np + prepared, spacing = _prepare_region_contours( + region_contours, + local_bounds, + spacing=spacing, + world_metric=world_metric, + ) points = [] faces = [] boundaries = [] components = [] offset = 0 - for contours, interior in regions: - region_points, region_faces, region_boundary = triangulate_planar_contours( + for contours, interior in prepared: + region_points, region_faces, region_boundary = _triangulate_contour_component( contours, interior, spacing, @@ -449,4 +434,3 @@ def triangulate_planar_regions(regions, spacing, return_components=False): np.concatenate(boundaries), ) return (*result, components) if return_components else result - diff --git a/src/anyimage/operators/cutout_tool/node_assets.py b/src/anyimage/operators/cutout_tool/node_assets.py deleted file mode 100644 index 930ab14..0000000 --- a/src/anyimage/operators/cutout_tool/node_assets.py +++ /dev/null @@ -1,10 +0,0 @@ -"""Load Geometry Nodes assets used by Cutout shapes.""" - -from ..common.node_asset import load_asset_node_group -from .shape_types import CUTOUT_NODE_GROUP_NAME, CUTOUT_SHAPES - - -def cutout_node_group(shape): - if shape not in CUTOUT_SHAPES: - raise ValueError(f"Unsupported Cutout shape: {shape}") - return load_asset_node_group(CUTOUT_NODE_GROUP_NAME) diff --git a/src/anyimage/operators/cutout_tool/region_to_shape.py b/src/anyimage/operators/cutout_tool/region_to_shape.py new file mode 100644 index 0000000..97c32c0 --- /dev/null +++ b/src/anyimage/operators/cutout_tool/region_to_shape.py @@ -0,0 +1,842 @@ +import json +from dataclasses import dataclass +from enum import StrEnum + +import bpy + +from ...preferences import addon_preferences, configured_max_ai_input_size +from ...properties import BEN2_MODEL_KEY, DEFAULT_MOGE2_MODEL_KEY +from ...runtime import JobOperatorBase +from ...server.media.resolution import limited_image_size +from ...server.media.resolution import ( + DEFAULT_MAX_AI_INPUT_SIZE, + MAX_MAX_AI_INPUT_SIZE, + MIN_MAX_AI_INPUT_SIZE, +) +from ..common.image_data import ( + cleanup_image_input, + color_data_name, + depth_data_name, + image_rgba, + load_normal_result_image, + image_empty_bounds, + prepare_region_input, + require_image_empty, + source_start_frame, + video_total_frames, +) +from ..common.image_interaction import report_image_edit_exception +from ..common.image_region import ( + ImageEditWarning, + RegionMask, + RegionPath, + rasterize_region_path, +) +from ..common.mesh_result import ( + create_image_material, + finalize_object_result, + load_asset_node_group, + load_image_geometry, + modifier_input_identifier, + set_modifier_input, +) +from ..common.ai_job import ( + invoke_ai_setup_if_needed, + moge2_parameters, + production_device, + require_environment, + require_input_path, + require_model, +) +from .geometry.depth_calibration import ( + DEFAULT_DEPTH_BASELINE, + DEFAULT_DEPTH_DIRECTION, + depth_balloon_projection_parameters, + metric_depth_reference, + use_adaptive_depth_calibration, +) +from .geometry.triangulation import ( + metric_transform, + triangulate_region_contours, +) +from .geometry.balloon import balloon_profile as calculate_balloon_profile +from .geometry.region_contours import extract_region_contours + +CUTOUT_ALPHA_THRESHOLD = 0.1 +CUTOUT_CLEANUP_UI_SCALE = 100.0 +BALLOON_ATTRIBUTE_NAME = "o_balloon" +CUTOUT_NODE_GROUP_NAME = "O Image Cutout" +CUTOUT_NODE_GROUP_NAMES = { + "SURFACE": "O Image Surface", + "BALLOON": "O Image Balloon", + "DEPTH_BALLOON": "O Image Depth Balloon", + "DEPTH_SURFACE": "O Image Depth Surface", +} +CUTOUT_SHAPE_INDICES = { + shape: index for index, shape in enumerate(CUTOUT_NODE_GROUP_NAMES) +} +CUTOUT_SHAPES = frozenset(CUTOUT_NODE_GROUP_NAMES) +DEPTH_CUTOUT_SHAPES = frozenset({"DEPTH_BALLOON", "DEPTH_SURFACE"}) +_HIDDEN = {"HIDDEN", "SKIP_SAVE"} + + +class NormalMode(StrEnum): + NONE = "NONE" + TANGENT = "TANGENT" + OBJECT = "OBJECT" + + +@dataclass(frozen=True) +class BaseShape: + vertices: object + faces: object + uv: object + balloon: object + +CUTOUT_AXIS_TO_SOURCE = ( + (0.0, -1.0, 0.0, 0.0), + (0.0, 0.0, 1.0, 0.0), + (-1.0, 0.0, 0.0, 0.0), + (0.0, 0.0, 0.0, 1.0), +) + + +def _cutout_depth_image(source_object, points, valid, camera_depth): + import numpy as np + + points = np.asarray(points, dtype=np.float32) + valid = np.asarray(valid, dtype=bool) + height, width = valid.shape + if points.shape != (height, width, 3): + raise ValueError("Depth Surface point dimensions do not match") + discontinuity = _depth_discontinuity_map(camera_depth, valid) + discontinuity[~valid] = -1.0 + pixels = np.empty((height, width, 4), dtype=np.float32) + pixels[:, :, :3] = points + pixels[:, :, 3] = discontinuity + image = bpy.data.images.new( + depth_data_name(source_object), + width=width, + height=height, + alpha=True, + float_buffer=True, + ) + image.colorspace_settings.name = "Non-Color" + image.alpha_mode = "CHANNEL_PACKED" + image.pixels.foreach_set(np.flipud(pixels).ravel()) + image.pack() + return image + + +def _cutout_world_matrix(source_matrix, crop_center): + from mathutils import Matrix + + center_x, center_y = (float(value) for value in crop_center) + return ( + source_matrix + @ Matrix.Translation((center_x, center_y, 0.0)) + @ Matrix(CUTOUT_AXIS_TO_SOURCE) + ) + + +def _orient_cutout_vertices(vertices): + import numpy as np + + vertices = np.asarray(vertices, dtype=np.float64) + return np.column_stack( + ( + -vertices[:, 2], + -vertices[:, 0], + vertices[:, 1], + ) + ) + + +def _cutout_domain( + region_contours, + bounds, + spacing, + world_metric, + balloon_algorithm="POISSON", +): + import numpy as np + + x_min, x_max, y_min, y_max = bounds + + planar, planar_faces, boundary, components = triangulate_region_contours( + region_contours, + bounds, + spacing=spacing, + world_metric=world_metric, + return_components=True, + ) + profile = calculate_balloon_profile( + balloon_algorithm, planar, planar_faces, boundary, components + ) + faces = np.asarray(planar_faces, dtype=np.int32) + local = planar @ np.linalg.inv(metric_transform(world_metric)) + uv = np.column_stack( + ( + (local[:, 0] - x_min) / (x_max - x_min), + (local[:, 1] - y_min) / (y_max - y_min), + ) + ).astype(np.float32) + crop_center = np.asarray( + ((x_min + x_max) * 0.5, (y_min + y_max) * 0.5), + dtype=np.float64, + ) + local -= crop_center + vertices = np.column_stack((local, np.zeros(len(local), dtype=np.float64))) + return vertices, faces, uv, boundary, planar, profile + + +def _world_plane_metric(matrix_world): + from mathutils import Vector + + linear = matrix_world.to_3x3() + x = linear @ Vector((1.0, 0.0, 0.0)) + y = linear @ Vector((0.0, 1.0, 0.0)) + z = linear @ Vector((0.0, 0.0, 1.0)) + return ((x.dot(x), x.dot(y)), (x.dot(y), y.dot(y))), z.length + + +def _cutout_cleanup_value(spacing, world_metric): + import numpy as np + + if spacing <= 0.0: + raise ValueError("Cutout point spacing must be greater than zero") + area_scale = abs(float(np.linalg.det(metric_transform(world_metric)))) + if area_scale <= 1e-12: + raise ValueError("The Image Empty has a degenerate world transform") + return CUTOUT_CLEANUP_UI_SCALE * float(spacing) ** 2 / area_scale + + +def _depth_uniform_scale( + source_object, + field, + bounds=None, +): + import numpy as np + + intrinsics = field.intrinsics + width, height = field.image_size + x_min, x_max, y_min, y_max = bounds or image_empty_bounds(source_object) + plane_width = x_max - x_min + plane_height = y_max - y_min + fx, fy = float(intrinsics[0, 0]), float(intrinsics[1, 1]) + if fx <= 0.0 or fy <= 0.0: + raise ValueError("Camera intrinsics contain an invalid focal length") + scale_x = plane_width * fx / (max(width - 1, 1) * field.reference_depth) + scale_y = plane_height * fy / (max(height - 1, 1) * field.reference_depth) + return float(np.sqrt(scale_x * scale_y)) + + +def _depth_discontinuity_map(depth, valid): + import numpy as np + + depth = np.asarray(depth, dtype=np.float32) + valid = np.asarray(valid, dtype=bool) & np.isfinite(depth) & (depth > 0.0) + padded_depth = np.pad(depth, 1, mode="edge") + padded_valid = np.pad(valid, 1, mode="edge") + windows = np.lib.stride_tricks.sliding_window_view(padded_depth, (3, 3)) + usable = np.lib.stride_tricks.sliding_window_view(padded_valid, (3, 3)) + minimum = np.min(np.where(usable, windows, np.inf), axis=(-2, -1)) + maximum = np.max(np.where(usable, windows, -np.inf), axis=(-2, -1)) + result = (maximum - minimum) / np.maximum(np.abs(depth), 1e-6) + result[~usable.all(axis=(-2, -1))] = 1.0 + return result.astype(np.float32) + + +def _counter_clockwise_faces(points, faces): + import numpy as np + + points = np.asarray(points, dtype=np.float64) + result = np.asarray(faces, dtype=np.int32).copy() + polygons = points[result] + following = np.roll(polygons, -1, axis=1) + signed_area = np.sum( + polygons[..., 0] * following[..., 1] + - polygons[..., 1] * following[..., 0], + axis=1, + ) + result[signed_area < 0.0] = result[signed_area < 0.0, ::-1] + return result + + +def _assign_uv(mesh, uv): + import numpy as np + + for layer in list(mesh.uv_layers): + mesh.uv_layers.remove(layer) + uv_layer = mesh.uv_layers.new(name="UVMap") + loop_vertices = np.empty(len(mesh.loops), dtype=np.int32) + mesh.loops.foreach_get("vertex_index", loop_vertices) + loop_uv = np.asarray(uv, dtype=np.float32)[loop_vertices] + uv_layer.data.foreach_set("uv", loop_uv.ravel()) + + +def _assign_float_point_attribute(mesh, name, values): + import numpy as np + + attribute = mesh.attributes.new(name=name, type="FLOAT", domain="POINT") + attribute.data.foreach_set("value", np.asarray(values, dtype=np.float32)) + + +def _crop_plane_bounds(source_object, crop_bounds, source_size): + source_width, source_height = (float(value) for value in source_size) + left, top, right, bottom = (float(value) for value in crop_bounds) + x_min, x_max, y_min, y_max = image_empty_bounds(source_object) + width = x_max - x_min + height = y_max - y_min + return ( + x_min + left * width / source_width, + x_min + right * width / source_width, + y_max - bottom * height / source_height, + y_max - top * height / source_height, + ) + + +def _cropped_color_image(source_object, crop_bounds, source_rgba): + import numpy as np + + source = source_object.data + rgba = np.asarray(source_rgba) + left, top, right, bottom = crop_bounds + width = right - left + height = bottom - top + if width <= 0 or height <= 0: + raise ValueError("The Cutout crop dimensions must be positive") + cropped = np.zeros((height, width, 4), dtype=np.float32) + source_height, source_width = rgba.shape[:2] + source_left = max(0, left) + source_top = max(0, top) + source_right = min(source_width, right) + source_bottom = min(source_height, bottom) + if source_right > source_left and source_bottom > source_top: + cropped[ + source_top - top : source_bottom - top, + source_left - left : source_right - left, + ] = rgba[source_top:source_bottom, source_left:source_right] + image = bpy.data.images.new( + color_data_name(source_object), + width=width, + height=height, + alpha=True, + ) + try: + image.colorspace_settings.name = source.colorspace_settings.name + image.alpha_mode = source.alpha_mode + image.pixels.foreach_set(np.flipud(cropped).ravel()) + image.pack() + image.update() + except Exception: + bpy.data.images.remove(image, do_unlink=True) + raise + return image + + +def _extract_cutout_contours( + region_mask, + mesh_detail, + threshold=CUTOUT_ALPHA_THRESHOLD, +): + contour_resolution = max(512, min(int(mesh_detail) * 8, 1024)) + return extract_region_contours( + region_mask, + max_resolution=contour_resolution, + threshold=threshold, + ) + + +def cutout_node_group(shape): + if shape not in CUTOUT_SHAPES: + raise ValueError(f"Unsupported Cutout shape: {shape}") + return load_asset_node_group(CUTOUT_NODE_GROUP_NAME) + + +def limited_cutout_mesh_detail(mesh_detail, crop_size, max_input_size): + crop_width, crop_height = (int(value) for value in crop_size) + limited_width, limited_height = limited_image_size( + (crop_width, crop_height), + max_input_size, + ) + scale = max(limited_width, limited_height) / max(crop_width, crop_height) + return max(4, min(1024, round(int(mesh_detail) * scale))) + + +def _cutout_point_spacing(source_object, detail): + import math + + x_min, x_max, y_min, y_max = image_empty_bounds(source_object) + metric, _world_z_scale = _world_plane_metric(source_object.matrix_world) + width = (x_max - x_min) * math.sqrt(max(metric[0][0], 0.0)) + height = (y_max - y_min) * math.sqrt(max(metric[1][1], 0.0)) + return max(width, height) / max(int(detail), 8) + + +def build_base_shape(context, source_object, detail, region_mask): + import numpy as np + + bounds = _crop_plane_bounds( + source_object, + region_mask.bounds, + tuple(source_object.data.size), + ) + region_contours = _extract_cutout_contours( + region_mask, + detail, + threshold=CUTOUT_ALPHA_THRESHOLD, + ) + world_metric, world_z_scale = _world_plane_metric(source_object.matrix_world) + if world_z_scale <= 1e-10: + raise ValueError("The Image Empty has a degenerate world transform") + preferences = addon_preferences(context) + balloon_algorithm = getattr( + preferences, "balloon_profile_algorithm", "POISSON" + ) + point_spacing = _cutout_point_spacing(source_object, detail) + domain = _cutout_domain( + region_contours, + bounds, + point_spacing, + world_metric, + balloon_algorithm, + ) + source_vertices, faces, uv, _boundary, planar, balloon = domain + return BaseShape( + vertices=_orient_cutout_vertices(source_vertices), + faces=_counter_clockwise_faces(planar, faces), + uv=np.asarray(uv, dtype=np.float32), + balloon=np.asarray(balloon, dtype=np.float64) / world_z_scale, + ) + + +def create_shape_object( + context, + source_object, + shape, + detail, + region_mask, + base_shape, + *, + field=None, + normal_image=None, + normal_mode=NormalMode.NONE, + source_rgba=None, +): + import numpy as np + + normal_mode = NormalMode(normal_mode) + bounds = _crop_plane_bounds( + source_object, + region_mask.bounds, + tuple(source_object.data.size), + ) + crop_center = ( + (bounds[0] + bounds[1]) * 0.5, + (bounds[2] + bounds[3]) * 0.5, + ) + world_metric, _world_z_scale = _world_plane_metric(source_object.matrix_world) + cleanup_value = _cutout_cleanup_value( + _cutout_point_spacing(source_object, detail), + world_metric, + ) + color_image = _cropped_color_image( + source_object, + region_mask.bounds, + image_rgba(source_object.data) if source_rgba is None else source_rgba, + ) + material = None + depth_image = None + mesh = None + mesh_object = None + try: + material = create_image_material( + source_object, + color_image, + 0.0, + normal_image=normal_image, + normal_space=( + NormalMode.TANGENT.value + if normal_mode == NormalMode.NONE + else normal_mode.value + ), + scene=context.scene, + ) + depth_reference = None + depth_uniform_scale = None + depth_direction = None + if shape in DEPTH_CUTOUT_SHAPES: + points = field.points + valid = field.valid + calibration_valid = field.calibration_valid + depth_image = _cutout_depth_image( + source_object, points, valid, points[..., 2] + ) + depth_uniform_scale = _depth_uniform_scale( + source_object, + field, + bounds, + ) + if not use_adaptive_depth_calibration( + field.region_transparent_fraction + ): + depth_reference = DEFAULT_DEPTH_BASELINE + if shape == "DEPTH_BALLOON": + depth_direction = DEFAULT_DEPTH_DIRECTION + elif shape == "DEPTH_SURFACE": + depth_reference = metric_depth_reference( + points[..., 2], + calibration_valid, + depth_uniform_scale, + ) + else: + depth_reference, depth_direction = ( + depth_balloon_projection_parameters( + base_shape.vertices, + base_shape.uv, + points[..., 2], + calibration_valid, + depth_uniform_scale, + ) + ) + + source_name = source_object.name + mesh = bpy.data.meshes.new(source_name) + mesh.from_pydata( + np.asarray(base_shape.vertices).tolist(), + [], + base_shape.faces, + ) + for face in mesh.polygons: + face.use_smooth = True + _assign_uv(mesh, base_shape.uv) + _assign_float_point_attribute( + mesh, + BALLOON_ATTRIBUTE_NAME, + base_shape.balloon, + ) + mesh.materials.append(material) + mesh.update() + + mesh_object = bpy.data.objects.new(source_name, mesh) + context.collection.objects.link(mesh_object) + mesh_object.matrix_world = _cutout_world_matrix( + source_object.matrix_world, + crop_center, + ) + mesh_object["anyimage_mesh_shape"] = shape + node_group = cutout_node_group(shape) + modifier = mesh_object.modifiers.new(name=node_group.name, type="NODES") + modifier.node_group = node_group + modifier.show_group_selector = False + for name, value in ( + ("Shape", CUTOUT_SHAPE_INDICES[shape]), + ("Cleanup", cleanup_value), + ("Smooth Iterations", 0), + ("Smooth", 1.0), + ( + "Thickness", + 1.0 if shape in {"BALLOON", "DEPTH_BALLOON"} else 0.0, + ), + ): + set_modifier_input( + modifier, + modifier_input_identifier(node_group, name), + value, + ) + if shape in DEPTH_CUTOUT_SHAPES: + for name, value in ( + ("Depth Scale", 1.0), + ("Uniform Scale", depth_uniform_scale), + ("Base Plane Depth", depth_reference), + ("Depth Image", depth_image), + ): + set_modifier_input( + modifier, + modifier_input_identifier(node_group, name), + value, + ) + if shape == "DEPTH_BALLOON": + set_modifier_input( + modifier, + modifier_input_identifier(node_group, "Depth Direction"), + depth_direction, + ) + return finalize_object_result( + context, + source_object, + mesh_object, + keep_source=True, + ) + except Exception: + if mesh_object is not None: + bpy.data.objects.remove(mesh_object, do_unlink=True) + if mesh is not None and mesh.users == 0: + bpy.data.meshes.remove(mesh) + if material is not None and material.users == 0: + bpy.data.materials.remove(material) + if color_image.users == 0: + bpy.data.images.remove(color_image, do_unlink=True) + if normal_image is not None and normal_image.users == 0: + bpy.data.images.remove(normal_image, do_unlink=True) + if depth_image is not None and depth_image.users == 0: + bpy.data.images.remove(depth_image, do_unlink=True) + raise + + +def region_to_shape( + context, + source_object, + shape, + detail, + region_mask, + *, + field=None, + normal_image=None, + normal_mode=NormalMode.NONE, + source_rgba=None, +): + crop_width = region_mask.bounds[2] - region_mask.bounds[0] + crop_height = region_mask.bounds[3] - region_mask.bounds[1] + detail = limited_cutout_mesh_detail( + detail, + (crop_width, crop_height), + configured_max_ai_input_size(context), + ) + base_shape = build_base_shape( + context, + source_object, + detail, + region_mask, + ) + result = create_shape_object( + context, + source_object, + shape, + detail, + region_mask, + base_shape, + field=field, + normal_image=normal_image, + normal_mode=normal_mode, + source_rgba=source_rgba, + ) + undo_message = f"Convert to {shape.replace('_', ' ').title()}" + if "FINISHED" not in bpy.ops.ed.undo_push(message=undo_message): + raise RuntimeError(f"Unable to create the {undo_message} Undo step") + return result + + +def _region_mask_from_image(path, bounds): + image = bpy.data.images.load(str(path), check_existing=False) + try: + return RegionMask( + image_rgba(image)[:, :, 3], + tuple(int(value) for value in bounds), + ) + finally: + bpy.data.images.remove(image, do_unlink=True) + + +def _generated_region_mask(result, input_path, source_bounds): + if not isinstance(result.value, dict) or "region" not in result.value: + return _region_mask_from_image(input_path, source_bounds) + local_bounds = tuple( + int(value) + for value in json.loads( + result.file("region_bounds").read_text(encoding="utf-8") + ) + ) + bounds = ( + source_bounds[0] + local_bounds[0], + source_bounds[1] + local_bounds[1], + source_bounds[0] + local_bounds[2], + source_bounds[1] + local_bounds[3], + ) + return _region_mask_from_image(result.file("region"), bounds) + + +def cutout_moge_settings(context): + preferences = addon_preferences(context) + model = getattr(preferences, "mesh_depth_model", DEFAULT_MOGE2_MODEL_KEY) + if not model.startswith("MOGE2_"): + model = DEFAULT_MOGE2_MODEL_KEY + return ( + model, + int(getattr(preferences, "mesh_resolution_level", 5)), + ) + + +class CutoutRegionToShape(JobOperatorBase, bpy.types.Operator): + bl_idname = "anyimage.cutout_region_to_shape" + bl_label = "Cutout Region to Shape" + bl_description = "Build the selected image Region as a Cutout shape" + bl_options = {"INTERNAL"} + job_type = "generate-region-data" + starting_message = "Generating Cutout Region data" + + input_path: bpy.props.StringProperty(options=_HIDDEN) + source_object_name: bpy.props.StringProperty(options=_HIDDEN) + region_path_json: bpy.props.StringProperty(options=_HIDDEN) + region_bounds_json: bpy.props.StringProperty(options=_HIDDEN) + shape: bpy.props.StringProperty(options=_HIDDEN) + mesh_detail: bpy.props.IntProperty( + default=128, + min=4, + max=1024, + options=_HIDDEN, + ) + model: bpy.props.StringProperty( + default=DEFAULT_MOGE2_MODEL_KEY, + options=_HIDDEN, + ) + resolution_level: bpy.props.IntProperty( + default=5, + min=0, + max=9, + options=_HIDDEN, + ) + max_input_size: bpy.props.IntProperty( + default=DEFAULT_MAX_AI_INPUT_SIZE, + min=MIN_MAX_AI_INPUT_SIZE, + max=MAX_MAX_AI_INPUT_SIZE, + options=_HIDDEN, + ) + normal_mode: bpy.props.StringProperty( + default=NormalMode.NONE.value, + options=_HIDDEN, + ) + refine_selection: bpy.props.BoolProperty(default=False, options=_HIDDEN) + + def _needs_generation(self): + return ( + self.refine_selection + or self.shape in DEPTH_CUTOUT_SHAPES + or NormalMode(self.normal_mode) != NormalMode.NONE + ) + + def execute(self, context): + try: + source_object = require_image_empty(self.source_object_name) + if video_total_frames(source_object.data) != 1: + raise ValueError("Cutout currently supports still images") + if self.shape not in CUTOUT_SHAPES: + raise ValueError(f"Unsupported Cutout shape: {self.shape}") + self.normal_mode = NormalMode(self.normal_mode).value + region_path = RegionPath.from_json(self.region_path_json) + source_rgba = image_rgba(source_object.data) + region_mask = rasterize_region_path( + tuple(source_object.data.size), + region_path, + source_alpha=source_rgba[:, :, 3], + alpha_threshold=CUTOUT_ALPHA_THRESHOLD, + antialias=True, + ) + context.view_layer.objects.active = source_object + source_object.select_set(True) + self.max_input_size = configured_max_ai_input_size(context) + if not self._needs_generation(): + region_to_shape( + context, + source_object, + self.shape, + self.mesh_detail, + region_mask, + source_rgba=source_rgba, + ) + return {"FINISHED"} + setup_result = invoke_ai_setup_if_needed() + if setup_result is not None: + return setup_result + self.model, self.resolution_level = cutout_moge_settings(context) + self.region_bounds_json = json.dumps( + region_mask.bounds, + separators=(",", ":"), + ) + self.input_path = str( + prepare_region_input(source_object.data, region_mask) + ) + except ( + ImageEditWarning, + OSError, + RuntimeError, + TypeError, + ValueError, + ) as error: + report_image_edit_exception(self, error) + return {"CANCELLED"} + result = super().execute(context) + if "CANCELLED" in result: + cleanup_image_input(self.input_path, True) + return result + + def request(self, context): + require_environment() + source_path = require_input_path(self.input_path) + output_depth = self.shape in DEPTH_CUTOUT_SHAPES + normal_mode = NormalMode(self.normal_mode) + if self.refine_selection: + require_model(BEN2_MODEL_KEY) + if output_depth or normal_mode != NormalMode.NONE: + require_model(self.model) + parameters = { + "input": str(source_path), + "device": production_device().lower(), + "refine_region": self.refine_selection, + "output_depth": output_depth, + "normal_mode": normal_mode.value, + "max_input_size": int(self.max_input_size), + } + if self.refine_selection: + parameters["refine_model"] = BEN2_MODEL_KEY + if output_depth or normal_mode != NormalMode.NONE: + start_frame = source_start_frame(self.source_object_name) + parameters.update( + model=self.model, + resolution_level=self.resolution_level, + ) + parameters.update( + moge2_parameters( + context.scene.anyimage_settings, + source_path, + self.model, + self.resolution_level, + start_frame, + ) + ) + return parameters + + def response(self, context, result): + source_object = require_image_empty(self.source_object_name) + source_bounds = tuple(json.loads(self.region_bounds_json)) + region_mask = _generated_region_mask( + result, + self.input_path, + source_bounds, + ) + value = result.value if isinstance(result.value, dict) else {} + normal_image = ( + load_normal_result_image(result.file("normal"), source_object) + if "normal" in value + else None + ) + field = ( + load_image_geometry(result.file("image_geometry")) + if self.shape in DEPTH_CUTOUT_SHAPES + else None + ) + region_to_shape( + context, + source_object, + self.shape, + self.mesh_detail, + region_mask, + field=field, + normal_image=normal_image, + normal_mode=NormalMode(self.normal_mode), + ) + return f"{self.shape.replace('_', ' ').title()} is ready" + + def cleanup(self): + cleanup_image_input(self.input_path, True) diff --git a/src/anyimage/operators/cutout_tool/selection_tool.py b/src/anyimage/operators/cutout_tool/select_region.py similarity index 65% rename from src/anyimage/operators/cutout_tool/selection_tool.py rename to src/anyimage/operators/cutout_tool/select_region.py index 9d3c932..57f147a 100644 --- a/src/anyimage/operators/cutout_tool/selection_tool.py +++ b/src/anyimage/operators/cutout_tool/select_region.py @@ -1,27 +1,42 @@ -import json - import bpy from ...properties import ( refine_selection_property, ) -from ..common.image import is_image_empty, video_total_frames -from ..common.image_edit import ( - ImageEditWarning, - apply_image_mask_settings, - draw_image_mask_settings, - polygon_bounds, +from ..common.image_data import image_rgba, require_image_empty, video_total_frames +from ..common.image_interaction import ( + apply_region_settings, + draw_region_settings, report_image_edit_exception, resolve_image_edit_click, ) -from ..common.image_interaction import hidden_string_property, serialize_matrix -from ..common.image_gesture import ImageMaskGesture, gesture_property -from ..common.image_mask import selected_image_rgba +from ..common.image_interaction import serialize_matrix +from ..common.image_interaction import ImageRegionGesture, gesture_property +from ..common.image_region import ( + ImageEditWarning, + RegionPath, + rasterize_region_path, +) CUTOUT_DETAIL_MIN = 4 CUTOUT_DETAIL_MAX = 1024 +def _validate_cutout_region( + source_object, + region_path, +): + image = source_object.data + rgba = image_rgba(image) + rasterize_region_path( + tuple(image.size), + region_path, + source_alpha=rgba[:, :, 3], + alpha_threshold=0.1, + antialias=True, + ) + + def cutout_mesh_detail(image_size, point_spacing): longest_edge = max(int(value) for value in image_size) detail = round(longest_edge / max(int(point_spacing), 1)) @@ -34,23 +49,21 @@ def _pie_button( label, icon, source_name, - selection_data, + region_path_json, refine_selection, - normal_analysis, - invert, + normal_mode, mesh_detail, ): operator = pie.operator( - "anyimage.convert_cutout_shape", + "anyimage.cutout_region_to_shape", text=label, icon=icon, ) operator.source_object_name = source_name - operator.selection_data = selection_data + operator.region_path_json = region_path_json operator.shape = shape operator.refine_selection = refine_selection - operator.normal_analysis = normal_analysis - operator.invert = invert + operator.normal_mode = normal_mode operator.mesh_detail = mesh_detail @@ -58,29 +71,27 @@ def _open_shape_pie( context, event, source_object, - polygon, + region_path, refine_selection=False, - invert=False, ): source_name = source_object.name - selection_data = json.dumps(polygon, separators=(",", ":")) + region_path_json = region_path.to_json() settings = context.scene.anyimage_settings point_spacing = settings.cutout_point_spacing - normal_analysis = bool(getattr(settings, "cutout_normal_analysis", False)) + normal_mode = str(getattr(settings, "cutout_normal_mode", "NONE")) mesh_detail = cutout_mesh_detail(source_object.data.size, point_spacing) def draw(menu, _context): pie = menu.layout.menu_pie() _pie_button( pie, - "CUTOUT", + "SURFACE", "Surface", "MESH_GRID", source_name, - selection_data, + region_path_json, refine_selection, - normal_analysis, - invert, + normal_mode, mesh_detail, ) _pie_button( @@ -89,10 +100,9 @@ def _open_shape_pie( "Balloon", "MESH_UVSPHERE", source_name, - selection_data, + region_path_json, refine_selection, - normal_analysis, - invert, + normal_mode, mesh_detail, ) _pie_button( @@ -101,10 +111,9 @@ def _open_shape_pie( "Depth Balloon", "META_BALL", source_name, - selection_data, + region_path_json, refine_selection, - normal_analysis, - invert, + normal_mode, mesh_detail, ) _pie_button( @@ -113,10 +122,9 @@ def _open_shape_pie( "Depth Surface", "SURFACE_NCURVE", source_name, - selection_data, + region_path_json, refine_selection, - normal_analysis, - invert, + normal_mode, mesh_detail, ) @@ -128,16 +136,16 @@ def _open_shape_pie( ) -class SelectCutoutRegion(ImageMaskGesture, bpy.types.Operator): +class SelectCutoutRegion(ImageRegionGesture, bpy.types.Operator): bl_idname = "anyimage.select_cutout_region" bl_label = "Select Cutout Region" bl_description = "Lasso an Image Empty region, then choose its mesh shape" bl_options = {"INTERNAL"} active_tool_id = "anyimage.cutout_lasso" - source_object_name: hidden_string_property() - source_matrix_data: hidden_string_property() - selection_data: hidden_string_property() + source_object_name: bpy.props.StringProperty(options={"HIDDEN", "SKIP_SAVE"}) + source_matrix_data: bpy.props.StringProperty(options={"HIDDEN", "SKIP_SAVE"}) + region_path_json: bpy.props.StringProperty(options={"HIDDEN", "SKIP_SAVE"}) refine_selection: refine_selection_property() invert: bpy.props.BoolProperty(default=False, options={"SKIP_SAVE"}) gesture: gesture_property() @@ -167,15 +175,15 @@ class SelectCutoutRegion(ImageMaskGesture, bpy.types.Operator): if video_total_frames(source_object.data) > 1: self.report({"WARNING"}, "Image tools only support a still image") return {"CANCELLED"} - apply_image_mask_settings(self, context) + apply_region_settings(self, context) self.invert = False self.source_object_name = source_object.name self.source_matrix_data = serialize_matrix(source_object.matrix_world) width, height = (int(value) for value in source_object.data.size) - self.selection_data = json.dumps( - ((0, 0), (width, 0), (width, height), (0, height)), - separators=(",", ":"), - ) + self.region_path_json = RegionPath( + points=((0, 0), (width, 0), (width, height), (0, height)), + fill_rule="NONZERO", + ).to_json() return self.execute(context) def modal(self, context, event): @@ -183,47 +191,27 @@ class SelectCutoutRegion(ImageMaskGesture, bpy.types.Operator): return super().modal(context, event) def execute(self, context): - source_object = bpy.data.objects.get(self.source_object_name) - if not is_image_empty(source_object): - self.report({"ERROR"}, "The source Image Empty is no longer available") + try: + source_object = require_image_empty(self.source_object_name) + except RuntimeError as error: + self.report({"ERROR"}, str(error)) return {"CANCELLED"} try: - polygon = json.loads(self.selection_data) - polygon_bounds(polygon) - selected_image_rgba( - source_object, polygon, self.invert, "NONZERO" - ) - except (ImageEditWarning, json.JSONDecodeError, TypeError, ValueError) as error: + region_path = RegionPath.from_json(self.region_path_json) + _validate_cutout_region(source_object, region_path) + except (ImageEditWarning, TypeError, ValueError) as error: report_image_edit_exception(self, error) return {"CANCELLED"} _open_shape_pie( context, self._pie_event, source_object, - polygon, + region_path, self.refine_selection, - self.invert, ) return {"FINISHED"} -class ActivateCutoutTool(bpy.types.Operator): - bl_idname = "anyimage.activate_cutout_tool" - bl_label = "Cutout Tool" - bl_description = "Activate the Cutout Lasso tool" - bl_options = {"INTERNAL"} - - @classmethod - def poll(cls, context): - return context.area is not None and context.area.type == "VIEW_3D" - - def execute(self, _context): - return bpy.ops.wm.tool_set_by_id( - name=CutoutTool.bl_idname, - space_type="VIEW_3D", - ) - - class CutoutTool(bpy.types.WorkSpaceTool): bl_space_type = "VIEW_3D" bl_context_mode = "OBJECT" @@ -248,5 +236,5 @@ class CutoutTool(bpy.types.WorkSpaceTool): def draw_settings(context, layout, _tool): settings = context.scene.anyimage_settings layout.prop(settings, "cutout_point_spacing") - layout.prop(settings, "cutout_normal_analysis") - draw_image_mask_settings(context, layout, "cutout_refine_selection") + layout.prop(settings, "cutout_normal_mode") + draw_region_settings(context, layout, "cutout_refine_selection") diff --git a/src/anyimage/operators/cutout_tool/shape_conversion.py b/src/anyimage/operators/cutout_tool/shape_conversion.py deleted file mode 100644 index db706b7..0000000 --- a/src/anyimage/operators/cutout_tool/shape_conversion.py +++ /dev/null @@ -1,214 +0,0 @@ -import json - -import bpy - -from ...preferences import addon_preferences, configured_max_ai_input_size -from ...properties import BEN2_MODEL_KEY, DEFAULT_MOGE2_MODEL_KEY -from ...runtime import JobOperatorBase -from ...server.media.resolution import ( - DEFAULT_MAX_AI_INPUT_SIZE, - MAX_MAX_AI_INPUT_SIZE, - MIN_MAX_AI_INPUT_SIZE, -) -from ..common.image import ( - is_image_empty, - prepare_image_input, - remove_temporary_input, - video_total_frames, -) -from ..common.image_edit import report_image_edit_exception -from ..common.operator import ( - HIDDEN, - cleanup_temporary_input, - input_path, - invoke_ai_setup_if_needed, - moge2_parameters, - production_device, - require_environment, - require_model, - source_start_frame, -) -from .build_shapes import build_cutout_response, build_local_cutout -from .shape_types import DEPTH_CUTOUT_SHAPES, LOCAL_CUTOUT_SHAPES - - -def cutout_moge_settings(context): - preferences = addon_preferences(context) - model = getattr(preferences, "mesh_depth_model", DEFAULT_MOGE2_MODEL_KEY) - if not model.startswith("MOGE2_"): - model = DEFAULT_MOGE2_MODEL_KEY - return ( - model, - int(getattr(preferences, "mesh_resolution_level", 5)), - ) - - -class GenerateCutoutShape(JobOperatorBase, bpy.types.Operator): - bl_idname = "anyimage.generate_cutout_shape" - bl_label = "Generate Cutout Shape" - job_type = "prepare-geometry-data" - starting_message = "Preparing Cutout shape" - - input_path: bpy.props.StringProperty(options=HIDDEN) - source_object_name: bpy.props.StringProperty(options=HIDDEN) - shape: bpy.props.StringProperty(options=HIDDEN) - selection_data: bpy.props.StringProperty(options=HIDDEN) - mesh_detail: bpy.props.IntProperty(default=128, min=4, max=1024, options=HIDDEN) - model: bpy.props.StringProperty(default=DEFAULT_MOGE2_MODEL_KEY, options=HIDDEN) - resolution_level: bpy.props.IntProperty(default=5, min=0, max=9, options=HIDDEN) - max_input_size: bpy.props.IntProperty( - default=DEFAULT_MAX_AI_INPUT_SIZE, - min=MIN_MAX_AI_INPUT_SIZE, - max=MAX_MAX_AI_INPUT_SIZE, - options=HIDDEN, - ) - normal_analysis: bpy.props.BoolProperty(default=False, options=HIDDEN) - delete_input: bpy.props.BoolProperty(default=False, options=HIDDEN) - refine_selection: bpy.props.BoolProperty(default=False, options=HIDDEN) - invert: bpy.props.BoolProperty(default=False, options=HIDDEN) - - def request(self, context): - require_environment() - if self.shape in DEPTH_CUTOUT_SHAPES or self.normal_analysis: - require_model(self.model) - if self.refine_selection: - require_model(BEN2_MODEL_KEY) - polygon = json.loads(self.selection_data) if self.selection_data else None - if polygon is None or len(polygon) < 3: - raise ValueError("The Cutout selection is invalid") - parameters = { - "input": str(input_path(self.input_path)), - "analysis_input": str(input_path(self.input_path)), - "shape": self.shape, - "polygon": polygon, - "mesh_detail": self.mesh_detail, - "device": production_device().lower(), - "video_frames": 1, - "start_frame": source_start_frame(self.source_object_name), - "max_input_size": int(self.max_input_size), - "refine_selection": self.refine_selection, - "normal_analysis": self.normal_analysis, - "invert": self.invert, - } - if self.shape in DEPTH_CUTOUT_SHAPES or self.normal_analysis: - parameters.update( - model=self.model, - resolution_level=self.resolution_level, - ) - if self.refine_selection: - parameters["refine_model"] = BEN2_MODEL_KEY - if self.shape not in DEPTH_CUTOUT_SHAPES: - return parameters - settings = context.scene.anyimage_settings - parameters.update( - moge2_parameters( - settings, - input_path(self.input_path), - self.model, - self.resolution_level, - source_start_frame(self.source_object_name), - ) - ) - parameters.update(shape=self.shape, polygon=polygon) - return parameters - - def response(self, context, result): - build_cutout_response(context, result, self) - return f"{self.shape.replace('_', ' ').title()} is ready" - - def cleanup(self): - cleanup_temporary_input(self.input_path, self.delete_input) - - -def start_cutout_job( - operator, - context, - shape, - selection_data, - refine_selection=False, - normal_analysis=False, - invert=False, -): - source_object = context.object - if video_total_frames(source_object.data) != 1: - operator.report({"ERROR"}, "Cutout currently supports still images") - return {"CANCELLED"} - setup_result = invoke_ai_setup_if_needed() - if setup_result is not None: - return setup_result - try: - source_path, delete_input = prepare_image_input( - source_object.data, - context.scene, - ) - model, resolution_level = cutout_moge_settings(context) - except (RuntimeError, ValueError) as error: - operator.report({"ERROR"}, str(error)) - return {"CANCELLED"} - result = bpy.ops.anyimage.generate_cutout_shape( - input_path=str(source_path), - source_object_name=source_object.name, - shape=shape, - selection_data=selection_data, - mesh_detail=operator.mesh_detail, - model=model, - resolution_level=resolution_level, - max_input_size=configured_max_ai_input_size(context), - normal_analysis=normal_analysis, - delete_input=delete_input, - refine_selection=refine_selection, - invert=invert, - ) - if "CANCELLED" in result and delete_input: - remove_temporary_input(source_path) - return {"CANCELLED"} if "CANCELLED" in result else {"FINISHED"} - - -class ConvertCutoutShape(bpy.types.Operator): - bl_idname = "anyimage.convert_cutout_shape" - bl_label = "Convert Cutout Shape" - bl_description = "Convert the saved Cutout selection into the chosen shape" - bl_options = {"UNDO"} - - source_object_name: bpy.props.StringProperty(options=HIDDEN) - selection_data: bpy.props.StringProperty(options=HIDDEN) - shape: bpy.props.StringProperty(options=HIDDEN) - mesh_detail: bpy.props.IntProperty(default=128, min=4, max=1024, options=HIDDEN) - refine_selection: bpy.props.BoolProperty(default=False, options=HIDDEN) - normal_analysis: bpy.props.BoolProperty(default=False, options=HIDDEN) - invert: bpy.props.BoolProperty(default=False, options=HIDDEN) - - def execute(self, context): - source_object = bpy.data.objects.get(self.source_object_name) - if not is_image_empty(source_object): - self.report({"ERROR"}, "The source Image Empty is no longer available") - return {"CANCELLED"} - context.view_layer.objects.active = source_object - source_object.select_set(True) - if ( - self.shape in LOCAL_CUTOUT_SHAPES - and not self.refine_selection - and not self.normal_analysis - ): - try: - build_local_cutout( - self, - context, - source_object, - self.shape, - self.selection_data, - self.invert, - ) - except Exception as error: - report_image_edit_exception(self, error) - return {"CANCELLED"} - return {"FINISHED"} - return start_cutout_job( - self, - context, - self.shape, - self.selection_data, - self.refine_selection, - self.normal_analysis, - self.invert, - ) diff --git a/src/anyimage/operators/cutout_tool/shape_types.py b/src/anyimage/operators/cutout_tool/shape_types.py deleted file mode 100644 index 7e86470..0000000 --- a/src/anyimage/operators/cutout_tool/shape_types.py +++ /dev/null @@ -1,23 +0,0 @@ -"""Cutout shape identifiers and Geometry Nodes asset names.""" - -CUTOUT_NODE_GROUP_NAME = "O Image Cutout" -CUTOUT_NODE_GROUP_NAMES = { - "CUTOUT": "O Image Surface", - "BALLOON": "O Image Balloon", - "DEPTH_BALLOON": "O Image Depth Balloon", - "DEPTH_SURFACE": "O Image Depth Surface", -} -CUTOUT_SHAPE_LABELS = { - "CUTOUT": "Surface", - "BALLOON": "Balloon", - "DEPTH_BALLOON": "Depth Balloon", - "DEPTH_SURFACE": "Depth Surface", -} -CUTOUT_SHAPE_INDICES = { - shape: index for index, shape in enumerate(CUTOUT_NODE_GROUP_NAMES) -} - -CUTOUT_SHAPES = frozenset(CUTOUT_NODE_GROUP_NAMES) -LOCAL_CUTOUT_SHAPES = frozenset({"CUTOUT", "BALLOON"}) -DEPTH_CUTOUT_SHAPES = frozenset({"DEPTH_BALLOON", "DEPTH_SURFACE"}) -BALLOON_ATTRIBUTE_NAME = "o_balloon" diff --git a/src/anyimage/operators/debug.py b/src/anyimage/operators/debug.py index 1f36901..9a0c836 100644 --- a/src/anyimage/operators/debug.py +++ b/src/anyimage/operators/debug.py @@ -1,18 +1,28 @@ +from pathlib import Path + import bpy from ..runtime import JobOperatorBase from ..properties import BEN2_MODEL_KEY, REALESRGAN_GENERAL_WDN_MODEL_KEY -from .common.operator import ( - background_input_path, +from .common.ai_job import ( + require_background_input_path, da3_parameters, - input_path, + require_input_path, moge2_parameters, - open_job_directory, require_environment, require_model, ) +def _open_job_directory(directory): + directory = Path(directory) + if not directory.is_dir(): + raise RuntimeError(f"Job directory does not exist: {directory}") + result = bpy.ops.wm.path_open(filepath=str(directory)) + if "FINISHED" not in result: + raise RuntimeError(f"Unable to open job directory: {directory}") + + class DebugDa3(JobOperatorBase, bpy.types.Operator): bl_idname = "anyimage.debug_da3" bl_label = "Debug DA3" @@ -27,7 +37,7 @@ class DebugDa3(JobOperatorBase, bpy.types.Operator): return { **da3_parameters( settings, - input_path(settings.input_path), + require_input_path(settings.input_path), model, settings.process_res, 1, @@ -38,7 +48,7 @@ class DebugDa3(JobOperatorBase, bpy.types.Operator): } def response(self, _context, result): - open_job_directory(result.directory) + _open_job_directory(result.directory) return f"Saved DA3 API prediction to {result.directory}" @@ -55,7 +65,7 @@ class DebugMoge2(JobOperatorBase, bpy.types.Operator): return { **moge2_parameters( settings, - input_path(settings.input_path), + require_input_path(settings.input_path), settings.moge_model, settings.moge_resolution_level, 1, @@ -64,7 +74,7 @@ class DebugMoge2(JobOperatorBase, bpy.types.Operator): } def response(self, _context, result): - open_job_directory(result.directory) + _open_job_directory(result.directory) return f"Exported MoGe-2 data to {result.directory}" @@ -79,7 +89,7 @@ class DebugBen2(JobOperatorBase, bpy.types.Operator): settings = context.scene.anyimage_settings require_model(BEN2_MODEL_KEY) return { - "input": str(background_input_path(settings.background_input_path)), + "input": str(require_background_input_path(settings.background_input_path)), "model": BEN2_MODEL_KEY, "device": settings.device.lower(), "video_frames": settings.video_frames, @@ -87,7 +97,7 @@ class DebugBen2(JobOperatorBase, bpy.types.Operator): } def response(self, _context, result): - open_job_directory(result.directory) + _open_job_directory(result.directory) return f"Removed background to {result.directory}" @@ -107,7 +117,7 @@ class DebugUpscale(JobOperatorBase, bpy.types.Operator): settings = context.scene.anyimage_settings require_model(self.model) return { - "input": str(input_path(settings.input_path)), + "input": str(require_input_path(settings.input_path)), "model": self.model, "device": settings.device.lower(), "video_frames": settings.video_frames, @@ -115,5 +125,5 @@ class DebugUpscale(JobOperatorBase, bpy.types.Operator): } def response(self, _context, result): - open_job_directory(result.directory) + _open_job_directory(result.directory) return f"Saved upscale API prediction to {result.directory}" diff --git a/src/anyimage/operators/image_tool/box.py b/src/anyimage/operators/image_tool/box.py index 1c7d293..66c9d2e 100644 --- a/src/anyimage/operators/image_tool/box.py +++ b/src/anyimage/operators/image_tool/box.py @@ -1,6 +1,6 @@ import bpy -from ..common.image_edit import draw_image_mask_settings +from ..common.image_interaction import draw_region_settings from .edit_mask import EditImageMask @@ -21,4 +21,4 @@ class ImageBoxTool(bpy.types.WorkSpaceTool): ) def draw_settings(context, layout, _tool): - draw_image_mask_settings(context, layout, "image_box_refine_selection") + draw_region_settings(context, layout, "image_box_refine_selection") diff --git a/src/anyimage/operators/image_tool/edit_mask.py b/src/anyimage/operators/image_tool/edit_mask.py index 297b15c..67b2a12 100644 --- a/src/anyimage/operators/image_tool/edit_mask.py +++ b/src/anyimage/operators/image_tool/edit_mask.py @@ -2,38 +2,119 @@ import json import bpy -from ...properties import refine_selection_property +from ...properties import BEN2_MODEL_KEY, refine_selection_property from ...runtime import JobOperatorBase -from ..common.image import ( - is_image_empty, - prepare_image_input, - remove_temporary_input, +from ..common.image_data import ( + cleanup_image_input, + create_pixel_result_image, + image_rgba, + image_pixels, + prepare_region_input, + replace_empty_image, + require_image_empty, video_total_frames, ) -from ..common.image_edit import ( - ImageEditWarning, - image_mask_refine_request, - polygon_bounds, +from ..common.image_interaction import ( + image_edit_poll, report_image_edit_exception, ) -from ..common.image_gesture import ImageMaskGesture, gesture_property -from ..common.image_interaction import hidden_string_property -from ..common.image_mask import ( - edit_image_object, - polygon_mask, - refined_mask_from_pixels, +from ..common.image_interaction import ImageRegionGesture, gesture_property +from ..common.image_region import ( + ImageEditWarning, + RegionPath, + crop_pixels_to_alpha, + rasterize_region_path, +) +from ..common.ai_job import ( + production_device, + require_environment, + require_model, ) -from ..common.operator import HIDDEN, cleanup_temporary_input, push_undo -class EditImageMask(ImageMaskGesture, bpy.types.Operator): + +_HIDDEN = {"HIDDEN", "SKIP_SAVE"} + + +def _refined_mask_from_pixels(mask_pixels, mask_size, bounds, source_size): + import numpy as np + + mask_width, mask_height = (int(value) for value in mask_size) + source_width, source_height = (int(value) for value in source_size) + local = np.flipud( + np.asarray(mask_pixels, dtype=np.float32).reshape( + (mask_height, mask_width, 4) + ) + )[:, :, 3] + result = np.zeros((source_height, source_width), dtype=np.float32) + left, top, right, bottom = (int(value) for value in bounds) + source_left = max(left, 0) + source_top = max(top, 0) + source_right = min(right, source_width) + source_bottom = min(bottom, source_height) + if source_right <= source_left or source_bottom <= source_top: + return result + result[source_top:source_bottom, source_left:source_right] = local[ + source_top - top : source_bottom - top, + source_left - left : source_right - left, + ] + return result + + +def _edit_image_mask_pixels( + source_pixels, + source_size, + region_mask, + invert=False, +): + """Apply a mask while preserving source RGB, then tightly crop the canvas.""" + import numpy as np + + width, height = (int(value) for value in source_size) + rgba = np.flipud( + np.asarray(source_pixels, dtype=np.float32).reshape((height, width, 4)) + ) + mask = np.asarray(region_mask, dtype=np.float32) + if invert: + mask = 1.0 - mask + mask = np.where(rgba[:, :, 3] > 0.0, mask, 0.0) + rgba[:, :, 3] *= mask + return crop_pixels_to_alpha( + np.flipud(rgba).ravel(), + (0, 0, width, height), + ) + + +def _edit_image_object(source_object, region_mask, invert=False): + image = source_object.data + source_size = tuple(image.size) + pixels, crop_bounds = _edit_image_mask_pixels( + image_pixels(image), + source_size, + region_mask, + invert, + ) + left, top, right, bottom = crop_bounds + result_image = create_pixel_result_image( + image, + pixels, + (right - left, bottom - top), + ) + return replace_empty_image( + source_object, + result_image, + placement_bounds=crop_bounds, + ) + + +class EditImageMask(ImageRegionGesture, bpy.types.Operator): bl_idname = "anyimage.edit_image_mask" bl_label = "Edit Image Mask" bl_description = "Edit and tightly crop an Image Empty with a gesture" bl_options = {"REGISTER", "UNDO"} - source_object_name: hidden_string_property() - source_matrix_data: hidden_string_property() - selection_data: hidden_string_property() + source_object_name: bpy.props.StringProperty(options={"HIDDEN", "SKIP_SAVE"}) + source_matrix_data: bpy.props.StringProperty(options={"HIDDEN", "SKIP_SAVE"}) + region_path_json: bpy.props.StringProperty(options={"HIDDEN", "SKIP_SAVE"}) refine_selection: refine_selection_property() invert: bpy.props.BoolProperty( name="Invert", @@ -45,35 +126,37 @@ class EditImageMask(ImageMaskGesture, bpy.types.Operator): def execute(self, context): - source_object = bpy.data.objects.get(self.source_object_name) - if not is_image_empty(source_object): - self.report({"ERROR"}, "The source Image Empty is no longer available") + try: + source_object = require_image_empty(self.source_object_name) + except RuntimeError as error: + self.report({"ERROR"}, str(error)) return {"CANCELLED"} try: - polygon = json.loads(self.selection_data) - polygon_bounds(polygon) + region_path = RegionPath.from_json(self.region_path_json) if video_total_frames(source_object.data) > 1: raise RuntimeError("Image tools only support a still image") + region_mask = rasterize_region_path( + tuple(source_object.data.size), + region_path, + source_alpha=image_rgba(source_object.data)[:, :, 3], + ) if self.refine_selection: - source_path, delete_input = prepare_image_input( - source_object.data, context.scene + source_path = prepare_region_input( + source_object.data, + region_mask, ) result = bpy.ops.anyimage.refine_image_mask( input_path=str(source_path), - delete_input=delete_input, source_object_name=self.source_object_name, - selection_data=self.selection_data, - invert=self.invert, + region_bounds_json=json.dumps(region_mask.bounds), ) if "CANCELLED" in result: - if delete_input: - remove_temporary_input(source_path) + cleanup_image_input(source_path, True) return {"CANCELLED"} return {"FINISHED"} - edit_image_object( + _edit_image_object( source_object, - polygon_mask(tuple(source_object.data.size), polygon), - self.invert, + region_mask.full_values(tuple(source_object.data.size)), ) except (json.JSONDecodeError, RuntimeError, TypeError, ValueError) as error: report_image_edit_exception(self, error) @@ -87,29 +170,39 @@ class RefineImageMask(JobOperatorBase, bpy.types.Operator): job_type = "refine-image-mask" starting_message = "Refining image selection" - input_path: bpy.props.StringProperty(options=HIDDEN) - delete_input: bpy.props.BoolProperty(default=False, options=HIDDEN) - source_object_name: hidden_string_property() - selection_data: hidden_string_property() - invert: bpy.props.BoolProperty(default=False, options=HIDDEN) + input_path: bpy.props.StringProperty(options=_HIDDEN) + source_object_name: bpy.props.StringProperty(options={"HIDDEN", "SKIP_SAVE"}) + region_bounds_json: bpy.props.StringProperty(options={"HIDDEN", "SKIP_SAVE"}) def request(self, _context): - return image_mask_refine_request(self.input_path, self.selection_data) + require_environment() + require_model(BEN2_MODEL_KEY) + return { + "input": str(self.input_path), + "model": BEN2_MODEL_KEY, + "device": production_device().lower(), + } def response(self, _context, result): - source_object = bpy.data.objects.get(self.source_object_name) - if not is_image_empty(source_object): - raise RuntimeError("The source Image Empty is no longer available") + source_object = require_image_empty(self.source_object_name) import numpy as np - bounds = tuple(json.loads(result.file("bounds").read_text(encoding="utf-8"))) + source_bounds = tuple(json.loads(self.region_bounds_json)) + local_bounds = tuple( + json.loads(result.file("bounds").read_text(encoding="utf-8")) + ) + bounds = ( + source_bounds[0] + local_bounds[0], + source_bounds[1] + local_bounds[1], + source_bounds[0] + local_bounds[2], + source_bounds[1] + local_bounds[3], + ) refined_image = bpy.data.images.load( str(result.file("image")), check_existing=False ) try: - refined_pixels = np.empty(len(refined_image.pixels), dtype=np.float32) - refined_image.pixels.foreach_get(refined_pixels) - mask = refined_mask_from_pixels( + refined_pixels = image_pixels(refined_image) + mask = _refined_mask_from_pixels( refined_pixels, tuple(refined_image.size), bounds, @@ -118,15 +211,16 @@ class RefineImageMask(JobOperatorBase, bpy.types.Operator): finally: bpy.data.images.remove(refined_image, do_unlink=True) try: - edit_image_object(source_object, mask, self.invert) + _edit_image_object(source_object, mask) except ImageEditWarning as error: self.report({"WARNING"}, str(error)) return str(error) - push_undo("Edit Image Mask") + if "FINISHED" not in bpy.ops.ed.undo_push(message="Edit Image Mask"): + raise RuntimeError("Unable to create the Edit Image Mask Undo step") return "Image selection refined" def cleanup(self): - cleanup_temporary_input(self.input_path, self.delete_input) + cleanup_image_input(self.input_path, True) class ActivateImageTool(bpy.types.Operator): diff --git a/src/anyimage/operators/image_tool/lasso.py b/src/anyimage/operators/image_tool/lasso.py index c54fb0e..a80d089 100644 --- a/src/anyimage/operators/image_tool/lasso.py +++ b/src/anyimage/operators/image_tool/lasso.py @@ -1,6 +1,6 @@ import bpy -from ..common.image_edit import draw_image_mask_settings +from ..common.image_interaction import draw_region_settings from .edit_mask import EditImageMask @@ -21,4 +21,4 @@ class ImageLassoTool(bpy.types.WorkSpaceTool): ) def draw_settings(context, layout, _tool): - draw_image_mask_settings(context, layout, "image_lasso_refine_selection") + draw_region_settings(context, layout, "image_lasso_refine_selection") diff --git a/src/anyimage/operators/perspective_tool/edit_perspective.py b/src/anyimage/operators/perspective_tool/edit_perspective.py index 4c3885a..f2e6432 100644 --- a/src/anyimage/operators/perspective_tool/edit_perspective.py +++ b/src/anyimage/operators/perspective_tool/edit_perspective.py @@ -6,24 +6,24 @@ import bpy from ..common.image_interaction import ( deserialize_matrix, drawing_in_region, - hidden_string_property, screen_path_to_image_pixels, serialize_matrix, ) -from ..common.image import ( +from ..common.image_region import crop_pixels_to_alpha +from ..common.image_data import ( + create_pixel_result_image, image_base_name, - is_image_empty, - plane_bounds, + image_pixels, + image_empty_bounds, + replace_empty_image, + require_image_empty, video_total_frames, ) -from ..common.image_edit import ( +from ..common.image_interaction import ( active_view3d_tool_id, - crop_pixels_to_alpha, draw_dashed_line, draw_polygon_fill, image_edit_poll, - polygon_bounds, - replace_empty_image_pixels, report_image_edit_exception, resolve_image_edit_click, ) @@ -69,7 +69,7 @@ def validate_perspective_quad(points, min_area=1e-6): ) ) if area < min_area: - raise ValueError("Draw a larger perspective region") + raise ValueError("Draw a larger perspective quad") return quad @@ -313,7 +313,7 @@ def homography_from_points(source, target): values.extend((u, v)) matrix = np.asarray(rows, dtype=np.float64) if np.linalg.cond(matrix) > 1e12: - raise ValueError("The perspective region is too degenerate to transform") + raise ValueError("The perspective quad is too degenerate to transform") coefficients = np.linalg.solve(matrix, np.asarray(values, dtype=np.float64)) return np.append(coefficients, 1.0).reshape((3, 3)) @@ -438,9 +438,13 @@ def perspective_quad_aspect(quad): ) -def perspective_placement_bounds(polygon, output_size, crop_bounds): +def perspective_placement_bounds(quad, output_size, crop_bounds): """Map a rectified Alpha crop back to an approximate source-image rectangle.""" - source_left, source_top, source_right, source_bottom = polygon_bounds(polygon) + quad = validate_perspective_quad(quad) + source_left = math.floor(float(quad[:, 0].min())) + source_top = math.floor(float(quad[:, 1].min())) + source_right = math.ceil(float(quad[:, 0].max())) + source_bottom = math.ceil(float(quad[:, 1].max())) output_width, output_height = (float(value) for value in output_size) crop_left, crop_top, crop_right, crop_bottom = crop_bounds source_width = source_right - source_left @@ -456,12 +460,12 @@ def perspective_placement_bounds(polygon, output_size, crop_bounds): def extract_perspective_pixels( source_pixels, source_size, - polygon, + quad, aspect_ratio, ): - output_size = perspective_output_size(polygon, aspect_ratio) + output_size = perspective_output_size(quad, aspect_ratio) pixels = warp_perspective_pixels( - source_pixels, source_size, polygon, output_size + source_pixels, source_size, quad, output_size ) pixels, crop_bounds = crop_pixels_to_alpha(pixels, (0, 0, *output_size)) cropped_size = ( @@ -469,34 +473,20 @@ def extract_perspective_pixels( crop_bounds[3] - crop_bounds[1], ) placement_bounds = perspective_placement_bounds( - polygon, output_size, crop_bounds + quad, output_size, crop_bounds ) return pixels, cropped_size, placement_bounds -def create_perspective_result( - source_object, - pixels, - output_size, - placement_bounds, -): - return replace_empty_image_pixels( - source_object, - pixels, - output_size, - placement_bounds, - ) - - class EditImagePerspective(bpy.types.Operator): bl_idname = "anyimage.edit_image_perspective" bl_label = "Edit Image Perspective" bl_description = "Deform four image corners into a rectangular image" bl_options = {"REGISTER", "UNDO"} - source_object_name: hidden_string_property() - source_matrix_data: hidden_string_property() - selection_data: hidden_string_property() + source_object_name: bpy.props.StringProperty(options={"HIDDEN", "SKIP_SAVE"}) + source_matrix_data: bpy.props.StringProperty(options={"HIDDEN", "SKIP_SAVE"}) + quad_json: bpy.props.StringProperty(options={"HIDDEN", "SKIP_SAVE"}) aspect_ratio: bpy.props.FloatProperty( default=1.0, min=MIN_INTERACTIVE_ASPECT, @@ -632,7 +622,7 @@ class EditImagePerspective(bpy.types.Operator): return {"CANCELLED"} self.source_object_name = source_object.name self.source_matrix_data = serialize_matrix(source_object.matrix_world) - self.selection_data = "" + self.quad_json = "" self._phase = "POINTS" self._preview_image = None self._source_pixels = None @@ -704,36 +694,37 @@ class EditImagePerspective(bpy.types.Operator): self._set_point_status(context) context.area.tag_redraw() return {"RUNNING_MODAL"} - source_object = bpy.data.objects.get(self.source_object_name) - if not is_image_empty(source_object): - self.report({"ERROR"}, "The source Image Empty is no longer available") + try: + source_object = require_image_empty(self.source_object_name) + except RuntimeError as error: + self.report({"ERROR"}, str(error)) self._finish(context) return {"CANCELLED"} try: canonical_perspective_quad(self._points, MIN_QUAD_AREA) matrix_world = deserialize_matrix(self.source_matrix_data) - polygon = screen_path_to_image_pixels( + quad = screen_path_to_image_pixels( self._points, context.region, context.region_data, matrix_world, - plane_bounds(source_object), + image_empty_bounds(source_object), tuple(source_object.data.size), ) - polygon = canonical_perspective_quad(polygon) + quad = canonical_perspective_quad(quad) if not perspective_quad_overlaps_image( - polygon, tuple(source_object.data.size) + quad, tuple(source_object.data.size) ): self.report( {"WARNING"}, - "Perspective region does not overlap the source image", + "Perspective quad does not overlap the source image", ) self._finish(context) return {"CANCELLED"} - self.selection_data = json.dumps( - polygon.tolist(), separators=(",", ":") + self.quad_json = json.dumps( + quad.tolist(), separators=(",", ":") ) - self.aspect_ratio = perspective_quad_aspect(polygon) + self.aspect_ratio = perspective_quad_aspect(quad) self._initial_aspect_ratio = self.aspect_ratio self._raw_aspect_ratio = self.aspect_ratio self._aspect_preset_label = None @@ -751,14 +742,11 @@ class EditImagePerspective(bpy.types.Operator): self._aspect_anchor_x = center_x import numpy as np - self._source_pixels = np.empty( - len(source_object.data.pixels), dtype=np.float32 - ) - source_object.data.pixels.foreach_get(self._source_pixels) + self._source_pixels = image_pixels(source_object.data) preview_pixels = warp_perspective_pixels( self._source_pixels, tuple(source_object.data.size), - polygon, + quad, (PREVIEW_TEXTURE_SIZE, PREVIEW_TEXTURE_SIZE), ) if not np.any( @@ -766,7 +754,7 @@ class EditImagePerspective(bpy.types.Operator): ): self.report( {"WARNING"}, - "Perspective region contains no visible image pixels", + "Perspective quad contains no visible image pixels", ) self._finish(context) return {"CANCELLED"} @@ -785,20 +773,21 @@ class EditImagePerspective(bpy.types.Operator): return {"PASS_THROUGH"} def execute(self, context): - source_object = bpy.data.objects.get(self.source_object_name) - if not is_image_empty(source_object): - self.report({"ERROR"}, "The source Image Empty is no longer available") + try: + source_object = require_image_empty(self.source_object_name) + except RuntimeError as error: + self.report({"ERROR"}, str(error)) return {"CANCELLED"} try: if video_total_frames(source_object.data) > 1: raise RuntimeError("Image Perspective only supports a still image") - polygon = validate_perspective_quad(json.loads(self.selection_data)) + quad = validate_perspective_quad(json.loads(self.quad_json)) if not perspective_quad_overlaps_image( - polygon, tuple(source_object.data.size) + quad, tuple(source_object.data.size) ): self.report( {"WARNING"}, - "Perspective region does not overlap the source image", + "Perspective quad does not overlap the source image", ) return {"CANCELLED"} import numpy as np @@ -807,21 +796,22 @@ class EditImagePerspective(bpy.types.Operator): if source_pixels is None or len(source_pixels) != len( source_object.data.pixels ): - source_pixels = np.empty( - len(source_object.data.pixels), dtype=np.float32 - ) - source_object.data.pixels.foreach_get(source_pixels) + source_pixels = image_pixels(source_object.data) pixels, output_size, placement_bounds = extract_perspective_pixels( source_pixels, tuple(source_object.data.size), - polygon, + quad, self.aspect_ratio, ) - create_perspective_result( - source_object, + result_image = create_pixel_result_image( + source_object.data, pixels, output_size, - placement_bounds, + ) + replace_empty_image( + source_object, + result_image, + placement_bounds=placement_bounds, ) except ( json.JSONDecodeError, diff --git a/src/anyimage/operators/remove_background.py b/src/anyimage/operators/remove_background.py index d312530..ba5c0dd 100644 --- a/src/anyimage/operators/remove_background.py +++ b/src/anyimage/operators/remove_background.py @@ -2,27 +2,27 @@ import bpy from ..properties import BEN2_MODEL_KEY from ..runtime import JobOperatorBase -from .common.operator import ( - HIDDEN, - background_input_path, - cleanup_temporary_input, +from .common.ai_job import ( + require_background_input_path, invoke_ai_setup_if_needed, production_device, - push_undo, require_environment, require_model, - source_start_frame, ) -from .common.image import ( +from .common.image_data import ( + cleanup_image_input, default_extract_frames, is_image_empty, prepare_image_input, - remove_temporary_input, + source_start_frame, video_total_frames, ) +_HIDDEN = {"HIDDEN", "SKIP_SAVE"} + + class RemoveImageBackground(bpy.types.Operator): bl_idname = "anyimage.remove_image_background" bl_label = "Remove Background" @@ -65,29 +65,22 @@ class RemoveImageBackground(bpy.types.Operator): if "CANCELLED" not in result: return {"FINISHED"} if delete_input: - remove_temporary_input(source_path) + cleanup_image_input(source_path, True) return {"CANCELLED"} - - - - def apply_background_response(result, source_object_name): - from .common.image import create_color_image, is_image_empty + from .common.image_data import ( + load_color_result_image, + replace_empty_image, + require_image_empty, + ) - source_object = bpy.data.objects.get(source_object_name) - if not is_image_empty(source_object): - raise RuntimeError("The source image empty is no longer available") - source_image = source_object.data - source_name = source_image.name - original_fake_user = source_image.use_fake_user - result_image = create_color_image(source_object, result.directory) - source_image.use_fake_user = original_fake_user - if not original_fake_user and int(source_image.users) == 0: - bpy.data.images.remove(source_image) - result_image.name = source_name - push_undo("Remove Background") + source_object = require_image_empty(source_object_name) + result_image = load_color_result_image(source_object, result.directory) + replace_empty_image(source_object, result_image) + if "FINISHED" not in bpy.ops.ed.undo_push(message="Remove Background"): + raise RuntimeError("Unable to create the Remove Background Undo step") class RunBackgroundRemoval(JobOperatorBase, bpy.types.Operator): @@ -96,16 +89,16 @@ class RunBackgroundRemoval(JobOperatorBase, bpy.types.Operator): job_type = "remove-background" starting_message = "Starting Image Empty background removal" - input_path: bpy.props.StringProperty(options=HIDDEN) - source_object_name: bpy.props.StringProperty(options=HIDDEN) - delete_input: bpy.props.BoolProperty(default=False, options=HIDDEN) + input_path: bpy.props.StringProperty(options=_HIDDEN) + source_object_name: bpy.props.StringProperty(options=_HIDDEN) + delete_input: bpy.props.BoolProperty(default=False, options=_HIDDEN) def request(self, context): require_environment() require_model(BEN2_MODEL_KEY) settings = context.scene.anyimage_settings return { - "input": str(background_input_path(self.input_path)), + "input": str(require_background_input_path(self.input_path)), "model": BEN2_MODEL_KEY, "device": production_device().lower(), "video_frames": settings.video_frames, @@ -117,4 +110,4 @@ class RunBackgroundRemoval(JobOperatorBase, bpy.types.Operator): return "Image background is removed" def cleanup(self): - cleanup_temporary_input(self.input_path, self.delete_input) + cleanup_image_input(self.input_path, self.delete_input) diff --git a/src/anyimage/operators/upscale.py b/src/anyimage/operators/upscale.py index 5061629..9893bd6 100644 --- a/src/anyimage/operators/upscale.py +++ b/src/anyimage/operators/upscale.py @@ -8,25 +8,43 @@ from ..server.media.resolution import ( MIN_MAX_AI_INPUT_SIZE, ) from ..runtime import JobOperatorBase -from .common.operator import ( - HIDDEN, - cleanup_temporary_input, - input_path, +from .common.ai_job import ( + require_input_path, invoke_ai_setup_if_needed, production_device, require_environment, require_model, - source_start_frame, ) -from .common.image import ( - create_color_image, +from .common.image_data import ( + cleanup_image_input, default_extract_frames, - image_base_name, is_image_empty, + load_color_result_image, prepare_image_input, + source_start_frame, + replace_empty_image, + require_image_empty, video_total_frames, ) +_HIDDEN = {"HIDDEN", "SKIP_SAVE"} + +UPSCALE_FACTOR = 4 + + +def upscale_output_size(image): + width, height = (int(value) for value in getattr(image, "size", (0, 0))) + if width <= 0 or height <= 0: + return None + return width * UPSCALE_FACTOR, height * UPSCALE_FACTOR + + +def upscale_input_size_allowed(image, max_input_size): + width, height = (int(value) for value in getattr(image, "size", (0, 0))) + if width <= 0 or height <= 0: + return True + return max(width, height) < int(max_input_size) + class UpscaleImage(JobOperatorBase, bpy.types.Operator): bl_idname = "anyimage.upscale_image" @@ -38,7 +56,7 @@ class UpscaleImage(JobOperatorBase, bpy.types.Operator): model: bpy.props.StringProperty( default="REALESRGAN_GENERAL_WDN_X4V3", - options=HIDDEN, + options=_HIDDEN, ) max_input_size: bpy.props.IntProperty( name="Max AI Input Size", @@ -48,18 +66,26 @@ class UpscaleImage(JobOperatorBase, bpy.types.Operator): max=MAX_MAX_AI_INPUT_SIZE, step=256, subtype="PIXEL", - options=HIDDEN, + options=_HIDDEN, ) - input_path: bpy.props.StringProperty(options=HIDDEN) - source_object_name: bpy.props.StringProperty(options=HIDDEN) - source_image_name: bpy.props.StringProperty(options=HIDDEN) - delete_input: bpy.props.BoolProperty(default=False, options=HIDDEN) + input_path: bpy.props.StringProperty(options=_HIDDEN) + source_object_name: bpy.props.StringProperty(options=_HIDDEN) + source_image_name: bpy.props.StringProperty(options=_HIDDEN) + delete_input: bpy.props.BoolProperty(default=False, options=_HIDDEN) @classmethod def poll(cls, context): from ..runtime import runtime - return is_image_empty(context.object) and not runtime.server_busy() + source_object = getattr(context, "object", None) + return ( + is_image_empty(source_object) + and upscale_input_size_allowed( + source_object.data, + configured_max_ai_input_size(context), + ) + and not runtime.server_busy() + ) def invoke(self, context, _event): setup_result = invoke_ai_setup_if_needed() @@ -74,23 +100,15 @@ class UpscaleImage(JobOperatorBase, bpy.types.Operator): if not is_image_empty(source_object): self.report({"ERROR"}, "Select an Image Empty to upscale") return {"CANCELLED"} - current_image = source_object.data - try: - original_image = getattr( - current_image, - "anyimage_upscale_source", - None, - ) - except ReferenceError: - original_image = None - source_image = original_image or current_image - self.source_image_name = source_image.name - self._rebuild_from_original = source_image is not current_image - if self._rebuild_from_original: + source_image = source_object.data + if not upscale_input_size_allowed(source_image, self.max_input_size): self.report( - {"WARNING"}, - "This image was already upscaled; rebuilding from the original", + {"ERROR"}, + "Image long edge must be smaller than Maximum AI Input Size " + f"({self.max_input_size} px)", ) + return {"CANCELLED"} + self.source_image_name = source_image.name if video_total_frames(source_image) > 1: context.scene.anyimage_settings.video_frames = ( default_extract_frames(source_object) @@ -109,7 +127,7 @@ class UpscaleImage(JobOperatorBase, bpy.types.Operator): self.delete_input = delete_input result = super().execute(context) if "CANCELLED" in result: - cleanup_temporary_input(self.input_path, self.delete_input) + cleanup_image_input(self.input_path, self.delete_input) return result def request(self, context): @@ -117,7 +135,7 @@ class UpscaleImage(JobOperatorBase, bpy.types.Operator): require_model(self.model) settings = context.scene.anyimage_settings return { - "input": str(input_path(self.input_path)), + "input": str(require_input_path(self.input_path)), "model": self.model, "device": production_device().lower(), "video_frames": settings.video_frames, @@ -126,28 +144,18 @@ class UpscaleImage(JobOperatorBase, bpy.types.Operator): } def response(self, _context, result): - source_object = bpy.data.objects.get(self.source_object_name) - if not is_image_empty(source_object): - raise RuntimeError("The source image empty is no longer available") + source_object = require_image_empty(self.source_object_name) source_image = bpy.data.images.get(self.source_image_name) if source_image is None: - raise RuntimeError("The original source image is no longer available") - replaced_image = source_object.data - result_image = create_color_image( + raise RuntimeError("The source image is no longer available") + result_image = load_color_result_image( source_object, result.directory, frame_prefix="upscale.", ) - if replaced_image is not source_image: - replaced_image.use_fake_user = False - if int(replaced_image.users) == 0: - bpy.data.images.remove(replaced_image, do_unlink=True) - result_image.name = f"{image_base_name(source_image)}_upscale.png" - result_image.anyimage_upscale_source = source_image + replace_empty_image(source_object, result_image) model_label = upscale_model_label(self.model) - if getattr(self, "_rebuild_from_original", False): - return f"Image is rebuilt from the original with {model_label}" return f"Image is upscaled with {model_label}" def cleanup(self): - cleanup_temporary_input(self.input_path, self.delete_input) + cleanup_image_input(self.input_path, self.delete_input) diff --git a/src/anyimage/preferences.py b/src/anyimage/preferences.py index f4b813f..5427d0a 100644 --- a/src/anyimage/preferences.py +++ b/src/anyimage/preferences.py @@ -96,8 +96,8 @@ class AnyImagePreferences(bpy.types.AddonPreferences): max_ai_input_size: bpy.props.IntProperty( name="Maximum AI Input Size", description=( - "Maximum long edge used by Upscale, Depth and Normal analysis; " - "larger inputs are resized proportionally without changing color textures" + "Maximum source long edge eligible for Upscale and maximum long edge " + "used by Depth and Normal analysis" ), default=DEFAULT_MAX_AI_INPUT_SIZE, min=MIN_MAX_AI_INPUT_SIZE, diff --git a/src/anyimage/properties.py b/src/anyimage/properties.py index cfaeea5..d4df92b 100644 --- a/src/anyimage/properties.py +++ b/src/anyimage/properties.py @@ -232,11 +232,24 @@ def refine_selection_property(): ) -def cutout_normal_analysis_property(): - return bpy.props.BoolProperty( +def cutout_normal_mode_property(): + return bpy.props.EnumProperty( name="Normal", - description="Use MoGe-2 to add a normal texture for the selected shape", - default=False, + description="Normal texture generated with MoGe-2", + items=( + ("NONE", "None", "Do not generate a normal texture"), + ( + "TANGENT", + "Tangent", + "Generate a normal texture relative to the image plane", + ), + ( + "OBJECT", + "Object", + "Generate a normal texture in Cutout object-local axes", + ), + ), + default="NONE", ) @@ -255,7 +268,7 @@ def cutout_point_spacing_property(): class AnyImageSettings(bpy.types.PropertyGroup): cutout_point_spacing: cutout_point_spacing_property() - cutout_normal_analysis: cutout_normal_analysis_property() + cutout_normal_mode: cutout_normal_mode_property() cutout_refine_selection: refine_selection_property() image_box_refine_selection: refine_selection_property() image_lasso_refine_selection: refine_selection_property() diff --git a/src/anyimage/server/app.py b/src/anyimage/server/app.py index 84b5296..d6cb0c3 100644 --- a/src/anyimage/server/app.py +++ b/src/anyimage/server/app.py @@ -10,7 +10,8 @@ from .jobs.debug_predictions import ( run_upscale as _debug_upscale, ) from .jobs.refine_image_mask import run as _refine_image_mask -from .jobs.prepare_geometry_data import run as _prepare_geometry_data +from .jobs.generate_depth_plane import run as _generate_depth_plane +from .jobs.generate_region_data import run as _generate_region_data from .jobs.upscale import run as _upscale_image from .model_manager import ModelManager @@ -50,9 +51,14 @@ def upscale_image(context, parameters): return _upscale_image(context, parameters) -@server.job("prepare-geometry-data") -def prepare_geometry_data(context, parameters): - return _prepare_geometry_data(context, parameters) +@server.job("generate-depth-plane") +def generate_depth_plane(context, parameters): + return _generate_depth_plane(context, parameters) + + +@server.job("generate-region-data") +def generate_region_data(context, parameters): + return _generate_region_data(context, parameters) @server.job("debug-da3") diff --git a/src/anyimage/server/geometry/generate_depth.py b/src/anyimage/server/geometry/generate_depth.py new file mode 100644 index 0000000..183554b --- /dev/null +++ b/src/anyimage/server/geometry/generate_depth.py @@ -0,0 +1,181 @@ +"""Generate model-derived depth and normal data for one raster image.""" + +from ..media import DEFAULT_MAX_AI_INPUT_SIZE, limit_image +from ..models import geometry_adapter, moge2 +from .normal_texture import write_camera_normal_texture + + +def _limited_model_input(source_path, output_path, max_input_size): + from PIL import Image + + with Image.open(source_path) as opened: + limited = limit_image(opened, max_input_size) + if limited is opened: + return source_path + limited.save(output_path, format="PNG", compress_level=1) + return output_path + + +def _alpha_mask(image_path, shape): + import numpy as np + from PIL import Image + + with Image.open(image_path) as opened: + alpha = opened.convert("RGBA").getchannel("A") + alpha = alpha.resize( + (int(shape[1]), int(shape[0])), + Image.Resampling.NEAREST, + ) + return np.asarray(alpha, dtype=np.uint8) >= 26 + + +def transparent_fraction(image_path, alpha_threshold=26): + import numpy as np + from PIL import Image + + with Image.open(image_path) as opened: + alpha = np.asarray( + opened.convert("RGBA").getchannel("A"), + dtype=np.uint8, + ) + return float(np.count_nonzero(alpha < alpha_threshold) / alpha.size) + + +def _write_image_geometry_archive( + frame, + output_path, + calibration_valid, + region_transparent_fraction, +): + import numpy as np + + depth = np.asarray(frame.depth, dtype=np.float32) + points = np.asarray(frame.points, dtype=np.float32) + intrinsics = np.asarray(frame.intrinsics, dtype=np.float32) + valid = np.asarray(frame.valid_mask, dtype=bool) + if depth.ndim != 2 or points.shape != (*depth.shape, 3): + raise ValueError("The depth model returned an invalid point field") + valid &= np.isfinite(depth) & (depth > 0.0) + valid &= np.isfinite(points).all(axis=-1) + if not valid.any(): + raise ValueError("The depth model returned no valid geometry") + calibration_valid = np.asarray(calibration_valid, dtype=bool) & valid + if not calibration_valid.any(): + raise ValueError("The depth field contains no valid Region pixels") + reference_values = depth[calibration_valid] + if frame.model_mask is not None: + model_mask = np.asarray(frame.model_mask, dtype=bool) + masked = depth[calibration_valid & model_mask] + if len(masked): + reference_values = masked + np.savez_compressed( + output_path, + points=points, + depth=depth, + intrinsics=intrinsics, + valid=valid, + calibration_valid=calibration_valid, + image_size=np.asarray((depth.shape[1], depth.shape[0]), dtype=np.int32), + reference_depth=np.asarray(np.median(reference_values), dtype=np.float32), + region_transparent_fraction=np.asarray( + region_transparent_fraction, + dtype=np.float32, + ), + ) + + +def _write_normal( + prediction, + color_path, + output_path, + normal_mode, +): + import numpy as np + from PIL import Image + + normal = np.asarray(prediction.normal[0], dtype=np.float32) + valid = _alpha_mask(color_path, normal.shape[:2]) + with Image.open(color_path) as source: + output_size = source.size + return write_camera_normal_texture( + normal, + valid, + (0, 0), + output_size, + output_size, + output_path, + normal_mode=normal_mode, + ) + + +def generate_depth( + context, + parameters, + input_path, + *, + output_depth, + normal_mode="NONE", +): + """Generate requested AI data from an already prepared image Region.""" + normal_mode = str(normal_mode).upper() + if normal_mode not in {"NONE", "TANGENT", "OBJECT"}: + raise ValueError(f"Unsupported normal mode: {normal_mode}") + model_path = _limited_model_input( + input_path, + context.directory / "model-input.png", + parameters.get("max_input_size", DEFAULT_MAX_AI_INPUT_SIZE), + ) + try: + model_manager = context.resource("model_manager") + adapter = geometry_adapter(parameters["model"]) + prediction = None + if output_depth: + context.progress(0.45, "Generating depth") + if adapter is moge2: + prediction = adapter.infer( + model_manager, + parameters, + [model_path], + include_points=True, + cancel_check=context.check_cancelled, + ) + else: + prediction = adapter.infer( + model_manager, + parameters, + [model_path], + cancel_check=context.check_cancelled, + ) + frame = adapter.geometry_frame(prediction) + calibration_valid = _alpha_mask(input_path, frame.valid_mask.shape) + archive_path = context.directory / "image_geometry.npz" + _write_image_geometry_archive( + frame, + archive_path, + calibration_valid, + transparent_fraction(input_path), + ) + else: + archive_path = None + + normal_path = None + if normal_mode != "NONE": + context.progress(0.72, "Generating surface normal") + if prediction is None or adapter is not moge2: + prediction = moge2.infer( + model_manager, + parameters, + [model_path], + include_points=False, + cancel_check=context.check_cancelled, + ) + normal_path = _write_normal( + prediction, + input_path, + context.directory / "normal.png", + normal_mode, + ) + return archive_path, normal_path + finally: + if model_path != input_path: + model_path.unlink(missing_ok=True) diff --git a/src/anyimage/server/geometry/normal_texture.py b/src/anyimage/server/geometry/normal_texture.py index 3da7624..d628b88 100644 --- a/src/anyimage/server/geometry/normal_texture.py +++ b/src/anyimage/server/geometry/normal_texture.py @@ -3,14 +3,14 @@ from pathlib import Path -def resized_selection_mask(selection_mask, shape): +def resized_valid_mask(valid_mask, shape): import numpy as np from PIL import Image - selection_mask = np.asarray(selection_mask, dtype=bool) - if selection_mask.shape == tuple(shape): - return selection_mask - resized = Image.fromarray(selection_mask.astype(np.uint8) * 255, mode="L") + valid_mask = np.asarray(valid_mask, dtype=bool) + if valid_mask.shape == tuple(shape): + return valid_mask + resized = Image.fromarray(valid_mask.astype(np.uint8) * 255, mode="L") resized = resized.resize( (int(shape[1]), int(shape[0])), Image.Resampling.NEAREST, @@ -25,8 +25,9 @@ def write_camera_normal_texture( crop_size, source_size, output_path, + normal_mode="TANGENT", ): - """Encode MoGe camera-axis normals into a full-image RGB texture.""" + """Encode MoGe camera-axis normals in Blender tangent or object space.""" import numpy as np from PIL import Image @@ -35,14 +36,26 @@ def write_camera_normal_texture( if normal.ndim != 3 or normal.shape[2] != 3: raise ValueError("Depth Balloon normal data must have three components") if valid.shape != normal.shape[:2]: - valid = resized_selection_mask(valid, normal.shape[:2]) + valid = resized_valid_mask(valid, normal.shape[:2]) + normal_mode = str(normal_mode).upper() + if normal_mode == "TANGENT": + converted = normal * np.asarray((1.0, -1.0, -1.0), dtype=np.float32) + empty_normal = (128, 128, 255) + elif normal_mode == "OBJECT": + converted = normal[..., (2, 0, 1)] * np.asarray( + (-1.0, -1.0, -1.0), + dtype=np.float32, + ) + empty_normal = (255, 128, 128) + else: + raise ValueError(f"Unsupported normal mode: {normal_mode}") encoded = np.clip( - (normal * np.asarray((1.0, -1.0, -1.0), dtype=np.float32) + 1.0) + (converted + 1.0) * 127.5, 0.0, 255.0, ).astype(np.uint8) - encoded[~valid] = (128, 128, 255) + encoded[~valid] = empty_normal crop_width, crop_height = (int(value) for value in crop_size) crop = Image.fromarray(encoded, mode="RGB") if crop.size != (crop_width, crop_height): @@ -51,7 +64,7 @@ def write_camera_normal_texture( Image.Resampling.BILINEAR, ) source_width, source_height = (int(value) for value in source_size) - full = Image.new("RGB", (source_width, source_height), (128, 128, 255)) + full = Image.new("RGB", (source_width, source_height), empty_normal) full.paste(crop, tuple(int(value) for value in origin)) output_path = Path(output_path) full.save(output_path) diff --git a/src/anyimage/server/jobs/generate_depth_plane.py b/src/anyimage/server/jobs/generate_depth_plane.py new file mode 100644 index 0000000..b84a80d --- /dev/null +++ b/src/anyimage/server/jobs/generate_depth_plane.py @@ -0,0 +1,22 @@ +"""Generate depth data and preserve color for one complete image.""" + +from ..geometry.generate_depth import generate_depth +from ..media import copy_color_images, input_frames + + +def run(context, parameters): + context.progress(0.05, "Preparing Depth Plane") + context.directory.mkdir(parents=True, exist_ok=True) + with input_frames(context, parameters) as images: + if len(images) != 1: + raise ValueError("Depth Plane currently supports one image") + copy_color_images(images, context.directory) + archive_path, _normal_path = generate_depth( + context, + parameters, + images[0], + output_depth=True, + ) + context.check_cancelled() + context.progress(1.0, "Depth Plane data is ready") + return {"image_geometry": archive_path.name} diff --git a/src/anyimage/server/jobs/generate_region_data.py b/src/anyimage/server/jobs/generate_region_data.py new file mode 100644 index 0000000..e82f952 --- /dev/null +++ b/src/anyimage/server/jobs/generate_region_data.py @@ -0,0 +1,51 @@ +"""Generate optional AI data for one Blender-rasterized Cutout Region.""" + +import json +from pathlib import Path + +from ..geometry.generate_depth import generate_depth +from ..region import generate_selection + + +def run(context, parameters): + context.progress(0.05, "Preparing Cutout Region") + context.directory.mkdir(parents=True, exist_ok=True) + input_path = Path(parameters["input"]) + region_path = input_path + result = {} + if parameters.get("refine_region"): + context.progress(0.2, "Generating refined Region") + region_path, bounds = generate_selection( + context, + input_path, + parameters["refine_model"], + parameters["device"], + context.directory / "region.png", + ) + bounds_path = context.directory / "region_bounds.json" + bounds_path.write_text( + json.dumps(bounds, separators=(",", ":")), + encoding="utf-8", + ) + result.update( + region=region_path.name, + region_bounds=bounds_path.name, + ) + + output_depth = bool(parameters.get("output_depth")) + normal_mode = str(parameters.get("normal_mode", "NONE")) + if output_depth or normal_mode != "NONE": + archive_path, normal_path = generate_depth( + context, + parameters, + region_path, + output_depth=output_depth, + normal_mode=normal_mode, + ) + if archive_path is not None: + result["image_geometry"] = archive_path.name + if normal_path is not None: + result["normal"] = normal_path.name + context.check_cancelled() + context.progress(1.0, "Cutout Region data is ready") + return result diff --git a/src/anyimage/server/jobs/prepare_geometry_data.py b/src/anyimage/server/jobs/prepare_geometry_data.py deleted file mode 100644 index 1026648..0000000 --- a/src/anyimage/server/jobs/prepare_geometry_data.py +++ /dev/null @@ -1,255 +0,0 @@ -from ..geometry.normal_texture import write_camera_normal_texture -from ..media import ( - DEFAULT_MAX_AI_INPUT_SIZE, - analysis_frames, - copy_color_images, - input_frames, - limit_image, -) -from ..models import geometry_adapter, moge2 -from ..selection.cutout_images import refine_selection, write_images, write_selection - - -def _limited_model_input(source_path, output_path, max_input_size): - from PIL import Image - - with Image.open(source_path) as opened: - limited = limit_image(opened, max_input_size) - if limited is opened: - return source_path - limited.save(output_path, format="PNG", compress_level=1) - return output_path - - -def _write_field(frame, output_path, calibration_valid=None): - import numpy as np - - depth = np.asarray(frame.depth, dtype=np.float32) - points = np.asarray(frame.points, dtype=np.float32) - intrinsics = np.asarray(frame.intrinsics, dtype=np.float32) - valid = np.asarray(frame.valid_mask, dtype=bool) - if depth.ndim != 2 or points.shape != (*depth.shape, 3): - raise ValueError("The depth model returned an invalid point field") - valid &= np.isfinite(depth) & (depth > 0.0) - valid &= np.isfinite(points).all(axis=-1) - if not valid.any(): - raise ValueError("The depth model returned no valid geometry") - if calibration_valid is None: - calibration_valid = valid.copy() - else: - calibration_valid = np.asarray(calibration_valid, dtype=bool) - if calibration_valid.shape != valid.shape: - raise ValueError("The depth calibration mask dimensions do not match") - calibration_valid &= valid - if not calibration_valid.any(): - raise ValueError("The depth field contains no valid calibration pixels") - reference_values = depth[calibration_valid] - if frame.model_mask is not None: - model_mask = np.asarray(frame.model_mask, dtype=bool) - masked = depth[calibration_valid & model_mask] - if len(masked): - reference_values = masked - np.savez_compressed( - output_path, - points=points, - depth=depth, - intrinsics=intrinsics, - valid=valid, - calibration_valid=calibration_valid, - image_size=np.asarray((depth.shape[1], depth.shape[0]), dtype=np.int32), - reference_depth=np.asarray(np.median(reference_values), dtype=np.float32), - ) - - -def _selection_calibration_mask(frame, selection_path): - import numpy as np - from PIL import Image - - valid = np.asarray(frame.valid_mask, dtype=bool) - with Image.open(selection_path) as opened: - selection = opened.convert("RGBA").getchannel("A") - selection = selection.resize( - (valid.shape[1], valid.shape[0]), - Image.Resampling.NEAREST, - ) - selected = np.asarray(selection, dtype=np.uint8) >= 26 - calibration_valid = valid & selected - if not calibration_valid.any(): - raise ValueError("The depth field contains no valid selected pixels") - return calibration_valid - - -def _write_camera_normal_from_prediction(prediction, color_path, output_path): - import numpy as np - from PIL import Image - - normal = np.asarray(prediction.normal[0], dtype=np.float32) - with Image.open(color_path) as source: - alpha = source.convert("RGBA").getchannel("A") - source_size = source.size - valid = ( - np.asarray( - alpha.resize((normal.shape[1], normal.shape[0]), Image.Resampling.BILINEAR) - ) - > 127 - ) - return write_camera_normal_texture( - normal, - valid, - (0, 0), - source_size, - source_size, - output_path, - ) - - -def _write_camera_normal(context, parameters, model_path, color_path=None): - prediction = moge2.infer( - context.resource("model_manager"), - parameters, - [model_path], - include_points=False, - cancel_check=context.check_cancelled, - ) - return _write_camera_normal_from_prediction( - prediction, - color_path or model_path, - context.directory / "normal.png", - ) - - -def _write_depth_field( - context, - parameters, - model_path, - normal_color_path=None, - include_normal=False, - selection_path=None, -): - model_manager = context.resource("model_manager") - adapter = geometry_adapter(parameters["model"]) - if adapter is moge2: - prediction = adapter.infer( - model_manager, - parameters, - [model_path], - include_points=True, - cancel_check=context.check_cancelled, - ) - else: - prediction = adapter.infer( - model_manager, - parameters, - [model_path], - cancel_check=context.check_cancelled, - ) - context.check_cancelled() - field_path = context.directory / "field.npz" - frame = adapter.geometry_frame(prediction) - calibration_valid = frame.valid_mask - if selection_path is not None: - calibration_valid = _selection_calibration_mask(frame, selection_path) - _write_field(frame, field_path, calibration_valid) - normal_path = None - if include_normal and adapter is moge2: - normal_path = _write_camera_normal_from_prediction( - prediction, - normal_color_path or model_path, - context.directory / "normal.png", - ) - return field_path, normal_path - - -def run(context, parameters): - context.progress(0.02, "Preparing geometry input") - context.directory.mkdir(parents=True, exist_ok=True) - with input_frames(context, parameters) as images: - if len(images) != 1: - raise ValueError("Empty Image to Mesh currently supports one image") - shape = parameters["shape"] - polygon = parameters.get("polygon") - analysis_path = None - selection_path = None - result = {} - if polygon: - selection_bounds = None - full_selection_path = context.directory / "selection-full.png" - if parameters.get("refine_selection"): - refine_selection( - context, - parameters, - images[0], - polygon, - full_selection_path, - ) - else: - _selection_path, selection_bounds = write_selection( - images[0], - polygon, - full_selection_path, - parameters.get("invert", False), - ) - color_path, selection_path, analysis_path, bounds_path = ( - write_images( - images[0], - full_selection_path, - context.directory, - selection_bounds, - ) - ) - result.update( - selection=selection_path.name, - bounds=bounds_path.name, - ) - else: - copy_color_images(images, context.directory) - - normal_analysis = bool(parameters.get("normal_analysis")) - with analysis_frames(context, parameters, images) as model_images: - if len(model_images) != 1: - raise ValueError("Empty Image to Mesh currently supports one image") - model_path = analysis_path or model_images[0] - limited_model_path = model_path - if normal_analysis or shape in { - "DEPTH_PLANE", - "DEPTH_BALLOON", - "DEPTH_SURFACE", - }: - limited_model_path = _limited_model_input( - model_path, - context.directory / "model-input.png", - parameters.get("max_input_size", DEFAULT_MAX_AI_INPUT_SIZE), - ) - if shape in {"CUTOUT", "BALLOON"} and normal_analysis: - context.progress(0.4, "Estimating surface normal") - result["normal"] = _write_camera_normal( - context, parameters, limited_model_path, model_path - ).name - elif shape in {"DEPTH_PLANE", "DEPTH_BALLOON", "DEPTH_SURFACE"}: - context.progress(0.4, "Estimating image depth") - field_path, normal_path = _write_depth_field( - context, - parameters, - limited_model_path, - normal_color_path=model_path, - include_normal=normal_analysis, - selection_path=selection_path, - ) - result["field"] = field_path.name - if ( - normal_analysis - and normal_path is None - and shape == "DEPTH_SURFACE" - ): - normal_path = _write_camera_normal( - context, parameters, limited_model_path, model_path - ) - if normal_path is not None: - result["normal"] = normal_path.name - - if limited_model_path != model_path: - limited_model_path.unlink(missing_ok=True) - - context.check_cancelled() - context.progress(1.0, "Geometry data is ready") - return result diff --git a/src/anyimage/server/jobs/refine_image_mask.py b/src/anyimage/server/jobs/refine_image_mask.py index 98563e9..b0a141e 100644 --- a/src/anyimage/server/jobs/refine_image_mask.py +++ b/src/anyimage/server/jobs/refine_image_mask.py @@ -1,12 +1,7 @@ import json import time -from ..models.ben2 import infer_foreground -from ..selection.polygon_mask import ( - apply_selection_mask, - create_selection_canvas, - crop_to_alpha, -) +from ..region import generate_selection def run(context, parameters): @@ -16,28 +11,14 @@ def run(context, parameters): context.check_cancelled() output_directory = context.directory output_directory.mkdir(parents=True, exist_ok=True) - canvas, selection_mask, bounds = create_selection_canvas( - parameters["input"], parameters["polygon"] - ) - selection_path = output_directory / "selection-input.png" - canvas.save(selection_path, format="PNG", compress_level=1) - model_manager = context.resource("model_manager") context.progress(0.2, f"Loading BEN2 for {parameters['device'].upper()}") - foreground, _load_ms, _inference_ms = infer_foreground( - selection_path, - model_manager.directory(parameters["model"]), + output_path, bounds = generate_selection( + context, + parameters["input"], + parameters["model"], parameters["device"], - model_cache=model_manager, - cancel_check=context.check_cancelled, - preserve_input_alpha=True, + output_directory / "image.png", ) - context.check_cancelled() - context.progress(0.9, "Applying selection mask") - output, bounds = crop_to_alpha( - apply_selection_mask(foreground, selection_mask), bounds - ) - output_path = output_directory / "image.png" - output.save(output_path, format="PNG", compress_level=1) bounds_path = output_directory / "bounds.json" bounds_path.write_text(json.dumps(bounds, separators=(",", ":")), encoding="utf-8") elapsed_ms = (time.perf_counter() - started) * 1000.0 diff --git a/src/anyimage/server/region/__init__.py b/src/anyimage/server/region/__init__.py new file mode 100644 index 0000000..af4b8b3 --- /dev/null +++ b/src/anyimage/server/region/__init__.py @@ -0,0 +1,5 @@ +"""AI generation for rasterized image Regions.""" + +from .generate_selection import generate_selection + +__all__ = ("generate_selection",) diff --git a/src/anyimage/server/region/generate_selection.py b/src/anyimage/server/region/generate_selection.py new file mode 100644 index 0000000..a53c892 --- /dev/null +++ b/src/anyimage/server/region/generate_selection.py @@ -0,0 +1,40 @@ +from ..models.ben2 import infer_foreground + + +def _crop_to_alpha(image): + import numpy as np + + pixels = np.asarray(image.convert("RGBA"), dtype=np.uint8) + visible_y, visible_x = np.nonzero(pixels[:, :, 3] > 0) + if not len(visible_x): + raise ValueError("The generated Region contains no visible pixels") + bounds = ( + int(visible_x.min()), + int(visible_y.min()), + int(visible_x.max()) + 1, + int(visible_y.max()) + 1, + ) + return image.crop(bounds), bounds + + +def generate_selection( + context, + input_path, + model, + device, + output_path, +): + """Generate a refined selection from an already rasterized Region.""" + model_manager = context.resource("model_manager") + foreground, _load_ms, _inference_ms = infer_foreground( + input_path, + model_manager.directory(model), + device, + model_cache=model_manager, + cancel_check=context.check_cancelled, + preserve_input_alpha=True, + ) + context.check_cancelled() + output, bounds = _crop_to_alpha(foreground) + output.save(output_path, format="PNG", compress_level=1) + return output_path, bounds diff --git a/src/anyimage/server/selection/__init__.py b/src/anyimage/server/selection/__init__.py deleted file mode 100644 index bea3f5a..0000000 --- a/src/anyimage/server/selection/__init__.py +++ /dev/null @@ -1,22 +0,0 @@ -"""Image selection rasterization and Cutout image preparation.""" - -from .cutout_images import refine_selection, write_images, write_selection -from .polygon_mask import ( - apply_selection_mask, - create_selection_canvas, - crop_to_alpha, - polygon_bounds, - polygon_selection_mask, -) - - -__all__ = ( - "apply_selection_mask", - "create_selection_canvas", - "crop_to_alpha", - "polygon_bounds", - "polygon_selection_mask", - "refine_selection", - "write_images", - "write_selection", -) diff --git a/src/anyimage/server/selection/cutout_images.py b/src/anyimage/server/selection/cutout_images.py deleted file mode 100644 index 7e1e974..0000000 --- a/src/anyimage/server/selection/cutout_images.py +++ /dev/null @@ -1,162 +0,0 @@ -"""Prepare Cutout selection images inside the Job Server.""" - -import json - -from ..models.ben2 import infer_foreground -from .polygon_mask import ( - apply_selection_mask, - create_selection_canvas, - polygon_selection_mask, -) - - -def _write_masked_color(input_path, polygon, output_path, invert=False): - import numpy as np - from PIL import Image, ImageFilter, ImageOps - - with Image.open(input_path) as opened: - source = opened.convert("RGBA") - selection = Image.fromarray( - polygon_selection_mask(source.size, polygon, "NONZERO").astype(np.uint8) * 255, - mode="L", - ).filter(ImageFilter.GaussianBlur(radius=0.75)) - if invert: - selection = ImageOps.invert(selection) - pixels = np.asarray(source, dtype=np.uint8).copy() - source_alpha = pixels[:, :, 3].copy() - pixels[:, :, 3] = np.asarray(selection, dtype=np.uint8) - pixels[source_alpha < 26] = 0 - pixels[pixels[:, :, 3] == 0, :3] = 0 - Image.fromarray(pixels, mode="RGBA").save( - output_path, - format="PNG", - compress_level=1, - ) - return output_path - - -def write_selection(input_path, polygon, output_path, invert=False): - import numpy as np - from PIL import Image, ImageFilter - - if invert: - _write_masked_color(input_path, polygon, output_path, invert=True) - with Image.open(input_path) as source: - return output_path, (0, 0, source.width, source.height) - canvas, selection_mask, bounds = create_selection_canvas( - input_path, polygon, "NONZERO" - ) - selection = Image.fromarray( - selection_mask.astype(np.uint8) * 255, - mode="L", - ).filter(ImageFilter.GaussianBlur(radius=0.75)) - pixels = np.asarray(canvas, dtype=np.uint8).copy() - source_alpha = pixels[:, :, 3].copy() - pixels[:, :, 3] = np.asarray(selection, dtype=np.uint8) - pixels[source_alpha < 26] = 0 - pixels[pixels[:, :, 3] == 0, :3] = 0 - Image.fromarray(pixels, mode="RGBA").save( - output_path, - format="PNG", - compress_level=1, - ) - return output_path, bounds - - -def refine_selection(context, parameters, input_path, polygon, output_path): - from PIL import Image, ImageChops - - canvas, selection_mask, bounds = create_selection_canvas( - input_path, polygon, "NONZERO" - ) - selection_path = context.directory / "selection-input.png" - canvas.save(selection_path, format="PNG", compress_level=1) - model_manager = context.resource("model_manager") - foreground, _load_ms, _inference_ms = infer_foreground( - selection_path, - model_manager.directory(parameters["refine_model"]), - parameters["device"], - model_cache=model_manager, - cancel_check=context.check_cancelled, - preserve_input_alpha=True, - ) - refined = apply_selection_mask(foreground, selection_mask) - with Image.open(input_path) as opened: - source = opened.convert("RGBA") - if parameters.get("invert"): - mask = Image.new("L", source.size, 0) - mask.paste(refined.getchannel("A"), bounds[:2]) - source.putalpha( - ImageChops.multiply(source.getchannel("A"), ImageChops.invert(mask)) - ) - output = source - else: - output = Image.new("RGBA", source.size, (0, 0, 0, 0)) - output.paste(refined, bounds[:2]) - output.save(output_path, format="PNG", compress_level=1) - return output_path - - -def write_images(input_path, selection_path, output_directory, selection_bounds=None): - import numpy as np - from PIL import Image - - with Image.open(input_path) as opened: - source = opened.convert("RGBA") - with Image.open(selection_path) as opened: - selection = opened.convert("RGBA") - if selection_bounds is None and selection.size != source.size: - raise ValueError("The Cutout selection does not match the source image") - if selection_bounds is None: - selection_bounds = (0, 0, source.width, source.height) - color_canvas = Image.new("RGBA", selection.size, (0, 0, 0, 0)) - color_canvas.paste(source, (-selection_bounds[0], -selection_bounds[1])) - source_alpha = np.asarray(color_canvas, dtype=np.uint8)[:, :, 3] - selection_pixels = np.asarray(selection, dtype=np.uint8).copy() - alpha = selection_pixels[:, :, 3] - alpha[alpha <= 1] = 0 - alpha[source_alpha < 26] = 0 - selection_pixels[alpha == 0, :3] = 0 - selection = Image.fromarray(selection_pixels, mode="RGBA") - visible_y, visible_x = np.nonzero(alpha >= 26) - if not len(visible_x): - raise ValueError("The Cutout selection contains no visible pixels") - content_bounds = ( - int(visible_x.min()), - int(visible_y.min()), - int(visible_x.max()) + 1, - int(visible_y.max()) + 1, - ) - selection_left, selection_top, _selection_right, _selection_bottom = ( - selection_bounds - ) - bounds = ( - selection_left + content_bounds[0], - selection_top + content_bounds[1], - selection_left + content_bounds[2], - selection_top + content_bounds[3], - ) - color = color_canvas.crop(content_bounds) - color_path = output_directory / "image.0001.png" - color.save(color_path, format="PNG", compress_level=1) - - selection_crop = selection.crop(content_bounds) - selection_path = output_directory / "selection.png" - selection_crop.save(selection_path, format="PNG", compress_level=1) - - analysis_pixels = np.asarray(color, dtype=np.uint8).copy() - mask_alpha = np.asarray(selection_crop, dtype=np.uint8)[:, :, 3] - analysis_pixels[:, :, 3] = np.minimum(analysis_pixels[:, :, 3], mask_alpha) - analysis_pixels[mask_alpha < 26] = 0 - analysis_path = output_directory / "analysis.png" - Image.fromarray(analysis_pixels, mode="RGBA").save( - analysis_path, - format="PNG", - compress_level=1, - ) - bounds_path = output_directory / "bounds.json" - bounds_path.write_text( - json.dumps(bounds, separators=(",", ":")), - encoding="utf-8", - ) - return color_path, selection_path, analysis_path, bounds_path diff --git a/src/anyimage/server/selection/polygon_mask.py b/src/anyimage/server/selection/polygon_mask.py deleted file mode 100644 index 250457a..0000000 --- a/src/anyimage/server/selection/polygon_mask.py +++ /dev/null @@ -1,151 +0,0 @@ -import math - - -def polygon_bounds(polygon): - import numpy as np - - points = np.asarray(polygon, dtype=np.float64) - if points.ndim != 2 or points.shape[0] < 3 or points.shape[1] != 2: - raise ValueError("The selection does not define a valid image region") - if not np.isfinite(points).all(): - raise ValueError("The selection contains invalid image coordinates") - left = int(math.floor(float(points[:, 0].min()))) - top = int(math.floor(float(points[:, 1].min()))) - right = int(math.ceil(float(points[:, 0].max()))) - bottom = int(math.ceil(float(points[:, 1].max()))) - if right <= left or bottom <= top: - raise ValueError("The selection does not define a valid image region") - return left, top, right, bottom - - -def polygon_selection_mask(image_size, polygon, fill_rule="EVEN_ODD"): - """Rasterize a polygon with an explicit fill rule.""" - import numpy as np - - width, height = (int(value) for value in image_size) - points = np.asarray(polygon, dtype=np.float64) - if width <= 0 or height <= 0: - raise ValueError("The selection canvas has no readable dimensions") - if points.ndim != 2 or points.shape[0] < 3 or points.shape[1] != 2: - raise ValueError("The selection does not define a valid image region") - if not np.isfinite(points).all(): - raise ValueError("The selection contains invalid image coordinates") - if fill_rule not in {"EVEN_ODD", "NONZERO"}: - raise ValueError(f"Unknown polygon fill rule: {fill_rule}") - - scanlines = np.arange(height, dtype=np.float64) + 0.5 - crossings = [[] for _index in range(height)] - for start, end in zip(points, np.roll(points, -1, axis=0)): - selected = np.flatnonzero( - (start[1] > scanlines) != (end[1] > scanlines) - ) - if not len(selected): - continue - crossing_x = start[0] + ( - (scanlines[selected] - start[1]) - * (end[0] - start[0]) - / (end[1] - start[1]) - ) - winding = 1 if end[1] > start[1] else -1 - for row, value in zip(selected, crossing_x): - crossings[int(row)].append((float(value), winding)) - - mask = np.zeros((height, width), dtype=bool) - for row, values in enumerate(crossings): - values.sort() - state = 0 - index = 0 - while index < len(values): - crossing_x = values[index][0] - winding = 0 - count = 0 - while index < len(values) and values[index][0] == crossing_x: - winding += values[index][1] - count += 1 - index += 1 - state = ( - state + winding - if fill_rule == "NONZERO" - else state ^ (count % 2) - ) - if index >= len(values) or state == 0: - continue - following_x = values[index][0] - first = max(0, int(math.ceil(crossing_x - 0.5))) - last = min(width, int(math.floor(following_x - 0.5)) + 1) - mask[row, first:last] = True - return mask - -def create_selection_canvas(input_path, polygon, fill_rule="EVEN_ODD"): - """Create an unbounded selection canvas with transparent black outside.""" - import numpy as np - from PIL import Image - - with Image.open(input_path) as opened_image: - source = opened_image.convert("RGBA") - left, top, right, bottom = polygon_bounds(polygon) - if right <= 0 or bottom <= 0 or left >= source.width or top >= source.height: - raise ValueError("The selection does not overlap the source image") - canvas = Image.new("RGBA", (right - left, bottom - top), (0, 0, 0, 0)) - canvas.paste(source, (-left, -top)) - relative = [(float(x) - left, float(y) - top) for x, y in polygon] - selection_mask = polygon_selection_mask(canvas.size, relative, fill_rule) - pixels = np.asarray(canvas, dtype=np.uint8).copy() - pixels[~selection_mask] = 0 - return Image.fromarray(pixels, mode="RGBA"), selection_mask, ( - left, - top, - right, - bottom, - ) - - -def apply_selection_mask(foreground, selection_mask): - """Clear pixels outside the source selection after background removal.""" - import numpy as np - from PIL import Image - - pixels = np.asarray(foreground.convert("RGBA"), dtype=np.uint8).copy() - if pixels.shape[:2] != selection_mask.shape: - pixels = np.asarray( - foreground.convert("RGBA").resize( - (selection_mask.shape[1], selection_mask.shape[0]), - Image.Resampling.BILINEAR, - ), - dtype=np.uint8, - ) - pixels[~selection_mask] = 0 - pixels[pixels[:, :, 3] == 0, :3] = 0 - return Image.fromarray(pixels, mode="RGBA") - - -def crop_to_alpha(image, bounds): - """Crop an RGBA mask image to its nonzero Alpha bounds.""" - import numpy as np - from PIL import Image - - pixels = np.asarray(image.convert("RGBA"), dtype=np.uint8) - visible_y, visible_x = np.nonzero(pixels[:, :, 3] > 0) - if not len(visible_x): - raise ValueError("The selection contains no visible image pixels") - content_left = int(visible_x.min()) - content_top = int(visible_y.min()) - content_right = int(visible_x.max()) + 1 - content_bottom = int(visible_y.max()) + 1 - output = Image.new( - "RGBA", - ( - content_right - content_left, - content_bottom - content_top, - ), - (0, 0, 0, 0), - ) - output.paste(image, (-content_left, -content_top)) - left, top, _right, _bottom = bounds - return output, ( - left + content_left, - top + content_top, - left + content_right, - top + content_bottom, - ) - diff --git a/tests/test_addon.py b/tests/test_addon.py index 6657256..765b887 100644 --- a/tests/test_addon.py +++ b/tests/test_addon.py @@ -84,35 +84,36 @@ class AddonTest(unittest.TestCase): cls.addon = importlib.import_module("anyimage") cls.job_runtime = importlib.import_module("anyimage.runtime") cls.model_operators = importlib.import_module("anyimage.operators.ai_setup") - cls.image_depth = importlib.import_module("anyimage.operators.common.material") - cls.image_data = importlib.import_module("anyimage.operators.common.image") + cls.image_depth = importlib.import_module("anyimage.operators.common.mesh_result") + cls.image_data = importlib.import_module("anyimage.operators.common.image_data") + cls.object_data = importlib.import_module("anyimage.operators.common.mesh_result") cls.debug = importlib.import_module("anyimage.operators.debug") cls.mesh_plane = importlib.import_module("anyimage.operators.convert_to_plane.mesh_plane") cls.mesh = importlib.import_module("anyimage.operators.convert_to_plane.conversion") cls.cutout_conversion = importlib.import_module( - "anyimage.operators.cutout_tool.shape_conversion" + "anyimage.operators.cutout_tool.region_to_shape" ) cls.cutout_build = importlib.import_module( - "anyimage.operators.cutout_tool.build_shapes" + "anyimage.operators.cutout_tool.region_to_shape" ) cls.mesh_shapes = importlib.import_module( - "anyimage.operators.cutout_tool.shape_types" + "anyimage.operators.cutout_tool.region_to_shape" ) cls.image_background = importlib.import_module( "anyimage.operators.remove_background" ) cls.image_upscale = importlib.import_module("anyimage.operators.upscale") cls.image_edit_common = importlib.import_module( - "anyimage.operators.common.image_edit" + "anyimage.operators.common.image_interaction" ) cls.image_mask = importlib.import_module( "anyimage.operators.image_tool.edit_mask" ) - cls.image_mask_data = importlib.import_module( - "anyimage.operators.common.image_mask" + cls.image_region = importlib.import_module( + "anyimage.operators.common.image_region" ) cls.image_gesture = importlib.import_module( - "anyimage.operators.common.image_gesture" + "anyimage.operators.common.image_interaction" ) cls.image_box = importlib.import_module("anyimage.operators.image_tool.box") cls.image_lasso = importlib.import_module( @@ -126,10 +127,10 @@ class AddonTest(unittest.TestCase): "anyimage.operators.perspective_tool.edit_perspective" ) cls.cutout_geometry = importlib.import_module( - "anyimage.operators.cutout_tool.planar_mesh" + "anyimage.operators.cutout_tool.geometry.triangulation" ) cls.cutout_depth = importlib.import_module( - "anyimage.operators.cutout_tool.depth_calibration" + "anyimage.operators.cutout_tool.geometry.depth_calibration" ) cls.image_interaction = importlib.import_module( "anyimage.operators.common.image_interaction" @@ -235,8 +236,8 @@ class AddonTest(unittest.TestCase): for operator, identifier in ( (self.addon.operators.GenerateDepthPlane, "anyimage.generate_depth_plane"), ( - self.addon.operators.GenerateCutoutShape, - "anyimage.generate_cutout_shape", + self.addon.operators.CutoutRegionToShape, + "anyimage.cutout_region_to_shape", ), ): self.assertEqual(operator.bl_idname, identifier) @@ -838,7 +839,7 @@ class AddonTest(unittest.TestCase): operator = self.mesh.ConvertToPlane self.assertIn(operator, self.addon.CLASSES) self.assertIn(self.mesh.ConvertToDepthPlane, self.addon.CLASSES) - self.assertIn(self.cutout_conversion.ConvertCutoutShape, self.addon.CLASSES) + self.assertIn(self.cutout_conversion.CutoutRegionToShape, self.addon.CLASSES) self.assertEqual(operator.bl_idname, "anyimage.convert_to_plane") self.assertNotIn("shadeless", operator.__annotations__) self.assertNotIn("keep_original", operator.__annotations__) @@ -885,7 +886,7 @@ class AddonTest(unittest.TestCase): "image_user_settings", return_value=(1, 0, 24, True), ), - patch.object(self.mesh, "create_material", return_value="Material"), + patch.object(self.mesh, "create_image_material", return_value="Material"), patch.object(self.mesh, "copy_color_image", return_value="Color"), patch.object(self.mesh, "build_mesh_plane") as build_mesh, ): @@ -917,167 +918,86 @@ class AddonTest(unittest.TestCase): parameters = operator.request(context) self.assertEqual(parameters["max_input_size"], 1536) - self.assertEqual(parameters["shape"], "DEPTH_PLANE") + self.assertNotIn("shape", parameters) + self.assertNotIn("polygon", parameters) - def test_cutout_and_balloon_without_refine_build_locally(self): - source = SimpleNamespace(select_set=lambda _selected: None) - context = SimpleNamespace( - view_layer=SimpleNamespace(objects=SimpleNamespace(active=None)) + def test_surface_and_balloon_without_ai_build_locally(self): + import numpy as np + + source = SimpleNamespace(data=SimpleNamespace(size=(10, 10)), select_set=lambda _value: None) + context = SimpleNamespace(view_layer=SimpleNamespace(objects=SimpleNamespace(active=None))) + region_mask = self.cutout_conversion.RegionMask( + np.ones((8, 8), dtype=np.float32), (1, 1, 9, 9) ) - - for shape in ("CUTOUT", "BALLOON"): + source_rgba = np.ones((10, 10, 4), dtype=np.float32) + for shape in ("SURFACE", "BALLOON"): with self.subTest(shape=shape): operator = SimpleNamespace( source_object_name="Source", - selection_data="[[0,0],[10,0],[0,10]]", + region_path_json=json.dumps( + {"points": [[1, 1], [9, 1], [9, 9], [1, 9]]} + ), shape=shape, mesh_detail=64, + normal_mode="NONE", refine_selection=False, - normal_analysis=False, - invert=True, report=lambda *_args: None, + _needs_generation=lambda: False, ) with ( - patch.object( - self.cutout_conversion.bpy.data.objects, - "get", - return_value=source, - ), - patch.object( - self.cutout_conversion, "is_image_empty", return_value=True - ), - patch.object( - self.cutout_conversion, "build_local_cutout" - ) as build_local, + patch.object(self.cutout_conversion, "require_image_empty", return_value=source), + patch.object(self.cutout_conversion, "video_total_frames", return_value=1), + patch.object(self.cutout_conversion, "image_rgba", return_value=source_rgba), patch.object( self.cutout_conversion, - "start_cutout_job", - side_effect=AssertionError("local Cutout must not start a Job"), + "rasterize_region_path", + return_value=region_mask, ), + patch.object(self.cutout_conversion, "configured_max_ai_input_size", return_value=2048), + patch.object(self.cutout_conversion, "region_to_shape") as convert, ): - result = self.cutout_conversion.ConvertCutoutShape.execute( - operator, context - ) + result = self.cutout_conversion.CutoutRegionToShape.execute(operator, context) self.assertEqual(result, {"FINISHED"}) - build_local.assert_called_once_with( - operator, + convert.assert_called_once_with( context, source, shape, - operator.selection_data, - True, + 64, + region_mask, + source_rgba=source_rgba, ) - def test_refined_cutout_and_balloon_submit_blendjob(self): - source = SimpleNamespace(select_set=lambda _selected: None) - context = SimpleNamespace( - view_layer=SimpleNamespace(objects=SimpleNamespace(active=None)) - ) - - for shape in ("CUTOUT", "BALLOON"): - with self.subTest(shape=shape): - operator = SimpleNamespace( - source_object_name="Source", - selection_data="[[0,0],[10,0],[0,10]]", - shape=shape, - mesh_detail=64, - refine_selection=True, - normal_analysis=False, - invert=False, - report=lambda *_args: None, - ) - with ( - patch.object( - self.cutout_conversion.bpy.data.objects, - "get", - return_value=source, - ), - patch.object( - self.cutout_conversion, "is_image_empty", return_value=True - ), - patch.object( - self.cutout_conversion, - "build_local_cutout", - side_effect=AssertionError("Refine must run through the Job"), - ), - patch.object( - self.cutout_conversion, - "start_cutout_job", - return_value={"FINISHED"}, - ) as start_job, - ): - result = self.cutout_conversion.ConvertCutoutShape.execute( - operator, context - ) - - self.assertEqual(result, {"FINISHED"}) - start_job.assert_called_once_with( - operator, - context, - shape, - operator.selection_data, - True, - False, - False, - ) - - def test_plain_balloon_job_requests_moge_normal(self): - source = SimpleNamespace( - name="Source", - data=SimpleNamespace(size=(2048, 1024)), - ) - context = SimpleNamespace(object=source, scene=SimpleNamespace()) - operator = SimpleNamespace(mesh_detail=64, report=lambda *_args: None) - - ops = SimpleNamespace(anyimage=SimpleNamespace()) + def test_cutout_request_describes_ai_capabilities(self): + operator = self.cutout_conversion.CutoutRegionToShape() + operator.input_path = __file__ + operator.source_object_name = "Source" + operator.shape = "DEPTH_SURFACE" + operator.model = self.addon.properties.DEFAULT_MOGE2_MODEL_KEY + operator.resolution_level = 5 + operator.max_input_size = 2048 + operator.normal_mode = "OBJECT" + operator.refine_selection = True + context = SimpleNamespace(scene=SimpleNamespace(anyimage_settings=SimpleNamespace())) with ( - patch.object(self.fake_bpy, "ops", ops, create=True), - patch.object( - self.cutout_conversion, "video_total_frames", return_value=1 - ), - patch.object( - self.cutout_conversion, - "invoke_ai_setup_if_needed", - return_value=None, - ) as setup, - patch.object( - self.cutout_conversion, - "prepare_image_input", - return_value=(Path("input.png"), False), - ), - patch.object( - self.cutout_conversion, - "cutout_moge_settings", - return_value=("MOGE2_VITL", 5), - ), - patch.object( - self.cutout_conversion.bpy.ops.anyimage, - "generate_cutout_shape", - return_value={"FINISHED"}, - create=True, - ) as generate_mesh, + patch.object(self.cutout_conversion, "require_environment"), + patch.object(self.cutout_conversion, "require_model") as require_model, + patch.object(self.cutout_conversion, "production_device", return_value="CPU"), + patch.object(self.cutout_conversion, "source_start_frame", return_value=1), + patch.object(self.cutout_conversion, "moge2_parameters", return_value={"model": operator.model, "resolution_level": 5}), ): - result = self.cutout_conversion.start_cutout_job( - operator, - context, - "BALLOON", - "[[0,0],[10,0],[0,10]]", - normal_analysis=True, - ) + parameters = operator.request(context) - self.assertEqual(result, {"FINISHED"}) - setup.assert_called_once_with() - self.assertEqual(generate_mesh.call_args.kwargs["shape"], "BALLOON") - self.assertEqual(generate_mesh.call_args.kwargs["mesh_detail"], 64) self.assertEqual( - generate_mesh.call_args.kwargs["model"], - "MOGE2_VITL", + [call.args[0] for call in require_model.call_args_list], + [self.addon.properties.BEN2_MODEL_KEY, operator.model], ) - self.assertEqual(generate_mesh.call_args.kwargs["resolution_level"], 5) - self.assertEqual(generate_mesh.call_args.kwargs["max_input_size"], 2048) - self.assertNotIn("process_res", generate_mesh.call_args.kwargs) - self.assertNotIn("normal_model", generate_mesh.call_args.kwargs) + self.assertTrue(parameters["refine_region"]) + self.assertTrue(parameters["output_depth"]) + self.assertEqual(parameters["normal_mode"], "OBJECT") + self.assertNotIn("polygon", parameters) + self.assertNotIn("shape", parameters) + self.assertNotIn("normal_analysis", parameters) def test_cutout_moge_settings_falls_back_from_da3(self): preferences = SimpleNamespace( @@ -1096,81 +1016,6 @@ class AddonTest(unittest.TestCase): (self.addon.properties.DEFAULT_MOGE2_MODEL_KEY, 7), ) - def test_all_cutout_shapes_request_one_moge_model_when_normal_is_enabled(self): - context = SimpleNamespace( - scene=SimpleNamespace(anyimage_settings=SimpleNamespace(video_frames=1)) - ) - for shape in self.mesh_shapes.CUTOUT_SHAPES: - with self.subTest(shape=shape): - operator = self.cutout_conversion.GenerateCutoutShape() - operator.input_path = __file__ - operator.source_object_name = "" - operator.shape = shape - operator.selection_data = "[[0,0],[10,0],[0,10]]" - operator.model = self.addon.properties.DEFAULT_MOGE2_MODEL_KEY - operator.mesh_detail = 64 - operator.resolution_level = 5 - operator.max_input_size = 2048 - operator.normal_analysis = True - operator.refine_selection = False - operator.invert = False - with ( - patch.object(self.cutout_conversion, "require_environment"), - patch.object( - self.cutout_conversion, "require_model" - ) as require_model, - patch.object( - self.cutout_conversion, "production_device", return_value="CPU" - ), - patch.object( - self.cutout_conversion, "source_start_frame", return_value=1 - ), - ): - parameters = operator.request(context) - - require_model.assert_called_once_with( - self.addon.properties.DEFAULT_MOGE2_MODEL_KEY - ) - self.assertEqual( - parameters["model"], - self.addon.properties.DEFAULT_MOGE2_MODEL_KEY, - ) - self.assertEqual(parameters["resolution_level"], 5) - self.assertNotIn("normal_model", parameters) - self.assertNotIn("normal_resolution_level", parameters) - self.assertEqual(parameters["mesh_detail"], 64) - self.assertEqual(parameters["max_input_size"], 2048) - - def test_plain_cutout_job_skips_moge_when_normal_is_disabled(self): - operator = self.cutout_conversion.GenerateCutoutShape() - operator.input_path = __file__ - operator.source_object_name = "" - operator.shape = "CUTOUT" - operator.selection_data = "[[0,0],[10,0],[0,10]]" - operator.mesh_detail = 64 - operator.max_input_size = 2048 - operator.normal_analysis = False - operator.refine_selection = False - operator.invert = False - - with ( - patch.object(self.cutout_conversion, "require_environment"), - patch.object(self.cutout_conversion, "require_model") as require_model, - patch.object( - self.cutout_conversion, "production_device", return_value="CPU" - ), - patch.object( - self.cutout_conversion, "source_start_frame", return_value=1 - ), - ): - parameters = operator.request(SimpleNamespace(scene=SimpleNamespace())) - - require_model.assert_not_called() - self.assertFalse(parameters["normal_analysis"]) - self.assertEqual(parameters["max_input_size"], 2048) - self.assertNotIn("model", parameters) - self.assertNotIn("resolution_level", parameters) - def test_image_menu_keeps_ai_features_clickable_and_offers_models(self): calls = [] labels = {} @@ -1226,7 +1071,7 @@ class AddonTest(unittest.TestCase): self.assertEqual( labels["anyimage.upscale_image"], - "Upscale (640 × 480)", + "Upscale (2560 × 1920)", ) def test_image_menu_opens_environment_settings_when_ai_is_ready(self): @@ -1284,7 +1129,7 @@ class AddonTest(unittest.TestCase): patch.object(self.image_upscale, "require_environment"), patch.object(self.image_upscale, "require_model") as require_model, patch.object( - self.image_upscale, "input_path", return_value=Path("input.png") + self.image_upscale, "require_input_path", return_value=Path("input.png") ), patch.object(self.image_upscale, "production_device", return_value="CUDA"), patch.object(self.image_upscale, "source_start_frame", return_value=4), @@ -1383,123 +1228,6 @@ class AddonTest(unittest.TestCase): self.assertIs(result, expected) self.assertNotIn("REGISTER", operator.bl_options) - def test_upscale_response_replaces_source_image(self): - original_image = SimpleNamespace(name="Original") - source_object = SimpleNamespace( - type="EMPTY", - empty_display_type="IMAGE", - data=original_image, - ) - self.fake_bpy.data = SimpleNamespace( - objects=SimpleNamespace( - get=lambda name: source_object if name == "Source" else None - ), - images=SimpleNamespace( - get=lambda name: original_image if name == "Original" else None - ), - ) - operator = self.image_upscale.UpscaleImage() - operator.source_object_name = "Source" - operator.source_image_name = "Original" - operator.model = "REALESRGAN_GENERAL_WDN_X4V3" - result = SimpleNamespace(directory=Path("result")) - result_image = SimpleNamespace(name="", anyimage_upscale_source=None) - with patch.object( - self.image_upscale, - "create_color_image", - return_value=result_image, - ) as create_color: - message = operator.response(None, result) - - create_color.assert_called_once_with( - source_object, - result.directory, - frame_prefix="upscale.", - ) - self.assertEqual(result_image.name, "Original_upscale.png") - self.assertIs( - result_image.anyimage_upscale_source, - original_image, - ) - self.assertEqual( - message, - "Image is upscaled with Real-ESRGAN General WDN x4v3", - ) - - def test_upscale_response_reports_removed_original_image(self): - source_object = SimpleNamespace( - type="EMPTY", - empty_display_type="IMAGE", - data=object(), - ) - self.fake_bpy.data = SimpleNamespace( - objects=SimpleNamespace(get=lambda _name: source_object), - images=SimpleNamespace(get=lambda _name: None), - ) - operator = self.image_upscale.UpscaleImage() - operator.source_object_name = "Source" - operator.source_image_name = "Removed" - - with self.assertRaisesRegex( - RuntimeError, - "original source image is no longer available", - ): - operator.response(None, SimpleNamespace(directory=Path("result"))) - - def test_depth_analysis_follows_the_oldest_upscale_source(self): - original = SimpleNamespace(name="Original") - first_upscale = SimpleNamespace( - name="First", - anyimage_upscale_source=original, - ) - second_upscale = SimpleNamespace( - name="Second", - anyimage_upscale_source=first_upscale, - ) - - self.assertIs( - self.image_data.analysis_source_image(second_upscale), - original, - ) - - def test_repeated_upscale_uses_original_image_and_reports_warning(self): - original_image = SimpleNamespace(name="Original", source="FILE") - current_image = SimpleNamespace( - source="FILE", - anyimage_upscale_source=original_image, - ) - source_object = SimpleNamespace( - name="Source", - type="EMPTY", - empty_display_type="IMAGE", - data=current_image, - ) - context = SimpleNamespace( - object=source_object, - scene=SimpleNamespace(anyimage_settings=SimpleNamespace(video_frames=1)), - ) - reports = [] - operator = self.image_upscale.UpscaleImage() - operator.report = lambda levels, message: reports.append((levels, message)) - with ( - patch.object( - self.image_upscale, - "prepare_image_input", - return_value=(Path("original.png"), False), - ) as prepare, - patch.object( - self.job_runtime.JobOperatorBase, - "execute", - return_value={"RUNNING_MODAL"}, - ), - ): - result = operator.execute(context) - - self.assertEqual(result, {"RUNNING_MODAL"}) - prepare.assert_called_once_with(original_image, context.scene) - self.assertEqual(operator.source_image_name, "Original") - self.assertIn("rebuilding from the original", reports[0][1]) - def test_packed_image_export_does_not_use_scene_render_transform(self): with tempfile.TemporaryDirectory() as directory: output_directory = Path(directory) @@ -1530,104 +1258,10 @@ class AddonTest(unittest.TestCase): ) self.assertEqual(image.file_format, "JPEG") - def test_color_result_inherits_source_color_space(self): - with tempfile.TemporaryDirectory() as directory: - output_directory = Path(directory) - (output_directory / "image.0001.png").touch() - source_image = SimpleNamespace( - name="source.png", - source="FILE", - use_fake_user=False, - colorspace_settings=SimpleNamespace(name="ACEScg"), - ) - - class ResultImage: - def __init__(self): - self.alpha_mode = "" - self.colorspace_settings = SimpleNamespace(name="sRGB") - self.name = "" - - def pack(self): - pass - - result_image = ResultImage() - source_object = SimpleNamespace( - name="Source", - type="EMPTY", - empty_display_type="IMAGE", - data=source_image, - ) - self.fake_bpy.data = SimpleNamespace( - images=SimpleNamespace( - load=lambda *_args, **_options: result_image, - remove=lambda *_args, **_options: None, - ) - ) - - created = self.image_data.create_color_image( - source_object, - output_directory, - ) - - self.assertIs(created, result_image) - self.assertEqual(result_image.colorspace_settings.name, "ACEScg") - self.assertEqual(result_image.name, "source_color.png") - self.assertIs(source_object.data, result_image) - - def test_background_result_replaces_only_an_unshared_image(self): - for remaining_users, removed_source, result_name in ( - (0, True, "Source.png"), - (1, False, "Source_color.png"), - ): - with self.subTest(remaining_users=remaining_users): - source_image = SimpleNamespace( - name="Source.png", - use_fake_user=False, - users=remaining_users, - ) - source_object = SimpleNamespace( - name="Source", - type="EMPTY", - empty_display_type="IMAGE", - data=source_image, - ) - result_image = SimpleNamespace(name="Source_color.png") - removed = [] - self.fake_bpy.data = SimpleNamespace( - objects={"Source": source_object}, - images=SimpleNamespace( - remove=lambda image: removed.append(image), - ), - ) - - def create_result(target, _directory): - source_image.use_fake_user = True - target.data = result_image - return result_image - - with ( - patch.object( - self.image_data, - "create_color_image", - side_effect=create_result, - ), - patch.object(self.image_background, "push_undo") as push_undo, - ): - self.image_background.apply_background_response( - SimpleNamespace(directory=Path("result")), - "Source", - ) - - self.assertIs(source_object.data, result_image) - self.assertFalse(source_image.use_fake_user) - self.assertEqual(removed, [source_image] if removed_source else []) - self.assertEqual(result_image.name, result_name) - push_undo.assert_called_once_with("Remove Background") - def test_cutout_contour_is_smooth_and_evenly_resampled(self): import numpy as np - contour = self.cutout_geometry.smooth_lasso_contour( + contour = self.cutout_geometry.smooth_region_contour( ((0, 0), (3, 0), (4, 0), (4, 2), (0, 2)), 0.25, ) @@ -1635,7 +1269,7 @@ class AddonTest(unittest.TestCase): lengths = np.linalg.norm(np.diff(closed, axis=0), axis=1) self.assertGreater(len(contour), 20) - self.assertGreater(self.cutout_geometry.polygon_area(contour), 0.0) + self.assertGreater(self.cutout_geometry.contour_area(contour), 0.0) self.assertLess(float(lengths.std() / lengths.mean()), 0.15) def test_cutout_contour_smooths_pixel_stairs_before_coarse_resampling(self): @@ -1646,7 +1280,7 @@ class AddonTest(unittest.TestCase): staircase.append((value + 1.0, float(value))) staircase.append((value + 1.0, value + 1.0)) staircase.append((0.0, 16.0)) - smooth = self.cutout_geometry.smooth_lasso_contour(staircase, 2.0) + smooth = self.cutout_geometry.smooth_region_contour(staircase, 2.0) def diagonal_noise(points): points = np.asarray(points) @@ -1659,8 +1293,8 @@ class AddonTest(unittest.TestCase): self.assertLess(diagonal_noise(smooth), 0.05) - def test_cutout_polygon_mask_keeps_nonzero_winding_overlap(self): - polygon = ( + def test_region_mask_keeps_nonzero_winding_overlap(self): + path = ( (1, 1), (7, 1), (7, 7), @@ -1674,15 +1308,21 @@ class AddonTest(unittest.TestCase): (1, 1), ) - even_odd = self.image_mask_data.polygon_mask((12, 9), polygon) - nonzero = self.image_mask_data.polygon_mask((12, 9), polygon, "NONZERO") + even_odd = self.image_region.rasterize_region_path( + (12, 9), + self.image_region.RegionPath(points=path, fill_rule="EVEN_ODD"), + ).full_values((12, 9)) + nonzero = self.image_region.rasterize_region_path( + (12, 9), + self.image_region.RegionPath(points=path, fill_rule="NONZERO"), + ).full_values((12, 9)) self.assertFalse(even_odd[3, 5]) self.assertTrue(nonzero[3, 5]) def preview_area(fill_rule): triangles = self.image_edit_common.scanline_fill_triangles( - polygon, fill_rule + path, fill_rule ) return sum( abs( @@ -1698,68 +1338,6 @@ class AddonTest(unittest.TestCase): self.assertAlmostEqual(preview_area('EVEN_ODD'), 36.0) self.assertAlmostEqual(preview_area('NONZERO'), 54.0) - def test_image_mask_preserves_source_rgb_outside_the_selection(self): - import numpy as np - - top_down = np.full((3, 3, 4), (0.2, 0.4, 0.6, 1.0), dtype=np.float32) - source = np.flipud(top_down).ravel() - selection = self.image_mask_data.polygon_mask( - (3, 3), ((0, 0), (3, 0), (0, 3)) - ) - pixels, bounds = self.image_mask_data.edit_image_mask_pixels( - source, - (3, 3), - selection, - ) - width = bounds[2] - bounds[0] - height = bounds[3] - bounds[1] - result = np.flipud(np.asarray(pixels).reshape((height, width, 4))) - - self.assertEqual(bounds, (0, 0, 2, 2)) - np.testing.assert_allclose(result[1, 1], (0.2, 0.4, 0.6, 0.0)) - self.assertEqual(float(result[0, 0, 3]), 1.0) - - def test_image_mask_preserves_hidden_rgb_from_source_transparency(self): - import numpy as np - - top_down = np.full((2, 2, 4), (0.2, 0.4, 0.6, 1.0), dtype=np.float32) - top_down[0, 0, 3] = 0.0 - pixels, _bounds = self.image_mask_data.edit_image_mask_pixels( - np.flipud(top_down).ravel(), - (2, 2), - np.ones((2, 2), dtype=np.float32), - ) - result = np.flipud(np.asarray(pixels).reshape((2, 2, 4))) - - np.testing.assert_allclose(result[0, 0], (0.2, 0.4, 0.6, 0.0)) - - def test_image_mask_invert_runs_after_refinement(self): - import numpy as np - - top_down = np.full((2, 2, 4), (0.2, 0.4, 0.6, 1.0), dtype=np.float32) - refined = np.asarray(((1.0, 0.25), (0.0, 1.0)), dtype=np.float32) - pixels, bounds = self.image_mask_data.edit_image_mask_pixels( - np.flipud(top_down).ravel(), - (2, 2), - refined, - invert=True, - ) - result = np.flipud(np.asarray(pixels).reshape((2, 2, 4))) - - self.assertEqual(bounds, (0, 0, 2, 2)) - self.assertEqual(float(result[0, 0, 3]), 0.0) - self.assertAlmostEqual(float(result[0, 1, 3]), 0.75) - np.testing.assert_allclose(result[0, 0], (0.2, 0.4, 0.6, 0.0)) - - def test_image_mask_rejects_a_fully_transparent_result(self): - import numpy as np - - with self.assertRaisesRegex(ValueError, "no visible"): - self.image_mask_data.edit_image_mask_pixels( - np.ones(2 * 2 * 4, dtype=np.float32), - (2, 2), - np.zeros((2, 2), dtype=np.float32), - ) def test_expected_empty_image_edits_are_reported_as_warnings(self): reports = [] @@ -1789,165 +1367,6 @@ class AddonTest(unittest.TestCase): ], ) - def test_image_mask_crops_to_visible_alpha_without_padding(self): - import numpy as np - - rgba = np.full((4, 5, 4), (0.8, 0.7, 0.6, 0.0), dtype=np.float32) - rgba[1, 2] = (0.2, 0.4, 0.6, 1.0) - rgba[2, 3] = (0.2, 0.4, 0.6, 1.0) - pixels, bounds = self.image_edit_common.crop_pixels_to_alpha( - np.flipud(rgba).ravel(), - (10, 20, 15, 24), - ) - result = np.flipud(np.asarray(pixels).reshape((2, 2, 4))) - - self.assertEqual(bounds, (12, 21, 14, 23)) - self.assertEqual(result.shape, (2, 2, 4)) - np.testing.assert_allclose(result[0, 1], (0.8, 0.7, 0.6, 0.0)) - - def test_image_mask_restores_refined_alpha_to_source_coordinates(self): - import numpy as np - - top_down = np.zeros((2, 2, 4), dtype=np.float32) - top_down[:, :, 3] = ((1.0, 0.5), (0.0, 1.0)) - mask = self.image_mask_data.refined_mask_from_pixels( - np.flipud(top_down).ravel(), - (2, 2), - (1, 1, 3, 3), - (4, 4), - ) - - expected = np.zeros((4, 4), dtype=np.float32) - expected[1:3, 1:3] = top_down[:, :, 3] - np.testing.assert_allclose(mask, expected) - - def test_image_edit_replaces_an_unshared_image_without_leaving_a_copy(self): - class Pixels: - values = None - - def foreach_set(self, values): - self.values = tuple(values) - - source_image = SimpleNamespace( - name="Source.png", - size=(4, 2), - colorspace_settings=SimpleNamespace(name="sRGB"), - use_fake_user=False, - users=0, - ) - source_object = SimpleNamespace( - type="EMPTY", - empty_display_type="IMAGE", - data=source_image, - empty_display_size=1.0, - empty_image_offset=(0.0, 0.0), - ) - result_image = SimpleNamespace( - name="", - alpha_mode="", - colorspace_settings=SimpleNamespace(name=""), - pixels=Pixels(), - pack=lambda: packed.append(True), - update=lambda: updated.append(True), - ) - packed = [] - updated = [] - removed = [] - self.fake_bpy.data = SimpleNamespace( - objects=[source_object], - images=SimpleNamespace( - new=lambda *_args, **_options: result_image, - remove=lambda image, **_options: removed.append(image), - ), - ) - pixels = tuple(float(index) for index in range(16)) - with patch.object( - self.image_edit_common, - "empty_display_alignment", - return_value=(2.0, (0.25, -0.25)), - ): - result = self.image_edit_common.replace_empty_image_pixels( - source_object, pixels, (2, 2), (0, 0, 2, 2) - ) - - self.assertIs(result, result_image) - self.assertIs(source_object.data, result_image) - self.assertEqual(removed, [source_image]) - self.assertEqual(result_image.name, "Source.png") - self.assertEqual(result_image.pixels.values, pixels) - self.assertEqual(packed, [True]) - self.assertEqual(updated, [True]) - self.assertEqual(source_object.empty_display_size, 2.0) - self.assertEqual(source_object.empty_image_offset, (0.25, -0.25)) - - def test_image_edit_forks_an_image_shared_by_multiple_empties(self): - class Pixels: - values = None - - def foreach_set(self, values): - self.values = tuple(values) - - class ResultImage: - colorspace_settings = SimpleNamespace(name="") - pixels = Pixels() - alpha_mode = "" - - def pack(self): - pass - - def update(self): - pass - - source_image = SimpleNamespace( - name="Source.png", - size=(4, 2), - colorspace_settings=SimpleNamespace(name="ACEScg"), - use_fake_user=False, - users=1, - ) - source_object = SimpleNamespace( - type="EMPTY", - empty_display_type="IMAGE", - data=source_image, - empty_display_size=1.0, - empty_image_offset=(0.0, 0.0), - ) - sibling = SimpleNamespace( - type="EMPTY", - empty_display_type="IMAGE", - data=source_image, - ) - result_image = ResultImage() - created = [] - - def new_image(*args, **options): - created.append((args, options)) - return result_image - - self.fake_bpy.data = SimpleNamespace( - objects=[source_object, sibling], - images=SimpleNamespace( - new=new_image, - remove=lambda *_args, **_options: None, - ), - ) - pixels = tuple(float(index) for index in range(16)) - with patch.object( - self.image_edit_common, - "empty_display_alignment", - return_value=(2.0, (0.25, -0.25)), - ): - result = self.image_edit_common.replace_empty_image_pixels( - source_object, pixels, (2, 2), (0, 0, 2, 2) - ) - - self.assertIs(result, result_image) - self.assertIs(source_object.data, result_image) - self.assertIs(sibling.data, source_image) - self.assertEqual(len(created), 1) - self.assertEqual(result_image.colorspace_settings.name, "ACEScg") - self.assertEqual(result_image.pixels.values, pixels) - def test_image_edit_tools_and_operators_are_registered(self): operator = self.image_mask.EditImageMask self.assertIn(operator, self.addon.CLASSES) @@ -1973,12 +1392,12 @@ class AddonTest(unittest.TestCase): self.assertIn("draw_settings", self.image_lasso.ImageLassoTool.__dict__) context = SimpleNamespace(scene=SimpleNamespace(anyimage_settings=object())) layout = SimpleNamespace(prop=lambda *_args: None) - with patch.object(self.image_box, "draw_image_mask_settings") as draw: + with patch.object(self.image_box, "draw_region_settings") as draw: self.image_box.ImageBoxTool.draw_settings(context, layout, None) draw.assert_called_once_with( context, layout, "image_box_refine_selection" ) - with patch.object(self.image_lasso, "draw_image_mask_settings") as draw: + with patch.object(self.image_lasso, "draw_region_settings") as draw: self.image_lasso.ImageLassoTool.draw_settings(context, layout, None) draw.assert_called_once_with( context, layout, "image_lasso_refine_selection" @@ -1987,7 +1406,7 @@ class AddonTest(unittest.TestCase): "anyimage.operators.cutout_tool" ).CutoutTool cutout_module = sys.modules[cutout_tool.__module__] - with patch.object(cutout_module, "draw_image_mask_settings") as draw: + with patch.object(cutout_module, "draw_region_settings") as draw: cutout_tool.draw_settings(context, layout, None) draw.assert_called_once_with(context, layout, "cutout_refine_selection") self.assertNotIn("draw_settings", self.image_line.ImageLineTool.__dict__) @@ -2010,7 +1429,7 @@ class AddonTest(unittest.TestCase): self.assertEqual(settings["cutout_point_spacing"]["default"], 32) self.assertEqual(settings["cutout_point_spacing"]["min"], 4) self.assertEqual(settings["cutout_point_spacing"]["max"], 32) - self.assertFalse(settings["cutout_normal_analysis"]["default"]) + self.assertEqual(settings["cutout_normal_mode"]["default"], "NONE") self.assertNotIn("image_extract_continuous", settings) self.assertNotIn("image_extract_padding", settings) self.assertNotIn("image_extract_remove_background", settings) @@ -2027,15 +1446,15 @@ class AddonTest(unittest.TestCase): gesture=gesture, refine_selection=True, ) - self.image_edit_common.apply_image_mask_settings(operator, context) + self.image_edit_common.apply_region_settings(operator, context) self.assertFalse(operator.refine_selection) box = SimpleNamespace(gesture="BOX", refine_selection=True) - self.image_edit_common.apply_image_mask_settings(box, context) + self.image_edit_common.apply_region_settings(box, context) self.assertFalse(box.refine_selection) lasso = SimpleNamespace(gesture="LASSO", refine_selection=False) - self.image_edit_common.apply_image_mask_settings(lasso, context) + self.image_edit_common.apply_region_settings(lasso, context) self.assertTrue(lasso.refine_selection) cutout = SimpleNamespace( @@ -2043,7 +1462,7 @@ class AddonTest(unittest.TestCase): active_tool_id="anyimage.cutout_lasso", refine_selection=False, ) - self.image_edit_common.apply_image_mask_settings(cutout, context) + self.image_edit_common.apply_region_settings(cutout, context) self.assertTrue(cutout.refine_selection) def test_image_edit_first_click_selects_another_image_without_editing(self): @@ -2382,15 +1801,17 @@ class AddonTest(unittest.TestCase): (), ) - def test_image_box_completion_serializes_the_pixel_selection(self): + def test_image_box_completion_serializes_the_image_region(self): source = SimpleNamespace(data=SimpleNamespace(size=(64, 32))) finished = [] operator = SimpleNamespace( source_object_name="Source", source_matrix_data="matrix", gesture="BOX", - selection_data="", - _screen_polygon=lambda: ((1, 1), (8, 1), (8, 6), (1, 6)), + region_path_json="", + invert=False, + preview_fill_rule="EVEN_ODD", + _screen_path=lambda: ((1, 1), (8, 1), (8, 6), (1, 6)), _finish=lambda context: finished.append(context), execute=lambda _context: {"FINISHED"}, ) @@ -2399,26 +1820,28 @@ class AddonTest(unittest.TestCase): with ( patch.object( - self.fake_bpy.data.objects, "get", return_value=source + self.image_gesture, "require_image_empty", return_value=source ), - patch.object(self.image_gesture, "is_image_empty", return_value=True), patch.object(self.image_gesture, "deserialize_matrix"), - patch.object(self.image_gesture, "plane_bounds"), + patch.object(self.image_gesture, "image_empty_bounds"), patch.object( self.image_gesture, "screen_path_to_image_pixels", return_value=polygon, ), ): - result = self.image_gesture.ImageMaskGesture._complete( + result = self.image_gesture.ImageRegionGesture._complete( operator, context ) self.assertEqual(result, {"FINISHED"}) - self.assertEqual(operator.selection_data, "[[1,2],[3,4],[5,6]]") + self.assertEqual( + json.loads(operator.region_path_json)["points"], + [[1.0, 2.0], [3.0, 4.0], [5.0, 6.0]], + ) self.assertEqual(finished, [context]) - def test_image_edit_selection_uses_white_dashed_segments(self): + def test_image_region_uses_white_dashed_segments(self): segments = self.image_edit_common.dashed_line_segments( ((0, 0), (10, 0)), dash_length=3.0, gap_length=2.0 ) @@ -2441,7 +1864,7 @@ class AddonTest(unittest.TestCase): _fill_triangles=(), _outline_polygon=None, _outline_segments=(), - _screen_polygon=lambda: polygon, + _screen_path=lambda: polygon, ) with ( patch.object(self.image_gesture, 'drawing_in_region', return_value=True), @@ -2458,8 +1881,8 @@ class AddonTest(unittest.TestCase): patch.object(self.image_gesture, 'draw_polygon_fill') as draw_fill, patch.object(self.image_gesture, 'draw_dashed_line') as draw_outline, ): - self.image_gesture.ImageMaskGesture._draw_selection(operator) - self.image_gesture.ImageMaskGesture._draw_selection(operator) + self.image_gesture.ImageRegionGesture._draw_region(operator) + self.image_gesture.ImageRegionGesture._draw_region(operator) build_fill.assert_called_once_with(polygon, 'NONZERO') build_outline.assert_called_once_with((*polygon, polygon[0])) @@ -2518,66 +1941,15 @@ class AddonTest(unittest.TestCase): image = self.image_gesture.image_line_half_plane_polygon( ((0, 5), (10, 5)), (10, 10) ) - self.assertEqual(self.image_edit_common.polygon_bounds(preview), (0, 5, 10, 10)) - self.assertEqual(self.image_edit_common.polygon_bounds(image), (0, 0, 10, 5)) - - def test_cutout_interior_points_have_uniform_spacing(self): - import numpy as np - - contour = np.asarray(((0, 0), (4, 0), (4, 4), (0, 4)), dtype=float) - points = self.cutout_geometry.uniform_interior_points(contour, 0.5) - distances = np.linalg.norm(points[:, None] - points[None, :], axis=2) - np.fill_diagonal(distances, np.inf) - nearest = distances.min(axis=1) - - self.assertGreater(len(points), 20) - self.assertAlmostEqual(float(np.median(nearest)), 0.5, delta=0.02) - self.assertGreater(float(nearest.min()), 0.49) - - def test_cutout_default_sampling_stays_coarse(self): - import numpy as np - - polygon = np.asarray( - ((100, 100), (900, 100), (950, 500), (800, 900), (200, 900), (50, 500)), - dtype=float, + self.assertEqual( + self.image_region.region_path_bounds(preview), + (0, 5, 10, 10), ) - contour, interior, _spacing = self.cutout_geometry.prepare_planar_points( - polygon, - (1000, 1000), - (0.0, 1.0, 0.0, 1.0), + self.assertEqual( + self.image_region.region_path_bounds(image), + (0, 0, 10, 5), ) - self.assertLess(len(contour), 150) - self.assertLess(len(interior), 400) - - def test_cutout_sampling_uses_world_distance(self): - import numpy as np - - polygon = np.asarray( - ((0, 0), (1000, 0), (1000, 1000), (0, 1000)), - dtype=float, - ) - unit_contour, unit_interior, unit_spacing = ( - self.cutout_geometry.prepare_planar_points( - polygon, - (1000, 1000), - (0.0, 1.0, 0.0, 1.0), - ) - ) - scaled_contour, scaled_interior, scaled_spacing = ( - self.cutout_geometry.prepare_planar_points( - polygon, - (1000, 1000), - (0.0, 2.0, 0.0, 2.0), - world_metric=((0.25, 0.0), (0.0, 0.25)), - ) - ) - - self.assertEqual(unit_spacing, 0.1) - self.assertEqual(scaled_spacing, 0.1) - self.assertEqual(len(unit_contour), len(scaled_contour)) - self.assertEqual(len(unit_interior), len(scaled_interior)) - def test_image_operators_stay_clickable_until_server_is_busy(self): context = SimpleNamespace( object=SimpleNamespace(data=SimpleNamespace()), @@ -2777,7 +2149,7 @@ class AddonTest(unittest.TestCase): self.outliner_menu_draws[0], self.addon.draw_image_context_menu, ) - self.assertTrue(hasattr(self.fake_bpy.types.Image, "anyimage_upscale_source")) + self.assertFalse(hasattr(self.fake_bpy.types.Image, "anyimage_upscale_source")) self.addon.unregister() self.assertEqual(self.registered, []) diff --git a/tests/test_cutout.py b/tests/test_cutout.py index 90c8aea..a9de31f 100644 --- a/tests/test_cutout.py +++ b/tests/test_cutout.py @@ -8,8 +8,9 @@ import bpy SOURCE_ROOT = Path(__file__).parents[1] / "src" sys.path.insert(0, str(SOURCE_ROOT)) -from anyimage.operators.common.image_gesture import ImageMaskGesture -from anyimage.operators.cutout_tool import selection_tool as cutout +from anyimage.operators.common.image_interaction import ImageRegionGesture +from anyimage.operators.cutout_tool import select_region as cutout +from anyimage.operators.common.image_region import RegionPath class Pie: @@ -31,10 +32,10 @@ class WindowManager: draw(menu, None) -def cutout_context(window_manager, point_spacing=32, normal_analysis=False): +def cutout_context(window_manager, point_spacing=32, normal_mode="NONE"): settings = SimpleNamespace( cutout_point_spacing=point_spacing, - cutout_normal_analysis=normal_analysis, + cutout_normal_mode=normal_mode, ) return SimpleNamespace( window_manager=window_manager, @@ -51,10 +52,15 @@ def test_cutout_pie_uses_four_semantic_shapes_in_cardinal_order(): context = cutout_context(window_manager) source = cutout_source() - cutout._open_shape_pie(context, object(), source, ((0, 0), (10, 0), (0, 10))) + cutout._open_shape_pie( + context, + object(), + source, + RegionPath(points=((0, 0), (10, 0), (0, 10))), + ) assert [call[2].shape for call in window_manager.pie.calls] == [ - "CUTOUT", + "SURFACE", "BALLOON", "DEPTH_BALLOON", "DEPTH_SURFACE", @@ -66,18 +72,18 @@ def test_cutout_pie_uses_four_semantic_shapes_in_cardinal_order(): "Depth Surface", ] assert all( - call[0] == "anyimage.convert_cutout_shape" for call in window_manager.pie.calls + call[0] == "anyimage.cutout_region_to_shape" + for call in window_manager.pie.calls ) assert not any(call[2].refine_selection for call in window_manager.pie.calls) - assert not any(call[2].normal_analysis for call in window_manager.pie.calls) - assert not any(call[2].invert for call in window_manager.pie.calls) + assert all(call[2].normal_mode == "NONE" for call in window_manager.pie.calls) assert all(call[2].mesh_detail == 64 for call in window_manager.pie.calls) def test_cutout_pie_passes_image_lasso_options_to_every_shape(): window_manager = WindowManager() context = cutout_context( - window_manager, point_spacing=16, normal_analysis=True + window_manager, point_spacing=16, normal_mode="OBJECT" ) source = cutout_source(size=(4096, 2048)) @@ -85,14 +91,19 @@ def test_cutout_pie_passes_image_lasso_options_to_every_shape(): context, object(), source, - ((0, 0), (10, 0), (0, 10)), + RegionPath( + points=((0, 0), (10, 0), (0, 10)), + invert=True, + ), refine_selection=True, - invert=True, ) assert all(call[2].refine_selection for call in window_manager.pie.calls) - assert all(call[2].normal_analysis for call in window_manager.pie.calls) - assert all(call[2].invert for call in window_manager.pie.calls) + assert all(call[2].normal_mode == "OBJECT" for call in window_manager.pie.calls) + assert all( + RegionPath.from_json(call[2].region_path_json).invert + for call in window_manager.pie.calls + ) assert all(call[2].mesh_detail == 256 for call in window_manager.pie.calls) @@ -116,8 +127,8 @@ def test_cutout_reuses_image_lasso_gesture_and_double_click_selects_all(monkeypa cutout, "resolve_image_edit_click", lambda *_args: (source, True) ) monkeypatch.setattr(cutout, "video_total_frames", lambda _image: 1) - monkeypatch.setattr(cutout, "apply_image_mask_settings", lambda *_args: None) - monkeypatch.setattr(cutout, "selected_image_rgba", lambda *_args: object()) + monkeypatch.setattr(cutout, "apply_region_settings", lambda *_args: None) + monkeypatch.setattr(cutout, "_validate_cutout_region", lambda *_args: None) operator = SimpleNamespace( full_image=True, @@ -129,10 +140,12 @@ def test_cutout_reuses_image_lasso_gesture_and_double_click_selects_all(monkeypa operator, context, SimpleNamespace(value="DOUBLE_CLICK") ) - assert issubclass(cutout.SelectCutoutRegion, ImageMaskGesture) + assert issubclass(cutout.SelectCutoutRegion, ImageRegionGesture) assert result == {"FINISHED"} - selection = json.loads(window_manager.pie.calls[0][2].selection_data) - assert selection == [[0, 0], [320, 0], [320, 180], [0, 180]] + region_path = RegionPath.from_json( + window_manager.pie.calls[0][2].region_path_json + ) + assert region_path.points == ((0, 0), (320, 0), (320, 180), (0, 180)) assert window_manager.pie.calls[0][2].mesh_detail == 10 assert not hasattr(cutout, "pending_selection") diff --git a/tests/test_image.py b/tests/test_image.py index c530156..2ca6ce8 100644 --- a/tests/test_image.py +++ b/tests/test_image.py @@ -27,21 +27,31 @@ def load_image_modules(): module_paths = ( ( - "anyimage.operators.common.image", - PROJECT_ROOT / "src" / "anyimage" / "operators" / "common" / "image.py", - ), - ( - "anyimage.operators.common.node_asset", + "anyimage.operators.common.image_data", PROJECT_ROOT / "src" / "anyimage" / "operators" / "common" - / "node_asset.py", + / "image_data.py", ), ( - "anyimage.operators.common.material", - PROJECT_ROOT / "src" / "anyimage" / "operators" / "common" / "material.py", + "anyimage.operators.common.mesh_result", + PROJECT_ROOT + / "src" + / "anyimage" + / "operators" + / "common" + / "mesh_result.py", + ), + ( + "anyimage.operators.convert_to_plane.resolution", + PROJECT_ROOT + / "src" + / "anyimage" + / "operators" + / "convert_to_plane" + / "resolution.py", ), ) packages = { @@ -50,6 +60,7 @@ def load_image_modules(): "anyimage", "anyimage.operators", "anyimage.operators.common", + "anyimage.operators.convert_to_plane", ) } for package in packages.values(): @@ -452,21 +463,7 @@ class ImageTest(unittest.TestCase): "img_normal.png", ) - def test_copy_depth_image_renames_still_source(self): - class FakeImage: - def __init__(self, name): - self.name = name - - def copy(self): - return FakeImage("img.png.001") - - source_object = SimpleNamespace(data=FakeImage("img.png")) - - depth_image = self.image_depth.copy_depth_image(source_object) - - self.assertEqual(depth_image.name, "img_depth.exr") - - def test_color_and_material_names_use_source_image_base_name(self): + def test_color_name_uses_source_image_base_name(self): source_object = SimpleNamespace( data=SimpleNamespace(name="img.png"), ) @@ -475,19 +472,11 @@ class ImageTest(unittest.TestCase): self.image_depth.color_data_name(source_object), "img_color.png", ) - self.assertEqual( - self.image_depth.material_data_name(source_object), - "img", - ) source_object.data.name = "img.png.001" self.assertEqual( self.image_depth.color_data_name(source_object), "img.001_color.png", ) - self.assertEqual( - self.image_depth.material_data_name(source_object), - "img.001", - ) def test_surface_displacement_is_set_to_displacement_only(self): class Material: @@ -501,14 +490,14 @@ class ImageTest(unittest.TestCase): "DISPLACEMENT", ) - def test_plane_bounds_keep_image_aspect_and_empty_offset(self): + def test_image_empty_bounds_keep_image_aspect_and_empty_offset(self): source = SimpleNamespace( data=SimpleNamespace(size=(800, 400)), empty_display_size=2.0, empty_image_offset=(-0.5, -0.5), ) - bounds = self.image_depth.plane_bounds(source) + bounds = self.image_depth.image_empty_bounds(source) self.assertEqual(bounds, (-1.0, 1.0, -0.5, 0.5)) diff --git a/tests/test_mesh.py b/tests/test_mesh.py index 09e73d0..ac57d0a 100644 --- a/tests/test_mesh.py +++ b/tests/test_mesh.py @@ -1,24 +1,31 @@ import builtins import sys -import tempfile from pathlib import Path from types import SimpleNamespace from unittest.mock import patch import numpy as np -from PIL import Image SOURCE_ROOT = Path(__file__).parents[1] / "src" sys.path.insert(0, str(SOURCE_ROOT)) -from anyimage.operators.cutout_tool import balloon_profile as cutout_profile -from anyimage.operators.cutout_tool import build_shapes as mesh -from anyimage.operators.cutout_tool import depth_calibration as cutout_depth -from anyimage.operators.cutout_tool import node_assets as cutout_nodes -from anyimage.operators.cutout_tool import shape_types as shapes -from anyimage.operators.cutout_tool.balloon_profile import poisson_balloon_profile -from anyimage.operators.common.normal_map import rasterize_tangent_normal -from anyimage.operators.cutout_tool.mask_regions import alpha_regions, image_alpha_regions +from anyimage.operators.cutout_tool import region_to_shape as mesh +from anyimage.operators.cutout_tool import region_to_shape as cutout_nodes +from anyimage.operators.cutout_tool import region_to_shape as shapes +from anyimage.operators.cutout_tool.geometry import balloon as cutout_profile +from anyimage.operators.cutout_tool.geometry import depth_calibration as cutout_depth +from anyimage.operators.cutout_tool.geometry.balloon import poisson_balloon_profile +from anyimage.operators.common.mesh_result import ImageGeometry, load_image_geometry +from anyimage.operators.common.image_region import ( + RegionMask, + RegionPath, + rasterize_region_path, +) +from anyimage.operators.convert_to_plane.normal_map import rasterize_tangent_normal +from anyimage.operators.cutout_tool.geometry.region_contours import ( + RegionContours, + extract_region_contours, +) from anyimage.server.models import moge2 as moge2_model @@ -27,6 +34,29 @@ def test_cutout_mesh_detail_uses_the_cropped_analysis_scale(): assert mesh.limited_cutout_mesh_detail(256, (1024, 512), 2048) == 256 assert mesh.limited_cutout_mesh_detail(8, (8192, 4096), 2048) == 4 + +def test_image_geometry_loader_uses_one_schema_for_plane_and_cutout(tmp_path): + path = tmp_path / "image_geometry.npz" + points = np.ones((2, 3, 3), dtype=np.float32) + valid = np.ones((2, 3), dtype=bool) + np.savez_compressed( + path, + points=points, + intrinsics=np.eye(3, dtype=np.float32), + valid=valid, + calibration_valid=valid, + image_size=np.asarray((3, 2), dtype=np.int32), + reference_depth=np.asarray(1.0, dtype=np.float32), + region_transparent_fraction=np.asarray(0.25, dtype=np.float32), + ) + + field = load_image_geometry(path) + + assert field.points.shape == (2, 3, 3) + assert field.image_size == (3, 2) + assert field.reference_depth == 1.0 + assert field.region_transparent_fraction == 0.25 + def test_cutout_node_groups_are_loaded_from_the_asset(): loaded = object() with patch.object( @@ -171,16 +201,19 @@ def test_depth_surface_discontinuity_map_uses_local_three_by_three_range(): def test_depth_surface_projection_uses_one_scale_for_all_camera_axes(): points = np.zeros((51, 101, 3), dtype=np.float32) points[..., 2] = np.linspace(1.0, 3.0, 51 * 101).reshape(51, 101) - field = ( - points, - np.asarray(((100.0, 0.0, 50.0), (0.0, 100.0, 25.0), (0.0, 0.0, 1.0))), - np.ones((51, 101), dtype=bool), - np.ones((51, 101), dtype=bool), - (101, 51), - 2.0, + field = ImageGeometry( + points=points, + intrinsics=np.asarray( + ((100.0, 0.0, 50.0), (0.0, 100.0, 25.0), (0.0, 0.0, 1.0)) + ), + valid=np.ones((51, 101), dtype=bool), + calibration_valid=np.ones((51, 101), dtype=bool), + image_size=(101, 51), + reference_depth=2.0, + region_transparent_fraction=0.0, ) - uniform_scale = mesh._surface_uniform_scale( + uniform_scale = mesh._depth_uniform_scale( SimpleNamespace(), field, bounds=(-2.0, 2.0, -1.0, 1.0), @@ -189,7 +222,7 @@ def test_depth_surface_projection_uses_one_scale_for_all_camera_axes(): assert uniform_scale == 2.0 reference_depth = cutout_depth.metric_depth_reference( points[..., 2], - field[3], + field.calibration_valid, uniform_scale, ) assert np.isclose(reference_depth, 5.8, atol=1e-6) @@ -212,20 +245,24 @@ def test_moge_surface_field_uses_the_model_validity_mask(): assert np.array_equal(frame.valid_mask, prediction.mask[0]) -def test_cutout_regions_group_holes_with_their_connected_component(): - with tempfile.TemporaryDirectory() as directory: - path = Path(directory) / "cutout.png" - pixels = np.zeros((120, 200, 4), dtype=np.uint8) - pixels[20:100, 50:160] = (20, 40, 60, 255) - pixels[45:75, 80:120] = 0 - pixels[5:15, 5:15] = (20, 40, 60, 255) - Image.fromarray(pixels, mode="RGBA").save(path) +def test_region_contours_group_holes_with_their_connected_component(): + alpha = np.zeros((120, 200), dtype=np.uint8) + alpha[20:100, 50:160] = 255 + alpha[45:75, 80:120] = 0 + alpha[5:15, 5:15] = 255 - regions, size = image_alpha_regions(path, max_resolution=200) + region_contours = extract_region_contours( + RegionMask(alpha, (0, 0, 200, 120)), + max_resolution=200, + ) - assert size == (200, 120) - assert sorted(len(region) for region in regions) == [1, 2] - contours = [contour for region in regions for contour in region] + assert region_contours.image_size == (200, 120) + assert sorted(len(component) for component in region_contours.components) == [1, 2] + contours = [ + contour + for component in region_contours.components + for contour in component + ] bounds = [ (*np.min(contour, axis=0), *np.max(contour, axis=0)) for contour in contours @@ -235,16 +272,19 @@ def test_cutout_regions_group_holes_with_their_connected_component(): assert (5.0, 5.0, 15.0, 15.0) in bounds -def test_alpha_regions_accepts_blender_float_alpha_with_a_hole(): +def test_region_contours_accept_blender_float_mask_with_a_hole(): alpha = np.zeros((120, 200), dtype=np.float32) alpha[20:100, 50:160] = 1.0 alpha[45:75, 80:120] = 0.0 - regions, size = alpha_regions(alpha, max_resolution=200) + region_contours = extract_region_contours( + RegionMask(alpha, (0, 0, 200, 120)), + max_resolution=200, + ) - assert size == (200, 120) - assert len(regions) == 1 - assert len(regions[0]) == 2 + assert region_contours.image_size == (200, 120) + assert len(region_contours.components) == 1 + assert len(region_contours.components[0]) == 2 def test_cutout_crop_bounds_map_to_the_source_plane(): @@ -268,10 +308,14 @@ def test_depth_surface_cutout_uses_half_alpha_boundary(): alpha[2:10, 2:14] = 0.49 alpha[4:8, 5:11] = 0.51 - regions, size = mesh._cutout_regions(alpha, 64, threshold=0.5) - contour = np.asarray(regions[0][0]) + region_contours = mesh._extract_cutout_contours( + RegionMask(alpha, (0, 0, 16, 12)), + 64, + threshold=0.5, + ) + contour = np.asarray(region_contours.components[0][0]) - assert size == (16, 12) + assert region_contours.image_size == (16, 12) np.testing.assert_allclose(contour.min(axis=0), (5.0, 4.0)) np.testing.assert_allclose(contour.max(axis=0), (11.0, 8.0)) @@ -283,91 +327,31 @@ def test_blender_cutout_alpha_pipeline_does_not_import_pillow(): raise ModuleNotFoundError("Pillow is unavailable in Blender") return original_import(name, *args, **kwargs) - selection = np.zeros((32, 48), dtype=bool) - selection[4:28, 6:42] = True with patch("builtins.__import__", side_effect=without_pillow): - antialiased = mesh._antialias_selection(selection) - regions, size = alpha_regions(antialiased, max_resolution=24) + region_mask = rasterize_region_path( + (48, 32), + RegionPath(points=((6, 4), (42, 4), (42, 28), (6, 28))), + antialias=True, + ) + antialiased = region_mask.full_values((48, 32)) + region_contours = extract_region_contours( + RegionMask(antialiased, (0, 0, 48, 32)), + max_resolution=24, + ) - assert size == (48, 32) - assert regions + assert region_contours.image_size == (48, 32) + assert region_contours.components assert np.any((antialiased > 0.0) & (antialiased < 1.0)) -def test_unrefined_cutout_geometry_uses_strict_tenth_source_alpha_threshold(): - alpha = np.full((40, 60), 0.1, dtype=np.float32) - alpha[10:30, 20:40] = 0.11 - polygon = ((0, 0), (60, 0), (60, 40), (0, 40)) - - regions, size = mesh._geometry_cutout_regions(alpha, polygon, 64) - contour = np.asarray(regions[0][0]) - - assert size == (60, 40) - np.testing.assert_allclose(contour.min(axis=0), (20.0, 10.0)) - np.testing.assert_allclose(contour.max(axis=0), (40.0, 30.0)) - -def test_cutout_geometry_excludes_zero_source_alpha_without_refine(): - alpha = np.zeros((80, 120), dtype=np.float32) - alpha[10:70, 10:110] = 1.0 - alpha[30:50, 45:75] = 0.0 - polygon = ((10, 10), (110, 10), (110, 70), (10, 70)) - - regions, size = mesh._geometry_cutout_regions(alpha, polygon, 64) - - assert size == (120, 80) - assert len(regions) == 1 - assert len(regions[0]) == 2 - - -def test_cutout_geometry_uses_final_alpha_with_refine(): - alpha = np.zeros((80, 120), dtype=np.float32) - alpha[10:70, 10:110] = 1.0 - alpha[30:50, 45:75] = 0.0 - polygon = ((10, 10), (110, 10), (110, 70), (10, 70)) - - regions, size = mesh._geometry_cutout_regions( - alpha, polygon, 64, refine_selection=True - ) - - assert size == (120, 80) - assert len(regions) == 1 - assert len(regions[0]) == 2 - -def test_unrefined_cutout_geometry_discards_out_of_frame_selection(): - alpha = np.zeros((60, 100), dtype=np.float32) - alpha[10:50] = 1.0 - polygon = ((-20, 10), (120, 10), (120, 50), (-20, 50)) - - regions, size = mesh._geometry_cutout_regions(alpha, polygon, 64) - - points = np.concatenate( - [np.asarray(contour) for region in regions for contour in region] - ) - assert size == (100, 60) - assert points[:, 0].min() >= 0.0 - assert points[:, 0].max() <= 100.0 - -def test_refined_cutout_geometry_discards_out_of_frame_selection(): - alpha = np.zeros((60, 100), dtype=np.float32) - alpha[10:50] = 1.0 - polygon = ((-20, 10), (120, 10), (120, 50), (-20, 50)) - - regions, size = mesh._geometry_cutout_regions( - alpha, polygon, 64, refine_selection=True - ) - - points = np.concatenate( - [np.asarray(contour) for region in regions for contour in region] - ) - assert size == (100, 60) - assert points[:, 0].min() >= 0.0 - assert points[:, 0].max() <= 100.0 - def test_blender_cutout_contour_resolution_tracks_mesh_detail(): - expected = ([(((0, 0), (1, 0), (0, 1)),)], (1, 1)) - alpha = np.ones((2, 2), dtype=np.float32) - with patch.object(mesh, "alpha_regions", return_value=expected) as trace: - assert mesh._cutout_regions(alpha, 32) == expected - assert mesh._cutout_regions(alpha, 256) == expected + expected = RegionContours( + components=(((((0, 0), (1, 0), (0, 1))),),), + image_size=(1, 1), + ) + region_mask = RegionMask(np.ones((2, 2), dtype=np.float32), (0, 0, 2, 2)) + with patch.object(mesh, "extract_region_contours", return_value=expected) as trace: + assert mesh._extract_cutout_contours(region_mask, 32) == expected + assert mesh._extract_cutout_contours(region_mask, 256) == expected assert trace.call_args_list[0].kwargs["max_resolution"] == 512 assert trace.call_args_list[1].kwargs["max_resolution"] == 1024 @@ -375,7 +359,7 @@ def test_blender_cutout_contour_resolution_tracks_mesh_detail(): def test_cutout_point_spacing_uses_the_complete_source_plane(): source = SimpleNamespace(matrix_world=object()) with ( - patch.object(mesh, "plane_bounds", return_value=(-2.0, 2.0, -1.0, 1.0)), + patch.object(mesh, "image_empty_bounds", return_value=(-2.0, 2.0, -1.0, 1.0)), patch.object( mesh, "_world_plane_metric", @@ -387,23 +371,28 @@ def test_cutout_point_spacing_uses_the_complete_source_plane(): assert spacing == 0.02 -def test_cutout_cleanup_area_uses_point_spacing_and_plane_area_scale(): +def test_cutout_cleanup_value_scales_the_ui_without_changing_node_area(): assert np.isclose( - mesh._cutout_cleanup_area( + mesh._cutout_cleanup_value( 0.25, ((1.0, 0.0), (0.0, 1.0)), ), - 0.0625, + 6.25, ) assert np.isclose( - mesh._cutout_cleanup_area( + mesh._cutout_cleanup_value( 0.25, ((4.0, 0.0), (0.0, 9.0)), ), - 0.0625 / 6.0, + 6.25 / 6.0, ) +def test_cutout_depth_calibration_uses_five_percent_transparency_threshold(): + assert not cutout_depth.use_adaptive_depth_calibration(0.0499) + assert cutout_depth.use_adaptive_depth_calibration(0.05) + + def test_poisson_balloon_profile_is_zero_on_boundary_and_positive_inside(): points = np.asarray( ((0, 0), (2, 0), (2, 2), (0, 2), (1, 1)), @@ -482,8 +471,7 @@ def test_cutout_domain_adapts_boundary_spacing_for_a_small_valid_region(): ) domain = mesh._cutout_domain( - regions, - (4096, 4096), + RegionContours(regions, (4096, 4096)), (-1.0, 1.0, -1.0, 1.0), 0.5, ((1.0, 0.0), (0.0, 1.0)), @@ -497,8 +485,7 @@ def test_cutout_domain_builds_a_full_image_at_minimum_detail(): regions = ((((0, 0), (4096, 0), (4096, 4096), (0, 4096)),),) domain = mesh._cutout_domain( - regions, - (4096, 4096), + RegionContours(regions, (4096, 4096)), (-1.0, 1.0, -1.0, 1.0), 0.5, ((1.0, 0.0), (0.0, 1.0)), @@ -511,8 +498,7 @@ def test_cutout_domain_centers_vertices_on_crop_bounds(): regions = ((((0, 0), (100, 0), (100, 50), (0, 50)),),) vertices, _faces, uv, _boundary, _planar, _profile = mesh._cutout_domain( - regions, - (100, 50), + RegionContours(regions, (100, 50)), (2.0, 6.0, -3.0, 1.0), 0.5, ((1.0, 0.0), (0.0, 1.0)), @@ -535,8 +521,7 @@ def test_cutout_domain_keeps_overlapping_regions_disconnected(): ), ) domain = mesh._cutout_domain( - regions, - (120, 80), + RegionContours(regions, (120, 80)), (0.0, 1.0, 0.0, 1.0), 0.025, ((1.0, 0.0), (0.0, 1.0)), diff --git a/tests/test_mesh_blender.py b/tests/test_mesh_blender.py index aadd807..014a4be 100644 --- a/tests/test_mesh_blender.py +++ b/tests/test_mesh_blender.py @@ -10,13 +10,22 @@ from mathutils import Matrix, Vector SOURCE_ROOT = Path(__file__).parents[1] / "src" sys.path.insert(0, str(SOURCE_ROOT)) -from anyimage.operators.common import material, node_graph -from anyimage.operators.cutout_tool import build_shapes as mesh -from anyimage.operators.cutout_tool import node_assets as cutout_nodes -from anyimage.operators.cutout_tool import shape_types as shapes +from anyimage.operators.common import image_data as common_image +from anyimage.operators.common import mesh_result as material +from anyimage.operators.common.mesh_result import ImageGeometry +from anyimage.operators.common.image_region import ( + RegionMask, + RegionPath, + rasterize_region_path, +) +from anyimage.operators.cutout_tool import region_to_shape as mesh +from anyimage.operators.cutout_tool import region_to_shape as cutout_nodes +from anyimage.operators.cutout_tool import region_to_shape as shapes from anyimage.operators.convert_to_plane import nodes as plane_nodes from anyimage.operators.convert_to_plane import build as plane_build -from anyimage.operators.common.normal_map import bake_tangent_normal_image +from anyimage.operators.convert_to_plane.normal_map import ( + bake_tangent_normal_image, +) def _inputs(group): @@ -35,6 +44,31 @@ def _group_node(group, node_tree): ) +def _create_cutout_shape(source, shape, detail, *, bounds=None, field=None): + if bounds is None: + bounds = (0, 0, *tuple(int(value) for value in source.data.size)) + left, top, right, bottom = bounds + region_mask = RegionMask( + np.ones((bottom - top, right - left), dtype=np.float32), + bounds, + ) + base_shape = mesh.build_base_shape( + bpy.context, + source, + detail, + region_mask, + ) + return mesh.create_shape_object( + bpy.context, + source, + shape, + detail, + region_mask, + base_shape, + field=field, + ) + + def test_depth_surface_image_packs_metric_points_and_edge_validity(): bpy.ops.wm.read_factory_settings(use_empty=True) source_image = bpy.data.images.new("Source.png", width=2, height=2) @@ -44,7 +78,7 @@ def test_depth_surface_image_packs_metric_points_and_edge_validity(): depth = np.asarray(((1.0, 1.0), (1.0, 2.0)), dtype=np.float32) valid = np.asarray(((True, False), (True, True))) - image = mesh._surface_depth_image(source, points, valid, depth) + image = mesh._cutout_depth_image(source, points, valid, depth) packed = np.empty(16, dtype=np.float32) image.pixels.foreach_get(packed) @@ -92,7 +126,7 @@ def test_cutout_modifier_wraps_four_menu_controlled_shape_groups(): plane = plane_nodes.plane_node_group("PLANE") depth_plane = plane_nodes.plane_node_group("DEPTH_PLANE") - cutout = cutout_nodes.cutout_node_group("CUTOUT") + cutout = cutout_nodes.cutout_node_group("SURFACE") assert plane.name == "O Mesh Plane" assert cutout.name == shapes.CUTOUT_NODE_GROUP_NAME assert all( @@ -117,7 +151,7 @@ def test_cutout_modifier_wraps_four_menu_controlled_shape_groups(): "Smooth Iterations", "Cleanup", "Smooth", - "Reference Depth", + "Base Plane Depth", "Depth Direction", "Uniform Scale", "Depth Image", @@ -129,7 +163,7 @@ def test_cutout_modifier_wraps_four_menu_controlled_shape_groups(): assert [item.name for item in panels[0].interface_items] == [ "Cleanup", "Smooth", - "Reference Depth", + "Base Plane Depth", "Depth Direction", ] assert [item.name for item in panels[1].interface_items] == [ @@ -148,9 +182,11 @@ def test_cutout_modifier_wraps_four_menu_controlled_shape_groups(): menu_switch = next( node for node in cutout.nodes if node.bl_idname == "GeometryNodeMenuSwitch" ) - assert [item.name for item in menu_switch.enum_items] == list( - shapes.CUTOUT_SHAPE_LABELS.values() - ) + labels = { + shape: shape.replace("_", " ").title() + for shape in shapes.CUTOUT_NODE_GROUP_NAMES + } + assert [item.name for item in menu_switch.enum_items] == list(labels.values()) assert any( link.from_socket.name == "Shape" and link.to_node == menu_switch @@ -171,7 +207,7 @@ def test_cutout_modifier_wraps_four_menu_controlled_shape_groups(): } assert set(child_nodes) == set(child_groups.values()) for shape, group in child_groups.items(): - label = shapes.CUTOUT_SHAPE_LABELS[shape] + label = labels[shape] assert any( link.from_node == child_nodes[group] and link.to_node == menu_switch @@ -179,14 +215,14 @@ def test_cutout_modifier_wraps_four_menu_controlled_shape_groups(): for link in cutout.links ) - assert _inputs(child_groups["CUTOUT"]) == ["Geometry", "Thickness"] + assert _inputs(child_groups["SURFACE"]) == ["Geometry", "Thickness"] assert _inputs(child_groups["BALLOON"]) == ["Geometry", "Thickness"] assert _inputs(child_groups["DEPTH_BALLOON"]) == [ "Geometry", "Thickness", "Depth Scale", "Double Sided", - "Reference Depth", + "Base Plane Depth", "Depth Direction", "Uniform Scale", "Depth Image", @@ -195,7 +231,7 @@ def test_cutout_modifier_wraps_four_menu_controlled_shape_groups(): "Geometry", "Thickness", "Depth Scale", - "Reference Depth", + "Base Plane Depth", "Uniform Scale", "Depth Image", ] @@ -216,7 +252,7 @@ def test_cutout_modifier_wraps_four_menu_controlled_shape_groups(): "Thickness": set(shapes.CUTOUT_SHAPES), "Depth Scale": {"DEPTH_BALLOON", "DEPTH_SURFACE"}, "Double Sided": {"DEPTH_BALLOON"}, - "Reference Depth": {"DEPTH_BALLOON", "DEPTH_SURFACE"}, + "Base Plane Depth": {"DEPTH_BALLOON", "DEPTH_SURFACE"}, "Depth Direction": {"DEPTH_BALLOON"}, "Uniform Scale": {"DEPTH_BALLOON", "DEPTH_SURFACE"}, "Depth Image": {"DEPTH_BALLOON", "DEPTH_SURFACE"}, @@ -252,7 +288,7 @@ def test_cutout_modifier_wraps_four_menu_controlled_shape_groups(): for shape, group in child_groups.items() } assert repeat_counts == { - "CUTOUT": 1, + "SURFACE": 1, "BALLOON": 0, "DEPTH_BALLOON": 0, "DEPTH_SURFACE": 1, @@ -278,6 +314,15 @@ def test_cutout_modifier_wraps_four_menu_controlled_shape_groups(): ) assert cleanup.default_value == 0.0 assert cleanup.subtype == "NONE" + cleanup_scale = next( + link.to_node + for link in cutout.links + if link.from_socket.name == "Cleanup" + and link.from_node.bl_idname == "NodeGroupInput" + ) + assert cleanup_scale.bl_idname == "ShaderNodeMath" + assert cleanup_scale.operation == "MULTIPLY" + assert np.isclose(cleanup_scale.inputs[1].default_value, 0.01) assert smooth.default_value == 1.0 assert smooth.subtype == "FACTOR" assert iterations.default_value == 0 @@ -304,7 +349,7 @@ def test_cutout_modifier_wraps_four_menu_controlled_shape_groups(): for node in extrudes ) for group in ( - child_groups["CUTOUT"], + child_groups["SURFACE"], child_groups["DEPTH_SURFACE"], depth_plane, ): @@ -377,8 +422,8 @@ def test_image_layer_switches_between_tangent_and_object_space_normal(): color = bpy.data.images.new("Color", width=2, height=2) normal_image = bpy.data.images.new("Normal", width=2, height=2) - object_material = material.create_material( - "Object Normal", + object_material = material.create_image_material( + SimpleNamespace(data=SimpleNamespace(name="Object Normal")), color, 1.0, normal_image=normal_image, @@ -387,8 +432,8 @@ def test_image_layer_switches_between_tangent_and_object_space_normal(): layer = _group_node(group, object_material.node_tree) assert layer.inputs["Object Space"].default_value is True - tangent_material = material.create_material( - "Tangent Normal", + tangent_material = material.create_image_material( + SimpleNamespace(data=SimpleNamespace(name="Tangent Normal")), color, 1.0, normal_image=normal_image, @@ -442,17 +487,10 @@ def test_balloon_build_starts_from_the_unified_single_sheet(): source.data = image source.empty_display_size = 2.0 bpy.context.collection.objects.link(source) - balloon_material = bpy.data.materials.new("Balloon Sheet.material") - - result = mesh.build_cutout_object( - bpy.context, + result = _create_cutout_shape( source, "BALLOON", 8, - balloon_material, - regions=((((0, 0), (64, 0), (64, 64), (0, 64)),),), - image_size=(64, 64), - keep_original=True, ) assert all(abs(vertex.co.x) < 1e-8 for vertex in result.data.vertices) @@ -478,22 +516,15 @@ def test_teddy_preference_writes_the_shared_balloon_attribute(): source.data = image source.empty_display_size = 2.0 bpy.context.collection.objects.link(source) - teddy_material = bpy.data.materials.new("Teddy Sheet.material") - with patch.object( mesh, "addon_preferences", return_value=SimpleNamespace(balloon_profile_algorithm="TEDDY"), ): - result = mesh.build_cutout_object( - bpy.context, + result = _create_cutout_shape( source, - "CUTOUT", + "SURFACE", 8, - teddy_material, - regions=((((0, 0), (64, 0), (64, 64), (0, 64)),),), - image_size=(64, 64), - keep_original=True, ) attribute = result.data.attributes[shapes.BALLOON_ATTRIBUTE_NAME] @@ -502,22 +533,24 @@ def test_teddy_preference_writes_the_shared_balloon_attribute(): assert profile.min() == 0.0 and profile.max() > 0.0 -def test_depth_surface_object_normal_is_remapped_to_cutout_axes(): +def test_cutout_material_uses_flat_tangent_normal(): bpy.ops.wm.read_factory_settings(use_empty=True) - image = bpy.data.images.new( - "Surface Normal", - width=1, - height=1, - alpha=True, - float_buffer=True, + source = bpy.data.objects.new("Source", None) + source.empty_display_type = "IMAGE" + source.data = bpy.data.images.new("Source", width=2, height=2) + color = bpy.data.images.new("Color", width=2, height=2) + normal = bpy.data.images.new("Normal", width=1, height=1) + + result = material.create_image_material( + source, + color, + 0.0, + normal_image=normal, + scene=bpy.context.scene, ) - image.pixels.foreach_set((0.5, 0.5, 1.0, 1.0)) - mesh._orient_cutout_object_normal(image) - - pixels = np.empty(4, dtype=np.float32) - image.pixels.foreach_get(pixels) - assert np.allclose(pixels, (0.0, 0.5, 0.5, 1.0), atol=1e-6) + layer = _group_node(material.material_node_group(), result.node_tree) + assert layer.inputs["Object Space"].default_value is False def test_internal_depth_plane_image_uses_standard_data_name(): @@ -540,8 +573,6 @@ def test_depth_surface_protected_smooth_evaluates(): source.data = image source.empty_display_size = 2.0 bpy.context.collection.objects.link(source) - surface_material = bpy.data.materials.new("Surface.material") - size = 8 depth = np.ones((size, size), dtype=np.float32) x, y = np.meshgrid(np.arange(size), np.arange(size)) @@ -553,25 +584,21 @@ def test_depth_surface_protected_smooth_evaluates(): points[..., 0] = (x - intrinsics[0, 2]) / intrinsics[0, 0] * depth points[..., 1] = (y - intrinsics[1, 2]) / intrinsics[1, 1] * depth points[..., 2] = depth - field = ( - points, - intrinsics, - np.ones((size, size), dtype=bool), - np.ones((size, size), dtype=bool), - (size, size), - 1.0, + field = ImageGeometry( + points=points, + intrinsics=intrinsics, + valid=np.ones((size, size), dtype=bool), + calibration_valid=np.ones((size, size), dtype=bool), + image_size=(size, size), + reference_depth=1.0, + region_transparent_fraction=0.0, ) - result = mesh.build_cutout_object( - bpy.context, + result = _create_cutout_shape( source, "DEPTH_SURFACE", 8, - surface_material, field=field, - regions=((((0, 0), (64, 0), (64, 64), (0, 64)),),), - image_size=(64, 64), - keep_original=True, ) assert all(abs(vertex.co.x) < 1e-8 for vertex in result.data.vertices) @@ -581,7 +608,7 @@ def test_depth_surface_protected_smooth_evaluates(): assert modifier[ mesh.modifier_input_identifier(modifier.node_group, "Shape") ] == shapes.CUTOUT_SHAPE_INDICES["DEPTH_SURFACE"] - expected_cleanup = mesh._cutout_cleanup_area( + expected_cleanup = mesh._cutout_cleanup_value( mesh._cutout_point_spacing(source, 8), mesh._world_plane_metric(source.matrix_world)[0], ) @@ -599,7 +626,7 @@ def test_depth_surface_protected_smooth_evaluates(): ] == 0 reference_id = mesh.modifier_input_identifier( modifier.node_group, - "Reference Depth", + "Base Plane Depth", ) assert modifier[reference_id] == 8.0 mesh.set_modifier_input( @@ -656,52 +683,9 @@ def test_depth_surface_protected_smooth_evaluates(): finally: evaluated.to_mesh_clear() - mesh.set_modifier_input( - modifier, - mesh.modifier_input_identifier( - modifier.node_group, "Smooth Iterations" - ), - 0, - ) - external_group = bpy.data.node_groups.new("External O Smooth", "GeometryNodeTree") - external_group.is_modifier = True - node_graph.interface_socket( - external_group, "Geometry", "OUTPUT", "NodeSocketGeometry" - ) - node_graph.interface_socket( - external_group, "Geometry", "INPUT", "NodeSocketGeometry" - ) - external_iterations = node_graph.interface_socket( - external_group, "Iterations", "INPUT", "NodeSocketInt" - ) - external_iterations.default_value = 4 - external_iterations.min_value = 0 - external_smooth = node_graph.interface_socket( - external_group, "Smooth", "INPUT", "NodeSocketFloat" - ) - external_smooth.default_value = 1.0 - external_input = external_group.nodes.new("NodeGroupInput") - external_output = external_group.nodes.new("NodeGroupOutput") - external_geometry = node_graph.expand_smooth( - external_group, external_input.outputs["Geometry"] - ) - external_group.links.new( - external_geometry, external_output.inputs["Geometry"] - ) - external_modifier = result.modifiers.new("External O Smooth", "NODES") - external_modifier.node_group = external_group - result.update_tag(refresh={"DATA"}) - bpy.context.view_layer.update() - evaluated = result.evaluated_get(bpy.context.evaluated_depsgraph_get()) - external_mesh = evaluated.to_mesh() - try: - external_positions = np.asarray( - [tuple(vertex.co) for vertex in external_mesh.vertices] - ) - finally: - evaluated.to_mesh_clear() - assert internal_positions.shape == external_positions.shape - assert np.allclose(internal_positions, external_positions, atol=1e-6) + assert internal_positions.ndim == 2 + assert internal_positions.shape[1] == 3 + assert np.isfinite(internal_positions).all() @@ -756,7 +740,7 @@ def test_cutout_thickness_keeps_one_connected_shell(): data.from_pydata(vertices, (), faces) obj = bpy.data.objects.new("Connected Cutout", data) bpy.context.collection.objects.link(obj) - modifier = _add_cutout_modifier(obj, "CUTOUT") + modifier = _add_cutout_modifier(obj, "SURFACE") mesh.set_modifier_input( modifier, @@ -923,7 +907,7 @@ def test_cutout_cleanup_uses_absolute_area_with_many_small_islands(): data.from_pydata(vertices, (), faces) obj = bpy.data.objects.new("Cleanup", data) bpy.context.collection.objects.link(obj) - modifier = _add_cutout_modifier(obj, "CUTOUT") + modifier = _add_cutout_modifier(obj, "SURFACE") cleanup_id = mesh.modifier_input_identifier(modifier.node_group, "Cleanup") @@ -931,7 +915,7 @@ def test_cutout_cleanup_uses_absolute_area_with_many_small_islands(): bpy.context.view_layer.update() assert len(_evaluated_positions(obj)) == 45 - mesh.set_modifier_input(modifier, cleanup_id, 0.001) + mesh.set_modifier_input(modifier, cleanup_id, 0.1) obj.update_tag(refresh={"DATA"}) bpy.context.view_layer.update() assert len(_evaluated_positions(obj)) == 9 @@ -945,7 +929,6 @@ def test_depth_balloon_uses_metric_plane_inputs_without_controller(): source.data = image source.empty_display_size = 2.0 bpy.context.collection.objects.link(source) - balloon_material = bpy.data.materials.new("Depth Balloon.material") crop_bounds = (50, 20, 90, 80) crop = ((0, 0), (40, 0), (40, 60), (0, 60)) local_bounds = mesh._crop_plane_bounds( @@ -965,26 +948,22 @@ def test_depth_balloon_uses_metric_plane_inputs_without_controller(): ((50.0, 0.0, 20.0), (0.0, 50.0, 30.0), (0.0, 0.0, 1.0)), dtype=np.float32, ) - field = ( - points, - intrinsics, - np.ones((61, 41), dtype=bool), - np.ones((61, 41), dtype=bool), - (41, 61), - 2.0, + field = ImageGeometry( + points=points, + intrinsics=intrinsics, + valid=np.ones((61, 41), dtype=bool), + calibration_valid=np.ones((61, 41), dtype=bool), + image_size=(41, 61), + reference_depth=2.0, + region_transparent_fraction=0.05, ) - result = mesh.build_cutout_object( - bpy.context, + result = _create_cutout_shape( source, "DEPTH_BALLOON", 8, - balloon_material, field=field, - regions=((crop,),), - image_size=(40, 60), - bounds=local_bounds, - keep_original=True, + bounds=crop_bounds, ) assert all(abs(vertex.co.x) < 1e-8 for vertex in result.data.vertices) @@ -1019,15 +998,15 @@ def test_depth_balloon_uses_metric_plane_inputs_without_controller(): depth_reference_id = mesh.modifier_input_identifier( modifier.node_group, - "Reference Depth", + "Base Plane Depth", ) - assert modifier[depth_reference_id] == 8.0 + assert not np.isclose(modifier[depth_reference_id], 8.0) depth_direction_id = mesh.modifier_input_identifier( modifier.node_group, "Depth Direction", ) direction = np.asarray(modifier[depth_direction_id], dtype=np.float64) - assert np.array_equal(direction, (0.0, 0.0, 1.0)) + assert not np.allclose(direction, (0.0, 0.0, 1.0)) uniform_scale_id = mesh.modifier_input_identifier( modifier.node_group, "Uniform Scale", @@ -1130,7 +1109,7 @@ def test_loaded_normal_image_uses_normal_texture_name(tmp_path): Image.new("RGB", (1, 1), (128, 128, 255)).save(path) source = SimpleNamespace(data=SimpleNamespace(name="Poster.png")) - image = mesh._load_normal_image(path, source) + image = common_image.load_normal_result_image(path, source) assert image.name == "Poster_normal.png" assert image.colorspace_settings.name == "Non-Color" @@ -1155,20 +1134,19 @@ def test_converted_color_images_are_new_and_use_blender_duplicate_suffix(): def test_local_cutout_selection_matches_uint8_tenth_alpha_boundary(): - bpy.ops.wm.read_factory_settings(use_empty=True) - image = bpy.data.images.new("Alpha Cutout", width=4, height=4, alpha=True) top_down = np.ones((4, 4, 4), dtype=np.float32) top_down[1, 1, 3] = 25.0 / 255.0 top_down[1, 2, 3] = 26.0 / 255.0 - image.pixels.foreach_set(np.flipud(top_down).ravel()) - source = bpy.data.objects.new("Alpha Cutout", None) - source.empty_display_type = "IMAGE" - source.data = image - - _rgba, selection = mesh._local_cutout_rgba( - source, - ((0, 0), (4, 0), (4, 4), (0, 4)), + region_mask = rasterize_region_path( + (4, 4), + RegionPath( + points=((0, 0), (4, 0), (4, 4), (0, 4)), + fill_rule="NONZERO", + ), + source_alpha=top_down[:, :, 3], + alpha_threshold=0.1, ) + selection = region_mask.full_values((4, 4)) assert selection[1, 1] == 0.0 assert selection[1, 2] > 0.1 @@ -1184,28 +1162,47 @@ def test_local_cutout_builds_from_blender_image_pixels_without_a_job(): bpy.context.collection.objects.link(source) source.select_set(True) bpy.context.view_layer.objects.active = source - selection = "[[-8,8],[72,8],[72,40],[-8,40]]" - source_bounds = mesh.plane_bounds(source) + region_path = RegionPath( + points=((-8, 8), (72, 8), (72, 40), (-8, 40)), + fill_rule="NONZERO", + ) + source_bounds = mesh.image_empty_bounds(source) - with patch.object(mesh, "push_undo") as undo: - mesh.build_local_cutout( - SimpleNamespace(mesh_detail=16), - bpy.context, - source, - "CUTOUT", - selection, + with patch.object( + mesh, + "image_rgba", + wraps=mesh.image_rgba, + ) as read_rgba: + rgba = mesh.image_rgba(image) + region_mask = rasterize_region_path( + tuple(image.size), + region_path, + source_alpha=rgba[:, :, 3], + alpha_threshold=mesh.CUTOUT_ALPHA_THRESHOLD, + antialias=True, ) + with patch.object(mesh, "configured_max_ai_input_size", return_value=2048): + mesh.region_to_shape( + bpy.context, + source, + "SURFACE", + 16, + region_mask, + source_rgba=rgba, + ) + + read_rgba.assert_called_once_with(image) result = bpy.context.view_layer.objects.active assert result is not source assert result.type == "MESH" - assert result["anyimage_mesh_shape"] == "CUTOUT" + assert result["anyimage_mesh_shape"] == "SURFACE" modifier = result.modifiers[0] assert modifier.node_group.name == shapes.CUTOUT_NODE_GROUP_NAME assert modifier[ mesh.modifier_input_identifier(modifier.node_group, "Shape") - ] == shapes.CUTOUT_SHAPE_INDICES["CUTOUT"] - expected_cleanup = mesh._cutout_cleanup_area( + ] == shapes.CUTOUT_SHAPE_INDICES["SURFACE"] + expected_cleanup = mesh._cutout_cleanup_value( mesh._cutout_point_spacing(source, 16), mesh._world_plane_metric(source.matrix_world)[0], ) @@ -1246,4 +1243,3 @@ def test_local_cutout_builds_from_blender_image_pixels_without_a_job(): color_texture.image.pixels.foreach_get(copied) copied = copied.reshape((34, 64, 4)) assert np.allclose(copied, 1.0) - undo.assert_called_once_with("Convert to Cutout") diff --git a/tests/test_pack.py b/tests/test_pack.py index e81e768..be9aaab 100644 --- a/tests/test_pack.py +++ b/tests/test_pack.py @@ -67,26 +67,21 @@ class PackTest(unittest.TestCase): self.assertNotIn("assets/O_AnyImage.blend1", names) for path in ( "operators/common/__init__.py", - "operators/common/image.py", - "operators/common/material.py", - "operators/common/node_asset.py", - "operators/common/normal_map.py", - "operators/common/operator.py", - "operators/common/image_edit.py", - "operators/common/image_gesture.py", + "operators/common/ai_job.py", + "operators/common/image_data.py", "operators/common/image_interaction.py", - "operators/common/image_mask.py", - "operators/common/node_graph.py", + "operators/common/image_region.py", + "operators/common/mesh_result.py", + "operators/convert_to_plane/normal_map.py", + "operators/convert_to_plane/resolution.py", "operators/cutout_tool/__init__.py", - "operators/cutout_tool/balloon_profile.py", - "operators/cutout_tool/build_shapes.py", - "operators/cutout_tool/depth_calibration.py", - "operators/cutout_tool/mask_regions.py", - "operators/cutout_tool/node_assets.py", - "operators/cutout_tool/planar_mesh.py", - "operators/cutout_tool/selection_tool.py", - "operators/cutout_tool/shape_conversion.py", - "operators/cutout_tool/shape_types.py", + "operators/cutout_tool/select_region.py", + "operators/cutout_tool/region_to_shape.py", + "operators/cutout_tool/geometry/__init__.py", + "operators/cutout_tool/geometry/balloon.py", + "operators/cutout_tool/geometry/depth_calibration.py", + "operators/cutout_tool/geometry/region_contours.py", + "operators/cutout_tool/geometry/triangulation.py", "operators/convert_to_plane/__init__.py", "operators/convert_to_plane/build.py", "operators/convert_to_plane/conversion.py", @@ -118,10 +113,15 @@ class PackTest(unittest.TestCase): self.assertIn("server/models/geometry.py", names) self.assertIn("server/jobs/remove_background.py", names) self.assertIn("server/jobs/refine_image_mask.py", names) - self.assertIn("server/selection/cutout_images.py", names) - self.assertIn("server/selection/polygon_mask.py", names) - self.assertIn("server/jobs/prepare_geometry_data.py", names) + self.assertIn("server/region/generate_selection.py", names) + self.assertIn("server/jobs/generate_region_data.py", names) + self.assertIn("server/jobs/generate_depth_plane.py", names) + self.assertIn("server/geometry/generate_depth.py", names) self.assertIn("server/geometry/normal_texture.py", names) + self.assertNotIn("operators/common/image_selection.py", names) + self.assertNotIn("server/selection/cutout_images.py", names) + self.assertNotIn("server/selection/polygon_mask.py", names) + self.assertNotIn("server/jobs/prepare_geometry_data.py", names) self.assertNotIn("server/geometry/cutout.py", names) self.assertIn("server/models/ben2.py", names) self.assertIn("server/models/da3.py", names) diff --git a/tests/test_server.py b/tests/test_server.py index 4959d35..aaa3f73 100644 --- a/tests/test_server.py +++ b/tests/test_server.py @@ -41,8 +41,8 @@ class ServerTest(unittest.TestCase): @classmethod def setUpClass(cls): cls.server_module = load_server_module() - importlib.import_module("server.selection.polygon_mask") - importlib.import_module("server.jobs.prepare_geometry_data") + importlib.import_module("server.geometry.generate_depth") + importlib.import_module("server.jobs.generate_region_data") from blendjob import JobServer from server.model_manager import ModelManager, ModelSessionCache @@ -261,7 +261,8 @@ class ServerTest(unittest.TestCase): "refine-image-mask", "remove-background", "upscale-image", - "prepare-geometry-data", + "generate-depth-plane", + "generate-region-data", "debug-da3", "debug-moge2", "debug-ben2", @@ -385,652 +386,116 @@ class ServerTest(unittest.TestCase): ) self.assertEqual(events, ["clear", "close"]) - def test_polygon_selection_mask_uses_even_odd_fill_for_self_intersection(self): - image_mask = sys.modules["server.selection.polygon_mask"] - polygon = ( - (1, 1), - (9, 1), - (9, 9), - (1, 9), - (1, 1), - (3, 3), - (3, 7), - (7, 7), - (7, 3), - (3, 3), - (1, 1), - ) - - mask = image_mask.polygon_selection_mask((10, 10), polygon) - - self.assertTrue(mask[2, 2]) - self.assertFalse(mask[4, 4]) - self.assertEqual(int(mask.sum()), 48) - - def test_polygon_selection_mask_nonzero_keeps_same_direction_overlap(self): - image_mask = sys.modules["server.selection.polygon_mask"] - polygon = ( - (1, 1), - (7, 1), - (7, 7), - (1, 7), - (1, 1), - (4, 1), - (10, 1), - (10, 7), - (4, 7), - (4, 1), - (1, 1), - ) - - even_odd = image_mask.polygon_selection_mask((12, 9), polygon) - nonzero = image_mask.polygon_selection_mask( - (12, 9), polygon, "NONZERO" - ) - - self.assertTrue(even_odd[3, 2]) - self.assertFalse(even_odd[3, 5]) - self.assertTrue(nonzero[3, 2]) - self.assertTrue(nonzero[3, 5]) - self.assertTrue(nonzero[3, 8]) - def test_image_mask_canvas_clears_rgba_outside_the_selection(self): + def test_region_transparent_fraction_uses_visible_alpha_threshold(self): import numpy as np from PIL import Image - image_mask = sys.modules["server.selection.polygon_mask"] - source_path = self.storage_root / "source.png" - pixels = np.full((2, 2, 4), (20, 40, 60, 255), dtype=np.uint8) - Image.fromarray(pixels, mode="RGBA").save(source_path) + depth = sys.modules["server.geometry.generate_depth"] + output_path = self.storage_root / "region.png" + pixels = np.full((10, 10, 4), 255, dtype=np.uint8) + pixels.reshape(-1, 4)[:5, 3] = 25 + Image.fromarray(pixels, mode="RGBA").save(output_path) - canvas, _selection, bounds = image_mask.create_selection_canvas( - source_path, - ((-1, -1), (2, -1), (-1, 2)), - ) - result = np.asarray(canvas) + self.assertEqual(depth.transparent_fraction(output_path), 0.05) - self.assertEqual(bounds, (-1, -1, 2, 2)) - self.assertEqual(result.shape, (3, 3, 4)) - self.assertEqual(tuple(result[2, 2]), (0, 0, 0, 0)) - self.assertEqual(tuple(result[1, 1]), (20, 40, 60, 255)) - - def test_image_mask_crops_tightly_to_alpha(self): - import numpy as np - from PIL import Image - - image_mask = sys.modules["server.selection.polygon_mask"] - pixels = np.zeros((4, 5, 4), dtype=np.uint8) - pixels[1:3, 2:4] = (20, 40, 60, 255) - result, bounds = image_mask.crop_to_alpha( - Image.fromarray(pixels, mode="RGBA"), - (10, 20, 15, 24), - ) - result = np.asarray(result) - - self.assertEqual(bounds, (12, 21, 14, 23)) - self.assertEqual(result.shape, (2, 2, 4)) - self.assertTrue(np.all(result[:, :, 3] == 255)) - - def test_mesh_refine_selection_keeps_source_size_and_selection_alpha(self): - cutout_job = importlib.import_module("server.selection.cutout_images") - import numpy as np - from PIL import Image - - mesh_job = sys.modules["server.jobs.prepare_geometry_data"] - source_path = self.storage_root / "source.png" - pixels = np.full((6, 8, 4), (20, 40, 60, 255), dtype=np.uint8) - Image.fromarray(pixels, mode="RGBA").save(source_path) - job_directory = self.storage_root / "job" - job_directory.mkdir() - context = SimpleNamespace( - directory=job_directory, - resource=lambda _name: SimpleNamespace( - directory=lambda _model: Path("model") - ), - check_cancelled=lambda: None, - ) - - def passthrough(path, *_args, **_kwargs): - return Image.open(path).convert("RGBA"), 0.0, 0.0 - - output_path = job_directory / "image.0001.png" - with patch.object( - cutout_job, - "infer_foreground", - side_effect=passthrough, - ): - cutout_job.refine_selection( - context, - {"refine_model": "BEN2_BASE", "device": "cpu"}, - source_path, - ((1, 1), (6, 1), (1, 5)), - output_path, - ) - result = np.asarray(Image.open(output_path).convert("RGBA")) - - self.assertEqual(result.shape, (6, 8, 4)) - self.assertEqual(tuple(result[1, 1]), (20, 40, 60, 255)) - self.assertEqual(tuple(result[5, 7]), (0, 0, 0, 0)) - - def test_mesh_cutout_images_crop_color_and_preserve_source_alpha(self): - cutout_job = importlib.import_module("server.selection.cutout_images") - import json - import numpy as np - from PIL import Image - - mesh_job = sys.modules["server.jobs.prepare_geometry_data"] - source_path = self.storage_root / "analysis-source.png" - selection_path = self.storage_root / "selection-source.png" - output_directory = self.storage_root / "cutout-images" - output_directory.mkdir() - source = np.full((6, 8, 4), (10, 80, 160, 255), dtype=np.uint8) - source[2, 3, 3] = 25 - source[2, 4, 3] = 26 - source[3, 5, 3] = 0 - Image.fromarray(source, mode="RGBA").save(source_path) - selection = np.zeros((6, 8, 4), dtype=np.uint8) - selection[1:5, 2:7] = (200, 20, 40, 255) - selection[1, 2] = (200, 20, 40, 0) - Image.fromarray(selection, mode="RGBA").save(selection_path) - - color_path, mask_path, analysis_path, bounds_path = ( - cutout_job.write_images( - source_path, - selection_path, - output_directory, - ) - ) - color = np.asarray(Image.open(color_path).convert("RGBA")) - mask = np.asarray(Image.open(mask_path).convert("RGBA")) - analysis = np.asarray(Image.open(analysis_path).convert("RGBA")) - - self.assertEqual(json.loads(bounds_path.read_text()), [2, 1, 7, 5]) - self.assertEqual(color.shape, (4, 5, 4)) - self.assertEqual(tuple(color[1, 1]), (10, 80, 160, 25)) - self.assertEqual(int(mask[1, 1, 3]), 0) - self.assertEqual(tuple(analysis[1, 1]), (0, 0, 0, 0)) - self.assertEqual(tuple(color[1, 2]), (10, 80, 160, 26)) - self.assertGreater(int(mask[1, 2, 3]), 25) - self.assertEqual(int(mask[0, 0, 3]), 0) - self.assertEqual(tuple(color[0, 0]), (10, 80, 160, 255)) - self.assertEqual(tuple(analysis[0, 0]), (0, 0, 0, 0)) - self.assertEqual(tuple(analysis[1, 2]), (10, 80, 160, 26)) - self.assertEqual(tuple(color[2, 3]), (10, 80, 160, 0)) - self.assertEqual(int(mask[2, 3, 3]), 0) - self.assertEqual(tuple(analysis[2, 3]), (0, 0, 0, 0)) - - def test_mesh_unrefined_cutout_discards_selection_outside_the_source(self): - cutout_job = importlib.import_module("server.selection.cutout_images") - import json - import numpy as np - from PIL import Image - - mesh_job = sys.modules["server.jobs.prepare_geometry_data"] - source_path = self.storage_root / 'outside-source.png' - full_selection_path = self.storage_root / 'outside-selection-full.png' - output_directory = self.storage_root / 'outside-cutout' - output_directory.mkdir() - Image.new('RGBA', (8, 6), (10, 80, 160, 255)).save(source_path) - polygon = ((-2, 1), (10, 1), (10, 5), (-2, 5)) - - _selection_path, selection_bounds = ( - cutout_job.write_selection( - source_path, - polygon, - full_selection_path, - ) - ) - color_path, mask_path, analysis_path, bounds_path = ( - cutout_job.write_images( - source_path, - full_selection_path, - output_directory, - selection_bounds, - ) - ) - color = np.asarray(Image.open(color_path).convert('RGBA')) - mask = np.asarray(Image.open(mask_path).convert('RGBA')) - analysis = np.asarray(Image.open(analysis_path).convert('RGBA')) - - self.assertEqual(selection_bounds, (-2, 1, 10, 5)) - self.assertEqual(json.loads(bounds_path.read_text()), [0, 1, 8, 5]) - self.assertEqual(color.shape, (4, 8, 4)) - self.assertEqual(tuple(color[1, 0]), (10, 80, 160, 255)) - self.assertEqual(tuple(color[1, 7]), (10, 80, 160, 255)) - self.assertGreater(int(mask[1, 0, 3]), 127) - self.assertEqual(tuple(analysis[1, 0]), (10, 80, 160, 255)) - - def test_mesh_job_discards_unrefined_outside_selection_with_normal(self): - import json - import numpy as np - from PIL import Image - - mesh_job = sys.modules["server.jobs.prepare_geometry_data"] - source_path = self.storage_root / "mesh-source.png" - pixels = np.full((6, 8, 4), (20, 40, 60, 255), dtype=np.uint8) - Image.fromarray(pixels, mode="RGBA").save(source_path) - job_directory = self.storage_root / "mesh-job" - context = SimpleNamespace( - directory=job_directory, - progress=lambda *_args: None, - check_cancelled=lambda: None, - resource=lambda _name: object(), - ) - normal = np.zeros((1, 4, 12, 3), dtype=np.float32) - normal[..., 0] = 1.0 - prediction = SimpleNamespace(normal=normal) - received = [] - received_images = [] - - def infer(_manager, parameters, images, **_kwargs): - received.append(parameters) - received_images.extend(images) - return prediction - - with patch.object(mesh_job.moge2, "infer", side_effect=infer): - result = mesh_job.run( - context, - { - "input": str(source_path), - "shape": "BALLOON", - "polygon": ((-2, 1), (10, 1), (10, 5), (-2, 5)), - "mesh_detail": 64, - "invert": False, - "device": "cpu", - "model": "MOGE2_VITS_NORMAL", - "resolution_level": 5, - "normal_analysis": True, - }, - ) - - color_path = job_directory / "image.0001.png" - color_pixels = np.asarray(Image.open(color_path).convert("RGBA")) - normal_pixels = np.asarray( - Image.open(job_directory / result["normal"]).convert("RGB") - ) - self.assertEqual( - result, - { - "selection": "selection.png", - "bounds": "bounds.json", - "normal": "normal.png", - }, - ) - self.assertTrue(color_path.is_file()) - self.assertFalse((job_directory / "contour.json").exists()) - self.assertEqual(received[0]["model"], "MOGE2_VITS_NORMAL") - self.assertEqual(received_images, [job_directory / "analysis.png"]) - analysis_pixels = np.asarray( - Image.open(received_images[0]).convert("RGBA") - ) - self.assertEqual( - json.loads((job_directory / "bounds.json").read_text()), - [0, 1, 8, 5], - ) - self.assertEqual(tuple(analysis_pixels[2, 2, :3]), (20, 40, 60)) - self.assertGreater(int(analysis_pixels[2, 2, 3]), 240) - self.assertEqual(tuple(analysis_pixels[2, 0, :3]), (20, 40, 60)) - self.assertEqual(tuple(color_pixels[2, 0, :3]), (20, 40, 60)) - self.assertGreater(int(color_pixels[2, 0, 3]), 240) - self.assertEqual(tuple(color_pixels[2, 7, :3]), (20, 40, 60)) - self.assertEqual(color_pixels.shape[:2], (4, 8)) - self.assertEqual(analysis_pixels.shape[:2], (4, 8)) - self.assertEqual(normal_pixels.shape[:2], (4, 8)) - self.assertEqual(tuple(normal_pixels[0, 0]), (255, 127, 127)) - self.assertEqual(tuple(normal_pixels[2, 0]), (255, 127, 127)) - - def test_cutout_limits_the_cropped_model_input_but_keeps_color_resolution(self): - import numpy as np - from PIL import Image - - mesh_job = sys.modules["server.jobs.prepare_geometry_data"] - source_path = self.storage_root / "limited-cutout-source.png" - Image.new("RGBA", (20, 10), (20, 40, 60, 255)).save(source_path) - job_directory = self.storage_root / "limited-cutout-job" - context = SimpleNamespace( - directory=job_directory, - progress=lambda *_args: None, - check_cancelled=lambda: None, - resource=lambda _name: object(), - ) - prediction = SimpleNamespace( - normal=np.zeros((1, 2, 8, 3), dtype=np.float32) - ) - inferred_sizes = [] - - def infer(_manager, _parameters, images, **_kwargs): - with Image.open(images[0]) as model_input: - inferred_sizes.append(model_input.size) - return prediction - - with patch.object(mesh_job.moge2, "infer", side_effect=infer): - result = mesh_job.run( - context, - { - "input": str(source_path), - "shape": "BALLOON", - "polygon": ((2, 3), (18, 3), (18, 7), (2, 7)), - "mesh_detail": 64, - "max_input_size": 8, - "invert": False, - "device": "cpu", - "model": "MOGE2_VITS_NORMAL", - "resolution_level": 5, - "normal_analysis": True, - }, - ) - - with Image.open(job_directory / "image.0001.png") as color: - color_size = color.size - with Image.open(job_directory / result["normal"]) as normal: - normal_size = normal.size - self.assertEqual(inferred_sizes, [(8, 2)]) - self.assertEqual(color_size, (16, 4)) - self.assertEqual(normal_size, (16, 4)) - self.assertFalse((job_directory / "model-input.png").exists()) - - def test_mesh_job_skips_moge_normal_when_option_is_disabled(self): - from PIL import Image - - mesh_job = sys.modules["server.jobs.prepare_geometry_data"] - source_path = self.storage_root / "mesh-no-normal-source.png" - Image.new("RGBA", (8, 6), (20, 40, 60, 255)).save(source_path) - job_directory = self.storage_root / "mesh-no-normal-job" - context = SimpleNamespace( - directory=job_directory, - progress=lambda *_args: None, - check_cancelled=lambda: None, - resource=lambda _name: object(), - ) - - with patch.object(mesh_job, "_write_camera_normal") as write_normal: - result = mesh_job.run( - context, - { - "input": str(source_path), - "shape": "BALLOON", - "polygon": ((0, 0), (8, 0), (8, 6), (0, 6)), - "mesh_detail": 64, - "normal_analysis": False, - "invert": False, - "device": "cpu", - }, - ) - - self.assertEqual( - result, - {"selection": "selection.png", "bounds": "bounds.json"}, - ) - write_normal.assert_not_called() - self.assertFalse((job_directory / "normal.png").exists()) - - def test_mesh_depth_balloon_uses_metric_point_field(self): - from PIL import Image - - mesh_job = sys.modules["server.jobs.prepare_geometry_data"] - source_path = self.storage_root / "mesh-depth-balloon-source.png" - Image.new("RGBA", (6, 6), (20, 40, 60, 255)).save(source_path) - job_directory = self.storage_root / "mesh-depth-balloon-job" - context = SimpleNamespace( - directory=job_directory, - progress=lambda *_args: None, - check_cancelled=lambda: None, - ) - field_path = job_directory / "field.npz" - - with patch.object( - mesh_job, - "_write_depth_field", - return_value=(field_path, None), - ) as write_surface: - result = mesh_job.run( - context, - { - "input": str(source_path), - "shape": "DEPTH_BALLOON", - "polygon": ((1, 1), (5, 1), (5, 5), (1, 5)), - "normal_analysis": False, - "video_frames": 1, - "invert": False, - }, - ) - - self.assertEqual( - result, - { - "selection": "selection.png", - "bounds": "bounds.json", - "field": "field.npz", - }, - ) - self.assertTrue(write_surface.called) - self.assertEqual( - write_surface.call_args.kwargs["selection_path"], - job_directory / "selection.png", - ) - - def test_moge_depth_and_normal_share_one_prediction(self): + def test_image_geometry_keeps_model_and_region_validity_separate(self): import numpy as np - mesh_job = sys.modules["server.jobs.prepare_geometry_data"] - job_directory = self.storage_root / "shared-moge-prediction" - job_directory.mkdir() - color_path = job_directory / "analysis.png" - normal_path = job_directory / "normal.png" - prediction = object() - valid = np.ones((2, 3), dtype=bool) + depth = sys.modules["server.geometry.generate_depth"] + output_path = self.storage_root / "image_geometry.npz" + valid = np.asarray(((True, True, False), (True, True, True))) + calibration = np.asarray(((False, True, False), (False, False, True))) frame = SimpleNamespace( - depth=np.ones((2, 3), dtype=np.float32), + depth=np.asarray(((1.0, 2.0, 3.0), (4.0, 5.0, 6.0))), points=np.ones((2, 3, 3), dtype=np.float32), intrinsics=np.eye(3, dtype=np.float32), valid_mask=valid, - model_mask=valid, - ) - context = SimpleNamespace( - directory=job_directory, - resource=lambda _name: object(), - check_cancelled=lambda: None, - ) - parameters = { - "model": "MOGE2_VITS_NORMAL", - "device": "cpu", - "resolution_level": 5, - } - - with ( - patch.object( - mesh_job.moge2, "infer", return_value=prediction - ) as infer, - patch.object( - mesh_job.moge2, "geometry_frame", return_value=frame - ), - patch.object(mesh_job, "_write_field") as write_field, - patch.object( - mesh_job, - "_write_camera_normal_from_prediction", - return_value=normal_path, - ) as write_normal, - ): - field_path, written_normal_path = mesh_job._write_depth_field( - context, - parameters, - color_path, - include_normal=True, - ) - - infer.assert_called_once() - write_field.assert_called_once() - write_normal.assert_called_once_with( - prediction, color_path, normal_path - ) - self.assertEqual(field_path, job_directory / "field.npz") - self.assertEqual(written_normal_path, normal_path) - - def test_depth_calibration_mask_is_separate_from_model_validity(self): - import numpy as np - from PIL import Image - - mesh_job = sys.modules["server.jobs.prepare_geometry_data"] - selection_path = self.storage_root / "depth-selection.png" - selection = np.zeros((2, 3, 4), dtype=np.uint8) - selection[0, 1, 3] = 255 - selection[1, 2, 3] = 255 - Image.fromarray(selection, mode="RGBA").save(selection_path) - frame = SimpleNamespace( - valid_mask=np.asarray( - ((True, True, True), (True, True, False)), - dtype=bool, - ) + model_mask=None, ) - original_valid = frame.valid_mask.copy() - result = mesh_job._selection_calibration_mask(frame, selection_path) + depth._write_image_geometry_archive(frame, output_path, calibration, 0.05) - np.testing.assert_array_equal( - result, - ((False, True, False), (False, False, False)), - ) - np.testing.assert_array_equal(frame.valid_mask, original_valid) - - def test_depth_field_writes_original_and_calibration_masks_separately(self): - import numpy as np - - mesh_job = sys.modules["server.jobs.prepare_geometry_data"] - output_path = self.storage_root / "field.npz" - valid = np.asarray(((True, True), (True, False)), dtype=bool) - calibration = np.asarray(((False, True), (False, False)), dtype=bool) - depth = np.asarray(((2.0, 2.1), (2.2, 0.0)), dtype=np.float32) - points = np.zeros((2, 2, 3), dtype=np.float32) - points[..., 2] = depth - frame = SimpleNamespace( - depth=depth, - points=points, - intrinsics=np.eye(3, dtype=np.float32), - valid_mask=valid, - model_mask=valid, - ) - - mesh_job._write_field(frame, output_path, calibration) - - with np.load(output_path, allow_pickle=False) as archive: + with np.load(output_path) as archive: np.testing.assert_array_equal(archive["valid"], valid) - np.testing.assert_array_equal( - archive["calibration_valid"], - calibration, + np.testing.assert_array_equal(archive["calibration_valid"], calibration) + self.assertAlmostEqual( + float(archive["region_transparent_fraction"]), + 0.05, + places=6, ) - def test_mesh_job_uses_upscale_source_only_for_analysis(self): - from PIL import Image - - mesh_job = sys.modules["server.jobs.prepare_geometry_data"] - display_path = self.storage_root / "upscaled.png" - analysis_path = self.storage_root / "original.png" - Image.new("RGBA", (12, 8), (200, 20, 40, 255)).save(display_path) - Image.new("RGBA", (3, 2), (10, 80, 160, 255)).save(analysis_path) - job_directory = self.storage_root / "mesh-upscale-source-job" + def test_generate_region_data_only_returns_requested_outputs(self): + job = sys.modules["server.jobs.generate_region_data"] + input_path = self.storage_root / "input.png" + input_path.write_bytes(b"region") context = SimpleNamespace( - directory=job_directory, + directory=self.storage_root / "job", progress=lambda *_args: None, check_cancelled=lambda: None, ) - field_path = job_directory / "field.npz" - + archive = context.directory / "image_geometry.npz" + normal = context.directory / "normal.png" with patch.object( - mesh_job, - "_write_depth_field", - return_value=(field_path, None), - ) as write_surface: - result = mesh_job.run( + job, + "generate_depth", + return_value=(archive, normal), + ) as generate: + result = job.run( context, { - "input": str(display_path), - "analysis_input": str(analysis_path), - "shape": "DEPTH_SURFACE", - "normal_analysis": False, - "video_frames": 1, + "input": str(input_path), + "output_depth": True, + "normal_mode": "OBJECT", }, ) - self.assertEqual(result, {"field": "field.npz"}) - self.assertEqual(write_surface.call_args.args[2], analysis_path) - with Image.open(job_directory / "image.0001.png") as color: - self.assertEqual(color.size, (12, 8)) - - def test_mesh_masked_color_keeps_same_direction_overlap(self): - cutout_job = importlib.import_module("server.selection.cutout_images") - import numpy as np - from PIL import Image - - mesh_job = sys.modules["server.jobs.prepare_geometry_data"] - source_path = self.storage_root / "opaque-overlap.png" - output_path = self.storage_root / "cutout-overlap.png" - Image.new("RGBA", (12, 9), (20, 40, 60, 255)).save(source_path) - polygon = ( - (1, 1), - (7, 1), - (7, 7), - (1, 7), - (1, 1), - (4, 1), - (10, 1), - (10, 7), - (4, 7), - (4, 1), - (1, 1), + self.assertEqual( + result, + {"image_geometry": "image_geometry.npz", "normal": "normal.png"}, + ) + generate.assert_called_once_with( + context, + { + "input": str(input_path), + "output_depth": True, + "normal_mode": "OBJECT", + }, + input_path, + output_depth=True, + normal_mode="OBJECT", ) - cutout_job._write_masked_color(source_path, polygon, output_path) - alpha = np.asarray(Image.open(output_path).convert("RGBA"))[:, :, 3] - - self.assertGreater(int(alpha[3, 2]), 200) - self.assertGreater(int(alpha[3, 5]), 200) - self.assertGreater(int(alpha[3, 8]), 200) - def test_mesh_masked_color_preserves_nonzero_winding_hole(self): - cutout_job = importlib.import_module("server.selection.cutout_images") - import numpy as np - from PIL import Image - - mesh_job = sys.modules["server.jobs.prepare_geometry_data"] - source_path = self.storage_root / "opaque-hole.png" - output_path = self.storage_root / "cutout-hole.png" - Image.new("RGBA", (10, 10), (20, 40, 60, 255)).save(source_path) - polygon = ( - (1, 1), - (9, 1), - (9, 9), - (1, 9), - (1, 1), - (3, 3), - (3, 7), - (7, 7), - (7, 3), - (3, 3), - (1, 1), + def test_generate_region_data_refines_an_already_rasterized_region(self): + job = sys.modules["server.jobs.generate_region_data"] + input_path = self.storage_root / "input.png" + input_path.write_bytes(b"region") + directory = self.storage_root / "job" + context = SimpleNamespace( + directory=directory, + progress=lambda *_args: None, + check_cancelled=lambda: None, ) - cutout_job._write_masked_color(source_path, polygon, output_path) - alpha = np.asarray(Image.open(output_path).convert("RGBA"))[:, :, 3] + def generate(_context, _input, _model, _device, output): + output.write_bytes(b"refined") + return output, (2, 3, 8, 9) - self.assertGreater(int(alpha[2, 2]), 200) - self.assertLess(int(alpha[4, 4]), 32) + with patch.object(job, "generate_selection", side_effect=generate): + result = job.run( + context, + { + "input": str(input_path), + "refine_region": True, + "refine_model": "BEN2_BASE", + "device": "cpu", + }, + ) - def test_mesh_masked_color_antialiases_the_cutout_edge(self): - cutout_job = importlib.import_module("server.selection.cutout_images") - import numpy as np - from PIL import Image - - mesh_job = sys.modules["server.jobs.prepare_geometry_data"] - source_path = self.storage_root / "opaque.png" - output_path = self.storage_root / "cutout.png" - Image.new("RGBA", (16, 16), (20, 40, 60, 255)).save(source_path) - - cutout_job._write_masked_color( - source_path, - ((1, 1), (14, 1), (1, 14)), - output_path, + self.assertEqual( + result, + {"region": "region.png", "region_bounds": "region_bounds.json"}, + ) + self.assertEqual( + (directory / "region_bounds.json").read_text(encoding="utf-8"), + "[2,3,8,9]", ) - alpha = np.asarray(Image.open(output_path).convert("RGBA"))[:, :, 3] - - self.assertTrue(np.any((alpha > 0) & (alpha < 255))) - -if __name__ == "__main__": - unittest.main() diff --git a/tests/test_worker.py b/tests/test_worker.py index 52fbea8..3e00f4b 100644 --- a/tests/test_worker.py +++ b/tests/test_worker.py @@ -927,6 +927,13 @@ class WorkerTest(unittest.TestCase): self.assertEqual(result.mode, "RGBA") self.assertEqual(result.size, (20, 12)) alpha = np.asarray(result.getchannel("A")) + expected_alpha = np.asarray( + Image.fromarray(pixels[:, :, 3], mode="L").resize( + result.size, + Image.Resampling.LANCZOS, + ) + ) + np.testing.assert_array_equal(alpha, expected_alpha) self.assertLess( int(alpha[:, 0].max()), int(alpha[:, -1].min()), diff --git a/tools/node_assets/build.py b/tools/node_assets/build.py index 0603a08..0b1d4e5 100644 --- a/tools/node_assets/build.py +++ b/tools/node_assets/build.py @@ -19,9 +19,6 @@ from build_cutout import ( ) from build_mesh_plane import build_mesh_plane_group from build_shadeless import build_shadeless_group -from anyimage.operators.common.node_graph import expand_smooth - - OUTPUT_PATH = ( PROJECT_ROOT / "src" @@ -40,7 +37,7 @@ def main(): shape: build_node_group(shape) for shape in CUTOUT_NODE_GROUP_NAMES } - build_cutout_modifier_group(shape_groups, expand_smooth) + build_cutout_modifier_group(shape_groups) OUTPUT_PATH.parent.mkdir(parents=True, exist_ok=True) bpy.ops.wm.save_as_mainfile(filepath=str(OUTPUT_PATH), check_existing=False) diff --git a/tools/node_assets/build_cutout.py b/tools/node_assets/build_cutout.py index b141302..906d57a 100644 --- a/tools/node_assets/build_cutout.py +++ b/tools/node_assets/build_cutout.py @@ -4,6 +4,8 @@ import math import bpy +from node_graph import expand_smooth + from anyimage.operators.cutout_tool.shape_types import ( BALLOON_ATTRIBUTE_NAME, CUTOUT_NODE_GROUP_NAME, @@ -727,7 +729,7 @@ def build_node_group(shape): options = group.interface.new_panel(name="Options", default_closed=True) reference_depth = _float_input( group, - "Reference Depth", + "Base Plane Depth", 1.0, 0.0, None, @@ -751,7 +753,7 @@ def build_node_group(shape): options = group.interface.new_panel(name="Options", default_closed=True) reference_depth = _float_input( group, - "Reference Depth", + "Base Plane Depth", 1.0, 0.0, None, @@ -788,7 +790,7 @@ def build_node_group(shape): "Depth Image", "Depth Scale", "Uniform Scale", - "Reference Depth", + "Base Plane Depth", "Depth Direction", }, ) @@ -813,7 +815,7 @@ def build_node_group(shape): controls.outputs["Uniform Scale"], controls.outputs["Depth Direction"], controls.outputs["Depth Image"], - controls.outputs["Reference Depth"], + controls.outputs["Base Plane Depth"], ) output_location = (5100.0, 600.0) else: @@ -823,7 +825,7 @@ def build_node_group(shape): controls.outputs["Thickness"], controls.outputs["Depth Scale"], controls.outputs["Uniform Scale"], - controls.outputs["Reference Depth"], + controls.outputs["Base Plane Depth"], controls.outputs["Depth Image"], ) output_location = (8200.0, 400.0) @@ -844,7 +846,7 @@ def build_node_group(shape): return group -def build_cutout_modifier_group(shape_groups, expand_smooth): +def build_cutout_modifier_group(shape_groups): group = bpy.data.node_groups.get(CUTOUT_NODE_GROUP_NAME) if group is not None: return group @@ -896,16 +898,14 @@ def build_cutout_modifier_group(shape_groups, expand_smooth): reference_depth = _float_input( group, - "Reference Depth", + "Base Plane Depth", 1.0, 0.0, None, subtype="DISTANCE", parent=options, ) - reference_depth.description = ( - "Metric reference depth shared by both Depth shapes" - ) + reference_depth.description = "Depth of the flat base plane for both Depth shapes" depth_direction = _vector_input( group, "Depth Direction", @@ -936,10 +936,13 @@ def build_cutout_modifier_group(shape_groups, expand_smooth): (-2700.0, 500.0), {"Geometry", "Cleanup"}, ) + cleanup_area = _math(group.nodes, "MULTIPLY", (-2500.0, 750.0)) + cleanup_area.inputs[1].default_value = 0.01 + group.links.new(base_controls.outputs["Cleanup"], cleanup_area.inputs[0]) geometry = _island_cleanup( group, base_controls.outputs["Geometry"], - base_controls.outputs["Cleanup"], + cleanup_area.outputs[0], (-2350.0, 250.0), ) diff --git a/src/anyimage/operators/common/node_graph.py b/tools/node_assets/node_graph.py similarity index 96% rename from src/anyimage/operators/common/node_graph.py rename to tools/node_assets/node_graph.py index dc6fe08..c13a1ab 100644 --- a/src/anyimage/operators/common/node_graph.py +++ b/tools/node_assets/node_graph.py @@ -1,14 +1,4 @@ -"""Shared Geometry Nodes construction helpers.""" - -def interface_socket(group, name, in_out, socket_type): - socket = group.interface.new_socket( - name=name, - in_out=in_out, - socket_type=socket_type, - ) - if in_out == "INPUT" and hasattr(socket, "structure_type"): - socket.structure_type = "SINGLE" - return socket +"""Node graph builders used only while generating bundled assets.""" def group_input(nodes, location, visible_names): @@ -189,4 +179,3 @@ def expand_smooth( node.location.x += x_offset return repeat_output.outputs["Geometry"] - diff --git a/tools/node_assets/validate.py b/tools/node_assets/validate.py index 25def21..11b88f6 100644 --- a/tools/node_assets/validate.py +++ b/tools/node_assets/validate.py @@ -11,10 +11,8 @@ sys.path.insert( str(next((PROJECT_ROOT / "src" / "anyimage" / "wheels").glob("blendjob-*.whl"))), ) -from anyimage.operators.common.node_asset import ( +from anyimage.operators.common.mesh_result import ( load_asset_node_group, - load_mesh_plane_node_group, - load_shadeless_node_group, ) from anyimage.operators.cutout_tool.shape_types import ( CUTOUT_NODE_GROUP_NAME, @@ -32,8 +30,8 @@ def input_names(group): def main(): bpy.ops.wm.read_factory_settings(use_empty=True) - mesh_plane = load_mesh_plane_node_group() - shadeless = load_shadeless_node_group() + mesh_plane = load_asset_node_group("O Mesh Plane") + shadeless = load_asset_node_group("O Shadeless") cutout = load_asset_node_group(CUTOUT_NODE_GROUP_NAME) cutout_groups = { name: bpy.data.node_groups[name] diff --git a/tools/node_assets/validate_cutout.py b/tools/node_assets/validate_cutout.py index 084a732..6752325 100644 --- a/tools/node_assets/validate_cutout.py +++ b/tools/node_assets/validate_cutout.py @@ -13,7 +13,7 @@ sys.path.insert( from anyimage.operators.cutout_tool import node_assets as cutout_nodes from anyimage.operators.cutout_tool import shape_types as shapes -from anyimage.operators.common.node_asset import ( +from anyimage.operators.common.mesh_result import ( modifier_input_identifier, set_modifier_input, ) @@ -125,7 +125,7 @@ def main(): "Smooth Iterations", "Cleanup", "Smooth", - "Reference Depth", + "Base Plane Depth", "Depth Direction", "Uniform Scale", "Depth Image", @@ -138,7 +138,7 @@ def main(): "Thickness", "Depth Scale", "Double Sided", - "Reference Depth", + "Base Plane Depth", "Depth Direction", "Uniform Scale", "Depth Image", @@ -147,7 +147,7 @@ def main(): "Geometry", "Thickness", "Depth Scale", - "Reference Depth", + "Base Plane Depth", "Uniform Scale", "Depth Image", ], @@ -171,7 +171,7 @@ def main(): "Thickness": set(shapes.CUTOUT_SHAPES), "Depth Scale": {"DEPTH_BALLOON", "DEPTH_SURFACE"}, "Double Sided": {"DEPTH_BALLOON"}, - "Reference Depth": {"DEPTH_BALLOON", "DEPTH_SURFACE"}, + "Base Plane Depth": {"DEPTH_BALLOON", "DEPTH_SURFACE"}, "Depth Direction": {"DEPTH_BALLOON"}, "Uniform Scale": {"DEPTH_BALLOON", "DEPTH_SURFACE"}, "Depth Image": {"DEPTH_BALLOON", "DEPTH_SURFACE"}, @@ -230,6 +230,15 @@ def main(): ) assert cleanup.default_value == 0.0 assert cleanup.subtype == "NONE" + cleanup_scale = next( + link.to_node + for link in wrapper.links + if link.from_socket.name == "Cleanup" + and link.from_node.bl_idname == "NodeGroupInput" + ) + assert cleanup_scale.bl_idname == "ShaderNodeMath" + assert cleanup_scale.operation == "MULTIPLY" + assert abs(cleanup_scale.inputs[1].default_value - 0.01) < 1e-7 assert smooth.default_value == 1.0 assert iterations.default_value == 0 assert iterations.min_value == 0 @@ -341,7 +350,7 @@ def main(): ("Depth Image", point_image), ("Depth Scale", 1.0), ("Uniform Scale", 1.0), - ("Reference Depth", 2.0), + ("Base Plane Depth", 2.0), ("Depth Direction", (0.0, 0.0, 1.0)), ): set_modifier_input( @@ -392,7 +401,7 @@ def main(): ("Depth Image", point_image), ("Depth Scale", 1.0), ("Uniform Scale", 1.0), - ("Reference Depth", 2.0), + ("Base Plane Depth", 2.0), ("Smooth Iterations", 0), ): set_modifier_input( diff --git a/tools/node_assets/validate_mesh_plane.py b/tools/node_assets/validate_mesh_plane.py index 41cbd3b..5e15c97 100644 --- a/tools/node_assets/validate_mesh_plane.py +++ b/tools/node_assets/validate_mesh_plane.py @@ -11,8 +11,7 @@ sys.path.insert( str(next((PROJECT_ROOT / "src" / "anyimage" / "wheels").glob("blendjob-*.whl"))), ) -from anyimage.operators.common.node_asset import ( - get_modifier_input, +from anyimage.operators.common.mesh_result import ( modifier_input_identifier, set_modifier_input, ) @@ -28,6 +27,13 @@ def evaluated_mesh(obj): return evaluated, evaluated.to_mesh() +def modifier_input_value(modifier, identifier): + inputs = getattr(getattr(modifier, "properties", None), "inputs", None) + if inputs is not None: + return getattr(inputs, identifier).value + return modifier[identifier] + + def main(): bpy.ops.wm.read_factory_settings(use_empty=True) mesh = create_mesh_plane_mesh("Mesh Plane", (-1.0, 1.0, -0.5, 0.5)) @@ -49,7 +55,7 @@ def main(): assert inputs["Subdivide"].max_value == 10 assert inputs["Thickness"].default_value == 0.0 assert inputs["Material"].hide_in_modifier - assert get_modifier_input(modifier, inputs["Material"].identifier) == material + assert modifier_input_value(modifier, inputs["Material"].identifier) == material evaluated, result = evaluated_mesh(obj) try: