feat: Implement cutout tool and depth generation features
- Added a new cutout tool for selecting regions in images, allowing users to choose mesh shapes based on selected areas. - Introduced depth generation capabilities for rasterized images, including normal data extraction. - Created jobs for generating depth planes and region data, enhancing the AI data generation workflow. - Developed a selection refinement process for cutout regions, improving accuracy in generated outputs. - Established a new region module for handling rasterized image regions and their associated AI generation tasks. - Enhanced node graph builders for asset generation, facilitating smoother integration of new features.
This commit is contained in:
@@ -54,6 +54,7 @@ uv lock --refresh
|
||||
## 命名规范
|
||||
|
||||
- 变量、函数、Docstring 使用英文；同一概念在代码和文档中使用统一表达
|
||||
- 功能代码的动作前缀应表达产物边界：`generate_*` 表示 AI 生成，`build_*` 只在内存中构建值或几何，`create_*` 表示创建 Blender Image、Mesh、Material、Object 等真实数据
|
||||
- 名称应说明业务职责，并与用户概念、公开协议和界面术语保持一致；不要使用脱离上下文后无法判断用途的宽泛名称
|
||||
- 文件和目录名应反映职责层级：集合层使用复数，单个功能或动作使用单数或动宾结构；同一层级保持一致
|
||||
- Python 类使用大驼峰，不添加项目名前缀。Operator 使用动宾结构，如 `ConvertToDepthPlane`、`RemoveImageBackground`；Menu、Panel、AddonPreferences、PropertyGroup 分别使用实际类型后缀，如 `AnyImageImageMenu`、`ServerPanel`、`AnyImagePreferences`、`AnyImageSettings`
|
||||
|
||||
14
README.md
14
README.md
@@ -9,14 +9,14 @@ AnyImage 是一款 Blender Extension，用于把图片带入可编辑的 2.5D
|
||||
- 在 3D View、Node Editor 与绘画模式中用 `Ctrl+V`（macOS 为 `Cmd+V`）粘贴并 Pack 剪贴板图片
|
||||
- 将 Image Empty 转换为带 `O Mesh Plane` Geometry Nodes 的普通 Plane
|
||||
- 使用 MoGe-2 或 DA3 将静态图转换为可调 Depth Plane
|
||||
- 用 Cutout 套索在图片可见 Alpha 内生成 Surface、Balloon、Depth Balloon 或 Depth Surface，并通过统一的 `O Image Cutout` 修改器切换 Shape、按创建分辨率清理碎岛和调整表面
|
||||
- 用 Cutout 套索在图片可见 Alpha 内生成 Surface、Balloon、Depth Balloon 或 Depth Surface，并通过统一的 `O Image Cutout` 修改器切换 Shape、按创建分辨率清理碎岛、根据透明选区校准浮雕基准和调整表面
|
||||
- 用 Image Box、Lasso、Line、Polyline 和 Perspective 清理或矫正静态图片
|
||||
- 用 BEN2 为单图、视频或序列帧去除背景
|
||||
- 用三档 ONNX 模型将单图、视频或序列帧放大 4×
|
||||
- 用三档 ONNX 模型将单图、视频或序列帧放大 4×，并独立重采样 Alpha
|
||||
|
||||
AI 推理运行在 Blender 之外的独立 Python 3.12 Environment 中。Blender 进程只负责 UI、任务提交和结果导入，不加载 ONNX Runtime 或模型权重。
|
||||
|
||||
全局 **Maximum AI Input Size** 默认限制 Upscale、Depth 与 Normal 分析的长边为 2048 px；超过时等比缩放分析输入，但不会降低颜色贴图分辨率。Cutout 在套索裁切后应用该限制，并以同一分析尺度计算网格密度。
|
||||
全局 **Maximum AI Input Size** 默认设为 2048 px。当前图片长边达到该值时 Upscale 不可用；Depth 与 Normal 的分析输入超过该值时等比缩小，但不会降低颜色贴图分辨率。Cutout 在套索裁切后应用该限制，并以同一分析尺度计算网格密度。
|
||||
|
||||
## 安装
|
||||
|
||||
@@ -28,7 +28,7 @@ AI 推理运行在 Blender 之外的独立 Python 3.12 Environment 中。Blender
|
||||
3. 选中 Image Empty，在右键 **AnyImage** 菜单中点击 **Set Up AI Environment…**。
|
||||
4. 安装完成后，在扩展 Preferences 中确认默认 MoGe-2、BEN2 与 Real-ESRGAN WDN 模型显示 Ready。
|
||||
|
||||
普通 Plane、基础 Image 编辑，以及不带 Refine / Normal 的 Cutout Surface 与 Balloon 无需 AI Environment。Cutout 的 Depth 与 Normal 统一使用 MoGe-2；两者同时启用时共享同一次 Prediction，不会重复运行模型。
|
||||
普通 Plane、基础 Image 编辑，以及不带 Refine / Normal 的 Cutout Surface 与 Balloon 无需 AI Environment。Cutout 的 Depth 与 Normal 统一使用 MoGe-2；两者同时启用时共享同一次 Prediction，不会重复运行模型。Cutout Normal 可选择 None、基于完整图片平面的 Tangent Space 浮雕，或直接编码到对象局部轴的 Object Space。
|
||||
|
||||
## 快速使用
|
||||
|
||||
@@ -37,9 +37,9 @@ AI 推理运行在 Blender 之外的独立 Python 3.12 Environment 中。Blender
|
||||
- **Convert to Plane**：同步创建普通图片网格；
|
||||
- **Convert to Depth Plane**：使用 Preferences 中选择的深度模型处理静态图；
|
||||
- **Remove Background**：生成透明图片或序列；
|
||||
- **Upscale**：使用默认高清模型生成 4× 结果。
|
||||
- **Upscale**：使用默认高清模型把当前图片编辑为 4× 结果。
|
||||
|
||||
3D Viewport 工具栏提供 Image Mask、Perspective 与 Cutout。工作流对比和当前输入范围见[选择工作流](docs/guides/workflows.md)。
|
||||
3D Viewport 工具栏提供 Image Mask、Perspective 与 Cutout。Remove Background、Upscale、Image Mask 和 Perspective 都只更新当前 Empty；共享 Image 会保留原数据，并为当前 Empty 创建独立结果。工作流对比和当前输入范围见[选择工作流](docs/guides/workflows.md)。
|
||||
|
||||
## 源码结构
|
||||
|
||||
@@ -52,7 +52,7 @@ src/anyimage/
|
||||
└── __init__.py # Blender 注册入口
|
||||
```
|
||||
|
||||
程序边界、功能文件结构和调用流程见[内部实现总览](docs/internals/index.md)。文档按开始使用、功能指南、参考和内部实现分层，不在 README 重复内部算法细节。
|
||||
程序边界、功能文件结构和调用流程见[内部实现总览](docs/internals/index.md)；开发新功能前先查看[Common 公共函数](docs/internals/common.md)，复用已经存在的 Image、ImageGeometry、Material 和对象替换入口。文档按开始使用、功能指南、参考和内部实现分层，不在 README 重复内部算法细节。
|
||||
|
||||
## 开发
|
||||
|
||||
|
||||
@@ -21,7 +21,7 @@ Cutout 用套索从一张 Image Empty 中选择区域，再生成 Surface、Ball
|
||||
|
||||
四种结果都保留源 Image Empty，并在同一位置创建新网格。它们共享选区轮廓、UV、材质和基础三角网格，因此可以从同一张图多次生成不同 Shape 进行比较。
|
||||
|
||||
无论是否启用 Refine Selection，套索超出图片的部分都不会生成网格，源图中 `Alpha <= 0.1` 的像素也始终从几何中排除。双击选择完整画面时同样只为 `Alpha > 0.1` 的区域生成网格。
|
||||
无论是否启用 Refine Selection，套索超出图片的部分都不会生成网格，源图中 `Alpha <= 0.1` 的像素也始终从 Region 中排除。双击使用完整画面时同样只为 `Alpha > 0.1` 的 Region 生成网格。
|
||||
|
||||
## 工具设置
|
||||
|
||||
@@ -38,19 +38,19 @@ Point Spacing 先按完整源图换算基础密度；选区裁切后的长边若
|
||||
四个 Shape 都使用同一个 `O Image Cutout` Geometry Nodes 修改器。创建时 Shape 菜单已经设为饼菜单所选类型；切换 Shape 后，Blender 只显示该类型实际使用的输入。常用参数包括：
|
||||
|
||||
- **Thickness**：Surface / Depth Surface 使用场景距离；Balloon / Depth Balloon 使用鼓起高度倍率。
|
||||
- **Cleanup**：绝对面积阈值；创建时根据 Point Spacing 和图片平面尺寸自动写入，面积小于该值的 Mesh Island 会在生成任意 Shape 前移除。
|
||||
- **Cleanup**：碎岛面积阈值；创建时根据 Point Spacing 和图片平面尺寸自动写入。为便于在修改器中调节，显示值放大了 `100` 倍，节点内部会还原为实际面积，因此这次显示缩放不会改变默认清理效果。
|
||||
- **Smooth Iterations**：位于外层，控制平滑次数，默认 `0`；**Smooth** 位于 Options 中，控制每次平滑的混合强度。四种 Shape 都可使用。
|
||||
- **Depth Scale**：Depth Shape 的深度强度；`0` 回到基础平面或 Balloon。
|
||||
|
||||
两个 Depth Shape 共用 **Reference Depth**；Depth Balloon 还提供 **Depth Direction**。插件自动写入的 Depth Image 与 Uniform Scale 位于折叠的 Data 面板，一般无需手动修改。
|
||||
两个 Depth Shape 共用 **Base Plane Depth**；Depth Balloon 还提供 **Depth Direction**。生成前会检查最终 Region 输入的 Alpha：`Alpha < 0.1` 的像素达到 `5%` 时，根据有效 Region 自适应 Base Plane Depth，并为 Depth Balloon 拟合 Direction；不足 `5%` 时使用稳定的固定基准。这个判断不依赖双击全图或 Refine Selection。插件自动写入的 Depth Image 与 Uniform Scale 位于折叠的 Data 面板，一般无需手动修改。
|
||||
|
||||
Balloon 的轮廓高度算法由扩展 Preferences 中的 **Balloon Profile** 控制：Poisson 更平滑，Teddy 更强调轮廓骨架。修改默认值只影响之后创建的 Shape。
|
||||
|
||||
## Refine 与 Normal
|
||||
|
||||
Refine Selection 只在“套索与源图可见 Alpha 的交集”内收紧用于建网格的最终选择，不把 AI 生成的颜色写回原图。材质继续使用当前 Image Empty 的源颜色和原 Alpha。
|
||||
Refine Selection 只在“套索与源图可见 Alpha 的交集”内收紧最终 Region，不把 AI 生成的颜色写回原图。材质继续使用当前 Image Empty 的源颜色和原 Alpha。
|
||||
|
||||
Normal 关闭时使用几何法线；开启后，Surface、Balloon 与 Depth Balloon 使用适配成品网格的 Tangent Space 法线，Depth Surface 使用 Object Space 法线。Cutout 始终使用 MoGe-2；Depth 与 Normal 同时需要时共享同一次 Prediction，不会重复运行模型。
|
||||
Normal 设为 **None** 时使用几何法线；**Tangent** 把 MoGe-2 结果作为基于完整图片平面的浮雕细节，**Object** 则直接编码到 Cutout 对象局部轴。两种模式都不根据 Geometry Nodes 的成品网格再次烘焙。Normal 保持模型分析分辨率，Color Image 仍保留原始裁切分辨率；Depth 与 Normal 同时需要时共享同一次 Prediction，不会重复运行模型。
|
||||
|
||||
## 当前范围
|
||||
|
||||
|
||||
@@ -20,7 +20,7 @@ AnyImage 把运行环境、模型和任务结果放在 Blender 之外，多个 B
|
||||
|
||||
## 全局 AI 输入尺寸
|
||||
|
||||
Environment 区的 **Maximum AI Input Size** 控制 Upscale、Depth 与 Normal 分析输入的最大长边，默认 `2048 px`，可在 `256–8192 px` 之间调整。超过上限的输入会保持宽高比缩小；颜色贴图仍使用原始分辨率。Cutout 在完成套索裁切后再应用上限，因此整张原图很大但选区较小时不会被无谓缩小。
|
||||
Environment 区的 **Maximum AI Input Size** 默认 `2048 px`，可在 `256–8192 px` 之间调整。它既是允许执行 Upscale 的当前图片长边上限，也是 Depth 与 Normal 分析输入的最大长边；Upscale 在达到上限时置灰，Depth 与 Normal 则保持宽高比缩小分析输入。颜色贴图仍使用当前图片的原始分辨率。Cutout 在完成套索裁切后再应用上限，因此整张图片很大但选区较小时不会被无谓缩小。
|
||||
## Device
|
||||
|
||||
扩展偏好设置中的 **Device** 控制 Depth Plane、Cutout 的 AI 增强、Remove Background 和 Upscale 等正式任务。
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
# 高清放大
|
||||
|
||||
Upscale 用 ONNX 模型把 Image Empty 的图片放大 4 倍，不创建网格。它支持单图、Movie 和 Image Sequence，并保留源 Alpha。
|
||||
Upscale 用 ONNX 模型把 Image Empty 的 RGB 放大 4 倍，不创建网格。它支持单图、Movie 和 Image Sequence，并以 Lanczos 独立放大源 Alpha 后重新合并。
|
||||
|
||||
## 使用
|
||||
|
||||
@@ -8,7 +8,7 @@ Upscale 用 ONNX 模型把 Image Empty 的图片放大 4 倍，不创建网格
|
||||
2. 按需要设置 Environment 区的 **Maximum AI Input Size**；默认 `2048 px`，最高 `8192 px`。
|
||||
3. 选中 Image Empty，在右键 **AnyImage** 菜单中点击 **Upscale**。
|
||||
|
||||
输入长边超过全局上限时，会先保持宽高比缩小，再执行 4× 放大。因此默认设置下结果长边最多为 `8192 px`。完成后，当前 Empty 会改用名称以 `_upscale.png` 结尾的结果图片。
|
||||
菜单项只显示本次操作预计生成的 4× 尺寸。当前图片长边必须小于全局上限；达到上限后 Upscale 会置灰，因此默认设置下单次结果长边小于 `8192 px`。完成后，当前 Empty 会直接使用放大结果；原 Image 没有其他引用时沿用原名并移除旧数据，存在其他引用时则为当前 Empty 保留独立结果。
|
||||
|
||||
## 模型选择
|
||||
|
||||
@@ -18,10 +18,10 @@ Upscale 用 ONNX 模型把 Image Empty 的图片放大 4 倍，不创建网格
|
||||
|
||||
后两项可在扩展偏好设置中按需下载。
|
||||
|
||||
对高清结果再次执行 Upscale 时，AnyImage 会从最初的原图重新生成并替换当前结果，不会连续放大已有高清图。
|
||||
AnyImage 不追踪高清结果的原图引用，Upscale 始终处理当前 Image。只要当前长边仍小于全局上限，就可以再次放大；达到上限后不能继续执行。
|
||||
|
||||
把高清结果继续转换为 Depth Plane 时，颜色材质可以继续使用当前高清图，分析输入则遵守全局上限；Cutout 同样只缩放裁切后的分析输入，不会把限制后的临时图用作 Color。
|
||||
把高清结果继续转换为 Depth Plane 时，颜色材质和分析都以当前 Image 为来源，分析输入仍遵守全局上限；Cutout 同样只缩放裁切后的分析输入，不会把限制后的临时图用作 Color。
|
||||
|
||||
如果显存或内存不足，请降低 **Maximum AI Input Size**，或改用 CPU。
|
||||
|
||||
分块推理、Alpha 合并与原图引用见 [Upscale 实现原理](../internals/operators/upscale.md)。
|
||||
分块推理、Alpha 合并与输入限制见 [Upscale 实现原理](../internals/operators/upscale.md)。
|
||||
|
||||
@@ -37,7 +37,7 @@ flowchart TD
|
||||
Menu --> Runtime["注册 JobRuntime"]
|
||||
```
|
||||
|
||||
`__init__.py` 按依赖顺序注册，并在注销时逆序清理。扩展级设置统一由 `preferences.addon_preferences()` 读取；`runtime.py` 只消费 `configured_storage_root` 回调，不持有 Blender 偏好设置的定位逻辑。交互中的临时状态和 Debug 参数保存在 `Scene.anyimage_settings`，Upscale 的原图引用保存在 `Image.anyimage_upscale_source`。 全局 `max_ai_input_size` 也在 `preferences.py` 定义与读取，由 Operator 冻结到每次 Job 请求。
|
||||
`__init__.py` 按依赖顺序注册，并在注销时逆序清理。扩展级设置统一由 `preferences.addon_preferences()` 读取；`runtime.py` 只消费 `configured_storage_root` 回调，不持有 Blender 偏好设置的定位逻辑。交互中的临时状态和 Debug 参数保存在 `Scene.anyimage_settings`。全局 `max_ai_input_size` 在 `preferences.py` 定义与读取，由 Operator 冻结到每次 Job 请求；Upscale 还用它判断当前 Image 是否允许执行。
|
||||
|
||||
## 同步与异步路径
|
||||
|
||||
@@ -60,6 +60,10 @@ sequenceDiagram
|
||||
|
||||
这条边界也决定了模块依赖方向：`server/` 不导入 `bpy` 或 `operators/`；Blender 侧可以读取 Server 产物，但不会复用 Server 中的 Blender 无关算法来创建 RNA 数据。
|
||||
|
||||
## 共享能力
|
||||
## 共享能力 { #shared-capabilities }
|
||||
|
||||
`operators/common/` 保存多个功能真实复用的能力：图片输入与命名、像素写回、手势状态、Mask 运算、Image Empty 坐标、材质、节点资产、法线烘焙和 Job 参数辅助。共享模块只承载跨功能概念；完整工作流仍留在各自功能包中，以便从用户入口追踪到实现。
|
||||
`operators/common/` 只保存多个生产功能真实复用的稳定原子能力；完整工作流、功能参数和专用算法留在所属模块。公共层不保留旧路径转发函数，也不为了测试或资产工具建立入口。
|
||||
|
||||
全部可由功能模块调用的函数见[Common 公共函数](common.md)。该索引只列 `common/` 外部的生产调用，不重复列出公共模块内部辅助函数；创建新功能前先核对该页，避免重新实现 Image 引用分流、像素方向、Job 输入生命周期、数据块命名、节点资产加载和对象激活逻辑。
|
||||
|
||||
新增公共函数前应确认至少有两个真实消费者，或它是两个公共流程共同依赖的原子能力。只有单一功能使用的逻辑应留在所属模块；删除或移动能力时同步清理注册、测试、工具脚本和文档，不保留兼容胶水。
|
||||
|
||||
102
docs/internals/common.md
Normal file
102
docs/internals/common.md
Normal file
@@ -0,0 +1,102 @@
|
||||
# Common 公共函数
|
||||
|
||||
`src/anyimage/operators/common/` 集中已经被多个功能复用的 Blender 侧原子能力，目的是让开发新功能时能够先查找并复用现有实现，避免重复造轮子。
|
||||
|
||||
本页只列功能模块直接调用的公共函数，不列 `common/` 内部辅助函数、测试或资产工具专用函数，也不列公共类型。
|
||||
|
||||
## Image 数据块
|
||||
|
||||
`image_data.py` 处理 Blender `Image` 数据块及其在 Image Empty 上的使用方式，包括像素读取、动画帧、Job 输入、结果 Image 创建和 Empty 引用替换。它不处理 Viewport 手势或选区算法。
|
||||
|
||||
| 函数 | 职责 |
|
||||
| --- | --- |
|
||||
| `image_data.is_image_empty()` | 判断对象是否为带有效 Image 数据的 Empty。 |
|
||||
| `image_data.require_image_empty()` | 按名称取得 Image Empty；对象已失效时抛错。 |
|
||||
| `image_data.image_pixels()` | 以 Blender 原始顺序读取一维 float32 像素。 |
|
||||
| `image_data.image_rgba()` | 读取为 Top-down RGBA float32 数组。 |
|
||||
| `image_data.video_total_frames()` | 返回 Movie 或 Sequence 总帧数，静态图返回 1。 |
|
||||
| `image_data.image_user_settings()` | 读取 Image Empty 的帧起点、偏移、时长和循环设置。 |
|
||||
| `image_data.default_extract_frames()` | 取得当前 Empty 默认应处理的帧数。 |
|
||||
| `image_data.source_start_frame()` | 从源 Image Empty 取得 Job 应处理的起始帧。 |
|
||||
| `image_data.prepare_image_input()` | 将 Packed、Dirty、Sequence 等输入准备为 Job 可读路径，并返回临时标记。 |
|
||||
| `image_data.prepare_region_input()` | 将 `RegionMask` 裁切为带 Mask Alpha 的临时 RGBA Job 输入。 |
|
||||
| `image_data.cleanup_image_input()` | 按临时标记清理由 `prepare_image_input()` 创建的文件或目录。 |
|
||||
| `image_data.image_base_name()` | 去除图片或视频扩展名和 Blender 数字后缀，得到数据块基础名。 |
|
||||
| `image_data.depth_data_name()` | 生成 Depth 结果数据块文件名。 |
|
||||
| `image_data.color_data_name()` | 生成 Color 结果数据块文件名。 |
|
||||
| `image_data.load_color_result_image()` | 加载、命名、Pack 并配置 Job 的 Color 或 Sequence 结果；不修改 Empty。 |
|
||||
| `image_data.load_normal_result_image()` | 加载、命名、设为 Non-Color 并 Pack Normal 结果。 |
|
||||
| `image_data.create_pixel_result_image()` | 从本地编辑后的 RGBA 像素创建 Packed Image；不修改 Empty。 |
|
||||
| `image_data.replace_empty_image()` | 执行 Image Empty 结果替换、共享引用分流、旧数据清理、帧设置继承和可选画布对齐。 |
|
||||
| `image_data.image_empty_bounds()` | 计算 Image Empty 在对象局部坐标中的实际显示边界。 |
|
||||
|
||||
`load_*_result_image()` 和 `create_pixel_result_image()` 只创建结果 Image；`replace_empty_image()` 负责把结果提交给当前 Empty。两者分别表达“创建产物”和“替换引用”。
|
||||
|
||||
## Image Region 与视图交互
|
||||
|
||||
这组公共能力使用固定术语：
|
||||
|
||||
- Region 是用户在图片空间圈定的业务范围；`RegionPath` 保存路径点、反选状态和填充规则。
|
||||
- `RegionMask` 是 Region 的栅格化结果，只包含局部像素权重 `values` 和源图 `bounds`。
|
||||
- Mask 只表示二维像素权重或 Alpha 运算，不作为手势、协议或业务范围的别名。
|
||||
- Selection 只用于 Blender 对象选择和界面动作 Refine Selection，不表示中间数据。
|
||||
|
||||
`image_interaction.py` 处理 3D Viewport 中的点击、坐标投影、`ImageRegionGesture`、工具设置和预览绘制；`image_region.py` 负责图片空间 Region 的值、栅格化和 Alpha 裁切。前者负责用户如何圈定 Region，后者负责 Region 如何变成 Mask。
|
||||
|
||||
### Viewport 交互
|
||||
|
||||
| 函数 | 职责 |
|
||||
| --- | --- |
|
||||
| `image_interaction.report_image_edit_exception()` | 将可预期的图片编辑警告和异常映射为 Operator Report。 |
|
||||
| `image_interaction.image_edit_poll()` | 判断当前上下文是否允许启动 Image 编辑工具。 |
|
||||
| `image_interaction.resolve_image_edit_click()` | 解析点击到 Image Empty，并保持或恢复选择状态。 |
|
||||
| `image_interaction.active_view3d_tool_id()` | 取得当前 View3D Workspace Tool ID。 |
|
||||
| `image_interaction.apply_region_settings()` | 将当前 Region 工具的 Refine 设置同步到手势 Operator。 |
|
||||
| `image_interaction.draw_region_settings()` | 绘制 Region 工具的公共 Refine 设置。 |
|
||||
| `image_interaction.draw_dashed_line()` | 绘制手势或透视框虚线。 |
|
||||
| `image_interaction.draw_polygon_fill()` | 绘制手势或透视框填充预览。 |
|
||||
| `image_interaction.gesture_property()` | 声明统一的 Box、Lasso、Line 和 Polyline RNA 属性。 |
|
||||
| `image_interaction.screen_path_to_image_pixels()` | 将屏幕路径投影为 Image Empty 像素坐标。 |
|
||||
| `image_interaction.serialize_matrix()` | 将 4×4 Matrix 序列化为 Operator 可保存的字符串。 |
|
||||
| `image_interaction.deserialize_matrix()` | 恢复已序列化的 4×4 Matrix。 |
|
||||
| `image_interaction.drawing_in_region()` | 判断 Modal 绘制是否仍位于启动时的 Area 和 Region。 |
|
||||
|
||||
### Region 栅格化
|
||||
|
||||
| 函数 | 职责 |
|
||||
| --- | --- |
|
||||
| `image_region.region_path_bounds()` | 校验 Region Path 并返回整数像素包围框。 |
|
||||
| `image_region.rasterize_region_path()` | 把 `RegionPath` 栅格化为 `RegionMask`，并可与源 Alpha 相交。 |
|
||||
| `image_region.crop_pixels_to_alpha()` | 按 Alpha 非零区域紧裁切 RGBA 像素。 |
|
||||
|
||||
## Mesh 结果构建
|
||||
|
||||
`mesh_result.py` 集中 Convert to Plane 与 Cutout 共同使用的结果构建能力：读取图片几何产物、创建材质、加载节点资产、设置修改器输入，并完成结果对象替换。
|
||||
|
||||
`ImageGeometry` 表示按图片像素组织的三维点、相机内参、有效区域、参考深度和选区统计；`load_image_geometry()` 从 `image_geometry.npz` 读取并校验这组数据。
|
||||
|
||||
| 函数 | 职责 |
|
||||
| --- | --- |
|
||||
| `mesh_result.load_image_geometry()` | 读取并校验 `image_geometry.npz`，返回用于构建 Mesh 的 `ImageGeometry`。 |
|
||||
| `mesh_result.create_image_material()` | 从 Image Empty 创建包含颜色、位移、Normal、Alpha 和动画设置的完整材质。 |
|
||||
| `mesh_result.load_asset_node_group()` | 从随扩展分发的 Blend 资产加载指定 Node Group。 |
|
||||
| `mesh_result.modifier_input_identifier()` | 按接口名称解析 Geometry Nodes Modifier 输入 Identifier。 |
|
||||
| `mesh_result.set_modifier_input()` | 跨受支持 Blender RNA 形式设置 Modifier 输入。 |
|
||||
| `mesh_result.finalize_object_result()` | 激活结果对象，并按需替换或保留源 Empty。 |
|
||||
|
||||
节点资产构建和验证专用函数位于 `tools/node_assets/`，不属于运行时公共能力。
|
||||
|
||||
## AI 任务准备
|
||||
|
||||
`ai_job.py` 处理多个 AI 功能共同使用的任务准备，包括输入路径校验、Environment 与模型检查、Device 选择，以及 DA3 和 MoGe-2 请求参数组装。它不包含业务 Operator、结果导入或 Undo 操作。
|
||||
|
||||
| 函数 | 职责 |
|
||||
| --- | --- |
|
||||
| `ai_job.require_input_path()` | 解析 Blender 路径并要求其为现存文件或目录。 |
|
||||
| `ai_job.require_background_input_path()` | 在通用路径校验之外限制 BEN2 支持的媒体类型。 |
|
||||
| `ai_job.production_device()` | 读取正式任务使用的计算设备。 |
|
||||
| `ai_job.require_environment()` | 要求 AnyImage AI Environment 已就绪。 |
|
||||
| `ai_job.invoke_ai_setup_if_needed()` | Environment 未就绪时启动安装交互。 |
|
||||
| `ai_job.require_model()` | 校验模型标识和下载状态，并返回模型记录。 |
|
||||
| `ai_job.da3_parameters()` | 组装 DA3 正式任务与 Debug 共用参数。 |
|
||||
| `ai_job.moge2_parameters()` | 组装 MoGe-2 正式任务与 Debug 共用参数。 |
|
||||
@@ -39,10 +39,11 @@ flowchart TD
|
||||
## 文档分层
|
||||
|
||||
- [程序架构](architecture.md)：注册、配置、线程和模块边界。
|
||||
- [Common 公共函数](common.md)：仅列生产功能从 `operators/common/` 外部调用的函数。
|
||||
- [Job Runtime 与 Server](runtime.md)：任务生命周期、媒体输入和模型 Session。
|
||||
- [Operator 实现索引](operators/index.md)：按 `src/anyimage/operators/` 中的完整功能拆分实现页。
|
||||
- [Job 产物](outputs.md)：文件用途和 Blender 导入关系。
|
||||
- [开发与发布](development.md)：本地环境、测试、打包和 CI。
|
||||
- [节点资产](node-assets.md)：Geometry Nodes 的构建、加载与验证。
|
||||
|
||||
内部实现文档描述模块之间的职责和主要数据流。函数级分支、参数校验与异常处理以源码和测试为准，避免文档重复实现细节。
|
||||
内部实现文档描述模块之间的职责和主要数据流。[Common 公共函数](common.md)按真实外部调用列出开发新功能时应优先复用的稳定入口；函数级分支、参数校验与异常处理仍以源码和测试为准。
|
||||
|
||||
@@ -7,12 +7,12 @@ AnyImage 将稳定、可复用的 Geometry Nodes 和 Shader 节点组构建到 `
|
||||
```text
|
||||
src/anyimage/
|
||||
├── assets/O_AnyImage.blend
|
||||
└── operators/common/
|
||||
├── node_asset.py # 资产路径、加载与 Modifier 输入
|
||||
└── node_graph.py # 共享节点图构造函数
|
||||
├── operators/common/mesh_result.py # 运行时资产加载与 Modifier 输入
|
||||
└── operators/convert_to_plane/nodes.py # Depth Plane 运行时节点图
|
||||
|
||||
tools/node_assets/
|
||||
├── build.py # 统一构建入口
|
||||
├── node_graph.py # Cutout 资产构建专用节点图函数
|
||||
├── build_mesh_plane.py
|
||||
├── build_shadeless.py
|
||||
├── build_cutout.py
|
||||
@@ -27,7 +27,7 @@ tools/node_assets/
|
||||
flowchart TD
|
||||
Scripts["tools/node_assets/build_*.py"] --> Build["Blender 后台构建"]
|
||||
Build --> Blend["assets/O_AnyImage.blend"]
|
||||
Blend --> Loader["operators/common/node_asset.py"]
|
||||
Blend --> Loader["operators/common/mesh_result.py"]
|
||||
Loader --> Modifier["运行时 Node Group / Modifier"]
|
||||
```
|
||||
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
# Plane 转换
|
||||
|
||||
Plane 转换包含两条路径：普通 Plane 在 Blender 内同步完成；Depth Plane 先通过 `prepare-geometry-data` Job 取得深度场，再回到 Blender 构建面片。两者共享图片命名、材质与基础四边形能力，但使用不同的 Geometry Nodes。
|
||||
Plane 转换包含两条路径：普通 Plane 在 Blender 内同步完成；Depth Plane 先通过 `generate-depth-plane` Job 取得深度场，再回到 Blender 构建面片。两者共享图片命名、材质与基础四边形能力，但使用不同的 Geometry Nodes。
|
||||
|
||||
## 对应文件
|
||||
|
||||
@@ -13,7 +13,8 @@ operators/convert_to_plane/
|
||||
└── nodes.py # 运行时 O Image Depth Plane 节点图
|
||||
|
||||
server/
|
||||
├── jobs/prepare_geometry_data.py
|
||||
├── jobs/generate_depth_plane.py
|
||||
├── geometry/generate_depth.py
|
||||
├── models/da3.py
|
||||
├── models/moge2.py
|
||||
└── geometry/projection.py
|
||||
@@ -27,8 +28,8 @@ flowchart TD
|
||||
Kind -- "Plane" --> Copy["复制 Image 与材质"]
|
||||
Copy --> Quad["创建四边形 + O Mesh Plane"]
|
||||
Kind -- "Depth Plane" --> Input["准备颜色与分析输入"]
|
||||
Input --> Job["prepare-geometry-data"]
|
||||
Job --> Field["image.0001 + field.npz"]
|
||||
Input --> Job["generate-depth-plane"]
|
||||
Job --> Field["image.0001 + image_geometry.npz"]
|
||||
Field --> Anchor["按 Empty 范围锚定相机点"]
|
||||
Anchor --> Depth["创建深度 Image + 材质 + 节点修改器"]
|
||||
Quad --> Replace["替换源 Empty"]
|
||||
@@ -45,7 +46,7 @@ flowchart TD
|
||||
|
||||
`ConvertToDepthPlane` 当前只接受静态图片。深度模型和分辨率来自扩展 Preferences：MoGe-2 使用 Resolution Level，DA3 使用按原图尺寸换算并对齐到模型 Patch 的处理分辨率。
|
||||
|
||||
`GenerateDepthPlane` 把颜色输入、分析输入和模型参数提交给 `prepare-geometry-data`。Server 将模型结果统一为 `GeometryFrame`，写出包含相机点、深度、内参、有效区域和参考深度的 `field.npz`。这一步不创建网格。 `max_input_size` 同时传入 Job；Server 通过 `media/resolution.py` 在模型推理前限制分析图长边，默认 `2048`。颜色输入仍由未缩放的原图复制，受限分析图不会成为材质 Color。
|
||||
`GenerateDepthPlane` 把当前 Image 随模型参数提交给 `generate-depth-plane`。`jobs/generate_depth_plane.py` 负责单图协议，`geometry/generate_depth.py` 负责 AI 深度生成；Server 将模型结果统一为 `GeometryFrame`，写出包含相机点、深度、内参、有效区域和参考深度的 `image_geometry.npz`。Blender 侧通过公共 `load_image_geometry()` 读取为 `ImageGeometry`，与 Cutout 使用同一协议校验。这一步不创建网格。`max_input_size` 同时传入 Job；Server 通过 `media/resolution.py` 在模型推理前限制分析图长边，默认 `2048`。颜色输入仍按当前 Image 的未缩放分辨率复制，受限分析图不会成为材质 Color。
|
||||
|
||||
Blender 响应阶段执行三件事：
|
||||
|
||||
@@ -56,3 +57,5 @@ Blender 响应阶段执行三件事：
|
||||
Depth 节点组先复用 `O Mesh Plane` 完成细分与材质，再生成背面厚度、限制正面向内穿透，最后在 Thickness 为零时删除背面和侧面。`Depth Amount` 控制位移强度；`Depth Image` 与 `Material` 是由插件填入的内部输入。
|
||||
|
||||
与 Cutout 的 Depth Surface 不同，Depth Plane 只使用锚定后的 Z 高度，不在 Geometry Nodes 中重建完整相机空间 XYZ。它强调一张图上的可编辑浮雕，而不是局部透视表面。
|
||||
|
||||
颜色材质、Normal 导入和结果对象激活分别复用 `create_image_material()`、`load_normal_result_image()` 与 `finalize_object_result()`；普通 Plane 和 Depth Plane 不再各自维护对象替换步骤。
|
||||
|
||||
@@ -1,80 +1,76 @@
|
||||
# Cutout
|
||||
# Cutout Tool
|
||||
|
||||
Cutout 从一条图片空间套索生成规则三角基础面，再交给统一的 `O Image Cutout` Geometry Nodes 修改器形成 Surface、Balloon、Depth Balloon 或 Depth Surface。公开节点组用 Shape 菜单路由到四个原有内部节点组；轮廓和 Mesh 始终由 Blender 主线程创建，Server 只在需要时准备 Refine、深度或法线数据。
|
||||
Cutout 由两个核心 Operator 组成：
|
||||
|
||||
## 对应文件
|
||||
- `SelectCutoutRegion`（`select_region.py`）把 Viewport 手势转换为图片空间 Region，并弹出 Shape 菜单。
|
||||
- `CutoutRegionToShape`（`region_to_shape.py`）准备所需数据，再构建并创建 Shape。
|
||||
|
||||
```text
|
||||
operators/cutout_tool/
|
||||
├── selection_tool.py # 套索交互、设置与 Shape 饼菜单
|
||||
├── shape_conversion.py # 本地 / Job 路径选择和任务参数
|
||||
├── build_shapes.py # 裁切贴图、Mesh、材质与修改器
|
||||
├── shape_types.py # Shape 与节点组映射
|
||||
├── mask_regions.py # Alpha 外轮廓与孔洞
|
||||
├── planar_mesh.py # 规则采样与约束三角化
|
||||
├── balloon_profile.py # Poisson / Teddy profile
|
||||
├── depth_calibration.py # Reference 与 Direction
|
||||
└── node_assets.py # Cutout 节点资产加载
|
||||
路径约定：Blender 侧文件相对 `src/anyimage/operators/cutout_tool/`，Server 侧文件相对 `src/anyimage/server/`。
|
||||
|
||||
server/
|
||||
├── jobs/prepare_geometry_data.py # Cutout / Depth Plane 数据编排
|
||||
├── selection/
|
||||
│ ├── polygon_mask.py # Polygon 栅格化与选区裁切
|
||||
│ └── cutout_images.py # Refine 与 Cutout 局部图片
|
||||
└── geometry/normal_texture.py # 相机轴法线编码
|
||||
```
|
||||
|
||||
## 路径选择
|
||||
## 执行路径
|
||||
|
||||
```mermaid
|
||||
flowchart TD
|
||||
Lasso["套索 + Shape"] --> Refine{"Refine Selection？"}
|
||||
Refine -- "是" --> Ben2["BEN2 × 1"]
|
||||
Refine -- "否" --> Selection["套索 Selection"]
|
||||
Ben2 --> Selection
|
||||
Selection --> NeedMoge{"需要 Depth 或 Normal？"}
|
||||
NeedMoge -- "否" --> Mesh["轮廓 → 规则点 → 约束三角化"]
|
||||
NeedMoge -- "是" --> Moge["MoGe-2 × 1\\n同一 Prediction"]
|
||||
Moge --> Data["Depth Field / Camera Normal"]
|
||||
Data --> Mesh
|
||||
Mesh --> Profile["计算 o_balloon"]
|
||||
Profile --> Nodes["加载 O Image Cutout 并设置 Shape"]
|
||||
Nodes --> Bake["按需烘焙最终 Tangent Normal"]
|
||||
Select["圈定 Cutout Region<br/>选择 Shape"]
|
||||
Raster["栅格化 Region"]
|
||||
Route{"需要 AI 数据？"}
|
||||
Local["使用本地 Region 数据"]
|
||||
AI["生成 Refine / Depth / Normal"]
|
||||
Ready["汇总 Shape 输入"]
|
||||
Build["构建基础 Shape"]
|
||||
Create["创建 Blender 对象"]
|
||||
|
||||
Select --> Raster
|
||||
Raster --> Route
|
||||
Route -- "否" --> Local --> Ready
|
||||
Route -- "是" --> AI --> Ready
|
||||
Ready --> Build --> Create
|
||||
```
|
||||
|
||||
Cutout 只接受静态图片。Refine Selection 是网格生成前唯一可选的选区分析：开启时执行一次 BEN2，关闭时直接使用套索 Selection。Surface 与 Balloon 在不需要 Normal 时不运行 MoGe-2；两个 Depth Shape 和任意开启 Normal 的 Shape 都统一使用 MoGe-2。Depth 与 Normal 同时启用时只执行一次推理，同一个 Prediction 同时提供相机点、深度和图片空间法线。
|
||||
## 节点说明
|
||||
|
||||
Cutout 不使用 DA3。若全局 Depth Model 当前选择 DA3，Cutout 会回退到默认 MoGe-2；选择 MoGe-2 档位时则沿用该档位和 Resolution Level。DA3 仍可用于独立的 Depth Plane 流程。
|
||||
### 圈定 Cutout Region
|
||||
|
||||
## 选择与局部数据
|
||||
`SelectCutoutRegion` 将 Lasso 路径投射到图片像素坐标，形成 `RegionPath`，然后把 Region 和用户选择的 Shape 交给 `CutoutRegionToShape`。双击图片时，Region 使用完整图片范围。
|
||||
|
||||
Cutout 复用 `ImageMaskGesture` 的采点和屏幕预览，并复用 `operators/common/image_mask.py` 的可见选区语义：最终几何范围始终是套索、图片边界与源图 `Alpha > 0.1` 区域的交集。套索采用 NONZERO 填充规则，使反向内环可以表达孔洞。`Point Spacing` 先根据完整源图最长边换算为基础 `mesh_detail`；获得裁切 Selection 后，再按 `min(1, max_input_size / crop_long_edge)` 缩放有效 `mesh_detail`。因此只有裁切图超过全局上限时网格才会变稀，Color Image 仍使用原始裁切尺寸。
|
||||
`RegionPath` 定义在 `operators/common/image_region.py`，包含图片空间路径点、反选状态和填充规则。它既供 Cutout 使用，也供 Image Tool 使用。
|
||||
|
||||
Job 路径将数据拆成三层：源颜色、最终选择和只供模型分析的 RGBA。写出最终选择前会再次以 `Alpha > 0.1` 与源图求交，因此未 Refine、Refine 和双击全图遵循相同边界。`bounds.json` 把可见选区的局部矩形映射回源图，Blender 据此恢复局部平面范围和 UV。未走 Job 时，`build_shapes.py` 在内存中完成相同的求交、裁切与坐标换算。
|
||||
### 栅格化 Region
|
||||
|
||||
## 基础网格
|
||||
`CutoutRegionToShape` 把 `RegionPath` 栅格化为 `RegionMask`。`RegionMask` 只包含局部像素权重 `values` 和它在源图中的 `bounds`，后续几何、颜色裁切和 AI 输入都以它为范围。
|
||||
|
||||
最终 Alpha 先转为带包含关系的外轮廓与孔洞。`planar_mesh.py` 在内部生成近似六角排列的规则点，并用轮廓作为约束进行 Delaunay 三角化。边界按目标间距重采样，短轮廓会提高边界采样密度，内部点仍保持用户设置的间距。
|
||||
Surface 或 Balloon 在未启用 Refine 与 Normal 时直接使用本地 Region 数据；选择 Depth Shape，或启用 Refine、Normal 时进入 AI 路径。两条路径最终都调用 `region_to_shape()`。
|
||||
|
||||
四种 Shape 共享同一张单面基础网格、UV 和 `o_balloon` 点属性。创建阶段以目标 Point Spacing 的平方作为一个采样单元面积，再除以 Image Empty 图片平面从局部到世界的面积缩放，得到修改器局部坐标中的 Cleanup 阈值。这个值只取决于工具分辨率和图片尺寸/变换，不受碎片数量或碎面面积分布影响。Balloon Profile 由 Preferences 选择 Poisson 或 Teddy；节点组只消费计算后的属性，不再重复推导轮廓。
|
||||
### 生成 AI 数据
|
||||
|
||||
## 统一修改器与四种 Shape
|
||||
`generate-region-data` Job（`jobs/generate_region_data.py`）根据请求生成所需数据：
|
||||
|
||||
创建结果时始终添加 `O Image Cutout`，并按饼菜单选择写入 Shape。节点组先统一执行 Cleanup，再由 Menu Switch 仅求值当前 Shape 的内部节点组，最后统一执行 Smooth；因此四种 Shape 都可使用 Cleanup、Smooth 和 Smooth Iterations。各分支独有输入直接连接到对应内部组，Blender 4.5 会在修改器中隐藏当前 Shape 不使用的输入。Cleanup 直接比较 Mesh Island 总面积与创建时写入的绝对面积阈值，不再统计当前网格面的中位面积。
|
||||
- `generate_selection()`（`region/generate_selection.py`）使用 BEN2 收紧 Region；
|
||||
- `generate_depth(..., normal_mode=...)`（`geometry/generate_depth.py`）使用 MoGe-2 生成 Depth，并可从同一次 Prediction 派生 Tangent 或 Object Normal。
|
||||
|
||||
| Shape | 内部节点组 | 主要几何来源 |
|
||||
Blender 传入的已是带 Region Alpha 的局部 RGBA。Depth 与 Normal 同时启用时只运行一次 MoGe-2，颜色贴图仍使用 Region 的原始裁切分辨率。
|
||||
|
||||
### 构建基础 Shape
|
||||
|
||||
`region_to_shape()` 汇总 Region、Depth 和 Normal 输入，再依次执行几何构建与 Blender 数据创建。
|
||||
|
||||
`build_base_shape()` 从 `RegionMask` 构建纯内存 `BaseShape`：先由 `region_contours.py` 提取包含外轮廓与孔洞的 `RegionContours`，再由 `triangulation.py` 按 Point Spacing 采样并约束三角化，最后使用 `balloon.py` 计算 `o_balloon`。`depth_calibration.py` 负责把 AI Depth 标定为 Blender 中使用的深度基准与方向。四种 Shape 共用这张单面基础网格。
|
||||
|
||||
### 创建 Blender 对象
|
||||
|
||||
`create_shape_object()` 创建裁切后的 Color Image、Material、Mesh Object 和统一的 `O Image Cutout` Modifier。Shape 的差异由 Modifier 中对应的几何分支表达：
|
||||
|
||||
| Shape | 形成方式 | Shape 自身所需 AI 数据 |
|
||||
| --- | --- | --- |
|
||||
| Surface | `O Image Surface` | 基础薄片与可调 Thickness |
|
||||
| Balloon | `O Image Balloon` | `o_balloon` 鼓起高度 |
|
||||
| Depth Balloon | `O Image Depth Balloon` | Balloon profile + 相对基准面的非负深度高度 |
|
||||
| Depth Surface | `O Image Depth Surface` | MoGe-2 相机点的完整投影 |
|
||||
| Surface | 保持基础 Shape 的平面轮廓，并可增加真实厚度 | 无 |
|
||||
| Balloon | 使用 `o_balloon` 从轮廓向内部平滑鼓起 | 无 |
|
||||
| Depth Balloon | 在 Balloon 上叠加相对基准面的图片深度，并沿 Depth Direction 形成浮雕 | MoGe-2 Depth |
|
||||
| Depth Surface | 使用完整相机点 XYZ 重建带透视位移的表面，并可增加厚度 | MoGe-2 Camera Points |
|
||||
|
||||
四个内部节点组不再标记为 Modifier，也不再各自执行 Cleanup。原本只在 `O Image Depth Surface` 内的 Smooth 后处理已移到公开节点组；统一节点组的 Smooth 默认值为 `1`；外层 `Smooth Iterations` 默认值为 `0`，只有用户增加迭代次数后才执行平滑。两个 Depth 内部节点组也统一使用 `Reference Depth`，总节点组只暴露一个共用输入。
|
||||
四种 Shape 共用 Region、基础三角面、UV、材质、Cleanup 和 Smooth。Cleanup 的 UI 数值放大 `100` 倍，节点内部乘以 `0.01` 还原，因此显示尺度不会改变实际清理阈值。
|
||||
|
||||
两个 Depth 形状把相机点 RGB 写入 Packed 浮点 `_depth.exr`，并以隐藏的 `Uniform Scale` 匹配 Image Empty 尺寸。未启用 Refine 时 Reference 使用固定值，Depth Balloon Direction 使用默认轴向；启用 Refine 后，`calibration_valid` 限制在模型有效区域与最终选择的交集，Reference 使用其深度第 95 百分位，Depth Balloon 还拟合局部基准方向。
|
||||
### Depth 与 Normal
|
||||
|
||||
Depth Balloon 保留图片平面 XY，只使用相机 Z 形成向外高度；Depth Surface 将完整 XYZ 映射到 Cutout 局部轴，因此能表达透视位移。两者都只采样深度 Image 的 RGB，Alpha 仅保存有效性和深度跳变信息。
|
||||
Depth Shape 根据最终 Region Alpha 选择基准：透明像素比例达到 `5%` 时从有效内容自适应计算 Base Plane Depth；不足 `5%` 时使用固定基准。Depth Balloon 还会在自适应路径中拟合 Depth Direction。
|
||||
|
||||
## 法线空间
|
||||
|
||||
MoGe-2 在 Job 阶段输出图片空间的相机轴法线预测；这一步不依赖 Blender 网格。Depth Surface 在导入时将它重排到 Cutout 对象轴并以 Object Space 使用；Surface、Balloon 与 Depth Balloon 则先创建基础网格并完成 Geometry Nodes 求值，再根据成品网格 UV、法线和切线基底烘成 Tangent Space。也就是说，网格前产生的是模型 Prediction，最终法线贴图的空间转换仍发生在网格生成之后。
|
||||
Normal 是材质层的浮雕细节，与四种 Shape 都可组合。None 使用几何法线；Tangent 以完整图片平面为切线基准；Object 直接编码到 Cutout 对象局部轴。Geometry Nodes 负责大轮廓和深度，Normal 负责表面细节。
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
# Image Tool
|
||||
|
||||
`operators/image_tool/` 对应工具栏中的 Box、Lasso、Line 与 Polyline。四个 WorkspaceTool 共用一个 `EditImageMask` Operator，把屏幕手势转换为图片 Alpha；只有 Box 与 Lasso 可以用 BEN2 细化选择。
|
||||
`operators/image_tool/` 对应工具栏中的 Box、Lasso、Line 与 Polyline。四个 WorkspaceTool 共用一个 `EditImageMask` Operator：手势先形成图片空间 Region，Region 栅格化为 Mask 后才修改图片 Alpha；只有 Box 与 Lasso 可以用 BEN2 执行 Refine Selection。
|
||||
|
||||
## 对应文件
|
||||
|
||||
@@ -20,20 +20,22 @@ operators/image_tool/
|
||||
|
||||
```mermaid
|
||||
flowchart TD
|
||||
Tool["Box / Lasso / Line / Polyline"] --> Gesture["ImageMaskGesture"]
|
||||
Gesture --> Polygon["图片像素 Polygon"]
|
||||
Polygon --> Refine{"Refine Selection?"}
|
||||
Refine -- "否" --> Mask["本地几何 Mask"]
|
||||
Refine -- "是" --> Job["refine-image-mask"]
|
||||
Tool["Box / Lasso / Line / Polyline"] --> Gesture["ImageRegionGesture"]
|
||||
Gesture --> Region["RegionPath"]
|
||||
Region --> Raster["栅格化 Region"]
|
||||
Raster --> Data["RegionMask"]
|
||||
Data --> Refine{"Refine Selection?"}
|
||||
Refine -- "否" --> Mask["Region Mask"]
|
||||
Refine -- "是" --> Input["带 Region Alpha 的裁切 RGBA"]
|
||||
Input --> Job["refine-image-mask"]
|
||||
Job --> Mask
|
||||
Mask --> Invert["可选反选"]
|
||||
Invert --> Alpha["源 Alpha × 最终 Mask"]
|
||||
Mask --> Alpha["源 Alpha × Mask"]
|
||||
Alpha --> Crop["紧裁切并写回 Empty"]
|
||||
```
|
||||
|
||||
`EditImageMask` 只接受静态图片。未启用 Refine 时，Polygon 在 Blender 侧直接栅格化；启用后，Operator 准备单图输入并提交 Job，响应阶段读取局部 Alpha 与源图 `bounds`，再映射回完整画布。反选始终在最终 Mask 上执行。
|
||||
`EditImageMask` 只接受静态图片。`ImageRegionGesture` 把屏幕路径投射为 `RegionPath`；`rasterize_region_path()` 再生成 `RegionMask`。未启用 Refine 时直接把 Region Mask 乘入源 Alpha；启用后，Operator 将 `RegionMask` 写成局部 RGBA 并提交 Job，响应阶段读取细化后的 Alpha 与 `bounds`，再映射回完整画布。
|
||||
|
||||
Box 和 Lasso 分别保存自己的 Refine 状态。Line 与 Polyline 强制关闭 Refine，只使用几何 Mask；Line 将有向线扩展为半平面，Polyline 显式闭合用户确认的顶点。
|
||||
Box 和 Lasso 分别保存自己的 Refine Selection 状态。Line 与 Polyline 强制关闭 Refine，只使用本地 Region Mask；Line 将有向线扩展为半平面 Path，Polyline 显式闭合用户确认的 Path。
|
||||
|
||||
## 手势与预览
|
||||
|
||||
@@ -43,4 +45,4 @@ Box 和 Lasso 分别保存自己的 Refine 状态。Line 与 Polyline 强制关
|
||||
|
||||
## 写回结果
|
||||
|
||||
最终像素保留源 RGB，只修改 Alpha，再按非零 Alpha 紧裁切。公共写回函数只替换当前 Empty 的 Image，并更新 `empty_display_size` 与 `empty_image_offset`，因此剩余像素继续对齐原图片平面，对象 `matrix_world` 不变。
|
||||
最终像素通过公共 `image_pixels()` / `image_rgba()` 读取，保留源 RGB，只修改 Alpha，再按非零 Alpha 紧裁切。公共像素写回函数创建结果后，通过统一的 Image Empty 替换入口处理共享引用，并更新 `empty_display_size` 与 `empty_image_offset`，因此剩余像素继续对齐原图片平面，对象 `matrix_world` 不变。
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
# Operator 实现索引
|
||||
|
||||
`src/anyimage/operators/` 负责 Blender 侧交互、Job 请求组装和结果导入。内部实现按完整功能包或顶层业务模块拆页；只提供共享能力的 `common/` 不单独成页，其职责统一放在[程序架构](../architecture.md)中说明。
|
||||
`src/anyimage/operators/` 负责 Blender 侧交互、Job 请求组装和结果导入。内部实现按完整功能包或顶层业务模块拆页；`common/` 的边界见[程序架构](../architecture.md)，实际公开入口见[Common 公共函数](../common.md)。
|
||||
|
||||
| Operator 路径 | 实现页 | 主要职责 |
|
||||
| --- | --- | --- |
|
||||
@@ -8,11 +8,11 @@
|
||||
| `operators/ai_setup.py` | [AI 环境管理](ai-setup.md) | 环境安装、模型下载与缓存释放 |
|
||||
| `operators/clipboard_image/` | [Clipboard Image](clipboard-image.md) | 系统剪贴板读取、粘贴与快捷键 |
|
||||
| `operators/convert_to_plane/` | [Convert to Plane](convert-to-plane.md) | Mesh Plane 与 Depth Plane 创建 |
|
||||
| `operators/cutout_tool/` | [Cutout Tool](cutout-tool.md) | 区域选择、轮廓生成与 Shape 构建 |
|
||||
| `operators/cutout_tool/` | [Cutout Tool](cutout-tool.md) | Region 选择、基础 Shape 构建与 Blender 对象创建 |
|
||||
| `operators/debug.py` | [Debug Operator](debug.md) | 模型接口与原始产物验证 |
|
||||
| `operators/image_tool/` | [Image Tool](image-tool.md) | Box、Lasso、Line 与 Polyline 编辑 |
|
||||
| `operators/perspective_tool/` | [Perspective Tool](perspective-tool.md) | 四点透视框与透视校正 |
|
||||
| `operators/remove_background.py` | [Remove Background](remove-background.md) | BEN2 去背与结果替换 |
|
||||
| `operators/upscale.py` | [Upscale](upscale.md) | 4× 高清与原图引用维护 |
|
||||
| `operators/upscale.py` | [Upscale](upscale.md) | 当前 Image 的 4× 高清与输入长边检查 |
|
||||
|
||||
`operators/__init__.py` 只聚合注册类，不承载独立业务流程。具体类的注册和逆序注销由扩展入口统一管理，整体边界见[程序架构](../architecture.md)。
|
||||
|
||||
@@ -10,7 +10,7 @@ operators/perspective_tool/
|
||||
└── edit_perspective.py # 四点交互、预览、Homography 与写回
|
||||
```
|
||||
|
||||
图片选择、裁切和 Empty 写回复用 `operators/common/` 的基础能力，不与 Image Mask Operator 共享业务状态。
|
||||
图片坐标投射、Alpha 裁切和 Empty 写回复用 `operators/common/` 的基础能力；四点数据始终称为 Quad，不与 Image Region 或 Image Mask Operator 共享业务状态。
|
||||
|
||||
## 矫正流程
|
||||
|
||||
@@ -29,4 +29,4 @@ flowchart TD
|
||||
|
||||
Homography 使用预乘 Alpha 插值，避免透明边缘混入黑色。输出最长边限制为 8192 px，总像素限制为 16,777,216，并按 256 行分块计算，控制中间数组大小。
|
||||
|
||||
Perspective 是 deform 工具，不提供 Refine 或反选。结果比例可能不同于原四边形的轴对齐范围，因此 Empty 写回保持中心和覆盖尺度近似一致，而不是假设当前 3D View 相机等于图片拍摄相机。
|
||||
Perspective 是 deform 工具，不提供 Refine 或反选。它通过公共 `image_pixels()` 读取供预览与最终 Warp 共用的源像素，并与 Image Mask 共用像素结果创建和 Image Empty 替换入口。结果比例可能不同于原四边形的轴对齐范围，因此 Empty 写回保持中心和覆盖尺度近似一致，而不是假设当前 3D View 相机等于图片拍摄相机。
|
||||
|
||||
@@ -8,7 +8,7 @@
|
||||
src/anyimage/
|
||||
├── operators/
|
||||
│ ├── remove_background.py # Blender 入口、任务参数与结果替换
|
||||
│ └── common/image.py # 媒体输入准备与结果 Image 创建
|
||||
│ └── common/image_data.py # 媒体输入准备与结果 Image 创建
|
||||
└── server/
|
||||
├── jobs/remove_background.py # 单图与多帧任务编排
|
||||
└── models/ben2.py # BEN2 前景适配
|
||||
@@ -28,4 +28,4 @@ flowchart TD
|
||||
|
||||
Packed 图片或内存中修改过的单图会先导出为临时输入；文件图片、视频和序列则沿用可读取的源路径。多帧输入由统一媒体层展开，结果保持连续帧命名，导入后继续沿用源 Image User 的帧起点、偏移与循环设置。
|
||||
|
||||
BEN2 推理前使用输入 Alpha 屏蔽透明区域，模型输出的前景 Alpha 再与源 Alpha 合成，RGB 保持为源图内容。响应阶段只替换当前 Empty 的 Image 数据；当旧 Image 没有其他用户和 Fake User 时，旧数据块会被清理并由结果继承名称。
|
||||
BEN2 推理前使用输入 Alpha 屏蔽透明区域，模型输出的前景 Alpha 再与源 Alpha 合成，RGB 保持为源图内容。响应阶段通过公共 Image Empty 替换入口导入结果；当旧 Image 没有其他用户和 Fake User 时，旧数据块会被清理并由结果继承名称。
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
# Upscale
|
||||
|
||||
本页对应 `src/anyimage/operators/upscale.py`。该模块读取扩展首选项中的模型与全局 AI 输入长边上限，提交 4× 高清任务，并维护高清结果到原始 Image 的引用。
|
||||
本页对应 `src/anyimage/operators/upscale.py`。该模块读取扩展首选项中的模型与全局 AI 输入长边上限，检查当前 Image 是否可处理，再提交 4× 高清任务。
|
||||
|
||||
## 对应文件
|
||||
|
||||
@@ -8,7 +8,7 @@
|
||||
src/anyimage/
|
||||
├── operators/
|
||||
│ ├── upscale.py # Blender 入口、任务参数与结果替换
|
||||
│ └── common/image.py # 媒体输入准备与结果 Image 创建
|
||||
│ └── common/image_data.py # 媒体输入准备与结果 Image 创建
|
||||
└── server/
|
||||
├── jobs/upscale.py # 单图与多帧任务编排
|
||||
├── media/resolution.py # 共用长边限制与等比缩放
|
||||
@@ -20,17 +20,18 @@ src/anyimage/
|
||||
|
||||
```mermaid
|
||||
flowchart TD
|
||||
Empty["Image Empty"] --> Source["解析原始 Image"]
|
||||
Source --> Prepare["准备媒体输入"]
|
||||
Empty["Image Empty"] --> Source["读取当前 Image"]
|
||||
Source --> Gate{"当前长边小于全局上限？"}
|
||||
Gate -- "否" --> Disabled["Upscale 置灰"]
|
||||
Gate -- "是" --> Prepare["准备媒体输入"]
|
||||
Prepare --> Job["upscale-image Job"]
|
||||
Job --> Limit["按全局输入长边上限等比缩放"]
|
||||
Limit --> RGB["分块执行 4× RGB 推理"]
|
||||
Limit --> Alpha["独立缩放 Alpha"]
|
||||
Job --> RGB["分块执行 4× RGB 推理"]
|
||||
Job --> Alpha["独立缩放 Alpha"]
|
||||
RGB --> Merge["合并 RGBA 结果"]
|
||||
Alpha --> Merge
|
||||
Merge --> Replace["导入并替换当前 Image"]
|
||||
```
|
||||
|
||||
模型以 256×256 分块处理 RGB，并带入相邻上下文以减弱块边界；Alpha 按同一受限输入尺寸单独缩放，最后合并为 RGBA。`max_input_size` 来自全局 **Maximum AI Input Size**，默认 `2048`，只约束送入模型的长边；4× 输出长边因此默认为最多 `8192`。输入尺寸不超过上限时不会发生预缩放。
|
||||
模型以 256×256 分块处理 RGB，并带入相邻上下文以减弱块边界；Alpha 不进入模型，使用 Lanczos 独立放大后合并为 RGBA。`max_input_size` 来自全局 **Maximum AI Input Size**，默认 `2048`。Operator 只允许当前 Image 长边小于该值时执行，因此正式流程不会预缩放输入，4× 输出长边小于上限的四倍。Server 仍保留相同上限作为任务边界。
|
||||
|
||||
结果以 `_upscale.png` 结尾，并通过 `Image.anyimage_upscale_source` 保存原图引用。再次对高清结果执行时，Operator 会从原图重新生成，而不是继续放大上一次输出。模型与全局输入上限在每次执行时从 Preferences 读取，再写入可序列化的隐藏属性交给 Job。Provider 资源错误会释放当前 Upscale Session，并提示降低全局输入上限或改用 CPU。
|
||||
菜单根据当前尺寸只显示预计的 4× 结果尺寸。结果通过与 Remove Background、Image Mask 和 Perspective 相同的 Image Empty 替换入口处理：原 Image 没有其他用户且未启用 Fake User 时，移除旧数据并让结果沿用原名；存在其他用户时保留原 Image，并只让当前 Empty 使用独立结果。结果不增加 Upscale 专属后缀，也不保存来源引用。后续 Upscale 与 Depth Plane 都读取当前 Image。模型与全局输入上限在每次执行时从 Preferences 读取，再写入可序列化的隐藏属性交给 Job。Provider 资源错误会释放当前 Upscale Session，并提示降低全局输入上限或改用 CPU。
|
||||
|
||||
@@ -21,16 +21,16 @@ flowchart TD
|
||||
| Job | 主要文件 | 消费方 |
|
||||
| --- | --- | --- |
|
||||
| `remove-background` | `image.0001.png` 起 | 替换 Image Empty 的颜色数据 |
|
||||
| `upscale-image` | `upscale.0001.png` 起 | 创建 `_upscale.png` Image 并记录原图引用 |
|
||||
| `refine-image-mask` | `selection-input.png`、`image.png`、`bounds.json` | Image Box / Lasso 读取 Alpha 与原图坐标 |
|
||||
| `prepare-geometry-data`：Depth Plane | `image.0001.*`、`field.npz` | 创建颜色图、深度图、材质和 Plane |
|
||||
| `prepare-geometry-data`：Cutout | `image.0001.png`、`selection.png`、`analysis.png`、`bounds.json`，以及按形状生成的 `field.npz` / `normal.png` | Blender 主线程构建局部网格与贴图 |
|
||||
| `upscale-image` | `upscale.0001.png` 起 | 按共享引用情况替换当前 Image Empty 的颜色数据，并沿用原 Image 命名语义 |
|
||||
| `refine-image-mask` | `image.png`、`bounds.json` | Image Box / Lasso 读取细化 Alpha 与原图坐标 |
|
||||
| `generate-depth-plane` | `image.0001.*`、`image_geometry.npz` | 创建颜色图、深度图、材质和 Plane |
|
||||
| `generate-region-data` | 可选的 `region.png`、`region_bounds.json`、`image_geometry.npz`、`normal.png` | Blender 主线程构建 Cutout Shape 与贴图 |
|
||||
|
||||
`prepare-geometry-data` 不输出 Blender 网格。它只准备颜色、选区和模型数组；轮廓提取、约束三角化、Geometry Nodes Modifier 与法线烘焙都在 Blender 响应阶段完成。
|
||||
两个几何 Job 都不输出 Blender 网格。`generate-depth-plane` 生成完整图片的颜色副本与深度场；`generate-region-data` 只生成请求的 AI 产物。轮廓提取、约束三角化以及 Mesh、Material 和 Geometry Nodes 创建都在 Blender 主线程完成。
|
||||
|
||||
## 深度场
|
||||
|
||||
`field.npz` 是 Depth Plane、Depth Balloon 与 Depth Surface 共用的模型中间结果：
|
||||
`image_geometry.npz` 是 Depth Plane、Depth Balloon 与 Depth Surface 共用的模型中间结果。Blender 侧统一通过 `operators/common/mesh_result.py` 的 `load_image_geometry()` 读取为 `ImageGeometry`，功能模块不分别解释字段：
|
||||
|
||||
| 字段 | 含义 |
|
||||
| --- | --- |
|
||||
@@ -38,21 +38,21 @@ flowchart TD
|
||||
| `depth` | 相机深度 |
|
||||
| `intrinsics` | 像素坐标下的 3×3 相机内参 |
|
||||
| `valid` | 模型与数值检查后的有效区域 |
|
||||
| `calibration_valid` | 用于结果定位或深度基准的区域；Cutout 时与选择相交 |
|
||||
| `calibration_valid` | 用于结果定位或深度基准的区域；Cutout 时与 Region 相交 |
|
||||
| `image_size` | 深度场宽高 |
|
||||
| `reference_depth` | 校准区域中的参考深度 |
|
||||
|
||||
Server 保留模型空间数据，不提前换算 Blender 单位。Depth Plane 在导入时根据 Image Empty 平面范围和相机内参锚定点场；Cutout 深度形状则计算 `Uniform Scale` 和局部坐标，再把相机点写入 Packed 浮点 Image 供 Geometry Nodes 采样。
|
||||
|
||||
## Cutout 三类图片
|
||||
## Cutout Region 产物
|
||||
|
||||
Cutout 把“显示内容”“几何范围”和“模型分析”分开：
|
||||
Blender 侧先把 `RegionPath` 栅格化为 `RegionMask`，再将 Region 裁切为带有效 Alpha 的临时 RGBA 输入。Server 不接收路径或 Mask 协议，只按需产生 AI 数据：
|
||||
|
||||
- `image.0001.png` 保存裁切矩形中的源颜色与原 Alpha，用作材质。
|
||||
- `selection.png` 保存最终选择 Alpha，用于轮廓与网格。
|
||||
- `analysis.png` 只保留选择内的有效 RGBA，供深度或法线模型分析。
|
||||
- 启用 Refine 时，`region.png` 保存细化后的局部 RGBA，`region_bounds.json` 保存它在输入裁切图中的范围；
|
||||
- Depth Shape 使用 `image_geometry.npz`；
|
||||
- Tangent 或 Object Normal 使用 `normal.png`。
|
||||
|
||||
`bounds.json` 把裁切矩形映射回源图像素坐标。这样 Server 可以只处理局部数据，Blender 仍能恢复局部平面范围、UV 和世界位置。
|
||||
Blender 将局部范围与原始 `RegionMask.bounds` 合成回源图坐标，并直接从源 Image 创建最终 Color Image。未启用 Refine、Depth 或 Normal 时不提交 Job。
|
||||
|
||||
## Debug 产物
|
||||
|
||||
@@ -65,7 +65,7 @@ Debug Job 保留模型适配层的直接输出，不执行正式功能的 Blende
|
||||
| `debug-ben2` | `foreground*.png` |
|
||||
| `debug-upscale` | `prediction*.png` |
|
||||
|
||||
这些文件用于验证模型、Device 和适配层。正式 Job 会继续派生选择、Alpha、深度场或可导入图片，因此两者的文件结构有意不同。
|
||||
这些文件用于验证模型、Device 和适配层。正式 Job 会继续派生 Region、Alpha、深度场或可导入图片，因此两者的文件结构有意不同。
|
||||
|
||||
## 单图与序列
|
||||
|
||||
|
||||
@@ -14,7 +14,7 @@ src/anyimage/
|
||||
├── model_manager.py # 模型目录与 Session 缓存
|
||||
├── media/input.py # 图片、视频和序列输入
|
||||
├── jobs/ # app.py 可提交的公开任务
|
||||
├── selection/ # 选区栅格化与 Cutout 图片准备
|
||||
├── region/ # BEN2 Region 细化
|
||||
├── models/ # 模型适配与 ONNX 推理
|
||||
├── geometry/ # 深度反投影与法线纹理编码
|
||||
└── outputs/ # Debug Prediction 序列化
|
||||
@@ -41,17 +41,18 @@ Runtime 在首次任务或显式启动时拉起 Server。任务在队列中串
|
||||
| `download-model`、`download-required-models` | 准备模型文件 |
|
||||
| `remove-background` | BEN2 透明前景 |
|
||||
| `upscale-image` | 4× 图片或帧序列 |
|
||||
| `refine-image-mask` | 为 Image Box / Lasso 细化选区 |
|
||||
| `prepare-geometry-data` | 为 Depth Plane 或 Cutout 准备颜色、选区、深度和法线数据 |
|
||||
| `refine-image-mask` | 为 Image Box / Lasso 细化 Region |
|
||||
| `generate-depth-plane` | 为 Depth Plane 生成颜色副本与深度场 |
|
||||
| `generate-region-data` | 按需为 Cutout Region 生成 Refine、Depth 或 Normal |
|
||||
| `debug-*` | 保存模型适配层的直接输出 |
|
||||
|
||||
## 媒体输入
|
||||
|
||||
`server/media/input.py` 把单图、视频和图片目录统一为有序帧路径，并用上下文管理器负责临时帧目录的生命周期。它还支持将颜色输入与分析输入分开：例如 Upscale 结果可以作为颜色来源，同时使用它保存的原图引用进行 Depth Plane 分析。帧展开是媒体输入的内部能力，不再暴露没有业务消费者的独立 Job。 `server/media/resolution.py` 提供默认 `2048` 的共用输入长边限制；Upscale 直接限制每帧，`prepare-geometry-data` 则在 Cutout 裁切和分析输入选择完成后限制 Depth / Normal 模型输入。该临时缩放图不参与颜色产物写入。
|
||||
`server/media/input.py` 把单图、视频和图片目录统一为有序帧路径，并用上下文管理器负责临时帧目录的生命周期。帧展开是媒体输入的内部能力，不再暴露没有业务消费者的独立 Job。`server/media/resolution.py` 提供默认 `2048` 的共用输入长边限制；Upscale Operator 只允许长边小于该值的当前 Image 进入正式任务，Depth Plane 与 Cutout 则只限制模型分析输入，不降低颜色贴图分辨率。
|
||||
|
||||
当前 `prepare-geometry-data` 只接受一帧，因此 Depth Plane 与 Cutout 都是静态图工作流。Remove Background 与 Upscale 通过同一媒体入口处理单图、视频和序列帧。
|
||||
`generate-depth-plane` 与 `generate-region-data` 只接受一帧，因此 Depth Plane 与 Cutout 都是静态图工作流。Remove Background 与 Upscale 通过同一媒体入口处理单图、视频和序列帧。
|
||||
|
||||
公开 Job 不与 Operator 一一对应：Depth Plane 和 Cutout 共享 `prepare-geometry-data`，四个 Debug Job 共享 `jobs/debug_predictions.py`。Job 只负责协议和编排；可复用的输入、选区、模型与几何数据能力分别位于 `media/`、`selection/`、`models/` 和 `geometry/`，Job 之间不互相导入实现。
|
||||
公开 Job 按稳定的计算边界划分，不按界面入口机械映射。Depth Plane 使用 `jobs/generate_depth_plane.py`，Cutout 使用 `jobs/generate_region_data.py`，四个 Debug Job 共享 `jobs/debug_predictions.py`。Job 只负责协议和编排；可复用的输入、AI Region、模型与几何数据能力分别位于 `media/`、`region/`、`models/` 和 `geometry/`，Job 之间不互相导入实现。
|
||||
|
||||
## 模型层
|
||||
|
||||
|
||||
@@ -34,11 +34,11 @@ Cutout 提供四种结果：
|
||||
| Depth Balloon | `O Image Depth Balloon` | MoGe-2 高度 |
|
||||
| Depth Surface | `O Image Depth Surface` | MoGe-2 相机点 |
|
||||
|
||||
四种 Shape 共用 `O Image Cutout` 修改器、规则三角基础面、UV 和 `o_balloon` 属性。Shape 菜单选择内部几何分支，Cleanup 在分支前执行，Smooth 在分支后执行；Point Spacing 控制网格密度，Normal 与 Refine Selection 按需启用 MoGe-2 和 BEN2。
|
||||
四种 Shape 共用 `O Image Cutout` 修改器、规则三角基础面、UV 和 `o_balloon` 属性。Shape 菜单选择内部几何分支，Cleanup 在分支前执行且使用便于调节的百倍显示值，Smooth 在分支后执行；Depth Shape 根据 Region 透明比例选择固定或自适应 Base Plane Depth，Normal 可选 None、Tangent 或 Object，Refine Selection 按需启用 BEN2。
|
||||
|
||||
### 图片结果
|
||||
|
||||
Image Mask 和 Perspective 直接更新当前 Empty 的图片并 Pack。Remove Background 输出透明颜色图或序列。Upscale 通过三档 ONNX 模型输出 4× RGBA，并保存原图引用，重复执行时从原图重新生成。
|
||||
Image Mask 和 Perspective 直接更新当前 Empty 的图片并 Pack。Remove Background 输出透明颜色图或序列。Upscale 通过三档 ONNX 模型把当前 Image 输出为 4× RGBA；当前长边达到全局输入上限时不可执行。
|
||||
|
||||
## 模型
|
||||
|
||||
|
||||
@@ -29,6 +29,7 @@ nav:
|
||||
- 内部实现:
|
||||
- 总览: internals/index.md
|
||||
- 程序架构: internals/architecture.md
|
||||
- Common 公共函数: internals/common.md
|
||||
- 基础设施:
|
||||
- Job Runtime 与 Server: internals/runtime.md
|
||||
- Job 产物: internals/outputs.md
|
||||
|
||||
@@ -66,9 +66,6 @@ def register():
|
||||
separator=True,
|
||||
)
|
||||
bpy.types.Scene.anyimage_settings = bpy.props.PointerProperty(type=AnyImageSettings)
|
||||
bpy.types.Image.anyimage_upscale_source = bpy.props.PointerProperty(
|
||||
type=bpy.types.Image
|
||||
)
|
||||
bpy.types.VIEW3D_MT_object_context_menu.prepend(draw_image_context_menu)
|
||||
bpy.types.OUTLINER_MT_object.prepend(draw_image_context_menu)
|
||||
clipboard_image.register_keymaps()
|
||||
@@ -88,7 +85,5 @@ def unregister():
|
||||
bpy.types.OUTLINER_MT_object.remove(draw_image_context_menu)
|
||||
if hasattr(bpy.types.Scene, "anyimage_settings"):
|
||||
del bpy.types.Scene.anyimage_settings
|
||||
if hasattr(bpy.types.Image, "anyimage_upscale_source"):
|
||||
del bpy.types.Image.anyimage_upscale_source
|
||||
for class_type in reversed(CLASSES):
|
||||
bpy.utils.unregister_class(class_type)
|
||||
|
||||
Binary file not shown.
@@ -1,6 +1,7 @@
|
||||
import bpy
|
||||
|
||||
from .operators.common.image import is_image_empty
|
||||
from .operators.common.image_data import is_image_empty
|
||||
from .operators.upscale import upscale_output_size
|
||||
|
||||
|
||||
class AnyImageImageMenu(bpy.types.Menu):
|
||||
@@ -53,11 +54,10 @@ def upscale_menu_label(context):
|
||||
source_object = getattr(context, "object", None)
|
||||
if not is_image_empty(source_object):
|
||||
return "Upscale"
|
||||
width, height = (
|
||||
int(value) for value in getattr(source_object.data, "size", (0, 0))
|
||||
)
|
||||
if width <= 0 or height <= 0:
|
||||
output_size = upscale_output_size(source_object.data)
|
||||
if output_size is None:
|
||||
return "Upscale"
|
||||
width, height = output_size
|
||||
return f"Upscale ({width} × {height})"
|
||||
|
||||
|
||||
|
||||
@@ -1,8 +1,6 @@
|
||||
from .remove_background import RemoveImageBackground, RunBackgroundRemoval
|
||||
from .cutout_tool import (
|
||||
ActivateCutoutTool,
|
||||
ConvertCutoutShape,
|
||||
GenerateCutoutShape,
|
||||
CutoutRegionToShape,
|
||||
SelectCutoutRegion,
|
||||
)
|
||||
from .debug import DebugBen2, DebugDa3, DebugMoge2, DebugUpscale
|
||||
@@ -33,10 +31,8 @@ CLASSES = (
|
||||
DebugBen2,
|
||||
DebugUpscale,
|
||||
GenerateDepthPlane,
|
||||
GenerateCutoutShape,
|
||||
CutoutRegionToShape,
|
||||
SelectCutoutRegion,
|
||||
ConvertCutoutShape,
|
||||
ActivateCutoutTool,
|
||||
RunBackgroundRemoval,
|
||||
ClearModels,
|
||||
*IMAGE_TOOL_CLASSES,
|
||||
|
||||
@@ -8,7 +8,7 @@ from ..properties import (
|
||||
model_download_size,
|
||||
model_record,
|
||||
)
|
||||
from .common.operator import HIDDEN, require_environment
|
||||
from .common.ai_job import require_environment
|
||||
|
||||
|
||||
def format_download_size(size):
|
||||
@@ -128,7 +128,7 @@ class DownloadModel(JobOperatorBase, bpy.types.Operator):
|
||||
bl_label = "Download Model"
|
||||
job_type = "download-model"
|
||||
|
||||
model: bpy.props.StringProperty(options=HIDDEN)
|
||||
model: bpy.props.StringProperty(options={"HIDDEN", "SKIP_SAVE"})
|
||||
|
||||
def request(self, _context):
|
||||
require_environment()
|
||||
|
||||
@@ -1 +1 @@
|
||||
"""Shared Blender-side image, material, normal, and operator helpers."""
|
||||
"""Shared Blender-side image data, interaction, mesh result, and AI Job helpers."""
|
||||
|
||||
@@ -1,4 +1,3 @@
|
||||
import shutil
|
||||
from pathlib import Path
|
||||
|
||||
import bpy
|
||||
@@ -6,24 +5,22 @@ import bpy
|
||||
from ...preferences import addon_preferences
|
||||
from ...properties import ai_ready, model_ready, model_record
|
||||
from ...runtime import runtime
|
||||
from .image import extract_frame_range
|
||||
|
||||
BEN2_IMAGE_SUFFIXES = {".jpg", ".jpeg", ".png", ".webp"}
|
||||
BEN2_VIDEO_SUFFIXES = {
|
||||
".mp4", ".mov", ".avi", ".mkv", ".webm", ".m4v", ".mts", ".mpg", ".mpeg"
|
||||
}
|
||||
HIDDEN = {"HIDDEN", "SKIP_SAVE"}
|
||||
|
||||
|
||||
def input_path(value):
|
||||
def require_input_path(value):
|
||||
path = Path(bpy.path.abspath(value))
|
||||
if path.is_file() or path.is_dir():
|
||||
return path
|
||||
raise ValueError("Select a valid image, video, or frame directory")
|
||||
|
||||
|
||||
def background_input_path(value):
|
||||
path = input_path(value)
|
||||
def require_background_input_path(value):
|
||||
path = require_input_path(value)
|
||||
if path.is_dir() or path.suffix.lower() in (
|
||||
BEN2_IMAGE_SUFFIXES | BEN2_VIDEO_SUFFIXES
|
||||
):
|
||||
@@ -33,11 +30,6 @@ def background_input_path(value):
|
||||
)
|
||||
|
||||
|
||||
def source_start_frame(source_object_name):
|
||||
source_object = bpy.data.objects.get(source_object_name)
|
||||
return extract_frame_range(source_object)[0] if source_object else 1
|
||||
|
||||
|
||||
def production_device():
|
||||
return getattr(addon_preferences(), "device", "AUTO")
|
||||
|
||||
@@ -62,35 +54,6 @@ def require_model(model_key):
|
||||
return model
|
||||
|
||||
|
||||
def cleanup_temporary_input(path, delete_input):
|
||||
if not delete_input or not path:
|
||||
return
|
||||
temporary_path = Path(path)
|
||||
try:
|
||||
if temporary_path.is_dir():
|
||||
shutil.rmtree(temporary_path, ignore_errors=True)
|
||||
return
|
||||
temporary_path.unlink(missing_ok=True)
|
||||
temporary_path.parent.rmdir()
|
||||
except OSError:
|
||||
pass
|
||||
|
||||
|
||||
def open_job_directory(directory):
|
||||
directory = Path(directory)
|
||||
if not directory.is_dir():
|
||||
raise RuntimeError(f"Job directory does not exist: {directory}")
|
||||
result = bpy.ops.wm.path_open(filepath=str(directory))
|
||||
if "FINISHED" not in result:
|
||||
raise RuntimeError(f"Unable to open job directory: {directory}")
|
||||
|
||||
|
||||
def push_undo(message):
|
||||
result = bpy.ops.ed.undo_push(message=message)
|
||||
if "FINISHED" not in result:
|
||||
raise RuntimeError(f"Unable to create the {message} Undo step")
|
||||
|
||||
|
||||
def da3_parameters(settings, path, model, process_res, start_frame):
|
||||
return {
|
||||
"input": str(path),
|
||||
@@ -5,9 +5,6 @@ from pathlib import Path
|
||||
|
||||
import bpy
|
||||
|
||||
DA3_PATCH_SIZE = 14
|
||||
DA3_MIN_PROCESS_RES = 252
|
||||
DA3_MAX_PROCESS_RES = 1008
|
||||
IMAGE_NAME_SUFFIXES = (
|
||||
".jpeg",
|
||||
".jpg",
|
||||
@@ -44,6 +41,33 @@ def is_image_empty(candidate):
|
||||
)
|
||||
|
||||
|
||||
def require_image_empty(object_name):
|
||||
"""Return a named Image Empty or raise when it is no longer available."""
|
||||
source_object = bpy.data.objects.get(object_name)
|
||||
if not is_image_empty(source_object):
|
||||
raise RuntimeError("The source Image Empty is no longer available")
|
||||
return source_object
|
||||
|
||||
|
||||
def image_pixels(image):
|
||||
"""Read one Blender Image into a flat float32 pixel array."""
|
||||
import numpy as np
|
||||
|
||||
pixels = np.empty(len(image.pixels), dtype=np.float32)
|
||||
image.pixels.foreach_get(pixels)
|
||||
return pixels
|
||||
|
||||
|
||||
def image_rgba(image):
|
||||
"""Read one Blender Image into a top-down RGBA float32 array."""
|
||||
import numpy as np
|
||||
|
||||
width, height = (int(value) for value in image.size)
|
||||
if width <= 0 or height <= 0:
|
||||
raise ValueError("The source image has no readable dimensions")
|
||||
return np.flipud(image_pixels(image).reshape((height, width, 4)))
|
||||
|
||||
|
||||
|
||||
def is_animated_image(image):
|
||||
return getattr(image, "source", None) in {"MOVIE", "SEQUENCE"}
|
||||
@@ -95,6 +119,11 @@ def default_extract_frames(source_object):
|
||||
return extract_frame_range(source_object)[1]
|
||||
|
||||
|
||||
def source_start_frame(source_object_name):
|
||||
source_object = bpy.data.objects.get(source_object_name)
|
||||
return extract_frame_range(source_object)[0] if source_object else 1
|
||||
|
||||
|
||||
def image_path(image):
|
||||
filepath_from_user = getattr(image, "filepath_from_user", None)
|
||||
filepath = (
|
||||
@@ -113,14 +142,17 @@ def image_is_packed(image):
|
||||
return bool(getattr(image, "packed_files", ()))
|
||||
|
||||
|
||||
def remove_temporary_input(path):
|
||||
"""Remove a temporary input file or frame directory."""
|
||||
def cleanup_image_input(path, temporary):
|
||||
"""Remove a prepared input and its temporary container when requested."""
|
||||
if not temporary or not path:
|
||||
return
|
||||
path = Path(path)
|
||||
try:
|
||||
if path.is_dir():
|
||||
shutil.rmtree(path, ignore_errors=True)
|
||||
else:
|
||||
path.unlink(missing_ok=True)
|
||||
path.parent.rmdir()
|
||||
except OSError:
|
||||
pass
|
||||
|
||||
@@ -149,62 +181,10 @@ def normal_data_name(source_object):
|
||||
return f"{image_base_name(source_object.data)}_normal.png"
|
||||
|
||||
|
||||
def copy_depth_image(source_object):
|
||||
"""Return a renamed copy of the source image for use as the depth map."""
|
||||
depth_image = source_object.data.copy()
|
||||
depth_image.name = depth_data_name(source_object)
|
||||
return depth_image
|
||||
|
||||
|
||||
def color_data_name(source_object):
|
||||
return f"{image_base_name(source_object.data)}_color.png"
|
||||
|
||||
|
||||
def material_data_name(source_object):
|
||||
return image_base_name(source_object.data)
|
||||
|
||||
|
||||
def scaled_process_resolution(image, scale_percent):
|
||||
"""Return the scaled long edge clamped and aligned for DA3 inference."""
|
||||
width, height = image.size
|
||||
if width <= 0 or height <= 0:
|
||||
raise ValueError("The source image has no readable dimensions")
|
||||
if scale_percent < 1.0 or scale_percent > 100.0:
|
||||
raise ValueError("Resolution Scale must be between 1% and 100%")
|
||||
process_res = max(width, height) * scale_percent / 100.0
|
||||
process_res = max(DA3_MIN_PROCESS_RES, process_res)
|
||||
process_res = min(DA3_MAX_PROCESS_RES, process_res)
|
||||
return int(process_res / DA3_PATCH_SIZE + 0.5) * DA3_PATCH_SIZE
|
||||
|
||||
|
||||
def process_resolution_scale_property():
|
||||
return bpy.props.FloatProperty(
|
||||
name="Resolution Scale",
|
||||
description=(
|
||||
"Scale the source image long edge for DA3 processing; the result "
|
||||
"is limited to 252-1008 px and aligned to a multiple of 14"
|
||||
),
|
||||
default=100.0,
|
||||
min=1.0,
|
||||
max=100.0,
|
||||
step=100,
|
||||
precision=0,
|
||||
subtype="PERCENTAGE",
|
||||
)
|
||||
|
||||
|
||||
def draw_process_resolution(layout, owner, image):
|
||||
process_res = scaled_process_resolution(
|
||||
image,
|
||||
owner.process_res_factor,
|
||||
)
|
||||
layout.prop(
|
||||
owner,
|
||||
"process_res_factor",
|
||||
text=f"Resolution Scale ({process_res} px)",
|
||||
)
|
||||
|
||||
|
||||
def save_packed_image(image, _scene):
|
||||
output_path = Path(tempfile.mkdtemp(prefix="anyimage-input-")) / "input.png"
|
||||
original_file_format = image.file_format
|
||||
@@ -280,40 +260,42 @@ def prepare_image_input(image, scene):
|
||||
return source_path, False
|
||||
|
||||
|
||||
def analysis_source_image(image):
|
||||
"""Return the oldest available source behind an upscaled image."""
|
||||
current = image
|
||||
visited = set()
|
||||
while current is not None and id(current) not in visited:
|
||||
visited.add(id(current))
|
||||
source = getattr(current, "anyimage_upscale_source", None)
|
||||
if source is None or source is current:
|
||||
break
|
||||
current = source
|
||||
return current
|
||||
def prepare_region_input(image, region_mask):
|
||||
"""Write one cropped RGBA Region input owned by its temporary directory."""
|
||||
import numpy as np
|
||||
|
||||
|
||||
def prepare_image_analysis_inputs(image, scene):
|
||||
"""Prepare display and analysis inputs, reusing one path when possible."""
|
||||
source_path, delete_input = prepare_image_input(image, scene)
|
||||
analysis_image = analysis_source_image(image)
|
||||
if analysis_image is image:
|
||||
return source_path, delete_input, source_path, False
|
||||
left, top, right, bottom = region_mask.bounds
|
||||
rgba = image_rgba(image)
|
||||
cropped = rgba[top:bottom, left:right].copy()
|
||||
if cropped.shape[:2] != region_mask.values.shape:
|
||||
raise ValueError("The Region does not match the source image")
|
||||
cropped[:, :, 3] *= region_mask.values
|
||||
cropped[cropped[:, :, 3] <= 0.0, :3] = 0.0
|
||||
output_directory = Path(tempfile.mkdtemp(prefix="anyimage-region-"))
|
||||
output_path = output_directory / "input.png"
|
||||
temporary_image = bpy.data.images.new(
|
||||
"AnyImage Region Input",
|
||||
width=right - left,
|
||||
height=bottom - top,
|
||||
alpha=True,
|
||||
)
|
||||
original_format = temporary_image.file_format
|
||||
try:
|
||||
analysis_path, delete_analysis_input = prepare_image_input(
|
||||
analysis_image,
|
||||
scene,
|
||||
)
|
||||
except Exception:
|
||||
if delete_input:
|
||||
remove_temporary_input(source_path)
|
||||
temporary_image.colorspace_settings.name = image.colorspace_settings.name
|
||||
temporary_image.alpha_mode = "STRAIGHT"
|
||||
temporary_image.pixels.foreach_set(np.flipud(cropped).ravel())
|
||||
temporary_image.file_format = "PNG"
|
||||
temporary_image.save(filepath=str(output_path), save_copy=True)
|
||||
except (OSError, RuntimeError):
|
||||
cleanup_image_input(output_path, True)
|
||||
raise
|
||||
return (
|
||||
source_path,
|
||||
delete_input,
|
||||
analysis_path,
|
||||
delete_analysis_input,
|
||||
)
|
||||
finally:
|
||||
temporary_image.file_format = original_format
|
||||
bpy.data.images.remove(temporary_image, do_unlink=True)
|
||||
if not output_path.is_file():
|
||||
cleanup_image_input(output_path, True)
|
||||
raise RuntimeError("Blender did not export the Region input")
|
||||
return output_path
|
||||
|
||||
|
||||
def configure_image_user(
|
||||
@@ -388,31 +370,98 @@ def force_sequence_source(image):
|
||||
return image
|
||||
|
||||
|
||||
def create_sequence_image(name, frame_directory, frame_prefix="frame."):
|
||||
"""Load a numbered frame directory as a SEQUENCE image datablock."""
|
||||
frame_paths = sorted_frame_paths(frame_directory, frame_prefix)
|
||||
if not frame_paths:
|
||||
raise RuntimeError(f"No image frames found in: {frame_directory}")
|
||||
try:
|
||||
image = bpy.data.images.load(
|
||||
str(frame_paths[0]),
|
||||
check_existing=False,
|
||||
)
|
||||
force_sequence_source(image)
|
||||
except (OSError, RuntimeError) as error:
|
||||
if "image" in locals():
|
||||
bpy.data.images.remove(image, do_unlink=True)
|
||||
raise RuntimeError("Unable to load the frame sequence") from error
|
||||
image.name = name
|
||||
pack_sequence_if_possible(image, len(frame_paths))
|
||||
def load_normal_result_image(path, source_object):
|
||||
"""Load, name, configure, and pack a tangent normal image."""
|
||||
image = bpy.data.images.load(str(path), check_existing=False)
|
||||
image.name = normal_data_name(source_object)
|
||||
image.colorspace_settings.name = "Non-Color"
|
||||
image.pack()
|
||||
return image
|
||||
|
||||
|
||||
def create_pixel_result_image(source_image, pixels, output_size, suffix=""):
|
||||
"""Create a packed Image datablock from edited RGBA pixels."""
|
||||
output_width, output_height = (int(value) for value in output_size)
|
||||
if output_width <= 0 or output_height <= 0:
|
||||
raise ValueError("The edited image dimensions must be positive")
|
||||
if len(pixels) != output_width * output_height * 4:
|
||||
raise ValueError("The edited pixels do not match the image dimensions")
|
||||
image = bpy.data.images.new(
|
||||
f"{image_base_name(source_image)}{suffix}",
|
||||
width=output_width,
|
||||
height=output_height,
|
||||
alpha=True,
|
||||
)
|
||||
try:
|
||||
image.colorspace_settings.name = source_image.colorspace_settings.name
|
||||
image.alpha_mode = "STRAIGHT"
|
||||
image.pixels.foreach_set(pixels)
|
||||
image.pack()
|
||||
image.update()
|
||||
except Exception:
|
||||
bpy.data.images.remove(image, do_unlink=True)
|
||||
raise
|
||||
return image
|
||||
|
||||
def create_color_image(
|
||||
|
||||
def replace_empty_image(source_object, result_image, *, placement_bounds=None):
|
||||
"""Replace one Empty image, retaining shared source images."""
|
||||
source_image = source_object.data
|
||||
source_name = source_image.name
|
||||
original_fake_user = source_image.use_fake_user
|
||||
image_user = getattr(source_object, "image_user", None)
|
||||
frame_count = (
|
||||
video_total_frames(result_image)
|
||||
if is_animated_image(result_image)
|
||||
else None
|
||||
)
|
||||
alignment = (
|
||||
empty_display_alignment(
|
||||
source_object,
|
||||
tuple(source_image.size),
|
||||
tuple(result_image.size),
|
||||
placement_bounds,
|
||||
)
|
||||
if placement_bounds is not None
|
||||
else None
|
||||
)
|
||||
if frame_count is not None:
|
||||
(
|
||||
original_frame_start,
|
||||
_original_frame_offset,
|
||||
_original_frame_duration,
|
||||
original_use_cyclic,
|
||||
) = image_user_settings(source_object)
|
||||
|
||||
source_image.use_fake_user = True
|
||||
try:
|
||||
source_object.data = result_image
|
||||
if frame_count is not None:
|
||||
configure_image_user(
|
||||
image_user,
|
||||
frame_count,
|
||||
frame_start=original_frame_start,
|
||||
use_cyclic=original_use_cyclic,
|
||||
)
|
||||
if alignment is not None:
|
||||
source_object.empty_display_size = alignment[0]
|
||||
source_object.empty_image_offset = alignment[1]
|
||||
except Exception:
|
||||
source_object.data = source_image
|
||||
source_image.use_fake_user = original_fake_user
|
||||
raise
|
||||
|
||||
source_image.use_fake_user = original_fake_user
|
||||
if not original_fake_user and int(source_image.users) == 0:
|
||||
bpy.data.images.remove(source_image, do_unlink=True)
|
||||
result_image.name = source_name
|
||||
return result_image
|
||||
|
||||
|
||||
|
||||
def load_color_result_image(
|
||||
source_object,
|
||||
output_directory,
|
||||
assign_to_source=True,
|
||||
*,
|
||||
frame_prefix="image.",
|
||||
):
|
||||
@@ -439,39 +488,20 @@ def create_color_image(
|
||||
bpy.data.images.remove(result_image, do_unlink=True)
|
||||
raise RuntimeError("Unable to load the color result") from error
|
||||
|
||||
original_fake_user = source_image.use_fake_user
|
||||
try:
|
||||
result_name = color_data_name(source_object)
|
||||
if animated:
|
||||
pack_sequence_if_possible(result_image, len(color_paths))
|
||||
else:
|
||||
result_image.pack()
|
||||
if assign_to_source:
|
||||
image_user = getattr(source_object, "image_user", None)
|
||||
(
|
||||
original_frame_start,
|
||||
_original_frame_offset,
|
||||
_original_frame_duration,
|
||||
original_use_cyclic,
|
||||
) = image_user_settings(source_object)
|
||||
source_image.use_fake_user = True
|
||||
source_object.data = result_image
|
||||
if animated:
|
||||
configure_image_user(
|
||||
image_user,
|
||||
len(color_paths),
|
||||
frame_start=original_frame_start,
|
||||
use_cyclic=original_use_cyclic,
|
||||
)
|
||||
except (OSError, RuntimeError) as error:
|
||||
source_image.use_fake_user = original_fake_user
|
||||
result_image.name = result_name
|
||||
except (OSError, RuntimeError, TypeError, ValueError) as error:
|
||||
bpy.data.images.remove(result_image, do_unlink=True)
|
||||
raise RuntimeError("Unable to prepare the color result") from error
|
||||
result_image.name = result_name
|
||||
return result_image
|
||||
|
||||
|
||||
def plane_bounds(source_object):
|
||||
def image_empty_bounds(source_object):
|
||||
image_width, image_height = source_object.data.size
|
||||
if image_width <= 0 or image_height <= 0:
|
||||
raise ValueError("The source image has no readable dimensions")
|
||||
@@ -494,3 +524,47 @@ def plane_bounds(source_object):
|
||||
y_min,
|
||||
y_min + height,
|
||||
)
|
||||
|
||||
|
||||
def empty_display_alignment(
|
||||
source_object,
|
||||
source_size,
|
||||
output_size,
|
||||
placement_bounds,
|
||||
):
|
||||
"""Return the display size and offset for an edited Image Empty."""
|
||||
source_width, source_height = source_size
|
||||
output_width, output_height = output_size
|
||||
left, top, right, bottom = placement_bounds
|
||||
if (
|
||||
source_width <= 0
|
||||
or source_height <= 0
|
||||
or output_width <= 0
|
||||
or output_height <= 0
|
||||
or right <= left
|
||||
or bottom <= top
|
||||
):
|
||||
raise ValueError("The edited image placement is invalid")
|
||||
x_min, x_max, y_min, y_max = image_empty_bounds(source_object)
|
||||
local_width = x_max - x_min
|
||||
local_height = y_max - y_min
|
||||
desired_x_min = x_min + left * local_width / source_width
|
||||
desired_x_max = x_min + right * local_width / source_width
|
||||
desired_y_max = y_max - top * local_height / source_height
|
||||
desired_y_min = y_max - bottom * local_height / source_height
|
||||
desired_width = desired_x_max - desired_x_min
|
||||
desired_height = desired_y_max - desired_y_min
|
||||
display_size = max(desired_width, desired_height)
|
||||
aspect = output_width / output_height
|
||||
if aspect >= 1.0:
|
||||
display_width = display_size
|
||||
display_height = display_size / aspect
|
||||
else:
|
||||
display_width = display_size * aspect
|
||||
display_height = display_size
|
||||
center_x = (desired_x_min + desired_x_max) * 0.5
|
||||
center_y = (desired_y_min + desired_y_max) * 0.5
|
||||
return display_size, (
|
||||
center_x / display_width - 0.5,
|
||||
center_y / display_height - 0.5,
|
||||
)
|
||||
@@ -1,567 +0,0 @@
|
||||
import json
|
||||
import math
|
||||
|
||||
import bpy
|
||||
|
||||
from ...properties import (
|
||||
BEN2_MODEL_KEY,
|
||||
ai_ready,
|
||||
ai_setup_label,
|
||||
ai_status,
|
||||
)
|
||||
from .operator import production_device, require_environment, require_model
|
||||
from .image import image_base_name, is_image_empty, plane_bounds
|
||||
|
||||
|
||||
class ImageEditWarning(ValueError):
|
||||
pass
|
||||
|
||||
|
||||
def report_image_edit_exception(operator, error):
|
||||
level = "WARNING" if isinstance(error, ImageEditWarning) else "ERROR"
|
||||
operator.report({level}, str(error))
|
||||
|
||||
|
||||
def image_edit_poll(context):
|
||||
return context.area is not None and context.area.type == "VIEW_3D"
|
||||
|
||||
|
||||
def _restore_selection(context, active_object, selected_objects):
|
||||
bpy.ops.object.select_all(action="DESELECT")
|
||||
for selected in selected_objects:
|
||||
if bpy.data.objects.get(selected.name) is not None:
|
||||
selected.select_set(True)
|
||||
if (
|
||||
active_object is not None
|
||||
and bpy.data.objects.get(active_object.name) is not None
|
||||
):
|
||||
context.view_layer.objects.active = active_object
|
||||
|
||||
|
||||
def resolve_image_edit_click(context, event, report):
|
||||
"""Resolve the first click into source selection or image editing."""
|
||||
active_object = context.active_object
|
||||
selected_objects = tuple(context.selected_objects)
|
||||
try:
|
||||
bpy.ops.view3d.select(
|
||||
location=(event.mouse_region_x, event.mouse_region_y),
|
||||
extend=False,
|
||||
deselect_all=False,
|
||||
)
|
||||
except RuntimeError:
|
||||
pass
|
||||
clicked_object = context.active_object
|
||||
if is_image_empty(clicked_object) and clicked_object != active_object:
|
||||
return clicked_object, False
|
||||
if is_image_empty(clicked_object):
|
||||
return clicked_object, True
|
||||
if is_image_empty(active_object):
|
||||
_restore_selection(context, active_object, selected_objects)
|
||||
return active_object, True
|
||||
report({"WARNING"}, "Click an Image Empty to use an Image tool")
|
||||
return None, False
|
||||
|
||||
|
||||
def active_view3d_tool_id(context):
|
||||
tools = getattr(getattr(context, "workspace", None), "tools", None)
|
||||
from_mode = getattr(tools, "from_space_view3d_mode", None)
|
||||
if not callable(from_mode):
|
||||
return None
|
||||
try:
|
||||
tool = from_mode(getattr(context, "mode", "OBJECT"), create=False)
|
||||
except (RuntimeError, TypeError):
|
||||
return None
|
||||
return getattr(tool, "idname", None)
|
||||
|
||||
|
||||
def polygon_bounds(polygon):
|
||||
import numpy as np
|
||||
|
||||
points = np.asarray(polygon, dtype=np.float64)
|
||||
if points.ndim != 2 or points.shape[0] < 3 or points.shape[1] != 2:
|
||||
raise ValueError("The selection does not define a valid image region")
|
||||
if not np.isfinite(points).all():
|
||||
raise ValueError("The selection contains invalid image coordinates")
|
||||
left = int(math.floor(float(points[:, 0].min())))
|
||||
top = int(math.floor(float(points[:, 1].min())))
|
||||
right = int(math.ceil(float(points[:, 0].max())))
|
||||
bottom = int(math.ceil(float(points[:, 1].max())))
|
||||
if right <= left or bottom <= top:
|
||||
raise ValueError("The selection does not define a valid image region")
|
||||
return left, top, right, bottom
|
||||
|
||||
|
||||
def crop_pixels_to_alpha(pixels, bounds):
|
||||
"""Crop Blender-order RGBA pixels to their nonzero Alpha bounds."""
|
||||
import numpy as np
|
||||
|
||||
left, top, right, bottom = bounds
|
||||
width = right - left
|
||||
height = bottom - top
|
||||
rgba = np.flipud(
|
||||
np.asarray(pixels, dtype=np.float32).reshape((height, width, 4))
|
||||
)
|
||||
visible_y, visible_x = np.nonzero(rgba[:, :, 3] > 0.0)
|
||||
if not len(visible_x):
|
||||
raise ImageEditWarning("The edit contains no visible image pixels")
|
||||
content_left = int(visible_x.min())
|
||||
content_top = int(visible_y.min())
|
||||
content_right = int(visible_x.max()) + 1
|
||||
content_bottom = int(visible_y.max()) + 1
|
||||
output = rgba[content_top:content_bottom, content_left:content_right].copy()
|
||||
return np.flipud(output).ravel(), (
|
||||
left + content_left,
|
||||
top + content_top,
|
||||
left + content_right,
|
||||
top + content_bottom,
|
||||
)
|
||||
|
||||
|
||||
def empty_display_alignment(
|
||||
source_object, source_size, output_size, placement_bounds
|
||||
):
|
||||
"""Calculate image canvas placement without changing the Empty transform."""
|
||||
source_width, source_height = (float(value) for value in source_size)
|
||||
output_width, output_height = (float(value) for value in output_size)
|
||||
left, top, right, bottom = (float(value) for value in placement_bounds)
|
||||
x_min, x_max, y_min, y_max = plane_bounds(source_object)
|
||||
local_width = x_max - x_min
|
||||
local_height = y_max - y_min
|
||||
desired_x_min = x_min + left * local_width / source_width
|
||||
desired_x_max = x_min + right * local_width / source_width
|
||||
desired_y_max = y_max - top * local_height / source_height
|
||||
desired_y_min = y_max - bottom * local_height / source_height
|
||||
desired_width = desired_x_max - desired_x_min
|
||||
desired_height = desired_y_max - desired_y_min
|
||||
display_size = max(desired_width, desired_height)
|
||||
aspect = output_width / output_height
|
||||
if aspect >= 1.0:
|
||||
display_width = display_size
|
||||
display_height = display_size / aspect
|
||||
else:
|
||||
display_width = display_size * aspect
|
||||
display_height = display_size
|
||||
center_x = (desired_x_min + desired_x_max) * 0.5
|
||||
center_y = (desired_y_min + desired_y_max) * 0.5
|
||||
return display_size, (
|
||||
center_x / display_width - 0.5,
|
||||
center_y / display_height - 0.5,
|
||||
)
|
||||
|
||||
|
||||
|
||||
def write_image_pixels(image, pixels):
|
||||
image.alpha_mode = "STRAIGHT"
|
||||
image.pixels.foreach_set(pixels)
|
||||
image.pack()
|
||||
image.update()
|
||||
|
||||
|
||||
def replace_empty_image_pixels(
|
||||
source_object, pixels, output_size, placement_bounds, suffix=""
|
||||
):
|
||||
source_image = source_object.data
|
||||
source_size = tuple(source_image.size)
|
||||
output_width, output_height = (int(value) for value in output_size)
|
||||
if output_width <= 0 or output_height <= 0:
|
||||
raise ValueError("The edited image dimensions must be positive")
|
||||
if len(pixels) != output_width * output_height * 4:
|
||||
raise ValueError("The edited pixels do not match the image dimensions")
|
||||
display_size, image_offset = empty_display_alignment(
|
||||
source_object,
|
||||
source_size,
|
||||
(output_width, output_height),
|
||||
placement_bounds,
|
||||
)
|
||||
image = bpy.data.images.new(
|
||||
f"{image_base_name(source_image)}{suffix}",
|
||||
width=output_width,
|
||||
height=output_height,
|
||||
alpha=True,
|
||||
)
|
||||
try:
|
||||
image.colorspace_settings.name = source_image.colorspace_settings.name
|
||||
write_image_pixels(image, pixels)
|
||||
except Exception:
|
||||
bpy.data.images.remove(image, do_unlink=True)
|
||||
raise
|
||||
source_name = source_image.name
|
||||
source_object.data = image
|
||||
if not source_image.use_fake_user and int(source_image.users) == 0:
|
||||
bpy.data.images.remove(source_image)
|
||||
image.name = source_name
|
||||
source_object.empty_display_size = display_size
|
||||
source_object.empty_image_offset = image_offset
|
||||
return image
|
||||
|
||||
|
||||
def image_mask_refine_request(input_path, selection_data):
|
||||
require_environment()
|
||||
require_model(BEN2_MODEL_KEY)
|
||||
polygon = json.loads(selection_data)
|
||||
polygon_bounds(polygon)
|
||||
return {
|
||||
"input": str(input_path),
|
||||
"polygon": polygon,
|
||||
"model": BEN2_MODEL_KEY,
|
||||
"device": production_device().lower(),
|
||||
}
|
||||
|
||||
|
||||
def refine_selection_setting_name(operator):
|
||||
if getattr(operator, "active_tool_id", None) == "anyimage.cutout_lasso":
|
||||
return "cutout_refine_selection"
|
||||
return {
|
||||
"BOX": "image_box_refine_selection",
|
||||
"LASSO": "image_lasso_refine_selection",
|
||||
}.get(operator.gesture)
|
||||
|
||||
|
||||
def apply_image_mask_settings(operator, context):
|
||||
settings = context.scene.anyimage_settings
|
||||
setting_name = refine_selection_setting_name(operator)
|
||||
operator.refine_selection = (
|
||||
setting_name is not None
|
||||
and ai_ready()
|
||||
and getattr(settings, setting_name)
|
||||
)
|
||||
|
||||
|
||||
def draw_image_mask_operator_settings(layout, operator):
|
||||
if operator.gesture not in {"BOX", "LASSO"}:
|
||||
layout.prop(operator, "invert")
|
||||
return
|
||||
status = ai_status()
|
||||
row = layout.row()
|
||||
row.enabled = status["ready"]
|
||||
row.prop(operator, "refine_selection")
|
||||
layout.prop(operator, "invert")
|
||||
if not status["ready"]:
|
||||
layout.operator(
|
||||
"anyimage.setup_ai_environment",
|
||||
text=ai_setup_label(status),
|
||||
icon="IMPORT",
|
||||
)
|
||||
|
||||
|
||||
def draw_image_mask_settings(context, layout, setting_name):
|
||||
settings = context.scene.anyimage_settings
|
||||
status = ai_status()
|
||||
row = layout.row()
|
||||
row.enabled = status["ready"]
|
||||
row.prop(settings, setting_name)
|
||||
if not status["ready"]:
|
||||
layout.operator(
|
||||
"anyimage.setup_ai_environment",
|
||||
text=ai_setup_label(status),
|
||||
icon="IMPORT",
|
||||
)
|
||||
|
||||
|
||||
def dashed_line_segments(vertices, dash_length=4.0, gap_length=4.0):
|
||||
if dash_length <= 0.0 or gap_length < 0.0:
|
||||
raise ValueError("Dash length must be positive and gap length cannot be negative")
|
||||
pattern_length = dash_length + gap_length
|
||||
segments = []
|
||||
distance_offset = 0.0
|
||||
for start, end in zip(vertices, vertices[1:]):
|
||||
start_x, start_y = (float(value) for value in start)
|
||||
end_x, end_y = (float(value) for value in end)
|
||||
segment_length = math.hypot(end_x - start_x, end_y - start_y)
|
||||
if segment_length <= 0.0:
|
||||
continue
|
||||
direction_x = (end_x - start_x) / segment_length
|
||||
direction_y = (end_y - start_y) / segment_length
|
||||
position = 0.0
|
||||
while position < segment_length:
|
||||
phase = (distance_offset + position) % pattern_length
|
||||
if phase < dash_length:
|
||||
run_length = min(dash_length - phase, segment_length - position)
|
||||
segments.extend(
|
||||
(
|
||||
(
|
||||
start_x + direction_x * position,
|
||||
start_y + direction_y * position,
|
||||
),
|
||||
(
|
||||
start_x + direction_x * (position + run_length),
|
||||
start_y + direction_y * (position + run_length),
|
||||
),
|
||||
)
|
||||
)
|
||||
else:
|
||||
run_length = min(pattern_length - phase, segment_length - position)
|
||||
position += run_length
|
||||
distance_offset += segment_length
|
||||
return tuple(segments)
|
||||
|
||||
|
||||
def draw_dashed_line(
|
||||
vertices,
|
||||
color=(1.0, 1.0, 1.0, 0.9),
|
||||
segments=None,
|
||||
):
|
||||
import gpu
|
||||
from gpu_extras.batch import batch_for_shader
|
||||
|
||||
if segments is None:
|
||||
segments = dashed_line_segments(vertices)
|
||||
if not segments:
|
||||
return
|
||||
shader = gpu.shader.from_builtin("UNIFORM_COLOR")
|
||||
batch = batch_for_shader(shader, "LINES", {"pos": segments})
|
||||
gpu.state.blend_set("ALPHA")
|
||||
gpu.state.line_width_set(1.0)
|
||||
shader.bind()
|
||||
shader.uniform_float("color", color)
|
||||
batch.draw(shader)
|
||||
gpu.state.blend_set("NONE")
|
||||
|
||||
|
||||
def _signed_area(vertices):
|
||||
return sum(
|
||||
start[0] * end[1] - end[0] * start[1]
|
||||
for start, end in zip(vertices, (*vertices[1:], vertices[0]))
|
||||
) * 0.5
|
||||
|
||||
|
||||
def _triangle_cross(start, middle, end):
|
||||
return (middle[0] - start[0]) * (end[1] - start[1]) - (
|
||||
middle[1] - start[1]
|
||||
) * (end[0] - start[0])
|
||||
|
||||
|
||||
def _point_in_triangle(point, start, middle, end, orientation, epsilon):
|
||||
return all(
|
||||
orientation * cross >= -epsilon
|
||||
for cross in (
|
||||
_triangle_cross(start, middle, point),
|
||||
_triangle_cross(middle, end, point),
|
||||
_triangle_cross(end, start, point),
|
||||
)
|
||||
)
|
||||
|
||||
|
||||
def _triangulated_polygon_vertices_python(vertices):
|
||||
vertices = tuple(tuple(float(value) for value in vertex) for vertex in vertices)
|
||||
if len(vertices) < 3:
|
||||
return ()
|
||||
area = _signed_area(vertices)
|
||||
if abs(area) <= 1e-8:
|
||||
return ()
|
||||
orientation = 1.0 if area > 0.0 else -1.0
|
||||
remaining = list(range(len(vertices)))
|
||||
triangles = []
|
||||
epsilon = 1e-8
|
||||
while len(remaining) > 3:
|
||||
for position, current in enumerate(remaining):
|
||||
previous = remaining[position - 1]
|
||||
following = remaining[(position + 1) % len(remaining)]
|
||||
start = vertices[previous]
|
||||
middle = vertices[current]
|
||||
end = vertices[following]
|
||||
if orientation * _triangle_cross(start, middle, end) <= epsilon:
|
||||
continue
|
||||
if any(
|
||||
_point_in_triangle(
|
||||
vertices[index],
|
||||
start,
|
||||
middle,
|
||||
end,
|
||||
orientation,
|
||||
epsilon,
|
||||
)
|
||||
for index in remaining
|
||||
if index not in {previous, current, following}
|
||||
):
|
||||
continue
|
||||
triangles.extend((start, middle, end))
|
||||
del remaining[position]
|
||||
break
|
||||
else:
|
||||
return ()
|
||||
triangles.extend(vertices[index] for index in remaining)
|
||||
return tuple(triangles)
|
||||
|
||||
|
||||
def triangulated_polygon_vertices(vertices):
|
||||
vertices = tuple(tuple(float(value) for value in vertex) for vertex in vertices)
|
||||
try:
|
||||
import mathutils
|
||||
|
||||
points = [
|
||||
mathutils.Vector((vertex[0], vertex[1], 0.0)) for vertex in vertices
|
||||
]
|
||||
triangles = mathutils.geometry.tessellate_polygon([points])
|
||||
return tuple(
|
||||
(float(point[0]), float(point[1]))
|
||||
for triangle in triangles
|
||||
for point in triangle
|
||||
)
|
||||
except (AttributeError, ImportError, RuntimeError, TypeError, ValueError):
|
||||
return _triangulated_polygon_vertices_python(vertices)
|
||||
|
||||
|
||||
def scanline_fill_triangles(vertices, fill_rule='EVEN_ODD', step=2.0):
|
||||
vertices = tuple(tuple(float(value) for value in vertex) for vertex in vertices)
|
||||
if len(vertices) < 3:
|
||||
return ()
|
||||
if fill_rule not in {'EVEN_ODD', 'NONZERO'}:
|
||||
raise ValueError(f'Unknown polygon fill rule: {fill_rule}')
|
||||
minimum_y = min(vertex[1] for vertex in vertices)
|
||||
maximum_y = max(vertex[1] for vertex in vertices)
|
||||
band_bottom = minimum_y
|
||||
triangles = []
|
||||
edges = tuple(zip(vertices, (*vertices[1:], vertices[0])))
|
||||
while band_bottom < maximum_y:
|
||||
band_top = min(band_bottom + step, maximum_y)
|
||||
sample_y = (band_bottom + band_top) * 0.5
|
||||
events = []
|
||||
for start, end in edges:
|
||||
if (start[1] <= sample_y < end[1]) or (
|
||||
end[1] <= sample_y < start[1]
|
||||
):
|
||||
factor = (sample_y - start[1]) / (end[1] - start[1])
|
||||
events.append(
|
||||
(
|
||||
start[0] + (end[0] - start[0]) * factor,
|
||||
1 if end[1] > start[1] else -1,
|
||||
)
|
||||
)
|
||||
events.sort(key=lambda event: event[0])
|
||||
intervals = []
|
||||
winding = 0
|
||||
interval_start = None
|
||||
index = 0
|
||||
while index < len(events):
|
||||
x = events[index][0]
|
||||
delta = 0
|
||||
count = 0
|
||||
while index < len(events) and abs(events[index][0] - x) <= 1e-8:
|
||||
delta += events[index][1]
|
||||
count += 1
|
||||
index += 1
|
||||
was_inside = winding != 0
|
||||
if fill_rule == 'EVEN_ODD':
|
||||
if count % 2:
|
||||
winding = 0 if was_inside else 1
|
||||
else:
|
||||
winding += delta
|
||||
is_inside = winding != 0
|
||||
if not was_inside and is_inside:
|
||||
interval_start = x
|
||||
elif was_inside and not is_inside and interval_start is not None:
|
||||
intervals.append((interval_start, x))
|
||||
interval_start = None
|
||||
for left, right in intervals:
|
||||
if right - left <= 1e-8:
|
||||
continue
|
||||
triangles.extend(
|
||||
(
|
||||
(left, band_bottom),
|
||||
(right, band_bottom),
|
||||
(right, band_top),
|
||||
(left, band_bottom),
|
||||
(right, band_top),
|
||||
(left, band_top),
|
||||
)
|
||||
)
|
||||
band_bottom = band_top
|
||||
return tuple(triangles)
|
||||
|
||||
|
||||
def _point_on_segment(point, start, end, epsilon=1e-8):
|
||||
return (
|
||||
min(start[0], end[0]) - epsilon
|
||||
<= point[0]
|
||||
<= max(start[0], end[0]) + epsilon
|
||||
and min(start[1], end[1]) - epsilon
|
||||
<= point[1]
|
||||
<= max(start[1], end[1]) + epsilon
|
||||
)
|
||||
|
||||
|
||||
def _segments_intersect(first_start, first_end, second_start, second_end):
|
||||
crosses = (
|
||||
_triangle_cross(first_start, first_end, second_start),
|
||||
_triangle_cross(first_start, first_end, second_end),
|
||||
_triangle_cross(second_start, second_end, first_start),
|
||||
_triangle_cross(second_start, second_end, first_end),
|
||||
)
|
||||
if (crosses[0] > 1e-8) != (crosses[1] > 1e-8) and (
|
||||
crosses[2] > 1e-8
|
||||
) != (crosses[3] > 1e-8):
|
||||
return True
|
||||
return any(
|
||||
abs(cross) <= 1e-8 and _point_on_segment(point, start, end)
|
||||
for cross, point, start, end in (
|
||||
(crosses[0], second_start, first_start, first_end),
|
||||
(crosses[1], second_end, first_start, first_end),
|
||||
(crosses[2], first_start, second_start, second_end),
|
||||
(crosses[3], first_end, second_start, second_end),
|
||||
)
|
||||
)
|
||||
|
||||
|
||||
def polygon_self_intersects(vertices, cell_size=32.0):
|
||||
vertices = tuple(tuple(float(value) for value in vertex) for vertex in vertices)
|
||||
if len(vertices) < 4:
|
||||
return False
|
||||
edges = tuple(zip(vertices, (*vertices[1:], vertices[0])))
|
||||
cells = {}
|
||||
last_edge = len(edges) - 1
|
||||
for edge_index, (start, end) in enumerate(edges):
|
||||
left = math.floor(min(start[0], end[0]) / cell_size)
|
||||
right = math.floor(max(start[0], end[0]) / cell_size)
|
||||
bottom = math.floor(min(start[1], end[1]) / cell_size)
|
||||
top = math.floor(max(start[1], end[1]) / cell_size)
|
||||
edge_cells = tuple(
|
||||
(x, y)
|
||||
for x in range(left, right + 1)
|
||||
for y in range(bottom, top + 1)
|
||||
)
|
||||
candidates = {
|
||||
candidate
|
||||
for cell in edge_cells
|
||||
for candidate in cells.get(cell, ())
|
||||
}
|
||||
for candidate in candidates:
|
||||
if abs(edge_index - candidate) <= 1 or {
|
||||
edge_index,
|
||||
candidate,
|
||||
} == {0, last_edge}:
|
||||
continue
|
||||
if _segments_intersect(start, end, *edges[candidate]):
|
||||
return True
|
||||
for cell in edge_cells:
|
||||
cells.setdefault(cell, []).append(edge_index)
|
||||
return False
|
||||
|
||||
|
||||
def preview_fill_triangles(vertices, fill_rule='EVEN_ODD'):
|
||||
if not polygon_self_intersects(vertices):
|
||||
triangles = triangulated_polygon_vertices(vertices)
|
||||
if triangles:
|
||||
return triangles
|
||||
return scanline_fill_triangles(vertices, fill_rule)
|
||||
|
||||
|
||||
def draw_polygon_fill(
|
||||
vertices,
|
||||
triangles=None,
|
||||
color=(1.0, 1.0, 1.0, 0.16),
|
||||
):
|
||||
import gpu
|
||||
from gpu_extras.batch import batch_for_shader
|
||||
|
||||
if triangles is None:
|
||||
triangles = triangulated_polygon_vertices(vertices)
|
||||
if not triangles:
|
||||
return
|
||||
shader = gpu.shader.from_builtin("UNIFORM_COLOR")
|
||||
batch = batch_for_shader(shader, "TRIS", {"pos": triangles})
|
||||
gpu.state.blend_set("ALPHA")
|
||||
shader.bind()
|
||||
shader.uniform_float("color", color)
|
||||
batch.draw(shader)
|
||||
gpu.state.blend_set("NONE")
|
||||
@@ -1,420 +0,0 @@
|
||||
import json
|
||||
import math
|
||||
|
||||
import bpy
|
||||
|
||||
from .image import is_image_empty, plane_bounds, video_total_frames
|
||||
from .image_edit import (
|
||||
active_view3d_tool_id,
|
||||
apply_image_mask_settings,
|
||||
dashed_line_segments,
|
||||
draw_dashed_line,
|
||||
draw_image_mask_operator_settings,
|
||||
draw_polygon_fill,
|
||||
image_edit_poll,
|
||||
polygon_bounds,
|
||||
preview_fill_triangles,
|
||||
report_image_edit_exception,
|
||||
resolve_image_edit_click,
|
||||
)
|
||||
from .image_interaction import (
|
||||
deserialize_matrix,
|
||||
drawing_in_region,
|
||||
screen_path_to_image_pixels,
|
||||
serialize_matrix,
|
||||
)
|
||||
|
||||
GESTURE_TOOL_IDS = {
|
||||
"BOX": "anyimage.image_box",
|
||||
"LASSO": "anyimage.image_lasso",
|
||||
"LINE": "anyimage.image_line",
|
||||
"POLYLINE": "anyimage.image_polyline",
|
||||
}
|
||||
LINE_SNAP_RADIANS = math.radians(5.0)
|
||||
POLYLINE_CLOSE_RADIUS = 8.0
|
||||
LASSO_POINT_DISTANCE = 2.0
|
||||
LASSO_SIMPLIFY_TOLERANCE = 0.75
|
||||
|
||||
|
||||
def append_lasso_point(path, point):
|
||||
point = (float(point[0]), float(point[1]))
|
||||
if not path:
|
||||
path.append(point)
|
||||
return True
|
||||
if math.dist(point, path[-1]) < LASSO_POINT_DISTANCE:
|
||||
return False
|
||||
if len(path) >= 2:
|
||||
anchor = path[-2]
|
||||
previous = path[-1]
|
||||
segment_x = point[0] - anchor[0]
|
||||
segment_y = point[1] - anchor[1]
|
||||
segment_length_squared = segment_x * segment_x + segment_y * segment_y
|
||||
if segment_length_squared > 0.0:
|
||||
factor = (
|
||||
(previous[0] - anchor[0]) * segment_x
|
||||
+ (previous[1] - anchor[1]) * segment_y
|
||||
) / segment_length_squared
|
||||
projected = (
|
||||
anchor[0] + segment_x * factor,
|
||||
anchor[1] + segment_y * factor,
|
||||
)
|
||||
forward = (
|
||||
(previous[0] - anchor[0]) * (point[0] - previous[0])
|
||||
+ (previous[1] - anchor[1]) * (point[1] - previous[1])
|
||||
)
|
||||
if (
|
||||
0.0 <= factor <= 1.0
|
||||
and forward >= 0.0
|
||||
and math.dist(previous, projected) <= LASSO_SIMPLIFY_TOLERANCE
|
||||
):
|
||||
path[-1] = point
|
||||
return True
|
||||
path.append(point)
|
||||
return True
|
||||
|
||||
def gesture_property():
|
||||
return bpy.props.EnumProperty(
|
||||
items=(
|
||||
("BOX", "Box", "Edit with a rectangular gesture"),
|
||||
("LASSO", "Lasso", "Edit with a freehand gesture"),
|
||||
("LINE", "Line", "Edit one side of a line"),
|
||||
("POLYLINE", "Polyline", "Edit with a clicked polygon"),
|
||||
),
|
||||
default="BOX",
|
||||
options={"HIDDEN", "SKIP_SAVE"},
|
||||
)
|
||||
|
||||
|
||||
def box_screen_path(start, end):
|
||||
x0, y0 = start
|
||||
x1, y1 = end
|
||||
if abs(x1 - x0) < 2.0 or abs(y1 - y0) < 2.0:
|
||||
return ()
|
||||
return ((x0, y0), (x1, y0), (x1, y1), (x0, y1))
|
||||
|
||||
|
||||
def snapped_line_endpoint(start, end, snap):
|
||||
if not snap:
|
||||
return tuple(float(value) for value in end)
|
||||
delta_x = float(end[0]) - float(start[0])
|
||||
delta_y = float(end[1]) - float(start[1])
|
||||
length = math.hypot(delta_x, delta_y)
|
||||
if length <= 0.0:
|
||||
return tuple(float(value) for value in end)
|
||||
angle = round(math.atan2(delta_y, delta_x) / LINE_SNAP_RADIANS)
|
||||
angle *= LINE_SNAP_RADIANS
|
||||
return (
|
||||
float(start[0]) + math.cos(angle) * length,
|
||||
float(start[1]) + math.sin(angle) * length,
|
||||
)
|
||||
|
||||
|
||||
def line_half_plane_polygon(start, end, rectangle_size):
|
||||
"""Clip a rectangle to the left side of one directed line."""
|
||||
width, height = (float(value) for value in rectangle_size)
|
||||
if math.dist(start, end) < 2.0:
|
||||
return ()
|
||||
vertices = [(0.0, 0.0), (width, 0.0), (width, height), (0.0, height)]
|
||||
|
||||
def side(point):
|
||||
return (end[0] - start[0]) * (point[1] - start[1]) - (end[1] - start[1]) * (
|
||||
point[0] - start[0]
|
||||
)
|
||||
|
||||
result = []
|
||||
previous = vertices[-1]
|
||||
previous_side = side(previous)
|
||||
for current in vertices:
|
||||
current_side = side(current)
|
||||
if (current_side >= 0.0) != (previous_side >= 0.0):
|
||||
factor = previous_side / (previous_side - current_side)
|
||||
result.append(
|
||||
(
|
||||
previous[0] + (current[0] - previous[0]) * factor,
|
||||
previous[1] + (current[1] - previous[1]) * factor,
|
||||
)
|
||||
)
|
||||
if current_side >= 0.0:
|
||||
result.append(current)
|
||||
previous = current
|
||||
previous_side = current_side
|
||||
return tuple(result)
|
||||
|
||||
|
||||
def image_line_half_plane_polygon(line, image_size):
|
||||
"""Keep the image-space half-plane shown by the screen-space preview."""
|
||||
if len(line) != 2:
|
||||
raise ValueError("Drag a longer line across the image plane")
|
||||
return line_half_plane_polygon(line[1], line[0], image_size)
|
||||
|
||||
|
||||
class ImageMaskGesture:
|
||||
active_tool_id = None
|
||||
preview_fill_rule = 'EVEN_ODD'
|
||||
_path = None
|
||||
_cursor = None
|
||||
_handle = None
|
||||
_fill_polygon = None
|
||||
_fill_triangles = ()
|
||||
_outline_polygon = None
|
||||
_outline_segments = ()
|
||||
_area_pointer = 0
|
||||
_region_pointer = 0
|
||||
_region_size = (0, 0)
|
||||
|
||||
def draw(self, _context):
|
||||
draw_image_mask_operator_settings(self.layout, self)
|
||||
|
||||
@classmethod
|
||||
def poll(cls, context):
|
||||
return image_edit_poll(context)
|
||||
|
||||
def _screen_polygon(self):
|
||||
if self.gesture == "BOX":
|
||||
return box_screen_path(self._path[0], self._cursor)
|
||||
if self.gesture == "LASSO":
|
||||
return tuple(self._path)
|
||||
if self.gesture == "LINE":
|
||||
return line_half_plane_polygon(
|
||||
self._path[0], self._cursor, self._region_size
|
||||
)
|
||||
vertices = list(self._path)
|
||||
if self._cursor is not None:
|
||||
vertices.append(self._cursor)
|
||||
return tuple(vertices)
|
||||
|
||||
def _draw_selection(self):
|
||||
if not self._path or not drawing_in_region(
|
||||
self._area_pointer, self._region_pointer
|
||||
):
|
||||
return
|
||||
polygon = self._screen_polygon()
|
||||
if polygon != self._outline_polygon:
|
||||
self._outline_polygon = polygon
|
||||
if self.gesture == 'LINE':
|
||||
outline = (self._path[0], self._cursor)
|
||||
else:
|
||||
outline = (*polygon, polygon[0]) if len(polygon) > 1 else ()
|
||||
self._outline_segments = dashed_line_segments(outline)
|
||||
if len(polygon) >= 3:
|
||||
if polygon != self._fill_polygon:
|
||||
self._fill_polygon = polygon
|
||||
self._fill_triangles = preview_fill_triangles(
|
||||
polygon, self.preview_fill_rule
|
||||
)
|
||||
draw_polygon_fill(
|
||||
polygon,
|
||||
self._fill_triangles,
|
||||
(1.0, 0.25, 0.2, 0.16) if self.invert else (1.0, 1.0, 1.0, 0.16),
|
||||
)
|
||||
if self.gesture == "LINE":
|
||||
draw_dashed_line((), segments=self._outline_segments)
|
||||
return
|
||||
if len(polygon) > 1:
|
||||
draw_dashed_line((), segments=self._outline_segments)
|
||||
if (
|
||||
self.gesture == "POLYLINE"
|
||||
and len(self._path) >= 3
|
||||
and self._cursor is not None
|
||||
and math.dist(self._cursor, self._path[0]) <= POLYLINE_CLOSE_RADIUS
|
||||
):
|
||||
self._draw_polyline_close_indicator()
|
||||
|
||||
def _draw_polyline_close_indicator(self):
|
||||
import gpu
|
||||
from gpu_extras.batch import batch_for_shader
|
||||
|
||||
center_x, center_y = self._path[0]
|
||||
vertices = tuple(
|
||||
(
|
||||
center_x + math.cos(index * math.tau / 24.0) * POLYLINE_CLOSE_RADIUS,
|
||||
center_y + math.sin(index * math.tau / 24.0) * POLYLINE_CLOSE_RADIUS,
|
||||
)
|
||||
for index in range(25)
|
||||
)
|
||||
shader = gpu.shader.from_builtin("UNIFORM_COLOR")
|
||||
batch = batch_for_shader(shader, "LINE_STRIP", {"pos": vertices})
|
||||
gpu.state.blend_set("ALPHA")
|
||||
gpu.state.line_width_set(2.0)
|
||||
shader.bind()
|
||||
shader.uniform_float("color", (1.0, 1.0, 1.0, 0.95))
|
||||
batch.draw(shader)
|
||||
gpu.state.line_width_set(1.0)
|
||||
gpu.state.blend_set("NONE")
|
||||
|
||||
def _set_status(self, context):
|
||||
invert = "On" if self.invert else "Off"
|
||||
if self.gesture == "POLYLINE":
|
||||
action = "click points; click start or Enter to finish; Backspace removes"
|
||||
elif self.gesture == "LINE":
|
||||
action = "drag line; Ctrl snaps 5°; release to edit"
|
||||
else:
|
||||
action = "drag and release to edit"
|
||||
context.workspace.status_text_set(
|
||||
f"Image {self.gesture.title()}: {action}; F toggles Invert ({invert}); "
|
||||
"RMB or Esc cancels"
|
||||
)
|
||||
|
||||
def _finish(self, context):
|
||||
if self._handle is not None:
|
||||
bpy.types.SpaceView3D.draw_handler_remove(self._handle, "WINDOW")
|
||||
self._handle = None
|
||||
context.workspace.status_text_set(None)
|
||||
if context.area is not None:
|
||||
context.area.tag_redraw()
|
||||
|
||||
def cancel(self, context):
|
||||
self._finish(context)
|
||||
|
||||
def invoke(self, context, event):
|
||||
source_object, should_edit = resolve_image_edit_click(
|
||||
context, event, self.report
|
||||
)
|
||||
if not should_edit:
|
||||
return {"FINISHED"} if source_object is not None else {"CANCELLED"}
|
||||
if source_object is None:
|
||||
return {"CANCELLED"}
|
||||
if video_total_frames(source_object.data) > 1:
|
||||
self.report({"WARNING"}, "Image tools only support a still image")
|
||||
return {"CANCELLED"}
|
||||
apply_image_mask_settings(self, context)
|
||||
self.invert = False
|
||||
self.source_object_name = source_object.name
|
||||
self.source_matrix_data = serialize_matrix(source_object.matrix_world)
|
||||
self.selection_data = ""
|
||||
point = (float(event.mouse_region_x), float(event.mouse_region_y))
|
||||
self._path = [point]
|
||||
self._cursor = point
|
||||
self._fill_polygon = None
|
||||
self._fill_triangles = ()
|
||||
self._outline_polygon = None
|
||||
self._outline_segments = ()
|
||||
self._area_pointer = context.area.as_pointer()
|
||||
self._region_pointer = context.region.as_pointer()
|
||||
self._region_size = (context.region.width, context.region.height)
|
||||
self._handle = bpy.types.SpaceView3D.draw_handler_add(
|
||||
self._draw_selection, (), "WINDOW", "POST_PIXEL"
|
||||
)
|
||||
self._set_status(context)
|
||||
context.window_manager.modal_handler_add(self)
|
||||
context.area.tag_redraw()
|
||||
return {"RUNNING_MODAL"}
|
||||
|
||||
def _complete(self, context):
|
||||
source_object = bpy.data.objects.get(self.source_object_name)
|
||||
if not is_image_empty(source_object):
|
||||
raise RuntimeError("The source Image Empty is no longer available")
|
||||
matrix = deserialize_matrix(self.source_matrix_data)
|
||||
if self.gesture == "LINE":
|
||||
line = screen_path_to_image_pixels(
|
||||
(self._path[0], self._cursor),
|
||||
context.region,
|
||||
context.region_data,
|
||||
matrix,
|
||||
plane_bounds(source_object),
|
||||
tuple(source_object.data.size),
|
||||
)
|
||||
polygon = image_line_half_plane_polygon(
|
||||
line, tuple(source_object.data.size)
|
||||
)
|
||||
else:
|
||||
path = self._screen_polygon()
|
||||
if len(path) < 3:
|
||||
self._finish(context)
|
||||
return {"CANCELLED"}
|
||||
polygon = screen_path_to_image_pixels(
|
||||
path,
|
||||
context.region,
|
||||
context.region_data,
|
||||
matrix,
|
||||
plane_bounds(source_object),
|
||||
tuple(source_object.data.size),
|
||||
)
|
||||
polygon_bounds(polygon)
|
||||
self.selection_data = json.dumps(polygon, separators=(",", ":"))
|
||||
self._finish(context)
|
||||
return self.execute(context)
|
||||
|
||||
def modal(self, context, event):
|
||||
active_tool = active_view3d_tool_id(context)
|
||||
expected_tool = self.active_tool_id or GESTURE_TOOL_IDS[self.gesture]
|
||||
if active_tool is not None and active_tool != expected_tool:
|
||||
self._finish(context)
|
||||
return {"CANCELLED"}
|
||||
if event.type in {"ESC", "RIGHTMOUSE"} and event.value == "PRESS":
|
||||
self._finish(context)
|
||||
return {"CANCELLED"}
|
||||
if event.type == "F" and event.value == "PRESS":
|
||||
self.invert = not self.invert
|
||||
self._set_status(context)
|
||||
context.area.tag_redraw()
|
||||
return {"RUNNING_MODAL"}
|
||||
if self.gesture == "POLYLINE":
|
||||
return self._modal_polyline(context, event)
|
||||
if event.type == "MOUSEMOVE":
|
||||
point = (float(event.mouse_region_x), float(event.mouse_region_y))
|
||||
if self.gesture == "LASSO":
|
||||
append_lasso_point(self._path, point)
|
||||
self._cursor = point
|
||||
elif self.gesture == "LINE":
|
||||
self._cursor = snapped_line_endpoint(self._path[0], point, event.ctrl)
|
||||
else:
|
||||
self._cursor = point
|
||||
context.area.tag_redraw()
|
||||
return {"RUNNING_MODAL"}
|
||||
if event.type == "LEFTMOUSE" and event.value == "RELEASE":
|
||||
point = (float(event.mouse_region_x), float(event.mouse_region_y))
|
||||
if self.gesture == "LASSO":
|
||||
append_lasso_point(self._path, point)
|
||||
self._cursor = point
|
||||
elif self.gesture == "LINE":
|
||||
self._cursor = snapped_line_endpoint(self._path[0], point, event.ctrl)
|
||||
if math.dist(self._path[0], self._cursor) < 2.0:
|
||||
self._finish(context)
|
||||
return {"CANCELLED"}
|
||||
else:
|
||||
self._cursor = point
|
||||
try:
|
||||
return self._complete(context)
|
||||
except (RuntimeError, TypeError, ValueError) as error:
|
||||
report_image_edit_exception(self, error)
|
||||
self._finish(context)
|
||||
return {"CANCELLED"}
|
||||
return {"PASS_THROUGH"}
|
||||
|
||||
def _modal_polyline(self, context, event):
|
||||
if event.type == "BACK_SPACE" and event.value == "PRESS":
|
||||
if len(self._path) > 1:
|
||||
self._path.pop()
|
||||
context.area.tag_redraw()
|
||||
return {"RUNNING_MODAL"}
|
||||
if event.type in {"RET", "NUMPAD_ENTER"} and event.value == "PRESS":
|
||||
return self._confirm_polyline(context)
|
||||
if event.type == "MOUSEMOVE":
|
||||
self._cursor = (
|
||||
float(event.mouse_region_x),
|
||||
float(event.mouse_region_y),
|
||||
)
|
||||
context.area.tag_redraw()
|
||||
return {"RUNNING_MODAL"}
|
||||
if event.type == "LEFTMOUSE" and event.value == "PRESS":
|
||||
point = (float(event.mouse_region_x), float(event.mouse_region_y))
|
||||
if (
|
||||
len(self._path) >= 3
|
||||
and math.dist(point, self._path[0]) <= POLYLINE_CLOSE_RADIUS
|
||||
):
|
||||
return self._confirm_polyline(context)
|
||||
if math.dist(point, self._path[-1]) >= 2.0:
|
||||
self._path.append(point)
|
||||
context.area.tag_redraw()
|
||||
return {"RUNNING_MODAL"}
|
||||
return {"PASS_THROUGH"}
|
||||
|
||||
def _confirm_polyline(self, context):
|
||||
self._cursor = self._path[0]
|
||||
try:
|
||||
return self._complete(context)
|
||||
except (RuntimeError, TypeError, ValueError) as error:
|
||||
report_image_edit_exception(self, error)
|
||||
self._finish(context)
|
||||
return {"CANCELLED"}
|
||||
@@ -3,6 +3,14 @@ import math
|
||||
|
||||
import bpy
|
||||
|
||||
from ...properties import ai_ready, ai_setup_label, ai_status
|
||||
from .image_data import (
|
||||
image_empty_bounds,
|
||||
is_image_empty,
|
||||
require_image_empty,
|
||||
video_total_frames,
|
||||
)
|
||||
from .image_region import ImageEditWarning, RegionPath
|
||||
|
||||
MIN_PATH_DISTANCE = 2.0
|
||||
|
||||
@@ -18,8 +26,8 @@ def simplify_screen_path(path):
|
||||
|
||||
def screen_path_to_image_pixels(
|
||||
path,
|
||||
region,
|
||||
region_data,
|
||||
view_region,
|
||||
view3d,
|
||||
matrix_world,
|
||||
local_bounds,
|
||||
image_size,
|
||||
@@ -34,13 +42,13 @@ def screen_path_to_image_pixels(
|
||||
pixels = []
|
||||
for point in simplify_screen_path(path):
|
||||
ray_origin = inverse @ view3d_utils.region_2d_to_origin_3d(
|
||||
region,
|
||||
region_data,
|
||||
view_region,
|
||||
view3d,
|
||||
point,
|
||||
)
|
||||
ray_direction = inverse.to_3x3() @ view3d_utils.region_2d_to_vector_3d(
|
||||
region,
|
||||
region_data,
|
||||
view_region,
|
||||
view3d,
|
||||
point,
|
||||
)
|
||||
if abs(ray_direction.z) <= 1e-10:
|
||||
@@ -72,10 +80,6 @@ def deserialize_matrix(data):
|
||||
return Matrix(tuple(tuple(values[row * 4 : row * 4 + 4]) for row in range(4)))
|
||||
|
||||
|
||||
def hidden_string_property():
|
||||
return bpy.props.StringProperty(options={"HIDDEN", "SKIP_SAVE"})
|
||||
|
||||
|
||||
def drawing_in_region(area_pointer, region_pointer):
|
||||
area = getattr(bpy.context, "area", None)
|
||||
region = getattr(bpy.context, "region", None)
|
||||
@@ -85,3 +89,817 @@ def drawing_in_region(area_pointer, region_pointer):
|
||||
and area.as_pointer() == area_pointer
|
||||
and region.as_pointer() == region_pointer
|
||||
)
|
||||
|
||||
|
||||
def report_image_edit_exception(operator, error):
|
||||
level = "WARNING" if isinstance(error, ImageEditWarning) else "ERROR"
|
||||
operator.report({level}, str(error))
|
||||
|
||||
|
||||
def image_edit_poll(context):
|
||||
return context.area is not None and context.area.type == "VIEW_3D"
|
||||
|
||||
|
||||
def _restore_selection(context, active_object, selected_objects):
|
||||
bpy.ops.object.select_all(action="DESELECT")
|
||||
for selected in selected_objects:
|
||||
if bpy.data.objects.get(selected.name) is not None:
|
||||
selected.select_set(True)
|
||||
if (
|
||||
active_object is not None
|
||||
and bpy.data.objects.get(active_object.name) is not None
|
||||
):
|
||||
context.view_layer.objects.active = active_object
|
||||
|
||||
|
||||
def resolve_image_edit_click(context, event, report):
|
||||
"""Resolve the first click into source selection or image editing."""
|
||||
active_object = context.active_object
|
||||
selected_objects = tuple(context.selected_objects)
|
||||
try:
|
||||
bpy.ops.view3d.select(
|
||||
location=(event.mouse_region_x, event.mouse_region_y),
|
||||
extend=False,
|
||||
deselect_all=False,
|
||||
)
|
||||
except RuntimeError:
|
||||
pass
|
||||
clicked_object = context.active_object
|
||||
if is_image_empty(clicked_object) and clicked_object != active_object:
|
||||
return clicked_object, False
|
||||
if is_image_empty(clicked_object):
|
||||
return clicked_object, True
|
||||
if is_image_empty(active_object):
|
||||
_restore_selection(context, active_object, selected_objects)
|
||||
return active_object, True
|
||||
report({"WARNING"}, "Click an Image Empty to use an Image tool")
|
||||
return None, False
|
||||
|
||||
|
||||
def active_view3d_tool_id(context):
|
||||
tools = getattr(getattr(context, "workspace", None), "tools", None)
|
||||
from_mode = getattr(tools, "from_space_view3d_mode", None)
|
||||
if not callable(from_mode):
|
||||
return None
|
||||
try:
|
||||
tool = from_mode(getattr(context, "mode", "OBJECT"), create=False)
|
||||
except (RuntimeError, TypeError):
|
||||
return None
|
||||
return getattr(tool, "idname", None)
|
||||
|
||||
|
||||
def refine_selection_setting_name(operator):
|
||||
if getattr(operator, "active_tool_id", None) == "anyimage.cutout_lasso":
|
||||
return "cutout_refine_selection"
|
||||
return {
|
||||
"BOX": "image_box_refine_selection",
|
||||
"LASSO": "image_lasso_refine_selection",
|
||||
}.get(operator.gesture)
|
||||
|
||||
|
||||
def apply_region_settings(operator, context):
|
||||
settings = context.scene.anyimage_settings
|
||||
setting_name = refine_selection_setting_name(operator)
|
||||
operator.refine_selection = (
|
||||
setting_name is not None
|
||||
and ai_ready()
|
||||
and getattr(settings, setting_name)
|
||||
)
|
||||
|
||||
|
||||
def draw_region_operator_settings(layout, operator):
|
||||
if operator.gesture not in {"BOX", "LASSO"}:
|
||||
layout.prop(operator, "invert")
|
||||
return
|
||||
status = ai_status()
|
||||
row = layout.row()
|
||||
row.enabled = status["ready"]
|
||||
row.prop(operator, "refine_selection")
|
||||
layout.prop(operator, "invert")
|
||||
if not status["ready"]:
|
||||
layout.operator(
|
||||
"anyimage.setup_ai_environment",
|
||||
text=ai_setup_label(status),
|
||||
icon="IMPORT",
|
||||
)
|
||||
|
||||
|
||||
def draw_region_settings(context, layout, setting_name):
|
||||
settings = context.scene.anyimage_settings
|
||||
status = ai_status()
|
||||
row = layout.row()
|
||||
row.enabled = status["ready"]
|
||||
row.prop(settings, setting_name)
|
||||
if not status["ready"]:
|
||||
layout.operator(
|
||||
"anyimage.setup_ai_environment",
|
||||
text=ai_setup_label(status),
|
||||
icon="IMPORT",
|
||||
)
|
||||
|
||||
|
||||
def dashed_line_segments(vertices, dash_length=4.0, gap_length=4.0):
|
||||
if dash_length <= 0.0 or gap_length < 0.0:
|
||||
raise ValueError("Dash length must be positive and gap length cannot be negative")
|
||||
pattern_length = dash_length + gap_length
|
||||
segments = []
|
||||
distance_offset = 0.0
|
||||
for start, end in zip(vertices, vertices[1:]):
|
||||
start_x, start_y = (float(value) for value in start)
|
||||
end_x, end_y = (float(value) for value in end)
|
||||
segment_length = math.hypot(end_x - start_x, end_y - start_y)
|
||||
if segment_length <= 0.0:
|
||||
continue
|
||||
direction_x = (end_x - start_x) / segment_length
|
||||
direction_y = (end_y - start_y) / segment_length
|
||||
position = 0.0
|
||||
while position < segment_length:
|
||||
phase = (distance_offset + position) % pattern_length
|
||||
if phase < dash_length:
|
||||
run_length = min(dash_length - phase, segment_length - position)
|
||||
segments.extend(
|
||||
(
|
||||
(
|
||||
start_x + direction_x * position,
|
||||
start_y + direction_y * position,
|
||||
),
|
||||
(
|
||||
start_x + direction_x * (position + run_length),
|
||||
start_y + direction_y * (position + run_length),
|
||||
),
|
||||
)
|
||||
)
|
||||
else:
|
||||
run_length = min(pattern_length - phase, segment_length - position)
|
||||
position += run_length
|
||||
distance_offset += segment_length
|
||||
return tuple(segments)
|
||||
|
||||
|
||||
def draw_dashed_line(
|
||||
vertices,
|
||||
color=(1.0, 1.0, 1.0, 0.9),
|
||||
segments=None,
|
||||
):
|
||||
import gpu
|
||||
from gpu_extras.batch import batch_for_shader
|
||||
|
||||
if segments is None:
|
||||
segments = dashed_line_segments(vertices)
|
||||
if not segments:
|
||||
return
|
||||
shader = gpu.shader.from_builtin("UNIFORM_COLOR")
|
||||
batch = batch_for_shader(shader, "LINES", {"pos": segments})
|
||||
gpu.state.blend_set("ALPHA")
|
||||
gpu.state.line_width_set(1.0)
|
||||
shader.bind()
|
||||
shader.uniform_float("color", color)
|
||||
batch.draw(shader)
|
||||
gpu.state.blend_set("NONE")
|
||||
|
||||
|
||||
def _signed_area(vertices):
|
||||
return sum(
|
||||
start[0] * end[1] - end[0] * start[1]
|
||||
for start, end in zip(vertices, (*vertices[1:], vertices[0]))
|
||||
) * 0.5
|
||||
|
||||
|
||||
def _triangle_cross(start, middle, end):
|
||||
return (middle[0] - start[0]) * (end[1] - start[1]) - (
|
||||
middle[1] - start[1]
|
||||
) * (end[0] - start[0])
|
||||
|
||||
|
||||
def _point_in_triangle(point, start, middle, end, orientation, epsilon):
|
||||
return all(
|
||||
orientation * cross >= -epsilon
|
||||
for cross in (
|
||||
_triangle_cross(start, middle, point),
|
||||
_triangle_cross(middle, end, point),
|
||||
_triangle_cross(end, start, point),
|
||||
)
|
||||
)
|
||||
|
||||
|
||||
def _triangulated_polygon_vertices_python(vertices):
|
||||
vertices = tuple(tuple(float(value) for value in vertex) for vertex in vertices)
|
||||
if len(vertices) < 3:
|
||||
return ()
|
||||
area = _signed_area(vertices)
|
||||
if abs(area) <= 1e-8:
|
||||
return ()
|
||||
orientation = 1.0 if area > 0.0 else -1.0
|
||||
remaining = list(range(len(vertices)))
|
||||
triangles = []
|
||||
epsilon = 1e-8
|
||||
while len(remaining) > 3:
|
||||
for position, current in enumerate(remaining):
|
||||
previous = remaining[position - 1]
|
||||
following = remaining[(position + 1) % len(remaining)]
|
||||
start = vertices[previous]
|
||||
middle = vertices[current]
|
||||
end = vertices[following]
|
||||
if orientation * _triangle_cross(start, middle, end) <= epsilon:
|
||||
continue
|
||||
if any(
|
||||
_point_in_triangle(
|
||||
vertices[index],
|
||||
start,
|
||||
middle,
|
||||
end,
|
||||
orientation,
|
||||
epsilon,
|
||||
)
|
||||
for index in remaining
|
||||
if index not in {previous, current, following}
|
||||
):
|
||||
continue
|
||||
triangles.extend((start, middle, end))
|
||||
del remaining[position]
|
||||
break
|
||||
else:
|
||||
return ()
|
||||
triangles.extend(vertices[index] for index in remaining)
|
||||
return tuple(triangles)
|
||||
|
||||
|
||||
def triangulated_polygon_vertices(vertices):
|
||||
vertices = tuple(tuple(float(value) for value in vertex) for vertex in vertices)
|
||||
try:
|
||||
import mathutils
|
||||
|
||||
points = [
|
||||
mathutils.Vector((vertex[0], vertex[1], 0.0)) for vertex in vertices
|
||||
]
|
||||
triangles = mathutils.geometry.tessellate_polygon([points])
|
||||
return tuple(
|
||||
(float(point[0]), float(point[1]))
|
||||
for triangle in triangles
|
||||
for point in triangle
|
||||
)
|
||||
except (AttributeError, ImportError, RuntimeError, TypeError, ValueError):
|
||||
return _triangulated_polygon_vertices_python(vertices)
|
||||
|
||||
|
||||
def scanline_fill_triangles(vertices, fill_rule='EVEN_ODD', step=2.0):
|
||||
vertices = tuple(tuple(float(value) for value in vertex) for vertex in vertices)
|
||||
if len(vertices) < 3:
|
||||
return ()
|
||||
if fill_rule not in {'EVEN_ODD', 'NONZERO'}:
|
||||
raise ValueError(f'Unknown polygon fill rule: {fill_rule}')
|
||||
minimum_y = min(vertex[1] for vertex in vertices)
|
||||
maximum_y = max(vertex[1] for vertex in vertices)
|
||||
band_bottom = minimum_y
|
||||
triangles = []
|
||||
edges = tuple(zip(vertices, (*vertices[1:], vertices[0])))
|
||||
while band_bottom < maximum_y:
|
||||
band_top = min(band_bottom + step, maximum_y)
|
||||
sample_y = (band_bottom + band_top) * 0.5
|
||||
events = []
|
||||
for start, end in edges:
|
||||
if (start[1] <= sample_y < end[1]) or (
|
||||
end[1] <= sample_y < start[1]
|
||||
):
|
||||
factor = (sample_y - start[1]) / (end[1] - start[1])
|
||||
events.append(
|
||||
(
|
||||
start[0] + (end[0] - start[0]) * factor,
|
||||
1 if end[1] > start[1] else -1,
|
||||
)
|
||||
)
|
||||
events.sort(key=lambda event: event[0])
|
||||
intervals = []
|
||||
winding = 0
|
||||
interval_start = None
|
||||
index = 0
|
||||
while index < len(events):
|
||||
x = events[index][0]
|
||||
delta = 0
|
||||
count = 0
|
||||
while index < len(events) and abs(events[index][0] - x) <= 1e-8:
|
||||
delta += events[index][1]
|
||||
count += 1
|
||||
index += 1
|
||||
was_inside = winding != 0
|
||||
if fill_rule == 'EVEN_ODD':
|
||||
if count % 2:
|
||||
winding = 0 if was_inside else 1
|
||||
else:
|
||||
winding += delta
|
||||
is_inside = winding != 0
|
||||
if not was_inside and is_inside:
|
||||
interval_start = x
|
||||
elif was_inside and not is_inside and interval_start is not None:
|
||||
intervals.append((interval_start, x))
|
||||
interval_start = None
|
||||
for left, right in intervals:
|
||||
if right - left <= 1e-8:
|
||||
continue
|
||||
triangles.extend(
|
||||
(
|
||||
(left, band_bottom),
|
||||
(right, band_bottom),
|
||||
(right, band_top),
|
||||
(left, band_bottom),
|
||||
(right, band_top),
|
||||
(left, band_top),
|
||||
)
|
||||
)
|
||||
band_bottom = band_top
|
||||
return tuple(triangles)
|
||||
|
||||
|
||||
def _point_on_segment(point, start, end, epsilon=1e-8):
|
||||
return (
|
||||
min(start[0], end[0]) - epsilon
|
||||
<= point[0]
|
||||
<= max(start[0], end[0]) + epsilon
|
||||
and min(start[1], end[1]) - epsilon
|
||||
<= point[1]
|
||||
<= max(start[1], end[1]) + epsilon
|
||||
)
|
||||
|
||||
|
||||
def _segments_intersect(first_start, first_end, second_start, second_end):
|
||||
crosses = (
|
||||
_triangle_cross(first_start, first_end, second_start),
|
||||
_triangle_cross(first_start, first_end, second_end),
|
||||
_triangle_cross(second_start, second_end, first_start),
|
||||
_triangle_cross(second_start, second_end, first_end),
|
||||
)
|
||||
if (crosses[0] > 1e-8) != (crosses[1] > 1e-8) and (
|
||||
crosses[2] > 1e-8
|
||||
) != (crosses[3] > 1e-8):
|
||||
return True
|
||||
return any(
|
||||
abs(cross) <= 1e-8 and _point_on_segment(point, start, end)
|
||||
for cross, point, start, end in (
|
||||
(crosses[0], second_start, first_start, first_end),
|
||||
(crosses[1], second_end, first_start, first_end),
|
||||
(crosses[2], first_start, second_start, second_end),
|
||||
(crosses[3], first_end, second_start, second_end),
|
||||
)
|
||||
)
|
||||
|
||||
|
||||
def polygon_self_intersects(vertices, cell_size=32.0):
|
||||
vertices = tuple(tuple(float(value) for value in vertex) for vertex in vertices)
|
||||
if len(vertices) < 4:
|
||||
return False
|
||||
edges = tuple(zip(vertices, (*vertices[1:], vertices[0])))
|
||||
cells = {}
|
||||
last_edge = len(edges) - 1
|
||||
for edge_index, (start, end) in enumerate(edges):
|
||||
left = math.floor(min(start[0], end[0]) / cell_size)
|
||||
right = math.floor(max(start[0], end[0]) / cell_size)
|
||||
bottom = math.floor(min(start[1], end[1]) / cell_size)
|
||||
top = math.floor(max(start[1], end[1]) / cell_size)
|
||||
edge_cells = tuple(
|
||||
(x, y)
|
||||
for x in range(left, right + 1)
|
||||
for y in range(bottom, top + 1)
|
||||
)
|
||||
candidates = {
|
||||
candidate
|
||||
for cell in edge_cells
|
||||
for candidate in cells.get(cell, ())
|
||||
}
|
||||
for candidate in candidates:
|
||||
if abs(edge_index - candidate) <= 1 or {
|
||||
edge_index,
|
||||
candidate,
|
||||
} == {0, last_edge}:
|
||||
continue
|
||||
if _segments_intersect(start, end, *edges[candidate]):
|
||||
return True
|
||||
for cell in edge_cells:
|
||||
cells.setdefault(cell, []).append(edge_index)
|
||||
return False
|
||||
|
||||
|
||||
def preview_fill_triangles(vertices, fill_rule='EVEN_ODD'):
|
||||
if not polygon_self_intersects(vertices):
|
||||
triangles = triangulated_polygon_vertices(vertices)
|
||||
if triangles:
|
||||
return triangles
|
||||
return scanline_fill_triangles(vertices, fill_rule)
|
||||
|
||||
|
||||
def draw_polygon_fill(
|
||||
vertices,
|
||||
triangles=None,
|
||||
color=(1.0, 1.0, 1.0, 0.16),
|
||||
):
|
||||
import gpu
|
||||
from gpu_extras.batch import batch_for_shader
|
||||
|
||||
if triangles is None:
|
||||
triangles = triangulated_polygon_vertices(vertices)
|
||||
if not triangles:
|
||||
return
|
||||
shader = gpu.shader.from_builtin("UNIFORM_COLOR")
|
||||
batch = batch_for_shader(shader, "TRIS", {"pos": triangles})
|
||||
gpu.state.blend_set("ALPHA")
|
||||
shader.bind()
|
||||
shader.uniform_float("color", color)
|
||||
batch.draw(shader)
|
||||
gpu.state.blend_set("NONE")
|
||||
|
||||
|
||||
GESTURE_TOOL_IDS = {
|
||||
"BOX": "anyimage.image_box",
|
||||
"LASSO": "anyimage.image_lasso",
|
||||
"LINE": "anyimage.image_line",
|
||||
"POLYLINE": "anyimage.image_polyline",
|
||||
}
|
||||
LINE_SNAP_RADIANS = math.radians(5.0)
|
||||
POLYLINE_CLOSE_RADIUS = 8.0
|
||||
LASSO_POINT_DISTANCE = 2.0
|
||||
LASSO_SIMPLIFY_TOLERANCE = 0.75
|
||||
|
||||
|
||||
def append_lasso_point(path, point):
|
||||
point = (float(point[0]), float(point[1]))
|
||||
if not path:
|
||||
path.append(point)
|
||||
return True
|
||||
if math.dist(point, path[-1]) < LASSO_POINT_DISTANCE:
|
||||
return False
|
||||
if len(path) >= 2:
|
||||
anchor = path[-2]
|
||||
previous = path[-1]
|
||||
segment_x = point[0] - anchor[0]
|
||||
segment_y = point[1] - anchor[1]
|
||||
segment_length_squared = segment_x * segment_x + segment_y * segment_y
|
||||
if segment_length_squared > 0.0:
|
||||
factor = (
|
||||
(previous[0] - anchor[0]) * segment_x
|
||||
+ (previous[1] - anchor[1]) * segment_y
|
||||
) / segment_length_squared
|
||||
projected = (
|
||||
anchor[0] + segment_x * factor,
|
||||
anchor[1] + segment_y * factor,
|
||||
)
|
||||
forward = (
|
||||
(previous[0] - anchor[0]) * (point[0] - previous[0])
|
||||
+ (previous[1] - anchor[1]) * (point[1] - previous[1])
|
||||
)
|
||||
if (
|
||||
0.0 <= factor <= 1.0
|
||||
and forward >= 0.0
|
||||
and math.dist(previous, projected) <= LASSO_SIMPLIFY_TOLERANCE
|
||||
):
|
||||
path[-1] = point
|
||||
return True
|
||||
path.append(point)
|
||||
return True
|
||||
|
||||
|
||||
def gesture_property():
|
||||
return bpy.props.EnumProperty(
|
||||
items=(
|
||||
("BOX", "Box", "Edit with a rectangular gesture"),
|
||||
("LASSO", "Lasso", "Edit with a freehand gesture"),
|
||||
("LINE", "Line", "Edit one side of a line"),
|
||||
("POLYLINE", "Polyline", "Edit with a clicked polygon"),
|
||||
),
|
||||
default="BOX",
|
||||
options={"HIDDEN", "SKIP_SAVE"},
|
||||
)
|
||||
|
||||
|
||||
def box_screen_path(start, end):
|
||||
x0, y0 = start
|
||||
x1, y1 = end
|
||||
if abs(x1 - x0) < 2.0 or abs(y1 - y0) < 2.0:
|
||||
return ()
|
||||
return ((x0, y0), (x1, y0), (x1, y1), (x0, y1))
|
||||
|
||||
|
||||
def snapped_line_endpoint(start, end, snap):
|
||||
if not snap:
|
||||
return tuple(float(value) for value in end)
|
||||
delta_x = float(end[0]) - float(start[0])
|
||||
delta_y = float(end[1]) - float(start[1])
|
||||
length = math.hypot(delta_x, delta_y)
|
||||
if length <= 0.0:
|
||||
return tuple(float(value) for value in end)
|
||||
angle = round(math.atan2(delta_y, delta_x) / LINE_SNAP_RADIANS)
|
||||
angle *= LINE_SNAP_RADIANS
|
||||
return (
|
||||
float(start[0]) + math.cos(angle) * length,
|
||||
float(start[1]) + math.sin(angle) * length,
|
||||
)
|
||||
|
||||
|
||||
def line_half_plane_polygon(start, end, rectangle_size):
|
||||
"""Clip a rectangle to the left side of one directed line."""
|
||||
width, height = (float(value) for value in rectangle_size)
|
||||
if math.dist(start, end) < 2.0:
|
||||
return ()
|
||||
vertices = [(0.0, 0.0), (width, 0.0), (width, height), (0.0, height)]
|
||||
|
||||
def side(point):
|
||||
return (end[0] - start[0]) * (point[1] - start[1]) - (end[1] - start[1]) * (
|
||||
point[0] - start[0]
|
||||
)
|
||||
|
||||
result = []
|
||||
previous = vertices[-1]
|
||||
previous_side = side(previous)
|
||||
for current in vertices:
|
||||
current_side = side(current)
|
||||
if (current_side >= 0.0) != (previous_side >= 0.0):
|
||||
factor = previous_side / (previous_side - current_side)
|
||||
result.append(
|
||||
(
|
||||
previous[0] + (current[0] - previous[0]) * factor,
|
||||
previous[1] + (current[1] - previous[1]) * factor,
|
||||
)
|
||||
)
|
||||
if current_side >= 0.0:
|
||||
result.append(current)
|
||||
previous = current
|
||||
previous_side = current_side
|
||||
return tuple(result)
|
||||
|
||||
|
||||
def image_line_half_plane_polygon(line, image_size):
|
||||
"""Keep the image-space half-plane shown by the screen-space preview."""
|
||||
if len(line) != 2:
|
||||
raise ValueError("Drag a longer line across the image plane")
|
||||
return line_half_plane_polygon(line[1], line[0], image_size)
|
||||
|
||||
|
||||
class ImageRegionGesture:
|
||||
active_tool_id = None
|
||||
preview_fill_rule = 'EVEN_ODD'
|
||||
_path = None
|
||||
_cursor = None
|
||||
_handle = None
|
||||
_fill_polygon = None
|
||||
_fill_triangles = ()
|
||||
_outline_polygon = None
|
||||
_outline_segments = ()
|
||||
_area_pointer = 0
|
||||
_region_pointer = 0
|
||||
_region_size = (0, 0)
|
||||
|
||||
def draw(self, _context):
|
||||
draw_region_operator_settings(self.layout, self)
|
||||
|
||||
@classmethod
|
||||
def poll(cls, context):
|
||||
return image_edit_poll(context)
|
||||
|
||||
def _screen_path(self):
|
||||
if self.gesture == "BOX":
|
||||
return box_screen_path(self._path[0], self._cursor)
|
||||
if self.gesture == "LASSO":
|
||||
return tuple(self._path)
|
||||
if self.gesture == "LINE":
|
||||
return line_half_plane_polygon(
|
||||
self._path[0], self._cursor, self._region_size
|
||||
)
|
||||
vertices = list(self._path)
|
||||
if self._cursor is not None:
|
||||
vertices.append(self._cursor)
|
||||
return tuple(vertices)
|
||||
|
||||
def _draw_region(self):
|
||||
if not self._path or not drawing_in_region(
|
||||
self._area_pointer, self._region_pointer
|
||||
):
|
||||
return
|
||||
polygon = self._screen_path()
|
||||
if polygon != self._outline_polygon:
|
||||
self._outline_polygon = polygon
|
||||
if self.gesture == 'LINE':
|
||||
outline = (self._path[0], self._cursor)
|
||||
else:
|
||||
outline = (*polygon, polygon[0]) if len(polygon) > 1 else ()
|
||||
self._outline_segments = dashed_line_segments(outline)
|
||||
if len(polygon) >= 3:
|
||||
if polygon != self._fill_polygon:
|
||||
self._fill_polygon = polygon
|
||||
self._fill_triangles = preview_fill_triangles(
|
||||
polygon, self.preview_fill_rule
|
||||
)
|
||||
draw_polygon_fill(
|
||||
polygon,
|
||||
self._fill_triangles,
|
||||
(1.0, 0.25, 0.2, 0.16) if self.invert else (1.0, 1.0, 1.0, 0.16),
|
||||
)
|
||||
if self.gesture == "LINE":
|
||||
draw_dashed_line((), segments=self._outline_segments)
|
||||
return
|
||||
if len(polygon) > 1:
|
||||
draw_dashed_line((), segments=self._outline_segments)
|
||||
if (
|
||||
self.gesture == "POLYLINE"
|
||||
and len(self._path) >= 3
|
||||
and self._cursor is not None
|
||||
and math.dist(self._cursor, self._path[0]) <= POLYLINE_CLOSE_RADIUS
|
||||
):
|
||||
self._draw_polyline_close_indicator()
|
||||
|
||||
def _draw_polyline_close_indicator(self):
|
||||
import gpu
|
||||
from gpu_extras.batch import batch_for_shader
|
||||
|
||||
center_x, center_y = self._path[0]
|
||||
vertices = tuple(
|
||||
(
|
||||
center_x + math.cos(index * math.tau / 24.0) * POLYLINE_CLOSE_RADIUS,
|
||||
center_y + math.sin(index * math.tau / 24.0) * POLYLINE_CLOSE_RADIUS,
|
||||
)
|
||||
for index in range(25)
|
||||
)
|
||||
shader = gpu.shader.from_builtin("UNIFORM_COLOR")
|
||||
batch = batch_for_shader(shader, "LINE_STRIP", {"pos": vertices})
|
||||
gpu.state.blend_set("ALPHA")
|
||||
gpu.state.line_width_set(2.0)
|
||||
shader.bind()
|
||||
shader.uniform_float("color", (1.0, 1.0, 1.0, 0.95))
|
||||
batch.draw(shader)
|
||||
gpu.state.line_width_set(1.0)
|
||||
gpu.state.blend_set("NONE")
|
||||
|
||||
def _set_status(self, context):
|
||||
invert = "On" if self.invert else "Off"
|
||||
if self.gesture == "POLYLINE":
|
||||
action = "click points; click start or Enter to finish; Backspace removes"
|
||||
elif self.gesture == "LINE":
|
||||
action = "drag line; Ctrl snaps 5°; release to edit"
|
||||
else:
|
||||
action = "drag and release to edit"
|
||||
context.workspace.status_text_set(
|
||||
f"Image {self.gesture.title()}: {action}; F toggles Invert ({invert}); "
|
||||
"RMB or Esc cancels"
|
||||
)
|
||||
|
||||
def _finish(self, context):
|
||||
if self._handle is not None:
|
||||
bpy.types.SpaceView3D.draw_handler_remove(self._handle, "WINDOW")
|
||||
self._handle = None
|
||||
context.workspace.status_text_set(None)
|
||||
if context.area is not None:
|
||||
context.area.tag_redraw()
|
||||
|
||||
def cancel(self, context):
|
||||
self._finish(context)
|
||||
|
||||
def invoke(self, context, event):
|
||||
source_object, should_edit = resolve_image_edit_click(
|
||||
context, event, self.report
|
||||
)
|
||||
if not should_edit:
|
||||
return {"FINISHED"} if source_object is not None else {"CANCELLED"}
|
||||
if source_object is None:
|
||||
return {"CANCELLED"}
|
||||
if video_total_frames(source_object.data) > 1:
|
||||
self.report({"WARNING"}, "Image tools only support a still image")
|
||||
return {"CANCELLED"}
|
||||
apply_region_settings(self, context)
|
||||
self.invert = False
|
||||
self.source_object_name = source_object.name
|
||||
self.source_matrix_data = serialize_matrix(source_object.matrix_world)
|
||||
self.region_path_json = ""
|
||||
point = (float(event.mouse_region_x), float(event.mouse_region_y))
|
||||
self._path = [point]
|
||||
self._cursor = point
|
||||
self._fill_polygon = None
|
||||
self._fill_triangles = ()
|
||||
self._outline_polygon = None
|
||||
self._outline_segments = ()
|
||||
self._area_pointer = context.area.as_pointer()
|
||||
self._region_pointer = context.region.as_pointer()
|
||||
self._region_size = (context.region.width, context.region.height)
|
||||
self._handle = bpy.types.SpaceView3D.draw_handler_add(
|
||||
self._draw_region, (), "WINDOW", "POST_PIXEL"
|
||||
)
|
||||
self._set_status(context)
|
||||
context.window_manager.modal_handler_add(self)
|
||||
context.area.tag_redraw()
|
||||
return {"RUNNING_MODAL"}
|
||||
|
||||
def _complete(self, context):
|
||||
source_object = require_image_empty(self.source_object_name)
|
||||
matrix = deserialize_matrix(self.source_matrix_data)
|
||||
if self.gesture == "LINE":
|
||||
line = screen_path_to_image_pixels(
|
||||
(self._path[0], self._cursor),
|
||||
context.region,
|
||||
context.region_data,
|
||||
matrix,
|
||||
image_empty_bounds(source_object),
|
||||
tuple(source_object.data.size),
|
||||
)
|
||||
region_path = image_line_half_plane_polygon(
|
||||
line, tuple(source_object.data.size)
|
||||
)
|
||||
else:
|
||||
path = self._screen_path()
|
||||
if len(path) < 3:
|
||||
self._finish(context)
|
||||
return {"CANCELLED"}
|
||||
region_path = screen_path_to_image_pixels(
|
||||
path,
|
||||
context.region,
|
||||
context.region_data,
|
||||
matrix,
|
||||
image_empty_bounds(source_object),
|
||||
tuple(source_object.data.size),
|
||||
)
|
||||
self.region_path_json = RegionPath(
|
||||
points=region_path,
|
||||
invert=self.invert,
|
||||
fill_rule=self.preview_fill_rule,
|
||||
).to_json()
|
||||
self._finish(context)
|
||||
return self.execute(context)
|
||||
|
||||
def modal(self, context, event):
|
||||
active_tool = active_view3d_tool_id(context)
|
||||
expected_tool = self.active_tool_id or GESTURE_TOOL_IDS[self.gesture]
|
||||
if active_tool is not None and active_tool != expected_tool:
|
||||
self._finish(context)
|
||||
return {"CANCELLED"}
|
||||
if event.type in {"ESC", "RIGHTMOUSE"} and event.value == "PRESS":
|
||||
self._finish(context)
|
||||
return {"CANCELLED"}
|
||||
if event.type == "F" and event.value == "PRESS":
|
||||
self.invert = not self.invert
|
||||
self._set_status(context)
|
||||
context.area.tag_redraw()
|
||||
return {"RUNNING_MODAL"}
|
||||
if self.gesture == "POLYLINE":
|
||||
return self._modal_polyline(context, event)
|
||||
if event.type == "MOUSEMOVE":
|
||||
point = (float(event.mouse_region_x), float(event.mouse_region_y))
|
||||
if self.gesture == "LASSO":
|
||||
append_lasso_point(self._path, point)
|
||||
self._cursor = point
|
||||
elif self.gesture == "LINE":
|
||||
self._cursor = snapped_line_endpoint(self._path[0], point, event.ctrl)
|
||||
else:
|
||||
self._cursor = point
|
||||
context.area.tag_redraw()
|
||||
return {"RUNNING_MODAL"}
|
||||
if event.type == "LEFTMOUSE" and event.value == "RELEASE":
|
||||
point = (float(event.mouse_region_x), float(event.mouse_region_y))
|
||||
if self.gesture == "LASSO":
|
||||
append_lasso_point(self._path, point)
|
||||
self._cursor = point
|
||||
elif self.gesture == "LINE":
|
||||
self._cursor = snapped_line_endpoint(self._path[0], point, event.ctrl)
|
||||
if math.dist(self._path[0], self._cursor) < 2.0:
|
||||
self._finish(context)
|
||||
return {"CANCELLED"}
|
||||
else:
|
||||
self._cursor = point
|
||||
try:
|
||||
return self._complete(context)
|
||||
except (RuntimeError, TypeError, ValueError) as error:
|
||||
report_image_edit_exception(self, error)
|
||||
self._finish(context)
|
||||
return {"CANCELLED"}
|
||||
return {"PASS_THROUGH"}
|
||||
|
||||
def _modal_polyline(self, context, event):
|
||||
if event.type == "BACK_SPACE" and event.value == "PRESS":
|
||||
if len(self._path) > 1:
|
||||
self._path.pop()
|
||||
context.area.tag_redraw()
|
||||
return {"RUNNING_MODAL"}
|
||||
if event.type in {"RET", "NUMPAD_ENTER"} and event.value == "PRESS":
|
||||
return self._confirm_polyline(context)
|
||||
if event.type == "MOUSEMOVE":
|
||||
self._cursor = (
|
||||
float(event.mouse_region_x),
|
||||
float(event.mouse_region_y),
|
||||
)
|
||||
context.area.tag_redraw()
|
||||
return {"RUNNING_MODAL"}
|
||||
if event.type == "LEFTMOUSE" and event.value == "PRESS":
|
||||
point = (float(event.mouse_region_x), float(event.mouse_region_y))
|
||||
if (
|
||||
len(self._path) >= 3
|
||||
and math.dist(point, self._path[0]) <= POLYLINE_CLOSE_RADIUS
|
||||
):
|
||||
return self._confirm_polyline(context)
|
||||
if math.dist(point, self._path[-1]) >= 2.0:
|
||||
self._path.append(point)
|
||||
context.area.tag_redraw()
|
||||
return {"RUNNING_MODAL"}
|
||||
return {"PASS_THROUGH"}
|
||||
|
||||
def _confirm_polyline(self, context):
|
||||
self._cursor = self._path[0]
|
||||
try:
|
||||
return self._complete(context)
|
||||
except (RuntimeError, TypeError, ValueError) as error:
|
||||
report_image_edit_exception(self, error)
|
||||
self._finish(context)
|
||||
return {"CANCELLED"}
|
||||
|
||||
@@ -1,141 +0,0 @@
|
||||
from .image_edit import (
|
||||
ImageEditWarning,
|
||||
crop_pixels_to_alpha,
|
||||
polygon_bounds,
|
||||
replace_empty_image_pixels,
|
||||
)
|
||||
|
||||
def polygon_mask(image_size, polygon, fill_rule="EVEN_ODD"):
|
||||
import numpy as np
|
||||
|
||||
width, height = (int(value) for value in image_size)
|
||||
if width <= 0 or height <= 0:
|
||||
raise ValueError("The source image has no readable dimensions")
|
||||
if fill_rule not in {"EVEN_ODD", "NONZERO"}:
|
||||
raise ValueError(f"Unknown polygon fill rule: {fill_rule}")
|
||||
left, top, right, bottom = polygon_bounds(polygon)
|
||||
left = max(left, 0)
|
||||
top = max(top, 0)
|
||||
right = min(right, width)
|
||||
bottom = min(bottom, height)
|
||||
mask = np.zeros((height, width), dtype=bool)
|
||||
if right <= left or bottom <= top:
|
||||
return mask
|
||||
|
||||
points = np.asarray(polygon, dtype=np.float64)
|
||||
x = np.arange(left, right, dtype=np.float64) + 0.5
|
||||
for row_start in range(top, bottom, 128):
|
||||
row_end = min(row_start + 128, bottom)
|
||||
y = np.arange(row_start, row_end, dtype=np.float64) + 0.5
|
||||
state = np.zeros(
|
||||
(len(y), len(x)),
|
||||
dtype=bool if fill_rule == "EVEN_ODD" else np.int16,
|
||||
)
|
||||
previous = points[-1]
|
||||
for current in points:
|
||||
crosses = (current[1] > y) != (previous[1] > y)
|
||||
edge_x = (previous[0] - current[0]) * (y - current[1]) / (
|
||||
previous[1] - current[1] + 1e-300
|
||||
) + current[0]
|
||||
selected = crosses[:, None] & (x[None, :] < edge_x[:, None])
|
||||
if fill_rule == "EVEN_ODD":
|
||||
state ^= selected
|
||||
else:
|
||||
winding = 1 if current[1] > previous[1] else -1
|
||||
state += selected * winding
|
||||
previous = current
|
||||
mask[row_start:row_end, left:right] = state != 0
|
||||
return mask
|
||||
|
||||
def refined_mask_from_pixels(mask_pixels, mask_size, bounds, source_size):
|
||||
import numpy as np
|
||||
|
||||
mask_width, mask_height = (int(value) for value in mask_size)
|
||||
source_width, source_height = (int(value) for value in source_size)
|
||||
local = np.flipud(
|
||||
np.asarray(mask_pixels, dtype=np.float32).reshape((mask_height, mask_width, 4))
|
||||
)[:, :, 3]
|
||||
result = np.zeros((source_height, source_width), dtype=np.float32)
|
||||
left, top, right, bottom = (int(value) for value in bounds)
|
||||
source_left = max(left, 0)
|
||||
source_top = max(top, 0)
|
||||
source_right = min(right, source_width)
|
||||
source_bottom = min(bottom, source_height)
|
||||
if source_right <= source_left or source_bottom <= source_top:
|
||||
return result
|
||||
result[source_top:source_bottom, source_left:source_right] = local[
|
||||
source_top - top : source_bottom - top,
|
||||
source_left - left : source_right - left,
|
||||
]
|
||||
return result
|
||||
|
||||
|
||||
def visible_selection_mask(
|
||||
source_alpha, selection_mask, invert=False, alpha_threshold=0.0
|
||||
):
|
||||
"""Restrict a selection to source pixels above an Alpha threshold."""
|
||||
import numpy as np
|
||||
|
||||
alpha = np.asarray(source_alpha)
|
||||
mask = np.asarray(selection_mask, dtype=np.float32)
|
||||
if mask.shape != alpha.shape:
|
||||
raise ValueError("The selection does not match the source image")
|
||||
mask = np.clip(mask, 0.0, 1.0)
|
||||
if invert:
|
||||
mask = 1.0 - mask
|
||||
return np.where(alpha > float(alpha_threshold), mask, 0.0)
|
||||
|
||||
|
||||
def edit_image_mask_pixels(source_pixels, source_size, selection_mask, invert=False):
|
||||
"""Apply a mask while preserving source RGB, then tightly crop the canvas."""
|
||||
import numpy as np
|
||||
|
||||
width, height = (int(value) for value in source_size)
|
||||
rgba = np.flipud(
|
||||
np.asarray(source_pixels, dtype=np.float32).reshape((height, width, 4))
|
||||
)
|
||||
mask = visible_selection_mask(rgba[:, :, 3], selection_mask, invert)
|
||||
rgba[:, :, 3] *= mask
|
||||
return crop_pixels_to_alpha(np.flipud(rgba).ravel(), (0, 0, width, height))
|
||||
|
||||
|
||||
def edit_image_object(source_object, selection_mask, invert=False):
|
||||
import numpy as np
|
||||
|
||||
image = source_object.data
|
||||
source_size = tuple(image.size)
|
||||
source_pixels = np.empty(len(image.pixels), dtype=np.float32)
|
||||
image.pixels.foreach_get(source_pixels)
|
||||
pixels, crop_bounds = edit_image_mask_pixels(
|
||||
source_pixels, source_size, selection_mask, invert
|
||||
)
|
||||
left, top, right, bottom = crop_bounds
|
||||
return replace_empty_image_pixels(
|
||||
source_object,
|
||||
pixels,
|
||||
(right - left, bottom - top),
|
||||
crop_bounds,
|
||||
)
|
||||
|
||||
|
||||
def selected_image_rgba(
|
||||
source_object, polygon, invert=False, fill_rule="EVEN_ODD"
|
||||
):
|
||||
import numpy as np
|
||||
|
||||
image = source_object.data
|
||||
width, height = (int(value) for value in image.size)
|
||||
pixels = np.empty(len(image.pixels), dtype=np.float32)
|
||||
image.pixels.foreach_get(pixels)
|
||||
rgba = np.flipud(pixels.reshape((height, width, 4)))
|
||||
selection = visible_selection_mask(
|
||||
rgba[:, :, 3],
|
||||
polygon_mask((width, height), polygon, fill_rule),
|
||||
invert,
|
||||
)
|
||||
rgba[:, :, 3] *= selection
|
||||
if not np.any(rgba[:, :, 3] > 0.0):
|
||||
raise ImageEditWarning("The selection contains no visible image pixels")
|
||||
return rgba
|
||||
|
||||
|
||||
230
src/anyimage/operators/common/image_region.py
Normal file
230
src/anyimage/operators/common/image_region.py
Normal file
@@ -0,0 +1,230 @@
|
||||
import json
|
||||
import math
|
||||
from dataclasses import dataclass
|
||||
|
||||
|
||||
class ImageEditWarning(ValueError):
|
||||
pass
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class RegionPath:
|
||||
points: tuple[tuple[float, float], ...]
|
||||
invert: bool = False
|
||||
fill_rule: str = "EVEN_ODD"
|
||||
|
||||
def __post_init__(self):
|
||||
points = tuple(
|
||||
(float(point[0]), float(point[1]))
|
||||
for point in self.points
|
||||
)
|
||||
region_path_bounds(points)
|
||||
if self.fill_rule not in {"EVEN_ODD", "NONZERO"}:
|
||||
raise ValueError(f"Unknown Region Path fill rule: {self.fill_rule}")
|
||||
object.__setattr__(self, "points", points)
|
||||
object.__setattr__(self, "invert", bool(self.invert))
|
||||
|
||||
def to_json(self):
|
||||
return json.dumps(
|
||||
{
|
||||
"points": self.points,
|
||||
"invert": self.invert,
|
||||
"fill_rule": self.fill_rule,
|
||||
},
|
||||
separators=(",", ":"),
|
||||
)
|
||||
|
||||
@classmethod
|
||||
def from_json(cls, value):
|
||||
data = json.loads(value)
|
||||
if not isinstance(data, dict):
|
||||
raise ValueError("The Region Path data is invalid")
|
||||
return cls(
|
||||
points=tuple(data.get("points", ())),
|
||||
invert=bool(data.get("invert", False)),
|
||||
fill_rule=str(data.get("fill_rule", "EVEN_ODD")),
|
||||
)
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class RegionMask:
|
||||
values: object
|
||||
bounds: tuple[int, int, int, int]
|
||||
|
||||
def __post_init__(self):
|
||||
import numpy as np
|
||||
|
||||
values = np.asarray(self.values, dtype=np.float32)
|
||||
bounds = tuple(int(value) for value in self.bounds)
|
||||
if values.ndim != 2:
|
||||
raise ValueError("The region mask must be two-dimensional")
|
||||
if len(bounds) != 4:
|
||||
raise ValueError("The region bounds are invalid")
|
||||
left, top, right, bottom = bounds
|
||||
if right <= left or bottom <= top:
|
||||
raise ValueError("The region bounds are empty")
|
||||
if values.shape != (bottom - top, right - left):
|
||||
raise ValueError("The region mask does not match its bounds")
|
||||
if not np.isfinite(values).all():
|
||||
raise ValueError("The region mask contains invalid values")
|
||||
object.__setattr__(self, "values", np.clip(values, 0.0, 1.0))
|
||||
object.__setattr__(self, "bounds", bounds)
|
||||
|
||||
def full_values(self, image_size):
|
||||
import numpy as np
|
||||
|
||||
width, height = (int(value) for value in image_size)
|
||||
result = np.zeros((height, width), dtype=np.float32)
|
||||
left, top, right, bottom = self.bounds
|
||||
source_left = max(left, 0)
|
||||
source_top = max(top, 0)
|
||||
source_right = min(right, width)
|
||||
source_bottom = min(bottom, height)
|
||||
if source_right <= source_left or source_bottom <= source_top:
|
||||
return result
|
||||
result[source_top:source_bottom, source_left:source_right] = self.values[
|
||||
source_top - top : source_bottom - top,
|
||||
source_left - left : source_right - left,
|
||||
]
|
||||
return result
|
||||
|
||||
|
||||
def region_path_bounds(points):
|
||||
import numpy as np
|
||||
|
||||
points = np.asarray(points, dtype=np.float64)
|
||||
if points.ndim != 2 or points.shape[0] < 3 or points.shape[1] != 2:
|
||||
raise ValueError("The region does not define a valid image path")
|
||||
if not np.isfinite(points).all():
|
||||
raise ValueError("The region contains invalid image coordinates")
|
||||
left = int(math.floor(float(points[:, 0].min())))
|
||||
top = int(math.floor(float(points[:, 1].min())))
|
||||
right = int(math.ceil(float(points[:, 0].max())))
|
||||
bottom = int(math.ceil(float(points[:, 1].max())))
|
||||
if right <= left or bottom <= top:
|
||||
raise ValueError("The region does not define a valid image area")
|
||||
return left, top, right, bottom
|
||||
|
||||
|
||||
def _rasterize_path(image_size, points, fill_rule):
|
||||
import numpy as np
|
||||
|
||||
width, height = (int(value) for value in image_size)
|
||||
if width <= 0 or height <= 0:
|
||||
raise ValueError("The source image has no readable dimensions")
|
||||
left, top, right, bottom = region_path_bounds(points)
|
||||
left = max(left, 0)
|
||||
top = max(top, 0)
|
||||
right = min(right, width)
|
||||
bottom = min(bottom, height)
|
||||
mask = np.zeros((height, width), dtype=bool)
|
||||
if right <= left or bottom <= top:
|
||||
return mask
|
||||
|
||||
points = np.asarray(points, dtype=np.float64)
|
||||
x = np.arange(left, right, dtype=np.float64) + 0.5
|
||||
for row_start in range(top, bottom, 128):
|
||||
row_end = min(row_start + 128, bottom)
|
||||
y = np.arange(row_start, row_end, dtype=np.float64) + 0.5
|
||||
state = np.zeros(
|
||||
(len(y), len(x)),
|
||||
dtype=bool if fill_rule == "EVEN_ODD" else np.int16,
|
||||
)
|
||||
previous = points[-1]
|
||||
for current in points:
|
||||
crosses = (current[1] > y) != (previous[1] > y)
|
||||
edge_x = (previous[0] - current[0]) * (y - current[1]) / (
|
||||
previous[1] - current[1] + 1e-300
|
||||
) + current[0]
|
||||
selected = crosses[:, None] & (x[None, :] < edge_x[:, None])
|
||||
if fill_rule == "EVEN_ODD":
|
||||
state ^= selected
|
||||
else:
|
||||
winding = 1 if current[1] > previous[1] else -1
|
||||
state += selected * winding
|
||||
previous = current
|
||||
mask[row_start:row_end, left:right] = state != 0
|
||||
return mask
|
||||
|
||||
|
||||
def _antialias_mask(mask):
|
||||
import numpy as np
|
||||
|
||||
values = np.asarray(mask, dtype=np.float32)
|
||||
kernel = np.asarray((0.029, 0.235, 0.472, 0.235, 0.029), dtype=np.float32)
|
||||
radius = len(kernel) // 2
|
||||
horizontal = np.zeros_like(values)
|
||||
padded = np.pad(values, ((0, 0), (radius, radius)), mode="edge")
|
||||
for offset, weight in enumerate(kernel):
|
||||
horizontal += padded[:, offset : offset + values.shape[1]] * weight
|
||||
result = np.zeros_like(values)
|
||||
padded = np.pad(horizontal, ((radius, radius), (0, 0)), mode="edge")
|
||||
for offset, weight in enumerate(kernel):
|
||||
result += padded[offset : offset + values.shape[0]] * weight
|
||||
return result
|
||||
|
||||
|
||||
def rasterize_region_path(
|
||||
image_size,
|
||||
region_path,
|
||||
*,
|
||||
source_alpha=None,
|
||||
alpha_threshold=0.0,
|
||||
antialias=False,
|
||||
):
|
||||
import numpy as np
|
||||
|
||||
width, height = (int(value) for value in image_size)
|
||||
mask = _rasterize_path(
|
||||
(width, height),
|
||||
region_path.points,
|
||||
region_path.fill_rule,
|
||||
)
|
||||
if antialias:
|
||||
mask = _antialias_mask(mask)
|
||||
else:
|
||||
mask = mask.astype(np.float32)
|
||||
if region_path.invert:
|
||||
mask = 1.0 - mask
|
||||
if source_alpha is not None:
|
||||
alpha = np.asarray(source_alpha)
|
||||
if alpha.shape != (height, width):
|
||||
raise ValueError("The source Alpha does not match the image region")
|
||||
mask = np.where(alpha > float(alpha_threshold), mask, 0.0)
|
||||
visible_y, visible_x = np.nonzero(mask > float(alpha_threshold))
|
||||
if not len(visible_x):
|
||||
raise ImageEditWarning("The region contains no visible image pixels")
|
||||
bounds = (
|
||||
int(visible_x.min()),
|
||||
int(visible_y.min()),
|
||||
int(visible_x.max()) + 1,
|
||||
int(visible_y.max()) + 1,
|
||||
)
|
||||
left, top, right, bottom = bounds
|
||||
return RegionMask(mask[top:bottom, left:right], bounds)
|
||||
|
||||
|
||||
def crop_pixels_to_alpha(pixels, bounds):
|
||||
"""Crop Blender-order RGBA pixels to their nonzero Alpha bounds."""
|
||||
import numpy as np
|
||||
|
||||
left, top, right, bottom = bounds
|
||||
width = right - left
|
||||
height = bottom - top
|
||||
rgba = np.flipud(
|
||||
np.asarray(pixels, dtype=np.float32).reshape((height, width, 4))
|
||||
)
|
||||
visible_y, visible_x = np.nonzero(rgba[:, :, 3] > 0.0)
|
||||
if not len(visible_x):
|
||||
raise ImageEditWarning("The edit contains no visible image pixels")
|
||||
content_left = int(visible_x.min())
|
||||
content_top = int(visible_y.min())
|
||||
content_right = int(visible_x.max()) + 1
|
||||
content_bottom = int(visible_y.max()) + 1
|
||||
output = rgba[content_top:content_bottom, content_left:content_right].copy()
|
||||
return np.flipud(output).ravel(), (
|
||||
left + content_left,
|
||||
top + content_top,
|
||||
left + content_right,
|
||||
top + content_bottom,
|
||||
)
|
||||
@@ -1,7 +1,130 @@
|
||||
from dataclasses import dataclass
|
||||
from pathlib import Path
|
||||
|
||||
import bpy
|
||||
|
||||
from .image import configure_image_user, is_animated_image
|
||||
from .node_asset import load_shadeless_node_group
|
||||
from .image_data import (
|
||||
configure_image_user,
|
||||
image_base_name,
|
||||
image_pixels,
|
||||
is_animated_image,
|
||||
)
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class ImageGeometry:
|
||||
points: object
|
||||
intrinsics: object
|
||||
valid: object
|
||||
calibration_valid: object
|
||||
image_size: tuple
|
||||
reference_depth: float
|
||||
region_transparent_fraction: float
|
||||
|
||||
|
||||
def load_image_geometry(path):
|
||||
"""Load and validate the shared image geometry Job artifact."""
|
||||
import numpy as np
|
||||
|
||||
with np.load(path, allow_pickle=False) as archive:
|
||||
geometry = ImageGeometry(
|
||||
points=np.asarray(archive["points"], dtype=np.float32),
|
||||
intrinsics=np.asarray(archive["intrinsics"], dtype=np.float32),
|
||||
valid=np.asarray(archive["valid"], dtype=bool),
|
||||
calibration_valid=np.asarray(archive["calibration_valid"], dtype=bool),
|
||||
image_size=tuple(int(value) for value in archive["image_size"]),
|
||||
reference_depth=float(archive["reference_depth"]),
|
||||
region_transparent_fraction=float(
|
||||
archive["region_transparent_fraction"]
|
||||
),
|
||||
)
|
||||
if (
|
||||
geometry.points.ndim != 3
|
||||
or geometry.points.shape[2] != 3
|
||||
or geometry.valid.shape != geometry.points.shape[:2]
|
||||
or geometry.calibration_valid.shape != geometry.points.shape[:2]
|
||||
):
|
||||
raise ValueError("The mesh point field is invalid")
|
||||
if (
|
||||
geometry.intrinsics.shape != (3, 3)
|
||||
or not geometry.reference_depth > 0.0
|
||||
or not 0.0 <= geometry.region_transparent_fraction <= 1.0
|
||||
):
|
||||
raise ValueError("The mesh camera field is invalid")
|
||||
return geometry
|
||||
|
||||
|
||||
NODE_ASSET_NAME = "O_AnyImage.blend"
|
||||
|
||||
|
||||
def node_asset_path():
|
||||
return Path(__file__).resolve().parents[2] / "assets" / NODE_ASSET_NAME
|
||||
|
||||
|
||||
def load_asset_node_group(group_name):
|
||||
existing = bpy.data.node_groups.get(group_name)
|
||||
if existing is not None:
|
||||
return existing
|
||||
|
||||
asset_path = node_asset_path()
|
||||
if not asset_path.is_file():
|
||||
raise RuntimeError(f"Missing node asset: {asset_path}")
|
||||
if bpy.app.version < (5, 0, 0):
|
||||
directory = str(asset_path) + "/NodeTree/"
|
||||
result = bpy.ops.wm.append(
|
||||
filepath=directory + group_name,
|
||||
directory=directory,
|
||||
filename=group_name,
|
||||
link=False,
|
||||
do_reuse_local_id=True,
|
||||
)
|
||||
if "FINISHED" not in result:
|
||||
raise RuntimeError("Unable to append the node asset")
|
||||
node_group = bpy.data.node_groups.get(group_name)
|
||||
if node_group is None:
|
||||
raise RuntimeError("Appended node asset is unavailable")
|
||||
node_group.use_fake_user = True
|
||||
return node_group
|
||||
|
||||
with bpy.data.libraries.load(str(asset_path), link=True) as (source, target):
|
||||
if group_name not in source.node_groups:
|
||||
raise RuntimeError(f"Node asset is missing {group_name}")
|
||||
target.node_groups = [group_name]
|
||||
linked_group = target.node_groups[0]
|
||||
node_group = bpy.data.pack_linked_ids_hierarchy(root_id=linked_group)
|
||||
if linked_group != node_group and linked_group.users == 0:
|
||||
bpy.data.node_groups.remove(linked_group)
|
||||
node_group.use_fake_user = True
|
||||
return node_group
|
||||
|
||||
|
||||
def modifier_input_identifier(node_group, name):
|
||||
for item in node_group.interface.items_tree:
|
||||
if item.item_type != "SOCKET" or item.in_out != "INPUT":
|
||||
continue
|
||||
if item.name == name:
|
||||
return item.identifier
|
||||
raise RuntimeError(f"Geometry Nodes input is missing: {name}")
|
||||
|
||||
|
||||
def _modifier_input_slots(modifier):
|
||||
"""Return the runtime input collection, or None for the legacy API."""
|
||||
properties = getattr(modifier, "properties", None)
|
||||
if properties is None:
|
||||
return None
|
||||
return getattr(properties, "inputs", None)
|
||||
|
||||
|
||||
def set_modifier_input(modifier, identifier, value):
|
||||
"""Assign a value to a Geometry Nodes modifier input socket."""
|
||||
inputs = _modifier_input_slots(modifier)
|
||||
if inputs is not None:
|
||||
# Blender 5.2+ exposes inputs as runtime RNA keyed by socket identifier.
|
||||
getattr(inputs, identifier).value = value
|
||||
else:
|
||||
# Blender 4.x exposes inputs as modifier ID properties.
|
||||
modifier[identifier] = value
|
||||
|
||||
|
||||
DISPLACEMENT_STRENGTH = 1.0
|
||||
DISPLACEMENT_MIDLEVEL = 0.0
|
||||
@@ -11,6 +134,7 @@ BACKGROUND_DEPTH_ATTRIBUTE = "o_background_depth"
|
||||
DEPTH_SCALE_ATTRIBUTE = "o_depth_scale"
|
||||
IMAGE_MATERIAL_NODE_GROUP_NAME = "O Image Layer"
|
||||
DEPTH_MATERIAL_NODE_GROUP_NAME = "O Image Depth Layer"
|
||||
SHADELESS_NODE_GROUP_NAME = "O Shadeless"
|
||||
VIEW_TRANSFORM_COLOR_SPACES = (
|
||||
("ACES 1.3", "ACES 1.3 sRGB"),
|
||||
("ACES 2.0", "ACES 2.0 sRGB"),
|
||||
@@ -44,10 +168,7 @@ def set_material_color_space(image, scene=None):
|
||||
or getattr(image, "is_dirty", False)
|
||||
)
|
||||
):
|
||||
import numpy as np
|
||||
|
||||
pixels = np.empty(len(image.pixels), dtype=np.float32)
|
||||
image.pixels.foreach_get(pixels)
|
||||
pixels = image_pixels(image)
|
||||
for color_space in dict.fromkeys((preferred, "sRGB", original)):
|
||||
if not color_space:
|
||||
continue
|
||||
@@ -251,8 +372,8 @@ def material_node_group(depth_plane=False):
|
||||
return group
|
||||
|
||||
|
||||
def create_material(
|
||||
name,
|
||||
def create_image_material(
|
||||
source_object,
|
||||
color_image,
|
||||
depth_scale,
|
||||
displacement_image=None,
|
||||
@@ -270,7 +391,7 @@ def create_material(
|
||||
normal_space="TANGENT",
|
||||
scene=None,
|
||||
):
|
||||
material = bpy.data.materials.new(name)
|
||||
material = bpy.data.materials.new(image_base_name(source_object.data))
|
||||
material.use_nodes = True
|
||||
if hasattr(material, "surface_render_method"):
|
||||
material.surface_render_method = "DITHERED"
|
||||
@@ -332,7 +453,7 @@ def create_material(
|
||||
links.new(normal_texture.outputs["Color"], material_inputs.inputs["Normal"])
|
||||
if shadeless:
|
||||
shader = nodes.new("ShaderNodeGroup")
|
||||
shader.node_tree = load_shadeless_node_group()
|
||||
shader.node_tree = load_asset_node_group(SHADELESS_NODE_GROUP_NAME)
|
||||
shader.name = SHADELESS_NODE_GROUP_NAME
|
||||
transparent = nodes.new("ShaderNodeBsdfTransparent")
|
||||
mix_shader = nodes.new("ShaderNodeMixShader")
|
||||
@@ -434,3 +555,22 @@ def set_displacement_only(material):
|
||||
if owner is not None and hasattr(owner, property_name):
|
||||
setattr(owner, property_name, "DISPLACEMENT")
|
||||
return
|
||||
|
||||
|
||||
def finalize_object_result(
|
||||
context,
|
||||
source_object,
|
||||
result_object,
|
||||
*,
|
||||
keep_source=False,
|
||||
):
|
||||
"""Activate a result object and optionally replace its source object."""
|
||||
source_name = source_object.name
|
||||
for selected in context.selected_objects:
|
||||
selected.select_set(False)
|
||||
result_object.select_set(True)
|
||||
context.view_layer.objects.active = result_object
|
||||
if not keep_source:
|
||||
bpy.data.objects.remove(source_object, do_unlink=True)
|
||||
result_object.name = source_name
|
||||
return result_object
|
||||
@@ -1,119 +0,0 @@
|
||||
from pathlib import Path
|
||||
|
||||
import bpy
|
||||
|
||||
|
||||
NODE_ASSET_NAME = "O_AnyImage.blend"
|
||||
MESH_PLANE_NODE_GROUP_NAME = "O Mesh Plane"
|
||||
SHADELESS_NODE_GROUP_NAME = "O Shadeless"
|
||||
|
||||
|
||||
def node_asset_path():
|
||||
return Path(__file__).resolve().parents[2] / "assets" / NODE_ASSET_NAME
|
||||
|
||||
|
||||
def load_asset_node_group(group_name):
|
||||
existing = bpy.data.node_groups.get(group_name)
|
||||
if existing is not None:
|
||||
for item in existing.interface.items_tree:
|
||||
if (
|
||||
item.item_type == "SOCKET"
|
||||
and item.in_out == "INPUT"
|
||||
and item.name == "Normal"
|
||||
):
|
||||
item.default_value = (0.5, 0.5, 1.0, 1.0)
|
||||
for node in existing.nodes:
|
||||
if node.bl_idname == "ShaderNodeNormalMap":
|
||||
node.space = "TANGENT"
|
||||
return existing
|
||||
|
||||
asset_path = node_asset_path()
|
||||
if not asset_path.is_file():
|
||||
raise RuntimeError(f"Missing node asset: {asset_path}")
|
||||
if bpy.app.version < (5, 0, 0):
|
||||
directory = str(asset_path) + "/NodeTree/"
|
||||
result = bpy.ops.wm.append(
|
||||
filepath=directory + group_name,
|
||||
directory=directory,
|
||||
filename=group_name,
|
||||
link=False,
|
||||
do_reuse_local_id=True,
|
||||
)
|
||||
if "FINISHED" not in result:
|
||||
raise RuntimeError("Unable to append the node asset")
|
||||
node_group = bpy.data.node_groups.get(group_name)
|
||||
if node_group is None:
|
||||
raise RuntimeError("Appended node asset is unavailable")
|
||||
node_group.use_fake_user = True
|
||||
return node_group
|
||||
|
||||
with bpy.data.libraries.load(str(asset_path), link=True) as (source, target):
|
||||
if group_name not in source.node_groups:
|
||||
raise RuntimeError(f"Node asset is missing {group_name}")
|
||||
target.node_groups = [group_name]
|
||||
linked_group = target.node_groups[0]
|
||||
node_group = bpy.data.pack_linked_ids_hierarchy(root_id=linked_group)
|
||||
if linked_group != node_group and linked_group.users == 0:
|
||||
bpy.data.node_groups.remove(linked_group)
|
||||
node_group.use_fake_user = True
|
||||
return node_group
|
||||
|
||||
|
||||
def load_shadeless_node_group():
|
||||
return load_asset_node_group(SHADELESS_NODE_GROUP_NAME)
|
||||
|
||||
|
||||
def load_mesh_plane_node_group():
|
||||
return load_asset_node_group(MESH_PLANE_NODE_GROUP_NAME)
|
||||
|
||||
|
||||
def modifier_input_identifier(node_group, name):
|
||||
for item in node_group.interface.items_tree:
|
||||
if item.item_type != "SOCKET" or item.in_out != "INPUT":
|
||||
continue
|
||||
if item.name == name:
|
||||
return item.identifier
|
||||
raise RuntimeError(f"Geometry Nodes input is missing: {name}")
|
||||
|
||||
|
||||
def modifier_input_slots(modifier):
|
||||
"""Return the runtime input collection, or None for the legacy API."""
|
||||
properties = getattr(modifier, "properties", None)
|
||||
if properties is None:
|
||||
return None
|
||||
return getattr(properties, "inputs", None)
|
||||
|
||||
|
||||
def set_modifier_input(modifier, identifier, value):
|
||||
"""Assign a value to a Geometry Nodes modifier input socket."""
|
||||
inputs = modifier_input_slots(modifier)
|
||||
if inputs is not None:
|
||||
# Blender 5.2+ exposes inputs as runtime RNA keyed by socket identifier.
|
||||
getattr(inputs, identifier).value = value
|
||||
else:
|
||||
# Blender 4.x exposes inputs as modifier ID properties.
|
||||
modifier[identifier] = value
|
||||
|
||||
|
||||
def get_modifier_input(modifier, identifier):
|
||||
"""Read a Geometry Nodes modifier input socket value."""
|
||||
inputs = modifier_input_slots(modifier)
|
||||
if inputs is not None:
|
||||
return getattr(inputs, identifier).value
|
||||
return modifier[identifier]
|
||||
|
||||
|
||||
def modifier_input_driver_path(modifier, identifier):
|
||||
"""Return the driver data path for a modifier input socket."""
|
||||
if modifier_input_slots(modifier) is not None:
|
||||
return f'modifiers["{modifier.name}"].properties.inputs.{identifier}.value'
|
||||
return f'modifiers["{modifier.name}"]["{identifier}"]'
|
||||
|
||||
|
||||
def drive_modifier_input(owner_object, modifier, identifier, expression="frame"):
|
||||
"""Add a scripted driver to a modifier input so it follows the scene frame."""
|
||||
path = modifier_input_driver_path(modifier, identifier)
|
||||
driver = owner_object.driver_add(path)
|
||||
driver.driver.type = "SCRIPTED"
|
||||
driver.driver.expression = expression
|
||||
return driver
|
||||
@@ -1,19 +1,20 @@
|
||||
import bpy
|
||||
|
||||
from ..common.image import (
|
||||
from ..common.mesh_result import load_image_geometry
|
||||
from ..common.image_data import (
|
||||
color_data_name,
|
||||
create_color_image,
|
||||
depth_data_name,
|
||||
is_image_empty,
|
||||
material_data_name,
|
||||
normal_data_name,
|
||||
plane_bounds,
|
||||
image_pixels,
|
||||
load_color_result_image,
|
||||
load_normal_result_image,
|
||||
image_empty_bounds,
|
||||
require_image_empty,
|
||||
)
|
||||
from ..common.material import create_material
|
||||
from ..common.node_asset import modifier_input_identifier, set_modifier_input
|
||||
from ..common.normal_map import bake_tangent_normal_image
|
||||
from ..common.operator import push_undo
|
||||
from ..common.mesh_result import create_image_material
|
||||
from ..common.mesh_result import modifier_input_identifier, set_modifier_input
|
||||
from ..common.mesh_result import finalize_object_result
|
||||
from .mesh_plane import create_mesh_plane_mesh, create_mesh_plane_object
|
||||
from .normal_map import bake_tangent_normal_image
|
||||
from .nodes import depth_plane_node_group
|
||||
|
||||
|
||||
@@ -23,10 +24,7 @@ def copy_color_image(source_object):
|
||||
if getattr(source, "source", None) == "GENERATED" or getattr(
|
||||
source, "is_dirty", False
|
||||
):
|
||||
import numpy as np
|
||||
|
||||
pixels = np.empty(len(source.pixels), dtype=np.float32)
|
||||
source.pixels.foreach_get(pixels)
|
||||
pixels = image_pixels(source)
|
||||
image.pixels.foreach_set(pixels)
|
||||
image.update()
|
||||
image.name = color_data_name(source_object)
|
||||
@@ -38,7 +36,7 @@ def build_mesh_plane(context, source_object, subdivisions, material):
|
||||
plane = create_mesh_plane_object(
|
||||
context,
|
||||
source_name,
|
||||
plane_bounds(source_object),
|
||||
image_empty_bounds(source_object),
|
||||
material,
|
||||
matrix_world=source_object.matrix_world,
|
||||
subdivisions=subdivisions,
|
||||
@@ -46,58 +44,32 @@ def build_mesh_plane(context, source_object, subdivisions, material):
|
||||
for polygon in plane.data.polygons:
|
||||
polygon.use_smooth = True
|
||||
plane["anyimage_mesh_shape"] = "PLANE"
|
||||
for selected in context.selected_objects:
|
||||
selected.select_set(False)
|
||||
plane.select_set(True)
|
||||
context.view_layer.objects.active = plane
|
||||
bpy.data.objects.remove(source_object, do_unlink=True)
|
||||
plane.name = source_name
|
||||
return plane
|
||||
|
||||
|
||||
def load_depth_field(path):
|
||||
import numpy as np
|
||||
|
||||
with np.load(path, allow_pickle=False) as archive:
|
||||
points = np.asarray(archive["points"], dtype=np.float32)
|
||||
intrinsics = np.asarray(archive["intrinsics"], dtype=np.float32)
|
||||
valid = np.asarray(archive["valid"], dtype=bool)
|
||||
calibration_valid = np.asarray(archive["calibration_valid"], dtype=bool)
|
||||
image_size = tuple(int(value) for value in archive["image_size"])
|
||||
reference_depth = float(archive["reference_depth"])
|
||||
if (
|
||||
points.ndim != 3
|
||||
or points.shape[2] != 3
|
||||
or valid.shape != points.shape[:2]
|
||||
or calibration_valid.shape != points.shape[:2]
|
||||
):
|
||||
raise ValueError("The mesh point field is invalid")
|
||||
if intrinsics.shape != (3, 3) or not reference_depth > 0.0:
|
||||
raise ValueError("The mesh camera field is invalid")
|
||||
return points, intrinsics, valid, calibration_valid, image_size, reference_depth
|
||||
return finalize_object_result(context, source_object, plane)
|
||||
|
||||
|
||||
def anchored_point_field(source_object, field):
|
||||
import numpy as np
|
||||
|
||||
points, intrinsics, valid, _calibration, image_size, reference_depth = field
|
||||
width, height = image_size
|
||||
x_min, x_max, y_min, y_max = plane_bounds(source_object)
|
||||
points = field.points
|
||||
intrinsics = field.intrinsics
|
||||
valid = field.valid
|
||||
width, height = field.image_size
|
||||
x_min, x_max, y_min, y_max = image_empty_bounds(source_object)
|
||||
plane_width = x_max - x_min
|
||||
plane_height = y_max - y_min
|
||||
fx, fy = float(intrinsics[0, 0]), float(intrinsics[1, 1])
|
||||
cx, cy = float(intrinsics[0, 2]), float(intrinsics[1, 2])
|
||||
if fx <= 0.0 or fy <= 0.0:
|
||||
raise ValueError("Camera intrinsics contain an invalid focal length")
|
||||
scale_x = plane_width * fx / (max(width - 1, 1) * reference_depth)
|
||||
scale_y = plane_height * fy / (max(height - 1, 1) * reference_depth)
|
||||
scale_x = plane_width * fx / (max(width - 1, 1) * field.reference_depth)
|
||||
scale_y = plane_height * fy / (max(height - 1, 1) * field.reference_depth)
|
||||
scale_z = float(np.sqrt(scale_x * scale_y))
|
||||
camera_x = x_min + plane_width * cx / max(width - 1, 1)
|
||||
camera_y = y_max - plane_height * cy / max(height - 1, 1)
|
||||
anchored = np.empty_like(points, dtype=np.float32)
|
||||
anchored[..., 0] = camera_x + points[..., 0] * scale_x
|
||||
anchored[..., 1] = camera_y - points[..., 1] * scale_y
|
||||
anchored[..., 2] = (reference_depth - points[..., 2]) * scale_z
|
||||
anchored[..., 2] = (field.reference_depth - points[..., 2]) * scale_z
|
||||
return anchored, valid, points[..., 2]
|
||||
|
||||
|
||||
@@ -122,17 +94,9 @@ def create_depth_image(source_object, anchored_points):
|
||||
return image
|
||||
|
||||
|
||||
def load_normal_image(path, source_object):
|
||||
image = bpy.data.images.load(str(path), check_existing=False)
|
||||
image.name = normal_data_name(source_object)
|
||||
image.colorspace_settings.name = "Non-Color"
|
||||
image.pack()
|
||||
return image
|
||||
|
||||
|
||||
def build_depth_plane(context, source_object, subdivisions, material, field):
|
||||
source_name = source_object.name
|
||||
mesh = create_mesh_plane_mesh(source_name, plane_bounds(source_object))
|
||||
mesh = create_mesh_plane_mesh(source_name, image_empty_bounds(source_object))
|
||||
for polygon in mesh.polygons:
|
||||
polygon.use_smooth = True
|
||||
mesh.materials.append(material)
|
||||
@@ -160,33 +124,21 @@ def build_depth_plane(context, source_object, subdivisions, material, field):
|
||||
modifier_input_identifier(node_group, name),
|
||||
value,
|
||||
)
|
||||
for selected in context.selected_objects:
|
||||
selected.select_set(False)
|
||||
plane.select_set(True)
|
||||
context.view_layer.objects.active = plane
|
||||
bpy.data.objects.remove(source_object, do_unlink=True)
|
||||
plane.name = source_name
|
||||
return plane
|
||||
return finalize_object_result(context, source_object, plane)
|
||||
|
||||
|
||||
def build_depth_plane_response(context, result, operator):
|
||||
source_object = bpy.data.objects.get(operator.source_object_name)
|
||||
if not is_image_empty(source_object):
|
||||
raise RuntimeError("The source Image Empty no longer exists")
|
||||
color_image = create_color_image(
|
||||
source_object,
|
||||
result.directory,
|
||||
assign_to_source=False,
|
||||
)
|
||||
source_object = require_image_empty(operator.source_object_name)
|
||||
color_image = load_color_result_image(source_object, result.directory)
|
||||
color_image.name = color_data_name(source_object)
|
||||
normal_path = result.directory / "normal.png"
|
||||
normal_image = (
|
||||
load_normal_image(normal_path, source_object)
|
||||
load_normal_result_image(normal_path, source_object)
|
||||
if normal_path.is_file()
|
||||
else None
|
||||
)
|
||||
material = create_material(
|
||||
material_data_name(source_object),
|
||||
material = create_image_material(
|
||||
source_object,
|
||||
color_image,
|
||||
0.0,
|
||||
normal_image=normal_image,
|
||||
@@ -197,7 +149,7 @@ def build_depth_plane_response(context, result, operator):
|
||||
source_object,
|
||||
operator.mesh_detail,
|
||||
material,
|
||||
load_depth_field(result.file("field")),
|
||||
load_image_geometry(result.file("image_geometry")),
|
||||
)
|
||||
if normal_image is not None:
|
||||
bake_tangent_normal_image(
|
||||
@@ -206,4 +158,5 @@ def build_depth_plane_response(context, result, operator):
|
||||
plane,
|
||||
normal_image,
|
||||
)
|
||||
push_undo("Convert to Depth Plane")
|
||||
if "FINISHED" not in bpy.ops.ed.undo_push(message="Convert to Depth Plane"):
|
||||
raise RuntimeError("Unable to create the Convert to Depth Plane Undo step")
|
||||
|
||||
@@ -8,30 +8,29 @@ from ...server.media.resolution import (
|
||||
MAX_MAX_AI_INPUT_SIZE,
|
||||
MIN_MAX_AI_INPUT_SIZE,
|
||||
)
|
||||
from ..common.image import (
|
||||
analysis_source_image,
|
||||
from ..common.image_data import (
|
||||
cleanup_image_input,
|
||||
image_user_settings,
|
||||
is_image_empty,
|
||||
material_data_name,
|
||||
prepare_image_analysis_inputs,
|
||||
remove_temporary_input,
|
||||
scaled_process_resolution,
|
||||
prepare_image_input,
|
||||
source_start_frame,
|
||||
video_total_frames,
|
||||
)
|
||||
from ..common.material import create_material
|
||||
from ..common.operator import (
|
||||
HIDDEN,
|
||||
cleanup_temporary_input,
|
||||
from ..common.mesh_result import create_image_material
|
||||
from ..common.ai_job import (
|
||||
da3_parameters,
|
||||
input_path,
|
||||
require_input_path,
|
||||
invoke_ai_setup_if_needed,
|
||||
moge2_parameters,
|
||||
production_device,
|
||||
require_environment,
|
||||
require_model,
|
||||
source_start_frame,
|
||||
)
|
||||
from .build import build_depth_plane_response, build_mesh_plane, copy_color_image
|
||||
from .resolution import scaled_process_resolution
|
||||
|
||||
|
||||
_HIDDEN = {"HIDDEN", "SKIP_SAVE"}
|
||||
|
||||
|
||||
def depth_plane_settings(context):
|
||||
@@ -46,48 +45,39 @@ def depth_plane_settings(context):
|
||||
class GenerateDepthPlane(JobOperatorBase, bpy.types.Operator):
|
||||
bl_idname = "anyimage.generate_depth_plane"
|
||||
bl_label = "Generate Depth Plane"
|
||||
job_type = "prepare-geometry-data"
|
||||
job_type = "generate-depth-plane"
|
||||
starting_message = "Preparing Depth Plane"
|
||||
|
||||
input_path: bpy.props.StringProperty(options=HIDDEN)
|
||||
analysis_input_path: bpy.props.StringProperty(options=HIDDEN)
|
||||
source_object_name: bpy.props.StringProperty(options=HIDDEN)
|
||||
mesh_detail: bpy.props.IntProperty(default=0, min=0, max=10, options=HIDDEN)
|
||||
model: bpy.props.StringProperty(default=DEFAULT_MOGE2_MODEL_KEY, options=HIDDEN)
|
||||
process_res: bpy.props.IntProperty(default=1008, min=1, options=HIDDEN)
|
||||
resolution_level: bpy.props.IntProperty(default=5, min=0, max=9, options=HIDDEN)
|
||||
input_path: bpy.props.StringProperty(options=_HIDDEN)
|
||||
source_object_name: bpy.props.StringProperty(options=_HIDDEN)
|
||||
mesh_detail: bpy.props.IntProperty(default=0, min=0, max=10, options=_HIDDEN)
|
||||
model: bpy.props.StringProperty(default=DEFAULT_MOGE2_MODEL_KEY, options=_HIDDEN)
|
||||
process_res: bpy.props.IntProperty(default=1008, min=1, options=_HIDDEN)
|
||||
resolution_level: bpy.props.IntProperty(default=5, min=0, max=9, options=_HIDDEN)
|
||||
max_input_size: bpy.props.IntProperty(
|
||||
default=DEFAULT_MAX_AI_INPUT_SIZE,
|
||||
min=MIN_MAX_AI_INPUT_SIZE,
|
||||
max=MAX_MAX_AI_INPUT_SIZE,
|
||||
options=HIDDEN,
|
||||
options=_HIDDEN,
|
||||
)
|
||||
delete_input: bpy.props.BoolProperty(default=False, options=HIDDEN)
|
||||
delete_analysis_input: bpy.props.BoolProperty(default=False, options=HIDDEN)
|
||||
delete_input: bpy.props.BoolProperty(default=False, options=_HIDDEN)
|
||||
|
||||
def request(self, context):
|
||||
require_environment()
|
||||
require_model(self.model)
|
||||
parameters = {
|
||||
"input": str(input_path(self.input_path)),
|
||||
"analysis_input": str(input_path(self.analysis_input_path)),
|
||||
"shape": "DEPTH_PLANE",
|
||||
"polygon": None,
|
||||
"mesh_detail": self.mesh_detail,
|
||||
"input": str(require_input_path(self.input_path)),
|
||||
"device": production_device().lower(),
|
||||
"video_frames": 1,
|
||||
"start_frame": source_start_frame(self.source_object_name),
|
||||
"max_input_size": int(self.max_input_size),
|
||||
"refine_selection": False,
|
||||
"normal_analysis": False,
|
||||
"invert": False,
|
||||
}
|
||||
settings = context.scene.anyimage_settings
|
||||
if self.model.startswith("MOGE2_"):
|
||||
parameters.update(
|
||||
moge2_parameters(
|
||||
settings,
|
||||
input_path(self.input_path),
|
||||
require_input_path(self.input_path),
|
||||
self.model,
|
||||
self.resolution_level,
|
||||
source_start_frame(self.source_object_name),
|
||||
@@ -97,13 +87,12 @@ class GenerateDepthPlane(JobOperatorBase, bpy.types.Operator):
|
||||
parameters.update(
|
||||
da3_parameters(
|
||||
settings,
|
||||
input_path(self.input_path),
|
||||
require_input_path(self.input_path),
|
||||
self.model,
|
||||
self.process_res,
|
||||
source_start_frame(self.source_object_name),
|
||||
)
|
||||
)
|
||||
parameters.update(shape="DEPTH_PLANE", polygon=None)
|
||||
return parameters
|
||||
|
||||
def response(self, context, result):
|
||||
@@ -111,11 +100,7 @@ class GenerateDepthPlane(JobOperatorBase, bpy.types.Operator):
|
||||
return "Depth Plane is ready"
|
||||
|
||||
def cleanup(self):
|
||||
cleanup_temporary_input(self.input_path, self.delete_input)
|
||||
cleanup_temporary_input(
|
||||
self.analysis_input_path,
|
||||
self.delete_analysis_input,
|
||||
)
|
||||
cleanup_image_input(self.input_path, self.delete_input)
|
||||
|
||||
|
||||
def start_depth_plane_job(operator, context):
|
||||
@@ -127,13 +112,13 @@ def start_depth_plane_job(operator, context):
|
||||
if setup_result is not None:
|
||||
return setup_result
|
||||
try:
|
||||
source_path, delete_input, analysis_path, delete_analysis_input = (
|
||||
prepare_image_analysis_inputs(source_object.data, context.scene)
|
||||
source_path, delete_input = prepare_image_input(
|
||||
source_object.data,
|
||||
context.scene,
|
||||
)
|
||||
analysis_image = analysis_source_image(source_object.data)
|
||||
model, process_factor, resolution_level = depth_plane_settings(context)
|
||||
process_res = (
|
||||
scaled_process_resolution(analysis_image, process_factor)
|
||||
scaled_process_resolution(source_object.data, process_factor)
|
||||
if not model.startswith("MOGE2_")
|
||||
else 1
|
||||
)
|
||||
@@ -142,7 +127,6 @@ def start_depth_plane_job(operator, context):
|
||||
return {"CANCELLED"}
|
||||
result = bpy.ops.anyimage.generate_depth_plane(
|
||||
input_path=str(source_path),
|
||||
analysis_input_path=str(analysis_path),
|
||||
source_object_name=source_object.name,
|
||||
mesh_detail=operator.mesh_detail,
|
||||
model=model,
|
||||
@@ -150,13 +134,10 @@ def start_depth_plane_job(operator, context):
|
||||
resolution_level=resolution_level,
|
||||
max_input_size=configured_max_ai_input_size(context),
|
||||
delete_input=delete_input,
|
||||
delete_analysis_input=delete_analysis_input,
|
||||
)
|
||||
if "CANCELLED" in result:
|
||||
if delete_input:
|
||||
remove_temporary_input(source_path)
|
||||
if delete_analysis_input:
|
||||
remove_temporary_input(analysis_path)
|
||||
cleanup_image_input(source_path, True)
|
||||
return {"CANCELLED"} if "CANCELLED" in result else {"FINISHED"}
|
||||
|
||||
|
||||
@@ -178,8 +159,8 @@ class ConvertToPlane(bpy.types.Operator):
|
||||
frame_start, frame_offset, frame_duration, use_cyclic = image_user_settings(
|
||||
source_object
|
||||
)
|
||||
material = create_material(
|
||||
material_data_name(source_object),
|
||||
material = create_image_material(
|
||||
source_object,
|
||||
color_image,
|
||||
1.0,
|
||||
frame_start=frame_start,
|
||||
|
||||
@@ -2,11 +2,8 @@
|
||||
|
||||
import bpy
|
||||
|
||||
from ..common.node_asset import (
|
||||
load_mesh_plane_node_group,
|
||||
modifier_input_identifier,
|
||||
set_modifier_input,
|
||||
)
|
||||
from ..common.mesh_result import modifier_input_identifier, set_modifier_input
|
||||
from .nodes import mesh_plane_node_group
|
||||
|
||||
|
||||
MESH_PLANE_NODE_GROUP_NAME = "O Mesh Plane"
|
||||
@@ -44,7 +41,7 @@ def add_mesh_plane_processing(
|
||||
subdivisions=0,
|
||||
thickness=0.0,
|
||||
):
|
||||
node_group = load_mesh_plane_node_group()
|
||||
node_group = mesh_plane_node_group()
|
||||
modifier = plane_object.modifiers.new(
|
||||
name=MESH_PLANE_NODE_GROUP_NAME,
|
||||
type="NODES",
|
||||
|
||||
@@ -2,8 +2,33 @@
|
||||
|
||||
import bpy
|
||||
|
||||
from ..common.node_asset import load_mesh_plane_node_group
|
||||
from ..common.node_graph import group_input, interface_socket
|
||||
from ..common.mesh_result import load_asset_node_group
|
||||
|
||||
MESH_PLANE_NODE_GROUP_NAME = "O Mesh Plane"
|
||||
|
||||
|
||||
def mesh_plane_node_group():
|
||||
return load_asset_node_group(MESH_PLANE_NODE_GROUP_NAME)
|
||||
|
||||
|
||||
def _interface_socket(group, name, in_out, socket_type):
|
||||
socket = group.interface.new_socket(
|
||||
name=name,
|
||||
in_out=in_out,
|
||||
socket_type=socket_type,
|
||||
)
|
||||
if in_out == "INPUT" and hasattr(socket, "structure_type"):
|
||||
socket.structure_type = "SINGLE"
|
||||
return socket
|
||||
|
||||
|
||||
def _group_input(nodes, location, visible_names):
|
||||
node = nodes.new("NodeGroupInput")
|
||||
node.location = location
|
||||
visible = set(visible_names)
|
||||
for output in node.outputs:
|
||||
output.hide = output.name not in visible
|
||||
return node
|
||||
|
||||
def depth_plane_node_group():
|
||||
group_name = "O Image Depth Plane"
|
||||
@@ -16,42 +41,42 @@ def depth_plane_node_group():
|
||||
group.nodes.clear()
|
||||
group.interface.clear()
|
||||
group.is_modifier = True
|
||||
interface_socket(group, "Geometry", "INPUT", "NodeSocketGeometry")
|
||||
subdivide = interface_socket(group, "Subdivide", "INPUT", "NodeSocketInt")
|
||||
_interface_socket(group, "Geometry", "INPUT", "NodeSocketGeometry")
|
||||
subdivide = _interface_socket(group, "Subdivide", "INPUT", "NodeSocketInt")
|
||||
subdivide.default_value = 7
|
||||
subdivide.min_value = 0
|
||||
subdivide.max_value = 10
|
||||
thickness = interface_socket(group, "Thickness", "INPUT", "NodeSocketFloat")
|
||||
thickness = _interface_socket(group, "Thickness", "INPUT", "NodeSocketFloat")
|
||||
thickness.default_value = 0.1
|
||||
thickness.min_value = 0.0
|
||||
thickness.subtype = "DISTANCE"
|
||||
depth_amount = interface_socket(group, "Depth Amount", "INPUT", "NodeSocketFloat")
|
||||
depth_amount = _interface_socket(group, "Depth Amount", "INPUT", "NodeSocketFloat")
|
||||
depth_amount.default_value = 1.0
|
||||
depth_amount.min_value = 0.0
|
||||
depth_image = interface_socket(group, "Depth Image", "INPUT", "NodeSocketImage")
|
||||
depth_image = _interface_socket(group, "Depth Image", "INPUT", "NodeSocketImage")
|
||||
depth_image.hide_in_modifier = True
|
||||
material = interface_socket(group, "Material", "INPUT", "NodeSocketMaterial")
|
||||
material = _interface_socket(group, "Material", "INPUT", "NodeSocketMaterial")
|
||||
material.hide_in_modifier = True
|
||||
interface_socket(group, "Geometry", "OUTPUT", "NodeSocketGeometry")
|
||||
_interface_socket(group, "Geometry", "OUTPUT", "NodeSocketGeometry")
|
||||
|
||||
nodes = group.nodes
|
||||
links = group.links
|
||||
geometry_input = group_input(
|
||||
geometry_input = _group_input(
|
||||
nodes,
|
||||
(-1850.0, 450.0),
|
||||
{"Geometry", "Subdivide", "Material"},
|
||||
)
|
||||
shape_input = group_input(
|
||||
shape_input = _group_input(
|
||||
nodes,
|
||||
(-1250.0, 850.0),
|
||||
{"Thickness", "Depth Amount"},
|
||||
)
|
||||
image_input = group_input(nodes, (-1050.0, 50.0), {"Depth Image"})
|
||||
cleanup_input = group_input(nodes, (650.0, -200.0), {"Thickness"})
|
||||
image_input = _group_input(nodes, (-1050.0, 50.0), {"Depth Image"})
|
||||
cleanup_input = _group_input(nodes, (650.0, -200.0), {"Thickness"})
|
||||
group_output = nodes.new("NodeGroupOutput")
|
||||
group_output.location = (1600.0, 450.0)
|
||||
mesh_plane = nodes.new("GeometryNodeGroup")
|
||||
mesh_plane.node_tree = load_mesh_plane_node_group()
|
||||
mesh_plane.node_tree = mesh_plane_node_group()
|
||||
mesh_plane.inputs["Thickness"].default_value = 0.0
|
||||
mesh_plane.location = (-1550.0, 450.0)
|
||||
links.new(geometry_input.outputs["Geometry"], mesh_plane.inputs["Geometry"])
|
||||
@@ -150,7 +175,7 @@ def depth_plane_node_group():
|
||||
|
||||
def plane_node_group(shape):
|
||||
if shape == "PLANE":
|
||||
return load_mesh_plane_node_group()
|
||||
return mesh_plane_node_group()
|
||||
if shape == "DEPTH_PLANE":
|
||||
return depth_plane_node_group()
|
||||
raise ValueError(f"Unsupported Plane shape: {shape}")
|
||||
|
||||
@@ -1,3 +1,6 @@
|
||||
from ..common.image_data import image_pixels
|
||||
|
||||
|
||||
def _normalized(values, fallback):
|
||||
import numpy as np
|
||||
|
||||
@@ -91,9 +94,7 @@ def bake_tangent_normal_image(context, source_object, target_object, image):
|
||||
import numpy as np
|
||||
|
||||
width, height = (int(value) for value in image.size)
|
||||
rgba = np.empty(width * height * 4, dtype=np.float32)
|
||||
image.pixels.foreach_get(rgba)
|
||||
rgba = rgba.reshape((height, width, 4))
|
||||
rgba = image_pixels(image).reshape((height, width, 4))
|
||||
source_normal = rgba[..., :3].astype(np.float64) * 2.0 - 1.0
|
||||
|
||||
source_to_target = (
|
||||
16
src/anyimage/operators/convert_to_plane/resolution.py
Normal file
16
src/anyimage/operators/convert_to_plane/resolution.py
Normal file
@@ -0,0 +1,16 @@
|
||||
DA3_PATCH_SIZE = 14
|
||||
DA3_MIN_PROCESS_RES = 252
|
||||
DA3_MAX_PROCESS_RES = 1008
|
||||
|
||||
|
||||
def scaled_process_resolution(image, scale_percent):
|
||||
"""Return the scaled long edge clamped and aligned for DA3 inference."""
|
||||
width, height = image.size
|
||||
if width <= 0 or height <= 0:
|
||||
raise ValueError("The source image has no readable dimensions")
|
||||
if scale_percent < 1.0 or scale_percent > 100.0:
|
||||
raise ValueError("Resolution Scale must be between 1% and 100%")
|
||||
process_res = max(width, height) * scale_percent / 100.0
|
||||
process_res = max(DA3_MIN_PROCESS_RES, process_res)
|
||||
process_res = min(DA3_MAX_PROCESS_RES, process_res)
|
||||
return int(process_res / DA3_PATCH_SIZE + 0.5) * DA3_PATCH_SIZE
|
||||
@@ -1,12 +1,8 @@
|
||||
"""Cutout selection, shape conversion, and geometry algorithms."""
|
||||
|
||||
from .selection_tool import ActivateCutoutTool, CutoutTool, SelectCutoutRegion
|
||||
from .shape_conversion import ConvertCutoutShape, GenerateCutoutShape
|
||||
from .region_to_shape import CutoutRegionToShape
|
||||
from .select_region import CutoutTool, SelectCutoutRegion
|
||||
|
||||
__all__ = (
|
||||
"ActivateCutoutTool",
|
||||
"ConvertCutoutShape",
|
||||
"CutoutRegionToShape",
|
||||
"CutoutTool",
|
||||
"GenerateCutoutShape",
|
||||
"SelectCutoutRegion",
|
||||
)
|
||||
|
||||
@@ -1,814 +0,0 @@
|
||||
import json
|
||||
|
||||
import bpy
|
||||
|
||||
from ...preferences import addon_preferences, configured_max_ai_input_size
|
||||
from ...server.media.resolution import limited_image_size
|
||||
from ..common.image import (
|
||||
color_data_name,
|
||||
create_color_image,
|
||||
depth_data_name,
|
||||
is_image_empty,
|
||||
material_data_name,
|
||||
normal_data_name,
|
||||
plane_bounds,
|
||||
video_total_frames,
|
||||
)
|
||||
from ..common.material import create_material
|
||||
from ..common.node_asset import modifier_input_identifier, set_modifier_input
|
||||
from ..common.normal_map import bake_tangent_normal_image
|
||||
from ..common.operator import (
|
||||
push_undo,
|
||||
)
|
||||
from .depth_calibration import (
|
||||
DEFAULT_DEPTH_DIRECTION,
|
||||
UNREFINED_DEPTH_REFERENCE,
|
||||
depth_balloon_projection_parameters,
|
||||
metric_depth_reference,
|
||||
)
|
||||
from .planar_mesh import (
|
||||
metric_transform,
|
||||
prepare_planar_regions,
|
||||
triangulate_planar_regions,
|
||||
)
|
||||
from .balloon_profile import balloon_profile as calculate_balloon_profile
|
||||
from .mask_regions import alpha_regions
|
||||
from .node_assets import cutout_node_group
|
||||
from ..common.image_edit import (
|
||||
ImageEditWarning,
|
||||
polygon_bounds,
|
||||
)
|
||||
from ..common.image_mask import polygon_mask, visible_selection_mask
|
||||
from .shape_types import (
|
||||
BALLOON_ATTRIBUTE_NAME,
|
||||
CUTOUT_SHAPE_INDICES,
|
||||
)
|
||||
|
||||
CUTOUT_ALPHA_THRESHOLD = 0.1
|
||||
|
||||
CUTOUT_AXIS_TO_SOURCE = (
|
||||
(0.0, -1.0, 0.0, 0.0),
|
||||
(0.0, 0.0, 1.0, 0.0),
|
||||
(-1.0, 0.0, 0.0, 0.0),
|
||||
(0.0, 0.0, 0.0, 1.0),
|
||||
)
|
||||
|
||||
|
||||
def _surface_depth_image(source_object, points, valid, camera_depth):
|
||||
import numpy as np
|
||||
|
||||
points = np.asarray(points, dtype=np.float32)
|
||||
valid = np.asarray(valid, dtype=bool)
|
||||
height, width = valid.shape
|
||||
if points.shape != (height, width, 3):
|
||||
raise ValueError("Depth Surface point dimensions do not match")
|
||||
discontinuity = _depth_discontinuity_map(camera_depth, valid)
|
||||
discontinuity[~valid] = -1.0
|
||||
pixels = np.empty((height, width, 4), dtype=np.float32)
|
||||
pixels[:, :, :3] = points
|
||||
pixels[:, :, 3] = discontinuity
|
||||
image = bpy.data.images.new(
|
||||
depth_data_name(source_object),
|
||||
width=width,
|
||||
height=height,
|
||||
alpha=True,
|
||||
float_buffer=True,
|
||||
)
|
||||
image.colorspace_settings.name = "Non-Color"
|
||||
image.alpha_mode = "CHANNEL_PACKED"
|
||||
image.pixels.foreach_set(np.flipud(pixels).ravel())
|
||||
image.pack()
|
||||
return image
|
||||
|
||||
|
||||
def _cutout_world_matrix(source_matrix, crop_center):
|
||||
from mathutils import Matrix
|
||||
|
||||
center_x, center_y = (float(value) for value in crop_center)
|
||||
return (
|
||||
source_matrix
|
||||
@ Matrix.Translation((center_x, center_y, 0.0))
|
||||
@ Matrix(CUTOUT_AXIS_TO_SOURCE)
|
||||
)
|
||||
|
||||
|
||||
def _orient_cutout_vertices(vertices):
|
||||
import numpy as np
|
||||
|
||||
vertices = np.asarray(vertices, dtype=np.float64)
|
||||
return np.column_stack(
|
||||
(
|
||||
-vertices[:, 2],
|
||||
-vertices[:, 0],
|
||||
vertices[:, 1],
|
||||
)
|
||||
)
|
||||
|
||||
|
||||
def _cutout_domain(
|
||||
regions_pixels,
|
||||
image_size,
|
||||
bounds,
|
||||
spacing,
|
||||
world_metric,
|
||||
balloon_algorithm="POISSON",
|
||||
):
|
||||
import numpy as np
|
||||
|
||||
x_min, x_max, y_min, y_max = bounds
|
||||
|
||||
regions, spacing = prepare_planar_regions(
|
||||
regions_pixels,
|
||||
image_size,
|
||||
bounds,
|
||||
spacing=spacing,
|
||||
world_metric=world_metric,
|
||||
)
|
||||
planar, planar_faces, boundary, components = triangulate_planar_regions(
|
||||
regions,
|
||||
spacing,
|
||||
return_components=True,
|
||||
)
|
||||
profile = calculate_balloon_profile(
|
||||
balloon_algorithm, planar, planar_faces, boundary, components
|
||||
)
|
||||
faces = np.asarray(planar_faces, dtype=np.int32)
|
||||
local = planar @ np.linalg.inv(metric_transform(world_metric))
|
||||
uv = np.column_stack(
|
||||
(
|
||||
(local[:, 0] - x_min) / (x_max - x_min),
|
||||
(local[:, 1] - y_min) / (y_max - y_min),
|
||||
)
|
||||
).astype(np.float32)
|
||||
crop_center = np.asarray(
|
||||
((x_min + x_max) * 0.5, (y_min + y_max) * 0.5),
|
||||
dtype=np.float64,
|
||||
)
|
||||
local -= crop_center
|
||||
vertices = np.column_stack((local, np.zeros(len(local), dtype=np.float64)))
|
||||
return vertices, faces, uv, boundary, planar, profile
|
||||
|
||||
|
||||
def _world_plane_metric(matrix_world):
|
||||
from mathutils import Vector
|
||||
|
||||
linear = matrix_world.to_3x3()
|
||||
x = linear @ Vector((1.0, 0.0, 0.0))
|
||||
y = linear @ Vector((0.0, 1.0, 0.0))
|
||||
z = linear @ Vector((0.0, 0.0, 1.0))
|
||||
return ((x.dot(x), x.dot(y)), (x.dot(y), y.dot(y))), z.length
|
||||
|
||||
|
||||
def _cutout_cleanup_area(spacing, world_metric):
|
||||
import numpy as np
|
||||
|
||||
if spacing <= 0.0:
|
||||
raise ValueError("Cutout point spacing must be greater than zero")
|
||||
area_scale = abs(float(np.linalg.det(metric_transform(world_metric))))
|
||||
if area_scale <= 1e-12:
|
||||
raise ValueError("The Image Empty has a degenerate world transform")
|
||||
return float(spacing) ** 2 / area_scale
|
||||
|
||||
|
||||
def _load_field(path):
|
||||
import numpy as np
|
||||
|
||||
with np.load(path, allow_pickle=False) as archive:
|
||||
points = np.asarray(archive["points"], dtype=np.float32)
|
||||
intrinsics = np.asarray(archive["intrinsics"], dtype=np.float32)
|
||||
valid = np.asarray(archive["valid"], dtype=bool)
|
||||
calibration_valid = np.asarray(archive["calibration_valid"], dtype=bool)
|
||||
image_size = tuple(int(value) for value in archive["image_size"])
|
||||
reference_depth = float(archive["reference_depth"])
|
||||
if (
|
||||
points.ndim != 3
|
||||
or points.shape[2] != 3
|
||||
or valid.shape != points.shape[:2]
|
||||
or calibration_valid.shape != points.shape[:2]
|
||||
):
|
||||
raise ValueError("The mesh point field is invalid")
|
||||
if intrinsics.shape != (3, 3) or not reference_depth > 0.0:
|
||||
raise ValueError("The mesh camera field is invalid")
|
||||
return points, intrinsics, valid, calibration_valid, image_size, reference_depth
|
||||
|
||||
|
||||
def _surface_uniform_scale(
|
||||
source_object,
|
||||
field,
|
||||
bounds=None,
|
||||
):
|
||||
import numpy as np
|
||||
|
||||
(
|
||||
_points,
|
||||
intrinsics,
|
||||
_valid,
|
||||
_calibration_valid,
|
||||
image_size,
|
||||
reference_depth,
|
||||
) = field
|
||||
width, height = image_size
|
||||
x_min, x_max, y_min, y_max = bounds or plane_bounds(source_object)
|
||||
plane_width = x_max - x_min
|
||||
plane_height = y_max - y_min
|
||||
fx, fy = float(intrinsics[0, 0]), float(intrinsics[1, 1])
|
||||
if fx <= 0.0 or fy <= 0.0:
|
||||
raise ValueError("Camera intrinsics contain an invalid focal length")
|
||||
scale_x = plane_width * fx / (max(width - 1, 1) * reference_depth)
|
||||
scale_y = plane_height * fy / (max(height - 1, 1) * reference_depth)
|
||||
return float(np.sqrt(scale_x * scale_y))
|
||||
|
||||
|
||||
def _depth_discontinuity_map(depth, valid):
|
||||
import numpy as np
|
||||
|
||||
depth = np.asarray(depth, dtype=np.float32)
|
||||
valid = np.asarray(valid, dtype=bool) & np.isfinite(depth) & (depth > 0.0)
|
||||
padded_depth = np.pad(depth, 1, mode="edge")
|
||||
padded_valid = np.pad(valid, 1, mode="edge")
|
||||
windows = np.lib.stride_tricks.sliding_window_view(padded_depth, (3, 3))
|
||||
usable = np.lib.stride_tricks.sliding_window_view(padded_valid, (3, 3))
|
||||
minimum = np.min(np.where(usable, windows, np.inf), axis=(-2, -1))
|
||||
maximum = np.max(np.where(usable, windows, -np.inf), axis=(-2, -1))
|
||||
result = (maximum - minimum) / np.maximum(np.abs(depth), 1e-6)
|
||||
result[~usable.all(axis=(-2, -1))] = 1.0
|
||||
return result.astype(np.float32)
|
||||
|
||||
|
||||
def _counter_clockwise_faces(points, faces):
|
||||
import numpy as np
|
||||
|
||||
points = np.asarray(points, dtype=np.float64)
|
||||
result = np.asarray(faces, dtype=np.int32).copy()
|
||||
polygons = points[result]
|
||||
following = np.roll(polygons, -1, axis=1)
|
||||
signed_area = np.sum(
|
||||
polygons[..., 0] * following[..., 1]
|
||||
- polygons[..., 1] * following[..., 0],
|
||||
axis=1,
|
||||
)
|
||||
result[signed_area < 0.0] = result[signed_area < 0.0, ::-1]
|
||||
return result
|
||||
|
||||
|
||||
def _assign_uv(mesh, uv):
|
||||
import numpy as np
|
||||
|
||||
for layer in list(mesh.uv_layers):
|
||||
mesh.uv_layers.remove(layer)
|
||||
uv_layer = mesh.uv_layers.new(name="UVMap")
|
||||
loop_vertices = np.empty(len(mesh.loops), dtype=np.int32)
|
||||
mesh.loops.foreach_get("vertex_index", loop_vertices)
|
||||
loop_uv = np.asarray(uv, dtype=np.float32)[loop_vertices]
|
||||
uv_layer.data.foreach_set("uv", loop_uv.ravel())
|
||||
|
||||
|
||||
def _assign_float_point_attribute(mesh, name, values):
|
||||
import numpy as np
|
||||
|
||||
attribute = mesh.attributes.new(name=name, type="FLOAT", domain="POINT")
|
||||
attribute.data.foreach_set("value", np.asarray(values, dtype=np.float32))
|
||||
|
||||
|
||||
def _image_rgba(image):
|
||||
import numpy as np
|
||||
|
||||
width, height = (int(value) for value in image.size)
|
||||
pixels = np.empty(len(image.pixels), dtype=np.float32)
|
||||
image.pixels.foreach_get(pixels)
|
||||
return np.flipud(pixels.reshape((height, width, 4)))
|
||||
|
||||
|
||||
def _mask_crop_bounds(alpha):
|
||||
import numpy as np
|
||||
|
||||
values = np.asarray(alpha)
|
||||
visible_y, visible_x = np.nonzero(values > CUTOUT_ALPHA_THRESHOLD)
|
||||
if not len(visible_x):
|
||||
raise ValueError("The Cutout selection contains no visible pixels")
|
||||
return (
|
||||
int(visible_x.min()),
|
||||
int(visible_y.min()),
|
||||
int(visible_x.max()) + 1,
|
||||
int(visible_y.max()) + 1,
|
||||
)
|
||||
|
||||
|
||||
def _crop_plane_bounds(source_object, crop_bounds, source_size):
|
||||
source_width, source_height = (float(value) for value in source_size)
|
||||
left, top, right, bottom = (float(value) for value in crop_bounds)
|
||||
x_min, x_max, y_min, y_max = plane_bounds(source_object)
|
||||
width = x_max - x_min
|
||||
height = y_max - y_min
|
||||
return (
|
||||
x_min + left * width / source_width,
|
||||
x_min + right * width / source_width,
|
||||
y_max - bottom * height / source_height,
|
||||
y_max - top * height / source_height,
|
||||
)
|
||||
|
||||
|
||||
def _cropped_color_image(source_object, crop_bounds):
|
||||
import numpy as np
|
||||
|
||||
source = source_object.data
|
||||
rgba = _image_rgba(source)
|
||||
left, top, right, bottom = crop_bounds
|
||||
width = right - left
|
||||
height = bottom - top
|
||||
if width <= 0 or height <= 0:
|
||||
raise ValueError("The Cutout crop dimensions must be positive")
|
||||
cropped = np.zeros((height, width, 4), dtype=np.float32)
|
||||
source_height, source_width = rgba.shape[:2]
|
||||
source_left = max(0, left)
|
||||
source_top = max(0, top)
|
||||
source_right = min(source_width, right)
|
||||
source_bottom = min(source_height, bottom)
|
||||
if source_right > source_left and source_bottom > source_top:
|
||||
cropped[
|
||||
source_top - top : source_bottom - top,
|
||||
source_left - left : source_right - left,
|
||||
] = rgba[source_top:source_bottom, source_left:source_right]
|
||||
image = bpy.data.images.new(
|
||||
color_data_name(source_object),
|
||||
width=width,
|
||||
height=height,
|
||||
alpha=True,
|
||||
)
|
||||
try:
|
||||
image.colorspace_settings.name = source.colorspace_settings.name
|
||||
image.alpha_mode = source.alpha_mode
|
||||
image.pixels.foreach_set(np.flipud(cropped).ravel())
|
||||
image.pack()
|
||||
image.update()
|
||||
except Exception:
|
||||
bpy.data.images.remove(image, do_unlink=True)
|
||||
raise
|
||||
return image
|
||||
|
||||
|
||||
def _cutout_regions(
|
||||
alpha, mesh_detail, threshold=CUTOUT_ALPHA_THRESHOLD
|
||||
):
|
||||
contour_resolution = max(512, min(int(mesh_detail) * 8, 1024))
|
||||
return alpha_regions(
|
||||
alpha,
|
||||
max_resolution=contour_resolution,
|
||||
threshold=threshold,
|
||||
)
|
||||
|
||||
|
||||
def _antialias_selection(selection):
|
||||
import numpy as np
|
||||
|
||||
values = np.asarray(selection, dtype=np.float32)
|
||||
kernel = np.asarray((0.029, 0.235, 0.472, 0.235, 0.029), dtype=np.float32)
|
||||
radius = len(kernel) // 2
|
||||
horizontal = np.zeros_like(values)
|
||||
padded = np.pad(values, ((0, 0), (radius, radius)), mode="edge")
|
||||
for offset, weight in enumerate(kernel):
|
||||
horizontal += padded[:, offset : offset + values.shape[1]] * weight
|
||||
result = np.zeros_like(values)
|
||||
padded = np.pad(horizontal, ((radius, radius), (0, 0)), mode="edge")
|
||||
for offset, weight in enumerate(kernel):
|
||||
result += padded[offset : offset + values.shape[0]] * weight
|
||||
return result
|
||||
|
||||
|
||||
def _geometry_cutout_regions(
|
||||
alpha,
|
||||
polygon,
|
||||
mesh_detail,
|
||||
invert=False,
|
||||
refine_selection=False,
|
||||
selection=None,
|
||||
):
|
||||
import numpy as np
|
||||
|
||||
values = np.asarray(alpha, dtype=np.float32)
|
||||
if values.ndim != 2:
|
||||
raise ValueError("Cutout Alpha must be two-dimensional")
|
||||
source_height, source_width = values.shape
|
||||
source_size = (source_width, source_height)
|
||||
if refine_selection:
|
||||
geometry_alpha = np.where(values > CUTOUT_ALPHA_THRESHOLD, values, 0.0)
|
||||
regions, _image_size = _cutout_regions(geometry_alpha, mesh_detail)
|
||||
return regions, source_size
|
||||
geometry_alpha = selection
|
||||
if geometry_alpha is None:
|
||||
geometry_alpha = _antialias_selection(
|
||||
polygon_mask(source_size, polygon, "NONZERO")
|
||||
)
|
||||
geometry_alpha = visible_selection_mask(
|
||||
values,
|
||||
geometry_alpha,
|
||||
invert,
|
||||
alpha_threshold=CUTOUT_ALPHA_THRESHOLD,
|
||||
)
|
||||
regions, _image_size = _cutout_regions(geometry_alpha, mesh_detail)
|
||||
return regions, source_size
|
||||
|
||||
def _local_cutout_rgba(source_object, polygon, invert=False):
|
||||
import numpy as np
|
||||
|
||||
rgba = _image_rgba(source_object.data).copy()
|
||||
height, width = rgba.shape[:2]
|
||||
selection = _antialias_selection(
|
||||
polygon_mask((width, height), polygon, "NONZERO")
|
||||
)
|
||||
selection = visible_selection_mask(
|
||||
rgba[:, :, 3],
|
||||
selection,
|
||||
invert,
|
||||
alpha_threshold=CUTOUT_ALPHA_THRESHOLD,
|
||||
)
|
||||
rgba[:, :, 3] *= selection
|
||||
rgba[rgba[:, :, 3] <= 0.0, :3] = 0.0
|
||||
if not np.any(rgba[:, :, 3] > 0.0):
|
||||
raise ImageEditWarning("The selection contains no visible image pixels")
|
||||
return rgba, selection
|
||||
|
||||
|
||||
def _new_material(
|
||||
context,
|
||||
source_object,
|
||||
color_image,
|
||||
normal_image=None,
|
||||
normal_space="TANGENT",
|
||||
):
|
||||
return create_material(
|
||||
material_data_name(source_object),
|
||||
color_image,
|
||||
0.0,
|
||||
normal_image=normal_image,
|
||||
normal_space=normal_space,
|
||||
scene=context.scene,
|
||||
)
|
||||
|
||||
|
||||
def build_cutout_object(
|
||||
context,
|
||||
source_object,
|
||||
shape,
|
||||
detail,
|
||||
material,
|
||||
field=None,
|
||||
regions=None,
|
||||
image_size=None,
|
||||
bounds=None,
|
||||
keep_original=False,
|
||||
refine_selection=False,
|
||||
):
|
||||
import numpy as np
|
||||
|
||||
bounds = bounds or plane_bounds(source_object)
|
||||
crop_center = (
|
||||
(bounds[0] + bounds[1]) * 0.5,
|
||||
(bounds[2] + bounds[3]) * 0.5,
|
||||
)
|
||||
world_metric, world_z_scale = _world_plane_metric(source_object.matrix_world)
|
||||
depth_image = None
|
||||
depth_reference = None
|
||||
depth_uniform_scale = None
|
||||
depth_direction = None
|
||||
preferences = addon_preferences(context)
|
||||
balloon_algorithm = getattr(
|
||||
preferences, "balloon_profile_algorithm", "POISSON"
|
||||
)
|
||||
point_spacing = _cutout_point_spacing(source_object, detail)
|
||||
cleanup_area = _cutout_cleanup_area(point_spacing, world_metric)
|
||||
domain = _cutout_domain(
|
||||
regions,
|
||||
image_size,
|
||||
bounds,
|
||||
point_spacing,
|
||||
world_metric,
|
||||
balloon_algorithm,
|
||||
)
|
||||
source_vertices, faces, uv, boundary, planar, balloon_profile = domain
|
||||
faces = _counter_clockwise_faces(planar, faces)
|
||||
vertices = _orient_cutout_vertices(source_vertices)
|
||||
if world_z_scale <= 1e-10:
|
||||
raise ValueError("The Image Empty has a degenerate world transform")
|
||||
balloon_profile /= world_z_scale
|
||||
if shape in {"DEPTH_BALLOON", "DEPTH_SURFACE"}:
|
||||
(
|
||||
points,
|
||||
_intrinsics,
|
||||
valid,
|
||||
calibration_valid,
|
||||
_image_size,
|
||||
_reference,
|
||||
) = field
|
||||
depth_image = _surface_depth_image(
|
||||
source_object, points, valid, points[..., 2]
|
||||
)
|
||||
depth_uniform_scale = _surface_uniform_scale(
|
||||
source_object,
|
||||
field,
|
||||
bounds,
|
||||
)
|
||||
if not refine_selection:
|
||||
depth_reference = UNREFINED_DEPTH_REFERENCE
|
||||
if shape == "DEPTH_BALLOON":
|
||||
depth_direction = DEFAULT_DEPTH_DIRECTION
|
||||
elif shape == "DEPTH_SURFACE":
|
||||
depth_reference = metric_depth_reference(
|
||||
points[..., 2],
|
||||
calibration_valid,
|
||||
depth_uniform_scale,
|
||||
)
|
||||
else:
|
||||
depth_reference, depth_direction = depth_balloon_projection_parameters(
|
||||
vertices,
|
||||
uv,
|
||||
points[..., 2],
|
||||
calibration_valid,
|
||||
depth_uniform_scale,
|
||||
)
|
||||
|
||||
source_name = source_object.name
|
||||
mesh = bpy.data.meshes.new(source_name)
|
||||
mesh.from_pydata(np.asarray(vertices).tolist(), [], faces)
|
||||
for polygon in mesh.polygons:
|
||||
polygon.use_smooth = True
|
||||
_assign_uv(mesh, uv)
|
||||
_assign_float_point_attribute(mesh, BALLOON_ATTRIBUTE_NAME, balloon_profile)
|
||||
mesh.materials.append(material)
|
||||
mesh.update()
|
||||
|
||||
mesh_object = bpy.data.objects.new(source_name, mesh)
|
||||
context.collection.objects.link(mesh_object)
|
||||
mesh_object.matrix_world = _cutout_world_matrix(
|
||||
source_object.matrix_world, crop_center
|
||||
)
|
||||
mesh_object["anyimage_mesh_shape"] = shape
|
||||
node_group = cutout_node_group(shape)
|
||||
modifier = mesh_object.modifiers.new(name=node_group.name, type="NODES")
|
||||
modifier.node_group = node_group
|
||||
modifier.show_group_selector = False
|
||||
set_modifier_input(
|
||||
modifier,
|
||||
modifier_input_identifier(node_group, "Shape"),
|
||||
CUTOUT_SHAPE_INDICES[shape],
|
||||
)
|
||||
set_modifier_input(
|
||||
modifier,
|
||||
modifier_input_identifier(node_group, "Cleanup"),
|
||||
cleanup_area,
|
||||
)
|
||||
set_modifier_input(
|
||||
modifier,
|
||||
modifier_input_identifier(node_group, "Smooth Iterations"),
|
||||
0,
|
||||
)
|
||||
set_modifier_input(
|
||||
modifier,
|
||||
modifier_input_identifier(node_group, "Smooth"),
|
||||
1.0,
|
||||
)
|
||||
thickness = 1.0 if shape in {"BALLOON", "DEPTH_BALLOON"} else 0.0
|
||||
set_modifier_input(
|
||||
modifier,
|
||||
modifier_input_identifier(node_group, "Thickness"),
|
||||
thickness,
|
||||
)
|
||||
if shape in {"DEPTH_BALLOON", "DEPTH_SURFACE"}:
|
||||
set_modifier_input(
|
||||
modifier,
|
||||
modifier_input_identifier(
|
||||
node_group,
|
||||
"Depth Scale",
|
||||
),
|
||||
1.0,
|
||||
)
|
||||
if shape == "DEPTH_SURFACE":
|
||||
set_modifier_input(
|
||||
modifier,
|
||||
modifier_input_identifier(node_group, "Uniform Scale"),
|
||||
depth_uniform_scale,
|
||||
)
|
||||
set_modifier_input(
|
||||
modifier,
|
||||
modifier_input_identifier(node_group, "Reference Depth"),
|
||||
depth_reference,
|
||||
)
|
||||
if shape == "DEPTH_BALLOON":
|
||||
set_modifier_input(
|
||||
modifier,
|
||||
modifier_input_identifier(node_group, "Reference Depth"),
|
||||
depth_reference,
|
||||
)
|
||||
set_modifier_input(
|
||||
modifier,
|
||||
modifier_input_identifier(node_group, "Depth Direction"),
|
||||
depth_direction,
|
||||
)
|
||||
set_modifier_input(
|
||||
modifier,
|
||||
modifier_input_identifier(node_group, "Uniform Scale"),
|
||||
depth_uniform_scale,
|
||||
)
|
||||
if depth_image is not None:
|
||||
set_modifier_input(
|
||||
modifier,
|
||||
modifier_input_identifier(node_group, "Depth Image"),
|
||||
depth_image,
|
||||
)
|
||||
for selected in context.selected_objects:
|
||||
selected.select_set(False)
|
||||
mesh_object.select_set(True)
|
||||
context.view_layer.objects.active = mesh_object
|
||||
if not keep_original:
|
||||
bpy.data.objects.remove(source_object, do_unlink=True)
|
||||
mesh_object.name = source_name
|
||||
return mesh_object
|
||||
|
||||
|
||||
def _load_normal_image(path, source_object):
|
||||
image = bpy.data.images.load(str(path), check_existing=False)
|
||||
image.name = normal_data_name(source_object)
|
||||
image.colorspace_settings.name = "Non-Color"
|
||||
image.pack()
|
||||
return image
|
||||
|
||||
|
||||
def _orient_cutout_object_normal(image):
|
||||
import numpy as np
|
||||
|
||||
width, height = (int(value) for value in image.size)
|
||||
pixels = np.empty(width * height * 4, dtype=np.float32)
|
||||
image.pixels.foreach_get(pixels)
|
||||
rgba = pixels.reshape((height, width, 4))
|
||||
source = rgba[:, :, :3] * 2.0 - 1.0
|
||||
oriented = np.empty_like(source)
|
||||
oriented[:, :, 0] = -source[:, :, 2]
|
||||
oriented[:, :, 1] = -source[:, :, 0]
|
||||
oriented[:, :, 2] = source[:, :, 1]
|
||||
rgba[:, :, :3] = np.clip(oriented * 0.5 + 0.5, 0.0, 1.0)
|
||||
image.pixels.foreach_set(rgba.ravel())
|
||||
image.update()
|
||||
|
||||
|
||||
def limited_cutout_mesh_detail(mesh_detail, crop_size, max_input_size):
|
||||
crop_width, crop_height = (int(value) for value in crop_size)
|
||||
limited_width, limited_height = limited_image_size(
|
||||
(crop_width, crop_height),
|
||||
max_input_size,
|
||||
)
|
||||
scale = max(limited_width, limited_height) / max(crop_width, crop_height)
|
||||
return max(4, min(1024, round(int(mesh_detail) * scale)))
|
||||
|
||||
def _cutout_point_spacing(source_object, detail):
|
||||
import math
|
||||
|
||||
x_min, x_max, y_min, y_max = plane_bounds(source_object)
|
||||
metric, _world_z_scale = _world_plane_metric(source_object.matrix_world)
|
||||
width = (x_max - x_min) * math.sqrt(max(metric[0][0], 0.0))
|
||||
height = (y_max - y_min) * math.sqrt(max(metric[1][1], 0.0))
|
||||
return max(width, height) / max(int(detail), 8)
|
||||
|
||||
|
||||
def build_local_cutout(
|
||||
operator,
|
||||
context,
|
||||
source_object,
|
||||
shape,
|
||||
selection_data,
|
||||
invert=False,
|
||||
):
|
||||
if video_total_frames(source_object.data) != 1:
|
||||
raise ValueError("Mesh conversion currently supports still images")
|
||||
polygon = json.loads(selection_data) if selection_data else None
|
||||
if not isinstance(polygon, list) or len(polygon) < 3:
|
||||
raise ValueError("The Cutout selection is invalid")
|
||||
_rgba, selection = _local_cutout_rgba(source_object, polygon, invert)
|
||||
source_size = tuple(source_object.data.size)
|
||||
crop_bounds = _mask_crop_bounds(selection)
|
||||
left, top, right, bottom = crop_bounds
|
||||
cropped_selection = selection[top:bottom, left:right]
|
||||
mesh_detail = limited_cutout_mesh_detail(
|
||||
operator.mesh_detail,
|
||||
(right - left, bottom - top),
|
||||
configured_max_ai_input_size(context),
|
||||
)
|
||||
regions, image_size = _cutout_regions(
|
||||
cropped_selection,
|
||||
mesh_detail,
|
||||
threshold=CUTOUT_ALPHA_THRESHOLD,
|
||||
)
|
||||
local_bounds = _crop_plane_bounds(
|
||||
source_object,
|
||||
crop_bounds,
|
||||
source_size,
|
||||
)
|
||||
color_image = None
|
||||
material = None
|
||||
try:
|
||||
color_image = _cropped_color_image(source_object, crop_bounds)
|
||||
material = _new_material(context, source_object, color_image)
|
||||
build_cutout_object(
|
||||
context,
|
||||
source_object,
|
||||
shape,
|
||||
mesh_detail,
|
||||
material,
|
||||
regions=regions,
|
||||
image_size=image_size,
|
||||
bounds=local_bounds,
|
||||
keep_original=True,
|
||||
)
|
||||
except Exception:
|
||||
if material is not None and material.users == 0:
|
||||
bpy.data.materials.remove(material)
|
||||
if color_image is not None and color_image.users == 0:
|
||||
bpy.data.images.remove(color_image)
|
||||
raise
|
||||
push_undo(f"Convert to {shape.replace('_', ' ').title()}")
|
||||
|
||||
|
||||
def build_cutout_response(context, result, operator):
|
||||
source_object = bpy.data.objects.get(operator.source_object_name)
|
||||
if not is_image_empty(source_object):
|
||||
raise RuntimeError("The source Image Empty no longer exists")
|
||||
source_size = tuple(source_object.data.size)
|
||||
crop_bounds = tuple(
|
||||
int(value)
|
||||
for value in json.loads(
|
||||
result.file("bounds").read_text(encoding="utf-8")
|
||||
)
|
||||
)
|
||||
local_bounds = _crop_plane_bounds(
|
||||
source_object,
|
||||
crop_bounds,
|
||||
source_size,
|
||||
)
|
||||
selection_image = bpy.data.images.load(
|
||||
str(result.file("selection")),
|
||||
check_existing=False,
|
||||
)
|
||||
try:
|
||||
selection_alpha = _image_rgba(selection_image)[:, :, 3]
|
||||
mesh_detail = limited_cutout_mesh_detail(
|
||||
operator.mesh_detail,
|
||||
selection_image.size,
|
||||
operator.max_input_size,
|
||||
)
|
||||
regions, image_size = _cutout_regions(
|
||||
selection_alpha,
|
||||
mesh_detail,
|
||||
threshold=CUTOUT_ALPHA_THRESHOLD,
|
||||
)
|
||||
finally:
|
||||
bpy.data.images.remove(selection_image)
|
||||
color_image = create_color_image(
|
||||
source_object,
|
||||
result.directory,
|
||||
assign_to_source=False,
|
||||
)
|
||||
normal_path = result.directory / "normal.png"
|
||||
|
||||
normal_image = (
|
||||
_load_normal_image(normal_path, source_object)
|
||||
if normal_path.is_file()
|
||||
else None
|
||||
)
|
||||
if normal_image is not None and operator.shape == "DEPTH_SURFACE":
|
||||
_orient_cutout_object_normal(normal_image)
|
||||
material = _new_material(
|
||||
context,
|
||||
source_object,
|
||||
color_image,
|
||||
normal_image,
|
||||
normal_space=(
|
||||
"OBJECT"
|
||||
if operator.shape == "DEPTH_SURFACE" and normal_image is not None
|
||||
else "TANGENT"
|
||||
),
|
||||
)
|
||||
field = (
|
||||
_load_field(result.file("field"))
|
||||
if operator.shape in {"DEPTH_BALLOON", "DEPTH_SURFACE"}
|
||||
else None
|
||||
)
|
||||
mesh_object = build_cutout_object(
|
||||
context,
|
||||
source_object,
|
||||
operator.shape,
|
||||
mesh_detail,
|
||||
material,
|
||||
field=field,
|
||||
regions=regions,
|
||||
image_size=image_size,
|
||||
bounds=local_bounds,
|
||||
keep_original=True,
|
||||
refine_selection=operator.refine_selection,
|
||||
)
|
||||
if normal_image is not None and operator.shape != "DEPTH_SURFACE":
|
||||
bake_tangent_normal_image(
|
||||
context,
|
||||
source_object,
|
||||
mesh_object,
|
||||
normal_image,
|
||||
)
|
||||
push_undo(f"Convert to {operator.shape.replace('_', ' ').title()}")
|
||||
1
src/anyimage/operators/cutout_tool/geometry/__init__.py
Normal file
1
src/anyimage/operators/cutout_tool/geometry/__init__.py
Normal file
@@ -0,0 +1 @@
|
||||
"""Cutout 的内存几何算法。"""
|
||||
@@ -231,31 +231,49 @@ def cotangent_poisson_field(
|
||||
import numpy as np
|
||||
|
||||
points = np.asarray(planar_points, dtype=np.float64)
|
||||
faces = np.asarray(planar_faces, dtype=np.int64)
|
||||
if faces.size == 0:
|
||||
faces = np.empty((0, 3), dtype=np.int64)
|
||||
if faces.ndim != 2 or faces.shape[1] != 3:
|
||||
raise ValueError("Balloon profile faces must be triangles")
|
||||
boundary_mask = np.asarray(boundary_mask, dtype=bool)
|
||||
loads = np.zeros(len(points), dtype=np.float64)
|
||||
edge_weights = {}
|
||||
|
||||
def add_edge(start, end, weight):
|
||||
edge = tuple(sorted((int(start), int(end))))
|
||||
edge_weights[edge] = edge_weights.get(edge, 0.0) + weight
|
||||
|
||||
for face in planar_faces:
|
||||
first, second, third = (int(index) for index in face)
|
||||
a, b, c = points[[first, second, third]]
|
||||
cross = abs(
|
||||
(b[0] - a[0]) * (c[1] - a[1])
|
||||
- (b[1] - a[1]) * (c[0] - a[0])
|
||||
triangles = points[faces]
|
||||
a, b, c = (triangles[:, index] for index in range(3))
|
||||
cross = np.abs(
|
||||
(b[:, 0] - a[:, 0]) * (c[:, 1] - a[:, 1])
|
||||
- (b[:, 1] - a[:, 1]) * (c[:, 0] - a[:, 0])
|
||||
)
|
||||
usable = cross > 1e-14
|
||||
faces = faces[usable]
|
||||
a, b, c = a[usable], b[usable], c[usable]
|
||||
cross = cross[usable]
|
||||
loads = np.bincount(
|
||||
faces.ravel(),
|
||||
weights=np.repeat(cross / 6.0, 3),
|
||||
minlength=len(points),
|
||||
)
|
||||
edges = np.concatenate(
|
||||
(
|
||||
faces[:, (1, 2)],
|
||||
faces[:, (0, 2)],
|
||||
faces[:, (0, 1)],
|
||||
),
|
||||
axis=0,
|
||||
)
|
||||
edge_weights = np.concatenate(
|
||||
(
|
||||
np.sum((b - a) * (c - a), axis=1),
|
||||
np.sum((a - b) * (c - b), axis=1),
|
||||
np.sum((a - c) * (b - c), axis=1),
|
||||
)
|
||||
) / np.tile(cross * 2.0, 3)
|
||||
edges.sort(axis=1)
|
||||
edges, inverse = np.unique(edges, axis=0, return_inverse=True)
|
||||
edge_weights = np.bincount(
|
||||
inverse,
|
||||
weights=edge_weights,
|
||||
minlength=len(edges),
|
||||
)
|
||||
if cross <= 1e-14:
|
||||
continue
|
||||
area = cross * 0.5
|
||||
loads[[first, second, third]] += area / 3.0
|
||||
cotangent_a = float((b - a).dot(c - a) / cross)
|
||||
cotangent_b = float((a - b).dot(c - b) / cross)
|
||||
cotangent_c = float((a - c).dot(b - c) / cross)
|
||||
add_edge(second, third, cotangent_a * 0.5)
|
||||
add_edge(first, third, cotangent_b * 0.5)
|
||||
add_edge(first, second, cotangent_c * 0.5)
|
||||
|
||||
interior = np.flatnonzero(~boundary_mask)
|
||||
field = np.zeros(len(points), dtype=np.float64)
|
||||
@@ -263,38 +281,46 @@ def cotangent_poisson_field(
|
||||
return field
|
||||
compact = np.full(len(points), -1, dtype=np.int64)
|
||||
compact[interior] = np.arange(len(interior), dtype=np.int64)
|
||||
diagonal = np.zeros(len(interior), dtype=np.float64)
|
||||
edge_starts = []
|
||||
edge_ends = []
|
||||
weights = []
|
||||
# Constrained triangulation can create negative cotangent weights.
|
||||
# Clamping them keeps the Dirichlet system positive definite.
|
||||
for (start, end), weight in edge_weights.items():
|
||||
weight = max(float(weight), 0.0)
|
||||
if weight <= 1e-14:
|
||||
continue
|
||||
start_compact = compact[start]
|
||||
end_compact = compact[end]
|
||||
if start_compact >= 0:
|
||||
diagonal[start_compact] += weight
|
||||
if end_compact >= 0:
|
||||
diagonal[end_compact] += weight
|
||||
if start_compact >= 0 and end_compact >= 0:
|
||||
edge_starts.append(start_compact)
|
||||
edge_ends.append(end_compact)
|
||||
weights.append(weight)
|
||||
edge_weights = np.maximum(edge_weights, 0.0)
|
||||
usable = edge_weights > 1e-14
|
||||
compact_edges = compact[edges[usable]]
|
||||
edge_weights = edge_weights[usable]
|
||||
start_compact = compact_edges[:, 0]
|
||||
end_compact = compact_edges[:, 1]
|
||||
start_usable = start_compact >= 0
|
||||
end_usable = end_compact >= 0
|
||||
diagonal = np.zeros(len(interior), dtype=np.float64)
|
||||
diagonal += np.bincount(
|
||||
start_compact[start_usable],
|
||||
weights=edge_weights[start_usable],
|
||||
minlength=len(interior),
|
||||
)
|
||||
diagonal += np.bincount(
|
||||
end_compact[end_usable],
|
||||
weights=edge_weights[end_usable],
|
||||
minlength=len(interior),
|
||||
)
|
||||
if np.any(diagonal <= 1e-14):
|
||||
raise ValueError("Unable to build a stable Balloon profile")
|
||||
|
||||
edge_starts = np.asarray(edge_starts, dtype=np.int64)
|
||||
edge_ends = np.asarray(edge_ends, dtype=np.int64)
|
||||
weights = np.asarray(weights, dtype=np.float64)
|
||||
interior_edges = start_usable & end_usable
|
||||
edge_starts = start_compact[interior_edges]
|
||||
edge_ends = end_compact[interior_edges]
|
||||
weights = edge_weights[interior_edges]
|
||||
directed_starts = np.concatenate((edge_starts, edge_ends))
|
||||
directed_ends = np.concatenate((edge_ends, edge_starts))
|
||||
directed_weights = np.concatenate((weights, weights))
|
||||
|
||||
def multiply(values):
|
||||
result = diagonal * values
|
||||
if len(weights):
|
||||
np.add.at(result, edge_starts, -weights * values[edge_ends])
|
||||
np.add.at(result, edge_ends, -weights * values[edge_starts])
|
||||
if len(directed_weights):
|
||||
result -= np.bincount(
|
||||
directed_starts,
|
||||
weights=directed_weights * values[directed_ends],
|
||||
minlength=len(values),
|
||||
)
|
||||
return result
|
||||
|
||||
right_hand_side = loads[interior]
|
||||
@@ -3,10 +3,19 @@
|
||||
|
||||
PLANE_VARIANCE_FLOOR_RATIO = 0.025
|
||||
DEPTH_REFERENCE_PERCENTILE = 95.0
|
||||
UNREFINED_DEPTH_REFERENCE = 8.0
|
||||
ADAPTIVE_DEPTH_TRANSPARENCY_RATIO = 0.05
|
||||
DEFAULT_DEPTH_BASELINE = 8.0
|
||||
DEFAULT_DEPTH_DIRECTION = (0.0, 0.0, 1.0)
|
||||
|
||||
|
||||
def use_adaptive_depth_calibration(region_transparent_fraction):
|
||||
"""Return whether the Region input has enough transparent context."""
|
||||
return (
|
||||
float(region_transparent_fraction)
|
||||
>= ADAPTIVE_DEPTH_TRANSPARENCY_RATIO
|
||||
)
|
||||
|
||||
|
||||
def metric_depth_reference(camera_depth, valid, uniform_scale):
|
||||
"""Return the shared upper-percentile scene depth for both Depth shapes."""
|
||||
import numpy as np
|
||||
@@ -1,4 +1,34 @@
|
||||
"""Convert a Cutout Alpha mask into outer contours and holes."""
|
||||
"""Extract Cutout Region contours from a Region Mask."""
|
||||
|
||||
from dataclasses import dataclass
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class RegionContours:
|
||||
components: tuple
|
||||
image_size: tuple[int, int]
|
||||
|
||||
def __post_init__(self):
|
||||
components = tuple(
|
||||
tuple(
|
||||
tuple((float(x), float(y)) for x, y in contour)
|
||||
for contour in component
|
||||
)
|
||||
for component in self.components
|
||||
)
|
||||
image_size = tuple(int(value) for value in self.image_size)
|
||||
if not components or any(not component for component in components):
|
||||
raise ValueError("The Region has no usable contour components")
|
||||
if any(
|
||||
len(contour) < 3
|
||||
for component in components
|
||||
for contour in component
|
||||
):
|
||||
raise ValueError("A Region contour has fewer than three points")
|
||||
if len(image_size) != 2 or min(image_size) <= 0:
|
||||
raise ValueError("The Region contour image size is invalid")
|
||||
object.__setattr__(self, "components", components)
|
||||
object.__setattr__(self, "image_size", image_size)
|
||||
|
||||
|
||||
def _choose_boundary_edge(start, direction, candidates):
|
||||
@@ -27,7 +57,7 @@ def _simplify_axis_aligned_contour(points):
|
||||
return simplified
|
||||
|
||||
|
||||
def trace_mask_contours(mask):
|
||||
def _trace_region_contours(mask):
|
||||
import numpy as np
|
||||
|
||||
mask = np.asarray(mask, dtype=bool)
|
||||
@@ -113,8 +143,8 @@ def _point_inside_contour(point, contour):
|
||||
return inside
|
||||
|
||||
|
||||
def mask_regions(mask):
|
||||
contours = trace_mask_contours(mask)
|
||||
def _group_region_contours(mask):
|
||||
contours = _trace_region_contours(mask)
|
||||
outer = [contour for contour in contours if _signed_contour_area(contour) > 0.0]
|
||||
holes = [contour for contour in contours if _signed_contour_area(contour) < 0.0]
|
||||
if not outer:
|
||||
@@ -143,7 +173,7 @@ def mask_regions(mask):
|
||||
return regions
|
||||
|
||||
|
||||
def _resize_alpha(values, size):
|
||||
def _resize_values(values, size):
|
||||
import numpy as np
|
||||
|
||||
target_width, target_height = (int(value) for value in size)
|
||||
@@ -174,10 +204,10 @@ def _resize_alpha(values, size):
|
||||
)
|
||||
|
||||
|
||||
def alpha_regions(alpha, max_resolution=512, threshold=0.5):
|
||||
def extract_region_contours(region_mask, max_resolution=512, threshold=0.5):
|
||||
import numpy as np
|
||||
|
||||
values = np.asarray(alpha)
|
||||
values = np.asarray(region_mask.values)
|
||||
if values.ndim != 2:
|
||||
raise ValueError("Cutout Alpha must be two-dimensional")
|
||||
if np.issubdtype(values.dtype, np.floating):
|
||||
@@ -188,33 +218,24 @@ def alpha_regions(alpha, max_resolution=512, threshold=0.5):
|
||||
scale = min(1.0, float(max_resolution) / max(source_size))
|
||||
sampled_size = tuple(max(1, round(value * scale)) for value in source_size)
|
||||
if sampled_size != source_size:
|
||||
values = _resize_alpha(values, sampled_size)
|
||||
values = _resize_values(values, sampled_size)
|
||||
mask = values >= 255.0 * threshold
|
||||
regions = mask_regions(mask)
|
||||
components = _group_region_contours(mask)
|
||||
scale_x = source_size[0] / sampled_size[0]
|
||||
scale_y = source_size[1] / sampled_size[1]
|
||||
return (
|
||||
[
|
||||
[
|
||||
[
|
||||
[
|
||||
return RegionContours(
|
||||
components=tuple(
|
||||
tuple(
|
||||
tuple(
|
||||
(
|
||||
min(max(float(x) * scale_x, 0.0), source_size[0]),
|
||||
min(max(float(y) * scale_y, 0.0), source_size[1]),
|
||||
]
|
||||
for x, y in contour
|
||||
]
|
||||
for contour in region
|
||||
]
|
||||
for region in regions
|
||||
],
|
||||
source_size,
|
||||
)
|
||||
|
||||
|
||||
def image_alpha_regions(path, max_resolution=512, threshold=0.5):
|
||||
import numpy as np
|
||||
from PIL import Image
|
||||
|
||||
with Image.open(path) as source:
|
||||
alpha = np.asarray(source.convert("RGBA").getchannel("A"), dtype=np.uint8)
|
||||
return alpha_regions(alpha, max_resolution=max_resolution, threshold=threshold)
|
||||
for x, y in contour
|
||||
)
|
||||
for contour in component
|
||||
)
|
||||
for component in components
|
||||
),
|
||||
image_size=source_size,
|
||||
)
|
||||
@@ -3,7 +3,7 @@ import math
|
||||
POINT_SPACING = 0.1
|
||||
|
||||
|
||||
def polygon_area(points):
|
||||
def contour_area(points):
|
||||
import numpy as np
|
||||
|
||||
points = np.asarray(points, dtype=np.float64)
|
||||
@@ -16,10 +16,10 @@ def polygon_area(points):
|
||||
)
|
||||
|
||||
|
||||
def resample_closed_polygon(polygon, spacing):
|
||||
def resample_closed_contour(contour, spacing):
|
||||
import numpy as np
|
||||
|
||||
points = np.asarray(polygon, dtype=np.float64)
|
||||
points = np.asarray(contour, dtype=np.float64)
|
||||
if len(points) < 3:
|
||||
raise ValueError("The lasso does not define a valid image region")
|
||||
segments = np.roll(points, -1, axis=0) - points
|
||||
@@ -40,9 +40,9 @@ def resample_closed_polygon(polygon, spacing):
|
||||
return points[indices] + segments[indices] * factors[:, None]
|
||||
|
||||
|
||||
def smooth_lasso_contour(polygon, spacing, iterations=16):
|
||||
contour = resample_closed_polygon(polygon, spacing * 0.25)
|
||||
original_area = abs(polygon_area(contour))
|
||||
def smooth_region_contour(contour, spacing, iterations=16):
|
||||
contour = resample_closed_contour(contour, spacing * 0.25)
|
||||
original_area = abs(contour_area(contour))
|
||||
for _index in range(iterations):
|
||||
import numpy as np
|
||||
|
||||
@@ -51,14 +51,14 @@ def smooth_lasso_contour(polygon, spacing, iterations=16):
|
||||
+ contour * 0.75
|
||||
+ np.roll(contour, -1, axis=0) * 0.125
|
||||
)
|
||||
smooth_area = abs(polygon_area(contour))
|
||||
smooth_area = abs(contour_area(contour))
|
||||
if original_area > 0.0 and smooth_area > 0.0:
|
||||
center = contour.mean(axis=0)
|
||||
contour = center + (contour - center) * math.sqrt(
|
||||
original_area / smooth_area
|
||||
)
|
||||
contour = resample_closed_polygon(contour, spacing)
|
||||
if polygon_area(contour) < 0.0:
|
||||
contour = resample_closed_contour(contour, spacing)
|
||||
if contour_area(contour) < 0.0:
|
||||
contour = contour[::-1].copy()
|
||||
return contour
|
||||
|
||||
@@ -98,10 +98,6 @@ def _edge_bucket_indices(starts, ends, minimum_y, bucket_height, bucket_count):
|
||||
return buckets
|
||||
|
||||
|
||||
def points_inside_polygon(points, polygon):
|
||||
return points_inside_contours(points, (polygon,))
|
||||
|
||||
|
||||
def points_inside_contours(points, contours):
|
||||
import numpy as np
|
||||
|
||||
@@ -236,10 +232,6 @@ def uniform_interior_points_for_contours(contours, spacing):
|
||||
)
|
||||
|
||||
|
||||
def uniform_interior_points(contour, spacing):
|
||||
return uniform_interior_points_for_contours((contour,), spacing)
|
||||
|
||||
|
||||
def metric_transform(world_metric=None):
|
||||
import numpy as np
|
||||
|
||||
@@ -259,7 +251,7 @@ def metric_transform(world_metric=None):
|
||||
|
||||
|
||||
def prepare_planar_contours(
|
||||
polygons,
|
||||
contours,
|
||||
image_size,
|
||||
local_bounds,
|
||||
spacing=POINT_SPACING,
|
||||
@@ -274,9 +266,9 @@ def prepare_planar_contours(
|
||||
local_width = x_max - x_min
|
||||
local_height = y_max - y_min
|
||||
transform = metric_transform(world_metric)
|
||||
contours = []
|
||||
for polygon in polygons:
|
||||
pixels = np.asarray(polygon, dtype=np.float64)
|
||||
prepared = []
|
||||
for contour in contours:
|
||||
pixels = np.asarray(contour, dtype=np.float64)
|
||||
if pixels.ndim != 2 or pixels.shape[0] < 3 or pixels.shape[1] != 2:
|
||||
continue
|
||||
local = np.empty_like(pixels)
|
||||
@@ -289,30 +281,29 @@ def prepare_planar_contours(
|
||||
if perimeter <= 1e-12:
|
||||
continue
|
||||
contour_spacing = min(spacing, perimeter / 8.0)
|
||||
smoothed = smooth_lasso_contour(planar, contour_spacing)
|
||||
smoothed = smooth_region_contour(planar, contour_spacing)
|
||||
smoothed_local = smoothed @ np.linalg.inv(transform)
|
||||
smoothed_local[:, 0] = np.clip(smoothed_local[:, 0], x_min, x_max)
|
||||
smoothed_local[:, 1] = np.clip(smoothed_local[:, 1], y_min, y_max)
|
||||
contours.append(smoothed_local @ transform)
|
||||
if not contours:
|
||||
prepared.append(smoothed_local @ transform)
|
||||
if not prepared:
|
||||
raise ValueError("The Cutout has no region large enough for this resolution")
|
||||
interior = uniform_interior_points_for_contours(contours, spacing)
|
||||
return contours, interior, spacing
|
||||
interior = uniform_interior_points_for_contours(prepared, spacing)
|
||||
return prepared, interior, spacing
|
||||
|
||||
def prepare_planar_regions(
|
||||
regions,
|
||||
image_size,
|
||||
def _prepare_region_contours(
|
||||
region_contours,
|
||||
local_bounds,
|
||||
spacing=POINT_SPACING,
|
||||
world_metric=None,
|
||||
):
|
||||
prepared = []
|
||||
errors = []
|
||||
for region in regions:
|
||||
for component in region_contours.components:
|
||||
try:
|
||||
contours, interior, _spacing = prepare_planar_contours(
|
||||
region,
|
||||
image_size,
|
||||
component,
|
||||
region_contours.image_size,
|
||||
local_bounds,
|
||||
spacing=spacing,
|
||||
world_metric=world_metric,
|
||||
@@ -328,25 +319,7 @@ def prepare_planar_regions(
|
||||
return prepared, spacing
|
||||
|
||||
|
||||
def prepare_planar_points(
|
||||
polygon,
|
||||
image_size,
|
||||
local_bounds,
|
||||
spacing=POINT_SPACING,
|
||||
world_metric=None,
|
||||
):
|
||||
contours, interior, spacing = prepare_planar_contours(
|
||||
(polygon,),
|
||||
image_size,
|
||||
local_bounds,
|
||||
spacing=spacing,
|
||||
world_metric=world_metric,
|
||||
)
|
||||
contour = contours[0]
|
||||
return contour, interior, spacing
|
||||
|
||||
|
||||
def triangulate_planar_contours(contours, interior, spacing):
|
||||
def _triangulate_contour_component(contours, interior, spacing):
|
||||
import numpy as np
|
||||
from mathutils import Vector
|
||||
from mathutils.geometry import delaunay_2d_cdt
|
||||
@@ -413,16 +386,28 @@ def boundary_vertices_from_faces(vertex_count, faces):
|
||||
return boundary
|
||||
|
||||
|
||||
def triangulate_planar_regions(regions, spacing, return_components=False):
|
||||
def triangulate_region_contours(
|
||||
region_contours,
|
||||
local_bounds,
|
||||
spacing=POINT_SPACING,
|
||||
world_metric=None,
|
||||
return_components=False,
|
||||
):
|
||||
import numpy as np
|
||||
|
||||
prepared, spacing = _prepare_region_contours(
|
||||
region_contours,
|
||||
local_bounds,
|
||||
spacing=spacing,
|
||||
world_metric=world_metric,
|
||||
)
|
||||
points = []
|
||||
faces = []
|
||||
boundaries = []
|
||||
components = []
|
||||
offset = 0
|
||||
for contours, interior in regions:
|
||||
region_points, region_faces, region_boundary = triangulate_planar_contours(
|
||||
for contours, interior in prepared:
|
||||
region_points, region_faces, region_boundary = _triangulate_contour_component(
|
||||
contours,
|
||||
interior,
|
||||
spacing,
|
||||
@@ -449,4 +434,3 @@ def triangulate_planar_regions(regions, spacing, return_components=False):
|
||||
np.concatenate(boundaries),
|
||||
)
|
||||
return (*result, components) if return_components else result
|
||||
|
||||
@@ -1,10 +0,0 @@
|
||||
"""Load Geometry Nodes assets used by Cutout shapes."""
|
||||
|
||||
from ..common.node_asset import load_asset_node_group
|
||||
from .shape_types import CUTOUT_NODE_GROUP_NAME, CUTOUT_SHAPES
|
||||
|
||||
|
||||
def cutout_node_group(shape):
|
||||
if shape not in CUTOUT_SHAPES:
|
||||
raise ValueError(f"Unsupported Cutout shape: {shape}")
|
||||
return load_asset_node_group(CUTOUT_NODE_GROUP_NAME)
|
||||
842
src/anyimage/operators/cutout_tool/region_to_shape.py
Normal file
842
src/anyimage/operators/cutout_tool/region_to_shape.py
Normal file
@@ -0,0 +1,842 @@
|
||||
import json
|
||||
from dataclasses import dataclass
|
||||
from enum import StrEnum
|
||||
|
||||
import bpy
|
||||
|
||||
from ...preferences import addon_preferences, configured_max_ai_input_size
|
||||
from ...properties import BEN2_MODEL_KEY, DEFAULT_MOGE2_MODEL_KEY
|
||||
from ...runtime import JobOperatorBase
|
||||
from ...server.media.resolution import limited_image_size
|
||||
from ...server.media.resolution import (
|
||||
DEFAULT_MAX_AI_INPUT_SIZE,
|
||||
MAX_MAX_AI_INPUT_SIZE,
|
||||
MIN_MAX_AI_INPUT_SIZE,
|
||||
)
|
||||
from ..common.image_data import (
|
||||
cleanup_image_input,
|
||||
color_data_name,
|
||||
depth_data_name,
|
||||
image_rgba,
|
||||
load_normal_result_image,
|
||||
image_empty_bounds,
|
||||
prepare_region_input,
|
||||
require_image_empty,
|
||||
source_start_frame,
|
||||
video_total_frames,
|
||||
)
|
||||
from ..common.image_interaction import report_image_edit_exception
|
||||
from ..common.image_region import (
|
||||
ImageEditWarning,
|
||||
RegionMask,
|
||||
RegionPath,
|
||||
rasterize_region_path,
|
||||
)
|
||||
from ..common.mesh_result import (
|
||||
create_image_material,
|
||||
finalize_object_result,
|
||||
load_asset_node_group,
|
||||
load_image_geometry,
|
||||
modifier_input_identifier,
|
||||
set_modifier_input,
|
||||
)
|
||||
from ..common.ai_job import (
|
||||
invoke_ai_setup_if_needed,
|
||||
moge2_parameters,
|
||||
production_device,
|
||||
require_environment,
|
||||
require_input_path,
|
||||
require_model,
|
||||
)
|
||||
from .geometry.depth_calibration import (
|
||||
DEFAULT_DEPTH_BASELINE,
|
||||
DEFAULT_DEPTH_DIRECTION,
|
||||
depth_balloon_projection_parameters,
|
||||
metric_depth_reference,
|
||||
use_adaptive_depth_calibration,
|
||||
)
|
||||
from .geometry.triangulation import (
|
||||
metric_transform,
|
||||
triangulate_region_contours,
|
||||
)
|
||||
from .geometry.balloon import balloon_profile as calculate_balloon_profile
|
||||
from .geometry.region_contours import extract_region_contours
|
||||
|
||||
CUTOUT_ALPHA_THRESHOLD = 0.1
|
||||
CUTOUT_CLEANUP_UI_SCALE = 100.0
|
||||
BALLOON_ATTRIBUTE_NAME = "o_balloon"
|
||||
CUTOUT_NODE_GROUP_NAME = "O Image Cutout"
|
||||
CUTOUT_NODE_GROUP_NAMES = {
|
||||
"SURFACE": "O Image Surface",
|
||||
"BALLOON": "O Image Balloon",
|
||||
"DEPTH_BALLOON": "O Image Depth Balloon",
|
||||
"DEPTH_SURFACE": "O Image Depth Surface",
|
||||
}
|
||||
CUTOUT_SHAPE_INDICES = {
|
||||
shape: index for index, shape in enumerate(CUTOUT_NODE_GROUP_NAMES)
|
||||
}
|
||||
CUTOUT_SHAPES = frozenset(CUTOUT_NODE_GROUP_NAMES)
|
||||
DEPTH_CUTOUT_SHAPES = frozenset({"DEPTH_BALLOON", "DEPTH_SURFACE"})
|
||||
_HIDDEN = {"HIDDEN", "SKIP_SAVE"}
|
||||
|
||||
|
||||
class NormalMode(StrEnum):
|
||||
NONE = "NONE"
|
||||
TANGENT = "TANGENT"
|
||||
OBJECT = "OBJECT"
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class BaseShape:
|
||||
vertices: object
|
||||
faces: object
|
||||
uv: object
|
||||
balloon: object
|
||||
|
||||
CUTOUT_AXIS_TO_SOURCE = (
|
||||
(0.0, -1.0, 0.0, 0.0),
|
||||
(0.0, 0.0, 1.0, 0.0),
|
||||
(-1.0, 0.0, 0.0, 0.0),
|
||||
(0.0, 0.0, 0.0, 1.0),
|
||||
)
|
||||
|
||||
|
||||
def _cutout_depth_image(source_object, points, valid, camera_depth):
|
||||
import numpy as np
|
||||
|
||||
points = np.asarray(points, dtype=np.float32)
|
||||
valid = np.asarray(valid, dtype=bool)
|
||||
height, width = valid.shape
|
||||
if points.shape != (height, width, 3):
|
||||
raise ValueError("Depth Surface point dimensions do not match")
|
||||
discontinuity = _depth_discontinuity_map(camera_depth, valid)
|
||||
discontinuity[~valid] = -1.0
|
||||
pixels = np.empty((height, width, 4), dtype=np.float32)
|
||||
pixels[:, :, :3] = points
|
||||
pixels[:, :, 3] = discontinuity
|
||||
image = bpy.data.images.new(
|
||||
depth_data_name(source_object),
|
||||
width=width,
|
||||
height=height,
|
||||
alpha=True,
|
||||
float_buffer=True,
|
||||
)
|
||||
image.colorspace_settings.name = "Non-Color"
|
||||
image.alpha_mode = "CHANNEL_PACKED"
|
||||
image.pixels.foreach_set(np.flipud(pixels).ravel())
|
||||
image.pack()
|
||||
return image
|
||||
|
||||
|
||||
def _cutout_world_matrix(source_matrix, crop_center):
|
||||
from mathutils import Matrix
|
||||
|
||||
center_x, center_y = (float(value) for value in crop_center)
|
||||
return (
|
||||
source_matrix
|
||||
@ Matrix.Translation((center_x, center_y, 0.0))
|
||||
@ Matrix(CUTOUT_AXIS_TO_SOURCE)
|
||||
)
|
||||
|
||||
|
||||
def _orient_cutout_vertices(vertices):
|
||||
import numpy as np
|
||||
|
||||
vertices = np.asarray(vertices, dtype=np.float64)
|
||||
return np.column_stack(
|
||||
(
|
||||
-vertices[:, 2],
|
||||
-vertices[:, 0],
|
||||
vertices[:, 1],
|
||||
)
|
||||
)
|
||||
|
||||
|
||||
def _cutout_domain(
|
||||
region_contours,
|
||||
bounds,
|
||||
spacing,
|
||||
world_metric,
|
||||
balloon_algorithm="POISSON",
|
||||
):
|
||||
import numpy as np
|
||||
|
||||
x_min, x_max, y_min, y_max = bounds
|
||||
|
||||
planar, planar_faces, boundary, components = triangulate_region_contours(
|
||||
region_contours,
|
||||
bounds,
|
||||
spacing=spacing,
|
||||
world_metric=world_metric,
|
||||
return_components=True,
|
||||
)
|
||||
profile = calculate_balloon_profile(
|
||||
balloon_algorithm, planar, planar_faces, boundary, components
|
||||
)
|
||||
faces = np.asarray(planar_faces, dtype=np.int32)
|
||||
local = planar @ np.linalg.inv(metric_transform(world_metric))
|
||||
uv = np.column_stack(
|
||||
(
|
||||
(local[:, 0] - x_min) / (x_max - x_min),
|
||||
(local[:, 1] - y_min) / (y_max - y_min),
|
||||
)
|
||||
).astype(np.float32)
|
||||
crop_center = np.asarray(
|
||||
((x_min + x_max) * 0.5, (y_min + y_max) * 0.5),
|
||||
dtype=np.float64,
|
||||
)
|
||||
local -= crop_center
|
||||
vertices = np.column_stack((local, np.zeros(len(local), dtype=np.float64)))
|
||||
return vertices, faces, uv, boundary, planar, profile
|
||||
|
||||
|
||||
def _world_plane_metric(matrix_world):
|
||||
from mathutils import Vector
|
||||
|
||||
linear = matrix_world.to_3x3()
|
||||
x = linear @ Vector((1.0, 0.0, 0.0))
|
||||
y = linear @ Vector((0.0, 1.0, 0.0))
|
||||
z = linear @ Vector((0.0, 0.0, 1.0))
|
||||
return ((x.dot(x), x.dot(y)), (x.dot(y), y.dot(y))), z.length
|
||||
|
||||
|
||||
def _cutout_cleanup_value(spacing, world_metric):
|
||||
import numpy as np
|
||||
|
||||
if spacing <= 0.0:
|
||||
raise ValueError("Cutout point spacing must be greater than zero")
|
||||
area_scale = abs(float(np.linalg.det(metric_transform(world_metric))))
|
||||
if area_scale <= 1e-12:
|
||||
raise ValueError("The Image Empty has a degenerate world transform")
|
||||
return CUTOUT_CLEANUP_UI_SCALE * float(spacing) ** 2 / area_scale
|
||||
|
||||
|
||||
def _depth_uniform_scale(
|
||||
source_object,
|
||||
field,
|
||||
bounds=None,
|
||||
):
|
||||
import numpy as np
|
||||
|
||||
intrinsics = field.intrinsics
|
||||
width, height = field.image_size
|
||||
x_min, x_max, y_min, y_max = bounds or image_empty_bounds(source_object)
|
||||
plane_width = x_max - x_min
|
||||
plane_height = y_max - y_min
|
||||
fx, fy = float(intrinsics[0, 0]), float(intrinsics[1, 1])
|
||||
if fx <= 0.0 or fy <= 0.0:
|
||||
raise ValueError("Camera intrinsics contain an invalid focal length")
|
||||
scale_x = plane_width * fx / (max(width - 1, 1) * field.reference_depth)
|
||||
scale_y = plane_height * fy / (max(height - 1, 1) * field.reference_depth)
|
||||
return float(np.sqrt(scale_x * scale_y))
|
||||
|
||||
|
||||
def _depth_discontinuity_map(depth, valid):
|
||||
import numpy as np
|
||||
|
||||
depth = np.asarray(depth, dtype=np.float32)
|
||||
valid = np.asarray(valid, dtype=bool) & np.isfinite(depth) & (depth > 0.0)
|
||||
padded_depth = np.pad(depth, 1, mode="edge")
|
||||
padded_valid = np.pad(valid, 1, mode="edge")
|
||||
windows = np.lib.stride_tricks.sliding_window_view(padded_depth, (3, 3))
|
||||
usable = np.lib.stride_tricks.sliding_window_view(padded_valid, (3, 3))
|
||||
minimum = np.min(np.where(usable, windows, np.inf), axis=(-2, -1))
|
||||
maximum = np.max(np.where(usable, windows, -np.inf), axis=(-2, -1))
|
||||
result = (maximum - minimum) / np.maximum(np.abs(depth), 1e-6)
|
||||
result[~usable.all(axis=(-2, -1))] = 1.0
|
||||
return result.astype(np.float32)
|
||||
|
||||
|
||||
def _counter_clockwise_faces(points, faces):
|
||||
import numpy as np
|
||||
|
||||
points = np.asarray(points, dtype=np.float64)
|
||||
result = np.asarray(faces, dtype=np.int32).copy()
|
||||
polygons = points[result]
|
||||
following = np.roll(polygons, -1, axis=1)
|
||||
signed_area = np.sum(
|
||||
polygons[..., 0] * following[..., 1]
|
||||
- polygons[..., 1] * following[..., 0],
|
||||
axis=1,
|
||||
)
|
||||
result[signed_area < 0.0] = result[signed_area < 0.0, ::-1]
|
||||
return result
|
||||
|
||||
|
||||
def _assign_uv(mesh, uv):
|
||||
import numpy as np
|
||||
|
||||
for layer in list(mesh.uv_layers):
|
||||
mesh.uv_layers.remove(layer)
|
||||
uv_layer = mesh.uv_layers.new(name="UVMap")
|
||||
loop_vertices = np.empty(len(mesh.loops), dtype=np.int32)
|
||||
mesh.loops.foreach_get("vertex_index", loop_vertices)
|
||||
loop_uv = np.asarray(uv, dtype=np.float32)[loop_vertices]
|
||||
uv_layer.data.foreach_set("uv", loop_uv.ravel())
|
||||
|
||||
|
||||
def _assign_float_point_attribute(mesh, name, values):
|
||||
import numpy as np
|
||||
|
||||
attribute = mesh.attributes.new(name=name, type="FLOAT", domain="POINT")
|
||||
attribute.data.foreach_set("value", np.asarray(values, dtype=np.float32))
|
||||
|
||||
|
||||
def _crop_plane_bounds(source_object, crop_bounds, source_size):
|
||||
source_width, source_height = (float(value) for value in source_size)
|
||||
left, top, right, bottom = (float(value) for value in crop_bounds)
|
||||
x_min, x_max, y_min, y_max = image_empty_bounds(source_object)
|
||||
width = x_max - x_min
|
||||
height = y_max - y_min
|
||||
return (
|
||||
x_min + left * width / source_width,
|
||||
x_min + right * width / source_width,
|
||||
y_max - bottom * height / source_height,
|
||||
y_max - top * height / source_height,
|
||||
)
|
||||
|
||||
|
||||
def _cropped_color_image(source_object, crop_bounds, source_rgba):
|
||||
import numpy as np
|
||||
|
||||
source = source_object.data
|
||||
rgba = np.asarray(source_rgba)
|
||||
left, top, right, bottom = crop_bounds
|
||||
width = right - left
|
||||
height = bottom - top
|
||||
if width <= 0 or height <= 0:
|
||||
raise ValueError("The Cutout crop dimensions must be positive")
|
||||
cropped = np.zeros((height, width, 4), dtype=np.float32)
|
||||
source_height, source_width = rgba.shape[:2]
|
||||
source_left = max(0, left)
|
||||
source_top = max(0, top)
|
||||
source_right = min(source_width, right)
|
||||
source_bottom = min(source_height, bottom)
|
||||
if source_right > source_left and source_bottom > source_top:
|
||||
cropped[
|
||||
source_top - top : source_bottom - top,
|
||||
source_left - left : source_right - left,
|
||||
] = rgba[source_top:source_bottom, source_left:source_right]
|
||||
image = bpy.data.images.new(
|
||||
color_data_name(source_object),
|
||||
width=width,
|
||||
height=height,
|
||||
alpha=True,
|
||||
)
|
||||
try:
|
||||
image.colorspace_settings.name = source.colorspace_settings.name
|
||||
image.alpha_mode = source.alpha_mode
|
||||
image.pixels.foreach_set(np.flipud(cropped).ravel())
|
||||
image.pack()
|
||||
image.update()
|
||||
except Exception:
|
||||
bpy.data.images.remove(image, do_unlink=True)
|
||||
raise
|
||||
return image
|
||||
|
||||
|
||||
def _extract_cutout_contours(
|
||||
region_mask,
|
||||
mesh_detail,
|
||||
threshold=CUTOUT_ALPHA_THRESHOLD,
|
||||
):
|
||||
contour_resolution = max(512, min(int(mesh_detail) * 8, 1024))
|
||||
return extract_region_contours(
|
||||
region_mask,
|
||||
max_resolution=contour_resolution,
|
||||
threshold=threshold,
|
||||
)
|
||||
|
||||
|
||||
def cutout_node_group(shape):
|
||||
if shape not in CUTOUT_SHAPES:
|
||||
raise ValueError(f"Unsupported Cutout shape: {shape}")
|
||||
return load_asset_node_group(CUTOUT_NODE_GROUP_NAME)
|
||||
|
||||
|
||||
def limited_cutout_mesh_detail(mesh_detail, crop_size, max_input_size):
|
||||
crop_width, crop_height = (int(value) for value in crop_size)
|
||||
limited_width, limited_height = limited_image_size(
|
||||
(crop_width, crop_height),
|
||||
max_input_size,
|
||||
)
|
||||
scale = max(limited_width, limited_height) / max(crop_width, crop_height)
|
||||
return max(4, min(1024, round(int(mesh_detail) * scale)))
|
||||
|
||||
|
||||
def _cutout_point_spacing(source_object, detail):
|
||||
import math
|
||||
|
||||
x_min, x_max, y_min, y_max = image_empty_bounds(source_object)
|
||||
metric, _world_z_scale = _world_plane_metric(source_object.matrix_world)
|
||||
width = (x_max - x_min) * math.sqrt(max(metric[0][0], 0.0))
|
||||
height = (y_max - y_min) * math.sqrt(max(metric[1][1], 0.0))
|
||||
return max(width, height) / max(int(detail), 8)
|
||||
|
||||
|
||||
def build_base_shape(context, source_object, detail, region_mask):
|
||||
import numpy as np
|
||||
|
||||
bounds = _crop_plane_bounds(
|
||||
source_object,
|
||||
region_mask.bounds,
|
||||
tuple(source_object.data.size),
|
||||
)
|
||||
region_contours = _extract_cutout_contours(
|
||||
region_mask,
|
||||
detail,
|
||||
threshold=CUTOUT_ALPHA_THRESHOLD,
|
||||
)
|
||||
world_metric, world_z_scale = _world_plane_metric(source_object.matrix_world)
|
||||
if world_z_scale <= 1e-10:
|
||||
raise ValueError("The Image Empty has a degenerate world transform")
|
||||
preferences = addon_preferences(context)
|
||||
balloon_algorithm = getattr(
|
||||
preferences, "balloon_profile_algorithm", "POISSON"
|
||||
)
|
||||
point_spacing = _cutout_point_spacing(source_object, detail)
|
||||
domain = _cutout_domain(
|
||||
region_contours,
|
||||
bounds,
|
||||
point_spacing,
|
||||
world_metric,
|
||||
balloon_algorithm,
|
||||
)
|
||||
source_vertices, faces, uv, _boundary, planar, balloon = domain
|
||||
return BaseShape(
|
||||
vertices=_orient_cutout_vertices(source_vertices),
|
||||
faces=_counter_clockwise_faces(planar, faces),
|
||||
uv=np.asarray(uv, dtype=np.float32),
|
||||
balloon=np.asarray(balloon, dtype=np.float64) / world_z_scale,
|
||||
)
|
||||
|
||||
|
||||
def create_shape_object(
|
||||
context,
|
||||
source_object,
|
||||
shape,
|
||||
detail,
|
||||
region_mask,
|
||||
base_shape,
|
||||
*,
|
||||
field=None,
|
||||
normal_image=None,
|
||||
normal_mode=NormalMode.NONE,
|
||||
source_rgba=None,
|
||||
):
|
||||
import numpy as np
|
||||
|
||||
normal_mode = NormalMode(normal_mode)
|
||||
bounds = _crop_plane_bounds(
|
||||
source_object,
|
||||
region_mask.bounds,
|
||||
tuple(source_object.data.size),
|
||||
)
|
||||
crop_center = (
|
||||
(bounds[0] + bounds[1]) * 0.5,
|
||||
(bounds[2] + bounds[3]) * 0.5,
|
||||
)
|
||||
world_metric, _world_z_scale = _world_plane_metric(source_object.matrix_world)
|
||||
cleanup_value = _cutout_cleanup_value(
|
||||
_cutout_point_spacing(source_object, detail),
|
||||
world_metric,
|
||||
)
|
||||
color_image = _cropped_color_image(
|
||||
source_object,
|
||||
region_mask.bounds,
|
||||
image_rgba(source_object.data) if source_rgba is None else source_rgba,
|
||||
)
|
||||
material = None
|
||||
depth_image = None
|
||||
mesh = None
|
||||
mesh_object = None
|
||||
try:
|
||||
material = create_image_material(
|
||||
source_object,
|
||||
color_image,
|
||||
0.0,
|
||||
normal_image=normal_image,
|
||||
normal_space=(
|
||||
NormalMode.TANGENT.value
|
||||
if normal_mode == NormalMode.NONE
|
||||
else normal_mode.value
|
||||
),
|
||||
scene=context.scene,
|
||||
)
|
||||
depth_reference = None
|
||||
depth_uniform_scale = None
|
||||
depth_direction = None
|
||||
if shape in DEPTH_CUTOUT_SHAPES:
|
||||
points = field.points
|
||||
valid = field.valid
|
||||
calibration_valid = field.calibration_valid
|
||||
depth_image = _cutout_depth_image(
|
||||
source_object, points, valid, points[..., 2]
|
||||
)
|
||||
depth_uniform_scale = _depth_uniform_scale(
|
||||
source_object,
|
||||
field,
|
||||
bounds,
|
||||
)
|
||||
if not use_adaptive_depth_calibration(
|
||||
field.region_transparent_fraction
|
||||
):
|
||||
depth_reference = DEFAULT_DEPTH_BASELINE
|
||||
if shape == "DEPTH_BALLOON":
|
||||
depth_direction = DEFAULT_DEPTH_DIRECTION
|
||||
elif shape == "DEPTH_SURFACE":
|
||||
depth_reference = metric_depth_reference(
|
||||
points[..., 2],
|
||||
calibration_valid,
|
||||
depth_uniform_scale,
|
||||
)
|
||||
else:
|
||||
depth_reference, depth_direction = (
|
||||
depth_balloon_projection_parameters(
|
||||
base_shape.vertices,
|
||||
base_shape.uv,
|
||||
points[..., 2],
|
||||
calibration_valid,
|
||||
depth_uniform_scale,
|
||||
)
|
||||
)
|
||||
|
||||
source_name = source_object.name
|
||||
mesh = bpy.data.meshes.new(source_name)
|
||||
mesh.from_pydata(
|
||||
np.asarray(base_shape.vertices).tolist(),
|
||||
[],
|
||||
base_shape.faces,
|
||||
)
|
||||
for face in mesh.polygons:
|
||||
face.use_smooth = True
|
||||
_assign_uv(mesh, base_shape.uv)
|
||||
_assign_float_point_attribute(
|
||||
mesh,
|
||||
BALLOON_ATTRIBUTE_NAME,
|
||||
base_shape.balloon,
|
||||
)
|
||||
mesh.materials.append(material)
|
||||
mesh.update()
|
||||
|
||||
mesh_object = bpy.data.objects.new(source_name, mesh)
|
||||
context.collection.objects.link(mesh_object)
|
||||
mesh_object.matrix_world = _cutout_world_matrix(
|
||||
source_object.matrix_world,
|
||||
crop_center,
|
||||
)
|
||||
mesh_object["anyimage_mesh_shape"] = shape
|
||||
node_group = cutout_node_group(shape)
|
||||
modifier = mesh_object.modifiers.new(name=node_group.name, type="NODES")
|
||||
modifier.node_group = node_group
|
||||
modifier.show_group_selector = False
|
||||
for name, value in (
|
||||
("Shape", CUTOUT_SHAPE_INDICES[shape]),
|
||||
("Cleanup", cleanup_value),
|
||||
("Smooth Iterations", 0),
|
||||
("Smooth", 1.0),
|
||||
(
|
||||
"Thickness",
|
||||
1.0 if shape in {"BALLOON", "DEPTH_BALLOON"} else 0.0,
|
||||
),
|
||||
):
|
||||
set_modifier_input(
|
||||
modifier,
|
||||
modifier_input_identifier(node_group, name),
|
||||
value,
|
||||
)
|
||||
if shape in DEPTH_CUTOUT_SHAPES:
|
||||
for name, value in (
|
||||
("Depth Scale", 1.0),
|
||||
("Uniform Scale", depth_uniform_scale),
|
||||
("Base Plane Depth", depth_reference),
|
||||
("Depth Image", depth_image),
|
||||
):
|
||||
set_modifier_input(
|
||||
modifier,
|
||||
modifier_input_identifier(node_group, name),
|
||||
value,
|
||||
)
|
||||
if shape == "DEPTH_BALLOON":
|
||||
set_modifier_input(
|
||||
modifier,
|
||||
modifier_input_identifier(node_group, "Depth Direction"),
|
||||
depth_direction,
|
||||
)
|
||||
return finalize_object_result(
|
||||
context,
|
||||
source_object,
|
||||
mesh_object,
|
||||
keep_source=True,
|
||||
)
|
||||
except Exception:
|
||||
if mesh_object is not None:
|
||||
bpy.data.objects.remove(mesh_object, do_unlink=True)
|
||||
if mesh is not None and mesh.users == 0:
|
||||
bpy.data.meshes.remove(mesh)
|
||||
if material is not None and material.users == 0:
|
||||
bpy.data.materials.remove(material)
|
||||
if color_image.users == 0:
|
||||
bpy.data.images.remove(color_image, do_unlink=True)
|
||||
if normal_image is not None and normal_image.users == 0:
|
||||
bpy.data.images.remove(normal_image, do_unlink=True)
|
||||
if depth_image is not None and depth_image.users == 0:
|
||||
bpy.data.images.remove(depth_image, do_unlink=True)
|
||||
raise
|
||||
|
||||
|
||||
def region_to_shape(
|
||||
context,
|
||||
source_object,
|
||||
shape,
|
||||
detail,
|
||||
region_mask,
|
||||
*,
|
||||
field=None,
|
||||
normal_image=None,
|
||||
normal_mode=NormalMode.NONE,
|
||||
source_rgba=None,
|
||||
):
|
||||
crop_width = region_mask.bounds[2] - region_mask.bounds[0]
|
||||
crop_height = region_mask.bounds[3] - region_mask.bounds[1]
|
||||
detail = limited_cutout_mesh_detail(
|
||||
detail,
|
||||
(crop_width, crop_height),
|
||||
configured_max_ai_input_size(context),
|
||||
)
|
||||
base_shape = build_base_shape(
|
||||
context,
|
||||
source_object,
|
||||
detail,
|
||||
region_mask,
|
||||
)
|
||||
result = create_shape_object(
|
||||
context,
|
||||
source_object,
|
||||
shape,
|
||||
detail,
|
||||
region_mask,
|
||||
base_shape,
|
||||
field=field,
|
||||
normal_image=normal_image,
|
||||
normal_mode=normal_mode,
|
||||
source_rgba=source_rgba,
|
||||
)
|
||||
undo_message = f"Convert to {shape.replace('_', ' ').title()}"
|
||||
if "FINISHED" not in bpy.ops.ed.undo_push(message=undo_message):
|
||||
raise RuntimeError(f"Unable to create the {undo_message} Undo step")
|
||||
return result
|
||||
|
||||
|
||||
def _region_mask_from_image(path, bounds):
|
||||
image = bpy.data.images.load(str(path), check_existing=False)
|
||||
try:
|
||||
return RegionMask(
|
||||
image_rgba(image)[:, :, 3],
|
||||
tuple(int(value) for value in bounds),
|
||||
)
|
||||
finally:
|
||||
bpy.data.images.remove(image, do_unlink=True)
|
||||
|
||||
|
||||
def _generated_region_mask(result, input_path, source_bounds):
|
||||
if not isinstance(result.value, dict) or "region" not in result.value:
|
||||
return _region_mask_from_image(input_path, source_bounds)
|
||||
local_bounds = tuple(
|
||||
int(value)
|
||||
for value in json.loads(
|
||||
result.file("region_bounds").read_text(encoding="utf-8")
|
||||
)
|
||||
)
|
||||
bounds = (
|
||||
source_bounds[0] + local_bounds[0],
|
||||
source_bounds[1] + local_bounds[1],
|
||||
source_bounds[0] + local_bounds[2],
|
||||
source_bounds[1] + local_bounds[3],
|
||||
)
|
||||
return _region_mask_from_image(result.file("region"), bounds)
|
||||
|
||||
|
||||
def cutout_moge_settings(context):
|
||||
preferences = addon_preferences(context)
|
||||
model = getattr(preferences, "mesh_depth_model", DEFAULT_MOGE2_MODEL_KEY)
|
||||
if not model.startswith("MOGE2_"):
|
||||
model = DEFAULT_MOGE2_MODEL_KEY
|
||||
return (
|
||||
model,
|
||||
int(getattr(preferences, "mesh_resolution_level", 5)),
|
||||
)
|
||||
|
||||
|
||||
class CutoutRegionToShape(JobOperatorBase, bpy.types.Operator):
|
||||
bl_idname = "anyimage.cutout_region_to_shape"
|
||||
bl_label = "Cutout Region to Shape"
|
||||
bl_description = "Build the selected image Region as a Cutout shape"
|
||||
bl_options = {"INTERNAL"}
|
||||
job_type = "generate-region-data"
|
||||
starting_message = "Generating Cutout Region data"
|
||||
|
||||
input_path: bpy.props.StringProperty(options=_HIDDEN)
|
||||
source_object_name: bpy.props.StringProperty(options=_HIDDEN)
|
||||
region_path_json: bpy.props.StringProperty(options=_HIDDEN)
|
||||
region_bounds_json: bpy.props.StringProperty(options=_HIDDEN)
|
||||
shape: bpy.props.StringProperty(options=_HIDDEN)
|
||||
mesh_detail: bpy.props.IntProperty(
|
||||
default=128,
|
||||
min=4,
|
||||
max=1024,
|
||||
options=_HIDDEN,
|
||||
)
|
||||
model: bpy.props.StringProperty(
|
||||
default=DEFAULT_MOGE2_MODEL_KEY,
|
||||
options=_HIDDEN,
|
||||
)
|
||||
resolution_level: bpy.props.IntProperty(
|
||||
default=5,
|
||||
min=0,
|
||||
max=9,
|
||||
options=_HIDDEN,
|
||||
)
|
||||
max_input_size: bpy.props.IntProperty(
|
||||
default=DEFAULT_MAX_AI_INPUT_SIZE,
|
||||
min=MIN_MAX_AI_INPUT_SIZE,
|
||||
max=MAX_MAX_AI_INPUT_SIZE,
|
||||
options=_HIDDEN,
|
||||
)
|
||||
normal_mode: bpy.props.StringProperty(
|
||||
default=NormalMode.NONE.value,
|
||||
options=_HIDDEN,
|
||||
)
|
||||
refine_selection: bpy.props.BoolProperty(default=False, options=_HIDDEN)
|
||||
|
||||
def _needs_generation(self):
|
||||
return (
|
||||
self.refine_selection
|
||||
or self.shape in DEPTH_CUTOUT_SHAPES
|
||||
or NormalMode(self.normal_mode) != NormalMode.NONE
|
||||
)
|
||||
|
||||
def execute(self, context):
|
||||
try:
|
||||
source_object = require_image_empty(self.source_object_name)
|
||||
if video_total_frames(source_object.data) != 1:
|
||||
raise ValueError("Cutout currently supports still images")
|
||||
if self.shape not in CUTOUT_SHAPES:
|
||||
raise ValueError(f"Unsupported Cutout shape: {self.shape}")
|
||||
self.normal_mode = NormalMode(self.normal_mode).value
|
||||
region_path = RegionPath.from_json(self.region_path_json)
|
||||
source_rgba = image_rgba(source_object.data)
|
||||
region_mask = rasterize_region_path(
|
||||
tuple(source_object.data.size),
|
||||
region_path,
|
||||
source_alpha=source_rgba[:, :, 3],
|
||||
alpha_threshold=CUTOUT_ALPHA_THRESHOLD,
|
||||
antialias=True,
|
||||
)
|
||||
context.view_layer.objects.active = source_object
|
||||
source_object.select_set(True)
|
||||
self.max_input_size = configured_max_ai_input_size(context)
|
||||
if not self._needs_generation():
|
||||
region_to_shape(
|
||||
context,
|
||||
source_object,
|
||||
self.shape,
|
||||
self.mesh_detail,
|
||||
region_mask,
|
||||
source_rgba=source_rgba,
|
||||
)
|
||||
return {"FINISHED"}
|
||||
setup_result = invoke_ai_setup_if_needed()
|
||||
if setup_result is not None:
|
||||
return setup_result
|
||||
self.model, self.resolution_level = cutout_moge_settings(context)
|
||||
self.region_bounds_json = json.dumps(
|
||||
region_mask.bounds,
|
||||
separators=(",", ":"),
|
||||
)
|
||||
self.input_path = str(
|
||||
prepare_region_input(source_object.data, region_mask)
|
||||
)
|
||||
except (
|
||||
ImageEditWarning,
|
||||
OSError,
|
||||
RuntimeError,
|
||||
TypeError,
|
||||
ValueError,
|
||||
) as error:
|
||||
report_image_edit_exception(self, error)
|
||||
return {"CANCELLED"}
|
||||
result = super().execute(context)
|
||||
if "CANCELLED" in result:
|
||||
cleanup_image_input(self.input_path, True)
|
||||
return result
|
||||
|
||||
def request(self, context):
|
||||
require_environment()
|
||||
source_path = require_input_path(self.input_path)
|
||||
output_depth = self.shape in DEPTH_CUTOUT_SHAPES
|
||||
normal_mode = NormalMode(self.normal_mode)
|
||||
if self.refine_selection:
|
||||
require_model(BEN2_MODEL_KEY)
|
||||
if output_depth or normal_mode != NormalMode.NONE:
|
||||
require_model(self.model)
|
||||
parameters = {
|
||||
"input": str(source_path),
|
||||
"device": production_device().lower(),
|
||||
"refine_region": self.refine_selection,
|
||||
"output_depth": output_depth,
|
||||
"normal_mode": normal_mode.value,
|
||||
"max_input_size": int(self.max_input_size),
|
||||
}
|
||||
if self.refine_selection:
|
||||
parameters["refine_model"] = BEN2_MODEL_KEY
|
||||
if output_depth or normal_mode != NormalMode.NONE:
|
||||
start_frame = source_start_frame(self.source_object_name)
|
||||
parameters.update(
|
||||
model=self.model,
|
||||
resolution_level=self.resolution_level,
|
||||
)
|
||||
parameters.update(
|
||||
moge2_parameters(
|
||||
context.scene.anyimage_settings,
|
||||
source_path,
|
||||
self.model,
|
||||
self.resolution_level,
|
||||
start_frame,
|
||||
)
|
||||
)
|
||||
return parameters
|
||||
|
||||
def response(self, context, result):
|
||||
source_object = require_image_empty(self.source_object_name)
|
||||
source_bounds = tuple(json.loads(self.region_bounds_json))
|
||||
region_mask = _generated_region_mask(
|
||||
result,
|
||||
self.input_path,
|
||||
source_bounds,
|
||||
)
|
||||
value = result.value if isinstance(result.value, dict) else {}
|
||||
normal_image = (
|
||||
load_normal_result_image(result.file("normal"), source_object)
|
||||
if "normal" in value
|
||||
else None
|
||||
)
|
||||
field = (
|
||||
load_image_geometry(result.file("image_geometry"))
|
||||
if self.shape in DEPTH_CUTOUT_SHAPES
|
||||
else None
|
||||
)
|
||||
region_to_shape(
|
||||
context,
|
||||
source_object,
|
||||
self.shape,
|
||||
self.mesh_detail,
|
||||
region_mask,
|
||||
field=field,
|
||||
normal_image=normal_image,
|
||||
normal_mode=NormalMode(self.normal_mode),
|
||||
)
|
||||
return f"{self.shape.replace('_', ' ').title()} is ready"
|
||||
|
||||
def cleanup(self):
|
||||
cleanup_image_input(self.input_path, True)
|
||||
@@ -1,27 +1,42 @@
|
||||
import json
|
||||
|
||||
import bpy
|
||||
|
||||
from ...properties import (
|
||||
refine_selection_property,
|
||||
)
|
||||
from ..common.image import is_image_empty, video_total_frames
|
||||
from ..common.image_edit import (
|
||||
ImageEditWarning,
|
||||
apply_image_mask_settings,
|
||||
draw_image_mask_settings,
|
||||
polygon_bounds,
|
||||
from ..common.image_data import image_rgba, require_image_empty, video_total_frames
|
||||
from ..common.image_interaction import (
|
||||
apply_region_settings,
|
||||
draw_region_settings,
|
||||
report_image_edit_exception,
|
||||
resolve_image_edit_click,
|
||||
)
|
||||
from ..common.image_interaction import hidden_string_property, serialize_matrix
|
||||
from ..common.image_gesture import ImageMaskGesture, gesture_property
|
||||
from ..common.image_mask import selected_image_rgba
|
||||
from ..common.image_interaction import serialize_matrix
|
||||
from ..common.image_interaction import ImageRegionGesture, gesture_property
|
||||
from ..common.image_region import (
|
||||
ImageEditWarning,
|
||||
RegionPath,
|
||||
rasterize_region_path,
|
||||
)
|
||||
|
||||
CUTOUT_DETAIL_MIN = 4
|
||||
CUTOUT_DETAIL_MAX = 1024
|
||||
|
||||
|
||||
def _validate_cutout_region(
|
||||
source_object,
|
||||
region_path,
|
||||
):
|
||||
image = source_object.data
|
||||
rgba = image_rgba(image)
|
||||
rasterize_region_path(
|
||||
tuple(image.size),
|
||||
region_path,
|
||||
source_alpha=rgba[:, :, 3],
|
||||
alpha_threshold=0.1,
|
||||
antialias=True,
|
||||
)
|
||||
|
||||
|
||||
def cutout_mesh_detail(image_size, point_spacing):
|
||||
longest_edge = max(int(value) for value in image_size)
|
||||
detail = round(longest_edge / max(int(point_spacing), 1))
|
||||
@@ -34,23 +49,21 @@ def _pie_button(
|
||||
label,
|
||||
icon,
|
||||
source_name,
|
||||
selection_data,
|
||||
region_path_json,
|
||||
refine_selection,
|
||||
normal_analysis,
|
||||
invert,
|
||||
normal_mode,
|
||||
mesh_detail,
|
||||
):
|
||||
operator = pie.operator(
|
||||
"anyimage.convert_cutout_shape",
|
||||
"anyimage.cutout_region_to_shape",
|
||||
text=label,
|
||||
icon=icon,
|
||||
)
|
||||
operator.source_object_name = source_name
|
||||
operator.selection_data = selection_data
|
||||
operator.region_path_json = region_path_json
|
||||
operator.shape = shape
|
||||
operator.refine_selection = refine_selection
|
||||
operator.normal_analysis = normal_analysis
|
||||
operator.invert = invert
|
||||
operator.normal_mode = normal_mode
|
||||
operator.mesh_detail = mesh_detail
|
||||
|
||||
|
||||
@@ -58,29 +71,27 @@ def _open_shape_pie(
|
||||
context,
|
||||
event,
|
||||
source_object,
|
||||
polygon,
|
||||
region_path,
|
||||
refine_selection=False,
|
||||
invert=False,
|
||||
):
|
||||
source_name = source_object.name
|
||||
selection_data = json.dumps(polygon, separators=(",", ":"))
|
||||
region_path_json = region_path.to_json()
|
||||
settings = context.scene.anyimage_settings
|
||||
point_spacing = settings.cutout_point_spacing
|
||||
normal_analysis = bool(getattr(settings, "cutout_normal_analysis", False))
|
||||
normal_mode = str(getattr(settings, "cutout_normal_mode", "NONE"))
|
||||
mesh_detail = cutout_mesh_detail(source_object.data.size, point_spacing)
|
||||
|
||||
def draw(menu, _context):
|
||||
pie = menu.layout.menu_pie()
|
||||
_pie_button(
|
||||
pie,
|
||||
"CUTOUT",
|
||||
"SURFACE",
|
||||
"Surface",
|
||||
"MESH_GRID",
|
||||
source_name,
|
||||
selection_data,
|
||||
region_path_json,
|
||||
refine_selection,
|
||||
normal_analysis,
|
||||
invert,
|
||||
normal_mode,
|
||||
mesh_detail,
|
||||
)
|
||||
_pie_button(
|
||||
@@ -89,10 +100,9 @@ def _open_shape_pie(
|
||||
"Balloon",
|
||||
"MESH_UVSPHERE",
|
||||
source_name,
|
||||
selection_data,
|
||||
region_path_json,
|
||||
refine_selection,
|
||||
normal_analysis,
|
||||
invert,
|
||||
normal_mode,
|
||||
mesh_detail,
|
||||
)
|
||||
_pie_button(
|
||||
@@ -101,10 +111,9 @@ def _open_shape_pie(
|
||||
"Depth Balloon",
|
||||
"META_BALL",
|
||||
source_name,
|
||||
selection_data,
|
||||
region_path_json,
|
||||
refine_selection,
|
||||
normal_analysis,
|
||||
invert,
|
||||
normal_mode,
|
||||
mesh_detail,
|
||||
)
|
||||
_pie_button(
|
||||
@@ -113,10 +122,9 @@ def _open_shape_pie(
|
||||
"Depth Surface",
|
||||
"SURFACE_NCURVE",
|
||||
source_name,
|
||||
selection_data,
|
||||
region_path_json,
|
||||
refine_selection,
|
||||
normal_analysis,
|
||||
invert,
|
||||
normal_mode,
|
||||
mesh_detail,
|
||||
)
|
||||
|
||||
@@ -128,16 +136,16 @@ def _open_shape_pie(
|
||||
)
|
||||
|
||||
|
||||
class SelectCutoutRegion(ImageMaskGesture, bpy.types.Operator):
|
||||
class SelectCutoutRegion(ImageRegionGesture, bpy.types.Operator):
|
||||
bl_idname = "anyimage.select_cutout_region"
|
||||
bl_label = "Select Cutout Region"
|
||||
bl_description = "Lasso an Image Empty region, then choose its mesh shape"
|
||||
bl_options = {"INTERNAL"}
|
||||
active_tool_id = "anyimage.cutout_lasso"
|
||||
|
||||
source_object_name: hidden_string_property()
|
||||
source_matrix_data: hidden_string_property()
|
||||
selection_data: hidden_string_property()
|
||||
source_object_name: bpy.props.StringProperty(options={"HIDDEN", "SKIP_SAVE"})
|
||||
source_matrix_data: bpy.props.StringProperty(options={"HIDDEN", "SKIP_SAVE"})
|
||||
region_path_json: bpy.props.StringProperty(options={"HIDDEN", "SKIP_SAVE"})
|
||||
refine_selection: refine_selection_property()
|
||||
invert: bpy.props.BoolProperty(default=False, options={"SKIP_SAVE"})
|
||||
gesture: gesture_property()
|
||||
@@ -167,15 +175,15 @@ class SelectCutoutRegion(ImageMaskGesture, bpy.types.Operator):
|
||||
if video_total_frames(source_object.data) > 1:
|
||||
self.report({"WARNING"}, "Image tools only support a still image")
|
||||
return {"CANCELLED"}
|
||||
apply_image_mask_settings(self, context)
|
||||
apply_region_settings(self, context)
|
||||
self.invert = False
|
||||
self.source_object_name = source_object.name
|
||||
self.source_matrix_data = serialize_matrix(source_object.matrix_world)
|
||||
width, height = (int(value) for value in source_object.data.size)
|
||||
self.selection_data = json.dumps(
|
||||
((0, 0), (width, 0), (width, height), (0, height)),
|
||||
separators=(",", ":"),
|
||||
)
|
||||
self.region_path_json = RegionPath(
|
||||
points=((0, 0), (width, 0), (width, height), (0, height)),
|
||||
fill_rule="NONZERO",
|
||||
).to_json()
|
||||
return self.execute(context)
|
||||
|
||||
def modal(self, context, event):
|
||||
@@ -183,47 +191,27 @@ class SelectCutoutRegion(ImageMaskGesture, bpy.types.Operator):
|
||||
return super().modal(context, event)
|
||||
|
||||
def execute(self, context):
|
||||
source_object = bpy.data.objects.get(self.source_object_name)
|
||||
if not is_image_empty(source_object):
|
||||
self.report({"ERROR"}, "The source Image Empty is no longer available")
|
||||
try:
|
||||
source_object = require_image_empty(self.source_object_name)
|
||||
except RuntimeError as error:
|
||||
self.report({"ERROR"}, str(error))
|
||||
return {"CANCELLED"}
|
||||
try:
|
||||
polygon = json.loads(self.selection_data)
|
||||
polygon_bounds(polygon)
|
||||
selected_image_rgba(
|
||||
source_object, polygon, self.invert, "NONZERO"
|
||||
)
|
||||
except (ImageEditWarning, json.JSONDecodeError, TypeError, ValueError) as error:
|
||||
region_path = RegionPath.from_json(self.region_path_json)
|
||||
_validate_cutout_region(source_object, region_path)
|
||||
except (ImageEditWarning, TypeError, ValueError) as error:
|
||||
report_image_edit_exception(self, error)
|
||||
return {"CANCELLED"}
|
||||
_open_shape_pie(
|
||||
context,
|
||||
self._pie_event,
|
||||
source_object,
|
||||
polygon,
|
||||
region_path,
|
||||
self.refine_selection,
|
||||
self.invert,
|
||||
)
|
||||
return {"FINISHED"}
|
||||
|
||||
|
||||
class ActivateCutoutTool(bpy.types.Operator):
|
||||
bl_idname = "anyimage.activate_cutout_tool"
|
||||
bl_label = "Cutout Tool"
|
||||
bl_description = "Activate the Cutout Lasso tool"
|
||||
bl_options = {"INTERNAL"}
|
||||
|
||||
@classmethod
|
||||
def poll(cls, context):
|
||||
return context.area is not None and context.area.type == "VIEW_3D"
|
||||
|
||||
def execute(self, _context):
|
||||
return bpy.ops.wm.tool_set_by_id(
|
||||
name=CutoutTool.bl_idname,
|
||||
space_type="VIEW_3D",
|
||||
)
|
||||
|
||||
|
||||
class CutoutTool(bpy.types.WorkSpaceTool):
|
||||
bl_space_type = "VIEW_3D"
|
||||
bl_context_mode = "OBJECT"
|
||||
@@ -248,5 +236,5 @@ class CutoutTool(bpy.types.WorkSpaceTool):
|
||||
def draw_settings(context, layout, _tool):
|
||||
settings = context.scene.anyimage_settings
|
||||
layout.prop(settings, "cutout_point_spacing")
|
||||
layout.prop(settings, "cutout_normal_analysis")
|
||||
draw_image_mask_settings(context, layout, "cutout_refine_selection")
|
||||
layout.prop(settings, "cutout_normal_mode")
|
||||
draw_region_settings(context, layout, "cutout_refine_selection")
|
||||
@@ -1,214 +0,0 @@
|
||||
import json
|
||||
|
||||
import bpy
|
||||
|
||||
from ...preferences import addon_preferences, configured_max_ai_input_size
|
||||
from ...properties import BEN2_MODEL_KEY, DEFAULT_MOGE2_MODEL_KEY
|
||||
from ...runtime import JobOperatorBase
|
||||
from ...server.media.resolution import (
|
||||
DEFAULT_MAX_AI_INPUT_SIZE,
|
||||
MAX_MAX_AI_INPUT_SIZE,
|
||||
MIN_MAX_AI_INPUT_SIZE,
|
||||
)
|
||||
from ..common.image import (
|
||||
is_image_empty,
|
||||
prepare_image_input,
|
||||
remove_temporary_input,
|
||||
video_total_frames,
|
||||
)
|
||||
from ..common.image_edit import report_image_edit_exception
|
||||
from ..common.operator import (
|
||||
HIDDEN,
|
||||
cleanup_temporary_input,
|
||||
input_path,
|
||||
invoke_ai_setup_if_needed,
|
||||
moge2_parameters,
|
||||
production_device,
|
||||
require_environment,
|
||||
require_model,
|
||||
source_start_frame,
|
||||
)
|
||||
from .build_shapes import build_cutout_response, build_local_cutout
|
||||
from .shape_types import DEPTH_CUTOUT_SHAPES, LOCAL_CUTOUT_SHAPES
|
||||
|
||||
|
||||
def cutout_moge_settings(context):
|
||||
preferences = addon_preferences(context)
|
||||
model = getattr(preferences, "mesh_depth_model", DEFAULT_MOGE2_MODEL_KEY)
|
||||
if not model.startswith("MOGE2_"):
|
||||
model = DEFAULT_MOGE2_MODEL_KEY
|
||||
return (
|
||||
model,
|
||||
int(getattr(preferences, "mesh_resolution_level", 5)),
|
||||
)
|
||||
|
||||
|
||||
class GenerateCutoutShape(JobOperatorBase, bpy.types.Operator):
|
||||
bl_idname = "anyimage.generate_cutout_shape"
|
||||
bl_label = "Generate Cutout Shape"
|
||||
job_type = "prepare-geometry-data"
|
||||
starting_message = "Preparing Cutout shape"
|
||||
|
||||
input_path: bpy.props.StringProperty(options=HIDDEN)
|
||||
source_object_name: bpy.props.StringProperty(options=HIDDEN)
|
||||
shape: bpy.props.StringProperty(options=HIDDEN)
|
||||
selection_data: bpy.props.StringProperty(options=HIDDEN)
|
||||
mesh_detail: bpy.props.IntProperty(default=128, min=4, max=1024, options=HIDDEN)
|
||||
model: bpy.props.StringProperty(default=DEFAULT_MOGE2_MODEL_KEY, options=HIDDEN)
|
||||
resolution_level: bpy.props.IntProperty(default=5, min=0, max=9, options=HIDDEN)
|
||||
max_input_size: bpy.props.IntProperty(
|
||||
default=DEFAULT_MAX_AI_INPUT_SIZE,
|
||||
min=MIN_MAX_AI_INPUT_SIZE,
|
||||
max=MAX_MAX_AI_INPUT_SIZE,
|
||||
options=HIDDEN,
|
||||
)
|
||||
normal_analysis: bpy.props.BoolProperty(default=False, options=HIDDEN)
|
||||
delete_input: bpy.props.BoolProperty(default=False, options=HIDDEN)
|
||||
refine_selection: bpy.props.BoolProperty(default=False, options=HIDDEN)
|
||||
invert: bpy.props.BoolProperty(default=False, options=HIDDEN)
|
||||
|
||||
def request(self, context):
|
||||
require_environment()
|
||||
if self.shape in DEPTH_CUTOUT_SHAPES or self.normal_analysis:
|
||||
require_model(self.model)
|
||||
if self.refine_selection:
|
||||
require_model(BEN2_MODEL_KEY)
|
||||
polygon = json.loads(self.selection_data) if self.selection_data else None
|
||||
if polygon is None or len(polygon) < 3:
|
||||
raise ValueError("The Cutout selection is invalid")
|
||||
parameters = {
|
||||
"input": str(input_path(self.input_path)),
|
||||
"analysis_input": str(input_path(self.input_path)),
|
||||
"shape": self.shape,
|
||||
"polygon": polygon,
|
||||
"mesh_detail": self.mesh_detail,
|
||||
"device": production_device().lower(),
|
||||
"video_frames": 1,
|
||||
"start_frame": source_start_frame(self.source_object_name),
|
||||
"max_input_size": int(self.max_input_size),
|
||||
"refine_selection": self.refine_selection,
|
||||
"normal_analysis": self.normal_analysis,
|
||||
"invert": self.invert,
|
||||
}
|
||||
if self.shape in DEPTH_CUTOUT_SHAPES or self.normal_analysis:
|
||||
parameters.update(
|
||||
model=self.model,
|
||||
resolution_level=self.resolution_level,
|
||||
)
|
||||
if self.refine_selection:
|
||||
parameters["refine_model"] = BEN2_MODEL_KEY
|
||||
if self.shape not in DEPTH_CUTOUT_SHAPES:
|
||||
return parameters
|
||||
settings = context.scene.anyimage_settings
|
||||
parameters.update(
|
||||
moge2_parameters(
|
||||
settings,
|
||||
input_path(self.input_path),
|
||||
self.model,
|
||||
self.resolution_level,
|
||||
source_start_frame(self.source_object_name),
|
||||
)
|
||||
)
|
||||
parameters.update(shape=self.shape, polygon=polygon)
|
||||
return parameters
|
||||
|
||||
def response(self, context, result):
|
||||
build_cutout_response(context, result, self)
|
||||
return f"{self.shape.replace('_', ' ').title()} is ready"
|
||||
|
||||
def cleanup(self):
|
||||
cleanup_temporary_input(self.input_path, self.delete_input)
|
||||
|
||||
|
||||
def start_cutout_job(
|
||||
operator,
|
||||
context,
|
||||
shape,
|
||||
selection_data,
|
||||
refine_selection=False,
|
||||
normal_analysis=False,
|
||||
invert=False,
|
||||
):
|
||||
source_object = context.object
|
||||
if video_total_frames(source_object.data) != 1:
|
||||
operator.report({"ERROR"}, "Cutout currently supports still images")
|
||||
return {"CANCELLED"}
|
||||
setup_result = invoke_ai_setup_if_needed()
|
||||
if setup_result is not None:
|
||||
return setup_result
|
||||
try:
|
||||
source_path, delete_input = prepare_image_input(
|
||||
source_object.data,
|
||||
context.scene,
|
||||
)
|
||||
model, resolution_level = cutout_moge_settings(context)
|
||||
except (RuntimeError, ValueError) as error:
|
||||
operator.report({"ERROR"}, str(error))
|
||||
return {"CANCELLED"}
|
||||
result = bpy.ops.anyimage.generate_cutout_shape(
|
||||
input_path=str(source_path),
|
||||
source_object_name=source_object.name,
|
||||
shape=shape,
|
||||
selection_data=selection_data,
|
||||
mesh_detail=operator.mesh_detail,
|
||||
model=model,
|
||||
resolution_level=resolution_level,
|
||||
max_input_size=configured_max_ai_input_size(context),
|
||||
normal_analysis=normal_analysis,
|
||||
delete_input=delete_input,
|
||||
refine_selection=refine_selection,
|
||||
invert=invert,
|
||||
)
|
||||
if "CANCELLED" in result and delete_input:
|
||||
remove_temporary_input(source_path)
|
||||
return {"CANCELLED"} if "CANCELLED" in result else {"FINISHED"}
|
||||
|
||||
|
||||
class ConvertCutoutShape(bpy.types.Operator):
|
||||
bl_idname = "anyimage.convert_cutout_shape"
|
||||
bl_label = "Convert Cutout Shape"
|
||||
bl_description = "Convert the saved Cutout selection into the chosen shape"
|
||||
bl_options = {"UNDO"}
|
||||
|
||||
source_object_name: bpy.props.StringProperty(options=HIDDEN)
|
||||
selection_data: bpy.props.StringProperty(options=HIDDEN)
|
||||
shape: bpy.props.StringProperty(options=HIDDEN)
|
||||
mesh_detail: bpy.props.IntProperty(default=128, min=4, max=1024, options=HIDDEN)
|
||||
refine_selection: bpy.props.BoolProperty(default=False, options=HIDDEN)
|
||||
normal_analysis: bpy.props.BoolProperty(default=False, options=HIDDEN)
|
||||
invert: bpy.props.BoolProperty(default=False, options=HIDDEN)
|
||||
|
||||
def execute(self, context):
|
||||
source_object = bpy.data.objects.get(self.source_object_name)
|
||||
if not is_image_empty(source_object):
|
||||
self.report({"ERROR"}, "The source Image Empty is no longer available")
|
||||
return {"CANCELLED"}
|
||||
context.view_layer.objects.active = source_object
|
||||
source_object.select_set(True)
|
||||
if (
|
||||
self.shape in LOCAL_CUTOUT_SHAPES
|
||||
and not self.refine_selection
|
||||
and not self.normal_analysis
|
||||
):
|
||||
try:
|
||||
build_local_cutout(
|
||||
self,
|
||||
context,
|
||||
source_object,
|
||||
self.shape,
|
||||
self.selection_data,
|
||||
self.invert,
|
||||
)
|
||||
except Exception as error:
|
||||
report_image_edit_exception(self, error)
|
||||
return {"CANCELLED"}
|
||||
return {"FINISHED"}
|
||||
return start_cutout_job(
|
||||
self,
|
||||
context,
|
||||
self.shape,
|
||||
self.selection_data,
|
||||
self.refine_selection,
|
||||
self.normal_analysis,
|
||||
self.invert,
|
||||
)
|
||||
@@ -1,23 +0,0 @@
|
||||
"""Cutout shape identifiers and Geometry Nodes asset names."""
|
||||
|
||||
CUTOUT_NODE_GROUP_NAME = "O Image Cutout"
|
||||
CUTOUT_NODE_GROUP_NAMES = {
|
||||
"CUTOUT": "O Image Surface",
|
||||
"BALLOON": "O Image Balloon",
|
||||
"DEPTH_BALLOON": "O Image Depth Balloon",
|
||||
"DEPTH_SURFACE": "O Image Depth Surface",
|
||||
}
|
||||
CUTOUT_SHAPE_LABELS = {
|
||||
"CUTOUT": "Surface",
|
||||
"BALLOON": "Balloon",
|
||||
"DEPTH_BALLOON": "Depth Balloon",
|
||||
"DEPTH_SURFACE": "Depth Surface",
|
||||
}
|
||||
CUTOUT_SHAPE_INDICES = {
|
||||
shape: index for index, shape in enumerate(CUTOUT_NODE_GROUP_NAMES)
|
||||
}
|
||||
|
||||
CUTOUT_SHAPES = frozenset(CUTOUT_NODE_GROUP_NAMES)
|
||||
LOCAL_CUTOUT_SHAPES = frozenset({"CUTOUT", "BALLOON"})
|
||||
DEPTH_CUTOUT_SHAPES = frozenset({"DEPTH_BALLOON", "DEPTH_SURFACE"})
|
||||
BALLOON_ATTRIBUTE_NAME = "o_balloon"
|
||||
@@ -1,18 +1,28 @@
|
||||
from pathlib import Path
|
||||
|
||||
import bpy
|
||||
|
||||
from ..runtime import JobOperatorBase
|
||||
from ..properties import BEN2_MODEL_KEY, REALESRGAN_GENERAL_WDN_MODEL_KEY
|
||||
from .common.operator import (
|
||||
background_input_path,
|
||||
from .common.ai_job import (
|
||||
require_background_input_path,
|
||||
da3_parameters,
|
||||
input_path,
|
||||
require_input_path,
|
||||
moge2_parameters,
|
||||
open_job_directory,
|
||||
require_environment,
|
||||
require_model,
|
||||
)
|
||||
|
||||
|
||||
def _open_job_directory(directory):
|
||||
directory = Path(directory)
|
||||
if not directory.is_dir():
|
||||
raise RuntimeError(f"Job directory does not exist: {directory}")
|
||||
result = bpy.ops.wm.path_open(filepath=str(directory))
|
||||
if "FINISHED" not in result:
|
||||
raise RuntimeError(f"Unable to open job directory: {directory}")
|
||||
|
||||
|
||||
class DebugDa3(JobOperatorBase, bpy.types.Operator):
|
||||
bl_idname = "anyimage.debug_da3"
|
||||
bl_label = "Debug DA3"
|
||||
@@ -27,7 +37,7 @@ class DebugDa3(JobOperatorBase, bpy.types.Operator):
|
||||
return {
|
||||
**da3_parameters(
|
||||
settings,
|
||||
input_path(settings.input_path),
|
||||
require_input_path(settings.input_path),
|
||||
model,
|
||||
settings.process_res,
|
||||
1,
|
||||
@@ -38,7 +48,7 @@ class DebugDa3(JobOperatorBase, bpy.types.Operator):
|
||||
}
|
||||
|
||||
def response(self, _context, result):
|
||||
open_job_directory(result.directory)
|
||||
_open_job_directory(result.directory)
|
||||
return f"Saved DA3 API prediction to {result.directory}"
|
||||
|
||||
|
||||
@@ -55,7 +65,7 @@ class DebugMoge2(JobOperatorBase, bpy.types.Operator):
|
||||
return {
|
||||
**moge2_parameters(
|
||||
settings,
|
||||
input_path(settings.input_path),
|
||||
require_input_path(settings.input_path),
|
||||
settings.moge_model,
|
||||
settings.moge_resolution_level,
|
||||
1,
|
||||
@@ -64,7 +74,7 @@ class DebugMoge2(JobOperatorBase, bpy.types.Operator):
|
||||
}
|
||||
|
||||
def response(self, _context, result):
|
||||
open_job_directory(result.directory)
|
||||
_open_job_directory(result.directory)
|
||||
return f"Exported MoGe-2 data to {result.directory}"
|
||||
|
||||
|
||||
@@ -79,7 +89,7 @@ class DebugBen2(JobOperatorBase, bpy.types.Operator):
|
||||
settings = context.scene.anyimage_settings
|
||||
require_model(BEN2_MODEL_KEY)
|
||||
return {
|
||||
"input": str(background_input_path(settings.background_input_path)),
|
||||
"input": str(require_background_input_path(settings.background_input_path)),
|
||||
"model": BEN2_MODEL_KEY,
|
||||
"device": settings.device.lower(),
|
||||
"video_frames": settings.video_frames,
|
||||
@@ -87,7 +97,7 @@ class DebugBen2(JobOperatorBase, bpy.types.Operator):
|
||||
}
|
||||
|
||||
def response(self, _context, result):
|
||||
open_job_directory(result.directory)
|
||||
_open_job_directory(result.directory)
|
||||
return f"Removed background to {result.directory}"
|
||||
|
||||
|
||||
@@ -107,7 +117,7 @@ class DebugUpscale(JobOperatorBase, bpy.types.Operator):
|
||||
settings = context.scene.anyimage_settings
|
||||
require_model(self.model)
|
||||
return {
|
||||
"input": str(input_path(settings.input_path)),
|
||||
"input": str(require_input_path(settings.input_path)),
|
||||
"model": self.model,
|
||||
"device": settings.device.lower(),
|
||||
"video_frames": settings.video_frames,
|
||||
@@ -115,5 +125,5 @@ class DebugUpscale(JobOperatorBase, bpy.types.Operator):
|
||||
}
|
||||
|
||||
def response(self, _context, result):
|
||||
open_job_directory(result.directory)
|
||||
_open_job_directory(result.directory)
|
||||
return f"Saved upscale API prediction to {result.directory}"
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
import bpy
|
||||
|
||||
from ..common.image_edit import draw_image_mask_settings
|
||||
from ..common.image_interaction import draw_region_settings
|
||||
from .edit_mask import EditImageMask
|
||||
|
||||
|
||||
@@ -21,4 +21,4 @@ class ImageBoxTool(bpy.types.WorkSpaceTool):
|
||||
)
|
||||
|
||||
def draw_settings(context, layout, _tool):
|
||||
draw_image_mask_settings(context, layout, "image_box_refine_selection")
|
||||
draw_region_settings(context, layout, "image_box_refine_selection")
|
||||
|
||||
@@ -2,38 +2,119 @@ import json
|
||||
|
||||
import bpy
|
||||
|
||||
from ...properties import refine_selection_property
|
||||
from ...properties import BEN2_MODEL_KEY, refine_selection_property
|
||||
from ...runtime import JobOperatorBase
|
||||
from ..common.image import (
|
||||
is_image_empty,
|
||||
prepare_image_input,
|
||||
remove_temporary_input,
|
||||
from ..common.image_data import (
|
||||
cleanup_image_input,
|
||||
create_pixel_result_image,
|
||||
image_rgba,
|
||||
image_pixels,
|
||||
prepare_region_input,
|
||||
replace_empty_image,
|
||||
require_image_empty,
|
||||
video_total_frames,
|
||||
)
|
||||
from ..common.image_edit import (
|
||||
ImageEditWarning,
|
||||
image_mask_refine_request,
|
||||
polygon_bounds,
|
||||
from ..common.image_interaction import (
|
||||
image_edit_poll,
|
||||
report_image_edit_exception,
|
||||
)
|
||||
from ..common.image_gesture import ImageMaskGesture, gesture_property
|
||||
from ..common.image_interaction import hidden_string_property
|
||||
from ..common.image_mask import (
|
||||
edit_image_object,
|
||||
polygon_mask,
|
||||
refined_mask_from_pixels,
|
||||
from ..common.image_interaction import ImageRegionGesture, gesture_property
|
||||
from ..common.image_region import (
|
||||
ImageEditWarning,
|
||||
RegionPath,
|
||||
crop_pixels_to_alpha,
|
||||
rasterize_region_path,
|
||||
)
|
||||
from ..common.ai_job import (
|
||||
production_device,
|
||||
require_environment,
|
||||
require_model,
|
||||
)
|
||||
from ..common.operator import HIDDEN, cleanup_temporary_input, push_undo
|
||||
|
||||
class EditImageMask(ImageMaskGesture, bpy.types.Operator):
|
||||
|
||||
_HIDDEN = {"HIDDEN", "SKIP_SAVE"}
|
||||
|
||||
|
||||
def _refined_mask_from_pixels(mask_pixels, mask_size, bounds, source_size):
|
||||
import numpy as np
|
||||
|
||||
mask_width, mask_height = (int(value) for value in mask_size)
|
||||
source_width, source_height = (int(value) for value in source_size)
|
||||
local = np.flipud(
|
||||
np.asarray(mask_pixels, dtype=np.float32).reshape(
|
||||
(mask_height, mask_width, 4)
|
||||
)
|
||||
)[:, :, 3]
|
||||
result = np.zeros((source_height, source_width), dtype=np.float32)
|
||||
left, top, right, bottom = (int(value) for value in bounds)
|
||||
source_left = max(left, 0)
|
||||
source_top = max(top, 0)
|
||||
source_right = min(right, source_width)
|
||||
source_bottom = min(bottom, source_height)
|
||||
if source_right <= source_left or source_bottom <= source_top:
|
||||
return result
|
||||
result[source_top:source_bottom, source_left:source_right] = local[
|
||||
source_top - top : source_bottom - top,
|
||||
source_left - left : source_right - left,
|
||||
]
|
||||
return result
|
||||
|
||||
|
||||
def _edit_image_mask_pixels(
|
||||
source_pixels,
|
||||
source_size,
|
||||
region_mask,
|
||||
invert=False,
|
||||
):
|
||||
"""Apply a mask while preserving source RGB, then tightly crop the canvas."""
|
||||
import numpy as np
|
||||
|
||||
width, height = (int(value) for value in source_size)
|
||||
rgba = np.flipud(
|
||||
np.asarray(source_pixels, dtype=np.float32).reshape((height, width, 4))
|
||||
)
|
||||
mask = np.asarray(region_mask, dtype=np.float32)
|
||||
if invert:
|
||||
mask = 1.0 - mask
|
||||
mask = np.where(rgba[:, :, 3] > 0.0, mask, 0.0)
|
||||
rgba[:, :, 3] *= mask
|
||||
return crop_pixels_to_alpha(
|
||||
np.flipud(rgba).ravel(),
|
||||
(0, 0, width, height),
|
||||
)
|
||||
|
||||
|
||||
def _edit_image_object(source_object, region_mask, invert=False):
|
||||
image = source_object.data
|
||||
source_size = tuple(image.size)
|
||||
pixels, crop_bounds = _edit_image_mask_pixels(
|
||||
image_pixels(image),
|
||||
source_size,
|
||||
region_mask,
|
||||
invert,
|
||||
)
|
||||
left, top, right, bottom = crop_bounds
|
||||
result_image = create_pixel_result_image(
|
||||
image,
|
||||
pixels,
|
||||
(right - left, bottom - top),
|
||||
)
|
||||
return replace_empty_image(
|
||||
source_object,
|
||||
result_image,
|
||||
placement_bounds=crop_bounds,
|
||||
)
|
||||
|
||||
|
||||
class EditImageMask(ImageRegionGesture, bpy.types.Operator):
|
||||
bl_idname = "anyimage.edit_image_mask"
|
||||
bl_label = "Edit Image Mask"
|
||||
bl_description = "Edit and tightly crop an Image Empty with a gesture"
|
||||
bl_options = {"REGISTER", "UNDO"}
|
||||
|
||||
source_object_name: hidden_string_property()
|
||||
source_matrix_data: hidden_string_property()
|
||||
selection_data: hidden_string_property()
|
||||
source_object_name: bpy.props.StringProperty(options={"HIDDEN", "SKIP_SAVE"})
|
||||
source_matrix_data: bpy.props.StringProperty(options={"HIDDEN", "SKIP_SAVE"})
|
||||
region_path_json: bpy.props.StringProperty(options={"HIDDEN", "SKIP_SAVE"})
|
||||
refine_selection: refine_selection_property()
|
||||
invert: bpy.props.BoolProperty(
|
||||
name="Invert",
|
||||
@@ -45,35 +126,37 @@ class EditImageMask(ImageMaskGesture, bpy.types.Operator):
|
||||
|
||||
|
||||
def execute(self, context):
|
||||
source_object = bpy.data.objects.get(self.source_object_name)
|
||||
if not is_image_empty(source_object):
|
||||
self.report({"ERROR"}, "The source Image Empty is no longer available")
|
||||
try:
|
||||
source_object = require_image_empty(self.source_object_name)
|
||||
except RuntimeError as error:
|
||||
self.report({"ERROR"}, str(error))
|
||||
return {"CANCELLED"}
|
||||
try:
|
||||
polygon = json.loads(self.selection_data)
|
||||
polygon_bounds(polygon)
|
||||
region_path = RegionPath.from_json(self.region_path_json)
|
||||
if video_total_frames(source_object.data) > 1:
|
||||
raise RuntimeError("Image tools only support a still image")
|
||||
region_mask = rasterize_region_path(
|
||||
tuple(source_object.data.size),
|
||||
region_path,
|
||||
source_alpha=image_rgba(source_object.data)[:, :, 3],
|
||||
)
|
||||
if self.refine_selection:
|
||||
source_path, delete_input = prepare_image_input(
|
||||
source_object.data, context.scene
|
||||
source_path = prepare_region_input(
|
||||
source_object.data,
|
||||
region_mask,
|
||||
)
|
||||
result = bpy.ops.anyimage.refine_image_mask(
|
||||
input_path=str(source_path),
|
||||
delete_input=delete_input,
|
||||
source_object_name=self.source_object_name,
|
||||
selection_data=self.selection_data,
|
||||
invert=self.invert,
|
||||
region_bounds_json=json.dumps(region_mask.bounds),
|
||||
)
|
||||
if "CANCELLED" in result:
|
||||
if delete_input:
|
||||
remove_temporary_input(source_path)
|
||||
cleanup_image_input(source_path, True)
|
||||
return {"CANCELLED"}
|
||||
return {"FINISHED"}
|
||||
edit_image_object(
|
||||
_edit_image_object(
|
||||
source_object,
|
||||
polygon_mask(tuple(source_object.data.size), polygon),
|
||||
self.invert,
|
||||
region_mask.full_values(tuple(source_object.data.size)),
|
||||
)
|
||||
except (json.JSONDecodeError, RuntimeError, TypeError, ValueError) as error:
|
||||
report_image_edit_exception(self, error)
|
||||
@@ -87,29 +170,39 @@ class RefineImageMask(JobOperatorBase, bpy.types.Operator):
|
||||
job_type = "refine-image-mask"
|
||||
starting_message = "Refining image selection"
|
||||
|
||||
input_path: bpy.props.StringProperty(options=HIDDEN)
|
||||
delete_input: bpy.props.BoolProperty(default=False, options=HIDDEN)
|
||||
source_object_name: hidden_string_property()
|
||||
selection_data: hidden_string_property()
|
||||
invert: bpy.props.BoolProperty(default=False, options=HIDDEN)
|
||||
input_path: bpy.props.StringProperty(options=_HIDDEN)
|
||||
source_object_name: bpy.props.StringProperty(options={"HIDDEN", "SKIP_SAVE"})
|
||||
region_bounds_json: bpy.props.StringProperty(options={"HIDDEN", "SKIP_SAVE"})
|
||||
|
||||
def request(self, _context):
|
||||
return image_mask_refine_request(self.input_path, self.selection_data)
|
||||
require_environment()
|
||||
require_model(BEN2_MODEL_KEY)
|
||||
return {
|
||||
"input": str(self.input_path),
|
||||
"model": BEN2_MODEL_KEY,
|
||||
"device": production_device().lower(),
|
||||
}
|
||||
|
||||
def response(self, _context, result):
|
||||
source_object = bpy.data.objects.get(self.source_object_name)
|
||||
if not is_image_empty(source_object):
|
||||
raise RuntimeError("The source Image Empty is no longer available")
|
||||
source_object = require_image_empty(self.source_object_name)
|
||||
import numpy as np
|
||||
|
||||
bounds = tuple(json.loads(result.file("bounds").read_text(encoding="utf-8")))
|
||||
source_bounds = tuple(json.loads(self.region_bounds_json))
|
||||
local_bounds = tuple(
|
||||
json.loads(result.file("bounds").read_text(encoding="utf-8"))
|
||||
)
|
||||
bounds = (
|
||||
source_bounds[0] + local_bounds[0],
|
||||
source_bounds[1] + local_bounds[1],
|
||||
source_bounds[0] + local_bounds[2],
|
||||
source_bounds[1] + local_bounds[3],
|
||||
)
|
||||
refined_image = bpy.data.images.load(
|
||||
str(result.file("image")), check_existing=False
|
||||
)
|
||||
try:
|
||||
refined_pixels = np.empty(len(refined_image.pixels), dtype=np.float32)
|
||||
refined_image.pixels.foreach_get(refined_pixels)
|
||||
mask = refined_mask_from_pixels(
|
||||
refined_pixels = image_pixels(refined_image)
|
||||
mask = _refined_mask_from_pixels(
|
||||
refined_pixels,
|
||||
tuple(refined_image.size),
|
||||
bounds,
|
||||
@@ -118,15 +211,16 @@ class RefineImageMask(JobOperatorBase, bpy.types.Operator):
|
||||
finally:
|
||||
bpy.data.images.remove(refined_image, do_unlink=True)
|
||||
try:
|
||||
edit_image_object(source_object, mask, self.invert)
|
||||
_edit_image_object(source_object, mask)
|
||||
except ImageEditWarning as error:
|
||||
self.report({"WARNING"}, str(error))
|
||||
return str(error)
|
||||
push_undo("Edit Image Mask")
|
||||
if "FINISHED" not in bpy.ops.ed.undo_push(message="Edit Image Mask"):
|
||||
raise RuntimeError("Unable to create the Edit Image Mask Undo step")
|
||||
return "Image selection refined"
|
||||
|
||||
def cleanup(self):
|
||||
cleanup_temporary_input(self.input_path, self.delete_input)
|
||||
cleanup_image_input(self.input_path, True)
|
||||
|
||||
|
||||
class ActivateImageTool(bpy.types.Operator):
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
import bpy
|
||||
|
||||
from ..common.image_edit import draw_image_mask_settings
|
||||
from ..common.image_interaction import draw_region_settings
|
||||
from .edit_mask import EditImageMask
|
||||
|
||||
|
||||
@@ -21,4 +21,4 @@ class ImageLassoTool(bpy.types.WorkSpaceTool):
|
||||
)
|
||||
|
||||
def draw_settings(context, layout, _tool):
|
||||
draw_image_mask_settings(context, layout, "image_lasso_refine_selection")
|
||||
draw_region_settings(context, layout, "image_lasso_refine_selection")
|
||||
|
||||
@@ -6,24 +6,24 @@ import bpy
|
||||
from ..common.image_interaction import (
|
||||
deserialize_matrix,
|
||||
drawing_in_region,
|
||||
hidden_string_property,
|
||||
screen_path_to_image_pixels,
|
||||
serialize_matrix,
|
||||
)
|
||||
from ..common.image import (
|
||||
from ..common.image_region import crop_pixels_to_alpha
|
||||
from ..common.image_data import (
|
||||
create_pixel_result_image,
|
||||
image_base_name,
|
||||
is_image_empty,
|
||||
plane_bounds,
|
||||
image_pixels,
|
||||
image_empty_bounds,
|
||||
replace_empty_image,
|
||||
require_image_empty,
|
||||
video_total_frames,
|
||||
)
|
||||
from ..common.image_edit import (
|
||||
from ..common.image_interaction import (
|
||||
active_view3d_tool_id,
|
||||
crop_pixels_to_alpha,
|
||||
draw_dashed_line,
|
||||
draw_polygon_fill,
|
||||
image_edit_poll,
|
||||
polygon_bounds,
|
||||
replace_empty_image_pixels,
|
||||
report_image_edit_exception,
|
||||
resolve_image_edit_click,
|
||||
)
|
||||
@@ -69,7 +69,7 @@ def validate_perspective_quad(points, min_area=1e-6):
|
||||
)
|
||||
)
|
||||
if area < min_area:
|
||||
raise ValueError("Draw a larger perspective region")
|
||||
raise ValueError("Draw a larger perspective quad")
|
||||
return quad
|
||||
|
||||
|
||||
@@ -313,7 +313,7 @@ def homography_from_points(source, target):
|
||||
values.extend((u, v))
|
||||
matrix = np.asarray(rows, dtype=np.float64)
|
||||
if np.linalg.cond(matrix) > 1e12:
|
||||
raise ValueError("The perspective region is too degenerate to transform")
|
||||
raise ValueError("The perspective quad is too degenerate to transform")
|
||||
coefficients = np.linalg.solve(matrix, np.asarray(values, dtype=np.float64))
|
||||
return np.append(coefficients, 1.0).reshape((3, 3))
|
||||
|
||||
@@ -438,9 +438,13 @@ def perspective_quad_aspect(quad):
|
||||
)
|
||||
|
||||
|
||||
def perspective_placement_bounds(polygon, output_size, crop_bounds):
|
||||
def perspective_placement_bounds(quad, output_size, crop_bounds):
|
||||
"""Map a rectified Alpha crop back to an approximate source-image rectangle."""
|
||||
source_left, source_top, source_right, source_bottom = polygon_bounds(polygon)
|
||||
quad = validate_perspective_quad(quad)
|
||||
source_left = math.floor(float(quad[:, 0].min()))
|
||||
source_top = math.floor(float(quad[:, 1].min()))
|
||||
source_right = math.ceil(float(quad[:, 0].max()))
|
||||
source_bottom = math.ceil(float(quad[:, 1].max()))
|
||||
output_width, output_height = (float(value) for value in output_size)
|
||||
crop_left, crop_top, crop_right, crop_bottom = crop_bounds
|
||||
source_width = source_right - source_left
|
||||
@@ -456,12 +460,12 @@ def perspective_placement_bounds(polygon, output_size, crop_bounds):
|
||||
def extract_perspective_pixels(
|
||||
source_pixels,
|
||||
source_size,
|
||||
polygon,
|
||||
quad,
|
||||
aspect_ratio,
|
||||
):
|
||||
output_size = perspective_output_size(polygon, aspect_ratio)
|
||||
output_size = perspective_output_size(quad, aspect_ratio)
|
||||
pixels = warp_perspective_pixels(
|
||||
source_pixels, source_size, polygon, output_size
|
||||
source_pixels, source_size, quad, output_size
|
||||
)
|
||||
pixels, crop_bounds = crop_pixels_to_alpha(pixels, (0, 0, *output_size))
|
||||
cropped_size = (
|
||||
@@ -469,34 +473,20 @@ def extract_perspective_pixels(
|
||||
crop_bounds[3] - crop_bounds[1],
|
||||
)
|
||||
placement_bounds = perspective_placement_bounds(
|
||||
polygon, output_size, crop_bounds
|
||||
quad, output_size, crop_bounds
|
||||
)
|
||||
return pixels, cropped_size, placement_bounds
|
||||
|
||||
|
||||
def create_perspective_result(
|
||||
source_object,
|
||||
pixels,
|
||||
output_size,
|
||||
placement_bounds,
|
||||
):
|
||||
return replace_empty_image_pixels(
|
||||
source_object,
|
||||
pixels,
|
||||
output_size,
|
||||
placement_bounds,
|
||||
)
|
||||
|
||||
|
||||
class EditImagePerspective(bpy.types.Operator):
|
||||
bl_idname = "anyimage.edit_image_perspective"
|
||||
bl_label = "Edit Image Perspective"
|
||||
bl_description = "Deform four image corners into a rectangular image"
|
||||
bl_options = {"REGISTER", "UNDO"}
|
||||
|
||||
source_object_name: hidden_string_property()
|
||||
source_matrix_data: hidden_string_property()
|
||||
selection_data: hidden_string_property()
|
||||
source_object_name: bpy.props.StringProperty(options={"HIDDEN", "SKIP_SAVE"})
|
||||
source_matrix_data: bpy.props.StringProperty(options={"HIDDEN", "SKIP_SAVE"})
|
||||
quad_json: bpy.props.StringProperty(options={"HIDDEN", "SKIP_SAVE"})
|
||||
aspect_ratio: bpy.props.FloatProperty(
|
||||
default=1.0,
|
||||
min=MIN_INTERACTIVE_ASPECT,
|
||||
@@ -632,7 +622,7 @@ class EditImagePerspective(bpy.types.Operator):
|
||||
return {"CANCELLED"}
|
||||
self.source_object_name = source_object.name
|
||||
self.source_matrix_data = serialize_matrix(source_object.matrix_world)
|
||||
self.selection_data = ""
|
||||
self.quad_json = ""
|
||||
self._phase = "POINTS"
|
||||
self._preview_image = None
|
||||
self._source_pixels = None
|
||||
@@ -704,36 +694,37 @@ class EditImagePerspective(bpy.types.Operator):
|
||||
self._set_point_status(context)
|
||||
context.area.tag_redraw()
|
||||
return {"RUNNING_MODAL"}
|
||||
source_object = bpy.data.objects.get(self.source_object_name)
|
||||
if not is_image_empty(source_object):
|
||||
self.report({"ERROR"}, "The source Image Empty is no longer available")
|
||||
try:
|
||||
source_object = require_image_empty(self.source_object_name)
|
||||
except RuntimeError as error:
|
||||
self.report({"ERROR"}, str(error))
|
||||
self._finish(context)
|
||||
return {"CANCELLED"}
|
||||
try:
|
||||
canonical_perspective_quad(self._points, MIN_QUAD_AREA)
|
||||
matrix_world = deserialize_matrix(self.source_matrix_data)
|
||||
polygon = screen_path_to_image_pixels(
|
||||
quad = screen_path_to_image_pixels(
|
||||
self._points,
|
||||
context.region,
|
||||
context.region_data,
|
||||
matrix_world,
|
||||
plane_bounds(source_object),
|
||||
image_empty_bounds(source_object),
|
||||
tuple(source_object.data.size),
|
||||
)
|
||||
polygon = canonical_perspective_quad(polygon)
|
||||
quad = canonical_perspective_quad(quad)
|
||||
if not perspective_quad_overlaps_image(
|
||||
polygon, tuple(source_object.data.size)
|
||||
quad, tuple(source_object.data.size)
|
||||
):
|
||||
self.report(
|
||||
{"WARNING"},
|
||||
"Perspective region does not overlap the source image",
|
||||
"Perspective quad does not overlap the source image",
|
||||
)
|
||||
self._finish(context)
|
||||
return {"CANCELLED"}
|
||||
self.selection_data = json.dumps(
|
||||
polygon.tolist(), separators=(",", ":")
|
||||
self.quad_json = json.dumps(
|
||||
quad.tolist(), separators=(",", ":")
|
||||
)
|
||||
self.aspect_ratio = perspective_quad_aspect(polygon)
|
||||
self.aspect_ratio = perspective_quad_aspect(quad)
|
||||
self._initial_aspect_ratio = self.aspect_ratio
|
||||
self._raw_aspect_ratio = self.aspect_ratio
|
||||
self._aspect_preset_label = None
|
||||
@@ -751,14 +742,11 @@ class EditImagePerspective(bpy.types.Operator):
|
||||
self._aspect_anchor_x = center_x
|
||||
import numpy as np
|
||||
|
||||
self._source_pixels = np.empty(
|
||||
len(source_object.data.pixels), dtype=np.float32
|
||||
)
|
||||
source_object.data.pixels.foreach_get(self._source_pixels)
|
||||
self._source_pixels = image_pixels(source_object.data)
|
||||
preview_pixels = warp_perspective_pixels(
|
||||
self._source_pixels,
|
||||
tuple(source_object.data.size),
|
||||
polygon,
|
||||
quad,
|
||||
(PREVIEW_TEXTURE_SIZE, PREVIEW_TEXTURE_SIZE),
|
||||
)
|
||||
if not np.any(
|
||||
@@ -766,7 +754,7 @@ class EditImagePerspective(bpy.types.Operator):
|
||||
):
|
||||
self.report(
|
||||
{"WARNING"},
|
||||
"Perspective region contains no visible image pixels",
|
||||
"Perspective quad contains no visible image pixels",
|
||||
)
|
||||
self._finish(context)
|
||||
return {"CANCELLED"}
|
||||
@@ -785,20 +773,21 @@ class EditImagePerspective(bpy.types.Operator):
|
||||
return {"PASS_THROUGH"}
|
||||
|
||||
def execute(self, context):
|
||||
source_object = bpy.data.objects.get(self.source_object_name)
|
||||
if not is_image_empty(source_object):
|
||||
self.report({"ERROR"}, "The source Image Empty is no longer available")
|
||||
try:
|
||||
source_object = require_image_empty(self.source_object_name)
|
||||
except RuntimeError as error:
|
||||
self.report({"ERROR"}, str(error))
|
||||
return {"CANCELLED"}
|
||||
try:
|
||||
if video_total_frames(source_object.data) > 1:
|
||||
raise RuntimeError("Image Perspective only supports a still image")
|
||||
polygon = validate_perspective_quad(json.loads(self.selection_data))
|
||||
quad = validate_perspective_quad(json.loads(self.quad_json))
|
||||
if not perspective_quad_overlaps_image(
|
||||
polygon, tuple(source_object.data.size)
|
||||
quad, tuple(source_object.data.size)
|
||||
):
|
||||
self.report(
|
||||
{"WARNING"},
|
||||
"Perspective region does not overlap the source image",
|
||||
"Perspective quad does not overlap the source image",
|
||||
)
|
||||
return {"CANCELLED"}
|
||||
import numpy as np
|
||||
@@ -807,21 +796,22 @@ class EditImagePerspective(bpy.types.Operator):
|
||||
if source_pixels is None or len(source_pixels) != len(
|
||||
source_object.data.pixels
|
||||
):
|
||||
source_pixels = np.empty(
|
||||
len(source_object.data.pixels), dtype=np.float32
|
||||
)
|
||||
source_object.data.pixels.foreach_get(source_pixels)
|
||||
source_pixels = image_pixels(source_object.data)
|
||||
pixels, output_size, placement_bounds = extract_perspective_pixels(
|
||||
source_pixels,
|
||||
tuple(source_object.data.size),
|
||||
polygon,
|
||||
quad,
|
||||
self.aspect_ratio,
|
||||
)
|
||||
create_perspective_result(
|
||||
source_object,
|
||||
result_image = create_pixel_result_image(
|
||||
source_object.data,
|
||||
pixels,
|
||||
output_size,
|
||||
placement_bounds,
|
||||
)
|
||||
replace_empty_image(
|
||||
source_object,
|
||||
result_image,
|
||||
placement_bounds=placement_bounds,
|
||||
)
|
||||
except (
|
||||
json.JSONDecodeError,
|
||||
|
||||
@@ -2,27 +2,27 @@ import bpy
|
||||
|
||||
from ..properties import BEN2_MODEL_KEY
|
||||
from ..runtime import JobOperatorBase
|
||||
from .common.operator import (
|
||||
HIDDEN,
|
||||
background_input_path,
|
||||
cleanup_temporary_input,
|
||||
from .common.ai_job import (
|
||||
require_background_input_path,
|
||||
invoke_ai_setup_if_needed,
|
||||
production_device,
|
||||
push_undo,
|
||||
require_environment,
|
||||
require_model,
|
||||
source_start_frame,
|
||||
)
|
||||
|
||||
from .common.image import (
|
||||
from .common.image_data import (
|
||||
cleanup_image_input,
|
||||
default_extract_frames,
|
||||
is_image_empty,
|
||||
prepare_image_input,
|
||||
remove_temporary_input,
|
||||
source_start_frame,
|
||||
video_total_frames,
|
||||
)
|
||||
|
||||
|
||||
_HIDDEN = {"HIDDEN", "SKIP_SAVE"}
|
||||
|
||||
|
||||
class RemoveImageBackground(bpy.types.Operator):
|
||||
bl_idname = "anyimage.remove_image_background"
|
||||
bl_label = "Remove Background"
|
||||
@@ -65,29 +65,22 @@ class RemoveImageBackground(bpy.types.Operator):
|
||||
if "CANCELLED" not in result:
|
||||
return {"FINISHED"}
|
||||
if delete_input:
|
||||
remove_temporary_input(source_path)
|
||||
cleanup_image_input(source_path, True)
|
||||
return {"CANCELLED"}
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
def apply_background_response(result, source_object_name):
|
||||
from .common.image import create_color_image, is_image_empty
|
||||
from .common.image_data import (
|
||||
load_color_result_image,
|
||||
replace_empty_image,
|
||||
require_image_empty,
|
||||
)
|
||||
|
||||
source_object = bpy.data.objects.get(source_object_name)
|
||||
if not is_image_empty(source_object):
|
||||
raise RuntimeError("The source image empty is no longer available")
|
||||
source_image = source_object.data
|
||||
source_name = source_image.name
|
||||
original_fake_user = source_image.use_fake_user
|
||||
result_image = create_color_image(source_object, result.directory)
|
||||
source_image.use_fake_user = original_fake_user
|
||||
if not original_fake_user and int(source_image.users) == 0:
|
||||
bpy.data.images.remove(source_image)
|
||||
result_image.name = source_name
|
||||
push_undo("Remove Background")
|
||||
source_object = require_image_empty(source_object_name)
|
||||
result_image = load_color_result_image(source_object, result.directory)
|
||||
replace_empty_image(source_object, result_image)
|
||||
if "FINISHED" not in bpy.ops.ed.undo_push(message="Remove Background"):
|
||||
raise RuntimeError("Unable to create the Remove Background Undo step")
|
||||
|
||||
|
||||
class RunBackgroundRemoval(JobOperatorBase, bpy.types.Operator):
|
||||
@@ -96,16 +89,16 @@ class RunBackgroundRemoval(JobOperatorBase, bpy.types.Operator):
|
||||
job_type = "remove-background"
|
||||
starting_message = "Starting Image Empty background removal"
|
||||
|
||||
input_path: bpy.props.StringProperty(options=HIDDEN)
|
||||
source_object_name: bpy.props.StringProperty(options=HIDDEN)
|
||||
delete_input: bpy.props.BoolProperty(default=False, options=HIDDEN)
|
||||
input_path: bpy.props.StringProperty(options=_HIDDEN)
|
||||
source_object_name: bpy.props.StringProperty(options=_HIDDEN)
|
||||
delete_input: bpy.props.BoolProperty(default=False, options=_HIDDEN)
|
||||
|
||||
def request(self, context):
|
||||
require_environment()
|
||||
require_model(BEN2_MODEL_KEY)
|
||||
settings = context.scene.anyimage_settings
|
||||
return {
|
||||
"input": str(background_input_path(self.input_path)),
|
||||
"input": str(require_background_input_path(self.input_path)),
|
||||
"model": BEN2_MODEL_KEY,
|
||||
"device": production_device().lower(),
|
||||
"video_frames": settings.video_frames,
|
||||
@@ -117,4 +110,4 @@ class RunBackgroundRemoval(JobOperatorBase, bpy.types.Operator):
|
||||
return "Image background is removed"
|
||||
|
||||
def cleanup(self):
|
||||
cleanup_temporary_input(self.input_path, self.delete_input)
|
||||
cleanup_image_input(self.input_path, self.delete_input)
|
||||
|
||||
@@ -8,25 +8,43 @@ from ..server.media.resolution import (
|
||||
MIN_MAX_AI_INPUT_SIZE,
|
||||
)
|
||||
from ..runtime import JobOperatorBase
|
||||
from .common.operator import (
|
||||
HIDDEN,
|
||||
cleanup_temporary_input,
|
||||
input_path,
|
||||
from .common.ai_job import (
|
||||
require_input_path,
|
||||
invoke_ai_setup_if_needed,
|
||||
production_device,
|
||||
require_environment,
|
||||
require_model,
|
||||
source_start_frame,
|
||||
)
|
||||
from .common.image import (
|
||||
create_color_image,
|
||||
from .common.image_data import (
|
||||
cleanup_image_input,
|
||||
default_extract_frames,
|
||||
image_base_name,
|
||||
is_image_empty,
|
||||
load_color_result_image,
|
||||
prepare_image_input,
|
||||
source_start_frame,
|
||||
replace_empty_image,
|
||||
require_image_empty,
|
||||
video_total_frames,
|
||||
)
|
||||
|
||||
_HIDDEN = {"HIDDEN", "SKIP_SAVE"}
|
||||
|
||||
UPSCALE_FACTOR = 4
|
||||
|
||||
|
||||
def upscale_output_size(image):
|
||||
width, height = (int(value) for value in getattr(image, "size", (0, 0)))
|
||||
if width <= 0 or height <= 0:
|
||||
return None
|
||||
return width * UPSCALE_FACTOR, height * UPSCALE_FACTOR
|
||||
|
||||
|
||||
def upscale_input_size_allowed(image, max_input_size):
|
||||
width, height = (int(value) for value in getattr(image, "size", (0, 0)))
|
||||
if width <= 0 or height <= 0:
|
||||
return True
|
||||
return max(width, height) < int(max_input_size)
|
||||
|
||||
|
||||
class UpscaleImage(JobOperatorBase, bpy.types.Operator):
|
||||
bl_idname = "anyimage.upscale_image"
|
||||
@@ -38,7 +56,7 @@ class UpscaleImage(JobOperatorBase, bpy.types.Operator):
|
||||
|
||||
model: bpy.props.StringProperty(
|
||||
default="REALESRGAN_GENERAL_WDN_X4V3",
|
||||
options=HIDDEN,
|
||||
options=_HIDDEN,
|
||||
)
|
||||
max_input_size: bpy.props.IntProperty(
|
||||
name="Max AI Input Size",
|
||||
@@ -48,18 +66,26 @@ class UpscaleImage(JobOperatorBase, bpy.types.Operator):
|
||||
max=MAX_MAX_AI_INPUT_SIZE,
|
||||
step=256,
|
||||
subtype="PIXEL",
|
||||
options=HIDDEN,
|
||||
options=_HIDDEN,
|
||||
)
|
||||
input_path: bpy.props.StringProperty(options=HIDDEN)
|
||||
source_object_name: bpy.props.StringProperty(options=HIDDEN)
|
||||
source_image_name: bpy.props.StringProperty(options=HIDDEN)
|
||||
delete_input: bpy.props.BoolProperty(default=False, options=HIDDEN)
|
||||
input_path: bpy.props.StringProperty(options=_HIDDEN)
|
||||
source_object_name: bpy.props.StringProperty(options=_HIDDEN)
|
||||
source_image_name: bpy.props.StringProperty(options=_HIDDEN)
|
||||
delete_input: bpy.props.BoolProperty(default=False, options=_HIDDEN)
|
||||
|
||||
@classmethod
|
||||
def poll(cls, context):
|
||||
from ..runtime import runtime
|
||||
|
||||
return is_image_empty(context.object) and not runtime.server_busy()
|
||||
source_object = getattr(context, "object", None)
|
||||
return (
|
||||
is_image_empty(source_object)
|
||||
and upscale_input_size_allowed(
|
||||
source_object.data,
|
||||
configured_max_ai_input_size(context),
|
||||
)
|
||||
and not runtime.server_busy()
|
||||
)
|
||||
|
||||
def invoke(self, context, _event):
|
||||
setup_result = invoke_ai_setup_if_needed()
|
||||
@@ -74,23 +100,15 @@ class UpscaleImage(JobOperatorBase, bpy.types.Operator):
|
||||
if not is_image_empty(source_object):
|
||||
self.report({"ERROR"}, "Select an Image Empty to upscale")
|
||||
return {"CANCELLED"}
|
||||
current_image = source_object.data
|
||||
try:
|
||||
original_image = getattr(
|
||||
current_image,
|
||||
"anyimage_upscale_source",
|
||||
None,
|
||||
)
|
||||
except ReferenceError:
|
||||
original_image = None
|
||||
source_image = original_image or current_image
|
||||
self.source_image_name = source_image.name
|
||||
self._rebuild_from_original = source_image is not current_image
|
||||
if self._rebuild_from_original:
|
||||
source_image = source_object.data
|
||||
if not upscale_input_size_allowed(source_image, self.max_input_size):
|
||||
self.report(
|
||||
{"WARNING"},
|
||||
"This image was already upscaled; rebuilding from the original",
|
||||
{"ERROR"},
|
||||
"Image long edge must be smaller than Maximum AI Input Size "
|
||||
f"({self.max_input_size} px)",
|
||||
)
|
||||
return {"CANCELLED"}
|
||||
self.source_image_name = source_image.name
|
||||
if video_total_frames(source_image) > 1:
|
||||
context.scene.anyimage_settings.video_frames = (
|
||||
default_extract_frames(source_object)
|
||||
@@ -109,7 +127,7 @@ class UpscaleImage(JobOperatorBase, bpy.types.Operator):
|
||||
self.delete_input = delete_input
|
||||
result = super().execute(context)
|
||||
if "CANCELLED" in result:
|
||||
cleanup_temporary_input(self.input_path, self.delete_input)
|
||||
cleanup_image_input(self.input_path, self.delete_input)
|
||||
return result
|
||||
|
||||
def request(self, context):
|
||||
@@ -117,7 +135,7 @@ class UpscaleImage(JobOperatorBase, bpy.types.Operator):
|
||||
require_model(self.model)
|
||||
settings = context.scene.anyimage_settings
|
||||
return {
|
||||
"input": str(input_path(self.input_path)),
|
||||
"input": str(require_input_path(self.input_path)),
|
||||
"model": self.model,
|
||||
"device": production_device().lower(),
|
||||
"video_frames": settings.video_frames,
|
||||
@@ -126,28 +144,18 @@ class UpscaleImage(JobOperatorBase, bpy.types.Operator):
|
||||
}
|
||||
|
||||
def response(self, _context, result):
|
||||
source_object = bpy.data.objects.get(self.source_object_name)
|
||||
if not is_image_empty(source_object):
|
||||
raise RuntimeError("The source image empty is no longer available")
|
||||
source_object = require_image_empty(self.source_object_name)
|
||||
source_image = bpy.data.images.get(self.source_image_name)
|
||||
if source_image is None:
|
||||
raise RuntimeError("The original source image is no longer available")
|
||||
replaced_image = source_object.data
|
||||
result_image = create_color_image(
|
||||
raise RuntimeError("The source image is no longer available")
|
||||
result_image = load_color_result_image(
|
||||
source_object,
|
||||
result.directory,
|
||||
frame_prefix="upscale.",
|
||||
)
|
||||
if replaced_image is not source_image:
|
||||
replaced_image.use_fake_user = False
|
||||
if int(replaced_image.users) == 0:
|
||||
bpy.data.images.remove(replaced_image, do_unlink=True)
|
||||
result_image.name = f"{image_base_name(source_image)}_upscale.png"
|
||||
result_image.anyimage_upscale_source = source_image
|
||||
replace_empty_image(source_object, result_image)
|
||||
model_label = upscale_model_label(self.model)
|
||||
if getattr(self, "_rebuild_from_original", False):
|
||||
return f"Image is rebuilt from the original with {model_label}"
|
||||
return f"Image is upscaled with {model_label}"
|
||||
|
||||
def cleanup(self):
|
||||
cleanup_temporary_input(self.input_path, self.delete_input)
|
||||
cleanup_image_input(self.input_path, self.delete_input)
|
||||
|
||||
@@ -96,8 +96,8 @@ class AnyImagePreferences(bpy.types.AddonPreferences):
|
||||
max_ai_input_size: bpy.props.IntProperty(
|
||||
name="Maximum AI Input Size",
|
||||
description=(
|
||||
"Maximum long edge used by Upscale, Depth and Normal analysis; "
|
||||
"larger inputs are resized proportionally without changing color textures"
|
||||
"Maximum source long edge eligible for Upscale and maximum long edge "
|
||||
"used by Depth and Normal analysis"
|
||||
),
|
||||
default=DEFAULT_MAX_AI_INPUT_SIZE,
|
||||
min=MIN_MAX_AI_INPUT_SIZE,
|
||||
|
||||
@@ -232,11 +232,24 @@ def refine_selection_property():
|
||||
)
|
||||
|
||||
|
||||
def cutout_normal_analysis_property():
|
||||
return bpy.props.BoolProperty(
|
||||
def cutout_normal_mode_property():
|
||||
return bpy.props.EnumProperty(
|
||||
name="Normal",
|
||||
description="Use MoGe-2 to add a normal texture for the selected shape",
|
||||
default=False,
|
||||
description="Normal texture generated with MoGe-2",
|
||||
items=(
|
||||
("NONE", "None", "Do not generate a normal texture"),
|
||||
(
|
||||
"TANGENT",
|
||||
"Tangent",
|
||||
"Generate a normal texture relative to the image plane",
|
||||
),
|
||||
(
|
||||
"OBJECT",
|
||||
"Object",
|
||||
"Generate a normal texture in Cutout object-local axes",
|
||||
),
|
||||
),
|
||||
default="NONE",
|
||||
)
|
||||
|
||||
|
||||
@@ -255,7 +268,7 @@ def cutout_point_spacing_property():
|
||||
|
||||
class AnyImageSettings(bpy.types.PropertyGroup):
|
||||
cutout_point_spacing: cutout_point_spacing_property()
|
||||
cutout_normal_analysis: cutout_normal_analysis_property()
|
||||
cutout_normal_mode: cutout_normal_mode_property()
|
||||
cutout_refine_selection: refine_selection_property()
|
||||
image_box_refine_selection: refine_selection_property()
|
||||
image_lasso_refine_selection: refine_selection_property()
|
||||
|
||||
@@ -10,7 +10,8 @@ from .jobs.debug_predictions import (
|
||||
run_upscale as _debug_upscale,
|
||||
)
|
||||
from .jobs.refine_image_mask import run as _refine_image_mask
|
||||
from .jobs.prepare_geometry_data import run as _prepare_geometry_data
|
||||
from .jobs.generate_depth_plane import run as _generate_depth_plane
|
||||
from .jobs.generate_region_data import run as _generate_region_data
|
||||
from .jobs.upscale import run as _upscale_image
|
||||
from .model_manager import ModelManager
|
||||
|
||||
@@ -50,9 +51,14 @@ def upscale_image(context, parameters):
|
||||
return _upscale_image(context, parameters)
|
||||
|
||||
|
||||
@server.job("prepare-geometry-data")
|
||||
def prepare_geometry_data(context, parameters):
|
||||
return _prepare_geometry_data(context, parameters)
|
||||
@server.job("generate-depth-plane")
|
||||
def generate_depth_plane(context, parameters):
|
||||
return _generate_depth_plane(context, parameters)
|
||||
|
||||
|
||||
@server.job("generate-region-data")
|
||||
def generate_region_data(context, parameters):
|
||||
return _generate_region_data(context, parameters)
|
||||
|
||||
|
||||
@server.job("debug-da3")
|
||||
|
||||
181
src/anyimage/server/geometry/generate_depth.py
Normal file
181
src/anyimage/server/geometry/generate_depth.py
Normal file
@@ -0,0 +1,181 @@
|
||||
"""Generate model-derived depth and normal data for one raster image."""
|
||||
|
||||
from ..media import DEFAULT_MAX_AI_INPUT_SIZE, limit_image
|
||||
from ..models import geometry_adapter, moge2
|
||||
from .normal_texture import write_camera_normal_texture
|
||||
|
||||
|
||||
def _limited_model_input(source_path, output_path, max_input_size):
|
||||
from PIL import Image
|
||||
|
||||
with Image.open(source_path) as opened:
|
||||
limited = limit_image(opened, max_input_size)
|
||||
if limited is opened:
|
||||
return source_path
|
||||
limited.save(output_path, format="PNG", compress_level=1)
|
||||
return output_path
|
||||
|
||||
|
||||
def _alpha_mask(image_path, shape):
|
||||
import numpy as np
|
||||
from PIL import Image
|
||||
|
||||
with Image.open(image_path) as opened:
|
||||
alpha = opened.convert("RGBA").getchannel("A")
|
||||
alpha = alpha.resize(
|
||||
(int(shape[1]), int(shape[0])),
|
||||
Image.Resampling.NEAREST,
|
||||
)
|
||||
return np.asarray(alpha, dtype=np.uint8) >= 26
|
||||
|
||||
|
||||
def transparent_fraction(image_path, alpha_threshold=26):
|
||||
import numpy as np
|
||||
from PIL import Image
|
||||
|
||||
with Image.open(image_path) as opened:
|
||||
alpha = np.asarray(
|
||||
opened.convert("RGBA").getchannel("A"),
|
||||
dtype=np.uint8,
|
||||
)
|
||||
return float(np.count_nonzero(alpha < alpha_threshold) / alpha.size)
|
||||
|
||||
|
||||
def _write_image_geometry_archive(
|
||||
frame,
|
||||
output_path,
|
||||
calibration_valid,
|
||||
region_transparent_fraction,
|
||||
):
|
||||
import numpy as np
|
||||
|
||||
depth = np.asarray(frame.depth, dtype=np.float32)
|
||||
points = np.asarray(frame.points, dtype=np.float32)
|
||||
intrinsics = np.asarray(frame.intrinsics, dtype=np.float32)
|
||||
valid = np.asarray(frame.valid_mask, dtype=bool)
|
||||
if depth.ndim != 2 or points.shape != (*depth.shape, 3):
|
||||
raise ValueError("The depth model returned an invalid point field")
|
||||
valid &= np.isfinite(depth) & (depth > 0.0)
|
||||
valid &= np.isfinite(points).all(axis=-1)
|
||||
if not valid.any():
|
||||
raise ValueError("The depth model returned no valid geometry")
|
||||
calibration_valid = np.asarray(calibration_valid, dtype=bool) & valid
|
||||
if not calibration_valid.any():
|
||||
raise ValueError("The depth field contains no valid Region pixels")
|
||||
reference_values = depth[calibration_valid]
|
||||
if frame.model_mask is not None:
|
||||
model_mask = np.asarray(frame.model_mask, dtype=bool)
|
||||
masked = depth[calibration_valid & model_mask]
|
||||
if len(masked):
|
||||
reference_values = masked
|
||||
np.savez_compressed(
|
||||
output_path,
|
||||
points=points,
|
||||
depth=depth,
|
||||
intrinsics=intrinsics,
|
||||
valid=valid,
|
||||
calibration_valid=calibration_valid,
|
||||
image_size=np.asarray((depth.shape[1], depth.shape[0]), dtype=np.int32),
|
||||
reference_depth=np.asarray(np.median(reference_values), dtype=np.float32),
|
||||
region_transparent_fraction=np.asarray(
|
||||
region_transparent_fraction,
|
||||
dtype=np.float32,
|
||||
),
|
||||
)
|
||||
|
||||
|
||||
def _write_normal(
|
||||
prediction,
|
||||
color_path,
|
||||
output_path,
|
||||
normal_mode,
|
||||
):
|
||||
import numpy as np
|
||||
from PIL import Image
|
||||
|
||||
normal = np.asarray(prediction.normal[0], dtype=np.float32)
|
||||
valid = _alpha_mask(color_path, normal.shape[:2])
|
||||
with Image.open(color_path) as source:
|
||||
output_size = source.size
|
||||
return write_camera_normal_texture(
|
||||
normal,
|
||||
valid,
|
||||
(0, 0),
|
||||
output_size,
|
||||
output_size,
|
||||
output_path,
|
||||
normal_mode=normal_mode,
|
||||
)
|
||||
|
||||
|
||||
def generate_depth(
|
||||
context,
|
||||
parameters,
|
||||
input_path,
|
||||
*,
|
||||
output_depth,
|
||||
normal_mode="NONE",
|
||||
):
|
||||
"""Generate requested AI data from an already prepared image Region."""
|
||||
normal_mode = str(normal_mode).upper()
|
||||
if normal_mode not in {"NONE", "TANGENT", "OBJECT"}:
|
||||
raise ValueError(f"Unsupported normal mode: {normal_mode}")
|
||||
model_path = _limited_model_input(
|
||||
input_path,
|
||||
context.directory / "model-input.png",
|
||||
parameters.get("max_input_size", DEFAULT_MAX_AI_INPUT_SIZE),
|
||||
)
|
||||
try:
|
||||
model_manager = context.resource("model_manager")
|
||||
adapter = geometry_adapter(parameters["model"])
|
||||
prediction = None
|
||||
if output_depth:
|
||||
context.progress(0.45, "Generating depth")
|
||||
if adapter is moge2:
|
||||
prediction = adapter.infer(
|
||||
model_manager,
|
||||
parameters,
|
||||
[model_path],
|
||||
include_points=True,
|
||||
cancel_check=context.check_cancelled,
|
||||
)
|
||||
else:
|
||||
prediction = adapter.infer(
|
||||
model_manager,
|
||||
parameters,
|
||||
[model_path],
|
||||
cancel_check=context.check_cancelled,
|
||||
)
|
||||
frame = adapter.geometry_frame(prediction)
|
||||
calibration_valid = _alpha_mask(input_path, frame.valid_mask.shape)
|
||||
archive_path = context.directory / "image_geometry.npz"
|
||||
_write_image_geometry_archive(
|
||||
frame,
|
||||
archive_path,
|
||||
calibration_valid,
|
||||
transparent_fraction(input_path),
|
||||
)
|
||||
else:
|
||||
archive_path = None
|
||||
|
||||
normal_path = None
|
||||
if normal_mode != "NONE":
|
||||
context.progress(0.72, "Generating surface normal")
|
||||
if prediction is None or adapter is not moge2:
|
||||
prediction = moge2.infer(
|
||||
model_manager,
|
||||
parameters,
|
||||
[model_path],
|
||||
include_points=False,
|
||||
cancel_check=context.check_cancelled,
|
||||
)
|
||||
normal_path = _write_normal(
|
||||
prediction,
|
||||
input_path,
|
||||
context.directory / "normal.png",
|
||||
normal_mode,
|
||||
)
|
||||
return archive_path, normal_path
|
||||
finally:
|
||||
if model_path != input_path:
|
||||
model_path.unlink(missing_ok=True)
|
||||
@@ -3,14 +3,14 @@
|
||||
from pathlib import Path
|
||||
|
||||
|
||||
def resized_selection_mask(selection_mask, shape):
|
||||
def resized_valid_mask(valid_mask, shape):
|
||||
import numpy as np
|
||||
from PIL import Image
|
||||
|
||||
selection_mask = np.asarray(selection_mask, dtype=bool)
|
||||
if selection_mask.shape == tuple(shape):
|
||||
return selection_mask
|
||||
resized = Image.fromarray(selection_mask.astype(np.uint8) * 255, mode="L")
|
||||
valid_mask = np.asarray(valid_mask, dtype=bool)
|
||||
if valid_mask.shape == tuple(shape):
|
||||
return valid_mask
|
||||
resized = Image.fromarray(valid_mask.astype(np.uint8) * 255, mode="L")
|
||||
resized = resized.resize(
|
||||
(int(shape[1]), int(shape[0])),
|
||||
Image.Resampling.NEAREST,
|
||||
@@ -25,8 +25,9 @@ def write_camera_normal_texture(
|
||||
crop_size,
|
||||
source_size,
|
||||
output_path,
|
||||
normal_mode="TANGENT",
|
||||
):
|
||||
"""Encode MoGe camera-axis normals into a full-image RGB texture."""
|
||||
"""Encode MoGe camera-axis normals in Blender tangent or object space."""
|
||||
import numpy as np
|
||||
from PIL import Image
|
||||
|
||||
@@ -35,14 +36,26 @@ def write_camera_normal_texture(
|
||||
if normal.ndim != 3 or normal.shape[2] != 3:
|
||||
raise ValueError("Depth Balloon normal data must have three components")
|
||||
if valid.shape != normal.shape[:2]:
|
||||
valid = resized_selection_mask(valid, normal.shape[:2])
|
||||
valid = resized_valid_mask(valid, normal.shape[:2])
|
||||
normal_mode = str(normal_mode).upper()
|
||||
if normal_mode == "TANGENT":
|
||||
converted = normal * np.asarray((1.0, -1.0, -1.0), dtype=np.float32)
|
||||
empty_normal = (128, 128, 255)
|
||||
elif normal_mode == "OBJECT":
|
||||
converted = normal[..., (2, 0, 1)] * np.asarray(
|
||||
(-1.0, -1.0, -1.0),
|
||||
dtype=np.float32,
|
||||
)
|
||||
empty_normal = (255, 128, 128)
|
||||
else:
|
||||
raise ValueError(f"Unsupported normal mode: {normal_mode}")
|
||||
encoded = np.clip(
|
||||
(normal * np.asarray((1.0, -1.0, -1.0), dtype=np.float32) + 1.0)
|
||||
(converted + 1.0)
|
||||
* 127.5,
|
||||
0.0,
|
||||
255.0,
|
||||
).astype(np.uint8)
|
||||
encoded[~valid] = (128, 128, 255)
|
||||
encoded[~valid] = empty_normal
|
||||
crop_width, crop_height = (int(value) for value in crop_size)
|
||||
crop = Image.fromarray(encoded, mode="RGB")
|
||||
if crop.size != (crop_width, crop_height):
|
||||
@@ -51,7 +64,7 @@ def write_camera_normal_texture(
|
||||
Image.Resampling.BILINEAR,
|
||||
)
|
||||
source_width, source_height = (int(value) for value in source_size)
|
||||
full = Image.new("RGB", (source_width, source_height), (128, 128, 255))
|
||||
full = Image.new("RGB", (source_width, source_height), empty_normal)
|
||||
full.paste(crop, tuple(int(value) for value in origin))
|
||||
output_path = Path(output_path)
|
||||
full.save(output_path)
|
||||
|
||||
22
src/anyimage/server/jobs/generate_depth_plane.py
Normal file
22
src/anyimage/server/jobs/generate_depth_plane.py
Normal file
@@ -0,0 +1,22 @@
|
||||
"""Generate depth data and preserve color for one complete image."""
|
||||
|
||||
from ..geometry.generate_depth import generate_depth
|
||||
from ..media import copy_color_images, input_frames
|
||||
|
||||
|
||||
def run(context, parameters):
|
||||
context.progress(0.05, "Preparing Depth Plane")
|
||||
context.directory.mkdir(parents=True, exist_ok=True)
|
||||
with input_frames(context, parameters) as images:
|
||||
if len(images) != 1:
|
||||
raise ValueError("Depth Plane currently supports one image")
|
||||
copy_color_images(images, context.directory)
|
||||
archive_path, _normal_path = generate_depth(
|
||||
context,
|
||||
parameters,
|
||||
images[0],
|
||||
output_depth=True,
|
||||
)
|
||||
context.check_cancelled()
|
||||
context.progress(1.0, "Depth Plane data is ready")
|
||||
return {"image_geometry": archive_path.name}
|
||||
51
src/anyimage/server/jobs/generate_region_data.py
Normal file
51
src/anyimage/server/jobs/generate_region_data.py
Normal file
@@ -0,0 +1,51 @@
|
||||
"""Generate optional AI data for one Blender-rasterized Cutout Region."""
|
||||
|
||||
import json
|
||||
from pathlib import Path
|
||||
|
||||
from ..geometry.generate_depth import generate_depth
|
||||
from ..region import generate_selection
|
||||
|
||||
|
||||
def run(context, parameters):
|
||||
context.progress(0.05, "Preparing Cutout Region")
|
||||
context.directory.mkdir(parents=True, exist_ok=True)
|
||||
input_path = Path(parameters["input"])
|
||||
region_path = input_path
|
||||
result = {}
|
||||
if parameters.get("refine_region"):
|
||||
context.progress(0.2, "Generating refined Region")
|
||||
region_path, bounds = generate_selection(
|
||||
context,
|
||||
input_path,
|
||||
parameters["refine_model"],
|
||||
parameters["device"],
|
||||
context.directory / "region.png",
|
||||
)
|
||||
bounds_path = context.directory / "region_bounds.json"
|
||||
bounds_path.write_text(
|
||||
json.dumps(bounds, separators=(",", ":")),
|
||||
encoding="utf-8",
|
||||
)
|
||||
result.update(
|
||||
region=region_path.name,
|
||||
region_bounds=bounds_path.name,
|
||||
)
|
||||
|
||||
output_depth = bool(parameters.get("output_depth"))
|
||||
normal_mode = str(parameters.get("normal_mode", "NONE"))
|
||||
if output_depth or normal_mode != "NONE":
|
||||
archive_path, normal_path = generate_depth(
|
||||
context,
|
||||
parameters,
|
||||
region_path,
|
||||
output_depth=output_depth,
|
||||
normal_mode=normal_mode,
|
||||
)
|
||||
if archive_path is not None:
|
||||
result["image_geometry"] = archive_path.name
|
||||
if normal_path is not None:
|
||||
result["normal"] = normal_path.name
|
||||
context.check_cancelled()
|
||||
context.progress(1.0, "Cutout Region data is ready")
|
||||
return result
|
||||
@@ -1,255 +0,0 @@
|
||||
from ..geometry.normal_texture import write_camera_normal_texture
|
||||
from ..media import (
|
||||
DEFAULT_MAX_AI_INPUT_SIZE,
|
||||
analysis_frames,
|
||||
copy_color_images,
|
||||
input_frames,
|
||||
limit_image,
|
||||
)
|
||||
from ..models import geometry_adapter, moge2
|
||||
from ..selection.cutout_images import refine_selection, write_images, write_selection
|
||||
|
||||
|
||||
def _limited_model_input(source_path, output_path, max_input_size):
|
||||
from PIL import Image
|
||||
|
||||
with Image.open(source_path) as opened:
|
||||
limited = limit_image(opened, max_input_size)
|
||||
if limited is opened:
|
||||
return source_path
|
||||
limited.save(output_path, format="PNG", compress_level=1)
|
||||
return output_path
|
||||
|
||||
|
||||
def _write_field(frame, output_path, calibration_valid=None):
|
||||
import numpy as np
|
||||
|
||||
depth = np.asarray(frame.depth, dtype=np.float32)
|
||||
points = np.asarray(frame.points, dtype=np.float32)
|
||||
intrinsics = np.asarray(frame.intrinsics, dtype=np.float32)
|
||||
valid = np.asarray(frame.valid_mask, dtype=bool)
|
||||
if depth.ndim != 2 or points.shape != (*depth.shape, 3):
|
||||
raise ValueError("The depth model returned an invalid point field")
|
||||
valid &= np.isfinite(depth) & (depth > 0.0)
|
||||
valid &= np.isfinite(points).all(axis=-1)
|
||||
if not valid.any():
|
||||
raise ValueError("The depth model returned no valid geometry")
|
||||
if calibration_valid is None:
|
||||
calibration_valid = valid.copy()
|
||||
else:
|
||||
calibration_valid = np.asarray(calibration_valid, dtype=bool)
|
||||
if calibration_valid.shape != valid.shape:
|
||||
raise ValueError("The depth calibration mask dimensions do not match")
|
||||
calibration_valid &= valid
|
||||
if not calibration_valid.any():
|
||||
raise ValueError("The depth field contains no valid calibration pixels")
|
||||
reference_values = depth[calibration_valid]
|
||||
if frame.model_mask is not None:
|
||||
model_mask = np.asarray(frame.model_mask, dtype=bool)
|
||||
masked = depth[calibration_valid & model_mask]
|
||||
if len(masked):
|
||||
reference_values = masked
|
||||
np.savez_compressed(
|
||||
output_path,
|
||||
points=points,
|
||||
depth=depth,
|
||||
intrinsics=intrinsics,
|
||||
valid=valid,
|
||||
calibration_valid=calibration_valid,
|
||||
image_size=np.asarray((depth.shape[1], depth.shape[0]), dtype=np.int32),
|
||||
reference_depth=np.asarray(np.median(reference_values), dtype=np.float32),
|
||||
)
|
||||
|
||||
|
||||
def _selection_calibration_mask(frame, selection_path):
|
||||
import numpy as np
|
||||
from PIL import Image
|
||||
|
||||
valid = np.asarray(frame.valid_mask, dtype=bool)
|
||||
with Image.open(selection_path) as opened:
|
||||
selection = opened.convert("RGBA").getchannel("A")
|
||||
selection = selection.resize(
|
||||
(valid.shape[1], valid.shape[0]),
|
||||
Image.Resampling.NEAREST,
|
||||
)
|
||||
selected = np.asarray(selection, dtype=np.uint8) >= 26
|
||||
calibration_valid = valid & selected
|
||||
if not calibration_valid.any():
|
||||
raise ValueError("The depth field contains no valid selected pixels")
|
||||
return calibration_valid
|
||||
|
||||
|
||||
def _write_camera_normal_from_prediction(prediction, color_path, output_path):
|
||||
import numpy as np
|
||||
from PIL import Image
|
||||
|
||||
normal = np.asarray(prediction.normal[0], dtype=np.float32)
|
||||
with Image.open(color_path) as source:
|
||||
alpha = source.convert("RGBA").getchannel("A")
|
||||
source_size = source.size
|
||||
valid = (
|
||||
np.asarray(
|
||||
alpha.resize((normal.shape[1], normal.shape[0]), Image.Resampling.BILINEAR)
|
||||
)
|
||||
> 127
|
||||
)
|
||||
return write_camera_normal_texture(
|
||||
normal,
|
||||
valid,
|
||||
(0, 0),
|
||||
source_size,
|
||||
source_size,
|
||||
output_path,
|
||||
)
|
||||
|
||||
|
||||
def _write_camera_normal(context, parameters, model_path, color_path=None):
|
||||
prediction = moge2.infer(
|
||||
context.resource("model_manager"),
|
||||
parameters,
|
||||
[model_path],
|
||||
include_points=False,
|
||||
cancel_check=context.check_cancelled,
|
||||
)
|
||||
return _write_camera_normal_from_prediction(
|
||||
prediction,
|
||||
color_path or model_path,
|
||||
context.directory / "normal.png",
|
||||
)
|
||||
|
||||
|
||||
def _write_depth_field(
|
||||
context,
|
||||
parameters,
|
||||
model_path,
|
||||
normal_color_path=None,
|
||||
include_normal=False,
|
||||
selection_path=None,
|
||||
):
|
||||
model_manager = context.resource("model_manager")
|
||||
adapter = geometry_adapter(parameters["model"])
|
||||
if adapter is moge2:
|
||||
prediction = adapter.infer(
|
||||
model_manager,
|
||||
parameters,
|
||||
[model_path],
|
||||
include_points=True,
|
||||
cancel_check=context.check_cancelled,
|
||||
)
|
||||
else:
|
||||
prediction = adapter.infer(
|
||||
model_manager,
|
||||
parameters,
|
||||
[model_path],
|
||||
cancel_check=context.check_cancelled,
|
||||
)
|
||||
context.check_cancelled()
|
||||
field_path = context.directory / "field.npz"
|
||||
frame = adapter.geometry_frame(prediction)
|
||||
calibration_valid = frame.valid_mask
|
||||
if selection_path is not None:
|
||||
calibration_valid = _selection_calibration_mask(frame, selection_path)
|
||||
_write_field(frame, field_path, calibration_valid)
|
||||
normal_path = None
|
||||
if include_normal and adapter is moge2:
|
||||
normal_path = _write_camera_normal_from_prediction(
|
||||
prediction,
|
||||
normal_color_path or model_path,
|
||||
context.directory / "normal.png",
|
||||
)
|
||||
return field_path, normal_path
|
||||
|
||||
|
||||
def run(context, parameters):
|
||||
context.progress(0.02, "Preparing geometry input")
|
||||
context.directory.mkdir(parents=True, exist_ok=True)
|
||||
with input_frames(context, parameters) as images:
|
||||
if len(images) != 1:
|
||||
raise ValueError("Empty Image to Mesh currently supports one image")
|
||||
shape = parameters["shape"]
|
||||
polygon = parameters.get("polygon")
|
||||
analysis_path = None
|
||||
selection_path = None
|
||||
result = {}
|
||||
if polygon:
|
||||
selection_bounds = None
|
||||
full_selection_path = context.directory / "selection-full.png"
|
||||
if parameters.get("refine_selection"):
|
||||
refine_selection(
|
||||
context,
|
||||
parameters,
|
||||
images[0],
|
||||
polygon,
|
||||
full_selection_path,
|
||||
)
|
||||
else:
|
||||
_selection_path, selection_bounds = write_selection(
|
||||
images[0],
|
||||
polygon,
|
||||
full_selection_path,
|
||||
parameters.get("invert", False),
|
||||
)
|
||||
color_path, selection_path, analysis_path, bounds_path = (
|
||||
write_images(
|
||||
images[0],
|
||||
full_selection_path,
|
||||
context.directory,
|
||||
selection_bounds,
|
||||
)
|
||||
)
|
||||
result.update(
|
||||
selection=selection_path.name,
|
||||
bounds=bounds_path.name,
|
||||
)
|
||||
else:
|
||||
copy_color_images(images, context.directory)
|
||||
|
||||
normal_analysis = bool(parameters.get("normal_analysis"))
|
||||
with analysis_frames(context, parameters, images) as model_images:
|
||||
if len(model_images) != 1:
|
||||
raise ValueError("Empty Image to Mesh currently supports one image")
|
||||
model_path = analysis_path or model_images[0]
|
||||
limited_model_path = model_path
|
||||
if normal_analysis or shape in {
|
||||
"DEPTH_PLANE",
|
||||
"DEPTH_BALLOON",
|
||||
"DEPTH_SURFACE",
|
||||
}:
|
||||
limited_model_path = _limited_model_input(
|
||||
model_path,
|
||||
context.directory / "model-input.png",
|
||||
parameters.get("max_input_size", DEFAULT_MAX_AI_INPUT_SIZE),
|
||||
)
|
||||
if shape in {"CUTOUT", "BALLOON"} and normal_analysis:
|
||||
context.progress(0.4, "Estimating surface normal")
|
||||
result["normal"] = _write_camera_normal(
|
||||
context, parameters, limited_model_path, model_path
|
||||
).name
|
||||
elif shape in {"DEPTH_PLANE", "DEPTH_BALLOON", "DEPTH_SURFACE"}:
|
||||
context.progress(0.4, "Estimating image depth")
|
||||
field_path, normal_path = _write_depth_field(
|
||||
context,
|
||||
parameters,
|
||||
limited_model_path,
|
||||
normal_color_path=model_path,
|
||||
include_normal=normal_analysis,
|
||||
selection_path=selection_path,
|
||||
)
|
||||
result["field"] = field_path.name
|
||||
if (
|
||||
normal_analysis
|
||||
and normal_path is None
|
||||
and shape == "DEPTH_SURFACE"
|
||||
):
|
||||
normal_path = _write_camera_normal(
|
||||
context, parameters, limited_model_path, model_path
|
||||
)
|
||||
if normal_path is not None:
|
||||
result["normal"] = normal_path.name
|
||||
|
||||
if limited_model_path != model_path:
|
||||
limited_model_path.unlink(missing_ok=True)
|
||||
|
||||
context.check_cancelled()
|
||||
context.progress(1.0, "Geometry data is ready")
|
||||
return result
|
||||
@@ -1,12 +1,7 @@
|
||||
import json
|
||||
import time
|
||||
|
||||
from ..models.ben2 import infer_foreground
|
||||
from ..selection.polygon_mask import (
|
||||
apply_selection_mask,
|
||||
create_selection_canvas,
|
||||
crop_to_alpha,
|
||||
)
|
||||
from ..region import generate_selection
|
||||
|
||||
|
||||
def run(context, parameters):
|
||||
@@ -16,28 +11,14 @@ def run(context, parameters):
|
||||
context.check_cancelled()
|
||||
output_directory = context.directory
|
||||
output_directory.mkdir(parents=True, exist_ok=True)
|
||||
canvas, selection_mask, bounds = create_selection_canvas(
|
||||
parameters["input"], parameters["polygon"]
|
||||
)
|
||||
selection_path = output_directory / "selection-input.png"
|
||||
canvas.save(selection_path, format="PNG", compress_level=1)
|
||||
model_manager = context.resource("model_manager")
|
||||
context.progress(0.2, f"Loading BEN2 for {parameters['device'].upper()}")
|
||||
foreground, _load_ms, _inference_ms = infer_foreground(
|
||||
selection_path,
|
||||
model_manager.directory(parameters["model"]),
|
||||
output_path, bounds = generate_selection(
|
||||
context,
|
||||
parameters["input"],
|
||||
parameters["model"],
|
||||
parameters["device"],
|
||||
model_cache=model_manager,
|
||||
cancel_check=context.check_cancelled,
|
||||
preserve_input_alpha=True,
|
||||
output_directory / "image.png",
|
||||
)
|
||||
context.check_cancelled()
|
||||
context.progress(0.9, "Applying selection mask")
|
||||
output, bounds = crop_to_alpha(
|
||||
apply_selection_mask(foreground, selection_mask), bounds
|
||||
)
|
||||
output_path = output_directory / "image.png"
|
||||
output.save(output_path, format="PNG", compress_level=1)
|
||||
bounds_path = output_directory / "bounds.json"
|
||||
bounds_path.write_text(json.dumps(bounds, separators=(",", ":")), encoding="utf-8")
|
||||
elapsed_ms = (time.perf_counter() - started) * 1000.0
|
||||
|
||||
5
src/anyimage/server/region/__init__.py
Normal file
5
src/anyimage/server/region/__init__.py
Normal file
@@ -0,0 +1,5 @@
|
||||
"""AI generation for rasterized image Regions."""
|
||||
|
||||
from .generate_selection import generate_selection
|
||||
|
||||
__all__ = ("generate_selection",)
|
||||
40
src/anyimage/server/region/generate_selection.py
Normal file
40
src/anyimage/server/region/generate_selection.py
Normal file
@@ -0,0 +1,40 @@
|
||||
from ..models.ben2 import infer_foreground
|
||||
|
||||
|
||||
def _crop_to_alpha(image):
|
||||
import numpy as np
|
||||
|
||||
pixels = np.asarray(image.convert("RGBA"), dtype=np.uint8)
|
||||
visible_y, visible_x = np.nonzero(pixels[:, :, 3] > 0)
|
||||
if not len(visible_x):
|
||||
raise ValueError("The generated Region contains no visible pixels")
|
||||
bounds = (
|
||||
int(visible_x.min()),
|
||||
int(visible_y.min()),
|
||||
int(visible_x.max()) + 1,
|
||||
int(visible_y.max()) + 1,
|
||||
)
|
||||
return image.crop(bounds), bounds
|
||||
|
||||
|
||||
def generate_selection(
|
||||
context,
|
||||
input_path,
|
||||
model,
|
||||
device,
|
||||
output_path,
|
||||
):
|
||||
"""Generate a refined selection from an already rasterized Region."""
|
||||
model_manager = context.resource("model_manager")
|
||||
foreground, _load_ms, _inference_ms = infer_foreground(
|
||||
input_path,
|
||||
model_manager.directory(model),
|
||||
device,
|
||||
model_cache=model_manager,
|
||||
cancel_check=context.check_cancelled,
|
||||
preserve_input_alpha=True,
|
||||
)
|
||||
context.check_cancelled()
|
||||
output, bounds = _crop_to_alpha(foreground)
|
||||
output.save(output_path, format="PNG", compress_level=1)
|
||||
return output_path, bounds
|
||||
@@ -1,22 +0,0 @@
|
||||
"""Image selection rasterization and Cutout image preparation."""
|
||||
|
||||
from .cutout_images import refine_selection, write_images, write_selection
|
||||
from .polygon_mask import (
|
||||
apply_selection_mask,
|
||||
create_selection_canvas,
|
||||
crop_to_alpha,
|
||||
polygon_bounds,
|
||||
polygon_selection_mask,
|
||||
)
|
||||
|
||||
|
||||
__all__ = (
|
||||
"apply_selection_mask",
|
||||
"create_selection_canvas",
|
||||
"crop_to_alpha",
|
||||
"polygon_bounds",
|
||||
"polygon_selection_mask",
|
||||
"refine_selection",
|
||||
"write_images",
|
||||
"write_selection",
|
||||
)
|
||||
@@ -1,162 +0,0 @@
|
||||
"""Prepare Cutout selection images inside the Job Server."""
|
||||
|
||||
import json
|
||||
|
||||
from ..models.ben2 import infer_foreground
|
||||
from .polygon_mask import (
|
||||
apply_selection_mask,
|
||||
create_selection_canvas,
|
||||
polygon_selection_mask,
|
||||
)
|
||||
|
||||
|
||||
def _write_masked_color(input_path, polygon, output_path, invert=False):
|
||||
import numpy as np
|
||||
from PIL import Image, ImageFilter, ImageOps
|
||||
|
||||
with Image.open(input_path) as opened:
|
||||
source = opened.convert("RGBA")
|
||||
selection = Image.fromarray(
|
||||
polygon_selection_mask(source.size, polygon, "NONZERO").astype(np.uint8) * 255,
|
||||
mode="L",
|
||||
).filter(ImageFilter.GaussianBlur(radius=0.75))
|
||||
if invert:
|
||||
selection = ImageOps.invert(selection)
|
||||
pixels = np.asarray(source, dtype=np.uint8).copy()
|
||||
source_alpha = pixels[:, :, 3].copy()
|
||||
pixels[:, :, 3] = np.asarray(selection, dtype=np.uint8)
|
||||
pixels[source_alpha < 26] = 0
|
||||
pixels[pixels[:, :, 3] == 0, :3] = 0
|
||||
Image.fromarray(pixels, mode="RGBA").save(
|
||||
output_path,
|
||||
format="PNG",
|
||||
compress_level=1,
|
||||
)
|
||||
return output_path
|
||||
|
||||
|
||||
def write_selection(input_path, polygon, output_path, invert=False):
|
||||
import numpy as np
|
||||
from PIL import Image, ImageFilter
|
||||
|
||||
if invert:
|
||||
_write_masked_color(input_path, polygon, output_path, invert=True)
|
||||
with Image.open(input_path) as source:
|
||||
return output_path, (0, 0, source.width, source.height)
|
||||
canvas, selection_mask, bounds = create_selection_canvas(
|
||||
input_path, polygon, "NONZERO"
|
||||
)
|
||||
selection = Image.fromarray(
|
||||
selection_mask.astype(np.uint8) * 255,
|
||||
mode="L",
|
||||
).filter(ImageFilter.GaussianBlur(radius=0.75))
|
||||
pixels = np.asarray(canvas, dtype=np.uint8).copy()
|
||||
source_alpha = pixels[:, :, 3].copy()
|
||||
pixels[:, :, 3] = np.asarray(selection, dtype=np.uint8)
|
||||
pixels[source_alpha < 26] = 0
|
||||
pixels[pixels[:, :, 3] == 0, :3] = 0
|
||||
Image.fromarray(pixels, mode="RGBA").save(
|
||||
output_path,
|
||||
format="PNG",
|
||||
compress_level=1,
|
||||
)
|
||||
return output_path, bounds
|
||||
|
||||
|
||||
def refine_selection(context, parameters, input_path, polygon, output_path):
|
||||
from PIL import Image, ImageChops
|
||||
|
||||
canvas, selection_mask, bounds = create_selection_canvas(
|
||||
input_path, polygon, "NONZERO"
|
||||
)
|
||||
selection_path = context.directory / "selection-input.png"
|
||||
canvas.save(selection_path, format="PNG", compress_level=1)
|
||||
model_manager = context.resource("model_manager")
|
||||
foreground, _load_ms, _inference_ms = infer_foreground(
|
||||
selection_path,
|
||||
model_manager.directory(parameters["refine_model"]),
|
||||
parameters["device"],
|
||||
model_cache=model_manager,
|
||||
cancel_check=context.check_cancelled,
|
||||
preserve_input_alpha=True,
|
||||
)
|
||||
refined = apply_selection_mask(foreground, selection_mask)
|
||||
with Image.open(input_path) as opened:
|
||||
source = opened.convert("RGBA")
|
||||
if parameters.get("invert"):
|
||||
mask = Image.new("L", source.size, 0)
|
||||
mask.paste(refined.getchannel("A"), bounds[:2])
|
||||
source.putalpha(
|
||||
ImageChops.multiply(source.getchannel("A"), ImageChops.invert(mask))
|
||||
)
|
||||
output = source
|
||||
else:
|
||||
output = Image.new("RGBA", source.size, (0, 0, 0, 0))
|
||||
output.paste(refined, bounds[:2])
|
||||
output.save(output_path, format="PNG", compress_level=1)
|
||||
return output_path
|
||||
|
||||
|
||||
def write_images(input_path, selection_path, output_directory, selection_bounds=None):
|
||||
import numpy as np
|
||||
from PIL import Image
|
||||
|
||||
with Image.open(input_path) as opened:
|
||||
source = opened.convert("RGBA")
|
||||
with Image.open(selection_path) as opened:
|
||||
selection = opened.convert("RGBA")
|
||||
if selection_bounds is None and selection.size != source.size:
|
||||
raise ValueError("The Cutout selection does not match the source image")
|
||||
if selection_bounds is None:
|
||||
selection_bounds = (0, 0, source.width, source.height)
|
||||
color_canvas = Image.new("RGBA", selection.size, (0, 0, 0, 0))
|
||||
color_canvas.paste(source, (-selection_bounds[0], -selection_bounds[1]))
|
||||
source_alpha = np.asarray(color_canvas, dtype=np.uint8)[:, :, 3]
|
||||
selection_pixels = np.asarray(selection, dtype=np.uint8).copy()
|
||||
alpha = selection_pixels[:, :, 3]
|
||||
alpha[alpha <= 1] = 0
|
||||
alpha[source_alpha < 26] = 0
|
||||
selection_pixels[alpha == 0, :3] = 0
|
||||
selection = Image.fromarray(selection_pixels, mode="RGBA")
|
||||
visible_y, visible_x = np.nonzero(alpha >= 26)
|
||||
if not len(visible_x):
|
||||
raise ValueError("The Cutout selection contains no visible pixels")
|
||||
content_bounds = (
|
||||
int(visible_x.min()),
|
||||
int(visible_y.min()),
|
||||
int(visible_x.max()) + 1,
|
||||
int(visible_y.max()) + 1,
|
||||
)
|
||||
selection_left, selection_top, _selection_right, _selection_bottom = (
|
||||
selection_bounds
|
||||
)
|
||||
bounds = (
|
||||
selection_left + content_bounds[0],
|
||||
selection_top + content_bounds[1],
|
||||
selection_left + content_bounds[2],
|
||||
selection_top + content_bounds[3],
|
||||
)
|
||||
color = color_canvas.crop(content_bounds)
|
||||
color_path = output_directory / "image.0001.png"
|
||||
color.save(color_path, format="PNG", compress_level=1)
|
||||
|
||||
selection_crop = selection.crop(content_bounds)
|
||||
selection_path = output_directory / "selection.png"
|
||||
selection_crop.save(selection_path, format="PNG", compress_level=1)
|
||||
|
||||
analysis_pixels = np.asarray(color, dtype=np.uint8).copy()
|
||||
mask_alpha = np.asarray(selection_crop, dtype=np.uint8)[:, :, 3]
|
||||
analysis_pixels[:, :, 3] = np.minimum(analysis_pixels[:, :, 3], mask_alpha)
|
||||
analysis_pixels[mask_alpha < 26] = 0
|
||||
analysis_path = output_directory / "analysis.png"
|
||||
Image.fromarray(analysis_pixels, mode="RGBA").save(
|
||||
analysis_path,
|
||||
format="PNG",
|
||||
compress_level=1,
|
||||
)
|
||||
bounds_path = output_directory / "bounds.json"
|
||||
bounds_path.write_text(
|
||||
json.dumps(bounds, separators=(",", ":")),
|
||||
encoding="utf-8",
|
||||
)
|
||||
return color_path, selection_path, analysis_path, bounds_path
|
||||
@@ -1,151 +0,0 @@
|
||||
import math
|
||||
|
||||
|
||||
def polygon_bounds(polygon):
|
||||
import numpy as np
|
||||
|
||||
points = np.asarray(polygon, dtype=np.float64)
|
||||
if points.ndim != 2 or points.shape[0] < 3 or points.shape[1] != 2:
|
||||
raise ValueError("The selection does not define a valid image region")
|
||||
if not np.isfinite(points).all():
|
||||
raise ValueError("The selection contains invalid image coordinates")
|
||||
left = int(math.floor(float(points[:, 0].min())))
|
||||
top = int(math.floor(float(points[:, 1].min())))
|
||||
right = int(math.ceil(float(points[:, 0].max())))
|
||||
bottom = int(math.ceil(float(points[:, 1].max())))
|
||||
if right <= left or bottom <= top:
|
||||
raise ValueError("The selection does not define a valid image region")
|
||||
return left, top, right, bottom
|
||||
|
||||
|
||||
def polygon_selection_mask(image_size, polygon, fill_rule="EVEN_ODD"):
|
||||
"""Rasterize a polygon with an explicit fill rule."""
|
||||
import numpy as np
|
||||
|
||||
width, height = (int(value) for value in image_size)
|
||||
points = np.asarray(polygon, dtype=np.float64)
|
||||
if width <= 0 or height <= 0:
|
||||
raise ValueError("The selection canvas has no readable dimensions")
|
||||
if points.ndim != 2 or points.shape[0] < 3 or points.shape[1] != 2:
|
||||
raise ValueError("The selection does not define a valid image region")
|
||||
if not np.isfinite(points).all():
|
||||
raise ValueError("The selection contains invalid image coordinates")
|
||||
if fill_rule not in {"EVEN_ODD", "NONZERO"}:
|
||||
raise ValueError(f"Unknown polygon fill rule: {fill_rule}")
|
||||
|
||||
scanlines = np.arange(height, dtype=np.float64) + 0.5
|
||||
crossings = [[] for _index in range(height)]
|
||||
for start, end in zip(points, np.roll(points, -1, axis=0)):
|
||||
selected = np.flatnonzero(
|
||||
(start[1] > scanlines) != (end[1] > scanlines)
|
||||
)
|
||||
if not len(selected):
|
||||
continue
|
||||
crossing_x = start[0] + (
|
||||
(scanlines[selected] - start[1])
|
||||
* (end[0] - start[0])
|
||||
/ (end[1] - start[1])
|
||||
)
|
||||
winding = 1 if end[1] > start[1] else -1
|
||||
for row, value in zip(selected, crossing_x):
|
||||
crossings[int(row)].append((float(value), winding))
|
||||
|
||||
mask = np.zeros((height, width), dtype=bool)
|
||||
for row, values in enumerate(crossings):
|
||||
values.sort()
|
||||
state = 0
|
||||
index = 0
|
||||
while index < len(values):
|
||||
crossing_x = values[index][0]
|
||||
winding = 0
|
||||
count = 0
|
||||
while index < len(values) and values[index][0] == crossing_x:
|
||||
winding += values[index][1]
|
||||
count += 1
|
||||
index += 1
|
||||
state = (
|
||||
state + winding
|
||||
if fill_rule == "NONZERO"
|
||||
else state ^ (count % 2)
|
||||
)
|
||||
if index >= len(values) or state == 0:
|
||||
continue
|
||||
following_x = values[index][0]
|
||||
first = max(0, int(math.ceil(crossing_x - 0.5)))
|
||||
last = min(width, int(math.floor(following_x - 0.5)) + 1)
|
||||
mask[row, first:last] = True
|
||||
return mask
|
||||
|
||||
def create_selection_canvas(input_path, polygon, fill_rule="EVEN_ODD"):
|
||||
"""Create an unbounded selection canvas with transparent black outside."""
|
||||
import numpy as np
|
||||
from PIL import Image
|
||||
|
||||
with Image.open(input_path) as opened_image:
|
||||
source = opened_image.convert("RGBA")
|
||||
left, top, right, bottom = polygon_bounds(polygon)
|
||||
if right <= 0 or bottom <= 0 or left >= source.width or top >= source.height:
|
||||
raise ValueError("The selection does not overlap the source image")
|
||||
canvas = Image.new("RGBA", (right - left, bottom - top), (0, 0, 0, 0))
|
||||
canvas.paste(source, (-left, -top))
|
||||
relative = [(float(x) - left, float(y) - top) for x, y in polygon]
|
||||
selection_mask = polygon_selection_mask(canvas.size, relative, fill_rule)
|
||||
pixels = np.asarray(canvas, dtype=np.uint8).copy()
|
||||
pixels[~selection_mask] = 0
|
||||
return Image.fromarray(pixels, mode="RGBA"), selection_mask, (
|
||||
left,
|
||||
top,
|
||||
right,
|
||||
bottom,
|
||||
)
|
||||
|
||||
|
||||
def apply_selection_mask(foreground, selection_mask):
|
||||
"""Clear pixels outside the source selection after background removal."""
|
||||
import numpy as np
|
||||
from PIL import Image
|
||||
|
||||
pixels = np.asarray(foreground.convert("RGBA"), dtype=np.uint8).copy()
|
||||
if pixels.shape[:2] != selection_mask.shape:
|
||||
pixels = np.asarray(
|
||||
foreground.convert("RGBA").resize(
|
||||
(selection_mask.shape[1], selection_mask.shape[0]),
|
||||
Image.Resampling.BILINEAR,
|
||||
),
|
||||
dtype=np.uint8,
|
||||
)
|
||||
pixels[~selection_mask] = 0
|
||||
pixels[pixels[:, :, 3] == 0, :3] = 0
|
||||
return Image.fromarray(pixels, mode="RGBA")
|
||||
|
||||
|
||||
def crop_to_alpha(image, bounds):
|
||||
"""Crop an RGBA mask image to its nonzero Alpha bounds."""
|
||||
import numpy as np
|
||||
from PIL import Image
|
||||
|
||||
pixels = np.asarray(image.convert("RGBA"), dtype=np.uint8)
|
||||
visible_y, visible_x = np.nonzero(pixels[:, :, 3] > 0)
|
||||
if not len(visible_x):
|
||||
raise ValueError("The selection contains no visible image pixels")
|
||||
content_left = int(visible_x.min())
|
||||
content_top = int(visible_y.min())
|
||||
content_right = int(visible_x.max()) + 1
|
||||
content_bottom = int(visible_y.max()) + 1
|
||||
output = Image.new(
|
||||
"RGBA",
|
||||
(
|
||||
content_right - content_left,
|
||||
content_bottom - content_top,
|
||||
),
|
||||
(0, 0, 0, 0),
|
||||
)
|
||||
output.paste(image, (-content_left, -content_top))
|
||||
left, top, _right, _bottom = bounds
|
||||
return output, (
|
||||
left + content_left,
|
||||
top + content_top,
|
||||
left + content_right,
|
||||
top + content_bottom,
|
||||
)
|
||||
|
||||
File diff suppressed because it is too large
Load Diff
@@ -8,8 +8,9 @@ import bpy
|
||||
SOURCE_ROOT = Path(__file__).parents[1] / "src"
|
||||
sys.path.insert(0, str(SOURCE_ROOT))
|
||||
|
||||
from anyimage.operators.common.image_gesture import ImageMaskGesture
|
||||
from anyimage.operators.cutout_tool import selection_tool as cutout
|
||||
from anyimage.operators.common.image_interaction import ImageRegionGesture
|
||||
from anyimage.operators.cutout_tool import select_region as cutout
|
||||
from anyimage.operators.common.image_region import RegionPath
|
||||
|
||||
|
||||
class Pie:
|
||||
@@ -31,10 +32,10 @@ class WindowManager:
|
||||
draw(menu, None)
|
||||
|
||||
|
||||
def cutout_context(window_manager, point_spacing=32, normal_analysis=False):
|
||||
def cutout_context(window_manager, point_spacing=32, normal_mode="NONE"):
|
||||
settings = SimpleNamespace(
|
||||
cutout_point_spacing=point_spacing,
|
||||
cutout_normal_analysis=normal_analysis,
|
||||
cutout_normal_mode=normal_mode,
|
||||
)
|
||||
return SimpleNamespace(
|
||||
window_manager=window_manager,
|
||||
@@ -51,10 +52,15 @@ def test_cutout_pie_uses_four_semantic_shapes_in_cardinal_order():
|
||||
context = cutout_context(window_manager)
|
||||
source = cutout_source()
|
||||
|
||||
cutout._open_shape_pie(context, object(), source, ((0, 0), (10, 0), (0, 10)))
|
||||
cutout._open_shape_pie(
|
||||
context,
|
||||
object(),
|
||||
source,
|
||||
RegionPath(points=((0, 0), (10, 0), (0, 10))),
|
||||
)
|
||||
|
||||
assert [call[2].shape for call in window_manager.pie.calls] == [
|
||||
"CUTOUT",
|
||||
"SURFACE",
|
||||
"BALLOON",
|
||||
"DEPTH_BALLOON",
|
||||
"DEPTH_SURFACE",
|
||||
@@ -66,18 +72,18 @@ def test_cutout_pie_uses_four_semantic_shapes_in_cardinal_order():
|
||||
"Depth Surface",
|
||||
]
|
||||
assert all(
|
||||
call[0] == "anyimage.convert_cutout_shape" for call in window_manager.pie.calls
|
||||
call[0] == "anyimage.cutout_region_to_shape"
|
||||
for call in window_manager.pie.calls
|
||||
)
|
||||
assert not any(call[2].refine_selection for call in window_manager.pie.calls)
|
||||
assert not any(call[2].normal_analysis for call in window_manager.pie.calls)
|
||||
assert not any(call[2].invert for call in window_manager.pie.calls)
|
||||
assert all(call[2].normal_mode == "NONE" for call in window_manager.pie.calls)
|
||||
assert all(call[2].mesh_detail == 64 for call in window_manager.pie.calls)
|
||||
|
||||
|
||||
def test_cutout_pie_passes_image_lasso_options_to_every_shape():
|
||||
window_manager = WindowManager()
|
||||
context = cutout_context(
|
||||
window_manager, point_spacing=16, normal_analysis=True
|
||||
window_manager, point_spacing=16, normal_mode="OBJECT"
|
||||
)
|
||||
source = cutout_source(size=(4096, 2048))
|
||||
|
||||
@@ -85,14 +91,19 @@ def test_cutout_pie_passes_image_lasso_options_to_every_shape():
|
||||
context,
|
||||
object(),
|
||||
source,
|
||||
((0, 0), (10, 0), (0, 10)),
|
||||
refine_selection=True,
|
||||
RegionPath(
|
||||
points=((0, 0), (10, 0), (0, 10)),
|
||||
invert=True,
|
||||
),
|
||||
refine_selection=True,
|
||||
)
|
||||
|
||||
assert all(call[2].refine_selection for call in window_manager.pie.calls)
|
||||
assert all(call[2].normal_analysis for call in window_manager.pie.calls)
|
||||
assert all(call[2].invert for call in window_manager.pie.calls)
|
||||
assert all(call[2].normal_mode == "OBJECT" for call in window_manager.pie.calls)
|
||||
assert all(
|
||||
RegionPath.from_json(call[2].region_path_json).invert
|
||||
for call in window_manager.pie.calls
|
||||
)
|
||||
assert all(call[2].mesh_detail == 256 for call in window_manager.pie.calls)
|
||||
|
||||
|
||||
@@ -116,8 +127,8 @@ def test_cutout_reuses_image_lasso_gesture_and_double_click_selects_all(monkeypa
|
||||
cutout, "resolve_image_edit_click", lambda *_args: (source, True)
|
||||
)
|
||||
monkeypatch.setattr(cutout, "video_total_frames", lambda _image: 1)
|
||||
monkeypatch.setattr(cutout, "apply_image_mask_settings", lambda *_args: None)
|
||||
monkeypatch.setattr(cutout, "selected_image_rgba", lambda *_args: object())
|
||||
monkeypatch.setattr(cutout, "apply_region_settings", lambda *_args: None)
|
||||
monkeypatch.setattr(cutout, "_validate_cutout_region", lambda *_args: None)
|
||||
|
||||
operator = SimpleNamespace(
|
||||
full_image=True,
|
||||
@@ -129,10 +140,12 @@ def test_cutout_reuses_image_lasso_gesture_and_double_click_selects_all(monkeypa
|
||||
operator, context, SimpleNamespace(value="DOUBLE_CLICK")
|
||||
)
|
||||
|
||||
assert issubclass(cutout.SelectCutoutRegion, ImageMaskGesture)
|
||||
assert issubclass(cutout.SelectCutoutRegion, ImageRegionGesture)
|
||||
assert result == {"FINISHED"}
|
||||
selection = json.loads(window_manager.pie.calls[0][2].selection_data)
|
||||
assert selection == [[0, 0], [320, 0], [320, 180], [0, 180]]
|
||||
region_path = RegionPath.from_json(
|
||||
window_manager.pie.calls[0][2].region_path_json
|
||||
)
|
||||
assert region_path.points == ((0, 0), (320, 0), (320, 180), (0, 180))
|
||||
assert window_manager.pie.calls[0][2].mesh_detail == 10
|
||||
assert not hasattr(cutout, "pending_selection")
|
||||
|
||||
|
||||
@@ -27,21 +27,31 @@ def load_image_modules():
|
||||
|
||||
module_paths = (
|
||||
(
|
||||
"anyimage.operators.common.image",
|
||||
PROJECT_ROOT / "src" / "anyimage" / "operators" / "common" / "image.py",
|
||||
),
|
||||
(
|
||||
"anyimage.operators.common.node_asset",
|
||||
"anyimage.operators.common.image_data",
|
||||
PROJECT_ROOT
|
||||
/ "src"
|
||||
/ "anyimage"
|
||||
/ "operators"
|
||||
/ "common"
|
||||
/ "node_asset.py",
|
||||
/ "image_data.py",
|
||||
),
|
||||
(
|
||||
"anyimage.operators.common.material",
|
||||
PROJECT_ROOT / "src" / "anyimage" / "operators" / "common" / "material.py",
|
||||
"anyimage.operators.common.mesh_result",
|
||||
PROJECT_ROOT
|
||||
/ "src"
|
||||
/ "anyimage"
|
||||
/ "operators"
|
||||
/ "common"
|
||||
/ "mesh_result.py",
|
||||
),
|
||||
(
|
||||
"anyimage.operators.convert_to_plane.resolution",
|
||||
PROJECT_ROOT
|
||||
/ "src"
|
||||
/ "anyimage"
|
||||
/ "operators"
|
||||
/ "convert_to_plane"
|
||||
/ "resolution.py",
|
||||
),
|
||||
)
|
||||
packages = {
|
||||
@@ -50,6 +60,7 @@ def load_image_modules():
|
||||
"anyimage",
|
||||
"anyimage.operators",
|
||||
"anyimage.operators.common",
|
||||
"anyimage.operators.convert_to_plane",
|
||||
)
|
||||
}
|
||||
for package in packages.values():
|
||||
@@ -452,21 +463,7 @@ class ImageTest(unittest.TestCase):
|
||||
"img_normal.png",
|
||||
)
|
||||
|
||||
def test_copy_depth_image_renames_still_source(self):
|
||||
class FakeImage:
|
||||
def __init__(self, name):
|
||||
self.name = name
|
||||
|
||||
def copy(self):
|
||||
return FakeImage("img.png.001")
|
||||
|
||||
source_object = SimpleNamespace(data=FakeImage("img.png"))
|
||||
|
||||
depth_image = self.image_depth.copy_depth_image(source_object)
|
||||
|
||||
self.assertEqual(depth_image.name, "img_depth.exr")
|
||||
|
||||
def test_color_and_material_names_use_source_image_base_name(self):
|
||||
def test_color_name_uses_source_image_base_name(self):
|
||||
source_object = SimpleNamespace(
|
||||
data=SimpleNamespace(name="img.png"),
|
||||
)
|
||||
@@ -475,19 +472,11 @@ class ImageTest(unittest.TestCase):
|
||||
self.image_depth.color_data_name(source_object),
|
||||
"img_color.png",
|
||||
)
|
||||
self.assertEqual(
|
||||
self.image_depth.material_data_name(source_object),
|
||||
"img",
|
||||
)
|
||||
source_object.data.name = "img.png.001"
|
||||
self.assertEqual(
|
||||
self.image_depth.color_data_name(source_object),
|
||||
"img.001_color.png",
|
||||
)
|
||||
self.assertEqual(
|
||||
self.image_depth.material_data_name(source_object),
|
||||
"img.001",
|
||||
)
|
||||
|
||||
def test_surface_displacement_is_set_to_displacement_only(self):
|
||||
class Material:
|
||||
@@ -501,14 +490,14 @@ class ImageTest(unittest.TestCase):
|
||||
"DISPLACEMENT",
|
||||
)
|
||||
|
||||
def test_plane_bounds_keep_image_aspect_and_empty_offset(self):
|
||||
def test_image_empty_bounds_keep_image_aspect_and_empty_offset(self):
|
||||
source = SimpleNamespace(
|
||||
data=SimpleNamespace(size=(800, 400)),
|
||||
empty_display_size=2.0,
|
||||
empty_image_offset=(-0.5, -0.5),
|
||||
)
|
||||
|
||||
bounds = self.image_depth.plane_bounds(source)
|
||||
bounds = self.image_depth.image_empty_bounds(source)
|
||||
|
||||
self.assertEqual(bounds, (-1.0, 1.0, -0.5, 0.5))
|
||||
|
||||
|
||||
@@ -1,24 +1,31 @@
|
||||
import builtins
|
||||
import sys
|
||||
import tempfile
|
||||
from pathlib import Path
|
||||
from types import SimpleNamespace
|
||||
from unittest.mock import patch
|
||||
|
||||
import numpy as np
|
||||
from PIL import Image
|
||||
|
||||
SOURCE_ROOT = Path(__file__).parents[1] / "src"
|
||||
sys.path.insert(0, str(SOURCE_ROOT))
|
||||
|
||||
from anyimage.operators.cutout_tool import balloon_profile as cutout_profile
|
||||
from anyimage.operators.cutout_tool import build_shapes as mesh
|
||||
from anyimage.operators.cutout_tool import depth_calibration as cutout_depth
|
||||
from anyimage.operators.cutout_tool import node_assets as cutout_nodes
|
||||
from anyimage.operators.cutout_tool import shape_types as shapes
|
||||
from anyimage.operators.cutout_tool.balloon_profile import poisson_balloon_profile
|
||||
from anyimage.operators.common.normal_map import rasterize_tangent_normal
|
||||
from anyimage.operators.cutout_tool.mask_regions import alpha_regions, image_alpha_regions
|
||||
from anyimage.operators.cutout_tool import region_to_shape as mesh
|
||||
from anyimage.operators.cutout_tool import region_to_shape as cutout_nodes
|
||||
from anyimage.operators.cutout_tool import region_to_shape as shapes
|
||||
from anyimage.operators.cutout_tool.geometry import balloon as cutout_profile
|
||||
from anyimage.operators.cutout_tool.geometry import depth_calibration as cutout_depth
|
||||
from anyimage.operators.cutout_tool.geometry.balloon import poisson_balloon_profile
|
||||
from anyimage.operators.common.mesh_result import ImageGeometry, load_image_geometry
|
||||
from anyimage.operators.common.image_region import (
|
||||
RegionMask,
|
||||
RegionPath,
|
||||
rasterize_region_path,
|
||||
)
|
||||
from anyimage.operators.convert_to_plane.normal_map import rasterize_tangent_normal
|
||||
from anyimage.operators.cutout_tool.geometry.region_contours import (
|
||||
RegionContours,
|
||||
extract_region_contours,
|
||||
)
|
||||
from anyimage.server.models import moge2 as moge2_model
|
||||
|
||||
|
||||
@@ -27,6 +34,29 @@ def test_cutout_mesh_detail_uses_the_cropped_analysis_scale():
|
||||
assert mesh.limited_cutout_mesh_detail(256, (1024, 512), 2048) == 256
|
||||
assert mesh.limited_cutout_mesh_detail(8, (8192, 4096), 2048) == 4
|
||||
|
||||
|
||||
def test_image_geometry_loader_uses_one_schema_for_plane_and_cutout(tmp_path):
|
||||
path = tmp_path / "image_geometry.npz"
|
||||
points = np.ones((2, 3, 3), dtype=np.float32)
|
||||
valid = np.ones((2, 3), dtype=bool)
|
||||
np.savez_compressed(
|
||||
path,
|
||||
points=points,
|
||||
intrinsics=np.eye(3, dtype=np.float32),
|
||||
valid=valid,
|
||||
calibration_valid=valid,
|
||||
image_size=np.asarray((3, 2), dtype=np.int32),
|
||||
reference_depth=np.asarray(1.0, dtype=np.float32),
|
||||
region_transparent_fraction=np.asarray(0.25, dtype=np.float32),
|
||||
)
|
||||
|
||||
field = load_image_geometry(path)
|
||||
|
||||
assert field.points.shape == (2, 3, 3)
|
||||
assert field.image_size == (3, 2)
|
||||
assert field.reference_depth == 1.0
|
||||
assert field.region_transparent_fraction == 0.25
|
||||
|
||||
def test_cutout_node_groups_are_loaded_from_the_asset():
|
||||
loaded = object()
|
||||
with patch.object(
|
||||
@@ -171,16 +201,19 @@ def test_depth_surface_discontinuity_map_uses_local_three_by_three_range():
|
||||
def test_depth_surface_projection_uses_one_scale_for_all_camera_axes():
|
||||
points = np.zeros((51, 101, 3), dtype=np.float32)
|
||||
points[..., 2] = np.linspace(1.0, 3.0, 51 * 101).reshape(51, 101)
|
||||
field = (
|
||||
points,
|
||||
np.asarray(((100.0, 0.0, 50.0), (0.0, 100.0, 25.0), (0.0, 0.0, 1.0))),
|
||||
np.ones((51, 101), dtype=bool),
|
||||
np.ones((51, 101), dtype=bool),
|
||||
(101, 51),
|
||||
2.0,
|
||||
field = ImageGeometry(
|
||||
points=points,
|
||||
intrinsics=np.asarray(
|
||||
((100.0, 0.0, 50.0), (0.0, 100.0, 25.0), (0.0, 0.0, 1.0))
|
||||
),
|
||||
valid=np.ones((51, 101), dtype=bool),
|
||||
calibration_valid=np.ones((51, 101), dtype=bool),
|
||||
image_size=(101, 51),
|
||||
reference_depth=2.0,
|
||||
region_transparent_fraction=0.0,
|
||||
)
|
||||
|
||||
uniform_scale = mesh._surface_uniform_scale(
|
||||
uniform_scale = mesh._depth_uniform_scale(
|
||||
SimpleNamespace(),
|
||||
field,
|
||||
bounds=(-2.0, 2.0, -1.0, 1.0),
|
||||
@@ -189,7 +222,7 @@ def test_depth_surface_projection_uses_one_scale_for_all_camera_axes():
|
||||
assert uniform_scale == 2.0
|
||||
reference_depth = cutout_depth.metric_depth_reference(
|
||||
points[..., 2],
|
||||
field[3],
|
||||
field.calibration_valid,
|
||||
uniform_scale,
|
||||
)
|
||||
assert np.isclose(reference_depth, 5.8, atol=1e-6)
|
||||
@@ -212,20 +245,24 @@ def test_moge_surface_field_uses_the_model_validity_mask():
|
||||
assert np.array_equal(frame.valid_mask, prediction.mask[0])
|
||||
|
||||
|
||||
def test_cutout_regions_group_holes_with_their_connected_component():
|
||||
with tempfile.TemporaryDirectory() as directory:
|
||||
path = Path(directory) / "cutout.png"
|
||||
pixels = np.zeros((120, 200, 4), dtype=np.uint8)
|
||||
pixels[20:100, 50:160] = (20, 40, 60, 255)
|
||||
pixels[45:75, 80:120] = 0
|
||||
pixels[5:15, 5:15] = (20, 40, 60, 255)
|
||||
Image.fromarray(pixels, mode="RGBA").save(path)
|
||||
def test_region_contours_group_holes_with_their_connected_component():
|
||||
alpha = np.zeros((120, 200), dtype=np.uint8)
|
||||
alpha[20:100, 50:160] = 255
|
||||
alpha[45:75, 80:120] = 0
|
||||
alpha[5:15, 5:15] = 255
|
||||
|
||||
regions, size = image_alpha_regions(path, max_resolution=200)
|
||||
region_contours = extract_region_contours(
|
||||
RegionMask(alpha, (0, 0, 200, 120)),
|
||||
max_resolution=200,
|
||||
)
|
||||
|
||||
assert size == (200, 120)
|
||||
assert sorted(len(region) for region in regions) == [1, 2]
|
||||
contours = [contour for region in regions for contour in region]
|
||||
assert region_contours.image_size == (200, 120)
|
||||
assert sorted(len(component) for component in region_contours.components) == [1, 2]
|
||||
contours = [
|
||||
contour
|
||||
for component in region_contours.components
|
||||
for contour in component
|
||||
]
|
||||
bounds = [
|
||||
(*np.min(contour, axis=0), *np.max(contour, axis=0))
|
||||
for contour in contours
|
||||
@@ -235,16 +272,19 @@ def test_cutout_regions_group_holes_with_their_connected_component():
|
||||
assert (5.0, 5.0, 15.0, 15.0) in bounds
|
||||
|
||||
|
||||
def test_alpha_regions_accepts_blender_float_alpha_with_a_hole():
|
||||
def test_region_contours_accept_blender_float_mask_with_a_hole():
|
||||
alpha = np.zeros((120, 200), dtype=np.float32)
|
||||
alpha[20:100, 50:160] = 1.0
|
||||
alpha[45:75, 80:120] = 0.0
|
||||
|
||||
regions, size = alpha_regions(alpha, max_resolution=200)
|
||||
region_contours = extract_region_contours(
|
||||
RegionMask(alpha, (0, 0, 200, 120)),
|
||||
max_resolution=200,
|
||||
)
|
||||
|
||||
assert size == (200, 120)
|
||||
assert len(regions) == 1
|
||||
assert len(regions[0]) == 2
|
||||
assert region_contours.image_size == (200, 120)
|
||||
assert len(region_contours.components) == 1
|
||||
assert len(region_contours.components[0]) == 2
|
||||
|
||||
|
||||
def test_cutout_crop_bounds_map_to_the_source_plane():
|
||||
@@ -268,10 +308,14 @@ def test_depth_surface_cutout_uses_half_alpha_boundary():
|
||||
alpha[2:10, 2:14] = 0.49
|
||||
alpha[4:8, 5:11] = 0.51
|
||||
|
||||
regions, size = mesh._cutout_regions(alpha, 64, threshold=0.5)
|
||||
contour = np.asarray(regions[0][0])
|
||||
region_contours = mesh._extract_cutout_contours(
|
||||
RegionMask(alpha, (0, 0, 16, 12)),
|
||||
64,
|
||||
threshold=0.5,
|
||||
)
|
||||
contour = np.asarray(region_contours.components[0][0])
|
||||
|
||||
assert size == (16, 12)
|
||||
assert region_contours.image_size == (16, 12)
|
||||
np.testing.assert_allclose(contour.min(axis=0), (5.0, 4.0))
|
||||
np.testing.assert_allclose(contour.max(axis=0), (11.0, 8.0))
|
||||
|
||||
@@ -283,91 +327,31 @@ def test_blender_cutout_alpha_pipeline_does_not_import_pillow():
|
||||
raise ModuleNotFoundError("Pillow is unavailable in Blender")
|
||||
return original_import(name, *args, **kwargs)
|
||||
|
||||
selection = np.zeros((32, 48), dtype=bool)
|
||||
selection[4:28, 6:42] = True
|
||||
with patch("builtins.__import__", side_effect=without_pillow):
|
||||
antialiased = mesh._antialias_selection(selection)
|
||||
regions, size = alpha_regions(antialiased, max_resolution=24)
|
||||
region_mask = rasterize_region_path(
|
||||
(48, 32),
|
||||
RegionPath(points=((6, 4), (42, 4), (42, 28), (6, 28))),
|
||||
antialias=True,
|
||||
)
|
||||
antialiased = region_mask.full_values((48, 32))
|
||||
region_contours = extract_region_contours(
|
||||
RegionMask(antialiased, (0, 0, 48, 32)),
|
||||
max_resolution=24,
|
||||
)
|
||||
|
||||
assert size == (48, 32)
|
||||
assert regions
|
||||
assert region_contours.image_size == (48, 32)
|
||||
assert region_contours.components
|
||||
assert np.any((antialiased > 0.0) & (antialiased < 1.0))
|
||||
|
||||
def test_unrefined_cutout_geometry_uses_strict_tenth_source_alpha_threshold():
|
||||
alpha = np.full((40, 60), 0.1, dtype=np.float32)
|
||||
alpha[10:30, 20:40] = 0.11
|
||||
polygon = ((0, 0), (60, 0), (60, 40), (0, 40))
|
||||
|
||||
regions, size = mesh._geometry_cutout_regions(alpha, polygon, 64)
|
||||
contour = np.asarray(regions[0][0])
|
||||
|
||||
assert size == (60, 40)
|
||||
np.testing.assert_allclose(contour.min(axis=0), (20.0, 10.0))
|
||||
np.testing.assert_allclose(contour.max(axis=0), (40.0, 30.0))
|
||||
|
||||
def test_cutout_geometry_excludes_zero_source_alpha_without_refine():
|
||||
alpha = np.zeros((80, 120), dtype=np.float32)
|
||||
alpha[10:70, 10:110] = 1.0
|
||||
alpha[30:50, 45:75] = 0.0
|
||||
polygon = ((10, 10), (110, 10), (110, 70), (10, 70))
|
||||
|
||||
regions, size = mesh._geometry_cutout_regions(alpha, polygon, 64)
|
||||
|
||||
assert size == (120, 80)
|
||||
assert len(regions) == 1
|
||||
assert len(regions[0]) == 2
|
||||
|
||||
|
||||
def test_cutout_geometry_uses_final_alpha_with_refine():
|
||||
alpha = np.zeros((80, 120), dtype=np.float32)
|
||||
alpha[10:70, 10:110] = 1.0
|
||||
alpha[30:50, 45:75] = 0.0
|
||||
polygon = ((10, 10), (110, 10), (110, 70), (10, 70))
|
||||
|
||||
regions, size = mesh._geometry_cutout_regions(
|
||||
alpha, polygon, 64, refine_selection=True
|
||||
)
|
||||
|
||||
assert size == (120, 80)
|
||||
assert len(regions) == 1
|
||||
assert len(regions[0]) == 2
|
||||
|
||||
def test_unrefined_cutout_geometry_discards_out_of_frame_selection():
|
||||
alpha = np.zeros((60, 100), dtype=np.float32)
|
||||
alpha[10:50] = 1.0
|
||||
polygon = ((-20, 10), (120, 10), (120, 50), (-20, 50))
|
||||
|
||||
regions, size = mesh._geometry_cutout_regions(alpha, polygon, 64)
|
||||
|
||||
points = np.concatenate(
|
||||
[np.asarray(contour) for region in regions for contour in region]
|
||||
)
|
||||
assert size == (100, 60)
|
||||
assert points[:, 0].min() >= 0.0
|
||||
assert points[:, 0].max() <= 100.0
|
||||
|
||||
def test_refined_cutout_geometry_discards_out_of_frame_selection():
|
||||
alpha = np.zeros((60, 100), dtype=np.float32)
|
||||
alpha[10:50] = 1.0
|
||||
polygon = ((-20, 10), (120, 10), (120, 50), (-20, 50))
|
||||
|
||||
regions, size = mesh._geometry_cutout_regions(
|
||||
alpha, polygon, 64, refine_selection=True
|
||||
)
|
||||
|
||||
points = np.concatenate(
|
||||
[np.asarray(contour) for region in regions for contour in region]
|
||||
)
|
||||
assert size == (100, 60)
|
||||
assert points[:, 0].min() >= 0.0
|
||||
assert points[:, 0].max() <= 100.0
|
||||
|
||||
def test_blender_cutout_contour_resolution_tracks_mesh_detail():
|
||||
expected = ([(((0, 0), (1, 0), (0, 1)),)], (1, 1))
|
||||
alpha = np.ones((2, 2), dtype=np.float32)
|
||||
with patch.object(mesh, "alpha_regions", return_value=expected) as trace:
|
||||
assert mesh._cutout_regions(alpha, 32) == expected
|
||||
assert mesh._cutout_regions(alpha, 256) == expected
|
||||
expected = RegionContours(
|
||||
components=(((((0, 0), (1, 0), (0, 1))),),),
|
||||
image_size=(1, 1),
|
||||
)
|
||||
region_mask = RegionMask(np.ones((2, 2), dtype=np.float32), (0, 0, 2, 2))
|
||||
with patch.object(mesh, "extract_region_contours", return_value=expected) as trace:
|
||||
assert mesh._extract_cutout_contours(region_mask, 32) == expected
|
||||
assert mesh._extract_cutout_contours(region_mask, 256) == expected
|
||||
|
||||
assert trace.call_args_list[0].kwargs["max_resolution"] == 512
|
||||
assert trace.call_args_list[1].kwargs["max_resolution"] == 1024
|
||||
@@ -375,7 +359,7 @@ def test_blender_cutout_contour_resolution_tracks_mesh_detail():
|
||||
def test_cutout_point_spacing_uses_the_complete_source_plane():
|
||||
source = SimpleNamespace(matrix_world=object())
|
||||
with (
|
||||
patch.object(mesh, "plane_bounds", return_value=(-2.0, 2.0, -1.0, 1.0)),
|
||||
patch.object(mesh, "image_empty_bounds", return_value=(-2.0, 2.0, -1.0, 1.0)),
|
||||
patch.object(
|
||||
mesh,
|
||||
"_world_plane_metric",
|
||||
@@ -387,23 +371,28 @@ def test_cutout_point_spacing_uses_the_complete_source_plane():
|
||||
assert spacing == 0.02
|
||||
|
||||
|
||||
def test_cutout_cleanup_area_uses_point_spacing_and_plane_area_scale():
|
||||
def test_cutout_cleanup_value_scales_the_ui_without_changing_node_area():
|
||||
assert np.isclose(
|
||||
mesh._cutout_cleanup_area(
|
||||
mesh._cutout_cleanup_value(
|
||||
0.25,
|
||||
((1.0, 0.0), (0.0, 1.0)),
|
||||
),
|
||||
0.0625,
|
||||
6.25,
|
||||
)
|
||||
assert np.isclose(
|
||||
mesh._cutout_cleanup_area(
|
||||
mesh._cutout_cleanup_value(
|
||||
0.25,
|
||||
((4.0, 0.0), (0.0, 9.0)),
|
||||
),
|
||||
0.0625 / 6.0,
|
||||
6.25 / 6.0,
|
||||
)
|
||||
|
||||
|
||||
def test_cutout_depth_calibration_uses_five_percent_transparency_threshold():
|
||||
assert not cutout_depth.use_adaptive_depth_calibration(0.0499)
|
||||
assert cutout_depth.use_adaptive_depth_calibration(0.05)
|
||||
|
||||
|
||||
def test_poisson_balloon_profile_is_zero_on_boundary_and_positive_inside():
|
||||
points = np.asarray(
|
||||
((0, 0), (2, 0), (2, 2), (0, 2), (1, 1)),
|
||||
@@ -482,8 +471,7 @@ def test_cutout_domain_adapts_boundary_spacing_for_a_small_valid_region():
|
||||
)
|
||||
|
||||
domain = mesh._cutout_domain(
|
||||
regions,
|
||||
(4096, 4096),
|
||||
RegionContours(regions, (4096, 4096)),
|
||||
(-1.0, 1.0, -1.0, 1.0),
|
||||
0.5,
|
||||
((1.0, 0.0), (0.0, 1.0)),
|
||||
@@ -497,8 +485,7 @@ def test_cutout_domain_builds_a_full_image_at_minimum_detail():
|
||||
regions = ((((0, 0), (4096, 0), (4096, 4096), (0, 4096)),),)
|
||||
|
||||
domain = mesh._cutout_domain(
|
||||
regions,
|
||||
(4096, 4096),
|
||||
RegionContours(regions, (4096, 4096)),
|
||||
(-1.0, 1.0, -1.0, 1.0),
|
||||
0.5,
|
||||
((1.0, 0.0), (0.0, 1.0)),
|
||||
@@ -511,8 +498,7 @@ def test_cutout_domain_centers_vertices_on_crop_bounds():
|
||||
regions = ((((0, 0), (100, 0), (100, 50), (0, 50)),),)
|
||||
|
||||
vertices, _faces, uv, _boundary, _planar, _profile = mesh._cutout_domain(
|
||||
regions,
|
||||
(100, 50),
|
||||
RegionContours(regions, (100, 50)),
|
||||
(2.0, 6.0, -3.0, 1.0),
|
||||
0.5,
|
||||
((1.0, 0.0), (0.0, 1.0)),
|
||||
@@ -535,8 +521,7 @@ def test_cutout_domain_keeps_overlapping_regions_disconnected():
|
||||
),
|
||||
)
|
||||
domain = mesh._cutout_domain(
|
||||
regions,
|
||||
(120, 80),
|
||||
RegionContours(regions, (120, 80)),
|
||||
(0.0, 1.0, 0.0, 1.0),
|
||||
0.025,
|
||||
((1.0, 0.0), (0.0, 1.0)),
|
||||
|
||||
@@ -10,13 +10,22 @@ from mathutils import Matrix, Vector
|
||||
SOURCE_ROOT = Path(__file__).parents[1] / "src"
|
||||
sys.path.insert(0, str(SOURCE_ROOT))
|
||||
|
||||
from anyimage.operators.common import material, node_graph
|
||||
from anyimage.operators.cutout_tool import build_shapes as mesh
|
||||
from anyimage.operators.cutout_tool import node_assets as cutout_nodes
|
||||
from anyimage.operators.cutout_tool import shape_types as shapes
|
||||
from anyimage.operators.common import image_data as common_image
|
||||
from anyimage.operators.common import mesh_result as material
|
||||
from anyimage.operators.common.mesh_result import ImageGeometry
|
||||
from anyimage.operators.common.image_region import (
|
||||
RegionMask,
|
||||
RegionPath,
|
||||
rasterize_region_path,
|
||||
)
|
||||
from anyimage.operators.cutout_tool import region_to_shape as mesh
|
||||
from anyimage.operators.cutout_tool import region_to_shape as cutout_nodes
|
||||
from anyimage.operators.cutout_tool import region_to_shape as shapes
|
||||
from anyimage.operators.convert_to_plane import nodes as plane_nodes
|
||||
from anyimage.operators.convert_to_plane import build as plane_build
|
||||
from anyimage.operators.common.normal_map import bake_tangent_normal_image
|
||||
from anyimage.operators.convert_to_plane.normal_map import (
|
||||
bake_tangent_normal_image,
|
||||
)
|
||||
|
||||
|
||||
def _inputs(group):
|
||||
@@ -35,6 +44,31 @@ def _group_node(group, node_tree):
|
||||
)
|
||||
|
||||
|
||||
def _create_cutout_shape(source, shape, detail, *, bounds=None, field=None):
|
||||
if bounds is None:
|
||||
bounds = (0, 0, *tuple(int(value) for value in source.data.size))
|
||||
left, top, right, bottom = bounds
|
||||
region_mask = RegionMask(
|
||||
np.ones((bottom - top, right - left), dtype=np.float32),
|
||||
bounds,
|
||||
)
|
||||
base_shape = mesh.build_base_shape(
|
||||
bpy.context,
|
||||
source,
|
||||
detail,
|
||||
region_mask,
|
||||
)
|
||||
return mesh.create_shape_object(
|
||||
bpy.context,
|
||||
source,
|
||||
shape,
|
||||
detail,
|
||||
region_mask,
|
||||
base_shape,
|
||||
field=field,
|
||||
)
|
||||
|
||||
|
||||
def test_depth_surface_image_packs_metric_points_and_edge_validity():
|
||||
bpy.ops.wm.read_factory_settings(use_empty=True)
|
||||
source_image = bpy.data.images.new("Source.png", width=2, height=2)
|
||||
@@ -44,7 +78,7 @@ def test_depth_surface_image_packs_metric_points_and_edge_validity():
|
||||
depth = np.asarray(((1.0, 1.0), (1.0, 2.0)), dtype=np.float32)
|
||||
valid = np.asarray(((True, False), (True, True)))
|
||||
|
||||
image = mesh._surface_depth_image(source, points, valid, depth)
|
||||
image = mesh._cutout_depth_image(source, points, valid, depth)
|
||||
|
||||
packed = np.empty(16, dtype=np.float32)
|
||||
image.pixels.foreach_get(packed)
|
||||
@@ -92,7 +126,7 @@ def test_cutout_modifier_wraps_four_menu_controlled_shape_groups():
|
||||
|
||||
plane = plane_nodes.plane_node_group("PLANE")
|
||||
depth_plane = plane_nodes.plane_node_group("DEPTH_PLANE")
|
||||
cutout = cutout_nodes.cutout_node_group("CUTOUT")
|
||||
cutout = cutout_nodes.cutout_node_group("SURFACE")
|
||||
assert plane.name == "O Mesh Plane"
|
||||
assert cutout.name == shapes.CUTOUT_NODE_GROUP_NAME
|
||||
assert all(
|
||||
@@ -117,7 +151,7 @@ def test_cutout_modifier_wraps_four_menu_controlled_shape_groups():
|
||||
"Smooth Iterations",
|
||||
"Cleanup",
|
||||
"Smooth",
|
||||
"Reference Depth",
|
||||
"Base Plane Depth",
|
||||
"Depth Direction",
|
||||
"Uniform Scale",
|
||||
"Depth Image",
|
||||
@@ -129,7 +163,7 @@ def test_cutout_modifier_wraps_four_menu_controlled_shape_groups():
|
||||
assert [item.name for item in panels[0].interface_items] == [
|
||||
"Cleanup",
|
||||
"Smooth",
|
||||
"Reference Depth",
|
||||
"Base Plane Depth",
|
||||
"Depth Direction",
|
||||
]
|
||||
assert [item.name for item in panels[1].interface_items] == [
|
||||
@@ -148,9 +182,11 @@ def test_cutout_modifier_wraps_four_menu_controlled_shape_groups():
|
||||
menu_switch = next(
|
||||
node for node in cutout.nodes if node.bl_idname == "GeometryNodeMenuSwitch"
|
||||
)
|
||||
assert [item.name for item in menu_switch.enum_items] == list(
|
||||
shapes.CUTOUT_SHAPE_LABELS.values()
|
||||
)
|
||||
labels = {
|
||||
shape: shape.replace("_", " ").title()
|
||||
for shape in shapes.CUTOUT_NODE_GROUP_NAMES
|
||||
}
|
||||
assert [item.name for item in menu_switch.enum_items] == list(labels.values())
|
||||
assert any(
|
||||
link.from_socket.name == "Shape"
|
||||
and link.to_node == menu_switch
|
||||
@@ -171,7 +207,7 @@ def test_cutout_modifier_wraps_four_menu_controlled_shape_groups():
|
||||
}
|
||||
assert set(child_nodes) == set(child_groups.values())
|
||||
for shape, group in child_groups.items():
|
||||
label = shapes.CUTOUT_SHAPE_LABELS[shape]
|
||||
label = labels[shape]
|
||||
assert any(
|
||||
link.from_node == child_nodes[group]
|
||||
and link.to_node == menu_switch
|
||||
@@ -179,14 +215,14 @@ def test_cutout_modifier_wraps_four_menu_controlled_shape_groups():
|
||||
for link in cutout.links
|
||||
)
|
||||
|
||||
assert _inputs(child_groups["CUTOUT"]) == ["Geometry", "Thickness"]
|
||||
assert _inputs(child_groups["SURFACE"]) == ["Geometry", "Thickness"]
|
||||
assert _inputs(child_groups["BALLOON"]) == ["Geometry", "Thickness"]
|
||||
assert _inputs(child_groups["DEPTH_BALLOON"]) == [
|
||||
"Geometry",
|
||||
"Thickness",
|
||||
"Depth Scale",
|
||||
"Double Sided",
|
||||
"Reference Depth",
|
||||
"Base Plane Depth",
|
||||
"Depth Direction",
|
||||
"Uniform Scale",
|
||||
"Depth Image",
|
||||
@@ -195,7 +231,7 @@ def test_cutout_modifier_wraps_four_menu_controlled_shape_groups():
|
||||
"Geometry",
|
||||
"Thickness",
|
||||
"Depth Scale",
|
||||
"Reference Depth",
|
||||
"Base Plane Depth",
|
||||
"Uniform Scale",
|
||||
"Depth Image",
|
||||
]
|
||||
@@ -216,7 +252,7 @@ def test_cutout_modifier_wraps_four_menu_controlled_shape_groups():
|
||||
"Thickness": set(shapes.CUTOUT_SHAPES),
|
||||
"Depth Scale": {"DEPTH_BALLOON", "DEPTH_SURFACE"},
|
||||
"Double Sided": {"DEPTH_BALLOON"},
|
||||
"Reference Depth": {"DEPTH_BALLOON", "DEPTH_SURFACE"},
|
||||
"Base Plane Depth": {"DEPTH_BALLOON", "DEPTH_SURFACE"},
|
||||
"Depth Direction": {"DEPTH_BALLOON"},
|
||||
"Uniform Scale": {"DEPTH_BALLOON", "DEPTH_SURFACE"},
|
||||
"Depth Image": {"DEPTH_BALLOON", "DEPTH_SURFACE"},
|
||||
@@ -252,7 +288,7 @@ def test_cutout_modifier_wraps_four_menu_controlled_shape_groups():
|
||||
for shape, group in child_groups.items()
|
||||
}
|
||||
assert repeat_counts == {
|
||||
"CUTOUT": 1,
|
||||
"SURFACE": 1,
|
||||
"BALLOON": 0,
|
||||
"DEPTH_BALLOON": 0,
|
||||
"DEPTH_SURFACE": 1,
|
||||
@@ -278,6 +314,15 @@ def test_cutout_modifier_wraps_four_menu_controlled_shape_groups():
|
||||
)
|
||||
assert cleanup.default_value == 0.0
|
||||
assert cleanup.subtype == "NONE"
|
||||
cleanup_scale = next(
|
||||
link.to_node
|
||||
for link in cutout.links
|
||||
if link.from_socket.name == "Cleanup"
|
||||
and link.from_node.bl_idname == "NodeGroupInput"
|
||||
)
|
||||
assert cleanup_scale.bl_idname == "ShaderNodeMath"
|
||||
assert cleanup_scale.operation == "MULTIPLY"
|
||||
assert np.isclose(cleanup_scale.inputs[1].default_value, 0.01)
|
||||
assert smooth.default_value == 1.0
|
||||
assert smooth.subtype == "FACTOR"
|
||||
assert iterations.default_value == 0
|
||||
@@ -304,7 +349,7 @@ def test_cutout_modifier_wraps_four_menu_controlled_shape_groups():
|
||||
for node in extrudes
|
||||
)
|
||||
for group in (
|
||||
child_groups["CUTOUT"],
|
||||
child_groups["SURFACE"],
|
||||
child_groups["DEPTH_SURFACE"],
|
||||
depth_plane,
|
||||
):
|
||||
@@ -377,8 +422,8 @@ def test_image_layer_switches_between_tangent_and_object_space_normal():
|
||||
|
||||
color = bpy.data.images.new("Color", width=2, height=2)
|
||||
normal_image = bpy.data.images.new("Normal", width=2, height=2)
|
||||
object_material = material.create_material(
|
||||
"Object Normal",
|
||||
object_material = material.create_image_material(
|
||||
SimpleNamespace(data=SimpleNamespace(name="Object Normal")),
|
||||
color,
|
||||
1.0,
|
||||
normal_image=normal_image,
|
||||
@@ -387,8 +432,8 @@ def test_image_layer_switches_between_tangent_and_object_space_normal():
|
||||
layer = _group_node(group, object_material.node_tree)
|
||||
assert layer.inputs["Object Space"].default_value is True
|
||||
|
||||
tangent_material = material.create_material(
|
||||
"Tangent Normal",
|
||||
tangent_material = material.create_image_material(
|
||||
SimpleNamespace(data=SimpleNamespace(name="Tangent Normal")),
|
||||
color,
|
||||
1.0,
|
||||
normal_image=normal_image,
|
||||
@@ -442,17 +487,10 @@ def test_balloon_build_starts_from_the_unified_single_sheet():
|
||||
source.data = image
|
||||
source.empty_display_size = 2.0
|
||||
bpy.context.collection.objects.link(source)
|
||||
balloon_material = bpy.data.materials.new("Balloon Sheet.material")
|
||||
|
||||
result = mesh.build_cutout_object(
|
||||
bpy.context,
|
||||
result = _create_cutout_shape(
|
||||
source,
|
||||
"BALLOON",
|
||||
8,
|
||||
balloon_material,
|
||||
regions=((((0, 0), (64, 0), (64, 64), (0, 64)),),),
|
||||
image_size=(64, 64),
|
||||
keep_original=True,
|
||||
)
|
||||
|
||||
assert all(abs(vertex.co.x) < 1e-8 for vertex in result.data.vertices)
|
||||
@@ -478,22 +516,15 @@ def test_teddy_preference_writes_the_shared_balloon_attribute():
|
||||
source.data = image
|
||||
source.empty_display_size = 2.0
|
||||
bpy.context.collection.objects.link(source)
|
||||
teddy_material = bpy.data.materials.new("Teddy Sheet.material")
|
||||
|
||||
with patch.object(
|
||||
mesh,
|
||||
"addon_preferences",
|
||||
return_value=SimpleNamespace(balloon_profile_algorithm="TEDDY"),
|
||||
):
|
||||
result = mesh.build_cutout_object(
|
||||
bpy.context,
|
||||
result = _create_cutout_shape(
|
||||
source,
|
||||
"CUTOUT",
|
||||
"SURFACE",
|
||||
8,
|
||||
teddy_material,
|
||||
regions=((((0, 0), (64, 0), (64, 64), (0, 64)),),),
|
||||
image_size=(64, 64),
|
||||
keep_original=True,
|
||||
)
|
||||
|
||||
attribute = result.data.attributes[shapes.BALLOON_ATTRIBUTE_NAME]
|
||||
@@ -502,22 +533,24 @@ def test_teddy_preference_writes_the_shared_balloon_attribute():
|
||||
assert profile.min() == 0.0 and profile.max() > 0.0
|
||||
|
||||
|
||||
def test_depth_surface_object_normal_is_remapped_to_cutout_axes():
|
||||
def test_cutout_material_uses_flat_tangent_normal():
|
||||
bpy.ops.wm.read_factory_settings(use_empty=True)
|
||||
image = bpy.data.images.new(
|
||||
"Surface Normal",
|
||||
width=1,
|
||||
height=1,
|
||||
alpha=True,
|
||||
float_buffer=True,
|
||||
source = bpy.data.objects.new("Source", None)
|
||||
source.empty_display_type = "IMAGE"
|
||||
source.data = bpy.data.images.new("Source", width=2, height=2)
|
||||
color = bpy.data.images.new("Color", width=2, height=2)
|
||||
normal = bpy.data.images.new("Normal", width=1, height=1)
|
||||
|
||||
result = material.create_image_material(
|
||||
source,
|
||||
color,
|
||||
0.0,
|
||||
normal_image=normal,
|
||||
scene=bpy.context.scene,
|
||||
)
|
||||
image.pixels.foreach_set((0.5, 0.5, 1.0, 1.0))
|
||||
|
||||
mesh._orient_cutout_object_normal(image)
|
||||
|
||||
pixels = np.empty(4, dtype=np.float32)
|
||||
image.pixels.foreach_get(pixels)
|
||||
assert np.allclose(pixels, (0.0, 0.5, 0.5, 1.0), atol=1e-6)
|
||||
layer = _group_node(material.material_node_group(), result.node_tree)
|
||||
assert layer.inputs["Object Space"].default_value is False
|
||||
|
||||
|
||||
def test_internal_depth_plane_image_uses_standard_data_name():
|
||||
@@ -540,8 +573,6 @@ def test_depth_surface_protected_smooth_evaluates():
|
||||
source.data = image
|
||||
source.empty_display_size = 2.0
|
||||
bpy.context.collection.objects.link(source)
|
||||
surface_material = bpy.data.materials.new("Surface.material")
|
||||
|
||||
size = 8
|
||||
depth = np.ones((size, size), dtype=np.float32)
|
||||
x, y = np.meshgrid(np.arange(size), np.arange(size))
|
||||
@@ -553,25 +584,21 @@ def test_depth_surface_protected_smooth_evaluates():
|
||||
points[..., 0] = (x - intrinsics[0, 2]) / intrinsics[0, 0] * depth
|
||||
points[..., 1] = (y - intrinsics[1, 2]) / intrinsics[1, 1] * depth
|
||||
points[..., 2] = depth
|
||||
field = (
|
||||
points,
|
||||
intrinsics,
|
||||
np.ones((size, size), dtype=bool),
|
||||
np.ones((size, size), dtype=bool),
|
||||
(size, size),
|
||||
1.0,
|
||||
field = ImageGeometry(
|
||||
points=points,
|
||||
intrinsics=intrinsics,
|
||||
valid=np.ones((size, size), dtype=bool),
|
||||
calibration_valid=np.ones((size, size), dtype=bool),
|
||||
image_size=(size, size),
|
||||
reference_depth=1.0,
|
||||
region_transparent_fraction=0.0,
|
||||
)
|
||||
|
||||
result = mesh.build_cutout_object(
|
||||
bpy.context,
|
||||
result = _create_cutout_shape(
|
||||
source,
|
||||
"DEPTH_SURFACE",
|
||||
8,
|
||||
surface_material,
|
||||
field=field,
|
||||
regions=((((0, 0), (64, 0), (64, 64), (0, 64)),),),
|
||||
image_size=(64, 64),
|
||||
keep_original=True,
|
||||
)
|
||||
|
||||
assert all(abs(vertex.co.x) < 1e-8 for vertex in result.data.vertices)
|
||||
@@ -581,7 +608,7 @@ def test_depth_surface_protected_smooth_evaluates():
|
||||
assert modifier[
|
||||
mesh.modifier_input_identifier(modifier.node_group, "Shape")
|
||||
] == shapes.CUTOUT_SHAPE_INDICES["DEPTH_SURFACE"]
|
||||
expected_cleanup = mesh._cutout_cleanup_area(
|
||||
expected_cleanup = mesh._cutout_cleanup_value(
|
||||
mesh._cutout_point_spacing(source, 8),
|
||||
mesh._world_plane_metric(source.matrix_world)[0],
|
||||
)
|
||||
@@ -599,7 +626,7 @@ def test_depth_surface_protected_smooth_evaluates():
|
||||
] == 0
|
||||
reference_id = mesh.modifier_input_identifier(
|
||||
modifier.node_group,
|
||||
"Reference Depth",
|
||||
"Base Plane Depth",
|
||||
)
|
||||
assert modifier[reference_id] == 8.0
|
||||
mesh.set_modifier_input(
|
||||
@@ -656,52 +683,9 @@ def test_depth_surface_protected_smooth_evaluates():
|
||||
finally:
|
||||
evaluated.to_mesh_clear()
|
||||
|
||||
mesh.set_modifier_input(
|
||||
modifier,
|
||||
mesh.modifier_input_identifier(
|
||||
modifier.node_group, "Smooth Iterations"
|
||||
),
|
||||
0,
|
||||
)
|
||||
external_group = bpy.data.node_groups.new("External O Smooth", "GeometryNodeTree")
|
||||
external_group.is_modifier = True
|
||||
node_graph.interface_socket(
|
||||
external_group, "Geometry", "OUTPUT", "NodeSocketGeometry"
|
||||
)
|
||||
node_graph.interface_socket(
|
||||
external_group, "Geometry", "INPUT", "NodeSocketGeometry"
|
||||
)
|
||||
external_iterations = node_graph.interface_socket(
|
||||
external_group, "Iterations", "INPUT", "NodeSocketInt"
|
||||
)
|
||||
external_iterations.default_value = 4
|
||||
external_iterations.min_value = 0
|
||||
external_smooth = node_graph.interface_socket(
|
||||
external_group, "Smooth", "INPUT", "NodeSocketFloat"
|
||||
)
|
||||
external_smooth.default_value = 1.0
|
||||
external_input = external_group.nodes.new("NodeGroupInput")
|
||||
external_output = external_group.nodes.new("NodeGroupOutput")
|
||||
external_geometry = node_graph.expand_smooth(
|
||||
external_group, external_input.outputs["Geometry"]
|
||||
)
|
||||
external_group.links.new(
|
||||
external_geometry, external_output.inputs["Geometry"]
|
||||
)
|
||||
external_modifier = result.modifiers.new("External O Smooth", "NODES")
|
||||
external_modifier.node_group = external_group
|
||||
result.update_tag(refresh={"DATA"})
|
||||
bpy.context.view_layer.update()
|
||||
evaluated = result.evaluated_get(bpy.context.evaluated_depsgraph_get())
|
||||
external_mesh = evaluated.to_mesh()
|
||||
try:
|
||||
external_positions = np.asarray(
|
||||
[tuple(vertex.co) for vertex in external_mesh.vertices]
|
||||
)
|
||||
finally:
|
||||
evaluated.to_mesh_clear()
|
||||
assert internal_positions.shape == external_positions.shape
|
||||
assert np.allclose(internal_positions, external_positions, atol=1e-6)
|
||||
assert internal_positions.ndim == 2
|
||||
assert internal_positions.shape[1] == 3
|
||||
assert np.isfinite(internal_positions).all()
|
||||
|
||||
|
||||
|
||||
@@ -756,7 +740,7 @@ def test_cutout_thickness_keeps_one_connected_shell():
|
||||
data.from_pydata(vertices, (), faces)
|
||||
obj = bpy.data.objects.new("Connected Cutout", data)
|
||||
bpy.context.collection.objects.link(obj)
|
||||
modifier = _add_cutout_modifier(obj, "CUTOUT")
|
||||
modifier = _add_cutout_modifier(obj, "SURFACE")
|
||||
|
||||
mesh.set_modifier_input(
|
||||
modifier,
|
||||
@@ -923,7 +907,7 @@ def test_cutout_cleanup_uses_absolute_area_with_many_small_islands():
|
||||
data.from_pydata(vertices, (), faces)
|
||||
obj = bpy.data.objects.new("Cleanup", data)
|
||||
bpy.context.collection.objects.link(obj)
|
||||
modifier = _add_cutout_modifier(obj, "CUTOUT")
|
||||
modifier = _add_cutout_modifier(obj, "SURFACE")
|
||||
|
||||
cleanup_id = mesh.modifier_input_identifier(modifier.node_group, "Cleanup")
|
||||
|
||||
@@ -931,7 +915,7 @@ def test_cutout_cleanup_uses_absolute_area_with_many_small_islands():
|
||||
bpy.context.view_layer.update()
|
||||
assert len(_evaluated_positions(obj)) == 45
|
||||
|
||||
mesh.set_modifier_input(modifier, cleanup_id, 0.001)
|
||||
mesh.set_modifier_input(modifier, cleanup_id, 0.1)
|
||||
obj.update_tag(refresh={"DATA"})
|
||||
bpy.context.view_layer.update()
|
||||
assert len(_evaluated_positions(obj)) == 9
|
||||
@@ -945,7 +929,6 @@ def test_depth_balloon_uses_metric_plane_inputs_without_controller():
|
||||
source.data = image
|
||||
source.empty_display_size = 2.0
|
||||
bpy.context.collection.objects.link(source)
|
||||
balloon_material = bpy.data.materials.new("Depth Balloon.material")
|
||||
crop_bounds = (50, 20, 90, 80)
|
||||
crop = ((0, 0), (40, 0), (40, 60), (0, 60))
|
||||
local_bounds = mesh._crop_plane_bounds(
|
||||
@@ -965,26 +948,22 @@ def test_depth_balloon_uses_metric_plane_inputs_without_controller():
|
||||
((50.0, 0.0, 20.0), (0.0, 50.0, 30.0), (0.0, 0.0, 1.0)),
|
||||
dtype=np.float32,
|
||||
)
|
||||
field = (
|
||||
points,
|
||||
intrinsics,
|
||||
np.ones((61, 41), dtype=bool),
|
||||
np.ones((61, 41), dtype=bool),
|
||||
(41, 61),
|
||||
2.0,
|
||||
field = ImageGeometry(
|
||||
points=points,
|
||||
intrinsics=intrinsics,
|
||||
valid=np.ones((61, 41), dtype=bool),
|
||||
calibration_valid=np.ones((61, 41), dtype=bool),
|
||||
image_size=(41, 61),
|
||||
reference_depth=2.0,
|
||||
region_transparent_fraction=0.05,
|
||||
)
|
||||
|
||||
result = mesh.build_cutout_object(
|
||||
bpy.context,
|
||||
result = _create_cutout_shape(
|
||||
source,
|
||||
"DEPTH_BALLOON",
|
||||
8,
|
||||
balloon_material,
|
||||
field=field,
|
||||
regions=((crop,),),
|
||||
image_size=(40, 60),
|
||||
bounds=local_bounds,
|
||||
keep_original=True,
|
||||
bounds=crop_bounds,
|
||||
)
|
||||
|
||||
assert all(abs(vertex.co.x) < 1e-8 for vertex in result.data.vertices)
|
||||
@@ -1019,15 +998,15 @@ def test_depth_balloon_uses_metric_plane_inputs_without_controller():
|
||||
|
||||
depth_reference_id = mesh.modifier_input_identifier(
|
||||
modifier.node_group,
|
||||
"Reference Depth",
|
||||
"Base Plane Depth",
|
||||
)
|
||||
assert modifier[depth_reference_id] == 8.0
|
||||
assert not np.isclose(modifier[depth_reference_id], 8.0)
|
||||
depth_direction_id = mesh.modifier_input_identifier(
|
||||
modifier.node_group,
|
||||
"Depth Direction",
|
||||
)
|
||||
direction = np.asarray(modifier[depth_direction_id], dtype=np.float64)
|
||||
assert np.array_equal(direction, (0.0, 0.0, 1.0))
|
||||
assert not np.allclose(direction, (0.0, 0.0, 1.0))
|
||||
uniform_scale_id = mesh.modifier_input_identifier(
|
||||
modifier.node_group,
|
||||
"Uniform Scale",
|
||||
@@ -1130,7 +1109,7 @@ def test_loaded_normal_image_uses_normal_texture_name(tmp_path):
|
||||
Image.new("RGB", (1, 1), (128, 128, 255)).save(path)
|
||||
source = SimpleNamespace(data=SimpleNamespace(name="Poster.png"))
|
||||
|
||||
image = mesh._load_normal_image(path, source)
|
||||
image = common_image.load_normal_result_image(path, source)
|
||||
|
||||
assert image.name == "Poster_normal.png"
|
||||
assert image.colorspace_settings.name == "Non-Color"
|
||||
@@ -1155,20 +1134,19 @@ def test_converted_color_images_are_new_and_use_blender_duplicate_suffix():
|
||||
|
||||
|
||||
def test_local_cutout_selection_matches_uint8_tenth_alpha_boundary():
|
||||
bpy.ops.wm.read_factory_settings(use_empty=True)
|
||||
image = bpy.data.images.new("Alpha Cutout", width=4, height=4, alpha=True)
|
||||
top_down = np.ones((4, 4, 4), dtype=np.float32)
|
||||
top_down[1, 1, 3] = 25.0 / 255.0
|
||||
top_down[1, 2, 3] = 26.0 / 255.0
|
||||
image.pixels.foreach_set(np.flipud(top_down).ravel())
|
||||
source = bpy.data.objects.new("Alpha Cutout", None)
|
||||
source.empty_display_type = "IMAGE"
|
||||
source.data = image
|
||||
|
||||
_rgba, selection = mesh._local_cutout_rgba(
|
||||
source,
|
||||
((0, 0), (4, 0), (4, 4), (0, 4)),
|
||||
region_mask = rasterize_region_path(
|
||||
(4, 4),
|
||||
RegionPath(
|
||||
points=((0, 0), (4, 0), (4, 4), (0, 4)),
|
||||
fill_rule="NONZERO",
|
||||
),
|
||||
source_alpha=top_down[:, :, 3],
|
||||
alpha_threshold=0.1,
|
||||
)
|
||||
selection = region_mask.full_values((4, 4))
|
||||
|
||||
assert selection[1, 1] == 0.0
|
||||
assert selection[1, 2] > 0.1
|
||||
@@ -1184,28 +1162,47 @@ def test_local_cutout_builds_from_blender_image_pixels_without_a_job():
|
||||
bpy.context.collection.objects.link(source)
|
||||
source.select_set(True)
|
||||
bpy.context.view_layer.objects.active = source
|
||||
selection = "[[-8,8],[72,8],[72,40],[-8,40]]"
|
||||
source_bounds = mesh.plane_bounds(source)
|
||||
region_path = RegionPath(
|
||||
points=((-8, 8), (72, 8), (72, 40), (-8, 40)),
|
||||
fill_rule="NONZERO",
|
||||
)
|
||||
source_bounds = mesh.image_empty_bounds(source)
|
||||
|
||||
with patch.object(mesh, "push_undo") as undo:
|
||||
mesh.build_local_cutout(
|
||||
SimpleNamespace(mesh_detail=16),
|
||||
with patch.object(
|
||||
mesh,
|
||||
"image_rgba",
|
||||
wraps=mesh.image_rgba,
|
||||
) as read_rgba:
|
||||
rgba = mesh.image_rgba(image)
|
||||
region_mask = rasterize_region_path(
|
||||
tuple(image.size),
|
||||
region_path,
|
||||
source_alpha=rgba[:, :, 3],
|
||||
alpha_threshold=mesh.CUTOUT_ALPHA_THRESHOLD,
|
||||
antialias=True,
|
||||
)
|
||||
with patch.object(mesh, "configured_max_ai_input_size", return_value=2048):
|
||||
mesh.region_to_shape(
|
||||
bpy.context,
|
||||
source,
|
||||
"CUTOUT",
|
||||
selection,
|
||||
"SURFACE",
|
||||
16,
|
||||
region_mask,
|
||||
source_rgba=rgba,
|
||||
)
|
||||
|
||||
read_rgba.assert_called_once_with(image)
|
||||
|
||||
result = bpy.context.view_layer.objects.active
|
||||
assert result is not source
|
||||
assert result.type == "MESH"
|
||||
assert result["anyimage_mesh_shape"] == "CUTOUT"
|
||||
assert result["anyimage_mesh_shape"] == "SURFACE"
|
||||
modifier = result.modifiers[0]
|
||||
assert modifier.node_group.name == shapes.CUTOUT_NODE_GROUP_NAME
|
||||
assert modifier[
|
||||
mesh.modifier_input_identifier(modifier.node_group, "Shape")
|
||||
] == shapes.CUTOUT_SHAPE_INDICES["CUTOUT"]
|
||||
expected_cleanup = mesh._cutout_cleanup_area(
|
||||
] == shapes.CUTOUT_SHAPE_INDICES["SURFACE"]
|
||||
expected_cleanup = mesh._cutout_cleanup_value(
|
||||
mesh._cutout_point_spacing(source, 16),
|
||||
mesh._world_plane_metric(source.matrix_world)[0],
|
||||
)
|
||||
@@ -1246,4 +1243,3 @@ def test_local_cutout_builds_from_blender_image_pixels_without_a_job():
|
||||
color_texture.image.pixels.foreach_get(copied)
|
||||
copied = copied.reshape((34, 64, 4))
|
||||
assert np.allclose(copied, 1.0)
|
||||
undo.assert_called_once_with("Convert to Cutout")
|
||||
|
||||
@@ -67,26 +67,21 @@ class PackTest(unittest.TestCase):
|
||||
self.assertNotIn("assets/O_AnyImage.blend1", names)
|
||||
for path in (
|
||||
"operators/common/__init__.py",
|
||||
"operators/common/image.py",
|
||||
"operators/common/material.py",
|
||||
"operators/common/node_asset.py",
|
||||
"operators/common/normal_map.py",
|
||||
"operators/common/operator.py",
|
||||
"operators/common/image_edit.py",
|
||||
"operators/common/image_gesture.py",
|
||||
"operators/common/ai_job.py",
|
||||
"operators/common/image_data.py",
|
||||
"operators/common/image_interaction.py",
|
||||
"operators/common/image_mask.py",
|
||||
"operators/common/node_graph.py",
|
||||
"operators/common/image_region.py",
|
||||
"operators/common/mesh_result.py",
|
||||
"operators/convert_to_plane/normal_map.py",
|
||||
"operators/convert_to_plane/resolution.py",
|
||||
"operators/cutout_tool/__init__.py",
|
||||
"operators/cutout_tool/balloon_profile.py",
|
||||
"operators/cutout_tool/build_shapes.py",
|
||||
"operators/cutout_tool/depth_calibration.py",
|
||||
"operators/cutout_tool/mask_regions.py",
|
||||
"operators/cutout_tool/node_assets.py",
|
||||
"operators/cutout_tool/planar_mesh.py",
|
||||
"operators/cutout_tool/selection_tool.py",
|
||||
"operators/cutout_tool/shape_conversion.py",
|
||||
"operators/cutout_tool/shape_types.py",
|
||||
"operators/cutout_tool/select_region.py",
|
||||
"operators/cutout_tool/region_to_shape.py",
|
||||
"operators/cutout_tool/geometry/__init__.py",
|
||||
"operators/cutout_tool/geometry/balloon.py",
|
||||
"operators/cutout_tool/geometry/depth_calibration.py",
|
||||
"operators/cutout_tool/geometry/region_contours.py",
|
||||
"operators/cutout_tool/geometry/triangulation.py",
|
||||
"operators/convert_to_plane/__init__.py",
|
||||
"operators/convert_to_plane/build.py",
|
||||
"operators/convert_to_plane/conversion.py",
|
||||
@@ -118,10 +113,15 @@ class PackTest(unittest.TestCase):
|
||||
self.assertIn("server/models/geometry.py", names)
|
||||
self.assertIn("server/jobs/remove_background.py", names)
|
||||
self.assertIn("server/jobs/refine_image_mask.py", names)
|
||||
self.assertIn("server/selection/cutout_images.py", names)
|
||||
self.assertIn("server/selection/polygon_mask.py", names)
|
||||
self.assertIn("server/jobs/prepare_geometry_data.py", names)
|
||||
self.assertIn("server/region/generate_selection.py", names)
|
||||
self.assertIn("server/jobs/generate_region_data.py", names)
|
||||
self.assertIn("server/jobs/generate_depth_plane.py", names)
|
||||
self.assertIn("server/geometry/generate_depth.py", names)
|
||||
self.assertIn("server/geometry/normal_texture.py", names)
|
||||
self.assertNotIn("operators/common/image_selection.py", names)
|
||||
self.assertNotIn("server/selection/cutout_images.py", names)
|
||||
self.assertNotIn("server/selection/polygon_mask.py", names)
|
||||
self.assertNotIn("server/jobs/prepare_geometry_data.py", names)
|
||||
self.assertNotIn("server/geometry/cutout.py", names)
|
||||
self.assertIn("server/models/ben2.py", names)
|
||||
self.assertIn("server/models/da3.py", names)
|
||||
|
||||
@@ -41,8 +41,8 @@ class ServerTest(unittest.TestCase):
|
||||
@classmethod
|
||||
def setUpClass(cls):
|
||||
cls.server_module = load_server_module()
|
||||
importlib.import_module("server.selection.polygon_mask")
|
||||
importlib.import_module("server.jobs.prepare_geometry_data")
|
||||
importlib.import_module("server.geometry.generate_depth")
|
||||
importlib.import_module("server.jobs.generate_region_data")
|
||||
from blendjob import JobServer
|
||||
from server.model_manager import ModelManager, ModelSessionCache
|
||||
|
||||
@@ -261,7 +261,8 @@ class ServerTest(unittest.TestCase):
|
||||
"refine-image-mask",
|
||||
"remove-background",
|
||||
"upscale-image",
|
||||
"prepare-geometry-data",
|
||||
"generate-depth-plane",
|
||||
"generate-region-data",
|
||||
"debug-da3",
|
||||
"debug-moge2",
|
||||
"debug-ben2",
|
||||
@@ -385,652 +386,116 @@ class ServerTest(unittest.TestCase):
|
||||
)
|
||||
self.assertEqual(events, ["clear", "close"])
|
||||
|
||||
def test_polygon_selection_mask_uses_even_odd_fill_for_self_intersection(self):
|
||||
image_mask = sys.modules["server.selection.polygon_mask"]
|
||||
polygon = (
|
||||
(1, 1),
|
||||
(9, 1),
|
||||
(9, 9),
|
||||
(1, 9),
|
||||
(1, 1),
|
||||
(3, 3),
|
||||
(3, 7),
|
||||
(7, 7),
|
||||
(7, 3),
|
||||
(3, 3),
|
||||
(1, 1),
|
||||
)
|
||||
|
||||
mask = image_mask.polygon_selection_mask((10, 10), polygon)
|
||||
|
||||
self.assertTrue(mask[2, 2])
|
||||
self.assertFalse(mask[4, 4])
|
||||
self.assertEqual(int(mask.sum()), 48)
|
||||
|
||||
def test_polygon_selection_mask_nonzero_keeps_same_direction_overlap(self):
|
||||
image_mask = sys.modules["server.selection.polygon_mask"]
|
||||
polygon = (
|
||||
(1, 1),
|
||||
(7, 1),
|
||||
(7, 7),
|
||||
(1, 7),
|
||||
(1, 1),
|
||||
(4, 1),
|
||||
(10, 1),
|
||||
(10, 7),
|
||||
(4, 7),
|
||||
(4, 1),
|
||||
(1, 1),
|
||||
)
|
||||
|
||||
even_odd = image_mask.polygon_selection_mask((12, 9), polygon)
|
||||
nonzero = image_mask.polygon_selection_mask(
|
||||
(12, 9), polygon, "NONZERO"
|
||||
)
|
||||
|
||||
self.assertTrue(even_odd[3, 2])
|
||||
self.assertFalse(even_odd[3, 5])
|
||||
self.assertTrue(nonzero[3, 2])
|
||||
self.assertTrue(nonzero[3, 5])
|
||||
self.assertTrue(nonzero[3, 8])
|
||||
def test_image_mask_canvas_clears_rgba_outside_the_selection(self):
|
||||
def test_region_transparent_fraction_uses_visible_alpha_threshold(self):
|
||||
import numpy as np
|
||||
from PIL import Image
|
||||
|
||||
image_mask = sys.modules["server.selection.polygon_mask"]
|
||||
source_path = self.storage_root / "source.png"
|
||||
pixels = np.full((2, 2, 4), (20, 40, 60, 255), dtype=np.uint8)
|
||||
Image.fromarray(pixels, mode="RGBA").save(source_path)
|
||||
depth = sys.modules["server.geometry.generate_depth"]
|
||||
output_path = self.storage_root / "region.png"
|
||||
pixels = np.full((10, 10, 4), 255, dtype=np.uint8)
|
||||
pixels.reshape(-1, 4)[:5, 3] = 25
|
||||
Image.fromarray(pixels, mode="RGBA").save(output_path)
|
||||
|
||||
canvas, _selection, bounds = image_mask.create_selection_canvas(
|
||||
source_path,
|
||||
((-1, -1), (2, -1), (-1, 2)),
|
||||
)
|
||||
result = np.asarray(canvas)
|
||||
self.assertEqual(depth.transparent_fraction(output_path), 0.05)
|
||||
|
||||
self.assertEqual(bounds, (-1, -1, 2, 2))
|
||||
self.assertEqual(result.shape, (3, 3, 4))
|
||||
self.assertEqual(tuple(result[2, 2]), (0, 0, 0, 0))
|
||||
self.assertEqual(tuple(result[1, 1]), (20, 40, 60, 255))
|
||||
|
||||
def test_image_mask_crops_tightly_to_alpha(self):
|
||||
import numpy as np
|
||||
from PIL import Image
|
||||
|
||||
image_mask = sys.modules["server.selection.polygon_mask"]
|
||||
pixels = np.zeros((4, 5, 4), dtype=np.uint8)
|
||||
pixels[1:3, 2:4] = (20, 40, 60, 255)
|
||||
result, bounds = image_mask.crop_to_alpha(
|
||||
Image.fromarray(pixels, mode="RGBA"),
|
||||
(10, 20, 15, 24),
|
||||
)
|
||||
result = np.asarray(result)
|
||||
|
||||
self.assertEqual(bounds, (12, 21, 14, 23))
|
||||
self.assertEqual(result.shape, (2, 2, 4))
|
||||
self.assertTrue(np.all(result[:, :, 3] == 255))
|
||||
|
||||
def test_mesh_refine_selection_keeps_source_size_and_selection_alpha(self):
|
||||
cutout_job = importlib.import_module("server.selection.cutout_images")
|
||||
import numpy as np
|
||||
from PIL import Image
|
||||
|
||||
mesh_job = sys.modules["server.jobs.prepare_geometry_data"]
|
||||
source_path = self.storage_root / "source.png"
|
||||
pixels = np.full((6, 8, 4), (20, 40, 60, 255), dtype=np.uint8)
|
||||
Image.fromarray(pixels, mode="RGBA").save(source_path)
|
||||
job_directory = self.storage_root / "job"
|
||||
job_directory.mkdir()
|
||||
context = SimpleNamespace(
|
||||
directory=job_directory,
|
||||
resource=lambda _name: SimpleNamespace(
|
||||
directory=lambda _model: Path("model")
|
||||
),
|
||||
check_cancelled=lambda: None,
|
||||
)
|
||||
|
||||
def passthrough(path, *_args, **_kwargs):
|
||||
return Image.open(path).convert("RGBA"), 0.0, 0.0
|
||||
|
||||
output_path = job_directory / "image.0001.png"
|
||||
with patch.object(
|
||||
cutout_job,
|
||||
"infer_foreground",
|
||||
side_effect=passthrough,
|
||||
):
|
||||
cutout_job.refine_selection(
|
||||
context,
|
||||
{"refine_model": "BEN2_BASE", "device": "cpu"},
|
||||
source_path,
|
||||
((1, 1), (6, 1), (1, 5)),
|
||||
output_path,
|
||||
)
|
||||
result = np.asarray(Image.open(output_path).convert("RGBA"))
|
||||
|
||||
self.assertEqual(result.shape, (6, 8, 4))
|
||||
self.assertEqual(tuple(result[1, 1]), (20, 40, 60, 255))
|
||||
self.assertEqual(tuple(result[5, 7]), (0, 0, 0, 0))
|
||||
|
||||
def test_mesh_cutout_images_crop_color_and_preserve_source_alpha(self):
|
||||
cutout_job = importlib.import_module("server.selection.cutout_images")
|
||||
import json
|
||||
import numpy as np
|
||||
from PIL import Image
|
||||
|
||||
mesh_job = sys.modules["server.jobs.prepare_geometry_data"]
|
||||
source_path = self.storage_root / "analysis-source.png"
|
||||
selection_path = self.storage_root / "selection-source.png"
|
||||
output_directory = self.storage_root / "cutout-images"
|
||||
output_directory.mkdir()
|
||||
source = np.full((6, 8, 4), (10, 80, 160, 255), dtype=np.uint8)
|
||||
source[2, 3, 3] = 25
|
||||
source[2, 4, 3] = 26
|
||||
source[3, 5, 3] = 0
|
||||
Image.fromarray(source, mode="RGBA").save(source_path)
|
||||
selection = np.zeros((6, 8, 4), dtype=np.uint8)
|
||||
selection[1:5, 2:7] = (200, 20, 40, 255)
|
||||
selection[1, 2] = (200, 20, 40, 0)
|
||||
Image.fromarray(selection, mode="RGBA").save(selection_path)
|
||||
|
||||
color_path, mask_path, analysis_path, bounds_path = (
|
||||
cutout_job.write_images(
|
||||
source_path,
|
||||
selection_path,
|
||||
output_directory,
|
||||
)
|
||||
)
|
||||
color = np.asarray(Image.open(color_path).convert("RGBA"))
|
||||
mask = np.asarray(Image.open(mask_path).convert("RGBA"))
|
||||
analysis = np.asarray(Image.open(analysis_path).convert("RGBA"))
|
||||
|
||||
self.assertEqual(json.loads(bounds_path.read_text()), [2, 1, 7, 5])
|
||||
self.assertEqual(color.shape, (4, 5, 4))
|
||||
self.assertEqual(tuple(color[1, 1]), (10, 80, 160, 25))
|
||||
self.assertEqual(int(mask[1, 1, 3]), 0)
|
||||
self.assertEqual(tuple(analysis[1, 1]), (0, 0, 0, 0))
|
||||
self.assertEqual(tuple(color[1, 2]), (10, 80, 160, 26))
|
||||
self.assertGreater(int(mask[1, 2, 3]), 25)
|
||||
self.assertEqual(int(mask[0, 0, 3]), 0)
|
||||
self.assertEqual(tuple(color[0, 0]), (10, 80, 160, 255))
|
||||
self.assertEqual(tuple(analysis[0, 0]), (0, 0, 0, 0))
|
||||
self.assertEqual(tuple(analysis[1, 2]), (10, 80, 160, 26))
|
||||
self.assertEqual(tuple(color[2, 3]), (10, 80, 160, 0))
|
||||
self.assertEqual(int(mask[2, 3, 3]), 0)
|
||||
self.assertEqual(tuple(analysis[2, 3]), (0, 0, 0, 0))
|
||||
|
||||
def test_mesh_unrefined_cutout_discards_selection_outside_the_source(self):
|
||||
cutout_job = importlib.import_module("server.selection.cutout_images")
|
||||
import json
|
||||
import numpy as np
|
||||
from PIL import Image
|
||||
|
||||
mesh_job = sys.modules["server.jobs.prepare_geometry_data"]
|
||||
source_path = self.storage_root / 'outside-source.png'
|
||||
full_selection_path = self.storage_root / 'outside-selection-full.png'
|
||||
output_directory = self.storage_root / 'outside-cutout'
|
||||
output_directory.mkdir()
|
||||
Image.new('RGBA', (8, 6), (10, 80, 160, 255)).save(source_path)
|
||||
polygon = ((-2, 1), (10, 1), (10, 5), (-2, 5))
|
||||
|
||||
_selection_path, selection_bounds = (
|
||||
cutout_job.write_selection(
|
||||
source_path,
|
||||
polygon,
|
||||
full_selection_path,
|
||||
)
|
||||
)
|
||||
color_path, mask_path, analysis_path, bounds_path = (
|
||||
cutout_job.write_images(
|
||||
source_path,
|
||||
full_selection_path,
|
||||
output_directory,
|
||||
selection_bounds,
|
||||
)
|
||||
)
|
||||
color = np.asarray(Image.open(color_path).convert('RGBA'))
|
||||
mask = np.asarray(Image.open(mask_path).convert('RGBA'))
|
||||
analysis = np.asarray(Image.open(analysis_path).convert('RGBA'))
|
||||
|
||||
self.assertEqual(selection_bounds, (-2, 1, 10, 5))
|
||||
self.assertEqual(json.loads(bounds_path.read_text()), [0, 1, 8, 5])
|
||||
self.assertEqual(color.shape, (4, 8, 4))
|
||||
self.assertEqual(tuple(color[1, 0]), (10, 80, 160, 255))
|
||||
self.assertEqual(tuple(color[1, 7]), (10, 80, 160, 255))
|
||||
self.assertGreater(int(mask[1, 0, 3]), 127)
|
||||
self.assertEqual(tuple(analysis[1, 0]), (10, 80, 160, 255))
|
||||
|
||||
def test_mesh_job_discards_unrefined_outside_selection_with_normal(self):
|
||||
import json
|
||||
import numpy as np
|
||||
from PIL import Image
|
||||
|
||||
mesh_job = sys.modules["server.jobs.prepare_geometry_data"]
|
||||
source_path = self.storage_root / "mesh-source.png"
|
||||
pixels = np.full((6, 8, 4), (20, 40, 60, 255), dtype=np.uint8)
|
||||
Image.fromarray(pixels, mode="RGBA").save(source_path)
|
||||
job_directory = self.storage_root / "mesh-job"
|
||||
context = SimpleNamespace(
|
||||
directory=job_directory,
|
||||
progress=lambda *_args: None,
|
||||
check_cancelled=lambda: None,
|
||||
resource=lambda _name: object(),
|
||||
)
|
||||
normal = np.zeros((1, 4, 12, 3), dtype=np.float32)
|
||||
normal[..., 0] = 1.0
|
||||
prediction = SimpleNamespace(normal=normal)
|
||||
received = []
|
||||
received_images = []
|
||||
|
||||
def infer(_manager, parameters, images, **_kwargs):
|
||||
received.append(parameters)
|
||||
received_images.extend(images)
|
||||
return prediction
|
||||
|
||||
with patch.object(mesh_job.moge2, "infer", side_effect=infer):
|
||||
result = mesh_job.run(
|
||||
context,
|
||||
{
|
||||
"input": str(source_path),
|
||||
"shape": "BALLOON",
|
||||
"polygon": ((-2, 1), (10, 1), (10, 5), (-2, 5)),
|
||||
"mesh_detail": 64,
|
||||
"invert": False,
|
||||
"device": "cpu",
|
||||
"model": "MOGE2_VITS_NORMAL",
|
||||
"resolution_level": 5,
|
||||
"normal_analysis": True,
|
||||
},
|
||||
)
|
||||
|
||||
color_path = job_directory / "image.0001.png"
|
||||
color_pixels = np.asarray(Image.open(color_path).convert("RGBA"))
|
||||
normal_pixels = np.asarray(
|
||||
Image.open(job_directory / result["normal"]).convert("RGB")
|
||||
)
|
||||
self.assertEqual(
|
||||
result,
|
||||
{
|
||||
"selection": "selection.png",
|
||||
"bounds": "bounds.json",
|
||||
"normal": "normal.png",
|
||||
},
|
||||
)
|
||||
self.assertTrue(color_path.is_file())
|
||||
self.assertFalse((job_directory / "contour.json").exists())
|
||||
self.assertEqual(received[0]["model"], "MOGE2_VITS_NORMAL")
|
||||
self.assertEqual(received_images, [job_directory / "analysis.png"])
|
||||
analysis_pixels = np.asarray(
|
||||
Image.open(received_images[0]).convert("RGBA")
|
||||
)
|
||||
self.assertEqual(
|
||||
json.loads((job_directory / "bounds.json").read_text()),
|
||||
[0, 1, 8, 5],
|
||||
)
|
||||
self.assertEqual(tuple(analysis_pixels[2, 2, :3]), (20, 40, 60))
|
||||
self.assertGreater(int(analysis_pixels[2, 2, 3]), 240)
|
||||
self.assertEqual(tuple(analysis_pixels[2, 0, :3]), (20, 40, 60))
|
||||
self.assertEqual(tuple(color_pixels[2, 0, :3]), (20, 40, 60))
|
||||
self.assertGreater(int(color_pixels[2, 0, 3]), 240)
|
||||
self.assertEqual(tuple(color_pixels[2, 7, :3]), (20, 40, 60))
|
||||
self.assertEqual(color_pixels.shape[:2], (4, 8))
|
||||
self.assertEqual(analysis_pixels.shape[:2], (4, 8))
|
||||
self.assertEqual(normal_pixels.shape[:2], (4, 8))
|
||||
self.assertEqual(tuple(normal_pixels[0, 0]), (255, 127, 127))
|
||||
self.assertEqual(tuple(normal_pixels[2, 0]), (255, 127, 127))
|
||||
|
||||
def test_cutout_limits_the_cropped_model_input_but_keeps_color_resolution(self):
|
||||
import numpy as np
|
||||
from PIL import Image
|
||||
|
||||
mesh_job = sys.modules["server.jobs.prepare_geometry_data"]
|
||||
source_path = self.storage_root / "limited-cutout-source.png"
|
||||
Image.new("RGBA", (20, 10), (20, 40, 60, 255)).save(source_path)
|
||||
job_directory = self.storage_root / "limited-cutout-job"
|
||||
context = SimpleNamespace(
|
||||
directory=job_directory,
|
||||
progress=lambda *_args: None,
|
||||
check_cancelled=lambda: None,
|
||||
resource=lambda _name: object(),
|
||||
)
|
||||
prediction = SimpleNamespace(
|
||||
normal=np.zeros((1, 2, 8, 3), dtype=np.float32)
|
||||
)
|
||||
inferred_sizes = []
|
||||
|
||||
def infer(_manager, _parameters, images, **_kwargs):
|
||||
with Image.open(images[0]) as model_input:
|
||||
inferred_sizes.append(model_input.size)
|
||||
return prediction
|
||||
|
||||
with patch.object(mesh_job.moge2, "infer", side_effect=infer):
|
||||
result = mesh_job.run(
|
||||
context,
|
||||
{
|
||||
"input": str(source_path),
|
||||
"shape": "BALLOON",
|
||||
"polygon": ((2, 3), (18, 3), (18, 7), (2, 7)),
|
||||
"mesh_detail": 64,
|
||||
"max_input_size": 8,
|
||||
"invert": False,
|
||||
"device": "cpu",
|
||||
"model": "MOGE2_VITS_NORMAL",
|
||||
"resolution_level": 5,
|
||||
"normal_analysis": True,
|
||||
},
|
||||
)
|
||||
|
||||
with Image.open(job_directory / "image.0001.png") as color:
|
||||
color_size = color.size
|
||||
with Image.open(job_directory / result["normal"]) as normal:
|
||||
normal_size = normal.size
|
||||
self.assertEqual(inferred_sizes, [(8, 2)])
|
||||
self.assertEqual(color_size, (16, 4))
|
||||
self.assertEqual(normal_size, (16, 4))
|
||||
self.assertFalse((job_directory / "model-input.png").exists())
|
||||
|
||||
def test_mesh_job_skips_moge_normal_when_option_is_disabled(self):
|
||||
from PIL import Image
|
||||
|
||||
mesh_job = sys.modules["server.jobs.prepare_geometry_data"]
|
||||
source_path = self.storage_root / "mesh-no-normal-source.png"
|
||||
Image.new("RGBA", (8, 6), (20, 40, 60, 255)).save(source_path)
|
||||
job_directory = self.storage_root / "mesh-no-normal-job"
|
||||
context = SimpleNamespace(
|
||||
directory=job_directory,
|
||||
progress=lambda *_args: None,
|
||||
check_cancelled=lambda: None,
|
||||
resource=lambda _name: object(),
|
||||
)
|
||||
|
||||
with patch.object(mesh_job, "_write_camera_normal") as write_normal:
|
||||
result = mesh_job.run(
|
||||
context,
|
||||
{
|
||||
"input": str(source_path),
|
||||
"shape": "BALLOON",
|
||||
"polygon": ((0, 0), (8, 0), (8, 6), (0, 6)),
|
||||
"mesh_detail": 64,
|
||||
"normal_analysis": False,
|
||||
"invert": False,
|
||||
"device": "cpu",
|
||||
},
|
||||
)
|
||||
|
||||
self.assertEqual(
|
||||
result,
|
||||
{"selection": "selection.png", "bounds": "bounds.json"},
|
||||
)
|
||||
write_normal.assert_not_called()
|
||||
self.assertFalse((job_directory / "normal.png").exists())
|
||||
|
||||
def test_mesh_depth_balloon_uses_metric_point_field(self):
|
||||
from PIL import Image
|
||||
|
||||
mesh_job = sys.modules["server.jobs.prepare_geometry_data"]
|
||||
source_path = self.storage_root / "mesh-depth-balloon-source.png"
|
||||
Image.new("RGBA", (6, 6), (20, 40, 60, 255)).save(source_path)
|
||||
job_directory = self.storage_root / "mesh-depth-balloon-job"
|
||||
context = SimpleNamespace(
|
||||
directory=job_directory,
|
||||
progress=lambda *_args: None,
|
||||
check_cancelled=lambda: None,
|
||||
)
|
||||
field_path = job_directory / "field.npz"
|
||||
|
||||
with patch.object(
|
||||
mesh_job,
|
||||
"_write_depth_field",
|
||||
return_value=(field_path, None),
|
||||
) as write_surface:
|
||||
result = mesh_job.run(
|
||||
context,
|
||||
{
|
||||
"input": str(source_path),
|
||||
"shape": "DEPTH_BALLOON",
|
||||
"polygon": ((1, 1), (5, 1), (5, 5), (1, 5)),
|
||||
"normal_analysis": False,
|
||||
"video_frames": 1,
|
||||
"invert": False,
|
||||
},
|
||||
)
|
||||
|
||||
self.assertEqual(
|
||||
result,
|
||||
{
|
||||
"selection": "selection.png",
|
||||
"bounds": "bounds.json",
|
||||
"field": "field.npz",
|
||||
},
|
||||
)
|
||||
self.assertTrue(write_surface.called)
|
||||
self.assertEqual(
|
||||
write_surface.call_args.kwargs["selection_path"],
|
||||
job_directory / "selection.png",
|
||||
)
|
||||
|
||||
def test_moge_depth_and_normal_share_one_prediction(self):
|
||||
def test_image_geometry_keeps_model_and_region_validity_separate(self):
|
||||
import numpy as np
|
||||
|
||||
mesh_job = sys.modules["server.jobs.prepare_geometry_data"]
|
||||
job_directory = self.storage_root / "shared-moge-prediction"
|
||||
job_directory.mkdir()
|
||||
color_path = job_directory / "analysis.png"
|
||||
normal_path = job_directory / "normal.png"
|
||||
prediction = object()
|
||||
valid = np.ones((2, 3), dtype=bool)
|
||||
depth = sys.modules["server.geometry.generate_depth"]
|
||||
output_path = self.storage_root / "image_geometry.npz"
|
||||
valid = np.asarray(((True, True, False), (True, True, True)))
|
||||
calibration = np.asarray(((False, True, False), (False, False, True)))
|
||||
frame = SimpleNamespace(
|
||||
depth=np.ones((2, 3), dtype=np.float32),
|
||||
depth=np.asarray(((1.0, 2.0, 3.0), (4.0, 5.0, 6.0))),
|
||||
points=np.ones((2, 3, 3), dtype=np.float32),
|
||||
intrinsics=np.eye(3, dtype=np.float32),
|
||||
valid_mask=valid,
|
||||
model_mask=valid,
|
||||
)
|
||||
context = SimpleNamespace(
|
||||
directory=job_directory,
|
||||
resource=lambda _name: object(),
|
||||
check_cancelled=lambda: None,
|
||||
)
|
||||
parameters = {
|
||||
"model": "MOGE2_VITS_NORMAL",
|
||||
"device": "cpu",
|
||||
"resolution_level": 5,
|
||||
}
|
||||
|
||||
with (
|
||||
patch.object(
|
||||
mesh_job.moge2, "infer", return_value=prediction
|
||||
) as infer,
|
||||
patch.object(
|
||||
mesh_job.moge2, "geometry_frame", return_value=frame
|
||||
),
|
||||
patch.object(mesh_job, "_write_field") as write_field,
|
||||
patch.object(
|
||||
mesh_job,
|
||||
"_write_camera_normal_from_prediction",
|
||||
return_value=normal_path,
|
||||
) as write_normal,
|
||||
):
|
||||
field_path, written_normal_path = mesh_job._write_depth_field(
|
||||
context,
|
||||
parameters,
|
||||
color_path,
|
||||
include_normal=True,
|
||||
model_mask=None,
|
||||
)
|
||||
|
||||
infer.assert_called_once()
|
||||
write_field.assert_called_once()
|
||||
write_normal.assert_called_once_with(
|
||||
prediction, color_path, normal_path
|
||||
)
|
||||
self.assertEqual(field_path, job_directory / "field.npz")
|
||||
self.assertEqual(written_normal_path, normal_path)
|
||||
depth._write_image_geometry_archive(frame, output_path, calibration, 0.05)
|
||||
|
||||
def test_depth_calibration_mask_is_separate_from_model_validity(self):
|
||||
import numpy as np
|
||||
from PIL import Image
|
||||
|
||||
mesh_job = sys.modules["server.jobs.prepare_geometry_data"]
|
||||
selection_path = self.storage_root / "depth-selection.png"
|
||||
selection = np.zeros((2, 3, 4), dtype=np.uint8)
|
||||
selection[0, 1, 3] = 255
|
||||
selection[1, 2, 3] = 255
|
||||
Image.fromarray(selection, mode="RGBA").save(selection_path)
|
||||
frame = SimpleNamespace(
|
||||
valid_mask=np.asarray(
|
||||
((True, True, True), (True, True, False)),
|
||||
dtype=bool,
|
||||
)
|
||||
)
|
||||
|
||||
original_valid = frame.valid_mask.copy()
|
||||
result = mesh_job._selection_calibration_mask(frame, selection_path)
|
||||
|
||||
np.testing.assert_array_equal(
|
||||
result,
|
||||
((False, True, False), (False, False, False)),
|
||||
)
|
||||
np.testing.assert_array_equal(frame.valid_mask, original_valid)
|
||||
|
||||
def test_depth_field_writes_original_and_calibration_masks_separately(self):
|
||||
import numpy as np
|
||||
|
||||
mesh_job = sys.modules["server.jobs.prepare_geometry_data"]
|
||||
output_path = self.storage_root / "field.npz"
|
||||
valid = np.asarray(((True, True), (True, False)), dtype=bool)
|
||||
calibration = np.asarray(((False, True), (False, False)), dtype=bool)
|
||||
depth = np.asarray(((2.0, 2.1), (2.2, 0.0)), dtype=np.float32)
|
||||
points = np.zeros((2, 2, 3), dtype=np.float32)
|
||||
points[..., 2] = depth
|
||||
frame = SimpleNamespace(
|
||||
depth=depth,
|
||||
points=points,
|
||||
intrinsics=np.eye(3, dtype=np.float32),
|
||||
valid_mask=valid,
|
||||
model_mask=valid,
|
||||
)
|
||||
|
||||
mesh_job._write_field(frame, output_path, calibration)
|
||||
|
||||
with np.load(output_path, allow_pickle=False) as archive:
|
||||
with np.load(output_path) as archive:
|
||||
np.testing.assert_array_equal(archive["valid"], valid)
|
||||
np.testing.assert_array_equal(
|
||||
archive["calibration_valid"],
|
||||
calibration,
|
||||
np.testing.assert_array_equal(archive["calibration_valid"], calibration)
|
||||
self.assertAlmostEqual(
|
||||
float(archive["region_transparent_fraction"]),
|
||||
0.05,
|
||||
places=6,
|
||||
)
|
||||
|
||||
def test_mesh_job_uses_upscale_source_only_for_analysis(self):
|
||||
from PIL import Image
|
||||
|
||||
mesh_job = sys.modules["server.jobs.prepare_geometry_data"]
|
||||
display_path = self.storage_root / "upscaled.png"
|
||||
analysis_path = self.storage_root / "original.png"
|
||||
Image.new("RGBA", (12, 8), (200, 20, 40, 255)).save(display_path)
|
||||
Image.new("RGBA", (3, 2), (10, 80, 160, 255)).save(analysis_path)
|
||||
job_directory = self.storage_root / "mesh-upscale-source-job"
|
||||
def test_generate_region_data_only_returns_requested_outputs(self):
|
||||
job = sys.modules["server.jobs.generate_region_data"]
|
||||
input_path = self.storage_root / "input.png"
|
||||
input_path.write_bytes(b"region")
|
||||
context = SimpleNamespace(
|
||||
directory=job_directory,
|
||||
directory=self.storage_root / "job",
|
||||
progress=lambda *_args: None,
|
||||
check_cancelled=lambda: None,
|
||||
)
|
||||
field_path = job_directory / "field.npz"
|
||||
|
||||
archive = context.directory / "image_geometry.npz"
|
||||
normal = context.directory / "normal.png"
|
||||
with patch.object(
|
||||
mesh_job,
|
||||
"_write_depth_field",
|
||||
return_value=(field_path, None),
|
||||
) as write_surface:
|
||||
result = mesh_job.run(
|
||||
job,
|
||||
"generate_depth",
|
||||
return_value=(archive, normal),
|
||||
) as generate:
|
||||
result = job.run(
|
||||
context,
|
||||
{
|
||||
"input": str(display_path),
|
||||
"analysis_input": str(analysis_path),
|
||||
"shape": "DEPTH_SURFACE",
|
||||
"normal_analysis": False,
|
||||
"video_frames": 1,
|
||||
"input": str(input_path),
|
||||
"output_depth": True,
|
||||
"normal_mode": "OBJECT",
|
||||
},
|
||||
)
|
||||
|
||||
self.assertEqual(result, {"field": "field.npz"})
|
||||
self.assertEqual(write_surface.call_args.args[2], analysis_path)
|
||||
with Image.open(job_directory / "image.0001.png") as color:
|
||||
self.assertEqual(color.size, (12, 8))
|
||||
|
||||
def test_mesh_masked_color_keeps_same_direction_overlap(self):
|
||||
cutout_job = importlib.import_module("server.selection.cutout_images")
|
||||
import numpy as np
|
||||
from PIL import Image
|
||||
|
||||
mesh_job = sys.modules["server.jobs.prepare_geometry_data"]
|
||||
source_path = self.storage_root / "opaque-overlap.png"
|
||||
output_path = self.storage_root / "cutout-overlap.png"
|
||||
Image.new("RGBA", (12, 9), (20, 40, 60, 255)).save(source_path)
|
||||
polygon = (
|
||||
(1, 1),
|
||||
(7, 1),
|
||||
(7, 7),
|
||||
(1, 7),
|
||||
(1, 1),
|
||||
(4, 1),
|
||||
(10, 1),
|
||||
(10, 7),
|
||||
(4, 7),
|
||||
(4, 1),
|
||||
(1, 1),
|
||||
self.assertEqual(
|
||||
result,
|
||||
{"image_geometry": "image_geometry.npz", "normal": "normal.png"},
|
||||
)
|
||||
generate.assert_called_once_with(
|
||||
context,
|
||||
{
|
||||
"input": str(input_path),
|
||||
"output_depth": True,
|
||||
"normal_mode": "OBJECT",
|
||||
},
|
||||
input_path,
|
||||
output_depth=True,
|
||||
normal_mode="OBJECT",
|
||||
)
|
||||
|
||||
cutout_job._write_masked_color(source_path, polygon, output_path)
|
||||
alpha = np.asarray(Image.open(output_path).convert("RGBA"))[:, :, 3]
|
||||
|
||||
self.assertGreater(int(alpha[3, 2]), 200)
|
||||
self.assertGreater(int(alpha[3, 5]), 200)
|
||||
self.assertGreater(int(alpha[3, 8]), 200)
|
||||
def test_mesh_masked_color_preserves_nonzero_winding_hole(self):
|
||||
cutout_job = importlib.import_module("server.selection.cutout_images")
|
||||
import numpy as np
|
||||
from PIL import Image
|
||||
|
||||
mesh_job = sys.modules["server.jobs.prepare_geometry_data"]
|
||||
source_path = self.storage_root / "opaque-hole.png"
|
||||
output_path = self.storage_root / "cutout-hole.png"
|
||||
Image.new("RGBA", (10, 10), (20, 40, 60, 255)).save(source_path)
|
||||
polygon = (
|
||||
(1, 1),
|
||||
(9, 1),
|
||||
(9, 9),
|
||||
(1, 9),
|
||||
(1, 1),
|
||||
(3, 3),
|
||||
(3, 7),
|
||||
(7, 7),
|
||||
(7, 3),
|
||||
(3, 3),
|
||||
(1, 1),
|
||||
def test_generate_region_data_refines_an_already_rasterized_region(self):
|
||||
job = sys.modules["server.jobs.generate_region_data"]
|
||||
input_path = self.storage_root / "input.png"
|
||||
input_path.write_bytes(b"region")
|
||||
directory = self.storage_root / "job"
|
||||
context = SimpleNamespace(
|
||||
directory=directory,
|
||||
progress=lambda *_args: None,
|
||||
check_cancelled=lambda: None,
|
||||
)
|
||||
|
||||
cutout_job._write_masked_color(source_path, polygon, output_path)
|
||||
alpha = np.asarray(Image.open(output_path).convert("RGBA"))[:, :, 3]
|
||||
def generate(_context, _input, _model, _device, output):
|
||||
output.write_bytes(b"refined")
|
||||
return output, (2, 3, 8, 9)
|
||||
|
||||
self.assertGreater(int(alpha[2, 2]), 200)
|
||||
self.assertLess(int(alpha[4, 4]), 32)
|
||||
|
||||
def test_mesh_masked_color_antialiases_the_cutout_edge(self):
|
||||
cutout_job = importlib.import_module("server.selection.cutout_images")
|
||||
import numpy as np
|
||||
from PIL import Image
|
||||
|
||||
mesh_job = sys.modules["server.jobs.prepare_geometry_data"]
|
||||
source_path = self.storage_root / "opaque.png"
|
||||
output_path = self.storage_root / "cutout.png"
|
||||
Image.new("RGBA", (16, 16), (20, 40, 60, 255)).save(source_path)
|
||||
|
||||
cutout_job._write_masked_color(
|
||||
source_path,
|
||||
((1, 1), (14, 1), (1, 14)),
|
||||
output_path,
|
||||
with patch.object(job, "generate_selection", side_effect=generate):
|
||||
result = job.run(
|
||||
context,
|
||||
{
|
||||
"input": str(input_path),
|
||||
"refine_region": True,
|
||||
"refine_model": "BEN2_BASE",
|
||||
"device": "cpu",
|
||||
},
|
||||
)
|
||||
alpha = np.asarray(Image.open(output_path).convert("RGBA"))[:, :, 3]
|
||||
|
||||
self.assertTrue(np.any((alpha > 0) & (alpha < 255)))
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
self.assertEqual(
|
||||
result,
|
||||
{"region": "region.png", "region_bounds": "region_bounds.json"},
|
||||
)
|
||||
self.assertEqual(
|
||||
(directory / "region_bounds.json").read_text(encoding="utf-8"),
|
||||
"[2,3,8,9]",
|
||||
)
|
||||
|
||||
@@ -927,6 +927,13 @@ class WorkerTest(unittest.TestCase):
|
||||
self.assertEqual(result.mode, "RGBA")
|
||||
self.assertEqual(result.size, (20, 12))
|
||||
alpha = np.asarray(result.getchannel("A"))
|
||||
expected_alpha = np.asarray(
|
||||
Image.fromarray(pixels[:, :, 3], mode="L").resize(
|
||||
result.size,
|
||||
Image.Resampling.LANCZOS,
|
||||
)
|
||||
)
|
||||
np.testing.assert_array_equal(alpha, expected_alpha)
|
||||
self.assertLess(
|
||||
int(alpha[:, 0].max()),
|
||||
int(alpha[:, -1].min()),
|
||||
|
||||
@@ -19,9 +19,6 @@ from build_cutout import (
|
||||
)
|
||||
from build_mesh_plane import build_mesh_plane_group
|
||||
from build_shadeless import build_shadeless_group
|
||||
from anyimage.operators.common.node_graph import expand_smooth
|
||||
|
||||
|
||||
OUTPUT_PATH = (
|
||||
PROJECT_ROOT
|
||||
/ "src"
|
||||
@@ -40,7 +37,7 @@ def main():
|
||||
shape: build_node_group(shape)
|
||||
for shape in CUTOUT_NODE_GROUP_NAMES
|
||||
}
|
||||
build_cutout_modifier_group(shape_groups, expand_smooth)
|
||||
build_cutout_modifier_group(shape_groups)
|
||||
OUTPUT_PATH.parent.mkdir(parents=True, exist_ok=True)
|
||||
bpy.ops.wm.save_as_mainfile(filepath=str(OUTPUT_PATH), check_existing=False)
|
||||
|
||||
|
||||
@@ -4,6 +4,8 @@ import math
|
||||
|
||||
import bpy
|
||||
|
||||
from node_graph import expand_smooth
|
||||
|
||||
from anyimage.operators.cutout_tool.shape_types import (
|
||||
BALLOON_ATTRIBUTE_NAME,
|
||||
CUTOUT_NODE_GROUP_NAME,
|
||||
@@ -727,7 +729,7 @@ def build_node_group(shape):
|
||||
options = group.interface.new_panel(name="Options", default_closed=True)
|
||||
reference_depth = _float_input(
|
||||
group,
|
||||
"Reference Depth",
|
||||
"Base Plane Depth",
|
||||
1.0,
|
||||
0.0,
|
||||
None,
|
||||
@@ -751,7 +753,7 @@ def build_node_group(shape):
|
||||
options = group.interface.new_panel(name="Options", default_closed=True)
|
||||
reference_depth = _float_input(
|
||||
group,
|
||||
"Reference Depth",
|
||||
"Base Plane Depth",
|
||||
1.0,
|
||||
0.0,
|
||||
None,
|
||||
@@ -788,7 +790,7 @@ def build_node_group(shape):
|
||||
"Depth Image",
|
||||
"Depth Scale",
|
||||
"Uniform Scale",
|
||||
"Reference Depth",
|
||||
"Base Plane Depth",
|
||||
"Depth Direction",
|
||||
},
|
||||
)
|
||||
@@ -813,7 +815,7 @@ def build_node_group(shape):
|
||||
controls.outputs["Uniform Scale"],
|
||||
controls.outputs["Depth Direction"],
|
||||
controls.outputs["Depth Image"],
|
||||
controls.outputs["Reference Depth"],
|
||||
controls.outputs["Base Plane Depth"],
|
||||
)
|
||||
output_location = (5100.0, 600.0)
|
||||
else:
|
||||
@@ -823,7 +825,7 @@ def build_node_group(shape):
|
||||
controls.outputs["Thickness"],
|
||||
controls.outputs["Depth Scale"],
|
||||
controls.outputs["Uniform Scale"],
|
||||
controls.outputs["Reference Depth"],
|
||||
controls.outputs["Base Plane Depth"],
|
||||
controls.outputs["Depth Image"],
|
||||
)
|
||||
output_location = (8200.0, 400.0)
|
||||
@@ -844,7 +846,7 @@ def build_node_group(shape):
|
||||
return group
|
||||
|
||||
|
||||
def build_cutout_modifier_group(shape_groups, expand_smooth):
|
||||
def build_cutout_modifier_group(shape_groups):
|
||||
group = bpy.data.node_groups.get(CUTOUT_NODE_GROUP_NAME)
|
||||
if group is not None:
|
||||
return group
|
||||
@@ -896,16 +898,14 @@ def build_cutout_modifier_group(shape_groups, expand_smooth):
|
||||
|
||||
reference_depth = _float_input(
|
||||
group,
|
||||
"Reference Depth",
|
||||
"Base Plane Depth",
|
||||
1.0,
|
||||
0.0,
|
||||
None,
|
||||
subtype="DISTANCE",
|
||||
parent=options,
|
||||
)
|
||||
reference_depth.description = (
|
||||
"Metric reference depth shared by both Depth shapes"
|
||||
)
|
||||
reference_depth.description = "Depth of the flat base plane for both Depth shapes"
|
||||
depth_direction = _vector_input(
|
||||
group,
|
||||
"Depth Direction",
|
||||
@@ -936,10 +936,13 @@ def build_cutout_modifier_group(shape_groups, expand_smooth):
|
||||
(-2700.0, 500.0),
|
||||
{"Geometry", "Cleanup"},
|
||||
)
|
||||
cleanup_area = _math(group.nodes, "MULTIPLY", (-2500.0, 750.0))
|
||||
cleanup_area.inputs[1].default_value = 0.01
|
||||
group.links.new(base_controls.outputs["Cleanup"], cleanup_area.inputs[0])
|
||||
geometry = _island_cleanup(
|
||||
group,
|
||||
base_controls.outputs["Geometry"],
|
||||
base_controls.outputs["Cleanup"],
|
||||
cleanup_area.outputs[0],
|
||||
(-2350.0, 250.0),
|
||||
)
|
||||
|
||||
|
||||
@@ -1,14 +1,4 @@
|
||||
"""Shared Geometry Nodes construction helpers."""
|
||||
|
||||
def interface_socket(group, name, in_out, socket_type):
|
||||
socket = group.interface.new_socket(
|
||||
name=name,
|
||||
in_out=in_out,
|
||||
socket_type=socket_type,
|
||||
)
|
||||
if in_out == "INPUT" and hasattr(socket, "structure_type"):
|
||||
socket.structure_type = "SINGLE"
|
||||
return socket
|
||||
"""Node graph builders used only while generating bundled assets."""
|
||||
|
||||
|
||||
def group_input(nodes, location, visible_names):
|
||||
@@ -189,4 +179,3 @@ def expand_smooth(
|
||||
node.location.x += x_offset
|
||||
return repeat_output.outputs["Geometry"]
|
||||
|
||||
|
||||
@@ -11,10 +11,8 @@ sys.path.insert(
|
||||
str(next((PROJECT_ROOT / "src" / "anyimage" / "wheels").glob("blendjob-*.whl"))),
|
||||
)
|
||||
|
||||
from anyimage.operators.common.node_asset import (
|
||||
from anyimage.operators.common.mesh_result import (
|
||||
load_asset_node_group,
|
||||
load_mesh_plane_node_group,
|
||||
load_shadeless_node_group,
|
||||
)
|
||||
from anyimage.operators.cutout_tool.shape_types import (
|
||||
CUTOUT_NODE_GROUP_NAME,
|
||||
@@ -32,8 +30,8 @@ def input_names(group):
|
||||
|
||||
def main():
|
||||
bpy.ops.wm.read_factory_settings(use_empty=True)
|
||||
mesh_plane = load_mesh_plane_node_group()
|
||||
shadeless = load_shadeless_node_group()
|
||||
mesh_plane = load_asset_node_group("O Mesh Plane")
|
||||
shadeless = load_asset_node_group("O Shadeless")
|
||||
cutout = load_asset_node_group(CUTOUT_NODE_GROUP_NAME)
|
||||
cutout_groups = {
|
||||
name: bpy.data.node_groups[name]
|
||||
|
||||
@@ -13,7 +13,7 @@ sys.path.insert(
|
||||
|
||||
from anyimage.operators.cutout_tool import node_assets as cutout_nodes
|
||||
from anyimage.operators.cutout_tool import shape_types as shapes
|
||||
from anyimage.operators.common.node_asset import (
|
||||
from anyimage.operators.common.mesh_result import (
|
||||
modifier_input_identifier,
|
||||
set_modifier_input,
|
||||
)
|
||||
@@ -125,7 +125,7 @@ def main():
|
||||
"Smooth Iterations",
|
||||
"Cleanup",
|
||||
"Smooth",
|
||||
"Reference Depth",
|
||||
"Base Plane Depth",
|
||||
"Depth Direction",
|
||||
"Uniform Scale",
|
||||
"Depth Image",
|
||||
@@ -138,7 +138,7 @@ def main():
|
||||
"Thickness",
|
||||
"Depth Scale",
|
||||
"Double Sided",
|
||||
"Reference Depth",
|
||||
"Base Plane Depth",
|
||||
"Depth Direction",
|
||||
"Uniform Scale",
|
||||
"Depth Image",
|
||||
@@ -147,7 +147,7 @@ def main():
|
||||
"Geometry",
|
||||
"Thickness",
|
||||
"Depth Scale",
|
||||
"Reference Depth",
|
||||
"Base Plane Depth",
|
||||
"Uniform Scale",
|
||||
"Depth Image",
|
||||
],
|
||||
@@ -171,7 +171,7 @@ def main():
|
||||
"Thickness": set(shapes.CUTOUT_SHAPES),
|
||||
"Depth Scale": {"DEPTH_BALLOON", "DEPTH_SURFACE"},
|
||||
"Double Sided": {"DEPTH_BALLOON"},
|
||||
"Reference Depth": {"DEPTH_BALLOON", "DEPTH_SURFACE"},
|
||||
"Base Plane Depth": {"DEPTH_BALLOON", "DEPTH_SURFACE"},
|
||||
"Depth Direction": {"DEPTH_BALLOON"},
|
||||
"Uniform Scale": {"DEPTH_BALLOON", "DEPTH_SURFACE"},
|
||||
"Depth Image": {"DEPTH_BALLOON", "DEPTH_SURFACE"},
|
||||
@@ -230,6 +230,15 @@ def main():
|
||||
)
|
||||
assert cleanup.default_value == 0.0
|
||||
assert cleanup.subtype == "NONE"
|
||||
cleanup_scale = next(
|
||||
link.to_node
|
||||
for link in wrapper.links
|
||||
if link.from_socket.name == "Cleanup"
|
||||
and link.from_node.bl_idname == "NodeGroupInput"
|
||||
)
|
||||
assert cleanup_scale.bl_idname == "ShaderNodeMath"
|
||||
assert cleanup_scale.operation == "MULTIPLY"
|
||||
assert abs(cleanup_scale.inputs[1].default_value - 0.01) < 1e-7
|
||||
assert smooth.default_value == 1.0
|
||||
assert iterations.default_value == 0
|
||||
assert iterations.min_value == 0
|
||||
@@ -341,7 +350,7 @@ def main():
|
||||
("Depth Image", point_image),
|
||||
("Depth Scale", 1.0),
|
||||
("Uniform Scale", 1.0),
|
||||
("Reference Depth", 2.0),
|
||||
("Base Plane Depth", 2.0),
|
||||
("Depth Direction", (0.0, 0.0, 1.0)),
|
||||
):
|
||||
set_modifier_input(
|
||||
@@ -392,7 +401,7 @@ def main():
|
||||
("Depth Image", point_image),
|
||||
("Depth Scale", 1.0),
|
||||
("Uniform Scale", 1.0),
|
||||
("Reference Depth", 2.0),
|
||||
("Base Plane Depth", 2.0),
|
||||
("Smooth Iterations", 0),
|
||||
):
|
||||
set_modifier_input(
|
||||
|
||||
@@ -11,8 +11,7 @@ sys.path.insert(
|
||||
str(next((PROJECT_ROOT / "src" / "anyimage" / "wheels").glob("blendjob-*.whl"))),
|
||||
)
|
||||
|
||||
from anyimage.operators.common.node_asset import (
|
||||
get_modifier_input,
|
||||
from anyimage.operators.common.mesh_result import (
|
||||
modifier_input_identifier,
|
||||
set_modifier_input,
|
||||
)
|
||||
@@ -28,6 +27,13 @@ def evaluated_mesh(obj):
|
||||
return evaluated, evaluated.to_mesh()
|
||||
|
||||
|
||||
def modifier_input_value(modifier, identifier):
|
||||
inputs = getattr(getattr(modifier, "properties", None), "inputs", None)
|
||||
if inputs is not None:
|
||||
return getattr(inputs, identifier).value
|
||||
return modifier[identifier]
|
||||
|
||||
|
||||
def main():
|
||||
bpy.ops.wm.read_factory_settings(use_empty=True)
|
||||
mesh = create_mesh_plane_mesh("Mesh Plane", (-1.0, 1.0, -0.5, 0.5))
|
||||
@@ -49,7 +55,7 @@ def main():
|
||||
assert inputs["Subdivide"].max_value == 10
|
||||
assert inputs["Thickness"].default_value == 0.0
|
||||
assert inputs["Material"].hide_in_modifier
|
||||
assert get_modifier_input(modifier, inputs["Material"].identifier) == material
|
||||
assert modifier_input_value(modifier, inputs["Material"].identifier) == material
|
||||
|
||||
evaluated, result = evaluated_mesh(obj)
|
||||
try:
|
||||
|
||||
Reference in New Issue
Block a user