Add new functionality for image rectification and depth metadata handling
- Implemented a preview tool for rectifying images with interactive aspect controls in `preview.py`. - Added tests for depth metadata loading to ensure consistent schema usage in `test_depth_metadata.py`. - Developed comprehensive tests for cutout calibration processes in `test_cutout_calibration.py`, including stability checks and depth calibration. - Created tests for frame compositing operations in `test_frame_compositing.py`, ensuring correct handling of depth and transparency. - Introduced material slot inheritance functionality in `material_slots.py` to manage geometry material assignments. - Built shared shader nodes for material processing in `shader.py`, including roughness and bump mapping setups.
This commit is contained in:
3
.env.example
Normal file
3
.env.example
Normal file
@@ -0,0 +1,3 @@
|
||||
# 复制为 .env，并填写本机 Blender 可执行文件路径。
|
||||
# Windows 路径可使用正斜杠。
|
||||
BLENDER_BIN="C:/Program Files/Blender Foundation/Blender 4.5/blender.exe"
|
||||
@@ -7,7 +7,7 @@
|
||||
- 说明性内容以中文为主，专有名词和约定俗成的表达可使用英文
|
||||
- 修改代码、依赖或执行方式后，
|
||||
- 不保留旧路径、转发函数、兼容层或胶水代码
|
||||
- 更新并运行相关测试，但不主动同步说明文档、构建文档或打包扩展
|
||||
- 仅更新并运行相关测试，但不主动同步说明文档、构建文档或打包扩展
|
||||
- 优先选择依赖少、直接、易验证的实现
|
||||
- 复用来自 `common` 的函数，避免重复造轮子
|
||||
- 确保文件名/文件夹名和内容边界一致
|
||||
@@ -75,7 +75,7 @@ uv run sync-models
|
||||
|
||||
- `README.md` 面向用户，不在其中堆叠内部细节。实现原理放入 `docs/internals` 对应主题
|
||||
- 文档只正面描述当前主题的职责与实现，不记录无关模块、未采用路径、历史差异、边界辩解、“不做什么”的排除性信息。
|
||||
- 实现原理用流程图替代文字，Mermaid 默认使用从上到下
|
||||
- 根据内容选择最清晰的表达方式，优先简洁文字；复杂流程或关系在有助于理解时使用图表。Mermaid 流程图默认从上到下。
|
||||
|
||||
## Blender 开发规范
|
||||
|
||||
|
||||
@@ -3,7 +3,7 @@
|
||||
| 模块 | 职责 |
|
||||
| --- | --- |
|
||||
| `__init__.py` | 注册类型、工具、菜单、快捷键与 Runtime |
|
||||
| `menu.py`、`panel.py` | Image Empty 上下文菜单与 Job Server 面板 |
|
||||
| `menu.py`、`panel.py` | Image Empty 与 Shader Image Texture 上下文菜单与 Job Server 面板 |
|
||||
| `preferences.py` | 扩展设置及读取入口 |
|
||||
| `properties.py` | Scene 手势状态、视频帧数、模型状态与枚举 |
|
||||
| `common/` | Blender 侧公共能力 |
|
||||
@@ -12,15 +12,6 @@
|
||||
|
||||
## 注册与配置
|
||||
|
||||
```mermaid
|
||||
flowchart TD
|
||||
A[register] --> B[按 CLASSES 顺序注册类型]
|
||||
B --> C[注册 Frame、Mask、Rectify、Cutout 工具]
|
||||
C --> D[挂载 Scene.anyimage_settings]
|
||||
D --> E[挂载 View3D 与 Outliner 菜单及剪贴板快捷键]
|
||||
E --> F[注册 JobRuntime]
|
||||
```
|
||||
|
||||
注销先关闭 Runtime 和交互入口，再移除 Scene 属性，按 `CLASSES` 逆序注销类型。设备、存储目录、模型、材质显示适配和功能上限统一通过 `preferences.py` 读取。
|
||||
|
||||
**Maximum AI Input Size** 限制模型分析图和 Upscale 可提交源图；**Maximum Frame Resolution** 限制 Frame 输出；**Maximum Cutout Mesh Resolution** 限制轮廓网格采样。三者各自作用于对应数据阶段。
|
||||
@@ -40,4 +31,4 @@ flowchart TD
|
||||
|
||||
Plane、Frame、Mask、Rectify 和无需 AI 的 Cutout 在 Blender 内执行。Server 的 `jobs/` 编排任务，`models/` 负责模型适配，`geometry/` 处理深度与法线产物。Server 模块通过内部相对导入共享能力，Blender 侧通过 `common/` 复用。
|
||||
|
||||
MoGe-2 输出统一为 `GeometryFrame`：包含 Depth、Validity、Intrinsics，以及按需提供的 Normal 和 Points。产物写入器消费同一个 Frame，保持深度、法线和相机数据一致。参见 [Runtime](runtime.md) 与 [产物协议](outputs.md)。
|
||||
MoGe-2 与 MoGe-3 输出统一为 `GeometryFrame`：包含 Depth、Validity、Intrinsics，以及按需提供的 Normal 和 Points。产物写入器消费同一个 Frame，保持深度、法线和相机数据一致。参见 [Runtime](runtime.md) 与 [产物协议](outputs.md)。
|
||||
|
||||
@@ -6,6 +6,9 @@
|
||||
| --- | --- |
|
||||
| `ai.py` | 环境和模型就绪检查、输入路径校验、`production_device()`、`moge2_parameters()` 与安装 UI |
|
||||
| `image.py` | `image_rgba()`、动画帧设置、Job 输入准备、像素采样、结果 Image 和 `replace_empty_image()` |
|
||||
| `color_image.py` | 材质颜色与 AI 分析输入的准备和清理 |
|
||||
| `hdr_image.py` | HDR 去背景分析输入及浮点颜色保留 |
|
||||
| `image_target.py` | Image Empty 与 Shader Image Texture 编辑目标 |
|
||||
| `selection.py` | `SelectionPath`、`SelectionMask`、non-zero winding 栅格化与 Alpha 修整 |
|
||||
| `viewport.py` | 原生点选、屏幕投影、`ImageGesture`、Lasso / Brush / Polyline 与 Overlay |
|
||||
| `depth.py` | 深度图片和元数据加载、相机坐标与显示范围的尺度换算 |
|
||||
@@ -15,22 +18,9 @@
|
||||
|
||||
## 图像生命周期
|
||||
|
||||
```mermaid
|
||||
flowchart TD
|
||||
A[Image 与 Image Empty] --> B[image_rgba 读取业务像素]
|
||||
B --> C{处理位置}
|
||||
C -- 本地 --> D[像素算法]
|
||||
C -- Server --> E[prepare_image_input 或 prepare_image_bounds_input]
|
||||
E --> F[Job 输出文件]
|
||||
F --> G[加载并 Pack]
|
||||
D --> G
|
||||
G --> H[replace_empty_image]
|
||||
H --> I[保留身份、共享引用、帧设置和画框]
|
||||
```
|
||||
|
||||
业务像素采用 Top-down RGBA，`image_pixels()` 使用 Blender 像素顺序。预乘采样用于投影插值；文件或 Packed 图像的业务读取保留隐藏 RGB。直接编辑结果通过统一入口处理共享 Image、失败恢复和源数据清理。
|
||||
|
||||
`prepare_image_input()` 返回路径及临时标记；`cleanup_image_input()` 与创建方配对。Cutout 的 `prepare_image_bounds_input()` 复用已读取的源 RGBA，输出指定几何 bounds 的矩形图片。
|
||||
`prepare_image_input()` 返回路径及临时标记；`cleanup_image_input()` 与创建方配对。`color_image.py` 的 `prepare_material_color_input()` 准备材质颜色，`material_analysis_input()` 提供分析输入；Cutout 按几何 bounds 裁切。
|
||||
|
||||
## 手势与 Selection
|
||||
|
||||
|
||||
@@ -15,11 +15,11 @@ uv run pytest
|
||||
| Frame、Mask、Rectify、Plane、Cutout | 对应 `test_*_tool.py`、`test_*_operators.py` 与对象、投影、网格测试 |
|
||||
| 图像身份、Alpha、材质与 Undo | 图像编辑集成、共享替换、材质适配与 Undo 测试 |
|
||||
| Server | `test_job_server.py`、`test_server_jobs.py`、`test_media_input.py`、`test_geometry_artifacts.py` |
|
||||
| 模型 | `test_model_manager.py`、目录、下载、推理与各 ONNX 模型测试 |
|
||||
| 节点资产 | 布局检查、Depth Plane 模式、深度分离与 Cutout 体积求值 |
|
||||
| 工具与包协议 | 扩展 Manifest、打包、模型同步、节点启动器和文档命令测试 |
|
||||
| 模型 | `test_model_manager.py`、下载、推理与各 ONNX 模型测试 |
|
||||
| 节点资产 | 布局前后功能等价、深度分离与 Cutout 体积求值 |
|
||||
| 工具与包协议 | 扩展 Manifest、打包、模型同步与节点启动器 |
|
||||
|
||||
Blender Stub 共用 `blender_test_support.py`；Server 和实际 Blender 几何 fixture 按使用范围存放在对应 support 模块。测试命名描述功能、触发条件与可观察结果。
|
||||
Blender Stub 共用 `tests/support/blender.py`；Server 和实际 Blender 几何 fixture 按使用范围存放在对应 support 模块。测试命名描述功能、触发条件与可观察结果。
|
||||
|
||||
## 发布命令
|
||||
|
||||
@@ -27,14 +27,14 @@ Blender Stub 共用 `blender_test_support.py`；Server 和实际 Blender 几何
|
||||
| --- | --- |
|
||||
| `uv run pack` | 生成 Windows、macOS arm64、Linux 扩展包 |
|
||||
| `uv run pack --prepare` | 为本机同步扩展 wheel 与 Manifest |
|
||||
| `uv run build-nodes` | 构建、验证节点资产并运行测试 |
|
||||
| `uv run sync-models` | 准备、校验并上传模型声明的文件 |
|
||||
| `uv run node-group build` | 构建、验证节点资产并运行测试 |
|
||||
| `uv run model sync` | 准备、校验并上传模型声明的文件 |
|
||||
| `uv run docs -dev` | 本地预览文档 |
|
||||
| `uv run docs` | 通过 Mike 发布项目版本并更新 latest |
|
||||
|
||||
`tools/pack.py` 从 `pyproject.toml` 读取依赖，按平台与 Blender Python 版本解析 wheel，使用 Blender 提供的 NumPy。产物命名为 `AnyImage.v<version>.<platform>.zip`，写入 `dist/`。本地 BlendJob wheel 内容变化后运行 `uv lock --refresh`，并核对锁文件、Manifest 与打包测试。
|
||||
|
||||
`tools/models/cli.py` 根据模型目录选择上游 ONNX 或本地导出流程；`tools/models/exports.py` 处理对应 PyTorch 权重导出。文件校验依据声明大小和 SHA-256，上传使用逐文件复制。`--prepare-only` 用于本地准备。
|
||||
`tools/models/cli.py` 根据模型目录选择上游 ONNX 或本地导出流程；`tools/models/exports.py` 处理对应 PyTorch 权重导出。文件校验依据声明大小和 SHA-256，上传使用逐文件复制。`uv run model prepare` 用于本地准备。
|
||||
|
||||
## CI 与文档导航
|
||||
|
||||
|
||||
@@ -2,16 +2,6 @@
|
||||
|
||||
AnyImage 在 Blender 主线程处理交互、像素编辑和数据创建，通过 BlendJob 把模型推理交给独立 Job Server。
|
||||
|
||||
```mermaid
|
||||
flowchart TD
|
||||
A[Image Empty 或剪贴板图片] --> B[Operator 与手势工具]
|
||||
B --> C{需要 AI？}
|
||||
C -- 是 --> D[准备文件与普通字典参数]
|
||||
D --> E[JobRuntime → Job Server]
|
||||
E --> F[图片与相机元数据]
|
||||
F --> G[Blender 主线程导入]
|
||||
C -- 否 --> G
|
||||
G --> H[Image 编辑或 Mesh 与材质创建]
|
||||
```
|
||||
|
||||
| 主题 | 对应职责 |
|
||||
|
||||
@@ -1,11 +1,26 @@
|
||||
# 节点资产
|
||||
|
||||
## 资产清单
|
||||
|
||||
节点组文件在 `tools/nodes/groups`
|
||||
|
||||
| 节点组 | 构建模块 |
|
||||
| ------------------------ | ------------------------- |
|
||||
| `O Image Plane` | `image_plane.py` |
|
||||
| `O Image Depth Plane` | `image_depth_plane.py` |
|
||||
| `O Image Relief Plane` | `image_relief_plane.py` |
|
||||
| `O Image Layer` | `image_layer.py` |
|
||||
| `O Image Depth Layer` | `image_depth_layer.py` |
|
||||
| `O Shadeless` | `shadeless.py` |
|
||||
| `O Image Cutout` | `image_cutout.py` |
|
||||
| `O Image Depth Cutout` | `image_depth_cutout.py` |
|
||||
| `O Image Depth Balloon` | `image_depth_balloon.py` |
|
||||
| `O Image Depth Panorama` | `image_depth_panorama.py` |
|
||||
|
||||
## 构建节点资产
|
||||
|
||||
生产节点组由 `uv run node-group build` 构建到 `src/anyimage/assets/O_AnyImage.blend`
|
||||
|
||||
节点组文件在 `tools/nodes/groups`
|
||||
|
||||
节点组规则：
|
||||
|
||||
- 字段默认直接连接，多个消费者可复用同一输出；按消费位置的几何、域、类型和输入属性版本验证求值等价性
|
||||
@@ -20,52 +35,9 @@
|
||||
- socket `subtype` 按业务语义设置：长度用 `DISTANCE`，`0–1` 比例用 `FACTOR`，`0–100` 用 `PERCENTAGE`；无对应范围语义时保留默认
|
||||
- 接口或布局变更须验证 socket 顺序与结构、未使用输出隐藏及几何求值；纯布局变更比较运算、默认值和实际连接，确认功能不变
|
||||
|
||||
```mermaid
|
||||
flowchart TD
|
||||
A[字段生产与消费] --> B{需要按名称访问}
|
||||
B -->|是| C[命名属性]
|
||||
B -->|否| D{目标上下文重新求值是否等价}
|
||||
D -->|是| E[字段直连并显式转换域]
|
||||
D -->|否| F[调整消费顺序或在源几何采样]
|
||||
F --> G{仍需保存源数据}
|
||||
G -->|否| E
|
||||
G -->|是| H[仅存储必要命名数据]
|
||||
H --> I[最后消费者后精确清理]
|
||||
```
|
||||
|
||||
## 字段与临时属性
|
||||
|
||||
平滑循环按固定源位置查询原边界索引，并在当前边界网格上消费位置；拓扑不变的边界与影响权重直接求值。Depth Plane 由 UV 矩形判断原边界，Panorama 的闭合源球面以 False 表示原边界。
|
||||
|
||||
厚度构建使用源顶点索引排列构造网格，完成挤出与合并后按索引从源表面采样位置、厚度和法线。Depth Cutout 的中心、圆化、结构及膨胀字段在投影源表面求值。
|
||||
|
||||
| 属性 | 域 / 类型 | 写入与最后消费 |
|
||||
| --- | --- | --- |
|
||||
| `.o_anyimage_depth_cut` | CORNER / BOOLEAN | 三个深度组在 Split Edges 前保存切分标记，供投影调整射线，投影后精确清理 |
|
||||
| `.o_anyimage_original_boundary` | POINT / BOOLEAN | Depth Cutout 在切分前保存任意输入轮廓，供切口平滑保护原边界，平滑后精确清理 |
|
||||
| `UVMap` | CORNER / FLOAT2 | Image Plane 与 Panorama 的输出 UV，供材质读取 |
|
||||
| `o_normal_scale` | POINT 或 FACE / FLOAT | Cutout 输出法线掩码，Depth Cutout 合并背面与侧壁后统一写入，供形状和材质消费 |
|
||||
|
||||
|
||||
## 资产清单
|
||||
|
||||
| 节点组 | 构建模块 |
|
||||
| --- | --- |
|
||||
| `O Image Plane` | `image_plane.py` |
|
||||
| `O Image Depth Plane` | `image_depth_plane.py` |
|
||||
| `O Image Relief Plane` | `image_relief_plane.py` |
|
||||
| `O Image Layer` | `image_layer.py` |
|
||||
| `O Image Depth Layer` | `image_depth_layer.py` |
|
||||
| `O Shadeless` | `shadeless.py` |
|
||||
| `O Image Cutout` | `image_cutout.py` |
|
||||
| `O Image Depth Cutout` | `image_depth_cutout.py` |
|
||||
| `O Image Depth Balloon` | `image_depth_balloon.py` |
|
||||
| `O Image Depth Panorama` | `image_depth_panorama.py` |
|
||||
|
||||
|
||||
## 构建资产脚本
|
||||
|
||||
`tools/nodes/build.py` 按依赖顺序构建资产，调用 `arrangement/` 统一排列几何节点组后保存。`common/` 提供共用构建函数，`preview/` 导出交互预览。
|
||||
`tools/nodes/build.py` 按依赖顺序构建资产，调用 `arrangement/` 统一排列几何节点组后保存。`common/` 提供共用构建函数，其中 `material_slots.py` 继承几何材质槽，`shader.py` 构建材质着色节点，`preview/` 导出交互预览。
|
||||
|
||||
保存后，`check.py` 在独立 Blender 进程中重新加载资产并做基本几何求值。接口、默认值和详细几何行为由 `tests/` 检查。
|
||||
|
||||
|
||||
@@ -6,7 +6,7 @@ AI 环境管理从当前安装状态出发，补齐独立运行环境和必需
|
||||
|
||||
```mermaid
|
||||
flowchart TD
|
||||
Entry["Set Up AI Environment"] --> Status["读取环境与必需模型状态"]
|
||||
Entry["Set Up AI Server"] --> Status["读取环境与必需模型状态"]
|
||||
Status --> Ready{"是否全部就绪？"}
|
||||
Ready -- "是" --> Done["结束"]
|
||||
Ready -- "否" --> Environment{"运行环境是否就绪？"}
|
||||
@@ -29,7 +29,7 @@ flowchart TD
|
||||
|
||||
环境尚未就绪时，`SetupAIEnvironment` 调用 Runtime 注册的安装 Operator。`use_mirror` 决定 uv、Python 和 Python 包的下载源，模型使用模型目录中声明的来源。
|
||||
|
||||
安装完成后继续进入模型下载，不在 Blender Python 中加载 ONNX Runtime 或模型依赖。
|
||||
安装完成后通过 `post_install()` 下载默认模型，模型依赖加载于独立 Server 环境。
|
||||
|
||||
### 下载模型
|
||||
|
||||
@@ -39,4 +39,4 @@ Job 在 `server/app.py` 注册；`server/model_catalog.py` 保存模型名称、
|
||||
|
||||
### 释放模型缓存
|
||||
|
||||
`ai_setup.py` 中的 `ClearModels` 调用 `runtime.clear_resource("model_manager")`，释放 Server 中已加载的模型 Session。它不会删除磁盘上的模型文件，下次推理仍可从现有文件重新加载。
|
||||
`ai_setup.py` 中的 `ClearModels` 调用 `runtime.clear_resource("model_manager")`，释放 Server 中已加载的模型 Session。下次推理从模型目录重新加载 Session。
|
||||
|
||||
14
docs/internals/operators/convert-to-panorama.md
Normal file
14
docs/internals/operators/convert-to-panorama.md
Normal file
@@ -0,0 +1,14 @@
|
||||
# Panorama 转换
|
||||
|
||||
`operators/convert_to_panorama/` 将静态 2:1 等距柱状 Image Empty 转换为径向深度表面。`operators.py` 提交任务，`object.py` 导入对象，`__init__.py` 汇总导出与注册清单。
|
||||
|
||||
```mermaid
|
||||
flowchart TD
|
||||
A[准备颜色与分析图] --> B[generate-panorama-geometry]
|
||||
B --> C[采样 12 个透视视图并推理]
|
||||
C --> D[融合径向距离与有效性]
|
||||
D --> E[写出 depth.exr 与 depth.json]
|
||||
E --> F[创建材质与 O Image Depth Panorama]
|
||||
```
|
||||
|
||||
任务使用 Preferences 选择的几何模型。源颜色由 Blender 侧保存，Server 的 `geometry/panorama.py` 负责视图采样与距离融合，`jobs/panorama.py` 编排推理和产物写入。
|
||||
@@ -1,23 +1,21 @@
|
||||
# Plane 转换
|
||||
|
||||
`operators/convert_to_plane/operators.py` 提供 `ConvertToPlane`、`ConvertToDepthPlane` 与 `GenerateDepthPlane`。对象和材质导入位于 `object.py`，基础矩形网格位于 `image_plane.py`。
|
||||
`operators/convert_to_plane/operators.py` 提供 `ConvertToPlane`、`ConvertToDepthPlane`、`ConvertToReliefPlane` 与 `GenerateDepthPlane`。`object.py` 导入对象和材质，`image_plane.py` 构建基础矩形网格，`__init__.py` 汇总导出与注册清单。
|
||||
|
||||
```mermaid
|
||||
flowchart TD
|
||||
A[Image Empty] --> B{转换类型}
|
||||
B -- Plane --> C[复制 Color 与动画设置]
|
||||
C --> D[中心原点四边形与 O Image Plane]
|
||||
B -- Depth Plane --> E[检查静态图与 AI 就绪状态]
|
||||
E --> F[generate-depth-plane-geometry]
|
||||
F --> G[Color、depth.exr、depth.json、Tangent Normal]
|
||||
G --> H[加载并 Pack 数据]
|
||||
H --> I[O Image Depth Plane 与材质]
|
||||
D --> J[替换源 Empty 并保留世界位置]
|
||||
I --> J
|
||||
A[Image Empty] --> B[准备材质颜色]
|
||||
B --> C{转换类型}
|
||||
C -- Plane --> D[四边形与 O Image Plane]
|
||||
C -- Depth 或 Relief --> E[静态图提交几何 Job]
|
||||
E --> F[加载深度、元数据与法线]
|
||||
F --> G[对应节点修改器与材质]
|
||||
D --> H[创建对象并替换源 Empty]
|
||||
G --> H
|
||||
```
|
||||
|
||||
普通 Plane 支持图片、视频和序列，材质保留 Image User 帧设置。基础四边形以局部中心为原点，对象矩阵补偿原 Image Empty 显示范围；节点组生成细分与厚度，材质位于 Mesh 第 0 个槽。
|
||||
普通 Plane 支持图片、视频和序列，材质保留 Image User 帧设置。基础网格以局部中心为原点，对象矩阵补偿源 Empty 的显示范围。
|
||||
|
||||
Depth Plane 从 Preferences 取得 MoGe-2 模型、分辨率级别与 AI 输入上限，一次推理生成相机 XYZ 深度和切线法线。`create_depth_plane_from_result()` 创建颜色材质与深度修改器，并在失败时清理未使用的数据块。
|
||||
Depth 与 Relief 从 Preferences 读取几何模型、分辨率级别和 AI 输入上限。`create_depth_plane_from_result()` 导入产物并创建修改器。`O Image Depth Plane` 使用相机 XYZ 与对象空间法线构建深度表面；`O Image Relief Plane` 使用深度 Z 与切线空间法线构建浮雕。
|
||||
|
||||
`O Image Depth Plane` 提供默认 Relief 和 Camera：Relief 读取深度 Z 形成固定底面的地形；Camera 使用相机 XYZ 投影并沿平滑法线形成壳。两个模式各自保存 Thickness。完整接口见[节点资产](../node-assets.md)，文件协议见[Job 产物](../outputs.md)。
|
||||
节点构建见[节点资产](../node-assets.md)，文件协议见[Job 产物](../outputs.md)。
|
||||
|
||||
@@ -4,7 +4,7 @@ Cutout 在静态 Image Empty 上圈定 Selection，按轮廓创建 Flat、Solid
|
||||
|
||||
```mermaid
|
||||
flowchart TD
|
||||
A[Lasso 拖动并提交 SelectionPath] --> B[选择 Shape]
|
||||
A[Lasso 或 Polyline 提交 SelectionPath] --> B[选择 Shape]
|
||||
B --> C[栅格化 SelectionMask 并读取源 RGBA]
|
||||
C --> D{需要 Depth 或 Normal？}
|
||||
D -- 是 --> E[准备 bounds 矩形 RGBA]
|
||||
@@ -28,11 +28,13 @@ flowchart TD
|
||||
|
||||
`geometry.py` 从内容 Alpha 提取连通轮廓和孔洞，按 Mesh Detail 采样并生成 `BaseShape`。Low、Medium、High、Ultra 分别以 32、16、8、4 个源像素作为基础间距，并受 Maximum Cutout Mesh Resolution 限制。
|
||||
|
||||
`fine_mesh.py` 负责 Fine Outline 的细节保留与约束网格，`polygon.py` 提供轮廓运算。Balloon Profile 按 Preferences 选择 Poisson 或 Teddy，生成所有预设共用的 POINT 属性 `o_balloon`。
|
||||
`fine_mesh.py` 负责 Fine Outline 的细节保留与约束网格，`polygon.py` 提供轮廓运算。`balloon.py` 负责 Balloon Profile，按 Preferences 选择 Poisson 或 Teddy，生成所有预设共用的 POINT 属性 `o_balloon`。
|
||||
|
||||
`depth_calibration.py` 负责深度参考平面、尺度与方向校准；`geometry.py` 保留轮廓和网格生成。
|
||||
|
||||
## AI 与对象创建
|
||||
|
||||
Server 接收 Selection bounds 内保留源 Alpha 的 RGBA，一次 MoGe-2 推理按 `generate_depth` 与 `normal_mode` 返回产物。深度边缘延展使用所选 Alpha Threshold，Normal 使用原预测；AI 输入上限只影响分析分辨率。
|
||||
Server 接收 Selection bounds 内保留源 Alpha 的 RGBA，一次 MoGe 推理按 `generate_depth` 与 `normal_mode` 返回产物。深度参考区域使用所选 Alpha Threshold，Normal 使用原预测；AI 输入上限只影响分析分辨率。
|
||||
|
||||
`object.py` 的 `create_shape_object()` 消费 BaseShape、Color、Depth 和 Normal，加载对应节点组。基础网格位于局部 XY，向图片内部为 -Z；选择 +X 时节点与对象矩阵配对变换，保持世界几何一致。Object Normal 同步按方向转换。
|
||||
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
# Frame 多图合成
|
||||
|
||||
`operators/frame_tool.py` 定义 `FrameImages` 和 `FrameTool`。Frame 将选中的静态 Image Empty 按当前视图合成为一张图片。
|
||||
`operators/frame_tool/operators.py` 定义 `FrameImages` 和 `FrameTool`。`projection.py` 负责视图投影、交叠和结果平面定位，`compositing.py` 负责像素深度排序与边界覆盖采样，`__init__.py` 导出类型及注册清单。Frame 将选中的静态 Image Empty 按当前视图合成为一张图片。
|
||||
|
||||
```mermaid
|
||||
flowchart TD
|
||||
|
||||
@@ -5,11 +5,12 @@
|
||||
| `operators/ai_setup.py` | [AI 环境管理](ai-setup.md) |
|
||||
| `operators/clipboard_image/` | [剪贴板图片](clipboard-image.md) |
|
||||
| `operators/convert_to_plane/` | [Plane 与 Depth Plane](convert-to-plane.md) |
|
||||
| `operators/convert_to_panorama/` | [Panorama](convert-to-panorama.md) |
|
||||
| `operators/cutout_tool/` | [Cutout](cutout-tool.md) |
|
||||
| `operators/frame_tool.py` | [Frame 多图合成](frame.md) |
|
||||
| `operators/frame_tool/operators.py` | [Frame 多图合成](frame.md) |
|
||||
| `operators/mask_tool.py` | [Mask Alpha 编辑](mask.md) |
|
||||
| `operators/rectify_tool.py` | [Rectify 透视校正](rectify.md) |
|
||||
| `operators/rectify_tool/operators.py` | [Rectify 透视校正](rectify.md) |
|
||||
| `operators/remove_background.py` | [Remove Background](remove-background.md) |
|
||||
| `operators/upscale.py` | [Upscale](upscale.md) |
|
||||
|
||||
单文件功能按业务命名；复杂功能包在 `operators.py` 定义 Operator、对应 WorkSpaceTool 和入口协调逻辑，其余模块承载具体实现。包的 `__init__.py` 汇总导出，类型在 `operators/__init__.py` 和扩展 `__init__.py` 汇总注册。AI Operator 使用普通字典提交任务，并在主线程响应中读取文件和修改 Blender 数据。
|
||||
单文件功能按业务命名；复杂功能包在 `operators.py` 定义 Operator、对应 WorkSpaceTool 和入口协调逻辑，其余模块承载具体实现。包的 `__init__.py` 汇总导出与 `CLASSES` 注册清单，类型在 `operators/__init__.py` 和扩展 `__init__.py` 汇总注册。AI Operator 使用普通字典提交任务，并在主线程响应中读取文件和修改 Blender 数据。
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
# Rectify 透视校正
|
||||
|
||||
`operators/rectify_tool.py` 定义 `RectifyImagePerspective` 与 `RectifyTool`，将图片内的四角区域校正为矩形。
|
||||
`operators/rectify_tool/operators.py` 定义 `RectifyImagePerspective` 与 `RectifyTool`，将图片内的四角区域校正为矩形。`geometry.py` 负责四角验证与像素校正，`preview.py` 负责预览图、Overlay 和宽高比控制，`__init__.py` 导出类型及注册清单。
|
||||
|
||||
```mermaid
|
||||
flowchart TD
|
||||
|
||||
@@ -1,37 +1,19 @@
|
||||
# Remove Background
|
||||
|
||||
Remove Background 从当前 Image Empty 准备单图或多帧输入，运行 BEN2 生成前景 Alpha，再用结果 Image 替换原图。
|
||||
|
||||
## 执行过程
|
||||
`operators/remove_background.py` 从 Image Empty 或当前 Shader Image Texture 准备输入。`RemoveImageBackground` 协调设置与输入，`RunBackgroundRemoval` 提交 Job 并应用结果。
|
||||
|
||||
```mermaid
|
||||
flowchart TD
|
||||
Empty["Image Empty"] --> Prepare["准备图片、视频或序列输入"]
|
||||
Prepare --> Job["remove-background Job"]
|
||||
Job --> Frames["逐帧运行 BEN2"]
|
||||
Frames --> Result["写出 PNG 结果序列"]
|
||||
Result --> Import["创建结果 Image"]
|
||||
Import --> Replace["替换 Empty.data"]
|
||||
A[ImageEditTarget 取得编辑目标] --> B[准备图像输入]
|
||||
B --> C[remove-background Job]
|
||||
C --> D[所选背景模型生成前景 Alpha]
|
||||
D --> E{输入类型}
|
||||
E -- 普通媒体 --> F[PNG 结果序列]
|
||||
E -- HDR --> G[alpha.npy]
|
||||
F --> H[应用结果与 Undo]
|
||||
G --> H
|
||||
```
|
||||
|
||||
Packed 图片和内存中修改过的单图会先导出临时文件；文件图片、视频和序列沿用原路径。
|
||||
`server/jobs/remove_background.py` 展开媒体并调用 `server/models/background.py`，支持 BiRefNet 和 BEN2。模型前景 Alpha 与源 Alpha 合成。
|
||||
|
||||
## 节点说明
|
||||
|
||||
### 准备媒体输入
|
||||
|
||||
入口是 `operators/remove_background.py` 中的 `RemoveImageBackground`。它检查当前对象、AI Environment 和帧数，再调用 `common/image.py` 的 `prepare_image_input()`。多帧输入会先写入 Scene 中的抽帧设置。
|
||||
|
||||
准备完成后，内部 Operator `RunBackgroundRemoval` 保存输入路径、源对象名和临时文件清理标记。
|
||||
|
||||
### 运行 BEN2
|
||||
|
||||
`RunBackgroundRemoval` 提交 `remove-background` Job。`server/jobs/remove_background.py` 使用公共媒体输入层展开单图、视频或序列，并逐帧调用 `server/models/ben2.py`。
|
||||
|
||||
BEN2 推理前先用源 Alpha 屏蔽透明区域，模型输出的前景 Alpha 再与源 Alpha 合成，RGB 保持源图内容。多帧结果按连续文件名写入 Job 目录。
|
||||
|
||||
### 导入并替换结果
|
||||
|
||||
Job 返回后，`remove_background.py` 的 `apply_background_response()` 调用 `common/image.py` 创建单图或序列 Image，并恢复源 Image User 的帧起点、偏移和循环设置。
|
||||
|
||||
`replace_empty_image()` 随后替换 `Empty.data`，提交一个 Remove Background Undo 步骤，并清理本次创建的临时输入。
|
||||
普通结果通过 `ImageEditTarget` 保留共享图像引用与帧设置。HDR 使用 `common/hdr_image.py` 的 `HdrBackgroundInput` 准备分析输入，并将返回 Alpha 应用于原浮点 RGB。Operator 负责临时输入清理。
|
||||
|
||||
@@ -1,21 +1,15 @@
|
||||
# Upscale
|
||||
|
||||
`operators/upscale.py` 从当前 Image Empty 生成 2× 结果并替换原图，支持单图、视频与序列帧。
|
||||
`operators/upscale.py` 从 Image Empty 或当前 Shader Image Texture 生成 2× 结果。普通媒体支持单图、视频和序列帧。
|
||||
|
||||
```mermaid
|
||||
flowchart TD
|
||||
A[检查源图长边小于 AI 输入上限] --> B[冻结 Preferences 模型与尺寸设置]
|
||||
B --> C[准备媒体输入]
|
||||
C --> D[upscale-image 逐帧处理]
|
||||
D --> E{所选模型}
|
||||
E -- Real-ESRGAN 或 SAFMN --> F[256×256 tile 与边缘上下文执行 4× RGB 推理]
|
||||
E -- PiSA-SR --> G[分阶段生成推理，细节强度 0.7]
|
||||
F --> H[统一缩放到源图 2×]
|
||||
G --> H
|
||||
H --> I[原 Alpha 独立 Lanczos 缩放并回填]
|
||||
I --> J[写出 upscale 序列并替换 Image]
|
||||
A[检查源图尺寸并冻结模型设置] --> B[准备媒体输入]
|
||||
B --> C[upscale-image 逐帧处理]
|
||||
C --> D[分块执行 4× RGB 推理]
|
||||
D --> E[统一缩放为源图 2×]
|
||||
E --> F[原 Alpha 独立 Lanczos 缩放]
|
||||
F --> G[ImageEditTarget 应用结果并提交 Undo]
|
||||
```
|
||||
|
||||
`server/models/upscale.py` 统一处理输入上限、模型分派和最终 RGB/Alpha 尺寸。`onnx_upscale.py` 处理常规 4× 模型，`onnx_pisa_sr.py` 处理 PiSA-SR。多帧与多 tile 只在最后一次运行请求内存收缩。
|
||||
|
||||
Provider 资源错误会释放当前 Upscale Session，并返回降低输入上限或切换 CPU 的提示。响应通过 `common/image.py` 加载、Pack、保留图像共享与帧设置，并提交 Undo。
|
||||
`server/models/upscale.py` 处理输入上限和最终尺寸，`onnx_upscale.py` 执行 Real-ESRGAN 与 HAT 模型。ModelManager 管理 Session；Provider 资源错误会释放当前 Session 并返回处理建议。
|
||||
|
||||
@@ -1,43 +1,40 @@
|
||||
# Job 产物
|
||||
|
||||
Server 把结果写入 Job 目录，返回以相对文件名为值的字典。Blender 主线程通过 JobResult 读取文件，再创建或更新 Image、Mesh、Material 和 Object。
|
||||
Server 将结果写入 Job 目录，返回以相对文件名为值的字典。Blender 主线程通过 JobResult 读取文件，创建或更新 Image、Mesh、Material 和 Object。
|
||||
|
||||
| Job | 文件 | 消费方 |
|
||||
| --- | --- | --- |
|
||||
| `remove-background` | `image.0001.png` 起 | Image Empty 替换 |
|
||||
| `upscale-image` | `upscale.0001.png` 起 | 2× Image Empty 替换 |
|
||||
| `generate-depth-plane-geometry` | `image.0001.*`、`depth.exr`、`depth.json`、`tangent-normal.png` | Plane 对象与材质 |
|
||||
| `generate-cutout-artifacts` | 按请求提供深度、元数据和法线 | Cutout 对象与材质 |
|
||||
| `remove-background` | PNG 序列或 `alpha.npy` | ImageEditTarget 或 HDR Alpha 编辑 |
|
||||
| `upscale-image` | `upscale.0001.png` 起 | 图像替换 |
|
||||
| `generate-depth-plane-geometry` | `depth.exr`、`depth.json` 和法线 | Depth / Relief Plane |
|
||||
| `generate-cutout-artifacts` | 按请求提供深度、元数据和法线 | Cutout |
|
||||
| `generate-panorama-geometry` | `depth.exr`、`depth.json` | Panorama |
|
||||
|
||||
## 深度与法线协议
|
||||
|
||||
| 返回字段 | 文件 | 内容 |
|
||||
| --- | --- | --- |
|
||||
| `depth` | `depth.exr` | float32 RGB 保存相机空间 XYZ，Alpha 保存 Validity |
|
||||
| `depth_metadata` | `depth.json` | `image_size`、3×3 `intrinsics`、`reference_depth` |
|
||||
| `tangent_normal` | `tangent-normal.png` | 切线空间法线 |
|
||||
| `object_normal` | `object-normal.png` | Cutout 原生 XY / Z 坐标的对象空间法线 |
|
||||
| 返回字段 | 内容 |
|
||||
| --- | --- |
|
||||
| `depth` | float32 EXR，透视产物 RGB 保存相机空间 XYZ，Alpha 保存源 Alpha × 模型有效性 |
|
||||
| `depth_metadata` | JSON，透视产物包含 `image_size`、3×3 `intrinsics`、`reference_depth` |
|
||||
| `tangent_normal` | `tangent-normal.png`，切线空间法线 |
|
||||
| `object_normal` | `object-normal.png`，对象空间法线 |
|
||||
|
||||
```mermaid
|
||||
flowchart TD
|
||||
A[矩形源 RGBA] --> B[限尺寸并合成黑底模型输入]
|
||||
B --> C[MoGe-2 GeometryFrame]
|
||||
C --> D[原预测生成 Normal]
|
||||
C --> E{Cutout 深度？}
|
||||
E -- 是 --> F[按源 Alpha 延展轮廓深度]
|
||||
E -- 否 --> G[完整相机 Points]
|
||||
F --> G
|
||||
G --> H[depth.exr 与 depth.json]
|
||||
D --> I[Blender 加载并 Pack]
|
||||
H --> I
|
||||
A[源 RGB 与 Alpha] --> B[限尺寸 RGB 推理]
|
||||
B --> C[GeometryFrame]
|
||||
C --> D[写出深度与法线]
|
||||
A --> E[Alpha 有效性与参考区域]
|
||||
E --> D
|
||||
D --> F[Blender 加载并 Pack]
|
||||
```
|
||||
|
||||
`depth_texture.py` 保存完整有限 Points；模型 Mask 转为 `0/1` Validity。参考深度取有效、有限、正深度且满足源 Alpha 条件的样本中位数：整图使用 Alpha ≥ 0.1，Cutout 使用所选 Alpha Threshold 并要求 Alpha > 0。非法字段和空参考区域抛出错误。
|
||||
参考深度取满足模型有效性、有限正深度与 Alpha 阈值的样本中位数。`common/depth.py` 加载 EXR，设置 Non-Color 和 CHANNEL_PACKED。Depth Plane 使用对象空间法线，Relief Plane 使用切线空间法线。
|
||||
|
||||
`common/depth.py` 加载深度并配置 Non-Color、CHANNEL_PACKED 与 Pack。Relief 和 Depth Balloon 读取 B/Z，Camera 与 Depth Solid 读取 XYZ，共用同一 EXR。
|
||||
Panorama 的 EXR RGB 保存径向距离，Alpha 保存源 Alpha 与融合有效性；元数据包含 `projection: equirectangular` 和 `image_size`。
|
||||
|
||||
## Selection 与颜色
|
||||
## 颜色与图像编辑
|
||||
|
||||
Cutout 提交几何 bounds 内的原始 RGBA。Blender 在创建网格时组合源 Alpha 和 `SelectionMask`；颜色图保留矩形源 RGBA，法线可见区域取源 Alpha。AI 分析尺寸与颜色分辨率分别处理。
|
||||
`common/color_image.py` 为 Plane、Panorama 和 Cutout 准备独立的材质颜色与 AI 分析输入。Cutout 使用 Selection bounds 内的源图；网格创建时组合源 Alpha 与 SelectionMask。颜色分辨率与分析尺寸分别处理。
|
||||
|
||||
Remove Background、Upscale、Frame、Mask、Rectify 通过 `common/image.py` 的编辑结果入口保留源 Image 名称和共享引用语义。Plane 与 Cutout 的材质贴图使用 Color、Depth、Normal 命名函数。
|
||||
`ImageEditTarget` 统一 Image Empty 和 Shader Image Texture 的编辑目标及共享引用。HDR 去背景通过 `HdrBackgroundInput` 保留 Blender 浮点 RGB，并应用 Server 返回的连续 Alpha。
|
||||
|
||||
@@ -1,53 +1,36 @@
|
||||
# Job Runtime 与 Server
|
||||
|
||||
`runtime.py` 配置 BlendJob `JobRuntime` 和独立 Python 3.12 Environment。`server/app.py` 注册 Job，`model_manager.py` 作为 `model_manager` Resource 管理下载文件与缓存 Session。
|
||||
`runtime.py` 配置 BlendJob `JobRuntime` 和独立 Python 3.12 Environment。`server/app.py` 注册 Job，`model_manager.py` 管理模型文件与缓存 Session。
|
||||
|
||||
```mermaid
|
||||
flowchart TD
|
||||
A[Operator request] --> B[JobRuntime]
|
||||
B --> C[Server 单 Worker 队列]
|
||||
C --> D[Job 调用媒体与模型模块]
|
||||
D --> E[写入 jobs/job-id 目录]
|
||||
E --> F[主线程 response]
|
||||
A[Operator request] --> B[JobRuntime 队列]
|
||||
B --> C[Server Job 调用媒体与模型模块]
|
||||
C --> D[写入 Job 目录]
|
||||
D --> E[主线程 response 导入结果]
|
||||
```
|
||||
|
||||
Job 在阶段和帧边界检查取消，进度通过 `context.progress()` 返回。视频临时帧由 `media/input.py` 的上下文管理器清理，Blender 导出的临时输入由对应 Operator 清理。
|
||||
|
||||
## Job 与数据模块
|
||||
|
||||
| Job | 实现与用途 |
|
||||
| --- | --- |
|
||||
| `download-model`、`download-required-models` | `app.py` 调用 ModelManager 下载单个或缺失模型 |
|
||||
| `remove-background` | `jobs/remove_background.py`，生成 BEN2 前景 |
|
||||
| `upscale-image` | `jobs/upscale.py`，生成 2× 图像或序列 |
|
||||
| `generate-depth-plane-geometry` | `jobs/depth_plane.py`，复制整图颜色并生成深度、元数据与切线法线 |
|
||||
| `generate-cutout-artifacts` | `jobs/cutout.py`，按请求生成 Cutout 深度和法线 |
|
||||
| `download-model`、`download-required-models` | `app.py` 下载单个或缺失模型 |
|
||||
| `remove-background` | `jobs/remove_background.py` 生成前景 Alpha |
|
||||
| `upscale-image` | `jobs/upscale.py` 生成 2× 图像或序列 |
|
||||
| `generate-depth-plane-geometry` | `jobs/depth_plane.py` 生成深度、元数据和法线 |
|
||||
| `generate-cutout-artifacts` | `jobs/cutout.py` 按请求生成深度和法线 |
|
||||
| `generate-panorama-geometry` | `jobs/panorama.py` 生成全景径向深度 |
|
||||
|
||||
`media/` 处理单图、序列、视频与输入尺寸。Depth Plane 和 Cutout 按单张静态图片工作；Remove Background 与 Upscale 逐帧处理动画输入。
|
||||
`media/` 展开单图、序列与视频，并限制分析尺寸。Job 在阶段与帧边界检查取消，通过 `context.progress()` 返回进度。媒体上下文清理抽取的临时帧，Operator 清理 Blender 导出的临时输入。
|
||||
|
||||
`geometry/prediction_artifacts.py` 将限尺寸后的 RGBA 合成到黑底，运行一次 MoGe-2，再调用深度、法线和元数据写入器。Cutout 的深度分支通过 `edge_depth.py` 延展轮廓附近的深度；法线使用原预测。文件字段见 [Job 产物](outputs.md)。
|
||||
`geometry/prediction_artifacts.py` 缩放源 RGB 后运行一次 MoGe，将同一预测交给深度、法线和元数据写入器。源 Alpha 用于产物有效性和参考深度采样。文件协议见 [Job 产物](outputs.md)。
|
||||
|
||||
## 模型与缓存
|
||||
|
||||
| 用途 | 模型 | Session 分类 |
|
||||
| --- | --- | --- |
|
||||
| Depth 与 Normal | MoGe-2 ViT-S / ViT-B / ViT-L Normal | geometry |
|
||||
| Remove Background | BEN2 | ben2 |
|
||||
| Upscale | Real-ESRGAN WDN、SAFMN、Real-ESRGAN x4plus、PiSA-SR | upscale |
|
||||
| Depth 与 Normal | MoGe-2 ViT-S / ViT-B Normal、MoGe-3 ViT-L | geometry |
|
||||
| Remove Background | BiRefNet Lite、BEN2、BiRefNet HR Matting | background |
|
||||
| Upscale | Real-ESRGAN WDN、Real-ESRGAN x4plus、HAT GAN x4 Sharper | upscale |
|
||||
|
||||
`model_catalog.py` 声明文件来源、许可、大小和 SHA-256。`model_download.py` 下载文件并校验；ModelManager 按模型路径和设备缓存各类最近使用的 Session，切换模型或设备时释放对应类别。
|
||||
`model_catalog.py` 声明文件来源、许可、大小和 SHA-256；`model_download.py` 下载并校验。ModelManager 按模型路径和设备缓存 Session，切换模型或设备时释放对应类别。Clear Models 或关闭 Server 时释放已加载资源。
|
||||
|
||||
```mermaid
|
||||
flowchart TD
|
||||
A[取得缓存 Session] --> B[运行单图、多帧或 tile]
|
||||
B --> C[最后一次推理]
|
||||
C --> D[CUDA 收缩空闲 arena]
|
||||
D --> E[保留 Session 供后续复用]
|
||||
E --> F[Clear Models 或关闭 Server 时释放]
|
||||
```
|
||||
|
||||
Windows 默认 DirectML，macOS arm64 默认 CPU 并支持选择 CoreML。MoGe-2 为 CoreML 配置静态 shape 分区，其余计算由 CPU 执行。PiSA-SR 在 CoreML 上返回明确的支持限制；其分阶段 Session 在切换阶段、取消和清理时关闭。
|
||||
|
||||
## Environment 与存储
|
||||
|
||||
首次安装通过 `post_install()` 下载默认 MoGe-2、BEN2 与 Real-ESRGAN WDN。Storage Root 默认 `~/.anyimage`，其中保存 `.venv/`、`models/`、`jobs/` 及环境状态和日志。设置入口与安装流程见 [AI 环境管理](operators/ai-setup.md)。
|
||||
首次安装通过 `post_install()` 下载默认 MoGe-2、BiRefNet Lite 与 Real-ESRGAN WDN。Storage Root 默认 `~/.anyimage`，保存环境、模型、Job 产物和日志。设置入口见 [AI 环境管理](operators/ai-setup.md)。
|
||||
|
||||
@@ -23,6 +23,7 @@ nav:
|
||||
- AI 环境管理: internals/operators/ai-setup.md
|
||||
- Clipboard Image: internals/operators/clipboard-image.md
|
||||
- Convert to Plane: internals/operators/convert-to-plane.md
|
||||
- Convert to Panorama: internals/operators/convert-to-panorama.md
|
||||
- Cutout Tool: internals/operators/cutout-tool.md
|
||||
- Frame: internals/operators/frame.md
|
||||
- Mask: internals/operators/mask.md
|
||||
|
||||
2
openspec/changes/add-cutout-polyline/.openspec.yaml
Normal file
2
openspec/changes/add-cutout-polyline/.openspec.yaml
Normal file
@@ -0,0 +1,2 @@
|
||||
schema: spec-driven
|
||||
created: 2026-09-12
|
||||
49
openspec/changes/add-cutout-polyline/design.md
Normal file
49
openspec/changes/add-cutout-polyline/design.md
Normal file
@@ -0,0 +1,49 @@
|
||||
## Context
|
||||
|
||||
Cutout 的 `SelectCutoutSelection` 已继承 `common.viewport.ImageGesture`，目前固定为 Lasso 并通过拖动键位启动。Mask 的 Polyline 已由该共用类实现。Cutout 完成选区后打开 Shape 饼菜单，Depth Solid 经异步生成后使用 `O Image Depth Cutout` 创建对象。
|
||||
|
||||
节点构建内部的 Uniform Thickness 输入在保存前显示名改为 Thickness；节点组同时存在 Balloon 使用的 Thickness 输入，初始化必须区分 socket 标识。
|
||||
|
||||
## Goals
|
||||
|
||||
支持 Cutout 逐点剪出及 Uniform 初始化；统一 Mask 的 Extend 命名和 Set 默认值。
|
||||
|
||||
## Decisions
|
||||
|
||||
### 共用手势与独立设置
|
||||
|
||||
在 `properties.py` 定义 Cutout 的 Lasso / Polyline 属性，由 Scene 保存，默认 Lasso。Cutout 在启动时读取手势，沿用 `ImageGesture` 的逐点输入、实时预览、闭合提示和取消清理。使用共用按下启动键位支持首点输入，并验证 Lasso 拖动完成以及空白处源对象保持行为。共用逻辑留在 `common`，Cutout 负责自己的状态文本和选区提交。
|
||||
|
||||
### 创建时应用 Uniform
|
||||
|
||||
```mermaid
|
||||
flowchart TD
|
||||
A[读取 Cutout Gesture] --> B[ImageGesture 收集并确认选区]
|
||||
B --> C[Shape 饼菜单携带本次 Gesture 快照]
|
||||
C --> D[CutoutSelectionToShape 保存 Gesture]
|
||||
D --> E[生成所需图像并在主线程创建对象]
|
||||
E --> F{Polyline 且 Depth Solid}
|
||||
F -->|是| G[设置新修改器 Mode 为 Uniform]
|
||||
F -->|否| H[按 Shape 初始化修改器]
|
||||
```
|
||||
|
||||
手势作为普通字符串从选择 Operator 显式传到饼菜单、转换 Operator 和对象创建链路，异步响应使用该快照，避免用户中途切换 Scene 设置影响结果。服务器生成协议继续处理深度和法线产物；修改器初始化由主线程完成。Uniform 自动设置针对本次新建的 Depth Solid 对象，之后用户可手动修改 Mode。
|
||||
|
||||
### 节点厚度默认值
|
||||
|
||||
将构建内部 Uniform Thickness 默认值设为 0.5，并验证最终资产对应 socket 的默认值和新修改器实际值。检查现有通用 Thickness 赋值只初始化其目标输入，避免同名 socket 覆盖 Uniform 默认值。按现有资产修订机制更新 Depth Solid 版本并运行 `uv run node-group build`。
|
||||
|
||||
### Mask 模式
|
||||
|
||||
统一为 `SET` / Set、`EXTEND` / Extend、`SUBTRACT` / Subtract，保留枚举数值 0、1、2。Scene 和 Operator 均通过共用属性默认 Set。Extend 保持 `max(alpha, mask)`，Set 保持 `alpha * mask`，Subtract 保持 `alpha * (1-mask)`；状态文本和测试同步命名。
|
||||
|
||||
## Risks / Trade-offs
|
||||
|
||||
- 按下启动可能改变 Cutout 的源对象选择时机 → 覆盖 Lasso、Polyline、空白点击和取消行为。
|
||||
- 异步生成期间手势被切换 → 在提交时保存本次手势，并覆盖设置改变后的响应。
|
||||
- 同名 Thickness socket 容易导致写入错误 → 验证具体 socket identifier、资产默认值及实际修改器值。
|
||||
- 旧脚本传入 `ADD` 将失效 → 调用方统一使用 `EXTEND`，保持原枚举数值且不增加别名。
|
||||
|
||||
## Migration Plan
|
||||
|
||||
实现代码与测试后重建节点资产，执行相关测试及全量测试。回退时将本次代码、测试和资产作为同一变更回退。
|
||||
27
openspec/changes/add-cutout-polyline/proposal.md
Normal file
27
openspec/changes/add-cutout-polyline/proposal.md
Normal file
@@ -0,0 +1,27 @@
|
||||
## Why
|
||||
|
||||
Cutout 目前只能自由绘制 Lasso，沿直线轮廓剪出时需要逐点定义边界。复用 Mask 的 Polyline 交互，并为这种剪出方式设置 Uniform 厚度，可减少手动调整；Mask 使用 Extend 和默认 Set，使名称与常用操作一致。
|
||||
|
||||
## What Changes
|
||||
|
||||
- Cutout 增加 Gesture 选项 Lasso / Polyline，默认 Lasso，复用共用逐点交互和选区提交链路。
|
||||
- Polyline 剪出创建使用 `O Image Depth Cutout` 的 Depth Solid 对象时，自动设置修改器 Mode 为 Uniform。
|
||||
- `O Image Depth Cutout` 的 Uniform Thickness 接口默认值改为 0.5（资产中显示为 Thickness）。
|
||||
- **BREAKING** Mask 的 `ADD` / Add 统一重命名为 `EXTEND` / Extend，保留扩展 Alpha 的运算语义；默认模式从 Subtract 改为 Set。
|
||||
|
||||
## Capabilities
|
||||
|
||||
### New Capabilities
|
||||
|
||||
- `cutout-polyline`: Cutout 手势选择、共用 Polyline 交互，以及创建结果的 Uniform 初始化。
|
||||
- `mask-mode-options`: Mask 的 Set / Extend / Subtract 名称、运算和默认值。
|
||||
|
||||
### Modified Capabilities
|
||||
|
||||
无。当前 `openspec/specs` 尚无已归档的对应规范。
|
||||
|
||||
## Impact
|
||||
|
||||
- `properties.py`、`common/viewport.py`、`operators/cutout_tool`、`operators/mask_tool.py` 和相关注册、交互、对象创建测试。
|
||||
- `tools/nodes/groups/image_depth_cutout.py`、节点资产 `O_AnyImage.blend` 及节点测试；实现阶段需运行节点构建与验证。
|
||||
- 无新增依赖。已有 change 中的 Mask、Polyline 和 Cutout 提案作为实现背景，本 change 定义本次行为。
|
||||
@@ -0,0 +1,57 @@
|
||||
## ADDED Requirements
|
||||
|
||||
### Requirement: Cutout offers persistent gesture selection
|
||||
|
||||
Cutout SHALL 提供 Lasso 和 Polyline 手势，默认 Lasso，并保存到 Scene 的独立 Cutout 设置。
|
||||
|
||||
#### Scenario: User selects Polyline
|
||||
- **WHEN** 用户将 Cutout Gesture 切换为 Polyline 并开始剪出
|
||||
- **THEN** 工具以首次左键按下的位置为起点，随后通过点击添加顶点
|
||||
|
||||
#### Scenario: User returns to Cutout
|
||||
- **WHEN** 用户选择 Polyline 后切换到其他工具再返回
|
||||
- **THEN** Cutout 继续使用 Polyline，Mask Gesture 保持自己的设置
|
||||
|
||||
#### Scenario: Default Lasso interaction
|
||||
- **WHEN** 新 Scene 中用户拖动 Cutout 并释放左键
|
||||
- **THEN** 工具以 Lasso 收集轮廓并在有效选区完成后打开 Shape 饼菜单
|
||||
|
||||
### Requirement: Cutout reuses shared Polyline interaction
|
||||
|
||||
Cutout MUST 复用 Mask 所用的共用 Polyline 手势，提供轮廓预览、起点闭合提示、左键添加顶点、Backspace 删除末点、点击起点或双击或 Enter 完成，以及 RMB、Esc 和切换工具取消。
|
||||
|
||||
#### Scenario: User completes a polygon
|
||||
- **WHEN** 至少三个顶点后，用户点击起点闭合范围内、双击或按 Enter / Numpad Enter
|
||||
- **THEN** 工具提交已确认的闭合 SelectionPath 并打开 Shape 饼菜单，闭合点击和双击第二击不添加重复顶点
|
||||
|
||||
#### Scenario: Too few points
|
||||
- **WHEN** 少于三个顶点时用户尝试完成
|
||||
- **THEN** 工具继续收集顶点，不打开 Shape 菜单
|
||||
|
||||
#### Scenario: User edits or cancels a polygon
|
||||
- **WHEN** 用户按 Backspace，或通过 RMB、Esc、切换工具取消
|
||||
- **THEN** Backspace 删除末点并至少保留起点；取消清理预览且不创建结果对象
|
||||
|
||||
#### Scenario: Empty click preserves the source
|
||||
- **WHEN** 用户在空白处启动 Cutout 手势
|
||||
- **THEN** 工具保持现有源对象选择规则，不因空白点击清空源对象
|
||||
|
||||
### Requirement: Polyline initializes Depth Solid with Uniform mode
|
||||
|
||||
通过 Polyline 创建 Depth Solid 时，系统 MUST 将新对象的 `O Image Depth Cutout` 修改器 Mode 初始化为 Uniform。手势 MUST 使用本次提交快照。
|
||||
|
||||
#### Scenario: Polyline depth result completes asynchronously
|
||||
- **WHEN** 用户提交 Polyline 并选择 Depth Solid，在生成结束前将 Cutout Gesture 改为 Lasso
|
||||
- **THEN** 本次结果的 Mode 仍为 Uniform，用户可在创建后手动调整
|
||||
|
||||
#### Scenario: Lasso or another shape is used
|
||||
- **WHEN** 用户通过 Lasso 创建 Depth Solid，或通过 Polyline 创建 Flat、Solid、Depth Balloon
|
||||
- **THEN** 对象沿用该 Shape 的现有 Mode 初始化规则
|
||||
|
||||
### Requirement: Uniform thickness defaults to half a unit
|
||||
|
||||
`O Image Depth Cutout` 的 Uniform Thickness socket MUST 默认 0.5，保存资产中对应显示为 Thickness 的 socket 和新建修改器 MUST 使用该值。
|
||||
|
||||
#### Scenario: Node asset is built and loaded
|
||||
- **WHEN** 完成节点资产构建并从资产加载节点组创建 Polyline Depth Solid
|
||||
- **THEN** Mode 为 Uniform，Uniform 厚度值为 0.5，通用 Thickness 初始化不会覆盖该值
|
||||
@@ -0,0 +1,25 @@
|
||||
## ADDED Requirements
|
||||
|
||||
### Requirement: Mask exposes Set Extend and Subtract
|
||||
|
||||
Mask MUST 按顺序提供 `SET` / Set、`EXTEND` / Extend、`SUBTRACT` / Subtract，枚举数值分别为 0、1、2；界面和状态文本 MUST 使用 Extend。
|
||||
|
||||
#### Scenario: User extends Alpha
|
||||
- **WHEN** 用户以 Extend 提交值为 mask 的选区到现有 alpha
|
||||
- **THEN** 结果 Alpha 为 `max(alpha, mask)`，RGB 保持不变
|
||||
|
||||
#### Scenario: User uses Set or Subtract
|
||||
- **WHEN** 用户使用 Set 或 Subtract
|
||||
- **THEN** 结果 Alpha 分别为 `alpha * mask` 或 `alpha * (1-mask)`，RGB 保持不变
|
||||
|
||||
### Requirement: Mask defaults to Set
|
||||
|
||||
Scene 的 Mask Mode 和 Mask Operator 的模式属性 MUST 默认 Set，并通过同一属性定义保持一致。
|
||||
|
||||
#### Scenario: New scene and operator defaults
|
||||
- **WHEN** 创建新 Scene 或未指定模式的 Mask Operator
|
||||
- **THEN** 模式默认值为 `SET`
|
||||
|
||||
#### Scenario: User chooses another mode
|
||||
- **WHEN** 用户主动设置 Extend 或 Subtract 后启动 Mask
|
||||
- **THEN** 本次操作使用用户保存的模式
|
||||
21
openspec/changes/add-cutout-polyline/tasks.md
Normal file
21
openspec/changes/add-cutout-polyline/tasks.md
Normal file
@@ -0,0 +1,21 @@
|
||||
## 1. Cutout 手势
|
||||
|
||||
- [x] 1.1 在 properties.py 添加默认 Lasso 的 Cutout Gesture 属性并接入 Scene、Operator 和工具设置，检查注册与逆序注销测试。
|
||||
- [x] 1.2 将 Cutout 接入共用 Polyline 启动、预览与完成逻辑，更新状态文本；覆盖逐点、闭合、双击、Enter、Backspace、取消、Lasso 回归和空白处源对象保持行为。
|
||||
|
||||
## 2. Uniform 初始化与资产
|
||||
|
||||
- [x] 2.1 将本次 Gesture 快照从选区提交经 Shape 菜单传入转换 Operator 和对象创建链路；覆盖异步期间切换设置的行为。
|
||||
- [x] 2.2 为 Polyline 创建的 Depth Solid 设置 Mode 为 Uniform，验证 Lasso 与其他 Shape 的初始化行为。
|
||||
- [x] 2.3 将节点内部 Uniform Thickness 默认值改为 0.5，检查同名 Thickness socket 的赋值目标并更新资产修订号；补充资产默认值与实际修改器值测试。
|
||||
- [x] 2.4 运行 `uv run node-group build`，更新并验证 O_AnyImage.blend，确认源码与保存资产的 Capture Attribute 数量为零。
|
||||
|
||||
## 3. Mask 模式
|
||||
|
||||
- [x] 3.1 将 Mask 的 ADD / Add 统一改为 EXTEND / Extend，保持枚举数值与 Alpha 运算语义，将共用模式属性默认值改为 SET。
|
||||
- [x] 3.2 更新模式、状态文本、设置持久化和注册测试，验证三种模式的 Alpha 结果及 RGB 保持行为。
|
||||
|
||||
## 4. 验证
|
||||
|
||||
- [x] 4.1 运行相关测试及 `uv run pytest`，确认正常退出。
|
||||
- [x] 4.2 检查运行时代码和测试中的旧 Mask 模式引用及最终差异，确认源码、测试和节点资产一致。
|
||||
@@ -1,6 +1,6 @@
|
||||
[project]
|
||||
name = "anyimage"
|
||||
version = "0.13.2"
|
||||
version = "0.13.3"
|
||||
description = "Fast image-guided structure building tools for Blender"
|
||||
readme = "README.md"
|
||||
requires-python = ">=3.11,<3.12"
|
||||
@@ -9,7 +9,7 @@ dependencies = ["blendjob==0.1.17", "numpy>=1.26,<2.0", "scipy==1.15.3"]
|
||||
|
||||
[dependency-groups]
|
||||
dev = [
|
||||
"av==17.0.0",
|
||||
|
||||
"bpy==4.5.3",
|
||||
"huggingface-hub==1.4.1",
|
||||
"mike>=2.1.3,<3",
|
||||
|
||||
Binary file not shown.
@@ -1,7 +1,7 @@
|
||||
schema_version = "1.0.0"
|
||||
|
||||
id = "anyimage"
|
||||
version = "0.13.2"
|
||||
version = "0.13.3"
|
||||
name = "AnyImage"
|
||||
tagline = "Build spatial structures from images, fast"
|
||||
maintainer = "OmooLab"
|
||||
|
||||
@@ -69,12 +69,10 @@ def require_model(model_key):
|
||||
return model
|
||||
|
||||
|
||||
def moge2_parameters(settings, path, model, resolution_level, start_frame):
|
||||
def moge2_parameters(path, model, resolution_level):
|
||||
return {
|
||||
"input": str(path),
|
||||
"model": model,
|
||||
"resolution_level": int(resolution_level),
|
||||
"video_frames": settings.video_frames,
|
||||
"start_frame": start_frame,
|
||||
"device": production_device().lower(),
|
||||
}
|
||||
|
||||
@@ -7,12 +7,12 @@ from pathlib import Path
|
||||
|
||||
import bpy
|
||||
|
||||
from .image import cleanup_image_input, image_base_name, image_is_packed, image_rgba, is_animated_image, video_total_frames
|
||||
from .image import cleanup_image_input, image_base_name, image_is_packed, image_rgba, is_animated_image
|
||||
|
||||
|
||||
def require_static_color_image(image):
|
||||
"""Reject animated inputs before preparing material resources."""
|
||||
if is_animated_image(image) or video_total_frames(image) != 1:
|
||||
if is_animated_image(image):
|
||||
raise ValueError("Material conversion supports static images only")
|
||||
|
||||
|
||||
|
||||
@@ -1,14 +1,12 @@
|
||||
"""Shared Blender Image and Image Empty lifecycle helpers."""
|
||||
|
||||
import math
|
||||
import re
|
||||
import shutil
|
||||
import tempfile
|
||||
from pathlib import Path
|
||||
|
||||
import bpy
|
||||
|
||||
from ..server.media.input import validate_input_path
|
||||
|
||||
VIEW_TRANSFORM_COLOR_SPACES = (
|
||||
("ACES 1.3", "ACES 1.3 sRGB"),
|
||||
@@ -36,19 +34,6 @@ MAX_PROJECTIVE_OUTPUT_PIXELS = 16_777_216
|
||||
PROJECTIVE_CHUNK_ROWS = 256
|
||||
|
||||
|
||||
VIDEO_NAME_SUFFIXES = (
|
||||
".mp4",
|
||||
".mov",
|
||||
".avi",
|
||||
".mkv",
|
||||
".webm",
|
||||
".m4v",
|
||||
".mts",
|
||||
".mpg",
|
||||
".mpeg",
|
||||
)
|
||||
|
||||
|
||||
def is_image_empty(candidate):
|
||||
return bool(
|
||||
candidate
|
||||
@@ -290,21 +275,10 @@ def warp_projective_pixels(source_pixels, source_size, quad, output_size):
|
||||
return np.flipud(straight_rgba(output)).ravel()
|
||||
|
||||
|
||||
|
||||
def is_animated_image(image):
|
||||
return getattr(image, "source", None) in {"MOVIE", "SEQUENCE"}
|
||||
|
||||
|
||||
def video_total_frames(image):
|
||||
"""Return the total animated frame count, or 1 for a still image."""
|
||||
source = getattr(image, "source", None)
|
||||
if source == "MOVIE":
|
||||
return int(getattr(image, "frame_duration", 1) or 1)
|
||||
if source == "SEQUENCE":
|
||||
return len(collect_sequence_frames(image)) or 1
|
||||
return 1
|
||||
|
||||
|
||||
def image_user_settings(source_object):
|
||||
"""Return an image owner's playback settings as a four-item tuple."""
|
||||
image_user = getattr(source_object, "image_user", None)
|
||||
@@ -318,28 +292,11 @@ def image_user_settings(source_object):
|
||||
)
|
||||
|
||||
|
||||
def extract_frame_range(image, image_user):
|
||||
"""Return the clamped (start_frame, frame_count) range extracted from the source."""
|
||||
total_frames = video_total_frames(image)
|
||||
if total_frames <= 1:
|
||||
return 1, total_frames
|
||||
frame_offset = int(getattr(image_user, "frame_offset", 0) or 0)
|
||||
image_frames = int(getattr(image_user, "frame_duration", 0) or 0)
|
||||
frame_count = (
|
||||
min(image_frames, total_frames)
|
||||
if image_frames >= 1
|
||||
else total_frames
|
||||
)
|
||||
start_frame = min(1 + frame_offset, total_frames - frame_count + 1)
|
||||
start_frame = max(start_frame, 1)
|
||||
return start_frame, frame_count
|
||||
|
||||
|
||||
def image_path(image):
|
||||
filepath_from_user = getattr(image, "filepath_from_user", None)
|
||||
filepath = (
|
||||
filepath_from_user()
|
||||
if callable(filepath_from_user) and getattr(image, "source", None) != "SEQUENCE"
|
||||
if callable(filepath_from_user)
|
||||
else getattr(image, "filepath", "")
|
||||
)
|
||||
if not filepath:
|
||||
@@ -371,7 +328,7 @@ def cleanup_image_input(path, temporary):
|
||||
def image_base_name(image):
|
||||
name = image.name
|
||||
lower_name = name.lower()
|
||||
for suffix in IMAGE_NAME_SUFFIXES + VIDEO_NAME_SUFFIXES:
|
||||
for suffix in IMAGE_NAME_SUFFIXES:
|
||||
suffix_index = lower_name.rfind(suffix)
|
||||
if suffix_index <= 0:
|
||||
continue
|
||||
@@ -409,137 +366,35 @@ def save_packed_image(image, _scene):
|
||||
return output_path
|
||||
|
||||
|
||||
def collect_sequence_frames(image):
|
||||
"""Return the numbered frame files of a SEQUENCE image, or [] when absent."""
|
||||
source_path = image_path(image)
|
||||
if not is_animated_image(image) or not source_path.is_file():
|
||||
return []
|
||||
match = re.match(
|
||||
r"^(?P<prefix>.*?)(?P<digits>\d+)(?P<suffix>\.[^.]+)$",
|
||||
source_path.name,
|
||||
)
|
||||
if match is None:
|
||||
return []
|
||||
prefix = match.group("prefix")
|
||||
suffix = match.group("suffix").lower()
|
||||
frames = []
|
||||
for path in source_path.parent.iterdir():
|
||||
frame_match = re.match(
|
||||
r"^(?P<prefix>.*?)(?P<digits>\d+)(?P<suffix>\.[^.]+)$",
|
||||
path.name,
|
||||
)
|
||||
if (
|
||||
frame_match is not None
|
||||
and frame_match.group("prefix") == prefix
|
||||
and frame_match.group("suffix").lower() == suffix
|
||||
):
|
||||
frames.append((int(frame_match.group("digits")), path))
|
||||
return [path for _digits, path in sorted(frames)]
|
||||
|
||||
|
||||
def prepare_image_input(image, scene):
|
||||
source_path = image_path(image)
|
||||
"""Prepare one still image for a Server job."""
|
||||
if is_animated_image(image):
|
||||
raise RuntimeError("Only still images are supported")
|
||||
if image_is_packed(image):
|
||||
return save_packed_image(image, scene), True
|
||||
if getattr(image, "source", None) == "MOVIE":
|
||||
if not source_path.is_file():
|
||||
raise RuntimeError(
|
||||
f"Source video file does not exist: {source_path}"
|
||||
)
|
||||
return source_path, False
|
||||
if getattr(image, "source", None) == "SEQUENCE":
|
||||
validate_input_path(source_path, images_only=True)
|
||||
sequence_frames = collect_sequence_frames(image)
|
||||
if sequence_frames:
|
||||
temporary_directory = Path(tempfile.mkdtemp(prefix="anyimage-input-"))
|
||||
try:
|
||||
for index, frame_path in enumerate(sequence_frames):
|
||||
suffix = frame_path.suffix.lower() or ".jpg"
|
||||
shutil.copy2(
|
||||
frame_path,
|
||||
temporary_directory / f"frame.{index + 1:04d}{suffix}",
|
||||
)
|
||||
except Exception:
|
||||
shutil.rmtree(temporary_directory, ignore_errors=True)
|
||||
raise
|
||||
return temporary_directory, True
|
||||
source_path = image_path(image)
|
||||
if not source_path.is_file():
|
||||
raise RuntimeError(
|
||||
f"Source image file does not exist: {source_path}"
|
||||
)
|
||||
raise RuntimeError(f"Source image file does not exist: {source_path}")
|
||||
return source_path, False
|
||||
|
||||
|
||||
def configure_image_user(
|
||||
image_user,
|
||||
frame_count=None,
|
||||
frame_start=1,
|
||||
frame_offset=0,
|
||||
use_cyclic=None,
|
||||
):
|
||||
"""Map the image to scene frames and enable auto refresh."""
|
||||
if image_user is None:
|
||||
return
|
||||
if hasattr(image_user, "frame_start"):
|
||||
image_user.frame_start = frame_start
|
||||
if hasattr(image_user, "frame_offset"):
|
||||
image_user.frame_offset = frame_offset
|
||||
if hasattr(image_user, "use_auto_refresh"):
|
||||
image_user.use_auto_refresh = True
|
||||
if use_cyclic is not None and hasattr(image_user, "use_cyclic"):
|
||||
image_user.use_cyclic = use_cyclic
|
||||
if frame_count is not None and hasattr(image_user, "frame_duration"):
|
||||
image_user.frame_duration = frame_count
|
||||
|
||||
|
||||
def pack_sequence_if_possible(image, expected_frames):
|
||||
"""Pack a frame sequence when Blender supports it; otherwise keep external files."""
|
||||
if expected_frames <= 1 or not image.has_data:
|
||||
return False
|
||||
def load_image_edit_result(source_image, output_path):
|
||||
"""Load and pack the image file returned by a Job."""
|
||||
output_path = Path(output_path)
|
||||
if not output_path.is_file():
|
||||
raise RuntimeError(f"Color result image does not exist: {output_path}")
|
||||
result_image = None
|
||||
try:
|
||||
image.pack()
|
||||
except (OSError, RuntimeError):
|
||||
return False
|
||||
if len(getattr(image, "packed_files", ())) >= expected_frames:
|
||||
return True
|
||||
try:
|
||||
image.unpack(method="REMOVE")
|
||||
except (OSError, RuntimeError):
|
||||
pass
|
||||
return False
|
||||
|
||||
|
||||
def sorted_frame_paths(directory, prefix=None):
|
||||
"""Return image files in a directory sorted by their trailing frame number."""
|
||||
def frame_key(path):
|
||||
match = re.search(r"(\d+)(?=\.[^.]+$)", path.name)
|
||||
return (int(match.group(1)) if match is not None else 0, path.name)
|
||||
|
||||
return sorted(
|
||||
(
|
||||
path
|
||||
for path in Path(directory).iterdir()
|
||||
if path.is_file()
|
||||
and path.suffix.lower() in IMAGE_NAME_SUFFIXES
|
||||
and (prefix is None or path.stem.startswith(prefix))
|
||||
),
|
||||
key=frame_key,
|
||||
)
|
||||
|
||||
|
||||
def force_sequence_source(image):
|
||||
"""Ensure a loaded image is a SEQUENCE datablock for its numbered frames."""
|
||||
if getattr(image, "source", None) == "SEQUENCE":
|
||||
return image
|
||||
try:
|
||||
image.source = "SEQUENCE"
|
||||
except (OSError, RuntimeError, TypeError) as error:
|
||||
raise RuntimeError(
|
||||
"Blender cannot load the frame sequence"
|
||||
) from error
|
||||
if getattr(image, "source", None) != "SEQUENCE":
|
||||
raise RuntimeError("Blender cannot load the frame sequence")
|
||||
return image
|
||||
result_image = bpy.data.images.load(str(output_path), check_existing=False)
|
||||
result_image.colorspace_settings.name = source_image.colorspace_settings.name
|
||||
result_image.alpha_mode = source_image.alpha_mode
|
||||
result_image.pack()
|
||||
result_image.name = source_image.name
|
||||
except (OSError, RuntimeError, TypeError, ValueError) as error:
|
||||
if result_image is not None:
|
||||
bpy.data.images.remove(result_image, do_unlink=True)
|
||||
raise RuntimeError("Unable to load the color result") from error
|
||||
return result_image
|
||||
|
||||
|
||||
def load_normal_result_image(path, source_object):
|
||||
@@ -556,22 +411,6 @@ def color_alpha_mode(color_space):
|
||||
return "STRAIGHT" if color_space in dict(VIEW_TRANSFORM_COLOR_SPACES).values() else "PREMUL"
|
||||
|
||||
|
||||
def configure_color_alpha(image):
|
||||
"""Apply the presentation mode to an owned Color image."""
|
||||
alpha_mode = color_alpha_mode(image.colorspace_settings.name)
|
||||
if image.alpha_mode != alpha_mode:
|
||||
pixels = (
|
||||
image_pixels(image)
|
||||
if getattr(image, "source", None) == "GENERATED"
|
||||
or getattr(image, "is_dirty", False)
|
||||
else None
|
||||
)
|
||||
image.alpha_mode = alpha_mode
|
||||
if pixels is not None:
|
||||
image.pixels.foreach_set(pixels)
|
||||
image.update()
|
||||
|
||||
|
||||
def create_image_edit_result(source_image, pixels, output_size):
|
||||
"""Create a packed Image datablock from edited RGBA pixels."""
|
||||
output_width, output_height = (int(value) for value in output_size)
|
||||
@@ -680,11 +519,6 @@ def replace_empty_image(source_object, result_image, *, placement_bounds=None,
|
||||
image_user = getattr(source_object, "image_user", None)
|
||||
original_timing = image_user_settings(source_object) if image_user is not None else None
|
||||
original_auto_refresh = getattr(image_user, "use_auto_refresh", False)
|
||||
frame_count = (
|
||||
video_total_frames(result_image)
|
||||
if is_animated_image(result_image)
|
||||
else None
|
||||
)
|
||||
alignment = (
|
||||
empty_display_alignment(
|
||||
source_object,
|
||||
@@ -695,13 +529,6 @@ def replace_empty_image(source_object, result_image, *, placement_bounds=None,
|
||||
if placement_bounds is not None
|
||||
else None
|
||||
)
|
||||
if frame_count is not None:
|
||||
(
|
||||
original_frame_start,
|
||||
_original_frame_offset,
|
||||
_original_frame_duration,
|
||||
original_use_cyclic,
|
||||
) = image_user_settings(source_object)
|
||||
|
||||
source_image.use_fake_user = True
|
||||
original_alignment = (
|
||||
@@ -720,13 +547,6 @@ def replace_empty_image(source_object, result_image, *, placement_bounds=None,
|
||||
final_image = source_image
|
||||
source_object.data = final_image
|
||||
source_object.use_empty_image_alpha = True
|
||||
if frame_count is not None:
|
||||
configure_image_user(
|
||||
image_user,
|
||||
frame_count,
|
||||
frame_start=original_frame_start,
|
||||
use_cyclic=original_use_cyclic,
|
||||
)
|
||||
if alignment is not None:
|
||||
source_object.empty_display_size = alignment[0]
|
||||
source_object.empty_image_offset = alignment[1]
|
||||
@@ -751,64 +571,6 @@ def replace_empty_image(source_object, result_image, *, placement_bounds=None,
|
||||
bpy.data.images.remove(result_image)
|
||||
|
||||
|
||||
def _load_packed_result_image(
|
||||
source_image,
|
||||
output_directory,
|
||||
*,
|
||||
frame_prefix,
|
||||
result_name,
|
||||
alpha_mode,
|
||||
):
|
||||
output_directory = Path(output_directory)
|
||||
color_paths = sorted_frame_paths(output_directory, frame_prefix)
|
||||
if not color_paths:
|
||||
raise RuntimeError(
|
||||
f"Color result has no images: {output_directory}"
|
||||
)
|
||||
source_color_space = source_image.colorspace_settings.name
|
||||
animated = is_animated_image(source_image)
|
||||
try:
|
||||
result_image = bpy.data.images.load(
|
||||
str(color_paths[0]),
|
||||
check_existing=False,
|
||||
)
|
||||
result_image.colorspace_settings.name = source_color_space
|
||||
result_image.alpha_mode = alpha_mode
|
||||
if animated:
|
||||
force_sequence_source(result_image)
|
||||
except (OSError, RuntimeError, TypeError, ValueError) as error:
|
||||
if "result_image" in locals():
|
||||
bpy.data.images.remove(result_image, do_unlink=True)
|
||||
raise RuntimeError("Unable to load the color result") from error
|
||||
|
||||
try:
|
||||
if animated:
|
||||
pack_sequence_if_possible(result_image, len(color_paths))
|
||||
else:
|
||||
result_image.pack()
|
||||
result_image.name = result_name
|
||||
except (OSError, RuntimeError, TypeError, ValueError) as error:
|
||||
bpy.data.images.remove(result_image, do_unlink=True)
|
||||
raise RuntimeError("Unable to prepare the color result") from error
|
||||
return result_image
|
||||
|
||||
|
||||
def load_image_edit_result(
|
||||
source_image,
|
||||
output_directory,
|
||||
*,
|
||||
frame_prefix="image.",
|
||||
):
|
||||
"""Load one packed Job result that directly edits the source Image."""
|
||||
return _load_packed_result_image(
|
||||
source_image,
|
||||
output_directory,
|
||||
frame_prefix=frame_prefix,
|
||||
result_name=source_image.name,
|
||||
alpha_mode=source_image.alpha_mode,
|
||||
)
|
||||
|
||||
|
||||
def image_empty_bounds(source_object):
|
||||
image_width, image_height = source_object.data.size
|
||||
if image_width <= 0 or image_height <= 0:
|
||||
|
||||
@@ -6,8 +6,6 @@ from uuid import uuid4
|
||||
import bpy
|
||||
|
||||
from .image import (
|
||||
configure_image_user,
|
||||
extract_frame_range,
|
||||
image_content_state,
|
||||
image_user_settings,
|
||||
is_animated_image,
|
||||
@@ -16,7 +14,6 @@ from .image import (
|
||||
prepare_image_input,
|
||||
replace_empty_image,
|
||||
restore_image_content,
|
||||
video_total_frames,
|
||||
)
|
||||
|
||||
|
||||
@@ -84,7 +81,6 @@ def replace_texture_image(node, result_image):
|
||||
original_timing = image_user_settings(node)
|
||||
original_auto_refresh = node.image_user.use_auto_refresh
|
||||
original_content = None
|
||||
frame_count = video_total_frames(result_image) if is_animated_image(result_image) else None
|
||||
try:
|
||||
if has_other_image_user(node):
|
||||
final_image = result_image
|
||||
@@ -94,13 +90,6 @@ def replace_texture_image(node, result_image):
|
||||
restore_image_content(source_image, result_content)
|
||||
final_image = source_image
|
||||
node.image = final_image
|
||||
if frame_count is not None:
|
||||
configure_image_user(
|
||||
node.image_user,
|
||||
frame_count,
|
||||
frame_start=original_timing[0],
|
||||
use_cyclic=original_timing[3],
|
||||
)
|
||||
except Exception:
|
||||
if original_content is not None:
|
||||
restore_image_content(source_image, original_content)
|
||||
@@ -122,8 +111,6 @@ class ImageEditTarget:
|
||||
owner: object
|
||||
image: object
|
||||
tree: object = None
|
||||
start_frame: int = 1
|
||||
frame_count: int = 1
|
||||
node_identity: str = ""
|
||||
|
||||
@classmethod
|
||||
@@ -132,7 +119,8 @@ class ImageEditTarget:
|
||||
if owner is None:
|
||||
raise RuntimeError("Select an Image Empty or a material Image Texture")
|
||||
image = owner_image(owner)
|
||||
start_frame, frame_count = extract_frame_range(image, owner.image_user)
|
||||
if is_animated_image(image):
|
||||
raise RuntimeError("Only still images are supported")
|
||||
tree = None if is_image_empty(owner) else owner.id_data
|
||||
identity = ""
|
||||
if tree is not None:
|
||||
@@ -141,7 +129,7 @@ class ImageEditTarget:
|
||||
identity = uuid4().hex
|
||||
owner["anyimage_identity"] = identity
|
||||
return cls(
|
||||
owner, image, tree, start_frame, frame_count, identity,
|
||||
owner, image, tree, identity,
|
||||
)
|
||||
|
||||
def validate(self):
|
||||
@@ -165,13 +153,11 @@ class ImageEditTarget:
|
||||
|
||||
def prepare(self, context):
|
||||
self.validate()
|
||||
if video_total_frames(self.image) > 1:
|
||||
context.scene.anyimage_settings.video_frames = self.frame_count
|
||||
return prepare_image_input(self.image, context.scene)
|
||||
|
||||
def apply(self, directory, *, frame_prefix="image."):
|
||||
def apply(self, output_path):
|
||||
self.validate()
|
||||
result_image = load_image_edit_result(self.image, directory, frame_prefix=frame_prefix)
|
||||
result_image = load_image_edit_result(self.image, output_path)
|
||||
return self.commit(result_image)
|
||||
|
||||
def commit(self, result_image, *, isolate_shared=False):
|
||||
|
||||
@@ -10,7 +10,7 @@ from .image import (
|
||||
image_empty_bounds,
|
||||
is_image_empty,
|
||||
require_image_empty,
|
||||
video_total_frames,
|
||||
is_animated_image,
|
||||
)
|
||||
from .selection import ImageEditWarning, SelectionPath
|
||||
|
||||
@@ -1019,7 +1019,7 @@ class ImageGesture:
|
||||
return {"CANCELLED"}
|
||||
if source_object is None:
|
||||
return {"CANCELLED"}
|
||||
if video_total_frames(source_object.data) > 1:
|
||||
if is_animated_image(source_object.data):
|
||||
self.report({"WARNING"}, "Image gestures only support a still image")
|
||||
return {"CANCELLED"}
|
||||
self.source_object_name = source_object.name
|
||||
|
||||
@@ -1,17 +1,9 @@
|
||||
from .remove_background import RemoveImageBackground, RunBackgroundRemoval
|
||||
from .cutout_tool import (
|
||||
CutoutSelectionToShape,
|
||||
SelectCutoutSelection,
|
||||
)
|
||||
from .frame_tool import FrameImages
|
||||
from .cutout_tool import CLASSES as CUTOUT_CLASSES
|
||||
from .frame_tool import CLASSES as FRAME_CLASSES
|
||||
from .mask_tool import EditImageAlpha
|
||||
from .rectify_tool import RectifyImagePerspective
|
||||
from .convert_to_plane import (
|
||||
ConvertToDepthPlane,
|
||||
ConvertToReliefPlane,
|
||||
ConvertToPlane,
|
||||
GenerateDepthPlane,
|
||||
)
|
||||
from .rectify_tool import CLASSES as RECTIFY_CLASSES
|
||||
from .convert_to_plane import CLASSES as PLANE_CLASSES
|
||||
from .ai_setup import (
|
||||
ClearModels,
|
||||
DownloadModel,
|
||||
@@ -25,21 +17,17 @@ from .convert_to_panorama import CLASSES as PANORAMA_CLASSES
|
||||
|
||||
CLASSES = (
|
||||
*PANORAMA_CLASSES,
|
||||
*PLANE_CLASSES,
|
||||
*CUTOUT_CLASSES,
|
||||
SetupAIEnvironment,
|
||||
OpenAIEnvironmentSettings,
|
||||
DownloadModel,
|
||||
DownloadRequiredModels,
|
||||
GenerateDepthPlane,
|
||||
CutoutSelectionToShape,
|
||||
SelectCutoutSelection,
|
||||
RunBackgroundRemoval,
|
||||
ClearModels,
|
||||
FrameImages,
|
||||
*FRAME_CLASSES,
|
||||
EditImageAlpha,
|
||||
RectifyImagePerspective,
|
||||
*RECTIFY_CLASSES,
|
||||
RemoveImageBackground,
|
||||
UpscaleImage,
|
||||
ConvertToPlane,
|
||||
ConvertToDepthPlane,
|
||||
ConvertToReliefPlane,
|
||||
)
|
||||
|
||||
@@ -21,11 +21,8 @@ class SetupAIEnvironment(bpy.types.Operator):
|
||||
bl_description = "Install the environment and required AI models"
|
||||
|
||||
use_mirror: bpy.props.BoolProperty(
|
||||
name="Use Mirror",
|
||||
description=(
|
||||
"Use the China mirrors for uv, Python and Python packages; "
|
||||
"model download sources are unchanged"
|
||||
),
|
||||
name="Use China Mirror",
|
||||
description="Download Python and packages from China mirrors",
|
||||
default=False,
|
||||
)
|
||||
|
||||
@@ -34,36 +31,24 @@ class SetupAIEnvironment(bpy.types.Operator):
|
||||
return not runtime.server_busy()
|
||||
|
||||
def invoke(self, context, _event):
|
||||
return context.window_manager.invoke_props_dialog(self, width=440)
|
||||
return context.window_manager.invoke_props_dialog(
|
||||
self, width=440, title="Set Up AI", confirm_text="Install"
|
||||
)
|
||||
|
||||
def draw(self, _context):
|
||||
layout = self.layout
|
||||
status = ai_status()
|
||||
if not status["environment_ready"]:
|
||||
required_size = sum(
|
||||
model_download_size(model_key)
|
||||
for model_key in DEFAULT_MODEL_KEYS
|
||||
model_download_size(model_key) for model_key in DEFAULT_MODEL_KEYS
|
||||
)
|
||||
layout.label(
|
||||
text="AnyImage AI features run in an isolated Python environment."
|
||||
)
|
||||
layout.label(
|
||||
text="This installs Python, ONNX Runtime and supporting packages."
|
||||
text="Set up AI features for depth, background removal and upscaling."
|
||||
)
|
||||
layout.label(
|
||||
text=(
|
||||
"It also downloads MoGe-2 ViT-S Normal, BiRefNet Lite and "
|
||||
"Real-ESRGAN General WDN x4v3 "
|
||||
f"({format_download_size(required_size)})."
|
||||
)
|
||||
)
|
||||
layout.label(
|
||||
text="Allow about 600 MB of downloads and 1 GB of disk space."
|
||||
)
|
||||
layout.label(
|
||||
text=(
|
||||
"Installation usually takes 5–20 minutes, "
|
||||
"depending on network speed."
|
||||
"Downloads the runtime and required models "
|
||||
f"({format_download_size(required_size)} for models)."
|
||||
)
|
||||
)
|
||||
layout.separator(type="LINE")
|
||||
@@ -71,7 +56,7 @@ class SetupAIEnvironment(bpy.types.Operator):
|
||||
return
|
||||
|
||||
if status["ready"]:
|
||||
layout.label(text="The AI Environment is ready.", icon="CHECKMARK")
|
||||
layout.label(text="AI Server is ready.", icon="CHECKMARK")
|
||||
return
|
||||
|
||||
labels = []
|
||||
@@ -83,12 +68,13 @@ class SetupAIEnvironment(bpy.types.Operator):
|
||||
for model_key in status["missing_models"]
|
||||
)
|
||||
layout.label(
|
||||
text="The AI Environment is already installed.",
|
||||
text="AI environment installed.",
|
||||
icon="CHECKMARK",
|
||||
)
|
||||
layout.label(text=f"Required models to download: {', '.join(labels)}")
|
||||
layout.label(text=f"Download size: {format_download_size(missing_size)}")
|
||||
layout.label(text="The download usually takes 2–10 minutes.")
|
||||
layout.label(text=f"Download: {', '.join(labels)}")
|
||||
layout.label(
|
||||
text=f"{format_download_size(missing_size)} · Usually 2–10 minutes."
|
||||
)
|
||||
|
||||
def execute(self, _context):
|
||||
status = ai_status()
|
||||
@@ -159,7 +145,8 @@ class DownloadRequiredModels(JobOperatorBase, bpy.types.Operator):
|
||||
|
||||
class ClearModels(bpy.types.Operator):
|
||||
bl_idname = "anyimage.clear_models"
|
||||
bl_label = "Clear Models"
|
||||
bl_label = "Unload Models"
|
||||
bl_description = "Unload AI models to free memory. Downloaded model files are kept."
|
||||
|
||||
def execute(self, _context):
|
||||
try:
|
||||
|
||||
@@ -1,11 +1,12 @@
|
||||
from .operators import (
|
||||
CLASSES,
|
||||
PasteClipboardImage,
|
||||
TrackNativeCopy,
|
||||
register_keymaps,
|
||||
unregister_keymaps,
|
||||
)
|
||||
|
||||
CLASSES = (TrackNativeCopy, PasteClipboardImage)
|
||||
|
||||
__all__ = (
|
||||
"CLASSES",
|
||||
"PasteClipboardImage",
|
||||
|
||||
@@ -148,9 +148,6 @@ class PasteClipboardImage(bpy.types.Operator):
|
||||
return {"FINISHED"}
|
||||
|
||||
|
||||
CLASSES = (TrackNativeCopy, PasteClipboardImage)
|
||||
|
||||
|
||||
def unregister_keymaps():
|
||||
global _native_copy_token
|
||||
for keymap, keymap_item in _keymap_items:
|
||||
|
||||
@@ -3,7 +3,7 @@
|
||||
import bpy
|
||||
|
||||
from ...common.ai import invoke_ai_setup_if_needed, production_device, report_ai_error, require_environment, require_input_path, require_model
|
||||
from ...common.image import is_image_empty, video_total_frames
|
||||
from ...common.image import is_image_empty, is_animated_image
|
||||
from ...common.color_image import prepare_material_color_input, material_analysis_input, cleanup_material_color_input
|
||||
from ...preferences import addon_preferences, configured_max_ai_input_size
|
||||
from ...preferences import configured_geometry_model
|
||||
@@ -26,7 +26,7 @@ class GeneratePanorama(JobOperatorBase, bpy.types.Operator):
|
||||
source_object_name: bpy.props.StringProperty(options=_HIDDEN)
|
||||
source_identity: bpy.props.StringProperty(options=_HIDDEN)
|
||||
image_identity: bpy.props.StringProperty(options=_HIDDEN)
|
||||
mesh_detail: bpy.props.IntProperty(default=7, min=0, max=8, options=_HIDDEN)
|
||||
mesh_detail: bpy.props.IntProperty(default=8, min=0, max=8, options=_HIDDEN)
|
||||
model: bpy.props.StringProperty(default=DEFAULT_GEOMETRY_MODEL_KEY, options=_HIDDEN)
|
||||
resolution_level: bpy.props.IntProperty(default=5, min=0, max=9, options=_HIDDEN)
|
||||
max_input_size: bpy.props.IntProperty(default=DEFAULT_MAX_AI_INPUT_SIZE,
|
||||
@@ -39,7 +39,7 @@ class GeneratePanorama(JobOperatorBase, bpy.types.Operator):
|
||||
"input": str(require_input_path(self.input_path)),
|
||||
"device": production_device().lower(), "model": self.model,
|
||||
"resolution_level": int(self.resolution_level),
|
||||
"max_input_size": int(self.max_input_size), "video_frames": 1, "start_frame": 1,
|
||||
"max_input_size": int(self.max_input_size),
|
||||
}
|
||||
|
||||
def response(self, context, result):
|
||||
@@ -53,10 +53,10 @@ class GeneratePanorama(JobOperatorBase, bpy.types.Operator):
|
||||
class ConvertToPanorama(bpy.types.Operator):
|
||||
bl_idname = "anyimage.convert_to_panorama"
|
||||
bl_label = "Convert to Panorama"
|
||||
bl_description = "Convert a static 2:1 equirectangular Image Empty into a radial depth surface"
|
||||
bl_description = "Turn a still 2:1 panoramic image into a surrounding 3D surface using AI-estimated depth."
|
||||
bl_options = {"REGISTER", "UNDO"}
|
||||
|
||||
mesh_detail: bpy.props.IntProperty(name="Subdivide", default=7, min=0, max=8)
|
||||
mesh_detail: bpy.props.IntProperty(name="Subdivide", default=8, min=0, max=8)
|
||||
|
||||
@classmethod
|
||||
def poll(cls, context):
|
||||
@@ -68,7 +68,7 @@ class ConvertToPanorama(bpy.types.Operator):
|
||||
source = context.object
|
||||
image = source.data
|
||||
width, height = image.size
|
||||
if image.source in {"MOVIE", "SEQUENCE"} or video_total_frames(image) != 1 or height <= 0 or width != height * 2:
|
||||
if is_animated_image(image) or height <= 0 or width != height * 2:
|
||||
self.report({"ERROR"}, "Panorama conversion requires a static 2:1 equirectangular image")
|
||||
return {"CANCELLED"}
|
||||
setup = invoke_ai_setup_if_needed(self)
|
||||
|
||||
@@ -7,7 +7,10 @@ from .operators import (
|
||||
GenerateDepthPlane,
|
||||
)
|
||||
|
||||
CLASSES = (GenerateDepthPlane, ConvertToPlane, ConvertToDepthPlane, ConvertToReliefPlane)
|
||||
|
||||
__all__ = (
|
||||
"CLASSES",
|
||||
"ConvertToDepthPlane",
|
||||
"ConvertToReliefPlane",
|
||||
"ConvertToPlane",
|
||||
|
||||
@@ -75,18 +75,14 @@ class GenerateDepthPlane(JobOperatorBase, bpy.types.Operator):
|
||||
parameters = {
|
||||
"input": str(input_path),
|
||||
"device": production_device().lower(),
|
||||
"video_frames": 1,
|
||||
"start_frame": 1,
|
||||
"max_input_size": int(self.max_input_size),
|
||||
"plane_type": self.plane_type,
|
||||
}
|
||||
parameters.update(
|
||||
moge2_parameters(
|
||||
context.scene.anyimage_settings,
|
||||
input_path,
|
||||
self.model,
|
||||
self.resolution_level,
|
||||
1,
|
||||
)
|
||||
)
|
||||
return parameters
|
||||
@@ -175,7 +171,10 @@ class ConvertToPlane(bpy.types.Operator):
|
||||
class ConvertToDepthPlane(bpy.types.Operator):
|
||||
bl_idname = "anyimage.convert_to_depth_plane"
|
||||
bl_label = "Convert to Depth Plane"
|
||||
bl_description = "Convert the complete Image Empty into a camera-projected depth plane"
|
||||
bl_description = (
|
||||
"Use AI-estimated depth to turn the image into a 3D surface "
|
||||
"that follows the source image's perspective."
|
||||
)
|
||||
bl_options = {"REGISTER", "UNDO"}
|
||||
|
||||
mesh_detail: bpy.props.IntProperty(name="Subdivide", default=6, min=0, max=10)
|
||||
@@ -193,7 +192,7 @@ class ConvertToDepthPlane(bpy.types.Operator):
|
||||
class ConvertToReliefPlane(bpy.types.Operator):
|
||||
bl_idname = "anyimage.convert_to_relief_plane"
|
||||
bl_label = "Convert to Relief Plane"
|
||||
bl_description = "Convert the complete Image Empty into a fixed-base relief plane"
|
||||
bl_description = "Use AI-estimated depth to create a raised relief on a flat base."
|
||||
bl_options = {"REGISTER", "UNDO"}
|
||||
|
||||
mesh_detail: bpy.props.IntProperty(name="Subdivide", default=6, min=0, max=10)
|
||||
|
||||
@@ -1,6 +1,9 @@
|
||||
from .operators import CutoutSelectionToShape, CutoutTool, SelectCutoutSelection
|
||||
|
||||
CLASSES = (CutoutSelectionToShape, SelectCutoutSelection)
|
||||
|
||||
__all__ = (
|
||||
"CLASSES",
|
||||
"CutoutSelectionToShape",
|
||||
"CutoutTool",
|
||||
"SelectCutoutSelection",
|
||||
|
||||
396
src/anyimage/operators/cutout_tool/balloon.py
Normal file
396
src/anyimage/operators/cutout_tool/balloon.py
Normal file
@@ -0,0 +1,396 @@
|
||||
"""Compute Poisson and Teddy height profiles for Cutout meshes."""
|
||||
|
||||
import math
|
||||
|
||||
|
||||
DEFAULT_INFLATION = 0.45
|
||||
|
||||
|
||||
def distances_to_contour(points, contour):
|
||||
import numpy as np
|
||||
|
||||
points = np.asarray(points, dtype=np.float64)
|
||||
contour = np.asarray(contour, dtype=np.float64)
|
||||
squared = np.full(len(points), np.inf, dtype=np.float64)
|
||||
for start, end in zip(contour, np.roll(contour, -1, axis=0)):
|
||||
segment = end - start
|
||||
length_squared = float(segment.dot(segment))
|
||||
if length_squared <= 1e-16:
|
||||
continue
|
||||
factor = np.clip(
|
||||
(points - start) @ segment / length_squared,
|
||||
0.0,
|
||||
1.0,
|
||||
)
|
||||
nearest = start + factor[:, None] * segment
|
||||
squared = np.minimum(
|
||||
squared,
|
||||
np.sum((points - nearest) ** 2, axis=1),
|
||||
)
|
||||
return np.sqrt(squared)
|
||||
|
||||
|
||||
def chordal_axis(contour, epsilon=1e-9):
|
||||
import numpy as np
|
||||
from mathutils import Vector
|
||||
from mathutils.geometry import delaunay_2d_cdt
|
||||
|
||||
contour = np.asarray(contour, dtype=np.float64)
|
||||
boundary_count = len(contour)
|
||||
boundary_edges = [
|
||||
(index, (index + 1) % boundary_count) for index in range(boundary_count)
|
||||
]
|
||||
result = delaunay_2d_cdt(
|
||||
[Vector(point) for point in contour],
|
||||
boundary_edges,
|
||||
[list(range(boundary_count))],
|
||||
1,
|
||||
epsilon,
|
||||
)
|
||||
points = np.asarray(
|
||||
[tuple(point) for point in result[0]],
|
||||
dtype=np.float64,
|
||||
)
|
||||
triangles = [tuple(face) for face in result[2] if len(face) == 3]
|
||||
boundary = {tuple(sorted(edge)) for edge in boundary_edges}
|
||||
node_indices = {}
|
||||
nodes = []
|
||||
segments = set()
|
||||
|
||||
def edge_node(edge):
|
||||
key = ("edge", *tuple(sorted(edge)))
|
||||
if key not in node_indices:
|
||||
node_indices[key] = len(nodes)
|
||||
nodes.append((points[edge[0]] + points[edge[1]]) * 0.5)
|
||||
return node_indices[key]
|
||||
|
||||
for face_index, face in enumerate(triangles):
|
||||
edges = [
|
||||
(face[0], face[1]),
|
||||
(face[1], face[2]),
|
||||
(face[2], face[0]),
|
||||
]
|
||||
internal = [edge for edge in edges if tuple(sorted(edge)) not in boundary]
|
||||
internal_nodes = [edge_node(edge) for edge in internal]
|
||||
if len(internal_nodes) == 2:
|
||||
segments.add(tuple(sorted(internal_nodes)))
|
||||
elif len(internal_nodes) == 3:
|
||||
key = ("face", face_index)
|
||||
center_index = len(nodes)
|
||||
node_indices[key] = center_index
|
||||
nodes.append(points[list(face)].mean(axis=0))
|
||||
for midpoint_index in internal_nodes:
|
||||
segments.add(tuple(sorted((center_index, midpoint_index))))
|
||||
|
||||
if not nodes:
|
||||
center = contour.mean(axis=0)
|
||||
return (
|
||||
np.asarray((center,), dtype=np.float64),
|
||||
[],
|
||||
np.asarray(
|
||||
(distances_to_contour((center,), contour)[0],),
|
||||
dtype=np.float64,
|
||||
),
|
||||
)
|
||||
|
||||
nodes = np.asarray(nodes, dtype=np.float64)
|
||||
radii = distances_to_contour(nodes, contour)
|
||||
active = set(segments)
|
||||
while len(active) > 1:
|
||||
degree = [0] * len(nodes)
|
||||
for start, end in active:
|
||||
degree[start] += 1
|
||||
degree[end] += 1
|
||||
removable = []
|
||||
for edge in active:
|
||||
start, end = edge
|
||||
leaf = start if degree[start] == 1 else end if degree[end] == 1 else None
|
||||
if leaf is None:
|
||||
continue
|
||||
length = float(np.linalg.norm(nodes[start] - nodes[end]))
|
||||
if length < max(radii[leaf] * 0.5, epsilon):
|
||||
removable.append(edge)
|
||||
if not removable:
|
||||
break
|
||||
active.difference_update(removable)
|
||||
|
||||
if active:
|
||||
used = sorted({index for edge in active for index in edge})
|
||||
remap = {old: new for new, old in enumerate(used)}
|
||||
nodes = nodes[used]
|
||||
radii = radii[used]
|
||||
active = {tuple(sorted((remap[start], remap[end]))) for start, end in active}
|
||||
else:
|
||||
widest = int(np.argmax(radii))
|
||||
nodes = nodes[[widest]]
|
||||
radii = radii[[widest]]
|
||||
|
||||
return nodes, sorted(active), radii
|
||||
|
||||
|
||||
def smooth_interior_heights(
|
||||
heights,
|
||||
planar_faces,
|
||||
boundary_mask,
|
||||
iterations=2,
|
||||
factor=0.3,
|
||||
):
|
||||
import numpy as np
|
||||
|
||||
values = np.asarray(heights, dtype=np.float64).copy()
|
||||
boundary_mask = np.asarray(boundary_mask, dtype=bool)
|
||||
neighbors = [set() for _index in range(len(values))]
|
||||
for face in planar_faces:
|
||||
for start, end in zip(face, face[1:] + face[:1]):
|
||||
neighbors[start].add(end)
|
||||
neighbors[end].add(start)
|
||||
for _iteration in range(iterations):
|
||||
updated = values.copy()
|
||||
for index in np.flatnonzero(~boundary_mask):
|
||||
adjacent = [
|
||||
neighbor for neighbor in neighbors[index] if not boundary_mask[neighbor]
|
||||
]
|
||||
if adjacent:
|
||||
average = float(values[adjacent].mean())
|
||||
updated[index] = values[index] * (1.0 - factor) + average * factor
|
||||
updated[boundary_mask] = 0.0
|
||||
values = updated
|
||||
return values
|
||||
|
||||
|
||||
def teddy_balloon_heights(
|
||||
planar_points,
|
||||
planar_faces,
|
||||
contour,
|
||||
boundary_mask,
|
||||
inflation,
|
||||
):
|
||||
import numpy as np
|
||||
|
||||
points = np.asarray(planar_points, dtype=np.float64)
|
||||
boundary_mask = np.asarray(boundary_mask, dtype=bool)
|
||||
nodes, segments, radii = chordal_axis(contour)
|
||||
squared_heights = np.zeros(len(points), dtype=np.float64)
|
||||
|
||||
for start, end in segments:
|
||||
axis_start = nodes[start]
|
||||
axis_end = nodes[end]
|
||||
axis = axis_end - axis_start
|
||||
length_squared = float(axis.dot(axis))
|
||||
if length_squared <= 1e-16:
|
||||
continue
|
||||
factors = np.clip(
|
||||
(points - axis_start) @ axis / length_squared,
|
||||
0.0,
|
||||
1.0,
|
||||
)
|
||||
centers = axis_start + factors[:, None] * axis
|
||||
local_radii = radii[start] + factors * (radii[end] - radii[start])
|
||||
distance_squared = np.sum((points - centers) ** 2, axis=1)
|
||||
squared_heights = np.maximum(
|
||||
squared_heights,
|
||||
local_radii * local_radii - distance_squared,
|
||||
)
|
||||
|
||||
degree = np.zeros(len(nodes), dtype=np.int64)
|
||||
for start, end in segments:
|
||||
degree[start] += 1
|
||||
degree[end] += 1
|
||||
for index in np.flatnonzero(degree != 2):
|
||||
distance_squared = np.sum((points - nodes[index]) ** 2, axis=1)
|
||||
squared_heights = np.maximum(
|
||||
squared_heights,
|
||||
radii[index] * radii[index] - distance_squared,
|
||||
)
|
||||
|
||||
missing = (~boundary_mask) & (squared_heights <= 0.0)
|
||||
if np.any(missing):
|
||||
fallback = distances_to_contour(points[missing], contour)
|
||||
squared_heights[missing] = fallback * fallback
|
||||
heights = inflation * np.sqrt(np.maximum(squared_heights, 0.0))
|
||||
heights[boundary_mask] = 0.0
|
||||
return smooth_interior_heights(
|
||||
heights,
|
||||
planar_faces,
|
||||
boundary_mask,
|
||||
)
|
||||
|
||||
|
||||
def cotangent_poisson_field(
|
||||
planar_points,
|
||||
planar_faces,
|
||||
boundary_mask,
|
||||
tolerance=1e-8,
|
||||
):
|
||||
import numpy as np
|
||||
|
||||
points = np.asarray(planar_points, dtype=np.float64)
|
||||
faces = np.asarray(planar_faces, dtype=np.int64)
|
||||
if faces.size == 0:
|
||||
faces = np.empty((0, 3), dtype=np.int64)
|
||||
if faces.ndim != 2 or faces.shape[1] != 3:
|
||||
raise ValueError("Balloon profile faces must be triangles")
|
||||
boundary_mask = np.asarray(boundary_mask, dtype=bool)
|
||||
triangles = points[faces]
|
||||
a, b, c = (triangles[:, index] for index in range(3))
|
||||
cross = np.abs(
|
||||
(b[:, 0] - a[:, 0]) * (c[:, 1] - a[:, 1])
|
||||
- (b[:, 1] - a[:, 1]) * (c[:, 0] - a[:, 0])
|
||||
)
|
||||
usable = cross > 1e-14
|
||||
faces = faces[usable]
|
||||
a, b, c = a[usable], b[usable], c[usable]
|
||||
cross = cross[usable]
|
||||
loads = np.bincount(
|
||||
faces.ravel(),
|
||||
weights=np.repeat(cross / 6.0, 3),
|
||||
minlength=len(points),
|
||||
)
|
||||
edges = np.concatenate(
|
||||
(
|
||||
faces[:, (1, 2)],
|
||||
faces[:, (0, 2)],
|
||||
faces[:, (0, 1)],
|
||||
),
|
||||
axis=0,
|
||||
)
|
||||
edge_weights = np.concatenate(
|
||||
(
|
||||
np.sum((b - a) * (c - a), axis=1),
|
||||
np.sum((a - b) * (c - b), axis=1),
|
||||
np.sum((a - c) * (b - c), axis=1),
|
||||
)
|
||||
) / np.tile(cross * 2.0, 3)
|
||||
edges.sort(axis=1)
|
||||
edges, inverse = np.unique(edges, axis=0, return_inverse=True)
|
||||
edge_weights = np.bincount(
|
||||
inverse,
|
||||
weights=edge_weights,
|
||||
minlength=len(edges),
|
||||
)
|
||||
|
||||
interior = np.flatnonzero(~boundary_mask)
|
||||
field = np.zeros(len(points), dtype=np.float64)
|
||||
if not len(interior):
|
||||
return field
|
||||
compact = np.full(len(points), -1, dtype=np.int64)
|
||||
compact[interior] = np.arange(len(interior), dtype=np.int64)
|
||||
# Constrained triangulation can create negative cotangent weights.
|
||||
# Clamping them keeps the Dirichlet system positive definite.
|
||||
edge_weights = np.maximum(edge_weights, 0.0)
|
||||
usable = edge_weights > 1e-14
|
||||
compact_edges = compact[edges[usable]]
|
||||
edge_weights = edge_weights[usable]
|
||||
start_compact = compact_edges[:, 0]
|
||||
end_compact = compact_edges[:, 1]
|
||||
start_usable = start_compact >= 0
|
||||
end_usable = end_compact >= 0
|
||||
diagonal = np.zeros(len(interior), dtype=np.float64)
|
||||
diagonal += np.bincount(
|
||||
start_compact[start_usable],
|
||||
weights=edge_weights[start_usable],
|
||||
minlength=len(interior),
|
||||
)
|
||||
diagonal += np.bincount(
|
||||
end_compact[end_usable],
|
||||
weights=edge_weights[end_usable],
|
||||
minlength=len(interior),
|
||||
)
|
||||
if np.any(diagonal <= 1e-14):
|
||||
raise ValueError("Unable to build a stable Balloon profile")
|
||||
|
||||
interior_edges = start_usable & end_usable
|
||||
edge_starts = start_compact[interior_edges]
|
||||
edge_ends = end_compact[interior_edges]
|
||||
weights = edge_weights[interior_edges]
|
||||
directed_starts = np.concatenate((edge_starts, edge_ends))
|
||||
directed_ends = np.concatenate((edge_ends, edge_starts))
|
||||
directed_weights = np.concatenate((weights, weights))
|
||||
|
||||
def multiply(values):
|
||||
result = diagonal * values
|
||||
if len(directed_weights):
|
||||
result -= np.bincount(
|
||||
directed_starts,
|
||||
weights=directed_weights * values[directed_ends],
|
||||
minlength=len(values),
|
||||
)
|
||||
return result
|
||||
|
||||
right_hand_side = loads[interior]
|
||||
solution = np.zeros(len(interior), dtype=np.float64)
|
||||
residual = right_hand_side.copy()
|
||||
preconditioned = residual / diagonal
|
||||
direction = preconditioned.copy()
|
||||
residual_product = float(residual.dot(preconditioned))
|
||||
target = max(float(np.linalg.norm(right_hand_side)) * tolerance, 1e-12)
|
||||
max_iterations = max(64, min(len(interior) * 4, 4000))
|
||||
for _iteration in range(max_iterations):
|
||||
if float(np.linalg.norm(residual)) <= target:
|
||||
break
|
||||
multiplied = multiply(direction)
|
||||
denominator = float(direction.dot(multiplied))
|
||||
if denominator <= 0.0 or not math.isfinite(denominator):
|
||||
raise ValueError("Unable to solve the Balloon profile")
|
||||
step = residual_product / denominator
|
||||
solution += step * direction
|
||||
residual -= step * multiplied
|
||||
preconditioned = residual / diagonal
|
||||
next_product = float(residual.dot(preconditioned))
|
||||
if not math.isfinite(next_product):
|
||||
raise ValueError("Unable to solve the Balloon profile")
|
||||
if abs(residual_product) <= 1e-30:
|
||||
break
|
||||
direction = preconditioned + next_product / residual_product * direction
|
||||
residual_product = next_product
|
||||
|
||||
field[interior] = np.maximum(solution, 0.0)
|
||||
return field
|
||||
|
||||
|
||||
def poisson_balloon_profile(
|
||||
planar_points,
|
||||
planar_faces,
|
||||
boundary_mask,
|
||||
inflation=DEFAULT_INFLATION,
|
||||
):
|
||||
import numpy as np
|
||||
|
||||
field = cotangent_poisson_field(
|
||||
planar_points,
|
||||
planar_faces,
|
||||
boundary_mask,
|
||||
)
|
||||
return 2.0 * float(inflation) * np.sqrt(field)
|
||||
|
||||
|
||||
def balloon_profile(
|
||||
algorithm,
|
||||
planar_points,
|
||||
planar_faces,
|
||||
boundary_mask,
|
||||
components,
|
||||
inflation=DEFAULT_INFLATION,
|
||||
):
|
||||
import numpy as np
|
||||
|
||||
algorithm = str(algorithm).upper()
|
||||
if algorithm == "POISSON":
|
||||
return poisson_balloon_profile(
|
||||
planar_points,
|
||||
planar_faces,
|
||||
boundary_mask,
|
||||
inflation,
|
||||
)
|
||||
if algorithm != "TEDDY":
|
||||
raise ValueError(f"Unknown Balloon profile algorithm: {algorithm}")
|
||||
result = np.zeros(len(planar_points), dtype=np.float64)
|
||||
for start, end, component_faces, outer_contour in components:
|
||||
result[start:end] = teddy_balloon_heights(
|
||||
np.asarray(planar_points)[start:end],
|
||||
component_faces,
|
||||
outer_contour,
|
||||
np.asarray(boundary_mask)[start:end],
|
||||
inflation,
|
||||
)
|
||||
return result
|
||||
169
src/anyimage/operators/cutout_tool/depth_calibration.py
Normal file
169
src/anyimage/operators/cutout_tool/depth_calibration.py
Normal file
@@ -0,0 +1,169 @@
|
||||
"""Calibrate metric reference depth and the Depth Balloon base plane."""
|
||||
|
||||
PLANE_VARIANCE_FLOOR_RATIO = 0.025
|
||||
DEPTH_REFERENCE_PERCENTILE = 95.0
|
||||
ADAPTIVE_DEPTH_TRANSPARENCY_RATIO = 0.05
|
||||
DEFAULT_DEPTH_BASELINE = 8.0
|
||||
DEFAULT_DEPTH_DIRECTION = (0.0, 0.0, 1.0)
|
||||
|
||||
|
||||
def use_adaptive_depth_calibration(selection_transparent_fraction):
|
||||
"""Return whether the Selection input has enough transparent context."""
|
||||
return float(selection_transparent_fraction) >= ADAPTIVE_DEPTH_TRANSPARENCY_RATIO
|
||||
|
||||
|
||||
def metric_depth_reference(camera_depth, valid, uniform_scale):
|
||||
"""Return the shared upper-percentile scene depth for both Depth shapes."""
|
||||
import numpy as np
|
||||
|
||||
camera_depth = np.asarray(camera_depth, dtype=np.float32)
|
||||
valid = np.asarray(valid, dtype=bool)
|
||||
if valid.shape != camera_depth.shape:
|
||||
raise ValueError("Depth values and validity mask dimensions do not match")
|
||||
usable = valid & np.isfinite(camera_depth) & (camera_depth > 0.0)
|
||||
if not usable.any():
|
||||
raise ValueError("The depth field contains no valid metric depth")
|
||||
if not uniform_scale > 0.0:
|
||||
raise ValueError("Depth Uniform Scale must be positive")
|
||||
reference = np.percentile(
|
||||
camera_depth[usable],
|
||||
DEPTH_REFERENCE_PERCENTILE,
|
||||
)
|
||||
return float(reference) * float(uniform_scale)
|
||||
|
||||
|
||||
def _sample_valid_bilinear(values, valid, x, y):
|
||||
import numpy as np
|
||||
|
||||
height, width = values.shape
|
||||
x = np.clip(x, 0.0, max(width - 1, 0))
|
||||
y = np.clip(y, 0.0, max(height - 1, 0))
|
||||
x0 = np.floor(x).astype(np.int64)
|
||||
y0 = np.floor(y).astype(np.int64)
|
||||
x1 = np.minimum(x0 + 1, width - 1)
|
||||
y1 = np.minimum(y0 + 1, height - 1)
|
||||
fx = x - x0
|
||||
fy = y - y0
|
||||
samples = np.stack(
|
||||
(
|
||||
values[y0, x0],
|
||||
values[y0, x1],
|
||||
values[y1, x0],
|
||||
values[y1, x1],
|
||||
),
|
||||
axis=1,
|
||||
)
|
||||
weights = np.stack(
|
||||
(
|
||||
(1.0 - fx) * (1.0 - fy),
|
||||
fx * (1.0 - fy),
|
||||
(1.0 - fx) * fy,
|
||||
fx * fy,
|
||||
),
|
||||
axis=1,
|
||||
)
|
||||
sample_valid = np.stack(
|
||||
(
|
||||
valid[y0, x0],
|
||||
valid[y0, x1],
|
||||
valid[y1, x0],
|
||||
valid[y1, x1],
|
||||
),
|
||||
axis=1,
|
||||
)
|
||||
weights *= sample_valid
|
||||
totals = weights.sum(axis=1)
|
||||
result = np.zeros(len(x), dtype=np.float32)
|
||||
usable = totals > 1e-8
|
||||
result[usable] = (samples[usable] * weights[usable]).sum(axis=1) / totals[usable]
|
||||
return result, usable
|
||||
|
||||
|
||||
def _weighted_plane(coordinates, values, weights):
|
||||
import numpy as np
|
||||
|
||||
total = float(np.sum(weights))
|
||||
if total <= 1e-12:
|
||||
return np.zeros(3, dtype=np.float32)
|
||||
center = np.sum(coordinates * weights[:, None], axis=0) / total
|
||||
value_center = float(np.sum(values * weights) / total)
|
||||
centered = coordinates - center
|
||||
covariance = (centered * weights[:, None]).T @ centered / total
|
||||
cross = centered.T @ (weights * (values - value_center)) / total
|
||||
eigenvalues, eigenvectors = np.linalg.eigh(covariance)
|
||||
largest = max(float(eigenvalues[-1]), 0.0)
|
||||
if largest <= 1e-12:
|
||||
slopes = np.zeros(2, dtype=np.float64)
|
||||
else:
|
||||
supported = np.maximum(eigenvalues, largest * PLANE_VARIANCE_FLOOR_RATIO)
|
||||
slopes = eigenvectors @ ((eigenvectors.T @ cross) / supported)
|
||||
intercept = value_center - float(center @ slopes)
|
||||
return np.asarray((*slopes, intercept), dtype=np.float32)
|
||||
|
||||
|
||||
def _stable_plane(coordinates, values):
|
||||
import numpy as np
|
||||
|
||||
coordinates = np.asarray(coordinates, dtype=np.float64)
|
||||
values = np.asarray(values, dtype=np.float64)
|
||||
if len(coordinates) < 2:
|
||||
return np.asarray((0.0, 0.0, np.median(values)), dtype=np.float32)
|
||||
weights = np.ones(len(values), dtype=np.float64)
|
||||
plane = np.zeros(3, dtype=np.float32)
|
||||
for _iteration in range(4):
|
||||
plane = _weighted_plane(coordinates, values, weights)
|
||||
residual = values - coordinates @ plane[:2] - plane[2]
|
||||
centered = residual - np.median(residual)
|
||||
scale = max(float(np.median(np.abs(centered))) * 1.4826, 1e-6)
|
||||
weights = np.minimum(
|
||||
1.0,
|
||||
(2.5 * scale) / np.maximum(np.abs(centered), 1e-12),
|
||||
)
|
||||
return plane
|
||||
|
||||
|
||||
def depth_balloon_projection_parameters(
|
||||
vertices,
|
||||
vertex_uv,
|
||||
camera_depth,
|
||||
valid,
|
||||
uniform_scale,
|
||||
):
|
||||
"""Fit a metric base plane and return its center depth and direction."""
|
||||
import numpy as np
|
||||
|
||||
vertices = np.asarray(vertices, dtype=np.float32)
|
||||
uv = np.asarray(vertex_uv, dtype=np.float32)
|
||||
if vertices.ndim != 2 or vertices.shape[1] != 3:
|
||||
raise ValueError("Depth Balloon vertices must have three components")
|
||||
if uv.shape != (len(vertices), 2):
|
||||
raise ValueError("Depth Balloon UV coordinates do not match its vertices")
|
||||
camera_depth = np.asarray(camera_depth, dtype=np.float32)
|
||||
valid = np.asarray(valid, dtype=bool)
|
||||
if valid.shape != camera_depth.shape:
|
||||
raise ValueError("Depth Balloon depth and mask dimensions do not match")
|
||||
if not uniform_scale > 0.0:
|
||||
raise ValueError("Depth Balloon Uniform Scale must be positive")
|
||||
height, width = camera_depth.shape
|
||||
sample_x = uv[:, 0] * max(width - 1, 0)
|
||||
sample_y = (1.0 - uv[:, 1]) * max(height - 1, 0)
|
||||
calibration_valid = valid & np.isfinite(camera_depth) & (camera_depth > 0.0)
|
||||
sampled, usable = _sample_valid_bilinear(
|
||||
camera_depth,
|
||||
calibration_valid,
|
||||
sample_x,
|
||||
sample_y,
|
||||
)
|
||||
if not usable.any():
|
||||
raise ValueError("MoGe-2 found no valid Depth Balloon samples")
|
||||
local_coordinates = vertices[:, :2]
|
||||
metric_depth = sampled[usable] * float(uniform_scale)
|
||||
plane = _stable_plane(local_coordinates[usable], metric_depth)
|
||||
reference = metric_depth_reference(
|
||||
camera_depth,
|
||||
calibration_valid,
|
||||
uniform_scale,
|
||||
)
|
||||
direction = np.asarray((plane[0], plane[1], 1.0), dtype=np.float64)
|
||||
direction /= np.linalg.norm(direction)
|
||||
return reference, tuple(float(value) for value in direction)
|
||||
@@ -5,6 +5,7 @@ from dataclasses import dataclass
|
||||
|
||||
from ...preferences import addon_preferences
|
||||
from ...common.image import image_empty_bounds
|
||||
from . import balloon
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
@@ -661,568 +662,6 @@ def triangulate_selection_contours(
|
||||
return (*result, components) if return_components else result
|
||||
|
||||
|
||||
DEFAULT_INFLATION = 0.45
|
||||
|
||||
|
||||
def distances_to_contour(points, contour):
|
||||
import numpy as np
|
||||
|
||||
points = np.asarray(points, dtype=np.float64)
|
||||
contour = np.asarray(contour, dtype=np.float64)
|
||||
squared = np.full(len(points), np.inf, dtype=np.float64)
|
||||
for start, end in zip(contour, np.roll(contour, -1, axis=0)):
|
||||
segment = end - start
|
||||
length_squared = float(segment.dot(segment))
|
||||
if length_squared <= 1e-16:
|
||||
continue
|
||||
factor = np.clip(
|
||||
(points - start) @ segment / length_squared,
|
||||
0.0,
|
||||
1.0,
|
||||
)
|
||||
nearest = start + factor[:, None] * segment
|
||||
squared = np.minimum(
|
||||
squared,
|
||||
np.sum((points - nearest) ** 2, axis=1),
|
||||
)
|
||||
return np.sqrt(squared)
|
||||
|
||||
|
||||
def chordal_axis(contour, epsilon=1e-9):
|
||||
import numpy as np
|
||||
from mathutils import Vector
|
||||
from mathutils.geometry import delaunay_2d_cdt
|
||||
|
||||
contour = np.asarray(contour, dtype=np.float64)
|
||||
boundary_count = len(contour)
|
||||
boundary_edges = [
|
||||
(index, (index + 1) % boundary_count) for index in range(boundary_count)
|
||||
]
|
||||
result = delaunay_2d_cdt(
|
||||
[Vector(point) for point in contour],
|
||||
boundary_edges,
|
||||
[list(range(boundary_count))],
|
||||
1,
|
||||
epsilon,
|
||||
)
|
||||
points = np.asarray(
|
||||
[tuple(point) for point in result[0]],
|
||||
dtype=np.float64,
|
||||
)
|
||||
triangles = [tuple(face) for face in result[2] if len(face) == 3]
|
||||
boundary = {tuple(sorted(edge)) for edge in boundary_edges}
|
||||
node_indices = {}
|
||||
nodes = []
|
||||
segments = set()
|
||||
|
||||
def edge_node(edge):
|
||||
key = ("edge", *tuple(sorted(edge)))
|
||||
if key not in node_indices:
|
||||
node_indices[key] = len(nodes)
|
||||
nodes.append((points[edge[0]] + points[edge[1]]) * 0.5)
|
||||
return node_indices[key]
|
||||
|
||||
for face_index, face in enumerate(triangles):
|
||||
edges = [
|
||||
(face[0], face[1]),
|
||||
(face[1], face[2]),
|
||||
(face[2], face[0]),
|
||||
]
|
||||
internal = [edge for edge in edges if tuple(sorted(edge)) not in boundary]
|
||||
internal_nodes = [edge_node(edge) for edge in internal]
|
||||
if len(internal_nodes) == 2:
|
||||
segments.add(tuple(sorted(internal_nodes)))
|
||||
elif len(internal_nodes) == 3:
|
||||
key = ("face", face_index)
|
||||
center_index = len(nodes)
|
||||
node_indices[key] = center_index
|
||||
nodes.append(points[list(face)].mean(axis=0))
|
||||
for midpoint_index in internal_nodes:
|
||||
segments.add(tuple(sorted((center_index, midpoint_index))))
|
||||
|
||||
if not nodes:
|
||||
center = contour.mean(axis=0)
|
||||
return (
|
||||
np.asarray((center,), dtype=np.float64),
|
||||
[],
|
||||
np.asarray(
|
||||
(distances_to_contour((center,), contour)[0],),
|
||||
dtype=np.float64,
|
||||
),
|
||||
)
|
||||
|
||||
nodes = np.asarray(nodes, dtype=np.float64)
|
||||
radii = distances_to_contour(nodes, contour)
|
||||
active = set(segments)
|
||||
while len(active) > 1:
|
||||
degree = [0] * len(nodes)
|
||||
for start, end in active:
|
||||
degree[start] += 1
|
||||
degree[end] += 1
|
||||
removable = []
|
||||
for edge in active:
|
||||
start, end = edge
|
||||
leaf = start if degree[start] == 1 else end if degree[end] == 1 else None
|
||||
if leaf is None:
|
||||
continue
|
||||
length = float(np.linalg.norm(nodes[start] - nodes[end]))
|
||||
if length < max(radii[leaf] * 0.5, epsilon):
|
||||
removable.append(edge)
|
||||
if not removable:
|
||||
break
|
||||
active.difference_update(removable)
|
||||
|
||||
if active:
|
||||
used = sorted({index for edge in active for index in edge})
|
||||
remap = {old: new for new, old in enumerate(used)}
|
||||
nodes = nodes[used]
|
||||
radii = radii[used]
|
||||
active = {tuple(sorted((remap[start], remap[end]))) for start, end in active}
|
||||
else:
|
||||
widest = int(np.argmax(radii))
|
||||
nodes = nodes[[widest]]
|
||||
radii = radii[[widest]]
|
||||
|
||||
return nodes, sorted(active), radii
|
||||
|
||||
|
||||
def smooth_interior_heights(
|
||||
heights,
|
||||
planar_faces,
|
||||
boundary_mask,
|
||||
iterations=2,
|
||||
factor=0.3,
|
||||
):
|
||||
import numpy as np
|
||||
|
||||
values = np.asarray(heights, dtype=np.float64).copy()
|
||||
boundary_mask = np.asarray(boundary_mask, dtype=bool)
|
||||
neighbors = [set() for _index in range(len(values))]
|
||||
for face in planar_faces:
|
||||
for start, end in zip(face, face[1:] + face[:1]):
|
||||
neighbors[start].add(end)
|
||||
neighbors[end].add(start)
|
||||
for _iteration in range(iterations):
|
||||
updated = values.copy()
|
||||
for index in np.flatnonzero(~boundary_mask):
|
||||
adjacent = [
|
||||
neighbor for neighbor in neighbors[index] if not boundary_mask[neighbor]
|
||||
]
|
||||
if adjacent:
|
||||
average = float(values[adjacent].mean())
|
||||
updated[index] = values[index] * (1.0 - factor) + average * factor
|
||||
updated[boundary_mask] = 0.0
|
||||
values = updated
|
||||
return values
|
||||
|
||||
|
||||
def teddy_balloon_heights(
|
||||
planar_points,
|
||||
planar_faces,
|
||||
contour,
|
||||
boundary_mask,
|
||||
inflation,
|
||||
):
|
||||
import numpy as np
|
||||
|
||||
points = np.asarray(planar_points, dtype=np.float64)
|
||||
boundary_mask = np.asarray(boundary_mask, dtype=bool)
|
||||
nodes, segments, radii = chordal_axis(contour)
|
||||
squared_heights = np.zeros(len(points), dtype=np.float64)
|
||||
|
||||
for start, end in segments:
|
||||
axis_start = nodes[start]
|
||||
axis_end = nodes[end]
|
||||
axis = axis_end - axis_start
|
||||
length_squared = float(axis.dot(axis))
|
||||
if length_squared <= 1e-16:
|
||||
continue
|
||||
factors = np.clip(
|
||||
(points - axis_start) @ axis / length_squared,
|
||||
0.0,
|
||||
1.0,
|
||||
)
|
||||
centers = axis_start + factors[:, None] * axis
|
||||
local_radii = radii[start] + factors * (radii[end] - radii[start])
|
||||
distance_squared = np.sum((points - centers) ** 2, axis=1)
|
||||
squared_heights = np.maximum(
|
||||
squared_heights,
|
||||
local_radii * local_radii - distance_squared,
|
||||
)
|
||||
|
||||
degree = np.zeros(len(nodes), dtype=np.int64)
|
||||
for start, end in segments:
|
||||
degree[start] += 1
|
||||
degree[end] += 1
|
||||
for index in np.flatnonzero(degree != 2):
|
||||
distance_squared = np.sum((points - nodes[index]) ** 2, axis=1)
|
||||
squared_heights = np.maximum(
|
||||
squared_heights,
|
||||
radii[index] * radii[index] - distance_squared,
|
||||
)
|
||||
|
||||
missing = (~boundary_mask) & (squared_heights <= 0.0)
|
||||
if np.any(missing):
|
||||
fallback = distances_to_contour(points[missing], contour)
|
||||
squared_heights[missing] = fallback * fallback
|
||||
heights = inflation * np.sqrt(np.maximum(squared_heights, 0.0))
|
||||
heights[boundary_mask] = 0.0
|
||||
return smooth_interior_heights(
|
||||
heights,
|
||||
planar_faces,
|
||||
boundary_mask,
|
||||
)
|
||||
|
||||
|
||||
def cotangent_poisson_field(
|
||||
planar_points,
|
||||
planar_faces,
|
||||
boundary_mask,
|
||||
tolerance=1e-8,
|
||||
):
|
||||
import numpy as np
|
||||
|
||||
points = np.asarray(planar_points, dtype=np.float64)
|
||||
faces = np.asarray(planar_faces, dtype=np.int64)
|
||||
if faces.size == 0:
|
||||
faces = np.empty((0, 3), dtype=np.int64)
|
||||
if faces.ndim != 2 or faces.shape[1] != 3:
|
||||
raise ValueError("Balloon profile faces must be triangles")
|
||||
boundary_mask = np.asarray(boundary_mask, dtype=bool)
|
||||
triangles = points[faces]
|
||||
a, b, c = (triangles[:, index] for index in range(3))
|
||||
cross = np.abs(
|
||||
(b[:, 0] - a[:, 0]) * (c[:, 1] - a[:, 1])
|
||||
- (b[:, 1] - a[:, 1]) * (c[:, 0] - a[:, 0])
|
||||
)
|
||||
usable = cross > 1e-14
|
||||
faces = faces[usable]
|
||||
a, b, c = a[usable], b[usable], c[usable]
|
||||
cross = cross[usable]
|
||||
loads = np.bincount(
|
||||
faces.ravel(),
|
||||
weights=np.repeat(cross / 6.0, 3),
|
||||
minlength=len(points),
|
||||
)
|
||||
edges = np.concatenate(
|
||||
(
|
||||
faces[:, (1, 2)],
|
||||
faces[:, (0, 2)],
|
||||
faces[:, (0, 1)],
|
||||
),
|
||||
axis=0,
|
||||
)
|
||||
edge_weights = np.concatenate(
|
||||
(
|
||||
np.sum((b - a) * (c - a), axis=1),
|
||||
np.sum((a - b) * (c - b), axis=1),
|
||||
np.sum((a - c) * (b - c), axis=1),
|
||||
)
|
||||
) / np.tile(cross * 2.0, 3)
|
||||
edges.sort(axis=1)
|
||||
edges, inverse = np.unique(edges, axis=0, return_inverse=True)
|
||||
edge_weights = np.bincount(
|
||||
inverse,
|
||||
weights=edge_weights,
|
||||
minlength=len(edges),
|
||||
)
|
||||
|
||||
interior = np.flatnonzero(~boundary_mask)
|
||||
field = np.zeros(len(points), dtype=np.float64)
|
||||
if not len(interior):
|
||||
return field
|
||||
compact = np.full(len(points), -1, dtype=np.int64)
|
||||
compact[interior] = np.arange(len(interior), dtype=np.int64)
|
||||
# Constrained triangulation can create negative cotangent weights.
|
||||
# Clamping them keeps the Dirichlet system positive definite.
|
||||
edge_weights = np.maximum(edge_weights, 0.0)
|
||||
usable = edge_weights > 1e-14
|
||||
compact_edges = compact[edges[usable]]
|
||||
edge_weights = edge_weights[usable]
|
||||
start_compact = compact_edges[:, 0]
|
||||
end_compact = compact_edges[:, 1]
|
||||
start_usable = start_compact >= 0
|
||||
end_usable = end_compact >= 0
|
||||
diagonal = np.zeros(len(interior), dtype=np.float64)
|
||||
diagonal += np.bincount(
|
||||
start_compact[start_usable],
|
||||
weights=edge_weights[start_usable],
|
||||
minlength=len(interior),
|
||||
)
|
||||
diagonal += np.bincount(
|
||||
end_compact[end_usable],
|
||||
weights=edge_weights[end_usable],
|
||||
minlength=len(interior),
|
||||
)
|
||||
if np.any(diagonal <= 1e-14):
|
||||
raise ValueError("Unable to build a stable Balloon profile")
|
||||
|
||||
interior_edges = start_usable & end_usable
|
||||
edge_starts = start_compact[interior_edges]
|
||||
edge_ends = end_compact[interior_edges]
|
||||
weights = edge_weights[interior_edges]
|
||||
directed_starts = np.concatenate((edge_starts, edge_ends))
|
||||
directed_ends = np.concatenate((edge_ends, edge_starts))
|
||||
directed_weights = np.concatenate((weights, weights))
|
||||
|
||||
def multiply(values):
|
||||
result = diagonal * values
|
||||
if len(directed_weights):
|
||||
result -= np.bincount(
|
||||
directed_starts,
|
||||
weights=directed_weights * values[directed_ends],
|
||||
minlength=len(values),
|
||||
)
|
||||
return result
|
||||
|
||||
right_hand_side = loads[interior]
|
||||
solution = np.zeros(len(interior), dtype=np.float64)
|
||||
residual = right_hand_side.copy()
|
||||
preconditioned = residual / diagonal
|
||||
direction = preconditioned.copy()
|
||||
residual_product = float(residual.dot(preconditioned))
|
||||
target = max(float(np.linalg.norm(right_hand_side)) * tolerance, 1e-12)
|
||||
max_iterations = max(64, min(len(interior) * 4, 4000))
|
||||
for _iteration in range(max_iterations):
|
||||
if float(np.linalg.norm(residual)) <= target:
|
||||
break
|
||||
multiplied = multiply(direction)
|
||||
denominator = float(direction.dot(multiplied))
|
||||
if denominator <= 0.0 or not math.isfinite(denominator):
|
||||
raise ValueError("Unable to solve the Balloon profile")
|
||||
step = residual_product / denominator
|
||||
solution += step * direction
|
||||
residual -= step * multiplied
|
||||
preconditioned = residual / diagonal
|
||||
next_product = float(residual.dot(preconditioned))
|
||||
if not math.isfinite(next_product):
|
||||
raise ValueError("Unable to solve the Balloon profile")
|
||||
if abs(residual_product) <= 1e-30:
|
||||
break
|
||||
direction = preconditioned + next_product / residual_product * direction
|
||||
residual_product = next_product
|
||||
|
||||
field[interior] = np.maximum(solution, 0.0)
|
||||
return field
|
||||
|
||||
|
||||
def poisson_balloon_profile(
|
||||
planar_points,
|
||||
planar_faces,
|
||||
boundary_mask,
|
||||
inflation=DEFAULT_INFLATION,
|
||||
):
|
||||
import numpy as np
|
||||
|
||||
field = cotangent_poisson_field(
|
||||
planar_points,
|
||||
planar_faces,
|
||||
boundary_mask,
|
||||
)
|
||||
return 2.0 * float(inflation) * np.sqrt(field)
|
||||
|
||||
|
||||
def balloon_profile(
|
||||
algorithm,
|
||||
planar_points,
|
||||
planar_faces,
|
||||
boundary_mask,
|
||||
components,
|
||||
inflation=DEFAULT_INFLATION,
|
||||
):
|
||||
import numpy as np
|
||||
|
||||
algorithm = str(algorithm).upper()
|
||||
if algorithm == "POISSON":
|
||||
return poisson_balloon_profile(
|
||||
planar_points,
|
||||
planar_faces,
|
||||
boundary_mask,
|
||||
inflation,
|
||||
)
|
||||
if algorithm != "TEDDY":
|
||||
raise ValueError(f"Unknown Balloon profile algorithm: {algorithm}")
|
||||
result = np.zeros(len(planar_points), dtype=np.float64)
|
||||
for start, end, component_faces, outer_contour in components:
|
||||
result[start:end] = teddy_balloon_heights(
|
||||
np.asarray(planar_points)[start:end],
|
||||
component_faces,
|
||||
outer_contour,
|
||||
np.asarray(boundary_mask)[start:end],
|
||||
inflation,
|
||||
)
|
||||
return result
|
||||
|
||||
|
||||
PLANE_VARIANCE_FLOOR_RATIO = 0.025
|
||||
DEPTH_REFERENCE_PERCENTILE = 95.0
|
||||
ADAPTIVE_DEPTH_TRANSPARENCY_RATIO = 0.05
|
||||
DEFAULT_DEPTH_BASELINE = 8.0
|
||||
DEFAULT_DEPTH_DIRECTION = (0.0, 0.0, 1.0)
|
||||
|
||||
|
||||
def use_adaptive_depth_calibration(selection_transparent_fraction):
|
||||
"""Return whether the Selection input has enough transparent context."""
|
||||
return float(selection_transparent_fraction) >= ADAPTIVE_DEPTH_TRANSPARENCY_RATIO
|
||||
|
||||
|
||||
def metric_depth_reference(camera_depth, valid, uniform_scale):
|
||||
"""Return the shared upper-percentile scene depth for both Depth shapes."""
|
||||
import numpy as np
|
||||
|
||||
camera_depth = np.asarray(camera_depth, dtype=np.float32)
|
||||
valid = np.asarray(valid, dtype=bool)
|
||||
if valid.shape != camera_depth.shape:
|
||||
raise ValueError("Depth values and validity mask dimensions do not match")
|
||||
usable = valid & np.isfinite(camera_depth) & (camera_depth > 0.0)
|
||||
if not usable.any():
|
||||
raise ValueError("The depth field contains no valid metric depth")
|
||||
if not uniform_scale > 0.0:
|
||||
raise ValueError("Depth Uniform Scale must be positive")
|
||||
reference = np.percentile(
|
||||
camera_depth[usable],
|
||||
DEPTH_REFERENCE_PERCENTILE,
|
||||
)
|
||||
return float(reference) * float(uniform_scale)
|
||||
|
||||
|
||||
def _sample_valid_bilinear(values, valid, x, y):
|
||||
import numpy as np
|
||||
|
||||
height, width = values.shape
|
||||
x = np.clip(x, 0.0, max(width - 1, 0))
|
||||
y = np.clip(y, 0.0, max(height - 1, 0))
|
||||
x0 = np.floor(x).astype(np.int64)
|
||||
y0 = np.floor(y).astype(np.int64)
|
||||
x1 = np.minimum(x0 + 1, width - 1)
|
||||
y1 = np.minimum(y0 + 1, height - 1)
|
||||
fx = x - x0
|
||||
fy = y - y0
|
||||
samples = np.stack(
|
||||
(
|
||||
values[y0, x0],
|
||||
values[y0, x1],
|
||||
values[y1, x0],
|
||||
values[y1, x1],
|
||||
),
|
||||
axis=1,
|
||||
)
|
||||
weights = np.stack(
|
||||
(
|
||||
(1.0 - fx) * (1.0 - fy),
|
||||
fx * (1.0 - fy),
|
||||
(1.0 - fx) * fy,
|
||||
fx * fy,
|
||||
),
|
||||
axis=1,
|
||||
)
|
||||
sample_valid = np.stack(
|
||||
(
|
||||
valid[y0, x0],
|
||||
valid[y0, x1],
|
||||
valid[y1, x0],
|
||||
valid[y1, x1],
|
||||
),
|
||||
axis=1,
|
||||
)
|
||||
weights *= sample_valid
|
||||
totals = weights.sum(axis=1)
|
||||
result = np.zeros(len(x), dtype=np.float32)
|
||||
usable = totals > 1e-8
|
||||
result[usable] = (samples[usable] * weights[usable]).sum(axis=1) / totals[usable]
|
||||
return result, usable
|
||||
|
||||
|
||||
def _weighted_plane(coordinates, values, weights):
|
||||
import numpy as np
|
||||
|
||||
total = float(np.sum(weights))
|
||||
if total <= 1e-12:
|
||||
return np.zeros(3, dtype=np.float32)
|
||||
center = np.sum(coordinates * weights[:, None], axis=0) / total
|
||||
value_center = float(np.sum(values * weights) / total)
|
||||
centered = coordinates - center
|
||||
covariance = (centered * weights[:, None]).T @ centered / total
|
||||
cross = centered.T @ (weights * (values - value_center)) / total
|
||||
eigenvalues, eigenvectors = np.linalg.eigh(covariance)
|
||||
largest = max(float(eigenvalues[-1]), 0.0)
|
||||
if largest <= 1e-12:
|
||||
slopes = np.zeros(2, dtype=np.float64)
|
||||
else:
|
||||
supported = np.maximum(eigenvalues, largest * PLANE_VARIANCE_FLOOR_RATIO)
|
||||
slopes = eigenvectors @ ((eigenvectors.T @ cross) / supported)
|
||||
intercept = value_center - float(center @ slopes)
|
||||
return np.asarray((*slopes, intercept), dtype=np.float32)
|
||||
|
||||
|
||||
def _stable_plane(coordinates, values):
|
||||
import numpy as np
|
||||
|
||||
coordinates = np.asarray(coordinates, dtype=np.float64)
|
||||
values = np.asarray(values, dtype=np.float64)
|
||||
if len(coordinates) < 2:
|
||||
return np.asarray((0.0, 0.0, np.median(values)), dtype=np.float32)
|
||||
weights = np.ones(len(values), dtype=np.float64)
|
||||
plane = np.zeros(3, dtype=np.float32)
|
||||
for _iteration in range(4):
|
||||
plane = _weighted_plane(coordinates, values, weights)
|
||||
residual = values - coordinates @ plane[:2] - plane[2]
|
||||
centered = residual - np.median(residual)
|
||||
scale = max(float(np.median(np.abs(centered))) * 1.4826, 1e-6)
|
||||
weights = np.minimum(
|
||||
1.0,
|
||||
(2.5 * scale) / np.maximum(np.abs(centered), 1e-12),
|
||||
)
|
||||
return plane
|
||||
|
||||
|
||||
def depth_balloon_projection_parameters(
|
||||
vertices,
|
||||
vertex_uv,
|
||||
camera_depth,
|
||||
valid,
|
||||
uniform_scale,
|
||||
):
|
||||
"""Fit a metric base plane and return its center depth and direction."""
|
||||
import numpy as np
|
||||
|
||||
vertices = np.asarray(vertices, dtype=np.float32)
|
||||
uv = np.asarray(vertex_uv, dtype=np.float32)
|
||||
if vertices.ndim != 2 or vertices.shape[1] != 3:
|
||||
raise ValueError("Depth Balloon vertices must have three components")
|
||||
if uv.shape != (len(vertices), 2):
|
||||
raise ValueError("Depth Balloon UV coordinates do not match its vertices")
|
||||
camera_depth = np.asarray(camera_depth, dtype=np.float32)
|
||||
valid = np.asarray(valid, dtype=bool)
|
||||
if valid.shape != camera_depth.shape:
|
||||
raise ValueError("Depth Balloon depth and mask dimensions do not match")
|
||||
if not uniform_scale > 0.0:
|
||||
raise ValueError("Depth Balloon Uniform Scale must be positive")
|
||||
height, width = camera_depth.shape
|
||||
sample_x = uv[:, 0] * max(width - 1, 0)
|
||||
sample_y = (1.0 - uv[:, 1]) * max(height - 1, 0)
|
||||
calibration_valid = valid & np.isfinite(camera_depth) & (camera_depth > 0.0)
|
||||
sampled, usable = _sample_valid_bilinear(
|
||||
camera_depth,
|
||||
calibration_valid,
|
||||
sample_x,
|
||||
sample_y,
|
||||
)
|
||||
if not usable.any():
|
||||
raise ValueError("MoGe-2 found no valid Depth Balloon samples")
|
||||
local_coordinates = vertices[:, :2]
|
||||
metric_depth = sampled[usable] * float(uniform_scale)
|
||||
plane = _stable_plane(local_coordinates[usable], metric_depth)
|
||||
reference = metric_depth_reference(
|
||||
camera_depth,
|
||||
calibration_valid,
|
||||
uniform_scale,
|
||||
)
|
||||
direction = np.asarray((plane[0], plane[1], 1.0), dtype=np.float64)
|
||||
direction /= np.linalg.norm(direction)
|
||||
return reference, tuple(float(value) for value in direction)
|
||||
|
||||
|
||||
CUTOUT_ALPHA_THRESHOLD = 0.9
|
||||
CUTOUT_CLEANUP_UI_SCALE = 100.0
|
||||
|
||||
@@ -1313,7 +752,7 @@ def cutout_domain(
|
||||
planar = np.concatenate(refined_points)
|
||||
planar_faces, components = refined_faces, refined_components
|
||||
boundary = boundary_vertices_from_faces(len(planar), planar_faces)
|
||||
profile = balloon_profile(
|
||||
profile = balloon.balloon_profile(
|
||||
balloon_algorithm, planar, planar_faces, boundary, components
|
||||
)
|
||||
faces = np.asarray(planar_faces, dtype=np.int32)
|
||||
|
||||
@@ -31,9 +31,9 @@ def _pie_button(
|
||||
selection_path_json,
|
||||
generate_normal,
|
||||
point_spacing,
|
||||
inward_axis,
|
||||
alpha_threshold,
|
||||
fine_outline,
|
||||
gesture,
|
||||
):
|
||||
operator = pie.operator(
|
||||
"anyimage.cutout_selection_to_shape",
|
||||
@@ -45,9 +45,9 @@ def _pie_button(
|
||||
operator.shape = shape
|
||||
operator.generate_normal = generate_normal
|
||||
operator.point_spacing = point_spacing
|
||||
operator.inward_axis = inward_axis
|
||||
operator.alpha_threshold = alpha_threshold
|
||||
operator.fine_outline = fine_outline
|
||||
operator.gesture = gesture
|
||||
|
||||
|
||||
def open_shape_pie(
|
||||
@@ -55,6 +55,8 @@ def open_shape_pie(
|
||||
event,
|
||||
source_object,
|
||||
selection_path,
|
||||
*,
|
||||
gesture="LASSO",
|
||||
):
|
||||
source_name = source_object.name
|
||||
selection_path_json = selection_path.to_json()
|
||||
@@ -64,7 +66,6 @@ def open_shape_pie(
|
||||
generate_normal = ready and bool(
|
||||
getattr(settings, "cutout_generate_normal", False)
|
||||
)
|
||||
inward_axis = str(getattr(settings, "cutout_inward_axis", "NEGATIVE_Z"))
|
||||
alpha_threshold = settings.cutout_alpha_threshold
|
||||
fine_outline = settings.cutout_fine_outline
|
||||
|
||||
@@ -81,9 +82,9 @@ def open_shape_pie(
|
||||
selection_path_json,
|
||||
generate_normal,
|
||||
point_spacing,
|
||||
inward_axis,
|
||||
alpha_threshold,
|
||||
fine_outline,
|
||||
gesture,
|
||||
)
|
||||
|
||||
context.window_manager.popup_menu_pie(
|
||||
|
||||
@@ -15,23 +15,24 @@ from ...common.object import (
|
||||
set_modifier_input,
|
||||
)
|
||||
from ...common.node import load_node_group
|
||||
from .geometry import (
|
||||
CUTOUT_ALPHA_THRESHOLD,
|
||||
from .depth_calibration import (
|
||||
DEFAULT_DEPTH_BASELINE,
|
||||
DEFAULT_DEPTH_DIRECTION,
|
||||
depth_balloon_projection_parameters,
|
||||
metric_depth_reference,
|
||||
use_adaptive_depth_calibration,
|
||||
)
|
||||
from .geometry import (
|
||||
CUTOUT_ALPHA_THRESHOLD,
|
||||
crop_plane_bounds,
|
||||
cutout_cleanup_value,
|
||||
cutout_sampling_spacing,
|
||||
depth_balloon_projection_parameters,
|
||||
metric_depth_reference,
|
||||
selection_calibration,
|
||||
use_adaptive_depth_calibration,
|
||||
world_plane_metric,
|
||||
)
|
||||
from .shape import (
|
||||
BALLOON_ATTRIBUTE_NAME,
|
||||
NORMAL_SCALE_ATTRIBUTE_NAME,
|
||||
CUTOUT_X_TO_Z,
|
||||
CUTOUT_NODE_GROUP_NAMES,
|
||||
DEPTH_CUTOUT_SHAPES,
|
||||
)
|
||||
@@ -43,38 +44,11 @@ class NormalMode(StrEnum):
|
||||
OBJECT = "OBJECT"
|
||||
|
||||
|
||||
class CutoutInwardAxis(StrEnum):
|
||||
NEGATIVE_Z = "NEGATIVE_Z"
|
||||
POSITIVE_X = "POSITIVE_X"
|
||||
|
||||
|
||||
def _cutout_world_matrix(source_matrix, crop_center, inward_axis):
|
||||
def _cutout_world_matrix(source_matrix, crop_center):
|
||||
from mathutils import Matrix
|
||||
|
||||
center_x, center_y = (float(value) for value in crop_center)
|
||||
matrix = source_matrix @ Matrix.Translation((center_x, center_y, 0.0))
|
||||
if CutoutInwardAxis(inward_axis) == CutoutInwardAxis.POSITIVE_X:
|
||||
matrix @= Matrix(CUTOUT_X_TO_Z)
|
||||
return matrix
|
||||
|
||||
|
||||
def orient_object_normal_image(image, inward_axis):
|
||||
"""Convert native Z-axis Cutout normals to the selected object axes."""
|
||||
if CutoutInwardAxis(inward_axis) == CutoutInwardAxis.NEGATIVE_Z:
|
||||
return image
|
||||
import numpy as np
|
||||
|
||||
pixels = np.empty(int(image.size[0]) * int(image.size[1]) * 4, dtype=np.float32)
|
||||
image.pixels.foreach_get(pixels)
|
||||
rgba = pixels.reshape((-1, 4))
|
||||
normal = rgba[:, :3] * 2.0 - 1.0
|
||||
rgba[:, :3] = np.column_stack(
|
||||
(-normal[:, 2], -normal[:, 0], normal[:, 1])
|
||||
) * 0.5 + 0.5
|
||||
image.pixels.foreach_set(pixels)
|
||||
image.update()
|
||||
image.pack()
|
||||
return image
|
||||
return source_matrix @ Matrix.Translation((center_x, center_y, 0.0))
|
||||
|
||||
|
||||
def _assign_uv(mesh, uv):
|
||||
@@ -99,7 +73,7 @@ def _assign_float_point_attribute(mesh, name, values):
|
||||
|
||||
|
||||
def load_cutout_node_group(shape):
|
||||
return load_node_group(CUTOUT_NODE_GROUP_NAMES[shape], asset_revision=23 if shape == "DEPTH_SOLID" else 12 if shape == "DEPTH_BALLOON" else 11)
|
||||
return load_node_group(CUTOUT_NODE_GROUP_NAMES[shape], asset_revision=26 if shape == "DEPTH_SOLID" else 14 if shape == "DEPTH_BALLOON" else 13)
|
||||
|
||||
|
||||
def _connect_cutout_normal_scale(material):
|
||||
@@ -134,13 +108,12 @@ def create_shape_object(
|
||||
depth_metadata=None,
|
||||
normal_image=None,
|
||||
normal_mode=NormalMode.NONE,
|
||||
inward_axis=CutoutInwardAxis.NEGATIVE_Z,
|
||||
color_image,
|
||||
gesture="LASSO",
|
||||
):
|
||||
import numpy as np
|
||||
|
||||
normal_mode = NormalMode(normal_mode)
|
||||
inward_axis = CutoutInwardAxis(inward_axis)
|
||||
bounds = crop_plane_bounds(
|
||||
source_object,
|
||||
content_bounds,
|
||||
@@ -229,7 +202,6 @@ def create_shape_object(
|
||||
mesh_object.matrix_world = _cutout_world_matrix(
|
||||
source_object.matrix_world,
|
||||
crop_center,
|
||||
inward_axis,
|
||||
)
|
||||
mesh_object["anyimage_mesh_shape"] = shape
|
||||
node_group = load_cutout_node_group(shape)
|
||||
@@ -237,10 +209,6 @@ def create_shape_object(
|
||||
modifier.node_group = node_group
|
||||
modifier.show_group_selector = False
|
||||
for name, value in (
|
||||
(
|
||||
"Inward Axis",
|
||||
0 if inward_axis == CutoutInwardAxis.NEGATIVE_Z else 1,
|
||||
),
|
||||
("Cleanup Threshold", cleanup_value),
|
||||
(
|
||||
"Thickness",
|
||||
@@ -249,9 +217,24 @@ def create_shape_object(
|
||||
):
|
||||
set_modifier_input(
|
||||
modifier,
|
||||
modifier_input_identifier(node_group, name),
|
||||
modifier_input_identifier(
|
||||
node_group, name,
|
||||
subtype="NONE" if name == "Thickness" and shape == "DEPTH_SOLID" else None,
|
||||
),
|
||||
value,
|
||||
)
|
||||
if shape in {"FLAT", "SOLID", "DEPTH_SOLID"} and gesture == "POLYLINE":
|
||||
set_modifier_input(
|
||||
modifier,
|
||||
modifier_input_identifier(node_group, "Mode"),
|
||||
1,
|
||||
)
|
||||
if shape == "SOLID":
|
||||
set_modifier_input(
|
||||
modifier,
|
||||
modifier_input_identifier(node_group, "Thickness", subtype="DISTANCE"),
|
||||
0.5,
|
||||
)
|
||||
if shape in DEPTH_CUTOUT_SHAPES:
|
||||
for name, value in (
|
||||
("Depth Scale", 1.0),
|
||||
|
||||
@@ -13,6 +13,7 @@ from ...preferences import (
|
||||
from ...properties import (
|
||||
cutout_alpha_threshold_property,
|
||||
cutout_fine_outline_property,
|
||||
cutout_gesture_property,
|
||||
)
|
||||
from ...runtime import JobOperatorBase
|
||||
from ...server.model_catalog import DEFAULT_GEOMETRY_MODEL_KEY
|
||||
@@ -35,7 +36,7 @@ from ...common.image import (
|
||||
image_rgba,
|
||||
load_normal_result_image,
|
||||
require_image_empty,
|
||||
video_total_frames,
|
||||
is_animated_image,
|
||||
)
|
||||
from ...common.depth import load_depth_metadata, load_depth_result_image
|
||||
from ...common.color_image import (
|
||||
@@ -47,7 +48,7 @@ from ...common.color_image import (
|
||||
)
|
||||
from ...common.viewport import (
|
||||
ImageGesture,
|
||||
image_edit_drag_keymap,
|
||||
image_edit_point_keymap,
|
||||
report_image_edit_exception,
|
||||
)
|
||||
from ...common.selection import (
|
||||
@@ -62,10 +63,8 @@ from .geometry import (
|
||||
)
|
||||
from .interaction import open_shape_pie
|
||||
from .object import (
|
||||
CutoutInwardAxis,
|
||||
NormalMode,
|
||||
create_shape_object,
|
||||
orient_object_normal_image,
|
||||
)
|
||||
from .shape import CUTOUT_SHAPES, DEPTH_CUTOUT_SHAPES
|
||||
|
||||
@@ -100,8 +99,8 @@ def create_cutout_shape(
|
||||
depth_metadata=None,
|
||||
normal_image=None,
|
||||
normal_mode=NormalMode.NONE,
|
||||
inward_axis=CutoutInwardAxis.NEGATIVE_Z,
|
||||
color_image,
|
||||
gesture="LASSO",
|
||||
):
|
||||
base_shape = build_base_shape(
|
||||
context,
|
||||
@@ -125,8 +124,8 @@ def create_cutout_shape(
|
||||
depth_metadata=depth_metadata,
|
||||
normal_image=normal_image,
|
||||
normal_mode=normal_mode,
|
||||
inward_axis=inward_axis,
|
||||
color_image=color_image,
|
||||
gesture=gesture,
|
||||
)
|
||||
undo_message = f"Convert to {shape.replace('_', ' ').title()}"
|
||||
if "FINISHED" not in bpy.ops.ed.undo_push(message=undo_message):
|
||||
@@ -191,6 +190,7 @@ class CutoutSelectionToShape(JobOperatorBase, bpy.types.Operator):
|
||||
selection_path_json: bpy.props.StringProperty(options=_HIDDEN)
|
||||
selection_bounds_json: bpy.props.StringProperty(options=_HIDDEN)
|
||||
shape: bpy.props.StringProperty(default="SOLID", options=_HIDDEN)
|
||||
gesture: bpy.props.StringProperty(default="LASSO", options=_HIDDEN)
|
||||
point_spacing: bpy.props.FloatProperty(
|
||||
default=32.0,
|
||||
min=0.01,
|
||||
@@ -215,23 +215,13 @@ class CutoutSelectionToShape(JobOperatorBase, bpy.types.Operator):
|
||||
generate_normal: bpy.props.BoolProperty(default=False, options=_HIDDEN)
|
||||
alpha_threshold: cutout_alpha_threshold_property()
|
||||
fine_outline: cutout_fine_outline_property()
|
||||
inward_axis: bpy.props.StringProperty(
|
||||
default=CutoutInwardAxis.NEGATIVE_Z.value,
|
||||
options=_HIDDEN,
|
||||
)
|
||||
|
||||
def _needs_generation(self):
|
||||
return self.shape in DEPTH_CUTOUT_SHAPES or self.generate_normal
|
||||
|
||||
def execute(self, context):
|
||||
try:
|
||||
self.inward_axis = getattr(
|
||||
self,
|
||||
"inward_axis",
|
||||
CutoutInwardAxis.NEGATIVE_Z.value,
|
||||
)
|
||||
source_object = require_image_empty(self.source_object_name)
|
||||
if video_total_frames(source_object.data) != 1:
|
||||
if is_animated_image(source_object.data):
|
||||
raise ValueError("Cutout currently supports still images")
|
||||
if self.shape not in CUTOUT_SHAPES:
|
||||
raise ValueError(f"Unsupported Cutout shape: {self.shape}")
|
||||
@@ -263,8 +253,9 @@ class CutoutSelectionToShape(JobOperatorBase, bpy.types.Operator):
|
||||
create_cutout_shape(
|
||||
context, source_object, self.shape, self.point_spacing,
|
||||
content_values, selection_mask.bounds,
|
||||
inward_axis=self.inward_axis, alpha_threshold=self.alpha_threshold,
|
||||
alpha_threshold=self.alpha_threshold,
|
||||
fine_outline=self.fine_outline, color_image=color_image,
|
||||
gesture=self.gesture,
|
||||
)
|
||||
return {"FINISHED"}
|
||||
setup_result = invoke_ai_setup_if_needed(self)
|
||||
@@ -317,21 +308,14 @@ class CutoutSelectionToShape(JobOperatorBase, bpy.types.Operator):
|
||||
)
|
||||
parameters.update(
|
||||
moge2_parameters(
|
||||
context.scene.anyimage_settings,
|
||||
source_path,
|
||||
self.model,
|
||||
self.resolution_level,
|
||||
1,
|
||||
)
|
||||
)
|
||||
return parameters
|
||||
|
||||
def response(self, context, result):
|
||||
inward_axis = getattr(
|
||||
self,
|
||||
"inward_axis",
|
||||
CutoutInwardAxis.NEGATIVE_Z.value,
|
||||
)
|
||||
source_object = require_image_empty(self.source_object_name)
|
||||
source_bounds = tuple(json.loads(self.selection_bounds_json))
|
||||
color_image, source_alpha, color_bounds = _generated_color_result(
|
||||
@@ -358,8 +342,6 @@ class CutoutSelectionToShape(JobOperatorBase, bpy.types.Operator):
|
||||
result.file(normal_key),
|
||||
source_object,
|
||||
)
|
||||
if normal_mode == NormalMode.OBJECT:
|
||||
orient_object_normal_image(normal_image, inward_axis)
|
||||
depth_metadata = None
|
||||
if shape_uses_depth(self.shape):
|
||||
depth_metadata = load_depth_metadata(result.file("depth_metadata"))
|
||||
@@ -379,10 +361,10 @@ class CutoutSelectionToShape(JobOperatorBase, bpy.types.Operator):
|
||||
depth_metadata=depth_metadata,
|
||||
normal_image=normal_image,
|
||||
normal_mode=normal_mode,
|
||||
inward_axis=inward_axis,
|
||||
alpha_threshold=self.alpha_threshold,
|
||||
fine_outline=self.fine_outline,
|
||||
color_image=color_image,
|
||||
gesture=self.gesture,
|
||||
)
|
||||
except Exception:
|
||||
if normal_image is not None and normal_image.users == 0:
|
||||
@@ -401,22 +383,32 @@ class CutoutSelectionToShape(JobOperatorBase, bpy.types.Operator):
|
||||
class SelectCutoutSelection(ImageGesture, bpy.types.Operator):
|
||||
bl_idname = "anyimage.select_cutout_selection"
|
||||
bl_label = "Start Cutout Selection"
|
||||
bl_description = "Lasso an Image Empty selection, then choose its mesh shape"
|
||||
bl_description = "Select an Image Empty area with a lasso or polyline, then choose its mesh shape"
|
||||
bl_options = {"INTERNAL"}
|
||||
active_tool_id = "anyimage.cutout_lasso"
|
||||
gesture = "LASSO"
|
||||
resolve_on_press = True
|
||||
gesture: cutout_gesture_property()
|
||||
|
||||
source_object_name: bpy.props.StringProperty(options={"HIDDEN", "SKIP_SAVE"})
|
||||
source_matrix_data: bpy.props.StringProperty(options={"HIDDEN", "SKIP_SAVE"})
|
||||
|
||||
_pie_event = None
|
||||
_polyline_confirm_type = None
|
||||
|
||||
def _read_gesture_settings(self, context):
|
||||
self.gesture = context.scene.anyimage_settings.cutout_gesture
|
||||
|
||||
def _set_status(self, context):
|
||||
action = (
|
||||
"click points; click start, double-click, or Enter for shapes; Backspace removes"
|
||||
if self.gesture == "POLYLINE" else "drag and release for shapes"
|
||||
)
|
||||
context.workspace.status_text_set(
|
||||
"Cutout Lasso: drag and release for shapes; RMB or Esc cancels"
|
||||
f"Cutout {self.gesture.title()}: {action}; RMB or Esc cancels"
|
||||
)
|
||||
|
||||
def invoke(self, context, event):
|
||||
self._polyline_confirm_type = None
|
||||
self._pie_event = event
|
||||
return super().invoke(context, event)
|
||||
|
||||
@@ -424,6 +416,19 @@ class SelectCutoutSelection(ImageGesture, bpy.types.Operator):
|
||||
self._pie_event = event
|
||||
return ImageGesture.modal(self, context, event)
|
||||
|
||||
def _confirm_polyline(self, context):
|
||||
if len(self._path) >= 3:
|
||||
self._polyline_confirm_type = self._pie_event.type
|
||||
return {"RUNNING_MODAL"}
|
||||
|
||||
def _modal_polyline(self, context, event):
|
||||
if self._polyline_confirm_type is not None:
|
||||
if event.type == self._polyline_confirm_type and event.value == "RELEASE":
|
||||
self._polyline_confirm_type = None
|
||||
return ImageGesture._confirm_polyline(self, context)
|
||||
return {"RUNNING_MODAL"}
|
||||
return ImageGesture._modal_polyline(self, context, event)
|
||||
|
||||
def _submit_selection_path(self, context, selection_path):
|
||||
try:
|
||||
source_object = require_image_empty(self.source_object_name)
|
||||
@@ -435,6 +440,7 @@ class SelectCutoutSelection(ImageGesture, bpy.types.Operator):
|
||||
self._pie_event,
|
||||
source_object,
|
||||
selection_path,
|
||||
gesture=self.gesture,
|
||||
)
|
||||
return {"FINISHED"}
|
||||
|
||||
@@ -444,19 +450,18 @@ class CutoutTool(bpy.types.WorkSpaceTool):
|
||||
bl_context_mode = "OBJECT"
|
||||
bl_idname = "anyimage.cutout_lasso"
|
||||
bl_label = "Cutout"
|
||||
bl_description = "Lasso an Image Empty and choose a mesh shape"
|
||||
bl_description = "Select an Image Empty area with a lasso or polyline and choose a mesh shape"
|
||||
bl_icon = "ops.mesh.primitive_sphere_add_gizmo"
|
||||
bl_operator = SelectCutoutSelection.bl_idname
|
||||
bl_keymap = image_edit_drag_keymap(
|
||||
bl_keymap = image_edit_point_keymap(
|
||||
SelectCutoutSelection.bl_idname,
|
||||
deselect_on_empty=False,
|
||||
)
|
||||
|
||||
def draw_settings(context, layout, _tool):
|
||||
settings = context.scene.anyimage_settings
|
||||
status = draw_ai_setup(layout)
|
||||
draw_ai_property(layout, settings, "cutout_generate_normal", status)
|
||||
layout.prop(settings, "cutout_gesture")
|
||||
layout.prop(settings, "cutout_mesh_detail")
|
||||
layout.prop(settings, "cutout_fine_outline")
|
||||
layout.prop(settings, "cutout_alpha_threshold")
|
||||
layout.prop(settings, "cutout_inward_axis")
|
||||
status = draw_ai_setup(layout)
|
||||
draw_ai_property(layout, settings, "cutout_generate_normal", status)
|
||||
|
||||
@@ -2,12 +2,6 @@
|
||||
|
||||
BALLOON_ATTRIBUTE_NAME = "o_balloon"
|
||||
NORMAL_SCALE_ATTRIBUTE_NAME = "o_normal_scale"
|
||||
CUTOUT_X_TO_Z = (
|
||||
(0.0, -1.0, 0.0, 0.0),
|
||||
(0.0, 0.0, 1.0, 0.0),
|
||||
(-1.0, 0.0, 0.0, 0.0),
|
||||
(0.0, 0.0, 0.0, 1.0),
|
||||
)
|
||||
CUTOUT_Z_TO_X = (
|
||||
(0.0, 0.0, -1.0, 0.0),
|
||||
(-1.0, 0.0, 0.0, 0.0),
|
||||
|
||||
@@ -1,648 +0,0 @@
|
||||
import math
|
||||
|
||||
import bpy
|
||||
|
||||
from ..common.image import (
|
||||
MAX_PROJECTIVE_OUTPUT_PIXELS,
|
||||
PROJECTIVE_CHUNK_ROWS,
|
||||
create_image_edit_result,
|
||||
homography_from_points,
|
||||
image_empty_bounds,
|
||||
image_pixels,
|
||||
is_image_empty,
|
||||
premultiplied_rgba,
|
||||
replace_empty_image,
|
||||
sample_rgba,
|
||||
transform_homography,
|
||||
video_total_frames,
|
||||
)
|
||||
from ..preferences import configured_max_frame_resolution
|
||||
from ..common.viewport import (
|
||||
active_image_edit_source,
|
||||
active_view3d_tool_id,
|
||||
draw_dashed_line,
|
||||
draw_polygon_fill,
|
||||
drawing_in_region,
|
||||
image_edit_drag_keymap,
|
||||
image_edit_poll,
|
||||
rectangle_screen_path,
|
||||
report_image_edit_exception,
|
||||
)
|
||||
from ..common.selection import ImageEditWarning, clip_polygon_halfplanes
|
||||
|
||||
FRAME_TOOL_ID = "anyimage.frame"
|
||||
PROJECTIVE_RELATIVE_EPSILON = 1e-12
|
||||
|
||||
|
||||
def _matrix_array(matrix):
|
||||
import numpy as np
|
||||
|
||||
return np.asarray(
|
||||
tuple(tuple(float(value) for value in row) for row in matrix),
|
||||
dtype=np.float64,
|
||||
)
|
||||
|
||||
|
||||
def frame_source_projection(
|
||||
projection_matrix,
|
||||
view_matrix,
|
||||
matrix_world,
|
||||
bounds,
|
||||
region_size,
|
||||
*,
|
||||
perspective,
|
||||
):
|
||||
"""Return normalized image-plane transforms for one frozen view."""
|
||||
import numpy as np
|
||||
|
||||
x_min, x_max, y_min, y_max = (float(value) for value in bounds)
|
||||
region_width, region_height = (float(value) for value in region_size)
|
||||
if min(x_max - x_min, y_max - y_min, region_width, region_height) <= 0.0:
|
||||
raise ValueError("The Image Empty projection is invalid")
|
||||
source_plane = np.asarray(
|
||||
(
|
||||
(x_max - x_min, 0.0, x_min),
|
||||
(0.0, y_min - y_max, y_max),
|
||||
(0.0, 0.0, 0.0),
|
||||
(0.0, 0.0, 1.0),
|
||||
),
|
||||
dtype=np.float64,
|
||||
)
|
||||
world = _matrix_array(matrix_world)
|
||||
source_to_world = world @ source_plane
|
||||
source_to_view = _matrix_array(view_matrix) @ source_to_world
|
||||
source_to_clip = _matrix_array(projection_matrix) @ source_to_world
|
||||
source_to_screen = np.stack(
|
||||
(
|
||||
0.5 * (source_to_clip[0] + source_to_clip[3]),
|
||||
0.5 * (source_to_clip[1] + source_to_clip[3]),
|
||||
source_to_clip[3],
|
||||
)
|
||||
)
|
||||
scale = float(np.max(np.abs(source_to_screen)))
|
||||
if not math.isfinite(scale) or scale <= 0.0:
|
||||
raise ValueError("The Image Empty projection is invalid")
|
||||
source_to_screen /= scale
|
||||
if not np.isfinite(source_to_screen).all():
|
||||
raise ValueError("The Image Empty projection is invalid")
|
||||
condition = float(np.linalg.cond(source_to_screen))
|
||||
if not math.isfinite(condition) or condition > 1.0 / np.finfo(np.float64).eps:
|
||||
raise ValueError("The Image Empty is edge-on to the current view")
|
||||
return {
|
||||
"bounds": (x_min, x_max, y_min, y_max),
|
||||
"origin_depth": -float((_matrix_array(view_matrix) @ world)[2, 3]),
|
||||
"perspective": bool(perspective),
|
||||
"region_size": (region_width, region_height),
|
||||
"screen_to_source": np.linalg.inv(source_to_screen),
|
||||
"source_to_screen": source_to_screen,
|
||||
"source_to_view": source_to_view,
|
||||
"source_to_world": source_to_world,
|
||||
}
|
||||
|
||||
|
||||
def _projective_points(points, matrix):
|
||||
"""Map points and return finite results plus per-point validity."""
|
||||
import numpy as np
|
||||
|
||||
points = np.asarray(points, dtype=np.float64)
|
||||
flat = points.reshape((-1, 2))
|
||||
homogeneous = np.column_stack((flat, np.ones(len(flat), dtype=np.float64)))
|
||||
mapped = homogeneous @ np.asarray(matrix, dtype=np.float64).T
|
||||
magnitude = np.max(np.abs(mapped), axis=1)
|
||||
valid = (
|
||||
np.isfinite(mapped).all(axis=1)
|
||||
& (np.abs(mapped[:, 2]) > PROJECTIVE_RELATIVE_EPSILON * magnitude)
|
||||
)
|
||||
result = np.full((len(flat), 2), np.nan, dtype=np.float64)
|
||||
result[valid] = mapped[valid, :2] / mapped[valid, 2, None]
|
||||
return result.reshape(points.shape), valid.reshape(points.shape[:-1])
|
||||
|
||||
|
||||
def frame_source_coordinates(source, screen_points):
|
||||
"""Return source pixels, view depth, validity, and normalized coordinates."""
|
||||
import numpy as np
|
||||
|
||||
screen_points = np.asarray(screen_points, dtype=np.float64)
|
||||
region_width, region_height = source["region_size"]
|
||||
normalized_screen = screen_points / (region_width, region_height)
|
||||
normalized_source, valid = _projective_points(
|
||||
normalized_screen,
|
||||
source["screen_to_source"],
|
||||
)
|
||||
flat_source = normalized_source.reshape((-1, 2))
|
||||
homogeneous = np.column_stack(
|
||||
(flat_source, np.ones(len(flat_source), dtype=np.float64))
|
||||
)
|
||||
depth = -(homogeneous @ source["source_to_view"][2])
|
||||
depth = depth.reshape(valid.shape)
|
||||
valid &= np.isfinite(depth)
|
||||
if source["perspective"]:
|
||||
valid &= depth > 0.0
|
||||
valid &= (
|
||||
(normalized_source[..., 0] >= 0.0)
|
||||
& (normalized_source[..., 0] <= 1.0)
|
||||
& (normalized_source[..., 1] >= 0.0)
|
||||
& (normalized_source[..., 1] <= 1.0)
|
||||
)
|
||||
image_width, image_height = source["image_size"]
|
||||
source_points = normalized_source * (image_width, image_height)
|
||||
source_points[~valid] = -1.0
|
||||
return source_points, depth, valid, normalized_source
|
||||
|
||||
|
||||
def frame_source_overlap(source, frame_quad):
|
||||
"""Clip the source plane before dividing its homogeneous projection."""
|
||||
import numpy as np
|
||||
|
||||
frame = np.asarray(frame_quad, dtype=np.float64) / source["region_size"]
|
||||
left, bottom = frame.min(axis=0)
|
||||
right, top = frame.max(axis=0)
|
||||
x, y, w = source["source_to_screen"]
|
||||
boundaries = [x - left * w, right * w - x, y - bottom * w, top * w - y]
|
||||
if source["perspective"]:
|
||||
boundaries.insert(0, -source["source_to_view"][2])
|
||||
polygon = clip_polygon_halfplanes(
|
||||
((0.0, 0.0), (1.0, 0.0), (1.0, 1.0), (0.0, 1.0)), boundaries
|
||||
)
|
||||
if len(polygon) < 3:
|
||||
return ()
|
||||
screen, valid = _projective_points(polygon, source["source_to_screen"])
|
||||
if not np.all(valid):
|
||||
return ()
|
||||
screen *= source["region_size"]
|
||||
# Center the coordinates to avoid cancellation in a distant small polygon.
|
||||
centered = screen - screen[0]
|
||||
area = abs(np.sum(centered[:, 0] * np.roll(centered[:, 1], -1)
|
||||
- centered[:, 1] * np.roll(centered[:, 0], -1))) * 0.5
|
||||
return tuple(map(tuple, screen)) if area > 1e-8 else ()
|
||||
|
||||
|
||||
def polygon_centroid(polygon):
|
||||
"""Return the area centroid of one non-degenerate screen polygon."""
|
||||
cross_sum = 0.0
|
||||
x_sum = 0.0
|
||||
y_sum = 0.0
|
||||
for first, second in zip(polygon, (*polygon[1:], polygon[0])):
|
||||
cross = first[0] * second[1] - second[0] * first[1]
|
||||
cross_sum += cross
|
||||
x_sum += (first[0] + second[0]) * cross
|
||||
y_sum += (first[1] + second[1]) * cross
|
||||
if abs(cross_sum) <= 1e-12:
|
||||
raise ValueError("Frame does not overlap the active Image Empty")
|
||||
return x_sum / (3.0 * cross_sum), y_sum / (3.0 * cross_sum)
|
||||
|
||||
|
||||
def _pixel_quad(image_size):
|
||||
width, height = (float(value) for value in image_size)
|
||||
if min(width, height) <= 0.0:
|
||||
raise ValueError("The source image has no readable dimensions")
|
||||
return ((0.0, 0.0), (width, 0.0), (width, height), (0.0, height))
|
||||
|
||||
|
||||
def frame_output_size(frame_quad, region_size, max_resolution):
|
||||
"""Choose output resolution from Frame pixels within the current Region."""
|
||||
frame_width = abs(float(frame_quad[1][0] - frame_quad[0][0]))
|
||||
frame_height = abs(float(frame_quad[0][1] - frame_quad[3][1]))
|
||||
region_width, region_height = (float(value) for value in region_size)
|
||||
maximum = float(max_resolution)
|
||||
if min(frame_width, frame_height, region_width, region_height, maximum) <= 0.0:
|
||||
raise ValueError("The Frame output dimensions must be positive")
|
||||
scale = min(
|
||||
1.0,
|
||||
region_width / frame_width,
|
||||
region_height / frame_height,
|
||||
maximum / max(frame_width, frame_height),
|
||||
math.sqrt(
|
||||
MAX_PROJECTIVE_OUTPUT_PIXELS / max(frame_width * frame_height, 1.0)
|
||||
),
|
||||
)
|
||||
return (
|
||||
max(2, int(round(frame_width * scale))),
|
||||
max(2, int(round(frame_height * scale))),
|
||||
)
|
||||
|
||||
|
||||
def _frame_source(source_object, region, view3d, view_matrix, *, active):
|
||||
import numpy as np
|
||||
|
||||
image_size = tuple(source_object.data.size)
|
||||
pixels = image_pixels(source_object.data)
|
||||
width, height = image_size
|
||||
bounds = image_empty_bounds(source_object)
|
||||
source = frame_source_projection(
|
||||
view3d.perspective_matrix,
|
||||
view_matrix,
|
||||
source_object.matrix_world,
|
||||
bounds,
|
||||
(region.width, region.height),
|
||||
perspective=view3d.is_perspective,
|
||||
)
|
||||
source.update(
|
||||
{
|
||||
"active": bool(active),
|
||||
"image_size": image_size,
|
||||
"name": source_object.name,
|
||||
"object": source_object,
|
||||
"pixels": premultiplied_rgba(pixels, image_size),
|
||||
"rgba": np.flipud(pixels.reshape((height, width, 4))),
|
||||
}
|
||||
)
|
||||
return source
|
||||
|
||||
|
||||
def _composite_frame_samples(sources, screen_points):
|
||||
"""Resolve independently sorted samples with an unweighted bottom RGB base."""
|
||||
import numpy as np
|
||||
|
||||
samples, depths, coordinates = [], [], []
|
||||
for source in sources:
|
||||
points, depth, valid, _ = frame_source_coordinates(source, screen_points)
|
||||
sample = sample_rgba(source["pixels"], points)
|
||||
sample[~valid] = 0
|
||||
depths.append(np.where(valid, depth, -math.inf))
|
||||
samples.append(sample)
|
||||
coordinates.append(points)
|
||||
samples = np.stack(samples)
|
||||
depths = np.stack(depths)
|
||||
order = np.argsort(-depths, axis=0, kind="stable")
|
||||
bottom = order[0]
|
||||
covered = np.isfinite(np.take_along_axis(depths, bottom[None], axis=0)[0])
|
||||
result = np.zeros_like(samples[0])
|
||||
for index, source in enumerate(sources):
|
||||
selected = covered & (bottom == index)
|
||||
if np.any(selected):
|
||||
result[..., :3][selected] = sample_rgba(
|
||||
source["rgba"], coordinates[index][selected]
|
||||
)[..., :3]
|
||||
for index in range(len(sources)):
|
||||
layer = np.take_along_axis(samples, order[index][None, ..., None], axis=0)[0]
|
||||
alpha = layer[..., 3:4]
|
||||
result[..., 3:4] = alpha + result[..., 3:4] * (1 - alpha)
|
||||
if index:
|
||||
result[..., :3] = layer[..., :3] + result[..., :3] * (1 - alpha)
|
||||
return result, covered
|
||||
|
||||
|
||||
def _frame_boundary_pixels(sources, points, output_to_screen):
|
||||
"""Conservatively detect projected contour and depth-order crossings."""
|
||||
import numpy as np
|
||||
|
||||
boundary = np.zeros(points.shape[:-1], dtype=bool)
|
||||
for source in sources:
|
||||
width, height = source["region_size"]
|
||||
mapping = source["screen_to_source"] @ np.diag((1 / width, 1 / height, 1)) @ output_to_screen
|
||||
u, v, w = mapping
|
||||
lines = [u, v, u - w, v - w, w]
|
||||
positive = np.ones_like(boundary)
|
||||
negative = np.ones_like(boundary)
|
||||
for line in (u, v, w - u, w - v, w):
|
||||
radius = 0.5 * (abs(line[0]) + abs(line[1]))
|
||||
value = points[..., 0] * line[0] + points[..., 1] * line[1] + line[2]
|
||||
positive &= value + radius >= 0
|
||||
negative &= value - radius <= 0
|
||||
source_boundary = np.zeros_like(boundary)
|
||||
if source["perspective"]:
|
||||
lines.append(source["source_to_view"][2] @ mapping)
|
||||
for line in lines:
|
||||
radius = 0.5 * (abs(line[0]) + abs(line[1]))
|
||||
value = points[..., 0] * line[0] + points[..., 1] * line[1] + line[2]
|
||||
source_boundary |= np.abs(value) < radius - 1e-10 * max(radius, 1e-20)
|
||||
boundary |= source_boundary & (positive | negative)
|
||||
if len(sources) > 1:
|
||||
first_order = None
|
||||
for offset in ((-0.499999, -0.499999), (0.499999, -0.499999),
|
||||
(-0.499999, 0.499999), (0.499999, 0.499999)):
|
||||
screen = transform_homography(points + offset, output_to_screen)
|
||||
depths = []
|
||||
for source in sources:
|
||||
_, depth, valid, _ = frame_source_coordinates(source, screen)
|
||||
depths.append(np.where(valid, depth, -math.inf))
|
||||
order = np.argsort(-np.stack(depths), axis=0, kind="stable")
|
||||
if first_order is None:
|
||||
first_order = order
|
||||
else:
|
||||
boundary |= np.any(order != first_order, axis=0)
|
||||
return boundary
|
||||
|
||||
|
||||
def composite_frame_pixels(sources, frame_quad, output_size):
|
||||
"""Composite projected Image Empty sources into one Blender-order RGBA image."""
|
||||
import numpy as np
|
||||
|
||||
output_width, output_height = (int(value) for value in output_size)
|
||||
if min(output_width, output_height) <= 0:
|
||||
raise ValueError("The Frame output dimensions must be positive")
|
||||
sources = tuple(
|
||||
sorted(sources, key=lambda source: (source["active"], source["name"]))
|
||||
)
|
||||
if not sources or not sources[-1]["active"]:
|
||||
raise ValueError("Frame requires one active Image Empty")
|
||||
output_quad = _pixel_quad(output_size)
|
||||
output_to_screen = homography_from_points(output_quad, frame_quad)
|
||||
output = np.zeros((output_height, output_width, 4), dtype=np.float32)
|
||||
covered = np.zeros((output_height, output_width), dtype=bool)
|
||||
target_x = np.arange(output_width, dtype=np.float64) + 0.5
|
||||
for row_start in range(0, output_height, PROJECTIVE_CHUNK_ROWS):
|
||||
row_end = min(row_start + PROJECTIVE_CHUNK_ROWS, output_height)
|
||||
target_y = np.arange(row_start, row_end, dtype=np.float64) + 0.5
|
||||
x, y = np.meshgrid(target_x, target_y)
|
||||
points = np.stack((x, y), axis=-1)
|
||||
composite, coverage = _composite_frame_samples(
|
||||
sources, transform_homography(points, output_to_screen)
|
||||
)
|
||||
boundary = _frame_boundary_pixels(sources, points, output_to_screen)
|
||||
if np.any(boundary):
|
||||
edge_points = points[boundary]
|
||||
accumulated = np.zeros((len(edge_points), 4), dtype=np.float32)
|
||||
counts = np.zeros(len(edge_points), dtype=np.int32)
|
||||
for dy in (-0.375, -0.125, 0.125, 0.375):
|
||||
for dx in (-0.375, -0.125, 0.125, 0.375):
|
||||
sample, hit = _composite_frame_samples(
|
||||
sources,
|
||||
transform_homography(edge_points + (dx, dy), output_to_screen),
|
||||
)
|
||||
accumulated += sample
|
||||
counts += hit
|
||||
accumulated[..., 3] /= 16
|
||||
accumulated[..., :3] /= np.maximum(counts, 1)[..., None]
|
||||
composite[boundary] = accumulated
|
||||
coverage[boundary] = counts > 0
|
||||
covered[row_start:row_end] = coverage
|
||||
output[row_start:row_end] = composite
|
||||
output[..., 3][output[..., 3] <= 1e-8] = 0.0
|
||||
output[~covered] = 0.0
|
||||
return np.flipud(output).ravel()
|
||||
|
||||
|
||||
def view_facing_frame_geometry(corners, output_size):
|
||||
"""Return a world matrix array and display size for frame-plane corners."""
|
||||
import numpy as np
|
||||
|
||||
corners = np.asarray(corners, dtype=np.float64)
|
||||
if corners.shape != (4, 3) or not np.isfinite(corners).all():
|
||||
raise ValueError("The Frame view plane is invalid")
|
||||
top_left, top_right, _bottom_right, bottom_left = corners
|
||||
right = top_right - top_left
|
||||
up = top_left - bottom_left
|
||||
width = float(np.linalg.norm(right))
|
||||
height = float(np.linalg.norm(up))
|
||||
if min(width, height) <= 1e-10:
|
||||
raise ValueError("Draw a larger Frame")
|
||||
right /= width
|
||||
up /= height
|
||||
forward = np.cross(right, up)
|
||||
forward_length = float(np.linalg.norm(forward))
|
||||
if forward_length <= 1e-10:
|
||||
raise ValueError("The Frame view plane is invalid")
|
||||
forward /= forward_length
|
||||
matrix = np.identity(4, dtype=np.float64)
|
||||
matrix[:3, 0] = right
|
||||
matrix[:3, 1] = up
|
||||
matrix[:3, 2] = forward
|
||||
matrix[:3, 3] = corners.mean(axis=0)
|
||||
display_size = width if output_size[0] >= output_size[1] else height
|
||||
return matrix, display_size
|
||||
|
||||
|
||||
def frame_result_transform(frame_quad, source_origin, region, view3d, output_size):
|
||||
"""Return the view-facing object matrix and Image Empty display size."""
|
||||
from mathutils import Matrix
|
||||
from bpy_extras import view3d_utils
|
||||
|
||||
corners = tuple(
|
||||
tuple(
|
||||
float(value)
|
||||
for value in view3d_utils.region_2d_to_location_3d(
|
||||
region,
|
||||
view3d,
|
||||
point,
|
||||
source_origin,
|
||||
)
|
||||
)
|
||||
for point in frame_quad
|
||||
)
|
||||
matrix, display_size = view_facing_frame_geometry(corners, output_size)
|
||||
return Matrix(tuple(tuple(row) for row in matrix)), display_size
|
||||
|
||||
|
||||
def frame_depth_location(active_source, overlap_polygon):
|
||||
"""Return the world point whose view depth places the Frame result."""
|
||||
import numpy as np
|
||||
|
||||
source_object = active_source["object"]
|
||||
if not active_source["perspective"] or active_source["origin_depth"] > 0.0:
|
||||
return source_object.matrix_world.translation.copy()
|
||||
centroid = np.asarray((polygon_centroid(overlap_polygon),), dtype=np.float64)
|
||||
_points, depth, valid, normalized_source = frame_source_coordinates(
|
||||
active_source,
|
||||
centroid,
|
||||
)
|
||||
if not bool(valid[0]) or depth[0] <= 0.0:
|
||||
raise ValueError("The Frame result depth is invalid")
|
||||
source_point = np.asarray(
|
||||
(normalized_source[0, 0], normalized_source[0, 1], 1.0),
|
||||
dtype=np.float64,
|
||||
)
|
||||
world = active_source["source_to_world"] @ source_point
|
||||
if not np.isfinite(world).all() or abs(world[3]) <= 1e-12:
|
||||
raise ValueError("The Frame result depth is invalid")
|
||||
return tuple(float(value) for value in world[:3] / world[3])
|
||||
|
||||
|
||||
def selected_frame_sources(context):
|
||||
"""Return selected still Image Empties with the active object last."""
|
||||
active = active_image_edit_source(context)
|
||||
sources = [
|
||||
source for source in context.selected_objects if is_image_empty(source)
|
||||
]
|
||||
for source in sources:
|
||||
if video_total_frames(source.data) > 1:
|
||||
raise ValueError("Frame only supports still images")
|
||||
return tuple(
|
||||
sorted(sources, key=lambda source: (source is active, source.name))
|
||||
)
|
||||
|
||||
|
||||
def apply_frame(context, source_objects, frame_quad):
|
||||
"""Bake selected projections into the active Image Empty."""
|
||||
active = context.active_object
|
||||
if active not in source_objects:
|
||||
raise ValueError("The active Image Empty is not a Frame source")
|
||||
view3d = context.region_data
|
||||
sources = tuple(
|
||||
_frame_source(
|
||||
source,
|
||||
context.region,
|
||||
view3d,
|
||||
view3d.view_matrix,
|
||||
active=source is active,
|
||||
)
|
||||
for source in source_objects
|
||||
)
|
||||
active_source = next(source for source in sources if source["active"])
|
||||
active_overlap = frame_source_overlap(active_source, frame_quad)
|
||||
if not active_overlap:
|
||||
raise ImageEditWarning("Frame does not overlap the active Image Empty")
|
||||
output_size = frame_output_size(
|
||||
frame_quad,
|
||||
(context.region.width, context.region.height),
|
||||
configured_max_frame_resolution(context),
|
||||
)
|
||||
pixels = composite_frame_pixels(sources, frame_quad, output_size)
|
||||
depth_location = frame_depth_location(active_source, active_overlap)
|
||||
matrix, display_size = frame_result_transform(
|
||||
frame_quad,
|
||||
depth_location,
|
||||
context.region,
|
||||
view3d,
|
||||
output_size,
|
||||
)
|
||||
result_image = create_image_edit_result(active.data, pixels, output_size)
|
||||
old_matrix = active.matrix_world.copy()
|
||||
old_display_size = active.empty_display_size
|
||||
old_offset = tuple(active.empty_image_offset)
|
||||
source_images = []
|
||||
for source in source_objects:
|
||||
if all(image is not source.data for image in source_images):
|
||||
source_images.append(source.data)
|
||||
try:
|
||||
active.matrix_world = matrix
|
||||
active.empty_display_size = display_size
|
||||
active.empty_image_offset = (-0.5, -0.5)
|
||||
replace_empty_image(
|
||||
active, result_image,
|
||||
removed_objects=tuple(source for source in source_objects if source != active),
|
||||
)
|
||||
except Exception:
|
||||
active.matrix_world = old_matrix
|
||||
active.empty_display_size = old_display_size
|
||||
active.empty_image_offset = old_offset
|
||||
raise
|
||||
for source in source_objects:
|
||||
if source is not active:
|
||||
bpy.data.objects.remove(source, do_unlink=True)
|
||||
for image in source_images:
|
||||
try:
|
||||
if image.users == 0 and not image.use_fake_user:
|
||||
bpy.data.images.remove(image, do_unlink=True)
|
||||
except (ReferenceError, RuntimeError):
|
||||
pass
|
||||
return active
|
||||
|
||||
|
||||
class FrameImages(bpy.types.Operator):
|
||||
bl_idname = "anyimage.frame_images"
|
||||
bl_label = "Frame Images"
|
||||
bl_description = "Bake selected Image Empties into a view-facing frame"
|
||||
bl_options = {"UNDO"}
|
||||
|
||||
_active_name = ""
|
||||
_source_names = ()
|
||||
_start = None
|
||||
_cursor = None
|
||||
_handle = None
|
||||
_area_pointer = 0
|
||||
_region_pointer = 0
|
||||
|
||||
@classmethod
|
||||
def poll(cls, context):
|
||||
return image_edit_poll(context)
|
||||
|
||||
def _draw_overlay(self):
|
||||
if not drawing_in_region(self._area_pointer, self._region_pointer):
|
||||
return
|
||||
frame = rectangle_screen_path(self._start, self._cursor)
|
||||
if not frame:
|
||||
return
|
||||
draw_polygon_fill(frame)
|
||||
draw_dashed_line((*frame, frame[0]))
|
||||
|
||||
def _finish(self, context):
|
||||
if self._handle is not None:
|
||||
bpy.types.SpaceView3D.draw_handler_remove(self._handle, "WINDOW")
|
||||
self._handle = None
|
||||
context.workspace.status_text_set(None)
|
||||
if context.area is not None:
|
||||
context.area.tag_redraw()
|
||||
|
||||
def cancel(self, context):
|
||||
self._finish(context)
|
||||
|
||||
def invoke(self, context, event):
|
||||
try:
|
||||
sources = selected_frame_sources(context)
|
||||
except (TypeError, ValueError) as error:
|
||||
report_image_edit_exception(self, error)
|
||||
return {"CANCELLED"}
|
||||
self._active_name = context.active_object.name
|
||||
self._source_names = tuple(source.name for source in sources)
|
||||
point = (float(event.mouse_region_x), float(event.mouse_region_y))
|
||||
self._start = point
|
||||
self._cursor = point
|
||||
self._area_pointer = context.area.as_pointer()
|
||||
self._region_pointer = context.region.as_pointer()
|
||||
self._handle = bpy.types.SpaceView3D.draw_handler_add(
|
||||
self._draw_overlay, (), "WINDOW", "POST_PIXEL"
|
||||
)
|
||||
context.workspace.status_text_set(
|
||||
"Frame: drag a rectangle to merge selected Image Empties; "
|
||||
"RMB or Esc cancels"
|
||||
)
|
||||
context.window_manager.modal_handler_add(self)
|
||||
context.area.tag_redraw()
|
||||
return {"RUNNING_MODAL"}
|
||||
|
||||
def modal(self, context, event):
|
||||
active_tool = active_view3d_tool_id(context)
|
||||
if active_tool is not None and active_tool != FRAME_TOOL_ID:
|
||||
self._finish(context)
|
||||
return {"CANCELLED"}
|
||||
if event.type in {"ESC", "RIGHTMOUSE"} and event.value == "PRESS":
|
||||
self._finish(context)
|
||||
return {"CANCELLED"}
|
||||
if event.type == "MOUSEMOVE":
|
||||
self._cursor = (
|
||||
float(event.mouse_region_x),
|
||||
float(event.mouse_region_y),
|
||||
)
|
||||
context.area.tag_redraw()
|
||||
return {"RUNNING_MODAL"}
|
||||
if event.type == "LEFTMOUSE" and event.value == "RELEASE":
|
||||
self._cursor = (
|
||||
float(event.mouse_region_x),
|
||||
float(event.mouse_region_y),
|
||||
)
|
||||
frame = rectangle_screen_path(self._start, self._cursor)
|
||||
if not frame:
|
||||
self._finish(context)
|
||||
return {"CANCELLED"}
|
||||
try:
|
||||
active = bpy.data.objects.get(self._active_name)
|
||||
sources = tuple(
|
||||
bpy.data.objects.get(name) for name in self._source_names
|
||||
)
|
||||
if (
|
||||
not is_image_empty(active)
|
||||
or any(not is_image_empty(source) for source in sources)
|
||||
or context.active_object is not active
|
||||
):
|
||||
raise ValueError("The selected Image Empties changed during Frame")
|
||||
apply_frame(context, sources, frame)
|
||||
except (RuntimeError, TypeError, ValueError) as error:
|
||||
report_image_edit_exception(self, error)
|
||||
self._finish(context)
|
||||
return {"CANCELLED"}
|
||||
self._finish(context)
|
||||
return {"FINISHED"}
|
||||
return {"PASS_THROUGH"}
|
||||
|
||||
|
||||
class FrameTool(bpy.types.WorkSpaceTool):
|
||||
bl_space_type = "VIEW_3D"
|
||||
bl_context_mode = "OBJECT"
|
||||
bl_idname = FRAME_TOOL_ID
|
||||
bl_label = "Frame"
|
||||
bl_description = "Bake selected Image Empties into a view-facing frame"
|
||||
bl_icon = "ops.sculpt.border_mask"
|
||||
bl_operator = FrameImages.bl_idname
|
||||
bl_keymap = image_edit_drag_keymap(FrameImages.bl_idname)
|
||||
7
src/anyimage/operators/frame_tool/__init__.py
Normal file
7
src/anyimage/operators/frame_tool/__init__.py
Normal file
@@ -0,0 +1,7 @@
|
||||
"""Frame conversion operators and workspace tool."""
|
||||
|
||||
from .operators import FrameImages, FrameTool
|
||||
|
||||
CLASSES = (FrameImages,)
|
||||
|
||||
__all__ = ("CLASSES", "FrameImages", "FrameTool")
|
||||
141
src/anyimage/operators/frame_tool/compositing.py
Normal file
141
src/anyimage/operators/frame_tool/compositing.py
Normal file
@@ -0,0 +1,141 @@
|
||||
"""Composite projected image samples with independent RGB and Alpha coverage."""
|
||||
|
||||
import math
|
||||
from ...common.image import (
|
||||
PROJECTIVE_CHUNK_ROWS,
|
||||
homography_from_points,
|
||||
sample_rgba,
|
||||
transform_homography,
|
||||
)
|
||||
from .projection import frame_source_coordinates
|
||||
|
||||
|
||||
def _pixel_quad(image_size):
|
||||
width, height = (float(value) for value in image_size)
|
||||
if min(width, height) <= 0.0:
|
||||
raise ValueError("The source image has no readable dimensions")
|
||||
return ((0.0, 0.0), (width, 0.0), (width, height), (0.0, height))
|
||||
|
||||
|
||||
def _composite_frame_samples(sources, screen_points):
|
||||
"""Resolve independently sorted samples with an unweighted bottom RGB base."""
|
||||
import numpy as np
|
||||
|
||||
samples, depths, coordinates = [], [], []
|
||||
for source in sources:
|
||||
points, depth, valid, _ = frame_source_coordinates(source, screen_points)
|
||||
sample = sample_rgba(source["pixels"], points)
|
||||
sample[~valid] = 0
|
||||
depths.append(np.where(valid, depth, -math.inf))
|
||||
samples.append(sample)
|
||||
coordinates.append(points)
|
||||
samples = np.stack(samples)
|
||||
depths = np.stack(depths)
|
||||
order = np.argsort(-depths, axis=0, kind="stable")
|
||||
bottom = order[0]
|
||||
covered = np.isfinite(np.take_along_axis(depths, bottom[None], axis=0)[0])
|
||||
result = np.zeros_like(samples[0])
|
||||
for index, source in enumerate(sources):
|
||||
selected = covered & (bottom == index)
|
||||
if np.any(selected):
|
||||
result[..., :3][selected] = sample_rgba(
|
||||
source["rgba"], coordinates[index][selected]
|
||||
)[..., :3]
|
||||
for index in range(len(sources)):
|
||||
layer = np.take_along_axis(samples, order[index][None, ..., None], axis=0)[0]
|
||||
alpha = layer[..., 3:4]
|
||||
result[..., 3:4] = alpha + result[..., 3:4] * (1 - alpha)
|
||||
if index:
|
||||
result[..., :3] = layer[..., :3] + result[..., :3] * (1 - alpha)
|
||||
return result, covered
|
||||
|
||||
|
||||
def _frame_boundary_pixels(sources, points, output_to_screen):
|
||||
"""Conservatively detect projected contour and depth-order crossings."""
|
||||
import numpy as np
|
||||
|
||||
boundary = np.zeros(points.shape[:-1], dtype=bool)
|
||||
for source in sources:
|
||||
width, height = source["region_size"]
|
||||
mapping = source["screen_to_source"] @ np.diag((1 / width, 1 / height, 1)) @ output_to_screen
|
||||
u, v, w = mapping
|
||||
lines = [u, v, u - w, v - w, w]
|
||||
positive = np.ones_like(boundary)
|
||||
negative = np.ones_like(boundary)
|
||||
for line in (u, v, w - u, w - v, w):
|
||||
radius = 0.5 * (abs(line[0]) + abs(line[1]))
|
||||
value = points[..., 0] * line[0] + points[..., 1] * line[1] + line[2]
|
||||
positive &= value + radius >= 0
|
||||
negative &= value - radius <= 0
|
||||
source_boundary = np.zeros_like(boundary)
|
||||
if source["perspective"]:
|
||||
lines.append(source["source_to_view"][2] @ mapping)
|
||||
for line in lines:
|
||||
radius = 0.5 * (abs(line[0]) + abs(line[1]))
|
||||
value = points[..., 0] * line[0] + points[..., 1] * line[1] + line[2]
|
||||
source_boundary |= np.abs(value) < radius - 1e-10 * max(radius, 1e-20)
|
||||
boundary |= source_boundary & (positive | negative)
|
||||
if len(sources) > 1:
|
||||
first_order = None
|
||||
for offset in ((-0.499999, -0.499999), (0.499999, -0.499999),
|
||||
(-0.499999, 0.499999), (0.499999, 0.499999)):
|
||||
screen = transform_homography(points + offset, output_to_screen)
|
||||
depths = []
|
||||
for source in sources:
|
||||
_, depth, valid, _ = frame_source_coordinates(source, screen)
|
||||
depths.append(np.where(valid, depth, -math.inf))
|
||||
order = np.argsort(-np.stack(depths), axis=0, kind="stable")
|
||||
if first_order is None:
|
||||
first_order = order
|
||||
else:
|
||||
boundary |= np.any(order != first_order, axis=0)
|
||||
return boundary
|
||||
|
||||
|
||||
def composite_frame_pixels(sources, frame_quad, output_size):
|
||||
"""Composite projected Image Empty sources into one Blender-order RGBA image."""
|
||||
import numpy as np
|
||||
|
||||
output_width, output_height = (int(value) for value in output_size)
|
||||
if min(output_width, output_height) <= 0:
|
||||
raise ValueError("The Frame output dimensions must be positive")
|
||||
sources = tuple(
|
||||
sorted(sources, key=lambda source: (source["active"], source["name"]))
|
||||
)
|
||||
if not sources or not sources[-1]["active"]:
|
||||
raise ValueError("Frame requires one active Image Empty")
|
||||
output_quad = _pixel_quad(output_size)
|
||||
output_to_screen = homography_from_points(output_quad, frame_quad)
|
||||
output = np.zeros((output_height, output_width, 4), dtype=np.float32)
|
||||
covered = np.zeros((output_height, output_width), dtype=bool)
|
||||
target_x = np.arange(output_width, dtype=np.float64) + 0.5
|
||||
for row_start in range(0, output_height, PROJECTIVE_CHUNK_ROWS):
|
||||
row_end = min(row_start + PROJECTIVE_CHUNK_ROWS, output_height)
|
||||
target_y = np.arange(row_start, row_end, dtype=np.float64) + 0.5
|
||||
x, y = np.meshgrid(target_x, target_y)
|
||||
points = np.stack((x, y), axis=-1)
|
||||
composite, coverage = _composite_frame_samples(
|
||||
sources, transform_homography(points, output_to_screen)
|
||||
)
|
||||
boundary = _frame_boundary_pixels(sources, points, output_to_screen)
|
||||
if np.any(boundary):
|
||||
edge_points = points[boundary]
|
||||
accumulated = np.zeros((len(edge_points), 4), dtype=np.float32)
|
||||
counts = np.zeros(len(edge_points), dtype=np.int32)
|
||||
for dy in (-0.375, -0.125, 0.125, 0.375):
|
||||
for dx in (-0.375, -0.125, 0.125, 0.375):
|
||||
sample, hit = _composite_frame_samples(
|
||||
sources,
|
||||
transform_homography(edge_points + (dx, dy), output_to_screen),
|
||||
)
|
||||
accumulated += sample
|
||||
counts += hit
|
||||
accumulated[..., 3] /= 16
|
||||
accumulated[..., :3] /= np.maximum(counts, 1)[..., None]
|
||||
composite[boundary] = accumulated
|
||||
coverage[boundary] = counts > 0
|
||||
covered[row_start:row_end] = coverage
|
||||
output[row_start:row_end] = composite
|
||||
output[..., 3][output[..., 3] <= 1e-8] = 0.0
|
||||
output[~covered] = 0.0
|
||||
return np.flipud(output).ravel()
|
||||
264
src/anyimage/operators/frame_tool/operators.py
Normal file
264
src/anyimage/operators/frame_tool/operators.py
Normal file
@@ -0,0 +1,264 @@
|
||||
"""Frame interaction, source preparation, and Blender image replacement."""
|
||||
|
||||
import bpy
|
||||
|
||||
from ...common.image import (
|
||||
create_image_edit_result,
|
||||
image_empty_bounds,
|
||||
image_pixels,
|
||||
is_image_empty,
|
||||
premultiplied_rgba,
|
||||
replace_empty_image,
|
||||
is_animated_image,
|
||||
)
|
||||
from ...preferences import configured_max_frame_resolution
|
||||
from ...common.viewport import (
|
||||
active_image_edit_source,
|
||||
active_view3d_tool_id,
|
||||
draw_dashed_line,
|
||||
draw_polygon_fill,
|
||||
drawing_in_region,
|
||||
image_edit_drag_keymap,
|
||||
image_edit_poll,
|
||||
rectangle_screen_path,
|
||||
report_image_edit_exception,
|
||||
)
|
||||
from ...common.selection import ImageEditWarning
|
||||
from .projection import (
|
||||
frame_depth_location,
|
||||
frame_output_size,
|
||||
frame_result_transform,
|
||||
frame_source_overlap,
|
||||
frame_source_projection,
|
||||
)
|
||||
from .compositing import composite_frame_pixels
|
||||
|
||||
|
||||
FRAME_TOOL_ID = "anyimage.frame"
|
||||
|
||||
|
||||
def _frame_source(source_object, region, view3d, view_matrix, *, active):
|
||||
import numpy as np
|
||||
|
||||
image_size = tuple(source_object.data.size)
|
||||
pixels = image_pixels(source_object.data)
|
||||
width, height = image_size
|
||||
bounds = image_empty_bounds(source_object)
|
||||
source = frame_source_projection(
|
||||
view3d.perspective_matrix,
|
||||
view_matrix,
|
||||
source_object.matrix_world,
|
||||
bounds,
|
||||
(region.width, region.height),
|
||||
perspective=view3d.is_perspective,
|
||||
)
|
||||
source.update(
|
||||
{
|
||||
"active": bool(active),
|
||||
"image_size": image_size,
|
||||
"name": source_object.name,
|
||||
"object": source_object,
|
||||
"pixels": premultiplied_rgba(pixels, image_size),
|
||||
"rgba": np.flipud(pixels.reshape((height, width, 4))),
|
||||
}
|
||||
)
|
||||
return source
|
||||
|
||||
|
||||
def selected_frame_sources(context):
|
||||
"""Return selected still Image Empties with the active object last."""
|
||||
active = active_image_edit_source(context)
|
||||
sources = [
|
||||
source for source in context.selected_objects if is_image_empty(source)
|
||||
]
|
||||
for source in sources:
|
||||
if is_animated_image(source.data):
|
||||
raise ValueError("Frame only supports still images")
|
||||
return tuple(
|
||||
sorted(sources, key=lambda source: (source is active, source.name))
|
||||
)
|
||||
|
||||
|
||||
def apply_frame(context, source_objects, frame_quad):
|
||||
"""Bake selected projections into the active Image Empty."""
|
||||
active = context.active_object
|
||||
if active not in source_objects:
|
||||
raise ValueError("The active Image Empty is not a Frame source")
|
||||
view3d = context.region_data
|
||||
sources = tuple(
|
||||
_frame_source(
|
||||
source,
|
||||
context.region,
|
||||
view3d,
|
||||
view3d.view_matrix,
|
||||
active=source is active,
|
||||
)
|
||||
for source in source_objects
|
||||
)
|
||||
active_source = next(source for source in sources if source["active"])
|
||||
active_overlap = frame_source_overlap(active_source, frame_quad)
|
||||
if not active_overlap:
|
||||
raise ImageEditWarning("Frame does not overlap the active Image Empty")
|
||||
output_size = frame_output_size(
|
||||
frame_quad,
|
||||
(context.region.width, context.region.height),
|
||||
configured_max_frame_resolution(context),
|
||||
)
|
||||
pixels = composite_frame_pixels(sources, frame_quad, output_size)
|
||||
depth_location = frame_depth_location(active_source, active_overlap)
|
||||
matrix, display_size = frame_result_transform(
|
||||
frame_quad,
|
||||
depth_location,
|
||||
context.region,
|
||||
view3d,
|
||||
output_size,
|
||||
)
|
||||
result_image = create_image_edit_result(active.data, pixels, output_size)
|
||||
old_matrix = active.matrix_world.copy()
|
||||
old_display_size = active.empty_display_size
|
||||
old_offset = tuple(active.empty_image_offset)
|
||||
source_images = []
|
||||
for source in source_objects:
|
||||
if all(image is not source.data for image in source_images):
|
||||
source_images.append(source.data)
|
||||
try:
|
||||
active.matrix_world = matrix
|
||||
active.empty_display_size = display_size
|
||||
active.empty_image_offset = (-0.5, -0.5)
|
||||
replace_empty_image(
|
||||
active, result_image,
|
||||
removed_objects=tuple(source for source in source_objects if source != active),
|
||||
)
|
||||
except Exception:
|
||||
active.matrix_world = old_matrix
|
||||
active.empty_display_size = old_display_size
|
||||
active.empty_image_offset = old_offset
|
||||
raise
|
||||
for source in source_objects:
|
||||
if source is not active:
|
||||
bpy.data.objects.remove(source, do_unlink=True)
|
||||
for image in source_images:
|
||||
try:
|
||||
if image.users == 0 and not image.use_fake_user:
|
||||
bpy.data.images.remove(image, do_unlink=True)
|
||||
except (ReferenceError, RuntimeError):
|
||||
pass
|
||||
return active
|
||||
|
||||
|
||||
class FrameImages(bpy.types.Operator):
|
||||
bl_idname = "anyimage.frame_images"
|
||||
bl_label = "Frame Images"
|
||||
bl_description = "Draw a rectangle to combine selected images into one image from the current view."
|
||||
bl_options = {"UNDO"}
|
||||
|
||||
_active_name = ""
|
||||
_source_names = ()
|
||||
_start = None
|
||||
_cursor = None
|
||||
_handle = None
|
||||
_area_pointer = 0
|
||||
_region_pointer = 0
|
||||
|
||||
@classmethod
|
||||
def poll(cls, context):
|
||||
return image_edit_poll(context)
|
||||
|
||||
def _draw_overlay(self):
|
||||
if not drawing_in_region(self._area_pointer, self._region_pointer):
|
||||
return
|
||||
frame = rectangle_screen_path(self._start, self._cursor)
|
||||
if not frame:
|
||||
return
|
||||
draw_polygon_fill(frame)
|
||||
draw_dashed_line((*frame, frame[0]))
|
||||
|
||||
def _finish(self, context):
|
||||
if self._handle is not None:
|
||||
bpy.types.SpaceView3D.draw_handler_remove(self._handle, "WINDOW")
|
||||
self._handle = None
|
||||
context.workspace.status_text_set(None)
|
||||
if context.area is not None:
|
||||
context.area.tag_redraw()
|
||||
|
||||
def cancel(self, context):
|
||||
self._finish(context)
|
||||
|
||||
def invoke(self, context, event):
|
||||
try:
|
||||
sources = selected_frame_sources(context)
|
||||
except (TypeError, ValueError) as error:
|
||||
report_image_edit_exception(self, error)
|
||||
return {"CANCELLED"}
|
||||
self._active_name = context.active_object.name
|
||||
self._source_names = tuple(source.name for source in sources)
|
||||
point = (float(event.mouse_region_x), float(event.mouse_region_y))
|
||||
self._start = point
|
||||
self._cursor = point
|
||||
self._area_pointer = context.area.as_pointer()
|
||||
self._region_pointer = context.region.as_pointer()
|
||||
self._handle = bpy.types.SpaceView3D.draw_handler_add(
|
||||
self._draw_overlay, (), "WINDOW", "POST_PIXEL"
|
||||
)
|
||||
context.workspace.status_text_set(
|
||||
"Frame: drag a rectangle to merge selected Image Empties; "
|
||||
"RMB or Esc cancels"
|
||||
)
|
||||
context.window_manager.modal_handler_add(self)
|
||||
context.area.tag_redraw()
|
||||
return {"RUNNING_MODAL"}
|
||||
|
||||
def modal(self, context, event):
|
||||
active_tool = active_view3d_tool_id(context)
|
||||
if active_tool is not None and active_tool != FRAME_TOOL_ID:
|
||||
self._finish(context)
|
||||
return {"CANCELLED"}
|
||||
if event.type in {"ESC", "RIGHTMOUSE"} and event.value == "PRESS":
|
||||
self._finish(context)
|
||||
return {"CANCELLED"}
|
||||
if event.type == "MOUSEMOVE":
|
||||
self._cursor = (
|
||||
float(event.mouse_region_x),
|
||||
float(event.mouse_region_y),
|
||||
)
|
||||
context.area.tag_redraw()
|
||||
return {"RUNNING_MODAL"}
|
||||
if event.type == "LEFTMOUSE" and event.value == "RELEASE":
|
||||
self._cursor = (
|
||||
float(event.mouse_region_x),
|
||||
float(event.mouse_region_y),
|
||||
)
|
||||
frame = rectangle_screen_path(self._start, self._cursor)
|
||||
if not frame:
|
||||
self._finish(context)
|
||||
return {"CANCELLED"}
|
||||
try:
|
||||
active = bpy.data.objects.get(self._active_name)
|
||||
sources = tuple(
|
||||
bpy.data.objects.get(name) for name in self._source_names
|
||||
)
|
||||
if (
|
||||
not is_image_empty(active)
|
||||
or any(not is_image_empty(source) for source in sources)
|
||||
or context.active_object is not active
|
||||
):
|
||||
raise ValueError("The selected Image Empties changed during Frame")
|
||||
apply_frame(context, sources, frame)
|
||||
except (RuntimeError, TypeError, ValueError) as error:
|
||||
report_image_edit_exception(self, error)
|
||||
self._finish(context)
|
||||
return {"CANCELLED"}
|
||||
self._finish(context)
|
||||
return {"FINISHED"}
|
||||
return {"PASS_THROUGH"}
|
||||
|
||||
|
||||
class FrameTool(bpy.types.WorkSpaceTool):
|
||||
bl_space_type = "VIEW_3D"
|
||||
bl_context_mode = "OBJECT"
|
||||
bl_idname = FRAME_TOOL_ID
|
||||
bl_label = "Frame"
|
||||
bl_description = "Draw a rectangle to combine selected images into one image from the current view."
|
||||
bl_icon = "ops.sculpt.border_mask"
|
||||
bl_operator = FrameImages.bl_idname
|
||||
bl_keymap = image_edit_drag_keymap(FrameImages.bl_idname)
|
||||
264
src/anyimage/operators/frame_tool/projection.py
Normal file
264
src/anyimage/operators/frame_tool/projection.py
Normal file
@@ -0,0 +1,264 @@
|
||||
"""Project image planes and place the view-facing Frame result."""
|
||||
|
||||
import math
|
||||
from ...common.image import MAX_PROJECTIVE_OUTPUT_PIXELS
|
||||
from ...common.selection import clip_polygon_halfplanes
|
||||
|
||||
|
||||
PROJECTIVE_RELATIVE_EPSILON = 1e-12
|
||||
|
||||
|
||||
def _matrix_array(matrix):
|
||||
import numpy as np
|
||||
|
||||
return np.asarray(
|
||||
tuple(tuple(float(value) for value in row) for row in matrix),
|
||||
dtype=np.float64,
|
||||
)
|
||||
|
||||
|
||||
def frame_source_projection(
|
||||
projection_matrix,
|
||||
view_matrix,
|
||||
matrix_world,
|
||||
bounds,
|
||||
region_size,
|
||||
*,
|
||||
perspective,
|
||||
):
|
||||
"""Return normalized image-plane transforms for one frozen view."""
|
||||
import numpy as np
|
||||
|
||||
x_min, x_max, y_min, y_max = (float(value) for value in bounds)
|
||||
region_width, region_height = (float(value) for value in region_size)
|
||||
if min(x_max - x_min, y_max - y_min, region_width, region_height) <= 0.0:
|
||||
raise ValueError("The Image Empty projection is invalid")
|
||||
source_plane = np.asarray(
|
||||
(
|
||||
(x_max - x_min, 0.0, x_min),
|
||||
(0.0, y_min - y_max, y_max),
|
||||
(0.0, 0.0, 0.0),
|
||||
(0.0, 0.0, 1.0),
|
||||
),
|
||||
dtype=np.float64,
|
||||
)
|
||||
world = _matrix_array(matrix_world)
|
||||
source_to_world = world @ source_plane
|
||||
source_to_view = _matrix_array(view_matrix) @ source_to_world
|
||||
source_to_clip = _matrix_array(projection_matrix) @ source_to_world
|
||||
source_to_screen = np.stack(
|
||||
(
|
||||
0.5 * (source_to_clip[0] + source_to_clip[3]),
|
||||
0.5 * (source_to_clip[1] + source_to_clip[3]),
|
||||
source_to_clip[3],
|
||||
)
|
||||
)
|
||||
scale = float(np.max(np.abs(source_to_screen)))
|
||||
if not math.isfinite(scale) or scale <= 0.0:
|
||||
raise ValueError("The Image Empty projection is invalid")
|
||||
source_to_screen /= scale
|
||||
if not np.isfinite(source_to_screen).all():
|
||||
raise ValueError("The Image Empty projection is invalid")
|
||||
condition = float(np.linalg.cond(source_to_screen))
|
||||
if not math.isfinite(condition) or condition > 1.0 / np.finfo(np.float64).eps:
|
||||
raise ValueError("The Image Empty is edge-on to the current view")
|
||||
return {
|
||||
"bounds": (x_min, x_max, y_min, y_max),
|
||||
"origin_depth": -float((_matrix_array(view_matrix) @ world)[2, 3]),
|
||||
"perspective": bool(perspective),
|
||||
"region_size": (region_width, region_height),
|
||||
"screen_to_source": np.linalg.inv(source_to_screen),
|
||||
"source_to_screen": source_to_screen,
|
||||
"source_to_view": source_to_view,
|
||||
"source_to_world": source_to_world,
|
||||
}
|
||||
|
||||
|
||||
def _projective_points(points, matrix):
|
||||
"""Map points and return finite results plus per-point validity."""
|
||||
import numpy as np
|
||||
|
||||
points = np.asarray(points, dtype=np.float64)
|
||||
flat = points.reshape((-1, 2))
|
||||
homogeneous = np.column_stack((flat, np.ones(len(flat), dtype=np.float64)))
|
||||
mapped = homogeneous @ np.asarray(matrix, dtype=np.float64).T
|
||||
magnitude = np.max(np.abs(mapped), axis=1)
|
||||
valid = (
|
||||
np.isfinite(mapped).all(axis=1)
|
||||
& (np.abs(mapped[:, 2]) > PROJECTIVE_RELATIVE_EPSILON * magnitude)
|
||||
)
|
||||
result = np.full((len(flat), 2), np.nan, dtype=np.float64)
|
||||
result[valid] = mapped[valid, :2] / mapped[valid, 2, None]
|
||||
return result.reshape(points.shape), valid.reshape(points.shape[:-1])
|
||||
|
||||
|
||||
def frame_source_coordinates(source, screen_points):
|
||||
"""Return source pixels, view depth, validity, and normalized coordinates."""
|
||||
import numpy as np
|
||||
|
||||
screen_points = np.asarray(screen_points, dtype=np.float64)
|
||||
region_width, region_height = source["region_size"]
|
||||
normalized_screen = screen_points / (region_width, region_height)
|
||||
normalized_source, valid = _projective_points(
|
||||
normalized_screen,
|
||||
source["screen_to_source"],
|
||||
)
|
||||
flat_source = normalized_source.reshape((-1, 2))
|
||||
homogeneous = np.column_stack(
|
||||
(flat_source, np.ones(len(flat_source), dtype=np.float64))
|
||||
)
|
||||
depth = -(homogeneous @ source["source_to_view"][2])
|
||||
depth = depth.reshape(valid.shape)
|
||||
valid &= np.isfinite(depth)
|
||||
if source["perspective"]:
|
||||
valid &= depth > 0.0
|
||||
valid &= (
|
||||
(normalized_source[..., 0] >= 0.0)
|
||||
& (normalized_source[..., 0] <= 1.0)
|
||||
& (normalized_source[..., 1] >= 0.0)
|
||||
& (normalized_source[..., 1] <= 1.0)
|
||||
)
|
||||
image_width, image_height = source["image_size"]
|
||||
source_points = normalized_source * (image_width, image_height)
|
||||
source_points[~valid] = -1.0
|
||||
return source_points, depth, valid, normalized_source
|
||||
|
||||
|
||||
def frame_source_overlap(source, frame_quad):
|
||||
"""Clip the source plane before dividing its homogeneous projection."""
|
||||
import numpy as np
|
||||
|
||||
frame = np.asarray(frame_quad, dtype=np.float64) / source["region_size"]
|
||||
left, bottom = frame.min(axis=0)
|
||||
right, top = frame.max(axis=0)
|
||||
x, y, w = source["source_to_screen"]
|
||||
boundaries = [x - left * w, right * w - x, y - bottom * w, top * w - y]
|
||||
if source["perspective"]:
|
||||
boundaries.insert(0, -source["source_to_view"][2])
|
||||
polygon = clip_polygon_halfplanes(
|
||||
((0.0, 0.0), (1.0, 0.0), (1.0, 1.0), (0.0, 1.0)), boundaries
|
||||
)
|
||||
if len(polygon) < 3:
|
||||
return ()
|
||||
screen, valid = _projective_points(polygon, source["source_to_screen"])
|
||||
if not np.all(valid):
|
||||
return ()
|
||||
screen *= source["region_size"]
|
||||
# Center the coordinates to avoid cancellation in a distant small polygon.
|
||||
centered = screen - screen[0]
|
||||
area = abs(np.sum(centered[:, 0] * np.roll(centered[:, 1], -1)
|
||||
- centered[:, 1] * np.roll(centered[:, 0], -1))) * 0.5
|
||||
return tuple(map(tuple, screen)) if area > 1e-8 else ()
|
||||
|
||||
|
||||
def polygon_centroid(polygon):
|
||||
"""Return the area centroid of one non-degenerate screen polygon."""
|
||||
cross_sum = 0.0
|
||||
x_sum = 0.0
|
||||
y_sum = 0.0
|
||||
for first, second in zip(polygon, (*polygon[1:], polygon[0])):
|
||||
cross = first[0] * second[1] - second[0] * first[1]
|
||||
cross_sum += cross
|
||||
x_sum += (first[0] + second[0]) * cross
|
||||
y_sum += (first[1] + second[1]) * cross
|
||||
if abs(cross_sum) <= 1e-12:
|
||||
raise ValueError("Frame does not overlap the active Image Empty")
|
||||
return x_sum / (3.0 * cross_sum), y_sum / (3.0 * cross_sum)
|
||||
|
||||
|
||||
def frame_output_size(frame_quad, region_size, max_resolution):
|
||||
"""Choose output resolution from Frame pixels within the current Region."""
|
||||
frame_width = abs(float(frame_quad[1][0] - frame_quad[0][0]))
|
||||
frame_height = abs(float(frame_quad[0][1] - frame_quad[3][1]))
|
||||
region_width, region_height = (float(value) for value in region_size)
|
||||
maximum = float(max_resolution)
|
||||
if min(frame_width, frame_height, region_width, region_height, maximum) <= 0.0:
|
||||
raise ValueError("The Frame output dimensions must be positive")
|
||||
scale = min(
|
||||
1.0,
|
||||
region_width / frame_width,
|
||||
region_height / frame_height,
|
||||
maximum / max(frame_width, frame_height),
|
||||
math.sqrt(
|
||||
MAX_PROJECTIVE_OUTPUT_PIXELS / max(frame_width * frame_height, 1.0)
|
||||
),
|
||||
)
|
||||
return (
|
||||
max(2, int(round(frame_width * scale))),
|
||||
max(2, int(round(frame_height * scale))),
|
||||
)
|
||||
|
||||
|
||||
def view_facing_frame_geometry(corners, output_size):
|
||||
"""Return a world matrix array and display size for frame-plane corners."""
|
||||
import numpy as np
|
||||
|
||||
corners = np.asarray(corners, dtype=np.float64)
|
||||
if corners.shape != (4, 3) or not np.isfinite(corners).all():
|
||||
raise ValueError("The Frame view plane is invalid")
|
||||
top_left, top_right, _bottom_right, bottom_left = corners
|
||||
right = top_right - top_left
|
||||
up = top_left - bottom_left
|
||||
width = float(np.linalg.norm(right))
|
||||
height = float(np.linalg.norm(up))
|
||||
if min(width, height) <= 1e-10:
|
||||
raise ValueError("Draw a larger Frame")
|
||||
right /= width
|
||||
up /= height
|
||||
forward = np.cross(right, up)
|
||||
forward_length = float(np.linalg.norm(forward))
|
||||
if forward_length <= 1e-10:
|
||||
raise ValueError("The Frame view plane is invalid")
|
||||
forward /= forward_length
|
||||
matrix = np.identity(4, dtype=np.float64)
|
||||
matrix[:3, 0] = right
|
||||
matrix[:3, 1] = up
|
||||
matrix[:3, 2] = forward
|
||||
matrix[:3, 3] = corners.mean(axis=0)
|
||||
display_size = width if output_size[0] >= output_size[1] else height
|
||||
return matrix, display_size
|
||||
|
||||
|
||||
def frame_result_transform(frame_quad, source_origin, region, view3d, output_size):
|
||||
"""Return the view-facing object matrix and Image Empty display size."""
|
||||
from mathutils import Matrix
|
||||
from bpy_extras import view3d_utils
|
||||
|
||||
corners = tuple(
|
||||
tuple(
|
||||
float(value)
|
||||
for value in view3d_utils.region_2d_to_location_3d(
|
||||
region,
|
||||
view3d,
|
||||
point,
|
||||
source_origin,
|
||||
)
|
||||
)
|
||||
for point in frame_quad
|
||||
)
|
||||
matrix, display_size = view_facing_frame_geometry(corners, output_size)
|
||||
return Matrix(tuple(tuple(row) for row in matrix)), display_size
|
||||
|
||||
|
||||
def frame_depth_location(active_source, overlap_polygon):
|
||||
"""Return the world point whose view depth places the Frame result."""
|
||||
import numpy as np
|
||||
|
||||
source_object = active_source["object"]
|
||||
if not active_source["perspective"] or active_source["origin_depth"] > 0.0:
|
||||
return source_object.matrix_world.translation.copy()
|
||||
centroid = np.asarray((polygon_centroid(overlap_polygon),), dtype=np.float64)
|
||||
_points, depth, valid, normalized_source = frame_source_coordinates(
|
||||
active_source,
|
||||
centroid,
|
||||
)
|
||||
if not bool(valid[0]) or depth[0] <= 0.0:
|
||||
raise ValueError("The Frame result depth is invalid")
|
||||
source_point = np.asarray(
|
||||
(normalized_source[0, 0], normalized_source[0, 1], 1.0),
|
||||
dtype=np.float64,
|
||||
)
|
||||
world = active_source["source_to_world"] @ source_point
|
||||
if not np.isfinite(world).all() or abs(world[3]) <= 1e-12:
|
||||
raise ValueError("The Frame result depth is invalid")
|
||||
return tuple(float(value) for value in world[:3] / world[3])
|
||||
@@ -39,7 +39,7 @@ def compose_alpha(source_alpha, mask_values, mode):
|
||||
mode = str(mode)
|
||||
if mode == "SET":
|
||||
result = alpha * mask
|
||||
elif mode == "ADD":
|
||||
elif mode == "EXTEND":
|
||||
result = np.maximum(alpha, mask)
|
||||
elif mode == "SUBTRACT":
|
||||
result = alpha * (1.0 - mask)
|
||||
@@ -72,7 +72,7 @@ def rasterize_brush_path(image_size, path, radius, project_path):
|
||||
class EditImageAlpha(ImageGesture, bpy.types.Operator):
|
||||
bl_idname = "anyimage.edit_image_alpha"
|
||||
bl_label = "Edit Image Alpha"
|
||||
bl_description = "Edit Image Empty Alpha with a lasso, brush, or polyline"
|
||||
bl_description = "Hide or reveal image areas with a lasso, brush, or polyline."
|
||||
bl_options = {"UNDO"}
|
||||
active_tool_id = MASK_TOOL_ID
|
||||
resolve_on_press = True
|
||||
@@ -157,7 +157,7 @@ class MaskTool(bpy.types.WorkSpaceTool):
|
||||
bl_context_mode = "OBJECT"
|
||||
bl_idname = MASK_TOOL_ID
|
||||
bl_label = "Mask"
|
||||
bl_description = "Edit Image Empty Alpha with a lasso, brush, or polyline"
|
||||
bl_description = "Hide or reveal image areas with a lasso, brush, or polyline."
|
||||
bl_icon = "ops.sculpt.lasso_mask"
|
||||
bl_operator = EditImageAlpha.bl_idname
|
||||
bl_cursor = "PAINT_CROSS"
|
||||
|
||||
7
src/anyimage/operators/rectify_tool/__init__.py
Normal file
7
src/anyimage/operators/rectify_tool/__init__.py
Normal file
@@ -0,0 +1,7 @@
|
||||
"""Perspective correction operators and workspace tool."""
|
||||
|
||||
from .operators import RectifyImagePerspective, RectifyTool
|
||||
|
||||
CLASSES = (RectifyImagePerspective,)
|
||||
|
||||
__all__ = ("CLASSES", "RectifyImagePerspective", "RectifyTool")
|
||||
144
src/anyimage/operators/rectify_tool/geometry.py
Normal file
144
src/anyimage/operators/rectify_tool/geometry.py
Normal file
@@ -0,0 +1,144 @@
|
||||
"""Validate perspective quads and resample the selected image region."""
|
||||
|
||||
import math
|
||||
|
||||
from ...common.selection import (
|
||||
clip_polygon_halfplanes,
|
||||
trim_pixels_to_alpha,
|
||||
)
|
||||
from ...common.image import (
|
||||
projective_output_size,
|
||||
warp_projective_pixels,
|
||||
)
|
||||
|
||||
|
||||
MIN_INTERACTIVE_ASPECT = 0.05
|
||||
MAX_INTERACTIVE_ASPECT = 20.0
|
||||
|
||||
|
||||
def validate_perspective_quad(points, min_area=1e-6):
|
||||
"""Return four convex corners without changing their correspondence."""
|
||||
import numpy as np
|
||||
|
||||
quad = np.asarray(points, dtype=np.float64)
|
||||
if quad.shape != (4, 2) or not np.isfinite(quad).all():
|
||||
raise ValueError("Rectify requires four valid image corners")
|
||||
edges = np.roll(quad, -1, axis=0) - quad
|
||||
if np.any(np.linalg.norm(edges, axis=1) < 2.0):
|
||||
raise ValueError("Rectify corners are too close together")
|
||||
crosses = np.cross(edges, np.roll(edges, -1, axis=0))
|
||||
if np.any(np.abs(crosses) <= 1e-8) or not (
|
||||
np.all(crosses > 0.0) or np.all(crosses < 0.0)
|
||||
):
|
||||
raise ValueError("Rectify corners must define one convex quadrilateral")
|
||||
area = 0.5 * abs(
|
||||
float(
|
||||
np.dot(quad[:, 0], np.roll(quad[:, 1], -1))
|
||||
- np.dot(quad[:, 1], np.roll(quad[:, 0], -1))
|
||||
)
|
||||
)
|
||||
if area < min_area:
|
||||
raise ValueError("Draw a larger Rectify quad")
|
||||
return quad
|
||||
|
||||
|
||||
def canonical_perspective_quad(points, min_area=1e-6):
|
||||
"""Order four unordered points as source-oriented top-left through bottom-left."""
|
||||
import numpy as np
|
||||
|
||||
points = np.asarray(points, dtype=np.float64)
|
||||
if points.shape != (4, 2) or not np.isfinite(points).all():
|
||||
raise ValueError("Rectify requires four valid image corners")
|
||||
distances = np.linalg.norm(points[:, None] - points[None, :], axis=2)
|
||||
distances += np.eye(4) * 1e9
|
||||
if float(distances.min()) < 2.0:
|
||||
raise ValueError("Rectify corners are too close together")
|
||||
center = points.mean(axis=0)
|
||||
ring = points[np.argsort(np.arctan2(points[:, 1] - center[1], points[:, 0] - center[0]))]
|
||||
ring = validate_perspective_quad(ring, min_area)
|
||||
candidates = []
|
||||
for winding in (ring, ring[::-1]):
|
||||
for start in range(4):
|
||||
candidate = np.roll(winding, -start, axis=0)
|
||||
top = candidate[1] - candidate[0]
|
||||
left = candidate[3] - candidate[0]
|
||||
score = top[0] / np.linalg.norm(top) + left[1] / np.linalg.norm(left)
|
||||
candidates.append((float(score), -candidate[0, 1], -candidate[0, 0], candidate))
|
||||
return max(candidates, key=lambda item: item[:3])[3]
|
||||
|
||||
|
||||
def perspective_quad_overlaps_image(points, image_size):
|
||||
import numpy as np
|
||||
|
||||
width, height = image_size
|
||||
if min(width, height) <= 0:
|
||||
raise ValueError("The source image has no readable dimensions")
|
||||
clipped = np.asarray(clip_polygon_halfplanes(
|
||||
points, ((1, 0, 0), (-1, 0, width), (0, 1, 0), (0, -1, height))
|
||||
))
|
||||
if clipped.ndim != 2 or len(clipped) < 3:
|
||||
return False
|
||||
area = 0.5 * abs(
|
||||
float(
|
||||
np.dot(clipped[:, 0], np.roll(clipped[:, 1], -1))
|
||||
- np.dot(clipped[:, 1], np.roll(clipped[:, 0], -1))
|
||||
)
|
||||
)
|
||||
return area > 1e-6
|
||||
|
||||
|
||||
def perspective_quad_aspect(quad):
|
||||
"""Estimate an initial output aspect from the visible quad edge lengths."""
|
||||
import numpy as np
|
||||
|
||||
quad = validate_perspective_quad(quad)
|
||||
width = 0.5 * (
|
||||
np.linalg.norm(quad[1] - quad[0]) + np.linalg.norm(quad[2] - quad[3])
|
||||
)
|
||||
height = 0.5 * (
|
||||
np.linalg.norm(quad[2] - quad[1]) + np.linalg.norm(quad[3] - quad[0])
|
||||
)
|
||||
return min(
|
||||
max(float(width / max(height, 1e-8)), MIN_INTERACTIVE_ASPECT),
|
||||
MAX_INTERACTIVE_ASPECT,
|
||||
)
|
||||
|
||||
|
||||
def perspective_placement_bounds(quad, output_size, trim_bounds):
|
||||
"""Map a rectified Alpha trim back to an approximate source-image rectangle."""
|
||||
quad = validate_perspective_quad(quad)
|
||||
source_left = math.floor(float(quad[:, 0].min()))
|
||||
source_top = math.floor(float(quad[:, 1].min()))
|
||||
source_right = math.ceil(float(quad[:, 0].max()))
|
||||
source_bottom = math.ceil(float(quad[:, 1].max()))
|
||||
output_width, output_height = (float(value) for value in output_size)
|
||||
trim_left, trim_top, trim_right, trim_bottom = trim_bounds
|
||||
source_width = source_right - source_left
|
||||
source_height = source_bottom - source_top
|
||||
return (
|
||||
source_left + trim_left * source_width / output_width,
|
||||
source_top + trim_top * source_height / output_height,
|
||||
source_left + trim_right * source_width / output_width,
|
||||
source_top + trim_bottom * source_height / output_height,
|
||||
)
|
||||
|
||||
|
||||
def extract_perspective_pixels(
|
||||
source_pixels,
|
||||
source_size,
|
||||
quad,
|
||||
aspect_ratio,
|
||||
):
|
||||
output_size = projective_output_size(quad, aspect_ratio)
|
||||
pixels = warp_projective_pixels(
|
||||
source_pixels, source_size, quad, output_size
|
||||
)
|
||||
pixels, trim_bounds = trim_pixels_to_alpha(pixels, (0, 0, *output_size))
|
||||
trimmed_size = (
|
||||
trim_bounds[2] - trim_bounds[0],
|
||||
trim_bounds[3] - trim_bounds[1],
|
||||
)
|
||||
placement_bounds = perspective_placement_bounds(
|
||||
quad, output_size, trim_bounds
|
||||
)
|
||||
return pixels, trimmed_size, placement_bounds
|
||||
@@ -1,9 +1,11 @@
|
||||
"""Rectify interaction and Blender image replacement."""
|
||||
|
||||
import json
|
||||
import math
|
||||
|
||||
import bpy
|
||||
|
||||
from ..common.viewport import (
|
||||
from ...common.viewport import (
|
||||
active_view3d_tool_id,
|
||||
deserialize_matrix,
|
||||
draw_dashed_line,
|
||||
@@ -16,311 +18,45 @@ from ..common.viewport import (
|
||||
screen_path_to_image_pixels,
|
||||
serialize_matrix,
|
||||
)
|
||||
from ..common.selection import clip_polygon_halfplanes, trim_pixels_to_alpha
|
||||
from ..common.image import (
|
||||
from ...common.image import (
|
||||
create_image_edit_result,
|
||||
image_base_name,
|
||||
image_pixels,
|
||||
image_empty_bounds,
|
||||
projective_output_size,
|
||||
replace_empty_image,
|
||||
require_image_empty,
|
||||
video_total_frames,
|
||||
is_animated_image,
|
||||
warp_projective_pixels,
|
||||
)
|
||||
MIN_QUAD_AREA = 16.0
|
||||
RECTIFY_TOOL_ID = "anyimage.rectify"
|
||||
PREVIEW_TEXTURE_SIZE = 512
|
||||
ASPECT_PIXELS_PER_DOUBLE = 240.0
|
||||
MIN_INTERACTIVE_ASPECT = 0.05
|
||||
MAX_INTERACTIVE_ASPECT = 20.0
|
||||
ASPECT_PRESETS = (
|
||||
("9 : 16", 9.0 / 16.0),
|
||||
("3 : 4", 3.0 / 4.0),
|
||||
("1 : 1", 1.0),
|
||||
("4 : 3", 4.0 / 3.0),
|
||||
("16 : 9", 16.0 / 9.0),
|
||||
from .geometry import (
|
||||
MAX_INTERACTIVE_ASPECT,
|
||||
MIN_INTERACTIVE_ASPECT,
|
||||
canonical_perspective_quad,
|
||||
extract_perspective_pixels,
|
||||
perspective_quad_aspect,
|
||||
perspective_quad_overlaps_image,
|
||||
validate_perspective_quad,
|
||||
)
|
||||
from .preview import (
|
||||
PREVIEW_TEXTURE_SIZE,
|
||||
aspect_ratio_from_mouse,
|
||||
create_perspective_preview_image,
|
||||
draw_aspect_hud,
|
||||
draw_preview_frame,
|
||||
interactive_preview_polygon,
|
||||
preview_draw_bounds,
|
||||
preview_texture_draw_options,
|
||||
snapped_aspect_ratio,
|
||||
)
|
||||
|
||||
|
||||
def validate_perspective_quad(points, min_area=1e-6):
|
||||
"""Return four convex corners without changing their correspondence."""
|
||||
import numpy as np
|
||||
|
||||
quad = np.asarray(points, dtype=np.float64)
|
||||
if quad.shape != (4, 2) or not np.isfinite(quad).all():
|
||||
raise ValueError("Rectify requires four valid image corners")
|
||||
edges = np.roll(quad, -1, axis=0) - quad
|
||||
if np.any(np.linalg.norm(edges, axis=1) < 2.0):
|
||||
raise ValueError("Rectify corners are too close together")
|
||||
crosses = np.cross(edges, np.roll(edges, -1, axis=0))
|
||||
if np.any(np.abs(crosses) <= 1e-8) or not (
|
||||
np.all(crosses > 0.0) or np.all(crosses < 0.0)
|
||||
):
|
||||
raise ValueError("Rectify corners must define one convex quadrilateral")
|
||||
area = 0.5 * abs(
|
||||
float(
|
||||
np.dot(quad[:, 0], np.roll(quad[:, 1], -1))
|
||||
- np.dot(quad[:, 1], np.roll(quad[:, 0], -1))
|
||||
)
|
||||
)
|
||||
if area < min_area:
|
||||
raise ValueError("Draw a larger Rectify quad")
|
||||
return quad
|
||||
|
||||
|
||||
def canonical_perspective_quad(points, min_area=1e-6):
|
||||
"""Order four unordered points as source-oriented top-left through bottom-left."""
|
||||
import numpy as np
|
||||
|
||||
points = np.asarray(points, dtype=np.float64)
|
||||
if points.shape != (4, 2) or not np.isfinite(points).all():
|
||||
raise ValueError("Rectify requires four valid image corners")
|
||||
distances = np.linalg.norm(points[:, None] - points[None, :], axis=2)
|
||||
distances += np.eye(4) * 1e9
|
||||
if float(distances.min()) < 2.0:
|
||||
raise ValueError("Rectify corners are too close together")
|
||||
center = points.mean(axis=0)
|
||||
ring = points[np.argsort(np.arctan2(points[:, 1] - center[1], points[:, 0] - center[0]))]
|
||||
ring = validate_perspective_quad(ring, min_area)
|
||||
candidates = []
|
||||
for winding in (ring, ring[::-1]):
|
||||
for start in range(4):
|
||||
candidate = np.roll(winding, -start, axis=0)
|
||||
top = candidate[1] - candidate[0]
|
||||
left = candidate[3] - candidate[0]
|
||||
score = top[0] / np.linalg.norm(top) + left[1] / np.linalg.norm(left)
|
||||
candidates.append((float(score), -candidate[0, 1], -candidate[0, 0], candidate))
|
||||
return max(candidates, key=lambda item: item[:3])[3]
|
||||
|
||||
|
||||
def preview_polygon(points):
|
||||
"""Return a non-crossing preview hull for the current unordered clicks."""
|
||||
import numpy as np
|
||||
|
||||
points = np.asarray(points, dtype=np.float64)
|
||||
if len(points) < 3:
|
||||
return points
|
||||
center = points.mean(axis=0)
|
||||
return points[np.argsort(np.arctan2(points[:, 1] - center[1], points[:, 0] - center[0]))]
|
||||
|
||||
|
||||
def interactive_preview_polygon(points, cursor):
|
||||
"""Return the clicked points plus one distinct moving cursor point."""
|
||||
vertices = list(points)
|
||||
if (
|
||||
cursor is not None
|
||||
and vertices
|
||||
and len(vertices) < 4
|
||||
and math.dist(vertices[-1], cursor) >= 2.0
|
||||
):
|
||||
vertices.append(cursor)
|
||||
return preview_polygon(vertices)
|
||||
|
||||
|
||||
def perspective_quad_overlaps_image(points, image_size):
|
||||
import numpy as np
|
||||
|
||||
width, height = image_size
|
||||
if min(width, height) <= 0:
|
||||
raise ValueError("The source image has no readable dimensions")
|
||||
clipped = np.asarray(clip_polygon_halfplanes(
|
||||
points, ((1, 0, 0), (-1, 0, width), (0, 1, 0), (0, -1, height))
|
||||
))
|
||||
if clipped.ndim != 2 or len(clipped) < 3:
|
||||
return False
|
||||
area = 0.5 * abs(
|
||||
float(
|
||||
np.dot(clipped[:, 0], np.roll(clipped[:, 1], -1))
|
||||
- np.dot(clipped[:, 1], np.roll(clipped[:, 0], -1))
|
||||
)
|
||||
)
|
||||
return area > 1e-6
|
||||
|
||||
|
||||
def aspect_ratio_from_mouse(initial_ratio, horizontal_delta):
|
||||
ratio = initial_ratio * math.pow(
|
||||
2.0, float(horizontal_delta) / ASPECT_PIXELS_PER_DOUBLE
|
||||
)
|
||||
return min(max(ratio, MIN_INTERACTIVE_ASPECT), MAX_INTERACTIVE_ASPECT)
|
||||
|
||||
|
||||
def snapped_aspect_ratio(aspect_ratio):
|
||||
label, ratio = min(
|
||||
ASPECT_PRESETS,
|
||||
key=lambda preset: abs(math.log(aspect_ratio / preset[1])),
|
||||
)
|
||||
return ratio, label
|
||||
|
||||
|
||||
def aspect_ratio_label(aspect_ratio, preset_label=None):
|
||||
if preset_label is not None:
|
||||
return preset_label
|
||||
if aspect_ratio >= 1.0:
|
||||
return f"{aspect_ratio:.2f} : 1"
|
||||
return f"1 : {1.0 / aspect_ratio:.2f}"
|
||||
|
||||
|
||||
def preview_draw_bounds(region_size, aspect_ratio):
|
||||
region_width, region_height = (float(value) for value in region_size)
|
||||
available_width = min(region_width * 0.65, 640.0)
|
||||
available_height = min(region_height * 0.65, 640.0)
|
||||
if aspect_ratio >= available_width / available_height:
|
||||
width = available_width
|
||||
height = width / aspect_ratio
|
||||
else:
|
||||
height = available_height
|
||||
width = height * aspect_ratio
|
||||
return (
|
||||
(region_width - width) * 0.5,
|
||||
(region_height - height) * 0.5,
|
||||
width,
|
||||
height,
|
||||
)
|
||||
|
||||
|
||||
def draw_preview_frame(bounds):
|
||||
import gpu
|
||||
from gpu_extras.batch import batch_for_shader
|
||||
|
||||
left, bottom, width, height = bounds
|
||||
vertices = (
|
||||
(left, bottom),
|
||||
(left + width, bottom),
|
||||
(left + width, bottom + height),
|
||||
(left, bottom + height),
|
||||
(left, bottom),
|
||||
)
|
||||
shader = gpu.shader.from_builtin("UNIFORM_COLOR")
|
||||
batch = batch_for_shader(shader, "LINE_STRIP", {"pos": vertices})
|
||||
gpu.state.blend_set("ALPHA")
|
||||
gpu.state.line_width_set(2.0)
|
||||
shader.bind()
|
||||
shader.uniform_float("color", (0.12, 0.52, 1.0, 1.0))
|
||||
batch.draw(shader)
|
||||
gpu.state.line_width_set(1.0)
|
||||
gpu.state.blend_set("NONE")
|
||||
|
||||
|
||||
def draw_centered_text(text, center_x, baseline_y, size, color):
|
||||
import blf
|
||||
|
||||
font_id = 0
|
||||
blf.size(font_id, size)
|
||||
width, _height = blf.dimensions(font_id, text)
|
||||
x = center_x - width * 0.5
|
||||
blf.position(font_id, x + 1.0, baseline_y - 1.0, 0.0)
|
||||
blf.color(font_id, 0.0, 0.0, 0.0, 0.9)
|
||||
blf.draw(font_id, text)
|
||||
blf.position(font_id, x, baseline_y, 0.0)
|
||||
blf.color(font_id, *color)
|
||||
blf.draw(font_id, text)
|
||||
|
||||
|
||||
def draw_aspect_hud(bounds, aspect_ratio, preset_label=None):
|
||||
left, bottom, width, height = bounds
|
||||
center_x = left + width * 0.5
|
||||
draw_centered_text(
|
||||
aspect_ratio_label(aspect_ratio, preset_label),
|
||||
center_x,
|
||||
bottom + height * 0.5 - 10.0,
|
||||
22,
|
||||
(0.25, 0.65, 1.0, 1.0),
|
||||
)
|
||||
draw_centered_text(
|
||||
"Move horizontally • Ctrl snap • LMB confirm • Backspace edit",
|
||||
center_x,
|
||||
bottom - 26.0,
|
||||
13,
|
||||
(0.85, 0.9, 1.0, 1.0),
|
||||
)
|
||||
|
||||
|
||||
def create_perspective_preview_image(source_image, pixels):
|
||||
image = bpy.data.images.new(
|
||||
f"{image_base_name(source_image)}.rectify-preview",
|
||||
width=PREVIEW_TEXTURE_SIZE,
|
||||
height=PREVIEW_TEXTURE_SIZE,
|
||||
alpha=True,
|
||||
)
|
||||
try:
|
||||
image.colorspace_settings.name = "sRGB"
|
||||
image.alpha_mode = "STRAIGHT"
|
||||
image.pixels.foreach_set(pixels)
|
||||
image.update()
|
||||
except Exception:
|
||||
bpy.data.images.remove(image, do_unlink=True)
|
||||
raise
|
||||
return image
|
||||
|
||||
|
||||
def preview_texture_draw_options(blender_version):
|
||||
if tuple(blender_version) >= (5, 0, 0):
|
||||
return {"is_scene_linear_with_rec709_srgb_target": True}
|
||||
return {}
|
||||
|
||||
|
||||
def perspective_quad_aspect(quad):
|
||||
"""Estimate an initial output aspect from the visible quad edge lengths."""
|
||||
import numpy as np
|
||||
|
||||
quad = validate_perspective_quad(quad)
|
||||
width = 0.5 * (
|
||||
np.linalg.norm(quad[1] - quad[0]) + np.linalg.norm(quad[2] - quad[3])
|
||||
)
|
||||
height = 0.5 * (
|
||||
np.linalg.norm(quad[2] - quad[1]) + np.linalg.norm(quad[3] - quad[0])
|
||||
)
|
||||
return min(
|
||||
max(float(width / max(height, 1e-8)), MIN_INTERACTIVE_ASPECT),
|
||||
MAX_INTERACTIVE_ASPECT,
|
||||
)
|
||||
|
||||
|
||||
def perspective_placement_bounds(quad, output_size, trim_bounds):
|
||||
"""Map a rectified Alpha trim back to an approximate source-image rectangle."""
|
||||
quad = validate_perspective_quad(quad)
|
||||
source_left = math.floor(float(quad[:, 0].min()))
|
||||
source_top = math.floor(float(quad[:, 1].min()))
|
||||
source_right = math.ceil(float(quad[:, 0].max()))
|
||||
source_bottom = math.ceil(float(quad[:, 1].max()))
|
||||
output_width, output_height = (float(value) for value in output_size)
|
||||
trim_left, trim_top, trim_right, trim_bottom = trim_bounds
|
||||
source_width = source_right - source_left
|
||||
source_height = source_bottom - source_top
|
||||
return (
|
||||
source_left + trim_left * source_width / output_width,
|
||||
source_top + trim_top * source_height / output_height,
|
||||
source_left + trim_right * source_width / output_width,
|
||||
source_top + trim_bottom * source_height / output_height,
|
||||
)
|
||||
|
||||
|
||||
def extract_perspective_pixels(
|
||||
source_pixels,
|
||||
source_size,
|
||||
quad,
|
||||
aspect_ratio,
|
||||
):
|
||||
output_size = projective_output_size(quad, aspect_ratio)
|
||||
pixels = warp_projective_pixels(
|
||||
source_pixels, source_size, quad, output_size
|
||||
)
|
||||
pixels, trim_bounds = trim_pixels_to_alpha(pixels, (0, 0, *output_size))
|
||||
trimmed_size = (
|
||||
trim_bounds[2] - trim_bounds[0],
|
||||
trim_bounds[3] - trim_bounds[1],
|
||||
)
|
||||
placement_bounds = perspective_placement_bounds(
|
||||
quad, output_size, trim_bounds
|
||||
)
|
||||
return pixels, trimmed_size, placement_bounds
|
||||
MIN_QUAD_AREA = 16.0
|
||||
RECTIFY_TOOL_ID = "anyimage.rectify"
|
||||
|
||||
|
||||
class RectifyImagePerspective(bpy.types.Operator):
|
||||
bl_idname = "anyimage.rectify_image_perspective"
|
||||
bl_label = "Rectify Image Perspective"
|
||||
bl_description = "Rectify four image corners into a rectangular image"
|
||||
bl_description = "Select four corners of an image area to correct its perspective into a rectangle."
|
||||
bl_options = {"REGISTER", "UNDO"}
|
||||
|
||||
source_object_name: bpy.props.StringProperty(options={"HIDDEN", "SKIP_SAVE"})
|
||||
@@ -456,7 +192,7 @@ class RectifyImagePerspective(bpy.types.Operator):
|
||||
return {"FINISHED"} if source_object is not None else {"CANCELLED"}
|
||||
if source_object is None:
|
||||
return {"CANCELLED"}
|
||||
if video_total_frames(source_object.data) > 1:
|
||||
if is_animated_image(source_object.data):
|
||||
self.report(
|
||||
{"WARNING"},
|
||||
"Rectify only supports a still image",
|
||||
@@ -621,7 +357,7 @@ class RectifyImagePerspective(bpy.types.Operator):
|
||||
self.report({"ERROR"}, str(error))
|
||||
return {"CANCELLED"}
|
||||
try:
|
||||
if video_total_frames(source_object.data) > 1:
|
||||
if is_animated_image(source_object.data):
|
||||
raise RuntimeError("Rectify only supports a still image")
|
||||
quad = validate_perspective_quad(json.loads(self.quad_json))
|
||||
if not perspective_quad_overlaps_image(
|
||||
@@ -632,8 +368,6 @@ class RectifyImagePerspective(bpy.types.Operator):
|
||||
"Rectify quad does not overlap the source image",
|
||||
)
|
||||
return {"CANCELLED"}
|
||||
import numpy as np
|
||||
|
||||
source_pixels = self._source_pixels
|
||||
if source_pixels is None or len(source_pixels) != len(
|
||||
source_object.data.pixels
|
||||
@@ -672,7 +406,7 @@ class RectifyTool(bpy.types.WorkSpaceTool):
|
||||
bl_context_mode = "OBJECT"
|
||||
bl_idname = RECTIFY_TOOL_ID
|
||||
bl_label = "Rectify"
|
||||
bl_description = "Rectify four image corners into a rectangular image"
|
||||
bl_description = "Select four corners of an image area to correct its perspective into a rectangle."
|
||||
bl_icon = "ops.sculpt.line_project"
|
||||
bl_operator = RectifyImagePerspective.bl_idname
|
||||
bl_keymap = image_edit_point_keymap(RectifyImagePerspective.bl_idname)
|
||||
168
src/anyimage/operators/rectify_tool/preview.py
Normal file
168
src/anyimage/operators/rectify_tool/preview.py
Normal file
@@ -0,0 +1,168 @@
|
||||
"""Rectify preview images, overlays, and interactive aspect controls."""
|
||||
|
||||
import math
|
||||
|
||||
import bpy
|
||||
|
||||
from ...common.image import image_base_name
|
||||
from .geometry import (
|
||||
MAX_INTERACTIVE_ASPECT,
|
||||
MIN_INTERACTIVE_ASPECT,
|
||||
)
|
||||
|
||||
|
||||
PREVIEW_TEXTURE_SIZE = 512
|
||||
ASPECT_PIXELS_PER_DOUBLE = 240.0
|
||||
ASPECT_PRESETS = (
|
||||
("9 : 16", 9.0 / 16.0),
|
||||
("3 : 4", 3.0 / 4.0),
|
||||
("1 : 1", 1.0),
|
||||
("4 : 3", 4.0 / 3.0),
|
||||
("16 : 9", 16.0 / 9.0),
|
||||
)
|
||||
|
||||
|
||||
def preview_polygon(points):
|
||||
"""Return a non-crossing preview hull for the current unordered clicks."""
|
||||
import numpy as np
|
||||
|
||||
points = np.asarray(points, dtype=np.float64)
|
||||
if len(points) < 3:
|
||||
return points
|
||||
center = points.mean(axis=0)
|
||||
return points[np.argsort(np.arctan2(points[:, 1] - center[1], points[:, 0] - center[0]))]
|
||||
|
||||
|
||||
def interactive_preview_polygon(points, cursor):
|
||||
"""Return the clicked points plus one distinct moving cursor point."""
|
||||
vertices = list(points)
|
||||
if (
|
||||
cursor is not None
|
||||
and vertices
|
||||
and len(vertices) < 4
|
||||
and math.dist(vertices[-1], cursor) >= 2.0
|
||||
):
|
||||
vertices.append(cursor)
|
||||
return preview_polygon(vertices)
|
||||
|
||||
|
||||
def aspect_ratio_from_mouse(initial_ratio, horizontal_delta):
|
||||
ratio = initial_ratio * math.pow(
|
||||
2.0, float(horizontal_delta) / ASPECT_PIXELS_PER_DOUBLE
|
||||
)
|
||||
return min(max(ratio, MIN_INTERACTIVE_ASPECT), MAX_INTERACTIVE_ASPECT)
|
||||
|
||||
|
||||
def snapped_aspect_ratio(aspect_ratio):
|
||||
label, ratio = min(
|
||||
ASPECT_PRESETS,
|
||||
key=lambda preset: abs(math.log(aspect_ratio / preset[1])),
|
||||
)
|
||||
return ratio, label
|
||||
|
||||
|
||||
def aspect_ratio_label(aspect_ratio, preset_label=None):
|
||||
if preset_label is not None:
|
||||
return preset_label
|
||||
if aspect_ratio >= 1.0:
|
||||
return f"{aspect_ratio:.2f} : 1"
|
||||
return f"1 : {1.0 / aspect_ratio:.2f}"
|
||||
|
||||
|
||||
def preview_draw_bounds(region_size, aspect_ratio):
|
||||
region_width, region_height = (float(value) for value in region_size)
|
||||
available_width = min(region_width * 0.65, 640.0)
|
||||
available_height = min(region_height * 0.65, 640.0)
|
||||
if aspect_ratio >= available_width / available_height:
|
||||
width = available_width
|
||||
height = width / aspect_ratio
|
||||
else:
|
||||
height = available_height
|
||||
width = height * aspect_ratio
|
||||
return (
|
||||
(region_width - width) * 0.5,
|
||||
(region_height - height) * 0.5,
|
||||
width,
|
||||
height,
|
||||
)
|
||||
|
||||
|
||||
def draw_preview_frame(bounds):
|
||||
import gpu
|
||||
from gpu_extras.batch import batch_for_shader
|
||||
|
||||
left, bottom, width, height = bounds
|
||||
vertices = (
|
||||
(left, bottom),
|
||||
(left + width, bottom),
|
||||
(left + width, bottom + height),
|
||||
(left, bottom + height),
|
||||
(left, bottom),
|
||||
)
|
||||
shader = gpu.shader.from_builtin("UNIFORM_COLOR")
|
||||
batch = batch_for_shader(shader, "LINE_STRIP", {"pos": vertices})
|
||||
gpu.state.blend_set("ALPHA")
|
||||
gpu.state.line_width_set(2.0)
|
||||
shader.bind()
|
||||
shader.uniform_float("color", (0.12, 0.52, 1.0, 1.0))
|
||||
batch.draw(shader)
|
||||
gpu.state.line_width_set(1.0)
|
||||
gpu.state.blend_set("NONE")
|
||||
|
||||
|
||||
def draw_centered_text(text, center_x, baseline_y, size, color):
|
||||
import blf
|
||||
|
||||
font_id = 0
|
||||
blf.size(font_id, size)
|
||||
width, _height = blf.dimensions(font_id, text)
|
||||
x = center_x - width * 0.5
|
||||
blf.position(font_id, x + 1.0, baseline_y - 1.0, 0.0)
|
||||
blf.color(font_id, 0.0, 0.0, 0.0, 0.9)
|
||||
blf.draw(font_id, text)
|
||||
blf.position(font_id, x, baseline_y, 0.0)
|
||||
blf.color(font_id, *color)
|
||||
blf.draw(font_id, text)
|
||||
|
||||
|
||||
def draw_aspect_hud(bounds, aspect_ratio, preset_label=None):
|
||||
left, bottom, width, height = bounds
|
||||
center_x = left + width * 0.5
|
||||
draw_centered_text(
|
||||
aspect_ratio_label(aspect_ratio, preset_label),
|
||||
center_x,
|
||||
bottom + height * 0.5 - 10.0,
|
||||
22,
|
||||
(0.25, 0.65, 1.0, 1.0),
|
||||
)
|
||||
draw_centered_text(
|
||||
"Move horizontally • Ctrl snap • LMB confirm • Backspace edit",
|
||||
center_x,
|
||||
bottom - 26.0,
|
||||
13,
|
||||
(0.85, 0.9, 1.0, 1.0),
|
||||
)
|
||||
|
||||
|
||||
def create_perspective_preview_image(source_image, pixels):
|
||||
image = bpy.data.images.new(
|
||||
f"{image_base_name(source_image)}.rectify-preview",
|
||||
width=PREVIEW_TEXTURE_SIZE,
|
||||
height=PREVIEW_TEXTURE_SIZE,
|
||||
alpha=True,
|
||||
)
|
||||
try:
|
||||
image.colorspace_settings.name = "sRGB"
|
||||
image.alpha_mode = "STRAIGHT"
|
||||
image.pixels.foreach_set(pixels)
|
||||
image.update()
|
||||
except Exception:
|
||||
bpy.data.images.remove(image, do_unlink=True)
|
||||
raise
|
||||
return image
|
||||
|
||||
|
||||
def preview_texture_draw_options(blender_version):
|
||||
if tuple(blender_version) >= (5, 0, 0):
|
||||
return {"is_scene_linear_with_rec709_srgb_target": True}
|
||||
return {}
|
||||
@@ -86,23 +86,20 @@ class RunBackgroundRemoval(JobOperatorBase, bpy.types.Operator):
|
||||
require_environment()
|
||||
model = configured_background_model(context)
|
||||
require_model(model)
|
||||
settings = context.scene.anyimage_settings
|
||||
parameters = {
|
||||
"input": str(require_input_path(self.input_path)),
|
||||
"model": model,
|
||||
"device": production_device().lower(),
|
||||
"video_frames": settings.video_frames,
|
||||
"start_frame": self._image_target.start_frame,
|
||||
}
|
||||
if getattr(self, "_hdr_input", None) is not None:
|
||||
parameters.update(output_kind="alpha", video_frames=1, start_frame=1)
|
||||
parameters["output_kind"] = "alpha"
|
||||
return parameters
|
||||
|
||||
def response(self, _context, result):
|
||||
if getattr(self, "_hdr_input", None) is not None:
|
||||
self._hdr_input.apply(self._image_target, result.file("alpha"), prepare_undo=True)
|
||||
else:
|
||||
self._image_target.apply(result.directory)
|
||||
self._image_target.apply(result.file("foreground"))
|
||||
if "FINISHED" not in bpy.ops.ed.undo_push(message="Remove Background"):
|
||||
raise RuntimeError("Unable to create the Remove Background Undo step")
|
||||
return "Image background is removed"
|
||||
|
||||
@@ -115,18 +115,15 @@ class UpscaleImage(JobOperatorBase, bpy.types.Operator):
|
||||
def request(self, context):
|
||||
require_environment()
|
||||
require_model(self.model)
|
||||
settings = context.scene.anyimage_settings
|
||||
return {
|
||||
"input": str(require_input_path(self.input_path)),
|
||||
"model": self.model,
|
||||
"device": production_device().lower(),
|
||||
"video_frames": settings.video_frames,
|
||||
"start_frame": self._image_target.start_frame,
|
||||
"max_input_size": int(self.max_input_size),
|
||||
}
|
||||
|
||||
def response(self, _context, result):
|
||||
self._image_target.apply(result.directory, frame_prefix="upscale.")
|
||||
self._image_target.apply(result.file("upscale"))
|
||||
model_label = upscale_model_label(self.model)
|
||||
return f"Image is upscaled with {model_label}"
|
||||
|
||||
|
||||
@@ -87,15 +87,15 @@ class ServerPanel(bpy.types.Panel):
|
||||
cached_row = layout.row(align=True)
|
||||
if len(cached_names) > 1:
|
||||
cached_list = cached_row.column(align=True)
|
||||
cached_list.label(text="Cached:")
|
||||
cached_list.label(text="Loaded Models:")
|
||||
for name in cached_names:
|
||||
cached_list.label(text=name)
|
||||
else:
|
||||
cached_row.label(
|
||||
text=(
|
||||
f"Cached: {cached_names[0]}"
|
||||
f"Loaded Models: {cached_names[0]}"
|
||||
if cached_names
|
||||
else "Cached: None"
|
||||
else "No Models Loaded"
|
||||
)
|
||||
)
|
||||
clear = cached_row.row(align=True)
|
||||
|
||||
@@ -39,9 +39,9 @@ BALLOON_PROFILE_ALGORITHM_ITEMS = (
|
||||
(
|
||||
"POISSON",
|
||||
"Poisson",
|
||||
"Smooth cotangent-Laplacian Balloon profile",
|
||||
"Create a smooth, rounded bulge within the Cutout outline.",
|
||||
),
|
||||
("TEDDY", "Teddy", "Chordal-axis Balloon profile"),
|
||||
("TEDDY", "Teddy", "Create a bulge shaped by the Cutout's central branches."),
|
||||
)
|
||||
|
||||
|
||||
@@ -136,27 +136,25 @@ class AnyImagePreferences(bpy.types.AddonPreferences):
|
||||
bl_idname = ADDON_PACKAGE
|
||||
|
||||
adapt_material_to_view_transform: bpy.props.BoolProperty(
|
||||
name="Adapt to Scene View Transform",
|
||||
name="Preserve Image Colors",
|
||||
description=(
|
||||
"Use the scene view transform's inverse color space with Straight alpha "
|
||||
"for new material color textures; otherwise use sRGB with Premultiplied alpha"
|
||||
"Compensate for the scene's view transform to keep image colors "
|
||||
"close to the original when creating materials."
|
||||
),
|
||||
default=True,
|
||||
)
|
||||
storage_root: bpy.props.StringProperty(
|
||||
name="Storage Root",
|
||||
name="Storage Folder",
|
||||
description=(
|
||||
"Shared user-writable storage for the environment, models and jobs; "
|
||||
"all Blender versions can use the same directory"
|
||||
"Choose where AnyImage stores its AI environment, downloaded models, and task files."
|
||||
),
|
||||
subtype="DIR_PATH",
|
||||
default=str(DEFAULT_STORAGE_ROOT),
|
||||
)
|
||||
device: bpy.props.EnumProperty(
|
||||
name="Device",
|
||||
name="AI Processing Device",
|
||||
description=(
|
||||
"Inference device used by production tasks; only devices supported "
|
||||
"on this machine are listed"
|
||||
"Choose the CPU or GPU used to run AI models."
|
||||
),
|
||||
items=device_items,
|
||||
default=0,
|
||||
@@ -170,8 +168,8 @@ class AnyImagePreferences(bpy.types.AddonPreferences):
|
||||
max_ai_input_size: bpy.props.IntProperty(
|
||||
name="Maximum AI Input Size",
|
||||
description=(
|
||||
"Maximum source long edge eligible for Upscale and maximum long edge "
|
||||
"used by Depth, Normal and Selection refinement"
|
||||
"Limit the longest image edge used for depth, normal, and selection refinement. "
|
||||
"Upscale requires a source image smaller than this limit."
|
||||
),
|
||||
default=DEFAULT_MAX_AI_INPUT_SIZE,
|
||||
min=MIN_MAX_AI_INPUT_SIZE,
|
||||
@@ -181,7 +179,10 @@ class AnyImagePreferences(bpy.types.AddonPreferences):
|
||||
)
|
||||
max_cutout_mesh_resolution: bpy.props.IntProperty(
|
||||
name="Maximum Cutout Mesh Resolution",
|
||||
description="Maximum sampling resolution along the longest Cutout edge",
|
||||
description=(
|
||||
"Limit the longest image edge used to sample the Cutout mesh. "
|
||||
"Higher values allow finer outlines and denser meshes."
|
||||
),
|
||||
default=DEFAULT_MAX_CUTOUT_MESH_RESOLUTION,
|
||||
min=MIN_CUTOUT_MESH_RESOLUTION,
|
||||
max=MAX_CUTOUT_MESH_RESOLUTION,
|
||||
@@ -190,7 +191,7 @@ class AnyImagePreferences(bpy.types.AddonPreferences):
|
||||
)
|
||||
max_frame_resolution: bpy.props.IntProperty(
|
||||
name="Maximum Frame Resolution",
|
||||
description="Maximum output resolution along the longest Frame edge",
|
||||
description="Set the maximum length, in pixels, of the combined image's longest edge.",
|
||||
default=DEFAULT_MAX_FRAME_RESOLUTION,
|
||||
min=MIN_FRAME_RESOLUTION,
|
||||
max=MAX_FRAME_RESOLUTION,
|
||||
@@ -199,7 +200,7 @@ class AnyImagePreferences(bpy.types.AddonPreferences):
|
||||
)
|
||||
balloon_profile_algorithm: bpy.props.EnumProperty(
|
||||
name="Balloon Profile",
|
||||
description="Algorithm used to generate o_balloon for every Cutout shape",
|
||||
description="Control how the Cutout surface bulges outward from its outline.",
|
||||
items=BALLOON_PROFILE_ALGORITHM_ITEMS,
|
||||
default="POISSON",
|
||||
)
|
||||
@@ -210,14 +211,17 @@ class AnyImagePreferences(bpy.types.AddonPreferences):
|
||||
default=DEFAULT_BACKGROUND_MODEL_KEY,
|
||||
)
|
||||
geometry_model: bpy.props.EnumProperty(
|
||||
name="Generate Depth Map Model",
|
||||
description="Model used to generate depth maps",
|
||||
name="Depth & Normal Model",
|
||||
description="Choose the AI model used to generate depth and surface normals.",
|
||||
items=geometry_model_items,
|
||||
default=0,
|
||||
)
|
||||
geometry_resolution_level: bpy.props.IntProperty(
|
||||
name="MoGe Resolution Level",
|
||||
description="MoGe-2 inference resolution used for geometry and normal generation",
|
||||
description=(
|
||||
"Set the processing resolution for depth and normal generation. "
|
||||
"Higher levels can capture finer details but take more time and memory."
|
||||
),
|
||||
default=5,
|
||||
min=0,
|
||||
max=9,
|
||||
|
||||
@@ -140,6 +140,18 @@ def upscale_model_label(model_key):
|
||||
return model.label if model is not None else model_key
|
||||
|
||||
|
||||
def cutout_gesture_property():
|
||||
return bpy.props.EnumProperty(
|
||||
name="Gesture",
|
||||
description="Shape used to select the Cutout area",
|
||||
items=(
|
||||
("LASSO", "Lasso", "Draw a closed freehand area"),
|
||||
("POLYLINE", "Polyline", "Click points to define a closed area"),
|
||||
),
|
||||
default="LASSO",
|
||||
)
|
||||
|
||||
|
||||
def mask_gesture_property():
|
||||
return bpy.props.EnumProperty(
|
||||
name="Gesture",
|
||||
@@ -156,19 +168,31 @@ def mask_gesture_property():
|
||||
def mask_mode_property():
|
||||
return bpy.props.EnumProperty(
|
||||
name="Mode",
|
||||
description="How the gesture changes image Alpha",
|
||||
description="Choose how the selected or painted area changes image visibility.",
|
||||
items=(
|
||||
("SET", "Set", "Keep current Alpha inside the gesture", "SELECT_SET", 0),
|
||||
("ADD", "Add", "Extend Alpha with the gesture", "SELECT_EXTEND", 1),
|
||||
(
|
||||
"SET",
|
||||
"Set",
|
||||
"Keep visible content inside the selected or painted area and hide everything outside.",
|
||||
"SELECT_SET",
|
||||
0,
|
||||
),
|
||||
(
|
||||
"EXTEND",
|
||||
"Extend",
|
||||
"Reveal the selected or painted area while keeping other areas unchanged.",
|
||||
"SELECT_EXTEND",
|
||||
1,
|
||||
),
|
||||
(
|
||||
"SUBTRACT",
|
||||
"Subtract",
|
||||
"Remove Alpha inside the gesture",
|
||||
"Hide the selected or painted area while keeping other areas unchanged.",
|
||||
"SELECT_SUBTRACT",
|
||||
2,
|
||||
),
|
||||
),
|
||||
default="SUBTRACT",
|
||||
default="SET",
|
||||
)
|
||||
|
||||
|
||||
@@ -183,24 +207,12 @@ def mask_radius_property():
|
||||
)
|
||||
|
||||
|
||||
def cutout_inward_axis_property():
|
||||
return bpy.props.EnumProperty(
|
||||
name="Inward Axis",
|
||||
description="Local axis pointing inward from the image sheet",
|
||||
items=(
|
||||
("NEGATIVE_Z", "-Z", "Match the source Image Empty local axes"),
|
||||
("POSITIVE_X", "+X", "Use the original Cutout local axes"),
|
||||
),
|
||||
default="NEGATIVE_Z",
|
||||
)
|
||||
|
||||
|
||||
def cutout_generate_normal_property():
|
||||
return bpy.props.BoolProperty(
|
||||
name="Normal Map",
|
||||
description=(
|
||||
"Generate a normal map for Flat and Solid; "
|
||||
"Depth Solid and Depth Balloon always generate normal maps"
|
||||
"Generate a normal map to add surface detail under lighting for Flat and Solid shapes. "
|
||||
"Depth Solid and Depth Balloon always generate normal maps."
|
||||
),
|
||||
default=False,
|
||||
)
|
||||
@@ -209,7 +221,10 @@ def cutout_generate_normal_property():
|
||||
def cutout_alpha_threshold_property():
|
||||
return bpy.props.FloatProperty(
|
||||
name="Alpha Threshold",
|
||||
description="Minimum Alpha kept in the Cutout outline",
|
||||
description=(
|
||||
"Exclude areas below this opacity from the Cutout outline. "
|
||||
"Higher values exclude more semi-transparent edges."
|
||||
),
|
||||
default=0.9,
|
||||
min=0.0,
|
||||
max=1.0,
|
||||
@@ -230,7 +245,7 @@ def cutout_fine_outline_property():
|
||||
def cutout_mesh_detail_property():
|
||||
return bpy.props.EnumProperty(
|
||||
name="Mesh Detail",
|
||||
description="Mesh sampling density",
|
||||
description="Control mesh density. Higher levels use more vertices to represent finer shapes.",
|
||||
items=(
|
||||
("LOW", "Low", "32 source image pixels between neighboring mesh points"),
|
||||
("MEDIUM", "Medium", "16 source image pixels between neighboring mesh points"),
|
||||
@@ -242,18 +257,11 @@ def cutout_mesh_detail_property():
|
||||
|
||||
|
||||
class AnyImageSettings(bpy.types.PropertyGroup):
|
||||
cutout_gesture: cutout_gesture_property()
|
||||
cutout_alpha_threshold: cutout_alpha_threshold_property()
|
||||
cutout_mesh_detail: cutout_mesh_detail_property()
|
||||
cutout_fine_outline: cutout_fine_outline_property()
|
||||
cutout_generate_normal: cutout_generate_normal_property()
|
||||
cutout_inward_axis: cutout_inward_axis_property()
|
||||
mask_gesture: mask_gesture_property()
|
||||
mask_mode: mask_mode_property()
|
||||
mask_radius: mask_radius_property()
|
||||
video_frames: bpy.props.IntProperty(
|
||||
name="Extract Frames",
|
||||
description="Maximum number of consecutive frames extracted from the start of a video or frame sequence",
|
||||
default=16,
|
||||
min=1,
|
||||
soft_max=64,
|
||||
)
|
||||
|
||||
@@ -12,7 +12,7 @@ ENVIRONMENT = {
|
||||
"numpy==2.4.2",
|
||||
"scipy==1.15.3",
|
||||
"pillow==12.1.1",
|
||||
"av==17.0.0",
|
||||
|
||||
"trimesh==4.11.2",
|
||||
"huggingface-hub==1.4.1",
|
||||
],
|
||||
|
||||
@@ -64,16 +64,13 @@ def generate_moge_artifacts(
|
||||
parameters.get("max_input_size", DEFAULT_MAX_AI_INPUT_SIZE),
|
||||
)
|
||||
try:
|
||||
frames = moge.infer(
|
||||
frame = moge.infer(
|
||||
context.resource("model_manager"),
|
||||
parameters,
|
||||
[model_path],
|
||||
model_path,
|
||||
include_points=generate_depth,
|
||||
cancel_check=context.check_cancelled,
|
||||
)
|
||||
if len(frames) != 1:
|
||||
raise ValueError("Artifact generation requires one geometry frame")
|
||||
frame = frames[0]
|
||||
result = {}
|
||||
if generate_depth:
|
||||
context.progress(0.45, "Generating depth texture")
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
"""Generate optional AI files from one rectangular Cutout image input."""
|
||||
|
||||
from pathlib import Path
|
||||
from ..media.input import validate_input_path
|
||||
|
||||
from ..geometry.prediction_artifacts import generate_moge_artifacts
|
||||
|
||||
@@ -8,7 +8,7 @@ from ..geometry.prediction_artifacts import generate_moge_artifacts
|
||||
def run(context, parameters):
|
||||
context.progress(0.05, "Preparing Cutout Selection")
|
||||
context.directory.mkdir(parents=True, exist_ok=True)
|
||||
input_path = Path(parameters["input"])
|
||||
input_path = validate_input_path(parameters["input"])
|
||||
result = {}
|
||||
generate_depth = bool(parameters.get("generate_depth", False))
|
||||
normal_mode = str(parameters.get("normal_mode", "NONE"))
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
"""Generate model-space depth and normals for one complete image."""
|
||||
|
||||
from ..geometry.prediction_artifacts import generate_moge_artifacts
|
||||
from ..media import input_frames
|
||||
from ..media.input import validate_input_path
|
||||
|
||||
|
||||
def run(context, parameters):
|
||||
@@ -10,16 +10,14 @@ def run(context, parameters):
|
||||
raise ValueError(f"Unsupported plane type: {plane_type}")
|
||||
context.progress(0.05, "Preparing depth conversion")
|
||||
context.directory.mkdir(parents=True, exist_ok=True)
|
||||
with input_frames(context, parameters) as images:
|
||||
if len(images) != 1:
|
||||
raise ValueError("Depth conversion currently supports one image")
|
||||
artifacts = generate_moge_artifacts(
|
||||
context,
|
||||
parameters,
|
||||
images[0],
|
||||
generate_depth=True,
|
||||
normal_mode="OBJECT" if plane_type == "DEPTH" else "TANGENT",
|
||||
)
|
||||
input_path = validate_input_path(parameters["input"])
|
||||
artifacts = generate_moge_artifacts(
|
||||
context,
|
||||
parameters,
|
||||
input_path,
|
||||
generate_depth=True,
|
||||
normal_mode="OBJECT" if plane_type == "DEPTH" else "TANGENT",
|
||||
)
|
||||
context.check_cancelled()
|
||||
context.progress(1.0, "Depth conversion data is ready")
|
||||
return {key: path.name for key, path in artifacts.items()}
|
||||
|
||||
@@ -1,12 +1,10 @@
|
||||
"""Generate a radial depth texture from the current RGBA panorama."""
|
||||
|
||||
import json
|
||||
from pathlib import Path
|
||||
|
||||
from ..geometry.depth_texture import write_float_exr
|
||||
from ..geometry.panorama import camera_bases, merge_distances, sample_panorama, view_rays
|
||||
from ..media import VIDEO_SUFFIXES, input_frames
|
||||
from ..media.input import open_image
|
||||
from ..media.input import open_image, validate_input_path
|
||||
from ..media.resolution import DEFAULT_MAX_AI_INPUT_SIZE, normalized_max_input_size
|
||||
from ..models.moge import infer_image as infer
|
||||
|
||||
@@ -15,63 +13,58 @@ def run(context, parameters):
|
||||
import numpy as np
|
||||
|
||||
context.check_cancelled()
|
||||
input_path = Path(parameters["input"])
|
||||
if input_path.is_dir() or input_path.suffix.lower() in VIDEO_SUFFIXES:
|
||||
raise ValueError("Panorama conversion requires one static 2:1 image")
|
||||
input_path = validate_input_path(parameters["input"])
|
||||
maximum = normalized_max_input_size(parameters.get("max_input_size", DEFAULT_MAX_AI_INPUT_SIZE))
|
||||
context.directory.mkdir(parents=True, exist_ok=True)
|
||||
outputs = [context.directory / name for name in ("depth.exr", "depth.json")]
|
||||
try:
|
||||
with input_frames(context, parameters) as images:
|
||||
if len(images) != 1:
|
||||
with open_image(input_path) as opened:
|
||||
if opened.width != opened.height * 2:
|
||||
raise ValueError("Panorama conversion requires one static 2:1 image")
|
||||
with open_image(images[0]) as opened:
|
||||
if getattr(opened, "n_frames", 1) != 1 or opened.width != opened.height * 2:
|
||||
raise ValueError("Panorama conversion requires one static 2:1 image")
|
||||
source = opened.convert("RGBA")
|
||||
width, height = source.size
|
||||
size = min(maximum, max(512, width // 4))
|
||||
image = np.asarray(source, dtype=np.float32) / 255.0
|
||||
manager = context.resource("model_manager")
|
||||
bases = camera_bases()
|
||||
rays = view_rays(size)
|
||||
distances, masks = [], []
|
||||
session = None
|
||||
for index, basis in enumerate(bases):
|
||||
context.check_cancelled()
|
||||
context.progress(0.05 + 0.65 * index / len(bases), f"Generating panorama view {index + 1}/12")
|
||||
directions = rays @ basis
|
||||
alpha = sample_panorama(image[..., 3], directions, order=0)
|
||||
visible = alpha > 0
|
||||
if not visible.any():
|
||||
distances.append(np.ones((size, size), dtype=np.float32))
|
||||
masks.append(visible)
|
||||
continue
|
||||
if session is None:
|
||||
session = manager.get_moge(manager.directory(parameters["model"]), parameters["device"])
|
||||
color = sample_panorama(image[..., :3], directions) * alpha[..., None]
|
||||
prediction = infer(session, color, parameters["resolution_level"],
|
||||
fov_x=90.0, source_valid=visible)
|
||||
distances.append(np.linalg.norm(prediction["points"], axis=-1))
|
||||
masks.append((prediction["mask"] > 0.5) & visible)
|
||||
del prediction
|
||||
context.progress(0.75, "Merging panoramic distance")
|
||||
distance, valid = merge_distances(distances, masks, bases, width, height,
|
||||
cancel_check=context.check_cancelled)
|
||||
alpha = np.asarray(source.getchannel("A"),
|
||||
dtype=np.float32) / 255.0
|
||||
alpha *= valid
|
||||
valid_depth = (alpha >= 0.9) & np.isfinite(distance) & (distance > 0)
|
||||
if not valid_depth.any():
|
||||
raise ValueError("The panorama contains no valid depth pixels")
|
||||
if not np.isfinite(distance).all() or (distance <= 0).any():
|
||||
raise ValueError("Panorama fusion produced invalid distances")
|
||||
rgba = np.empty((height, width, 4), dtype=np.float32)
|
||||
rgba[..., :3], rgba[..., 3] = distance[..., None], alpha
|
||||
write_float_exr(rgba, outputs[0])
|
||||
outputs[1].write_text(json.dumps({
|
||||
"projection": "equirectangular", "image_size": [width, height],
|
||||
}, separators=(",", ":")), encoding="utf-8")
|
||||
source = opened.convert("RGBA")
|
||||
width, height = source.size
|
||||
size = min(maximum, max(512, width // 4))
|
||||
image = np.asarray(source, dtype=np.float32) / 255.0
|
||||
manager = context.resource("model_manager")
|
||||
bases = camera_bases()
|
||||
rays = view_rays(size)
|
||||
distances, masks = [], []
|
||||
session = None
|
||||
for index, basis in enumerate(bases):
|
||||
context.check_cancelled()
|
||||
context.progress(0.05 + 0.65 * index / len(bases), f"Generating panorama view {index + 1}/12")
|
||||
directions = rays @ basis
|
||||
alpha = sample_panorama(image[..., 3], directions, order=0)
|
||||
visible = alpha > 0
|
||||
if not visible.any():
|
||||
distances.append(np.ones((size, size), dtype=np.float32))
|
||||
masks.append(visible)
|
||||
continue
|
||||
if session is None:
|
||||
session = manager.get_moge(manager.directory(parameters["model"]), parameters["device"])
|
||||
color = sample_panorama(image[..., :3], directions) * alpha[..., None]
|
||||
prediction = infer(session, color, parameters["resolution_level"],
|
||||
fov_x=90.0, source_valid=visible)
|
||||
distances.append(np.linalg.norm(prediction["points"], axis=-1))
|
||||
masks.append((prediction["mask"] > 0.5) & visible)
|
||||
del prediction
|
||||
context.progress(0.75, "Merging panoramic distance")
|
||||
distance, valid = merge_distances(distances, masks, bases, width, height,
|
||||
cancel_check=context.check_cancelled)
|
||||
alpha = np.asarray(source.getchannel("A"),
|
||||
dtype=np.float32) / 255.0
|
||||
alpha *= valid
|
||||
valid_depth = (alpha >= 0.9) & np.isfinite(distance) & (distance > 0)
|
||||
if not valid_depth.any():
|
||||
raise ValueError("The panorama contains no valid depth pixels")
|
||||
if not np.isfinite(distance).all() or (distance <= 0).any():
|
||||
raise ValueError("Panorama fusion produced invalid distances")
|
||||
rgba = np.empty((height, width, 4), dtype=np.float32)
|
||||
rgba[..., :3], rgba[..., 3] = distance[..., None], alpha
|
||||
write_float_exr(rgba, outputs[0])
|
||||
outputs[1].write_text(json.dumps({
|
||||
"projection": "equirectangular", "image_size": [width, height],
|
||||
}, separators=(",", ":")), encoding="utf-8")
|
||||
context.check_cancelled()
|
||||
context.progress(1.0, "Panorama conversion data is ready")
|
||||
return {"depth": outputs[0].name, "depth_metadata": outputs[1].name}
|
||||
|
||||
@@ -1,9 +1,8 @@
|
||||
import time
|
||||
|
||||
from ..media import input_frames
|
||||
from ..media.input import open_image
|
||||
from ..media.input import open_image, validate_input_path
|
||||
from ..model_catalog import BACKGROUND_MODELS
|
||||
from ..models.background import infer_alpha, infer_foreground, infer_foreground_frames, inference_device
|
||||
from ..models.background import infer_alpha, infer_foreground, inference_device
|
||||
|
||||
|
||||
def run(context, parameters):
|
||||
@@ -24,62 +23,45 @@ def run(context, parameters):
|
||||
if output_kind not in {"foreground", "alpha"}:
|
||||
raise ValueError(f"Unknown background removal output: {output_kind}")
|
||||
|
||||
with input_frames(context, parameters) as images:
|
||||
context.progress(0.12, f"Loading {parameters['model']} for {device.upper()}")
|
||||
if output_kind == "alpha":
|
||||
import numpy as np
|
||||
input_path = validate_input_path(parameters["input"])
|
||||
context.progress(0.12, f"Loading {parameters['model']} for {device.upper()}")
|
||||
if output_kind == "alpha":
|
||||
import numpy as np
|
||||
|
||||
if len(images) != 1:
|
||||
raise ValueError("Alpha output requires one still image")
|
||||
with open_image(images[0]) as source:
|
||||
alpha, load_ms, inference_ms = infer_alpha(
|
||||
source, model_directory, device, model_cache=model_manager,
|
||||
cancel_check=context.check_cancelled,
|
||||
)
|
||||
context.check_cancelled()
|
||||
np.save(output_directory / "alpha.npy", alpha, allow_pickle=False)
|
||||
elif len(images) == 1:
|
||||
context.progress(0.35, "Removing background")
|
||||
foreground, load_ms, inference_ms = infer_foreground(
|
||||
images[0],
|
||||
model_directory,
|
||||
device,
|
||||
model_cache=model_manager,
|
||||
with open_image(input_path) as source:
|
||||
alpha, load_ms, inference_ms = infer_alpha(
|
||||
source, model_directory, device, model_cache=model_manager,
|
||||
cancel_check=context.check_cancelled,
|
||||
preserve_input_alpha=True,
|
||||
)
|
||||
context.check_cancelled()
|
||||
context.progress(0.92, "Writing transparent PNG")
|
||||
output_path = output_directory / "image.0001.png"
|
||||
foreground.convert("RGBA").save(
|
||||
output_path,
|
||||
format="PNG",
|
||||
compress_level=1,
|
||||
)
|
||||
if not output_path.is_file():
|
||||
raise RuntimeError("Background removal did not write a foreground image")
|
||||
else:
|
||||
load_ms, inference_ms = infer_foreground_frames(
|
||||
images,
|
||||
model_directory,
|
||||
device,
|
||||
output_directory,
|
||||
context.progress,
|
||||
0.12,
|
||||
0.7,
|
||||
model_cache=model_manager,
|
||||
cancel_check=context.check_cancelled,
|
||||
output_prefix="image",
|
||||
preserve_input_alpha=True,
|
||||
)
|
||||
context.check_cancelled()
|
||||
np.save(output_directory / "alpha.npy", alpha, allow_pickle=False)
|
||||
else:
|
||||
context.progress(0.35, "Removing background")
|
||||
foreground, load_ms, inference_ms = infer_foreground(
|
||||
input_path,
|
||||
model_directory,
|
||||
device,
|
||||
model_cache=model_manager,
|
||||
cancel_check=context.check_cancelled,
|
||||
preserve_input_alpha=True,
|
||||
)
|
||||
context.check_cancelled()
|
||||
context.progress(0.92, "Writing transparent PNG")
|
||||
output_path = output_directory / "foreground.png"
|
||||
foreground.convert("RGBA").save(
|
||||
output_path,
|
||||
format="PNG",
|
||||
compress_level=1,
|
||||
)
|
||||
if not output_path.is_file():
|
||||
raise RuntimeError("Background removal did not write a foreground image")
|
||||
|
||||
context.check_cancelled()
|
||||
total_ms = (time.perf_counter() - total_started) * 1000.0
|
||||
frame_text = "" if len(images) == 1 else f" from {len(images)} frame(s)"
|
||||
message = (
|
||||
f"Background removed{frame_text} in {inference_ms:.0f} ms "
|
||||
f"Background removed in {inference_ms:.0f} ms "
|
||||
f"(model load {load_ms:.0f} ms, total {total_ms:.0f} ms)"
|
||||
)
|
||||
context.log(message)
|
||||
context.progress(1.0, message)
|
||||
return {"alpha": "alpha.npy"} if output_kind == "alpha" else {"image": "."}
|
||||
return {"alpha": "alpha.npy"} if output_kind == "alpha" else {"foreground": "foreground.png"}
|
||||
|
||||
@@ -1,13 +1,8 @@
|
||||
from ..media import DEFAULT_MAX_AI_INPUT_SIZE, input_frames
|
||||
from ..media import DEFAULT_MAX_AI_INPUT_SIZE
|
||||
from ..media.input import validate_input_path
|
||||
from ..models.upscale import infer_one
|
||||
|
||||
|
||||
def frame_progress_message(message, index, total):
|
||||
if total > 1:
|
||||
return f"{message} ({index + 1}/{total})"
|
||||
return message
|
||||
|
||||
|
||||
def run(context, parameters):
|
||||
context.log(
|
||||
f"Upscale model: {parameters['model']}; "
|
||||
@@ -15,27 +10,17 @@ def run(context, parameters):
|
||||
f"{parameters.get('max_input_size', DEFAULT_MAX_AI_INPUT_SIZE)} px"
|
||||
)
|
||||
context.progress(0.02, "Preparing upscale input")
|
||||
context.directory.mkdir(parents=True, exist_ok=True)
|
||||
with input_frames(context, parameters) as images:
|
||||
total = len(images)
|
||||
for index, image in enumerate(images):
|
||||
context.check_cancelled()
|
||||
context.progress(
|
||||
0.08 + index / total * 0.84,
|
||||
frame_progress_message("Upscaling image", index, total),
|
||||
)
|
||||
prediction = infer_one(
|
||||
context.resource("model_manager"),
|
||||
parameters,
|
||||
image,
|
||||
cancel_check=context.check_cancelled,
|
||||
release_memory=index == total - 1,
|
||||
)
|
||||
prediction.save(
|
||||
context.directory / f"upscale.{index + 1:04d}.png",
|
||||
format="PNG",
|
||||
compress_level=1,
|
||||
)
|
||||
context.check_cancelled()
|
||||
input_path = validate_input_path(parameters["input"])
|
||||
context.directory.mkdir(parents=True, exist_ok=True)
|
||||
context.progress(0.08, "Upscaling image")
|
||||
prediction = infer_one(
|
||||
context.resource("model_manager"),
|
||||
parameters,
|
||||
input_path,
|
||||
cancel_check=context.check_cancelled,
|
||||
)
|
||||
context.check_cancelled()
|
||||
prediction.save(context.directory / "upscale.png", format="PNG", compress_level=1)
|
||||
context.progress(1.0, "Image upscale complete")
|
||||
return {"upscale": "."}
|
||||
return {"upscale": "upscale.png"}
|
||||
|
||||
@@ -1,14 +1,7 @@
|
||||
from .input import (
|
||||
IMAGE_SUFFIXES,
|
||||
VIDEO_SUFFIXES,
|
||||
TaskCancelled,
|
||||
analysis_frames,
|
||||
check_cancelled,
|
||||
collect_images,
|
||||
extract_video_frames,
|
||||
first_frames,
|
||||
input_frames,
|
||||
prepare_input,
|
||||
)
|
||||
|
||||
|
||||
@@ -23,15 +16,8 @@ from .resolution import (
|
||||
|
||||
__all__ = (
|
||||
"IMAGE_SUFFIXES",
|
||||
"VIDEO_SUFFIXES",
|
||||
"TaskCancelled",
|
||||
"analysis_frames",
|
||||
"check_cancelled",
|
||||
"collect_images",
|
||||
"extract_video_frames",
|
||||
"first_frames",
|
||||
"input_frames",
|
||||
"prepare_input",
|
||||
"DEFAULT_MAX_AI_INPUT_SIZE",
|
||||
"MAX_MAX_AI_INPUT_SIZE",
|
||||
"MIN_MAX_AI_INPUT_SIZE",
|
||||
|
||||
@@ -1,46 +1,21 @@
|
||||
import shutil
|
||||
import tempfile
|
||||
from contextlib import contextmanager
|
||||
from pathlib import Path
|
||||
|
||||
|
||||
IMAGE_SUFFIXES = {".jpg", ".jpeg", ".png", ".webp"}
|
||||
VIDEO_SUFFIXES = {
|
||||
".mp4",
|
||||
".mov",
|
||||
".avi",
|
||||
".mkv",
|
||||
".webm",
|
||||
".m4v",
|
||||
".mts",
|
||||
".mpg",
|
||||
".mpeg",
|
||||
}
|
||||
|
||||
|
||||
def validate_input_path(value, *, images_only=False):
|
||||
"""Validate supported media without importing image or video libraries."""
|
||||
def validate_input_path(value):
|
||||
"""Validate one supported image file without importing image libraries."""
|
||||
path = Path(value)
|
||||
if path.is_dir() and not images_only:
|
||||
if not collect_images(path):
|
||||
supported = ", ".join(sorted(IMAGE_SUFFIXES))
|
||||
raise RuntimeError(
|
||||
f"No supported image frames found in: {path}. Supported images: {supported}."
|
||||
)
|
||||
return path
|
||||
if not path.is_file():
|
||||
kind = "video" if path.suffix.lower() in VIDEO_SUFFIXES else "image"
|
||||
raise RuntimeError(f"Source {kind} file does not exist: {path}")
|
||||
raise RuntimeError(f"Source image file does not exist: {path}")
|
||||
suffix = path.suffix.lower()
|
||||
supported = IMAGE_SUFFIXES if images_only else IMAGE_SUFFIXES | VIDEO_SUFFIXES
|
||||
if suffix not in supported:
|
||||
message = (
|
||||
if suffix not in IMAGE_SUFFIXES:
|
||||
raise RuntimeError(
|
||||
f"Unsupported input format: {suffix or '(no extension)'}. "
|
||||
f"Supported images: {', '.join(sorted(IMAGE_SUFFIXES))}."
|
||||
)
|
||||
if not images_only:
|
||||
message += f" Supported videos: {', '.join(sorted(VIDEO_SUFFIXES))}."
|
||||
raise RuntimeError(message)
|
||||
return path
|
||||
|
||||
|
||||
@@ -54,6 +29,8 @@ def open_image(path):
|
||||
try:
|
||||
opened = Image.open(path)
|
||||
opened.load()
|
||||
if getattr(opened, "n_frames", 1) != 1:
|
||||
raise RuntimeError("Only still images are supported")
|
||||
except (OSError, ValueError, SyntaxError) as error:
|
||||
raise RuntimeError(f"Unable to read image: {path}") from error
|
||||
yield opened
|
||||
@@ -69,134 +46,3 @@ class TaskCancelled(RuntimeError):
|
||||
def check_cancelled(cancel_check):
|
||||
if cancel_check is not None and cancel_check():
|
||||
raise TaskCancelled("Task cancelled")
|
||||
|
||||
|
||||
def collect_images(directory):
|
||||
directory = Path(directory)
|
||||
return sorted(
|
||||
path
|
||||
for path in directory.iterdir()
|
||||
if path.is_file() and path.suffix.lower() in IMAGE_SUFFIXES
|
||||
)
|
||||
|
||||
|
||||
def first_frames(images, maximum_frames, start_frame=1):
|
||||
"""Return up to maximum_frames image paths in order from start_frame."""
|
||||
if maximum_frames <= 0:
|
||||
return images
|
||||
start_index = max(start_frame - 1, 0)
|
||||
return images[start_index:start_index + maximum_frames]
|
||||
|
||||
|
||||
def extract_video_frames(
|
||||
video_path,
|
||||
output_directory,
|
||||
maximum_frames,
|
||||
av,
|
||||
start_frame=1,
|
||||
cancel_check=None,
|
||||
):
|
||||
output_directory = Path(output_directory)
|
||||
output_directory.mkdir(parents=True, exist_ok=True)
|
||||
try:
|
||||
container = av.open(str(video_path))
|
||||
except Exception as error:
|
||||
raise RuntimeError(f"Unable to read video: {video_path}") from error
|
||||
image_paths = []
|
||||
skipped = 0
|
||||
try:
|
||||
try:
|
||||
stream = container.streams.video[0]
|
||||
frames = iter(container.decode(stream))
|
||||
except Exception as error:
|
||||
raise RuntimeError(f"Unable to read video: {video_path}") from error
|
||||
while True:
|
||||
try:
|
||||
frame = next(frames)
|
||||
except StopIteration:
|
||||
break
|
||||
except Exception as error:
|
||||
raise RuntimeError(f"Unable to read video: {video_path}") from error
|
||||
check_cancelled(cancel_check)
|
||||
if skipped < start_frame - 1:
|
||||
skipped += 1
|
||||
continue
|
||||
image_path = output_directory / f"frame.{len(image_paths) + 1:04d}.jpg"
|
||||
try:
|
||||
decoded = frame.to_image()
|
||||
except Exception as error:
|
||||
raise RuntimeError(f"Unable to read video: {video_path}") from error
|
||||
decoded.save(image_path, format="JPEG")
|
||||
image_paths.append(image_path)
|
||||
if len(image_paths) >= maximum_frames:
|
||||
break
|
||||
finally:
|
||||
container.close()
|
||||
|
||||
if not image_paths:
|
||||
raise RuntimeError(f"Unable to read video: {video_path}")
|
||||
return image_paths
|
||||
|
||||
|
||||
def prepare_input(
|
||||
input_path,
|
||||
video_frames,
|
||||
temporary_root,
|
||||
start_frame=1,
|
||||
cancel_check=None,
|
||||
):
|
||||
check_cancelled(cancel_check)
|
||||
input_path = validate_input_path(input_path)
|
||||
if input_path.is_dir():
|
||||
images = collect_images(input_path)
|
||||
return first_frames(images, video_frames, start_frame), None
|
||||
suffix = input_path.suffix.lower()
|
||||
if suffix in IMAGE_SUFFIXES:
|
||||
return [input_path], None
|
||||
|
||||
import av
|
||||
|
||||
temporary_directory = Path(
|
||||
tempfile.mkdtemp(prefix="anyimage-video-", dir=temporary_root)
|
||||
)
|
||||
try:
|
||||
images = extract_video_frames(
|
||||
input_path,
|
||||
temporary_directory,
|
||||
video_frames,
|
||||
av,
|
||||
start_frame=start_frame,
|
||||
cancel_check=cancel_check,
|
||||
)
|
||||
except Exception:
|
||||
shutil.rmtree(temporary_directory, ignore_errors=True)
|
||||
raise
|
||||
return images, temporary_directory
|
||||
|
||||
|
||||
@contextmanager
|
||||
def input_frames(context, parameters):
|
||||
"""Yield prepared input frames and clean up extracted video frames."""
|
||||
images, temporary_directory = prepare_input(
|
||||
parameters["input"],
|
||||
parameters.get("video_frames", 16),
|
||||
context.directory,
|
||||
start_frame=parameters.get("start_frame", 1),
|
||||
cancel_check=context.check_cancelled,
|
||||
)
|
||||
try:
|
||||
yield images
|
||||
finally:
|
||||
if temporary_directory is not None:
|
||||
shutil.rmtree(temporary_directory, ignore_errors=True)
|
||||
|
||||
|
||||
@contextmanager
|
||||
def analysis_frames(context, parameters, fallback_images):
|
||||
"""Yield model-analysis frames, falling back to the color input."""
|
||||
analysis_input = parameters.get("analysis_input")
|
||||
if not analysis_input or analysis_input == parameters["input"]:
|
||||
yield fallback_images
|
||||
return
|
||||
with input_frames(context, dict(parameters, input=analysis_input)) as images:
|
||||
yield images
|
||||
|
||||
@@ -22,7 +22,7 @@ BIREFNET_LITE = ModelSpec(
|
||||
directory_name="birefnet-lite",
|
||||
repository="ZhengPeng7/BiRefNet_lite",
|
||||
license_name="MIT",
|
||||
description="Compact 1024px background removal with continuous alpha",
|
||||
description="Remove backgrounds at 1024 px processing resolution while preserving soft, semi-transparent edges.",
|
||||
download_patterns=("model.onnx", "LICENSE.txt"),
|
||||
ready_patterns=("model.onnx", "LICENSE.txt"),
|
||||
r2_directory="birefnet-lite",
|
||||
@@ -40,7 +40,7 @@ BEN2_MODEL = ModelSpec(
|
||||
repository="onnx-community/BEN2-ONNX",
|
||||
license_name="Apache 2.0",
|
||||
huggingface_repository="onnx-community/BEN2-ONNX",
|
||||
description="General-purpose FP16 ONNX background removal model",
|
||||
description="Remove backgrounds from general images while preserving foreground transparency.",
|
||||
download_patterns=("onnx/model_fp16.onnx",),
|
||||
ready_patterns=("onnx/model_fp16.onnx",),
|
||||
r2_directory="BEN2-ONNX",
|
||||
@@ -58,7 +58,7 @@ REALESRGAN_GENERAL_WDN_X4V3 = ModelSpec(
|
||||
directory_name="realesr-general-wdn-x4v3",
|
||||
repository="xinntao/Real-ESRGAN",
|
||||
license_name="BSD 3-Clause",
|
||||
description="Fast and conservative 2x output from a 4x ONNX model",
|
||||
description="Enlarge images to twice their width and height with fast, restrained detail enhancement.",
|
||||
download_patterns=("realesr-general-wdn-x4v3.onnx",),
|
||||
ready_patterns=("realesr-general-wdn-x4v3.onnx",),
|
||||
r2_directory="realesr-general-wdn-x4v3",
|
||||
@@ -77,7 +77,7 @@ REALESRGAN_X4PLUS = ModelSpec(
|
||||
repository="jonathanst29/tinier-upscale-models",
|
||||
license_name="BSD 3-Clause",
|
||||
huggingface_repository="jonathanst29/tinier-upscale-models",
|
||||
description="Detailed 2x output from a general-purpose 4x ONNX model",
|
||||
description="Enlarge images to twice their width and height with stronger detail enhancement.",
|
||||
download_patterns=("realesrgan-x4plus.onnx",),
|
||||
ready_patterns=("realesrgan-x4plus.onnx",),
|
||||
r2_directory="realesrgan-x4plus",
|
||||
@@ -97,7 +97,10 @@ HAT_GAN_X4_SHARPER = ModelSpec(
|
||||
directory_name="hat-gan-x4-sharper",
|
||||
repository="XPixelGroup/HAT",
|
||||
license_name="Apache 2.0",
|
||||
description="Detailed texture enhancement with 2x output; higher processing cost",
|
||||
description=(
|
||||
"Enlarge images to twice their width and height with detailed texture enhancement. "
|
||||
"Requires more processing time."
|
||||
),
|
||||
download_patterns=("model.onnx", "LICENSE.txt"),
|
||||
ready_patterns=("model.onnx", "LICENSE.txt"),
|
||||
r2_directory="hat-gan-x4-sharper",
|
||||
@@ -129,7 +132,7 @@ MOGE2_MODELS = {
|
||||
repository="Ruicheng/moge-2-vits-normal-onnx",
|
||||
license_name="MIT",
|
||||
huggingface_repository="Ruicheng/moge-2-vits-normal-onnx",
|
||||
description="Fastest MoGe-2 model using ONNX Runtime",
|
||||
description="Generate depth and surface normals with the fastest MoGe-2 option.",
|
||||
download_patterns=("model.onnx",),
|
||||
ready_patterns=("model.onnx",),
|
||||
r2_directory="moge-2-vits-normal-onnx",
|
||||
@@ -146,7 +149,7 @@ MOGE2_MODELS = {
|
||||
repository="Ruicheng/moge-2-vitb-normal-onnx",
|
||||
license_name="MIT",
|
||||
huggingface_repository="Ruicheng/moge-2-vitb-normal-onnx",
|
||||
description="Balanced MoGe-2 model using ONNX Runtime",
|
||||
description="Generate depth and surface normals with a balance of processing speed and detail.",
|
||||
download_patterns=("model.onnx",),
|
||||
ready_patterns=("model.onnx",),
|
||||
r2_directory="moge-2-vitb-normal-onnx",
|
||||
@@ -163,7 +166,7 @@ MOGE3_VITL = ModelSpec(
|
||||
directory_name="moge-3-vitl-onnx",
|
||||
repository="Ruicheng/moge-3-vitl",
|
||||
license_name="MIT",
|
||||
description="MoGe-3 geometry with three refinement steps using ONNX Runtime",
|
||||
description="Generate depth and surface normals with MoGe-3, refining the prediction in three steps.",
|
||||
download_patterns=("backbone.onnx", "refiner.onnx"),
|
||||
ready_patterns=("backbone.onnx", "refiner.onnx"),
|
||||
r2_directory="moge-3-vitl-onnx",
|
||||
@@ -180,7 +183,10 @@ BIREFNET_HR_MATTING = ModelSpec(
|
||||
directory_name="birefnet-hr-matting",
|
||||
repository="ZhengPeng7/BiRefNet_HR-matting",
|
||||
license_name="MIT",
|
||||
description="2048px matting with continuous alpha; CPU only, high memory and processing cost",
|
||||
description=(
|
||||
"Remove backgrounds at 2048 px processing resolution while preserving soft, semi-transparent edges. "
|
||||
"Runs on CPU and requires more time and memory."
|
||||
),
|
||||
download_patterns=("model.onnx", "LICENSE.txt"),
|
||||
ready_patterns=("model.onnx", "LICENSE.txt"),
|
||||
r2_directory="birefnet-hr-matting",
|
||||
|
||||
@@ -75,7 +75,7 @@ def infer_foreground(
|
||||
from PIL import Image, ImageChops
|
||||
|
||||
check_cancelled(cancel_check)
|
||||
input_path = validate_input_path(Path(input_path), images_only=True)
|
||||
input_path = validate_input_path(Path(input_path))
|
||||
with open_image(input_path) as opened_image:
|
||||
source = opened_image.convert("RGBA")
|
||||
alpha, load_ms, inference_ms = infer_alpha(
|
||||
@@ -91,47 +91,3 @@ def infer_foreground(
|
||||
foreground.paste(source.convert("RGB"), (0, 0), source.getchannel("A"))
|
||||
foreground.putalpha(mask)
|
||||
return foreground, load_ms, inference_ms
|
||||
|
||||
|
||||
def infer_foreground_frames(
|
||||
images,
|
||||
model_dir,
|
||||
device,
|
||||
output_directory,
|
||||
progress_callback,
|
||||
start_progress,
|
||||
progress_span,
|
||||
model_cache=None,
|
||||
cancel_check=None,
|
||||
output_prefix="foreground",
|
||||
preserve_input_alpha=False,
|
||||
):
|
||||
"""Remove each frame's background and write numbered transparent PNG files."""
|
||||
total_load_ms = 0.0
|
||||
total_inference_ms = 0.0
|
||||
for index, image_path in enumerate(images):
|
||||
check_cancelled(cancel_check)
|
||||
progress_callback(
|
||||
start_progress + progress_span * index / len(images),
|
||||
f"Removing background ({index + 1}/{len(images)})",
|
||||
)
|
||||
foreground, load_ms, inference_ms = infer_foreground(
|
||||
image_path,
|
||||
model_dir,
|
||||
device,
|
||||
model_cache=model_cache,
|
||||
cancel_check=cancel_check,
|
||||
preserve_input_alpha=preserve_input_alpha,
|
||||
release_memory=index == len(images) - 1,
|
||||
)
|
||||
output_path = output_directory / f"{output_prefix}.{index + 1:04d}.png"
|
||||
foreground.convert("RGBA").save(
|
||||
output_path,
|
||||
format="PNG",
|
||||
compress_level=1,
|
||||
)
|
||||
if not output_path.is_file():
|
||||
raise RuntimeError(f"Background removal did not write: {output_path}")
|
||||
total_load_ms += load_ms
|
||||
total_inference_ms += inference_ms
|
||||
return total_load_ms, total_inference_ms
|
||||
|
||||
@@ -38,30 +38,16 @@ def _geometry_frame(prediction, include_points):
|
||||
def infer(
|
||||
model_manager,
|
||||
parameters,
|
||||
images,
|
||||
image_path,
|
||||
*,
|
||||
include_points,
|
||||
progress_callback=None,
|
||||
cancel_check=None,
|
||||
):
|
||||
frames = []
|
||||
for index, image_path in enumerate(images):
|
||||
if cancel_check is not None:
|
||||
cancel_check()
|
||||
if progress_callback is not None:
|
||||
progress_callback(index, len(images))
|
||||
frames.append(
|
||||
_geometry_frame(
|
||||
infer_one(
|
||||
model_manager,
|
||||
parameters,
|
||||
image_path,
|
||||
release_memory=index == len(images) - 1,
|
||||
),
|
||||
include_points,
|
||||
)
|
||||
)
|
||||
return frames
|
||||
"""Infer one image and return its geometry fields."""
|
||||
if cancel_check is not None:
|
||||
cancel_check()
|
||||
prediction = infer_one(model_manager, parameters, image_path)
|
||||
return _geometry_frame(prediction, include_points)
|
||||
|
||||
|
||||
def infer_one(
|
||||
|
||||
@@ -9,14 +9,6 @@ from tests.support.blender import mock_bpy_imports
|
||||
|
||||
def load_common_modules():
|
||||
fake_bpy = ModuleType("bpy")
|
||||
fake_bpy.props = SimpleNamespace(
|
||||
BoolProperty=lambda **options: options,
|
||||
EnumProperty=lambda **options: options,
|
||||
FloatProperty=lambda **options: options,
|
||||
IntProperty=lambda **options: options,
|
||||
StringProperty=lambda **options: options,
|
||||
)
|
||||
fake_bpy.types = SimpleNamespace(Menu=object, Operator=object)
|
||||
fake_bpy.path = SimpleNamespace(abspath=lambda path: path)
|
||||
fake_bpy.context = SimpleNamespace(scene=None)
|
||||
|
||||
@@ -58,9 +50,7 @@ def load_common_modules():
|
||||
name: ModuleType(name)
|
||||
for name in (
|
||||
"anyimage",
|
||||
"anyimage.operators",
|
||||
"anyimage.common",
|
||||
"anyimage.operators.convert_to_plane",
|
||||
)
|
||||
}
|
||||
for package in packages.values():
|
||||
@@ -80,4 +70,4 @@ def load_common_modules():
|
||||
specification.loader.exec_module(module)
|
||||
loaded.append(module)
|
||||
|
||||
return SimpleNamespace(image=loaded[0], node=loaded[1], material=loaded[2], object=loaded[3])
|
||||
return SimpleNamespace(image=loaded[0], material=loaded[2], object=loaded[3])
|
||||
|
||||
@@ -35,9 +35,9 @@ def test_owned_color_images_preserve_source_and_use_premul(floating):
|
||||
|
||||
def test_job_loader_preserves_edit_alpha_policy(tmp_path):
|
||||
rgba = np.array([[[51, 102, 204, 64]]], dtype=np.uint8)
|
||||
Image.fromarray(rgba).save(tmp_path / "image.0001.png")
|
||||
Image.fromarray(rgba).save(tmp_path / "foreground.png")
|
||||
original = bpy.data.images.new("Job source", width=1, height=1, alpha=True)
|
||||
result = images.load_image_edit_result(original, tmp_path)
|
||||
result = images.load_image_edit_result(original, tmp_path / "foreground.png")
|
||||
try:
|
||||
assert result.alpha_mode == "STRAIGHT"
|
||||
assert result.packed_file is not None
|
||||
@@ -47,27 +47,6 @@ def test_job_loader_preserves_edit_alpha_policy(tmp_path):
|
||||
bpy.data.images.remove(original)
|
||||
|
||||
|
||||
def test_failed_empty_replacement_restores_blending_and_timing():
|
||||
original = SimpleNamespace(name="Original", use_fake_user=False, users=1)
|
||||
timing = SimpleNamespace(frame_start=3, frame_offset=2, frame_duration=7, use_cyclic=True)
|
||||
source = SimpleNamespace(data=original, use_empty_image_alpha=False, image_user=timing)
|
||||
result = SimpleNamespace(source="SEQUENCE", users=1)
|
||||
|
||||
def fail_configuration(*args, **kwargs):
|
||||
timing.frame_start = 99
|
||||
raise RuntimeError("Failed configuration")
|
||||
|
||||
with patch.object(images, "has_other_image_empty", return_value=True), patch.object(images, "video_total_frames", return_value=3), patch.object(
|
||||
images, "configure_image_user", side_effect=fail_configuration
|
||||
):
|
||||
with pytest.raises(RuntimeError, match="Failed configuration"):
|
||||
images.replace_empty_image(source, result)
|
||||
assert source.data is original
|
||||
assert not source.use_empty_image_alpha
|
||||
assert not original.use_fake_user
|
||||
assert images.image_user_settings(source) == (3, 2, 7, True)
|
||||
|
||||
|
||||
def test_business_pixels_ignore_premul_read_interpretation():
|
||||
original = bpy.data.images.new("Premul read source", width=1, height=1, alpha=True)
|
||||
original.alpha_mode = "PREMUL"
|
||||
@@ -188,10 +167,11 @@ def test_isolated_decode_failure_releases_copy():
|
||||
assert source.alpha_mode == "PREMUL"
|
||||
|
||||
|
||||
def test_mask_pack_reopen_add_and_repeat_frame_preserve_hidden_color(tmp_path):
|
||||
def test_mask_pack_reopen_extend_and_repeat_frame_preserve_hidden_color(tmp_path):
|
||||
from anyimage.common.selection import SelectionMask
|
||||
from anyimage.operators.mask_tool import apply_alpha_mask
|
||||
from anyimage.operators import frame_tool as frame
|
||||
from anyimage.operators.frame_tool.projection import frame_source_projection
|
||||
from anyimage.operators.frame_tool.compositing import composite_frame_pixels
|
||||
|
||||
source = bpy.data.images.new("Roundtrip source", width=2, height=2, alpha=True)
|
||||
rgba = np.tile(np.array([0.2, 0.4, 0.8, 1], dtype=np.float32), (2, 2, 1))
|
||||
@@ -208,15 +188,15 @@ def test_mask_pack_reopen_add_and_repeat_frame_preserve_hidden_color(tmp_path):
|
||||
restored = images.image_rgba(result)
|
||||
np.testing.assert_allclose(restored[..., :3], rgba[..., :3], atol=1 / 255)
|
||||
np.testing.assert_array_equal(restored[..., 3], 0)
|
||||
added = images.create_image_edit_result(result, apply_alpha_mask(restored, full, "ADD"), (2, 2))
|
||||
added = images.create_image_edit_result(result, apply_alpha_mask(restored, full, "EXTEND"), (2, 2))
|
||||
bpy.data.images.remove(result)
|
||||
result = added
|
||||
for _ in range(2):
|
||||
source = frame.frame_source_projection(np.eye(4), np.eye(4), np.eye(4), (-1, 1, -1, 1), (2, 2), perspective=False)
|
||||
source = frame_source_projection(np.eye(4), np.eye(4), np.eye(4), (-1, 1, -1, 1), (2, 2), perspective=False)
|
||||
rgba_read = images.image_rgba(result)
|
||||
source.update(rgba=rgba_read, pixels=images.premultiplied_rgba(images.image_pixels(result), (2, 2)),
|
||||
image_size=(2, 2), active=True, name=result.name)
|
||||
pixels = frame.composite_frame_pixels((source,), ((0, 2), (2, 2), (2, 0), (0, 0)), (2, 2))
|
||||
pixels = composite_frame_pixels((source,), ((0, 2), (2, 2), (2, 0), (0, 0)), (2, 2))
|
||||
next_result = images.create_image_edit_result(result, pixels, (2, 2))
|
||||
bpy.data.images.remove(result)
|
||||
result = next_result
|
||||
@@ -225,3 +205,19 @@ def test_mask_pack_reopen_add_and_repeat_frame_preserve_hidden_color(tmp_path):
|
||||
np.testing.assert_allclose(images.image_rgba(result), rgba, atol=1 / 255)
|
||||
finally:
|
||||
bpy.data.images.remove(result)
|
||||
|
||||
|
||||
def test_job_loader_reads_only_the_returned_file(tmp_path):
|
||||
Image.new("RGBA", (2, 2), (255, 0, 0, 255)).save(tmp_path / "foreground.png")
|
||||
Image.new("RGBA", (4, 4), (0, 0, 255, 255)).save(tmp_path / "upscale.png")
|
||||
original = bpy.data.images.new("Source", width=1, height=1, alpha=True)
|
||||
result = images.load_image_edit_result(original, tmp_path / "upscale.png")
|
||||
try:
|
||||
assert tuple(result.size) == (4, 4)
|
||||
assert result.source == "FILE"
|
||||
assert result.packed_file is not None
|
||||
with pytest.raises(RuntimeError, match="Color result image does not exist"):
|
||||
images.load_image_edit_result(original, tmp_path / "missing.png")
|
||||
finally:
|
||||
bpy.data.images.remove(result)
|
||||
bpy.data.images.remove(original)
|
||||
|
||||
@@ -171,8 +171,10 @@ def test_hdr_analysis_preview_is_separate_from_full_precision_color(source):
|
||||
assert not preview.exists()
|
||||
|
||||
|
||||
@pytest.mark.parametrize("alpha_mode", ["STRAIGHT", "PREMUL", "CHANNEL_PACKED", "NONE"])
|
||||
@pytest.mark.parametrize("source_kind", ["generated", "file", "dirty", "packed"])
|
||||
@pytest.mark.parametrize("alpha_mode,source_kind", [
|
||||
("STRAIGHT", "generated"), ("PREMUL", "file"),
|
||||
("CHANNEL_PACKED", "dirty"), ("NONE", "packed"),
|
||||
])
|
||||
def test_float_color_conversion_preserves_rendered_color_and_opacity(source, tmp_path, alpha_mode, source_kind):
|
||||
from anyimage.common.hdr_image import save_hdr_result
|
||||
|
||||
|
||||
25
tests/anyimage/common/test_depth_metadata.py
Normal file
25
tests/anyimage/common/test_depth_metadata.py
Normal file
@@ -0,0 +1,25 @@
|
||||
import json
|
||||
|
||||
import numpy as np
|
||||
|
||||
from anyimage.common.depth import load_depth_metadata
|
||||
|
||||
|
||||
def test_depth_metadata_loader_uses_one_schema_for_plane_and_cutout(tmp_path):
|
||||
path = tmp_path / "depth.json"
|
||||
path.write_text(
|
||||
json.dumps(
|
||||
{
|
||||
"image_size": [3, 2],
|
||||
"intrinsics": np.eye(3).tolist(),
|
||||
"reference_depth": 1.0,
|
||||
}
|
||||
),
|
||||
encoding="utf-8",
|
||||
)
|
||||
|
||||
metadata = load_depth_metadata(path)
|
||||
|
||||
assert metadata["image_size"] == (3, 2)
|
||||
assert metadata["intrinsics"] == ((1.0, 0.0, 0.0), (0.0, 1.0, 0.0), (0.0, 0.0, 1.0))
|
||||
assert metadata["reference_depth"] == 1.0
|
||||
@@ -33,9 +33,12 @@ def edit_result(source):
|
||||
)
|
||||
|
||||
|
||||
@pytest.mark.parametrize("alpha_mode", ["STRAIGHT", "PREMUL", "CHANNEL_PACKED", "NONE"])
|
||||
@pytest.mark.parametrize("shared", [False, True])
|
||||
@pytest.mark.parametrize("entry", ["pixels", "image.", "upscale."])
|
||||
@pytest.mark.parametrize("alpha_mode,shared,entry", [
|
||||
("STRAIGHT", False, "pixels"),
|
||||
("PREMUL", True, "foreground.png"),
|
||||
("CHANNEL_PACKED", False, "upscale.png"),
|
||||
("NONE", True, "pixels"),
|
||||
])
|
||||
def test_image_edit_replacement_preserves_alpha_mode(source, tmp_path, alpha_mode, shared, entry):
|
||||
original = source.data
|
||||
original.alpha_mode = alpha_mode
|
||||
@@ -44,8 +47,8 @@ def test_image_edit_replacement_preserves_alpha_mode(source, tmp_path, alpha_mod
|
||||
if entry == "pixels":
|
||||
result = edit_result(source)
|
||||
else:
|
||||
Image.new("RGBA", (4, 3), (100, 150, 200, 128)).save(tmp_path / f"{entry}0001.png")
|
||||
result = images.load_image_edit_result(original, tmp_path, frame_prefix=entry)
|
||||
Image.new("RGBA", (4, 3), (100, 150, 200, 128)).save(tmp_path / entry)
|
||||
result = images.load_image_edit_result(original, tmp_path / entry)
|
||||
assert result.alpha_mode == alpha_mode
|
||||
images.replace_empty_image(source, result)
|
||||
assert source.data.alpha_mode == alpha_mode
|
||||
@@ -131,45 +134,6 @@ def test_frame_sharing_uses_surviving_objects(source, outside):
|
||||
assert tuple(original.size) == (2, 2)
|
||||
|
||||
|
||||
@pytest.mark.parametrize("shared", [False, True])
|
||||
def test_failed_display_configuration_restores_content(source, tmp_path, shared):
|
||||
original = source.data
|
||||
if shared:
|
||||
create_empty("Other", original)
|
||||
# Preserve unsaved pixels as well as the previous packed representation.
|
||||
original.pixels.foreach_set(np.tile([0.1, 0.9, 0.2, 0.5], 4).astype(np.float32))
|
||||
packed = bytes(original.packed_file.data)
|
||||
pixels = images.image_pixels(original)
|
||||
alpha = original.alpha_mode
|
||||
source.image_user.frame_start = 5
|
||||
source.image_user.use_auto_refresh = False
|
||||
for index in (1, 2):
|
||||
Image.new("RGBA", (4, 3), (100, 150, 200, 255)).save(tmp_path / f"image.{index:04d}.png")
|
||||
result = bpy.data.images.load(str(tmp_path / "image.0001.png"))
|
||||
result.source = "SEQUENCE"
|
||||
count = len(bpy.data.images) - 1
|
||||
|
||||
def fail_configuration(user, *args, **kwargs):
|
||||
user.frame_start = 99
|
||||
user.use_auto_refresh = True
|
||||
raise RuntimeError("Display failed")
|
||||
|
||||
with patch.object(images, "configure_image_user", side_effect=fail_configuration):
|
||||
with pytest.raises(RuntimeError, match="Display failed"):
|
||||
images.replace_empty_image(source, result)
|
||||
assert source.data == original
|
||||
assert tuple(original.size) == (2, 2)
|
||||
assert original.alpha_mode == alpha
|
||||
assert original.is_dirty
|
||||
assert bytes(original.packed_file.data) == packed
|
||||
np.testing.assert_array_equal(images.image_pixels(original), pixels)
|
||||
assert source.image_user.frame_start == 5
|
||||
assert not source.image_user.use_auto_refresh
|
||||
assert not source.use_empty_image_alpha
|
||||
assert not original.use_fake_user
|
||||
assert len(bpy.data.images) == count
|
||||
|
||||
|
||||
def test_failed_content_write_restores_source(source):
|
||||
original = source.data
|
||||
packed = bytes(original.packed_file.data)
|
||||
@@ -193,47 +157,6 @@ def test_failed_content_write_restores_source(source):
|
||||
assert len(bpy.data.images) == 1
|
||||
|
||||
|
||||
def test_animated_result_preserves_id_and_reopens(source, tmp_path):
|
||||
original = source.data
|
||||
source.image_user.frame_start = 4
|
||||
source.image_user.use_cyclic = True
|
||||
for index in (1, 2):
|
||||
Image.new("RGBA", (4, 3), (index * 80, 100, 50, 255)).save(tmp_path / f"image.{index:04d}.png")
|
||||
original.source = "SEQUENCE"
|
||||
result = images.load_image_edit_result(source.data, tmp_path)
|
||||
expected_path = result.filepath_raw
|
||||
expected_packed = bytes(result.packed_file.data) if result.packed_file else None
|
||||
final = images.replace_empty_image(source, result)
|
||||
assert final == original
|
||||
assert final.source == "SEQUENCE"
|
||||
assert final.filepath_raw == expected_path
|
||||
assert (bytes(final.packed_file.data) if final.packed_file else None) == expected_packed
|
||||
assert len(images.collect_sequence_frames(final)) == 2
|
||||
assert source.image_user.frame_start == 4
|
||||
assert source.image_user.frame_duration == 2
|
||||
assert source.image_user.use_cyclic
|
||||
assert source.image_user.use_auto_refresh
|
||||
path = str(tmp_path / "animation.blend")
|
||||
bpy.ops.wm.save_as_mainfile(filepath=path)
|
||||
bpy.ops.wm.open_mainfile(filepath=path)
|
||||
reopened = bpy.data.objects["Source"].data
|
||||
assert reopened.name == "Source.png"
|
||||
assert reopened.source == "SEQUENCE"
|
||||
frames = images.collect_sequence_frames(reopened)
|
||||
assert len(frames) == 2
|
||||
for index, frame in enumerate(frames, 1):
|
||||
loaded = bpy.data.images.load(str(frame), check_existing=False)
|
||||
try:
|
||||
assert tuple(loaded.size) == (4, 3)
|
||||
np.testing.assert_allclose(
|
||||
images.image_pixels(loaded)[:4],
|
||||
np.array([index * 80, 100, 50, 255]) / 255,
|
||||
atol=1 / 255,
|
||||
)
|
||||
finally:
|
||||
bpy.data.images.remove(loaded)
|
||||
|
||||
|
||||
@pytest.mark.parametrize("floating", [False, True])
|
||||
def test_failed_generated_commit_restores_unsaved_pixels(source, floating):
|
||||
original = bpy.data.images.new("Generated", width=3, height=2, alpha=True, float_buffer=floating)
|
||||
|
||||
@@ -1,6 +1,4 @@
|
||||
import tempfile
|
||||
import unittest
|
||||
from pathlib import Path
|
||||
from types import SimpleNamespace
|
||||
from tests.anyimage.common.support import load_common_modules
|
||||
|
||||
@@ -21,165 +19,6 @@ class ImageDataTest(unittest.TestCase):
|
||||
)
|
||||
|
||||
|
||||
def test_video_total_frames_reads_movie_duration(self):
|
||||
image = SimpleNamespace(source="MOVIE", frame_duration=97)
|
||||
|
||||
self.assertEqual(self.modules.image.video_total_frames(image), 97)
|
||||
|
||||
|
||||
def test_video_total_frames_counts_sequence_files(self):
|
||||
with tempfile.TemporaryDirectory() as directory:
|
||||
root = Path(directory)
|
||||
(root / "foreground-0001.png").touch()
|
||||
(root / "foreground-0002.png").touch()
|
||||
image = SimpleNamespace(
|
||||
source="SEQUENCE",
|
||||
filepath=str(root / "foreground-0001.png"),
|
||||
)
|
||||
|
||||
self.assertEqual(self.modules.image.video_total_frames(image), 2)
|
||||
|
||||
|
||||
def test_video_total_frames_returns_one_for_still_image(self):
|
||||
image = SimpleNamespace(source="FILE", frame_duration=1)
|
||||
|
||||
self.assertEqual(self.modules.image.video_total_frames(image), 1)
|
||||
|
||||
|
||||
def test_extract_frame_count_uses_empty_image_frames(self):
|
||||
source_object = SimpleNamespace(
|
||||
data=SimpleNamespace(source="MOVIE", frame_duration=97),
|
||||
image_user=SimpleNamespace(frame_duration=24),
|
||||
)
|
||||
|
||||
self.assertEqual(self.modules.image.extract_frame_range(source_object.data, getattr(source_object, "image_user", None))[1], 24)
|
||||
|
||||
|
||||
def test_extract_frame_count_caps_at_video_total(self):
|
||||
source_object = SimpleNamespace(
|
||||
data=SimpleNamespace(source="MOVIE", frame_duration=97),
|
||||
image_user=SimpleNamespace(frame_duration=200),
|
||||
)
|
||||
|
||||
self.assertEqual(self.modules.image.extract_frame_range(source_object.data, getattr(source_object, "image_user", None))[1], 97)
|
||||
|
||||
|
||||
def test_extract_frame_count_falls_back_to_video_total(self):
|
||||
source_object = SimpleNamespace(
|
||||
data=SimpleNamespace(source="MOVIE", frame_duration=97),
|
||||
)
|
||||
|
||||
self.assertEqual(self.modules.image.extract_frame_range(source_object.data, getattr(source_object, "image_user", None))[1], 97)
|
||||
|
||||
|
||||
def test_extract_frame_range_honors_offset_within_clip(self):
|
||||
source_object = SimpleNamespace(
|
||||
data=SimpleNamespace(source="MOVIE", frame_duration=97),
|
||||
image_user=SimpleNamespace(
|
||||
frame_start=4,
|
||||
frame_offset=70,
|
||||
frame_duration=24,
|
||||
),
|
||||
)
|
||||
|
||||
self.assertEqual(
|
||||
self.modules.image.extract_frame_range(source_object.data, source_object.image_user),
|
||||
(71, 24),
|
||||
)
|
||||
|
||||
|
||||
def test_extract_frame_range_clamps_large_offset_to_clip_tail(self):
|
||||
source_object = SimpleNamespace(
|
||||
data=SimpleNamespace(source="MOVIE", frame_duration=97),
|
||||
image_user=SimpleNamespace(
|
||||
frame_start=1,
|
||||
frame_offset=90,
|
||||
frame_duration=24,
|
||||
),
|
||||
)
|
||||
|
||||
self.assertEqual(
|
||||
self.modules.image.extract_frame_range(source_object.data, source_object.image_user),
|
||||
(74, 24),
|
||||
)
|
||||
|
||||
|
||||
def test_extract_frame_range_clamps_negative_offset_to_start(self):
|
||||
source_object = SimpleNamespace(
|
||||
data=SimpleNamespace(source="MOVIE", frame_duration=97),
|
||||
image_user=SimpleNamespace(
|
||||
frame_start=1,
|
||||
frame_offset=-5,
|
||||
frame_duration=24,
|
||||
),
|
||||
)
|
||||
|
||||
self.assertEqual(
|
||||
self.modules.image.extract_frame_range(source_object.data, source_object.image_user),
|
||||
(1, 24),
|
||||
)
|
||||
|
||||
|
||||
def test_extract_frame_range_returns_full_clip_for_still(self):
|
||||
source_object = SimpleNamespace(
|
||||
data=SimpleNamespace(source="FILE", frame_duration=1),
|
||||
image_user=SimpleNamespace(
|
||||
frame_start=1,
|
||||
frame_offset=50,
|
||||
frame_duration=24,
|
||||
),
|
||||
)
|
||||
|
||||
self.assertEqual(
|
||||
self.modules.image.extract_frame_range(source_object.data, source_object.image_user),
|
||||
(1, 1),
|
||||
)
|
||||
|
||||
|
||||
def test_extract_frame_range_honors_single_frame_setting(self):
|
||||
source_object = SimpleNamespace(
|
||||
data=SimpleNamespace(source="MOVIE", frame_duration=97),
|
||||
image_user=SimpleNamespace(
|
||||
frame_start=1,
|
||||
frame_offset=70,
|
||||
frame_duration=1,
|
||||
),
|
||||
)
|
||||
|
||||
self.assertEqual(
|
||||
self.modules.image.extract_frame_range(source_object.data, source_object.image_user),
|
||||
(71, 1),
|
||||
)
|
||||
|
||||
|
||||
def test_configure_image_user_preserves_cyclic_setting(self):
|
||||
image_user = SimpleNamespace(
|
||||
frame_start=0,
|
||||
frame_offset=5,
|
||||
use_auto_refresh=False,
|
||||
use_cyclic=False,
|
||||
frame_duration=1,
|
||||
)
|
||||
|
||||
self.modules.image.configure_image_user(
|
||||
image_user,
|
||||
97,
|
||||
frame_start=4,
|
||||
frame_offset=70,
|
||||
)
|
||||
|
||||
self.assertFalse(image_user.use_cyclic)
|
||||
self.assertTrue(image_user.use_auto_refresh)
|
||||
self.assertEqual(image_user.frame_start, 4)
|
||||
self.assertEqual(image_user.frame_offset, 70)
|
||||
self.assertEqual(image_user.frame_duration, 97)
|
||||
self.modules.image.configure_image_user(
|
||||
image_user,
|
||||
use_cyclic=True,
|
||||
)
|
||||
self.assertTrue(image_user.use_cyclic)
|
||||
|
||||
|
||||
def test_image_user_settings_reads_empty_values(self):
|
||||
source_object = SimpleNamespace(
|
||||
image_user=SimpleNamespace(
|
||||
@@ -208,84 +47,6 @@ class ImageDataTest(unittest.TestCase):
|
||||
)
|
||||
|
||||
|
||||
def test_image_base_name_strips_video_extension(self):
|
||||
image = SimpleNamespace(name="微信视频2026-07-23_192756_414.mp4")
|
||||
|
||||
self.assertEqual(
|
||||
self.modules.image.image_base_name(image),
|
||||
"微信视频2026-07-23_192756_414",
|
||||
)
|
||||
|
||||
|
||||
|
||||
|
||||
def test_force_sequence_source_keeps_loaded_sequence(self):
|
||||
image = SimpleNamespace(source="SEQUENCE")
|
||||
|
||||
result = self.modules.image.force_sequence_source(image)
|
||||
|
||||
self.assertIs(result, image)
|
||||
self.assertEqual(image.source, "SEQUENCE")
|
||||
|
||||
|
||||
def test_force_sequence_source_switches_single_image(self):
|
||||
image = SimpleNamespace(source="FILE")
|
||||
|
||||
result = self.modules.image.force_sequence_source(image)
|
||||
|
||||
self.assertIs(result, image)
|
||||
self.assertEqual(image.source, "SEQUENCE")
|
||||
|
||||
|
||||
def test_force_sequence_source_reports_unloadable_sequence(self):
|
||||
class BlockedImage:
|
||||
@property
|
||||
def source(self):
|
||||
return "FILE"
|
||||
|
||||
@source.setter
|
||||
def source(self, value):
|
||||
raise RuntimeError("blocked")
|
||||
|
||||
with self.assertRaisesRegex(RuntimeError, "cannot load"):
|
||||
self.modules.image.force_sequence_source(BlockedImage())
|
||||
|
||||
|
||||
def test_collect_sequence_frames_returns_numbered_files_in_order(self):
|
||||
with tempfile.TemporaryDirectory() as directory:
|
||||
root = Path(directory)
|
||||
(root / "frame-0001.jpg").touch()
|
||||
(root / "frame-0002.jpg").touch()
|
||||
(root / "frame-0003.jpg").touch()
|
||||
(root / "notes.txt").touch()
|
||||
image = SimpleNamespace(
|
||||
source="SEQUENCE",
|
||||
filepath=str(root / "frame-0002.jpg"),
|
||||
)
|
||||
|
||||
frames = self.modules.image.collect_sequence_frames(image)
|
||||
|
||||
self.assertEqual(
|
||||
[path.name for path in frames],
|
||||
["frame-0001.jpg", "frame-0002.jpg", "frame-0003.jpg"],
|
||||
)
|
||||
|
||||
|
||||
def test_sorted_frame_paths_uses_numeric_frame_order(self):
|
||||
with tempfile.TemporaryDirectory() as directory:
|
||||
root = Path(directory)
|
||||
(root / "frame-2.jpg").touch()
|
||||
(root / "frame-10.jpg").touch()
|
||||
(root / "frame-1.jpg").touch()
|
||||
|
||||
frames = self.modules.image.sorted_frame_paths(root)
|
||||
|
||||
self.assertEqual(
|
||||
[path.name for path in frames],
|
||||
["frame-1.jpg", "frame-2.jpg", "frame-10.jpg"],
|
||||
)
|
||||
|
||||
|
||||
def test_depth_data_uses_source_image_name(self):
|
||||
source_object = SimpleNamespace(
|
||||
name="obj_img",
|
||||
@@ -315,8 +76,6 @@ class ImageDataTest(unittest.TestCase):
|
||||
)
|
||||
|
||||
|
||||
|
||||
|
||||
def test_image_empty_bounds_keep_image_aspect_and_empty_offset(self):
|
||||
source = SimpleNamespace(
|
||||
data=SimpleNamespace(size=(800, 400)),
|
||||
|
||||
@@ -5,7 +5,6 @@ import numpy as np
|
||||
|
||||
from anyimage.common import image as image_data
|
||||
from anyimage.common import material as material_data
|
||||
from anyimage.operators.convert_to_plane import object as plane_object
|
||||
from anyimage.operators.cutout_tool import object as cutout_object
|
||||
from anyimage.operators.cutout_tool import shape as cutout_shape
|
||||
from tests.support.image_objects import _create_cutout_shape, _group_node, _inputs
|
||||
|
||||
@@ -18,21 +18,19 @@ def test_target_uses_edit_tree_and_survives_renaming(texture, tmp_path):
|
||||
texture.id_data.nodes.active = other
|
||||
context.space_data.edit_tree = None
|
||||
write_result(tmp_path)
|
||||
target.apply(tmp_path)
|
||||
target.apply(tmp_path / "foreground.png")
|
||||
assert tuple(texture.image.size) == (8, 6)
|
||||
assert other.image is None
|
||||
|
||||
|
||||
@pytest.mark.parametrize("color_space,alpha_mode", [
|
||||
("sRGB", "PREMUL"),
|
||||
("AgX Base sRGB", "STRAIGHT"),
|
||||
("Filmic sRGB", "STRAIGHT"),
|
||||
("sRGB", "STRAIGHT"),
|
||||
("AgX Base sRGB", "PREMUL"),
|
||||
@pytest.mark.parametrize("color_space,alpha_mode,filename,shared", [
|
||||
("sRGB", "PREMUL", "foreground.png", True),
|
||||
("AgX Base sRGB", "STRAIGHT", "upscale.png", False),
|
||||
("Filmic sRGB", "STRAIGHT", "foreground.png", False),
|
||||
("sRGB", "STRAIGHT", "upscale.png", True),
|
||||
("AgX Base sRGB", "PREMUL", "foreground.png", True),
|
||||
])
|
||||
@pytest.mark.parametrize("prefix", ["image.", "upscale."])
|
||||
@pytest.mark.parametrize("shared", [False, True])
|
||||
def test_image_actions_preserve_source_color_space_and_alpha(texture, tmp_path, color_space, alpha_mode, prefix, shared):
|
||||
def test_image_actions_preserve_source_color_space_and_alpha(texture, tmp_path, color_space, alpha_mode, filename, shared):
|
||||
texture.image.colorspace_settings.name = color_space
|
||||
texture.image.alpha_mode = alpha_mode
|
||||
original = texture.image
|
||||
@@ -41,8 +39,8 @@ def test_image_actions_preserve_source_color_space_and_alpha(texture, tmp_path,
|
||||
other = texture.id_data.nodes.new("ShaderNodeTexImage")
|
||||
other.image = original
|
||||
target = targets.ImageEditTarget.capture(texture_context(texture))
|
||||
write_result(tmp_path, prefix)
|
||||
target.apply(tmp_path, frame_prefix=prefix)
|
||||
write_result(tmp_path, filename)
|
||||
target.apply(tmp_path / filename)
|
||||
result = texture.image
|
||||
assert result.colorspace_settings.name == color_space
|
||||
assert result.alpha_mode == alpha_mode
|
||||
@@ -102,7 +100,7 @@ def test_target_rejects_changed_identity_before_loading_result(texture, tmp_path
|
||||
bpy.data.images.remove(texture.image)
|
||||
count = len(bpy.data.images)
|
||||
with pytest.raises(RuntimeError, match="no longer available"):
|
||||
target.apply(tmp_path)
|
||||
target.apply(tmp_path / "foreground.png")
|
||||
assert len(bpy.data.images) == count
|
||||
|
||||
|
||||
@@ -137,7 +135,7 @@ def test_texture_commit_preserves_structure_and_isolates_users(texture, tmp_path
|
||||
shared = reference in {"same_tree", "other_material", "empty", "brush"}
|
||||
links = [(link.from_socket, link.to_socket) for link in texture.id_data.links]
|
||||
write_result(tmp_path)
|
||||
target.apply(tmp_path)
|
||||
target.apply(tmp_path / "foreground.png")
|
||||
assert (texture.image == original) == (not shared)
|
||||
assert tuple(texture.image.size) == (8, 6)
|
||||
assert texture.image.colorspace_settings.name == "Non-Color"
|
||||
@@ -163,63 +161,46 @@ def test_texture_commit_preserves_structure_and_isolates_users(texture, tmp_path
|
||||
assert len(reopened.id_data.links) == len(links)
|
||||
|
||||
|
||||
@pytest.mark.parametrize("shared", [False, True])
|
||||
def test_animated_texture_preserves_playback_and_reopens(texture, tmp_path, shared):
|
||||
write_result(tmp_path, count=4)
|
||||
texture.image.source = "SEQUENCE"
|
||||
texture.image.filepath_raw = str(tmp_path / "image.0001.png")
|
||||
texture.image_user.frame_start = 9
|
||||
texture.image_user.frame_offset = 1
|
||||
texture.image_user.frame_duration = 2
|
||||
texture.image_user.use_cyclic = True
|
||||
context = texture_context(texture)
|
||||
target = targets.ImageEditTarget.capture(context)
|
||||
assert (target.start_frame, target.frame_count) == (2, 2)
|
||||
if shared:
|
||||
other = texture.id_data.nodes.new("ShaderNodeTexImage")
|
||||
other.image = texture.image
|
||||
with patch.object(targets, "prepare_image_input", return_value=(tmp_path, False)):
|
||||
target.prepare(context)
|
||||
assert context.scene.anyimage_settings.video_frames == 2
|
||||
target.apply(tmp_path)
|
||||
assert images.image_user_settings(texture) == (9, 0, 4, True)
|
||||
assert texture.image_user.use_auto_refresh
|
||||
assert texture.image.source == "SEQUENCE"
|
||||
path = str(tmp_path / "animated.blend")
|
||||
bpy.ops.wm.save_as_mainfile(filepath=path)
|
||||
bpy.ops.wm.open_mainfile(filepath=path)
|
||||
reopened = bpy.data.materials["Texture material"].node_tree.nodes["Source texture"]
|
||||
assert images.image_user_settings(reopened) == (9, 0, 4, True)
|
||||
assert len(images.collect_sequence_frames(reopened.image)) == 4
|
||||
|
||||
|
||||
@pytest.mark.parametrize("source", ["MOVIE", "SEQUENCE"])
|
||||
def test_target_rejects_animated_image(texture, source):
|
||||
texture.image.source = source
|
||||
with pytest.raises(RuntimeError, match="Only still images"):
|
||||
targets.ImageEditTarget.capture(texture_context(texture))
|
||||
|
||||
|
||||
@pytest.mark.parametrize("shared", [False, True])
|
||||
def test_texture_failed_commit_restores_dirty_pixels_and_playback(texture, tmp_path, shared):
|
||||
def test_failed_texture_assignment_restores_source(texture, tmp_path, shared):
|
||||
original = texture.image
|
||||
if shared:
|
||||
other = texture.id_data.nodes.new("ShaderNodeTexImage")
|
||||
other.image = original
|
||||
original.pixels.foreach_set(np.tile([0.1, 0.3, 0.7, 0.5], 12).astype(np.float32))
|
||||
packed = bytes(original.packed_file.data)
|
||||
pixels = images.image_pixels(original)
|
||||
timing = images.image_user_settings(texture)
|
||||
write_result(tmp_path, count=2)
|
||||
result = bpy.data.images.load(str(tmp_path / "image.0001.png"))
|
||||
result.source = "SEQUENCE"
|
||||
result = images.load_image_edit_result(original, write_result(tmp_path))
|
||||
count = len(bpy.data.images) - 1
|
||||
|
||||
def fail_configuration(user, *args, **kwargs):
|
||||
user.frame_start = 80
|
||||
user.use_auto_refresh = True
|
||||
raise RuntimeError("Playback failed")
|
||||
class FailingTarget:
|
||||
failed = False
|
||||
|
||||
with patch.object(targets, "configure_image_user", side_effect=fail_configuration):
|
||||
with pytest.raises(RuntimeError, match="Playback failed"):
|
||||
targets.replace_texture_image(texture, result)
|
||||
def __getattr__(self, name):
|
||||
return getattr(texture, name)
|
||||
|
||||
@property
|
||||
def image(self):
|
||||
return texture.image
|
||||
|
||||
@image.setter
|
||||
def image(self, value):
|
||||
texture.image = value
|
||||
if not self.failed:
|
||||
self.failed = True
|
||||
raise RuntimeError("Assignment failed")
|
||||
|
||||
with patch.object(targets, "has_other_image_user", return_value=shared):
|
||||
with pytest.raises(RuntimeError, match="Assignment failed"):
|
||||
targets.replace_texture_image(FailingTarget(), result)
|
||||
assert texture.image == original
|
||||
assert tuple(original.size) == (4, 3)
|
||||
assert bytes(original.packed_file.data) == packed
|
||||
np.testing.assert_array_equal(images.image_pixels(original), pixels)
|
||||
assert images.image_user_settings(texture) == timing
|
||||
assert not texture.image_user.use_auto_refresh
|
||||
assert len(bpy.data.images) == count
|
||||
|
||||
@@ -7,20 +7,6 @@ from tests.support.blender import BlenderTestCase
|
||||
|
||||
|
||||
class InputPreparationTest(BlenderTestCase):
|
||||
def test_sequence_format_is_checked_before_copy(self):
|
||||
with tempfile.TemporaryDirectory() as directory:
|
||||
path = Path(directory) / "frame.0001.exr"
|
||||
path.touch()
|
||||
image = SimpleNamespace(source="SEQUENCE")
|
||||
with (
|
||||
patch.object(self.image_data, "image_path", return_value=path),
|
||||
patch.object(self.image_data, "image_is_packed", return_value=False),
|
||||
patch.object(self.image_data.tempfile, "mkdtemp") as create,
|
||||
):
|
||||
with self.assertRaisesRegex(RuntimeError, "Unsupported input format: .exr"):
|
||||
self.image_data.prepare_image_input(image, None)
|
||||
create.assert_not_called()
|
||||
self.assertTrue(path.exists())
|
||||
|
||||
def test_packed_export_failure_removes_temporary_directory(self):
|
||||
for failure in (OSError("disk full"), None):
|
||||
@@ -34,21 +20,11 @@ class InputPreparationTest(BlenderTestCase):
|
||||
self.assertEqual(image.file_format, "OPEN_EXR")
|
||||
self.assertFalse(output.exists())
|
||||
|
||||
def test_sequence_copy_failure_preserves_originals(self):
|
||||
with tempfile.TemporaryDirectory() as directory:
|
||||
root = Path(directory)
|
||||
source = root / "frame.0001.png"
|
||||
source.write_bytes(b"source")
|
||||
output = root / "prepared"
|
||||
output.mkdir()
|
||||
with (
|
||||
patch.object(self.image_data, "image_path", return_value=source),
|
||||
patch.object(self.image_data, "image_is_packed", return_value=False),
|
||||
patch.object(self.image_data, "collect_sequence_frames", return_value=[source]),
|
||||
patch.object(self.image_data.tempfile, "mkdtemp", return_value=str(output)),
|
||||
patch.object(self.image_data.shutil, "copy2", side_effect=OSError("disk full")),
|
||||
):
|
||||
with self.assertRaises(OSError):
|
||||
self.image_data.prepare_image_input(SimpleNamespace(source="SEQUENCE"), None)
|
||||
self.assertFalse(output.exists())
|
||||
self.assertEqual(source.read_bytes(), b"source")
|
||||
|
||||
def test_animated_inputs_are_rejected_before_export(self):
|
||||
for source in ("MOVIE", "SEQUENCE"):
|
||||
with self.subTest(source=source), patch.object(self.image_data, "save_packed_image") as export:
|
||||
image = SimpleNamespace(source=source, packed_file=object())
|
||||
with self.assertRaisesRegex(RuntimeError, "Only still images"):
|
||||
self.image_data.prepare_image_input(image, None)
|
||||
export.assert_not_called()
|
||||
|
||||
@@ -19,7 +19,6 @@ def test_material_alpha_strength_follows_preference(image_layer, monkeypatch, en
|
||||
assert shader.node_tree.name == "O Shadeless"
|
||||
assert shader.inputs["Color"].links[0].from_node.type == "TEX_IMAGE"
|
||||
assert shader.inputs["Alpha"].links[0].from_socket.name == "Alpha"
|
||||
assert not any(node.type in {"MIX_SHADER", "BSDF_TRANSPARENT"} for node in result.node_tree.nodes)
|
||||
assert len([node for node in result.node_tree.nodes if node.type == "GROUP"]) == 1
|
||||
return
|
||||
layer = next(node for node in result.node_tree.nodes if node.type == "GROUP" and node.node_tree == image_layer)
|
||||
|
||||
@@ -29,36 +29,6 @@ class MaterialSettingsTest(unittest.TestCase):
|
||||
)
|
||||
|
||||
|
||||
def test_material_color_space_falls_back_when_unsupported(self):
|
||||
class ColorSpaceSettings:
|
||||
def __init__(self):
|
||||
self._name = "sRGB"
|
||||
|
||||
@property
|
||||
def name(self):
|
||||
return self._name
|
||||
|
||||
@name.setter
|
||||
def name(self, value):
|
||||
if value != "sRGB":
|
||||
raise TypeError("unsupported color space")
|
||||
self._name = value
|
||||
|
||||
image = SimpleNamespace(
|
||||
colorspace_settings=ColorSpaceSettings(),
|
||||
users=0, use_fake_user=False, alpha_mode="STRAIGHT",
|
||||
)
|
||||
scene = SimpleNamespace(
|
||||
view_settings=SimpleNamespace(view_transform="ACES 2.0")
|
||||
)
|
||||
|
||||
result = self.modules.material.configure_material_color_image(image, scene)
|
||||
|
||||
self.assertIs(result, image)
|
||||
self.assertEqual(image.alpha_mode, "PREMUL")
|
||||
self.assertEqual(image.colorspace_settings.name, "sRGB")
|
||||
|
||||
|
||||
def test_surface_displacement_is_set_to_displacement_only(self):
|
||||
class Material:
|
||||
surface_displacement_method = "BUMP"
|
||||
|
||||
@@ -35,18 +35,6 @@ def color_texture(result):
|
||||
return next(node.image for node in result.node_tree.nodes if node.type == "TEX_IMAGE")
|
||||
|
||||
|
||||
def test_adaptation_preference_defaults_and_reads_current_value():
|
||||
with patch.object(preferences, "addon_preferences", return_value=None):
|
||||
assert preferences.configured_material_view_adaptation()
|
||||
with patch.object(preferences, "addon_preferences", return_value=SimpleNamespace()):
|
||||
assert preferences.configured_material_view_adaptation()
|
||||
settings = SimpleNamespace(adapt_material_to_view_transform=False)
|
||||
with patch.object(preferences, "addon_preferences", return_value=settings):
|
||||
assert not preferences.configured_material_view_adaptation()
|
||||
settings.adapt_material_to_view_transform = True
|
||||
assert preferences.configured_material_view_adaptation()
|
||||
|
||||
|
||||
@pytest.mark.parametrize("enabled", [False, True])
|
||||
@pytest.mark.parametrize("view,space", [*material.VIEW_TRANSFORM_COLOR_SPACES, ("Standard", "sRGB"), ("Unknown", "sRGB")])
|
||||
def test_color_space_and_alpha_follow_preference(color_data, enabled, view, space):
|
||||
@@ -85,11 +73,11 @@ def test_material_pixels_survive_pack_and_library_reload(color_data, tmp_path, e
|
||||
source.pixels.foreach_set((expected.astype(np.float32) / 255).ravel())
|
||||
else:
|
||||
source = bpy.data.images.new("Material pixels", width=2, height=1, alpha=True, float_buffer=kind == "float")
|
||||
source.alpha_mode = "PREMUL"
|
||||
source.pixels.foreach_set((expected.astype(np.float32) / 255).ravel())
|
||||
if kind == "packed":
|
||||
source.pack()
|
||||
source.reload()
|
||||
images.configure_color_alpha(source)
|
||||
before = images.image_pixels(source)
|
||||
owner = bpy.data.objects.new("Color source owner", None)
|
||||
owner.empty_display_type = "IMAGE"
|
||||
@@ -175,51 +163,13 @@ def test_material_configures_the_image_selected_by_its_caller(color_data, entry,
|
||||
def test_edit_results_preserve_source_alpha_with_adaptation_enabled(color_data, tmp_path):
|
||||
source = bpy.data.images.new("Editing source", width=1, height=1, alpha=True)
|
||||
rgba = np.array([0.2, 0.4, 0.8, 0.25], dtype=np.float32)
|
||||
Image.fromarray(np.array([[[51, 102, 204, 64]]], dtype=np.uint8)).save(tmp_path / "image.0001.png")
|
||||
Image.fromarray(np.array([[[51, 102, 204, 64]]], dtype=np.uint8)).save(tmp_path / "foreground.png")
|
||||
with patch.object(preferences, "addon_preferences", return_value=SimpleNamespace(adapt_material_to_view_transform=True)):
|
||||
edited = images.create_image_edit_result(source, rgba, (1, 1))
|
||||
loaded = images.load_image_edit_result(source, tmp_path)
|
||||
loaded = images.load_image_edit_result(source, tmp_path / "foreground.png")
|
||||
for result in (edited, loaded):
|
||||
assert result.colorspace_settings.name == "sRGB"
|
||||
assert result.alpha_mode == source.alpha_mode
|
||||
np.testing.assert_allclose(images.image_pixels(result), rgba, atol=1 / 255)
|
||||
assert source.colorspace_settings.name == "sRGB"
|
||||
assert source.alpha_mode == "STRAIGHT"
|
||||
|
||||
|
||||
@pytest.mark.parametrize("supported", [{"sRGB", "Non-Color"}, {"Non-Color"}])
|
||||
def test_unavailable_color_space_falls_back_or_preserves_original(supported):
|
||||
class ColorSpace:
|
||||
value = "Non-Color"
|
||||
|
||||
@property
|
||||
def name(self):
|
||||
return self.value
|
||||
|
||||
@name.setter
|
||||
def name(self, value):
|
||||
if value not in supported:
|
||||
raise TypeError("Unsupported color space")
|
||||
self.value = value
|
||||
|
||||
image = SimpleNamespace(colorspace_settings=ColorSpace(), users=0, use_fake_user=False, alpha_mode="STRAIGHT")
|
||||
with patch.object(material, "configured_material_view_adaptation", return_value=True):
|
||||
if "sRGB" in supported:
|
||||
result = material.configure_material_color_image(image, view_scene("AgX"))
|
||||
assert result.colorspace_settings.name == "sRGB"
|
||||
assert result.alpha_mode == "PREMUL"
|
||||
else:
|
||||
with pytest.raises(RuntimeError, match="material color space"):
|
||||
material.configure_material_color_image(image, view_scene("AgX"))
|
||||
assert image.colorspace_settings.name == "Non-Color"
|
||||
assert image.alpha_mode == "STRAIGHT"
|
||||
|
||||
|
||||
def test_missing_scene_uses_srgb(color_data):
|
||||
image = bpy.data.images.new("Missing scene", width=1, height=1)
|
||||
with patch.object(material, "bpy", SimpleNamespace(context=SimpleNamespace(scene=None))), patch.object(
|
||||
material, "configured_material_view_adaptation", return_value=True
|
||||
):
|
||||
result = material.configure_material_color_image(image)
|
||||
assert result.colorspace_settings.name == "sRGB"
|
||||
assert result.alpha_mode == "PREMUL"
|
||||
|
||||
@@ -9,8 +9,8 @@ class ModifierInputsTest(unittest.TestCase):
|
||||
|
||||
def test_runtime_modifier_enum_input_uses_string_identifier(self):
|
||||
enum_items = (
|
||||
SimpleNamespace(identifier="SURFACE"),
|
||||
SimpleNamespace(identifier="BALLOON"),
|
||||
SimpleNamespace(identifier="Balloon"),
|
||||
SimpleNamespace(identifier="Shell"),
|
||||
)
|
||||
value_property = SimpleNamespace(type="ENUM", enum_items=enum_items)
|
||||
input_slot = SimpleNamespace(
|
||||
@@ -19,13 +19,13 @@ class ModifierInputsTest(unittest.TestCase):
|
||||
)
|
||||
modifier = SimpleNamespace(
|
||||
properties=SimpleNamespace(
|
||||
inputs=SimpleNamespace(Shape=input_slot),
|
||||
inputs=SimpleNamespace(Mode=input_slot),
|
||||
)
|
||||
)
|
||||
|
||||
self.modules.object.set_modifier_input(modifier, "Shape", 1)
|
||||
self.modules.object.set_modifier_input(modifier, "Mode", 1)
|
||||
|
||||
self.assertEqual(input_slot.value, "BALLOON")
|
||||
self.assertEqual(input_slot.value, "Shell")
|
||||
|
||||
|
||||
def test_modifier_inputs_support_rna_and_dictionary_values(self):
|
||||
@@ -44,9 +44,9 @@ class ModifierInputsTest(unittest.TestCase):
|
||||
|
||||
self.assertEqual(input_slot.value, 2.0)
|
||||
|
||||
class LegacyModifier(dict):
|
||||
class DictionaryModifier(dict):
|
||||
properties = None
|
||||
|
||||
legacy_modifier = LegacyModifier()
|
||||
self.modules.object.set_modifier_input(legacy_modifier, "Shape", 1)
|
||||
self.assertEqual(legacy_modifier["Shape"], 1)
|
||||
dictionary_modifier = DictionaryModifier()
|
||||
self.modules.object.set_modifier_input(dictionary_modifier, "Mode", 1)
|
||||
self.assertEqual(dictionary_modifier["Mode"], 1)
|
||||
|
||||
@@ -1,14 +0,0 @@
|
||||
from tests.support.paths import PROJECT_ROOT
|
||||
import unittest
|
||||
from tests.anyimage.common.support import load_common_modules
|
||||
|
||||
class NodeLoadingTest(unittest.TestCase):
|
||||
@classmethod
|
||||
def setUpClass(cls):
|
||||
cls.modules = load_common_modules()
|
||||
|
||||
def test_node_asset_path_resolves_bundled_library(self):
|
||||
self.assertEqual(
|
||||
self.modules.node.node_asset_path(),
|
||||
PROJECT_ROOT / "src" / "anyimage" / "assets" / "O_AnyImage.blend",
|
||||
)
|
||||
@@ -21,11 +21,11 @@ class AiImageOperatorsTest(BlenderTestCase):
|
||||
self.assertEqual(operator_type.bl_options, {"UNDO"})
|
||||
operator = operator_type()
|
||||
operator.input_path = "input.png"
|
||||
operator._image_target = SimpleNamespace(start_frame=4)
|
||||
operator._image_target = SimpleNamespace()
|
||||
operator.model = "HAT_GAN_X4_SHARPER"
|
||||
operator.max_input_size = 1024
|
||||
context = SimpleNamespace(
|
||||
scene=SimpleNamespace(anyimage_settings=SimpleNamespace(video_frames=7))
|
||||
scene=SimpleNamespace(anyimage_settings=SimpleNamespace())
|
||||
)
|
||||
with (
|
||||
patch.object(self.upscale, "require_environment"),
|
||||
@@ -44,8 +44,6 @@ class AiImageOperatorsTest(BlenderTestCase):
|
||||
"input": "input.png",
|
||||
"model": "HAT_GAN_X4_SHARPER",
|
||||
"device": "cuda",
|
||||
"video_frames": 7,
|
||||
"start_frame": 4,
|
||||
"max_input_size": 1024,
|
||||
},
|
||||
)
|
||||
@@ -89,7 +87,7 @@ class AiImageOperatorsTest(BlenderTestCase):
|
||||
|
||||
|
||||
def test_remove_background_loads_a_plain_image_edit_result(self):
|
||||
result = SimpleNamespace(directory=Path("result"))
|
||||
result = SimpleNamespace(file=lambda key: Path("result") / f"{key}.png")
|
||||
operator = self.remove_background.RunBackgroundRemoval()
|
||||
operator._image_target = Mock()
|
||||
ops = SimpleNamespace(
|
||||
@@ -100,12 +98,12 @@ class AiImageOperatorsTest(BlenderTestCase):
|
||||
):
|
||||
operator.response(None, result)
|
||||
|
||||
operator._image_target.apply.assert_called_once_with(Path("result"))
|
||||
operator._image_target.apply.assert_called_once_with(Path("result/foreground.png"))
|
||||
ops.ed.undo_push.assert_called_once_with(message="Remove Background")
|
||||
|
||||
|
||||
def test_upscale_loads_a_plain_image_edit_result(self):
|
||||
result = SimpleNamespace(directory=Path("result"))
|
||||
result = SimpleNamespace(file=lambda key: Path("result") / f"{key}.png")
|
||||
operator = self.upscale.UpscaleImage()
|
||||
operator._image_target = Mock()
|
||||
operator.model = "HAT_GAN_X4_SHARPER"
|
||||
@@ -119,7 +117,6 @@ class AiImageOperatorsTest(BlenderTestCase):
|
||||
message = operator.response(None, result)
|
||||
|
||||
operator._image_target.apply.assert_called_once_with(
|
||||
Path("result"),
|
||||
frame_prefix="upscale.",
|
||||
Path("result/upscale.png"),
|
||||
)
|
||||
self.assertEqual(message, "Image is upscaled with HAT Sharper")
|
||||
|
||||
@@ -19,10 +19,10 @@ class AiInputFailuresTest(BlenderTestCase):
|
||||
):
|
||||
operator = operator_type()
|
||||
operator.report = Mock()
|
||||
target = Mock(image=SimpleNamespace(size=(16, 16)), start_frame=1)
|
||||
target = Mock(image=SimpleNamespace(size=(16, 16)))
|
||||
target.prepare.return_value = (path, False)
|
||||
context = SimpleNamespace(scene=SimpleNamespace(
|
||||
anyimage_settings=SimpleNamespace(video_frames=1)))
|
||||
anyimage_settings=SimpleNamespace()))
|
||||
with ExitStack() as stack:
|
||||
stack.enter_context(patch.object(module.ImageEditTarget, "capture", return_value=target))
|
||||
stack.enter_context(patch.object(module, "require_environment"))
|
||||
|
||||
@@ -2,11 +2,43 @@ import importlib
|
||||
import tempfile
|
||||
from pathlib import Path
|
||||
from types import SimpleNamespace
|
||||
from unittest.mock import patch
|
||||
from unittest.mock import Mock, patch
|
||||
from tests.support.blender import BlenderTestCase
|
||||
|
||||
|
||||
class AiSetupTest(BlenderTestCase):
|
||||
def test_setup_dialog_uses_install_title_and_button(self):
|
||||
show_dialog = Mock(return_value={"RUNNING_MODAL"})
|
||||
context = SimpleNamespace(
|
||||
window_manager=SimpleNamespace(invoke_props_dialog=show_dialog)
|
||||
)
|
||||
operator = self.ai_setup.SetupAIEnvironment()
|
||||
|
||||
self.assertEqual(operator.invoke(context, None), {"RUNNING_MODAL"})
|
||||
show_dialog.assert_called_once_with(
|
||||
operator, width=440, title="Set Up AI", confirm_text="Install"
|
||||
)
|
||||
|
||||
def test_setup_dialog_shows_install_summary_and_mirror_option(self):
|
||||
labels = []
|
||||
properties = []
|
||||
operator = self.ai_setup.SetupAIEnvironment()
|
||||
operator.layout = SimpleNamespace(
|
||||
label=lambda **kwargs: labels.append(kwargs["text"]),
|
||||
separator=lambda **_kwargs: None,
|
||||
prop=lambda _owner, name: properties.append(name),
|
||||
)
|
||||
with patch.object(
|
||||
self.ai_setup, "ai_status", return_value={"environment_ready": False}
|
||||
):
|
||||
operator.draw(None)
|
||||
|
||||
self.assertEqual(labels, [
|
||||
"Set up AI features for depth, background removal and upscaling.",
|
||||
"Downloads the runtime and required models (259 MB for models).",
|
||||
])
|
||||
self.assertEqual(properties, ["use_mirror"])
|
||||
|
||||
def test_runtime_exposes_declared_environment_and_post_install(self):
|
||||
runtime = self.anyimage.runtime
|
||||
self.assertEqual(runtime.environment["python"], "3.12")
|
||||
|
||||
@@ -2,8 +2,8 @@ from unittest.mock import patch
|
||||
import numpy as np
|
||||
|
||||
|
||||
from anyimage.operators.cutout_tool import geometry as cutout_geometry
|
||||
from anyimage.operators.cutout_tool.geometry import poisson_balloon_profile
|
||||
from anyimage.operators.cutout_tool import balloon
|
||||
from anyimage.operators.cutout_tool.balloon import poisson_balloon_profile
|
||||
|
||||
|
||||
def test_poisson_balloon_profile_is_zero_on_boundary_and_positive_inside():
|
||||
@@ -62,11 +62,11 @@ def test_teddy_balloon_profile_uses_chordal_axis_radius():
|
||||
nodes = np.asarray(((1, 1), (3, 1)), dtype=float)
|
||||
|
||||
with patch.object(
|
||||
cutout_geometry,
|
||||
balloon,
|
||||
"chordal_axis",
|
||||
return_value=(nodes, [(0, 1)], np.asarray((1.0, 1.0))),
|
||||
):
|
||||
profile = cutout_geometry.teddy_balloon_heights(
|
||||
profile = balloon.teddy_balloon_heights(
|
||||
points,
|
||||
faces,
|
||||
points[:4],
|
||||
|
||||
@@ -23,7 +23,7 @@ def clipboard():
|
||||
bpy.ops.wm.read_factory_settings(use_empty=True)
|
||||
|
||||
|
||||
@pytest.mark.parametrize("edit", ["paint", "packed", "hidden_rgb", "size", "space", "alpha", "legacy", "unreadable"])
|
||||
@pytest.mark.parametrize("edit", ["paint", "packed", "hidden_rgb", "size", "space", "alpha", "unreadable"])
|
||||
def test_changed_cache_is_skipped_without_modifying_candidate(clipboard, edit, monkeypatch):
|
||||
original, reused = actions.acquire_packed_image(clipboard, ".png")
|
||||
expected = images.image_pixels(original)
|
||||
@@ -41,8 +41,6 @@ def test_changed_cache_is_skipped_without_modifying_candidate(clipboard, edit, m
|
||||
original.colorspace_settings.name = "Non-Color"
|
||||
elif edit == "alpha":
|
||||
original.alpha_mode = "PREMUL"
|
||||
elif edit == "legacy":
|
||||
del original[actions.CLIPBOARD_CONTENT_PROPERTY]
|
||||
else:
|
||||
read = actions._clipboard_content_hash
|
||||
def fail_candidate(image):
|
||||
|
||||
@@ -84,7 +84,6 @@ class ClipboardImageTest(unittest.TestCase):
|
||||
)
|
||||
for name in clipboard_image.PasteClipboardImage.__annotations__:
|
||||
self.assertFalse(name.startswith("anyimage_"), name)
|
||||
self.assertFalse(name.startswith("o_"), name)
|
||||
|
||||
def test_registers_and_removes_clipboard_shortcuts(self):
|
||||
class KeyMapItems:
|
||||
@@ -345,10 +344,6 @@ class ClipboardImageTest(unittest.TestCase):
|
||||
)
|
||||
|
||||
self.assertFalse(options["default"])
|
||||
annotations = self.addon.clipboard_image.PasteClipboardImage.__annotations__
|
||||
self.assertNotIn("subdivisions", annotations)
|
||||
self.assertNotIn("thickness", annotations)
|
||||
self.assertNotIn("unshaded", annotations)
|
||||
|
||||
def test_native_copy_takes_priority_while_clipboard_is_unchanged(self):
|
||||
clipboard_image = self.addon.clipboard_image
|
||||
|
||||
@@ -1,34 +1,11 @@
|
||||
import json
|
||||
from types import SimpleNamespace
|
||||
|
||||
import numpy as np
|
||||
|
||||
|
||||
from anyimage.common.selection import SelectionMask
|
||||
from anyimage.common.depth import (
|
||||
depth_uniform_scale,
|
||||
load_depth_metadata,
|
||||
)
|
||||
from anyimage.common.depth import depth_uniform_scale
|
||||
from anyimage.operators.cutout_tool import geometry as cutout_geometry
|
||||
|
||||
|
||||
def test_depth_metadata_loader_uses_one_schema_for_plane_and_cutout(tmp_path):
|
||||
path = tmp_path / "depth.json"
|
||||
path.write_text(
|
||||
json.dumps(
|
||||
{
|
||||
"image_size": [3, 2],
|
||||
"intrinsics": np.eye(3).tolist(),
|
||||
"reference_depth": 1.0,
|
||||
}
|
||||
),
|
||||
encoding="utf-8",
|
||||
)
|
||||
|
||||
metadata = load_depth_metadata(path)
|
||||
|
||||
assert metadata["image_size"] == (3, 2)
|
||||
assert metadata["intrinsics"] == ((1.0, 0.0, 0.0), (0.0, 1.0, 0.0), (0.0, 0.0, 1.0))
|
||||
assert metadata["reference_depth"] == 1.0
|
||||
from anyimage.operators.cutout_tool import depth_calibration
|
||||
|
||||
|
||||
def test_base_plane_fit_is_stable_for_a_slender_diagonal_domain():
|
||||
@@ -42,7 +19,7 @@ def test_base_plane_fit_is_stable_for_a_slender_diagonal_domain():
|
||||
)
|
||||
values = 0.4 * along + np.sign(across) * 0.05
|
||||
|
||||
plane = cutout_geometry._stable_plane(coordinates, values)
|
||||
plane = depth_calibration._stable_plane(coordinates, values)
|
||||
selected_plane = coordinates @ plane[:2] + plane[2]
|
||||
corners = np.asarray(((-1.0, -1.0), (-1.0, 1.0), (1.0, -1.0), (1.0, 1.0)))
|
||||
corner_plane = corners @ plane[:2] + plane[2]
|
||||
@@ -63,7 +40,7 @@ def test_depth_balloon_plane_calibration_matches_cutout_axes():
|
||||
uv.append((column * 0.5, 1.0 - row * 0.5))
|
||||
camera_depth[row, column] = 2.0 + 0.2 * local_x + 0.1 * local_y
|
||||
|
||||
reference, direction = cutout_geometry.depth_balloon_projection_parameters(
|
||||
reference, direction = depth_calibration.depth_balloon_projection_parameters(
|
||||
vertices,
|
||||
uv,
|
||||
camera_depth,
|
||||
@@ -93,7 +70,7 @@ def test_depth_balloon_reference_uses_the_shared_upper_depth_percentile():
|
||||
vertices.append((float(x), float(size - 1 - y), 0.0))
|
||||
uv.append((x / (size - 1), 1.0 - y / (size - 1)))
|
||||
|
||||
reference, direction = cutout_geometry.depth_balloon_projection_parameters(
|
||||
reference, direction = depth_calibration.depth_balloon_projection_parameters(
|
||||
vertices,
|
||||
uv,
|
||||
camera_depth,
|
||||
@@ -126,7 +103,7 @@ def test_depth_surface_projection_uses_one_scale_for_all_camera_axes():
|
||||
)
|
||||
|
||||
assert uniform_scale == 2.0
|
||||
reference_depth = cutout_geometry.metric_depth_reference(
|
||||
reference_depth = depth_calibration.metric_depth_reference(
|
||||
points[..., 2],
|
||||
np.ones((51, 101), dtype=bool),
|
||||
uniform_scale,
|
||||
@@ -157,8 +134,8 @@ def test_cutout_threshold_controls_contour_and_calibration():
|
||||
|
||||
|
||||
def test_cutout_depth_calibration_uses_five_percent_transparency_threshold():
|
||||
assert not cutout_geometry.use_adaptive_depth_calibration(0.0499)
|
||||
assert cutout_geometry.use_adaptive_depth_calibration(0.05)
|
||||
assert not depth_calibration.use_adaptive_depth_calibration(0.0499)
|
||||
assert depth_calibration.use_adaptive_depth_calibration(0.05)
|
||||
|
||||
|
||||
def test_selection_calibration_uses_the_final_selection_mask():
|
||||
@@ -3,7 +3,8 @@ import numpy as np
|
||||
import pytest
|
||||
from anyimage.operators.cutout_tool.object import load_cutout_node_group
|
||||
from anyimage.operators.cutout_tool.operators import normal_mode_for_shape
|
||||
from anyimage.operators.cutout_tool.geometry import refine_balloon_triangles, poisson_balloon_profile
|
||||
from anyimage.operators.cutout_tool.geometry import refine_balloon_triangles
|
||||
from anyimage.operators.cutout_tool.balloon import poisson_balloon_profile
|
||||
|
||||
|
||||
@pytest.mark.parametrize("enabled", [False, True])
|
||||
@@ -20,18 +21,18 @@ def test_new_cutout_does_not_reuse_a_saved_wrapper():
|
||||
saved.interface.new_socket(name="Shape", in_out="INPUT", socket_type="NodeSocketMenu")
|
||||
saved.use_fake_user = True
|
||||
current = load_cutout_node_group("SOLID")
|
||||
assert current != saved and current.get("anyimage_asset_revision") == 11
|
||||
assert current != saved and current.get("anyimage_asset_revision") == 13
|
||||
assert saved in bpy.data.node_groups.values()
|
||||
assert current == load_cutout_node_group("FLAT")
|
||||
assert any(item.name == "Shape" for item in saved.interface.items_tree)
|
||||
|
||||
|
||||
def test_depth_cutout_loads_boundary_blur_revision_and_preserves_existing_group():
|
||||
def test_depth_cutout_loads_uniform_thickness_revision_and_preserves_existing_group():
|
||||
bpy.ops.wm.read_factory_settings(use_empty=True)
|
||||
saved = bpy.data.node_groups.new("O Image Depth Cutout", "GeometryNodeTree")
|
||||
saved["anyimage_asset_revision"] = 11
|
||||
current = load_cutout_node_group("DEPTH_SOLID")
|
||||
assert current != saved and current["anyimage_asset_revision"] == 23
|
||||
assert current != saved and current["anyimage_asset_revision"] == 26
|
||||
assert saved in bpy.data.node_groups.values()
|
||||
assert saved["anyimage_asset_revision"] == 11
|
||||
assert load_cutout_node_group("DEPTH_SOLID") == current
|
||||
|
||||
@@ -5,6 +5,7 @@ from scipy.sparse.csgraph import connected_components
|
||||
from scipy.spatial import cKDTree
|
||||
|
||||
from anyimage.operators.cutout_tool import geometry as g
|
||||
from anyimage.operators.cutout_tool.balloon import poisson_balloon_profile
|
||||
from anyimage.operators.cutout_tool import fine_mesh as fine
|
||||
|
||||
|
||||
@@ -42,7 +43,7 @@ def assert_supported(points, faces, boundary, components):
|
||||
).tocsr()
|
||||
count, _labels = connected_components(graph[interior][:, interior], directed=False)
|
||||
assert count == len(components)
|
||||
profile = g.poisson_balloon_profile(points, faces, boundary)
|
||||
profile = poisson_balloon_profile(points, faces, boundary)
|
||||
assert np.isfinite(profile).all()
|
||||
assert np.all(profile[interior] > 0)
|
||||
|
||||
|
||||
@@ -47,7 +47,7 @@ def test_cutout_node_groups_are_loaded_from_the_asset():
|
||||
for shape in cutout_shape.CUTOUT_NODE_GROUP_NAMES:
|
||||
assert cutout_object.load_cutout_node_group(shape) is loaded
|
||||
load.assert_called_with(cutout_shape.CUTOUT_NODE_GROUP_NAMES[shape],
|
||||
asset_revision=23 if shape == "DEPTH_SOLID" else 12 if shape == "DEPTH_BALLOON" else 11)
|
||||
asset_revision=26 if shape == "DEPTH_SOLID" else 14 if shape == "DEPTH_BALLOON" else 13)
|
||||
assert load.call_count == 4
|
||||
|
||||
|
||||
|
||||
@@ -1,5 +1,9 @@
|
||||
import json
|
||||
from types import SimpleNamespace
|
||||
from types import MethodType
|
||||
from unittest.mock import Mock
|
||||
|
||||
import pytest
|
||||
|
||||
|
||||
from anyimage.common.selection import SelectionPath
|
||||
@@ -21,8 +25,10 @@ class Pie:
|
||||
class WindowManager:
|
||||
def __init__(self):
|
||||
self.pie = Pie()
|
||||
self.events = []
|
||||
|
||||
def popup_menu_pie(self, _event, draw, **_options):
|
||||
self.events.append(_event)
|
||||
menu = SimpleNamespace(layout=SimpleNamespace(menu_pie=lambda: self.pie))
|
||||
draw(menu, None)
|
||||
|
||||
@@ -31,14 +37,12 @@ def cutout_context(
|
||||
window_manager,
|
||||
mesh_detail="LOW",
|
||||
generate_normal=False,
|
||||
inward_axis="NEGATIVE_Z",
|
||||
alpha_threshold=0.9,
|
||||
fine_outline=True,
|
||||
):
|
||||
settings = SimpleNamespace(
|
||||
cutout_mesh_detail=mesh_detail,
|
||||
cutout_generate_normal=generate_normal,
|
||||
cutout_inward_axis=inward_axis,
|
||||
cutout_alpha_threshold=alpha_threshold,
|
||||
cutout_fine_outline=fine_outline,
|
||||
)
|
||||
@@ -94,7 +98,6 @@ def test_cutout_pie_uses_two_local_shapes_without_ai(monkeypatch):
|
||||
)
|
||||
assert not any(call[2].generate_normal for call in window_manager.pie.calls)
|
||||
assert all(call[2].point_spacing == 32 for call in window_manager.pie.calls)
|
||||
assert all(call[2].inward_axis == "NEGATIVE_Z" for call in window_manager.pie.calls)
|
||||
|
||||
|
||||
def test_cutout_pie_passes_image_lasso_options_to_all_ai_shapes(monkeypatch):
|
||||
@@ -103,7 +106,6 @@ def test_cutout_pie_passes_image_lasso_options_to_all_ai_shapes(monkeypatch):
|
||||
window_manager,
|
||||
mesh_detail="MEDIUM",
|
||||
generate_normal=True,
|
||||
inward_axis="POSITIVE_X",
|
||||
alpha_threshold=0.1,
|
||||
fine_outline=False,
|
||||
)
|
||||
@@ -149,7 +151,6 @@ def test_cutout_pie_passes_image_lasso_options_to_all_ai_shapes(monkeypatch):
|
||||
for call in window_manager.pie.calls
|
||||
)
|
||||
assert all(call[2].point_spacing == 16 for call in window_manager.pie.calls)
|
||||
assert all(call[2].inward_axis == "POSITIVE_X" for call in window_manager.pie.calls)
|
||||
|
||||
|
||||
def test_cutout_normal_space_is_derived_from_shape():
|
||||
@@ -175,20 +176,19 @@ def test_cutout_lasso_submission_passes_only_the_selected_path(monkeypatch):
|
||||
monkeypatch.setattr(cutout_main, "require_image_empty", lambda _name: source)
|
||||
monkeypatch.setattr(interaction, "ai_ready", lambda: False)
|
||||
path = SelectionPath(points=((0, 0), (10, 0), (0, 10)))
|
||||
operator = SimpleNamespace(source_object_name=source.name, _pie_event=object())
|
||||
operator = SimpleNamespace(source_object_name=source.name, _pie_event=object(), gesture="LASSO")
|
||||
result = cutout_main.SelectCutoutSelection._submit_selection_path(operator, context, path)
|
||||
assert result == {"FINISHED"}
|
||||
assert SelectionPath.from_json(window_manager.pie.calls[0][2].selection_path_json) == path
|
||||
|
||||
|
||||
def test_cutout_tool_only_binds_lasso_drag_and_native_selection():
|
||||
def test_cutout_tool_starts_on_press_and_keeps_shift_selection():
|
||||
keymap = cutout_main.CutoutTool.bl_keymap
|
||||
assert keymap[0][1]["value"] == "CLICK_DRAG"
|
||||
assert [entry[0] for entry in keymap[1:]] == ["view3d.select", "view3d.select"]
|
||||
assert all(entry[1]["value"] != "DOUBLE_CLICK" for entry in keymap)
|
||||
assert "full_image" not in cutout_main.SelectCutoutSelection.__annotations__
|
||||
assert keymap[1][2] == {"properties": [("deselect_all", False)]}
|
||||
assert keymap[2][2] == {"properties": [("toggle", True)]}
|
||||
assert keymap[0][1]["value"] == "PRESS"
|
||||
assert len(keymap) == 2
|
||||
assert keymap[1][0] == "view3d.select"
|
||||
assert keymap[1][2] == {"properties": [("toggle", True)]}
|
||||
assert cutout_main.SelectCutoutSelection.resolve_on_press
|
||||
|
||||
|
||||
def test_cutout_f_has_no_option_shortcut(monkeypatch):
|
||||
@@ -209,7 +209,7 @@ def test_cutout_f_has_no_option_shortcut(monkeypatch):
|
||||
|
||||
def test_cutout_status_has_no_letter_shortcut():
|
||||
status = []
|
||||
operator = SimpleNamespace()
|
||||
operator = SimpleNamespace(gesture="LASSO")
|
||||
context = SimpleNamespace(
|
||||
workspace=SimpleNamespace(status_text_set=status.append)
|
||||
)
|
||||
@@ -220,3 +220,95 @@ def test_cutout_status_has_no_letter_shortcut():
|
||||
"Cutout Lasso: drag and release for shapes; RMB or Esc cancels"
|
||||
]
|
||||
assert "F" not in status[0]
|
||||
|
||||
|
||||
def test_cutout_menu_keeps_submitted_gesture_during_redraw(monkeypatch):
|
||||
window_manager = WindowManager()
|
||||
context = cutout_context(window_manager)
|
||||
monkeypatch.setattr(interaction, "ai_ready", lambda: True)
|
||||
|
||||
def delayed_popup(_event, draw, **_options):
|
||||
context.scene.anyimage_settings.cutout_gesture = "LASSO"
|
||||
draw(SimpleNamespace(layout=SimpleNamespace(menu_pie=lambda: window_manager.pie)), None)
|
||||
|
||||
window_manager.popup_menu_pie = delayed_popup
|
||||
interaction.open_shape_pie(
|
||||
context, object(), cutout_source(),
|
||||
SelectionPath(points=((0, 0), (10, 0), (0, 10))), gesture="POLYLINE",
|
||||
)
|
||||
assert all(call[2].gesture == "POLYLINE" for call in window_manager.pie.calls)
|
||||
|
||||
|
||||
@pytest.mark.parametrize("finish", ("START", "DOUBLE_CLICK", "RET", "NUMPAD_ENTER"))
|
||||
def test_cutout_polyline_edits_and_submits_through_shared_gesture(monkeypatch, finish):
|
||||
window_manager = WindowManager()
|
||||
context = cutout_context(window_manager)
|
||||
context.area = SimpleNamespace(tag_redraw=Mock())
|
||||
context.region = context.region_data = object()
|
||||
context.scene.anyimage_settings.cutout_gesture = "POLYLINE"
|
||||
source = cutout_source()
|
||||
monkeypatch.setattr(viewport, "active_view3d_tool_id", lambda _context: None)
|
||||
monkeypatch.setattr(viewport, "require_image_empty", lambda _name: source)
|
||||
monkeypatch.setattr(cutout_main, "require_image_empty", lambda _name: source)
|
||||
monkeypatch.setattr(viewport, "deserialize_matrix", lambda _data: None)
|
||||
monkeypatch.setattr(viewport, "image_empty_bounds", lambda _source: None)
|
||||
monkeypatch.setattr(viewport, "screen_path_to_image_pixels", lambda path, *_args: path)
|
||||
monkeypatch.setattr(interaction, "ai_ready", lambda: False)
|
||||
operator = SimpleNamespace(
|
||||
source_object_name=source.name, source_matrix_data="matrix",
|
||||
active_tool_id="anyimage.cutout_lasso", _path=[(0, 0)],
|
||||
_polyline_confirm_type=None,
|
||||
_finish=Mock(),
|
||||
)
|
||||
for name in ("_modal_polyline", "_confirm_polyline", "_complete", "_submit_selection_path"):
|
||||
setattr(operator, name, MethodType(getattr(cutout_main.SelectCutoutSelection, name), operator))
|
||||
cutout_main.SelectCutoutSelection._read_gesture_settings(operator, context)
|
||||
|
||||
def event(kind, value="PRESS", point=(100, 100)):
|
||||
return cutout_main.SelectCutoutSelection.modal(operator, context, SimpleNamespace(
|
||||
type=kind, value=value, mouse_region_x=point[0], mouse_region_y=point[1],
|
||||
))
|
||||
|
||||
assert event("RET") == {"RUNNING_MODAL"}
|
||||
event("LEFTMOUSE", point=(100, 0))
|
||||
event("LEFTMOUSE", point=(100, 100))
|
||||
event("BACK_SPACE")
|
||||
assert operator._path == [(0, 0), (100, 0)]
|
||||
event("LEFTMOUSE", point=(100, 100))
|
||||
if finish == "START":
|
||||
result = event("LEFTMOUSE", point=(0, 0))
|
||||
elif finish == "DOUBLE_CLICK":
|
||||
result = event("LEFTMOUSE", "DOUBLE_CLICK")
|
||||
else:
|
||||
result = event(finish)
|
||||
assert result == {"RUNNING_MODAL"}
|
||||
assert window_manager.events == []
|
||||
operator._finish.assert_not_called()
|
||||
points = tuple(operator._path)
|
||||
assert event("MOUSEMOVE", point=(200, 200)) == {"RUNNING_MODAL"}
|
||||
assert event("BACK_SPACE") == {"RUNNING_MODAL"}
|
||||
assert tuple(operator._path) == points
|
||||
release_type = "LEFTMOUSE" if finish in {"START", "DOUBLE_CLICK"} else finish
|
||||
assert event("SPACE", "RELEASE") == {"RUNNING_MODAL"}
|
||||
result = event(release_type, "RELEASE")
|
||||
assert result == {"FINISHED"}
|
||||
assert len(window_manager.events) == 1
|
||||
assert window_manager.events[0].value == "RELEASE"
|
||||
operator._finish.assert_called_once_with(context)
|
||||
submitted = window_manager.pie.calls[0][2]
|
||||
assert submitted.gesture == "POLYLINE"
|
||||
assert SelectionPath.from_json(submitted.selection_path_json).points == ((0, 0), (100, 0), (100, 100))
|
||||
|
||||
|
||||
@pytest.mark.parametrize("reason", ("ESC", "RIGHTMOUSE", "TOOL_SWITCH"))
|
||||
def test_cutout_polyline_cancel_cleans_preview(monkeypatch, reason):
|
||||
context = SimpleNamespace()
|
||||
operator = SimpleNamespace(
|
||||
gesture="POLYLINE", active_tool_id="anyimage.cutout_lasso", _finish=Mock(),
|
||||
_polyline_confirm_type="LEFTMOUSE",
|
||||
)
|
||||
operator._cancel_and_finish = MethodType(viewport.ImageGesture._cancel_and_finish, operator)
|
||||
monkeypatch.setattr(viewport, "active_view3d_tool_id", lambda _context: "other" if reason == "TOOL_SWITCH" else None)
|
||||
result = cutout_main.SelectCutoutSelection.modal(operator, context, SimpleNamespace(type=reason, value="PRESS"))
|
||||
assert result == {"CANCELLED"}
|
||||
operator._finish.assert_called_once_with(context)
|
||||
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user