Implement Balloon Depth Sampling and Validation Tools
- Added depth.py for MoGe-2 depth sampling and balloon deformation functionality. - Introduced validate_balloon_asset.py to validate balloon shape modifiers and material properties in Blender. - Enhanced test_server.py with new unit tests for balloon depth analysis and job progress tracking. - Updated existing tests to improve code clarity and maintainability.
This commit is contained in:
13
README.md
13
README.md
@@ -8,7 +8,7 @@ SnapForm 是一个 Blender 扩展,可以把一张照片、一段视频或一
|
|||||||
|
|
||||||
## 它能做什么
|
## 它能做什么
|
||||||
|
|
||||||
- 从 Image Empty 套索提取不规则区域,并通过 Mode 选择 Poisson 或 Teddy / Chordal Axis 生成封闭 Balloon 网格;源 Empty 保持不变
|
- 从 Image Empty 套索提取不规则区域,通过 Poisson 或 Teddy / Chordal Axis 生成封闭 Balloon,并可用 BEN2 分析轮廓、MoGe-2 添加可调深度;源 Empty 保持不变
|
||||||
- 一键把 Image Empty 转换为立体面片:让 AI 估算深度,或直接用原图作为深度
|
- 一键把 Image Empty 转换为立体面片:让 AI 估算深度,或直接用原图作为深度
|
||||||
- 把单张 Image Empty 转换为透视纹理网格,并自动匹配原对象的 Transform 与 Offset
|
- 把单张 Image Empty 转换为透视纹理网格,并自动匹配原对象的 Transform 与 Offset
|
||||||
- 一键去除图片背景,生成透明前景图
|
- 一键去除图片背景,生成透明前景图
|
||||||
@@ -33,7 +33,7 @@ Environment Bootstrap 使用连续的 0–100%:uv 为 1–20%、Python 为 20
|
|||||||
时显示 `Download Required Models`,一次补齐缺失项,不会重复下载已经就绪的模型。
|
时显示 `Download Required Models`,一次补齐缺失项,不会重复下载已经就绪的模型。
|
||||||
Environment 未安装时,同一位置显示两个安装按钮。Environment 或必需模型未就绪时,
|
Environment 未安装时,同一位置显示两个安装按钮。Environment 或必需模型未就绪时,
|
||||||
菜单仍显示 `Convert to Depth Plane`、`Convert to Surface` 和 `Remove Background`,但三个入口都会保持禁用。
|
菜单仍显示 `Convert to Depth Plane`、`Convert to Surface` 和 `Remove Background`,但三个入口都会保持禁用。
|
||||||
`Balloon Mesh (Lasso)` 默认是 Blender 本地几何功能,在上述状态下仍可使用;只有开启 `AI Refine` 时才需要 Environment 与 BEN2。
|
`Balloon Mesh (Lasso)` 默认是 Blender 本地几何功能,在上述状态下仍可使用;开启 `Contour` 或 `Depth` 后,才需要 Environment 与对应的 BEN2 或 MoGe-2 模型。
|
||||||
这些 UI 状态直接检查 Storage Root 的 `models/` 文件,不会仅因打开菜单或面板而启动 Job Server。
|
这些 UI 状态直接检查 Storage Root 的 `models/` 文件,不会仅因打开菜单或面板而启动 Job Server。
|
||||||
|
|
||||||
## 快速上手
|
## 快速上手
|
||||||
@@ -56,11 +56,16 @@ Environment 未安装时,同一位置显示两个安装按钮。Environment
|
|||||||
|
|
||||||
按住左键沿目标轮廓绘制,松开后生成一个带原图 UV 的封闭网格。每次拖动都是一个独立完成的顶层 `Extract Balloon` Operator,工具本身继续保持激活,下一次拖动会创建另一个 Balloon;因此 `Ctrl+Z` 只撤销最近一次生成,`Ctrl+Shift+Z` 重做。每次生成后源 Image Empty 都保持选中和 Active。未绘制时没有持续 Modal 拦截事件,视图导航和其他原生快捷键由 Blender 正常处理;绘制中可用右键或 `Esc` 取消当前轨迹。选择左侧工具栏中的其他工具即可离开 Balloon Lasso。
|
按住左键沿目标轮廓绘制,松开后生成一个带原图 UV 的封闭网格。每次拖动都是一个独立完成的顶层 `Extract Balloon` Operator,工具本身继续保持激活,下一次拖动会创建另一个 Balloon;因此 `Ctrl+Z` 只撤销最近一次生成,`Ctrl+Shift+Z` 重做。每次生成后源 Image Empty 都保持选中和 Active。未绘制时没有持续 Modal 拦截事件,视图导航和其他原生快捷键由 Blender 正常处理;绘制中可用右键或 `Esc` 取消当前轨迹。选择左侧工具栏中的其他工具即可离开 Balloon Lasso。
|
||||||
|
|
||||||
- `Mode`、`Thickness`、`Resolution (m)`、`Reuse Material` 和 `AI Refine` 显示在 Tool Header / Tool Settings
|
- `Mode`、`Thickness`、`Resolution (m)`、`Reuse Material`、`Contour`、`Depth` 和启用 `Depth` 后出现的 `Depth Model` 显示在 Tool Header / Tool Settings
|
||||||
- `S`:调用 Blender 原生 Radial Control 调节 Thickness;通过 `zoom_path` 把默认 `1.0` 映射为约 100 px 的交互圆,软上限为 `5.0`,不设硬上限
|
- `S`:调用 Blender 原生 Radial Control 调节 Thickness;通过 `zoom_path` 把默认 `1.0` 映射为约 100 px 的交互圆,软上限为 `5.0`,不设硬上限
|
||||||
- `D`:调用 Blender 原生 Radial Control 调节 Resolution;通过独立 `zoom_path` 把默认 `0.1 m` 映射为约 100 px 的交互圆,数值越小网格越密
|
- `D`:调用 Blender 原生 Radial Control 调节 Resolution;通过独立 `zoom_path` 把默认 `0.1 m` 映射为约 100 px 的交互圆,数值越小网格越密
|
||||||
|
|
||||||
`Resolution (m)` 会根据 Scene Unit Scale 换算为网格使用的世界距离,因此不同单位设置下仍代表相同的物理采样间距。两种模式共用轮廓平滑、`.color` 图片副本和 `O Texture Layer` 材质。开启 `Reuse Material` 后,连续创建会按图片基名复用同一张 `.color` 与兼容的 Balloon 材质;关闭后每次新建。Balloon、Surface 与 Depth Plane 的材质名均取自图片数据名,而不是 Image Empty 对象名。`AI Refine` 默认关闭,此时 Balloon 完全在 Blender 本地生成,不需要 Environment;开启后会把套索外接范围裁切为图片,并将不规则选区外的 RGB 与 Alpha 全部清零后送入 BEN2。BEN2 Alpha 严格大于 `0.5` 的最大连通前景会成为新的套索轮廓,再生成 Balloon。当前只支持静态图片,不处理 Movie 或 Image Sequence。
|
`Resolution (m)` 会根据 Scene Unit Scale 换算为网格使用的世界距离,因此不同单位设置下仍代表相同的物理采样间距。Poisson 模式使用尺度无关的求解判断,支持 `0.02 m` 等较密采样。两个 AI 选项默认关闭;`Contour` 使用 BEN2 Alpha 严格大于 `0.5` 的最大连通前景重建轮廓,`Depth` 使用所选 MoGe-2 模型。MoGe 的物理距离会先减去稳健拟合的基面,再换算为不受绝对拍摄距离影响的相对起伏;近侧与远侧信号都保留,不削顶、不截平。
|
||||||
|
|
||||||
|
Depth Balloon 使用局部 X 作为深度轴,并创建 `<Balloon> Base Plane` Single Arrow Empty。Empty 的旋转定义基面朝向,沿自身局部 Z 移动则调整 middle level。默认深度校准为 `修正深度 = (0.5 × 相对基面深度 + 原始 Balloon 最大半厚度) × Depth Strength`;原始最大半厚度来自生成后的固定网格,不受修改器 `Thickness` 影响。这在 `Thickness = 1` 时等价于旧参数的 `Depth Strength = 0.5` 和两倍半厚度基面偏移。节点再以 Smoother Step 中心程度插值修正厚度与修正深度;`Thickness` 只控制厚度分支,`Depth Strength` 只控制深度分支。`Double Sided Depth` 默认关闭,使背面维持稳定厚度。
|
||||||
|
|
||||||
|
MoGe normal 会转换成适配 Balloon UV 的 tangent-space `normal.png`,由专用 `O Balloon Layer` 材质补回网格分辨率未能表达的细节;`Normal Strength` 默认 `0.5`,可在材质节点中调整。Balloon 材质通过 Custom Properties 直接保存源 Image datablock 引用,不依赖材质名或节点组名识别;勾选 `Reuse Material` 时优先复用同一 Image、同一材质类型的标记材质。Depth Balloon 共用一张原图尺寸的 normal,后续套索会把自己的有效区域合并进去,重叠区域采用最新结果。旧材质仍保留节点识别回退。当前只支持静态图片。
|
||||||
|
|
||||||
### 去除背景
|
### 去除背景
|
||||||
|
|
||||||
选中 Image Empty 后,右键打开 `SnapForm` 子菜单并选择 `Remove Background`,完成后 Empty 会自动
|
选中 Image Empty 后,右键打开 `SnapForm` 子菜单并选择 `Remove Background`,完成后 Empty 会自动
|
||||||
|
|||||||
@@ -106,6 +106,7 @@ BLENDER_BIN="/c/Program Files/Blender Foundation/Blender 4.5/blender.exe"
|
|||||||
"$BLENDER_BIN" --background --python tools/build_geometry_nodes_asset.py
|
"$BLENDER_BIN" --background --python tools/build_geometry_nodes_asset.py
|
||||||
"$BLENDER_BIN" --background --python tools/validate_depth_plane_asset.py
|
"$BLENDER_BIN" --background --python tools/validate_depth_plane_asset.py
|
||||||
"$BLENDER_BIN" --background --python tools/validate_geometry_nodes_asset.py
|
"$BLENDER_BIN" --background --python tools/validate_geometry_nodes_asset.py
|
||||||
|
"$BLENDER_BIN" --background --python tools/validate_balloon_asset.py
|
||||||
"$BLENDER_BIN" --background --python tools/validate_moge_depth_plane.py
|
"$BLENDER_BIN" --background --python tools/validate_moge_depth_plane.py
|
||||||
"$BLENDER_BIN" --background --python tools/validate_moge_surface.py
|
"$BLENDER_BIN" --background --python tools/validate_moge_surface.py
|
||||||
```
|
```
|
||||||
|
|||||||
@@ -20,7 +20,11 @@ Depth Plane 与 Surface 可使用 DA3 或 MoGe-2。DA3 提供相对深度和相
|
|||||||
|
|
||||||
转换任务完成后,SnapForm 在一个 Undo 步骤中创建网格、图片、材质和修改器。源 Image Empty 是否保留由转换选项决定。
|
转换任务完成后,SnapForm 在一个 Undo 步骤中创建网格、图片、材质和修改器。源 Image Empty 是否保留由转换选项决定。
|
||||||
|
|
||||||
Balloon 默认在 Blender 内部生成,无需 Environment 或模型。它支持连续绘制,每次 Stroke 都生成一个可单独 Undo/Redo 的结果。Tool Settings 提供模式、厚度、采样距离、材质复用与 `AI Refine`。开启 `AI Refine` 后,Server 会裁切套索外接范围,把不规则选区外的 RGB 与 Alpha 清零,再用 BEN2 Alpha 严格大于 `0.5` 的最大连通前景重建轮廓;此模式需要 Environment 与 BEN2。
|
Balloon 默认在 Blender 内部生成,无需 Environment 或模型。Tool Settings 提供模式、厚度、采样距离、材质复用、`Contour`、`Depth` 与 MoGe-2 `Depth Model`。Depth 会把物理距离相对化并稳健拟合初始基面,保留未截断的正负起伏。Base Plane 子 Empty 的旋转定义基面朝向,沿自身局部 Z 移动定义 middle level。
|
||||||
|
|
||||||
|
`O Balloon Shape` 计算 `修正厚度 = abs(原 X) × Thickness`,并将修正深度校准为 `(0.5 × 相对基面深度 + 原始 Balloon 最大半厚度) × Depth Strength`。固定的原始半厚度不乘 `Thickness`,因此 Thickness 与 Depth Strength 分别只控制各自分支。节点以归一化原 X 距离经 Smoother Step 得到中心程度,再插值厚度与深度。`Double Sided Depth` 默认关闭,背面保持 Balloon 厚度。MoGe normal 会输出为原图尺寸的 tangent-space 纹理,并通过 `O Balloon Layer` 的 `Normal Strength`(默认 `0.5`)补偿网格无法表达的细节。
|
||||||
|
|
||||||
|
新建 Balloon 材质会用 Custom Properties 保存源 Image datablock 和可选 normal Image 的直接引用。`Reuse Material` 优先按这些标记匹配,不依赖材质或节点组名称;普通与 Depth 材质分别复用。Depth Balloon 的共享 normal 会逐次合并新套索的有效区域,避免复用材质时丢失其他套索的法线。
|
||||||
|
|
||||||
## 动画
|
## 动画
|
||||||
|
|
||||||
|
|||||||
@@ -9,7 +9,7 @@
|
|||||||
| `generate-depth` | `results.npz`、深度与颜色序列 | Depth Plane 所需深度、相机和贴图 |
|
| `generate-depth` | `results.npz`、深度与颜色序列 | Depth Plane 所需深度、相机和贴图 |
|
||||||
| `generate-surface` | `mesh.npz`、`normal.png`、颜色层 | Surface 网格、相机与材质输入 |
|
| `generate-surface` | `mesh.npz`、`normal.png`、颜色层 | Surface 网格、相机与材质输入 |
|
||||||
| `remove-background` | `color.0001.png` 起 | 带 Alpha 的颜色层 |
|
| `remove-background` | `color.0001.png` 起 | 带 Alpha 的颜色层 |
|
||||||
| `refine-balloon` | `lasso-input.png`、`foreground.png`、`contour.json` | 套索约束输入、BEN2 前景与重建轮廓 |
|
| `analyze-balloon` | `lasso-input.png`,按选项输出 `foreground.png`、`contour.json`、`depth.npz`、`normal.png` | 套索约束输入、BEN2 轮廓与 MoGe-2 深度、法线 |
|
||||||
| `debug-da3` | `prediction.npz` | DA3 API 返回 |
|
| `debug-da3` | `prediction.npz` | DA3 API 返回 |
|
||||||
| `debug-moge2` | `prediction*.npz` | MoGe-2 API 返回 |
|
| `debug-moge2` | `prediction*.npz` | MoGe-2 API 返回 |
|
||||||
| `debug-ben2` | `foreground*.png` | BEN2 API 返回 |
|
| `debug-ben2` | `foreground*.png` | BEN2 API 返回 |
|
||||||
@@ -18,6 +18,8 @@
|
|||||||
|
|
||||||
DA3 Depth 结果包含 `depth`、`conf`、`extrinsics` 与 `intrinsics`。MoGe-2 Depth 结果包含 `depth`、`normal`、`mask` 与 `intrinsics`。
|
DA3 Depth 结果包含 `depth`、`conf`、`extrinsics` 与 `intrinsics`。MoGe-2 Depth 结果包含 `depth`、`normal`、`mask` 与 `intrinsics`。
|
||||||
|
|
||||||
|
Balloon Depth 的 `depth.npz` 包含相对 `depth_signal`、归一化拟合平面 `baseline_vector`、`valid`、`reference_depth`、裁切原点与输入尺寸。Geometry Nodes 将归一化原 X 距离经 Smoother Step 后作为 Thickness 与 Depth 的混合系数。`normal.png` 是铺回原图尺寸的 tangent-space 法线;套索和有效区域外为中性法线。
|
||||||
|
|
||||||
Surface 的 `mesh.npz` 包含 `vertices`、`faces`、`uv`、`intrinsics`、`image_size` 与 `reference_depth`,以及 Geometry Nodes 使用的网格属性。
|
Surface 的 `mesh.npz` 包含 `vertices`、`faces`、`uv`、`intrinsics`、`image_size` 与 `reference_depth`,以及 Geometry Nodes 使用的网格属性。
|
||||||
|
|
||||||
Debug 文件保持模型 API 的数据形态,用于比较推理结果;正式 Job 则提供 Blender 导入所需的派生数据。
|
Debug 文件保持模型 API 的数据形态,用于比较推理结果;正式 Job 则提供 Blender 导入所需的派生数据。
|
||||||
|
|||||||
@@ -57,7 +57,7 @@ BlendJob 提供 Runtime、HTTP transport、队列、状态和取消协议。Snap
|
|||||||
|
|
||||||
Environment 使用 Python 3.12、ONNX Runtime、NumPy、Pillow、PyAV、trimesh、FastAPI 和 Uvicorn。Windows 使用 DirectML 或 CPU;其他平台根据支持情况使用 CoreML、OpenVINO 或 CPU。
|
Environment 使用 Python 3.12、ONNX Runtime、NumPy、Pillow、PyAV、trimesh、FastAPI 和 Uvicorn。Windows 使用 DirectML 或 CPU;其他平台根据支持情况使用 CoreML、OpenVINO 或 CPU。
|
||||||
|
|
||||||
Balloon 的 `refine-balloon` Job 会在 BEN2 推理前裁切套索外接范围,并将不规则选区外的 RGBA 清零;推理后只回传 Alpha 大于 `0.5` 的最大连通轮廓,由 Blender 主线程生成网格。
|
Balloon 的 `analyze-balloon` Job 会裁切套索输入并清零选区外 RGBA。`Contour` 用 BEN2 回传最大连通轮廓;`Depth` 用 MoGe-2 对有效相机距离稳健拟合二维基面,并把物理距离换算为不受绝对拍摄距离影响、未截断的相对深度信号。Job 同时把 MoGe normal 转换为 tangent-space 纹理,铺回原图尺寸并以中性法线填充无效区域。Blender 主线程负责采样深度属性、创建 Base Plane Empty、组装 `O Balloon Shape` 修改器与 `O Balloon Layer` 材质。
|
||||||
|
|
||||||
正式功能支持:
|
正式功能支持:
|
||||||
|
|
||||||
|
|||||||
@@ -2,7 +2,7 @@ from .background import ProcessImageBackground, RemoveImageBackground
|
|||||||
from .balloon import (
|
from .balloon import (
|
||||||
ActivateBalloonTool,
|
ActivateBalloonTool,
|
||||||
ExtractBalloon,
|
ExtractBalloon,
|
||||||
RefineBalloon,
|
AnalyzeBalloon,
|
||||||
)
|
)
|
||||||
from .debug import DebugBen2, DebugDa3, DebugMoge2
|
from .debug import DebugBen2, DebugDa3, DebugMoge2
|
||||||
from .depth_plane import (
|
from .depth_plane import (
|
||||||
@@ -31,7 +31,7 @@ CLASSES = (
|
|||||||
ExtractOriginalFrames,
|
ExtractOriginalFrames,
|
||||||
ClearModels,
|
ClearModels,
|
||||||
ExtractBalloon,
|
ExtractBalloon,
|
||||||
RefineBalloon,
|
AnalyzeBalloon,
|
||||||
ActivateBalloonTool,
|
ActivateBalloonTool,
|
||||||
RemoveImageBackground,
|
RemoveImageBackground,
|
||||||
ConvertToDepthPlane,
|
ConvertToDepthPlane,
|
||||||
|
|||||||
@@ -2,5 +2,5 @@ from .operator import (
|
|||||||
ActivateBalloonTool,
|
ActivateBalloonTool,
|
||||||
BalloonTool,
|
BalloonTool,
|
||||||
ExtractBalloon,
|
ExtractBalloon,
|
||||||
RefineBalloon,
|
AnalyzeBalloon,
|
||||||
)
|
)
|
||||||
|
|||||||
622
src/snapform/operators/balloon/depth.py
Normal file
622
src/snapform/operators/balloon/depth.py
Normal file
@@ -0,0 +1,622 @@
|
|||||||
|
"""MoGe-2 depth sampling and adjustable Balloon deformation."""
|
||||||
|
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
import bpy
|
||||||
|
|
||||||
|
|
||||||
|
BALLOON_DEPTH_ATTRIBUTE = "o_balloon_depth"
|
||||||
|
BALLOON_SHAPE_NODE_GROUP_NAME = "O Balloon Shape"
|
||||||
|
BALLOON_SHAPE_NODE_GROUP_VERSION = 16
|
||||||
|
BALLOON_SHAPE_VERSION_PROPERTY = "snapform_balloon_shape_version"
|
||||||
|
|
||||||
|
|
||||||
|
def load_balloon_depth_data(path):
|
||||||
|
import numpy as np
|
||||||
|
|
||||||
|
try:
|
||||||
|
with np.load(Path(path), allow_pickle=False) as archive:
|
||||||
|
required = {
|
||||||
|
"depth_signal",
|
||||||
|
"baseline_vector",
|
||||||
|
"valid",
|
||||||
|
"origin",
|
||||||
|
"reference_depth",
|
||||||
|
"crop_size",
|
||||||
|
"source_size",
|
||||||
|
}
|
||||||
|
missing = required.difference(archive.files)
|
||||||
|
if missing:
|
||||||
|
raise ValueError(
|
||||||
|
f"Balloon depth data is missing: {', '.join(sorted(missing))}"
|
||||||
|
)
|
||||||
|
data = {
|
||||||
|
"depth_signal": np.asarray(
|
||||||
|
archive["depth_signal"],
|
||||||
|
dtype=np.float32,
|
||||||
|
),
|
||||||
|
"baseline_vector": np.asarray(
|
||||||
|
archive["baseline_vector"],
|
||||||
|
dtype=np.float32,
|
||||||
|
),
|
||||||
|
"valid": np.asarray(archive["valid"], dtype=bool),
|
||||||
|
"origin": tuple(int(value) for value in archive["origin"]),
|
||||||
|
"reference_depth": float(archive["reference_depth"]),
|
||||||
|
"crop_size": tuple(int(value) for value in archive["crop_size"]),
|
||||||
|
"source_size": tuple(int(value) for value in archive["source_size"]),
|
||||||
|
}
|
||||||
|
except OSError as error:
|
||||||
|
raise ValueError(f"Unable to read Balloon depth data: {path}") from error
|
||||||
|
if data["depth_signal"].ndim != 2:
|
||||||
|
raise ValueError("Balloon depth signal must be two-dimensional")
|
||||||
|
if data["valid"].shape != data["depth_signal"].shape:
|
||||||
|
raise ValueError("Balloon depth mask dimensions do not match")
|
||||||
|
if data["baseline_vector"].shape != (3,):
|
||||||
|
raise ValueError("Balloon base plane vector must have three components")
|
||||||
|
if any(value <= 0 for value in (*data["crop_size"], *data["source_size"])):
|
||||||
|
raise ValueError("Balloon depth image dimensions must be positive")
|
||||||
|
return data
|
||||||
|
|
||||||
|
|
||||||
|
def _sample_valid_bilinear(values, valid, x, y):
|
||||||
|
import numpy as np
|
||||||
|
|
||||||
|
height, width = values.shape
|
||||||
|
x = np.clip(x, 0.0, max(width - 1, 0))
|
||||||
|
y = np.clip(y, 0.0, max(height - 1, 0))
|
||||||
|
x0 = np.floor(x).astype(np.int64)
|
||||||
|
y0 = np.floor(y).astype(np.int64)
|
||||||
|
x1 = np.minimum(x0 + 1, width - 1)
|
||||||
|
y1 = np.minimum(y0 + 1, height - 1)
|
||||||
|
fx = x - x0
|
||||||
|
fy = y - y0
|
||||||
|
samples = np.stack(
|
||||||
|
(
|
||||||
|
values[y0, x0],
|
||||||
|
values[y0, x1],
|
||||||
|
values[y1, x0],
|
||||||
|
values[y1, x1],
|
||||||
|
),
|
||||||
|
axis=1,
|
||||||
|
)
|
||||||
|
weights = np.stack(
|
||||||
|
(
|
||||||
|
(1.0 - fx) * (1.0 - fy),
|
||||||
|
fx * (1.0 - fy),
|
||||||
|
(1.0 - fx) * fy,
|
||||||
|
fx * fy,
|
||||||
|
),
|
||||||
|
axis=1,
|
||||||
|
)
|
||||||
|
sample_valid = np.stack(
|
||||||
|
(
|
||||||
|
valid[y0, x0],
|
||||||
|
valid[y0, x1],
|
||||||
|
valid[y1, x0],
|
||||||
|
valid[y1, x1],
|
||||||
|
),
|
||||||
|
axis=1,
|
||||||
|
)
|
||||||
|
weights *= sample_valid
|
||||||
|
totals = weights.sum(axis=1)
|
||||||
|
result = np.zeros(len(x), dtype=np.float32)
|
||||||
|
usable = totals > 1e-8
|
||||||
|
result[usable] = (samples[usable] * weights[usable]).sum(axis=1) / totals[usable]
|
||||||
|
return result, usable
|
||||||
|
|
||||||
|
|
||||||
|
def balloon_depth_fields(vertices, vertex_uv, depth_data):
|
||||||
|
"""Sample camera depth and convert the fitted base plane to local space."""
|
||||||
|
import numpy as np
|
||||||
|
|
||||||
|
vertices = np.asarray(vertices, dtype=np.float32)
|
||||||
|
uv = np.asarray(vertex_uv, dtype=np.float32)
|
||||||
|
if vertices.ndim != 2 or vertices.shape[1] != 3:
|
||||||
|
raise ValueError("Balloon vertices must have three components")
|
||||||
|
if uv.shape != (len(vertices), 2):
|
||||||
|
raise ValueError("Balloon UV coordinates do not match its vertices")
|
||||||
|
depth_signal = np.asarray(depth_data["depth_signal"], dtype=np.float32)
|
||||||
|
valid = np.asarray(depth_data["valid"], dtype=bool)
|
||||||
|
origin_x, origin_y = depth_data["origin"]
|
||||||
|
crop_width, crop_height = depth_data["crop_size"]
|
||||||
|
source_width, source_height = depth_data["source_size"]
|
||||||
|
depth_height, depth_width = depth_signal.shape
|
||||||
|
|
||||||
|
crop_x = uv[:, 0] * source_width - origin_x
|
||||||
|
crop_y = (1.0 - uv[:, 1]) * source_height - origin_y
|
||||||
|
sample_x = crop_x * max(depth_width - 1, 0) / max(crop_width - 1, 1)
|
||||||
|
sample_y = crop_y * max(depth_height - 1, 0) / max(crop_height - 1, 1)
|
||||||
|
sampled, usable = _sample_valid_bilinear(
|
||||||
|
depth_signal,
|
||||||
|
valid,
|
||||||
|
sample_x,
|
||||||
|
sample_y,
|
||||||
|
)
|
||||||
|
inside_crop = (
|
||||||
|
(crop_x >= 0.0)
|
||||||
|
& (crop_x <= crop_width)
|
||||||
|
& (crop_y >= 0.0)
|
||||||
|
& (crop_y <= crop_height)
|
||||||
|
)
|
||||||
|
usable &= inside_crop
|
||||||
|
sampled[~usable] = 0.0
|
||||||
|
coordinates = np.stack(
|
||||||
|
(
|
||||||
|
sample_x / max(depth_width - 1, 1) * 2.0 - 1.0,
|
||||||
|
sample_y / max(depth_height - 1, 1) * 2.0 - 1.0,
|
||||||
|
np.ones(len(vertices), dtype=np.float32),
|
||||||
|
),
|
||||||
|
axis=1,
|
||||||
|
).astype(np.float32)
|
||||||
|
|
||||||
|
base_height = float(np.max(np.abs(vertices[:, 2])))
|
||||||
|
if not base_height > 0.0:
|
||||||
|
return np.zeros(len(vertices), dtype=np.float32), np.zeros(3, dtype=np.float32)
|
||||||
|
depth_scale = base_height
|
||||||
|
normalized_plane = np.asarray(
|
||||||
|
depth_data["baseline_vector"],
|
||||||
|
dtype=np.float32,
|
||||||
|
)
|
||||||
|
baseline_values = coordinates @ normalized_plane * depth_scale
|
||||||
|
local_design = np.stack(
|
||||||
|
(
|
||||||
|
vertices[:, 0],
|
||||||
|
vertices[:, 1],
|
||||||
|
np.ones(len(vertices), dtype=np.float32),
|
||||||
|
),
|
||||||
|
axis=1,
|
||||||
|
)
|
||||||
|
if np.count_nonzero(usable) >= 3:
|
||||||
|
local_plane = np.linalg.lstsq(
|
||||||
|
local_design[usable],
|
||||||
|
baseline_values[usable],
|
||||||
|
rcond=None,
|
||||||
|
)[0].astype(np.float32)
|
||||||
|
else:
|
||||||
|
local_plane = np.zeros(3, dtype=np.float32)
|
||||||
|
depth_values = sampled * depth_scale
|
||||||
|
if usable.any():
|
||||||
|
plane_without_offset = local_design[:, :2] @ local_plane[:2]
|
||||||
|
residual = depth_values - plane_without_offset
|
||||||
|
depth_values[usable] -= float(np.median(residual[usable]))
|
||||||
|
local_plane[2] = 0.0
|
||||||
|
return depth_values, local_plane
|
||||||
|
|
||||||
|
|
||||||
|
def _assign_point_attribute(mesh, name, values):
|
||||||
|
import numpy as np
|
||||||
|
|
||||||
|
values = np.asarray(values, dtype=np.float32)
|
||||||
|
if len(values) != len(mesh.vertices):
|
||||||
|
raise ValueError(f"Balloon attribute {name} does not match its vertices")
|
||||||
|
attribute = mesh.attributes.new(name=name, type="FLOAT", domain="POINT")
|
||||||
|
attribute.data.foreach_set("value", values)
|
||||||
|
|
||||||
|
|
||||||
|
def assign_balloon_shape_attributes(mesh, _vertices, depth_values=None):
|
||||||
|
if depth_values is not None:
|
||||||
|
_assign_point_attribute(mesh, BALLOON_DEPTH_ATTRIBUTE, depth_values)
|
||||||
|
|
||||||
|
|
||||||
|
def create_balloon_base_plane(context, balloon, vertices, local_plane):
|
||||||
|
import numpy as np
|
||||||
|
from mathutils import Vector
|
||||||
|
|
||||||
|
vertices = np.asarray(vertices, dtype=np.float32)
|
||||||
|
slope_y, slope_z, offset = (float(value) for value in local_plane)
|
||||||
|
extent = vertices[:, 1:].max(axis=0) - vertices[:, 1:].min(axis=0)
|
||||||
|
normal = Vector((1.0, -slope_y, -slope_z)).normalized()
|
||||||
|
|
||||||
|
controller = bpy.data.objects.new(f"{balloon.name} Base Plane", None)
|
||||||
|
context.collection.objects.link(controller)
|
||||||
|
controller.parent = balloon
|
||||||
|
controller.location = (0.0, 0.0, 0.0)
|
||||||
|
controller.rotation_euler = normal.to_track_quat("Z", "Y").to_euler()
|
||||||
|
controller.empty_display_type = "SINGLE_ARROW"
|
||||||
|
controller.empty_display_size = max(float(extent.max()) * 0.15, 0.01)
|
||||||
|
controller.show_in_front = True
|
||||||
|
controller.hide_render = True
|
||||||
|
controller["snapform_source_type"] = "BALLOON_BASE_PLANE"
|
||||||
|
controller["snapform_base_plane"] = (slope_y, slope_z, offset)
|
||||||
|
return controller
|
||||||
|
|
||||||
|
|
||||||
|
def _interface_socket(group, name, in_out, socket_type):
|
||||||
|
for item in group.interface.items_tree:
|
||||||
|
if item.item_type == "SOCKET" and item.name == name and item.in_out == in_out:
|
||||||
|
return item
|
||||||
|
return group.interface.new_socket(
|
||||||
|
name=name,
|
||||||
|
in_out=in_out,
|
||||||
|
socket_type=socket_type,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def _remove_obsolete_interface_sockets(group):
|
||||||
|
obsolete = {"Base X", "Base Y", "Base Offset", "Far Clamp"}
|
||||||
|
for item in list(group.interface.items_tree):
|
||||||
|
if (
|
||||||
|
item.item_type == "SOCKET"
|
||||||
|
and item.in_out == "INPUT"
|
||||||
|
and item.name in obsolete
|
||||||
|
):
|
||||||
|
group.interface.remove(item)
|
||||||
|
|
||||||
|
|
||||||
|
def _float_input(group, name, default, minimum, maximum, description):
|
||||||
|
socket = _interface_socket(group, name, "INPUT", "NodeSocketFloat")
|
||||||
|
socket.default_value = default
|
||||||
|
socket.min_value = minimum
|
||||||
|
socket.max_value = maximum
|
||||||
|
socket.description = description
|
||||||
|
if hasattr(socket, "structure_type"):
|
||||||
|
socket.structure_type = "SINGLE"
|
||||||
|
return socket
|
||||||
|
|
||||||
|
|
||||||
|
def _bool_input(group, name, default, description):
|
||||||
|
socket = _interface_socket(group, name, "INPUT", "NodeSocketBool")
|
||||||
|
socket.default_value = default
|
||||||
|
socket.description = description
|
||||||
|
if hasattr(socket, "structure_type"):
|
||||||
|
socket.structure_type = "SINGLE"
|
||||||
|
return socket
|
||||||
|
|
||||||
|
|
||||||
|
def balloon_shape_node_group():
|
||||||
|
group = bpy.data.node_groups.get(BALLOON_SHAPE_NODE_GROUP_NAME)
|
||||||
|
if (
|
||||||
|
group is not None
|
||||||
|
and group.get(BALLOON_SHAPE_VERSION_PROPERTY)
|
||||||
|
== BALLOON_SHAPE_NODE_GROUP_VERSION
|
||||||
|
):
|
||||||
|
return group
|
||||||
|
if group is None:
|
||||||
|
group = bpy.data.node_groups.new(
|
||||||
|
BALLOON_SHAPE_NODE_GROUP_NAME,
|
||||||
|
"GeometryNodeTree",
|
||||||
|
)
|
||||||
|
else:
|
||||||
|
group.nodes.clear()
|
||||||
|
_remove_obsolete_interface_sockets(group)
|
||||||
|
|
||||||
|
group.is_modifier = True
|
||||||
|
geometry_input = _interface_socket(
|
||||||
|
group,
|
||||||
|
"Geometry",
|
||||||
|
"INPUT",
|
||||||
|
"NodeSocketGeometry",
|
||||||
|
)
|
||||||
|
if hasattr(geometry_input, "structure_type"):
|
||||||
|
geometry_input.structure_type = "SINGLE"
|
||||||
|
_float_input(
|
||||||
|
group,
|
||||||
|
"Thickness",
|
||||||
|
1.0,
|
||||||
|
0.0,
|
||||||
|
10.0,
|
||||||
|
"Scale the symmetric base Balloon thickness along local X.",
|
||||||
|
)
|
||||||
|
_float_input(
|
||||||
|
group,
|
||||||
|
"Depth Strength",
|
||||||
|
1.0,
|
||||||
|
0.0,
|
||||||
|
10.0,
|
||||||
|
"Scale the signed MoGe-2 depth variation.",
|
||||||
|
)
|
||||||
|
_bool_input(
|
||||||
|
group,
|
||||||
|
"Double Sided Depth",
|
||||||
|
False,
|
||||||
|
"Apply AI depth to both sides; keep disabled for a stable Balloon back.",
|
||||||
|
)
|
||||||
|
base_plane_input = _interface_socket(
|
||||||
|
group,
|
||||||
|
"Base Plane",
|
||||||
|
"INPUT",
|
||||||
|
"NodeSocketObject",
|
||||||
|
)
|
||||||
|
base_plane_input.description = (
|
||||||
|
"Rotate the child Empty for middle-plane orientation; move it along local Z "
|
||||||
|
"to set the neutral depth level."
|
||||||
|
)
|
||||||
|
_interface_socket(group, "Geometry", "OUTPUT", "NodeSocketGeometry")
|
||||||
|
|
||||||
|
nodes = group.nodes
|
||||||
|
links = group.links
|
||||||
|
group_input = nodes.new("NodeGroupInput")
|
||||||
|
group_input.location = (-1500.0, 500.0)
|
||||||
|
position = nodes.new("GeometryNodeInputPosition")
|
||||||
|
position.location = (-1500.0, 200.0)
|
||||||
|
separate = nodes.new("ShaderNodeSeparateXYZ")
|
||||||
|
separate.location = (-1300.0, 200.0)
|
||||||
|
base_scale = nodes.new("ShaderNodeMath")
|
||||||
|
base_scale.operation = "MULTIPLY"
|
||||||
|
base_scale.location = (-1050.0, 350.0)
|
||||||
|
base_magnitude = nodes.new("ShaderNodeMath")
|
||||||
|
base_magnitude.operation = "ABSOLUTE"
|
||||||
|
base_magnitude.location = (-800.0, 350.0)
|
||||||
|
|
||||||
|
side = nodes.new("ShaderNodeMath")
|
||||||
|
side.operation = "SIGN"
|
||||||
|
side.location = (-1050.0, 100.0)
|
||||||
|
front = nodes.new("ShaderNodeMath")
|
||||||
|
front.operation = "LESS_THAN"
|
||||||
|
front.inputs[1].default_value = 0.0
|
||||||
|
front.location = (750.0, -350.0)
|
||||||
|
|
||||||
|
depth_attribute = nodes.new("GeometryNodeInputNamedAttribute")
|
||||||
|
depth_attribute.data_type = "FLOAT"
|
||||||
|
depth_attribute.inputs["Name"].default_value = BALLOON_DEPTH_ATTRIBUTE
|
||||||
|
depth_attribute.location = (-1500.0, -150.0)
|
||||||
|
profile_factor = nodes.new("ShaderNodeMath")
|
||||||
|
profile_factor.operation = "ABSOLUTE"
|
||||||
|
profile_factor.location = (-800.0, 100.0)
|
||||||
|
profile_statistics = nodes.new("GeometryNodeAttributeStatistic")
|
||||||
|
profile_statistics.data_type = "FLOAT"
|
||||||
|
profile_statistics.domain = "POINT"
|
||||||
|
profile_statistics.location = (-600.0, 50.0)
|
||||||
|
profile_max_safe = nodes.new("ShaderNodeMath")
|
||||||
|
profile_max_safe.operation = "MAXIMUM"
|
||||||
|
profile_max_safe.inputs[1].default_value = 1e-6
|
||||||
|
profile_max_safe.location = (-350.0, 100.0)
|
||||||
|
profile_normalized = nodes.new("ShaderNodeMath")
|
||||||
|
profile_normalized.operation = "DIVIDE"
|
||||||
|
profile_normalized.location = (-100.0, 100.0)
|
||||||
|
profile_smooth = nodes.new("ShaderNodeMapRange")
|
||||||
|
profile_smooth.data_type = "FLOAT"
|
||||||
|
profile_smooth.interpolation_type = "SMOOTHERSTEP"
|
||||||
|
profile_smooth.clamp = True
|
||||||
|
profile_smooth.inputs["From Min"].default_value = 0.0
|
||||||
|
profile_smooth.inputs["From Max"].default_value = 1.0
|
||||||
|
profile_smooth.inputs["To Min"].default_value = 0.0
|
||||||
|
profile_smooth.inputs["To Max"].default_value = 1.0
|
||||||
|
profile_smooth.location = (100.0, 50.0)
|
||||||
|
profile_active = nodes.new("ShaderNodeMath")
|
||||||
|
profile_active.operation = "MULTIPLY"
|
||||||
|
profile_active.location = (350.0, 50.0)
|
||||||
|
|
||||||
|
object_info = nodes.new("GeometryNodeObjectInfo")
|
||||||
|
object_info.transform_space = "RELATIVE"
|
||||||
|
object_info.location = (-1500.0, -450.0)
|
||||||
|
plane_normal = nodes.new("FunctionNodeRotateVector")
|
||||||
|
plane_normal.inputs["Vector"].default_value = (0.0, 0.0, 1.0)
|
||||||
|
plane_normal.location = (-1250.0, -500.0)
|
||||||
|
effective_normal = nodes.new("GeometryNodeSwitch")
|
||||||
|
effective_normal.input_type = "VECTOR"
|
||||||
|
effective_normal.inputs["False"].default_value = (1.0, 0.0, 0.0)
|
||||||
|
effective_normal.location = (-1050.0, -500.0)
|
||||||
|
normal_separate = nodes.new("ShaderNodeSeparateXYZ")
|
||||||
|
normal_separate.location = (-1000.0, -550.0)
|
||||||
|
zero_position = nodes.new("ShaderNodeCombineXYZ")
|
||||||
|
zero_position.location = (-1050.0, -250.0)
|
||||||
|
plane_dot = nodes.new("ShaderNodeVectorMath")
|
||||||
|
plane_dot.operation = "DOT_PRODUCT"
|
||||||
|
plane_dot.location = (-800.0, -300.0)
|
||||||
|
plane_offset = nodes.new("ShaderNodeVectorMath")
|
||||||
|
plane_offset.operation = "DOT_PRODUCT"
|
||||||
|
plane_offset.location = (-800.0, -500.0)
|
||||||
|
plane_numerator = nodes.new("ShaderNodeMath")
|
||||||
|
plane_numerator.operation = "SUBTRACT"
|
||||||
|
plane_numerator.location = (-550.0, -300.0)
|
||||||
|
plane_divide = nodes.new("ShaderNodeMath")
|
||||||
|
plane_divide.operation = "DIVIDE"
|
||||||
|
plane_divide.location = (-300.0, -300.0)
|
||||||
|
|
||||||
|
depth_residual = nodes.new("ShaderNodeMath")
|
||||||
|
depth_residual.operation = "ADD"
|
||||||
|
depth_residual.location = (-50.0, -200.0)
|
||||||
|
depth_calibration = nodes.new("ShaderNodeMath")
|
||||||
|
depth_calibration.operation = "MULTIPLY"
|
||||||
|
depth_calibration.inputs[1].default_value = 0.5
|
||||||
|
depth_calibration.label = "Calibrate Relative Depth"
|
||||||
|
depth_calibration.location = (200.0, -200.0)
|
||||||
|
corrected_depth = nodes.new("ShaderNodeMath")
|
||||||
|
corrected_depth.operation = "ADD"
|
||||||
|
corrected_depth.location = (450.0, -200.0)
|
||||||
|
depth_scale = nodes.new("ShaderNodeMath")
|
||||||
|
depth_scale.operation = "MULTIPLY"
|
||||||
|
depth_scale.location = (700.0, -150.0)
|
||||||
|
depth_mix = nodes.new("ShaderNodeMix")
|
||||||
|
depth_mix.data_type = "FLOAT"
|
||||||
|
depth_mix.clamp_factor = True
|
||||||
|
depth_mix.location = (450.0, -50.0)
|
||||||
|
signed_depth = nodes.new("ShaderNodeMath")
|
||||||
|
signed_depth.operation = "MULTIPLY"
|
||||||
|
signed_depth.location = (850.0, 50.0)
|
||||||
|
front_choice = nodes.new("GeometryNodeSwitch")
|
||||||
|
front_choice.input_type = "FLOAT"
|
||||||
|
front_choice.location = (1100.0, 0.0)
|
||||||
|
double_choice = nodes.new("GeometryNodeSwitch")
|
||||||
|
double_choice.input_type = "FLOAT"
|
||||||
|
double_choice.location = (1350.0, 100.0)
|
||||||
|
combine = nodes.new("ShaderNodeCombineXYZ")
|
||||||
|
combine.location = (1600.0, 250.0)
|
||||||
|
set_position = nodes.new("GeometryNodeSetPosition")
|
||||||
|
set_position.location = (1850.0, 450.0)
|
||||||
|
group_output = nodes.new("NodeGroupOutput")
|
||||||
|
group_output.location = (2450.0, 450.0)
|
||||||
|
|
||||||
|
base_frame = nodes.new("NodeFrame")
|
||||||
|
base_frame.label = "Base Thickness"
|
||||||
|
plane_frame = nodes.new("NodeFrame")
|
||||||
|
plane_frame.label = "Middle Plane"
|
||||||
|
mix_frame = nodes.new("NodeFrame")
|
||||||
|
mix_frame.label = "Depth Mix"
|
||||||
|
output_frame = nodes.new("NodeFrame")
|
||||||
|
output_frame.label = "Side Output"
|
||||||
|
for node in (
|
||||||
|
separate,
|
||||||
|
base_scale,
|
||||||
|
base_magnitude,
|
||||||
|
profile_factor,
|
||||||
|
profile_statistics,
|
||||||
|
profile_max_safe,
|
||||||
|
profile_normalized,
|
||||||
|
profile_smooth,
|
||||||
|
):
|
||||||
|
node.parent = base_frame
|
||||||
|
for node in (
|
||||||
|
object_info,
|
||||||
|
plane_normal,
|
||||||
|
effective_normal,
|
||||||
|
normal_separate,
|
||||||
|
zero_position,
|
||||||
|
plane_dot,
|
||||||
|
plane_offset,
|
||||||
|
plane_numerator,
|
||||||
|
plane_divide,
|
||||||
|
):
|
||||||
|
node.parent = plane_frame
|
||||||
|
for node in (
|
||||||
|
depth_residual,
|
||||||
|
depth_calibration,
|
||||||
|
corrected_depth,
|
||||||
|
depth_scale,
|
||||||
|
profile_active,
|
||||||
|
depth_mix,
|
||||||
|
):
|
||||||
|
node.parent = mix_frame
|
||||||
|
for node in (
|
||||||
|
side,
|
||||||
|
front,
|
||||||
|
signed_depth,
|
||||||
|
front_choice,
|
||||||
|
double_choice,
|
||||||
|
combine,
|
||||||
|
set_position,
|
||||||
|
):
|
||||||
|
node.parent = output_frame
|
||||||
|
|
||||||
|
base_frame.location = (-1800.0, 500.0)
|
||||||
|
separate.location = (0.0, 200.0)
|
||||||
|
base_scale.location = (220.0, 350.0)
|
||||||
|
base_magnitude.location = (440.0, 350.0)
|
||||||
|
profile_factor.location = (220.0, 50.0)
|
||||||
|
profile_statistics.location = (440.0, 0.0)
|
||||||
|
profile_max_safe.location = (660.0, 0.0)
|
||||||
|
profile_normalized.location = (880.0, 0.0)
|
||||||
|
profile_smooth.location = (1100.0, 0.0)
|
||||||
|
|
||||||
|
plane_frame.location = (-1800.0, -600.0)
|
||||||
|
object_info.location = (0.0, -300.0)
|
||||||
|
plane_normal.location = (220.0, -300.0)
|
||||||
|
effective_normal.location = (440.0, -300.0)
|
||||||
|
normal_separate.location = (660.0, -300.0)
|
||||||
|
zero_position.location = (220.0, 0.0)
|
||||||
|
plane_dot.location = (440.0, 0.0)
|
||||||
|
plane_offset.location = (660.0, -100.0)
|
||||||
|
plane_numerator.location = (880.0, 0.0)
|
||||||
|
plane_divide.location = (1100.0, 0.0)
|
||||||
|
|
||||||
|
mix_frame.location = (-200.0, 200.0)
|
||||||
|
profile_active.location = (0.0, 300.0)
|
||||||
|
depth_residual.location = (0.0, 0.0)
|
||||||
|
depth_calibration.location = (220.0, 0.0)
|
||||||
|
corrected_depth.location = (440.0, 0.0)
|
||||||
|
depth_scale.location = (660.0, 0.0)
|
||||||
|
depth_mix.location = (880.0, 150.0)
|
||||||
|
|
||||||
|
output_frame.location = (900.0, 200.0)
|
||||||
|
side.location = (0.0, 0.0)
|
||||||
|
front.location = (0.0, -250.0)
|
||||||
|
signed_depth.location = (220.0, 200.0)
|
||||||
|
front_choice.location = (440.0, 100.0)
|
||||||
|
double_choice.location = (660.0, 150.0)
|
||||||
|
combine.location = (880.0, 150.0)
|
||||||
|
set_position.location = (1100.0, 150.0)
|
||||||
|
group_input.location = (-2200.0, 500.0)
|
||||||
|
position.location = (-2200.0, 200.0)
|
||||||
|
depth_attribute.location = (-2200.0, -200.0)
|
||||||
|
group_output.location = (2500.0, 350.0)
|
||||||
|
|
||||||
|
links.new(position.outputs["Position"], separate.inputs["Vector"])
|
||||||
|
links.new(separate.outputs["X"], base_scale.inputs[0])
|
||||||
|
links.new(group_input.outputs["Thickness"], base_scale.inputs[1])
|
||||||
|
links.new(base_scale.outputs[0], base_magnitude.inputs[0])
|
||||||
|
links.new(separate.outputs["X"], side.inputs[0])
|
||||||
|
links.new(separate.outputs["X"], front.inputs[0])
|
||||||
|
links.new(separate.outputs["X"], profile_factor.inputs[0])
|
||||||
|
links.new(group_input.outputs["Geometry"], profile_statistics.inputs["Geometry"])
|
||||||
|
links.new(profile_factor.outputs[0], profile_statistics.inputs["Attribute"])
|
||||||
|
links.new(profile_statistics.outputs["Max"], profile_max_safe.inputs[0])
|
||||||
|
links.new(profile_factor.outputs[0], profile_normalized.inputs[0])
|
||||||
|
links.new(profile_max_safe.outputs[0], profile_normalized.inputs[1])
|
||||||
|
links.new(profile_normalized.outputs[0], profile_smooth.inputs["Value"])
|
||||||
|
links.new(profile_smooth.outputs["Result"], profile_active.inputs[0])
|
||||||
|
links.new(depth_attribute.outputs["Exists"], profile_active.inputs[1])
|
||||||
|
|
||||||
|
links.new(group_input.outputs["Base Plane"], object_info.inputs["Object"])
|
||||||
|
links.new(object_info.outputs["Rotation"], plane_normal.inputs["Rotation"])
|
||||||
|
links.new(depth_attribute.outputs["Exists"], effective_normal.inputs["Switch"])
|
||||||
|
links.new(plane_normal.outputs["Vector"], effective_normal.inputs["True"])
|
||||||
|
links.new(effective_normal.outputs["Output"], normal_separate.inputs["Vector"])
|
||||||
|
links.new(separate.outputs["Y"], zero_position.inputs["Y"])
|
||||||
|
links.new(separate.outputs["Z"], zero_position.inputs["Z"])
|
||||||
|
links.new(zero_position.outputs["Vector"], plane_dot.inputs[0])
|
||||||
|
links.new(effective_normal.outputs["Output"], plane_dot.inputs[1])
|
||||||
|
links.new(object_info.outputs["Location"], plane_offset.inputs[0])
|
||||||
|
links.new(effective_normal.outputs["Output"], plane_offset.inputs[1])
|
||||||
|
links.new(plane_offset.outputs["Value"], plane_numerator.inputs[0])
|
||||||
|
links.new(plane_dot.outputs["Value"], plane_numerator.inputs[1])
|
||||||
|
links.new(plane_numerator.outputs[0], plane_divide.inputs[0])
|
||||||
|
links.new(normal_separate.outputs["X"], plane_divide.inputs[1])
|
||||||
|
|
||||||
|
links.new(depth_attribute.outputs["Attribute"], depth_residual.inputs[0])
|
||||||
|
links.new(plane_divide.outputs[0], depth_residual.inputs[1])
|
||||||
|
links.new(depth_residual.outputs[0], depth_calibration.inputs[0])
|
||||||
|
links.new(depth_calibration.outputs[0], corrected_depth.inputs[0])
|
||||||
|
links.new(profile_statistics.outputs["Max"], corrected_depth.inputs[1])
|
||||||
|
links.new(corrected_depth.outputs[0], depth_scale.inputs[0])
|
||||||
|
links.new(group_input.outputs["Depth Strength"], depth_scale.inputs[1])
|
||||||
|
links.new(profile_active.outputs[0], depth_mix.inputs["Factor"])
|
||||||
|
links.new(base_magnitude.outputs[0], depth_mix.inputs["A"])
|
||||||
|
links.new(depth_scale.outputs[0], depth_mix.inputs["B"])
|
||||||
|
links.new(side.outputs[0], signed_depth.inputs[0])
|
||||||
|
links.new(depth_mix.outputs["Result"], signed_depth.inputs[1])
|
||||||
|
links.new(front.outputs[0], front_choice.inputs["Switch"])
|
||||||
|
links.new(base_scale.outputs[0], front_choice.inputs["False"])
|
||||||
|
links.new(signed_depth.outputs[0], front_choice.inputs["True"])
|
||||||
|
links.new(
|
||||||
|
group_input.outputs["Double Sided Depth"],
|
||||||
|
double_choice.inputs["Switch"],
|
||||||
|
)
|
||||||
|
links.new(front_choice.outputs["Output"], double_choice.inputs["False"])
|
||||||
|
links.new(signed_depth.outputs[0], double_choice.inputs["True"])
|
||||||
|
links.new(double_choice.outputs["Output"], combine.inputs["X"])
|
||||||
|
links.new(separate.outputs["Y"], combine.inputs["Y"])
|
||||||
|
links.new(separate.outputs["Z"], combine.inputs["Z"])
|
||||||
|
links.new(group_input.outputs["Geometry"], set_position.inputs["Geometry"])
|
||||||
|
links.new(combine.outputs["Vector"], set_position.inputs["Position"])
|
||||||
|
links.new(set_position.outputs["Geometry"], group_output.inputs["Geometry"])
|
||||||
|
group.color_tag = "GEOMETRY"
|
||||||
|
group.description = (
|
||||||
|
"Mix Balloon thickness with depth around an adjustable child Empty middle plane."
|
||||||
|
)
|
||||||
|
group[BALLOON_SHAPE_VERSION_PROPERTY] = BALLOON_SHAPE_NODE_GROUP_VERSION
|
||||||
|
group.use_fake_user = True
|
||||||
|
return group
|
||||||
|
|
||||||
|
|
||||||
|
def add_balloon_shape_modifier(
|
||||||
|
balloon,
|
||||||
|
thickness,
|
||||||
|
base_plane=None,
|
||||||
|
):
|
||||||
|
from ..material import modifier_input_identifier, set_modifier_input
|
||||||
|
|
||||||
|
node_group = balloon_shape_node_group()
|
||||||
|
modifier = balloon.modifiers.new(name=node_group.name, type="NODES")
|
||||||
|
modifier.node_group = node_group
|
||||||
|
modifier.show_group_selector = False
|
||||||
|
values = {
|
||||||
|
"Thickness": thickness,
|
||||||
|
"Depth Strength": 1.0,
|
||||||
|
"Double Sided Depth": False,
|
||||||
|
"Base Plane": base_plane,
|
||||||
|
}
|
||||||
|
for name, value in values.items():
|
||||||
|
identifier = modifier_input_identifier(node_group, name)
|
||||||
|
set_modifier_input(modifier, identifier, value)
|
||||||
|
return modifier
|
||||||
@@ -3,10 +3,19 @@ import math
|
|||||||
|
|
||||||
import bpy
|
import bpy
|
||||||
|
|
||||||
|
from .depth import (
|
||||||
|
add_balloon_shape_modifier,
|
||||||
|
assign_balloon_shape_attributes,
|
||||||
|
balloon_depth_fields,
|
||||||
|
create_balloon_base_plane,
|
||||||
|
load_balloon_depth_data,
|
||||||
|
)
|
||||||
from .geometry import INFLATION, POINT_SPACING, triangulate_balloon
|
from .geometry import INFLATION, POINT_SPACING, triangulate_balloon
|
||||||
from ...properties import (
|
from ...properties import (
|
||||||
BEN2_MODEL_KEY,
|
BEN2_MODEL_KEY,
|
||||||
balloon_ai_refine_property as ai_refine_property,
|
balloon_contour_analysis_property as contour_analysis_property,
|
||||||
|
balloon_depth_analysis_property as depth_analysis_property,
|
||||||
|
balloon_depth_model_property as depth_model_property,
|
||||||
balloon_mode_property as mode_property,
|
balloon_mode_property as mode_property,
|
||||||
balloon_resolution_property as resolution_property,
|
balloon_resolution_property as resolution_property,
|
||||||
balloon_reuse_material_property as reuse_material_property,
|
balloon_reuse_material_property as reuse_material_property,
|
||||||
@@ -25,16 +34,24 @@ from ..image import (
|
|||||||
color_data_name,
|
color_data_name,
|
||||||
is_image_empty,
|
is_image_empty,
|
||||||
material_data_name,
|
material_data_name,
|
||||||
|
normal_data_name,
|
||||||
plane_bounds,
|
plane_bounds,
|
||||||
prepare_image_input,
|
prepare_image_input,
|
||||||
remove_temporary_input,
|
remove_temporary_input,
|
||||||
video_total_frames,
|
video_total_frames,
|
||||||
)
|
)
|
||||||
from ..material import TEXTURE_MATERIAL_NODE_GROUP_NAME, create_material
|
from ..material import (
|
||||||
|
BALLOON_MATERIAL_NODE_GROUP_NAME,
|
||||||
|
TEXTURE_MATERIAL_NODE_GROUP_NAME,
|
||||||
|
create_material,
|
||||||
|
)
|
||||||
from ..surface import assign_vertex_uv
|
from ..surface import assign_vertex_uv
|
||||||
|
|
||||||
|
|
||||||
MIN_PATH_DISTANCE = 2.0
|
MIN_PATH_DISTANCE = 2.0
|
||||||
|
BALLOON_MATERIAL_PROPERTY = "snapform_balloon_material"
|
||||||
|
BALLOON_SOURCE_IMAGE_PROPERTY = "snapform_source_image"
|
||||||
|
BALLOON_NORMAL_IMAGE_PROPERTY = "snapform_balloon_normal_image"
|
||||||
|
|
||||||
|
|
||||||
def simplify_screen_path(path):
|
def simplify_screen_path(path):
|
||||||
@@ -150,7 +167,45 @@ def material_uses_balloon_texture(material, color_name):
|
|||||||
return has_texture_layer and has_color_image
|
return has_texture_layer and has_color_image
|
||||||
|
|
||||||
|
|
||||||
def reusable_balloon_material(material_name, color_name):
|
def marked_balloon_material(source_image, normal_required=False):
|
||||||
|
return next(
|
||||||
|
(
|
||||||
|
material
|
||||||
|
for material in bpy.data.materials
|
||||||
|
if callable(getattr(material, "get", None))
|
||||||
|
and bool(material.get(BALLOON_MATERIAL_PROPERTY))
|
||||||
|
and material.get(BALLOON_SOURCE_IMAGE_PROPERTY) is source_image
|
||||||
|
and (
|
||||||
|
material.get(BALLOON_NORMAL_IMAGE_PROPERTY) is not None
|
||||||
|
)
|
||||||
|
== normal_required
|
||||||
|
),
|
||||||
|
None,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def mark_balloon_material(material, source_image, normal_image=None):
|
||||||
|
material[BALLOON_MATERIAL_PROPERTY] = True
|
||||||
|
material[BALLOON_SOURCE_IMAGE_PROPERTY] = source_image
|
||||||
|
if normal_image is not None:
|
||||||
|
material[BALLOON_NORMAL_IMAGE_PROPERTY] = normal_image
|
||||||
|
|
||||||
|
|
||||||
|
def reusable_balloon_material(
|
||||||
|
material_name,
|
||||||
|
color_name,
|
||||||
|
source_image=None,
|
||||||
|
normal_required=False,
|
||||||
|
):
|
||||||
|
marked = (
|
||||||
|
marked_balloon_material(source_image, normal_required)
|
||||||
|
if source_image is not None
|
||||||
|
else None
|
||||||
|
)
|
||||||
|
if marked is not None:
|
||||||
|
return marked
|
||||||
|
if normal_required:
|
||||||
|
return None
|
||||||
return next(
|
return next(
|
||||||
(
|
(
|
||||||
material
|
material
|
||||||
@@ -162,6 +217,35 @@ def reusable_balloon_material(material_name, color_name):
|
|||||||
)
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def load_balloon_normal_image(path, source_object):
|
||||||
|
image = bpy.data.images.load(
|
||||||
|
str(path),
|
||||||
|
check_existing=False,
|
||||||
|
)
|
||||||
|
image.name = normal_data_name(source_object)
|
||||||
|
image.colorspace_settings.name = "Non-Color"
|
||||||
|
return image
|
||||||
|
|
||||||
|
|
||||||
|
def merge_balloon_normal_image(target, update):
|
||||||
|
import numpy as np
|
||||||
|
|
||||||
|
if tuple(target.size) != tuple(update.size):
|
||||||
|
raise ValueError("Reusable Balloon normal image dimensions do not match")
|
||||||
|
target_pixels = np.empty(len(target.pixels), dtype=np.float32)
|
||||||
|
update_pixels = np.empty(len(update.pixels), dtype=np.float32)
|
||||||
|
target.pixels.foreach_get(target_pixels)
|
||||||
|
update.pixels.foreach_get(update_pixels)
|
||||||
|
target_rgba = target_pixels.reshape((-1, 4))
|
||||||
|
update_rgba = update_pixels.reshape((-1, 4))
|
||||||
|
neutral = np.asarray((128.0 / 255.0, 128.0 / 255.0, 1.0))
|
||||||
|
changed = np.max(np.abs(update_rgba[:, :3] - neutral), axis=1) > (1.5 / 255.0)
|
||||||
|
target_rgba[changed] = update_rgba[changed]
|
||||||
|
target.pixels.foreach_set(target_pixels)
|
||||||
|
target.pack()
|
||||||
|
target.update()
|
||||||
|
|
||||||
|
|
||||||
def reusable_color_image(color_name):
|
def reusable_color_image(color_name):
|
||||||
return next(
|
return next(
|
||||||
(
|
(
|
||||||
@@ -173,6 +257,27 @@ def reusable_color_image(color_name):
|
|||||||
)
|
)
|
||||||
|
|
||||||
|
|
||||||
|
BALLOON_AXIS_TO_SOURCE = (
|
||||||
|
(0.0, -1.0, 0.0, 0.0),
|
||||||
|
(0.0, 0.0, 1.0, 0.0),
|
||||||
|
(-1.0, 0.0, 0.0, 0.0),
|
||||||
|
(0.0, 0.0, 0.0, 1.0),
|
||||||
|
)
|
||||||
|
|
||||||
|
def orient_balloon_vertices(vertices):
|
||||||
|
"""Rotate legacy XY/Z Balloon coordinates into a YZ/-X frame."""
|
||||||
|
return [(-z, -x, y) for x, y, z in vertices]
|
||||||
|
|
||||||
|
|
||||||
|
def orient_balloon_world_matrix(source_matrix):
|
||||||
|
from mathutils import Matrix
|
||||||
|
|
||||||
|
return source_matrix @ Matrix(BALLOON_AXIS_TO_SOURCE)
|
||||||
|
|
||||||
|
def orient_balloon_base_plane(local_plane):
|
||||||
|
slope_x, slope_y, offset = local_plane
|
||||||
|
return (slope_x, -slope_y, -offset)
|
||||||
|
|
||||||
def extract_balloon_object(
|
def extract_balloon_object(
|
||||||
context,
|
context,
|
||||||
source_object,
|
source_object,
|
||||||
@@ -183,6 +288,8 @@ def extract_balloon_object(
|
|||||||
resolution=POINT_SPACING,
|
resolution=POINT_SPACING,
|
||||||
select_result=True,
|
select_result=True,
|
||||||
reuse_material=True,
|
reuse_material=True,
|
||||||
|
depth_data=None,
|
||||||
|
normal_path=None,
|
||||||
):
|
):
|
||||||
image = source_object.data
|
image = source_object.data
|
||||||
image_size = tuple(image.size)
|
image_size = tuple(image.size)
|
||||||
@@ -199,45 +306,87 @@ def extract_balloon_object(
|
|||||||
spacing=point_spacing,
|
spacing=point_spacing,
|
||||||
world_metric=world_metric,
|
world_metric=world_metric,
|
||||||
world_z_scale=world_z_scale,
|
world_z_scale=world_z_scale,
|
||||||
inflation=inflation,
|
inflation=INFLATION,
|
||||||
algorithm=algorithm,
|
algorithm=algorithm,
|
||||||
)
|
)
|
||||||
|
|
||||||
|
depth_values = None
|
||||||
|
local_base_plane = (0.0, 0.0, 0.0)
|
||||||
|
if depth_data is not None:
|
||||||
|
depth_values, local_base_plane = balloon_depth_fields(
|
||||||
|
vertices,
|
||||||
|
vertex_uv,
|
||||||
|
depth_data,
|
||||||
|
)
|
||||||
|
local_base_plane = orient_balloon_base_plane(local_base_plane)
|
||||||
|
vertices = orient_balloon_vertices(vertices)
|
||||||
source_name = source_object.name
|
source_name = source_object.name
|
||||||
balloon_name = source_name
|
balloon_name = source_name
|
||||||
material_name = material_data_name(source_object)
|
material_name = material_data_name(source_object)
|
||||||
color_name = color_data_name(source_object)
|
color_name = color_data_name(source_object)
|
||||||
color_image = None
|
color_image = None
|
||||||
|
uses_normal = normal_path is not None
|
||||||
material = (
|
material = (
|
||||||
reusable_balloon_material(material_name, color_name)
|
reusable_balloon_material(
|
||||||
|
material_name,
|
||||||
|
color_name,
|
||||||
|
image,
|
||||||
|
normal_required=uses_normal,
|
||||||
|
)
|
||||||
if reuse_material
|
if reuse_material
|
||||||
else None
|
else None
|
||||||
)
|
)
|
||||||
|
normal_image = None
|
||||||
owns_color_image = False
|
owns_color_image = False
|
||||||
|
owns_normal_image = False
|
||||||
owns_material = False
|
owns_material = False
|
||||||
mesh = None
|
mesh = None
|
||||||
balloon = None
|
balloon = None
|
||||||
|
base_plane = None
|
||||||
try:
|
try:
|
||||||
|
if material is not None and not uses_normal:
|
||||||
|
mark_balloon_material(material, image)
|
||||||
|
if material is not None and uses_normal:
|
||||||
|
normal_image = material.get(BALLOON_NORMAL_IMAGE_PROPERTY)
|
||||||
|
update_normal = load_balloon_normal_image(normal_path, source_object)
|
||||||
|
try:
|
||||||
|
merge_balloon_normal_image(normal_image, update_normal)
|
||||||
|
finally:
|
||||||
|
bpy.data.images.remove(update_normal)
|
||||||
if material is None:
|
if material is None:
|
||||||
color_image = (
|
color_image = reusable_color_image(color_name) if reuse_material else None
|
||||||
reusable_color_image(color_name)
|
|
||||||
if reuse_material
|
|
||||||
else None
|
|
||||||
)
|
|
||||||
if color_image is None:
|
if color_image is None:
|
||||||
color_image = image.copy()
|
color_image = image.copy()
|
||||||
color_image.name = color_name
|
color_image.name = color_name
|
||||||
owns_color_image = True
|
owns_color_image = True
|
||||||
|
if normal_path is not None:
|
||||||
|
normal_image = load_balloon_normal_image(
|
||||||
|
normal_path,
|
||||||
|
source_object,
|
||||||
|
)
|
||||||
|
normal_image.pack()
|
||||||
|
owns_normal_image = True
|
||||||
material = create_material(
|
material = create_material(
|
||||||
material_name,
|
material_name,
|
||||||
color_image,
|
color_image,
|
||||||
0.0,
|
1.0,
|
||||||
surface_group_name=TEXTURE_MATERIAL_NODE_GROUP_NAME,
|
surface_group_name=(
|
||||||
|
BALLOON_MATERIAL_NODE_GROUP_NAME
|
||||||
|
if normal_image is not None
|
||||||
|
else TEXTURE_MATERIAL_NODE_GROUP_NAME
|
||||||
|
),
|
||||||
|
normal_image=normal_image,
|
||||||
scene=context.scene,
|
scene=context.scene,
|
||||||
)
|
)
|
||||||
|
mark_balloon_material(material, image, normal_image)
|
||||||
owns_material = True
|
owns_material = True
|
||||||
mesh = bpy.data.meshes.new(balloon_name)
|
mesh = bpy.data.meshes.new(balloon_name)
|
||||||
mesh.from_pydata(vertices, [], faces)
|
mesh.from_pydata(vertices, [], faces)
|
||||||
|
assign_balloon_shape_attributes(
|
||||||
|
mesh,
|
||||||
|
vertices,
|
||||||
|
depth_values,
|
||||||
|
)
|
||||||
mesh.update()
|
mesh.update()
|
||||||
assign_vertex_uv(mesh, vertex_uv)
|
assign_vertex_uv(mesh, vertex_uv)
|
||||||
for face in mesh.polygons:
|
for face in mesh.polygons:
|
||||||
@@ -245,26 +394,37 @@ def extract_balloon_object(
|
|||||||
|
|
||||||
balloon = bpy.data.objects.new(balloon_name, mesh)
|
balloon = bpy.data.objects.new(balloon_name, mesh)
|
||||||
context.collection.objects.link(balloon)
|
context.collection.objects.link(balloon)
|
||||||
balloon.matrix_world = matrix_world.copy()
|
balloon.matrix_world = orient_balloon_world_matrix(matrix_world)
|
||||||
balloon["snapform_source_type"] = "BALLOON"
|
balloon["snapform_source_type"] = "BALLOON"
|
||||||
balloon["snapform_balloon_algorithm"] = algorithm
|
balloon["snapform_balloon_algorithm"] = algorithm
|
||||||
balloon["snapform_source_object"] = source_name
|
balloon["snapform_source_object"] = source_name
|
||||||
balloon["snapform_inflation"] = inflation
|
balloon["snapform_inflation"] = inflation
|
||||||
balloon["snapform_resolution"] = point_spacing
|
balloon["snapform_resolution"] = point_spacing
|
||||||
balloon.data.materials.append(material)
|
balloon.data.materials.append(material)
|
||||||
|
if depth_data is not None:
|
||||||
|
base_plane = create_balloon_base_plane(
|
||||||
|
context,
|
||||||
|
balloon,
|
||||||
|
vertices,
|
||||||
|
local_base_plane,
|
||||||
|
)
|
||||||
|
add_balloon_shape_modifier(balloon, thickness, base_plane)
|
||||||
|
if depth_data is not None:
|
||||||
|
balloon["snapform_balloon_depth_analysis"] = True
|
||||||
|
balloon["snapform_reference_depth"] = depth_data["reference_depth"]
|
||||||
except Exception:
|
except Exception:
|
||||||
|
if base_plane is not None:
|
||||||
|
bpy.data.objects.remove(base_plane, do_unlink=True)
|
||||||
if balloon is not None:
|
if balloon is not None:
|
||||||
bpy.data.objects.remove(balloon, do_unlink=True)
|
bpy.data.objects.remove(balloon, do_unlink=True)
|
||||||
if mesh is not None and mesh.users == 0:
|
if mesh is not None and mesh.users == 0:
|
||||||
bpy.data.meshes.remove(mesh)
|
bpy.data.meshes.remove(mesh)
|
||||||
if owns_material and material is not None and material.users == 0:
|
if owns_material and material is not None and material.users == 0:
|
||||||
bpy.data.materials.remove(material)
|
bpy.data.materials.remove(material)
|
||||||
if (
|
if owns_color_image and color_image is not None and color_image.users == 0:
|
||||||
owns_color_image
|
|
||||||
and color_image is not None
|
|
||||||
and color_image.users == 0
|
|
||||||
):
|
|
||||||
bpy.data.images.remove(color_image)
|
bpy.data.images.remove(color_image)
|
||||||
|
if owns_normal_image and normal_image is not None and normal_image.users == 0:
|
||||||
|
bpy.data.images.remove(normal_image)
|
||||||
raise
|
raise
|
||||||
|
|
||||||
bpy.ops.object.select_all(action="DESELECT")
|
bpy.ops.object.select_all(action="DESELECT")
|
||||||
@@ -323,7 +483,9 @@ class ExtractBalloon(bpy.types.Operator):
|
|||||||
resolution: resolution_property()
|
resolution: resolution_property()
|
||||||
source_object_name: hidden_string_property()
|
source_object_name: hidden_string_property()
|
||||||
reuse_material: reuse_material_property()
|
reuse_material: reuse_material_property()
|
||||||
ai_refine: ai_refine_property()
|
contour_analysis: contour_analysis_property()
|
||||||
|
depth_analysis: depth_analysis_property()
|
||||||
|
depth_model: depth_model_property()
|
||||||
source_matrix_data: hidden_string_property()
|
source_matrix_data: hidden_string_property()
|
||||||
selection_data: hidden_string_property()
|
selection_data: hidden_string_property()
|
||||||
|
|
||||||
@@ -338,7 +500,10 @@ class ExtractBalloon(bpy.types.Operator):
|
|||||||
layout.prop(self, "thickness")
|
layout.prop(self, "thickness")
|
||||||
layout.prop(self, "resolution", text="Resolution (m)")
|
layout.prop(self, "resolution", text="Resolution (m)")
|
||||||
layout.prop(self, "reuse_material")
|
layout.prop(self, "reuse_material")
|
||||||
layout.prop(self, "ai_refine")
|
layout.prop(self, "contour_analysis")
|
||||||
|
layout.prop(self, "depth_analysis")
|
||||||
|
if self.depth_analysis:
|
||||||
|
layout.prop(self, "depth_model")
|
||||||
|
|
||||||
@classmethod
|
@classmethod
|
||||||
def poll(cls, context):
|
def poll(cls, context):
|
||||||
@@ -349,9 +514,8 @@ class ExtractBalloon(bpy.types.Operator):
|
|||||||
)
|
)
|
||||||
|
|
||||||
def _draw_lasso(self):
|
def _draw_lasso(self):
|
||||||
if (
|
if not self._path or not drawing_in_region(
|
||||||
not self._path
|
self._area_pointer, self._region_pointer
|
||||||
or not drawing_in_region(self._area_pointer, self._region_pointer)
|
|
||||||
):
|
):
|
||||||
return
|
return
|
||||||
vertices = list(self._path)
|
vertices = list(self._path)
|
||||||
@@ -377,7 +541,9 @@ class ExtractBalloon(bpy.types.Operator):
|
|||||||
self.thickness = settings.balloon_thickness
|
self.thickness = settings.balloon_thickness
|
||||||
self.resolution = settings.balloon_resolution
|
self.resolution = settings.balloon_resolution
|
||||||
self.reuse_material = settings.balloon_reuse_material
|
self.reuse_material = settings.balloon_reuse_material
|
||||||
self.ai_refine = settings.balloon_ai_refine
|
self.contour_analysis = settings.balloon_contour_analysis
|
||||||
|
self.depth_analysis = settings.balloon_depth_analysis
|
||||||
|
self.depth_model = settings.balloon_depth_model
|
||||||
self.source_object_name = source_object.name
|
self.source_object_name = source_object.name
|
||||||
self.source_matrix_data = serialize_matrix(source_object.matrix_world)
|
self.source_matrix_data = serialize_matrix(source_object.matrix_world)
|
||||||
self.selection_data = ""
|
self.selection_data = ""
|
||||||
@@ -449,14 +615,14 @@ class ExtractBalloon(bpy.types.Operator):
|
|||||||
polygon = json.loads(self.selection_data)
|
polygon = json.loads(self.selection_data)
|
||||||
if len(polygon) < 3:
|
if len(polygon) < 3:
|
||||||
raise ValueError("The stored lasso region is invalid")
|
raise ValueError("The stored lasso region is invalid")
|
||||||
if self.ai_refine:
|
if self.contour_analysis or self.depth_analysis:
|
||||||
if video_total_frames(source_object.data) > 1:
|
if video_total_frames(source_object.data) > 1:
|
||||||
raise RuntimeError("AI Refine only supports a still image")
|
raise RuntimeError("AI Analysis only supports a still image")
|
||||||
source_path, delete_input = prepare_image_input(
|
source_path, delete_input = prepare_image_input(
|
||||||
source_object.data,
|
source_object.data,
|
||||||
context.scene,
|
context.scene,
|
||||||
)
|
)
|
||||||
result = bpy.ops.snapform.refine_balloon(
|
result = bpy.ops.snapform.analyze_balloon(
|
||||||
input_path=str(source_path),
|
input_path=str(source_path),
|
||||||
delete_input=delete_input,
|
delete_input=delete_input,
|
||||||
source_object_name=self.source_object_name,
|
source_object_name=self.source_object_name,
|
||||||
@@ -466,6 +632,9 @@ class ExtractBalloon(bpy.types.Operator):
|
|||||||
thickness=self.thickness,
|
thickness=self.thickness,
|
||||||
resolution=self.resolution,
|
resolution=self.resolution,
|
||||||
reuse_material=self.reuse_material,
|
reuse_material=self.reuse_material,
|
||||||
|
contour_analysis=self.contour_analysis,
|
||||||
|
depth_analysis=self.depth_analysis,
|
||||||
|
depth_model=self.depth_model,
|
||||||
)
|
)
|
||||||
if "CANCELLED" in result:
|
if "CANCELLED" in result:
|
||||||
if delete_input:
|
if delete_input:
|
||||||
@@ -495,12 +664,11 @@ class ExtractBalloon(bpy.types.Operator):
|
|||||||
return {"FINISHED"}
|
return {"FINISHED"}
|
||||||
|
|
||||||
|
|
||||||
|
class AnalyzeBalloon(JobOperatorBase, bpy.types.Operator):
|
||||||
class RefineBalloon(JobOperatorBase, bpy.types.Operator):
|
bl_idname = "snapform.analyze_balloon"
|
||||||
bl_idname = "snapform.refine_balloon"
|
bl_label = "Analyze Balloon"
|
||||||
bl_label = "Refine Balloon"
|
job_type = "analyze-balloon"
|
||||||
job_type = "refine-balloon"
|
starting_message = "Starting Balloon AI analysis"
|
||||||
starting_message = "Starting Balloon AI refinement"
|
|
||||||
|
|
||||||
input_path: bpy.props.StringProperty(options=HIDDEN)
|
input_path: bpy.props.StringProperty(options=HIDDEN)
|
||||||
delete_input: bpy.props.BoolProperty(default=False, options=HIDDEN)
|
delete_input: bpy.props.BoolProperty(default=False, options=HIDDEN)
|
||||||
@@ -511,17 +679,35 @@ class RefineBalloon(JobOperatorBase, bpy.types.Operator):
|
|||||||
thickness: thickness_property()
|
thickness: thickness_property()
|
||||||
resolution: resolution_property()
|
resolution: resolution_property()
|
||||||
reuse_material: reuse_material_property()
|
reuse_material: reuse_material_property()
|
||||||
|
contour_analysis: bpy.props.BoolProperty(default=False, options=HIDDEN)
|
||||||
|
depth_analysis: bpy.props.BoolProperty(default=False, options=HIDDEN)
|
||||||
|
depth_model: depth_model_property()
|
||||||
|
resolution_level: bpy.props.IntProperty(
|
||||||
|
default=5,
|
||||||
|
min=0,
|
||||||
|
max=9,
|
||||||
|
options=HIDDEN,
|
||||||
|
)
|
||||||
|
|
||||||
def request(self, _context):
|
def request(self, _context):
|
||||||
require_environment()
|
require_environment()
|
||||||
|
if self.contour_analysis:
|
||||||
require_model(BEN2_MODEL_KEY)
|
require_model(BEN2_MODEL_KEY)
|
||||||
|
if self.depth_analysis:
|
||||||
|
require_model(self.depth_model)
|
||||||
|
if not self.contour_analysis and not self.depth_analysis:
|
||||||
|
raise ValueError("Select Contour or Depth")
|
||||||
polygon = json.loads(self.selection_data)
|
polygon = json.loads(self.selection_data)
|
||||||
if len(polygon) < 3:
|
if len(polygon) < 3:
|
||||||
raise ValueError("The stored lasso region is invalid")
|
raise ValueError("The stored lasso region is invalid")
|
||||||
return {
|
return {
|
||||||
"input": str(self.input_path),
|
"input": str(self.input_path),
|
||||||
"model": BEN2_MODEL_KEY,
|
"contour_analysis": self.contour_analysis,
|
||||||
|
"depth_analysis": self.depth_analysis,
|
||||||
|
"contour_model": BEN2_MODEL_KEY,
|
||||||
|
"depth_model": self.depth_model,
|
||||||
"device": production_device().lower(),
|
"device": production_device().lower(),
|
||||||
|
"resolution_level": self.resolution_level,
|
||||||
"polygon": polygon,
|
"polygon": polygon,
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -529,12 +715,17 @@ class RefineBalloon(JobOperatorBase, bpy.types.Operator):
|
|||||||
source_object = bpy.data.objects.get(self.source_object_name)
|
source_object = bpy.data.objects.get(self.source_object_name)
|
||||||
if not is_image_empty(source_object):
|
if not is_image_empty(source_object):
|
||||||
raise RuntimeError("The source Image Empty is no longer available")
|
raise RuntimeError("The source Image Empty is no longer available")
|
||||||
payload = json.loads(
|
polygon = json.loads(self.selection_data)
|
||||||
result.file("contour").read_text(encoding="utf-8")
|
if self.contour_analysis:
|
||||||
)
|
payload = json.loads(result.file("contour").read_text(encoding="utf-8"))
|
||||||
polygon = payload.get("points", ())
|
polygon = payload.get("points", ())
|
||||||
if len(polygon) < 3:
|
if len(polygon) < 3:
|
||||||
raise RuntimeError("BEN2 did not return a valid Balloon contour")
|
raise RuntimeError("BEN2 did not return a valid Balloon contour")
|
||||||
|
depth_data = (
|
||||||
|
load_balloon_depth_data(result.file("depth"))
|
||||||
|
if self.depth_analysis
|
||||||
|
else None
|
||||||
|
)
|
||||||
matrix_world = deserialize_matrix(self.source_matrix_data)
|
matrix_world = deserialize_matrix(self.source_matrix_data)
|
||||||
resolution_world = meters_to_world_distance(
|
resolution_world = meters_to_world_distance(
|
||||||
context.scene,
|
context.scene,
|
||||||
@@ -550,15 +741,21 @@ class RefineBalloon(JobOperatorBase, bpy.types.Operator):
|
|||||||
resolution=resolution_world,
|
resolution=resolution_world,
|
||||||
select_result=False,
|
select_result=False,
|
||||||
reuse_material=self.reuse_material,
|
reuse_material=self.reuse_material,
|
||||||
|
depth_data=depth_data,
|
||||||
|
normal_path=(result.file("normal") if self.depth_analysis else None),
|
||||||
)
|
)
|
||||||
balloon["snapform_resolution_m"] = self.resolution
|
balloon["snapform_resolution_m"] = self.resolution
|
||||||
balloon["snapform_balloon_ai_refined"] = True
|
if self.contour_analysis:
|
||||||
|
balloon["snapform_balloon_contour_analysis"] = True
|
||||||
|
if self.depth_analysis:
|
||||||
|
balloon["snapform_balloon_depth_model"] = self.depth_model
|
||||||
push_undo("Extract Balloon")
|
push_undo("Extract Balloon")
|
||||||
return "AI-refined Balloon is ready"
|
return "AI-analyzed Balloon is ready"
|
||||||
|
|
||||||
def cleanup(self):
|
def cleanup(self):
|
||||||
cleanup_temporary_input(self.input_path, self.delete_input)
|
cleanup_temporary_input(self.input_path, self.delete_input)
|
||||||
|
|
||||||
|
|
||||||
class ActivateBalloonTool(bpy.types.Operator):
|
class ActivateBalloonTool(bpy.types.Operator):
|
||||||
bl_idname = "snapform.activate_balloon_tool"
|
bl_idname = "snapform.activate_balloon_tool"
|
||||||
bl_label = "Balloon Lasso Tool"
|
bl_label = "Balloon Lasso Tool"
|
||||||
@@ -610,7 +807,7 @@ class BalloonTool(bpy.types.WorkSpaceTool):
|
|||||||
(
|
(
|
||||||
"zoom_path",
|
"zoom_path",
|
||||||
"scene.snapform_settings.balloon_thickness_radial_zoom",
|
"scene.snapform_settings.balloon_thickness_radial_zoom",
|
||||||
)
|
),
|
||||||
]
|
]
|
||||||
},
|
},
|
||||||
),
|
),
|
||||||
@@ -626,10 +823,11 @@ class BalloonTool(bpy.types.WorkSpaceTool):
|
|||||||
(
|
(
|
||||||
"zoom_path",
|
"zoom_path",
|
||||||
"scene.snapform_settings.balloon_resolution_radial_zoom",
|
"scene.snapform_settings.balloon_resolution_radial_zoom",
|
||||||
)
|
),
|
||||||
]
|
]
|
||||||
},
|
},
|
||||||
), )
|
),
|
||||||
|
)
|
||||||
|
|
||||||
def draw_settings(context, layout, tool):
|
def draw_settings(context, layout, tool):
|
||||||
settings = context.scene.snapform_settings
|
settings = context.scene.snapform_settings
|
||||||
@@ -637,4 +835,7 @@ class BalloonTool(bpy.types.WorkSpaceTool):
|
|||||||
layout.prop(settings, "balloon_thickness")
|
layout.prop(settings, "balloon_thickness")
|
||||||
layout.prop(settings, "balloon_resolution", text="Resolution (m)")
|
layout.prop(settings, "balloon_resolution", text="Resolution (m)")
|
||||||
layout.prop(settings, "balloon_reuse_material")
|
layout.prop(settings, "balloon_reuse_material")
|
||||||
layout.prop(settings, "balloon_ai_refine")
|
layout.prop(settings, "balloon_contour_analysis")
|
||||||
|
layout.prop(settings, "balloon_depth_analysis")
|
||||||
|
if settings.balloon_depth_analysis:
|
||||||
|
layout.prop(settings, "balloon_depth_model", text="Depth Model")
|
||||||
|
|||||||
@@ -46,7 +46,11 @@ def cotangent_poisson_field(
|
|||||||
edge_starts = []
|
edge_starts = []
|
||||||
edge_ends = []
|
edge_ends = []
|
||||||
weights = []
|
weights = []
|
||||||
|
# 约束剖分的瘦三角形可能产生负 cotangent 权重;钳制后保持 Dirichlet 系统正定。
|
||||||
for (start, end), weight in edge_weights.items():
|
for (start, end), weight in edge_weights.items():
|
||||||
|
weight = max(float(weight), 0.0)
|
||||||
|
if weight <= 1e-14:
|
||||||
|
continue
|
||||||
start_compact = compact[start]
|
start_compact = compact[start]
|
||||||
end_compact = compact[end]
|
end_compact = compact[end]
|
||||||
if start_compact >= 0:
|
if start_compact >= 0:
|
||||||
@@ -87,7 +91,7 @@ def cotangent_poisson_field(
|
|||||||
break
|
break
|
||||||
multiplied = multiply(direction)
|
multiplied = multiply(direction)
|
||||||
denominator = float(direction.dot(multiplied))
|
denominator = float(direction.dot(multiplied))
|
||||||
if denominator <= 1e-20 or not math.isfinite(denominator):
|
if denominator <= 0.0 or not math.isfinite(denominator):
|
||||||
raise ValueError("Unable to solve the Poisson Balloon field")
|
raise ValueError("Unable to solve the Poisson Balloon field")
|
||||||
step = residual_product / denominator
|
step = residual_product / denominator
|
||||||
solution += step * direction
|
solution += step * direction
|
||||||
|
|||||||
@@ -17,6 +17,7 @@ DEPTH_SCALE_ATTRIBUTE = "o_depth_scale"
|
|||||||
SURFACE_MATERIAL_NODE_GROUP_NAME = "O DA3 Surface Layer"
|
SURFACE_MATERIAL_NODE_GROUP_NAME = "O DA3 Surface Layer"
|
||||||
TEXTURE_MATERIAL_NODE_GROUP_NAME = "O Texture Layer"
|
TEXTURE_MATERIAL_NODE_GROUP_NAME = "O Texture Layer"
|
||||||
MOGE_SURFACE_MATERIAL_NODE_GROUP_NAME = "O MoGe2 Surface Layer"
|
MOGE_SURFACE_MATERIAL_NODE_GROUP_NAME = "O MoGe2 Surface Layer"
|
||||||
|
BALLOON_MATERIAL_NODE_GROUP_NAME = "O Balloon Layer"
|
||||||
DEPTH_PLANE_MATERIAL_NODE_GROUP_NAME = "O DA3 Depth Plane Layer"
|
DEPTH_PLANE_MATERIAL_NODE_GROUP_NAME = "O DA3 Depth Plane Layer"
|
||||||
MOGE_MATERIAL_NODE_GROUP_NAME = "O MoGe2 Depth Plane Layer"
|
MOGE_MATERIAL_NODE_GROUP_NAME = "O MoGe2 Depth Plane Layer"
|
||||||
SHADELESS_NODE_GROUP_NAME = "O Shadeless"
|
SHADELESS_NODE_GROUP_NAME = "O Shadeless"
|
||||||
@@ -250,7 +251,11 @@ def material_node_group(depth_plane=False, surface_group_name=None):
|
|||||||
("Alpha", "NodeSocketFloat"),
|
("Alpha", "NodeSocketFloat"),
|
||||||
("Bump Scale", "NodeSocketFloat"),
|
("Bump Scale", "NodeSocketFloat"),
|
||||||
]
|
]
|
||||||
if group_name == MOGE_SURFACE_MATERIAL_NODE_GROUP_NAME:
|
normal_group = group_name in {
|
||||||
|
MOGE_SURFACE_MATERIAL_NODE_GROUP_NAME,
|
||||||
|
BALLOON_MATERIAL_NODE_GROUP_NAME,
|
||||||
|
}
|
||||||
|
if normal_group:
|
||||||
input_sockets.insert(2, ("Normal", "NodeSocketColor"))
|
input_sockets.insert(2, ("Normal", "NodeSocketColor"))
|
||||||
input_sockets.insert(3, ("Normal Strength", "NodeSocketFloat"))
|
input_sockets.insert(3, ("Normal Strength", "NodeSocketFloat"))
|
||||||
if depth_plane:
|
if depth_plane:
|
||||||
@@ -271,7 +276,9 @@ def material_node_group(depth_plane=False, surface_group_name=None):
|
|||||||
socket.min_value = -10000.0
|
socket.min_value = -10000.0
|
||||||
socket.max_value = 10000.0
|
socket.max_value = 10000.0
|
||||||
elif name == "Normal Strength":
|
elif name == "Normal Strength":
|
||||||
socket.default_value = 1.0
|
socket.default_value = (
|
||||||
|
0.5 if group_name == BALLOON_MATERIAL_NODE_GROUP_NAME else 1.0
|
||||||
|
)
|
||||||
socket.min_value = 0.0
|
socket.min_value = 0.0
|
||||||
socket.max_value = 10.0
|
socket.max_value = 10.0
|
||||||
output_sockets = [
|
output_sockets = [
|
||||||
@@ -304,9 +311,13 @@ def material_node_group(depth_plane=False, surface_group_name=None):
|
|||||||
roughness_curve.location = (-250.0, 350.0)
|
roughness_curve.location = (-250.0, 350.0)
|
||||||
bump = nodes.new("ShaderNodeBump")
|
bump = nodes.new("ShaderNodeBump")
|
||||||
bump.location = (-250.0, 0.0)
|
bump.location = (-250.0, 0.0)
|
||||||
if group_name == MOGE_SURFACE_MATERIAL_NODE_GROUP_NAME:
|
if normal_group:
|
||||||
normal_map = nodes.new("ShaderNodeNormalMap")
|
normal_map = nodes.new("ShaderNodeNormalMap")
|
||||||
normal_map.space = "OBJECT"
|
normal_map.space = (
|
||||||
|
"TANGENT"
|
||||||
|
if group_name == BALLOON_MATERIAL_NODE_GROUP_NAME
|
||||||
|
else "OBJECT"
|
||||||
|
)
|
||||||
normal_map.location = (-500.0, -150.0)
|
normal_map.location = (-500.0, -150.0)
|
||||||
links.new(group_input.outputs["Normal"], normal_map.inputs["Color"])
|
links.new(group_input.outputs["Normal"], normal_map.inputs["Color"])
|
||||||
links.new(
|
links.new(
|
||||||
|
|||||||
@@ -93,10 +93,7 @@ def models_status(refresh=False):
|
|||||||
|
|
||||||
def _model_files_ready(model):
|
def _model_files_ready(model):
|
||||||
directory = runtime.storage_root() / "models" / model.directory_name
|
directory = runtime.storage_root() / "models" / model.directory_name
|
||||||
return all(
|
return all(any(directory.glob(pattern)) for pattern in model.ready_patterns)
|
||||||
any(directory.glob(pattern))
|
|
||||||
for pattern in model.ready_patterns
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
def model_ready(model_key):
|
def model_ready(model_key):
|
||||||
@@ -147,6 +144,15 @@ def moge2_model_enum_items(_owner, _context):
|
|||||||
)
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def balloon_depth_model_enum_items(_owner, _context):
|
||||||
|
models = [
|
||||||
|
model
|
||||||
|
for model in model_catalog()
|
||||||
|
if model.get("family") == "moge2" and model.get("enabled")
|
||||||
|
]
|
||||||
|
models.sort(key=lambda model: model.get("key") != DEFAULT_MOGE2_MODEL_KEY)
|
||||||
|
return _model_items(models)
|
||||||
|
|
||||||
def depth_plane_model_enum_items(_owner, _context):
|
def depth_plane_model_enum_items(_owner, _context):
|
||||||
models = [
|
models = [
|
||||||
model
|
model
|
||||||
@@ -165,8 +171,7 @@ def surface_model_enum_items(_owner, _context):
|
|||||||
models = [
|
models = [
|
||||||
model
|
model
|
||||||
for model in model_catalog()
|
for model in model_catalog()
|
||||||
if model.get("family") in {"moge2", "da3"}
|
if model.get("family") in {"moge2", "da3"} and model.get("enabled")
|
||||||
and model.get("enabled")
|
|
||||||
]
|
]
|
||||||
models.sort(key=lambda model: model.get("key") != DEFAULT_MOGE2_MODEL_KEY)
|
models.sort(key=lambda model: model.get("key") != DEFAULT_MOGE2_MODEL_KEY)
|
||||||
return _model_items(models)
|
return _model_items(models)
|
||||||
@@ -228,17 +233,33 @@ def balloon_reuse_material_property():
|
|||||||
)
|
)
|
||||||
|
|
||||||
|
|
||||||
def balloon_ai_refine_property():
|
def balloon_contour_analysis_property():
|
||||||
return bpy.props.BoolProperty(
|
return bpy.props.BoolProperty(
|
||||||
name="AI Refine",
|
name="Contour",
|
||||||
description=(
|
description=(
|
||||||
"Use BEN2 Alpha above 0.5 to refine the lasso before building "
|
"Use BEN2 Alpha above 0.5 to analyze the lasso contour before "
|
||||||
"the Balloon"
|
"building the Balloon"
|
||||||
),
|
),
|
||||||
default=False,
|
default=False,
|
||||||
)
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def balloon_depth_analysis_property():
|
||||||
|
return bpy.props.BoolProperty(
|
||||||
|
name="Depth",
|
||||||
|
description=("Use MoGe-2 to add adjustable image depth to the Balloon mesh"),
|
||||||
|
default=False,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def balloon_depth_model_property():
|
||||||
|
return bpy.props.EnumProperty(
|
||||||
|
name="Depth Model",
|
||||||
|
description="MoGe-2 model used for Balloon depth",
|
||||||
|
items=balloon_depth_model_enum_items,
|
||||||
|
default=0,
|
||||||
|
)
|
||||||
|
|
||||||
def balloon_radial_zoom_property(scale):
|
def balloon_radial_zoom_property(scale):
|
||||||
return bpy.props.FloatVectorProperty(
|
return bpy.props.FloatVectorProperty(
|
||||||
size=2,
|
size=2,
|
||||||
@@ -247,12 +268,15 @@ def balloon_radial_zoom_property(scale):
|
|||||||
options={"HIDDEN", "SKIP_SAVE"},
|
options={"HIDDEN", "SKIP_SAVE"},
|
||||||
)
|
)
|
||||||
|
|
||||||
|
|
||||||
class SnapFormSettings(bpy.types.PropertyGroup):
|
class SnapFormSettings(bpy.types.PropertyGroup):
|
||||||
balloon_mode: balloon_mode_property()
|
balloon_mode: balloon_mode_property()
|
||||||
balloon_thickness: balloon_thickness_property()
|
balloon_thickness: balloon_thickness_property()
|
||||||
balloon_resolution: balloon_resolution_property()
|
balloon_resolution: balloon_resolution_property()
|
||||||
balloon_reuse_material: balloon_reuse_material_property()
|
balloon_reuse_material: balloon_reuse_material_property()
|
||||||
balloon_ai_refine: balloon_ai_refine_property()
|
balloon_contour_analysis: balloon_contour_analysis_property()
|
||||||
|
balloon_depth_analysis: balloon_depth_analysis_property()
|
||||||
|
balloon_depth_model: balloon_depth_model_property()
|
||||||
balloon_thickness_radial_zoom: balloon_radial_zoom_property(100.0)
|
balloon_thickness_radial_zoom: balloon_radial_zoom_property(100.0)
|
||||||
balloon_resolution_radial_zoom: balloon_radial_zoom_property(1000.0)
|
balloon_resolution_radial_zoom: balloon_radial_zoom_property(1000.0)
|
||||||
input_path: bpy.props.StringProperty(
|
input_path: bpy.props.StringProperty(
|
||||||
|
|||||||
@@ -5,7 +5,7 @@ from blendjob import JobServer
|
|||||||
from .jobs import generate_depth as _generate_depth
|
from .jobs import generate_depth as _generate_depth
|
||||||
from .jobs import generate_surface as _generate_surface
|
from .jobs import generate_surface as _generate_surface
|
||||||
from .jobs.background import run as _remove_background
|
from .jobs.background import run as _remove_background
|
||||||
from .jobs.balloon import run as _refine_balloon
|
from .jobs.balloon import run as _analyze_balloon
|
||||||
from .jobs.debug_ben2 import run as _debug_ben2
|
from .jobs.debug_ben2 import run as _debug_ben2
|
||||||
from .jobs.debug_da3 import run as _debug_da3
|
from .jobs.debug_da3 import run as _debug_da3
|
||||||
from .jobs.debug_moge2 import run as _debug_moge2
|
from .jobs.debug_moge2 import run as _debug_moge2
|
||||||
@@ -33,9 +33,9 @@ def download_required_models(context, parameters):
|
|||||||
return manager.download_missing(context, parameters["models"])
|
return manager.download_missing(context, parameters["models"])
|
||||||
|
|
||||||
|
|
||||||
@server.job("refine-balloon")
|
@server.job("analyze-balloon")
|
||||||
def refine_balloon(context, parameters):
|
def analyze_balloon(context, parameters):
|
||||||
return _refine_balloon(context, parameters)
|
return _analyze_balloon(context, parameters)
|
||||||
|
|
||||||
|
|
||||||
@server.job("remove-background")
|
@server.job("remove-background")
|
||||||
|
|||||||
@@ -3,6 +3,7 @@ import math
|
|||||||
import time
|
import time
|
||||||
from pathlib import Path
|
from pathlib import Path
|
||||||
|
|
||||||
|
from ..models import moge2
|
||||||
from ..models.ben2 import infer_foreground
|
from ..models.ben2 import infer_foreground
|
||||||
|
|
||||||
|
|
||||||
@@ -43,7 +44,7 @@ def create_lasso_input(input_path, polygon, output_path):
|
|||||||
masked = Image.fromarray(pixels, mode="RGBA")
|
masked = Image.fromarray(pixels, mode="RGBA")
|
||||||
output_path = Path(output_path)
|
output_path = Path(output_path)
|
||||||
masked.save(output_path, format="PNG", compress_level=1)
|
masked.save(output_path, format="PNG", compress_level=1)
|
||||||
return output_path, selection_mask, (left, top)
|
return output_path, selection_mask, (left, top), (width, height)
|
||||||
|
|
||||||
|
|
||||||
def largest_component(mask):
|
def largest_component(mask):
|
||||||
@@ -178,15 +179,165 @@ def contour_from_alpha(alpha, selection_mask, origin=(0, 0)):
|
|||||||
return [[x + offset_x, y + offset_y] for x, y in contour]
|
return [[x + offset_x, y + offset_y] for x, y in contour]
|
||||||
|
|
||||||
|
|
||||||
|
def resized_selection_mask(selection_mask, shape):
|
||||||
|
import numpy as np
|
||||||
|
from PIL import Image
|
||||||
|
|
||||||
|
selection_mask = np.asarray(selection_mask, dtype=bool)
|
||||||
|
if selection_mask.shape == tuple(shape):
|
||||||
|
return selection_mask
|
||||||
|
resized = Image.fromarray(selection_mask.astype(np.uint8) * 255, mode="L")
|
||||||
|
resized = resized.resize(
|
||||||
|
(int(shape[1]), int(shape[0])),
|
||||||
|
Image.Resampling.NEAREST,
|
||||||
|
)
|
||||||
|
return np.asarray(resized, dtype=np.uint8) > 0
|
||||||
|
|
||||||
|
|
||||||
|
def normalized_moge_depth(depth, model_mask, selection_mask):
|
||||||
|
"""Normalize MoGe depth while preserving its adjustable fitted base plane."""
|
||||||
|
import numpy as np
|
||||||
|
|
||||||
|
depth = np.asarray(depth, dtype=np.float32)
|
||||||
|
model_mask = np.asarray(model_mask, dtype=bool)
|
||||||
|
selection_mask = resized_selection_mask(selection_mask, depth.shape)
|
||||||
|
if model_mask.shape != depth.shape:
|
||||||
|
raise ValueError("MoGe-2 depth and mask dimensions do not match")
|
||||||
|
valid = selection_mask & model_mask & np.isfinite(depth) & (depth > 0.0)
|
||||||
|
if not valid.any():
|
||||||
|
raise ValueError("MoGe-2 found no valid depth inside the lasso")
|
||||||
|
|
||||||
|
height, width = depth.shape
|
||||||
|
y, x = np.indices(depth.shape, dtype=np.float64)
|
||||||
|
x = x / max(width - 1, 1) * 2.0 - 1.0
|
||||||
|
y = y / max(height - 1, 1) * 2.0 - 1.0
|
||||||
|
design = np.stack((x[valid], y[valid], np.ones(valid.sum())), axis=1)
|
||||||
|
selected = depth[valid].astype(np.float64)
|
||||||
|
weights = np.ones_like(selected)
|
||||||
|
coefficients = np.zeros(3, dtype=np.float64)
|
||||||
|
for _iteration in range(4):
|
||||||
|
weighted_design = design * np.sqrt(weights)[:, None]
|
||||||
|
weighted_depth = selected * np.sqrt(weights)
|
||||||
|
coefficients = np.linalg.lstsq(
|
||||||
|
weighted_design,
|
||||||
|
weighted_depth,
|
||||||
|
rcond=None,
|
||||||
|
)[0]
|
||||||
|
residual = selected - design @ coefficients
|
||||||
|
centered = residual - np.median(residual)
|
||||||
|
scale = max(float(np.median(np.abs(centered))) * 1.4826, 1e-6)
|
||||||
|
weights = np.minimum(
|
||||||
|
1.0,
|
||||||
|
(2.5 * scale) / np.maximum(np.abs(centered), 1e-12),
|
||||||
|
)
|
||||||
|
|
||||||
|
baseline = coefficients[0] * x + coefficients[1] * y + coefficients[2]
|
||||||
|
reference = float(np.median(baseline[valid]))
|
||||||
|
residual = baseline[valid] - selected
|
||||||
|
near_residual = np.maximum(residual, 0.0)
|
||||||
|
near_total = float(np.sum(near_residual))
|
||||||
|
relative_span = (
|
||||||
|
float(np.sum(np.square(near_residual))) / near_total
|
||||||
|
if near_total > 1e-12
|
||||||
|
else float(np.sqrt(np.mean(np.square(residual))))
|
||||||
|
)
|
||||||
|
relative_span = max(relative_span, 1e-6)
|
||||||
|
depth_signal = np.zeros(depth.shape, dtype=np.float32)
|
||||||
|
depth_signal[valid] = (reference - selected) / relative_span
|
||||||
|
baseline_vector = np.asarray(
|
||||||
|
(
|
||||||
|
-coefficients[0] / relative_span,
|
||||||
|
-coefficients[1] / relative_span,
|
||||||
|
(reference - coefficients[2]) / relative_span,
|
||||||
|
),
|
||||||
|
dtype=np.float32,
|
||||||
|
)
|
||||||
|
return depth_signal, valid, reference, baseline_vector
|
||||||
|
|
||||||
|
|
||||||
|
def write_depth_analysis(
|
||||||
|
prediction,
|
||||||
|
selection_mask,
|
||||||
|
origin,
|
||||||
|
source_size,
|
||||||
|
output_path,
|
||||||
|
):
|
||||||
|
import numpy as np
|
||||||
|
|
||||||
|
if prediction.depth.shape[0] != 1 or prediction.mask.shape[0] != 1:
|
||||||
|
raise ValueError("Balloon Depth requires one MoGe-2 frame")
|
||||||
|
depth_signal, valid, reference, baseline_vector = normalized_moge_depth(
|
||||||
|
prediction.depth[0],
|
||||||
|
prediction.mask[0],
|
||||||
|
selection_mask,
|
||||||
|
)
|
||||||
|
crop_size = (selection_mask.shape[1], selection_mask.shape[0])
|
||||||
|
np.savez_compressed(
|
||||||
|
output_path,
|
||||||
|
depth_signal=depth_signal,
|
||||||
|
baseline_vector=baseline_vector,
|
||||||
|
valid=valid,
|
||||||
|
reference_depth=np.asarray(reference, dtype=np.float32),
|
||||||
|
origin=np.asarray(origin, dtype=np.int32),
|
||||||
|
crop_size=np.asarray(crop_size, dtype=np.int32),
|
||||||
|
source_size=np.asarray(source_size, dtype=np.int32),
|
||||||
|
)
|
||||||
|
return Path(output_path)
|
||||||
|
|
||||||
|
|
||||||
|
def write_balloon_normal_texture(
|
||||||
|
normal,
|
||||||
|
valid,
|
||||||
|
origin,
|
||||||
|
crop_size,
|
||||||
|
source_size,
|
||||||
|
output_path,
|
||||||
|
):
|
||||||
|
"""Write MoGe camera normals as a full-image tangent-space texture."""
|
||||||
|
import numpy as np
|
||||||
|
from PIL import Image
|
||||||
|
|
||||||
|
normal = np.asarray(normal, dtype=np.float32)
|
||||||
|
valid = np.asarray(valid, dtype=bool)
|
||||||
|
if normal.ndim != 3 or normal.shape[2] != 3:
|
||||||
|
raise ValueError("Balloon normal data must have three components")
|
||||||
|
if valid.shape != normal.shape[:2]:
|
||||||
|
valid = resized_selection_mask(valid, normal.shape[:2])
|
||||||
|
encoded = np.clip(
|
||||||
|
(normal * np.asarray((1.0, -1.0, -1.0), dtype=np.float32) + 1.0)
|
||||||
|
* 127.5,
|
||||||
|
0.0,
|
||||||
|
255.0,
|
||||||
|
).astype(np.uint8)
|
||||||
|
encoded[~valid] = (128, 128, 255)
|
||||||
|
crop_width, crop_height = (int(value) for value in crop_size)
|
||||||
|
crop = Image.fromarray(encoded, mode="RGB")
|
||||||
|
if crop.size != (crop_width, crop_height):
|
||||||
|
crop = crop.resize(
|
||||||
|
(crop_width, crop_height),
|
||||||
|
Image.Resampling.BILINEAR,
|
||||||
|
)
|
||||||
|
source_width, source_height = (int(value) for value in source_size)
|
||||||
|
full = Image.new("RGB", (source_width, source_height), (128, 128, 255))
|
||||||
|
full.paste(crop, tuple(int(value) for value in origin))
|
||||||
|
output_path = Path(output_path)
|
||||||
|
full.save(output_path)
|
||||||
|
return output_path
|
||||||
|
|
||||||
|
|
||||||
def run(context, parameters):
|
def run(context, parameters):
|
||||||
"""Refine a Balloon lasso with BEN2 and return its largest contour."""
|
"""Analyze a Balloon lasso with optional BEN2 contour and MoGe-2 depth."""
|
||||||
total_started = time.perf_counter()
|
total_started = time.perf_counter()
|
||||||
|
contour_analysis = bool(parameters.get("contour_analysis"))
|
||||||
|
depth_analysis = bool(parameters.get("depth_analysis"))
|
||||||
|
if not contour_analysis and not depth_analysis:
|
||||||
|
raise ValueError("Select Contour or Depth")
|
||||||
|
|
||||||
context.progress(0.02, "Preparing lasso input")
|
context.progress(0.02, "Preparing lasso input")
|
||||||
context.check_cancelled()
|
context.check_cancelled()
|
||||||
|
|
||||||
output_directory = context.directory
|
output_directory = context.directory
|
||||||
output_directory.mkdir(parents=True, exist_ok=True)
|
output_directory.mkdir(parents=True, exist_ok=True)
|
||||||
input_path, selection_mask, origin = create_lasso_input(
|
input_path, selection_mask, origin, source_size = create_lasso_input(
|
||||||
parameters["input"],
|
parameters["input"],
|
||||||
parameters["polygon"],
|
parameters["polygon"],
|
||||||
output_directory / "lasso-input.png",
|
output_directory / "lasso-input.png",
|
||||||
@@ -195,9 +346,11 @@ def run(context, parameters):
|
|||||||
|
|
||||||
device = parameters["device"]
|
device = parameters["device"]
|
||||||
model_manager = context.resource("model_manager")
|
model_manager = context.resource("model_manager")
|
||||||
model_directory = model_manager.directory(parameters["model"])
|
result = {}
|
||||||
context.progress(0.18, f"Loading BEN2 for {device.upper()}")
|
if contour_analysis:
|
||||||
foreground, load_ms, inference_ms = infer_foreground(
|
model_directory = model_manager.directory(parameters["contour_model"])
|
||||||
|
context.progress(0.15, f"Loading BEN2 for {device.upper()}")
|
||||||
|
foreground, _load_ms, _inference_ms = infer_foreground(
|
||||||
input_path,
|
input_path,
|
||||||
model_directory,
|
model_directory,
|
||||||
device,
|
device,
|
||||||
@@ -205,8 +358,10 @@ def run(context, parameters):
|
|||||||
cancel_check=context.check_cancelled,
|
cancel_check=context.check_cancelled,
|
||||||
)
|
)
|
||||||
context.check_cancelled()
|
context.check_cancelled()
|
||||||
|
context.progress(
|
||||||
context.progress(0.9, "Tracing BEN2 foreground")
|
0.55 if depth_analysis else 0.9,
|
||||||
|
"Tracing BEN2 foreground",
|
||||||
|
)
|
||||||
import numpy as np
|
import numpy as np
|
||||||
from PIL import Image
|
from PIL import Image
|
||||||
|
|
||||||
@@ -225,11 +380,49 @@ def run(context, parameters):
|
|||||||
json.dumps({"points": contour}, separators=(",", ":")),
|
json.dumps({"points": contour}, separators=(",", ":")),
|
||||||
encoding="utf-8",
|
encoding="utf-8",
|
||||||
)
|
)
|
||||||
|
result["contour"] = "contour.json"
|
||||||
|
|
||||||
|
if depth_analysis:
|
||||||
|
context.progress(
|
||||||
|
0.12 if not contour_analysis else 0.58,
|
||||||
|
"Estimating Balloon depth with MoGe-2",
|
||||||
|
)
|
||||||
|
prediction = moge2.infer(
|
||||||
|
model_manager,
|
||||||
|
{
|
||||||
|
"model": parameters["depth_model"],
|
||||||
|
"device": device,
|
||||||
|
"resolution_level": parameters.get("resolution_level", 5),
|
||||||
|
},
|
||||||
|
[input_path],
|
||||||
|
include_points=False,
|
||||||
|
progress_callback=None,
|
||||||
|
cancel_check=context.check_cancelled,
|
||||||
|
)
|
||||||
|
context.check_cancelled()
|
||||||
|
context.progress(0.9, "Writing Balloon depth")
|
||||||
|
depth_path = write_depth_analysis(
|
||||||
|
prediction,
|
||||||
|
selection_mask,
|
||||||
|
origin,
|
||||||
|
source_size,
|
||||||
|
output_directory / "depth.npz",
|
||||||
|
)
|
||||||
|
result["depth"] = depth_path.name
|
||||||
|
import numpy as np
|
||||||
|
|
||||||
|
with np.load(depth_path, allow_pickle=False) as data:
|
||||||
|
normal_path = write_balloon_normal_texture(
|
||||||
|
prediction.normal[0],
|
||||||
|
data["valid"],
|
||||||
|
origin,
|
||||||
|
(selection_mask.shape[1], selection_mask.shape[0]),
|
||||||
|
source_size,
|
||||||
|
output_directory / "normal.png",
|
||||||
|
)
|
||||||
|
result["normal"] = normal_path.name
|
||||||
|
|
||||||
total_ms = (time.perf_counter() - total_started) * 1000.0
|
total_ms = (time.perf_counter() - total_started) * 1000.0
|
||||||
message = (
|
message = f"Balloon AI analysis finished in {total_ms:.0f} ms"
|
||||||
f"Balloon contour refined in {inference_ms:.0f} ms "
|
|
||||||
f"(model load {load_ms:.0f} ms, total {total_ms:.0f} ms)"
|
|
||||||
)
|
|
||||||
context.progress(1.0, message)
|
context.progress(1.0, message)
|
||||||
return {"contour": "contour.json"}
|
return result
|
||||||
|
|||||||
@@ -82,34 +82,25 @@ class AddonTest(unittest.TestCase):
|
|||||||
cls.addon = importlib.import_module("snapform")
|
cls.addon = importlib.import_module("snapform")
|
||||||
cls.job_runtime = importlib.import_module("snapform.runtime")
|
cls.job_runtime = importlib.import_module("snapform.runtime")
|
||||||
cls.model_operators = importlib.import_module("snapform.operators.models")
|
cls.model_operators = importlib.import_module("snapform.operators.models")
|
||||||
cls.image_depth = importlib.import_module(
|
cls.image_depth = importlib.import_module("snapform.operators.material")
|
||||||
"snapform.operators.material"
|
cls.image_data = importlib.import_module("snapform.operators.image")
|
||||||
)
|
|
||||||
cls.image_data = importlib.import_module(
|
|
||||||
"snapform.operators.image"
|
|
||||||
)
|
|
||||||
cls.depth_operator = importlib.import_module(
|
cls.depth_operator = importlib.import_module(
|
||||||
"snapform.operators.depth_plane.operator"
|
"snapform.operators.depth_plane.operator"
|
||||||
)
|
)
|
||||||
cls.depth_build = importlib.import_module(
|
cls.depth_build = importlib.import_module(
|
||||||
"snapform.operators.depth_plane.build"
|
"snapform.operators.depth_plane.build"
|
||||||
)
|
)
|
||||||
cls.image_background = importlib.import_module(
|
cls.image_background = importlib.import_module("snapform.operators.background")
|
||||||
"snapform.operators.background"
|
cls.image_mesh = importlib.import_module("snapform.operators.surface")
|
||||||
)
|
|
||||||
cls.image_mesh = importlib.import_module(
|
|
||||||
"snapform.operators.surface"
|
|
||||||
)
|
|
||||||
cls.balloon = importlib.import_module("snapform.operators.balloon.operator")
|
cls.balloon = importlib.import_module("snapform.operators.balloon.operator")
|
||||||
cls.balloon_geometry = importlib.import_module(
|
cls.balloon_geometry = importlib.import_module(
|
||||||
"snapform.operators.balloon.geometry"
|
"snapform.operators.balloon.geometry"
|
||||||
)
|
)
|
||||||
|
cls.balloon_depth = importlib.import_module("snapform.operators.balloon.depth")
|
||||||
cls.balloon_poisson = importlib.import_module(
|
cls.balloon_poisson = importlib.import_module(
|
||||||
"snapform.operators.balloon.poisson"
|
"snapform.operators.balloon.poisson"
|
||||||
)
|
)
|
||||||
cls.balloon_teddy = importlib.import_module(
|
cls.balloon_teddy = importlib.import_module("snapform.operators.balloon.teddy")
|
||||||
"snapform.operators.balloon.teddy"
|
|
||||||
)
|
|
||||||
cls.paths_after_import = tuple(sys.path)
|
cls.paths_after_import = tuple(sys.path)
|
||||||
cls.fake_bpy = fake_bpy
|
cls.fake_bpy = fake_bpy
|
||||||
|
|
||||||
@@ -298,7 +289,9 @@ class AddonTest(unittest.TestCase):
|
|||||||
window_manager = SimpleNamespace(
|
window_manager = SimpleNamespace(
|
||||||
progress_begin=lambda *_args: None,
|
progress_begin=lambda *_args: None,
|
||||||
progress_update=lambda _value: None,
|
progress_update=lambda _value: None,
|
||||||
event_timer_add=lambda *_args, **_kwargs: timers.append(object()) or timers[-1],
|
event_timer_add=lambda *_args, **_kwargs: (
|
||||||
|
timers.append(object()) or timers[-1]
|
||||||
|
),
|
||||||
event_timer_remove=lambda _timer: None,
|
event_timer_remove=lambda _timer: None,
|
||||||
modal_handler_add=lambda _operator: None,
|
modal_handler_add=lambda _operator: None,
|
||||||
)
|
)
|
||||||
@@ -320,7 +313,9 @@ class AddonTest(unittest.TestCase):
|
|||||||
with patch.object(runtime, "storage_root_factory", root):
|
with patch.object(runtime, "storage_root_factory", root):
|
||||||
self.assertEqual(runtime.storage_root(), root.resolve())
|
self.assertEqual(runtime.storage_root(), root.resolve())
|
||||||
self.assertEqual(runtime.environment_directory(), root / ".venv")
|
self.assertEqual(runtime.environment_directory(), root / ".venv")
|
||||||
self.assertEqual(runtime.install_status_path(), root / "install-status.json")
|
self.assertEqual(
|
||||||
|
runtime.install_status_path(), root / "install-status.json"
|
||||||
|
)
|
||||||
self.assertEqual(runtime.install_log_path(), root / "install.log")
|
self.assertEqual(runtime.install_log_path(), root / "install.log")
|
||||||
self.assertEqual(runtime.server_log_path(), root / "server.log")
|
self.assertEqual(runtime.server_log_path(), root / "server.log")
|
||||||
|
|
||||||
@@ -328,8 +323,10 @@ class AddonTest(unittest.TestCase):
|
|||||||
runtime = self.addon.runtime
|
runtime = self.addon.runtime
|
||||||
with tempfile.TemporaryDirectory() as directory:
|
with tempfile.TemporaryDirectory() as directory:
|
||||||
root = Path(directory)
|
root = Path(directory)
|
||||||
python = root / ".venv" / (
|
python = (
|
||||||
"Scripts/python.exe" if sys.platform == "win32" else "bin/python"
|
root
|
||||||
|
/ ".venv"
|
||||||
|
/ ("Scripts/python.exe" if sys.platform == "win32" else "bin/python")
|
||||||
)
|
)
|
||||||
python.parent.mkdir(parents=True)
|
python.parent.mkdir(parents=True)
|
||||||
python.touch()
|
python.touch()
|
||||||
@@ -370,9 +367,7 @@ class AddonTest(unittest.TestCase):
|
|||||||
def test_snapform_post_install_is_ordinary_code_using_runtime_request(self):
|
def test_snapform_post_install_is_ordinary_code_using_runtime_request(self):
|
||||||
calls = []
|
calls = []
|
||||||
fake_runtime = SimpleNamespace(
|
fake_runtime = SimpleNamespace(
|
||||||
request=lambda job_type, parameters: calls.append(
|
request=lambda job_type, parameters: calls.append((job_type, parameters))
|
||||||
(job_type, parameters)
|
|
||||||
)
|
|
||||||
)
|
)
|
||||||
self.job_runtime.post_install(fake_runtime)
|
self.job_runtime.post_install(fake_runtime)
|
||||||
self.assertEqual(
|
self.assertEqual(
|
||||||
@@ -394,9 +389,7 @@ class AddonTest(unittest.TestCase):
|
|||||||
RuntimeError,
|
RuntimeError,
|
||||||
"MoGe-2 ViT-S Normal: network unavailable; BEN2 Base: network unavailable",
|
"MoGe-2 ViT-S Normal: network unavailable; BEN2 Base: network unavailable",
|
||||||
):
|
):
|
||||||
self.job_runtime.post_install(
|
self.job_runtime.post_install(SimpleNamespace(request=request))
|
||||||
SimpleNamespace(request=request)
|
|
||||||
)
|
|
||||||
|
|
||||||
self.assertEqual(calls, ["MOGE2_VITS_NORMAL", "BEN2_BASE"])
|
self.assertEqual(calls, ["MOGE2_VITS_NORMAL", "BEN2_BASE"])
|
||||||
|
|
||||||
@@ -414,14 +407,18 @@ class AddonTest(unittest.TestCase):
|
|||||||
|
|
||||||
controller = runtime_module.EnvironmentController(runtime)
|
controller = runtime_module.EnvironmentController(runtime)
|
||||||
controller.process = SimpleNamespace(poll=lambda: 0, returncode=0)
|
controller.process = SimpleNamespace(poll=lambda: 0, returncode=0)
|
||||||
with tempfile.TemporaryDirectory() as directory, patch.object(
|
with (
|
||||||
|
tempfile.TemporaryDirectory() as directory,
|
||||||
|
patch.object(
|
||||||
runtime,
|
runtime,
|
||||||
"post_install",
|
"post_install",
|
||||||
post_install,
|
post_install,
|
||||||
), patch.object(
|
),
|
||||||
|
patch.object(
|
||||||
runtime,
|
runtime,
|
||||||
"install_status_path",
|
"install_status_path",
|
||||||
return_value=Path(directory) / "missing.json",
|
return_value=Path(directory) / "missing.json",
|
||||||
|
),
|
||||||
):
|
):
|
||||||
self.assertEqual(controller.status("environment")["state"], "running")
|
self.assertEqual(controller.status("environment")["state"], "running")
|
||||||
self.assertTrue(started.wait(1.0))
|
self.assertTrue(started.wait(1.0))
|
||||||
@@ -441,14 +438,18 @@ class AddonTest(unittest.TestCase):
|
|||||||
|
|
||||||
controller = runtime_module.EnvironmentController(runtime)
|
controller = runtime_module.EnvironmentController(runtime)
|
||||||
controller.process = SimpleNamespace(poll=lambda: 0, returncode=0)
|
controller.process = SimpleNamespace(poll=lambda: 0, returncode=0)
|
||||||
with tempfile.TemporaryDirectory() as directory, patch.object(
|
with (
|
||||||
|
tempfile.TemporaryDirectory() as directory,
|
||||||
|
patch.object(
|
||||||
runtime,
|
runtime,
|
||||||
"post_install",
|
"post_install",
|
||||||
post_install,
|
post_install,
|
||||||
), patch.object(
|
),
|
||||||
|
patch.object(
|
||||||
runtime,
|
runtime,
|
||||||
"install_status_path",
|
"install_status_path",
|
||||||
return_value=Path(directory) / "missing.json",
|
return_value=Path(directory) / "missing.json",
|
||||||
|
),
|
||||||
):
|
):
|
||||||
controller.status("environment")
|
controller.status("environment")
|
||||||
controller.post_thread.join(1.0)
|
controller.post_thread.join(1.0)
|
||||||
@@ -477,17 +478,31 @@ class AddonTest(unittest.TestCase):
|
|||||||
def test_conversion_model_lists_default_to_moge2_vits(self):
|
def test_conversion_model_lists_default_to_moge2_vits(self):
|
||||||
properties = importlib.import_module("snapform.properties")
|
properties = importlib.import_module("snapform.properties")
|
||||||
catalog = (
|
catalog = (
|
||||||
{"key": "SMALL", "family": "da3", "enabled": True,
|
{
|
||||||
"label": "DA3 Small", "description": "DA3", "license": "Apache"},
|
"key": "SMALL",
|
||||||
{"key": "MOGE2_VITS_NORMAL", "family": "moge2", "enabled": True,
|
"family": "da3",
|
||||||
"label": "MoGe-2 ViT-S", "description": "MoGe", "license": "MIT"},
|
"enabled": True,
|
||||||
|
"label": "DA3 Small",
|
||||||
|
"description": "DA3",
|
||||||
|
"license": "Apache",
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"key": "MOGE2_VITS_NORMAL",
|
||||||
|
"family": "moge2",
|
||||||
|
"enabled": True,
|
||||||
|
"label": "MoGe-2 ViT-S",
|
||||||
|
"description": "MoGe",
|
||||||
|
"license": "MIT",
|
||||||
|
},
|
||||||
)
|
)
|
||||||
with patch.object(properties, "model_catalog", return_value=catalog):
|
with patch.object(properties, "model_catalog", return_value=catalog):
|
||||||
depth_items = properties.depth_plane_model_enum_items(None, None)
|
depth_items = properties.depth_plane_model_enum_items(None, None)
|
||||||
surface_items = properties.surface_model_enum_items(None, None)
|
surface_items = properties.surface_model_enum_items(None, None)
|
||||||
|
balloon_items = properties.balloon_depth_model_enum_items(None, None)
|
||||||
|
|
||||||
self.assertEqual(depth_items[0][0], "MOGE2_VITS_NORMAL")
|
self.assertEqual(depth_items[0][0], "MOGE2_VITS_NORMAL")
|
||||||
self.assertEqual(surface_items[0][0], "MOGE2_VITS_NORMAL")
|
self.assertEqual(surface_items[0][0], "MOGE2_VITS_NORMAL")
|
||||||
|
self.assertEqual([item[0] for item in balloon_items], ["MOGE2_VITS_NORMAL"])
|
||||||
|
|
||||||
def test_required_model_operator_requests_both_models(self):
|
def test_required_model_operator_requests_both_models(self):
|
||||||
operator = self.addon.operators.DownloadRequiredModels()
|
operator = self.addon.operators.DownloadRequiredModels()
|
||||||
@@ -521,13 +536,16 @@ class AddonTest(unittest.TestCase):
|
|||||||
|
|
||||||
menu = self.addon.SnapFormImageMenu()
|
menu = self.addon.SnapFormImageMenu()
|
||||||
menu.layout = Layout()
|
menu.layout = Layout()
|
||||||
with patch.object(
|
with (
|
||||||
|
patch.object(
|
||||||
self.addon.runtime,
|
self.addon.runtime,
|
||||||
"environment_ready",
|
"environment_ready",
|
||||||
return_value=True,
|
return_value=True,
|
||||||
), patch(
|
),
|
||||||
|
patch(
|
||||||
"snapform.properties.required_models_ready",
|
"snapform.properties.required_models_ready",
|
||||||
return_value=False,
|
return_value=False,
|
||||||
|
),
|
||||||
):
|
):
|
||||||
menu.draw(None)
|
menu.draw(None)
|
||||||
|
|
||||||
@@ -565,13 +583,16 @@ class AddonTest(unittest.TestCase):
|
|||||||
|
|
||||||
menu = self.addon.SnapFormImageMenu()
|
menu = self.addon.SnapFormImageMenu()
|
||||||
menu.layout = Layout()
|
menu.layout = Layout()
|
||||||
with patch.object(
|
with (
|
||||||
|
patch.object(
|
||||||
self.addon.runtime,
|
self.addon.runtime,
|
||||||
"environment_ready",
|
"environment_ready",
|
||||||
return_value=False,
|
return_value=False,
|
||||||
), patch(
|
),
|
||||||
|
patch(
|
||||||
"snapform.properties.required_models_ready",
|
"snapform.properties.required_models_ready",
|
||||||
return_value=False,
|
return_value=False,
|
||||||
|
),
|
||||||
):
|
):
|
||||||
menu.draw(None)
|
menu.draw(None)
|
||||||
|
|
||||||
@@ -617,20 +638,18 @@ class AddonTest(unittest.TestCase):
|
|||||||
import numpy as np
|
import numpy as np
|
||||||
|
|
||||||
polygon = np.asarray(
|
polygon = np.asarray(
|
||||||
((100, 100), (900, 100), (950, 500), (800, 900),
|
((100, 100), (900, 100), (950, 500), (800, 900), (200, 900), (50, 500)),
|
||||||
(200, 900), (50, 500)),
|
|
||||||
dtype=float,
|
dtype=float,
|
||||||
)
|
)
|
||||||
contour, interior, _spacing = (
|
contour, interior, _spacing = self.balloon_geometry.prepare_planar_points(
|
||||||
self.balloon_geometry.prepare_planar_points(
|
|
||||||
polygon,
|
polygon,
|
||||||
(1000, 1000),
|
(1000, 1000),
|
||||||
(0.0, 1.0, 0.0, 1.0),
|
(0.0, 1.0, 0.0, 1.0),
|
||||||
)
|
)
|
||||||
)
|
|
||||||
|
|
||||||
self.assertLess(len(contour), 150)
|
self.assertLess(len(contour), 150)
|
||||||
self.assertLess(len(interior), 400)
|
self.assertLess(len(interior), 400)
|
||||||
|
|
||||||
def test_balloon_sampling_uses_world_distance(self):
|
def test_balloon_sampling_uses_world_distance(self):
|
||||||
import numpy as np
|
import numpy as np
|
||||||
|
|
||||||
@@ -715,6 +734,66 @@ class AddonTest(unittest.TestCase):
|
|||||||
|
|
||||||
self.assertGreater(wide_height, narrow_height)
|
self.assertGreater(wide_height, narrow_height)
|
||||||
|
|
||||||
|
def test_poisson_balloon_supports_resolution_002(self):
|
||||||
|
import numpy as np
|
||||||
|
|
||||||
|
spacing = 0.02
|
||||||
|
size = 51
|
||||||
|
axis = np.arange(size, dtype=float) * spacing
|
||||||
|
points = np.asarray(
|
||||||
|
[(x, y) for y in axis for x in axis],
|
||||||
|
dtype=float,
|
||||||
|
)
|
||||||
|
faces = []
|
||||||
|
for row in range(size - 1):
|
||||||
|
for column in range(size - 1):
|
||||||
|
bottom_left = row * size + column
|
||||||
|
bottom_right = bottom_left + 1
|
||||||
|
top_left = bottom_left + size
|
||||||
|
top_right = top_left + 1
|
||||||
|
faces.append((bottom_left, bottom_right, top_right))
|
||||||
|
faces.append((bottom_left, top_right, top_left))
|
||||||
|
boundary = np.zeros(len(points), dtype=bool)
|
||||||
|
boundary[:size] = True
|
||||||
|
boundary[-size:] = True
|
||||||
|
boundary[::size] = True
|
||||||
|
boundary[size - 1 :: size] = True
|
||||||
|
|
||||||
|
heights = self.balloon_poisson.poisson_balloon_heights(
|
||||||
|
points,
|
||||||
|
faces,
|
||||||
|
boundary,
|
||||||
|
0.45,
|
||||||
|
)
|
||||||
|
|
||||||
|
self.assertTrue(np.isfinite(heights).all())
|
||||||
|
self.assertTrue((heights[boundary] == 0.0).all())
|
||||||
|
self.assertGreater(heights[(size * size) // 2], 0.0)
|
||||||
|
|
||||||
|
def test_poisson_solver_has_no_absolute_scale_cutoff(self):
|
||||||
|
import numpy as np
|
||||||
|
|
||||||
|
points = np.asarray(
|
||||||
|
((0, 0), (2, 0), (2, 2), (0, 2), (1, 1)),
|
||||||
|
dtype=float,
|
||||||
|
)
|
||||||
|
faces = ((0, 1, 4), (1, 2, 4), (2, 3, 4), (3, 0, 4))
|
||||||
|
boundary = np.asarray((True, True, True, True, False))
|
||||||
|
original = self.balloon_poisson.poisson_balloon_heights(
|
||||||
|
points,
|
||||||
|
faces,
|
||||||
|
boundary,
|
||||||
|
0.45,
|
||||||
|
)
|
||||||
|
tiny = self.balloon_poisson.poisson_balloon_heights(
|
||||||
|
points * 1e-5,
|
||||||
|
faces,
|
||||||
|
boundary,
|
||||||
|
0.45,
|
||||||
|
)
|
||||||
|
|
||||||
|
self.assertAlmostEqual(tiny[4], original[4] * 1e-5, places=12)
|
||||||
|
|
||||||
def test_teddy_balloon_uses_chordal_axis_radius(self):
|
def test_teddy_balloon_uses_chordal_axis_radius(self):
|
||||||
import numpy as np
|
import numpy as np
|
||||||
|
|
||||||
@@ -742,15 +821,18 @@ class AddonTest(unittest.TestCase):
|
|||||||
|
|
||||||
points = np.asarray(((0, 0),), dtype=float)
|
points = np.asarray(((0, 0),), dtype=float)
|
||||||
boundary = np.asarray((True,))
|
boundary = np.asarray((True,))
|
||||||
with patch.object(
|
with (
|
||||||
|
patch.object(
|
||||||
self.balloon_geometry,
|
self.balloon_geometry,
|
||||||
"poisson_balloon_heights",
|
"poisson_balloon_heights",
|
||||||
return_value=np.asarray((1.0,)),
|
return_value=np.asarray((1.0,)),
|
||||||
) as poisson, patch.object(
|
) as poisson,
|
||||||
|
patch.object(
|
||||||
self.balloon_geometry,
|
self.balloon_geometry,
|
||||||
"teddy_balloon_heights",
|
"teddy_balloon_heights",
|
||||||
return_value=np.asarray((2.0,)),
|
return_value=np.asarray((2.0,)),
|
||||||
) as teddy:
|
) as teddy,
|
||||||
|
):
|
||||||
poisson_result = self.balloon_geometry.balloon_heights(
|
poisson_result = self.balloon_geometry.balloon_heights(
|
||||||
"POISSON", points, (), points, boundary, 0.45
|
"POISSON", points, (), points, boundary, 0.45
|
||||||
)
|
)
|
||||||
@@ -762,6 +844,7 @@ class AddonTest(unittest.TestCase):
|
|||||||
self.assertEqual(teddy_result[0], 2.0)
|
self.assertEqual(teddy_result[0], 2.0)
|
||||||
poisson.assert_called_once()
|
poisson.assert_called_once()
|
||||||
teddy.assert_called_once()
|
teddy.assert_called_once()
|
||||||
|
|
||||||
def test_balloon_geometry_has_a_shared_closed_seam(self):
|
def test_balloon_geometry_has_a_shared_closed_seam(self):
|
||||||
from collections import Counter
|
from collections import Counter
|
||||||
|
|
||||||
@@ -801,29 +884,204 @@ class AddonTest(unittest.TestCase):
|
|||||||
self.addon.CLASSES.index(self.balloon.ActivateBalloonTool),
|
self.addon.CLASSES.index(self.balloon.ActivateBalloonTool),
|
||||||
)
|
)
|
||||||
|
|
||||||
def test_balloon_ai_refine_is_registered_and_forwarded_to_ben2(self):
|
def test_balloon_ai_analysis_is_registered_and_forwarded(self):
|
||||||
import inspect
|
import inspect
|
||||||
|
|
||||||
self.assertIn(self.balloon.RefineBalloon, self.addon.CLASSES)
|
self.assertIn(self.balloon.AnalyzeBalloon, self.addon.CLASSES)
|
||||||
operator = self.balloon.ExtractBalloon
|
operator = self.balloon.ExtractBalloon
|
||||||
self.assertIn("ai_refine", operator.__annotations__)
|
self.assertIn("contour_analysis", operator.__annotations__)
|
||||||
|
self.assertIn("depth_analysis", operator.__annotations__)
|
||||||
|
self.assertIn("depth_model", operator.__annotations__)
|
||||||
settings = self.addon.properties.SnapFormSettings.__annotations__
|
settings = self.addon.properties.SnapFormSettings.__annotations__
|
||||||
self.assertIn("balloon_ai_refine", settings)
|
for name in (
|
||||||
self.assertFalse(settings["balloon_ai_refine"]["default"])
|
"balloon_contour_analysis",
|
||||||
|
"balloon_depth_analysis",
|
||||||
|
):
|
||||||
|
self.assertIn(name, settings)
|
||||||
|
self.assertFalse(settings[name]["default"])
|
||||||
|
self.assertIn("balloon_depth_model", settings)
|
||||||
|
self.assertEqual(settings["balloon_contour_analysis"]["name"], "Contour")
|
||||||
|
self.assertEqual(settings["balloon_depth_analysis"]["name"], "Depth")
|
||||||
|
|
||||||
execute_source = inspect.getsource(operator.execute)
|
execute_source = inspect.getsource(operator.execute)
|
||||||
invoke_source = inspect.getsource(operator.invoke)
|
invoke_source = inspect.getsource(operator.invoke)
|
||||||
request_source = inspect.getsource(self.balloon.RefineBalloon.request)
|
request_source = inspect.getsource(self.balloon.AnalyzeBalloon.request)
|
||||||
response_source = inspect.getsource(self.balloon.RefineBalloon.response)
|
response_source = inspect.getsource(self.balloon.AnalyzeBalloon.response)
|
||||||
tool_source = inspect.getsource(self.balloon.BalloonTool.draw_settings)
|
tool_source = inspect.getsource(self.balloon.BalloonTool.draw_settings)
|
||||||
self.assertIn("bpy.ops.snapform.refine_balloon", execute_source)
|
self.assertIn("bpy.ops.snapform.analyze_balloon", execute_source)
|
||||||
self.assertIn("prepare_image_input(", execute_source)
|
self.assertIn("prepare_image_input(", execute_source)
|
||||||
self.assertIn("settings.balloon_ai_refine", invoke_source)
|
self.assertIn(
|
||||||
|
"settings.balloon_contour_analysis",
|
||||||
|
invoke_source,
|
||||||
|
)
|
||||||
|
self.assertIn(
|
||||||
|
"settings.balloon_depth_analysis",
|
||||||
|
invoke_source,
|
||||||
|
)
|
||||||
|
self.assertIn("self.depth_model = settings.balloon_depth_model", invoke_source)
|
||||||
|
self.assertIn("require_model(BEN2_MODEL_KEY)", request_source)
|
||||||
|
self.assertIn(
|
||||||
|
"require_model(self.depth_model)",
|
||||||
|
request_source,
|
||||||
|
)
|
||||||
self.assertIn('"polygon": polygon', request_source)
|
self.assertIn('"polygon": polygon', request_source)
|
||||||
self.assertIn('result.file("contour")', response_source)
|
self.assertIn('result.file("contour")', response_source)
|
||||||
self.assertIn("extract_balloon_object(", response_source)
|
self.assertIn('result.file("depth")', response_source)
|
||||||
|
self.assertIn("depth_data=depth_data", response_source)
|
||||||
self.assertIn('push_undo("Extract Balloon")', response_source)
|
self.assertIn('push_undo("Extract Balloon")', response_source)
|
||||||
self.assertIn('"balloon_ai_refine"', tool_source)
|
self.assertIn('"balloon_contour_analysis"', tool_source)
|
||||||
|
self.assertIn('"balloon_depth_analysis"', tool_source)
|
||||||
|
self.assertIn('"balloon_depth_model"', tool_source)
|
||||||
|
|
||||||
|
def test_balloon_analysis_requires_only_enabled_models(self):
|
||||||
|
operator = self.balloon.AnalyzeBalloon()
|
||||||
|
operator.input_path = "input.png"
|
||||||
|
operator.selection_data = "[[0,0],[2,0],[0,2]]"
|
||||||
|
operator.resolution_level = 5
|
||||||
|
operator.depth_model = "MOGE2_VITB_NORMAL"
|
||||||
|
required = []
|
||||||
|
|
||||||
|
with (
|
||||||
|
patch.object(self.balloon, "require_environment"),
|
||||||
|
patch.object(
|
||||||
|
self.balloon,
|
||||||
|
"require_model",
|
||||||
|
side_effect=required.append,
|
||||||
|
),
|
||||||
|
patch.object(
|
||||||
|
self.balloon,
|
||||||
|
"production_device",
|
||||||
|
return_value="CUDA",
|
||||||
|
),
|
||||||
|
):
|
||||||
|
operator.contour_analysis = True
|
||||||
|
operator.depth_analysis = False
|
||||||
|
contour_request = operator.request(None)
|
||||||
|
operator.contour_analysis = False
|
||||||
|
operator.depth_analysis = True
|
||||||
|
depth_request = operator.request(None)
|
||||||
|
|
||||||
|
self.assertEqual(
|
||||||
|
required,
|
||||||
|
[
|
||||||
|
self.addon.properties.BEN2_MODEL_KEY,
|
||||||
|
"MOGE2_VITB_NORMAL",
|
||||||
|
],
|
||||||
|
)
|
||||||
|
self.assertTrue(contour_request["contour_analysis"])
|
||||||
|
self.assertFalse(contour_request["depth_analysis"])
|
||||||
|
self.assertFalse(depth_request["contour_analysis"])
|
||||||
|
self.assertTrue(depth_request["depth_analysis"])
|
||||||
|
self.assertEqual(depth_request["depth_model"], "MOGE2_VITB_NORMAL")
|
||||||
|
|
||||||
|
def test_balloon_depth_fields_and_shape_nodes(self):
|
||||||
|
import inspect
|
||||||
|
import numpy as np
|
||||||
|
|
||||||
|
depth_data = {
|
||||||
|
"depth_signal": np.asarray(
|
||||||
|
((1.0, -2.0), (0.5, 0.0)),
|
||||||
|
dtype=np.float32,
|
||||||
|
),
|
||||||
|
"baseline_vector": np.zeros(3, dtype=np.float32),
|
||||||
|
"valid": np.ones((2, 2), dtype=bool),
|
||||||
|
"origin": (0, 0),
|
||||||
|
"crop_size": (2, 2),
|
||||||
|
"source_size": (2, 2),
|
||||||
|
}
|
||||||
|
vertices = (
|
||||||
|
(0.0, 0.0, 2.0),
|
||||||
|
(0.0, 0.0, 2.0),
|
||||||
|
(0.0, 0.0, -2.0),
|
||||||
|
(0.0, 0.0, -2.0),
|
||||||
|
)
|
||||||
|
uv = ((0.0, 1.0), (1.0, 1.0), (0.0, 0.0), (1.0, 0.0))
|
||||||
|
|
||||||
|
values, local_plane = self.balloon_depth.balloon_depth_fields(
|
||||||
|
vertices,
|
||||||
|
uv,
|
||||||
|
depth_data,
|
||||||
|
)
|
||||||
|
|
||||||
|
np.testing.assert_allclose(values, (1.5, -4.5, 0.5, -0.5))
|
||||||
|
self.assertAlmostEqual(float(np.median(values)), 0.0)
|
||||||
|
np.testing.assert_allclose(local_plane, (0.0, 0.0, 0.0))
|
||||||
|
self.assertEqual(
|
||||||
|
self.balloon.orient_balloon_vertices(((1.0, 2.0, 3.0),)),
|
||||||
|
[(-3.0, -1.0, 2.0)],
|
||||||
|
)
|
||||||
|
restored = np.asarray(self.balloon.BALLOON_AXIS_TO_SOURCE) @ np.asarray(
|
||||||
|
(-3.0, -1.0, 2.0, 1.0)
|
||||||
|
)
|
||||||
|
np.testing.assert_allclose(restored, (1.0, 2.0, 3.0, 1.0))
|
||||||
|
self.assertEqual(
|
||||||
|
self.balloon.orient_balloon_base_plane((0.2, 0.3, 0.4)),
|
||||||
|
(0.2, -0.3, -0.4),
|
||||||
|
)
|
||||||
|
source = inspect.getsource(self.balloon_depth.balloon_shape_node_group)
|
||||||
|
controller_source = inspect.getsource(
|
||||||
|
self.balloon_depth.create_balloon_base_plane
|
||||||
|
)
|
||||||
|
self.assertIn('"Thickness"', source)
|
||||||
|
self.assertIn('"Depth Strength"', source)
|
||||||
|
self.assertNotIn('"Far Clamp"', source)
|
||||||
|
self.assertIn('"Double Sided Depth"', source)
|
||||||
|
self.assertIn('"Base Plane"', source)
|
||||||
|
self.assertIn('nodes.new("GeometryNodeObjectInfo")', source)
|
||||||
|
self.assertIn("controller.parent = balloon", controller_source)
|
||||||
|
self.assertIn(
|
||||||
|
'controller.empty_display_type = "SINGLE_ARROW"', controller_source
|
||||||
|
)
|
||||||
|
self.assertIn('profile_factor.operation = "ABSOLUTE"', source)
|
||||||
|
self.assertIn('profile_statistics.data_type = "FLOAT"', source)
|
||||||
|
self.assertIn('profile_max_safe.operation = "MAXIMUM"', source)
|
||||||
|
self.assertIn('profile_normalized.operation = "DIVIDE"', source)
|
||||||
|
self.assertIn('profile_smooth.interpolation_type = "SMOOTHERSTEP"', source)
|
||||||
|
self.assertIn('profile_active.operation = "MULTIPLY"', source)
|
||||||
|
self.assertIn('effective_normal.input_type = "VECTOR"', source)
|
||||||
|
self.assertNotIn("o_balloon_depth_mix", source)
|
||||||
|
self.assertIn('front.operation = "LESS_THAN"', source)
|
||||||
|
self.assertIn('depth_residual.operation = "ADD"', source)
|
||||||
|
self.assertIn('depth_calibration.inputs[1].default_value = 0.5', source)
|
||||||
|
self.assertIn('corrected_depth.operation = "ADD"', source)
|
||||||
|
self.assertIn(
|
||||||
|
'profile_statistics.outputs["Max"], corrected_depth.inputs[1]',
|
||||||
|
source,
|
||||||
|
)
|
||||||
|
self.assertNotIn("baseline_thickness", source)
|
||||||
|
self.assertIn('separate.outputs["X"], base_scale.inputs[0]', source)
|
||||||
|
self.assertIn('double_choice.outputs["Output"], combine.inputs["X"]', source)
|
||||||
|
self.assertIn('depth_mix = nodes.new("ShaderNodeMix")', source)
|
||||||
|
self.assertIn('depth_mix.data_type = "FLOAT"', source)
|
||||||
|
self.assertIn('depth_mix.clamp_factor = True', source)
|
||||||
|
self.assertIn('base_magnitude.outputs[0], depth_mix.inputs["A"]', source)
|
||||||
|
self.assertIn('depth_scale.outputs[0], depth_mix.inputs["B"]', source)
|
||||||
|
self.assertNotIn("profile_inverse", source)
|
||||||
|
self.assertNotIn("thickness_component", source)
|
||||||
|
self.assertNotIn("depth_component", source)
|
||||||
|
self.assertNotIn("blended_magnitude", source)
|
||||||
|
self.assertIn('plane_offset.operation = "DOT_PRODUCT"', source)
|
||||||
|
self.assertIn("controller.location = (0.0, 0.0, 0.0)", controller_source)
|
||||||
|
self.assertNotIn("clamp_distance", source)
|
||||||
|
self.assertNotIn("safety_clamped", source)
|
||||||
|
self.assertNotIn("crossing_choice", source)
|
||||||
|
self.assertIn('front_choice.input_type = "FLOAT"', source)
|
||||||
|
self.assertIn('double_choice.input_type = "FLOAT"', source)
|
||||||
|
self.assertNotIn("double_inverse", source)
|
||||||
|
self.assertNotIn("depth_limited", source)
|
||||||
|
self.assertNotIn("side_gate", source)
|
||||||
|
self.assertIn('base_frame.label = "Base Thickness"', source)
|
||||||
|
self.assertIn('plane_frame.label = "Middle Plane"', source)
|
||||||
|
self.assertIn('mix_frame.label = "Depth Mix"', source)
|
||||||
|
self.assertIn('output_frame.label = "Side Output"', source)
|
||||||
|
self.assertIn('side.operation = "SIGN"', source)
|
||||||
|
self.assertIn("BALLOON_DEPTH_ATTRIBUTE", source)
|
||||||
|
self.assertEqual(
|
||||||
|
self.balloon_depth.BALLOON_DEPTH_ATTRIBUTE,
|
||||||
|
"o_balloon_depth",
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
def test_balloon_redo_properties_use_mode_and_meter_resolution(self):
|
def test_balloon_redo_properties_use_mode_and_meter_resolution(self):
|
||||||
import inspect
|
import inspect
|
||||||
|
|
||||||
@@ -949,8 +1207,7 @@ class AddonTest(unittest.TestCase):
|
|||||||
import json
|
import json
|
||||||
|
|
||||||
matrix = tuple(
|
matrix = tuple(
|
||||||
tuple(row * 4 + column for column in range(4))
|
tuple(row * 4 + column for column in range(4)) for row in range(4)
|
||||||
for row in range(4)
|
|
||||||
)
|
)
|
||||||
values = json.loads(self.balloon.serialize_matrix(matrix))
|
values = json.loads(self.balloon.serialize_matrix(matrix))
|
||||||
|
|
||||||
@@ -962,34 +1219,57 @@ class AddonTest(unittest.TestCase):
|
|||||||
source = inspect.getsource(self.balloon.extract_balloon_object)
|
source = inspect.getsource(self.balloon.extract_balloon_object)
|
||||||
self.assertNotIn("objects.remove(source_object", source)
|
self.assertNotIn("objects.remove(source_object", source)
|
||||||
self.assertIn("balloon_name = source_name", source)
|
self.assertIn("balloon_name = source_name", source)
|
||||||
self.assertIn("selected_object = balloon if select_result else source_object", source)
|
self.assertIn(
|
||||||
self.assertIn("balloon.matrix_world = matrix_world.copy()", source)
|
"selected_object = balloon if select_result else source_object", source
|
||||||
|
)
|
||||||
|
self.assertIn(
|
||||||
|
"balloon.matrix_world = orient_balloon_world_matrix(matrix_world)",
|
||||||
|
source,
|
||||||
|
)
|
||||||
self.assertIn("create_material(", source)
|
self.assertIn("create_material(", source)
|
||||||
self.assertIn("material_name = material_data_name(source_object)", source)
|
self.assertIn("material_name = material_data_name(source_object)", source)
|
||||||
self.assertIn(
|
self.assertIn(
|
||||||
"reusable_balloon_material(material_name, color_name)",
|
"normal_required=uses_normal",
|
||||||
source,
|
source,
|
||||||
)
|
)
|
||||||
|
self.assertIn("mark_balloon_material(material, image, normal_image)", source)
|
||||||
|
self.assertIn("merge_balloon_normal_image(normal_image, update_normal)", source)
|
||||||
self.assertIn("if reuse_material", source)
|
self.assertIn("if reuse_material", source)
|
||||||
self.assertIn("color_image = image.copy()", source)
|
self.assertIn("color_image = image.copy()", source)
|
||||||
self.assertIn("color_image.name = color_name", source)
|
self.assertIn("color_image.name = color_name", source)
|
||||||
self.assertIn(
|
self.assertIn(
|
||||||
"material_name,\n color_image,\n 0.0,",
|
"material_name,\n color_image,\n 1.0,",
|
||||||
source,
|
source,
|
||||||
)
|
)
|
||||||
self.assertIn(
|
self.assertIn(
|
||||||
"surface_group_name=TEXTURE_MATERIAL_NODE_GROUP_NAME",
|
"else TEXTURE_MATERIAL_NODE_GROUP_NAME",
|
||||||
source,
|
source,
|
||||||
)
|
)
|
||||||
self.assertIn("owns_material", source)
|
self.assertIn("owns_material", source)
|
||||||
self.assertIn("owns_color_image", source)
|
self.assertIn("owns_color_image", source)
|
||||||
|
self.assertIn("balloon_depth_fields(", source)
|
||||||
|
self.assertIn("assign_balloon_shape_attributes(", source)
|
||||||
|
self.assertIn("inflation=INFLATION", source)
|
||||||
|
self.assertIn("create_balloon_base_plane(", source)
|
||||||
|
self.assertIn(
|
||||||
|
"add_balloon_shape_modifier(balloon, thickness, base_plane)", source
|
||||||
|
)
|
||||||
|
self.assertIn("BALLOON_MATERIAL_NODE_GROUP_NAME", source)
|
||||||
|
self.assertIn("normal_path=", source)
|
||||||
self.assertEqual(
|
self.assertEqual(
|
||||||
self.image_depth.TEXTURE_MATERIAL_NODE_GROUP_NAME,
|
self.image_depth.TEXTURE_MATERIAL_NODE_GROUP_NAME,
|
||||||
"O Texture Layer",
|
"O Texture Layer",
|
||||||
)
|
)
|
||||||
|
|
||||||
def test_balloon_material_reuse_requires_matching_image_and_node_group(self):
|
def test_balloon_material_reuse_requires_matching_image_and_node_group(self):
|
||||||
|
class MarkedMaterial(dict):
|
||||||
|
def __init__(self, name, **properties):
|
||||||
|
super().__init__(properties)
|
||||||
|
self.name = name
|
||||||
|
self.node_tree = None
|
||||||
|
|
||||||
color_image = SimpleNamespace(name="cow.color")
|
color_image = SimpleNamespace(name="cow.color")
|
||||||
|
other_image = SimpleNamespace(name="other")
|
||||||
texture_node = SimpleNamespace(type="TEX_IMAGE", image=color_image)
|
texture_node = SimpleNamespace(type="TEX_IMAGE", image=color_image)
|
||||||
texture_layer = SimpleNamespace(
|
texture_layer = SimpleNamespace(
|
||||||
type="GROUP",
|
type="GROUP",
|
||||||
@@ -999,6 +1279,23 @@ class AddonTest(unittest.TestCase):
|
|||||||
name="cow.001",
|
name="cow.001",
|
||||||
node_tree=SimpleNamespace(nodes=[texture_node, texture_layer]),
|
node_tree=SimpleNamespace(nodes=[texture_node, texture_layer]),
|
||||||
)
|
)
|
||||||
|
marked = MarkedMaterial(
|
||||||
|
"unrelated-name",
|
||||||
|
snapform_balloon_material=True,
|
||||||
|
snapform_source_image=color_image,
|
||||||
|
)
|
||||||
|
marked_other = MarkedMaterial(
|
||||||
|
"cow",
|
||||||
|
snapform_balloon_material=True,
|
||||||
|
snapform_source_image=other_image,
|
||||||
|
)
|
||||||
|
normal_image = SimpleNamespace(name="cow.normal")
|
||||||
|
marked_normal = MarkedMaterial(
|
||||||
|
"depth-material",
|
||||||
|
snapform_balloon_material=True,
|
||||||
|
snapform_source_image=color_image,
|
||||||
|
snapform_balloon_normal_image=normal_image,
|
||||||
|
)
|
||||||
wrong_group = SimpleNamespace(
|
wrong_group = SimpleNamespace(
|
||||||
name="cow",
|
name="cow",
|
||||||
node_tree=SimpleNamespace(
|
node_tree=SimpleNamespace(
|
||||||
@@ -1013,14 +1310,37 @@ class AddonTest(unittest.TestCase):
|
|||||||
)
|
)
|
||||||
self.fake_bpy.data = SimpleNamespace(
|
self.fake_bpy.data = SimpleNamespace(
|
||||||
objects=SimpleNamespace(get=lambda _name: None),
|
objects=SimpleNamespace(get=lambda _name: None),
|
||||||
materials=[wrong_group, compatible],
|
materials=[
|
||||||
|
marked_other,
|
||||||
|
wrong_group,
|
||||||
|
compatible,
|
||||||
|
marked_normal,
|
||||||
|
marked,
|
||||||
|
],
|
||||||
images=[color_image],
|
images=[color_image],
|
||||||
)
|
)
|
||||||
|
|
||||||
|
self.assertIs(
|
||||||
|
self.balloon.reusable_balloon_material(
|
||||||
|
"cow",
|
||||||
|
"cow.color",
|
||||||
|
color_image,
|
||||||
|
),
|
||||||
|
marked,
|
||||||
|
)
|
||||||
self.assertIs(
|
self.assertIs(
|
||||||
self.balloon.reusable_balloon_material("cow", "cow.color"),
|
self.balloon.reusable_balloon_material("cow", "cow.color"),
|
||||||
compatible,
|
compatible,
|
||||||
)
|
)
|
||||||
|
self.assertIs(
|
||||||
|
self.balloon.reusable_balloon_material(
|
||||||
|
"cow",
|
||||||
|
"cow.color",
|
||||||
|
color_image,
|
||||||
|
normal_required=True,
|
||||||
|
),
|
||||||
|
marked_normal,
|
||||||
|
)
|
||||||
self.assertIs(
|
self.assertIs(
|
||||||
self.balloon.reusable_color_image("cow.color"),
|
self.balloon.reusable_color_image("cow.color"),
|
||||||
color_image,
|
color_image,
|
||||||
@@ -1056,31 +1376,40 @@ class AddonTest(unittest.TestCase):
|
|||||||
(True, True, True),
|
(True, True, True),
|
||||||
)
|
)
|
||||||
for environment_ready, models_ready, expected in states:
|
for environment_ready, models_ready, expected in states:
|
||||||
with self.subTest(
|
with (
|
||||||
|
self.subTest(
|
||||||
environment_ready=environment_ready,
|
environment_ready=environment_ready,
|
||||||
models_ready=models_ready,
|
models_ready=models_ready,
|
||||||
expected=expected,
|
expected=expected,
|
||||||
), patch.object(
|
),
|
||||||
|
patch.object(
|
||||||
self.addon.runtime,
|
self.addon.runtime,
|
||||||
"environment_ready",
|
"environment_ready",
|
||||||
return_value=environment_ready,
|
return_value=environment_ready,
|
||||||
), patch.object(
|
),
|
||||||
|
patch.object(
|
||||||
self.addon.runtime,
|
self.addon.runtime,
|
||||||
"server_busy",
|
"server_busy",
|
||||||
return_value=False,
|
return_value=False,
|
||||||
), patch(
|
),
|
||||||
|
patch(
|
||||||
"snapform.properties.required_models_ready",
|
"snapform.properties.required_models_ready",
|
||||||
return_value=models_ready,
|
return_value=models_ready,
|
||||||
|
),
|
||||||
):
|
):
|
||||||
for module, operator in operators:
|
for module, operator in operators:
|
||||||
with self.subTest(operator=operator.__name__), patch.object(
|
with (
|
||||||
|
self.subTest(operator=operator.__name__),
|
||||||
|
patch.object(
|
||||||
module,
|
module,
|
||||||
"is_image_empty",
|
"is_image_empty",
|
||||||
return_value=True,
|
return_value=True,
|
||||||
), patch.object(
|
),
|
||||||
|
patch.object(
|
||||||
module,
|
module,
|
||||||
"video_total_frames",
|
"video_total_frames",
|
||||||
return_value=1,
|
return_value=1,
|
||||||
|
),
|
||||||
):
|
):
|
||||||
self.assertEqual(operator.poll(context), expected)
|
self.assertEqual(operator.poll(context), expected)
|
||||||
|
|
||||||
@@ -1095,18 +1424,20 @@ class AddonTest(unittest.TestCase):
|
|||||||
(moge / "model.onnx").touch()
|
(moge / "model.onnx").touch()
|
||||||
(ben2 / "model_fp16.onnx").touch()
|
(ben2 / "model_fp16.onnx").touch()
|
||||||
|
|
||||||
with patch.object(
|
with (
|
||||||
|
patch.object(
|
||||||
self.addon.runtime,
|
self.addon.runtime,
|
||||||
"storage_root",
|
"storage_root",
|
||||||
return_value=root,
|
return_value=root,
|
||||||
), patch.object(
|
),
|
||||||
|
patch.object(
|
||||||
self.addon.runtime,
|
self.addon.runtime,
|
||||||
"resource",
|
"resource",
|
||||||
side_effect=AssertionError("UI must not start the Server"),
|
side_effect=AssertionError("UI must not start the Server"),
|
||||||
|
),
|
||||||
):
|
):
|
||||||
self.assertTrue(properties.required_models_ready())
|
self.assertTrue(properties.required_models_ready())
|
||||||
|
|
||||||
|
|
||||||
def test_runtime_request_uses_request_response_shape(self):
|
def test_runtime_request_uses_request_response_shape(self):
|
||||||
result = object()
|
result = object()
|
||||||
responses = []
|
responses = []
|
||||||
@@ -1224,13 +1555,12 @@ class AddonTest(unittest.TestCase):
|
|||||||
operator.resolution_level = 6
|
operator.resolution_level = 6
|
||||||
operator.remove_background = False
|
operator.remove_background = False
|
||||||
settings = SimpleNamespace(video_frames=12)
|
settings = SimpleNamespace(video_frames=12)
|
||||||
context = SimpleNamespace(
|
context = SimpleNamespace(scene=SimpleNamespace(snapform_settings=settings))
|
||||||
scene=SimpleNamespace(snapform_settings=settings)
|
|
||||||
)
|
|
||||||
|
|
||||||
with patch.object(
|
with (
|
||||||
self.depth_operator, "require_environment"
|
patch.object(self.depth_operator, "require_environment"),
|
||||||
), patch.object(self.depth_operator, "require_model"):
|
patch.object(self.depth_operator, "require_model"),
|
||||||
|
):
|
||||||
parameters = operator.request(context)
|
parameters = operator.request(context)
|
||||||
|
|
||||||
self.assertEqual(parameters["input"], __file__)
|
self.assertEqual(parameters["input"], __file__)
|
||||||
@@ -1247,9 +1577,7 @@ class AddonTest(unittest.TestCase):
|
|||||||
result = object()
|
result = object()
|
||||||
context = object()
|
context = object()
|
||||||
|
|
||||||
with patch.object(
|
with patch.object(self.depth_operator, "build_moge2_depth_response") as build:
|
||||||
self.depth_operator, "build_moge2_depth_response"
|
|
||||||
) as build:
|
|
||||||
operator.response(context, result)
|
operator.response(context, result)
|
||||||
|
|
||||||
build.assert_called_once_with(
|
build.assert_called_once_with(
|
||||||
@@ -1280,8 +1608,9 @@ class AddonTest(unittest.TestCase):
|
|||||||
calls = []
|
calls = []
|
||||||
self.fake_bpy.ops = SimpleNamespace(
|
self.fake_bpy.ops = SimpleNamespace(
|
||||||
snapform=SimpleNamespace(
|
snapform=SimpleNamespace(
|
||||||
generate_moge2_depth=lambda **options: calls.append(options)
|
generate_moge2_depth=lambda **options: (
|
||||||
or {"RUNNING_MODAL"}
|
calls.append(options) or {"RUNNING_MODAL"}
|
||||||
|
)
|
||||||
)
|
)
|
||||||
)
|
)
|
||||||
|
|
||||||
@@ -1296,5 +1625,6 @@ class AddonTest(unittest.TestCase):
|
|||||||
self.assertEqual(calls[0]["source_object_name"], "Reference")
|
self.assertEqual(calls[0]["source_object_name"], "Reference")
|
||||||
self.assertFalse(any(key.startswith("snapform_") for key in calls[0]))
|
self.assertFalse(any(key.startswith("snapform_") for key in calls[0]))
|
||||||
|
|
||||||
|
|
||||||
if __name__ == "__main__":
|
if __name__ == "__main__":
|
||||||
unittest.main()
|
unittest.main()
|
||||||
|
|||||||
@@ -15,7 +15,8 @@ PROJECT_ROOT = Path(__file__).resolve().parents[1]
|
|||||||
def load_server_module():
|
def load_server_module():
|
||||||
server_directory = PROJECT_ROOT / "src" / "snapform" / "server"
|
server_directory = PROJECT_ROOT / "src" / "snapform" / "server"
|
||||||
specification = importlib.util.spec_from_file_location(
|
specification = importlib.util.spec_from_file_location(
|
||||||
"server", server_directory / "__init__.py",
|
"server",
|
||||||
|
server_directory / "__init__.py",
|
||||||
submodule_search_locations=[str(server_directory)],
|
submodule_search_locations=[str(server_directory)],
|
||||||
)
|
)
|
||||||
module = importlib.util.module_from_spec(specification)
|
module = importlib.util.module_from_spec(specification)
|
||||||
@@ -51,14 +52,17 @@ class ServerTest(unittest.TestCase):
|
|||||||
manager = self.server_module.ModelManager(Path("."))
|
manager = self.server_module.ModelManager(Path("."))
|
||||||
downloads = []
|
downloads = []
|
||||||
context = SimpleNamespace(progress=lambda *_args: None)
|
context = SimpleNamespace(progress=lambda *_args: None)
|
||||||
with patch.object(
|
with (
|
||||||
|
patch.object(
|
||||||
manager,
|
manager,
|
||||||
"ready",
|
"ready",
|
||||||
side_effect=lambda key: key == "BEN2_BASE",
|
side_effect=lambda key: key == "BEN2_BASE",
|
||||||
), patch.object(
|
),
|
||||||
|
patch.object(
|
||||||
manager,
|
manager,
|
||||||
"_download",
|
"_download",
|
||||||
side_effect=lambda _context, key, _progress: downloads.append(key),
|
side_effect=lambda _context, key, _progress: downloads.append(key),
|
||||||
|
),
|
||||||
):
|
):
|
||||||
result = manager.download_missing(
|
result = manager.download_missing(
|
||||||
context,
|
context,
|
||||||
@@ -215,7 +219,7 @@ class ServerTest(unittest.TestCase):
|
|||||||
{
|
{
|
||||||
"download-model",
|
"download-model",
|
||||||
"download-required-models",
|
"download-required-models",
|
||||||
"refine-balloon",
|
"analyze-balloon",
|
||||||
"remove-background",
|
"remove-background",
|
||||||
"extract-frames",
|
"extract-frames",
|
||||||
"generate-depth",
|
"generate-depth",
|
||||||
@@ -254,15 +258,11 @@ class ServerTest(unittest.TestCase):
|
|||||||
|
|
||||||
def test_cancelling_job_keeps_server_and_model_cache_alive(self):
|
def test_cancelling_job_keeps_server_and_model_cache_alive(self):
|
||||||
fake_models = SimpleNamespace(
|
fake_models = SimpleNamespace(
|
||||||
snapshot=lambda: {
|
snapshot=lambda: {"loaded": {"ben2": "BEN2_BASE", "da3": ""}},
|
||||||
"loaded": {"ben2": "BEN2_BASE", "da3": ""}
|
|
||||||
},
|
|
||||||
clear=lambda: None,
|
clear=lambda: None,
|
||||||
close=lambda: None,
|
close=lambda: None,
|
||||||
)
|
)
|
||||||
with patch.object(
|
with patch.object(self.server_module, "ModelManager", return_value=fake_models):
|
||||||
self.server_module, "ModelManager", return_value=fake_models
|
|
||||||
):
|
|
||||||
server = self.create_server()
|
server = self.create_server()
|
||||||
started = threading.Event()
|
started = threading.Event()
|
||||||
|
|
||||||
@@ -287,9 +287,7 @@ class ServerTest(unittest.TestCase):
|
|||||||
)
|
)
|
||||||
|
|
||||||
def test_job_progress_is_kept_in_server_memory(self):
|
def test_job_progress_is_kept_in_server_memory(self):
|
||||||
server = self.server_module.JobServer(
|
server = self.server_module.JobServer("Test", storage_root=self.storage_root)
|
||||||
"Test", storage_root=self.storage_root
|
|
||||||
)
|
|
||||||
|
|
||||||
@server.job("example")
|
@server.job("example")
|
||||||
def example(context, _parameters):
|
def example(context, _parameters):
|
||||||
@@ -326,9 +324,7 @@ class ServerTest(unittest.TestCase):
|
|||||||
clear=lambda: events.append("clear"),
|
clear=lambda: events.append("clear"),
|
||||||
close=lambda: events.append("close"),
|
close=lambda: events.append("close"),
|
||||||
)
|
)
|
||||||
server = self.server_module.JobServer(
|
server = self.server_module.JobServer("Test", storage_root=self.storage_root)
|
||||||
"Test", storage_root=self.storage_root
|
|
||||||
)
|
|
||||||
server.add_resource("model_manager", resource)
|
server.add_resource("model_manager", resource)
|
||||||
|
|
||||||
@server.job("example")
|
@server.job("example")
|
||||||
@@ -356,9 +352,7 @@ class ServerTest(unittest.TestCase):
|
|||||||
clear=lambda: None,
|
clear=lambda: None,
|
||||||
close=lambda: None,
|
close=lambda: None,
|
||||||
)
|
)
|
||||||
with patch.object(
|
with patch.object(self.server_module, "ModelManager", return_value=fake_models):
|
||||||
self.server_module, "ModelManager", return_value=fake_models
|
|
||||||
):
|
|
||||||
server = self.create_server()
|
server = self.create_server()
|
||||||
with tempfile.TemporaryDirectory() as directory:
|
with tempfile.TemporaryDirectory() as directory:
|
||||||
root = Path(directory)
|
root = Path(directory)
|
||||||
@@ -376,9 +370,7 @@ class ServerTest(unittest.TestCase):
|
|||||||
job_id="job",
|
job_id="job",
|
||||||
)
|
)
|
||||||
status = self.wait_for_job(context)
|
status = self.wait_for_job(context)
|
||||||
self.assertTrue(
|
self.assertTrue((context.directory / "color.0001.jpg").is_file())
|
||||||
(context.directory / "color.0001.jpg").is_file()
|
|
||||||
)
|
|
||||||
|
|
||||||
self.assertEqual(status["state"], "succeeded")
|
self.assertEqual(status["state"], "succeeded")
|
||||||
|
|
||||||
@@ -395,7 +387,7 @@ class ServerTest(unittest.TestCase):
|
|||||||
pixels[1, 1, 3] = 64
|
pixels[1, 1, 3] = 64
|
||||||
Image.fromarray(pixels, mode="RGBA").save(source_path)
|
Image.fromarray(pixels, mode="RGBA").save(source_path)
|
||||||
|
|
||||||
_path, _selection, origin = balloon.create_lasso_input(
|
_path, _selection, origin, source_size = balloon.create_lasso_input(
|
||||||
source_path,
|
source_path,
|
||||||
((1, 1), (5, 1), (1, 5)),
|
((1, 1), (5, 1), (1, 5)),
|
||||||
output_path,
|
output_path,
|
||||||
@@ -403,6 +395,7 @@ class ServerTest(unittest.TestCase):
|
|||||||
result = np.asarray(Image.open(output_path).convert("RGBA"))
|
result = np.asarray(Image.open(output_path).convert("RGBA"))
|
||||||
|
|
||||||
self.assertEqual(origin, (1, 1))
|
self.assertEqual(origin, (1, 1))
|
||||||
|
self.assertEqual(source_size, (6, 6))
|
||||||
self.assertEqual(result.shape, (4, 4, 4))
|
self.assertEqual(result.shape, (4, 4, 4))
|
||||||
self.assertEqual(tuple(result[0, 0]), (20, 40, 60, 64))
|
self.assertEqual(tuple(result[0, 0]), (20, 40, 60, 64))
|
||||||
self.assertEqual(tuple(result[3, 3]), (0, 0, 0, 0))
|
self.assertEqual(tuple(result[3, 3]), (0, 0, 0, 0))
|
||||||
@@ -437,7 +430,9 @@ class ServerTest(unittest.TestCase):
|
|||||||
{
|
{
|
||||||
"input": str(source_path),
|
"input": str(source_path),
|
||||||
"polygon": ((1, 1), (5, 1), (1, 5)),
|
"polygon": ((1, 1), (5, 1), (1, 5)),
|
||||||
"model": "BEN2_BASE",
|
"contour_analysis": True,
|
||||||
|
"depth_analysis": False,
|
||||||
|
"contour_model": "BEN2_BASE",
|
||||||
"device": "cpu",
|
"device": "cpu",
|
||||||
},
|
},
|
||||||
)
|
)
|
||||||
@@ -448,6 +443,148 @@ class ServerTest(unittest.TestCase):
|
|||||||
self.assertTrue((job_directory / "foreground.png").is_file())
|
self.assertTrue((job_directory / "foreground.png").is_file())
|
||||||
self.assertTrue((job_directory / "contour.json").is_file())
|
self.assertTrue((job_directory / "contour.json").is_file())
|
||||||
|
|
||||||
|
def test_balloon_depth_analysis_normalizes_moge_distance(self):
|
||||||
|
import numpy as np
|
||||||
|
|
||||||
|
balloon = sys.modules["server.jobs.balloon"]
|
||||||
|
y, x = np.indices((7, 9), dtype=np.float32)
|
||||||
|
depth = 2.0 + x * 0.1 + y * 0.2
|
||||||
|
depth[3, 4] -= 0.5
|
||||||
|
depth[1, 1] += 3.0
|
||||||
|
model_mask = np.ones_like(depth, dtype=bool)
|
||||||
|
selection_mask = np.ones_like(depth, dtype=bool)
|
||||||
|
selection_mask[0, 0] = False
|
||||||
|
|
||||||
|
depth_signal, valid, reference, baseline_vector = balloon.normalized_moge_depth(
|
||||||
|
depth,
|
||||||
|
model_mask,
|
||||||
|
selection_mask,
|
||||||
|
)
|
||||||
|
normalized_x = x / (depth.shape[1] - 1) * 2.0 - 1.0
|
||||||
|
normalized_y = y / (depth.shape[0] - 1) * 2.0 - 1.0
|
||||||
|
fitted_plane = (
|
||||||
|
baseline_vector[0] * normalized_x
|
||||||
|
+ baseline_vector[1] * normalized_y
|
||||||
|
+ baseline_vector[2]
|
||||||
|
)
|
||||||
|
displacement = depth_signal - fitted_plane
|
||||||
|
|
||||||
|
self.assertFalse(valid[0, 0])
|
||||||
|
self.assertEqual(depth_signal[0, 0], 0.0)
|
||||||
|
self.assertGreater(float(displacement[3, 4]), 1.0)
|
||||||
|
self.assertLess(float(displacement[3, 4]), 1.01)
|
||||||
|
self.assertLess(displacement[1, 1], -1.0)
|
||||||
|
self.assertGreater(reference, 2.0)
|
||||||
|
self.assertEqual(baseline_vector.shape, (3,))
|
||||||
|
self.assertLess(float(displacement.min()), -1.0)
|
||||||
|
self.assertLess(float(displacement.max()), 1.01)
|
||||||
|
self.assertLess(float(np.percentile(displacement, 90.0)), 0.01)
|
||||||
|
|
||||||
|
positive = displacement[valid & (displacement > 0.0)]
|
||||||
|
self.assertAlmostEqual(
|
||||||
|
float(np.sum(np.square(positive)) / np.sum(positive)),
|
||||||
|
1.0,
|
||||||
|
places=5,
|
||||||
|
)
|
||||||
|
|
||||||
|
def test_balloon_relative_depth_ignores_absolute_distance_and_scale(self):
|
||||||
|
import numpy as np
|
||||||
|
|
||||||
|
balloon = sys.modules["server.jobs.balloon"]
|
||||||
|
y, x = np.indices((8, 10), dtype=np.float32)
|
||||||
|
depth = 4.0 + x * 0.04 + y * 0.03
|
||||||
|
depth[2:6, 3:7] -= np.asarray(
|
||||||
|
(
|
||||||
|
(0.1, 0.2, 0.3, 0.1),
|
||||||
|
(0.2, 0.4, 0.8, 0.2),
|
||||||
|
(0.15, 0.3, 0.5, 0.15),
|
||||||
|
(0.05, 0.1, 0.2, 0.05),
|
||||||
|
),
|
||||||
|
dtype=np.float32,
|
||||||
|
)
|
||||||
|
mask = np.ones_like(depth, dtype=bool)
|
||||||
|
|
||||||
|
original = balloon.normalized_moge_depth(depth, mask, mask)
|
||||||
|
farther = balloon.normalized_moge_depth(depth + 100.0, mask, mask)
|
||||||
|
rescaled = balloon.normalized_moge_depth(depth * 7.0, mask, mask)
|
||||||
|
|
||||||
|
np.testing.assert_allclose(farther[0], original[0], atol=2e-4)
|
||||||
|
np.testing.assert_allclose(farther[3], original[3], atol=2e-4)
|
||||||
|
np.testing.assert_allclose(rescaled[0], original[0], atol=2e-4)
|
||||||
|
np.testing.assert_allclose(rescaled[3], original[3], atol=2e-4)
|
||||||
|
def test_balloon_depth_job_sends_masked_crop_only_to_moge(self):
|
||||||
|
import numpy as np
|
||||||
|
from PIL import Image
|
||||||
|
|
||||||
|
balloon = sys.modules["server.jobs.balloon"]
|
||||||
|
source_path = self.storage_root / "depth-source.png"
|
||||||
|
pixels = np.full((6, 6, 4), (80, 100, 120, 255), dtype=np.uint8)
|
||||||
|
Image.fromarray(pixels, mode="RGBA").save(source_path)
|
||||||
|
job_directory = self.storage_root / "depth-job"
|
||||||
|
context = SimpleNamespace(
|
||||||
|
directory=job_directory,
|
||||||
|
progress=lambda *_args: None,
|
||||||
|
check_cancelled=lambda: None,
|
||||||
|
resource=lambda _name: SimpleNamespace(),
|
||||||
|
)
|
||||||
|
received = []
|
||||||
|
|
||||||
|
def infer(_manager, _parameters, images, **_kwargs):
|
||||||
|
image = Image.open(images[0]).convert("RGBA")
|
||||||
|
received.append(np.asarray(image))
|
||||||
|
height, width = image.size[1], image.size[0]
|
||||||
|
depth = np.linspace(
|
||||||
|
1.0,
|
||||||
|
2.0,
|
||||||
|
height * width,
|
||||||
|
dtype=np.float32,
|
||||||
|
).reshape(height, width)
|
||||||
|
return SimpleNamespace(
|
||||||
|
depth=depth[None],
|
||||||
|
mask=np.ones((1, height, width), dtype=bool),
|
||||||
|
normal=np.broadcast_to(
|
||||||
|
np.asarray((0.0, 0.0, -1.0), dtype=np.float32),
|
||||||
|
(1, height, width, 3),
|
||||||
|
).copy(),
|
||||||
|
)
|
||||||
|
|
||||||
|
with (
|
||||||
|
patch.object(balloon.moge2, "infer", side_effect=infer),
|
||||||
|
patch.object(
|
||||||
|
balloon,
|
||||||
|
"infer_foreground",
|
||||||
|
side_effect=AssertionError("BEN2 should not run"),
|
||||||
|
),
|
||||||
|
):
|
||||||
|
result = balloon.run(
|
||||||
|
context,
|
||||||
|
{
|
||||||
|
"input": str(source_path),
|
||||||
|
"polygon": ((1, 1), (5, 1), (1, 5)),
|
||||||
|
"contour_analysis": False,
|
||||||
|
"depth_analysis": True,
|
||||||
|
"depth_model": "MOGE2_VITS_NORMAL",
|
||||||
|
"resolution_level": 5,
|
||||||
|
"device": "cpu",
|
||||||
|
},
|
||||||
|
)
|
||||||
|
|
||||||
|
self.assertEqual(
|
||||||
|
result,
|
||||||
|
{"depth": "depth.npz", "normal": "normal.png"},
|
||||||
|
)
|
||||||
|
self.assertEqual(tuple(received[0][3, 3]), (0, 0, 0, 0))
|
||||||
|
self.assertFalse((job_directory / "foreground.png").exists())
|
||||||
|
with np.load(job_directory / "depth.npz", allow_pickle=False) as data:
|
||||||
|
self.assertEqual(data["depth_signal"].shape, (4, 4))
|
||||||
|
self.assertEqual(data["baseline_vector"].shape, (3,))
|
||||||
|
self.assertEqual(tuple(data["origin"]), (1, 1))
|
||||||
|
self.assertEqual(tuple(data["source_size"]), (6, 6))
|
||||||
|
self.assertTrue(data["valid"].any())
|
||||||
|
normal = np.asarray(Image.open(job_directory / "normal.png").convert("RGB"))
|
||||||
|
self.assertEqual(normal.shape, (6, 6, 3))
|
||||||
|
self.assertEqual(tuple(normal[0, 0]), (128, 128, 255))
|
||||||
|
|
||||||
def test_balloon_contour_uses_alpha_above_half_and_largest_component(self):
|
def test_balloon_contour_uses_alpha_above_half_and_largest_component(self):
|
||||||
import numpy as np
|
import numpy as np
|
||||||
|
|
||||||
@@ -469,5 +606,6 @@ class ServerTest(unittest.TestCase):
|
|||||||
{(12, 21), (15, 21), (15, 24), (12, 24)},
|
{(12, 21), (15, 21), (15, 24), (12, 24)},
|
||||||
)
|
)
|
||||||
|
|
||||||
|
|
||||||
if __name__ == "__main__":
|
if __name__ == "__main__":
|
||||||
unittest.main()
|
unittest.main()
|
||||||
|
|||||||
136
tools/validate_balloon_asset.py
Normal file
136
tools/validate_balloon_asset.py
Normal file
@@ -0,0 +1,136 @@
|
|||||||
|
from pathlib import Path
|
||||||
|
import sys
|
||||||
|
|
||||||
|
import bpy
|
||||||
|
|
||||||
|
|
||||||
|
PROJECT_ROOT = Path(__file__).parents[1]
|
||||||
|
sys.path.insert(0, str(PROJECT_ROOT / "src"))
|
||||||
|
sys.path.insert(
|
||||||
|
0,
|
||||||
|
str(next((PROJECT_ROOT / "src" / "snapform" / "wheels").glob("blendjob-*.whl"))),
|
||||||
|
)
|
||||||
|
|
||||||
|
from snapform.operators.balloon.depth import (
|
||||||
|
BALLOON_DEPTH_ATTRIBUTE,
|
||||||
|
add_balloon_shape_modifier,
|
||||||
|
balloon_shape_node_group,
|
||||||
|
)
|
||||||
|
from snapform.operators.balloon.operator import (
|
||||||
|
mark_balloon_material,
|
||||||
|
merge_balloon_normal_image,
|
||||||
|
reusable_balloon_material,
|
||||||
|
)
|
||||||
|
from snapform.operators.material import (
|
||||||
|
BALLOON_MATERIAL_NODE_GROUP_NAME,
|
||||||
|
material_node_group,
|
||||||
|
set_modifier_input,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def input_socket(group, name):
|
||||||
|
return next(
|
||||||
|
item
|
||||||
|
for item in group.interface.items_tree
|
||||||
|
if item.item_type == "SOCKET"
|
||||||
|
and item.in_out == "INPUT"
|
||||||
|
and item.name == name
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def main():
|
||||||
|
bpy.ops.wm.read_factory_settings(use_empty=True)
|
||||||
|
group = balloon_shape_node_group()
|
||||||
|
assert group.is_modifier
|
||||||
|
assert input_socket(group, "Double Sided Depth").default_value is False
|
||||||
|
smoother = next(
|
||||||
|
node
|
||||||
|
for node in group.nodes
|
||||||
|
if node.bl_idname == "ShaderNodeMapRange"
|
||||||
|
and node.interpolation_type == "SMOOTHERSTEP"
|
||||||
|
)
|
||||||
|
assert smoother.clamp
|
||||||
|
assert not any(
|
||||||
|
node.bl_idname == "ShaderNodeMath"
|
||||||
|
and node.operation == "MAXIMUM"
|
||||||
|
and node.label == "Clamp Depth"
|
||||||
|
for node in group.nodes
|
||||||
|
)
|
||||||
|
|
||||||
|
mesh = bpy.data.meshes.new("Balloon Validation Mesh")
|
||||||
|
mesh.from_pydata(
|
||||||
|
(
|
||||||
|
(-1.0, 0.0, 0.0),
|
||||||
|
(-0.5, 0.0, 0.0),
|
||||||
|
(0.0, 0.0, 0.0),
|
||||||
|
(1.0, 0.0, 0.0),
|
||||||
|
),
|
||||||
|
(),
|
||||||
|
(),
|
||||||
|
)
|
||||||
|
depth = mesh.attributes.new(BALLOON_DEPTH_ATTRIBUTE, "FLOAT", "POINT")
|
||||||
|
depth.data.foreach_set("value", (0.0, 2.0, 2.0, 2.0))
|
||||||
|
obj = bpy.data.objects.new("Balloon Validation", mesh)
|
||||||
|
bpy.context.scene.collection.objects.link(obj)
|
||||||
|
modifier = add_balloon_shape_modifier(obj, 1.0)
|
||||||
|
evaluated = obj.evaluated_get(bpy.context.evaluated_depsgraph_get())
|
||||||
|
result = evaluated.to_mesh()
|
||||||
|
try:
|
||||||
|
x = [vertex.co.x for vertex in result.vertices]
|
||||||
|
assert abs(x[0] + 1.0) < 1e-5
|
||||||
|
assert abs(x[1] + 1.25) < 1e-5
|
||||||
|
assert abs(x[2]) < 1e-5
|
||||||
|
assert abs(x[3] - 1.0) < 1e-5
|
||||||
|
finally:
|
||||||
|
evaluated.to_mesh_clear()
|
||||||
|
|
||||||
|
set_modifier_input(modifier, input_socket(group, "Thickness").identifier, 2.0)
|
||||||
|
obj.update_tag(refresh={"DATA"})
|
||||||
|
bpy.context.view_layer.update()
|
||||||
|
evaluated = obj.evaluated_get(bpy.context.evaluated_depsgraph_get())
|
||||||
|
result = evaluated.to_mesh()
|
||||||
|
try:
|
||||||
|
x = [vertex.co.x for vertex in result.vertices]
|
||||||
|
assert abs(x[0] + 1.0) < 1e-5
|
||||||
|
assert abs(x[3] - 2.0) < 1e-5
|
||||||
|
finally:
|
||||||
|
evaluated.to_mesh_clear()
|
||||||
|
|
||||||
|
material_group = material_node_group(
|
||||||
|
surface_group_name=BALLOON_MATERIAL_NODE_GROUP_NAME,
|
||||||
|
)
|
||||||
|
normal_map = next(
|
||||||
|
node
|
||||||
|
for node in material_group.nodes
|
||||||
|
if node.bl_idname == "ShaderNodeNormalMap"
|
||||||
|
)
|
||||||
|
assert normal_map.space == "TANGENT"
|
||||||
|
assert abs(input_socket(material_group, "Normal Strength").default_value - 0.5) < 1e-6
|
||||||
|
|
||||||
|
source_image = bpy.data.images.new("Reuse Source", 2, 1)
|
||||||
|
shared_normal = bpy.data.images.new("Shared Normal", 2, 1)
|
||||||
|
shared_normal.pixels.foreach_set(
|
||||||
|
(128 / 255, 128 / 255, 1, 1, 128 / 255, 128 / 255, 1, 1)
|
||||||
|
)
|
||||||
|
material = bpy.data.materials.new("Unrelated Material Name")
|
||||||
|
mark_balloon_material(material, source_image, shared_normal)
|
||||||
|
assert reusable_balloon_material(
|
||||||
|
"unused",
|
||||||
|
"unused",
|
||||||
|
source_image,
|
||||||
|
normal_required=True,
|
||||||
|
) is material
|
||||||
|
|
||||||
|
update_normal = bpy.data.images.new("Normal Update", 2, 1)
|
||||||
|
update_normal.pixels.foreach_set(
|
||||||
|
(1, 0.25, 0.75, 1, 128 / 255, 128 / 255, 1, 1)
|
||||||
|
)
|
||||||
|
merge_balloon_normal_image(shared_normal, update_normal)
|
||||||
|
pixels = list(shared_normal.pixels)
|
||||||
|
assert abs(pixels[0] - 1.0) < 1e-5
|
||||||
|
assert abs(pixels[4] - 128 / 255) < 1e-5
|
||||||
|
print(f"Validated Blender {bpy.app.version_string}: {group.name}")
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
main()
|
||||||
Reference in New Issue
Block a user