diff --git a/README.md b/README.md index c55b6ab..e2df08d 100644 --- a/README.md +++ b/README.md @@ -10,7 +10,7 @@ AnyImage 是一个 Blender 扩展，可以把一张照片、一段视频或一 - 在 3D View、Node Editor 与绘画模式中按 `Ctrl+V`（macOS 为 `Cmd+V`）直接粘贴并 Pack 剪贴板图片 - 不运行 AI，把 Image Empty 转换为可实时调整 Thickness 与 Subdivide 的 Mesh Plane -- 从 Image Empty 套索提取不规则区域，通过 Poisson 或 Teddy / Chordal Axis 成成封闭 Balloon，并可用 BEN2 分析轮廓、MoGe-2 添加可调深度；源 Empty 保持不变 +- 用 Cutout 套索从 Image Empty 生成 Surface、Balloon、Depth Balloon 或 Depth Surface；四种结果共用单面基础薄片、局部方向与清理逻辑 - 用 Image Box / Lasso / Line / Polyline 直接清理 Image Empty；按 `F` 反选，也可用 BEN2 细化选区 - Image 编辑会直接更新未共享的图片；图片被多个 Image Empty 共用时，只为当前 Empty 分离编辑结果 - 用 Image Perspective 勾勒四个角点，把透视四边形矫正为矩形裁切图，并沿用源 Image Empty 的空间朝向 @@ -36,7 +36,7 @@ Environment Bootstrap 使用连续的 0–100%：uv 为 1–20%、Python 为 20 选中 Image Empty 后，`AnyImage` 会显示在右键菜单最上方。Mesh / Depth / Surface 转换为一组，Remove Background 与 Upscale 作为图片处理归为下一组，环境入口位于最后。Environment 或必需模型未就绪时，点击 `Convert to Depth Plane…`、`Convert to Surface…`、`Remove Background` 或 `Upscale` 会打开对应准备对话框：Environment 未安装时入口显示 `Set Up AI Environment…`，安装 Environment 与默认模型；Environment 已安装但默认模型缺失时入口显示 `Download Required Models…`，只补齐缺失模型。全部就绪后，菜单入口改为 `AI Environment Settings…`，并打开 AnyImage 扩展偏好设置。 -其他 AI 模型统一在 AnyImage 扩展偏好设置中按需下载。`Balloon Lasso Tool` 的本地几何功能始终可用；AI 未就绪时，只禁用 Tool Settings 中的 `Contour`、`Depth` 与 Depth Model，并在同处根据状态显示 `Set Up AI Environment…` 或 `Download Required Models…`。共享的 AI 状态检测直接检查 Environment 和 Storage Root 的 `models/` 文件，不会仅因打开菜单或设置而启动 Job Server。 +扩展偏好设置先显示 Environment 状态与 Diagnostics，再依次提供默认展开、可折叠的 Depth & Normal、Remove Background、Upscale 和 Cutout 功能区。Depth Model 按三档 MoGe-2、两档 DA3 排列；选择 DA3 时会提示该模型不能生成 normal。可选模型的状态与下载按钮位于对应模型选择项下方，BEN2 则归在 Remove Background。共享的 AI 状态检测直接检查 Environment 和 Storage Root 的 `models/` 文件，不会仅因打开菜单或设置而启动 Job Server。 ## 快速上手 @@ -46,7 +46,7 @@ Environment Bootstrap 使用连续的 0–100%：uv 为 1–20%、Python 为 20 ### 转换为普通图片平面 -选中 Image Empty 后，右键选择 `AnyImage > Convert to Mesh Plane`。该操作不运行 AI，会创建带图片材质和 `O Mesh Plane` Geometry Nodes 修改器的基础四边形。`Thickness` 与 `Subdivide` 都在修改器中实时成成；`Subdivide` 使用二分级别（`3` 为短边 `8` 段），`Thickness = 0` 时结果为真正的单面。操作会立即执行；创建后可在 Last Operation 中调整 `Shadeless` 和 `Keep Original`，调整时保持源图片的世界位置、旋转与缩放。完整说明见 [Mesh Plane](docs/mesh-plane.md)。 +选中 Image Empty 后，右键选择 `AnyImage > Convert to Mesh Plane`。该操作不运行 AI，会创建带图片材质和 `O Mesh Plane` Geometry Nodes 修改器的基础四边形。`Thickness` 与 `Subdivide` 都在修改器中实时生成；`Subdivide` 使用二分级别（`3` 为短边 `8` 段）。`Thickness = 0` 时结果为真正的单面。操作会立即执行；创建后可在 Last Operation 中调整 `Shadeless` 和 `Keep Original`，调整时保持源图片的世界位置、旋转与缩放。完整说明见 [Mesh Plane](docs/mesh-plane.md)。 ### 转换为立体面片 @@ -68,7 +68,7 @@ Lasso 绘制时与 Box、Line、Polyline 一样显示白色虚线轮廓和淡灰 ### 四点透视矫正 -`Image Perspective` 是 Image Mask 工具组之后的独立工具，位于该组与 Balloon Lasso 之间。四个角可以按任意顺序点击，完成后会在源图片坐标中自动求出不交错的凸四边形，并以原图 X/Y 方向确定输出旋转。绘制时显示白色虚线和半透明选区面，`Backspace` 删除最近一个角点，右键或 `Esc` 取消；切换其他工具也会立即退出。首次点击遵循同一选择规则。Perspective 不依赖当前 View 的相机内参，因此透视和正交 View 都可以使用。 +`Image Perspective` 是紧随 Image Mask 工具组之后的独立工具。四个角可以按任意顺序点击，完成后会在源图片坐标中自动求出不交错的凸四边形，并以原图 X/Y 方向确定输出旋转。绘制时显示白色虚线和半透明选区面，`Backspace` 删除最近一个角点，右键或 `Esc` 取消；切换其他工具也会立即退出。首次点击遵循同一选择规则。Perspective 不依赖当前 View 的相机内参，因此透视和正交 View 都可以使用。 第四个角点按下时会在 Viewport 中央显示带蓝色边框的矫正图片预览，中央显示当前比例，下方显示鼠标操作提示；鼠标同时移到 Viewport 中心，为左右两个方向保留相同的调节空间。水平移动连续调整宽高比，按住 `Ctrl` 吸附到 `1:1`、`4:3`、`3:4`、`16:9`、`9:16`，下一次左键确认并创建；`Backspace` 返回第三点，右键或 `Esc` 取消。初始比例取四边形两组对边的平均长度比。预览使用一次成成的 512 × 512 临时图片，移动鼠标时只改变 UI 显示比例。 @@ -80,27 +80,17 @@ Image Perspective 是 deform 工具，不提供 Refine 或反选。它与其他 ### Cutout 基础面与四种形状 -在 Object Mode 的 3D Viewport 左侧工具栏选择 `Cutout`，沿 Image Empty 绘制后可从四个方向选择 Cutout、Balloon、Depth Balloon 或 Depth Surface；双击会选择完整图片。`Point Spacing` 以原图像素为单位控制规则三角格点间距，裁切范围大小不会改变密度。未启用 `Refine Selection` 时，几何只跟随套索，忽略原图 Alpha；即使启用 `Normal` 并改走 Job，图片边框外的选区也会保留。启用 Refine 后才使用 BEN2 Alpha 收紧几何选区，框外部分会随最终 AI Alpha 被裁掉。自交轨迹按非零环绕规则填充：同方向的重叠区域保留，只有反方向抵消的内环形成孔洞。未 Refine 的普通选区必须与原图可见像素相交。基础面内部使用均匀六角格点，平滑轮廓作为约束边界；小选区会自动缩短边界采样间距，避免在较大的 `Point Spacing` 下被误判为空，同时不增加内部格点密度。Cutout 的 Thickness 以场景距离沿图片平面法线对称加厚，前盖、背面和轮廓侧壁均保持朝外。套索或 Refine 的最终 Alpha 会确定紧凑矩形裁剪范围和几何轮廓，也用于构造 MoGe-2 局部分析图：选区内保留当前图片的原始 RGB 与有效 Alpha，图片边框外和选区外 RGBA 全部置零。材质 Color 使用同一裁剪范围内未乘选区 Alpha 的当前图片像素，只保留图片原有 Alpha；因此不会因几何 mask 再次产生透明边。当前图片若为 Upscale 结果，颜色、深度和法线都基于高清图的所选区域。裁剪后的 Color 与 normal 可以是非正方形，网格位置和 UV 会映射回原 Image Empty 中对应区域。几何只保留最终 Alpha 大于等于 `0.5` 的区域，边缘低置信度面会被剔除。未启用 Refine 且关闭 `Normal` 时，Cutout 与 Balloon 会直接在 Blender 内完成遮罩、轮廓提取和建面，不启动 BlendJob。`Normal` 默认关闭；启用后才通过 MoGe-2 成成 normal 纹理，并改用 Job。Depth Surface 直接使用 MoGe-2 原成的 object-space normal；Cutout、Balloon 与 Depth Balloon 会依据成品网格的平滑 TBN 另行烘制 tangent-space normal。 +在 Object Mode 的 3D Viewport 左侧工具栏选择 `Cutout`，沿 Image Empty 绘制后可从四个方向选择 Surface、Balloon、Depth Balloon 或 Depth Surface；双击会选择完整图片。`Point Spacing` 以完整原图像素为单位控制规则三角格点间距，裁切范围大小不会改变密度。未启用 `Refine Selection` 时，几何只跟随套索；启用后才使用 BEN2 Alpha 收紧选区。`Normal` 默认关闭；启用后才通过 MoGe-2 生成 normal 纹理。未启用 Refine 且关闭 Normal 时，Surface 与 Balloon 完全在 Blender 内生成，不启动 BlendJob。 -完整说明见 [Cutout](docs/cutout.md)。 +四种结果的源网格统一为真正的单面薄片，并分别使用 `O Image Surface`、`O Image Balloon`、`O Image Depth Balloon`、`O Image Depth Surface` 四个独立 Geometry Nodes。局部纸面统一位于 YZ 平面，局部 `+X` 朝纸面内侧，局部 `-X` 是外侧和深度箭头方向；对象原点统一位于裁切图片中心，对象矩阵补偿后薄片仍落在源 Image Empty 的原位置。`Thickness = 0` 时保留单面；Surface 与 Depth Surface 的 Thickness 是场景距离，Balloon 与 Depth Balloon 的 Thickness 是 `o_balloon` 高度倍率，默认 `1` 保留算法原值。加厚按整片连通区域处理，内部三角边不生成侧壁。Surface 与 Depth Surface 会以边界三角边的中位长度为目标宽度，沿 Thickness 方向自动增加侧壁分段，使侧壁面片尽量接近正方形；Balloon 与 Depth Balloon 不增加这些分段线。 -`O Image Depth Surface` 将 `O Smooth` 的 Repeat Zone、双域边界采样和法线权重节点直接展开在主节点图中，并固定按 `Pin Sharp = 1`、`Pin Boundary = 1` 计算，不暴露这两个参数；`Iterations` 默认 `1`、最小可设为 `0`。`Depth Continuity` 默认 `0`，采用最宽松的断层阈值。Thickness 保持正面的深度表面不动，只向背面挤出。 +四个修改器都有默认折叠的 `Options` 面板，`Cleanup` 统一放在其中。节点按 Mesh Island 累计离散面面积，并以基础面三角形的中位面积自动估算 Point Spacing 尺度；默认 `1` 会删除面积小于约一个点间距平方的孤立小面，因此无需在 Cutout 创建后写入绝对场景单位阈值。Depth Balloon 的 `Base Plane`，以及 Depth Surface 的 `Depth Continuity`、`Smooth` 也收纳在该面板。 -### 套索成成 Balloon +四种 Cutout 形态都会在创建基础网格时计算 Balloon profile，并统一写入 `FLOAT/POINT` 属性 `o_balloon`；不再写入 `o_boundary`。AnyImage 扩展 Preferences 的 **Balloon Profile** 可选择 Poisson 或 Teddy，两个 Balloon 节点组只读取属性，不再自行估算轮廓。两个 Depth 形状都从内部打包的浮点深度图片采样，不把深度写成点属性；内部深度 Image datablock 统一使用 `_depth.exr` 后缀，`Depth Image` socket 在修改器中隐藏。 -选中单张图片的 Image Empty，右键打开 `AnyImage` 子菜单并选择 `Balloon Lasso Tool`。Blender 会激活 3D Viewport 左侧工具栏底部、位于独立 Image Perspective 之后的 Balloon Lasso；Image Mask 工具组与 Add Cube 之间保留一个间隔。Balloon 使用 Poly Build 的绿色图标、Annotate 同款铅笔光标，Tool Header 与状态栏共同提示当前工具状态，不再用黄色外框或强制修改 Shading、Overlay。Blender 的公开 Workspace Tool API 不支持按 Active Object 动态隐藏工具，因此图标在 Object Mode 中保持可见，但只有 Active Object 是 Image Empty 时才能启动 Stroke。 +Depth Balloon 的 Base Plane 位于当前裁切后网格的中心，而不是原图中心。默认朝向由选区内有效 MoGe 深度进行稳健平面回归；细长选区会保留长轴趋势，同时抑制缺少空间支撑的短轴倾角。Single Arrow 沿局部 `-X` 指向实际深度方向；沿箭头移动的场景距离与前表面的同向位移严格一致。Geometry Nodes 会先用 Base Plane 校正深度，再让 `Depth Scale` 乘整个校正结果；因此 `Depth Scale = 0` 时 Base Plane 和深度图都不再改变深度分支。`Double Sided` 默认关闭，使背面保持 Balloon 厚度，开启后则把受 Base Plane 调整后的最终前表面关于对象局部 YZ 平面完整镜像；Base Plane 只定义正面的朝向与基线，不会移动对称中心。Depth Surface 先按深度图片位移和断层剔除，再清理新产生的小岛、加厚并执行展开的 O Smooth 结构；`Iterations` 默认 `1`，`Depth Continuity` 默认 `0`。 -按住左键沿目标轮廓绘制，松开后成成一个带原图 UV 的封闭网格。每次拖动都是一个独立完成的顶层 `Extract Balloon` Operator，工具本身继续保持激活，下一次拖动会创建另一个 Balloon；因此 `Ctrl+Z` 只撤销最近一次成成，`Ctrl+Shift+Z` 重做。每次成成后源 Image Empty 都保持选中和 Active。未绘制时没有持续 Modal 拦截事件，视图导航和其他原成快捷键由 Blender 正常处理；绘制中可用右键或 `Esc` 取消当前轨迹。选择左侧工具栏中的其他工具即可离开 Balloon Lasso。 - -- `Mode`、`Thickness`、`Resolution (m)`、`Reuse Material`、`Contour`、`Depth` 和启用 `Depth` 后出现的 `Depth Model` 显示在 Tool Header / Tool Settings -- `S`：调用 Blender 原成 Radial Control 调节 Thickness；通过 `zoom_path` 把默认 `1.0` 映射为约 100 px 的交互圆，软上限为 `5.0`，不设硬上限 -- `D`：调用 Blender 原成 Radial Control 调节 Resolution；通过独立 `zoom_path` 把默认 `0.1 m` 映射为约 100 px 的交互圆，数值越小网格越密 - -`Resolution (m)` 会根据 Scene Unit Scale 换算为网格使用的世界距离，因此不同单位设置下仍代表相同的物理采样间距。Poisson 模式使用尺度无关的求解判断，支持 `0.02 m` 等较密采样。两个 AI 选项默认关闭；`Contour` 使用 BEN2 Alpha 严格大于 `0.5` 的最大连通前景重建轮廓，`Depth` 使用所选 MoGe-2 模型。MoGe 的物理距离会先减去稳健拟合的基面，再换算为不受绝对拍摄距离影响的相对起伏；近侧与远侧信号都保留，不削顶、不截平。 - -Depth Balloon 使用局部 X 作为深度轴，并创建 ` Base Plane` Single Arrow Empty。Empty 的旋转定义基面朝向，沿自身局部 Z 移动则调整 middle level。默认深度校准为 `修正深度 = (0.5 × 相对基面深度 + 原始 Balloon 最大半厚度) × Depth Strength`；原始最大半厚度来自成成后的固定网格，不受修改器 `Thickness` 影响。这在 `Thickness = 1` 时等价于旧参数的 `Depth Strength = 0.5` 和两倍半厚度基面偏移。节点再以 Smoother Step 中心程度插值修正厚度与修正深度；`Thickness` 只控制厚度分支，`Depth Strength` 只控制深度分支。`Double Sided Depth` 默认关闭，使背面维持稳定厚度。 - -MoGe normal 会转换成适配 Balloon UV 的 tangent-space `normal.png`，由统一的 `O Image Layer` 材质补回网格分辨率未能表达的细节；`Normal Scale` 默认 `1`，`Bump Scale` 默认 `0`，未连接 normal 时使用 `[0.5, 0.5, 1.0, 1.0]` 中性值。Balloon 材质通过 Custom Properties 直接保存源 Image datablock 引用，不依赖材质名或节点组名识别；勾选 `Reuse Material` 时优先复用同一 Image、同一材质类型的标记材质。Depth Balloon 共用一张原图尺寸的 normal，后续套索会把自己的有效区域合并进去，重叠区域采用最新结果。旧材质仍保留节点识别回退。当前只支持静态图片。 +Surface、Balloon 与 Depth Balloon 的 MoGe normal 会依据最终网格 TBN 烘成 tangent-space 纹理；Depth Surface 直接使用适配新局部轴向的 object-space normal。完整说明见 [Cutout](docs/cutout.md)。 ### 去除背景 @@ -147,7 +137,7 @@ Layer Color 同时驱动 Principled BSDF 的 Base Color、Subsurface Radius 和 生成的颜色、深度和法线 Image datablock 分别使用 `_color.png`、`_depth.exr`、`_normal.png` 和 `_upscale.png` 后缀，让 Blender 保持对真实图片格式的识别。 -AnyImage 支持视频（mp4、mkv、mov 等）和序列帧，会逐帧处理并成成逐帧对应的立体面片。色色图片与 `.depth` 按源 Empty Image 的类型载入：普通单图为 `IMAGE`，Movie/Sequence 即使只处理一帧也仍为 `SEQUENCE`。处理范围默认 +AnyImage 支持视频（mp4、mkv、mov 等）和序列帧，会逐帧处理并成成逐帧对应的立体面片。`_color.png` 与 `_depth.exr` Image datablock 按源 Empty Image 的类型载入：普通单图为 `IMAGE`，Movie/Sequence 即使只处理一帧也仍为 `SEQUENCE`。处理范围默认 跟随 Empty 上设置的 `Image Frames` 与 `Offset`，不需要额外配置。DA3 的转换任务复用同一个 ONNX Session 逐帧推理，显存峰值不随视频帧数平方增长。Upscale 也会逐帧成成受输出上限约束的 4× 色色序列并写回 Empty。Debug 只保存模型 API 的直接返回：DA3 多帧输入 保留联合推理，MoGe-2、BEN2 与三档高清模型按各自的单图 API 逐帧保存。高清 Debug 输出 4× RGB `prediction*.png`，不做强度混合或图片替换；三个模型都由适配层自动分块，因此接受任意图片宽高。 diff --git a/docs/cutout.md b/docs/cutout.md index b05ef60..51f217a 100644 --- a/docs/cutout.md +++ b/docs/cutout.md @@ -1,12 +1,12 @@ # Cutout -Cutout 用套索从 Image Empty 中选择范围，再从四个方向选择生成结果：左侧 **Cutout**、右侧 **Balloon**、下方 **Depth Balloon**、上方 **Depth Surface**。双击图片会把完整画面作为选区。 +Cutout 用套索从 Image Empty 中选择范围，再从四个方向选择生成结果：左侧 **Surface**、右侧 **Balloon**、下方 **Depth Balloon**、上方 **Depth Surface**。双击图片会把完整画面作为选区。 ## 绘制选区 在 Object Mode 的 3D Viewport 左侧工具栏选择 **Cutout**，按住左键绘制，松开后选择形状。绘制时按 `F` 切换反选，右键或 `Esc` 取消当前轨迹。 -自交轨迹使用非零环绕填充规则。沿相同方向围住的区域即使重叠多次也会保留；只有反方向环绕使计数抵消时才形成孔洞。因此交错部分不会再被当成洞扣除，同时仍能表达中间的反向内环孔洞。普通选区可以越过 Image Empty 边框，框外轨迹仍会生成网格；选区必须与原图中的可见像素相交才有效。反选仍限制在原图范围，避免把框外无限区域截成任意矩形。未启用 **Refine Selection** 时，选区边缘会做轻量抗锯齿；启用后，BEN2 先在同一个几何选区内计算前景 Alpha，再应用反选。 +自交轨迹使用非零环绕填充规则。沿相同方向围住的区域即使重叠多次也会保留；只有反方向环绕使计数抵消时才形成孔洞。因此交错部分不会再被当成洞扣除，同时仍能表达中间的反向内环孔洞。普通选区可以越过 Image Empty 边框，框外轨迹仍会生成网格；选区必须与原图中的可见像素相交才有效。反选仍限制在原图范围，避免把框外无限区域截成任意矩形。Cutout 的 **Refine Selection** 独立保存，不与 Image Box 或 Image Lasso 联动。未启用时，选区边缘会做轻量抗锯齿；启用后，BEN2 先在同一个几何选区内计算前景 Alpha，再应用反选。 ## 基础面密度 @@ -14,18 +14,24 @@ Cutout 用套索从 Image Empty 中选择范围，再从四个方向选择生成 基础面先从最终 Alpha 提取外边界与孔洞，再以接近等边三角形的六角格点填充内部。轮廓按目标间距重采样和平滑，边界点仍作为约束参与 Delaunay，因此内部点保持均匀，外缘不会退化成规则栅格的阶梯锯齿。 -四种结果都启用平滑着色。Cutout 与 Depth Surface 的修改器也在输出阶段显式设置 Smooth Shading；Balloon 与 Depth Balloon 在创建源网格时设置。 +四种结果都启用平滑着色。Surface 与 Depth Surface 的修改器也在输出阶段显式设置 Smooth Shading；Balloon 与 Depth Balloon 在创建源网格时设置。 -Cutout 的基础三角面统一按图片正面逆时针排列，Thickness 沿图片平面法线向前后各偏移一半；背面与轮廓侧壁使用相反绕序，所有封闭面的法线都朝向几何外部。Depth Surface 则保持正面深度表面不动，只沿法线反方向挤出背面。Thickness 是 Blender 场景中的实际距离，因此节点接口使用 Distance 单位显示。 +Surface 的基础三角面统一按图片正面逆时针排列，Thickness 沿图片平面法线向前后各偏移一半；背面与轮廓侧壁使用相反绕序，所有封闭面的法线都朝向几何外部。Depth Surface 则保持正面深度表面不动，只沿法线反方向挤出背面。Surface 与 Depth Surface 的 Thickness 是 Blender 场景中的实际距离；其侧壁分段数按 `ceil(Thickness / 边界三角边中位长度)` 自动计算，并限制在最多 128 段，使大多数侧壁单元尽量接近正方形。分段只发生在连接前后表面的侧壁环带。Balloon 与 Depth Balloon 的 Thickness 是 `o_balloon` 高度倍率，`1` 保持算法原值，`0` 保持单面薄片；这两种形态不加入自适应分段，因此分段线不会改变 Balloon profile。 + +四个形状节点组统一采用从左到右的几何主线；参数使用靠近消费者的局部 Group Input，未连接 socket 全部隐藏。Cleanup、profile、深度采样、Base Plane、分段计数和平滑等字段分支放在主线上下，并在就近节点汇入；Repeat Zone 的输入、内部处理和输出保持清楚的包裹顺序。节点不使用自定义 label 或彩色 Frame，也不允许反向连线和节点重叠。`Surface` 对应节点组名称为 `O Image Surface`；载入旧资产时会把 `O Image Cutout` 原位迁移为新名称。 Depth Surface 的 `O Image Depth Surface` 按“深度 → 断层剔除 → 向后厚度 → 零厚度清理 → 展开的 O Smooth 结构 → 平滑着色”从左到右排列，使内置平滑与在完整输出后连接 `O Smooth` 获得相同的输入几何。平滑段把 O Essentials 参考资产的 Repeat Zone、两条不同求值域的边界判断、边界采样与法线一致性权重节点直接放入主节点图，每轮重新计算后再移动点，不再嵌套 `O Smooth` 节点组；内部固定按 `Pin Sharp = 1`、`Pin Boundary = 1` 计算，不暴露这两个参数，`Iterations` 默认 `1`、最小可设为 `0`。Depth Continuity 使用与 `O MoGe2 Surface` 相同的百分比阈值曲线，默认 `0`，对应最宽松的断层阈值；提高数值会逐步剔除更多过远的深度跳变面。 +两个 Depth 节点组的内部浮点深度 Image datablock 统一使用源图片基础名称加 `_depth.exr` 后缀，不再使用旧 `.depth` 名称。四个节点组都提供默认折叠的 `Options` 面板，并把 `Cleanup` 收纳其中。`O Image Depth Balloon` 的 `Base Plane` 也位于 Options；`O Image Depth Surface` 的 `Depth Continuity` 与 `Smooth` 同样位于 Options。Depth Balloon 先计算相对 Base Plane 的校正深度，再用 `Depth Scale` 乘整个校正结果；设为 `0` 时，Base Plane 的位置和旋转以及深度图都不再改变深度分支。开启 `Double Sided` 后，Base Plane 只影响正面的朝向与基线，背面再由最终正面关于对象局部 YZ 平面（`X = 0`）镜像得到；Base Plane 不会改变前后对称中心。 + +Base Plane 的默认朝向使用选区内有效 MoGe 深度进行四轮稳健平面回归。回归在图像坐标的主轴和短轴上分别评估空间支撑；当细长结构的短轴方差过小时，会限制该方向的反演增益，避免少量横向深度噪声被放大成极端倾角，同时保留长轴上有充分样本支持的深度趋势。Blender 将归一化平面换算到裁切网格的本地坐标时执行同样的稳定处理。 + ## 计算说明 -未启用 **Refine Selection** 时，几何范围只使用抗锯齿套索 mask，原图 Alpha 不参与轮廓；原图 Alpha 只用于确认选区内至少存在可见像素，并继续控制材质透明度。无论是否启用 **Normal**，Job 都会为真实出框的套索建立可带负坐标的扩展选区画布，因此图片边框外的几何不会被原图尺寸裁掉。套索或 Refine 的最终 Alpha 会确定紧凑矩形裁剪范围，并以 `0.5` 为几何边界：小于 `0.5` 的区域不建面。Color 贴图使用当前 Image Empty 在同一矩形内的原始像素，不乘套索或 Refine Alpha，只保留图片本身已有的 Alpha，图片外像素保持透明。深度和法线分析使用裁剪后的局部分析图：最终选区内保留当前图片的原始 RGB 与有效 Alpha，图片外和选区外 RGBA 全部置零。若当前图片来自 Upscale，材质、深度和法线都直接使用高清图所选区域，不再回溯放大前原图。Color 与 normal 允许保持裁剪后的非正方形尺寸；Blender 按裁剪 bounds 调整局部平面范围和 UV，使结果仍落在源 Image Empty 的原位置。关闭 **Normal** 时，Cutout 与 Balloon 直接在 Blender 内处理，不准备临时输入、不启动 BlendJob；启用 **Normal** 后才提交 Job。MoGe 返回的相机轴法线先按源 Image Empty 的对象坐标编码。Depth Surface 原样保留这张 object-space 贴图；Cutout、Balloon 与 Depth Balloon 则在最终几何节点结果可用后，使用成品网格的 UV、平滑角法线和切线基底重新烘成 tangent-space 贴图。启用 Refine 时，Job 先计算 BEN2 Alpha 并返回裁剪后的几何 mask。Depth 类型始终通过 Job 取得深度，但也只在启用 **Normal** 时写入 normal。四种形状的网格都在 Blender 主线程构建，不再通过 `contour.json` 传递边界。 +未启用 **Refine Selection** 时，几何范围只使用抗锯齿套索 mask，原图 Alpha 不参与轮廓；原图 Alpha 只用于确认选区内至少存在可见像素，并继续控制材质透明度。无论是否启用 **Normal**，Job 都会为真实出框的套索建立可带负坐标的扩展选区画布，因此图片边框外的几何不会被原图尺寸裁掉。套索或 Refine 的最终 Alpha 会确定紧凑矩形裁剪范围，并以 `0.5` 为几何边界：小于 `0.5` 的区域不建面。Color 贴图使用当前 Image Empty 在同一矩形内的原始像素，不乘套索或 Refine Alpha，只保留图片本身已有的 Alpha，图片外像素保持透明。深度和法线分析使用裁剪后的局部分析图：最终选区内保留当前图片的原始 RGB 与有效 Alpha，图片外和选区外 RGBA 全部置零。若当前图片来自 Upscale，材质、深度和法线都直接使用高清图所选区域，不再回溯放大前原图。Color 与 normal 允许保持裁剪后的非正方形尺寸；Blender 按裁剪 bounds 调整局部平面范围和 UV，使结果仍落在源 Image Empty 的原位置。关闭 **Normal** 时，Surface 与 Balloon 直接在 Blender 内处理，不准备临时输入、不启动 BlendJob；启用 **Normal** 后才提交 Job。MoGe 返回的相机轴法线先按源 Image Empty 的对象坐标编码。Depth Surface 原样保留这张 object-space 贴图；Surface、Balloon 与 Depth Balloon 则在最终几何节点结果可用后，使用成品网格的 UV、平滑角法线和切线基底重新烘成 tangent-space 贴图。启用 Refine 时，Job 先计算 BEN2 Alpha 并返回裁剪后的几何 mask。Depth 类型始终通过 Job 取得深度，但也只在启用 **Normal** 时写入 normal。四种形状的网格都在 Blender 主线程构建，不再通过 `contour.json` 传递边界。 最终 Alpha 先由 NumPy 批量找出四向边界，只沿实际周长追踪轮廓，并按包含关系把孔洞归入外轮廓。未 Refine 时直接复用套索 mask；只有真实出框时才建立扩展几何画布。启用 Refine 后完全使用原图画布内的最终 AI Alpha，所有框外套索区域都会被裁掉。格点准备使用批量数组生成；点内判断按 Y 范围分桶，只比较当前扫描带可能相交的轮廓边；边界安全距离也只检查每条边附近的少量格点。短轮廓会自动使用不大于周长八分之一的边界采样间距，避免有效小区域因当前 `Point Spacing` 较大而消失；内部格点仍严格使用用户设置的 `Point Spacing`。UV 与封边边界使用 Blender `foreach_*` 和 NumPy 批量写入，不逐元素遍历。Pillow 只会加速占比较小的 Alpha 模糊和缩放，因此 Blender 本地路径保持纯 NumPy，不新增 wheel。最终约束 Delaunay 和网格创建仍随实际点数增长，极密网格应按成品需要使用。 Cutout 当前只支持静态图片。**Refine Selection**、Depth 与 normal 分析需要已配置的 AI Environment 和对应模型。 -Cutout、Balloon、Depth Balloon、Depth Surface 以及普通 Surface 的图片材质共用 `O Image Layer`。节点组始终提供 Normal 输入和 Object Space 开关；没有 normal 纹理时使用 `[0.5, 0.5, 1.0, 1.0]` 中性默认值。这个开关只选择 Blender Normal Map 节点的解码空间，不承担 Object-to-Tangent 转换：三种 tangent-space 结果在接入材质前已经完成像素烘制，Depth Surface 才打开 Object Space 并直接使用 MoGe 原图。 +Surface、Balloon、Depth Balloon、Depth Surface 以及普通 Surface 的图片材质共用 `O Image Layer`。节点组始终提供 Normal 输入和 Object Space 开关；没有 normal 纹理时使用 `[0.5, 0.5, 1.0, 1.0]` 中性默认值。这个开关只选择 Blender Normal Map 节点的解码空间，不承担 Object-to-Tangent 转换：三种 tangent-space 结果在接入材质前已经完成像素烘制，Depth Surface 才打开 Object Space 并直接使用 MoGe 原图。 diff --git a/docs/development.md b/docs/development.md index 1429f8f..3fcfb3b 100644 --- a/docs/development.md +++ b/docs/development.md @@ -43,7 +43,7 @@ Server 的模型模块返回 Prediction，几何模块处理数组与网格， Blender 侧需要落盘的 Packed Image 与浮点深度 EXR 都通过 `Image.save(save_copy=True)` 导出，不调用 `save_render`，避免把 Scene Render 的 View Transform、Look 或 Exposure 耦合到模型输入和 Non-Color 数据；颜色结果载入后继承源 Image 的 Color Space。Server 中的 `PIL.Image.save()` 只负责普通文件编码，不属于 Blender Render 色彩管理。 -Image Box、Lasso、Line、Polyline 与 Perspective 位于 `operators/image_edit/` 功能包。模块职责、交互状态、像素管线、Refine 和 Empty 定位见 [Image 工具技术实现](image-tools-technical.md)。 +Image Box、Lasso、Line、Polyline 与 Perspective 位于 `operators/image_edit/` 功能包。Cutout、Image Box 与 Image Lasso 分别读取 `Scene.anyimage_settings.cutout_refine_selection`、`image_box_refine_selection` 与 `image_lasso_refine_selection`，三个工具的 Refine 状态互不联动；Line 与 Polyline 在公共设置入口中强制关闭 Refine。模块职责、交互状态、像素管线、Refine 和 Empty 定位见 [Image 工具技术实现](image-tools-technical.md)。 ## 项目结构 @@ -117,12 +117,12 @@ BLENDER_BIN="/c/Program Files/Blender Foundation/Blender 4.5/blender.exe" "$BLENDER_BIN" --background --python tools/validate_depth_plane_asset.py "$BLENDER_BIN" --background --python tools/validate_mesh_plane_asset.py "$BLENDER_BIN" --background --python tools/validate_geometry_nodes_asset.py -"$BLENDER_BIN" --background --python tools/validate_balloon_asset.py +"$BLENDER_BIN" --background --python tools/validate_cutout_asset.py "$BLENDER_BIN" --background --python tools/validate_moge_depth_plane.py "$BLENDER_BIN" --background --python tools/validate_moge_surface.py ``` -修改节点组、材质组或资产接口后，运行对应验证脚本和完整测试。Python、文档或打包逻辑变化无需重建 `.blend` 资产。 +修改节点组、材质组或资产接口后，运行对应验证脚本和完整测试。 Cutout 四个形状节点组还会验证所有连线从左向右、节点之间保留可读间距、局部 Group Input 未使用输出隐藏，以及 Repeat Zone 数量和几何求值不变。Python、文档或打包逻辑变化无需重建 `.blend` 资产。 ## 上游与许可证 diff --git a/docs/features.md b/docs/features.md index 698e9ae..1f78cb8 100644 --- a/docs/features.md +++ b/docs/features.md @@ -28,7 +28,7 @@ Depth Plane 与 Surface 可使用 DA3 或 MoGe-2。DA3 提供相对深度和相 `Convert to Mesh Plane` 位于 `Convert to Depth Plane` 上方，不需要 AI Environment。它生成基础四边形，并由 `O Mesh Plane` 修改器实时处理 `Thickness` 与 `Subdivide`；`Thickness` 为 `0` 时保持单面。操作会立即执行；创建后可在 Last Operation 中调整 `Shadeless` 与 `Keep Original`。 -Cutout 复用 Image Lasso 的反选与 Refine 流程，但自交选区使用非零环绕填充：同方向重叠保留，反方向内环形成孔洞。基础面以均匀六角格点填充平滑约束轮廓，`Point Spacing` 以完整源图像素为单位控制内部密度，不随局部裁切范围重新归一化；短轮廓会自适应边界采样以免有效小区域消失。未启用 Refine 的 Cutout 与 Balloon 只按套索建面，不把原图 Alpha 计入几何；同时关闭 `Normal` 时在 Blender 内完成遮罩、轮廓和建面，不启动 BlendJob。套索或 Refine Alpha 以 `0.5` 为几何剔除阈值，并确定紧凑矩形裁剪范围。材质 Color 使用当前 Empty Image 在该范围内的原始像素，不乘选择 Alpha，只保留图片原有 Alpha；MoGe 深度与法线则使用同一裁剪范围的局部分析图，选区内保留当前图片 RGB，范围外 RGBA 置零。Upscale 图片直接使用高清图的所选区域进行材质与 AI 分析。裁剪贴图可以是非正方形，局部平面范围和 UV 会映射回源图对应区域。`Normal` 默认关闭，启用后才用 MoGe-2 写入 normal；Depth Surface 保留 MoGe 原生 object-space 像素，Cutout、Balloon 与 Depth Balloon 根据成品网格 TBN 烘成 tangent-space 像素。Depth 类型始终使用 Job 生成深度，但同样由该选项决定是否写入 normal。Refine 启用后由 BEN2 返回裁剪后的最终 Alpha，Blender 再从该 Alpha 建面。详细操作与计算边界见 [Cutout](cutout.md)。 +Cutout 复用 Image Lasso 的反选与 Refine 流程，但自交选区使用非零环绕填充：同方向重叠保留，反方向内环形成孔洞。基础面以均匀六角格点填充平滑约束轮廓，`Point Spacing` 以完整源图像素为单位控制内部密度，不随局部裁切范围重新归一化；短轮廓会自适应边界采样以免有效小区域消失。未启用 Refine 的 Surface 与 Balloon 只按套索建面，不把原图 Alpha 计入几何；同时关闭 `Normal` 时在 Blender 内完成遮罩、轮廓和建面，不启动 BlendJob。套索或 Refine Alpha 以 `0.5` 为几何剔除阈值，并确定紧凑矩形裁剪范围。材质 Color 使用当前 Empty Image 在该范围内的原始像素，不乘选择 Alpha，只保留图片原有 Alpha；MoGe 深度与法线则使用同一裁剪范围的局部分析图，选区内保留当前图片 RGB，范围外 RGBA 置零。Upscale 图片直接使用高清图的所选区域进行材质与 AI 分析。裁剪贴图可以是非正方形，局部平面范围和 UV 会映射回源图对应区域。`Normal` 默认关闭，启用后才用 MoGe-2 写入 normal；Depth Surface 保留 MoGe 原生 object-space 像素，Surface、Balloon 与 Depth Balloon 根据成品网格 TBN 烘成 tangent-space 像素。Depth 类型始终使用 Job 生成深度，但同样由该选项决定是否写入 normal。Refine 启用后由 BEN2 返回裁剪后的最终 Alpha，Blender 再从该 Alpha 建面。详细操作与计算边界见 [Cutout](cutout.md)。 `O Image Depth Surface` 的平滑阶段直接展开 O Essentials `O Smooth` 的 Repeat Zone 与双域边界判断结构，不嵌套组节点；内部固定按 `Pin Sharp = 1`、`Pin Boundary = 1` 计算，不暴露这两个参数，`Iterations` 默认 `1`、最小可设为 `0`。`Depth Continuity` 默认 `0`；Thickness 固定正面，只向背面挤出。 @@ -36,15 +36,15 @@ Cutout 复用 Image Lasso 的反选与 Refine 流程，但自交选区使用非 转换任务完成后，AnyImage 在一个 Undo 步骤中创建网格、图片、材质和修改器。源 Image Empty 是否保留由转换选项决定。 -Image Box、Lasso、Line 与 Polyline 组成 Mask 工具组，并共用 `EditImageMask`。四者用 `F` 切换反选；`Refine Selection` 只用于 Box / Lasso。Perspective 是位于 Mask 组与 Balloon Lasso 之间的独立工具，将任意顺序输入的四个角矫正为指定比例的矩形。结果直接更新当前 Image Empty，并 Pack 到 `.blend`。 +Image Box、Lasso、Line 与 Polyline 组成 Mask 工具组，并共用 `EditImageMask`。四者用 `F` 切换反选；`Refine Selection` 只用于 Box / Lasso。 + +Perspective 是紧随 Mask 工具组之后的独立工具，将任意顺序输入的四个角矫正为指定比例的矩形。结果直接更新当前 Image Empty，并 Pack 到 `.blend`。 操作步骤见 [Image 编辑工具](image-tools.md)；处理管线与模块职责见 [Image 工具技术实现](image-tools-technical.md)。 -Balloon 默认在 Blender 内部生成，无需 Environment 或模型。Tool Settings 提供模式、厚度、采样距离、材质复用、`Contour`、`Depth` 与 MoGe-2 `Depth Model`；AI 未就绪时只禁用这三个控件，并在 Tool Settings 中显示安装按钮。Depth 会把物理距离相对化并稳健拟合初始基面，保留未截断的正负起伏。Base Plane 子 Empty 的旋转定义基面朝向，沿自身局部 Z 移动定义 middle level。 +Cutout 工具从四个方向生成 Surface、Balloon、Depth Balloon 与 Depth Surface。四种结果使用相同的单面基础薄片、局部 YZ 纸面方向和裁切中心原点，并统一写入 `o_balloon` 点属性；Preferences 可在 Poisson 与 Teddy 两种 Balloon Profile 算法之间选择。 -`O Balloon Shape` 计算 `修正厚度 = abs(原 X) × Thickness`，并将修正深度校准为 `(0.5 × 相对基面深度 + 原始 Balloon 最大半厚度) × Depth Strength`。固定的原始半厚度不乘 `Thickness`，因此 Thickness 与 Depth Strength 分别只控制各自分支。节点以归一化原 X 距离经 Smoother Step 得到中心程度，再插值厚度与深度。`Double Sided Depth` 默认关闭，背面保持 Balloon 厚度。MoGe 相机轴 normal 会在 Blender 评估最终 Balloon 后按平滑 TBN 烘为原图尺寸的 tangent-space 纹理，并通过统一 `O Image Layer` 的 `Normal Scale` 补偿网格无法表达的细节；`Normal Scale` 默认 `1`，`Bump Scale` 默认 `0`，未连接 normal 时使用中性默认值。 - -新建 Balloon 材质会用 Custom Properties 保存源 Image datablock 和可选 normal Image 的直接引用。`Reuse Material` 优先按这些标记匹配，不依赖材质或节点组名称；普通与 Depth 材质分别复用。Depth Balloon 的共享 normal 会逐次合并新套索的有效区域，避免复用材质时丢失其他套索的法线。 +四个 Geometry Nodes 修改器都提供默认折叠的 `Options` 面板，并把 `Cleanup` 收纳其中。Depth Balloon 额外把 `Base Plane` 放入 Options；Depth Surface 把 `Depth Continuity` 与 `Smooth` 放入 Options。Depth Balloon 使用 `Depth Scale` 控制相对基面的深度；`Double Sided` 开启后，先让 Base Plane 决定正面的朝向与基线，再把最终正面关于对象局部 YZ 平面完整镜像；Base Plane 不会改变对称中心。 ## 动画 diff --git a/docs/image-tools-technical.md b/docs/image-tools-technical.md index 6a84d83..b593a98 100644 --- a/docs/image-tools-technical.md +++ b/docs/image-tools-technical.md @@ -48,7 +48,7 @@ flowchart TD Crop --> Write["替换 Image 并更新 Empty 显示"] ``` -Box / Lasso 的顺序固定为 **Refine → Invert**。BEN2 只提供内容 Alpha；最终 RGB 始终来自源图。Blender 侧只修改 Alpha，不清理透明区 RGB，以避免 Viewport 纹理过滤产生黑边。Line / Polyline 强制关闭 Refine，只执行几何 mask 与 Invert。 +Cutout、Box 与 Lasso 分别使用 `Scene.anyimage_settings` 上的独立 Refine 属性，工具之间不传递状态。Box / Lasso 的顺序固定为 **Refine → Invert**。BEN2 只提供内容 Alpha；最终 RGB 始终来自源图。Blender 侧只修改 Alpha，不清理透明区 RGB，以避免 Viewport 纹理过滤产生黑边。Line / Polyline 调用公共设置入口时强制关闭 Refine，只执行几何 mask 与 Invert。 未形成有效区域的单击直接返回 `CANCELLED`，不报告消息；最终 Alpha 全空时，Operator 报告 `WARNING` 并保持源图不变。其他数据、状态或运行时异常仍报告 `ERROR`。 diff --git a/docs/image-tools.md b/docs/image-tools.md index f6d6871..fb61e25 100644 --- a/docs/image-tools.md +++ b/docs/image-tools.md @@ -1,6 +1,6 @@ # Image 编辑工具 -Image Box、Lasso、Line、Polyline 与 Perspective 直接编辑静态 Image Empty，并把结果 Pack 到当前 `.blend`。前四个工具组成 Mask 工具组；Perspective 是紧随其后的独立工具，Balloon Lasso 位于 Perspective 之后。 +Image Box、Lasso、Line、Polyline 与 Perspective 直接编辑静态 Image Empty，并把结果 Pack 到当前 `.blend`。前四个工具组成 Mask 工具组；Perspective 是紧随其后的独立工具。 ## 选择工具 @@ -17,7 +17,7 @@ Image Box、Lasso、Line、Polyline 与 Perspective 直接编辑静态 Image Emp
**截图占位：Image 工具组与 Tool Settings** -显示包含四个工具的 Image Mask 组、独立 Perspective、Balloon Lasso，以及 Box / Lasso 的 Refine Selection。 +显示包含四个工具的 Image Mask 组、独立 Perspective，以及 Box / Lasso 的 Refine Selection。 目标文件：`images/image-tools/tool-group.png`
@@ -48,7 +48,7 @@ Image Box、Lasso、Line、Polyline 与 Perspective 直接编辑静态 Image Emp - 关闭：保留选区内； - 开启：保留选区外。 -Box / Lasso 启用 **Refine Selection** 后，BEN2 会先把几何选区收紧到其中的前景，再应用反选。例如框住一个苹果并开启反选，最终会清理苹果本身、保留其余内容。Line / Polyline 不提供 Refine，以保持直线边界和明确的裁切方向。 +Cutout、Box 与 Lasso 分别保存各自的 **Refine Selection** 设置；在一个工具中切换不会改变另外两个工具。启用后，BEN2 会先把几何选区收紧到其中的前景，再应用反选。例如框住一个苹果并开启反选，最终会清理苹果本身、保留其余内容。Line / Polyline 不提供 Refine，以保持直线边界和明确的裁切方向。 结果会直接更新当前 Empty 的图片，并把画布紧裁切到最终可见 Alpha。只有一个 Image Empty 使用该图片时，工具更新原 Image；两个或更多 Image Empty 共用它时，工具为当前 Empty 创建独立结果，其他 Empty 保持不变。对象 Transform 不变，显示尺寸与 Image Offset 会更新，使剩余像素保持在原图片平面上的位置。透明区只修改 Alpha，保留源 RGB 供纹理过滤使用。未形成有效区域的单击会静默结束；有效选区没有可见像素时只显示警告，不修改图片。 diff --git a/docs/installation.md b/docs/installation.md index 22f2e5a..555f51e 100644 --- a/docs/installation.md +++ b/docs/installation.md @@ -17,13 +17,18 @@ AnyImage 的安装分为两步：先安装 Blender Extension，再安装独立 ## 检查扩展偏好设置 -在 AnyImage 的扩展偏好设置中可以看到两个选项： +AnyImage 的扩展偏好设置按功能分组。Environment 区包含： - **Storage Root**：保存运行环境、模型和任务文件的位置。默认是用户目录下的 `.anyimage`，一般无需修改。 - **Device**：正式转换任务使用的计算设备。AnyImage 默认选择本机可用的最快设备；如果 GPU Provider 不可用，会使用 CPU。 + +Environment 检测后显示 Diagnostics。Depth & Normal、Remove Background、Upscale 与 Cutout 是默认展开的可折叠功能区，各自包含对应设置和模型状态。其中： + - **Default Upscale Model**：Upscale 使用的模型，默认是 General WDN x4v3。 - **Maximum Upscale Size**：Upscale 输出长边上限，默认 4096 px，最高 8192 px。 +Depth Model 依次列出三档 MoGe-2 与两档 DA3；选择 DA3 时会提示该模型不能生成 normal。可选模型的 **Ready** 状态或 **Download** 按钮直接位于模型选择项下方；BEN2 等固定模型则显示在所属功能区。Cutout 区只保留 Balloon Profile。 + 如果准备把 Storage Root 放到其他磁盘，建议在首次安装 Environment 之前修改。更换目录后，旧目录中的环境和模型不会自动移动。 ## 配置 AI Environment diff --git a/docs/models-and-storage.md b/docs/models-and-storage.md index 4ce77a5..8ca5122 100644 --- a/docs/models-and-storage.md +++ b/docs/models-and-storage.md @@ -2,12 +2,15 @@ AnyImage 把运行环境、模型和任务结果放在 Blender 之外，多个 Blender 版本可以共享同一套数据。 -## Models 设置 +## 功能与模型设置 -打开 **Edit > Preferences > Extensions > AnyImage**，可以查看和下载模型。右键菜单中的 **AI Environment Settings…** 会直接打开这里。 +打开 **Edit > Preferences > Extensions > AnyImage**，可以查看和下载模型。右键菜单中的 **AI Environment Settings…** 会直接打开这里。Environment 检测后紧接 Diagnostics；其余设置分为默认展开、可折叠的 Depth & Normal、Remove Background、Upscale 和 Cutout 功能区，模型不会再集中显示在单独列表中。 - **Ready** 表示模型文件已经就绪。 - **Download** 用于按需下载模型。 +- Depth & Normal 与 Upscale 的下载按钮位于当前模型选择项下方；切换选择后，可直接下载对应模型。 +- Depth Model 先列出三档 MoGe-2，再列出两档 DA3；DA3 不输出 normal，选择后会在设置下方提示。 +- Remove Background 使用的 BEN2 固定模型显示在对应功能区；Cutout 区只包含 Balloon Profile。 - 默认安装会准备 MoGe-2 ViT-S Normal、BEN2 Base 和 Real-ESRGAN General WDN x4v3。 - DA3、更大的 MoGe-2，以及可选的 SAFMN Real x4、Real-ESRGAN x4plus 需要时再下载即可。 - 可公开取得的 ONNX 制品使用经过大小与 SHA-256 校验的原址，并以 OmooLab R2 为镜像；不会把源码仓库名称推测成 Hugging Face 下载地址。 diff --git a/docs/outputs.md b/docs/outputs.md index 2ef9b8d..9d42cf6 100644 --- a/docs/outputs.md +++ b/docs/outputs.md @@ -12,7 +12,6 @@ | `upscale-image` | `upscale.0001.png` 起 | 所选高清模型在输出上限内生成的 4× 高清层，并保留源图 Alpha | | `refine-image-mask` | `selection-input.png`、`image.png`、`bounds.json` | BEN2 细化后的局部内容 mask 与原图坐标 bounds | | `generate-mesh` | `image.0001.png`、`selection.png`、`bounds.json`，按形状输出 `normal.png`、`depth.npz` 或 `field.npz` | 当前图片所选区域的矩形 Color、几何 mask、原图坐标 bounds，以及同尺寸的局部深度或法线结果 | -| `analyze-balloon` | `lasso-input.png`，按选项输出 `foreground.png`、`contour.json`、`depth.npz`、`normal.png` | 套索约束输入、BEN2 轮廓与 MoGe-2 深度、法线 | | `debug-da3` | `prediction.npz` | DA3 API 返回 | | `debug-moge2` | `prediction*.npz` | MoGe-2 API 返回 | | `debug-ben2` | `foreground*.png` | BEN2 API 返回 | diff --git a/docs/server.md b/docs/server.md index ac5314e..42a3d4b 100644 --- a/docs/server.md +++ b/docs/server.md @@ -56,13 +56,13 @@ BlendJob 提供 Runtime、HTTP transport、队列、状态和取消协议。AnyI Environment 使用 Python 3.12、ONNX Runtime、NumPy、Pillow、PyAV、trimesh、FastAPI 和 Uvicorn。Windows 使用 DirectML 或 CPU；其他平台根据支持情况使用 CoreML、OpenVINO 或 CPU。 -Balloon 的 `analyze-balloon` Job 会裁切套索输入并清零选区外 RGBA。`Contour` 用 BEN2 回传最大连通轮廓；`Depth` 用 MoGe-2 对有效相机距离稳健拟合二维基面，并把物理距离换算为不受绝对拍摄距离影响、未截断的相对深度信号。Job 同时把 MoGe 相机轴 normal 铺回原图尺寸，并以中性法线填充无效区域。Blender 主线程负责采样深度属性、创建 Base Plane Empty、组装 `O Balloon Shape` 修改器，再根据最终评估网格的 TBN 把相机轴 normal 烘成 tangent-space Image 并接入统一的 `O Image Layer` 材质。 +Cutout 的 `generate-mesh` Job 会把套索或 BEN2 Alpha 收紧为裁切范围，同时分别写出未乘选区 Alpha 的 Color、几何 selection 与局部分析图。Cutout 和 Balloon 只在启用 Normal 时运行 MoGe-2；Depth Balloon 返回浮点深度分析，Depth Surface 返回深度场。Blender 主线程统一创建单面基础网格、计算 `o_balloon`、绑定内部深度图片并组装四个独立 Geometry Nodes；Cutout、Balloon 与 Depth Balloon 的 camera-axis normal 会在最终网格求值后烘成 tangent-space Image。 -Cutout 的 `generate-mesh` 不复用上述裁切图作为 AI 输入。套索或 BEN2 结果仅写成几何 mask；MoGe 深度与法线始终分析完整图片。输入为 Upscale 结果时，请求会额外携带放大前的 `analysis_input`，推理读取该输入，颜色与几何 mask 仍读取当前高清图；Depth Balloon 会先把套索坐标缩放到分析分辨率，再限制有效深度。Blender 材质使用当前 Empty Image 的未裁切副本。 +Depth Balloon 的基面拟合使用四轮稳健加权回归，并对细长选区中方差不足的短轴设置相对主轴的方差下限。该约束只处理病态方向，不会移除长轴上由多数有效深度点支持的趋势。Blender 会把回归平面的符号和半深度倍率转换到 Cutout 的局部 YZ 纸面坐标；Geometry Nodes 计算 `(Base Plane - 深度基值) × Depth Scale`，因此 `Depth Scale = 0` 会同时消除深度图和 Base Plane 对深度分支的影响。 Image Box 与 Lasso 的 `Refine Selection` 使用 `refine-image-mask` Job。它与标准 `remove-background` Job 共用单图 BEN2 函数；Server 在推理前清空 polygon 外的 RGBA，推理后再次应用几何 mask，并返回局部 Alpha 与源图 bounds。Line、Polyline 与 Perspective 不提交 Job。 -所有 BEN2 调用都在公共模型入口用输入 Alpha 遮住透明 RGB。Remove Background、Depth Plane、Surface 与 Image Refine 的正式结果保留源 RGB，只采用 BEN2 Alpha；Balloon 同样只消费 BEN2 Alpha。该边界既防止模型识别透明区隐藏内容，也避免把透明黑写回 Blender 后产生过滤黑边。 +所有 BEN2 调用都在公共模型入口用输入 Alpha 遮住透明 RGB。Remove Background、Depth Plane、Surface 与 Image Refine 的正式结果保留源 RGB，只采用 BEN2 Alpha；Cutout 的 Refine Selection 同样只消费 BEN2 Alpha。该边界既防止模型识别透明区隐藏内容，也避免把透明黑写回 Blender 后产生过滤黑边。 正式功能支持： diff --git a/src/anyimage/operators/balloon/__init__.py b/src/anyimage/operators/balloon/__init__.py deleted file mode 100644 index b37aca9..0000000 --- a/src/anyimage/operators/balloon/__init__.py +++ /dev/null @@ -1 +0,0 @@ -"""Balloon profile geometry used by the unified mesh converter.""" diff --git a/src/anyimage/operators/balloon/depth.py b/src/anyimage/operators/balloon/depth.py deleted file mode 100644 index 26bca8b..0000000 --- a/src/anyimage/operators/balloon/depth.py +++ /dev/null @@ -1,621 +0,0 @@ -"""MoGe-2 depth sampling and adjustable Balloon deformation.""" - -from pathlib import Path - -import bpy - -BALLOON_DEPTH_ATTRIBUTE = "o_balloon_depth" -BALLOON_SHAPE_NODE_GROUP_NAME = "O Balloon Shape" -BALLOON_SHAPE_NODE_GROUP_VERSION = 16 -BALLOON_SHAPE_VERSION_PROPERTY = "anyimage_balloon_shape_version" - - -def load_balloon_depth_data(path): - import numpy as np - - try: - with np.load(Path(path), allow_pickle=False) as archive: - required = { - "depth_signal", - "baseline_vector", - "valid", - "origin", - "reference_depth", - "crop_size", - "source_size", - } - missing = required.difference(archive.files) - if missing: - raise ValueError( - f"Balloon depth data is missing: {', '.join(sorted(missing))}" - ) - data = { - "depth_signal": np.asarray( - archive["depth_signal"], - dtype=np.float32, - ), - "baseline_vector": np.asarray( - archive["baseline_vector"], - dtype=np.float32, - ), - "valid": np.asarray(archive["valid"], dtype=bool), - "origin": tuple(int(value) for value in archive["origin"]), - "reference_depth": float(archive["reference_depth"]), - "crop_size": tuple(int(value) for value in archive["crop_size"]), - "source_size": tuple(int(value) for value in archive["source_size"]), - } - except OSError as error: - raise ValueError(f"Unable to read Balloon depth data: {path}") from error - if data["depth_signal"].ndim != 2: - raise ValueError("Balloon depth signal must be two-dimensional") - if data["valid"].shape != data["depth_signal"].shape: - raise ValueError("Balloon depth mask dimensions do not match") - if data["baseline_vector"].shape != (3,): - raise ValueError("Balloon base plane vector must have three components") - if any(value <= 0 for value in (*data["crop_size"], *data["source_size"])): - raise ValueError("Balloon depth image dimensions must be positive") - return data - - -def _sample_valid_bilinear(values, valid, x, y): - import numpy as np - - height, width = values.shape - x = np.clip(x, 0.0, max(width - 1, 0)) - y = np.clip(y, 0.0, max(height - 1, 0)) - x0 = np.floor(x).astype(np.int64) - y0 = np.floor(y).astype(np.int64) - x1 = np.minimum(x0 + 1, width - 1) - y1 = np.minimum(y0 + 1, height - 1) - fx = x - x0 - fy = y - y0 - samples = np.stack( - ( - values[y0, x0], - values[y0, x1], - values[y1, x0], - values[y1, x1], - ), - axis=1, - ) - weights = np.stack( - ( - (1.0 - fx) * (1.0 - fy), - fx * (1.0 - fy), - (1.0 - fx) * fy, - fx * fy, - ), - axis=1, - ) - sample_valid = np.stack( - ( - valid[y0, x0], - valid[y0, x1], - valid[y1, x0], - valid[y1, x1], - ), - axis=1, - ) - weights *= sample_valid - totals = weights.sum(axis=1) - result = np.zeros(len(x), dtype=np.float32) - usable = totals > 1e-8 - result[usable] = (samples[usable] * weights[usable]).sum(axis=1) / totals[usable] - return result, usable - - -def balloon_depth_fields(vertices, vertex_uv, depth_data): - """Sample camera depth and convert the fitted base plane to local space.""" - import numpy as np - - vertices = np.asarray(vertices, dtype=np.float32) - uv = np.asarray(vertex_uv, dtype=np.float32) - if vertices.ndim != 2 or vertices.shape[1] != 3: - raise ValueError("Balloon vertices must have three components") - if uv.shape != (len(vertices), 2): - raise ValueError("Balloon UV coordinates do not match its vertices") - depth_signal = np.asarray(depth_data["depth_signal"], dtype=np.float32) - valid = np.asarray(depth_data["valid"], dtype=bool) - origin_x, origin_y = depth_data["origin"] - crop_width, crop_height = depth_data["crop_size"] - source_width, source_height = depth_data["source_size"] - depth_height, depth_width = depth_signal.shape - - crop_x = uv[:, 0] * source_width - origin_x - crop_y = (1.0 - uv[:, 1]) * source_height - origin_y - sample_x = crop_x * max(depth_width - 1, 0) / max(crop_width - 1, 1) - sample_y = crop_y * max(depth_height - 1, 0) / max(crop_height - 1, 1) - sampled, usable = _sample_valid_bilinear( - depth_signal, - valid, - sample_x, - sample_y, - ) - inside_crop = ( - (crop_x >= 0.0) - & (crop_x <= crop_width) - & (crop_y >= 0.0) - & (crop_y <= crop_height) - ) - usable &= inside_crop - sampled[~usable] = 0.0 - coordinates = np.stack( - ( - sample_x / max(depth_width - 1, 1) * 2.0 - 1.0, - sample_y / max(depth_height - 1, 1) * 2.0 - 1.0, - np.ones(len(vertices), dtype=np.float32), - ), - axis=1, - ).astype(np.float32) - - base_height = float(np.max(np.abs(vertices[:, 2]))) - if not base_height > 0.0: - return np.zeros(len(vertices), dtype=np.float32), np.zeros(3, dtype=np.float32) - depth_scale = base_height - normalized_plane = np.asarray( - depth_data["baseline_vector"], - dtype=np.float32, - ) - baseline_values = coordinates @ normalized_plane * depth_scale - local_design = np.stack( - ( - vertices[:, 0], - vertices[:, 1], - np.ones(len(vertices), dtype=np.float32), - ), - axis=1, - ) - if np.count_nonzero(usable) >= 3: - local_plane = np.linalg.lstsq( - local_design[usable], - baseline_values[usable], - rcond=None, - )[0].astype(np.float32) - else: - local_plane = np.zeros(3, dtype=np.float32) - depth_values = sampled * depth_scale - if usable.any(): - plane_without_offset = local_design[:, :2] @ local_plane[:2] - residual = depth_values - plane_without_offset - depth_values[usable] -= float(np.median(residual[usable])) - local_plane[2] = 0.0 - return depth_values, local_plane - - -def _assign_point_attribute(mesh, name, values): - import numpy as np - - values = np.asarray(values, dtype=np.float32) - if len(values) != len(mesh.vertices): - raise ValueError(f"Balloon attribute {name} does not match its vertices") - attribute = mesh.attributes.new(name=name, type="FLOAT", domain="POINT") - attribute.data.foreach_set("value", values) - - -def assign_balloon_shape_attributes(mesh, _vertices, depth_values=None): - if depth_values is not None: - _assign_point_attribute(mesh, BALLOON_DEPTH_ATTRIBUTE, depth_values) - - -def create_balloon_base_plane(context, balloon, vertices, local_plane): - import numpy as np - from mathutils import Vector - - vertices = np.asarray(vertices, dtype=np.float32) - slope_y, slope_z, offset = (float(value) for value in local_plane) - extent = vertices[:, 1:].max(axis=0) - vertices[:, 1:].min(axis=0) - normal = Vector((1.0, -slope_y, -slope_z)).normalized() - - controller = bpy.data.objects.new(f"{balloon.name} Base Plane", None) - context.collection.objects.link(controller) - controller.parent = balloon - controller.location = (0.0, 0.0, 0.0) - controller.rotation_euler = normal.to_track_quat("Z", "Y").to_euler() - controller.empty_display_type = "SINGLE_ARROW" - controller.empty_display_size = max(float(extent.max()) * 0.15, 0.01) - controller.show_in_front = True - controller.hide_render = True - controller["anyimage_source_type"] = "BALLOON_BASE_PLANE" - controller["anyimage_base_plane"] = (slope_y, slope_z, offset) - return controller - - -def _interface_socket(group, name, in_out, socket_type): - for item in group.interface.items_tree: - if item.item_type == "SOCKET" and item.name == name and item.in_out == in_out: - return item - return group.interface.new_socket( - name=name, - in_out=in_out, - socket_type=socket_type, - ) - - -def _remove_obsolete_interface_sockets(group): - obsolete = {"Base X", "Base Y", "Base Offset", "Far Clamp"} - for item in list(group.interface.items_tree): - if ( - item.item_type == "SOCKET" - and item.in_out == "INPUT" - and item.name in obsolete - ): - group.interface.remove(item) - - -def _float_input(group, name, default, minimum, maximum, description): - socket = _interface_socket(group, name, "INPUT", "NodeSocketFloat") - socket.default_value = default - socket.min_value = minimum - socket.max_value = maximum - socket.description = description - if hasattr(socket, "structure_type"): - socket.structure_type = "SINGLE" - return socket - - -def _bool_input(group, name, default, description): - socket = _interface_socket(group, name, "INPUT", "NodeSocketBool") - socket.default_value = default - socket.description = description - if hasattr(socket, "structure_type"): - socket.structure_type = "SINGLE" - return socket - - -def balloon_shape_node_group(): - group = bpy.data.node_groups.get(BALLOON_SHAPE_NODE_GROUP_NAME) - if ( - group is not None - and group.get(BALLOON_SHAPE_VERSION_PROPERTY) - == BALLOON_SHAPE_NODE_GROUP_VERSION - ): - return group - if group is None: - group = bpy.data.node_groups.new( - BALLOON_SHAPE_NODE_GROUP_NAME, - "GeometryNodeTree", - ) - else: - group.nodes.clear() - _remove_obsolete_interface_sockets(group) - - group.is_modifier = True - geometry_input = _interface_socket( - group, - "Geometry", - "INPUT", - "NodeSocketGeometry", - ) - if hasattr(geometry_input, "structure_type"): - geometry_input.structure_type = "SINGLE" - _float_input( - group, - "Thickness", - 1.0, - 0.0, - 10.0, - "Scale the symmetric base Balloon thickness along local X.", - ) - _float_input( - group, - "Depth Strength", - 1.0, - 0.0, - 10.0, - "Scale the signed MoGe-2 depth variation.", - ) - _bool_input( - group, - "Double Sided Depth", - False, - "Apply AI depth to both sides; keep disabled for a stable Balloon back.", - ) - base_plane_input = _interface_socket( - group, - "Base Plane", - "INPUT", - "NodeSocketObject", - ) - base_plane_input.description = ( - "Rotate the child Empty for middle-plane orientation; move it along local Z " - "to set the neutral depth level." - ) - _interface_socket(group, "Geometry", "OUTPUT", "NodeSocketGeometry") - - nodes = group.nodes - links = group.links - group_input = nodes.new("NodeGroupInput") - group_input.location = (-1500.0, 500.0) - position = nodes.new("GeometryNodeInputPosition") - position.location = (-1500.0, 200.0) - separate = nodes.new("ShaderNodeSeparateXYZ") - separate.location = (-1300.0, 200.0) - base_scale = nodes.new("ShaderNodeMath") - base_scale.operation = "MULTIPLY" - base_scale.location = (-1050.0, 350.0) - base_magnitude = nodes.new("ShaderNodeMath") - base_magnitude.operation = "ABSOLUTE" - base_magnitude.location = (-800.0, 350.0) - - side = nodes.new("ShaderNodeMath") - side.operation = "SIGN" - side.location = (-1050.0, 100.0) - front = nodes.new("ShaderNodeMath") - front.operation = "LESS_THAN" - front.inputs[1].default_value = 0.0 - front.location = (750.0, -350.0) - - depth_attribute = nodes.new("GeometryNodeInputNamedAttribute") - depth_attribute.data_type = "FLOAT" - depth_attribute.inputs["Name"].default_value = BALLOON_DEPTH_ATTRIBUTE - depth_attribute.location = (-1500.0, -150.0) - profile_factor = nodes.new("ShaderNodeMath") - profile_factor.operation = "ABSOLUTE" - profile_factor.location = (-800.0, 100.0) - profile_statistics = nodes.new("GeometryNodeAttributeStatistic") - profile_statistics.data_type = "FLOAT" - profile_statistics.domain = "POINT" - profile_statistics.location = (-600.0, 50.0) - profile_max_safe = nodes.new("ShaderNodeMath") - profile_max_safe.operation = "MAXIMUM" - profile_max_safe.inputs[1].default_value = 1e-6 - profile_max_safe.location = (-350.0, 100.0) - profile_normalized = nodes.new("ShaderNodeMath") - profile_normalized.operation = "DIVIDE" - profile_normalized.location = (-100.0, 100.0) - profile_smooth = nodes.new("ShaderNodeMapRange") - profile_smooth.data_type = "FLOAT" - profile_smooth.interpolation_type = "SMOOTHERSTEP" - profile_smooth.clamp = True - profile_smooth.inputs["From Min"].default_value = 0.0 - profile_smooth.inputs["From Max"].default_value = 1.0 - profile_smooth.inputs["To Min"].default_value = 0.0 - profile_smooth.inputs["To Max"].default_value = 1.0 - profile_smooth.location = (100.0, 50.0) - profile_active = nodes.new("ShaderNodeMath") - profile_active.operation = "MULTIPLY" - profile_active.location = (350.0, 50.0) - - object_info = nodes.new("GeometryNodeObjectInfo") - object_info.transform_space = "RELATIVE" - object_info.location = (-1500.0, -450.0) - plane_normal = nodes.new("FunctionNodeRotateVector") - plane_normal.inputs["Vector"].default_value = (0.0, 0.0, 1.0) - plane_normal.location = (-1250.0, -500.0) - effective_normal = nodes.new("GeometryNodeSwitch") - effective_normal.input_type = "VECTOR" - effective_normal.inputs["False"].default_value = (1.0, 0.0, 0.0) - effective_normal.location = (-1050.0, -500.0) - normal_separate = nodes.new("ShaderNodeSeparateXYZ") - normal_separate.location = (-1000.0, -550.0) - zero_position = nodes.new("ShaderNodeCombineXYZ") - zero_position.location = (-1050.0, -250.0) - plane_dot = nodes.new("ShaderNodeVectorMath") - plane_dot.operation = "DOT_PRODUCT" - plane_dot.location = (-800.0, -300.0) - plane_offset = nodes.new("ShaderNodeVectorMath") - plane_offset.operation = "DOT_PRODUCT" - plane_offset.location = (-800.0, -500.0) - plane_numerator = nodes.new("ShaderNodeMath") - plane_numerator.operation = "SUBTRACT" - plane_numerator.location = (-550.0, -300.0) - plane_divide = nodes.new("ShaderNodeMath") - plane_divide.operation = "DIVIDE" - plane_divide.location = (-300.0, -300.0) - - depth_residual = nodes.new("ShaderNodeMath") - depth_residual.operation = "ADD" - depth_residual.location = (-50.0, -200.0) - depth_calibration = nodes.new("ShaderNodeMath") - depth_calibration.operation = "MULTIPLY" - depth_calibration.inputs[1].default_value = 0.5 - depth_calibration.label = "Calibrate Relative Depth" - depth_calibration.location = (200.0, -200.0) - corrected_depth = nodes.new("ShaderNodeMath") - corrected_depth.operation = "ADD" - corrected_depth.location = (450.0, -200.0) - depth_scale = nodes.new("ShaderNodeMath") - depth_scale.operation = "MULTIPLY" - depth_scale.location = (700.0, -150.0) - depth_mix = nodes.new("ShaderNodeMix") - depth_mix.data_type = "FLOAT" - depth_mix.clamp_factor = True - depth_mix.location = (450.0, -50.0) - signed_depth = nodes.new("ShaderNodeMath") - signed_depth.operation = "MULTIPLY" - signed_depth.location = (850.0, 50.0) - front_choice = nodes.new("GeometryNodeSwitch") - front_choice.input_type = "FLOAT" - front_choice.location = (1100.0, 0.0) - double_choice = nodes.new("GeometryNodeSwitch") - double_choice.input_type = "FLOAT" - double_choice.location = (1350.0, 100.0) - combine = nodes.new("ShaderNodeCombineXYZ") - combine.location = (1600.0, 250.0) - set_position = nodes.new("GeometryNodeSetPosition") - set_position.location = (1850.0, 450.0) - group_output = nodes.new("NodeGroupOutput") - group_output.location = (2450.0, 450.0) - - base_frame = nodes.new("NodeFrame") - base_frame.label = "Base Thickness" - plane_frame = nodes.new("NodeFrame") - plane_frame.label = "Middle Plane" - mix_frame = nodes.new("NodeFrame") - mix_frame.label = "Depth Mix" - output_frame = nodes.new("NodeFrame") - output_frame.label = "Side Output" - for node in ( - separate, - base_scale, - base_magnitude, - profile_factor, - profile_statistics, - profile_max_safe, - profile_normalized, - profile_smooth, - ): - node.parent = base_frame - for node in ( - object_info, - plane_normal, - effective_normal, - normal_separate, - zero_position, - plane_dot, - plane_offset, - plane_numerator, - plane_divide, - ): - node.parent = plane_frame - for node in ( - depth_residual, - depth_calibration, - corrected_depth, - depth_scale, - profile_active, - depth_mix, - ): - node.parent = mix_frame - for node in ( - side, - front, - signed_depth, - front_choice, - double_choice, - combine, - set_position, - ): - node.parent = output_frame - - base_frame.location = (-1800.0, 500.0) - separate.location = (0.0, 200.0) - base_scale.location = (220.0, 350.0) - base_magnitude.location = (440.0, 350.0) - profile_factor.location = (220.0, 50.0) - profile_statistics.location = (440.0, 0.0) - profile_max_safe.location = (660.0, 0.0) - profile_normalized.location = (880.0, 0.0) - profile_smooth.location = (1100.0, 0.0) - - plane_frame.location = (-1800.0, -600.0) - object_info.location = (0.0, -300.0) - plane_normal.location = (220.0, -300.0) - effective_normal.location = (440.0, -300.0) - normal_separate.location = (660.0, -300.0) - zero_position.location = (220.0, 0.0) - plane_dot.location = (440.0, 0.0) - plane_offset.location = (660.0, -100.0) - plane_numerator.location = (880.0, 0.0) - plane_divide.location = (1100.0, 0.0) - - mix_frame.location = (-200.0, 200.0) - profile_active.location = (0.0, 300.0) - depth_residual.location = (0.0, 0.0) - depth_calibration.location = (220.0, 0.0) - corrected_depth.location = (440.0, 0.0) - depth_scale.location = (660.0, 0.0) - depth_mix.location = (880.0, 150.0) - - output_frame.location = (900.0, 200.0) - side.location = (0.0, 0.0) - front.location = (0.0, -250.0) - signed_depth.location = (220.0, 200.0) - front_choice.location = (440.0, 100.0) - double_choice.location = (660.0, 150.0) - combine.location = (880.0, 150.0) - set_position.location = (1100.0, 150.0) - group_input.location = (-2200.0, 500.0) - position.location = (-2200.0, 200.0) - depth_attribute.location = (-2200.0, -200.0) - group_output.location = (2500.0, 350.0) - - links.new(position.outputs["Position"], separate.inputs["Vector"]) - links.new(separate.outputs["X"], base_scale.inputs[0]) - links.new(group_input.outputs["Thickness"], base_scale.inputs[1]) - links.new(base_scale.outputs[0], base_magnitude.inputs[0]) - links.new(separate.outputs["X"], side.inputs[0]) - links.new(separate.outputs["X"], front.inputs[0]) - links.new(separate.outputs["X"], profile_factor.inputs[0]) - links.new(group_input.outputs["Geometry"], profile_statistics.inputs["Geometry"]) - links.new(profile_factor.outputs[0], profile_statistics.inputs["Attribute"]) - links.new(profile_statistics.outputs["Max"], profile_max_safe.inputs[0]) - links.new(profile_factor.outputs[0], profile_normalized.inputs[0]) - links.new(profile_max_safe.outputs[0], profile_normalized.inputs[1]) - links.new(profile_normalized.outputs[0], profile_smooth.inputs["Value"]) - links.new(profile_smooth.outputs["Result"], profile_active.inputs[0]) - links.new(depth_attribute.outputs["Exists"], profile_active.inputs[1]) - - links.new(group_input.outputs["Base Plane"], object_info.inputs["Object"]) - links.new(object_info.outputs["Rotation"], plane_normal.inputs["Rotation"]) - links.new(depth_attribute.outputs["Exists"], effective_normal.inputs["Switch"]) - links.new(plane_normal.outputs["Vector"], effective_normal.inputs["True"]) - links.new(effective_normal.outputs["Output"], normal_separate.inputs["Vector"]) - links.new(separate.outputs["Y"], zero_position.inputs["Y"]) - links.new(separate.outputs["Z"], zero_position.inputs["Z"]) - links.new(zero_position.outputs["Vector"], plane_dot.inputs[0]) - links.new(effective_normal.outputs["Output"], plane_dot.inputs[1]) - links.new(object_info.outputs["Location"], plane_offset.inputs[0]) - links.new(effective_normal.outputs["Output"], plane_offset.inputs[1]) - links.new(plane_offset.outputs["Value"], plane_numerator.inputs[0]) - links.new(plane_dot.outputs["Value"], plane_numerator.inputs[1]) - links.new(plane_numerator.outputs[0], plane_divide.inputs[0]) - links.new(normal_separate.outputs["X"], plane_divide.inputs[1]) - - links.new(depth_attribute.outputs["Attribute"], depth_residual.inputs[0]) - links.new(plane_divide.outputs[0], depth_residual.inputs[1]) - links.new(depth_residual.outputs[0], depth_calibration.inputs[0]) - links.new(depth_calibration.outputs[0], corrected_depth.inputs[0]) - links.new(profile_statistics.outputs["Max"], corrected_depth.inputs[1]) - links.new(corrected_depth.outputs[0], depth_scale.inputs[0]) - links.new(group_input.outputs["Depth Strength"], depth_scale.inputs[1]) - links.new(profile_active.outputs[0], depth_mix.inputs["Factor"]) - links.new(base_magnitude.outputs[0], depth_mix.inputs["A"]) - links.new(depth_scale.outputs[0], depth_mix.inputs["B"]) - links.new(side.outputs[0], signed_depth.inputs[0]) - links.new(depth_mix.outputs["Result"], signed_depth.inputs[1]) - links.new(front.outputs[0], front_choice.inputs["Switch"]) - links.new(base_scale.outputs[0], front_choice.inputs["False"]) - links.new(signed_depth.outputs[0], front_choice.inputs["True"]) - links.new( - group_input.outputs["Double Sided Depth"], - double_choice.inputs["Switch"], - ) - links.new(front_choice.outputs["Output"], double_choice.inputs["False"]) - links.new(signed_depth.outputs[0], double_choice.inputs["True"]) - links.new(double_choice.outputs["Output"], combine.inputs["X"]) - links.new(separate.outputs["Y"], combine.inputs["Y"]) - links.new(separate.outputs["Z"], combine.inputs["Z"]) - links.new(group_input.outputs["Geometry"], set_position.inputs["Geometry"]) - links.new(combine.outputs["Vector"], set_position.inputs["Position"]) - links.new(set_position.outputs["Geometry"], group_output.inputs["Geometry"]) - group.color_tag = "GEOMETRY" - group.description = ( - "Mix Balloon thickness with depth around an adjustable child Empty middle plane." - ) - group[BALLOON_SHAPE_VERSION_PROPERTY] = BALLOON_SHAPE_NODE_GROUP_VERSION - group.use_fake_user = True - return group - - -def add_balloon_shape_modifier( - balloon, - thickness, - base_plane=None, -): - from ..material import modifier_input_identifier, set_modifier_input - - node_group = balloon_shape_node_group() - modifier = balloon.modifiers.new(name=node_group.name, type="NODES") - modifier.node_group = node_group - modifier.show_group_selector = False - values = { - "Thickness": thickness, - "Depth Strength": 1.0, - "Double Sided Depth": False, - "Base Plane": base_plane, - } - for name, value in values.items(): - identifier = modifier_input_identifier(node_group, name) - set_modifier_input(modifier, identifier, value) - return modifier diff --git a/src/anyimage/operators/balloon/operator.py b/src/anyimage/operators/balloon/operator.py deleted file mode 100644 index e32c5ca..0000000 --- a/src/anyimage/operators/balloon/operator.py +++ /dev/null @@ -1,857 +0,0 @@ -import json -import math - -import bpy - -from ...properties import ( - BEN2_MODEL_KEY, - ai_ready, - ai_setup_label, - ai_status, -) -from ...properties import ( - balloon_contour_analysis_property as contour_analysis_property, -) -from ...properties import ( - balloon_depth_analysis_property as depth_analysis_property, -) -from ...properties import ( - balloon_depth_model_property as depth_model_property, -) -from ...properties import ( - balloon_mode_property as mode_property, -) -from ...properties import ( - balloon_resolution_property as resolution_property, -) -from ...properties import ( - balloon_reuse_material_property as reuse_material_property, -) -from ...properties import ( - balloon_thickness_property as thickness_property, -) -from ...runtime import JobOperatorBase -from ..common import ( - HIDDEN, - cleanup_temporary_input, - production_device, - push_undo, - require_environment, - require_model, -) -from ..image import ( - color_data_name, - is_image_empty, - material_data_name, - normal_data_name, - plane_bounds, - prepare_image_input, - remove_temporary_input, - video_total_frames, -) -from ..material import ( - IMAGE_MATERIAL_NODE_GROUP_NAME, - create_material, -) -from ..normal_map import bake_tangent_normal_image -from ..surface import assign_vertex_uv -from .depth import ( - add_balloon_shape_modifier, - assign_balloon_shape_attributes, - balloon_depth_fields, - create_balloon_base_plane, - load_balloon_depth_data, -) -from .geometry import INFLATION, POINT_SPACING, triangulate_balloon - -MIN_PATH_DISTANCE = 2.0 -BALLOON_MATERIAL_PROPERTY = "anyimage_balloon_material" -BALLOON_SOURCE_IMAGE_PROPERTY = "anyimage_source_image" -BALLOON_NORMAL_IMAGE_PROPERTY = "anyimage_balloon_normal_image" - - -def simplify_screen_path(path): - pixels = [] - for point in path: - point = (float(point[0]), float(point[1])) - if not pixels or math.dist(point, pixels[-1]) >= MIN_PATH_DISTANCE: - pixels.append(point) - return pixels - - -def screen_path_to_image_pixels( - path, - region, - region_data, - matrix_world, - local_bounds, - image_size, -): - from bpy_extras import view3d_utils - - inverse = matrix_world.inverted() - x_min, x_max, y_min, y_max = local_bounds - image_width, image_height = image_size - local_width = x_max - x_min - local_height = y_max - y_min - pixels = [] - for point in simplify_screen_path(path): - ray_origin = inverse @ view3d_utils.region_2d_to_origin_3d( - region, - region_data, - point, - ) - ray_direction = inverse.to_3x3() @ view3d_utils.region_2d_to_vector_3d( - region, - region_data, - point, - ) - if abs(ray_direction.z) <= 1e-10: - raise ValueError("The Image Empty is edge-on to the current view") - distance = -ray_origin.z / ray_direction.z - local = ray_origin + ray_direction * distance - pixels.append( - ( - (local.x - x_min) * image_width / local_width, - (y_max - local.y) * image_height / local_height, - ) - ) - return pixels - - -def world_plane_metric(matrix_world): - from mathutils import Vector - - linear = matrix_world.to_3x3() - basis_x = linear @ Vector((1.0, 0.0, 0.0)) - basis_y = linear @ Vector((0.0, 1.0, 0.0)) - basis_z = linear @ Vector((0.0, 0.0, 1.0)) - metric = ( - (basis_x.dot(basis_x), basis_x.dot(basis_y)), - (basis_x.dot(basis_y), basis_y.dot(basis_y)), - ) - return metric, basis_z.length - - -def serialize_matrix(matrix): - return json.dumps( - [float(matrix[row][column]) for row in range(4) for column in range(4)], - separators=(",", ":"), - ) - - -def deserialize_matrix(data): - from mathutils import Matrix - - values = json.loads(data) - if len(values) != 16: - raise ValueError("The stored Image Empty transform is invalid") - return Matrix(tuple(tuple(values[row * 4 : row * 4 + 4]) for row in range(4))) - - -def balloon_geometry_parameters(thickness, resolution): - if thickness <= 0.0: - raise ValueError("Balloon thickness must be greater than zero") - if resolution <= 0.0: - raise ValueError("Balloon resolution must be greater than zero") - return resolution, INFLATION * thickness - - -def data_name_matches(name, base_name): - if name == base_name: - return True - suffix = name.removeprefix(f"{base_name}.") - return suffix != name and suffix.isdigit() - - -def marked_balloon_material(source_image, normal_required=False): - return next( - ( - material - for material in bpy.data.materials - if callable(getattr(material, "get", None)) - and bool(material.get(BALLOON_MATERIAL_PROPERTY)) - and material.get(BALLOON_SOURCE_IMAGE_PROPERTY) is source_image - and (material.get(BALLOON_NORMAL_IMAGE_PROPERTY) is not None) - == normal_required - ), - None, - ) - - -def mark_balloon_material(material, source_image, normal_image=None): - material[BALLOON_MATERIAL_PROPERTY] = True - material[BALLOON_SOURCE_IMAGE_PROPERTY] = source_image - if normal_image is not None: - material[BALLOON_NORMAL_IMAGE_PROPERTY] = normal_image - - -def reusable_balloon_material(source_image, normal_required=False): - return marked_balloon_material(source_image, normal_required) - - -def load_balloon_normal_image(path, source_object): - image = bpy.data.images.load( - str(path), - check_existing=False, - ) - image.name = normal_data_name(source_object) - image.colorspace_settings.name = "Non-Color" - return image - - -def merge_balloon_normal_image(target, update): - import numpy as np - - if tuple(target.size) != tuple(update.size): - raise ValueError("Reusable Balloon normal image dimensions do not match") - target_pixels = np.empty(len(target.pixels), dtype=np.float32) - update_pixels = np.empty(len(update.pixels), dtype=np.float32) - target.pixels.foreach_get(target_pixels) - update.pixels.foreach_get(update_pixels) - target_rgba = target_pixels.reshape((-1, 4)) - update_rgba = update_pixels.reshape((-1, 4)) - neutral = np.asarray((128.0 / 255.0, 128.0 / 255.0, 1.0)) - changed = np.max(np.abs(update_rgba[:, :3] - neutral), axis=1) > (1.5 / 255.0) - target_rgba[changed] = update_rgba[changed] - target.pixels.foreach_set(target_pixels) - target.pack() - target.update() - - -def reusable_color_image(color_name): - return next( - ( - image - for image in bpy.data.images - if data_name_matches(image.name, color_name) - ), - None, - ) - - -BALLOON_AXIS_TO_SOURCE = ( - (0.0, -1.0, 0.0, 0.0), - (0.0, 0.0, 1.0, 0.0), - (-1.0, 0.0, 0.0, 0.0), - (0.0, 0.0, 0.0, 1.0), -) - - -def orient_balloon_vertices(vertices): - """Rotate legacy XY/Z Balloon coordinates into a YZ/-X frame.""" - return [(-z, -x, y) for x, y, z in vertices] - - -def orient_balloon_world_matrix(source_matrix): - from mathutils import Matrix - - return source_matrix @ Matrix(BALLOON_AXIS_TO_SOURCE) - - -def orient_balloon_base_plane(local_plane): - slope_x, slope_y, offset = local_plane - return (slope_x, -slope_y, -offset) - - -def extract_balloon_object( - context, - source_object, - regions, - matrix_world, - algorithm, - thickness=1.0, - resolution=POINT_SPACING, - select_result=True, - reuse_material=True, - depth_data=None, - normal_path=None, - material_image=None, - local_bounds=None, - bake_normal_to_tangent=False, -): - image = material_image or source_object.data - image_size = tuple(image.size) - local_bounds = local_bounds or plane_bounds(source_object) - world_metric, world_z_scale = world_plane_metric(matrix_world) - point_spacing, inflation = balloon_geometry_parameters( - thickness, - resolution, - ) - vertices, faces, vertex_uv = triangulate_balloon( - regions, - image_size, - local_bounds, - spacing=point_spacing, - world_metric=world_metric, - world_z_scale=world_z_scale, - inflation=INFLATION, - algorithm=algorithm, - ) - - depth_values = None - local_base_plane = (0.0, 0.0, 0.0) - if depth_data is not None: - depth_values, local_base_plane = balloon_depth_fields( - vertices, - vertex_uv, - depth_data, - ) - local_base_plane = orient_balloon_base_plane(local_base_plane) - vertices = orient_balloon_vertices(vertices) - source_name = source_object.name - balloon_name = source_name - material_name = material_data_name(source_object) - color_name = color_data_name(source_object) - color_image = None - uses_normal = normal_path is not None - material = ( - reusable_balloon_material( - image, - normal_required=uses_normal, - ) - if reuse_material - else None - ) - normal_image = None - owns_color_image = False - owns_normal_image = False - owns_material = False - mesh = None - balloon = None - base_plane = None - try: - if material is not None and not uses_normal: - mark_balloon_material(material, image) - if material is not None and uses_normal: - normal_image = material.get(BALLOON_NORMAL_IMAGE_PROPERTY) - update_normal = load_balloon_normal_image(normal_path, source_object) - try: - merge_balloon_normal_image(normal_image, update_normal) - finally: - bpy.data.images.remove(update_normal) - if material is None: - color_image = material_image - if color_image is None and reuse_material: - color_image = reusable_color_image(color_name) - if color_image is None: - color_image = image.copy() - color_image.name = color_name - owns_color_image = True - if normal_path is not None: - normal_image = load_balloon_normal_image( - normal_path, - source_object, - ) - normal_image.pack() - owns_normal_image = True - material = create_material( - material_name, - color_image, - 0.0, - surface_group_name=IMAGE_MATERIAL_NODE_GROUP_NAME, - normal_image=normal_image, - scene=context.scene, - ) - mark_balloon_material(material, image, normal_image) - owns_material = True - mesh = bpy.data.meshes.new(balloon_name) - mesh.from_pydata(vertices, [], faces) - assign_balloon_shape_attributes( - mesh, - vertices, - depth_values, - ) - mesh.update() - assign_vertex_uv(mesh, vertex_uv) - for face in mesh.polygons: - face.use_smooth = True - - balloon = bpy.data.objects.new(balloon_name, mesh) - context.collection.objects.link(balloon) - balloon.matrix_world = orient_balloon_world_matrix(matrix_world) - balloon["anyimage_source_type"] = "BALLOON" - balloon["anyimage_balloon_algorithm"] = algorithm - balloon["anyimage_source_object"] = source_name - balloon["anyimage_inflation"] = inflation - balloon["anyimage_resolution"] = point_spacing - balloon.data.materials.append(material) - if depth_data is not None: - base_plane = create_balloon_base_plane( - context, - balloon, - vertices, - local_base_plane, - ) - add_balloon_shape_modifier( - balloon, - thickness, - base_plane, - ) - if bake_normal_to_tangent and normal_image is not None: - bake_tangent_normal_image( - context, - source_object, - balloon, - normal_image, - ) - if depth_data is not None: - balloon["anyimage_balloon_depth_analysis"] = True - balloon["anyimage_reference_depth"] = depth_data["reference_depth"] - except Exception: - if base_plane is not None: - bpy.data.objects.remove(base_plane, do_unlink=True) - if balloon is not None: - bpy.data.objects.remove(balloon, do_unlink=True) - if mesh is not None and mesh.users == 0: - bpy.data.meshes.remove(mesh) - if owns_material and material is not None and material.users == 0: - bpy.data.materials.remove(material) - if owns_color_image and color_image is not None and color_image.users == 0: - bpy.data.images.remove(color_image) - if owns_normal_image and normal_image is not None and normal_image.users == 0: - bpy.data.images.remove(normal_image) - raise - - bpy.ops.object.select_all(action="DESELECT") - selected_object = balloon if select_result else source_object - selected_object.select_set(True) - context.view_layer.objects.active = selected_object - return balloon - - -def hidden_string_property(): - return bpy.props.StringProperty(options={"HIDDEN", "SKIP_SAVE"}) - - -def meters_to_world_distance(scene, meters): - unit_settings = getattr(scene, "unit_settings", None) - scale_length = getattr(unit_settings, "scale_length", 1.0) - if scale_length <= 0.0: - scale_length = 1.0 - return meters / scale_length - - -def draw_line_strip(vertices, color, width=2.0): - import gpu - from gpu_extras.batch import batch_for_shader - - shader = gpu.shader.from_builtin("UNIFORM_COLOR") - batch = batch_for_shader(shader, "LINE_STRIP", {"pos": vertices}) - gpu.state.blend_set("ALPHA") - gpu.state.line_width_set(width) - shader.bind() - shader.uniform_float("color", color) - batch.draw(shader) - gpu.state.line_width_set(1.0) - gpu.state.blend_set("NONE") - - -def drawing_in_region(area_pointer, region_pointer): - area = getattr(bpy.context, "area", None) - region = getattr(bpy.context, "region", None) - return ( - area is not None - and region is not None - and area.as_pointer() == area_pointer - and region.as_pointer() == region_pointer - ) - - -class ExtractBalloon(bpy.types.Operator): - bl_idname = "anyimage.extract_balloon" - bl_label = "Extract Balloon" - bl_description = "Build one closed Balloon mesh from a saved lasso region" - bl_options = {"REGISTER", "UNDO"} - - mode: mode_property() - thickness: thickness_property() - resolution: resolution_property() - source_object_name: hidden_string_property() - reuse_material: reuse_material_property() - contour_analysis: contour_analysis_property() - depth_analysis: depth_analysis_property() - depth_model: depth_model_property() - source_matrix_data: hidden_string_property() - selection_data: hidden_string_property() - - _path = None - _handle = None - _area_pointer = 0 - _region_pointer = 0 - - def draw(self, context): - layout = self.layout - layout.prop(self, "mode") - layout.prop(self, "thickness") - layout.prop(self, "resolution", text="Resolution (m)") - layout.prop(self, "reuse_material") - status = ai_status() - ready = status["ready"] - contour_row = layout.row() - contour_row.enabled = ready - contour_row.prop(self, "contour_analysis") - depth_row = layout.row() - depth_row.enabled = ready - depth_row.prop(self, "depth_analysis") - if self.depth_analysis: - model_row = layout.row() - model_row.enabled = ready - model_row.prop(self, "depth_model") - if not ready: - layout.operator( - "anyimage.setup_ai_environment", - text=ai_setup_label(status), - icon="IMPORT", - ) - - @classmethod - def poll(cls, context): - return ( - context.area is not None - and context.area.type == "VIEW_3D" - and is_image_empty(context.active_object) - ) - - def _draw_lasso(self): - if not self._path or not drawing_in_region( - self._area_pointer, self._region_pointer - ): - return - vertices = list(self._path) - if len(vertices) > 2: - vertices.append(vertices[0]) - draw_line_strip(vertices, (0.15, 0.65, 1.0, 0.95), 2.0) - - def _finish(self, context): - if self._handle is not None: - bpy.types.SpaceView3D.draw_handler_remove(self._handle, "WINDOW") - self._handle = None - context.workspace.status_text_set(None) - if context.area is not None: - context.area.tag_redraw() - - def cancel(self, context): - self._finish(context) - - def invoke(self, context, event): - source_object = context.active_object - settings = context.scene.anyimage_settings - self.mode = settings.balloon_mode - self.thickness = settings.balloon_thickness - self.resolution = settings.balloon_resolution - self.reuse_material = settings.balloon_reuse_material - ready = ai_ready() - self.contour_analysis = ready and settings.balloon_contour_analysis - self.depth_analysis = ready and settings.balloon_depth_analysis - self.depth_model = settings.balloon_depth_model - self.source_object_name = source_object.name - self.source_matrix_data = serialize_matrix(source_object.matrix_world) - self.selection_data = "" - self._path = [(event.mouse_region_x, event.mouse_region_y)] - self._area_pointer = context.area.as_pointer() - self._region_pointer = context.region.as_pointer() - self._handle = bpy.types.SpaceView3D.draw_handler_add( - self._draw_lasso, - (), - "WINDOW", - "POST_PIXEL", - ) - context.workspace.status_text_set( - "Balloon Lasso: release LMB to extract, RMB or Esc to cancel" - ) - context.window_manager.modal_handler_add(self) - context.area.tag_redraw() - return {"RUNNING_MODAL"} - - def modal(self, context, event): - if event.type in {"ESC", "RIGHTMOUSE"} and event.value == "PRESS": - self._finish(context) - return {"CANCELLED"} - - if event.type == "MOUSEMOVE": - self._path.append((event.mouse_region_x, event.mouse_region_y)) - context.area.tag_redraw() - return {"RUNNING_MODAL"} - - if event.type == "LEFTMOUSE" and event.value == "RELEASE": - path = simplify_screen_path(self._path) - if len(path) < 3: - self.report({"WARNING"}, "Draw a larger lasso") - self._finish(context) - return {"CANCELLED"} - - source_object = bpy.data.objects.get(self.source_object_name) - if not is_image_empty(source_object): - self.report({"ERROR"}, "The source Image Empty is no longer available") - self._finish(context) - return {"CANCELLED"} - try: - matrix_world = deserialize_matrix(self.source_matrix_data) - polygon = screen_path_to_image_pixels( - path, - context.region, - context.region_data, - matrix_world, - plane_bounds(source_object), - tuple(source_object.data.size), - ) - self.selection_data = json.dumps(polygon, separators=(",", ":")) - except (RuntimeError, TypeError, ValueError) as error: - self.report({"ERROR"}, str(error)) - self._finish(context) - return {"CANCELLED"} - - self._finish(context) - return self.execute(context) - - return {"PASS_THROUGH"} - - def execute(self, context): - source_object = bpy.data.objects.get(self.source_object_name) - if not is_image_empty(source_object): - self.report({"ERROR"}, "The source Image Empty is no longer available") - return {"CANCELLED"} - try: - polygon = json.loads(self.selection_data) - if len(polygon) < 3: - raise ValueError("The stored lasso region is invalid") - if self.contour_analysis or self.depth_analysis: - if video_total_frames(source_object.data) > 1: - raise RuntimeError("AI Analysis only supports a still image") - source_path, delete_input = prepare_image_input( - source_object.data, - context.scene, - ) - result = bpy.ops.anyimage.analyze_balloon( - input_path=str(source_path), - delete_input=delete_input, - source_object_name=self.source_object_name, - source_matrix_data=self.source_matrix_data, - selection_data=self.selection_data, - mode=self.mode, - thickness=self.thickness, - resolution=self.resolution, - reuse_material=self.reuse_material, - contour_analysis=self.contour_analysis, - depth_analysis=self.depth_analysis, - depth_model=self.depth_model, - ) - if "CANCELLED" in result: - if delete_input: - remove_temporary_input(source_path) - return {"CANCELLED"} - return {"FINISHED"} - matrix_world = deserialize_matrix(self.source_matrix_data) - resolution_world = meters_to_world_distance( - context.scene, - self.resolution, - ) - balloon = extract_balloon_object( - context, - source_object, - ((polygon,),), - matrix_world, - self.mode, - thickness=self.thickness, - resolution=resolution_world, - select_result=False, - reuse_material=self.reuse_material, - ) - balloon["anyimage_resolution_m"] = self.resolution - except (json.JSONDecodeError, RuntimeError, TypeError, ValueError) as error: - self.report({"ERROR"}, str(error)) - return {"CANCELLED"} - return {"FINISHED"} - - -class AnalyzeBalloon(JobOperatorBase, bpy.types.Operator): - bl_idname = "anyimage.analyze_balloon" - bl_label = "Analyze Balloon" - job_type = "analyze-balloon" - starting_message = "Starting Balloon AI analysis" - - input_path: bpy.props.StringProperty(options=HIDDEN) - delete_input: bpy.props.BoolProperty(default=False, options=HIDDEN) - source_object_name: hidden_string_property() - source_matrix_data: hidden_string_property() - selection_data: hidden_string_property() - mode: mode_property() - thickness: thickness_property() - resolution: resolution_property() - reuse_material: reuse_material_property() - contour_analysis: bpy.props.BoolProperty(default=False, options=HIDDEN) - depth_analysis: bpy.props.BoolProperty(default=False, options=HIDDEN) - depth_model: depth_model_property() - resolution_level: bpy.props.IntProperty( - default=5, - min=0, - max=9, - options=HIDDEN, - ) - - def request(self, _context): - require_environment() - if self.contour_analysis: - require_model(BEN2_MODEL_KEY) - if self.depth_analysis: - require_model(self.depth_model) - if not self.contour_analysis and not self.depth_analysis: - raise ValueError("Select Contour or Depth") - polygon = json.loads(self.selection_data) - if len(polygon) < 3: - raise ValueError("The stored lasso region is invalid") - return { - "input": str(self.input_path), - "contour_analysis": self.contour_analysis, - "depth_analysis": self.depth_analysis, - "contour_model": BEN2_MODEL_KEY, - "depth_model": self.depth_model, - "device": production_device().lower(), - "resolution_level": self.resolution_level, - "polygon": polygon, - } - - def response(self, context, result): - source_object = bpy.data.objects.get(self.source_object_name) - if not is_image_empty(source_object): - raise RuntimeError("The source Image Empty is no longer available") - polygon = json.loads(self.selection_data) - if self.contour_analysis: - payload = json.loads(result.file("contour").read_text(encoding="utf-8")) - polygon = payload.get("points", ()) - if len(polygon) < 3: - raise RuntimeError("BEN2 did not return a valid Balloon contour") - depth_data = ( - load_balloon_depth_data(result.file("depth")) - if self.depth_analysis - else None - ) - matrix_world = deserialize_matrix(self.source_matrix_data) - resolution_world = meters_to_world_distance( - context.scene, - self.resolution, - ) - balloon = extract_balloon_object( - context, - source_object, - ((polygon,),), - matrix_world, - self.mode, - thickness=self.thickness, - resolution=resolution_world, - select_result=False, - reuse_material=self.reuse_material, - depth_data=depth_data, - normal_path=(result.file("normal") if self.depth_analysis else None), - bake_normal_to_tangent=self.depth_analysis, - ) - balloon["anyimage_resolution_m"] = self.resolution - if self.contour_analysis: - balloon["anyimage_balloon_contour_analysis"] = True - if self.depth_analysis: - balloon["anyimage_balloon_depth_model"] = self.depth_model - push_undo("Extract Balloon") - return "AI-analyzed Balloon is ready" - - def cleanup(self): - cleanup_temporary_input(self.input_path, self.delete_input) - - -class ActivateBalloonTool(bpy.types.Operator): - bl_idname = "anyimage.activate_balloon_tool" - bl_label = "Balloon Lasso Tool" - bl_description = "Activate the Balloon Lasso tool in the 3D Viewport" - bl_options = {"INTERNAL"} - - @classmethod - def poll(cls, context): - return ( - context.area is not None - and context.area.type == "VIEW_3D" - and is_image_empty(context.active_object) - ) - - def execute(self, context): - return bpy.ops.wm.tool_set_by_id( - name=BalloonTool.bl_idname, - space_type="VIEW_3D", - ) - - -class BalloonTool(bpy.types.WorkSpaceTool): - bl_space_type = "VIEW_3D" - bl_context_mode = "OBJECT" - bl_idname = "anyimage.balloon_lasso" - bl_label = "Balloon Lasso" - bl_description = ( - "Draw an Image Empty region and extract one independently undoable " - "Balloon per stroke" - ) - bl_icon = "ops.mesh.polybuild_hover" - bl_cursor = "PAINT_BRUSH" - bl_operator = ExtractBalloon.bl_idname - bl_keymap = ( - ( - ExtractBalloon.bl_idname, - {"type": "LEFTMOUSE", "value": "PRESS"}, - None, - ), - ( - "wm.radial_control", - {"type": "S", "value": "PRESS"}, - { - "properties": [ - ( - "data_path_primary", - "scene.anyimage_settings.balloon_thickness", - ), - ( - "zoom_path", - "scene.anyimage_settings.balloon_thickness_radial_zoom", - ), - ] - }, - ), - ( - "wm.radial_control", - {"type": "D", "value": "PRESS"}, - { - "properties": [ - ( - "data_path_primary", - "scene.anyimage_settings.balloon_resolution", - ), - ( - "zoom_path", - "scene.anyimage_settings.balloon_resolution_radial_zoom", - ), - ] - }, - ), - ) - - def draw_settings(context, layout, tool): - settings = context.scene.anyimage_settings - layout.prop(settings, "balloon_mode", text="") - layout.prop(settings, "balloon_thickness") - layout.prop(settings, "balloon_resolution", text="Resolution (m)") - layout.prop(settings, "balloon_reuse_material") - status = ai_status() - ready = status["ready"] - contour_row = layout.row() - contour_row.enabled = ready - contour_row.prop(settings, "balloon_contour_analysis") - depth_row = layout.row() - depth_row.enabled = ready - depth_row.prop(settings, "balloon_depth_analysis") - if settings.balloon_depth_analysis: - model_row = layout.row() - model_row.enabled = ready - model_row.prop(settings, "balloon_depth_model", text="Depth Model") - if not ready: - layout.operator( - "anyimage.setup_ai_environment", - text=ai_setup_label(status), - icon="IMPORT", - ) diff --git a/src/anyimage/operators/cutout.py b/src/anyimage/operators/cutout.py index 49cd935..dcc5569 100644 --- a/src/anyimage/operators/cutout.py +++ b/src/anyimage/operators/cutout.py @@ -3,9 +3,9 @@ import json import bpy from ..properties import ( - image_edit_refine_selection_property as refine_selection_property, + refine_selection_property, ) -from .balloon.operator import hidden_string_property, serialize_matrix +from .image_edit.interaction import hidden_string_property, serialize_matrix from .image import is_image_empty, video_total_frames from .image_edit.common import ( ImageEditWarning, @@ -78,7 +78,7 @@ def _open_shape_pie( _pie_button( pie, CUTOUT, - "Cutout", + "Surface", "MESH_GRID", source_name, selection_data, @@ -253,4 +253,4 @@ class CutoutTool(bpy.types.WorkSpaceTool): settings = context.scene.anyimage_settings layout.prop(settings, "cutout_point_spacing") layout.prop(settings, "cutout_normal_analysis") - draw_image_mask_settings(context, layout) + draw_image_mask_settings(context, layout, "cutout_refine_selection") diff --git a/src/anyimage/operators/cutout_depth.py b/src/anyimage/operators/cutout_depth.py new file mode 100644 index 0000000..e3848ae --- /dev/null +++ b/src/anyimage/operators/cutout_depth.py @@ -0,0 +1,226 @@ +"""MoGe-2 depth sampling and Depth Balloon base-plane controls.""" + +from pathlib import Path + +import bpy + + +PLANE_VARIANCE_FLOOR_RATIO = 0.025 + + +def load_depth_balloon_data(path): + import numpy as np + + try: + with np.load(Path(path), allow_pickle=False) as archive: + required = { + "depth_signal", + "baseline_vector", + "valid", + "origin", + "reference_depth", + "crop_size", + "source_size", + } + missing = required.difference(archive.files) + if missing: + raise ValueError( + f"Depth Balloon data is missing: {', '.join(sorted(missing))}" + ) + data = { + "depth_signal": np.asarray( + archive["depth_signal"], + dtype=np.float32, + ), + "baseline_vector": np.asarray( + archive["baseline_vector"], + dtype=np.float32, + ), + "valid": np.asarray(archive["valid"], dtype=bool), + "origin": tuple(int(value) for value in archive["origin"]), + "reference_depth": float(archive["reference_depth"]), + "crop_size": tuple(int(value) for value in archive["crop_size"]), + "source_size": tuple(int(value) for value in archive["source_size"]), + } + except OSError as error: + raise ValueError(f"Unable to read Depth Balloon data: {path}") from error + if data["depth_signal"].ndim != 2: + raise ValueError("Depth Balloon signal must be two-dimensional") + if data["valid"].shape != data["depth_signal"].shape: + raise ValueError("Depth Balloon mask dimensions do not match") + if data["baseline_vector"].shape != (3,): + raise ValueError("Depth Balloon base plane vector must have three components") + if any(value <= 0 for value in (*data["crop_size"], *data["source_size"])): + raise ValueError("Depth Balloon image dimensions must be positive") + return data + + +def _sample_valid_bilinear(values, valid, x, y): + import numpy as np + + height, width = values.shape + x = np.clip(x, 0.0, max(width - 1, 0)) + y = np.clip(y, 0.0, max(height - 1, 0)) + x0 = np.floor(x).astype(np.int64) + y0 = np.floor(y).astype(np.int64) + x1 = np.minimum(x0 + 1, width - 1) + y1 = np.minimum(y0 + 1, height - 1) + fx = x - x0 + fy = y - y0 + samples = np.stack( + ( + values[y0, x0], + values[y0, x1], + values[y1, x0], + values[y1, x1], + ), + axis=1, + ) + weights = np.stack( + ( + (1.0 - fx) * (1.0 - fy), + fx * (1.0 - fy), + (1.0 - fx) * fy, + fx * fy, + ), + axis=1, + ) + sample_valid = np.stack( + ( + valid[y0, x0], + valid[y0, x1], + valid[y1, x0], + valid[y1, x1], + ), + axis=1, + ) + weights *= sample_valid + totals = weights.sum(axis=1) + result = np.zeros(len(x), dtype=np.float32) + usable = totals > 1e-8 + result[usable] = (samples[usable] * weights[usable]).sum(axis=1) / totals[usable] + return result, usable + + +def _stable_plane(coordinates, values): + import numpy as np + + coordinates = np.asarray(coordinates, dtype=np.float64) + values = np.asarray(values, dtype=np.float64) + if len(coordinates) < 2: + return np.zeros(3, dtype=np.float32) + center = coordinates.mean(axis=0) + value_center = float(values.mean()) + centered = coordinates - center + covariance = centered.T @ centered / len(centered) + cross = centered.T @ (values - value_center) / len(centered) + eigenvalues, eigenvectors = np.linalg.eigh(covariance) + largest = max(float(eigenvalues[-1]), 0.0) + if largest <= 1e-12: + slopes = np.zeros(2, dtype=np.float64) + else: + supported = np.maximum(eigenvalues, largest * PLANE_VARIANCE_FLOOR_RATIO) + slopes = eigenvectors @ ((eigenvectors.T @ cross) / supported) + intercept = value_center - float(center @ slopes) + return np.asarray((*slopes, intercept), dtype=np.float32) + + +def depth_balloon_fields(vertices, vertex_uv, depth_data, depth_scale): + """Sample camera depth and fit its base plane at a fixed local scale.""" + import numpy as np + + vertices = np.asarray(vertices, dtype=np.float32) + uv = np.asarray(vertex_uv, dtype=np.float32) + if vertices.ndim != 2 or vertices.shape[1] != 3: + raise ValueError("Depth Balloon vertices must have three components") + if uv.shape != (len(vertices), 2): + raise ValueError("Depth Balloon UV coordinates do not match its vertices") + depth_signal = np.asarray(depth_data["depth_signal"], dtype=np.float32) + valid = np.asarray(depth_data["valid"], dtype=bool) + crop_width, crop_height = depth_data["crop_size"] + depth_height, depth_width = depth_signal.shape + + crop_x = uv[:, 0] * max(crop_width - 1, 0) + crop_y = (1.0 - uv[:, 1]) * max(crop_height - 1, 0) + sample_x = crop_x * max(depth_width - 1, 0) / max(crop_width - 1, 1) + sample_y = crop_y * max(depth_height - 1, 0) / max(crop_height - 1, 1) + sampled, usable = _sample_valid_bilinear( + depth_signal, + valid, + sample_x, + sample_y, + ) + inside_crop = ( + (crop_x >= 0.0) + & (crop_x <= max(crop_width - 1, 0)) + & (crop_y >= 0.0) + & (crop_y <= max(crop_height - 1, 0)) + ) + usable &= inside_crop + sampled[~usable] = 0.0 + coordinates = np.stack( + ( + sample_x / max(depth_width - 1, 1) * 2.0 - 1.0, + sample_y / max(depth_height - 1, 1) * 2.0 - 1.0, + np.ones(len(vertices), dtype=np.float32), + ), + axis=1, + ).astype(np.float32) + + depth_scale = float(depth_scale) + if not depth_scale > 0.0: + raise ValueError("Depth Balloon reference scale must be positive") + normalized_plane = np.asarray( + depth_data["baseline_vector"], + dtype=np.float32, + ) + baseline_values = ( + coordinates @ normalized_plane * depth_scale * 0.5 + ) + local_coordinates = vertices[:, :2] + if np.count_nonzero(usable) >= 3: + local_plane = _stable_plane( + local_coordinates[usable], + baseline_values[usable], + ) + else: + local_plane = np.zeros(3, dtype=np.float32) + depth_values = sampled * depth_scale + if usable.any(): + plane_without_offset = local_coordinates @ local_plane[:2] + residual = depth_values - plane_without_offset + depth_values[usable] -= float(np.median(residual[usable])) + return depth_values, local_plane + + +def orient_depth_balloon_plane(local_plane): + slope_x, slope_y, offset = (float(value) for value in local_plane) + return -slope_x, slope_y, offset + + +def create_depth_balloon_base_plane(context, mesh_object, vertices, local_plane): + import numpy as np + from mathutils import Vector + + vertices = np.asarray(vertices, dtype=np.float32) + slope_y, slope_z, offset = (float(value) for value in local_plane) + extent = vertices[:, 1:].max(axis=0) - vertices[:, 1:].min(axis=0) + center_y = 0.0 + center_z = 0.0 + center_x = offset + normal = Vector((-1.0, slope_y, slope_z)).normalized() + + controller = bpy.data.objects.new(f"{mesh_object.name} Base Plane", None) + context.collection.objects.link(controller) + controller.parent = mesh_object + controller.location = (float(center_x), float(center_y), float(center_z)) + controller.rotation_euler = normal.to_track_quat("Z", "Y").to_euler() + controller.empty_display_type = "SINGLE_ARROW" + controller.empty_display_size = max(float(extent.max()) * 0.15, 0.01) + controller.show_in_front = True + controller.hide_render = True + controller["anyimage_source_type"] = "DEPTH_BALLOON_BASE_PLANE" + controller["anyimage_base_plane"] = (slope_y, slope_z, offset) + return controller + + diff --git a/src/anyimage/operators/balloon/geometry.py b/src/anyimage/operators/cutout_geometry.py similarity index 80% rename from src/anyimage/operators/balloon/geometry.py rename to src/anyimage/operators/cutout_geometry.py index c093a0b..6065d89 100644 --- a/src/anyimage/operators/balloon/geometry.py +++ b/src/anyimage/operators/cutout_geometry.py @@ -1,11 +1,6 @@ import math -from .poisson import poisson_balloon_heights -from .teddy import teddy_balloon_heights - - POINT_SPACING = 0.1 -INFLATION = 0.45 def polygon_area(points): @@ -31,12 +26,12 @@ def resample_closed_polygon(polygon, spacing): lengths = np.linalg.norm(segments, axis=1) points = points[lengths > 1e-8] if len(points) < 3: - raise ValueError("The lasso region is too small to extract a Balloon") + raise ValueError("The lasso region is too small to create a Cutout") segments = np.roll(points, -1, axis=0) - points lengths = np.linalg.norm(segments, axis=1) perimeter = float(lengths.sum()) if perimeter <= spacing * 3.0: - raise ValueError("The lasso region is too small to extract a Balloon") + raise ValueError("The lasso region is too small to create a Cutout") count = max(3, int(round(perimeter / spacing))) distances = np.arange(count, dtype=np.float64) * perimeter / count cumulative = np.concatenate(([0.0], np.cumsum(lengths))) @@ -245,7 +240,6 @@ def uniform_interior_points(contour, spacing): return uniform_interior_points_for_contours((contour,), spacing) - def metric_transform(world_metric=None): import numpy as np @@ -274,7 +268,7 @@ def prepare_planar_contours( import numpy as np if spacing <= 0.0: - raise ValueError("Balloon point spacing must be greater than zero") + raise ValueError("Cutout point spacing must be greater than zero") image_width, image_height = image_size x_min, x_max, y_min, y_max = local_bounds local_width = x_max - x_min @@ -330,7 +324,6 @@ def prepare_planar_regions( return prepared, spacing - def prepare_planar_points( polygon, image_size, @@ -416,12 +409,13 @@ def boundary_vertices_from_faces(vertex_count, faces): return boundary -def triangulate_planar_regions(regions, spacing): +def triangulate_planar_regions(regions, spacing, return_components=False): import numpy as np points = [] faces = [] boundaries = [] + components = [] offset = 0 for contours, interior in regions: region_points, region_faces, region_boundary = triangulate_planar_contours( @@ -429,6 +423,14 @@ def triangulate_planar_regions(regions, spacing): interior, spacing, ) + components.append( + ( + offset, + offset + len(region_points), + region_faces, + contours[0], + ) + ) points.append(region_points) faces.extend( tuple(offset + value for value in face) for face in region_faces @@ -437,122 +439,11 @@ def triangulate_planar_regions(regions, spacing): offset += len(region_points) if not points: raise ValueError("Unable to triangulate the Cutout regions") - return ( + result = ( np.concatenate(points, axis=0), faces, np.concatenate(boundaries), ) + return (*result, components) if return_components else result -def balloon_heights( - algorithm, - planar_points, - planar_faces, - contour, - boundary_mask, - inflation, -): - if algorithm == "POISSON": - return poisson_balloon_heights( - planar_points, - planar_faces, - boundary_mask, - inflation, - ) - if algorithm == "TEDDY": - return teddy_balloon_heights( - planar_points, - planar_faces, - contour, - boundary_mask, - inflation, - ) - raise ValueError(f"Unknown Balloon algorithm: {algorithm}") - -def build_closed_geometry( - planar_points, - planar_faces, - boundary_mask, - heights, - local_bounds, - world_metric=None, - world_z_scale=1.0, -): - import numpy as np - - points = np.asarray(planar_points, dtype=np.float64) - boundary_mask = np.asarray(boundary_mask, dtype=bool) - heights = np.asarray(heights, dtype=np.float64) - local_points = points @ np.linalg.inv(metric_transform(world_metric)) - if world_z_scale <= 1e-10: - raise ValueError("The Image Empty has a degenerate world transform") - local_heights = heights / world_z_scale - x_min, x_max, y_min, y_max = local_bounds - width = x_max - x_min - height = y_max - y_min - vertices = [] - vertex_uv = [] - front_indices = [] - back_indices = [] - for index, point in enumerate(local_points): - u = (point[0] - x_min) / width - v = (point[1] - y_min) / height - front_indices.append(len(vertices)) - vertices.append((point[0], point[1], local_heights[index])) - vertex_uv.append((u, v)) - if boundary_mask[index]: - back_indices.append(front_indices[-1]) - else: - back_indices.append(len(vertices)) - vertices.append((point[0], point[1], -local_heights[index])) - vertex_uv.append((u, v)) - faces = [] - for triangle in planar_faces: - a, b, c = (points[index] for index in triangle) - signed_area = (b[0] - a[0]) * (c[1] - a[1]) - ( - b[1] - a[1] - ) * (c[0] - a[0]) - ordered = triangle if signed_area > 0.0 else triangle[::-1] - faces.append(tuple(front_indices[index] for index in ordered)) - faces.append( - tuple(back_indices[index] for index in reversed(ordered)) - ) - return vertices, faces, np.asarray(vertex_uv, dtype=np.float32) - -def triangulate_balloon( - regions, - image_size, - local_bounds, - spacing=POINT_SPACING, - world_metric=None, - world_z_scale=1.0, - inflation=INFLATION, - algorithm="POISSON", -): - planar_regions, spacing = prepare_planar_regions( - regions, - image_size, - local_bounds, - spacing=spacing, - world_metric=world_metric, - ) - planar_points, planar_faces, boundary_mask = triangulate_planar_regions( - planar_regions, spacing - ) - heights = balloon_heights( - algorithm, - planar_points, - planar_faces, - planar_regions[0][0][0], - boundary_mask, - inflation, - ) - return build_closed_geometry( - planar_points, - planar_faces, - boundary_mask, - heights, - local_bounds, - world_metric=world_metric, - world_z_scale=world_z_scale, - ) diff --git a/src/anyimage/operators/cutout_nodes.py b/src/anyimage/operators/cutout_nodes.py new file mode 100644 index 0000000..231f327 --- /dev/null +++ b/src/anyimage/operators/cutout_nodes.py @@ -0,0 +1,863 @@ +import math + +import bpy + + +CUTOUT = "CUTOUT" +BALLOON = "BALLOON" +DEPTH_BALLOON = "DEPTH_BALLOON" +DEPTH_SURFACE = "DEPTH_SURFACE" +BALLOON_ATTRIBUTE = "o_balloon" +NODE_GROUP_VERSION = 18 +GROUP_NAMES = { + CUTOUT: "O Image Surface", + BALLOON: "O Image Balloon", + DEPTH_BALLOON: "O Image Depth Balloon", + DEPTH_SURFACE: "O Image Depth Surface", +} + + +def _socket(group, name, in_out, socket_type, parent=None): + socket = group.interface.new_socket( + name=name, + in_out=in_out, + socket_type=socket_type, + parent=parent, + ) + if in_out == "INPUT" and hasattr(socket, "structure_type"): + socket.structure_type = "SINGLE" + return socket + + +def _group_input(nodes, location, visible_names): + node = nodes.new("NodeGroupInput") + node.location = location + visible = set(visible_names) + for output in node.outputs: + output.hide = output.name not in visible + return node + + +def _localize_group_inputs(group, controls): + """Place short, socket-specific Group Inputs beside their consumers.""" + links = group.links + outgoing = [link for link in links if link.from_node == controls] + by_socket = {} + for link in outgoing: + if link.from_socket.name == "Geometry": + continue + by_socket.setdefault(link.from_socket.name, []).append(link) + + for socket_name, socket_links in by_socket.items(): + ordered = sorted(socket_links, key=lambda link: link.to_node.location.x) + clusters = [] + for link in ordered: + if ( + not clusters + or link.to_node.location.x + - clusters[-1][-1].to_node.location.x + > 900.0 + ): + clusters.append([]) + clusters[-1].append(link) + for cluster in clusters: + targets = [link.to_node for link in cluster] + location = [ + min(node.location.x for node in targets) - 260.0, + max(node.location.y for node in targets) + 320.0, + ] + while any( + abs(node.location.x - location[0]) < 190.0 + and abs(node.location.y - location[1]) < 190.0 + for node in group.nodes + ): + location[1] += 220.0 + local = _group_input(group.nodes, location, {socket_name}) + for link in cluster: + target = link.to_socket + links.remove(link) + links.new(local.outputs[socket_name], target) + for output in controls.outputs: + output.hide = output.name != "Geometry" + + +def _float_input( + group, + name, + default, + minimum=0.0, + maximum=None, + subtype=None, + parent=None, +): + socket = _socket(group, name, "INPUT", "NodeSocketFloat", parent) + socket.default_value = default + socket.min_value = minimum + if maximum is not None: + socket.max_value = maximum + if subtype is not None: + socket.subtype = subtype + return socket + + +def _bool_input(group, name, default=False, parent=None): + socket = _socket(group, name, "INPUT", "NodeSocketBool", parent) + socket.default_value = default + return socket + + +def _math(nodes, operation, location): + node = nodes.new("ShaderNodeMath") + node.operation = operation + node.location = location + return node + + +def _island_cleanup(group, geometry, cleanup, location): + nodes = group.nodes + links = group.links + x, y = location + mesh_island = nodes.new("GeometryNodeInputMeshIsland") + mesh_island.location = (x, y + 250.0) + face_area = nodes.new("GeometryNodeInputMeshFaceArea") + face_area.location = (x, y + 100.0) + island_area = nodes.new("GeometryNodeAccumulateField") + island_area.data_type = "FLOAT" + island_area.domain = "FACE" + island_area.location = (x + 220.0, y + 200.0) + links.new(face_area.outputs["Area"], island_area.inputs["Value"]) + links.new(mesh_island.outputs["Island Index"], island_area.inputs["Group ID"]) + + statistics = nodes.new("GeometryNodeAttributeStatistic") + statistics.data_type = "FLOAT" + statistics.domain = "FACE" + statistics.location = (x + 220.0, y - 100.0) + links.new(geometry, statistics.inputs["Geometry"]) + links.new(face_area.outputs["Area"], statistics.inputs["Attribute"]) + reference_scale = _math(nodes, "MULTIPLY", (x + 450.0, y - 50.0)) + reference_scale.inputs[1].default_value = 4.0 / math.sqrt(3.0) + links.new(statistics.outputs["Median"], reference_scale.inputs[0]) + threshold = _math(nodes, "MULTIPLY", (x + 650.0, y - 50.0)) + links.new(reference_scale.outputs[0], threshold.inputs[0]) + links.new(cleanup, threshold.inputs[1]) + tiny = _math(nodes, "LESS_THAN", (x + 650.0, y + 200.0)) + links.new(island_area.outputs["Total"], tiny.inputs[0]) + links.new(threshold.outputs[0], tiny.inputs[1]) + delete = nodes.new("GeometryNodeDeleteGeometry") + delete.domain = "FACE" + delete.location = (x + 900.0, y) + links.new(geometry, delete.inputs["Geometry"]) + links.new(tiny.outputs[0], delete.inputs["Selection"]) + return delete.outputs["Geometry"] + + +def _profile_field(group, geometry, location): + nodes = group.nodes + links = group.links + x, y = location + profile = nodes.new("GeometryNodeInputNamedAttribute") + profile.data_type = "FLOAT" + profile.inputs["Name"].default_value = BALLOON_ATTRIBUTE + profile.location = (x, y + 250.0) + profile_max = nodes.new("GeometryNodeAttributeStatistic") + profile_max.data_type = "FLOAT" + profile_max.domain = "POINT" + profile_max.location = (x + 220.0, y) + links.new(geometry, profile_max.inputs["Geometry"]) + links.new(profile.outputs["Attribute"], profile_max.inputs["Attribute"]) + safe_max = _math(nodes, "MAXIMUM", (x + 450.0, y)) + safe_max.inputs[1].default_value = 1e-6 + links.new(profile_max.outputs["Max"], safe_max.inputs[0]) + normalized = _math(nodes, "DIVIDE", (x + 450.0, y + 250.0)) + links.new(profile.outputs["Attribute"], normalized.inputs[0]) + links.new(safe_max.outputs[0], normalized.inputs[1]) + smooth = nodes.new("ShaderNodeMapRange") + smooth.data_type = "FLOAT" + smooth.interpolation_type = "SMOOTHERSTEP" + smooth.clamp = True + smooth.inputs["From Min"].default_value = 0.0 + smooth.inputs["From Max"].default_value = 1.0 + smooth.inputs["To Min"].default_value = 0.0 + smooth.inputs["To Max"].default_value = 1.0 + smooth.location = (x + 650.0, y + 250.0) + links.new(normalized.outputs[0], smooth.inputs["Value"]) + return profile.outputs["Attribute"], smooth.outputs["Result"] + + +def _extrude_layers(group, geometry, location): + node = group.nodes.new("GeometryNodeExtrudeMesh") + node.mode = "FACES" + node.inputs["Individual"].default_value = False + node.inputs["Offset"].default_value = (1.0, 0.0, 0.0) + # Keep distinct front and back vertices before overriding their X positions. + node.inputs["Offset Scale"].default_value = 1.0 + node.location = location + group.links.new(geometry, node.inputs["Mesh"]) + return node + + +def _offset_geometry_x(group, geometry, x_value, location): + nodes = group.nodes + links = group.links + offset = nodes.new("ShaderNodeCombineXYZ") + offset.location = location + links.new(x_value, offset.inputs["X"]) + set_position = nodes.new("GeometryNodeSetPosition") + set_position.location = (location[0] + 200.0, location[1]) + links.new(geometry, set_position.inputs["Geometry"]) + links.new(offset.outputs["Vector"], set_position.inputs["Offset"]) + return set_position.outputs["Geometry"] + + +def _set_layer_x(group, geometry, extrude, front_x, back_x, location): + nodes = group.nodes + links = group.links + choose_x = nodes.new("GeometryNodeSwitch") + choose_x.input_type = "FLOAT" + choose_x.location = (location[0], location[1] - 200.0) + links.new(extrude.outputs["Top"], choose_x.inputs["Switch"]) + links.new(front_x, choose_x.inputs["False"]) + links.new(back_x, choose_x.inputs["True"]) + position = nodes.new("GeometryNodeInputPosition") + position.location = (location[0], location[1] + 150.0) + separate = nodes.new("ShaderNodeSeparateXYZ") + separate.location = (location[0] + 200.0, location[1] + 150.0) + links.new(position.outputs["Position"], separate.inputs["Vector"]) + combine = nodes.new("ShaderNodeCombineXYZ") + combine.location = (location[0] + 400.0, location[1]) + links.new(choose_x.outputs["Output"], combine.inputs["X"]) + links.new(separate.outputs["Y"], combine.inputs["Y"]) + links.new(separate.outputs["Z"], combine.inputs["Z"]) + set_position = nodes.new("GeometryNodeSetPosition") + set_position.location = (location[0] + 600.0, location[1]) + links.new(extrude.outputs["Mesh"], set_position.inputs["Geometry"]) + links.new(combine.outputs["Vector"], set_position.inputs["Position"]) + + side = nodes.new("GeometryNodeSeparateGeometry") + side.domain = "FACE" + side.location = (location[0] + 850.0, location[1] + 200.0) + links.new(set_position.outputs["Geometry"], side.inputs["Geometry"]) + links.new(extrude.outputs["Side"], side.inputs["Selection"]) + flip_side = nodes.new("GeometryNodeFlipFaces") + flip_side.location = (location[0] + 850.0, location[1] + 400.0) + links.new(side.outputs["Selection"], flip_side.inputs["Mesh"]) + front = _offset_geometry_x( + group, geometry, front_x, (location[0], location[1] - 450.0) + ) + back = _offset_geometry_x( + group, geometry, back_x, (location[0], location[1] - 700.0) + ) + flip_back = nodes.new("GeometryNodeFlipFaces") + flip_back.location = (location[0] + 500.0, location[1] - 700.0) + links.new(back, flip_back.inputs["Mesh"]) + + join = nodes.new("GeometryNodeJoinGeometry") + join.location = (location[0] + 1100.0, location[1]) + links.new(front, join.inputs["Geometry"]) + links.new(flip_back.outputs["Mesh"], join.inputs["Geometry"]) + links.new(flip_side.outputs["Mesh"], join.inputs["Geometry"]) + merge = nodes.new("GeometryNodeMergeByDistance") + merge.mode = "ALL" + merge.inputs["Distance"].default_value = 1e-6 + merge.location = (location[0] + 1300.0, location[1]) + links.new(join.outputs["Geometry"], merge.inputs["Geometry"]) + return merge.outputs["Geometry"] + + +def _set_segmented_layer_x( + group, + geometry, + front_x, + back_x, + thickness, + location, +): + nodes = group.nodes + links = group.links + x, y = location + + edge_vertices = nodes.new("GeometryNodeInputMeshEdgeVertices") + edge_vertices.location = (x, y + 850.0) + edge_length = nodes.new("ShaderNodeVectorMath") + edge_length.operation = "DISTANCE" + edge_length.location = (x + 220.0, y + 850.0) + links.new(edge_vertices.outputs["Position 1"], edge_length.inputs[0]) + links.new(edge_vertices.outputs["Position 2"], edge_length.inputs[1]) + edge_neighbors = nodes.new("GeometryNodeInputMeshEdgeNeighbors") + edge_neighbors.location = (x, y + 650.0) + boundary = _math(nodes, "COMPARE", (x + 220.0, y + 650.0)) + boundary.inputs[1].default_value = 1.0 + boundary.inputs[2].default_value = 0.1 + links.new(edge_neighbors.outputs["Face Count"], boundary.inputs[0]) + statistics = nodes.new("GeometryNodeAttributeStatistic") + statistics.data_type = "FLOAT" + statistics.domain = "EDGE" + statistics.location = (x + 450.0, y + 750.0) + links.new(geometry, statistics.inputs["Geometry"]) + links.new(boundary.outputs[0], statistics.inputs["Selection"]) + links.new(edge_length.outputs["Value"], statistics.inputs["Attribute"]) + safe_edge = _math(nodes, "MAXIMUM", (x + 680.0, y + 750.0)) + safe_edge.inputs[1].default_value = 1e-6 + links.new(statistics.outputs["Median"], safe_edge.inputs[0]) + segment_ratio = _math(nodes, "DIVIDE", (x + 900.0, y + 750.0)) + links.new(thickness, segment_ratio.inputs[0]) + links.new(safe_edge.outputs[0], segment_ratio.inputs[1]) + segment_count = nodes.new("FunctionNodeFloatToInt") + segment_count.rounding_mode = "CEILING" + segment_count.location = (x + 1120.0, y + 750.0) + links.new(segment_ratio.outputs[0], segment_count.inputs["Float"]) + at_least_one = nodes.new("FunctionNodeIntegerMath") + at_least_one.operation = "MAXIMUM" + at_least_one.inputs[1].default_value = 1 + at_least_one.location = (x + 1340.0, y + 750.0) + links.new(segment_count.outputs["Integer"], at_least_one.inputs[0]) + segment_limit = nodes.new("FunctionNodeIntegerMath") + segment_limit.operation = "MINIMUM" + segment_limit.inputs[1].default_value = 128 + segment_limit.location = (x + 1560.0, y + 750.0) + links.new(at_least_one.outputs[0], segment_limit.inputs[0]) + step = _math(nodes, "DIVIDE", (x + 1560.0, y + 500.0)) + links.new(thickness, step.inputs[0]) + links.new(segment_limit.outputs[0], step.inputs[1]) + + front = _offset_geometry_x(group, geometry, front_x, (x, y + 100.0)) + repeat_output = nodes.new("GeometryNodeRepeatOutput") + repeat_output.location = (x + 2380.0, y + 100.0) + repeat_input = nodes.new("GeometryNodeRepeatInput") + repeat_input.pair_with_output(repeat_output) + repeat_output.repeat_items.new("GEOMETRY", "Sides") + repeat_input.location = (x + 1560.0, y + 100.0) + links.new(segment_limit.outputs[0], repeat_input.inputs["Iterations"]) + links.new(geometry, repeat_input.inputs["Geometry"]) + negative_step = _math(nodes, "MULTIPLY", (x + 1760.0, y - 100.0)) + negative_step.inputs[1].default_value = -1.0 + links.new(step.outputs[0], negative_step.inputs[0]) + extrude = nodes.new("GeometryNodeExtrudeMesh") + extrude.mode = "FACES" + extrude.inputs["Individual"].default_value = False + extrude.inputs["Selection"].default_value = True + extrude.inputs["Offset"].default_value = (0.0, 0.0, 0.0) + extrude.location = (x + 1960.0, y + 100.0) + links.new(negative_step.outputs[0], extrude.inputs["Offset Scale"]) + links.new(repeat_input.outputs["Geometry"], extrude.inputs["Mesh"]) + split_extrude = nodes.new("GeometryNodeSeparateGeometry") + split_extrude.domain = "FACE" + split_extrude.location = (x + 2160.0, y + 100.0) + links.new(extrude.outputs["Mesh"], split_extrude.inputs["Geometry"]) + links.new(extrude.outputs["Side"], split_extrude.inputs["Selection"]) + join_sides = nodes.new("GeometryNodeJoinGeometry") + join_sides.location = (x + 2160.0, y - 150.0) + links.new(repeat_input.outputs["Sides"], join_sides.inputs["Geometry"]) + links.new(split_extrude.outputs["Selection"], join_sides.inputs["Geometry"]) + links.new(split_extrude.outputs["Inverted"], repeat_output.inputs["Geometry"]) + links.new(join_sides.outputs["Geometry"], repeat_output.inputs["Sides"]) + + positioned_side = _offset_geometry_x( + group, + repeat_output.outputs["Sides"], + front_x, + (x + 2450.0, y + 350.0), + ) + flip_side = nodes.new("GeometryNodeFlipFaces") + flip_side.location = (x + 2850.0, y + 350.0) + links.new(positioned_side, flip_side.inputs["Mesh"]) + + back = _offset_geometry_x( + group, geometry, back_x, (x + 2450.0, y - 400.0) + ) + flip_back = nodes.new("GeometryNodeFlipFaces") + flip_back.location = (x + 2850.0, y - 400.0) + links.new(back, flip_back.inputs["Mesh"]) + join = nodes.new("GeometryNodeJoinGeometry") + join.location = (x + 3050.0, y + 50.0) + links.new(front, join.inputs["Geometry"]) + links.new(flip_back.outputs["Mesh"], join.inputs["Geometry"]) + links.new(flip_side.outputs["Mesh"], join.inputs["Geometry"]) + merge = nodes.new("GeometryNodeMergeByDistance") + merge.mode = "ALL" + merge.inputs["Distance"].default_value = 1e-6 + merge.location = (x + 3250.0, y + 50.0) + links.new(join.outputs["Geometry"], merge.inputs["Geometry"]) + return merge.outputs["Geometry"] + + +def _set_geometry_x(group, geometry, x_value, location): + nodes = group.nodes + links = group.links + position = nodes.new("GeometryNodeInputPosition") + position.location = location + separate = nodes.new("ShaderNodeSeparateXYZ") + separate.location = (location[0] + 200.0, location[1]) + links.new(position.outputs["Position"], separate.inputs["Vector"]) + combine = nodes.new("ShaderNodeCombineXYZ") + combine.location = (location[0] + 400.0, location[1]) + links.new(x_value, combine.inputs["X"]) + links.new(separate.outputs["Y"], combine.inputs["Y"]) + links.new(separate.outputs["Z"], combine.inputs["Z"]) + set_position = nodes.new("GeometryNodeSetPosition") + set_position.location = (location[0] + 600.0, location[1]) + links.new(geometry, set_position.inputs["Geometry"]) + links.new(combine.outputs["Vector"], set_position.inputs["Position"]) + return set_position.outputs["Geometry"] + + +def _remove_zero_thickness( + group, + zero_geometry, + shaped_geometry, + thickness, + location, +): + nodes = group.nodes + links = group.links + zero = _math(nodes, "LESS_THAN", (location[0], location[1] - 200.0)) + zero.inputs[1].default_value = 1e-6 + links.new(thickness, zero.inputs[0]) + choose = nodes.new("GeometryNodeSwitch") + choose.input_type = "GEOMETRY" + choose.location = location + links.new(zero.outputs[0], choose.inputs["Switch"]) + links.new(shaped_geometry, choose.inputs["False"]) + links.new(zero_geometry, choose.inputs["True"]) + return choose.outputs["Output"] + + +def _image_fields(group, depth_image, location): + nodes = group.nodes + links = group.links + uv = nodes.new("GeometryNodeInputNamedAttribute") + uv.data_type = "FLOAT_VECTOR" + uv.inputs["Name"].default_value = "UVMap" + uv.location = location + texture = nodes.new("GeometryNodeImageTexture") + texture.interpolation = "Linear" + texture.extension = "EXTEND" + texture.location = (location[0] + 220.0, location[1]) + links.new(depth_image, texture.inputs["Image"]) + links.new(uv.outputs["Attribute"], texture.inputs["Vector"]) + separate = nodes.new("ShaderNodeSeparateXYZ") + separate.location = (location[0] + 450.0, location[1]) + links.new(texture.outputs["Color"], separate.inputs["Vector"]) + return separate.outputs["X"], separate.outputs["Y"], texture.outputs["Alpha"] + + +def _plane_x(group, base_plane, location): + nodes = group.nodes + links = group.links + object_info = nodes.new("GeometryNodeObjectInfo") + object_info.transform_space = "RELATIVE" + object_info.location = location + links.new(base_plane, object_info.inputs["Object"]) + plane_normal = nodes.new("FunctionNodeRotateVector") + plane_normal.inputs["Vector"].default_value = (0.0, 0.0, 1.0) + plane_normal.location = (location[0] + 220.0, location[1]) + links.new(object_info.outputs["Rotation"], plane_normal.inputs["Rotation"]) + position = nodes.new("GeometryNodeInputPosition") + position.location = (location[0], location[1] + 250.0) + separate = nodes.new("ShaderNodeSeparateXYZ") + separate.location = (location[0] + 220.0, location[1] + 250.0) + links.new(position.outputs["Position"], separate.inputs["Vector"]) + zero_x = nodes.new("ShaderNodeCombineXYZ") + zero_x.location = (location[0] + 440.0, location[1] + 250.0) + links.new(separate.outputs["Y"], zero_x.inputs["Y"]) + links.new(separate.outputs["Z"], zero_x.inputs["Z"]) + point_dot = nodes.new("ShaderNodeVectorMath") + point_dot.operation = "DOT_PRODUCT" + point_dot.location = (location[0] + 660.0, location[1] + 250.0) + links.new(zero_x.outputs["Vector"], point_dot.inputs[0]) + links.new(plane_normal.outputs["Vector"], point_dot.inputs[1]) + offset_dot = nodes.new("ShaderNodeVectorMath") + offset_dot.operation = "DOT_PRODUCT" + offset_dot.location = (location[0] + 660.0, location[1]) + links.new(object_info.outputs["Location"], offset_dot.inputs[0]) + links.new(plane_normal.outputs["Vector"], offset_dot.inputs[1]) + numerator = _math(nodes, "SUBTRACT", (location[0] + 880.0, location[1] + 150.0)) + links.new(offset_dot.outputs["Value"], numerator.inputs[0]) + links.new(point_dot.outputs["Value"], numerator.inputs[1]) + normal_x = nodes.new("ShaderNodeSeparateXYZ") + normal_x.location = (location[0] + 440.0, location[1] - 100.0) + links.new(plane_normal.outputs["Vector"], normal_x.inputs["Vector"]) + divide = _math(nodes, "DIVIDE", (location[0] + 1100.0, location[1] + 150.0)) + links.new(numerator.outputs[0], divide.inputs[0]) + links.new(normal_x.outputs["X"], divide.inputs[1]) + return divide.outputs[0] + + +def _shade_output(group, geometry, location): + shade = group.nodes.new("GeometryNodeSetShadeSmooth") + shade.domain = "FACE" + shade.inputs["Shade Smooth"].default_value = True + shade.location = location + output = group.nodes.new("NodeGroupOutput") + output.location = (location[0] + 250.0, location[1]) + group.links.new(geometry, shade.inputs["Geometry"]) + group.links.new(shade.outputs["Geometry"], output.inputs["Geometry"]) + + +def _build_cutout(group, geometry, thickness): + nodes = group.nodes + links = group.links + half = _math(nodes, "MULTIPLY", (-200.0, 50.0)) + half.inputs[1].default_value = 0.5 + links.new(thickness, half.inputs[0]) + front = _math(nodes, "MULTIPLY", (20.0, 0.0)) + front.inputs[1].default_value = -1.0 + links.new(half.outputs[0], front.inputs[0]) + shaped = _set_segmented_layer_x( + group, + geometry, + front.outputs[0], + half.outputs[0], + thickness, + (250.0, 300.0), + ) + return _remove_zero_thickness( + group, geometry, shaped, thickness, (3750.0, 300.0) + ) + + +def _build_balloon(group, geometry, thickness): + nodes = group.nodes + links = group.links + profile, _profile_factor = _profile_field( + group, geometry, (-200.0, 0.0) + ) + front = _math(nodes, "MULTIPLY", (2400.0, 500.0)) + front.inputs[1].default_value = -1.0 + links.new(profile, front.inputs[0]) + front_scale = _math(nodes, "MULTIPLY", (2600.0, 500.0)) + links.new(front.outputs[0], front_scale.inputs[0]) + links.new(thickness, front_scale.inputs[1]) + back = _math(nodes, "MULTIPLY", (2600.0, 300.0)) + links.new(profile, back.inputs[0]) + links.new(thickness, back.inputs[1]) + extrude = _extrude_layers(group, geometry, (2400.0, 750.0)) + shaped = _set_layer_x( + group, + geometry, + extrude, + front_scale.outputs[0], + back.outputs[0], + (2850.0, 650.0), + ) + return _remove_zero_thickness( + group, geometry, shaped, thickness, (4400.0, 650.0) + ) + + +def _build_depth_balloon( + group, + geometry, + thickness, + depth_scale, + double_sided, + base_plane, + depth_image, + depth_reference, +): + nodes = group.nodes + links = group.links + profile, profile_factor = _profile_field( + group, geometry, (-200.0, 250.0) + ) + sampled_depth, _auxiliary, _alpha = _image_fields( + group, depth_image, (200.0, -250.0) + ) + depth_distance = _math(nodes, "MULTIPLY", (850.0, -250.0)) + links.new(sampled_depth, depth_distance.inputs[0]) + links.new(depth_reference, depth_distance.inputs[1]) + calibrated = _math(nodes, "MULTIPLY", (1050.0, -250.0)) + calibrated.inputs[1].default_value = 0.5 + links.new(depth_distance.outputs[0], calibrated.inputs[0]) + depth_base = _math(nodes, "ADD", (1250.0, -200.0)) + links.new(calibrated.outputs[0], depth_base.inputs[0]) + links.new(depth_reference, depth_base.inputs[1]) + plane_x = _plane_x(group, base_plane, (200.0, -650.0)) + corrected_depth = _math(nodes, "SUBTRACT", (1450.0, -150.0)) + links.new(plane_x, corrected_depth.inputs[0]) + links.new(depth_base.outputs[0], corrected_depth.inputs[1]) + scaled_depth = _math(nodes, "MULTIPLY", (1650.0, -150.0)) + links.new(corrected_depth.outputs[0], scaled_depth.inputs[0]) + links.new(depth_scale, scaled_depth.inputs[1]) + + base_front = _math(nodes, "MULTIPLY", (2400.0, 500.0)) + links.new(profile, base_front.inputs[0]) + links.new(thickness, base_front.inputs[1]) + negate_front = _math(nodes, "MULTIPLY", (2600.0, 500.0)) + negate_front.inputs[1].default_value = -1.0 + links.new(base_front.outputs[0], negate_front.inputs[0]) + base_back = _math(nodes, "MULTIPLY", (2600.0, 300.0)) + links.new(profile, base_back.inputs[0]) + links.new(thickness, base_back.inputs[1]) + front_mix = nodes.new("ShaderNodeMix") + front_mix.data_type = "FLOAT" + front_mix.clamp_factor = True + front_mix.location = (2600.0, 50.0) + links.new(profile_factor, front_mix.inputs["Factor"]) + links.new(negate_front.outputs[0], front_mix.inputs["A"]) + links.new(scaled_depth.outputs[0], front_mix.inputs["B"]) + mirrored_front = _math(nodes, "MULTIPLY", (2850.0, -150.0)) + mirrored_front.inputs[1].default_value = -1.0 + links.new(front_mix.outputs["Result"], mirrored_front.inputs[0]) + back_choice = nodes.new("GeometryNodeSwitch") + back_choice.input_type = "FLOAT" + back_choice.location = (3050.0, 100.0) + links.new(double_sided, back_choice.inputs["Switch"]) + links.new(base_back.outputs[0], back_choice.inputs["False"]) + links.new(mirrored_front.outputs[0], back_choice.inputs["True"]) + extrude = _extrude_layers(group, geometry, (2400.0, 750.0)) + shaped = _set_layer_x( + group, + geometry, + extrude, + front_mix.outputs["Result"], + back_choice.outputs["Output"], + (3300.0, 650.0), + ) + front_only = _set_geometry_x( + group, geometry, front_mix.outputs["Result"], (3500.0, 100.0) + ) + return _remove_zero_thickness( + group, front_only, shaped, thickness, (4850.0, 600.0) + ) + + +def _depth_continuity_threshold(group, continuity, location): + nodes = group.nodes + links = group.links + percent = _math(nodes, "MULTIPLY", location) + percent.inputs[1].default_value = 0.01 + links.new(continuity, percent.inputs[0]) + inverse = _math(nodes, "SUBTRACT", (location[0] + 200.0, location[1])) + inverse.inputs[0].default_value = 0.5 + links.new(percent.outputs[0], inverse.inputs[1]) + exponent = _math(nodes, "MULTIPLY", (location[0] + 400.0, location[1])) + exponent.inputs[1].default_value = 10.0 + links.new(inverse.outputs[0], exponent.inputs[0]) + power = _math(nodes, "POWER", (location[0] + 600.0, location[1])) + power.inputs[0].default_value = 2.0 + links.new(exponent.outputs[0], power.inputs[1]) + scale = _math(nodes, "MULTIPLY", (location[0] + 800.0, location[1])) + scale.inputs[1].default_value = 0.04 + links.new(power.outputs[0], scale.inputs[0]) + return scale.outputs[0] + + +def _build_depth_surface( + group, + geometry, + thickness, + depth_scale, + continuity, + cleanup, + depth_image, + expand_smooth, +): + nodes = group.nodes + links = group.links + depth, discontinuity, _alpha = _image_fields( + group, depth_image, (-200.0, -200.0) + ) + threshold = _depth_continuity_threshold(group, continuity, (450.0, -250.0)) + jump = _math(nodes, "GREATER_THAN", (1450.0, -100.0)) + links.new(discontinuity, jump.inputs[0]) + links.new(threshold, jump.inputs[1]) + delete = nodes.new("GeometryNodeDeleteGeometry") + delete.domain = "FACE" + delete.location = (1650.0, 400.0) + links.new(geometry, delete.inputs["Geometry"]) + links.new(jump.outputs[0], delete.inputs["Selection"]) + geometry = _island_cleanup( + group, delete.outputs["Geometry"], cleanup, (1900.0, 0.0) + ) + + scaled = _math(nodes, "MULTIPLY", (2900.0, -100.0)) + links.new(depth, scaled.inputs[0]) + links.new(depth_scale, scaled.inputs[1]) + front = _math(nodes, "MULTIPLY", (3100.0, -100.0)) + front.inputs[1].default_value = -1.0 + links.new(scaled.outputs[0], front.inputs[0]) + back = _math(nodes, "ADD", (3300.0, -100.0)) + links.new(front.outputs[0], back.inputs[0]) + links.new(thickness, back.inputs[1]) + shaped = _set_segmented_layer_x( + group, + geometry, + front.outputs[0], + back.outputs[0], + thickness, + (3500.0, 400.0), + ) + front_only = _set_geometry_x( + group, geometry, front.outputs[0], (3700.0, -200.0) + ) + geometry = _remove_zero_thickness( + group, front_only, shaped, thickness, (7000.0, 400.0) + ) + return expand_smooth(group, geometry, x_offset=7300.0) + + +def node_group(shape, expand_smooth): + group_name = GROUP_NAMES[shape] + if shape == CUTOUT and bpy.data.node_groups.get(group_name) is None: + legacy = bpy.data.node_groups.get("O Image Cutout") + if legacy is not None: + legacy.name = group_name + group = bpy.data.node_groups.get(group_name) + if group is not None and group.get("anyimage_cutout_version") == NODE_GROUP_VERSION: + return group + if group is None: + group = bpy.data.node_groups.new(group_name, "GeometryNodeTree") + else: + group.nodes.clear() + group.interface.clear() + group.is_modifier = True + _socket(group, "Geometry", "INPUT", "NodeSocketGeometry") + if shape in {BALLOON, DEPTH_BALLOON}: + thickness = _float_input(group, "Thickness", 1.0) + thickness.description = ( + "Multiplier for the injected Poisson Balloon profile; zero keeps " + "the source sheet" + ) + else: + thickness = _float_input( + group, "Thickness", 0.0, 0.0, None, subtype="DISTANCE" + ) + thickness.description = "Final thickness in scene distance" + depth_image = None + if shape == DEPTH_BALLOON: + depth_scale = _float_input(group, "Depth Scale", 1.0) + depth_scale.description = "Scale the Base Plane-corrected image depth" + double_sided = _bool_input(group, "Double Sided", False) + double_sided.description = ( + "Mirror the complete front surface across the local YZ plane" + ) + depth_reference = _float_input( + group, + "Depth Reference", + 0.25, + 0.0, + None, + subtype="DISTANCE", + ) + depth_reference.description = "Internal scale for image depth" + depth_reference.hide_in_modifier = True + depth_image = _socket(group, "Depth Image", "INPUT", "NodeSocketImage") + depth_image.description = "Internally generated Depth Balloon data" + depth_image.hide_in_modifier = True + elif shape == DEPTH_SURFACE: + depth_scale = _float_input(group, "Depth Scale", 1.0) + iterations = _socket(group, "Iterations", "INPUT", "NodeSocketInt") + iterations.default_value = 1 + iterations.min_value = 0 + iterations.max_value = 20 + depth_image = _socket(group, "Depth Image", "INPUT", "NodeSocketImage") + depth_image.description = "Internally generated Depth Surface data" + depth_image.hide_in_modifier = True + + options = group.interface.new_panel(name="Options", default_closed=True) + cleanup = _float_input(group, "Cleanup", 1.0, 0.0, 100.0, parent=options) + cleanup.description = ( + "Remove disconnected islands smaller than this multiple of the " + "estimated point-spacing area" + ) + if shape == DEPTH_BALLOON: + base_plane = _socket( + group, "Base Plane", "INPUT", "NodeSocketObject", options + ) + base_plane.description = ( + "Fitted depth plane; its arrow points in the deformation direction" + ) + elif shape == DEPTH_SURFACE: + continuity = _float_input( + group, + "Depth Continuity", + 0.0, + 0.0, + 100.0, + subtype="PERCENTAGE", + parent=options, + ) + smooth = _float_input( + group, + "Smooth", + 1.0, + 0.0, + 1.0, + subtype="FACTOR", + parent=options, + ) + _socket(group, "Geometry", "OUTPUT", "NodeSocketGeometry") + + controls = _group_input( + group.nodes, + (-1600.0, 500.0), + { + "Geometry", + "Cleanup", + "Thickness", + "Double Sided", + "Base Plane", + "Depth Image", + "Depth Scale", + "Depth Continuity", + "Depth Reference", + }, + ) + geometry = _island_cleanup( + group, + controls.outputs["Geometry"], + controls.outputs["Cleanup"], + (-1300.0, 300.0), + ) + if shape == CUTOUT: + geometry = _build_cutout(group, geometry, controls.outputs["Thickness"]) + output_location = (4050.0, 300.0) + elif shape == BALLOON: + geometry = _build_balloon( + group, + geometry, + controls.outputs["Thickness"], + ) + output_location = (4650.0, 650.0) + elif shape == DEPTH_BALLOON: + geometry = _build_depth_balloon( + group, + geometry, + controls.outputs["Thickness"], + controls.outputs["Depth Scale"], + controls.outputs["Double Sided"], + controls.outputs["Base Plane"], + controls.outputs["Depth Image"], + controls.outputs["Depth Reference"], + ) + output_location = (5100.0, 600.0) + else: + geometry = _build_depth_surface( + group, + geometry, + controls.outputs["Thickness"], + controls.outputs["Depth Scale"], + controls.outputs["Depth Continuity"], + controls.outputs["Cleanup"], + controls.outputs["Depth Image"], + expand_smooth, + ) + output_location = (10700.0, 400.0) + _localize_group_inputs(group, controls) + _shade_output(group, geometry, output_location) + group.color_tag = "GEOMETRY" + group.description = { + CUTOUT: "Clean and symmetrically thicken a Surface sheet.", + BALLOON: "Inflate a Surface sheet from its boundary profile.", + DEPTH_BALLOON: ( + "Mix Balloon profile and hidden image depth around a Base Plane." + ), + DEPTH_SURFACE: ( + "Displace a Surface sheet from hidden image depth, cull, " + "thicken, and smooth." + ), + }[shape] + group["anyimage_cutout_version"] = NODE_GROUP_VERSION + group.use_fake_user = True + return group diff --git a/src/anyimage/operators/balloon/poisson.py b/src/anyimage/operators/cutout_profile.py similarity index 70% rename from src/anyimage/operators/balloon/poisson.py rename to src/anyimage/operators/cutout_profile.py index 5a19a03..0af2038 100644 --- a/src/anyimage/operators/balloon/poisson.py +++ b/src/anyimage/operators/cutout_profile.py @@ -1,5 +1,11 @@ import math +from .cutout_teddy import teddy_balloon_heights + + +DEFAULT_INFLATION = 0.45 +DEFAULT_ALGORITHM = "POISSON" + def cotangent_poisson_field( planar_points, @@ -46,7 +52,8 @@ def cotangent_poisson_field( edge_starts = [] edge_ends = [] weights = [] - # 约束剖分的瘦三角形可能产生负 cotangent 权重；钳制后保持 Dirichlet 系统正定。 + # Constrained triangulation can create negative cotangent weights. + # Clamping them keeps the Dirichlet system positive definite. for (start, end), weight in edge_weights.items(): weight = max(float(weight), 0.0) if weight <= 1e-14: @@ -62,7 +69,7 @@ def cotangent_poisson_field( edge_ends.append(end_compact) weights.append(weight) if np.any(diagonal <= 1e-14): - raise ValueError("Unable to build a stable Poisson Balloon field") + raise ValueError("Unable to build a stable Balloon profile") edge_starts = np.asarray(edge_starts, dtype=np.int64) edge_ends = np.asarray(edge_ends, dtype=np.int64) @@ -81,10 +88,7 @@ def cotangent_poisson_field( preconditioned = residual / diagonal direction = preconditioned.copy() residual_product = float(residual.dot(preconditioned)) - target = max( - float(np.linalg.norm(right_hand_side)) * tolerance, - 1e-12, - ) + target = max(float(np.linalg.norm(right_hand_side)) * tolerance, 1e-12) max_iterations = max(64, min(len(interior) * 4, 4000)) for _iteration in range(max_iterations): if float(np.linalg.norm(residual)) <= target: @@ -92,30 +96,28 @@ def cotangent_poisson_field( multiplied = multiply(direction) denominator = float(direction.dot(multiplied)) if denominator <= 0.0 or not math.isfinite(denominator): - raise ValueError("Unable to solve the Poisson Balloon field") + raise ValueError("Unable to solve the Balloon profile") step = residual_product / denominator solution += step * direction residual -= step * multiplied preconditioned = residual / diagonal next_product = float(residual.dot(preconditioned)) if not math.isfinite(next_product): - raise ValueError("Unable to solve the Poisson Balloon field") + raise ValueError("Unable to solve the Balloon profile") if abs(residual_product) <= 1e-30: break - direction = preconditioned + ( - next_product / residual_product - ) * direction + direction = preconditioned + next_product / residual_product * direction residual_product = next_product field[interior] = np.maximum(solution, 0.0) return field -def poisson_balloon_heights( +def poisson_balloon_profile( planar_points, planar_faces, boundary_mask, - inflation, + inflation=DEFAULT_INFLATION, ): import numpy as np @@ -124,4 +126,36 @@ def poisson_balloon_heights( planar_faces, boundary_mask, ) - return 2.0 * inflation * np.sqrt(field) + return 2.0 * float(inflation) * np.sqrt(field) + + +def balloon_profile( + algorithm, + planar_points, + planar_faces, + boundary_mask, + components, + inflation=DEFAULT_INFLATION, +): + import numpy as np + + algorithm = str(algorithm).upper() + if algorithm == "POISSON": + return poisson_balloon_profile( + planar_points, + planar_faces, + boundary_mask, + inflation, + ) + if algorithm != "TEDDY": + raise ValueError(f"Unknown Balloon profile algorithm: {algorithm}") + result = np.zeros(len(planar_points), dtype=np.float64) + for start, end, component_faces, outer_contour in components: + result[start:end] = teddy_balloon_heights( + np.asarray(planar_points)[start:end], + component_faces, + outer_contour, + np.asarray(boundary_mask)[start:end], + inflation, + ) + return result diff --git a/src/anyimage/operators/balloon/teddy.py b/src/anyimage/operators/cutout_teddy.py similarity index 99% rename from src/anyimage/operators/balloon/teddy.py rename to src/anyimage/operators/cutout_teddy.py index a97ad82..c891ff8 100644 --- a/src/anyimage/operators/balloon/teddy.py +++ b/src/anyimage/operators/cutout_teddy.py @@ -1,4 +1,3 @@ - def distances_to_contour(points, contour): import numpy as np diff --git a/src/anyimage/operators/image_edit/box.py b/src/anyimage/operators/image_edit/box.py index 3bef843..055ef0d 100644 --- a/src/anyimage/operators/image_edit/box.py +++ b/src/anyimage/operators/image_edit/box.py @@ -21,4 +21,4 @@ class ImageBoxTool(bpy.types.WorkSpaceTool): ) def draw_settings(context, layout, _tool): - draw_image_mask_settings(context, layout) + draw_image_mask_settings(context, layout, "image_box_refine_selection") diff --git a/src/anyimage/operators/image_edit/common.py b/src/anyimage/operators/image_edit/common.py index 5ed3dc5..2367ec6 100644 --- a/src/anyimage/operators/image_edit/common.py +++ b/src/anyimage/operators/image_edit/common.py @@ -208,12 +208,22 @@ def image_mask_refine_request(input_path, selection_data): } +def refine_selection_setting_name(operator): + if getattr(operator, "active_tool_id", None) == "anyimage.cutout_lasso": + return "cutout_refine_selection" + return { + "BOX": "image_box_refine_selection", + "LASSO": "image_lasso_refine_selection", + }.get(operator.gesture) + + def apply_image_mask_settings(operator, context): settings = context.scene.anyimage_settings + setting_name = refine_selection_setting_name(operator) operator.refine_selection = ( - operator.gesture in {"BOX", "LASSO"} + setting_name is not None and ai_ready() - and settings.image_edit_refine_selection + and getattr(settings, setting_name) ) @@ -234,12 +244,12 @@ def draw_image_mask_operator_settings(layout, operator): ) -def draw_image_mask_settings(context, layout): +def draw_image_mask_settings(context, layout, setting_name): settings = context.scene.anyimage_settings status = ai_status() row = layout.row() row.enabled = status["ready"] - row.prop(settings, "image_edit_refine_selection") + row.prop(settings, setting_name) if not status["ready"]: layout.operator( "anyimage.setup_ai_environment", diff --git a/src/anyimage/operators/image_edit/interaction.py b/src/anyimage/operators/image_edit/interaction.py new file mode 100644 index 0000000..c78691a --- /dev/null +++ b/src/anyimage/operators/image_edit/interaction.py @@ -0,0 +1,87 @@ +import json +import math + +import bpy + + +MIN_PATH_DISTANCE = 2.0 + + +def simplify_screen_path(path): + pixels = [] + for point in path: + point = (float(point[0]), float(point[1])) + if not pixels or math.dist(point, pixels[-1]) >= MIN_PATH_DISTANCE: + pixels.append(point) + return pixels + + +def screen_path_to_image_pixels( + path, + region, + region_data, + matrix_world, + local_bounds, + image_size, +): + from bpy_extras import view3d_utils + + inverse = matrix_world.inverted() + x_min, x_max, y_min, y_max = local_bounds + image_width, image_height = image_size + local_width = x_max - x_min + local_height = y_max - y_min + pixels = [] + for point in simplify_screen_path(path): + ray_origin = inverse @ view3d_utils.region_2d_to_origin_3d( + region, + region_data, + point, + ) + ray_direction = inverse.to_3x3() @ view3d_utils.region_2d_to_vector_3d( + region, + region_data, + point, + ) + if abs(ray_direction.z) <= 1e-10: + raise ValueError("The Image Empty is edge-on to the current view") + distance = -ray_origin.z / ray_direction.z + local = ray_origin + ray_direction * distance + pixels.append( + ( + (local.x - x_min) * image_width / local_width, + (y_max - local.y) * image_height / local_height, + ) + ) + return pixels + + +def serialize_matrix(matrix): + return json.dumps( + [float(matrix[row][column]) for row in range(4) for column in range(4)], + separators=(",", ":"), + ) + + +def deserialize_matrix(data): + from mathutils import Matrix + + values = json.loads(data) + if len(values) != 16: + raise ValueError("The stored Image Empty transform is invalid") + return Matrix(tuple(tuple(values[row * 4 : row * 4 + 4]) for row in range(4))) + + +def hidden_string_property(): + return bpy.props.StringProperty(options={"HIDDEN", "SKIP_SAVE"}) + + +def drawing_in_region(area_pointer, region_pointer): + area = getattr(bpy.context, "area", None) + region = getattr(bpy.context, "region", None) + return ( + area is not None + and region is not None + and area.as_pointer() == area_pointer + and region.as_pointer() == region_pointer + ) diff --git a/src/anyimage/operators/image_edit/lasso.py b/src/anyimage/operators/image_edit/lasso.py index 39b1986..1183e3c 100644 --- a/src/anyimage/operators/image_edit/lasso.py +++ b/src/anyimage/operators/image_edit/lasso.py @@ -21,4 +21,4 @@ class ImageLassoTool(bpy.types.WorkSpaceTool): ) def draw_settings(context, layout, _tool): - draw_image_mask_settings(context, layout) + draw_image_mask_settings(context, layout, "image_lasso_refine_selection") diff --git a/src/anyimage/operators/image_edit/mask.py b/src/anyimage/operators/image_edit/mask.py index 7452e22..8765c3c 100644 --- a/src/anyimage/operators/image_edit/mask.py +++ b/src/anyimage/operators/image_edit/mask.py @@ -4,10 +4,10 @@ import math import bpy from ...properties import ( - image_edit_refine_selection_property as refine_selection_property, + refine_selection_property, ) from ...runtime import JobOperatorBase -from ..balloon.operator import ( +from .interaction import ( deserialize_matrix, drawing_in_region, hidden_string_property, diff --git a/src/anyimage/operators/image_edit/perspective.py b/src/anyimage/operators/image_edit/perspective.py index b978588..11e8080 100644 --- a/src/anyimage/operators/image_edit/perspective.py +++ b/src/anyimage/operators/image_edit/perspective.py @@ -3,7 +3,7 @@ import math import bpy -from ..balloon.operator import ( +from .interaction import ( deserialize_matrix, drawing_in_region, hidden_string_property, diff --git a/src/anyimage/operators/mesh.py b/src/anyimage/operators/mesh.py index cbaddf3..4bf727a 100644 --- a/src/anyimage/operators/mesh.py +++ b/src/anyimage/operators/mesh.py @@ -5,13 +5,19 @@ from ..preferences import addon_preferences from ..properties import BEN2_MODEL_KEY, DEFAULT_MOGE2_MODEL_KEY from ..runtime import JobOperatorBase from ..server.geometry.cutout import alpha_regions -from .balloon.depth import load_balloon_depth_data -from .balloon.geometry import ( +from . import cutout_nodes +from .cutout_depth import ( + depth_balloon_fields, + create_depth_balloon_base_plane, + load_depth_balloon_data, + orient_depth_balloon_plane, +) +from .cutout_geometry import ( metric_transform, prepare_planar_regions, triangulate_planar_regions, ) -from .balloon.operator import extract_balloon_object +from .cutout_profile import balloon_profile as calculate_balloon_profile from .common import ( HIDDEN, cleanup_temporary_input, @@ -29,6 +35,7 @@ from .image import ( analysis_source_image, color_data_name, create_color_image, + depth_data_name, image_base_name, image_user_settings, is_image_empty, @@ -65,13 +72,14 @@ DEPTH_SURFACE = "DEPTH_SURFACE" CUTOUT_SHAPES = {CUTOUT, BALLOON, DEPTH_BALLOON, DEPTH_SURFACE} LOCAL_CUTOUT_SHAPES = {CUTOUT, BALLOON} DEPTH_SHAPES = {DEPTH_PLANE, DEPTH_BALLOON, DEPTH_SURFACE} -PROFILE_ATTRIBUTE = "o_profile_offset" -DEPTH_ATTRIBUTE = "o_depth_offset" -FRONT_FACE_ATTRIBUTE = "o_front_face" -DEPTH_DISCONTINUITY_ATTRIBUTE = "o_depth_discontinuity" -DEPTH_PROTECTION_ATTRIBUTE = "o_depth_protection" -BOUNDARY_ATTRIBUTE = "o_boundary" +BALLOON_ATTRIBUTE = cutout_nodes.BALLOON_ATTRIBUTE NODE_GROUP_VERSION = 13 +CUTOUT_AXIS_TO_SOURCE = ( + (0.0, -1.0, 0.0, 0.0), + (0.0, 0.0, 1.0, 0.0), + (-1.0, 0.0, 0.0, 0.0), + (0.0, 0.0, 0.0, 1.0), +) DEPTH_EDGE_THRESHOLD = 12.0 / 255.0 @@ -105,7 +113,7 @@ def _depth_field_image(source_object, anchored): pixels[:, :, :3] = values[:, :, None] pixels[:, :, 3] = 1.0 image = bpy.data.images.new( - f"{image_base_name(source_object.data)}.depth", + depth_data_name(source_object), width=width, height=height, alpha=True, @@ -117,7 +125,93 @@ def _depth_field_image(source_object, anchored): return image -def _cutout_domain(regions_pixels, image_size, bounds, spacing, world_metric): +def _cutout_depth_image(source_object, depth, auxiliary=None, valid=None): + import numpy as np + + depth = np.asarray(depth, dtype=np.float32) + height, width = depth.shape + pixels = np.zeros((height, width, 4), dtype=np.float32) + pixels[:, :, 0] = depth + if auxiliary is not None: + pixels[:, :, 1] = np.asarray(auxiliary, dtype=np.float32) + pixels[:, :, 3] = ( + np.asarray(valid, dtype=np.float32) + if valid is not None + else 1.0 + ) + image = bpy.data.images.new( + depth_data_name(source_object), + width=width, + height=height, + alpha=True, + float_buffer=True, + ) + image.colorspace_settings.name = "Non-Color" + image.pixels.foreach_set(np.flipud(pixels).ravel()) + image.pack() + return image + + +def _surface_depth_image(source_object, anchored, valid, camera_depth): + return _cutout_depth_image( + source_object, + anchored[..., 2], + _depth_discontinuity_map(camera_depth, valid), + valid, + ) + + +def _depth_balloon_image(source_object, depth_data): + return _cutout_depth_image( + source_object, + depth_data["depth_signal"], + valid=depth_data["valid"], + ) + + +def _cutout_world_matrix(source_matrix, crop_center): + from mathutils import Matrix + + center_x, center_y = (float(value) for value in crop_center) + return ( + source_matrix + @ Matrix.Translation((center_x, center_y, 0.0)) + @ Matrix(CUTOUT_AXIS_TO_SOURCE) + ) + + +def _orient_cutout_vertices(vertices): + import numpy as np + + vertices = np.asarray(vertices, dtype=np.float64) + return np.column_stack( + ( + -vertices[:, 2], + -vertices[:, 0], + vertices[:, 1], + ) + ) + + +def _depth_balloon_reference(vertices): + import numpy as np + + vertices = np.asarray(vertices, dtype=np.float64) + extents = np.ptp(vertices[:, :2], axis=0) + positive = extents[extents > 1e-8] + if not len(positive): + return 1e-6 + return max(float(positive.min()) * 0.25, 1e-6) + + +def _cutout_domain( + regions_pixels, + image_size, + bounds, + spacing, + world_metric, + balloon_algorithm="POISSON", +): import numpy as np x_min, x_max, y_min, y_max = bounds @@ -129,19 +223,29 @@ def _cutout_domain(regions_pixels, image_size, bounds, spacing, world_metric): spacing=spacing, world_metric=world_metric, ) - planar, planar_faces, boundary = triangulate_planar_regions( - regions, spacing + planar, planar_faces, boundary, components = triangulate_planar_regions( + regions, + spacing, + return_components=True, + ) + profile = calculate_balloon_profile( + balloon_algorithm, planar, planar_faces, boundary, components ) faces = np.asarray(planar_faces, dtype=np.int32) local = planar @ np.linalg.inv(metric_transform(world_metric)) - vertices = np.column_stack((local, np.zeros(len(local), dtype=np.float64))) uv = np.column_stack( ( (local[:, 0] - x_min) / (x_max - x_min), (local[:, 1] - y_min) / (y_max - y_min), ) ).astype(np.float32) - return vertices, faces, uv, boundary, planar + crop_center = np.asarray( + ((x_min + x_max) * 0.5, (y_min + y_max) * 0.5), + dtype=np.float64, + ) + local -= crop_center + vertices = np.column_stack((local, np.zeros(len(local), dtype=np.float64))) + return vertices, faces, uv, boundary, planar, profile def _world_plane_metric(matrix_world): @@ -154,7 +258,6 @@ def _world_plane_metric(matrix_world): return ((x.dot(x), x.dot(y)), (x.dot(y), y.dot(y))), z.length - def _load_field(path): import numpy as np @@ -195,66 +298,6 @@ def _anchored_point_field(source_object, field, bounds=None): return anchored, valid, points[..., 2] -def _sample_field(values, valid, uv): - import numpy as np - - height, width = valid.shape - x = np.clip(uv[:, 0], 0.0, 1.0) * max(width - 1, 0) - y = (1.0 - np.clip(uv[:, 1], 0.0, 1.0)) * max(height - 1, 0) - x0 = np.floor(x).astype(np.int32) - y0 = np.floor(y).astype(np.int32) - x1 = np.minimum(x0 + 1, width - 1) - y1 = np.minimum(y0 + 1, height - 1) - weights = np.stack( - ( - (x1 - x) * (y1 - y), - (x - x0) * (y1 - y), - (x1 - x) * (y - y0), - (x - x0) * (y - y0), - ), - axis=1, - ) - samples = np.stack( - (values[y0, x0], values[y0, x1], values[y1, x0], values[y1, x1]), axis=1 - ) - usable = np.stack( - (valid[y0, x0], valid[y0, x1], valid[y1, x0], valid[y1, x1]), axis=1 - ) - weights *= usable - total = weights.sum(axis=1) - vector_field = values.ndim == 3 - result = np.zeros( - (len(uv), values.shape[-1]) if vector_field else len(uv), - dtype=np.float32, - ) - selected = total > 1e-8 - if vector_field: - result[selected] = (samples[selected] * weights[selected, :, None]).sum( - axis=1 - ) / total[selected, None] - else: - result[selected] = (samples[selected] * weights[selected]).sum(axis=1) / total[ - selected - ] - if not selected.all(): - nearest_x = np.rint(x[~selected]).astype(np.int32) - nearest_y = np.rint(y[~selected]).astype(np.int32) - result[~selected] = values[nearest_y, nearest_x] - return result - - -def _depth_discontinuity_values(faces, depth): - import numpy as np - - triangle_depth = np.asarray(depth, dtype=np.float32)[faces] - valid = np.isfinite(triangle_depth).all(axis=1) & (triangle_depth > 0.0).all(axis=1) - scale = np.maximum(np.min(np.abs(triangle_depth), axis=1), 1e-6) - difference = np.max(triangle_depth, axis=1) - np.min(triangle_depth, axis=1) - result = np.full(len(faces), float("inf"), dtype=np.float32) - result[valid] = difference[valid] / scale[valid] - return result - - def _depth_discontinuity_map(depth, valid): import numpy as np @@ -271,22 +314,6 @@ def _depth_discontinuity_map(depth, valid): return result.astype(np.float32) -def _vertex_normals(vertices, faces): - import numpy as np - - triangles = vertices[faces] - normals = np.cross( - triangles[:, 1] - triangles[:, 0], triangles[:, 2] - triangles[:, 0] - ) - result = np.zeros_like(vertices, dtype=np.float64) - for corner in range(3): - np.add.at(result, faces[:, corner], normals) - lengths = np.linalg.norm(result, axis=1) - result[lengths > 1e-10] /= lengths[lengths > 1e-10, None] - result[lengths <= 1e-10] = (0.0, 0.0, 1.0) - return result - - def _counter_clockwise_faces(points, faces): import numpy as np @@ -303,83 +330,6 @@ def _counter_clockwise_faces(points, faces): return result -def _boundary_edges(faces): - import numpy as np - - faces = np.asarray(faces, dtype=np.int32) - directed = np.concatenate( - (faces[:, (0, 1)], faces[:, (1, 2)], faces[:, (2, 0)]), axis=0 - ) - keys = np.sort(directed, axis=1) - _unique, inverse, users = np.unique( - keys, axis=0, return_inverse=True, return_counts=True - ) - return directed[users[inverse] == 1] - - -def _layered_geometry(domain, shape, field=None): - import numpy as np - - vertices, faces, uv, boundary, planar = domain - faces = _counter_clockwise_faces(planar, faces) - depth = np.zeros_like(vertices, dtype=np.float64) - discontinuity = np.zeros(len(faces), dtype=np.float32) - vertex_discontinuity = np.zeros(len(vertices), dtype=np.float32) - carrier = vertices - if field is not None: - anchored, valid, camera_depth = field - target = _sample_field(anchored, valid, uv) - if shape == DEPTH_SURFACE: - vertex_discontinuity = _sample_field( - _depth_discontinuity_map(camera_depth, valid), valid, uv - ) - discontinuity = np.max(vertex_discontinuity[faces], axis=1) - depth = target - vertices - carrier = target - elif shape == DEPTH_PLANE: - depth[:, 2] = target[:, 2] - - normals = _vertex_normals(carrier, faces) - front = np.arange(len(vertices), dtype=np.int32) - back = front + len(vertices) - output_vertices = np.concatenate((vertices, vertices), axis=0) - output_uv = np.concatenate((uv, uv), axis=0) - if shape == DEPTH_SURFACE: - profile_offset = np.concatenate((np.zeros_like(normals), -normals), axis=0) - depth_offset = np.concatenate((depth, depth), axis=0) - else: - profile_offset = np.concatenate((normals * 0.5, -normals * 0.5), axis=0) - depth_offset = np.concatenate((depth, np.zeros_like(depth)), axis=0) - - output_faces = [tuple(int(value) for value in face) for face in faces] - output_faces.extend( - tuple(int(back[value]) for value in reversed(face)) for face in faces - ) - output_faces.extend( - (int(front[start]), int(back[start]), int(back[end]), int(front[end])) - for start, end in _boundary_edges(faces) - ) - front_faces = [True] * len(faces) + [False] * (len(output_faces) - len(faces)) - output_boundary = np.concatenate((boundary, boundary), axis=0) - depth_protection = np.concatenate( - (vertex_discontinuity, vertex_discontinuity), axis=0 - ) - face_discontinuity = np.concatenate( - (discontinuity, discontinuity, np.zeros(len(output_faces) - len(faces) * 2)) - ) - return ( - output_vertices, - output_faces, - output_uv, - profile_offset, - depth_offset, - front_faces, - output_boundary, - depth_protection, - face_discontinuity, - ) - - def _assign_uv(mesh, uv): import numpy as np @@ -392,20 +342,6 @@ def _assign_uv(mesh, uv): uv_layer.data.foreach_set("uv", loop_uv.ravel()) -def _assign_vector_attribute(mesh, name, values): - import numpy as np - - attribute = mesh.attributes.new(name=name, type="FLOAT_VECTOR", domain="POINT") - attribute.data.foreach_set("vector", np.asarray(values, dtype=np.float32).ravel()) - - -def _assign_bool_point_attribute(mesh, name, values): - import numpy as np - - attribute = mesh.attributes.new(name=name, type="BOOLEAN", domain="POINT") - attribute.data.foreach_set("value", np.asarray(values, dtype=bool)) - - def _assign_float_point_attribute(mesh, name, values): import numpy as np @@ -413,20 +349,6 @@ def _assign_float_point_attribute(mesh, name, values): attribute.data.foreach_set("value", np.asarray(values, dtype=np.float32)) -def _assign_face_attribute(mesh, name, values): - import numpy as np - - attribute = mesh.attributes.new(name=name, type="BOOLEAN", domain="FACE") - attribute.data.foreach_set("value", np.asarray(values, dtype=bool)) - - -def _assign_float_face_attribute(mesh, name, values): - import numpy as np - - attribute = mesh.attributes.new(name=name, type="FLOAT", domain="FACE") - attribute.data.foreach_set("value", np.asarray(values, dtype=np.float32)) - - def _interface_socket(group, name, in_out, socket_type): socket = group.interface.new_socket( name=name, @@ -760,198 +682,9 @@ def _mesh_node_group(shape): return load_mesh_plane_node_group() if shape == DEPTH_PLANE: return _depth_plane_node_group() - group_name = { - CUTOUT: "O Image Cutout", - DEPTH_SURFACE: "O Image Depth Surface", - }[shape] - group = bpy.data.node_groups.get(group_name) - if group is not None and group.get("anyimage_mesh_version") == NODE_GROUP_VERSION: - return group - if group is None: - group = bpy.data.node_groups.new(group_name, "GeometryNodeTree") - else: - group.nodes.clear() - group.interface.clear() - group.is_modifier = True - _interface_socket(group, "Geometry", "INPUT", "NodeSocketGeometry") - thickness = _interface_socket(group, "Thickness", "INPUT", "NodeSocketFloat") - thickness.default_value = 0.0 - thickness.min_value = 0.0 - thickness.subtype = "DISTANCE" - if shape == DEPTH_SURFACE: - depth_amount = _interface_socket( - group, "Depth Scale", "INPUT", "NodeSocketFloat" - ) - depth_amount.default_value = 1.0 - depth_amount.min_value = 0.0 - iterations = _interface_socket( - group, "Iterations", "INPUT", "NodeSocketInt" - ) - iterations.default_value = 1 - iterations.min_value = 0 - iterations.max_value = 20 - smooth = _interface_socket(group, "Smooth", "INPUT", "NodeSocketFloat") - smooth.default_value = 1.0 - smooth.min_value = 0.0 - smooth.max_value = 1.0 - smooth.subtype = "FACTOR" - continuity = _interface_socket( - group, "Depth Continuity", "INPUT", "NodeSocketFloat" - ) - continuity.default_value = 0.0 - continuity.min_value = 0.0 - continuity.max_value = 100.0 - continuity.subtype = "PERCENTAGE" - _interface_socket(group, "Geometry", "OUTPUT", "NodeSocketGeometry") - - nodes = group.nodes - links = group.links - shape_input = _group_input( - nodes, - (-1900.0, 250.0), - {"Geometry", "Thickness", "Depth Scale"}, - ) - group_output = nodes.new("NodeGroupOutput") - - profile_attribute = nodes.new("GeometryNodeInputNamedAttribute") - profile_attribute.data_type = "FLOAT_VECTOR" - profile_attribute.inputs["Name"].default_value = PROFILE_ATTRIBUTE - profile_attribute.location = (-1900.0, 650.0) - profile_scale = nodes.new("ShaderNodeVectorMath") - profile_scale.operation = "SCALE" - profile_scale.location = (-1650.0, 650.0) - links.new(profile_attribute.outputs["Attribute"], profile_scale.inputs[0]) - links.new(shape_input.outputs["Thickness"], profile_scale.inputs[3]) - offset = profile_scale.outputs["Vector"] - - if shape == DEPTH_SURFACE: - depth_attribute = nodes.new("GeometryNodeInputNamedAttribute") - depth_attribute.data_type = "FLOAT_VECTOR" - depth_attribute.inputs["Name"].default_value = DEPTH_ATTRIBUTE - depth_attribute.location = (-1900.0, 400.0) - depth_scale = nodes.new("ShaderNodeVectorMath") - depth_scale.operation = "SCALE" - depth_scale.location = (-1650.0, 400.0) - links.new(depth_attribute.outputs["Attribute"], depth_scale.inputs[0]) - links.new(shape_input.outputs["Depth Scale"], depth_scale.inputs[3]) - offset = depth_scale.outputs["Vector"] - - set_position = nodes.new("GeometryNodeSetPosition") - set_position.location = (-1150.0, 550.0) - links.new(shape_input.outputs["Geometry"], set_position.inputs["Geometry"]) - links.new(offset, set_position.inputs["Offset"]) - geometry = set_position.outputs["Geometry"] - - if shape == DEPTH_SURFACE: - culling_input = _group_input( - nodes, - (-1050.0, -400.0), - {"Depth Continuity"}, - ) - discontinuity = nodes.new("GeometryNodeInputNamedAttribute") - discontinuity.data_type = "FLOAT" - discontinuity.inputs["Name"].default_value = DEPTH_DISCONTINUITY_ATTRIBUTE - discontinuity.location = (-1150.0, 50.0) - continuity_scale = nodes.new("ShaderNodeMath") - continuity_scale.operation = "MULTIPLY" - continuity_scale.inputs[1].default_value = 0.01 - continuity_scale.location = (-900.0, -250.0) - inverse_continuity = nodes.new("ShaderNodeMath") - inverse_continuity.operation = "SUBTRACT" - inverse_continuity.inputs[0].default_value = 0.5 - inverse_continuity.location = (-700.0, -250.0) - exponent_scale = nodes.new("ShaderNodeMath") - exponent_scale.operation = "MULTIPLY" - exponent_scale.inputs[1].default_value = 10.0 - exponent_scale.location = (-500.0, -250.0) - threshold_power = nodes.new("ShaderNodeMath") - threshold_power.operation = "POWER" - threshold_power.inputs[0].default_value = 2.0 - threshold_power.location = (-300.0, -250.0) - threshold_scale = nodes.new("ShaderNodeMath") - threshold_scale.operation = "MULTIPLY" - threshold_scale.inputs[1].default_value = 0.04 - threshold_scale.location = (-100.0, -250.0) - depth_jump = nodes.new("ShaderNodeMath") - depth_jump.operation = "GREATER_THAN" - depth_jump.location = (100.0, 50.0) - delete_depth = nodes.new("GeometryNodeDeleteGeometry") - delete_depth.domain = "FACE" - delete_depth.location = (350.0, 550.0) - links.new( - culling_input.outputs["Depth Continuity"], - continuity_scale.inputs[0], - ) - links.new(continuity_scale.outputs[0], inverse_continuity.inputs[1]) - links.new(inverse_continuity.outputs[0], exponent_scale.inputs[0]) - links.new(exponent_scale.outputs[0], threshold_power.inputs[1]) - links.new(threshold_power.outputs[0], threshold_scale.inputs[0]) - links.new(discontinuity.outputs["Attribute"], depth_jump.inputs[0]) - links.new(threshold_scale.outputs[0], depth_jump.inputs[1]) - links.new(geometry, delete_depth.inputs["Geometry"]) - links.new(depth_jump.outputs[0], delete_depth.inputs["Selection"]) - geometry = delete_depth.outputs["Geometry"] - - apply_thickness = nodes.new("GeometryNodeSetPosition") - apply_thickness.location = (550.0, 550.0) - links.new(geometry, apply_thickness.inputs["Geometry"]) - links.new(profile_scale.outputs["Vector"], apply_thickness.inputs["Offset"]) - geometry = apply_thickness.outputs["Geometry"] - - front_face = nodes.new("GeometryNodeInputNamedAttribute") - front_face.data_type = "BOOLEAN" - front_face.inputs["Name"].default_value = FRONT_FACE_ATTRIBUTE - not_front = nodes.new("FunctionNodeBooleanMath") - not_front.operation = "NOT" - zero_thickness = nodes.new("ShaderNodeMath") - zero_thickness.operation = "LESS_THAN" - zero_thickness.inputs[1].default_value = 1e-6 - delete_selection = nodes.new("FunctionNodeBooleanMath") - delete_selection.operation = "AND" - delete_geometry = nodes.new("GeometryNodeDeleteGeometry") - delete_geometry.domain = "FACE" - finish_x = 800.0 if shape == DEPTH_SURFACE else -750.0 - finish_input = _group_input( - nodes, - (finish_x + 150.0, -250.0), - {"Thickness"}, - ) - front_face.location = (finish_x, 100.0) - not_front.location = (finish_x + 200.0, 100.0) - zero_thickness.location = (finish_x + 200.0, -100.0) - delete_selection.location = (finish_x + 400.0, 0.0) - delete_geometry.location = (finish_x + 650.0, 550.0) - links.new(front_face.outputs["Attribute"], not_front.inputs[0]) - links.new(finish_input.outputs["Thickness"], zero_thickness.inputs[0]) - links.new(not_front.outputs[0], delete_selection.inputs[0]) - links.new(zero_thickness.outputs[0], delete_selection.inputs[1]) - links.new(geometry, delete_geometry.inputs["Geometry"]) - links.new(delete_selection.outputs[0], delete_geometry.inputs["Selection"]) - - geometry = delete_geometry.outputs["Geometry"] - if shape == DEPTH_SURFACE: - geometry = _expand_smooth(group, geometry, x_offset=2250.0) - shade_x = 5550.0 - else: - shade_x = finish_x + 900.0 - - shade_smooth = nodes.new("GeometryNodeSetShadeSmooth") - shade_smooth.domain = "FACE" - shade_smooth.inputs["Shade Smooth"].default_value = True - shade_smooth.location = (shade_x, 550.0) - group_output.location = (shade_x + 250.0, 550.0) - links.new(geometry, shade_smooth.inputs["Geometry"]) - links.new(shade_smooth.outputs["Geometry"], group_output.inputs["Geometry"]) - group.color_tag = "GEOMETRY" - group.description = ( - "Cull depth discontinuities, extrude thickness backward, then smooth inline." - if shape == DEPTH_SURFACE - else "Apply cutout thickness and smooth shading." - ) - group["anyimage_mesh_version"] = NODE_GROUP_VERSION - group.use_fake_user = True - return group - + if shape in CUTOUT_SHAPES: + return cutout_nodes.node_group(shape, _expand_smooth) + raise ValueError(f"Unsupported mesh shape: {shape}") def _copy_color_image(source_object): @@ -1182,6 +915,7 @@ def _build_mesh_object( detail, material, field=None, + depth_data=None, regions=None, image_size=None, bounds=None, @@ -1190,64 +924,86 @@ def _build_mesh_object( import numpy as np bounds = bounds or plane_bounds(source_object) - world_metric, _world_z_scale = _world_plane_metric(source_object.matrix_world) - domain = ( - _plane_domain(bounds, detail, world_metric) - if shape in {PLANE, DEPTH_PLANE} - else _cutout_domain( + crop_center = ( + (bounds[0] + bounds[1]) * 0.5, + (bounds[2] + bounds[3]) * 0.5, + ) + world_metric, world_z_scale = _world_plane_metric(source_object.matrix_world) + depth_image = None + depth_reference = None + base_plane_values = None + if shape in {PLANE, DEPTH_PLANE}: + domain = _plane_domain(bounds, detail, world_metric) + vertices, faces, uv, _boundary, _planar = domain + anchored = ( + _anchored_point_field(source_object, field, bounds) + if field + else None + ) + if shape == DEPTH_PLANE: + depth_image = _depth_field_image(source_object, anchored[0]) + else: + preferences = addon_preferences(context) + balloon_algorithm = getattr( + preferences, "balloon_profile_algorithm", "POISSON" + ) + domain = _cutout_domain( regions, image_size, bounds, _cutout_point_spacing(source_object, detail), world_metric, + balloon_algorithm, ) - ) - anchored = ( - _anchored_point_field(source_object, field, bounds) - if field - else None - ) - depth_image = None - face_discontinuity = None - if shape in {PLANE, DEPTH_PLANE}: - vertices, faces, uv, boundary, _planar = domain - if shape == DEPTH_PLANE: - depth_image = _depth_field_image(source_object, anchored[0]) - else: - ( - vertices, - faces, - uv, - profile, - depth, - front_faces, - boundary, - depth_protection, - face_discontinuity, - ) = _layered_geometry(domain, shape, anchored) + source_vertices, faces, uv, boundary, planar, balloon_profile = domain + faces = _counter_clockwise_faces(planar, faces) + vertices = _orient_cutout_vertices(source_vertices) + if world_z_scale <= 1e-10: + raise ValueError("The Image Empty has a degenerate world transform") + balloon_profile /= world_z_scale + if shape == DEPTH_SURFACE: + anchored, valid, camera_depth = _anchored_point_field( + source_object, + field, + bounds, + ) + depth_image = _surface_depth_image( + source_object, + anchored, + valid, + camera_depth, + ) + elif shape == DEPTH_BALLOON: + if depth_data is None: + raise ValueError("Depth Balloon data is missing") + depth_image = _depth_balloon_image(source_object, depth_data) + depth_reference = _depth_balloon_reference(source_vertices) + _sampled, local_plane = depth_balloon_fields( + source_vertices, + uv, + depth_data, + depth_reference, + ) + base_plane_values = orient_depth_balloon_plane(local_plane) + source_name = source_object.name mesh = bpy.data.meshes.new(source_name) mesh.from_pydata(np.asarray(vertices).tolist(), [], faces) for polygon in mesh.polygons: polygon.use_smooth = True _assign_uv(mesh, uv) - if shape not in {PLANE, DEPTH_PLANE}: - _assign_vector_attribute(mesh, PROFILE_ATTRIBUTE, profile) - _assign_vector_attribute(mesh, DEPTH_ATTRIBUTE, depth) - _assign_face_attribute(mesh, FRONT_FACE_ATTRIBUTE, front_faces) - _assign_bool_point_attribute(mesh, BOUNDARY_ATTRIBUTE, boundary) - _assign_float_point_attribute( - mesh, DEPTH_PROTECTION_ATTRIBUTE, depth_protection - ) - if face_discontinuity is not None: - _assign_float_face_attribute( - mesh, DEPTH_DISCONTINUITY_ATTRIBUTE, face_discontinuity - ) + if shape in CUTOUT_SHAPES: + _assign_float_point_attribute(mesh, BALLOON_ATTRIBUTE, balloon_profile) mesh.materials.append(material) mesh.update() + mesh_object = bpy.data.objects.new(source_name, mesh) context.collection.objects.link(mesh_object) - mesh_object.matrix_world = source_object.matrix_world.copy() + mesh_object.matrix_world = ( + _cutout_world_matrix(source_object.matrix_world, crop_center) + if shape in CUTOUT_SHAPES + else source_object.matrix_world.copy() + ) mesh_object["anyimage_mesh_shape"] = shape node_group = _mesh_node_group(shape) modifier = mesh_object.modifiers.new(name=node_group.name, type="NODES") @@ -1265,7 +1021,17 @@ def _build_mesh_object( modifier_input_identifier(node_group, "Material"), material, ) - thickness = 0.1 if shape == DEPTH_PLANE else 0.0 + if shape in CUTOUT_SHAPES: + set_modifier_input( + modifier, + modifier_input_identifier(node_group, "Cleanup"), + 1.0, + ) + thickness = ( + 1.0 + if shape in {BALLOON, DEPTH_BALLOON} + else 0.1 if shape == DEPTH_PLANE else 0.0 + ) set_modifier_input( modifier, modifier_input_identifier(node_group, "Thickness"), @@ -1280,6 +1046,24 @@ def _build_mesh_object( ), 1.0, ) + base_plane = None + if shape == DEPTH_BALLOON: + set_modifier_input( + modifier, + modifier_input_identifier(node_group, "Depth Reference"), + depth_reference, + ) + base_plane = create_depth_balloon_base_plane( + context, + mesh_object, + vertices, + base_plane_values, + ) + set_modifier_input( + modifier, + modifier_input_identifier(node_group, "Base Plane"), + base_plane, + ) if depth_image is not None: set_modifier_input( modifier, @@ -1304,6 +1088,23 @@ def _load_normal_image(path, source_object): return image +def _orient_cutout_object_normal(image): + import numpy as np + + width, height = (int(value) for value in image.size) + pixels = np.empty(width * height * 4, dtype=np.float32) + image.pixels.foreach_get(pixels) + rgba = pixels.reshape((height, width, 4)) + source = rgba[:, :, :3] * 2.0 - 1.0 + oriented = np.empty_like(source) + oriented[:, :, 0] = -source[:, :, 2] + oriented[:, :, 1] = -source[:, :, 0] + oriented[:, :, 2] = source[:, :, 1] + rgba[:, :, :3] = np.clip(oriented * 0.5 + 0.5, 0.0, 1.0) + image.pixels.foreach_set(rgba.ravel()) + image.update() + + def _cutout_point_spacing(source_object, detail): import math @@ -1361,38 +1162,18 @@ def _build_local_cutout( material = None try: color_image = _cropped_color_image(source_object, crop_bounds) - if shape == BALLOON: - result_object = extract_balloon_object( - context, - source_object, - regions, - source_object.matrix_world.copy(), - "POISSON", - thickness=1.0, - resolution=_cutout_point_spacing( - source_object, operator.mesh_detail - ), - select_result=True, - reuse_material=False, - depth_data=None, - normal_path=None, - material_image=color_image, - local_bounds=local_bounds, - ) - result_object["anyimage_mesh_shape"] = shape - else: - material = _new_material(context, source_object, color_image) - _build_mesh_object( - context, - source_object, - shape, - operator.mesh_detail, - material, - regions=regions, - image_size=image_size, - bounds=local_bounds, - keep_original=True, - ) + material = _new_material(context, source_object, color_image) + _build_mesh_object( + context, + source_object, + shape, + operator.mesh_detail, + material, + regions=regions, + image_size=image_size, + bounds=local_bounds, + keep_original=True, + ) except Exception: if material is not None and material.users == 0: bpy.data.materials.remove(material) @@ -1401,6 +1182,7 @@ def _build_local_cutout( raise push_undo(f"Convert to {shape.replace('_', ' ').title()}") + def _build_response(context, result, operator): source_object = bpy.data.objects.get(operator.source_object_name) if not is_image_empty(source_object): @@ -1448,72 +1230,57 @@ def _build_response(context, result, operator): ) normal_path = result.directory / "normal.png" - if operator.shape in {BALLOON, DEPTH_BALLOON}: - depth_data = ( - load_balloon_depth_data(result.file("depth")) - if operator.shape == DEPTH_BALLOON - else None - ) - balloon = extract_balloon_object( + normal_image = ( + _load_normal_image(normal_path, source_object) + if normal_path.is_file() + else None + ) + if normal_image is not None and operator.shape == DEPTH_SURFACE: + _orient_cutout_object_normal(normal_image) + material = _new_material( + context, + source_object, + color_image, + normal_image, + normal_space=( + "OBJECT" + if operator.shape == DEPTH_SURFACE and normal_image is not None + else "TANGENT" + ), + ) + field = ( + _load_field(result.file("field")) + if operator.shape in {DEPTH_PLANE, DEPTH_SURFACE} + else None + ) + depth_data = ( + load_depth_balloon_data(result.file("depth")) + if operator.shape == DEPTH_BALLOON + else None + ) + mesh_object = _build_mesh_object( + context, + source_object, + operator.shape, + operator.mesh_detail, + material, + field=field, + depth_data=depth_data, + regions=regions, + image_size=image_size, + bounds=local_bounds, + keep_original=operator.shape in CUTOUT_SHAPES, + ) + if normal_image is not None and operator.shape != DEPTH_SURFACE: + bake_tangent_normal_image( context, source_object, - regions, - source_object.matrix_world.copy(), - "POISSON", - thickness=1.0, - resolution=_cutout_point_spacing(source_object, operator.mesh_detail), - select_result=True, - reuse_material=False, - depth_data=depth_data, - normal_path=normal_path if normal_path.is_file() else None, - material_image=color_image, - local_bounds=local_bounds, - bake_normal_to_tangent=normal_path.is_file(), - ) - balloon["anyimage_mesh_shape"] = operator.shape - else: - normal_image = ( - _load_normal_image(normal_path, source_object) - if normal_path.is_file() - else None - ) - material = _new_material( - context, - source_object, - color_image, + mesh_object, normal_image, - normal_space=( - "OBJECT" - if operator.shape == DEPTH_SURFACE and normal_image is not None - else "TANGENT" - ), ) - field = ( - _load_field(result.file("field")) - if operator.shape in DEPTH_SHAPES - else None - ) - mesh_object = _build_mesh_object( - context, - source_object, - operator.shape, - operator.mesh_detail, - material, - field=field, - regions=regions, - image_size=image_size, - bounds=local_bounds, - keep_original=operator.shape in CUTOUT_SHAPES, - ) - if normal_image is not None and operator.shape == CUTOUT: - bake_tangent_normal_image( - context, - source_object, - mesh_object, - normal_image, - ) push_undo(f"Convert to {operator.shape.replace('_', ' ').title()}") + def _depth_settings(context, shape): preferences = addon_preferences(context) model = getattr(preferences, "mesh_depth_model", DEFAULT_MOGE2_MODEL_KEY) diff --git a/src/anyimage/preferences.py b/src/anyimage/preferences.py index acdf0c2..674ea20 100644 --- a/src/anyimage/preferences.py +++ b/src/anyimage/preferences.py @@ -18,6 +18,15 @@ UPSCALE_MODEL_ITEMS = tuple( for index, model in enumerate(UPSCALE_MODELS.values()) ) +BALLOON_PROFILE_ALGORITHM_ITEMS = ( + ( + "POISSON", + "Poisson", + "Smooth cotangent-Laplacian Balloon profile", + ), + ("TEDDY", "Teddy", "Chordal-axis Balloon profile"), +) + def addon_preferences(context=None): preference_context = ( @@ -85,6 +94,12 @@ class AnyImagePreferences(bpy.types.AddonPreferences): step=256, subtype="PIXEL", ) + balloon_profile_algorithm: bpy.props.EnumProperty( + name="Balloon Profile", + description="Algorithm used to generate o_balloon for every Cutout shape", + items=BALLOON_PROFILE_ALGORITHM_ITEMS, + default="POISSON", + ) mesh_depth_model: bpy.props.EnumProperty( name="Mesh Depth Model", description="Depth model used by Depth Plane, Depth Balloon and Depth Surface", @@ -114,20 +129,12 @@ class AnyImagePreferences(bpy.types.AddonPreferences): from .runtime import runtime layout = self.layout + status = ai_status() + models = {model["key"]: model for model in model_catalog()} + + self._draw_section(layout, "Environment") layout.prop(self, "storage_root") layout.prop(self, "device") - layout.prop(self, "upscale_model") - layout.prop(self, "upscale_max_output_size") - layout.separator(type="LINE") - layout.label(text="Mesh Conversion") - layout.prop(self, "mesh_depth_model") - mesh_depth_model = getattr(self, "mesh_depth_model", DEFAULT_MOGE2_MODEL_KEY) - if mesh_depth_model.startswith("MOGE2_"): - layout.prop(self, "mesh_resolution_level") - else: - layout.prop(self, "mesh_process_res_factor") - layout.separator() - status = ai_status() layout.label(text=f"Environment: {runtime.environment_directory()}") if not status["environment_ready"]: layout.label(text="Environment is not installed", icon="ERROR") @@ -148,30 +155,109 @@ class AnyImagePreferences(bpy.types.AddonPreferences): text=ai_setup_label(status), icon="IMPORT", ) - layout.separator(type="LINE") - layout.label(text="Models") - models = model_catalog() - for family in ("moge2", "da3", "ben2", "upscale"): - for model in models: - if model.get("family") == family and ( - family != "da3" or model.get("enabled") - ): - self._draw_model(layout, model) - layout.separator() + + self._draw_section(layout, "Diagnostics") layout.operator( "anyimage.open_server_log", text="Open Server Log", icon="TEXT", ) - def _draw_model(self, layout, model): + section = self._draw_panel(layout, "depth_normal", "Depth & Normal") + if section is not None: + self._draw_depth_normal( + section, + models, + status["environment_ready"], + ) + + section = self._draw_panel( + layout, + "remove_background", + "Remove Background", + ) + if section is not None: + self._draw_model( + section, + models.get("BEN2_BASE"), + status["environment_ready"], + ) + + section = self._draw_panel(layout, "upscale", "Upscale") + if section is not None: + self._draw_upscale( + section, + models, + status["environment_ready"], + ) + + section = self._draw_panel(layout, "cutout", "Cutout") + if section is not None: + section.prop(self, "balloon_profile_algorithm") + + def _draw_depth_normal(self, layout, models, environment_ready): + layout.prop(self, "mesh_depth_model") + mesh_depth_model = getattr( + self, "mesh_depth_model", DEFAULT_MOGE2_MODEL_KEY + ) + if mesh_depth_model not in models: + mesh_depth_model = DEFAULT_MOGE2_MODEL_KEY + self._draw_model( + layout, + models.get(mesh_depth_model), + environment_ready, + ) + if mesh_depth_model.startswith("MOGE2_"): + layout.prop(self, "mesh_resolution_level") + else: + layout.prop(self, "mesh_process_res_factor") + layout.label( + text="DA3 models cannot generate normals", + icon="INFO", + ) + + def _draw_upscale(self, layout, models, environment_ready): + layout.prop(self, "upscale_model") + upscale_model = getattr( + self, "upscale_model", DEFAULT_UPSCALE_MODEL_KEY + ) + if upscale_model not in models: + upscale_model = DEFAULT_UPSCALE_MODEL_KEY + self._draw_model( + layout, + models.get(upscale_model), + environment_ready, + ) + layout.prop(self, "upscale_max_output_size") + + @staticmethod + def _draw_section(layout, label): + layout.separator(type="LINE") + layout.label(text=label) + + @staticmethod + def _draw_panel(layout, panel_id, label): + header, body = layout.panel( + f"anyimage_preferences_{panel_id}", + default_closed=False, + ) + header.label(text=label) + return body + + def _draw_model(self, layout, model, environment_ready=True): from .runtime import runtime + if model is None: + layout.label(text="Model is unavailable", icon="ERROR") + return row = layout.row(align=True) row.label(text=model["label"]) if model.get("ready"): row.label(text="Ready", icon="CHECKMARK") return + if not environment_ready: + row.label(text="Install Environment First", icon="INFO") + return download_row = row.row(align=True) download_row.enabled = not runtime.server_busy() operator = download_row.operator( diff --git a/src/anyimage/properties.py b/src/anyimage/properties.py index c71594e..aa680df 100644 --- a/src/anyimage/properties.py +++ b/src/anyimage/properties.py @@ -187,23 +187,13 @@ def moge2_model_enum_items(_owner, _context): ) -def balloon_depth_model_enum_items(_owner, _context): - models = [ - model - for model in model_catalog() - if model.get("family") == "moge2" and model.get("enabled") - ] - models.sort(key=lambda model: model.get("key") != DEFAULT_MOGE2_MODEL_KEY) - return _model_items(models) - - def depth_plane_model_enum_items(_owner, _context): models = [ model for model in model_catalog() if model.get("family") in {"moge2", "da3"} and model.get("enabled") ] - models.sort(key=lambda model: model.get("key") != DEFAULT_MOGE2_MODEL_KEY) + models.sort(key=lambda model: model.get("family") != "moge2") items = _model_items(models) items.append( ("ORIGINAL", "Original", "Use the original image as depth", len(items)) @@ -217,7 +207,7 @@ def surface_model_enum_items(_owner, _context): for model in model_catalog() if model.get("family") in {"moge2", "da3"} and model.get("enabled") ] - models.sort(key=lambda model: model.get("key") != DEFAULT_MOGE2_MODEL_KEY) + models.sort(key=lambda model: model.get("family") != "moge2") return _model_items(models) @@ -250,74 +240,7 @@ def upscale_model_label(model_key): return model.label if model is not None else model_key -BALLOON_MODE_ITEMS = ( - ( - "POISSON", - "Poisson", - "Use a global Poisson field for smooth inflation", - ), - ( - "TEDDY", - "Teddy / Chordal Axis", - "Use a Teddy-style chordal axis and local oval sections", - ), -) - - -def balloon_mode_property(): - return bpy.props.EnumProperty( - name="Mode", - description="Algorithm used to inflate the saved lasso region", - items=BALLOON_MODE_ITEMS, - default="POISSON", - ) - - -def balloon_thickness_property(): - return bpy.props.FloatProperty( - name="Thickness", - description="Scale the Balloon thickness while keeping the lasso region", - default=1.0, - min=0.05, - soft_max=5.0, - ) - - -def balloon_resolution_property(): - return bpy.props.FloatProperty( - name="Resolution (m)", - description=( - "Distance in meters between sampled mesh points; smaller values " - "create more geometry" - ), - default=0.1, - min=0.005, - soft_min=0.02, - soft_max=0.5, - max=10.0, - ) - - -def balloon_reuse_material_property(): - return bpy.props.BoolProperty( - name="Reuse Material", - description="Reuse a compatible Balloon material made from the same image", - default=True, - ) - - -def balloon_contour_analysis_property(): - return bpy.props.BoolProperty( - name="Contour", - description=( - "Use BEN2 Alpha above 0.5 to analyze the lasso contour before " - "building the Balloon" - ), - default=False, - ) - - -def image_edit_refine_selection_property(): +def refine_selection_property(): return bpy.props.BoolProperty( name="Refine Selection", description="Use BEN2 to fit the gesture mask to foreground content", @@ -332,6 +255,7 @@ def cutout_normal_analysis_property(): default=False, ) + def cutout_point_spacing_property(): return bpy.props.IntProperty( name="Point Spacing", @@ -345,45 +269,12 @@ def cutout_point_spacing_property(): ) -def balloon_depth_analysis_property(): - return bpy.props.BoolProperty( - name="Depth", - description=("Use MoGe-2 to add adjustable image depth to the Balloon mesh"), - default=False, - ) - - -def balloon_depth_model_property(): - return bpy.props.EnumProperty( - name="Depth Model", - description="MoGe-2 model used for Balloon depth", - items=balloon_depth_model_enum_items, - default=0, - ) - - -def balloon_radial_zoom_property(scale): - return bpy.props.FloatVectorProperty( - size=2, - default=(scale, scale), - min=0.01, - options={"HIDDEN", "SKIP_SAVE"}, - ) - - class AnyImageSettings(bpy.types.PropertyGroup): cutout_point_spacing: cutout_point_spacing_property() cutout_normal_analysis: cutout_normal_analysis_property() - image_edit_refine_selection: image_edit_refine_selection_property() - balloon_mode: balloon_mode_property() - balloon_thickness: balloon_thickness_property() - balloon_resolution: balloon_resolution_property() - balloon_reuse_material: balloon_reuse_material_property() - balloon_contour_analysis: balloon_contour_analysis_property() - balloon_depth_analysis: balloon_depth_analysis_property() - balloon_depth_model: balloon_depth_model_property() - balloon_thickness_radial_zoom: balloon_radial_zoom_property(100.0) - balloon_resolution_radial_zoom: balloon_radial_zoom_property(1000.0) + cutout_refine_selection: refine_selection_property() + image_box_refine_selection: refine_selection_property() + image_lasso_refine_selection: refine_selection_property() input_path: bpy.props.StringProperty( name="Image or Video", description="A single image or video file used for reconstruction", diff --git a/src/anyimage/server/jobs/balloon.py b/src/anyimage/server/jobs/balloon.py deleted file mode 100644 index 6019fe8..0000000 --- a/src/anyimage/server/jobs/balloon.py +++ /dev/null @@ -1,426 +0,0 @@ -import json -import math -import time -from pathlib import Path - -from ..models import moge2 -from ..models.ben2 import infer_foreground -from .image_mask import polygon_selection_mask - - -ALPHA_THRESHOLD = 0.5 - - -def create_lasso_input( - input_path, polygon, output_path, fill_rule="EVEN_ODD" -): - """Crop a lasso and clear every RGBA channel outside its polygon.""" - import numpy as np - from PIL import Image - - points = np.asarray(polygon, dtype=np.float64) - if points.ndim != 2 or points.shape[0] < 3 or points.shape[1] != 2: - raise ValueError("The lasso does not define a valid image region") - if not np.isfinite(points).all(): - raise ValueError("The lasso contains invalid image coordinates") - - with Image.open(input_path) as opened_image: - source = opened_image.convert("RGBA") - width, height = source.size - left = max(0, int(math.floor(float(points[:, 0].min())))) - top = max(0, int(math.floor(float(points[:, 1].min())))) - right = min(width, int(math.ceil(float(points[:, 0].max())))) - bottom = min(height, int(math.ceil(float(points[:, 1].max())))) - if right <= left or bottom <= top: - raise ValueError("The lasso does not overlap the source image") - - relative = points - np.asarray((left, top), dtype=np.float64) - cropped = source.crop((left, top, right, bottom)) - selection_mask = polygon_selection_mask(cropped.size, relative, fill_rule) - pixels = np.asarray(cropped, dtype=np.uint8).copy() - pixels[~selection_mask] = 0 - masked = Image.fromarray(pixels, mode="RGBA") - output_path = Path(output_path) - masked.save(output_path, format="PNG", compress_level=1) - return output_path, selection_mask, (left, top), (width, height) - - -def largest_component(mask): - """Return the largest four-connected component of a boolean image.""" - import numpy as np - - mask = np.asarray(mask, dtype=bool) - if mask.ndim != 2: - raise ValueError("The BEN2 Alpha mask must be two-dimensional") - height, width = mask.shape - visited = np.zeros_like(mask) - largest = [] - for start_y, start_x in np.argwhere(mask): - if visited[start_y, start_x]: - continue - visited[start_y, start_x] = True - pending = [(int(start_x), int(start_y))] - component = [] - while pending: - x, y = pending.pop() - component.append((x, y)) - for next_x, next_y in ( - (x - 1, y), - (x + 1, y), - (x, y - 1), - (x, y + 1), - ): - if ( - 0 <= next_x < width - and 0 <= next_y < height - and mask[next_y, next_x] - and not visited[next_y, next_x] - ): - visited[next_y, next_x] = True - pending.append((next_x, next_y)) - if len(component) > len(largest): - largest = component - if not largest: - raise ValueError("BEN2 found no foreground inside the lasso") - result = np.zeros_like(mask) - component_x, component_y = zip(*largest) - result[component_y, component_x] = True - return result - - -def _choose_boundary_edge(start, direction, candidates): - right = (-direction[1], direction[0]) - left = (direction[1], -direction[0]) - back = (-direction[0], -direction[1]) - preference = {right: 0, direction: 1, left: 2, back: 3} - return min( - candidates, - key=lambda end: preference.get( - (end[0] - start[0], end[1] - start[1]), - 4, - ), - ) - - -def _simplify_axis_aligned_contour(points): - simplified = [] - for index, current in enumerate(points): - previous = points[index - 1] - following = points[(index + 1) % len(points)] - first = (current[0] - previous[0], current[1] - previous[1]) - second = (following[0] - current[0], following[1] - current[1]) - if first[0] * second[1] - first[1] * second[0] != 0: - simplified.append(current) - return simplified - - -def trace_component_contour(component): - """Trace the outer pixel-edge loop of one four-connected component.""" - import numpy as np - - component = np.asarray(component, dtype=bool) - height, width = component.shape - edges = {} - - def add_edge(start, end): - edges.setdefault(start, []).append(end) - - for y, x in np.argwhere(component): - x = int(x) - y = int(y) - if y == 0 or not component[y - 1, x]: - add_edge((x, y), (x + 1, y)) - if x == width - 1 or not component[y, x + 1]: - add_edge((x + 1, y), (x + 1, y + 1)) - if y == height - 1 or not component[y + 1, x]: - add_edge((x + 1, y + 1), (x, y + 1)) - if x == 0 or not component[y, x - 1]: - add_edge((x, y + 1), (x, y)) - if not edges: - raise ValueError("BEN2 foreground has no usable boundary") - - start = min(edges, key=lambda point: (point[1], point[0])) - current = start - direction = (1, 0) - contour = [] - edge_count = sum(len(values) for values in edges.values()) - for _index in range(edge_count + 1): - contour.append(current) - candidates = edges.get(current, ()) - if not candidates: - raise ValueError("BEN2 foreground boundary is open") - following = _choose_boundary_edge(current, direction, candidates) - direction = (following[0] - current[0], following[1] - current[1]) - current = following - if current == start: - break - else: - raise ValueError("BEN2 foreground boundary could not be closed") - contour = _simplify_axis_aligned_contour(contour) - if len(contour) < 3: - raise ValueError("BEN2 foreground is too small for a Balloon") - return contour - - -def contour_from_alpha(alpha, selection_mask, origin=(0, 0)): - """Build a lasso contour from BEN2 Alpha values strictly above 0.5.""" - import numpy as np - - alpha = np.asarray(alpha, dtype=np.float32) - selection_mask = np.asarray(selection_mask, dtype=bool) - if alpha.shape != selection_mask.shape: - raise ValueError("BEN2 Alpha does not match the lasso input") - foreground = (alpha / 255.0 > ALPHA_THRESHOLD) & selection_mask - component = largest_component(foreground) - contour = trace_component_contour(component) - offset_x, offset_y = origin - return [[x + offset_x, y + offset_y] for x, y in contour] - - -def resized_selection_mask(selection_mask, shape): - import numpy as np - from PIL import Image - - selection_mask = np.asarray(selection_mask, dtype=bool) - if selection_mask.shape == tuple(shape): - return selection_mask - resized = Image.fromarray(selection_mask.astype(np.uint8) * 255, mode="L") - resized = resized.resize( - (int(shape[1]), int(shape[0])), - Image.Resampling.NEAREST, - ) - return np.asarray(resized, dtype=np.uint8) > 0 - - -def normalized_moge_depth(depth, model_mask, selection_mask): - """Normalize MoGe depth while preserving its adjustable fitted base plane.""" - import numpy as np - - depth = np.asarray(depth, dtype=np.float32) - model_mask = np.asarray(model_mask, dtype=bool) - selection_mask = resized_selection_mask(selection_mask, depth.shape) - if model_mask.shape != depth.shape: - raise ValueError("MoGe-2 depth and mask dimensions do not match") - valid = selection_mask & model_mask & np.isfinite(depth) & (depth > 0.0) - if not valid.any(): - raise ValueError("MoGe-2 found no valid depth inside the lasso") - - height, width = depth.shape - y, x = np.indices(depth.shape, dtype=np.float64) - x = x / max(width - 1, 1) * 2.0 - 1.0 - y = y / max(height - 1, 1) * 2.0 - 1.0 - design = np.stack((x[valid], y[valid], np.ones(valid.sum())), axis=1) - selected = depth[valid].astype(np.float64) - weights = np.ones_like(selected) - coefficients = np.zeros(3, dtype=np.float64) - for _iteration in range(4): - weighted_design = design * np.sqrt(weights)[:, None] - weighted_depth = selected * np.sqrt(weights) - coefficients = np.linalg.lstsq( - weighted_design, - weighted_depth, - rcond=None, - )[0] - residual = selected - design @ coefficients - centered = residual - np.median(residual) - scale = max(float(np.median(np.abs(centered))) * 1.4826, 1e-6) - weights = np.minimum( - 1.0, - (2.5 * scale) / np.maximum(np.abs(centered), 1e-12), - ) - - baseline = coefficients[0] * x + coefficients[1] * y + coefficients[2] - reference = float(np.median(baseline[valid])) - residual = baseline[valid] - selected - near_residual = np.maximum(residual, 0.0) - near_total = float(np.sum(near_residual)) - relative_span = ( - float(np.sum(np.square(near_residual))) / near_total - if near_total > 1e-12 - else float(np.sqrt(np.mean(np.square(residual)))) - ) - relative_span = max(relative_span, 1e-6) - depth_signal = np.zeros(depth.shape, dtype=np.float32) - depth_signal[valid] = (reference - selected) / relative_span - baseline_vector = np.asarray( - ( - -coefficients[0] / relative_span, - -coefficients[1] / relative_span, - (reference - coefficients[2]) / relative_span, - ), - dtype=np.float32, - ) - return depth_signal, valid, reference, baseline_vector - - -def write_depth_analysis( - prediction, - selection_mask, - origin, - source_size, - output_path, -): - import numpy as np - - if prediction.depth.shape[0] != 1 or prediction.mask.shape[0] != 1: - raise ValueError("Balloon Depth requires one MoGe-2 frame") - depth_signal, valid, reference, baseline_vector = normalized_moge_depth( - prediction.depth[0], - prediction.mask[0], - selection_mask, - ) - crop_size = (selection_mask.shape[1], selection_mask.shape[0]) - np.savez_compressed( - output_path, - depth_signal=depth_signal, - baseline_vector=baseline_vector, - valid=valid, - reference_depth=np.asarray(reference, dtype=np.float32), - origin=np.asarray(origin, dtype=np.int32), - crop_size=np.asarray(crop_size, dtype=np.int32), - source_size=np.asarray(source_size, dtype=np.int32), - ) - return Path(output_path) - - -def write_camera_normal_texture( - normal, - valid, - origin, - crop_size, - source_size, - output_path, -): - """Encode MoGe camera-axis normals into a full-image RGB texture.""" - import numpy as np - from PIL import Image - - normal = np.asarray(normal, dtype=np.float32) - valid = np.asarray(valid, dtype=bool) - if normal.ndim != 3 or normal.shape[2] != 3: - raise ValueError("Balloon normal data must have three components") - if valid.shape != normal.shape[:2]: - valid = resized_selection_mask(valid, normal.shape[:2]) - encoded = np.clip( - (normal * np.asarray((1.0, -1.0, -1.0), dtype=np.float32) + 1.0) - * 127.5, - 0.0, - 255.0, - ).astype(np.uint8) - encoded[~valid] = (128, 128, 255) - crop_width, crop_height = (int(value) for value in crop_size) - crop = Image.fromarray(encoded, mode="RGB") - if crop.size != (crop_width, crop_height): - crop = crop.resize( - (crop_width, crop_height), - Image.Resampling.BILINEAR, - ) - source_width, source_height = (int(value) for value in source_size) - full = Image.new("RGB", (source_width, source_height), (128, 128, 255)) - full.paste(crop, tuple(int(value) for value in origin)) - output_path = Path(output_path) - full.save(output_path) - return output_path - - -def run(context, parameters): - """Analyze a Balloon lasso with optional BEN2 contour and MoGe-2 depth.""" - total_started = time.perf_counter() - contour_analysis = bool(parameters.get("contour_analysis")) - depth_analysis = bool(parameters.get("depth_analysis")) - if not contour_analysis and not depth_analysis: - raise ValueError("Select Contour or Depth") - - context.progress(0.02, "Preparing lasso input") - context.check_cancelled() - output_directory = context.directory - output_directory.mkdir(parents=True, exist_ok=True) - input_path, selection_mask, origin, source_size = create_lasso_input( - parameters["input"], - parameters["polygon"], - output_directory / "lasso-input.png", - ) - context.check_cancelled() - - device = parameters["device"] - model_manager = context.resource("model_manager") - result = {} - if contour_analysis: - model_directory = model_manager.directory(parameters["contour_model"]) - context.progress(0.15, f"Loading BEN2 for {device.upper()}") - foreground, _load_ms, _inference_ms = infer_foreground( - input_path, - model_directory, - device, - model_cache=model_manager, - cancel_check=context.check_cancelled, - ) - context.check_cancelled() - context.progress( - 0.55 if depth_analysis else 0.9, - "Tracing BEN2 foreground", - ) - import numpy as np - from PIL import Image - - alpha = np.asarray(foreground.getchannel("A"), dtype=np.uint8) - contour = contour_from_alpha(alpha, selection_mask, origin) - gated_alpha = np.where(selection_mask, alpha, 0).astype(np.uint8) - foreground_pixels = foreground.convert("RGBA") - foreground_pixels.putalpha(Image.fromarray(gated_alpha, mode="L")) - foreground_pixels.save( - output_directory / "foreground.png", - format="PNG", - compress_level=1, - ) - contour_path = output_directory / "contour.json" - contour_path.write_text( - json.dumps({"points": contour}, separators=(",", ":")), - encoding="utf-8", - ) - result["contour"] = "contour.json" - - if depth_analysis: - context.progress( - 0.12 if not contour_analysis else 0.58, - "Estimating Balloon depth with MoGe-2", - ) - prediction = moge2.infer( - model_manager, - { - "model": parameters["depth_model"], - "device": device, - "resolution_level": parameters.get("resolution_level", 5), - }, - [input_path], - include_points=False, - progress_callback=None, - cancel_check=context.check_cancelled, - ) - context.check_cancelled() - context.progress(0.9, "Writing Balloon depth") - depth_path = write_depth_analysis( - prediction, - selection_mask, - origin, - source_size, - output_directory / "depth.npz", - ) - result["depth"] = depth_path.name - import numpy as np - - with np.load(depth_path, allow_pickle=False) as data: - normal_path = write_camera_normal_texture( - prediction.normal[0], - data["valid"], - origin, - (selection_mask.shape[1], selection_mask.shape[0]), - source_size, - output_directory / "normal.png", - ) - result["normal"] = normal_path.name - - total_ms = (time.perf_counter() - total_started) * 1000.0 - message = f"Balloon AI analysis finished in {total_ms:.0f} ms" - context.progress(1.0, message) - return result diff --git a/src/anyimage/server/jobs/cutout_depth.py b/src/anyimage/server/jobs/cutout_depth.py new file mode 100644 index 0000000..afa8d8a --- /dev/null +++ b/src/anyimage/server/jobs/cutout_depth.py @@ -0,0 +1,176 @@ +from pathlib import Path + + +PLANE_VARIANCE_FLOOR_RATIO = 0.025 + + +def resized_selection_mask(selection_mask, shape): + import numpy as np + from PIL import Image + + selection_mask = np.asarray(selection_mask, dtype=bool) + if selection_mask.shape == tuple(shape): + return selection_mask + resized = Image.fromarray(selection_mask.astype(np.uint8) * 255, mode="L") + resized = resized.resize( + (int(shape[1]), int(shape[0])), + Image.Resampling.NEAREST, + ) + return np.asarray(resized, dtype=np.uint8) > 0 + + +def _stable_weighted_plane(coordinates, values, weights): + import numpy as np + + total = float(np.sum(weights)) + if total <= 1e-12: + return np.zeros(3, dtype=np.float64) + center = np.sum(coordinates * weights[:, None], axis=0) / total + value_center = float(np.sum(values * weights) / total) + centered = coordinates - center + covariance = (centered * weights[:, None]).T @ centered / total + cross = centered.T @ (weights * (values - value_center)) / total + eigenvalues, eigenvectors = np.linalg.eigh(covariance) + largest = max(float(eigenvalues[-1]), 0.0) + if largest <= 1e-12: + slopes = np.zeros(2, dtype=np.float64) + else: + supported = np.maximum(eigenvalues, largest * PLANE_VARIANCE_FLOOR_RATIO) + slopes = eigenvectors @ ((eigenvectors.T @ cross) / supported) + intercept = value_center - float(center @ slopes) + return np.asarray((*slopes, intercept), dtype=np.float64) + + +def normalized_moge_depth(depth, model_mask, selection_mask): + """Normalize MoGe depth while preserving its adjustable fitted base plane.""" + import numpy as np + + depth = np.asarray(depth, dtype=np.float32) + model_mask = np.asarray(model_mask, dtype=bool) + selection_mask = resized_selection_mask(selection_mask, depth.shape) + if model_mask.shape != depth.shape: + raise ValueError("MoGe-2 depth and mask dimensions do not match") + valid = selection_mask & model_mask & np.isfinite(depth) & (depth > 0.0) + if not valid.any(): + raise ValueError("MoGe-2 found no valid depth inside the lasso") + + height, width = depth.shape + y, x = np.indices(depth.shape, dtype=np.float64) + x = x / max(width - 1, 1) * 2.0 - 1.0 + y = y / max(height - 1, 1) * 2.0 - 1.0 + coordinates = np.stack((x[valid], y[valid]), axis=1) + selected = depth[valid].astype(np.float64) + weights = np.ones_like(selected) + coefficients = np.zeros(3, dtype=np.float64) + for _iteration in range(4): + coefficients = _stable_weighted_plane( + coordinates, + selected, + weights, + ) + residual = ( + selected + - coordinates @ coefficients[:2] + - coefficients[2] + ) + centered = residual - np.median(residual) + scale = max(float(np.median(np.abs(centered))) * 1.4826, 1e-6) + weights = np.minimum( + 1.0, + (2.5 * scale) / np.maximum(np.abs(centered), 1e-12), + ) + + baseline = coefficients[0] * x + coefficients[1] * y + coefficients[2] + reference = float(np.median(baseline[valid])) + residual = baseline[valid] - selected + near_residual = np.maximum(residual, 0.0) + near_total = float(np.sum(near_residual)) + relative_span = ( + float(np.sum(np.square(near_residual))) / near_total + if near_total > 1e-12 + else float(np.sqrt(np.mean(np.square(residual)))) + ) + relative_span = max(relative_span, 1e-6) + depth_signal = np.zeros(depth.shape, dtype=np.float32) + depth_signal[valid] = (reference - selected) / relative_span + baseline_vector = np.asarray( + ( + -coefficients[0] / relative_span, + -coefficients[1] / relative_span, + (reference - coefficients[2]) / relative_span, + ), + dtype=np.float32, + ) + return depth_signal, valid, reference, baseline_vector + + +def write_depth_analysis( + prediction, + selection_mask, + origin, + source_size, + output_path, +): + import numpy as np + + if prediction.depth.shape[0] != 1 or prediction.mask.shape[0] != 1: + raise ValueError("Depth Balloon requires one MoGe-2 frame") + depth_signal, valid, reference, baseline_vector = normalized_moge_depth( + prediction.depth[0], + prediction.mask[0], + selection_mask, + ) + crop_size = (selection_mask.shape[1], selection_mask.shape[0]) + np.savez_compressed( + output_path, + depth_signal=depth_signal, + baseline_vector=baseline_vector, + valid=valid, + reference_depth=np.asarray(reference, dtype=np.float32), + origin=np.asarray(origin, dtype=np.int32), + crop_size=np.asarray(crop_size, dtype=np.int32), + source_size=np.asarray(source_size, dtype=np.int32), + ) + return Path(output_path) + + +def write_camera_normal_texture( + normal, + valid, + origin, + crop_size, + source_size, + output_path, +): + """Encode MoGe camera-axis normals into a full-image RGB texture.""" + import numpy as np + from PIL import Image + + normal = np.asarray(normal, dtype=np.float32) + valid = np.asarray(valid, dtype=bool) + if normal.ndim != 3 or normal.shape[2] != 3: + raise ValueError("Depth Balloon normal data must have three components") + if valid.shape != normal.shape[:2]: + valid = resized_selection_mask(valid, normal.shape[:2]) + encoded = np.clip( + (normal * np.asarray((1.0, -1.0, -1.0), dtype=np.float32) + 1.0) + * 127.5, + 0.0, + 255.0, + ).astype(np.uint8) + encoded[~valid] = (128, 128, 255) + crop_width, crop_height = (int(value) for value in crop_size) + crop = Image.fromarray(encoded, mode="RGB") + if crop.size != (crop_width, crop_height): + crop = crop.resize( + (crop_width, crop_height), + Image.Resampling.BILINEAR, + ) + source_width, source_height = (int(value) for value in source_size) + full = Image.new("RGB", (source_width, source_height), (128, 128, 255)) + full.paste(crop, tuple(int(value) for value in origin)) + output_path = Path(output_path) + full.save(output_path) + return output_path + + diff --git a/src/anyimage/server/jobs/mesh.py b/src/anyimage/server/jobs/mesh.py index 03e8c7e..427cadd 100644 --- a/src/anyimage/server/jobs/mesh.py +++ b/src/anyimage/server/jobs/mesh.py @@ -3,7 +3,7 @@ import json from ..media import copy_color_images from ..models import geometry_adapter, moge2 from .background import remove_image_background -from .balloon import ( +from .cutout_depth import ( write_camera_normal_texture, write_depth_analysis, ) @@ -247,7 +247,7 @@ def _write_camera_normal(context, parameters, color_path): ) -def _write_balloon_depth( +def _write_depth_balloon( context, parameters, input_path, @@ -371,7 +371,7 @@ def run(context, parameters): ).name elif shape == "DEPTH_BALLOON": context.progress(0.4, "Estimating Balloon depth") - depth_path, normal_path = _write_balloon_depth( + depth_path, normal_path = _write_depth_balloon( context, parameters, model_path, diff --git a/tests/test_addon.py b/tests/test_addon.py index f2d4fe4..3be3609 100644 --- a/tests/test_addon.py +++ b/tests/test_addon.py @@ -111,15 +111,13 @@ class AddonTest(unittest.TestCase): cls.image_perspective = importlib.import_module( "anyimage.operators.image_edit.perspective" ) - cls.balloon = importlib.import_module("anyimage.operators.balloon.operator") - cls.balloon_geometry = importlib.import_module( - "anyimage.operators.balloon.geometry" + cls.cutout_geometry = importlib.import_module( + "anyimage.operators.cutout_geometry" ) - cls.balloon_depth = importlib.import_module("anyimage.operators.balloon.depth") - cls.balloon_poisson = importlib.import_module( - "anyimage.operators.balloon.poisson" + cls.cutout_depth = importlib.import_module("anyimage.operators.cutout_depth") + cls.image_interaction = importlib.import_module( + "anyimage.operators.image_edit.interaction" ) - cls.balloon_teddy = importlib.import_module("anyimage.operators.balloon.teddy") cls.paths_after_import = tuple(sys.path) cls.fake_bpy = fake_bpy @@ -527,7 +525,7 @@ class AddonTest(unittest.TestCase): self.assertNotIn("stage", controller.post_status) self.assertNotIn("stages", controller.post_status) - def test_conversion_model_lists_default_to_moge2_vits(self): + def test_conversion_model_lists_moge2_before_da3(self): properties = importlib.import_module("anyimage.properties") catalog = ( { @@ -538,6 +536,14 @@ class AddonTest(unittest.TestCase): "description": "DA3", "license": "Apache", }, + { + "key": "BASE", + "family": "da3", + "enabled": True, + "label": "DA3 Base", + "description": "DA3", + "license": "Apache", + }, { "key": "MOGE2_VITS_NORMAL", "family": "moge2", @@ -546,15 +552,36 @@ class AddonTest(unittest.TestCase): "description": "MoGe", "license": "MIT", }, + { + "key": "MOGE2_VITB_NORMAL", + "family": "moge2", + "enabled": True, + "label": "MoGe-2 ViT-B", + "description": "MoGe", + "license": "MIT", + }, + { + "key": "MOGE2_VITL_NORMAL", + "family": "moge2", + "enabled": True, + "label": "MoGe-2 ViT-L", + "description": "MoGe", + "license": "MIT", + }, ) with patch.object(properties, "model_catalog", return_value=catalog): depth_items = properties.depth_plane_model_enum_items(None, None) surface_items = properties.surface_model_enum_items(None, None) - balloon_items = properties.balloon_depth_model_enum_items(None, None) - self.assertEqual(depth_items[0][0], "MOGE2_VITS_NORMAL") - self.assertEqual(surface_items[0][0], "MOGE2_VITS_NORMAL") - self.assertEqual([item[0] for item in balloon_items], ["MOGE2_VITS_NORMAL"]) + expected = ( + "MOGE2_VITS_NORMAL", + "MOGE2_VITB_NORMAL", + "MOGE2_VITL_NORMAL", + "SMALL", + "BASE", + ) + self.assertEqual(tuple(item[0] for item in depth_items[:-1]), expected) + self.assertEqual(tuple(item[0] for item in surface_items), expected) def test_required_model_operator_requests_both_models(self): operator = self.addon.operators.DownloadRequiredModels() @@ -664,9 +691,13 @@ class AddonTest(unittest.TestCase): self.assertFalse(hasattr(panel, "ModelsPanel")) def test_preferences_draws_upscale_model_download(self): + import inspect + preference_module = importlib.import_module("anyimage.preferences") downloads = [] properties = [] + labels = [] + panels = [] class Layout: enabled = True @@ -678,11 +709,15 @@ class AddonTest(unittest.TestCase): return None def label(self, **_options): - return None + labels.append(_options.get("text")) def row(self, **_options): return self + def panel(self, panel_id, **options): + panels.append((panel_id, options.get("default_closed"))) + return self, self + def operator(self, identifier, **_options): operator = SimpleNamespace(model="") if identifier == "anyimage.download_model": @@ -691,18 +726,22 @@ class AddonTest(unittest.TestCase): preferences = preference_module.AnyImagePreferences() preferences.layout = Layout() + preferences.mesh_depth_model = "BASE" + preferences.upscale_model = "SAFMN_REAL_X4" catalog = tuple( { "key": key, "label": key, - "family": "upscale", + "family": family, "enabled": True, "ready": False, } - for key in ( - "REALESRGAN_GENERAL_WDN_X4V3", - "SAFMN_REAL_X4", - "REALESRGAN_X4PLUS", + for key, family in ( + ("BASE", "da3"), + ("MOGE2_VITS_NORMAL", "moge2"), + ("BEN2_BASE", "ben2"), + ("REALESRGAN_GENERAL_WDN_X4V3", "upscale"), + ("SAFMN_REAL_X4", "upscale"), ) ) with ( @@ -726,10 +765,57 @@ class AddonTest(unittest.TestCase): self.assertEqual( [download.model for download in downloads], - [model["key"] for model in catalog], + ["BASE", "BEN2_BASE", "SAFMN_REAL_X4"], + ) + self.assertEqual( + [ + label + for label in labels + if label + in { + "Environment", + "Depth & Normal", + "Cutout", + "Remove Background", + "Upscale", + "Diagnostics", + } + ], + [ + "Environment", + "Diagnostics", + "Depth & Normal", + "Remove Background", + "Upscale", + "Cutout", + ], + ) + self.assertEqual( + panels, + [ + ("anyimage_preferences_depth_normal", False), + ("anyimage_preferences_remove_background", False), + ("anyimage_preferences_upscale", False), + ("anyimage_preferences_cutout", False), + ], + ) + self.assertIn("DA3 models cannot generate normals", labels) + self.assertNotIn("Normal Model", labels) + self.assertNotIn( + "separator", + inspect.getsource(preference_module.AnyImagePreferences._draw_panel), ) self.assertIn("upscale_model", properties) self.assertIn("upscale_max_output_size", properties) + self.assertIn("balloon_profile_algorithm", properties) + balloon_profile = preference_module.AnyImagePreferences.__annotations__[ + "balloon_profile_algorithm" + ] + self.assertEqual(balloon_profile["default"], "POISSON") + self.assertEqual( + tuple(item[0] for item in balloon_profile["items"]), + ("POISSON", "TEDDY"), + ) self.assertEqual( preference_module.AnyImagePreferences.__annotations__["upscale_model"][ "name" @@ -1518,10 +1604,10 @@ class AddonTest(unittest.TestCase): self.assertEqual(result_image.name, result_name) push_undo.assert_called_once_with("Remove Background") - def test_balloon_contour_is_smooth_and_evenly_resampled(self): + def test_cutout_contour_is_smooth_and_evenly_resampled(self): import numpy as np - contour = self.balloon_geometry.smooth_lasso_contour( + contour = self.cutout_geometry.smooth_lasso_contour( ((0, 0), (3, 0), (4, 0), (4, 2), (0, 2)), 0.25, ) @@ -1529,10 +1615,10 @@ class AddonTest(unittest.TestCase): lengths = np.linalg.norm(np.diff(closed, axis=0), axis=1) self.assertGreater(len(contour), 20) - self.assertGreater(self.balloon_geometry.polygon_area(contour), 0.0) + self.assertGreater(self.cutout_geometry.polygon_area(contour), 0.0) self.assertLess(float(lengths.std() / lengths.mean()), 0.15) - def test_balloon_contour_smooths_pixel_stairs_before_coarse_resampling(self): + def test_cutout_contour_smooths_pixel_stairs_before_coarse_resampling(self): import numpy as np staircase = [(0.0, 0.0)] @@ -1540,7 +1626,7 @@ class AddonTest(unittest.TestCase): staircase.append((value + 1.0, float(value))) staircase.append((value + 1.0, value + 1.0)) staircase.append((0.0, 16.0)) - smooth = self.balloon_geometry.smooth_lasso_contour(staircase, 2.0) + smooth = self.cutout_geometry.smooth_lasso_contour(staircase, 2.0) def diagonal_noise(points): points = np.asarray(points) @@ -1865,6 +1951,31 @@ class AddonTest(unittest.TestCase): self.assertNotIn("bl_cursor", tool.__dict__) self.assertIn("draw_settings", self.image_box.ImageBoxTool.__dict__) self.assertIn("draw_settings", self.image_lasso.ImageLassoTool.__dict__) + self.assertIn( + "draw_image_mask_settings", + inspect.getsource(self.image_box.ImageBoxTool.draw_settings), + ) + self.assertIn( + '"image_box_refine_selection"', + inspect.getsource(self.image_box.ImageBoxTool.draw_settings), + ) + self.assertIn( + "draw_image_mask_settings", + inspect.getsource(self.image_lasso.ImageLassoTool.draw_settings), + ) + self.assertIn( + '"image_lasso_refine_selection"', + inspect.getsource(self.image_lasso.ImageLassoTool.draw_settings), + ) + cutout_tool = importlib.import_module("anyimage.operators.cutout").CutoutTool + self.assertIn( + "draw_image_mask_settings", + inspect.getsource(cutout_tool.draw_settings), + ) + self.assertIn( + '"cutout_refine_selection"', + inspect.getsource(cutout_tool.draw_settings), + ) self.assertNotIn("draw_settings", self.image_line.ImageLineTool.__dict__) self.assertNotIn( "draw_settings", self.image_polyline.ImagePolylineTool.__dict__ @@ -1874,7 +1985,14 @@ class AddonTest(unittest.TestCase): self.assertNotIn("continuous", operator.__annotations__) self.assertNotIn("padding", operator.__annotations__) settings = self.addon.properties.AnyImageSettings.__annotations__ - self.assertFalse(settings["image_edit_refine_selection"]["default"]) + for name in ( + "cutout_refine_selection", + "image_box_refine_selection", + "image_lasso_refine_selection", + ): + self.assertFalse(settings[name]["default"]) + self.assertNotIn("refine_selection", settings) + self.assertNotIn("image_edit_refine_selection", settings) self.assertEqual(settings["cutout_point_spacing"]["default"], 32) self.assertEqual(settings["cutout_point_spacing"]["min"], 4) self.assertEqual(settings["cutout_point_spacing"]["max"], 32) @@ -1892,13 +2010,22 @@ class AddonTest(unittest.TestCase): self.assertIn('event.type == "F"', modal_source) self.assertIn("self.invert = not self.invert", modal_source) - self.assertIn("math.dist(point, self._path[-1]) >= 2.0", modal_source) + self.assertIn("append_lasso_point(self._path, point)", modal_source) + lasso_source = inspect.getsource(self.image_mask.append_lasso_point) + self.assertIn( + "math.dist(point, path[-1]) < LASSO_POINT_DISTANCE", + lasso_source, + ) mask_source = inspect.getsource(self.image_mask.polygon_mask) self.assertNotIn("np.indices", mask_source) self.assertIn("range(top, bottom, 128)", mask_source) - def test_image_line_and_polyline_disable_refine_selection(self): - settings = SimpleNamespace(image_edit_refine_selection=True) + def test_refine_selection_is_independent_for_each_area_tool(self): + settings = SimpleNamespace( + cutout_refine_selection=True, + image_box_refine_selection=False, + image_lasso_refine_selection=True, + ) context = SimpleNamespace(scene=SimpleNamespace(anyimage_settings=settings)) with patch.object(self.image_edit_common, "ai_ready", return_value=True): for gesture in ("LINE", "POLYLINE"): @@ -1909,9 +2036,21 @@ class AddonTest(unittest.TestCase): self.image_edit_common.apply_image_mask_settings(operator, context) self.assertFalse(operator.refine_selection) - operator = SimpleNamespace(gesture="BOX", refine_selection=False) - self.image_edit_common.apply_image_mask_settings(operator, context) - self.assertTrue(operator.refine_selection) + box = SimpleNamespace(gesture="BOX", refine_selection=True) + self.image_edit_common.apply_image_mask_settings(box, context) + self.assertFalse(box.refine_selection) + + lasso = SimpleNamespace(gesture="LASSO", refine_selection=False) + self.image_edit_common.apply_image_mask_settings(lasso, context) + self.assertTrue(lasso.refine_selection) + + cutout = SimpleNamespace( + gesture="LASSO", + active_tool_id="anyimage.cutout_lasso", + refine_selection=False, + ) + self.image_edit_common.apply_image_mask_settings(cutout, context) + self.assertTrue(cutout.refine_selection) def test_image_line_click_is_cancelled_before_projection(self): import inspect @@ -2392,11 +2531,11 @@ class AddonTest(unittest.TestCase): self.assertEqual(self.image_edit_common.polygon_bounds(preview), (0, 5, 10, 10)) self.assertEqual(self.image_edit_common.polygon_bounds(image), (0, 0, 10, 5)) - def test_balloon_interior_points_have_uniform_spacing(self): + def test_cutout_interior_points_have_uniform_spacing(self): import numpy as np contour = np.asarray(((0, 0), (4, 0), (4, 4), (0, 4)), dtype=float) - points = self.balloon_geometry.uniform_interior_points(contour, 0.5) + points = self.cutout_geometry.uniform_interior_points(contour, 0.5) distances = np.linalg.norm(points[:, None] - points[None, :], axis=2) np.fill_diagonal(distances, np.inf) nearest = distances.min(axis=1) @@ -2405,14 +2544,14 @@ class AddonTest(unittest.TestCase): self.assertAlmostEqual(float(np.median(nearest)), 0.5, delta=0.02) self.assertGreater(float(nearest.min()), 0.49) - def test_balloon_default_sampling_stays_coarse(self): + def test_cutout_default_sampling_stays_coarse(self): import numpy as np polygon = np.asarray( ((100, 100), (900, 100), (950, 500), (800, 900), (200, 900), (50, 500)), dtype=float, ) - contour, interior, _spacing = self.balloon_geometry.prepare_planar_points( + contour, interior, _spacing = self.cutout_geometry.prepare_planar_points( polygon, (1000, 1000), (0.0, 1.0, 0.0, 1.0), @@ -2421,7 +2560,7 @@ class AddonTest(unittest.TestCase): self.assertLess(len(contour), 150) self.assertLess(len(interior), 400) - def test_balloon_sampling_uses_world_distance(self): + def test_cutout_sampling_uses_world_distance(self): import numpy as np polygon = np.asarray( @@ -2429,14 +2568,14 @@ class AddonTest(unittest.TestCase): dtype=float, ) unit_contour, unit_interior, unit_spacing = ( - self.balloon_geometry.prepare_planar_points( + self.cutout_geometry.prepare_planar_points( polygon, (1000, 1000), (0.0, 1.0, 0.0, 1.0), ) ) scaled_contour, scaled_interior, scaled_spacing = ( - self.balloon_geometry.prepare_planar_points( + self.cutout_geometry.prepare_planar_points( polygon, (1000, 1000), (0.0, 2.0, 0.0, 2.0), @@ -2449,697 +2588,6 @@ class AddonTest(unittest.TestCase): self.assertEqual(len(unit_contour), len(scaled_contour)) self.assertEqual(len(unit_interior), len(scaled_interior)) - def test_poisson_balloon_is_zero_on_boundary_and_positive_inside(self): - import numpy as np - - points = np.asarray( - ((0, 0), (2, 0), (2, 2), (0, 2), (1, 1)), - dtype=float, - ) - faces = ((0, 1, 4), (1, 2, 4), (2, 3, 4), (3, 0, 4)) - boundary = np.asarray((True, True, True, True, False)) - heights = self.balloon_poisson.poisson_balloon_heights( - points, faces, boundary, 0.45 - ) - - self.assertTrue((heights[:4] == 0.0).all()) - self.assertGreater(heights[4], 0.0) - - def test_poisson_balloon_scales_in_world_units(self): - import numpy as np - - points = np.asarray( - ((0, 0), (2, 0), (2, 2), (0, 2), (1, 1)), - dtype=float, - ) - faces = ((0, 1, 4), (1, 2, 4), (2, 3, 4), (3, 0, 4)) - boundary = np.asarray((True, True, True, True, False)) - original = self.balloon_poisson.poisson_balloon_heights( - points, faces, boundary, 0.45 - ) - scaled = self.balloon_poisson.poisson_balloon_heights( - points * 2.0, faces, boundary, 0.45 - ) - - self.assertAlmostEqual(scaled[4], original[4] * 2.0, places=6) - - def test_poisson_balloon_makes_wide_regions_thicker(self): - import numpy as np - - faces = ((0, 1, 4), (1, 2, 4), (2, 3, 4), (3, 0, 4)) - boundary = np.asarray((True, True, True, True, False)) - wide = np.asarray( - ((0, 0), (4, 0), (4, 2), (0, 2), (2, 1)), - dtype=float, - ) - narrow = np.asarray( - ((0, 0), (4, 0), (4, 0.6), (0, 0.6), (2, 0.3)), - dtype=float, - ) - wide_height = self.balloon_poisson.poisson_balloon_heights( - wide, faces, boundary, 0.45 - )[4] - narrow_height = self.balloon_poisson.poisson_balloon_heights( - narrow, faces, boundary, 0.45 - )[4] - - self.assertGreater(wide_height, narrow_height) - - def test_balloon_keeps_overlapping_regions_disconnected(self): - import numpy as np - - regions = ( - ( - ((20, 0), (90, 0), (90, 70), (20, 70)), - ((50, 20), (50, 50), (70, 50), (70, 20)), - ), - ( - ((0, 20), (40, 20), (40, 45), (0, 45)), - ), - ) - vertices, faces, uv = self.balloon_geometry.triangulate_balloon( - regions, - (100, 100), - (0.0, 1.0, 0.0, 1.0), - spacing=0.1, - world_metric=((1.0, 0.0), (0.0, 1.0)), - ) - centers = uv[np.asarray(faces)].mean(axis=1) - in_hole = ( - (centers[:, 0] > 0.55) - & (centers[:, 0] < 0.65) - & (centers[:, 1] > 0.57) - & (centers[:, 1] < 0.73) - ) - in_left_region = ( - (centers[:, 0] > 0.02) - & (centers[:, 0] < 0.12) - & (centers[:, 1] > 0.55) - & (centers[:, 1] < 0.8) - ) - heights = np.asarray(vertices)[:, 2] - - self.assertFalse(in_hole.any()) - self.assertTrue(in_left_region.any()) - self.assertGreater(float(heights.max()), 0.0) - self.assertLess(float(heights.min()), 0.0) - adjacency = {} - for face in faces: - for vertex in face: - adjacency.setdefault(int(vertex), set()) - for start, end in zip(face, np.roll(face, -1)): - adjacency[int(start)].add(int(end)) - adjacency[int(end)].add(int(start)) - remaining = set(adjacency) - component_count = 0 - while remaining: - component_count += 1 - pending = [remaining.pop()] - while pending: - vertex = pending.pop() - connected = adjacency[vertex] & remaining - remaining.difference_update(connected) - pending.extend(connected) - - self.assertEqual(component_count, 2) - - def test_poisson_balloon_supports_resolution_002(self): - import numpy as np - - spacing = 0.02 - size = 51 - axis = np.arange(size, dtype=float) * spacing - points = np.asarray( - [(x, y) for y in axis for x in axis], - dtype=float, - ) - faces = [] - for row in range(size - 1): - for column in range(size - 1): - bottom_left = row * size + column - bottom_right = bottom_left + 1 - top_left = bottom_left + size - top_right = top_left + 1 - faces.append((bottom_left, bottom_right, top_right)) - faces.append((bottom_left, top_right, top_left)) - boundary = np.zeros(len(points), dtype=bool) - boundary[:size] = True - boundary[-size:] = True - boundary[::size] = True - boundary[size - 1 :: size] = True - - heights = self.balloon_poisson.poisson_balloon_heights( - points, - faces, - boundary, - 0.45, - ) - - self.assertTrue(np.isfinite(heights).all()) - self.assertTrue((heights[boundary] == 0.0).all()) - self.assertGreater(heights[(size * size) // 2], 0.0) - - def test_poisson_solver_has_no_absolute_scale_cutoff(self): - import numpy as np - - points = np.asarray( - ((0, 0), (2, 0), (2, 2), (0, 2), (1, 1)), - dtype=float, - ) - faces = ((0, 1, 4), (1, 2, 4), (2, 3, 4), (3, 0, 4)) - boundary = np.asarray((True, True, True, True, False)) - original = self.balloon_poisson.poisson_balloon_heights( - points, - faces, - boundary, - 0.45, - ) - tiny = self.balloon_poisson.poisson_balloon_heights( - points * 1e-5, - faces, - boundary, - 0.45, - ) - - self.assertAlmostEqual(tiny[4], original[4] * 1e-5, places=12) - - def test_teddy_balloon_uses_chordal_axis_radius(self): - import numpy as np - - points = np.asarray( - ((0, 0), (4, 0), (4, 2), (0, 2), (2, 1)), - dtype=float, - ) - faces = ((0, 1, 4), (1, 2, 4), (2, 3, 4), (3, 0, 4)) - boundary = np.asarray((True, True, True, True, False)) - nodes = np.asarray(((1, 1), (3, 1)), dtype=float) - with patch.object( - self.balloon_teddy, - "chordal_axis", - return_value=(nodes, [(0, 1)], np.asarray((1.0, 1.0))), - ): - heights = self.balloon_teddy.teddy_balloon_heights( - points, faces, points[:4], boundary, 0.45 - ) - - self.assertTrue((heights[:4] == 0.0).all()) - self.assertAlmostEqual(heights[4], 0.45, places=6) - - def test_balloon_dispatches_distinct_height_algorithms(self): - import numpy as np - - points = np.asarray(((0, 0),), dtype=float) - boundary = np.asarray((True,)) - with ( - patch.object( - self.balloon_geometry, - "poisson_balloon_heights", - return_value=np.asarray((1.0,)), - ) as poisson, - patch.object( - self.balloon_geometry, - "teddy_balloon_heights", - return_value=np.asarray((2.0,)), - ) as teddy, - ): - poisson_result = self.balloon_geometry.balloon_heights( - "POISSON", points, (), points, boundary, 0.45 - ) - teddy_result = self.balloon_geometry.balloon_heights( - "TEDDY", points, (), points, boundary, 0.45 - ) - - self.assertEqual(poisson_result[0], 1.0) - self.assertEqual(teddy_result[0], 2.0) - poisson.assert_called_once() - teddy.assert_called_once() - - def test_balloon_geometry_has_a_shared_closed_seam(self): - from collections import Counter - - import numpy as np - - planar = np.asarray( - ((0, 0), (2, 0), (2, 2), (0, 2), (1, 1)), - dtype=float, - ) - faces_2d = ((0, 1, 4), (1, 2, 4), (2, 3, 4), (3, 0, 4)) - boundary = np.asarray((True, True, True, True, False)) - heights = np.asarray((0, 0, 0, 0, 1), dtype=float) - vertices, faces, uv = self.balloon_geometry.build_closed_geometry( - planar, - faces_2d, - boundary, - heights, - (0.0, 2.0, 0.0, 2.0), - ) - edges = Counter() - for face in faces: - for start, end in zip(face, face[1:] + face[:1]): - edges[tuple(sorted((start, end)))] += 1 - - self.assertEqual(len(vertices), 6) - self.assertEqual(len(vertices), len(uv)) - self.assertEqual(uv.dtype, np.float32) - self.assertTrue(edges) - self.assertTrue(all(users == 2 for users in edges.values())) - - def test_balloon_tool_operators_are_not_registered(self): - self.assertNotIn(self.balloon.ExtractBalloon, self.addon.CLASSES) - self.assertNotIn(self.balloon.ActivateBalloonTool, self.addon.CLASSES) - self.assertNotIn(self.balloon.BalloonTool, self.addon.CLASSES) - self.assertIn(self.mesh.GenerateMesh, self.addon.CLASSES) - - def test_balloon_ai_analysis_is_registered_and_forwarded(self): - import inspect - - self.assertNotIn(self.balloon.AnalyzeBalloon, self.addon.CLASSES) - operator = self.balloon.ExtractBalloon - self.assertIn("contour_analysis", operator.__annotations__) - self.assertIn("depth_analysis", operator.__annotations__) - self.assertIn("depth_model", operator.__annotations__) - settings = self.addon.properties.AnyImageSettings.__annotations__ - for name in ( - "balloon_contour_analysis", - "balloon_depth_analysis", - ): - self.assertIn(name, settings) - self.assertFalse(settings[name]["default"]) - self.assertIn("balloon_depth_model", settings) - self.assertEqual(settings["balloon_contour_analysis"]["name"], "Contour") - self.assertEqual(settings["balloon_depth_analysis"]["name"], "Depth") - - execute_source = inspect.getsource(operator.execute) - invoke_source = inspect.getsource(operator.invoke) - request_source = inspect.getsource(self.balloon.AnalyzeBalloon.request) - response_source = inspect.getsource(self.balloon.AnalyzeBalloon.response) - tool_source = inspect.getsource(self.balloon.BalloonTool.draw_settings) - self.assertIn("bpy.ops.anyimage.analyze_balloon", execute_source) - self.assertIn("ready = ai_ready()", invoke_source) - self.assertIn("status = ai_status()", tool_source) - self.assertIn('"anyimage.setup_ai_environment"', tool_source) - self.assertIn("ai_setup_label(status)", tool_source) - self.assertIn("prepare_image_input(", execute_source) - self.assertIn( - "settings.balloon_contour_analysis", - invoke_source, - ) - self.assertIn( - "settings.balloon_depth_analysis", - invoke_source, - ) - self.assertIn("self.depth_model = settings.balloon_depth_model", invoke_source) - self.assertIn("require_model(BEN2_MODEL_KEY)", request_source) - self.assertIn( - "require_model(self.depth_model)", - request_source, - ) - self.assertIn('"polygon": polygon', request_source) - self.assertIn('result.file("contour")', response_source) - self.assertIn('result.file("depth")', response_source) - self.assertIn("depth_data=depth_data", response_source) - self.assertIn('push_undo("Extract Balloon")', response_source) - self.assertIn('"balloon_contour_analysis"', tool_source) - self.assertIn('"balloon_depth_analysis"', tool_source) - self.assertIn('"balloon_depth_model"', tool_source) - - def test_balloon_analysis_requires_only_enabled_models(self): - operator = self.balloon.AnalyzeBalloon() - operator.input_path = "input.png" - operator.selection_data = "[[0,0],[2,0],[0,2]]" - operator.resolution_level = 5 - operator.depth_model = "MOGE2_VITB_NORMAL" - required = [] - - with ( - patch.object(self.balloon, "require_environment"), - patch.object( - self.balloon, - "require_model", - side_effect=required.append, - ), - patch.object( - self.balloon, - "production_device", - return_value="CUDA", - ), - ): - operator.contour_analysis = True - operator.depth_analysis = False - contour_request = operator.request(None) - operator.contour_analysis = False - operator.depth_analysis = True - depth_request = operator.request(None) - - self.assertEqual( - required, - [ - self.addon.properties.BEN2_MODEL_KEY, - "MOGE2_VITB_NORMAL", - ], - ) - self.assertTrue(contour_request["contour_analysis"]) - self.assertFalse(contour_request["depth_analysis"]) - self.assertFalse(depth_request["contour_analysis"]) - self.assertTrue(depth_request["depth_analysis"]) - self.assertEqual(depth_request["depth_model"], "MOGE2_VITB_NORMAL") - - def test_balloon_depth_fields_and_shape_nodes(self): - import inspect - - import numpy as np - - depth_data = { - "depth_signal": np.asarray( - ((1.0, -2.0), (0.5, 0.0)), - dtype=np.float32, - ), - "baseline_vector": np.zeros(3, dtype=np.float32), - "valid": np.ones((2, 2), dtype=bool), - "origin": (0, 0), - "crop_size": (2, 2), - "source_size": (2, 2), - } - vertices = ( - (0.0, 0.0, 2.0), - (0.0, 0.0, 2.0), - (0.0, 0.0, -2.0), - (0.0, 0.0, -2.0), - ) - uv = ((0.0, 1.0), (1.0, 1.0), (0.0, 0.0), (1.0, 0.0)) - - values, local_plane = self.balloon_depth.balloon_depth_fields( - vertices, - uv, - depth_data, - ) - - np.testing.assert_allclose(values, (1.5, -4.5, 0.5, -0.5)) - self.assertAlmostEqual(float(np.median(values)), 0.0) - np.testing.assert_allclose(local_plane, (0.0, 0.0, 0.0)) - self.assertEqual( - self.balloon.orient_balloon_vertices(((1.0, 2.0, 3.0),)), - [(-3.0, -1.0, 2.0)], - ) - restored = np.asarray(self.balloon.BALLOON_AXIS_TO_SOURCE) @ np.asarray( - (-3.0, -1.0, 2.0, 1.0) - ) - np.testing.assert_allclose(restored, (1.0, 2.0, 3.0, 1.0)) - self.assertEqual( - self.balloon.orient_balloon_base_plane((0.2, 0.3, 0.4)), - (0.2, -0.3, -0.4), - ) - source = inspect.getsource(self.balloon_depth.balloon_shape_node_group) - controller_source = inspect.getsource( - self.balloon_depth.create_balloon_base_plane - ) - modifier_source = inspect.getsource( - self.balloon_depth.add_balloon_shape_modifier - ) - self.assertIn('"Thickness"', source) - self.assertIn('"Depth Strength"', source) - self.assertIn('"Depth Strength": 1.0', modifier_source) - self.assertNotIn('"Far Clamp"', source) - self.assertIn('"Double Sided Depth"', source) - self.assertIn('"Base Plane"', source) - self.assertIn('nodes.new("GeometryNodeObjectInfo")', source) - self.assertIn("controller.parent = balloon", controller_source) - self.assertIn( - 'controller.empty_display_type = "SINGLE_ARROW"', controller_source - ) - self.assertIn('profile_factor.operation = "ABSOLUTE"', source) - self.assertIn('profile_statistics.data_type = "FLOAT"', source) - self.assertIn('profile_max_safe.operation = "MAXIMUM"', source) - self.assertIn('profile_normalized.operation = "DIVIDE"', source) - self.assertIn('profile_smooth.interpolation_type = "SMOOTHERSTEP"', source) - self.assertIn('profile_active.operation = "MULTIPLY"', source) - self.assertIn('effective_normal.input_type = "VECTOR"', source) - self.assertNotIn("o_balloon_depth_mix", source) - self.assertIn('front.operation = "LESS_THAN"', source) - self.assertIn('depth_residual.operation = "ADD"', source) - self.assertIn("depth_calibration.inputs[1].default_value = 0.5", source) - self.assertIn('corrected_depth.operation = "ADD"', source) - self.assertIn( - 'profile_statistics.outputs["Max"], corrected_depth.inputs[1]', - source, - ) - self.assertNotIn("baseline_thickness", source) - self.assertIn('separate.outputs["X"], base_scale.inputs[0]', source) - self.assertIn('double_choice.outputs["Output"], combine.inputs["X"]', source) - self.assertIn('depth_mix = nodes.new("ShaderNodeMix")', source) - self.assertIn('depth_mix.data_type = "FLOAT"', source) - self.assertIn("depth_mix.clamp_factor = True", source) - self.assertIn('base_magnitude.outputs[0], depth_mix.inputs["A"]', source) - self.assertIn('depth_scale.outputs[0], depth_mix.inputs["B"]', source) - self.assertNotIn("profile_inverse", source) - self.assertNotIn("thickness_component", source) - self.assertNotIn("depth_component", source) - self.assertNotIn("blended_magnitude", source) - self.assertIn('plane_offset.operation = "DOT_PRODUCT"', source) - self.assertIn("controller.location = (0.0, 0.0, 0.0)", controller_source) - self.assertNotIn("clamp_distance", source) - self.assertNotIn("safety_clamped", source) - self.assertNotIn("crossing_choice", source) - self.assertIn('front_choice.input_type = "FLOAT"', source) - self.assertIn('double_choice.input_type = "FLOAT"', source) - self.assertNotIn("double_inverse", source) - self.assertNotIn("depth_limited", source) - self.assertNotIn("side_gate", source) - self.assertIn('base_frame.label = "Base Thickness"', source) - self.assertIn('plane_frame.label = "Middle Plane"', source) - self.assertIn('mix_frame.label = "Depth Mix"', source) - self.assertIn('output_frame.label = "Side Output"', source) - self.assertIn('side.operation = "SIGN"', source) - self.assertIn("BALLOON_DEPTH_ATTRIBUTE", source) - self.assertEqual( - self.balloon_depth.BALLOON_DEPTH_ATTRIBUTE, - "o_balloon_depth", - ) - - def test_balloon_redo_properties_use_mode_and_meter_resolution(self): - import inspect - - spacing, inflation = self.balloon.balloon_geometry_parameters(2.0, 0.2) - scene = SimpleNamespace( - unit_settings=SimpleNamespace(scale_length=0.01), - ) - - self.assertEqual(spacing, 0.2) - self.assertEqual(inflation, 0.9) - self.assertEqual( - self.balloon.meters_to_world_distance(scene, 0.1), - 10.0, - ) - operator = self.balloon.ExtractBalloon - self.assertEqual(operator.bl_options, {"REGISTER", "UNDO"}) - self.assertIn("mode", operator.__annotations__) - self.assertIn("thickness", operator.__annotations__) - self.assertIn("resolution", operator.__annotations__) - self.assertIn("reuse_material", operator.__annotations__) - self.assertIn("source_matrix_data", operator.__annotations__) - settings = self.addon.properties.AnyImageSettings.__annotations__ - self.assertIn("balloon_mode", settings) - self.assertIn("balloon_thickness", settings) - self.assertIn("balloon_resolution", settings) - self.assertIn("balloon_reuse_material", settings) - self.assertIn("balloon_thickness_radial_zoom", settings) - self.assertIn("balloon_resolution_radial_zoom", settings) - self.assertEqual( - settings["balloon_thickness_radial_zoom"]["default"], - (100.0, 100.0), - ) - self.assertEqual( - settings["balloon_resolution_radial_zoom"]["default"], - (1000.0, 1000.0), - ) - resolution_source = inspect.getsource( - self.addon.properties.balloon_resolution_property - ) - thickness_source = inspect.getsource( - self.addon.properties.balloon_thickness_property - ) - self.assertIn('name="Resolution (m)"', resolution_source) - self.assertNotIn("subtype=", resolution_source) - self.assertNotIn("unit=", resolution_source) - self.assertIn("soft_max=5.0", thickness_source) - self.assertNotIn(" max=", thickness_source) - - def test_balloon_workspace_tool_uses_one_top_level_operator_per_stroke(self): - import inspect - - execute_source = inspect.getsource(self.balloon.ExtractBalloon.execute) - invoke_source = inspect.getsource(self.balloon.ExtractBalloon.invoke) - modal_source = inspect.getsource(self.balloon.ExtractBalloon.modal) - - self.assertIn("json.loads(self.selection_data)", execute_source) - self.assertIn("deserialize_matrix(self.source_matrix_data)", execute_source) - self.assertIn("extract_balloon_object(", execute_source) - self.assertIn("select_result=False", execute_source) - self.assertIn("reuse_material=self.reuse_material", execute_source) - self.assertIn("meters_to_world_distance(", execute_source) - self.assertIn("serialize_matrix(source_object.matrix_world)", invoke_source) - self.assertIn("settings.balloon_thickness", invoke_source) - self.assertIn("settings.balloon_resolution", invoke_source) - self.assertIn("settings.balloon_reuse_material", invoke_source) - self.assertIn("return self.execute(context)", modal_source) - self.assertIn('return {"RUNNING_MODAL"}', modal_source) - self.assertIn('return {"PASS_THROUGH"}', modal_source) - self.assertNotIn("bpy.ops.anyimage.extract_balloon", modal_source) - - tool = self.balloon.BalloonTool - self.assertEqual(tool.bl_space_type, "VIEW_3D") - self.assertEqual(tool.bl_context_mode, "OBJECT") - self.assertEqual(tool.bl_operator, self.balloon.ExtractBalloon.bl_idname) - self.assertEqual(tool.bl_icon, "ops.mesh.polybuild_hover") - self.assertEqual(tool.bl_cursor, "PAINT_BRUSH") - keymap_operators = tuple(item[0] for item in tool.bl_keymap) - self.assertEqual(keymap_operators[0], self.balloon.ExtractBalloon.bl_idname) - self.assertEqual( - keymap_operators[1:], - ("wm.radial_control", "wm.radial_control"), - ) - self.assertEqual( - tool.bl_keymap[1][2]["properties"], - [ - ( - "data_path_primary", - "scene.anyimage_settings.balloon_thickness", - ), - ( - "zoom_path", - "scene.anyimage_settings.balloon_thickness_radial_zoom", - ), - ], - ) - self.assertEqual( - tool.bl_keymap[2][2]["properties"], - [ - ( - "data_path_primary", - "scene.anyimage_settings.balloon_resolution", - ), - ( - "zoom_path", - "scene.anyimage_settings.balloon_resolution_radial_zoom", - ), - ], - ) - - def test_balloon_draw_handlers_are_filtered_to_the_invoking_viewport(self): - import inspect - - source = inspect.getsource(self.balloon.drawing_in_region) - operator_source = inspect.getsource(self.balloon.ExtractBalloon) - - self.assertIn("area.as_pointer() == area_pointer", source) - self.assertIn("region.as_pointer() == region_pointer", source) - self.assertIn("drawing_in_region(", operator_source) - self.assertNotIn("wireframe_opacity", operator_source) - self.assertNotIn("shading.type", operator_source) - - def test_balloon_matrix_serialization_keeps_all_components(self): - import json - - matrix = tuple( - tuple(row * 4 + column for column in range(4)) for row in range(4) - ) - values = json.loads(self.balloon.serialize_matrix(matrix)) - - self.assertEqual(values, list(range(16))) - - def test_extract_balloon_preserves_the_source_object(self): - import inspect - - source = inspect.getsource(self.balloon.extract_balloon_object) - self.assertNotIn("objects.remove(source_object", source) - self.assertIn("balloon_name = source_name", source) - self.assertIn( - "selected_object = balloon if select_result else source_object", source - ) - self.assertIn( - "balloon.matrix_world = orient_balloon_world_matrix(matrix_world)", - source, - ) - self.assertIn("create_material(", source) - self.assertIn("material_name = material_data_name(source_object)", source) - self.assertIn( - "normal_required=uses_normal", - source, - ) - self.assertIn("mark_balloon_material(material, image, normal_image)", source) - self.assertIn("merge_balloon_normal_image(normal_image, update_normal)", source) - self.assertIn("if reuse_material", source) - self.assertIn("color_image = image.copy()", source) - self.assertIn("color_image.name = color_name", source) - self.assertIn( - "material_name,\n color_image,\n 0.0,", - source, - ) - self.assertIn( - "surface_group_name=IMAGE_MATERIAL_NODE_GROUP_NAME", - source, - ) - self.assertIn("owns_material", source) - self.assertIn("owns_color_image", source) - self.assertIn("balloon_depth_fields(", source) - self.assertIn("assign_balloon_shape_attributes(", source) - self.assertIn("inflation=INFLATION", source) - self.assertIn("create_balloon_base_plane(", source) - self.assertIn("add_balloon_shape_modifier(", source) - self.assertNotIn("depth_strength", source) - self.assertIn("IMAGE_MATERIAL_NODE_GROUP_NAME", source) - self.assertIn("normal_path=", source) - self.assertEqual( - self.image_depth.TEXTURE_MATERIAL_NODE_GROUP_NAME, - "O Image Layer", - ) - - def test_balloon_material_reuse_uses_explicit_source_markers(self): - class MarkedMaterial(dict): - def __init__(self, name, **properties): - super().__init__(properties) - self.name = name - - color_image = SimpleNamespace(name="cow.color") - other_image = SimpleNamespace(name="other") - marked = MarkedMaterial( - "color-material", - anyimage_balloon_material=True, - anyimage_source_image=color_image, - ) - marked_other = MarkedMaterial( - "other-material", - anyimage_balloon_material=True, - anyimage_source_image=other_image, - ) - normal_image = SimpleNamespace(name="cow.normal") - marked_normal = MarkedMaterial( - "normal-material", - anyimage_balloon_material=True, - anyimage_source_image=color_image, - anyimage_balloon_normal_image=normal_image, - ) - self.fake_bpy.data = SimpleNamespace( - materials=[marked_other, marked_normal, marked], - images=[color_image], - ) - - self.assertIs( - self.balloon.reusable_balloon_material(color_image), - marked, - ) - self.assertIs( - self.balloon.reusable_balloon_material( - color_image, - normal_required=True, - ), - marked_normal, - ) - self.assertIs( - self.balloon.reusable_color_image("cow.color"), - color_image, - ) - def test_surface_and_depth_plane_material_names_use_image_data(self): import inspect diff --git a/tests/test_cutout.py b/tests/test_cutout.py index a641576..d3d3f80 100644 --- a/tests/test_cutout.py +++ b/tests/test_cutout.py @@ -59,6 +59,12 @@ def test_cutout_pie_uses_four_semantic_shapes_in_cardinal_order(): mesh.DEPTH_BALLOON, mesh.DEPTH_SURFACE, ] + assert [call[1]["text"] for call in window_manager.pie.calls] == [ + "Surface", + "Balloon", + "Depth Balloon", + "Depth Surface", + ] assert all( call[0] == "anyimage.convert_cutout_shape" for call in window_manager.pie.calls ) diff --git a/tests/test_mesh.py b/tests/test_mesh.py index 6297bc4..83e31ac 100644 --- a/tests/test_mesh.py +++ b/tests/test_mesh.py @@ -11,11 +11,67 @@ from PIL import Image SOURCE_ROOT = Path(__file__).parents[1] / "src" sys.path.insert(0, str(SOURCE_ROOT)) -from anyimage.operators import mesh +from anyimage.operators import cutout_depth, cutout_teddy, mesh +from anyimage.operators.cutout_profile import poisson_balloon_profile from anyimage.operators.normal_map import rasterize_tangent_normal from anyimage.server.geometry.cutout import alpha_regions, image_alpha_regions +def test_base_plane_fit_is_stable_for_a_slender_diagonal_domain(): + along = np.linspace(-1.0, 1.0, 101) + across = np.resize(np.asarray((-0.01, 0.0, 0.01)), len(along)) + coordinates = np.column_stack( + ( + along - across, + along + across, + ) + ) + values = 0.4 * along + np.sign(across) * 0.05 + + plane = cutout_depth._stable_plane(coordinates, values) + selected_plane = coordinates @ plane[:2] + plane[2] + corners = np.asarray( + ((-1.0, -1.0), (-1.0, 1.0), (1.0, -1.0), (1.0, 1.0)) + ) + corner_plane = corners @ plane[:2] + plane[2] + + assert np.ptp(corner_plane) <= np.ptp(selected_plane) * 1.25 + assert np.corrcoef(selected_plane, along)[0, 1] > 0.99 + + +def test_depth_balloon_plane_calibration_matches_cutout_axes(): + vertices = np.asarray( + ( + (-1.0, -1.0, 0.0), + (1.0, -1.0, 0.0), + (1.0, 1.0, 0.0), + (-1.0, 1.0, 0.0), + ), + dtype=np.float32, + ) + uv = np.asarray( + ((0.0, 0.0), (1.0, 0.0), (1.0, 1.0), (0.0, 1.0)), + dtype=np.float32, + ) + depth_data = { + "depth_signal": np.zeros((2, 2), dtype=np.float32), + "baseline_vector": np.asarray((0.4, -0.2, 0.1), dtype=np.float32), + "valid": np.ones((2, 2), dtype=bool), + "crop_size": (2, 2), + } + + _depth, local_plane = cutout_depth.depth_balloon_fields( + vertices, + uv, + depth_data, + 1.0, + ) + oriented = cutout_depth.orient_depth_balloon_plane(local_plane) + + assert np.allclose(local_plane, (0.2, 0.1, 0.05), atol=1e-6) + assert np.allclose(oriented, (-0.2, 0.1, 0.05), atol=1e-6) + + def mesh_component_count(faces): adjacency = {} for face in faces: @@ -66,63 +122,6 @@ def test_plane_source_topology_is_one_quad_at_every_detail_level(): assert low[1].shape == high[1].shape == (1, 4) -def test_cutout_front_back_and_side_faces_point_outward(): - vertices = np.asarray( - ((0.0, 0.0, 0.0), (0.0, 1.0, 0.0), (1.0, 0.0, 0.0)), - dtype=np.float64, - ) - faces = np.asarray(((0, 1, 2),), dtype=np.int32) - uv = vertices[:, :2].astype(np.float32) - boundary = np.ones(3, dtype=bool) - layered = mesh._layered_geometry( - (vertices, faces, uv, boundary, vertices[:, :2]), - mesh.CUTOUT, - ) - positioned = layered[0] + layered[3] - output_faces = layered[1] - - def face_normal(face): - points = positioned[np.asarray(face[:3], dtype=np.int32)] - return np.cross(points[1] - points[0], points[2] - points[0]) - - assert face_normal(output_faces[0])[2] > 0.0 - assert face_normal(output_faces[1])[2] < 0.0 - center = vertices[:, :2].mean(axis=0) - for face in output_faces[2:]: - normal = face_normal(face) - side_center = positioned[np.asarray(face, dtype=np.int32), :2].mean(axis=0) - assert np.dot(normal[:2], side_center - center) > 0.0 - - -def test_depth_surface_moves_both_sides_with_camera_projection_and_extrudes_back(): - domain = mesh._plane_domain((-1.0, 1.0, -1.0, 1.0), 4) - field = np.zeros((2, 2, 3), dtype=np.float32) - field[..., 2] = 1.0 - valid = np.ones((2, 2), dtype=bool) - camera_depth = np.ones((2, 2), dtype=np.float32) - - depth_surface = mesh._layered_geometry( - domain, mesh.DEPTH_SURFACE, (field, valid, camera_depth) - ) - base_count = len(domain[0]) - - assert np.allclose(depth_surface[4][:base_count, 2], 1.0) - assert np.allclose(depth_surface[4][base_count:, 2], 1.0) - assert np.allclose(depth_surface[3][:base_count], 0.0) - assert np.allclose(depth_surface[3][base_count:, 2], -1.0) - assert all(depth_surface[5][: len(domain[1])]) - assert not any(depth_surface[5][len(domain[1]) :]) - - -def test_depth_surface_records_depth_discontinuities_for_geometry_nodes(): - faces = np.asarray(((0, 1, 2), (1, 2, 3)), dtype=np.int32) - depth = np.asarray((1.0, 1.01, 1.02, 2.0), dtype=np.float32) - - discontinuity = mesh._depth_discontinuity_values(faces, depth) - - assert discontinuity[0] < mesh.DEPTH_EDGE_THRESHOLD - assert discontinuity[1] > mesh.DEPTH_EDGE_THRESHOLD - def test_depth_surface_discontinuity_map_uses_local_three_by_three_range(): depth = np.ones((5, 5), dtype=np.float32) @@ -298,6 +297,78 @@ def test_cutout_point_spacing_uses_the_complete_source_plane(): assert spacing == 0.02 +def test_poisson_balloon_profile_is_zero_on_boundary_and_positive_inside(): + points = np.asarray( + ((0, 0), (2, 0), (2, 2), (0, 2), (1, 1)), + dtype=float, + ) + faces = ((0, 1, 4), (1, 2, 4), (2, 3, 4), (3, 0, 4)) + boundary = np.asarray((True, True, True, True, False)) + + profile = poisson_balloon_profile(points, faces, boundary) + + assert np.all(profile[:4] == 0.0) + assert profile[4] > 0.0 + + +def test_poisson_balloon_profile_scales_in_world_units(): + points = np.asarray( + ((0, 0), (2, 0), (2, 2), (0, 2), (1, 1)), + dtype=float, + ) + faces = ((0, 1, 4), (1, 2, 4), (2, 3, 4), (3, 0, 4)) + boundary = np.asarray((True, True, True, True, False)) + + original = poisson_balloon_profile(points, faces, boundary) + scaled = poisson_balloon_profile(points * 2.0, faces, boundary) + + assert np.isclose(scaled[4], original[4] * 2.0, atol=1e-6) + + +def test_poisson_balloon_profile_makes_wide_regions_thicker(): + faces = ((0, 1, 4), (1, 2, 4), (2, 3, 4), (3, 0, 4)) + boundary = np.asarray((True, True, True, True, False)) + wide = np.asarray( + ((0, 0), (4, 0), (4, 2), (0, 2), (2, 1)), + dtype=float, + ) + narrow = np.asarray( + ((0, 0), (4, 0), (4, 0.6), (0, 0.6), (2, 0.3)), + dtype=float, + ) + + wide_profile = poisson_balloon_profile(wide, faces, boundary) + narrow_profile = poisson_balloon_profile(narrow, faces, boundary) + + assert wide_profile[4] > narrow_profile[4] +def test_teddy_balloon_profile_uses_chordal_axis_radius(): + points = np.asarray( + ((0, 0), (4, 0), (4, 2), (0, 2), (2, 1)), + dtype=float, + ) + faces = ((0, 1, 4), (1, 2, 4), (2, 3, 4), (3, 0, 4)) + boundary = np.asarray((True, True, True, True, False)) + nodes = np.asarray(((1, 1), (3, 1)), dtype=float) + + with patch.object( + cutout_teddy, + "chordal_axis", + return_value=(nodes, [(0, 1)], np.asarray((1.0, 1.0))), + ): + profile = cutout_teddy.teddy_balloon_heights( + points, + faces, + points[:4], + boundary, + 0.45, + ) + + assert np.all(profile[:4] == 0.0) + assert np.isclose(profile[4], 0.45, atol=1e-6) + + + + def test_cutout_domain_adapts_boundary_spacing_for_a_small_valid_region(): regions = ( (((2046, 2046), (2050, 2046), (2050, 2050), (2046, 2050)),), @@ -329,6 +400,23 @@ def test_cutout_domain_builds_a_full_image_at_minimum_detail(): assert len(domain[0]) >= 4 assert len(domain[1]) >= 2 +def test_cutout_domain_centers_vertices_on_crop_bounds(): + regions = ((((0, 0), (100, 0), (100, 50), (0, 50)),),) + + vertices, _faces, uv, _boundary, _planar, _profile = mesh._cutout_domain( + regions, + (100, 50), + (2.0, 6.0, -3.0, 1.0), + 0.5, + ((1.0, 0.0), (0.0, 1.0)), + ) + + center = (vertices[:, :2].min(axis=0) + vertices[:, :2].max(axis=0)) * 0.5 + assert np.allclose(center, 0.0, atol=1e-6) + expected_uv = vertices[:, :2] / np.asarray((4.0, 4.0)) + 0.5 + assert np.allclose(uv, expected_uv, atol=1e-6) + + def test_cutout_domain_keeps_overlapping_regions_disconnected(): regions = ( ( diff --git a/tests/test_mesh_blender.py b/tests/test_mesh_blender.py index bbf0a1f..75327dc 100644 --- a/tests/test_mesh_blender.py +++ b/tests/test_mesh_blender.py @@ -5,7 +5,7 @@ from unittest.mock import patch import bpy import numpy as np -from mathutils import Matrix +from mathutils import Matrix, Vector SOURCE_ROOT = Path(__file__).parents[1] / "src" sys.path.insert(0, str(SOURCE_ROOT)) @@ -66,12 +66,31 @@ def test_shape_node_groups_expose_only_shape_parameters(): plane = mesh._mesh_node_group(mesh.PLANE) depth_plane = mesh._mesh_node_group(mesh.DEPTH_PLANE) + legacy_surface = bpy.data.node_groups.new( + "O Image Cutout", + "GeometryNodeTree", + ) cutout = mesh._mesh_node_group(mesh.CUTOUT) + balloon = mesh._mesh_node_group(mesh.BALLOON) + depth_balloon = mesh._mesh_node_group(mesh.DEPTH_BALLOON) depth_surface = mesh._mesh_node_group(mesh.DEPTH_SURFACE) assert plane.name == "O Mesh Plane" + assert cutout.name == "O Image Surface" + assert cutout == legacy_surface assert _inputs(plane) == ["Geometry", "Subdivide", "Thickness", "Material"] - assert _inputs(cutout) == ["Geometry", "Thickness"] + assert _inputs(cutout) == ["Geometry", "Thickness", "Cleanup"] + assert _inputs(balloon) == ["Geometry", "Thickness", "Cleanup"] + assert _inputs(depth_balloon) == [ + "Geometry", + "Thickness", + "Depth Scale", + "Double Sided", + "Depth Reference", + "Depth Image", + "Cleanup", + "Base Plane", + ] assert _inputs(depth_plane) == [ "Geometry", "Subdivide", @@ -85,9 +104,46 @@ def test_shape_node_groups_expose_only_shape_parameters(): "Thickness", "Depth Scale", "Iterations", - "Smooth", + "Depth Image", + "Cleanup", "Depth Continuity", + "Smooth", ] + assert len({group.as_pointer() for group in ( + cutout, + balloon, + depth_balloon, + depth_surface, + )}) == 4 + expected_options = { + cutout: ["Cleanup"], + balloon: ["Cleanup"], + depth_balloon: ["Cleanup", "Base Plane"], + depth_surface: ["Cleanup", "Depth Continuity", "Smooth"], + } + for group, expected in expected_options.items(): + panels = [ + item + for item in group.interface.items_tree + if item.item_type == "PANEL" + ] + assert [panel.name for panel in panels] == ["Options"] + assert panels[0].default_closed + assert [item.name for item in panels[0].interface_items] == expected + for group in (cutout, balloon, depth_balloon, depth_surface): + extrudes = [ + node + for node in group.nodes + if node.bl_idname == "GeometryNodeExtrudeMesh" + ] + assert extrudes + assert all( + not node.inputs["Individual"].default_value for node in extrudes + ) + assert all( + node.inputs["Offset Scale"].default_value == 1.0 + for node in extrudes + ) for group in (depth_plane, cutout, depth_surface): thickness = next( item @@ -95,6 +151,14 @@ def test_shape_node_groups_expose_only_shape_parameters(): if item.item_type == "SOCKET" and item.name == "Thickness" ) assert thickness.subtype == "DISTANCE" + for group in (balloon, depth_balloon): + thickness = next( + item + for item in group.interface.items_tree + if item.item_type == "SOCKET" and item.name == "Thickness" + ) + assert thickness.subtype == "NONE" + assert thickness.default_value == 1.0 for group, name in ( (depth_plane, "Depth Amount"), (depth_surface, "Depth Scale"), @@ -145,6 +209,18 @@ def test_shape_node_groups_expose_only_shape_parameters(): assert any( node.bl_idname == "GeometryNodeRepeatOutput" for node in depth_surface.nodes ) + repeat_counts = { + group: sum( + node.bl_idname == "GeometryNodeRepeatInput" for node in group.nodes + ) + for group in (cutout, balloon, depth_balloon, depth_surface) + } + assert repeat_counts == { + cutout: 1, + balloon: 0, + depth_balloon: 0, + depth_surface: 2, + } assert any( node.bl_idname == "GeometryNodeBlurAttribute" for node in depth_surface.nodes ) @@ -155,7 +231,7 @@ def test_shape_node_groups_expose_only_shape_parameters(): assert sum( node.bl_idname == "GeometryNodeInputMeshEdgeNeighbors" for node in depth_surface.nodes - ) == 2 + ) == 3 assert sum( node.bl_idname == "GeometryNodeInputMeshVertexNeighbors" for node in depth_surface.nodes @@ -191,12 +267,51 @@ def test_shape_node_groups_expose_only_shape_parameters(): and node.inputs[0].default_value == 1.0 for node in depth_surface.nodes ) - assert any( - node.bl_idname == "GeometryNodeInputNamedAttribute" - and node.inputs["Name"].default_value == mesh.DEPTH_DISCONTINUITY_ATTRIBUTE - for node in depth_surface.nodes + depth_reference = next( + item + for item in depth_balloon.interface.items_tree + if item.item_type == "SOCKET" + and item.in_out == "INPUT" + and item.name == "Depth Reference" ) - for group in (depth_plane, cutout, depth_surface): + assert depth_reference.hide_in_modifier + assert depth_reference.subtype == "DISTANCE" + for group in (depth_balloon, depth_surface): + depth_image = next( + item + for item in group.interface.items_tree + if item.item_type == "SOCKET" + and item.in_out == "INPUT" + and item.name == "Depth Image" + ) + assert depth_image.hide_in_modifier + assert any( + node.bl_idname == "GeometryNodeImageTexture" for node in group.nodes + ) + for group in (cutout, balloon, depth_balloon, depth_surface): + assert not any( + node.bl_idname == "GeometryNodeInputShortestEdgePaths" + for node in group.nodes + ) + assert not any( + node.bl_idname == "GeometryNodeInputNamedAttribute" + and node.inputs["Name"].default_value == "o_boundary" + for node in group.nodes + ) + for group in (balloon, depth_balloon): + profile = next( + node + for node in group.nodes + if node.bl_idname == "GeometryNodeInputNamedAttribute" + and node.inputs["Name"].default_value == mesh.BALLOON_ATTRIBUTE + ) + assert profile.data_type == "FLOAT" + assert any( + node.bl_idname == "ShaderNodeMapRange" + and node.interpolation_type == "SMOOTHERSTEP" + for node in group.nodes + ) + for group in (depth_plane, cutout, balloon, depth_balloon, depth_surface): input_sockets = [ item for item in group.interface.items_tree @@ -211,21 +326,21 @@ def test_shape_node_groups_expose_only_shape_parameters(): for output in group_input.outputs if output.name ) - geometry_links = [ - link - for link in group.links - if link.from_socket.type == "GEOMETRY" - and link.to_socket.type == "GEOMETRY" - ] assert all( link.from_node.location.x <= link.to_node.location.x - for link in geometry_links + for link in group.links ) assert all(not node.label for node in group.nodes) - node_locations = {tuple(node.location) for node in depth_surface.nodes} - assert len(node_locations) == len(depth_surface.nodes) - depth_plane_locations = {tuple(node.location) for node in depth_plane.nodes} - assert len(depth_plane_locations) == len(depth_plane.nodes) + node_locations = {tuple(node.location) for node in group.nodes} + assert len(node_locations) == len(group.nodes) + if group in (cutout, balloon, depth_balloon, depth_surface): + group_nodes = list(group.nodes) + for index, node in enumerate(group_nodes): + for other in group_nodes[index + 1:]: + assert not ( + abs(node.location.x - other.location.x) < 150.0 + and abs(node.location.y - other.location.y) < 100.0 + ) assert any( node.bl_idname == "GeometryNodeSetShadeSmooth" and node.inputs["Shade Smooth"].default_value @@ -344,9 +459,7 @@ def test_build_plane_keeps_one_quad_and_generates_topology_in_geometry_nodes(): assert len(result.data.vertices) == 4 assert len(result.data.polygons) == 1 assert all(polygon.use_smooth for polygon in result.data.polygons) - assert mesh.PROFILE_ATTRIBUTE not in result.data.attributes - assert mesh.DEPTH_ATTRIBUTE not in result.data.attributes - assert mesh.FRONT_FACE_ATTRIBUTE not in result.data.attributes + assert mesh.BALLOON_ATTRIBUTE not in result.data.attributes assert result.modifiers[0].node_group.name == "O Mesh Plane" evaluated = result.evaluated_get(bpy.context.evaluated_depsgraph_get()) evaluated_mesh = evaluated.to_mesh() @@ -355,6 +468,115 @@ def test_build_plane_keeps_one_quad_and_generates_topology_in_geometry_nodes(): finally: evaluated.to_mesh_clear() + +def test_balloon_build_starts_from_the_unified_single_sheet(): + bpy.ops.wm.read_factory_settings(use_empty=True) + image = bpy.data.images.new("Balloon Sheet", width=64, height=64, alpha=True) + source = bpy.data.objects.new("Balloon Sheet", None) + source.empty_display_type = "IMAGE" + source.data = image + source.empty_display_size = 2.0 + bpy.context.collection.objects.link(source) + balloon_material = bpy.data.materials.new("Balloon Sheet.material") + + result = mesh._build_mesh_object( + bpy.context, + source, + mesh.BALLOON, + 8, + balloon_material, + regions=((((0, 0), (64, 0), (64, 64), (0, 64)),),), + image_size=(64, 64), + keep_original=True, + ) + + assert all(abs(vertex.co.x) < 1e-8 for vertex in result.data.vertices) + assert len(result.data.vertices) > 4 + assert len(result.data.polygons) > 1 + assert mesh.BALLOON_ATTRIBUTE in result.data.attributes + profile_attribute = result.data.attributes[mesh.BALLOON_ATTRIBUTE] + assert profile_attribute.data_type == "FLOAT" + assert profile_attribute.domain == "POINT" + profile = np.empty(len(result.data.vertices), dtype=np.float32) + profile_attribute.data.foreach_get("value", profile) + assert profile.min() == 0.0 and profile.max() > 0.0 + assert result.modifiers[0].node_group.name == "O Image Balloon" + inward = result.matrix_world.to_3x3() @ Vector((1.0, 0.0, 0.0)) + assert np.allclose(tuple(inward), (0.0, 0.0, -1.0), atol=1e-6) + + +def test_teddy_preference_writes_the_shared_balloon_attribute(): + bpy.ops.wm.read_factory_settings(use_empty=True) + image = bpy.data.images.new("Teddy Sheet", width=64, height=64, alpha=True) + source = bpy.data.objects.new("Teddy Sheet", None) + source.empty_display_type = "IMAGE" + source.data = image + source.empty_display_size = 2.0 + bpy.context.collection.objects.link(source) + teddy_material = bpy.data.materials.new("Teddy Sheet.material") + + with patch.object( + mesh, + "addon_preferences", + return_value=SimpleNamespace(balloon_profile_algorithm="TEDDY"), + ): + result = mesh._build_mesh_object( + bpy.context, + source, + mesh.CUTOUT, + 8, + teddy_material, + regions=((((0, 0), (64, 0), (64, 64), (0, 64)),),), + image_size=(64, 64), + keep_original=True, + ) + + attribute = result.data.attributes[mesh.BALLOON_ATTRIBUTE] + profile = np.empty(len(result.data.vertices), dtype=np.float32) + attribute.data.foreach_get("value", profile) + assert profile.min() == 0.0 and profile.max() > 0.0 + + +def test_depth_surface_object_normal_is_remapped_to_cutout_axes(): + bpy.ops.wm.read_factory_settings(use_empty=True) + image = bpy.data.images.new( + "Surface Normal", + width=1, + height=1, + alpha=True, + float_buffer=True, + ) + image.pixels.foreach_set((0.5, 0.5, 1.0, 1.0)) + + mesh._orient_cutout_object_normal(image) + + pixels = np.empty(4, dtype=np.float32) + image.pixels.foreach_get(pixels) + assert np.allclose(pixels, (0.0, 0.5, 0.5, 1.0), atol=1e-6) + + +def test_internal_mesh_depth_images_use_standard_data_name(): + bpy.ops.wm.read_factory_settings(use_empty=True) + cutout_source = bpy.data.objects.new("Cutout Source", None) + cutout_source.data = bpy.data.images.new( + "Cutout Source.png", width=2, height=2, alpha=True + ) + cutout_depth = mesh._cutout_depth_image( + cutout_source, + np.ones((2, 2), dtype=np.float32), + ) + assert cutout_depth.name == "Cutout Source_depth.exr" + + plane_source = bpy.data.objects.new("Plane Source", None) + plane_source.data = bpy.data.images.new( + "Plane Source.png", width=2, height=2, alpha=True + ) + anchored = np.zeros((2, 2, 3), dtype=np.float32) + plane_depth = mesh._depth_field_image(plane_source, anchored) + assert plane_depth.name == "Plane Source_depth.exr" + + + def test_depth_surface_protected_smooth_evaluates(): bpy.ops.wm.read_factory_settings(use_empty=True) image = bpy.data.images.new("Surface", width=64, height=64) @@ -396,8 +618,8 @@ def test_depth_surface_protected_smooth_evaluates(): keep_original=True, ) - assert mesh.BOUNDARY_ATTRIBUTE in result.data.attributes - assert mesh.DEPTH_PROTECTION_ATTRIBUTE in result.data.attributes + assert all(abs(vertex.co.x) < 1e-8 for vertex in result.data.vertices) + assert mesh.BALLOON_ATTRIBUTE in result.data.attributes modifier = result.modifiers[0] mesh.set_modifier_input( modifier, @@ -407,9 +629,9 @@ def test_depth_surface_protected_smooth_evaluates(): evaluated = result.evaluated_get(bpy.context.evaluated_depsgraph_get()) baseline_mesh = evaluated.to_mesh() try: - baseline_z = np.asarray([vertex.co.z for vertex in baseline_mesh.vertices]) - baseline_min = baseline_z.min() - baseline_max = baseline_z.max() + baseline_x = np.asarray([vertex.co.x for vertex in baseline_mesh.vertices]) + baseline_min = baseline_x.min() + baseline_max = baseline_x.max() finally: evaluated.to_mesh_clear() @@ -424,9 +646,9 @@ def test_depth_surface_protected_smooth_evaluates(): evaluated_mesh = evaluated.to_mesh() try: assert len(evaluated_mesh.polygons) > 0 - z = np.asarray([vertex.co.z for vertex in evaluated_mesh.vertices]) - assert np.isclose(z.max(), baseline_max, atol=1e-5) - assert np.isclose(z.min(), baseline_min - 1.0, atol=1e-5) + x = np.asarray([vertex.co.x for vertex in evaluated_mesh.vertices]) + assert np.isclose(x.min(), baseline_min, atol=1e-5) + assert np.isclose(x.max(), baseline_max + 1.0, atol=1e-5) finally: evaluated.to_mesh_clear() @@ -497,6 +719,361 @@ def test_depth_surface_protected_smooth_evaluates(): assert np.allclose(internal_positions, external_positions, atol=1e-6) + +def _evaluated_positions(obj): + evaluated = obj.evaluated_get(bpy.context.evaluated_depsgraph_get()) + result = evaluated.to_mesh() + try: + return np.asarray([tuple(vertex.co) for vertex in result.vertices]) + finally: + evaluated.to_mesh_clear() + + +def test_cutout_thickness_keeps_one_connected_shell(): + bpy.ops.wm.read_factory_settings(use_empty=True) + vertices = [ + (0.0, float(y), float(z)) + for z in range(3) + for y in range(3) + ] + faces = [] + for z in range(2): + for y in range(2): + a = z * 3 + y + faces.extend(((a, a + 4, a + 1), (a, a + 3, a + 4))) + data = bpy.data.meshes.new("Connected Cutout") + data.from_pydata(vertices, (), faces) + obj = bpy.data.objects.new("Connected Cutout", data) + bpy.context.collection.objects.link(obj) + modifier = obj.modifiers.new("O Image Surface", "NODES") + modifier.node_group = mesh._mesh_node_group(mesh.CUTOUT) + mesh.set_modifier_input( + modifier, + mesh.modifier_input_identifier(modifier.node_group, "Cleanup"), + 0.0, + ) + mesh.set_modifier_input( + modifier, + mesh.modifier_input_identifier(modifier.node_group, "Thickness"), + 1.0, + ) + + evaluated = obj.evaluated_get(bpy.context.evaluated_depsgraph_get()) + result = evaluated.to_mesh() + try: + assert len(result.vertices) == 18 + assert len(result.polygons) == 24 + x = np.asarray([vertex.co.x for vertex in result.vertices]) + assert np.isclose(x.min(), -0.5, atol=1e-6) + assert np.isclose(x.max(), 0.5, atol=1e-6) + front_faces = [ + polygon for polygon in result.polygons if polygon.normal.x < -0.9 + ] + back_faces = [ + polygon for polygon in result.polygons if polygon.normal.x > 0.9 + ] + side_faces = [ + polygon for polygon in result.polygons if abs(polygon.normal.x) < 0.1 + ] + assert len(front_faces) == 8 + assert len(back_faces) == 8 + assert len(side_faces) == 8 + center = Vector((0.0, 1.0, 1.0)) + for polygon in side_faces: + radial = polygon.center - center + assert polygon.normal.dot(radial) > 0.0 + finally: + evaluated.to_mesh_clear() + + mesh.set_modifier_input( + modifier, + mesh.modifier_input_identifier(modifier.node_group, "Thickness"), + 2.4, + ) + obj.update_tag(refresh={"DATA"}) + bpy.context.view_layer.update() + evaluated = obj.evaluated_get(bpy.context.evaluated_depsgraph_get()) + result = evaluated.to_mesh() + try: + assert len(result.vertices) == 34 + assert len(result.polygons) == 40 + side_faces = [ + polygon for polygon in result.polygons if abs(polygon.normal.x) < 0.1 + ] + assert len(side_faces) == 24 + for polygon in side_faces: + coordinates = [result.vertices[index].co for index in polygon.vertices] + edge_lengths = [ + (coordinates[(index + 1) % len(coordinates)] - coordinate).length + for index, coordinate in enumerate(coordinates) + ] + assert max(edge_lengths) / min(edge_lengths) <= 1.25 + 1e-6 + finally: + evaluated.to_mesh_clear() + + +def test_balloon_uses_injected_profile_at_default_height(): + bpy.ops.wm.read_factory_settings(use_empty=True) + vertices = [ + (0.0, float(y), float(z)) + for z in range(3) + for y in range(3) + ] + faces = [] + for z in range(2): + for y in range(2): + a = z * 3 + y + faces.extend(((a, a + 1, a + 4), (a, a + 4, a + 3))) + data = bpy.data.meshes.new("Balloon Profile") + data.from_pydata(vertices, (), faces) + mesh._assign_float_point_attribute( + data, + mesh.BALLOON_ATTRIBUTE, + (0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0), + ) + obj = bpy.data.objects.new("Balloon Profile", data) + bpy.context.collection.objects.link(obj) + modifier = obj.modifiers.new("O Image Balloon", "NODES") + modifier.node_group = mesh._mesh_node_group(mesh.BALLOON) + mesh.set_modifier_input( + modifier, + mesh.modifier_input_identifier(modifier.node_group, "Cleanup"), + 0.0, + ) + mesh.set_modifier_input( + modifier, + mesh.modifier_input_identifier(modifier.node_group, "Thickness"), + 1.0, + ) + + bpy.context.view_layer.update() + positions = _evaluated_positions(obj) + assert np.isclose(positions[:, 0].min(), -1.0, atol=1e-6) + assert np.isclose(positions[:, 0].max(), 1.0, atol=1e-6) + assert np.isclose( + positions[:, 0].min(), + -positions[:, 0].max(), + atol=1e-5, + ) + evaluated = obj.evaluated_get(bpy.context.evaluated_depsgraph_get()) + result = evaluated.to_mesh() + try: + baseline_face_count = len(result.polygons) + finally: + evaluated.to_mesh_clear() + mesh.set_modifier_input( + modifier, + mesh.modifier_input_identifier(modifier.node_group, "Thickness"), + 2.0, + ) + obj.update_tag(refresh={"DATA"}) + bpy.context.view_layer.update() + evaluated = obj.evaluated_get(bpy.context.evaluated_depsgraph_get()) + result = evaluated.to_mesh() + try: + assert len(result.polygons) == baseline_face_count + x = np.asarray([vertex.co.x for vertex in result.vertices]) + assert np.isclose(x.min(), -2.0, atol=1e-6) + assert np.isclose(x.max(), 2.0, atol=1e-6) + finally: + evaluated.to_mesh_clear() + mesh.set_modifier_input( + modifier, + mesh.modifier_input_identifier(modifier.node_group, "Thickness"), + 0.0, + ) + obj.update_tag(refresh={"DATA"}) + bpy.context.view_layer.update() + zero_positions = _evaluated_positions(obj) + assert len(zero_positions) == len(vertices) + assert np.allclose(zero_positions[:, 0], 0.0, atol=1e-6) + + +def test_cutout_cleanup_removes_only_small_mesh_island(): + bpy.ops.wm.read_factory_settings(use_empty=True) + vertices = [ + (0.0, float(y), float(z)) + for z in range(3) + for y in range(3) + ] + vertices.extend( + ((0.0, 5.0, 5.0), (0.0, 5.01, 5.0), (0.0, 5.0, 5.01)) + ) + faces = [] + for z in range(2): + for y in range(2): + a = z * 3 + y + faces.extend(((a, a + 1, a + 4), (a, a + 4, a + 3))) + faces.append((9, 10, 11)) + data = bpy.data.meshes.new("Cleanup") + data.from_pydata(vertices, (), faces) + obj = bpy.data.objects.new("Cleanup", data) + bpy.context.collection.objects.link(obj) + modifier = obj.modifiers.new("O Image Surface", "NODES") + modifier.node_group = mesh._mesh_node_group(mesh.CUTOUT) + cleanup_id = mesh.modifier_input_identifier(modifier.node_group, "Cleanup") + + mesh.set_modifier_input(modifier, cleanup_id, 0.0) + bpy.context.view_layer.update() + assert len(_evaluated_positions(obj)) == 12 + + mesh.set_modifier_input(modifier, cleanup_id, 1.0) + obj.update_tag(refresh={"DATA"}) + bpy.context.view_layer.update() + assert len(_evaluated_positions(obj)) == 9 + + +def test_depth_balloon_base_plane_uses_crop_center_and_matching_arrow_direction(): + bpy.ops.wm.read_factory_settings(use_empty=True) + image = bpy.data.images.new("Depth Balloon", width=100, height=100, alpha=True) + source = bpy.data.objects.new("Depth Balloon", None) + source.empty_display_type = "IMAGE" + source.data = image + source.empty_display_size = 2.0 + bpy.context.collection.objects.link(source) + balloon_material = bpy.data.materials.new("Depth Balloon.material") + crop_bounds = (50, 20, 90, 80) + crop = ((0, 0), (40, 0), (40, 60), (0, 60)) + local_bounds = mesh._crop_plane_bounds( + source, + crop_bounds, + (100, 100), + ) + depth_data = { + "depth_signal": np.ones((61, 41), dtype=np.float32), + "baseline_vector": np.zeros(3, dtype=np.float32), + "valid": np.ones((61, 41), dtype=bool), + "origin": (50, 20), + "reference_depth": 1.0, + "crop_size": (40, 60), + "source_size": (100, 100), + } + + result = mesh._build_mesh_object( + bpy.context, + source, + mesh.DEPTH_BALLOON, + 8, + balloon_material, + depth_data=depth_data, + regions=((crop,),), + image_size=(40, 60), + bounds=local_bounds, + keep_original=True, + ) + + assert all(abs(vertex.co.x) < 1e-8 for vertex in result.data.vertices) + assert mesh.BALLOON_ATTRIBUTE in result.data.attributes + assert "o_balloon_depth" not in result.data.attributes + assert "o_depth_offset" not in result.data.attributes + modifier = result.modifiers[0] + base_plane_id = mesh.modifier_input_identifier( + modifier.node_group, + "Base Plane", + ) + depth_image_id = mesh.modifier_input_identifier( + modifier.node_group, + "Depth Image", + ) + base_plane = modifier[base_plane_id] + assert isinstance(modifier[depth_image_id], bpy.types.Image) + expected_center = np.zeros(2, dtype=np.float64) + assert np.allclose(base_plane.location[1:], expected_center, atol=1e-5) + crop_center = Vector( + ( + (local_bounds[0] + local_bounds[1]) * 0.5, + (local_bounds[2] + local_bounds[3]) * 0.5, + 0.0, + ) + ) + expected_origin = source.matrix_world @ crop_center + assert np.allclose( + tuple(result.matrix_world.translation), + tuple(expected_origin), + atol=1e-5, + ) + arrow = base_plane.rotation_euler.to_matrix() @ Vector((0.0, 0.0, 1.0)) + assert np.allclose(tuple(arrow), (-1.0, 0.0, 0.0), atol=1e-5) + + depth_reference_id = mesh.modifier_input_identifier( + modifier.node_group, + "Depth Reference", + ) + assert modifier[depth_reference_id] > 0.0 + thickness_id = mesh.modifier_input_identifier( + modifier.node_group, + "Thickness", + ) + positions = _evaluated_positions(result) + center_distance = np.linalg.norm(positions[:, 1:] - expected_center, axis=1) + nearest_center = center_distance <= center_distance.min() + 1e-6 + depth_at_thickness_one = positions[nearest_center, 0].min() + mesh.set_modifier_input(modifier, thickness_id, 2.0) + result.update_tag(refresh={"DATA"}) + bpy.context.view_layer.update() + positions = _evaluated_positions(result) + center_distance = np.linalg.norm(positions[:, 1:] - expected_center, axis=1) + nearest_center = center_distance <= center_distance.min() + 1e-6 + depth_at_thickness_two = positions[nearest_center, 0].min() + assert np.isclose( + depth_at_thickness_one, + depth_at_thickness_two, + atol=1e-5, + ) + mesh.set_modifier_input(modifier, thickness_id, 1.0) + result.update_tag(refresh={"DATA"}) + bpy.context.view_layer.update() + + before_positions = _evaluated_positions(result) + base_plane.location += arrow * 0.2 + result.update_tag(refresh={"DATA"}) + bpy.context.view_layer.update() + after_positions = _evaluated_positions(result) + assert np.isclose( + (after_positions[:, 0] - before_positions[:, 0]).min(), + -0.2, + atol=1e-5, + ) + + depth_scale_id = mesh.modifier_input_identifier( + modifier.node_group, + "Depth Scale", + ) + mesh.set_modifier_input(modifier, depth_scale_id, 0.0) + result.update_tag(refresh={"DATA"}) + bpy.context.view_layer.update() + zero_scale_before = _evaluated_positions(result) + base_plane.location += arrow * 0.15 + base_plane.rotation_euler.rotate_axis("Y", np.radians(12.0)) + result.update_tag(refresh={"DATA"}) + bpy.context.view_layer.update() + zero_scale_after = _evaluated_positions(result) + assert np.allclose(zero_scale_after, zero_scale_before, atol=1e-5) + mesh.set_modifier_input(modifier, depth_scale_id, 1.0) + + double_sided_id = mesh.modifier_input_identifier( + modifier.node_group, + "Double Sided", + ) + mesh.set_modifier_input(modifier, double_sided_id, True) + result.update_tag(refresh={"DATA"}) + bpy.context.view_layer.update() + mirrored = _evaluated_positions(result) + layers = {} + for x, y, z in mirrored: + layers.setdefault((round(float(y), 6), round(float(z), 6)), []).append(x) + assert layers + for x_values in layers.values(): + if len(x_values) == 1: + assert np.isclose(x_values[0], 0.0, atol=1e-5) + continue + assert np.isclose( + min(x_values) + max(x_values), + 0.0, + atol=1e-5, + ) + + def test_loaded_normal_image_uses_normal_texture_name(tmp_path): from PIL import Image @@ -558,9 +1135,15 @@ def test_local_cutout_builds_from_blender_image_pixels_without_a_job(): assert len(result.data.vertices) > 4 assert len(result.data.polygons) > 1 assert all(polygon.use_smooth for polygon in result.data.polygons) - vertex_x = [vertex.co.x for vertex in result.data.vertices] - assert min(vertex_x) < source_bounds[0] - assert max(vertex_x) > source_bounds[1] + assert all(abs(vertex.co.x) < 1e-8 for vertex in result.data.vertices) + assert mesh.BALLOON_ATTRIBUTE in result.data.attributes + source_inverse = source.matrix_world.inverted() + source_x = [ + (source_inverse @ (result.matrix_world @ vertex.co)).x + for vertex in result.data.vertices + ] + assert min(source_x) < source_bounds[0] + assert max(source_x) > source_bounds[1] assert bpy.data.objects.get(source.name) is source assert result.data.materials[0].node_tree is not None color_texture = next( diff --git a/tests/test_pack.py b/tests/test_pack.py index 0378384..0d5c011 100644 --- a/tests/test_pack.py +++ b/tests/test_pack.py @@ -67,7 +67,13 @@ class PackTest(unittest.TestCase): self.assertIn("operators/depth_plane/operator.py", names) self.assertIn("operators/mesh_plane.py", names) self.assertIn("operators/image.py", names) - self.assertIn("operators/balloon/operator.py", names) + self.assertIn("operators/cutout_geometry.py", names) + self.assertIn("operators/cutout_nodes.py", names) + self.assertIn("operators/cutout_depth.py", names) + self.assertIn("operators/cutout_profile.py", names) + self.assertIn("operators/cutout_teddy.py", names) + self.assertNotIn("operators/cutout_poisson.py", names) + self.assertNotIn("operators/balloon/operator.py", names) self.assertIn("operators/image_edit/__init__.py", names) self.assertIn("operators/image_edit/common.py", names) self.assertIn("operators/image_edit/mask.py", names) @@ -76,6 +82,7 @@ class PackTest(unittest.TestCase): self.assertIn("operators/image_edit/line.py", names) self.assertIn("operators/image_edit/polyline.py", names) self.assertIn("operators/image_edit/perspective.py", names) + self.assertIn("operators/image_edit/interaction.py", names) self.assertIn("operators/surface.py", names) self.assertIn( "assets/O_AnyImage.blend", @@ -87,6 +94,8 @@ class PackTest(unittest.TestCase): self.assertIn("server/geometry/surface.py", names) self.assertIn("server/jobs/background.py", names) self.assertIn("server/jobs/image_mask.py", names) + self.assertIn("server/jobs/cutout_depth.py", names) + self.assertNotIn("server/jobs/balloon.py", names) self.assertIn("server/models/ben2.py", names) self.assertIn("server/models/da3.py", names) self.assertIn("server/models/moge2.py", names) diff --git a/tests/test_server.py b/tests/test_server.py index 9b506bd..19adc00 100644 --- a/tests/test_server.py +++ b/tests/test_server.py @@ -41,7 +41,7 @@ class ServerTest(unittest.TestCase): @classmethod def setUpClass(cls): cls.server_module = load_server_module() - importlib.import_module("server.jobs.balloon") + importlib.import_module("server.jobs.cutout_depth") importlib.import_module("server.jobs.mesh") from blendjob import JobServer from server.model_manager import ModelManager, ModelSessionCache @@ -415,32 +415,6 @@ class ServerTest(unittest.TestCase): self.assertEqual(status["state"], "succeeded") - def test_balloon_lasso_input_crops_and_clears_outside_rgba(self): - import numpy as np - from PIL import Image - - balloon = sys.modules["server.jobs.balloon"] - with tempfile.TemporaryDirectory() as directory: - root = Path(directory) - source_path = root / "source.png" - output_path = root / "lasso.png" - pixels = np.full((6, 6, 4), (20, 40, 60, 255), dtype=np.uint8) - pixels[1, 1, 3] = 64 - Image.fromarray(pixels, mode="RGBA").save(source_path) - - _path, _selection, origin, source_size = balloon.create_lasso_input( - source_path, - ((1, 1), (5, 1), (1, 5)), - output_path, - ) - result = np.asarray(Image.open(output_path).convert("RGBA")) - - self.assertEqual(origin, (1, 1)) - self.assertEqual(source_size, (6, 6)) - self.assertEqual(result.shape, (4, 4, 4)) - self.assertEqual(tuple(result[0, 0]), (20, 40, 60, 64)) - self.assertEqual(tuple(result[3, 3]), (0, 0, 0, 0)) - def test_polygon_selection_mask_uses_even_odd_fill_for_self_intersection(self): image_mask = sys.modules["server.jobs.image_mask"] polygon = ( @@ -783,7 +757,7 @@ class ServerTest(unittest.TestCase): ) with patch.object(mesh_job.moge2, "infer", side_effect=infer): - depth_path, normal_path = mesh_job._write_balloon_depth( + depth_path, normal_path = mesh_job._write_depth_balloon( context, {"device": "cpu"}, source_path, @@ -909,53 +883,10 @@ class ServerTest(unittest.TestCase): ) alpha = np.asarray(Image.open(output_path).convert("RGBA"))[:, :, 3] - def test_balloon_job_sends_the_masked_crop_to_ben2(self): - import numpy as np - from PIL import Image - - balloon = sys.modules["server.jobs.balloon"] - source_path = self.storage_root / "source.png" - pixels = np.full((6, 6, 4), (20, 40, 60, 255), dtype=np.uint8) - Image.fromarray(pixels, mode="RGBA").save(source_path) - job_directory = self.storage_root / "job" - context = SimpleNamespace( - directory=job_directory, - progress=lambda *_args: None, - check_cancelled=lambda: None, - resource=lambda _name: SimpleNamespace( - directory=lambda _model: self.storage_root / "model" - ), - ) - received = [] - - def infer(input_path, *_args, **_kwargs): - image = Image.open(input_path).convert("RGBA") - received.append(np.asarray(image)) - return image, 0.0, 1.0 - - with patch.object(balloon, "infer_foreground", side_effect=infer): - result = balloon.run( - context, - { - "input": str(source_path), - "polygon": ((1, 1), (5, 1), (1, 5)), - "contour_analysis": True, - "depth_analysis": False, - "contour_model": "BEN2_BASE", - "device": "cpu", - }, - ) - - self.assertEqual(result, {"contour": "contour.json"}) - self.assertEqual(tuple(received[0][3, 3]), (0, 0, 0, 0)) - self.assertTrue((job_directory / "lasso-input.png").is_file()) - self.assertTrue((job_directory / "foreground.png").is_file()) - self.assertTrue((job_directory / "contour.json").is_file()) - def test_balloon_depth_analysis_normalizes_moge_distance(self): import numpy as np - balloon = sys.modules["server.jobs.balloon"] + balloon = sys.modules["server.jobs.cutout_depth"] y, x = np.indices((7, 9), dtype=np.float32) depth = 2.0 + x * 0.1 + y * 0.2 depth[3, 4] -= 0.5 @@ -996,10 +927,44 @@ class ServerTest(unittest.TestCase): places=5, ) + def test_balloon_depth_plane_is_stable_for_a_slender_diagonal_mask(self): + import numpy as np + + balloon = sys.modules["server.jobs.cutout_depth"] + y, x = np.indices((101, 101), dtype=np.float64) + normalized_x = x / 50.0 - 1.0 + normalized_y = y / 50.0 - 1.0 + distance = y - x + selection = np.abs(distance) <= 1.0 + along = (normalized_x + normalized_y) * 0.5 + depth = 4.0 + 0.4 * along + depth += np.where(distance > 0.0, 0.05, 0.0) + depth -= np.where(distance < 0.0, 0.05, 0.0) + + _signal, valid, _reference, baseline = balloon.normalized_moge_depth( + depth, + np.ones_like(selection), + selection, + ) + fitted = ( + baseline[0] * normalized_x + + baseline[1] * normalized_y + + baseline[2] + ) + + self.assertLessEqual( + float(np.ptp(fitted)), + float(np.ptp(fitted[valid])) * 1.25, + ) + self.assertGreater( + abs(float(np.corrcoef(fitted[valid], along[valid])[0, 1])), + 0.99, + ) + def test_balloon_relative_depth_ignores_absolute_distance_and_scale(self): import numpy as np - balloon = sys.modules["server.jobs.balloon"] + balloon = sys.modules["server.jobs.cutout_depth"] y, x = np.indices((8, 10), dtype=np.float32) depth = 4.0 + x * 0.04 + y * 0.03 depth[2:6, 3:7] -= np.asarray( @@ -1022,100 +987,5 @@ class ServerTest(unittest.TestCase): np.testing.assert_allclose(rescaled[0], original[0], atol=2e-4) np.testing.assert_allclose(rescaled[3], original[3], atol=2e-4) - def test_balloon_depth_job_sends_masked_crop_only_to_moge(self): - import numpy as np - from PIL import Image - - balloon = sys.modules["server.jobs.balloon"] - source_path = self.storage_root / "depth-source.png" - pixels = np.full((6, 6, 4), (80, 100, 120, 255), dtype=np.uint8) - Image.fromarray(pixels, mode="RGBA").save(source_path) - job_directory = self.storage_root / "depth-job" - context = SimpleNamespace( - directory=job_directory, - progress=lambda *_args: None, - check_cancelled=lambda: None, - resource=lambda _name: SimpleNamespace(), - ) - received = [] - - def infer(_manager, _parameters, images, **_kwargs): - image = Image.open(images[0]).convert("RGBA") - received.append(np.asarray(image)) - height, width = image.size[1], image.size[0] - depth = np.linspace( - 1.0, - 2.0, - height * width, - dtype=np.float32, - ).reshape(height, width) - return SimpleNamespace( - depth=depth[None], - mask=np.ones((1, height, width), dtype=bool), - normal=np.broadcast_to( - np.asarray((0.0, 0.0, -1.0), dtype=np.float32), - (1, height, width, 3), - ).copy(), - ) - - with ( - patch.object(balloon.moge2, "infer", side_effect=infer), - patch.object( - balloon, - "infer_foreground", - side_effect=AssertionError("BEN2 should not run"), - ), - ): - result = balloon.run( - context, - { - "input": str(source_path), - "polygon": ((1, 1), (5, 1), (1, 5)), - "contour_analysis": False, - "depth_analysis": True, - "depth_model": "MOGE2_VITS_NORMAL", - "resolution_level": 5, - "device": "cpu", - }, - ) - - self.assertEqual( - result, - {"depth": "depth.npz", "normal": "normal.png"}, - ) - self.assertEqual(tuple(received[0][3, 3]), (0, 0, 0, 0)) - self.assertFalse((job_directory / "foreground.png").exists()) - with np.load(job_directory / "depth.npz", allow_pickle=False) as data: - self.assertEqual(data["depth_signal"].shape, (4, 4)) - self.assertEqual(data["baseline_vector"].shape, (3,)) - self.assertEqual(tuple(data["origin"]), (1, 1)) - self.assertEqual(tuple(data["source_size"]), (6, 6)) - self.assertTrue(data["valid"].any()) - normal = np.asarray(Image.open(job_directory / "normal.png").convert("RGB")) - self.assertEqual(normal.shape, (6, 6, 3)) - self.assertEqual(tuple(normal[0, 0]), (128, 128, 255)) - - def test_balloon_contour_uses_alpha_above_half_and_largest_component(self): - import numpy as np - - balloon = sys.modules["server.jobs.balloon"] - alpha = np.zeros((6, 7), dtype=np.uint8) - alpha[0, 0] = 255 - alpha[1:4, 2:5] = 128 - alpha[1, 1] = 127 - selection_mask = np.ones_like(alpha, dtype=bool) - - contour = balloon.contour_from_alpha( - alpha, - selection_mask, - origin=(10, 20), - ) - - self.assertEqual( - set(map(tuple, contour)), - {(12, 21), (15, 21), (15, 24), (12, 24)}, - ) - - if __name__ == "__main__": unittest.main() diff --git a/tools/validate_balloon_asset.py b/tools/validate_balloon_asset.py deleted file mode 100644 index 7e0248c..0000000 --- a/tools/validate_balloon_asset.py +++ /dev/null @@ -1,135 +0,0 @@ -from pathlib import Path -import sys - -import bpy - - -PROJECT_ROOT = Path(__file__).parents[1] -sys.path.insert(0, str(PROJECT_ROOT / "src")) -sys.path.insert( - 0, - str(next((PROJECT_ROOT / "src" / "anyimage" / "wheels").glob("blendjob-*.whl"))), -) - -from anyimage.operators.balloon.depth import ( - BALLOON_DEPTH_ATTRIBUTE, - add_balloon_shape_modifier, - balloon_shape_node_group, -) -from anyimage.operators.balloon.operator import ( - mark_balloon_material, - merge_balloon_normal_image, - reusable_balloon_material, -) -from anyimage.operators.material import ( - BALLOON_MATERIAL_NODE_GROUP_NAME, - material_node_group, - set_modifier_input, -) - - -def input_socket(group, name): - return next( - item - for item in group.interface.items_tree - if item.item_type == "SOCKET" - and item.in_out == "INPUT" - and item.name == name - ) - - -def main(): - bpy.ops.wm.read_factory_settings(use_empty=True) - group = balloon_shape_node_group() - assert group.is_modifier - assert input_socket(group, "Double Sided Depth").default_value is False - smoother = next( - node - for node in group.nodes - if node.bl_idname == "ShaderNodeMapRange" - and node.interpolation_type == "SMOOTHERSTEP" - ) - assert smoother.clamp - assert not any( - node.bl_idname == "ShaderNodeMath" - and node.operation == "MAXIMUM" - and node.label == "Clamp Depth" - for node in group.nodes - ) - - mesh = bpy.data.meshes.new("Balloon Validation Mesh") - mesh.from_pydata( - ( - (-1.0, 0.0, 0.0), - (-0.5, 0.0, 0.0), - (0.0, 0.0, 0.0), - (1.0, 0.0, 0.0), - ), - (), - (), - ) - depth = mesh.attributes.new(BALLOON_DEPTH_ATTRIBUTE, "FLOAT", "POINT") - depth.data.foreach_set("value", (0.0, 2.0, 2.0, 2.0)) - obj = bpy.data.objects.new("Balloon Validation", mesh) - bpy.context.scene.collection.objects.link(obj) - modifier = add_balloon_shape_modifier(obj, 1.0) - evaluated = obj.evaluated_get(bpy.context.evaluated_depsgraph_get()) - result = evaluated.to_mesh() - try: - x = [vertex.co.x for vertex in result.vertices] - assert abs(x[0] + 1.0) < 1e-5 - assert abs(x[1] + 1.25) < 1e-5 - assert abs(x[2]) < 1e-5 - assert abs(x[3] - 1.0) < 1e-5 - finally: - evaluated.to_mesh_clear() - - set_modifier_input(modifier, input_socket(group, "Thickness").identifier, 2.0) - obj.update_tag(refresh={"DATA"}) - bpy.context.view_layer.update() - evaluated = obj.evaluated_get(bpy.context.evaluated_depsgraph_get()) - result = evaluated.to_mesh() - try: - x = [vertex.co.x for vertex in result.vertices] - assert abs(x[0] + 1.0) < 1e-5 - assert abs(x[3] - 2.0) < 1e-5 - finally: - evaluated.to_mesh_clear() - - material_group = material_node_group( - surface_group_name=BALLOON_MATERIAL_NODE_GROUP_NAME, - ) - normal_maps = { - node.space - for node in material_group.nodes - if node.bl_idname == "ShaderNodeNormalMap" - } - assert normal_maps == {"TANGENT", "OBJECT"} - assert input_socket(material_group, "Object Space").default_value is False - assert input_socket(material_group, "Normal Scale").default_value == 1.0 - - source_image = bpy.data.images.new("Reuse Source", 2, 1) - shared_normal = bpy.data.images.new("Shared Normal", 2, 1) - shared_normal.pixels.foreach_set( - (128 / 255, 128 / 255, 1, 1, 128 / 255, 128 / 255, 1, 1) - ) - material = bpy.data.materials.new("Unrelated Material Name") - mark_balloon_material(material, source_image, shared_normal) - assert reusable_balloon_material( - source_image, - normal_required=True, - ) is material - - update_normal = bpy.data.images.new("Normal Update", 2, 1) - update_normal.pixels.foreach_set( - (1, 0.25, 0.75, 1, 128 / 255, 128 / 255, 1, 1) - ) - merge_balloon_normal_image(shared_normal, update_normal) - pixels = list(shared_normal.pixels) - assert abs(pixels[0] - 1.0) < 1e-5 - assert abs(pixels[4] - 128 / 255) < 1e-5 - print(f"Validated Blender {bpy.app.version_string}: {group.name}") - - -if __name__ == "__main__": - main() diff --git a/tools/validate_cutout_asset.py b/tools/validate_cutout_asset.py new file mode 100644 index 0000000..2270c2b --- /dev/null +++ b/tools/validate_cutout_asset.py @@ -0,0 +1,165 @@ +from pathlib import Path +import sys + +import bpy + + +PROJECT_ROOT = Path(__file__).parents[1] +sys.path.insert(0, str(PROJECT_ROOT / "src")) +sys.path.insert( + 0, + str(next((PROJECT_ROOT / "src" / "anyimage" / "wheels").glob("blendjob-*.whl"))), +) + +from anyimage.operators import mesh +from anyimage.operators.material import modifier_input_identifier, set_modifier_input + + +def evaluated_mesh(obj): + evaluated = obj.evaluated_get(bpy.context.evaluated_depsgraph_get()) + result = evaluated.to_mesh() + return evaluated, result + + +def grid_object(name): + vertices = [ + (0.0, float(y), float(z)) + for z in range(3) + for y in range(3) + ] + faces = [] + for z in range(2): + for y in range(2): + a = z * 3 + y + faces.extend(((a, a + 4, a + 1), (a, a + 3, a + 4))) + data = bpy.data.meshes.new(name) + data.from_pydata(vertices, (), faces) + obj = bpy.data.objects.new(name, data) + bpy.context.scene.collection.objects.link(obj) + return obj + + +def add_modifier(obj, shape, thickness): + group = mesh._mesh_node_group(shape) + modifier = obj.modifiers.new(group.name, "NODES") + modifier.node_group = group + set_modifier_input( + modifier, + modifier_input_identifier(group, "Cleanup"), + 0.0, + ) + set_modifier_input( + modifier, + modifier_input_identifier(group, "Thickness"), + thickness, + ) + return modifier + + +def main(): + bpy.ops.wm.read_factory_settings(use_empty=True) + groups = { + shape: mesh._mesh_node_group(shape) + for shape in ( + mesh.CUTOUT, + mesh.BALLOON, + mesh.DEPTH_BALLOON, + mesh.DEPTH_SURFACE, + ) + } + assert [groups[shape].name for shape in groups] == [ + "O Image Surface", + "O Image Balloon", + "O Image Depth Balloon", + "O Image Depth Surface", + ] + repeat_counts = { + shape: sum( + node.bl_idname == "GeometryNodeRepeatInput" + for node in group.nodes + ) + for shape, group in groups.items() + } + assert repeat_counts == { + mesh.CUTOUT: 1, + mesh.BALLOON: 0, + mesh.DEPTH_BALLOON: 0, + mesh.DEPTH_SURFACE: 2, + } + for group in groups.values(): + assert group.is_modifier + assert all(not node.label for node in group.nodes) + assert all( + link.from_node.location.x <= link.to_node.location.x + for link in group.links + ) + group_nodes = list(group.nodes) + assert len({tuple(node.location) for node in group_nodes}) == len(group_nodes) + for index, node in enumerate(group_nodes): + for other in group_nodes[index + 1:]: + assert not ( + abs(node.location.x - other.location.x) < 150.0 + and abs(node.location.y - other.location.y) < 100.0 + ) + for group_input in ( + node for node in group_nodes if node.bl_idname == "NodeGroupInput" + ): + assert all( + output.is_linked or output.hide + for output in group_input.outputs + if output.name + ) + for item in group.interface.items_tree: + if item.item_type == "SOCKET" and item.in_out == "INPUT": + assert item.structure_type == "SINGLE" + + surface = grid_object("Surface Validation") + add_modifier(surface, mesh.CUTOUT, 2.4) + evaluated, result = evaluated_mesh(surface) + try: + assert len(result.vertices) == 34 + assert len(result.polygons) == 40 + side_faces = [ + polygon for polygon in result.polygons + if abs(polygon.normal.x) < 0.1 + ] + assert len(side_faces) == 24 + finally: + evaluated.to_mesh_clear() + + balloon = grid_object("Balloon Validation") + profile = balloon.data.attributes.new(mesh.BALLOON_ATTRIBUTE, "FLOAT", "POINT") + profile.data.foreach_set( + "value", + (0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0), + ) + modifier = add_modifier(balloon, mesh.BALLOON, 1.0) + evaluated, result = evaluated_mesh(balloon) + try: + initial_faces = len(result.polygons) + initial_extent = max(abs(vertex.co.x) for vertex in result.vertices) + finally: + evaluated.to_mesh_clear() + set_modifier_input( + modifier, + modifier_input_identifier(modifier.node_group, "Thickness"), + 2.0, + ) + balloon.update_tag(refresh={"DATA"}) + bpy.context.view_layer.update() + evaluated, result = evaluated_mesh(balloon) + try: + assert len(result.polygons) == initial_faces + extent = max(abs(vertex.co.x) for vertex in result.vertices) + assert abs(extent - initial_extent * 2.0) < 1e-5 + finally: + evaluated.to_mesh_clear() + + print( + f"Validated Blender {bpy.app.version_string}: " + f"{', '.join(group.name for group in groups.values())}" + ) + + +if __name__ == "__main__": + main() \ No newline at end of file diff --git a/tools/validate_moge_depth_plane.py b/tools/validate_moge_depth_plane.py index 04c1b75..bdae5f5 100644 --- a/tools/validate_moge_depth_plane.py +++ b/tools/validate_moge_depth_plane.py @@ -161,7 +161,7 @@ def main(): static_depth = next( node.image for node in image_textures - if node.image.name == "MoGe Test.depth" + if node.image.name == "MoGe Test_depth.exr" ) assert static_depth.source != "SEQUENCE" geometry_textures = [