Add geometry nodes asset and validation script
- Implemented a new geometry nodes asset in `build_geometry_nodes_asset.py` to handle mesh refinement and topology fixing. - Created a validation script in `validate_geometry_nodes_asset.py` to ensure the integrity and functionality of the geometry node group. - Added input handling for various parameters including resolution, iterations, and smoothing options. - Included checks for node group structure and attributes to validate the geometry processing pipeline.
This commit is contained in:
19
README.md
19
README.md
@@ -5,6 +5,7 @@ SnapForm 是一个 Blender 扩展,可以把一张照片、一段视频或一
|
||||
## 它能做什么
|
||||
|
||||
- 一键把 Image Empty 转换为立体面片:让 AI 估算深度,或直接用原图作为深度
|
||||
- 把单张 Image Empty 转换为透视纹理网格,并自动匹配原对象的 Transform 与 Offset
|
||||
- 一键去除图片背景,生成透明前景图
|
||||
- 支持视频与序列帧输入,逐帧处理,动画也能得到逐帧对应的立体面片
|
||||
- 全部操作都在右键菜单里完成,不需要写任何代码
|
||||
@@ -27,7 +28,7 @@ DA3、BEN2 源码包和 uv 已内置在扩展里,安装时直接从本地解
|
||||
### 转换为立体面片
|
||||
|
||||
1. 在 3D 视图中选中一个 Image Empty(空物体,显示方式为 Image)
|
||||
2. 右键选择 `Convert to Depth Plane`
|
||||
2. 右键打开 `SnapForm` 子菜单,选择 `Convert to Depth Plane`
|
||||
3. 在对话框中选择:
|
||||
- `Depth from`:`Original` 直接用原图,`AI Generated` 让 AI 估算深度
|
||||
- `Background Depth Distance` 为 `0` 时不剔除背景;大于 `0` 时把背景设为固定距离,AI 与 Original 均支持
|
||||
@@ -36,7 +37,21 @@ DA3、BEN2 源码包和 uv 已内置在扩展里,安装时直接从本地解
|
||||
|
||||
### 去除背景
|
||||
|
||||
选中 Image Empty 后,右键选择 `Remove Background`,完成后 Empty 会自动换成透明前景图。
|
||||
选中 Image Empty 后,右键打开 `SnapForm` 子菜单并选择 `Remove Background`,完成后 Empty 会自动
|
||||
换成透明前景图。该操作在子菜单中使用独立分割区域。
|
||||
|
||||
### 转换为透视网格
|
||||
|
||||
选中单张图片的 Image Empty,右键打开 `SnapForm` 子菜单并选择 `Convert to Mesh`。SnapForm 会按 DA3
|
||||
相机内参与深度逐像素反投影,并把生成网格匹配到原 Empty 的可见尺寸、Offset 与 Transform。DA3
|
||||
confidence、边拉伸与 BEN2 背景结果会保存为 Mesh attributes,再由一个可复用的 Geometry Nodes
|
||||
修改器完成剔面、快速重拓扑、Relax 与 UV 重投。匹配相机默认不创建;开启后会作为 Mesh 的子级。
|
||||
当前此功能仅支持单张图片。
|
||||
|
||||
转换对话框只保留模型设置、`Create Matching Camera`、`Unshaded` 与 `Delete Original`。剔除和重拓扑参数直接在
|
||||
Geometry Nodes 修改器中调整;`Unshaded` 可使用与 Depth Plane 相同的 Emission 材质。普通材质的
|
||||
Bump Distance 会按透视网格的深度空间缩放。Mesh attributes 统一使用 `o_` 前缀;Host 准备过程显示
|
||||
`Post-processing n/6`,可看到当前阶段与总步骤。
|
||||
|
||||
### 视频与序列帧
|
||||
|
||||
|
||||
@@ -81,7 +81,7 @@
|
||||
## Image Empty 去背景
|
||||
|
||||
1. 确认 `Host > Models` 中的 `BEN2 Base` 已就绪;首次安装 Runtime 时会默认下载。
|
||||
2. 在 3D View 选中 Image Empty,右键选择 `Remove Background`。
|
||||
2. 在 3D View 选中 Image Empty,右键打开 `SnapForm` 子菜单并选择 `Remove Background`。
|
||||
3. 完成后以 Fake User 保留原 Image datablock,新建并 Pack 一张透明 PNG 到当前 `.blend`,然后只把
|
||||
当前 Empty 的 `data` 换成新图;Empty 的 Transform、显示尺寸、Offset 及其他设置保持不变。
|
||||
4. 新 Image 使用原图去掉格式后再追加 `.foreground` 的名称。例如 `img.png` 生成
|
||||
@@ -103,7 +103,7 @@ Cyclic 沿用原 Empty 的设置。序列帧无法被 Blender 整体 Pack,因
|
||||
## Image Empty 转深度面片
|
||||
|
||||
1. 在 3D View 选中 `Object Type = Empty`、`Display As = Image` 的对象。
|
||||
2. 右键选择 `Convert to Depth Plane`,通过 `Depth from` 选择深度来源:
|
||||
2. 右键打开 `SnapForm` 子菜单并选择 `Convert to Depth Plane`,通过 `Depth from` 选择深度来源:
|
||||
- `AI Generated`:显示 Model、Use Ray Pose 和 Resolution Scale。实际 Process Res 为图片长边像素数
|
||||
乘以 Resolution Scale;范围为 `0.1–1.0`,默认 `1.0`。计算后的 Process Res 会直接显示在
|
||||
`Resolution Scale (… px)` 标签中。
|
||||
@@ -144,6 +144,58 @@ Cyclic 沿用原 Empty 的设置。序列帧无法被 Blender 整体 Pack,因
|
||||
`8×12` 个面。随后的 Simple Subdivision 修改器使用原生名称,Viewport Levels 固定为
|
||||
`1`,Render Levels 固定为 `2`,不再随 Resolution Scale 或 Subdivision 动态变化。
|
||||
|
||||
## Image Empty 转透视网格
|
||||
|
||||
1. 在 3D View 选中仅包含一帧的 Image Empty,右键打开 `SnapForm` 子菜单并选择
|
||||
`Convert to Mesh`。该入口独立于
|
||||
`Convert to Depth Plane`,当前不处理视频或序列帧。
|
||||
2. Host 请求 DA3 的 `depth`、`conf` 与 `intrinsics`,同时固定运行 BEN2 取得前景 alpha。Depth 不做
|
||||
smoothing,也不在 Host 中剔面或减面;Host 只反投影完整有效拓扑,并计算 confidence percentile、
|
||||
camera-space edge stretch 与 background 标记。
|
||||
3. Blender 把勾选 `Usage > Modifier` 的 `O Refine DA3 Mesh` 节点组直接作为 Geometry Nodes 修改器
|
||||
添加到 Mesh,不另设修改器名称。节点组内没有子节点组,依次完成:
|
||||
- 修改器顶层依次显示 `Resolution`、`Relax Iterations`、`Smooth`;默认展开的 `Culling` Panel
|
||||
包含 `Confidence`、`Stretch`、`Background`。`Stretch = 100%` 明确选择全部面,剔面后同时删除
|
||||
Face Count 为零的孤立顶点;
|
||||
- 每轮 Remesh Iteration 依次执行 Connected Merge、循环拓扑修复与一次 Smooth,使新拓扑在进入下一轮
|
||||
Merge 前先稳定;拓扑修复删除非流形面、单面拓扑和小面积 mesh islands;
|
||||
- 默认 `Resolution` 为原始有效三角边长中位数乘以 `sqrt(10) * 2`。`sqrt(10)` 来自原先 90% 面数缩减
|
||||
的二维边距估计,再乘以 2 为迭代合并留出更平滑、稳定的默认尺度;节点内部 Connected Merge 的
|
||||
Distance 为 `Resolution * 0.5`;
|
||||
- Smooth 采用 `O Smooth` 的边界检测和模糊法线权重,固定开放边界与薄面,只保留一个
|
||||
`Smooth` 程度系数;节点组不再额外执行 Flip Faces;
|
||||
- 从修改前的原始表面以 nearest-surface 重新采样 `UVMap`,最终明确写为 Corner 域 `2D Vector`。
|
||||
4. 节点组随插件放在 `O_SnapForm.blend`。Blender 4.5 首次 Append 后复用同一 datablock;
|
||||
Blender 5.0 以 linked ID 载入后调用 Pack,后续 Mesh 继续复用。插件不会为每个 Mesh 新建节点组,
|
||||
也不会在添加后重命名 Geometry Nodes 修改器。节点组只启用 `Usage > Modifier`,不 Mark as Asset;
|
||||
修改器关闭 `show_group_selector`。`Fix Topology` 位于默认折叠的 `Options` Panel。
|
||||
节点图按 Culling、Remesh、Topology Fix、Island Cleanup 与 UV 分区摆放;各区就近使用独立的
|
||||
Group Input,并隐藏没有连线的接口,避免跨越整个节点图的输入连线。
|
||||
5. Host 准备过程固定显示 6 个编号阶段:验证 DA3 数据、反投影、准备 DA3/BEN2 attributes、构建原始
|
||||
拓扑、投影 UV/attributes、写出网格。状态栏显示 `Post-processing n/6: ...`。
|
||||
6. Host 在 job 根目录写出 `mesh.npz` 与 `texture.png`。NPZ 保存 camera-space `vertices`、
|
||||
`faces`、`uv`、`intrinsics`、`image_size` 与有效深度中位数 `reference_depth`。Blender 读取这些
|
||||
数据后直接创建 Mesh 与原图材质,不需要 Open3D 或 OpenCV 进入 Blender Python 环境。
|
||||
Mesh 保留下列结构化 attributes:
|
||||
- 点域:`o_depth`、`o_confidence`、`o_confidence_percentile`、`o_background`;
|
||||
- 面域:`o_face_confidence`、`o_edge_stretch`、`o_background_face`。
|
||||
7. 网格以 `reference_depth` 截面匹配 Image Empty 的显示矩形:该截面的投影四角与 Empty 的可见
|
||||
边界重合,主点同时匹配 Empty Offset。网格对象直接沿用源 Empty 的 `matrix_world`,因此位置、旋转、
|
||||
缩放及父级关系产生的世界变换保持一致。更近的顶点沿 Empty 局部 `+Z` 方向凸出,三角面正面朝向
|
||||
观察相机,启用 Backface Culling 时可从正面看到。
|
||||
8. 转换对话框不再提供后处理参数,只保留模型设置、`Create Matching Camera`、`Unshaded` 与
|
||||
`Delete Original`。`Unshaded` 使用与 Depth Plane 相同的 Emission/Transparent 材质路径;普通材质的
|
||||
Bump Distance 按 `abs(object.scale.z) × camera scale_z` 缩放,与透视网格实际深度尺度一致。
|
||||
相机默认关闭;开启后会在 Empty 局部 `+Z` 侧创建一台朝向局部 `-Z` 的透视相机,并成为生成 Mesh
|
||||
的子级,使二者移动时保持投影关系。开启 `Delete Original` 时,网格成功创建后删除源 Empty。
|
||||
网格、材质与相机会归入同一个 `Convert to Mesh` Undo 步骤。
|
||||
|
||||
Debug 的 Textured Mesh GLB 会把 depth、confidence 与 confidence percentile 写入自定义顶点
|
||||
attributes。其他 DA3 Debug 三维输出会同时写出未压缩的 `prediction.attributes.npz`,保存完整
|
||||
depth、confidence、intrinsics、extrinsics 与 confidence percentile。当前 trimesh 的 glTF 点图元
|
||||
导出器只写 Position/Color,无法可靠把任意 attribute 嵌入点云 GLB,因此点云先以 companion NPZ
|
||||
保留完整结构;后续若改为 SnapForm 自有 glTF writer,可再把这些字段直接嵌入点图元。
|
||||
|
||||
所有任务使用统一的状态栏进度显示。变化中的阶段说明显示在右侧的进度条内部,取消按钮
|
||||
位于进度条右侧。状态字段支持阶段、当前操作与预计剩余秒数,界面统一组合为紧凑文案并保证
|
||||
进度单调递增。任务进程与 Host 请求在后台启动,按钮点击后立即进入进度状态。Image Depth 的阶段包括 Runtime 启动、输入准备、模型加载、深度估算、缓存
|
||||
@@ -234,6 +286,7 @@ Blender 为 Packed Image 或序列帧生成的输入副本也归档到对应 job
|
||||
- `download-model`
|
||||
- `export-reconstruction`
|
||||
- `generate-depth-plane`
|
||||
- `generate-image-mesh`
|
||||
- `remove-background`
|
||||
- `remove-image-background`
|
||||
- `generate-original-depth-plane`
|
||||
|
||||
@@ -7,8 +7,10 @@ from .image_depth import (
|
||||
BuildOriginalDepthPlane,
|
||||
ConvertImageToDepthPlane,
|
||||
RemoveImageBackground,
|
||||
SnapFormImageMenu,
|
||||
draw_image_context_menu,
|
||||
)
|
||||
from .image_mesh import BuildImageMesh, ConvertImageToMesh
|
||||
from .operators import (
|
||||
CancelJob,
|
||||
ControlHost,
|
||||
@@ -34,11 +36,14 @@ CLASSES = (
|
||||
RunJob,
|
||||
CancelJob,
|
||||
ControlHost,
|
||||
SnapFormImageMenu,
|
||||
ApplyRemovedBackground,
|
||||
RemoveImageBackground,
|
||||
BuildDepthPlane,
|
||||
BuildOriginalDepthPlane,
|
||||
ConvertImageToDepthPlane,
|
||||
BuildImageMesh,
|
||||
ConvertImageToMesh,
|
||||
HostPanel,
|
||||
ModelsPanel,
|
||||
DebugPanel,
|
||||
|
||||
BIN
src/snapform/assets/O_SnapForm.blend
LFS
Normal file
BIN
src/snapform/assets/O_SnapForm.blend
LFS
Normal file
Binary file not shown.
@@ -1783,22 +1783,37 @@ class ConvertImageToDepthPlane(bpy.types.Operator):
|
||||
return {"FINISHED"}
|
||||
|
||||
|
||||
def draw_image_context_menu(self, context):
|
||||
active_object = context.object
|
||||
if not is_image_empty(active_object):
|
||||
return
|
||||
self.layout.separator()
|
||||
row = self.layout.row()
|
||||
row.operator_context = "INVOKE_DEFAULT"
|
||||
row.operator(
|
||||
class SnapFormImageMenu(bpy.types.Menu):
|
||||
bl_idname = "SNAPFORM_MT_image_context"
|
||||
bl_label = "SnapForm"
|
||||
|
||||
def draw(self, _context):
|
||||
layout = self.layout
|
||||
layout.operator_context = "INVOKE_DEFAULT"
|
||||
layout.operator(
|
||||
"snapform.convert_image_to_depth_plane",
|
||||
text="Convert to Depth Plane",
|
||||
icon="MOD_DISPLACE",
|
||||
)
|
||||
row = self.layout.row()
|
||||
row.operator_context = "INVOKE_DEFAULT"
|
||||
row.operator(
|
||||
layout.operator(
|
||||
"snapform.convert_image_to_mesh",
|
||||
text="Convert to Mesh",
|
||||
icon="MESH_GRID",
|
||||
)
|
||||
layout.separator()
|
||||
layout.operator(
|
||||
"snapform.remove_image_background",
|
||||
text="Remove Background",
|
||||
icon="IMAGE_ALPHA",
|
||||
)
|
||||
|
||||
|
||||
def draw_image_context_menu(self, context):
|
||||
if not is_image_empty(context.object):
|
||||
return
|
||||
self.layout.separator()
|
||||
self.layout.menu(
|
||||
SnapFormImageMenu.bl_idname,
|
||||
text="SnapForm",
|
||||
icon="PLUGIN",
|
||||
)
|
||||
|
||||
564
src/snapform/image_mesh.py
Normal file
564
src/snapform/image_mesh.py
Normal file
@@ -0,0 +1,564 @@
|
||||
from pathlib import Path
|
||||
|
||||
import bpy
|
||||
|
||||
from .image_depth import (
|
||||
create_material,
|
||||
image_depth_model_items,
|
||||
is_image_empty,
|
||||
plane_bounds,
|
||||
prepare_image_input,
|
||||
remove_temporary_input,
|
||||
scaled_process_resolution,
|
||||
video_total_frames,
|
||||
)
|
||||
|
||||
|
||||
_last_mesh_model = "BASE"
|
||||
|
||||
POINT_ATTRIBUTES = {
|
||||
"o_depth": ("FLOAT", "point_depth"),
|
||||
"o_confidence": ("FLOAT", "point_confidence"),
|
||||
"o_confidence_percentile": (
|
||||
"FLOAT",
|
||||
"point_confidence_percentile",
|
||||
),
|
||||
"o_background": ("BOOLEAN", "point_background"),
|
||||
}
|
||||
FACE_ATTRIBUTES = {
|
||||
"o_face_confidence": (
|
||||
"FLOAT",
|
||||
"face_confidence_percentile",
|
||||
),
|
||||
"o_edge_stretch": ("FLOAT", "face_edge_stretch"),
|
||||
"o_background_face": ("BOOLEAN", "face_background"),
|
||||
}
|
||||
NODE_GROUP_NAME = "O Refine DA3 Mesh"
|
||||
NODE_ASSET_NAME = "O_SnapForm.blend"
|
||||
|
||||
|
||||
def load_image_mesh_data(npz_path):
|
||||
import numpy as np
|
||||
|
||||
try:
|
||||
with np.load(npz_path, allow_pickle=False) as archive:
|
||||
required = (
|
||||
"vertices",
|
||||
"faces",
|
||||
"uv",
|
||||
"intrinsics",
|
||||
"image_size",
|
||||
"reference_depth",
|
||||
"point_depth",
|
||||
"point_confidence",
|
||||
"point_confidence_percentile",
|
||||
"point_background",
|
||||
"face_confidence_percentile",
|
||||
"face_edge_stretch",
|
||||
"face_background",
|
||||
)
|
||||
missing = [key for key in required if key not in archive.files]
|
||||
if missing:
|
||||
raise ValueError(
|
||||
f"Mesh data is missing: {', '.join(missing)}"
|
||||
)
|
||||
vertices = np.asarray(archive["vertices"], dtype=np.float32)
|
||||
faces = np.asarray(archive["faces"], dtype=np.int32)
|
||||
uv = np.asarray(archive["uv"], dtype=np.float32)
|
||||
intrinsics = np.asarray(
|
||||
archive["intrinsics"],
|
||||
dtype=np.float32,
|
||||
)
|
||||
image_size = tuple(
|
||||
int(value) for value in archive["image_size"].tolist()
|
||||
)
|
||||
reference_depth = float(archive["reference_depth"])
|
||||
attributes = {
|
||||
key: np.asarray(archive[key])
|
||||
for _name, (_data_type, key) in {
|
||||
**POINT_ATTRIBUTES,
|
||||
**FACE_ATTRIBUTES,
|
||||
}.items()
|
||||
}
|
||||
except OSError as error:
|
||||
raise ValueError(f"Unable to read mesh data: {npz_path}") from error
|
||||
if vertices.ndim != 2 or vertices.shape[1] != 3:
|
||||
raise ValueError(f"Unsupported vertices shape: {vertices.shape}")
|
||||
if faces.ndim != 2 or faces.shape[1] != 3:
|
||||
raise ValueError(f"Unsupported faces shape: {faces.shape}")
|
||||
if uv.shape != (len(vertices), 2):
|
||||
raise ValueError("Mesh UV and vertex counts do not match")
|
||||
if intrinsics.shape != (3, 3):
|
||||
raise ValueError(f"Unsupported intrinsics shape: {intrinsics.shape}")
|
||||
if len(image_size) != 2 or min(image_size) <= 0:
|
||||
raise ValueError(f"Unsupported image size: {image_size}")
|
||||
if not reference_depth > 0.0:
|
||||
raise ValueError("Mesh reference depth must be positive")
|
||||
for _name, (_data_type, key) in POINT_ATTRIBUTES.items():
|
||||
if len(attributes[key]) != len(vertices):
|
||||
raise ValueError(f"Point attribute has the wrong length: {key}")
|
||||
for _name, (_data_type, key) in FACE_ATTRIBUTES.items():
|
||||
if len(attributes[key]) != len(faces):
|
||||
raise ValueError(f"Face attribute has the wrong length: {key}")
|
||||
return (
|
||||
vertices,
|
||||
faces,
|
||||
uv,
|
||||
intrinsics,
|
||||
image_size,
|
||||
reference_depth,
|
||||
attributes,
|
||||
)
|
||||
|
||||
|
||||
def empty_anchored_vertices(
|
||||
source_object,
|
||||
camera_vertices,
|
||||
intrinsics,
|
||||
image_size,
|
||||
reference_depth,
|
||||
):
|
||||
import numpy as np
|
||||
|
||||
width, height = image_size
|
||||
x_min, x_max, y_min, y_max = plane_bounds(source_object)
|
||||
plane_width = x_max - x_min
|
||||
plane_height = y_max - y_min
|
||||
pixel_width = max(width - 1, 1)
|
||||
pixel_height = max(height - 1, 1)
|
||||
fx = float(intrinsics[0, 0])
|
||||
fy = float(intrinsics[1, 1])
|
||||
cx = float(intrinsics[0, 2])
|
||||
cy = float(intrinsics[1, 2])
|
||||
if fx <= 0.0 or fy <= 0.0:
|
||||
raise ValueError("Camera intrinsics contain an invalid focal length")
|
||||
scale_x = plane_width * fx / (pixel_width * reference_depth)
|
||||
scale_y = plane_height * fy / (pixel_height * reference_depth)
|
||||
scale_z = float(np.sqrt(scale_x * scale_y))
|
||||
camera_x = x_min + plane_width * cx / pixel_width
|
||||
camera_y = y_max - plane_height * cy / pixel_height
|
||||
vertices = np.empty_like(camera_vertices, dtype=np.float32)
|
||||
vertices[:, 0] = camera_x + camera_vertices[:, 0] * scale_x
|
||||
vertices[:, 1] = camera_y - camera_vertices[:, 1] * scale_y
|
||||
vertices[:, 2] = (
|
||||
reference_depth - camera_vertices[:, 2]
|
||||
) * scale_z
|
||||
camera_location = (
|
||||
camera_x,
|
||||
camera_y,
|
||||
reference_depth * scale_z,
|
||||
)
|
||||
camera_parameters = {
|
||||
"scale_x": scale_x,
|
||||
"scale_y": scale_y,
|
||||
"scale_z": scale_z,
|
||||
"location": camera_location,
|
||||
}
|
||||
return vertices, camera_parameters
|
||||
|
||||
|
||||
def assign_vertex_uv(mesh, vertex_uv):
|
||||
import numpy as np
|
||||
|
||||
uv_layer = mesh.uv_layers.new(name="UVMap")
|
||||
loop_vertices = np.empty(len(mesh.loops), dtype=np.int32)
|
||||
mesh.loops.foreach_get("vertex_index", loop_vertices)
|
||||
loop_uv = vertex_uv[loop_vertices]
|
||||
uv_layer.data.foreach_set("uv", loop_uv.reshape(-1))
|
||||
|
||||
|
||||
def assign_mesh_attribute(mesh, name, data_type, domain, values):
|
||||
import numpy as np
|
||||
|
||||
attribute = mesh.attributes.new(
|
||||
name=name,
|
||||
type=data_type,
|
||||
domain=domain,
|
||||
)
|
||||
array = np.asarray(values)
|
||||
if data_type == "BOOLEAN":
|
||||
array = array.astype(bool)
|
||||
else:
|
||||
array = array.astype(np.float32)
|
||||
attribute.data.foreach_set("value", array.reshape(-1))
|
||||
|
||||
|
||||
def assign_snapform_attributes(mesh, attributes):
|
||||
for name, (data_type, key) in POINT_ATTRIBUTES.items():
|
||||
assign_mesh_attribute(
|
||||
mesh,
|
||||
name,
|
||||
data_type,
|
||||
"POINT",
|
||||
attributes[key],
|
||||
)
|
||||
for name, (data_type, key) in FACE_ATTRIBUTES.items():
|
||||
assign_mesh_attribute(
|
||||
mesh,
|
||||
name,
|
||||
data_type,
|
||||
"FACE",
|
||||
attributes[key],
|
||||
)
|
||||
|
||||
|
||||
def node_asset_path():
|
||||
return Path(__file__).with_name("assets") / NODE_ASSET_NAME
|
||||
|
||||
|
||||
def load_geometry_node_group():
|
||||
existing = bpy.data.node_groups.get(NODE_GROUP_NAME)
|
||||
if existing is not None:
|
||||
return existing
|
||||
asset_path = node_asset_path()
|
||||
if not asset_path.is_file():
|
||||
raise RuntimeError(f"Missing Geometry Nodes asset: {asset_path}")
|
||||
use_pack = bpy.app.version >= (5, 0, 0)
|
||||
if not use_pack:
|
||||
directory = str(asset_path) + "/NodeTree/"
|
||||
result = bpy.ops.wm.append(
|
||||
filepath=directory + NODE_GROUP_NAME,
|
||||
directory=directory,
|
||||
filename=NODE_GROUP_NAME,
|
||||
link=False,
|
||||
do_reuse_local_id=True,
|
||||
)
|
||||
if "FINISHED" not in result:
|
||||
raise RuntimeError("Unable to append the Geometry Nodes asset")
|
||||
node_group = bpy.data.node_groups.get(NODE_GROUP_NAME)
|
||||
if node_group is None:
|
||||
raise RuntimeError("Appended Geometry Nodes asset is unavailable")
|
||||
node_group.use_fake_user = True
|
||||
return node_group
|
||||
with bpy.data.libraries.load(str(asset_path), link=use_pack) as (
|
||||
source,
|
||||
target,
|
||||
):
|
||||
if NODE_GROUP_NAME not in source.node_groups:
|
||||
raise RuntimeError(
|
||||
f"Geometry Nodes asset is missing {NODE_GROUP_NAME}"
|
||||
)
|
||||
target.node_groups = [NODE_GROUP_NAME]
|
||||
node_group = target.node_groups[0]
|
||||
if use_pack:
|
||||
linked_group = node_group
|
||||
node_group = bpy.data.pack_linked_ids_hierarchy(root_id=linked_group)
|
||||
if linked_group != node_group and linked_group.users == 0:
|
||||
bpy.data.node_groups.remove(linked_group)
|
||||
node_group.use_fake_user = True
|
||||
return node_group
|
||||
|
||||
|
||||
def modifier_input_identifier(node_group, name):
|
||||
for item in node_group.interface.items_tree:
|
||||
if item.item_type == "SOCKET" and item.in_out == "INPUT":
|
||||
if item.name == name:
|
||||
return item.identifier
|
||||
raise RuntimeError(f"Geometry Nodes input is missing: {name}")
|
||||
|
||||
|
||||
def default_remesh_resolution(vertices, faces):
|
||||
import numpy as np
|
||||
|
||||
if not len(faces):
|
||||
return 0.01
|
||||
triangles = np.asarray(vertices, dtype=np.float32)[faces]
|
||||
edge_lengths = np.linalg.norm(
|
||||
triangles - np.roll(triangles, 1, axis=1),
|
||||
axis=2,
|
||||
)
|
||||
valid = edge_lengths[np.isfinite(edge_lengths) & (edge_lengths > 0.0)]
|
||||
if not len(valid):
|
||||
return 0.01
|
||||
# Twice the 90% reduction estimate gives the iterative remesh more room
|
||||
# to settle into a smooth, stable default topology.
|
||||
return float(np.median(valid) * np.sqrt(10.0) * 2.0)
|
||||
|
||||
|
||||
def material_depth_scale(mesh_object, camera_parameters):
|
||||
return abs(float(mesh_object.scale.z)) * abs(
|
||||
float(camera_parameters["scale_z"])
|
||||
)
|
||||
|
||||
|
||||
def add_geometry_nodes_processing(mesh_object, vertices, faces):
|
||||
node_group = load_geometry_node_group()
|
||||
modifier = mesh_object.modifiers.new(name=node_group.name, type="NODES")
|
||||
modifier.node_group = node_group
|
||||
modifier.show_group_selector = False
|
||||
identifier = modifier_input_identifier(node_group, "Resolution")
|
||||
modifier[identifier] = default_remesh_resolution(vertices, faces)
|
||||
|
||||
|
||||
def create_matching_camera(
|
||||
context,
|
||||
source_object,
|
||||
mesh_object,
|
||||
name,
|
||||
intrinsics,
|
||||
image_size,
|
||||
camera_parameters,
|
||||
):
|
||||
from mathutils import Matrix
|
||||
|
||||
width, height = image_size
|
||||
scale_x = camera_parameters["scale_x"]
|
||||
scale_z = camera_parameters["scale_z"]
|
||||
effective_fx = float(intrinsics[0, 0]) * scale_z / scale_x
|
||||
camera_data = bpy.data.cameras.new(f"{name}.camera")
|
||||
camera_data.sensor_fit = "HORIZONTAL"
|
||||
camera_data.sensor_width = 36.0
|
||||
camera_data.lens = effective_fx * camera_data.sensor_width / max(
|
||||
width - 1,
|
||||
1,
|
||||
)
|
||||
cx = float(intrinsics[0, 2])
|
||||
cy = float(intrinsics[1, 2])
|
||||
camera_data.shift_x = (cx - (width - 1) * 0.5) / max(width - 1, 1)
|
||||
camera_data.shift_y = ((height - 1) * 0.5 - cy) / max(width - 1, 1)
|
||||
camera_object = bpy.data.objects.new(f"{name}.camera", camera_data)
|
||||
context.collection.objects.link(camera_object)
|
||||
local_matrix = Matrix.Translation(camera_parameters["location"])
|
||||
camera_object.matrix_world = source_object.matrix_world @ local_matrix
|
||||
camera_world = camera_object.matrix_world.copy()
|
||||
camera_object.parent = mesh_object
|
||||
camera_object.matrix_world = camera_world
|
||||
return camera_object
|
||||
|
||||
|
||||
def build_image_mesh(
|
||||
context,
|
||||
source_object,
|
||||
npz_path,
|
||||
delete_source,
|
||||
create_camera,
|
||||
unshaded,
|
||||
):
|
||||
(
|
||||
camera_vertices,
|
||||
faces,
|
||||
uv,
|
||||
intrinsics,
|
||||
image_size,
|
||||
reference_depth,
|
||||
attributes,
|
||||
) = load_image_mesh_data(npz_path)
|
||||
vertices, camera_parameters = empty_anchored_vertices(
|
||||
source_object,
|
||||
camera_vertices,
|
||||
intrinsics,
|
||||
image_size,
|
||||
reference_depth,
|
||||
)
|
||||
source_name = source_object.name
|
||||
mesh = bpy.data.meshes.new(source_name)
|
||||
mesh.from_pydata(vertices.tolist(), [], faces.tolist())
|
||||
assign_vertex_uv(mesh, uv)
|
||||
assign_snapform_attributes(mesh, attributes)
|
||||
mesh.update()
|
||||
for polygon in mesh.polygons:
|
||||
polygon.use_smooth = True
|
||||
mesh_object = bpy.data.objects.new(source_name, mesh)
|
||||
context.collection.objects.link(mesh_object)
|
||||
mesh_object.matrix_world = source_object.matrix_world.copy()
|
||||
depth_scale = material_depth_scale(mesh_object, camera_parameters)
|
||||
material = create_material(
|
||||
source_name,
|
||||
source_object.data,
|
||||
depth_scale,
|
||||
unshaded=unshaded,
|
||||
)
|
||||
mesh.materials.append(material)
|
||||
mesh_object["snapform_reference_depth"] = reference_depth
|
||||
mesh_object["snapform_source_type"] = "IMAGE_MESH"
|
||||
add_geometry_nodes_processing(mesh_object, vertices, faces)
|
||||
if create_camera:
|
||||
create_matching_camera(
|
||||
context,
|
||||
source_object,
|
||||
mesh_object,
|
||||
source_name,
|
||||
intrinsics,
|
||||
image_size,
|
||||
camera_parameters,
|
||||
)
|
||||
for selected in context.selected_objects:
|
||||
selected.select_set(False)
|
||||
mesh_object.select_set(True)
|
||||
context.view_layer.objects.active = mesh_object
|
||||
if delete_source:
|
||||
bpy.data.objects.remove(source_object, do_unlink=True)
|
||||
mesh_object.name = source_name
|
||||
return mesh_object
|
||||
|
||||
|
||||
class BuildImageMesh(bpy.types.Operator):
|
||||
bl_idname = "snapform.build_image_mesh"
|
||||
bl_label = "Build Image Mesh"
|
||||
bl_options = {"INTERNAL"}
|
||||
|
||||
snapform_source_object_name: bpy.props.StringProperty(
|
||||
options={"HIDDEN", "SKIP_SAVE"},
|
||||
)
|
||||
snapform_npz_path: bpy.props.StringProperty(
|
||||
options={"HIDDEN", "SKIP_SAVE"},
|
||||
)
|
||||
snapform_delete_source: bpy.props.BoolProperty(
|
||||
default=True,
|
||||
options={"HIDDEN", "SKIP_SAVE"},
|
||||
)
|
||||
snapform_create_camera: bpy.props.BoolProperty(
|
||||
default=False,
|
||||
options={"HIDDEN", "SKIP_SAVE"},
|
||||
)
|
||||
snapform_unshaded: bpy.props.BoolProperty(
|
||||
default=False,
|
||||
options={"HIDDEN", "SKIP_SAVE"},
|
||||
)
|
||||
|
||||
def execute(self, context):
|
||||
source_object = bpy.data.objects.get(self.snapform_source_object_name)
|
||||
if not is_image_empty(source_object):
|
||||
self.report({"ERROR"}, "The source Image Empty no longer exists")
|
||||
return {"CANCELLED"}
|
||||
try:
|
||||
build_image_mesh(
|
||||
context,
|
||||
source_object,
|
||||
self.snapform_npz_path,
|
||||
self.snapform_delete_source,
|
||||
self.snapform_create_camera,
|
||||
self.snapform_unshaded,
|
||||
)
|
||||
except (OSError, RuntimeError, ValueError) as error:
|
||||
self.report({"ERROR"}, str(error))
|
||||
return {"CANCELLED"}
|
||||
return {"FINISHED"}
|
||||
|
||||
|
||||
class ConvertImageToMesh(bpy.types.Operator):
|
||||
bl_idname = "snapform.convert_image_to_mesh"
|
||||
bl_label = "Convert to Mesh"
|
||||
bl_description = "Convert an Image Empty into a perspective textured mesh"
|
||||
bl_options = {"REGISTER"}
|
||||
|
||||
snapform_model: bpy.props.EnumProperty(
|
||||
name="Model",
|
||||
description="Model used to estimate image depth",
|
||||
items=image_depth_model_items,
|
||||
)
|
||||
snapform_use_ray_pose: bpy.props.BoolProperty(
|
||||
name="Use Ray Pose",
|
||||
description="Use ray-based camera pose estimation",
|
||||
default=True,
|
||||
)
|
||||
snapform_process_res_factor: bpy.props.FloatProperty(
|
||||
name="Resolution Scale",
|
||||
description="Scale applied to the source image long-edge resolution",
|
||||
default=1.0,
|
||||
min=0.1,
|
||||
max=1.0,
|
||||
step=10,
|
||||
precision=2,
|
||||
)
|
||||
snapform_create_camera: bpy.props.BoolProperty(
|
||||
name="Create Matching Camera",
|
||||
description="Create a perspective camera matching the source projection",
|
||||
default=False,
|
||||
)
|
||||
snapform_unshaded: bpy.props.BoolProperty(
|
||||
name="Unshaded",
|
||||
description="Display the source color through an Emission shader",
|
||||
default=False,
|
||||
)
|
||||
snapform_delete_original: bpy.props.BoolProperty(
|
||||
name="Delete Original",
|
||||
description="Delete the source Image Empty after conversion",
|
||||
default=True,
|
||||
)
|
||||
|
||||
@classmethod
|
||||
def poll(cls, context):
|
||||
from .models import BEN2_MODEL, MODELS
|
||||
from .operators import active_job
|
||||
from .runtime import host_busy, model_ready
|
||||
|
||||
return (
|
||||
is_image_empty(context.object)
|
||||
and video_total_frames(context.object.data) == 1
|
||||
and active_job() is None
|
||||
and not host_busy()
|
||||
and any(model_ready(model_key) for model_key in MODELS)
|
||||
and model_ready(BEN2_MODEL.key)
|
||||
)
|
||||
|
||||
def invoke(self, context, _event):
|
||||
global _last_mesh_model
|
||||
|
||||
if not self.properties.is_property_set("snapform_model"):
|
||||
available_keys = [
|
||||
item[0] for item in image_depth_model_items(self, context)
|
||||
]
|
||||
if available_keys:
|
||||
self.snapform_model = (
|
||||
_last_mesh_model
|
||||
if _last_mesh_model in available_keys
|
||||
else available_keys[0]
|
||||
)
|
||||
return context.window_manager.invoke_props_dialog(self, width=360)
|
||||
|
||||
def draw(self, context):
|
||||
layout = self.layout
|
||||
layout.prop(self, "snapform_model")
|
||||
layout.prop(self, "snapform_use_ray_pose")
|
||||
process_res = scaled_process_resolution(
|
||||
context.object.data,
|
||||
self.snapform_process_res_factor,
|
||||
)
|
||||
layout.prop(
|
||||
self,
|
||||
"snapform_process_res_factor",
|
||||
text=f"Resolution Scale ({process_res} px)",
|
||||
)
|
||||
layout.prop(self, "snapform_create_camera")
|
||||
layout.prop(self, "snapform_unshaded")
|
||||
layout.prop(self, "snapform_delete_original")
|
||||
|
||||
def execute(self, context):
|
||||
from .operators import IMAGE_MESH_JOB
|
||||
|
||||
source_object = context.object
|
||||
if video_total_frames(source_object.data) != 1:
|
||||
self.report({"ERROR"}, "Convert to Mesh currently supports still images")
|
||||
return {"CANCELLED"}
|
||||
try:
|
||||
process_res = scaled_process_resolution(
|
||||
source_object.data,
|
||||
self.snapform_process_res_factor,
|
||||
)
|
||||
source_path, delete_input = prepare_image_input(
|
||||
source_object.data,
|
||||
context.scene,
|
||||
)
|
||||
except (RuntimeError, ValueError) as error:
|
||||
self.report({"ERROR"}, str(error))
|
||||
return {"CANCELLED"}
|
||||
result = bpy.ops.snapform.run_job(
|
||||
job_type=IMAGE_MESH_JOB,
|
||||
snapform_input_path=str(source_path),
|
||||
snapform_source_object_name=source_object.name,
|
||||
snapform_model_key=self.snapform_model,
|
||||
snapform_use_ray_pose=self.snapform_use_ray_pose,
|
||||
snapform_process_res=process_res,
|
||||
snapform_delete_source=self.snapform_delete_original,
|
||||
snapform_delete_input=delete_input,
|
||||
snapform_mesh_create_camera=self.snapform_create_camera,
|
||||
snapform_unshaded=self.snapform_unshaded,
|
||||
)
|
||||
if "CANCELLED" in result:
|
||||
if delete_input:
|
||||
remove_temporary_input(source_path)
|
||||
return {"CANCELLED"}
|
||||
global _last_mesh_model
|
||||
_last_mesh_model = self.snapform_model
|
||||
return {"FINISHED"}
|
||||
@@ -17,6 +17,7 @@ from .models import (
|
||||
|
||||
RECONSTRUCTION_JOB = "RECONSTRUCT"
|
||||
IMAGE_DEPTH_JOB = "IMAGE_DEPTH"
|
||||
IMAGE_MESH_JOB = "IMAGE_MESH"
|
||||
BACKGROUND_REMOVAL_JOB = "REMOVE_BACKGROUND"
|
||||
IMAGE_BACKGROUND_JOB = "IMAGE_BACKGROUND"
|
||||
ORIGINAL_BACKGROUND_JOB = "ORIGINAL_BACKGROUND"
|
||||
@@ -78,6 +79,7 @@ class ActiveJob:
|
||||
unshaded: bool = False
|
||||
remove_background: bool = False
|
||||
background_depth_distance: float = 0.0
|
||||
mesh_create_camera: bool = False
|
||||
temporary_input_path: Path | None = None
|
||||
cancelled: bool = False
|
||||
uses_host: bool = False
|
||||
@@ -139,6 +141,7 @@ class RunJob(bpy.types.Operator):
|
||||
("DOWNLOAD_MODEL", "Download Model", ""),
|
||||
(RECONSTRUCTION_JOB, "Export DA3 Result", ""),
|
||||
(IMAGE_DEPTH_JOB, "Convert Image to Depth Plane", ""),
|
||||
(IMAGE_MESH_JOB, "Convert Image to Mesh", ""),
|
||||
(BACKGROUND_REMOVAL_JOB, "Test BEN2 Background Removal", ""),
|
||||
(IMAGE_BACKGROUND_JOB, "Remove Image Empty Background", ""),
|
||||
(ORIGINAL_BACKGROUND_JOB, "Build Original Background Plane", ""),
|
||||
@@ -203,7 +206,14 @@ class RunJob(bpy.types.Operator):
|
||||
min=0.0,
|
||||
options={"HIDDEN", "SKIP_SAVE"},
|
||||
)
|
||||
|
||||
snapform_mesh_create_camera: bpy.props.BoolProperty(
|
||||
default=False,
|
||||
options={"HIDDEN", "SKIP_SAVE"},
|
||||
)
|
||||
snapform_remove_background: bpy.props.BoolProperty(
|
||||
default=False,
|
||||
options={"HIDDEN", "SKIP_SAVE"},
|
||||
)
|
||||
def execute(self, context):
|
||||
global _active_job
|
||||
|
||||
@@ -227,6 +237,8 @@ class RunJob(bpy.types.Operator):
|
||||
export_format = settings.export_format
|
||||
elif self.job_type == IMAGE_DEPTH_JOB:
|
||||
export_format = "mini_npz"
|
||||
elif self.job_type == IMAGE_MESH_JOB:
|
||||
export_format = "mesh_data"
|
||||
|
||||
job = ActiveJob(
|
||||
kind=self.job_type,
|
||||
@@ -243,12 +255,21 @@ class RunJob(bpy.types.Operator):
|
||||
delete_source=self.snapform_delete_source,
|
||||
displacement_via=self.snapform_displacement_via,
|
||||
unshaded=self.snapform_unshaded,
|
||||
remove_background=self.snapform_background_depth_distance > 0.0,
|
||||
remove_background=(
|
||||
self.snapform_background_depth_distance > 0.0
|
||||
or bool(getattr(self, "snapform_remove_background", False))
|
||||
),
|
||||
background_depth_distance=self.snapform_background_depth_distance,
|
||||
mesh_create_camera=getattr(
|
||||
self,
|
||||
"snapform_mesh_create_camera",
|
||||
False,
|
||||
),
|
||||
temporary_input_path=(
|
||||
Path(self.snapform_input_path)
|
||||
if self.job_type in {
|
||||
IMAGE_DEPTH_JOB,
|
||||
IMAGE_MESH_JOB,
|
||||
IMAGE_BACKGROUND_JOB,
|
||||
ORIGINAL_BACKGROUND_JOB,
|
||||
ORIGINAL_FRAMES_JOB,
|
||||
@@ -317,6 +338,7 @@ class RunJob(bpy.types.Operator):
|
||||
getattr(self, "snapform_delete_input", False)
|
||||
and self.job_type in {
|
||||
IMAGE_DEPTH_JOB,
|
||||
IMAGE_MESH_JOB,
|
||||
IMAGE_BACKGROUND_JOB,
|
||||
ORIGINAL_BACKGROUND_JOB,
|
||||
ORIGINAL_FRAMES_JOB,
|
||||
@@ -332,6 +354,8 @@ class RunJob(bpy.types.Operator):
|
||||
getattr(self, "snapform_background_depth_distance", 0.0)
|
||||
)
|
||||
remove_background = background_depth_distance > 0.0
|
||||
if self.job_type == IMAGE_MESH_JOB:
|
||||
remove_background = True
|
||||
if self.job_type == "INSTALL_RUNTIME":
|
||||
self._require_online_access()
|
||||
return runtime.install_runtime_command(paths["status"]), paths
|
||||
@@ -349,6 +373,7 @@ class RunJob(bpy.types.Operator):
|
||||
|
||||
if self.job_type in {
|
||||
IMAGE_DEPTH_JOB,
|
||||
IMAGE_MESH_JOB,
|
||||
IMAGE_BACKGROUND_JOB,
|
||||
ORIGINAL_BACKGROUND_JOB,
|
||||
ORIGINAL_FRAMES_JOB,
|
||||
@@ -410,15 +435,18 @@ class RunJob(bpy.types.Operator):
|
||||
if self.job_type not in {
|
||||
RECONSTRUCTION_JOB,
|
||||
IMAGE_DEPTH_JOB,
|
||||
IMAGE_MESH_JOB,
|
||||
}:
|
||||
raise RuntimeError(f"Unsupported task type: {self.job_type}")
|
||||
export_format = settings.export_format
|
||||
if self.job_type == IMAGE_DEPTH_JOB:
|
||||
export_format = "mini_npz"
|
||||
elif self.job_type == IMAGE_MESH_JOB:
|
||||
export_format = "mesh_data"
|
||||
use_ray_pose = None
|
||||
process_res = None
|
||||
model_key = selected_model_key(settings, export_format)
|
||||
if self.job_type == IMAGE_DEPTH_JOB:
|
||||
if self.job_type in {IMAGE_DEPTH_JOB, IMAGE_MESH_JOB}:
|
||||
model_key = self.snapform_model_key
|
||||
use_ray_pose = self.snapform_use_ray_pose
|
||||
process_res = self.snapform_process_res
|
||||
@@ -426,7 +454,7 @@ class RunJob(bpy.types.Operator):
|
||||
if not runtime.model_ready(model_key):
|
||||
raise RuntimeError(f"Download {model.label} first")
|
||||
if (
|
||||
self.job_type == IMAGE_DEPTH_JOB
|
||||
self.job_type in {IMAGE_DEPTH_JOB, IMAGE_MESH_JOB}
|
||||
and remove_background
|
||||
and not runtime.model_ready(BEN2_MODEL.key)
|
||||
):
|
||||
@@ -446,13 +474,13 @@ class RunJob(bpy.types.Operator):
|
||||
process_res=process_res,
|
||||
process_res_method=(
|
||||
IMAGE_DEPTH_PROCESS_RES_METHOD
|
||||
if self.job_type == IMAGE_DEPTH_JOB
|
||||
if self.job_type in {IMAGE_DEPTH_JOB, IMAGE_MESH_JOB}
|
||||
else None
|
||||
),
|
||||
use_ray_pose=use_ray_pose,
|
||||
start_frame=start_frame,
|
||||
remove_background=(
|
||||
self.job_type == IMAGE_DEPTH_JOB
|
||||
self.job_type in {IMAGE_DEPTH_JOB, IMAGE_MESH_JOB}
|
||||
and remove_background
|
||||
),
|
||||
background_depth_distance=(
|
||||
@@ -469,6 +497,7 @@ class RunJob(bpy.types.Operator):
|
||||
"""Debug panels use the scene device; production tasks use Preferences."""
|
||||
if self.job_type in {
|
||||
IMAGE_DEPTH_JOB,
|
||||
IMAGE_MESH_JOB,
|
||||
IMAGE_BACKGROUND_JOB,
|
||||
ORIGINAL_BACKGROUND_JOB,
|
||||
}:
|
||||
@@ -490,6 +519,8 @@ class RunJob(bpy.types.Operator):
|
||||
return f"Starting {settings.export_format} export"
|
||||
if self.job_type == IMAGE_DEPTH_JOB:
|
||||
return "Starting depth estimation"
|
||||
if self.job_type == IMAGE_MESH_JOB:
|
||||
return "Starting perspective mesh generation"
|
||||
if self.job_type == BACKGROUND_REMOVAL_JOB:
|
||||
return "Starting BEN2 background removal"
|
||||
if self.job_type == IMAGE_BACKGROUND_JOB:
|
||||
@@ -663,6 +694,21 @@ class RunJob(bpy.types.Operator):
|
||||
self._redraw_ui(context, force=True)
|
||||
return {"CANCELLED"}
|
||||
status["message"] = "Depth plane is ready"
|
||||
if job.kind == IMAGE_MESH_JOB:
|
||||
try:
|
||||
settings.status = "Building perspective mesh"
|
||||
settings.progress = 0.96
|
||||
self._redraw_ui(context, force=True)
|
||||
self._build_image_mesh(job)
|
||||
except (OSError, RuntimeError, ValueError) as import_error:
|
||||
settings.status = "Mesh import failed"
|
||||
settings.progress = 0.0
|
||||
self.report({"ERROR"}, str(import_error))
|
||||
runtime.mark_host_job_complete(job.status_path.stem)
|
||||
close_active_job()
|
||||
self._redraw_ui(context, force=True)
|
||||
return {"CANCELLED"}
|
||||
status["message"] = "Perspective mesh is ready"
|
||||
if job.kind in {ORIGINAL_BACKGROUND_JOB, ORIGINAL_FRAMES_JOB}:
|
||||
try:
|
||||
settings.status = "Building depth plane"
|
||||
@@ -706,6 +752,20 @@ class RunJob(bpy.types.Operator):
|
||||
self._redraw_ui(context, force=True)
|
||||
return {"FINISHED"}
|
||||
|
||||
def _build_image_mesh(self, job):
|
||||
result = bpy.ops.snapform.build_image_mesh(
|
||||
snapform_source_object_name=job.source_object_name,
|
||||
snapform_npz_path=str(job.result_directory / "mesh.npz"),
|
||||
snapform_delete_source=job.delete_source,
|
||||
snapform_create_camera=job.mesh_create_camera,
|
||||
snapform_unshaded=job.unshaded,
|
||||
)
|
||||
if "FINISHED" not in result:
|
||||
raise RuntimeError("Unable to build the perspective mesh")
|
||||
undo_result = bpy.ops.ed.undo_push(message="Convert to Mesh")
|
||||
if "FINISHED" not in undo_result:
|
||||
raise RuntimeError("Unable to create the mesh Undo step")
|
||||
|
||||
def _build_depth_plane(self, context, job):
|
||||
result_directory = job.result_directory
|
||||
npz_path = result_directory / "results.npz"
|
||||
|
||||
@@ -37,6 +37,7 @@ JOB_FOLDER_NAMES = {
|
||||
"DOWNLOAD_MODEL": "download-model",
|
||||
"RECONSTRUCT": "export-reconstruction",
|
||||
"IMAGE_DEPTH": "generate-depth-plane",
|
||||
"IMAGE_MESH": "generate-image-mesh",
|
||||
"REMOVE_BACKGROUND": "remove-background",
|
||||
"IMAGE_BACKGROUND": "remove-image-background",
|
||||
"ORIGINAL_BACKGROUND": "generate-original-depth-plane",
|
||||
@@ -812,6 +813,7 @@ def reconstruct_request(
|
||||
use_ray_pose=None,
|
||||
remove_background=False,
|
||||
background_depth_distance=0.0,
|
||||
mesh_filter_edges=None,
|
||||
start_frame=1,
|
||||
device=None,
|
||||
):
|
||||
@@ -852,6 +854,8 @@ def reconstruct_request(
|
||||
"background_depth_distance": float(background_depth_distance),
|
||||
"mesh_filter_edges": bool(
|
||||
getattr(settings, "mesh_filter_edges", True)
|
||||
if mesh_filter_edges is None
|
||||
else mesh_filter_edges
|
||||
),
|
||||
"mesh_max_edge_length": float(
|
||||
getattr(settings, "mesh_max_edge_length", 0.1)
|
||||
|
||||
@@ -24,7 +24,14 @@ VIDEO_SUFFIXES = {
|
||||
".mpeg",
|
||||
}
|
||||
SCENE_FILENAME = "scene.glb"
|
||||
EXPORT_FORMATS = ("glb", "gs_ply", "depth_vis", "mini_npz", "mesh")
|
||||
EXPORT_FORMATS = (
|
||||
"glb",
|
||||
"gs_ply",
|
||||
"depth_vis",
|
||||
"mini_npz",
|
||||
"mesh",
|
||||
"mesh_data",
|
||||
)
|
||||
HF_ENDPOINT = "https://huggingface.co"
|
||||
MODELSCOPE_ENDPOINT = "https://modelscope.cn"
|
||||
MODELSCOPE_REVISION = "master"
|
||||
@@ -368,41 +375,357 @@ def unproject_depth_frame(depth, intrinsics, extrinsics):
|
||||
return world_points[:, :3].reshape(height, width, 3)
|
||||
|
||||
|
||||
def unproject_camera_depth(depth, intrinsics):
|
||||
import numpy as np
|
||||
|
||||
height, width = depth.shape
|
||||
x, y = np.meshgrid(
|
||||
np.arange(width, dtype=np.float32),
|
||||
np.arange(height, dtype=np.float32),
|
||||
)
|
||||
pixels = np.stack((x, y, np.ones_like(x)), axis=-1).reshape(-1, 3)
|
||||
rays = (np.linalg.inv(intrinsics) @ pixels.T).T
|
||||
return (rays * depth.reshape(-1, 1)).reshape(height, width, 3)
|
||||
|
||||
|
||||
def smooth_depth_map(depth, smoothing):
|
||||
import numpy as np
|
||||
|
||||
factor = min(max(float(smoothing) / 100.0, 0.0), 1.0)
|
||||
if factor <= 0.0:
|
||||
return np.asarray(depth, dtype=np.float32).copy()
|
||||
import cv2
|
||||
|
||||
values = np.asarray(depth, dtype=np.float32)
|
||||
valid = np.isfinite(values) & (values > 0.0)
|
||||
if not valid.any():
|
||||
raise ValueError("DA3 returned no valid depth values")
|
||||
low, high = np.percentile(values[valid], (5.0, 95.0))
|
||||
depth_range = max(float(high - low), 1e-6)
|
||||
diameter = 3 + 2 * int(round(3.0 * factor))
|
||||
working_values = values.copy()
|
||||
working_values[~valid] = float(np.median(values[valid]))
|
||||
smoothed = cv2.bilateralFilter(
|
||||
working_values,
|
||||
diameter,
|
||||
depth_range * (0.01 + 0.09 * factor),
|
||||
1.0 + 4.0 * factor,
|
||||
)
|
||||
smoothed[~valid] = values[~valid]
|
||||
return smoothed
|
||||
|
||||
|
||||
def compact_mesh(vertices, faces):
|
||||
import numpy as np
|
||||
|
||||
vertices = np.asarray(vertices, dtype=np.float32)
|
||||
faces = np.asarray(faces, dtype=np.int32)
|
||||
used_vertices = np.unique(faces)
|
||||
index_map = np.full(len(vertices), -1, dtype=np.int32)
|
||||
index_map[used_vertices] = np.arange(len(used_vertices), dtype=np.int32)
|
||||
vertices = vertices[used_vertices]
|
||||
faces = index_map[faces]
|
||||
return vertices, faces
|
||||
|
||||
|
||||
def project_camera_uv(vertices, intrinsics, image_size):
|
||||
import numpy as np
|
||||
|
||||
width, height = image_size
|
||||
z = vertices[:, 2]
|
||||
if np.any(~np.isfinite(z)) or np.any(z <= 0.0):
|
||||
raise ValueError("Simplified mesh contains invalid camera depth")
|
||||
u = intrinsics[0, 0] * vertices[:, 0] / z + intrinsics[0, 2]
|
||||
v = intrinsics[1, 1] * vertices[:, 1] / z + intrinsics[1, 2]
|
||||
denominator_x = max(width - 1, 1)
|
||||
denominator_y = max(height - 1, 1)
|
||||
return np.column_stack(
|
||||
(
|
||||
np.clip(u / denominator_x, 0.0, 1.0),
|
||||
np.clip(1.0 - v / denominator_y, 0.0, 1.0),
|
||||
)
|
||||
).astype(np.float32)
|
||||
|
||||
|
||||
def project_camera_pixels(vertices, intrinsics):
|
||||
import numpy as np
|
||||
|
||||
z = vertices[:, 2]
|
||||
if np.any(~np.isfinite(z)) or np.any(z <= 0.0):
|
||||
raise ValueError("Mesh contains invalid camera depth")
|
||||
return np.column_stack(
|
||||
(
|
||||
intrinsics[0, 0] * vertices[:, 0] / z + intrinsics[0, 2],
|
||||
intrinsics[1, 1] * vertices[:, 1] / z + intrinsics[1, 2],
|
||||
)
|
||||
)
|
||||
|
||||
|
||||
def sample_image_nearest(values, pixels):
|
||||
import numpy as np
|
||||
|
||||
height, width = values.shape
|
||||
x = np.clip(np.rint(pixels[:, 0]).astype(np.int32), 0, width - 1)
|
||||
y = np.clip(np.rint(pixels[:, 1]).astype(np.int32), 0, height - 1)
|
||||
return np.asarray(values)[y, x]
|
||||
|
||||
|
||||
def confidence_percentile_map(depth, confidence):
|
||||
import numpy as np
|
||||
|
||||
valid = np.isfinite(depth) & (depth > 0.0) & np.isfinite(confidence)
|
||||
result = np.zeros(depth.shape, dtype=np.float32)
|
||||
if not valid.any():
|
||||
return result
|
||||
sorted_values = np.sort(np.asarray(confidence)[valid])
|
||||
ranks = np.searchsorted(
|
||||
sorted_values,
|
||||
np.asarray(confidence)[valid],
|
||||
side="right",
|
||||
)
|
||||
result[valid] = ranks.astype(np.float32) / len(sorted_values)
|
||||
return result
|
||||
|
||||
|
||||
def triangle_edge_stretch_values(points, triangles):
|
||||
import numpy as np
|
||||
|
||||
flat_points = np.asarray(points, dtype=np.float32).reshape(-1, 3)
|
||||
triangle_points = flat_points[triangles]
|
||||
triangle_depth = triangle_points[..., 2]
|
||||
safe_depth = np.where(
|
||||
np.abs(triangle_depth) > 1e-8,
|
||||
triangle_depth,
|
||||
np.nan,
|
||||
)
|
||||
triangle_rays = triangle_points / safe_depth[..., None]
|
||||
next_points = np.roll(triangle_points, -1, axis=1)
|
||||
next_rays = np.roll(triangle_rays, -1, axis=1)
|
||||
next_depth = np.roll(triangle_depth, -1, axis=1)
|
||||
local_depth = (triangle_depth + next_depth) * 0.5
|
||||
flat_edges = (
|
||||
triangle_rays * local_depth[..., None]
|
||||
- next_rays * local_depth[..., None]
|
||||
)
|
||||
flat_lengths = np.linalg.norm(flat_edges, axis=-1)
|
||||
actual_lengths = np.linalg.norm(
|
||||
triangle_points - next_points,
|
||||
axis=-1,
|
||||
)
|
||||
stretch = actual_lengths / np.maximum(flat_lengths, 1e-8)
|
||||
stretch[np.isclose(stretch, 1.0, rtol=1e-5, atol=1e-6)] = 1.0
|
||||
return stretch
|
||||
|
||||
|
||||
def image_mesh_attributes(
|
||||
vertices,
|
||||
faces,
|
||||
intrinsics,
|
||||
depth,
|
||||
confidence,
|
||||
confidence_percentiles,
|
||||
foreground_mask,
|
||||
):
|
||||
import numpy as np
|
||||
|
||||
pixels = project_camera_pixels(vertices, intrinsics)
|
||||
point_confidence = sample_image_nearest(confidence, pixels).astype(
|
||||
np.float32
|
||||
)
|
||||
point_confidence = np.nan_to_num(point_confidence, nan=0.0)
|
||||
point_percentile = sample_image_nearest(
|
||||
confidence_percentiles,
|
||||
pixels,
|
||||
).astype(np.float32)
|
||||
point_background = ~sample_image_nearest(foreground_mask, pixels).astype(
|
||||
bool
|
||||
)
|
||||
face_confidence = point_percentile[faces].min(axis=1)
|
||||
face_edge_stretch = triangle_edge_stretch_values(
|
||||
vertices,
|
||||
faces,
|
||||
).max(axis=1)
|
||||
face_background = point_background[faces].any(axis=1)
|
||||
return {
|
||||
"point_depth": vertices[:, 2].astype(np.float32),
|
||||
"point_confidence": point_confidence,
|
||||
"point_confidence_percentile": point_percentile,
|
||||
"point_background": point_background,
|
||||
"face_confidence_percentile": face_confidence.astype(np.float32),
|
||||
"face_edge_stretch": face_edge_stretch.astype(np.float32),
|
||||
"face_background": face_background,
|
||||
}
|
||||
|
||||
|
||||
def export_prediction_attributes(prediction, output_directory):
|
||||
import numpy as np
|
||||
|
||||
values = {}
|
||||
for name in ("depth", "conf", "intrinsics", "extrinsics"):
|
||||
value = getattr(prediction, name, None)
|
||||
if value is not None:
|
||||
values[name] = np.asarray(value)
|
||||
if "depth" in values and "conf" in values:
|
||||
values["confidence_percentile"] = np.stack(
|
||||
[
|
||||
confidence_percentile_map(depth, confidence)
|
||||
for depth, confidence in zip(
|
||||
values["depth"],
|
||||
values["conf"],
|
||||
)
|
||||
]
|
||||
)
|
||||
output_path = Path(output_directory) / "prediction.attributes.npz"
|
||||
np.savez(output_path, **values)
|
||||
return output_path
|
||||
|
||||
|
||||
def export_image_mesh_data(
|
||||
prediction,
|
||||
image_path,
|
||||
output_directory,
|
||||
foreground_path=None,
|
||||
progress_callback=None,
|
||||
):
|
||||
import numpy as np
|
||||
from PIL import Image
|
||||
|
||||
total_steps = 6
|
||||
|
||||
def progress(step, label):
|
||||
if progress_callback is not None:
|
||||
progress_callback(step, total_steps, label)
|
||||
|
||||
progress(1, "Validating DA3 data")
|
||||
required = ("depth", "conf", "intrinsics")
|
||||
missing = [
|
||||
name
|
||||
for name in required
|
||||
if getattr(prediction, name, None) is None
|
||||
]
|
||||
if missing:
|
||||
raise ValueError(f"DA3 mesh output is missing: {', '.join(missing)}")
|
||||
depth_frames = np.asarray(prediction.depth, dtype=np.float32)
|
||||
confidence_frames = np.asarray(prediction.conf, dtype=np.float32)
|
||||
if depth_frames.shape[0] != 1:
|
||||
raise ValueError("Convert to Mesh currently supports one image")
|
||||
raw_depth = depth_frames[0]
|
||||
confidence = confidence_frames[0]
|
||||
intrinsics = np.asarray(prediction.intrinsics[0], dtype=np.float32)
|
||||
progress(2, "Unprojecting camera geometry")
|
||||
points = unproject_camera_depth(raw_depth, intrinsics)
|
||||
progress(3, "Preparing DA3 and BEN2 attributes")
|
||||
foreground_mask = np.ones(raw_depth.shape, dtype=bool)
|
||||
if foreground_path is not None:
|
||||
foreground_mask = foreground_mesh_mask(
|
||||
foreground_path,
|
||||
raw_depth.shape,
|
||||
)
|
||||
confidence_percentiles = confidence_percentile_map(
|
||||
raw_depth,
|
||||
confidence,
|
||||
)
|
||||
progress(4, "Building editable source topology")
|
||||
faces = mesh_triangle_faces(
|
||||
points,
|
||||
raw_depth,
|
||||
confidence,
|
||||
None,
|
||||
False,
|
||||
0.0,
|
||||
)
|
||||
if not len(faces):
|
||||
raise ValueError(
|
||||
"DA3 output does not contain any valid mesh faces"
|
||||
)
|
||||
progress(5, "Projecting UV and structured attributes")
|
||||
vertices, faces = compact_mesh(
|
||||
points.reshape(-1, 3),
|
||||
faces,
|
||||
)
|
||||
height, width = raw_depth.shape
|
||||
uv = project_camera_uv(vertices, intrinsics, (width, height))
|
||||
attributes = image_mesh_attributes(
|
||||
vertices,
|
||||
faces,
|
||||
intrinsics,
|
||||
raw_depth,
|
||||
confidence,
|
||||
confidence_percentiles,
|
||||
foreground_mask,
|
||||
)
|
||||
valid_depth = raw_depth[np.isfinite(raw_depth) & (raw_depth > 0.0)]
|
||||
reference_depth = float(np.median(valid_depth))
|
||||
output_directory = Path(output_directory)
|
||||
output_directory.mkdir(parents=True, exist_ok=True)
|
||||
progress(6, "Writing mesh data")
|
||||
result_path = output_directory / "mesh.npz"
|
||||
np.savez_compressed(
|
||||
result_path,
|
||||
vertices=vertices,
|
||||
faces=faces,
|
||||
uv=uv,
|
||||
intrinsics=intrinsics,
|
||||
image_size=np.asarray((width, height), dtype=np.int32),
|
||||
reference_depth=np.asarray(reference_depth, dtype=np.float32),
|
||||
**attributes,
|
||||
)
|
||||
texture_path = output_directory / "texture.png"
|
||||
with Image.open(image_path) as image:
|
||||
image.convert("RGBA").save(texture_path, format="PNG", compress_level=1)
|
||||
return result_path, texture_path
|
||||
|
||||
|
||||
def depth_edge_mask(depth, threshold=12.0 / 255.0):
|
||||
import numpy as np
|
||||
|
||||
padded = np.pad(depth.astype(np.float32), 1, mode="edge")
|
||||
grad_x = (
|
||||
-padded[:-2, :-2]
|
||||
+ padded[:-2, 2:]
|
||||
- 2.0 * padded[1:-1, :-2]
|
||||
+ 2.0 * padded[1:-1, 2:]
|
||||
- padded[2:, :-2]
|
||||
+ padded[2:, 2:]
|
||||
values = np.asarray(depth, dtype=np.float32)
|
||||
edges = np.zeros(values.shape, dtype=bool)
|
||||
for axis in (0, 1):
|
||||
first = np.take(values, range(values.shape[axis] - 1), axis=axis)
|
||||
second = np.take(values, range(1, values.shape[axis]), axis=axis)
|
||||
valid = (
|
||||
np.isfinite(first)
|
||||
& np.isfinite(second)
|
||||
& (first > 0.0)
|
||||
& (second > 0.0)
|
||||
)
|
||||
grad_y = (
|
||||
-padded[:-2, :-2]
|
||||
- 2.0 * padded[:-2, 1:-1]
|
||||
- padded[:-2, 2:]
|
||||
+ padded[2:, :-2]
|
||||
+ 2.0 * padded[2:, 1:-1]
|
||||
+ padded[2:, 2:]
|
||||
scale = np.maximum(np.minimum(np.abs(first), np.abs(second)), 1e-6)
|
||||
discontinuity = valid & (np.abs(first - second) / scale > threshold)
|
||||
lower = [slice(None), slice(None)]
|
||||
upper = [slice(None), slice(None)]
|
||||
lower[axis] = slice(0, -1)
|
||||
upper[axis] = slice(1, None)
|
||||
edges[tuple(lower)] |= discontinuity
|
||||
edges[tuple(upper)] |= discontinuity
|
||||
return edges
|
||||
|
||||
|
||||
def edge_culling_stretch_limit(culling):
|
||||
factor = min(max(float(culling) / 100.0, 0.0001), 1.0)
|
||||
return 1.0 / factor
|
||||
|
||||
|
||||
def foreground_mesh_mask(foreground_path, depth_shape):
|
||||
import numpy as np
|
||||
from PIL import Image, ImageFilter
|
||||
|
||||
height, width = depth_shape
|
||||
with Image.open(foreground_path) as source:
|
||||
alpha = source.convert("RGBA").getchannel("A")
|
||||
alpha = alpha.resize(
|
||||
(width, height),
|
||||
resample=Image.Resampling.BILINEAR,
|
||||
)
|
||||
magnitude = np.hypot(grad_x, grad_y)
|
||||
finite = np.isfinite(magnitude)
|
||||
if not finite.any():
|
||||
return np.ones(depth.shape, dtype=bool)
|
||||
minimum = float(magnitude[finite].min())
|
||||
maximum = float(magnitude[finite].max())
|
||||
if maximum <= minimum:
|
||||
return np.zeros(depth.shape, dtype=bool)
|
||||
normalized = (magnitude - minimum) / (maximum - minimum)
|
||||
return normalized >= threshold
|
||||
alpha = alpha.filter(ImageFilter.GaussianBlur(radius=0.75))
|
||||
return np.asarray(alpha) >= 128
|
||||
|
||||
|
||||
def confidence_percentile_threshold(depth, confidence, percentile):
|
||||
import numpy as np
|
||||
|
||||
if float(percentile) <= 0.0:
|
||||
return None
|
||||
valid = np.isfinite(depth) & (depth > 0.0) & np.isfinite(confidence)
|
||||
if not valid.any():
|
||||
raise ValueError("DA3 returned no valid depth confidence values")
|
||||
@@ -416,6 +739,9 @@ def mesh_triangle_faces(
|
||||
confidence_threshold,
|
||||
filter_edges,
|
||||
max_edge_length,
|
||||
edge_threshold=12.0 / 255.0,
|
||||
edge_stretch_limit=None,
|
||||
vertex_mask=None,
|
||||
):
|
||||
import numpy as np
|
||||
|
||||
@@ -434,24 +760,47 @@ def mesh_triangle_faces(
|
||||
np.isfinite(points).all(axis=-1)
|
||||
& np.isfinite(depth)
|
||||
& (depth > 0.0)
|
||||
& (confidence >= confidence_threshold)
|
||||
)
|
||||
if confidence_threshold is not None:
|
||||
valid &= np.isfinite(confidence) & (
|
||||
confidence >= confidence_threshold
|
||||
)
|
||||
if filter_edges:
|
||||
valid &= ~depth_edge_mask(depth)
|
||||
face_valid = valid.reshape(-1)[quads].all(axis=1)
|
||||
if max_edge_length > 0.0:
|
||||
flat_points = points.reshape(-1, 3)
|
||||
quad_points = flat_points[quads]
|
||||
next_points = np.roll(quad_points, -1, axis=1)
|
||||
edge_lengths = np.linalg.norm(quad_points - next_points, axis=-1)
|
||||
face_valid &= (edge_lengths <= max_edge_length).all(axis=1)
|
||||
quads = quads[face_valid]
|
||||
if not len(quads):
|
||||
return np.empty((0, 3), dtype=np.int32)
|
||||
return np.concatenate(
|
||||
valid &= ~depth_edge_mask(depth, edge_threshold)
|
||||
if vertex_mask is not None:
|
||||
if np.shape(vertex_mask) != depth.shape:
|
||||
raise ValueError("Foreground mask and depth dimensions do not match")
|
||||
valid &= np.asarray(vertex_mask, dtype=bool)
|
||||
triangles = np.concatenate(
|
||||
(quads[:, (0, 1, 2)], quads[:, (0, 2, 3)]),
|
||||
axis=0,
|
||||
)
|
||||
face_valid = valid.reshape(-1)[triangles].all(axis=1)
|
||||
if edge_stretch_limit is not None:
|
||||
face_valid &= triangle_edge_stretch_mask(
|
||||
points,
|
||||
triangles,
|
||||
edge_stretch_limit,
|
||||
)
|
||||
if max_edge_length > 0.0:
|
||||
flat_points = points.reshape(-1, 3)
|
||||
triangle_points = flat_points[triangles]
|
||||
next_points = np.roll(triangle_points, -1, axis=1)
|
||||
edge_lengths = np.linalg.norm(
|
||||
triangle_points - next_points,
|
||||
axis=-1,
|
||||
)
|
||||
face_valid &= (edge_lengths <= max_edge_length).all(axis=1)
|
||||
triangles = triangles[face_valid]
|
||||
if not len(triangles):
|
||||
return np.empty((0, 3), dtype=np.int32)
|
||||
return triangles
|
||||
|
||||
|
||||
def triangle_edge_stretch_mask(points, triangles, stretch_limit):
|
||||
stretch = triangle_edge_stretch_values(points, triangles)
|
||||
tolerance = float(stretch_limit) * (1.0 + 1e-5)
|
||||
return (stretch <= tolerance).all(axis=1)
|
||||
|
||||
|
||||
def export_textured_mesh_glb(
|
||||
@@ -512,6 +861,16 @@ def export_textured_mesh_glb(
|
||||
visual=visual,
|
||||
process=False,
|
||||
)
|
||||
mesh.vertex_attributes["o_depth"] = depth[index].reshape(-1)
|
||||
mesh.vertex_attributes["o_confidence"] = confidence[
|
||||
index
|
||||
].reshape(-1)
|
||||
mesh.vertex_attributes["o_confidence_percentile"] = (
|
||||
confidence_percentile_map(
|
||||
depth[index],
|
||||
confidence[index],
|
||||
).reshape(-1)
|
||||
)
|
||||
mesh.remove_unreferenced_vertices()
|
||||
meshes.append(mesh)
|
||||
referenced_points.append(mesh.vertices)
|
||||
@@ -1160,6 +1519,9 @@ def exported_paths(output_directory, export_format):
|
||||
if export_format == "mesh":
|
||||
result_path = output_directory / "mesh.glb"
|
||||
return [result_path] if result_path.is_file() else []
|
||||
if export_format == "mesh_data":
|
||||
result_path = output_directory / "mesh.npz"
|
||||
return [result_path] if result_path.is_file() else []
|
||||
if export_format == "mini_npz":
|
||||
result_path = output_directory / "results.npz"
|
||||
if result_path.is_file():
|
||||
@@ -1401,6 +1763,23 @@ def reconstruct(args, model_cache=None, cancel_check=None):
|
||||
bool(getattr(args, "mesh_filter_edges", True)),
|
||||
float(getattr(args, "mesh_max_edge_length", 0.1)),
|
||||
)
|
||||
elif args.export_format == "mesh_data":
|
||||
def mesh_progress(step, total, label):
|
||||
write_status(
|
||||
args.status,
|
||||
0.88 + 0.09 * step / total,
|
||||
f"Post-processing {step}/{total}: {label}",
|
||||
)
|
||||
|
||||
export_image_mesh_data(
|
||||
prediction,
|
||||
images[0],
|
||||
output_directory,
|
||||
foreground_path=(
|
||||
foreground_images[0] if foreground_images else None
|
||||
),
|
||||
progress_callback=mesh_progress,
|
||||
)
|
||||
else:
|
||||
export(
|
||||
prediction,
|
||||
@@ -1408,6 +1787,13 @@ def reconstruct(args, model_cache=None, cancel_check=None):
|
||||
str(output_directory),
|
||||
**export_kwargs,
|
||||
)
|
||||
if args.export_format in {"glb", "gs_ply", "depth_vis", "mesh"}:
|
||||
write_status(
|
||||
args.status,
|
||||
0.97,
|
||||
"Writing structured debug attributes",
|
||||
)
|
||||
export_prediction_attributes(prediction, output_directory)
|
||||
check_cancelled(cancel_check)
|
||||
|
||||
result_paths = wait_for_exported_paths(
|
||||
|
||||
@@ -8,6 +8,8 @@ from tempfile import TemporaryDirectory
|
||||
from types import ModuleType, SimpleNamespace
|
||||
from unittest.mock import patch
|
||||
|
||||
import numpy as np
|
||||
|
||||
|
||||
PROJECT_ROOT = Path(__file__).resolve().parents[1]
|
||||
SOURCE_ROOT = PROJECT_ROOT / "src"
|
||||
@@ -28,6 +30,7 @@ class AddonTest(unittest.TestCase):
|
||||
)
|
||||
fake_bpy.types = SimpleNamespace(
|
||||
AddonPreferences=object,
|
||||
Menu=object,
|
||||
Operator=object,
|
||||
Panel=object,
|
||||
PropertyGroup=object,
|
||||
@@ -79,6 +82,7 @@ class AddonTest(unittest.TestCase):
|
||||
self.context_menu_draws.clear()
|
||||
self.addon.operators._active_job = None
|
||||
self.addon.image_depth._last_depth_plane_model = "BASE"
|
||||
self.addon.image_mesh._last_mesh_model = "BASE"
|
||||
self.fake_bpy.data = SimpleNamespace(
|
||||
objects=SimpleNamespace(get=lambda _name: None),
|
||||
)
|
||||
@@ -125,6 +129,135 @@ class AddonTest(unittest.TestCase):
|
||||
)
|
||||
self.assertFalse(hasattr(self.addon, "ImportDepthPlane"))
|
||||
|
||||
def test_registers_perspective_mesh_operators(self):
|
||||
self.assertIn(self.addon.ConvertImageToMesh, self.addon.CLASSES)
|
||||
self.assertIn(self.addon.BuildImageMesh, self.addon.CLASSES)
|
||||
self.assertEqual(self.addon.BuildImageMesh.bl_options, {"INTERNAL"})
|
||||
self.assertIn(self.addon.SnapFormImageMenu, self.addon.CLASSES)
|
||||
|
||||
def test_image_context_menu_groups_actions_and_separates_background(self):
|
||||
calls = []
|
||||
|
||||
class Layout:
|
||||
operator_context = None
|
||||
|
||||
def operator(self, identifier, **_options):
|
||||
calls.append(identifier)
|
||||
|
||||
def separator(self):
|
||||
calls.append("separator")
|
||||
|
||||
menu = self.addon.SnapFormImageMenu()
|
||||
menu.layout = Layout()
|
||||
menu.draw(None)
|
||||
|
||||
self.assertEqual(
|
||||
calls,
|
||||
[
|
||||
"snapform.convert_image_to_depth_plane",
|
||||
"snapform.convert_image_to_mesh",
|
||||
"separator",
|
||||
"snapform.remove_image_background",
|
||||
],
|
||||
)
|
||||
|
||||
def test_image_mesh_reference_plane_matches_image_empty_bounds(self):
|
||||
source_object = SimpleNamespace(
|
||||
data=SimpleNamespace(size=(401, 201)),
|
||||
empty_display_size=4.0,
|
||||
empty_image_offset=(-0.25, -0.75),
|
||||
)
|
||||
intrinsics = np.asarray(
|
||||
[[200.0, 0.0, 200.0], [0.0, 200.0, 100.0], [0.0, 0.0, 1.0]],
|
||||
dtype=np.float32,
|
||||
)
|
||||
camera_vertices = np.asarray(
|
||||
[
|
||||
[-2.0, -1.0, 2.0],
|
||||
[2.0, -1.0, 2.0],
|
||||
[-2.0, 1.0, 2.0],
|
||||
[2.0, 1.0, 2.0],
|
||||
],
|
||||
dtype=np.float32,
|
||||
)
|
||||
|
||||
vertices, camera = self.addon.image_mesh.empty_anchored_vertices(
|
||||
source_object,
|
||||
camera_vertices,
|
||||
intrinsics,
|
||||
(401, 201),
|
||||
2.0,
|
||||
)
|
||||
|
||||
x_min, x_max, y_min, y_max = self.addon.image_depth.plane_bounds(
|
||||
source_object
|
||||
)
|
||||
np.testing.assert_allclose(
|
||||
vertices,
|
||||
[
|
||||
[x_min, y_max, 0.0],
|
||||
[x_max, y_max, 0.0],
|
||||
[x_min, y_min, 0.0],
|
||||
[x_max, y_min, 0.0],
|
||||
],
|
||||
)
|
||||
np.testing.assert_allclose(
|
||||
camera["location"],
|
||||
((x_min + x_max) * 0.5, (y_min + y_max) * 0.5, 2.0024922),
|
||||
)
|
||||
|
||||
def test_mesh_conversion_dialog_only_keeps_model_and_camera_options(self):
|
||||
annotations = self.addon.ConvertImageToMesh.__annotations__
|
||||
|
||||
for property_name in (
|
||||
"snapform_confidence_percentile",
|
||||
"snapform_edge_culling",
|
||||
"snapform_face_reduction",
|
||||
"snapform_depth_smoothing",
|
||||
"snapform_remove_background",
|
||||
"snapform_postprocess_method",
|
||||
):
|
||||
with self.subTest(property_name=property_name):
|
||||
self.assertNotIn(property_name, annotations)
|
||||
self.assertFalse(annotations["snapform_create_camera"]["default"])
|
||||
self.assertFalse(annotations["snapform_unshaded"]["default"])
|
||||
self.assertEqual(annotations["snapform_unshaded"]["name"], "Unshaded")
|
||||
|
||||
def test_image_mesh_uses_o_prefixed_attributes_and_bundled_nodes(self):
|
||||
image_mesh = self.addon.image_mesh
|
||||
names = set(image_mesh.POINT_ATTRIBUTES) | set(
|
||||
image_mesh.FACE_ATTRIBUTES
|
||||
)
|
||||
|
||||
self.assertTrue(all(name.startswith("o_") for name in names))
|
||||
self.assertTrue(image_mesh.node_asset_path().is_file())
|
||||
self.assertEqual(image_mesh.NODE_GROUP_NAME, "O Refine DA3 Mesh")
|
||||
self.assertEqual(image_mesh.NODE_ASSET_NAME, "O_SnapForm.blend")
|
||||
|
||||
def test_image_mesh_material_depth_scale_matches_fitted_geometry(self):
|
||||
mesh_object = SimpleNamespace(scale=SimpleNamespace(z=-2.0))
|
||||
|
||||
self.assertEqual(
|
||||
self.addon.image_mesh.material_depth_scale(
|
||||
mesh_object,
|
||||
{"scale_z": 0.25},
|
||||
),
|
||||
0.5,
|
||||
)
|
||||
|
||||
def test_image_mesh_default_resolution_uses_double_reduction_estimate(self):
|
||||
vertices = np.asarray(
|
||||
((0.0, 0.0, 0.0), (1.0, 0.0, 0.0), (0.0, 1.0, 0.0)),
|
||||
dtype=np.float32,
|
||||
)
|
||||
faces = np.asarray(((0, 1, 2),), dtype=np.int32)
|
||||
edge_lengths = np.asarray((1.0, 1.0, np.sqrt(2.0)))
|
||||
|
||||
self.assertAlmostEqual(
|
||||
self.addon.image_mesh.default_remesh_resolution(vertices, faces),
|
||||
float(np.median(edge_lengths) * np.sqrt(10.0) * 2.0),
|
||||
)
|
||||
|
||||
def test_registers_image_background_operators(self):
|
||||
self.assertIn(self.addon.RemoveImageBackground, self.addon.CLASSES)
|
||||
self.assertIn(self.addon.ApplyRemovedBackground, self.addon.CLASSES)
|
||||
@@ -1486,6 +1619,51 @@ class AddonTest(unittest.TestCase):
|
||||
)
|
||||
self.assertEqual(parameters["use_ray_pose"], 0)
|
||||
|
||||
def test_reconstruct_request_enables_ben2_for_image_mesh(self):
|
||||
runtime = importlib.import_module("snapform.runtime")
|
||||
settings = SimpleNamespace(
|
||||
input_path="input.png",
|
||||
model="BASE",
|
||||
gs_model="GIANT_1_1",
|
||||
process_res=504,
|
||||
process_res_method="upper_bound_resize",
|
||||
conf_thresh_percentile=40.0,
|
||||
num_max_points=1_000_000,
|
||||
show_cameras=True,
|
||||
gs_views_interval=1,
|
||||
video_frames=16,
|
||||
device="AUTO",
|
||||
use_ray_pose=False,
|
||||
ref_view_strategy="saddle_balanced",
|
||||
)
|
||||
with patch.object(
|
||||
runtime,
|
||||
"runtime_python",
|
||||
return_value=Path("python"),
|
||||
), patch.object(
|
||||
runtime,
|
||||
"model_directory",
|
||||
side_effect=lambda key: Path(key.lower()),
|
||||
):
|
||||
request = runtime.reconstruct_request(
|
||||
settings,
|
||||
"mesh_data",
|
||||
Path("status.json"),
|
||||
Path("result"),
|
||||
remove_background=True,
|
||||
)
|
||||
|
||||
parameters = request.payload["parameters"]
|
||||
self.assertNotIn("mesh_confidence_percentile", parameters)
|
||||
self.assertNotIn("mesh_edge_culling", parameters)
|
||||
self.assertNotIn("mesh_face_reduction", parameters)
|
||||
self.assertNotIn("mesh_depth_smoothing", parameters)
|
||||
self.assertNotIn("mesh_postprocess_method", parameters)
|
||||
self.assertEqual(
|
||||
parameters["remove_background_model_dir"],
|
||||
str(Path("ben2_base")),
|
||||
)
|
||||
|
||||
def test_export_format_property_includes_all_da3_formats(self):
|
||||
export_format = self.addon.SnapFormSettings.__annotations__[
|
||||
"export_format"
|
||||
|
||||
@@ -21,7 +21,7 @@ def load_image_depth():
|
||||
IntProperty=lambda **options: options,
|
||||
StringProperty=lambda **options: options,
|
||||
)
|
||||
fake_bpy.types = SimpleNamespace(Operator=object)
|
||||
fake_bpy.types = SimpleNamespace(Menu=object, Operator=object)
|
||||
fake_bpy.path = SimpleNamespace(abspath=lambda path: path)
|
||||
with patch.dict(sys.modules, {"bpy": fake_bpy}):
|
||||
specification = importlib.util.spec_from_file_location(
|
||||
|
||||
@@ -56,6 +56,10 @@ class PackTest(unittest.TestCase):
|
||||
self.assertIn("__init__.py", names)
|
||||
self.assertIn("blender_manifest.toml", names)
|
||||
self.assertIn("image_depth.py", names)
|
||||
self.assertIn(
|
||||
"assets/O_SnapForm.blend",
|
||||
names,
|
||||
)
|
||||
self.assertIn("worker.py", names)
|
||||
self.assertIn("ai_host.py", names)
|
||||
self.assertNotIn("snapform/__init__.py", names)
|
||||
|
||||
@@ -982,7 +982,14 @@ class WorkerTest(unittest.TestCase):
|
||||
"export_textured_mesh_glb",
|
||||
side_effect=fake_mesh_export,
|
||||
):
|
||||
for export_format in self.worker.EXPORT_FORMATS:
|
||||
debug_formats = (
|
||||
"glb",
|
||||
"gs_ply",
|
||||
"depth_vis",
|
||||
"mini_npz",
|
||||
"mesh",
|
||||
)
|
||||
for export_format in debug_formats:
|
||||
args = SimpleNamespace(
|
||||
input=str(input_path),
|
||||
video_frames=16,
|
||||
@@ -1172,6 +1179,124 @@ class WorkerTest(unittest.TestCase):
|
||||
|
||||
self.assertEqual(threshold, 2.5)
|
||||
|
||||
def test_zero_mesh_confidence_disables_confidence_filter(self):
|
||||
depth = np.ones((2, 2), dtype=np.float32)
|
||||
confidence = np.full((2, 2), np.nan, dtype=np.float32)
|
||||
x, y = np.meshgrid([0.0, 1.0], [0.0, 1.0])
|
||||
points = np.stack((x, y, depth), axis=-1)
|
||||
|
||||
threshold = self.worker.confidence_percentile_threshold(
|
||||
depth,
|
||||
confidence,
|
||||
0.0,
|
||||
)
|
||||
faces = self.worker.mesh_triangle_faces(
|
||||
points,
|
||||
depth,
|
||||
confidence,
|
||||
threshold,
|
||||
filter_edges=False,
|
||||
max_edge_length=0.0,
|
||||
)
|
||||
|
||||
self.assertIsNone(threshold)
|
||||
self.assertEqual(len(faces), 2)
|
||||
|
||||
def test_mesh_filters_triangles_independently(self):
|
||||
depth = np.ones((2, 2), dtype=np.float32)
|
||||
confidence = np.ones((2, 2), dtype=np.float32)
|
||||
confidence[1, 0] = 0.0
|
||||
x, y = np.meshgrid(
|
||||
np.arange(2, dtype=np.float32),
|
||||
np.arange(2, dtype=np.float32),
|
||||
)
|
||||
points = np.stack((x, y, depth), axis=-1)
|
||||
|
||||
faces = self.worker.mesh_triangle_faces(
|
||||
points,
|
||||
depth,
|
||||
confidence,
|
||||
confidence_threshold=0.5,
|
||||
filter_edges=False,
|
||||
max_edge_length=0.0,
|
||||
)
|
||||
|
||||
np.testing.assert_array_equal(faces, [[0, 3, 1]])
|
||||
|
||||
def test_mesh_background_mask_combines_with_confidence(self):
|
||||
depth = np.ones((2, 3), dtype=np.float32)
|
||||
confidence = np.ones((2, 3), dtype=np.float32)
|
||||
x, y = np.meshgrid(
|
||||
np.arange(3, dtype=np.float32),
|
||||
np.arange(2, dtype=np.float32),
|
||||
)
|
||||
points = np.stack((x, y, depth), axis=-1)
|
||||
foreground = np.ones((2, 3), dtype=bool)
|
||||
foreground[:, 0] = False
|
||||
|
||||
faces = self.worker.mesh_triangle_faces(
|
||||
points,
|
||||
depth,
|
||||
confidence,
|
||||
confidence_threshold=None,
|
||||
filter_edges=False,
|
||||
max_edge_length=0.0,
|
||||
vertex_mask=foreground,
|
||||
)
|
||||
|
||||
np.testing.assert_array_equal(faces, [[1, 4, 5], [1, 5, 2]])
|
||||
|
||||
def test_depth_edge_filter_ignores_smooth_slope(self):
|
||||
smooth = np.tile(
|
||||
np.linspace(2.0, 2.04, 5, dtype=np.float32),
|
||||
(3, 1),
|
||||
)
|
||||
discontinuous = smooth.copy()
|
||||
discontinuous[:, 3:] += 1.0
|
||||
|
||||
self.assertFalse(self.worker.depth_edge_mask(smooth).any())
|
||||
self.assertTrue(self.worker.depth_edge_mask(discontinuous)[:, 2:4].all())
|
||||
|
||||
def test_higher_edge_culling_allows_less_stretch(self):
|
||||
limits = [
|
||||
self.worker.edge_culling_stretch_limit(value)
|
||||
for value in (0.0, 50.0, 100.0)
|
||||
]
|
||||
|
||||
self.assertEqual(limits, [10000.0, 2.0, 1.0])
|
||||
|
||||
def test_stretched_edge_culling_uses_camera_space_edge_ratio(self):
|
||||
flat_points = np.asarray(
|
||||
[
|
||||
[[0.0, 0.0, 1.0], [1.0, 0.0, 1.0]],
|
||||
[[0.0, 1.0, 1.0], [1.0, 1.0, 1.0]],
|
||||
],
|
||||
dtype=np.float32,
|
||||
)
|
||||
stretched_points = flat_points.copy()
|
||||
stretched_points[0, 1] = (2.0, 0.0, 2.0)
|
||||
triangles = np.asarray([[0, 2, 3], [0, 3, 1]], dtype=np.int32)
|
||||
|
||||
flat_mask = self.worker.triangle_edge_stretch_mask(
|
||||
flat_points,
|
||||
triangles,
|
||||
1.0,
|
||||
)
|
||||
loose_mask = self.worker.triangle_edge_stretch_mask(
|
||||
stretched_points,
|
||||
triangles,
|
||||
2.0,
|
||||
)
|
||||
strict_mask = self.worker.triangle_edge_stretch_mask(
|
||||
stretched_points,
|
||||
triangles,
|
||||
1.0,
|
||||
)
|
||||
|
||||
self.assertTrue(flat_mask.all())
|
||||
self.assertTrue(loose_mask.all())
|
||||
self.assertFalse(strict_mask.all())
|
||||
|
||||
def test_mesh_faces_apply_confidence_threshold_and_face_direction(self):
|
||||
points = np.array(
|
||||
[
|
||||
@@ -1228,6 +1353,113 @@ class WorkerTest(unittest.TestCase):
|
||||
|
||||
self.assertEqual(face_counts, [8, 2, 0])
|
||||
|
||||
def test_mesh_compaction_removes_unreferenced_vertices(self):
|
||||
vertices = np.arange(18, dtype=np.float32).reshape(6, 3)
|
||||
faces = np.asarray([[1, 3, 5]], dtype=np.int32)
|
||||
|
||||
compact_vertices, compact_faces = self.worker.compact_mesh(
|
||||
vertices,
|
||||
faces,
|
||||
)
|
||||
|
||||
np.testing.assert_array_equal(
|
||||
compact_vertices,
|
||||
vertices[[1, 3, 5]],
|
||||
)
|
||||
np.testing.assert_array_equal(compact_faces, [[0, 1, 2]])
|
||||
|
||||
def test_image_mesh_data_keeps_camera_geometry_and_uv(self):
|
||||
class FakeImage:
|
||||
def __enter__(self):
|
||||
return self
|
||||
|
||||
def __exit__(self, *_args):
|
||||
return None
|
||||
|
||||
def convert(self, mode):
|
||||
self.mode = mode
|
||||
return self
|
||||
|
||||
def save(self, path, **_options):
|
||||
Path(path).write_bytes(b"texture")
|
||||
|
||||
fake_pil = ModuleType("PIL")
|
||||
fake_pil.Image = SimpleNamespace(open=lambda _path: FakeImage())
|
||||
depth = np.full((1, 3, 3), 2.0, dtype=np.float32)
|
||||
confidence = np.ones((1, 3, 3), dtype=np.float32)
|
||||
intrinsics = np.asarray(
|
||||
[[[2.0, 0.0, 1.0], [0.0, 2.0, 1.0], [0.0, 0.0, 1.0]]],
|
||||
dtype=np.float32,
|
||||
)
|
||||
prediction = SimpleNamespace(
|
||||
depth=depth,
|
||||
conf=confidence,
|
||||
intrinsics=intrinsics,
|
||||
)
|
||||
progress = []
|
||||
|
||||
with tempfile.TemporaryDirectory() as directory, patch.dict(
|
||||
sys.modules,
|
||||
{"PIL": fake_pil},
|
||||
), patch.object(
|
||||
self.worker,
|
||||
"compact_mesh",
|
||||
side_effect=lambda vertices, faces: (
|
||||
vertices,
|
||||
faces,
|
||||
),
|
||||
):
|
||||
result_path, texture_path = self.worker.export_image_mesh_data(
|
||||
prediction,
|
||||
Path(directory) / "input.png",
|
||||
Path(directory) / "result",
|
||||
progress_callback=lambda step, total, label: progress.append(
|
||||
(step, total, label)
|
||||
),
|
||||
)
|
||||
with np.load(result_path) as result:
|
||||
self.assertEqual(result["vertices"].shape, (9, 3))
|
||||
self.assertEqual(result["faces"].shape, (8, 3))
|
||||
self.assertEqual(result["uv"].shape, (9, 2))
|
||||
self.assertEqual(float(result["reference_depth"]), 2.0)
|
||||
self.assertNotIn("postprocess_method", result.files)
|
||||
self.assertEqual(result["point_depth"].shape, (9,))
|
||||
self.assertEqual(
|
||||
result["point_confidence_percentile"].shape,
|
||||
(9,),
|
||||
)
|
||||
self.assertEqual(
|
||||
result["face_edge_stretch"].shape,
|
||||
(8,),
|
||||
)
|
||||
self.assertEqual(result["face_background"].shape, (8,))
|
||||
np.testing.assert_allclose(result["uv"][0], [0.0, 1.0])
|
||||
np.testing.assert_allclose(result["uv"][-1], [1.0, 0.0])
|
||||
self.assertTrue(texture_path.is_file())
|
||||
self.assertEqual([item[0] for item in progress], list(range(1, 7)))
|
||||
self.assertTrue(all(item[1] == 6 for item in progress))
|
||||
|
||||
def test_prediction_attributes_keep_debug_structure(self):
|
||||
prediction = SimpleNamespace(
|
||||
depth=np.ones((1, 2, 3), dtype=np.float32),
|
||||
conf=np.arange(6, dtype=np.float32).reshape(1, 2, 3),
|
||||
intrinsics=np.eye(3, dtype=np.float32)[None],
|
||||
extrinsics=np.eye(4, dtype=np.float32)[None],
|
||||
)
|
||||
|
||||
with tempfile.TemporaryDirectory() as directory:
|
||||
output_path = self.worker.export_prediction_attributes(
|
||||
prediction,
|
||||
directory,
|
||||
)
|
||||
with np.load(output_path) as attributes:
|
||||
self.assertEqual(attributes["depth"].shape, (1, 2, 3))
|
||||
self.assertEqual(attributes["conf"].shape, (1, 2, 3))
|
||||
self.assertEqual(
|
||||
attributes["confidence_percentile"].shape,
|
||||
(1, 2, 3),
|
||||
)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
|
||||
633
tools/build_geometry_nodes_asset.py
Normal file
633
tools/build_geometry_nodes_asset.py
Normal file
@@ -0,0 +1,633 @@
|
||||
from pathlib import Path
|
||||
|
||||
import bpy
|
||||
|
||||
|
||||
GROUP_NAME = "O Refine DA3 Mesh"
|
||||
OUTPUT_PATH = (
|
||||
Path(__file__).parents[1]
|
||||
/ "src"
|
||||
/ "snapform"
|
||||
/ "assets"
|
||||
/ "O_SnapForm.blend"
|
||||
)
|
||||
|
||||
|
||||
def input_socket(node, name, fallback):
|
||||
value = node.inputs.get(name)
|
||||
return value if value is not None else node.inputs[fallback]
|
||||
|
||||
|
||||
def new_socket(interface, name, in_out, socket_type, parent=None):
|
||||
return interface.new_socket(
|
||||
name=name,
|
||||
in_out=in_out,
|
||||
socket_type=socket_type,
|
||||
parent=parent,
|
||||
)
|
||||
|
||||
|
||||
def new_compare(nodes, data_type, operation, location, value):
|
||||
node = nodes.new("FunctionNodeCompare")
|
||||
node.data_type = data_type
|
||||
node.operation = operation
|
||||
node.location = location
|
||||
input_socket(node, "B", 1).default_value = value
|
||||
return node
|
||||
|
||||
|
||||
def new_group_input(nodes, location, visible_names):
|
||||
node = nodes.new("NodeGroupInput")
|
||||
node.location = location
|
||||
visible = set(visible_names)
|
||||
for output in node.outputs:
|
||||
output.hide = output.name not in visible
|
||||
return node
|
||||
|
||||
|
||||
def add_edge_boundary_field(nodes, links, location):
|
||||
x, y = location
|
||||
vertex_neighbors = nodes.new("GeometryNodeInputMeshVertexNeighbors")
|
||||
vertex_neighbors.location = (x, y + 100.0)
|
||||
loose_vertex = new_compare(
|
||||
nodes,
|
||||
"INT",
|
||||
"EQUAL",
|
||||
(x + 180.0, y + 100.0),
|
||||
1,
|
||||
)
|
||||
links.new(
|
||||
vertex_neighbors.outputs["Vertex Count"],
|
||||
input_socket(loose_vertex, "A", 0),
|
||||
)
|
||||
edge_neighbors = nodes.new("GeometryNodeInputMeshEdgeNeighbors")
|
||||
edge_neighbors.location = (x, y - 100.0)
|
||||
boundary_edge = new_compare(
|
||||
nodes,
|
||||
"INT",
|
||||
"EQUAL",
|
||||
(x + 180.0, y - 100.0),
|
||||
1,
|
||||
)
|
||||
links.new(
|
||||
edge_neighbors.outputs["Face Count"],
|
||||
input_socket(boundary_edge, "A", 0),
|
||||
)
|
||||
edge_to_point = nodes.new("GeometryNodeFieldOnDomain")
|
||||
edge_to_point.data_type = "BOOLEAN"
|
||||
edge_to_point.domain = "EDGE"
|
||||
edge_to_point.location = (x + 370.0, y - 100.0)
|
||||
links.new(boundary_edge.outputs["Result"], edge_to_point.inputs["Value"])
|
||||
combine = nodes.new("FunctionNodeBooleanMath")
|
||||
combine.operation = "OR"
|
||||
combine.location = (x + 560.0, y)
|
||||
links.new(loose_vertex.outputs["Result"], combine.inputs[0])
|
||||
links.new(edge_to_point.outputs["Value"], combine.inputs[1])
|
||||
return combine.outputs[0]
|
||||
|
||||
|
||||
def add_topology_fix_zone(group, geometry, iterations, location):
|
||||
nodes = group.nodes
|
||||
links = group.links
|
||||
x, y = location
|
||||
repeat_output = nodes.new("GeometryNodeRepeatOutput")
|
||||
repeat_output.location = (x + 1450.0, y)
|
||||
repeat_input = nodes.new("GeometryNodeRepeatInput")
|
||||
repeat_input.pair_with_output(repeat_output)
|
||||
repeat_input.location = (x, y)
|
||||
links.new(geometry, repeat_input.inputs["Geometry"])
|
||||
links.new(iterations, repeat_input.inputs["Iterations"])
|
||||
|
||||
edge_neighbors = nodes.new("GeometryNodeInputMeshEdgeNeighbors")
|
||||
edge_neighbors.location = (x + 100.0, y + 420.0)
|
||||
non_manifold = new_compare(
|
||||
nodes,
|
||||
"INT",
|
||||
"GREATER_THAN",
|
||||
(x + 280.0, y + 500.0),
|
||||
2,
|
||||
)
|
||||
boundary = new_compare(
|
||||
nodes,
|
||||
"INT",
|
||||
"LESS_THAN",
|
||||
(x + 280.0, y + 320.0),
|
||||
2,
|
||||
)
|
||||
links.new(
|
||||
edge_neighbors.outputs["Face Count"],
|
||||
input_socket(non_manifold, "A", 0),
|
||||
)
|
||||
links.new(
|
||||
edge_neighbors.outputs["Face Count"],
|
||||
input_socket(boundary, "A", 0),
|
||||
)
|
||||
non_manifold_face = nodes.new("GeometryNodeFieldOnDomain")
|
||||
non_manifold_face.data_type = "BOOLEAN"
|
||||
non_manifold_face.domain = "FACE"
|
||||
non_manifold_face.location = (x + 480.0, y + 500.0)
|
||||
boundary_face = nodes.new("GeometryNodeFieldOnDomain")
|
||||
boundary_face.data_type = "BOOLEAN"
|
||||
boundary_face.domain = "FACE"
|
||||
boundary_face.location = (x + 480.0, y + 320.0)
|
||||
links.new(non_manifold.outputs["Result"], non_manifold_face.inputs["Value"])
|
||||
links.new(boundary.outputs["Result"], boundary_face.inputs["Value"])
|
||||
broken_face = nodes.new("FunctionNodeBooleanMath")
|
||||
broken_face.operation = "AND"
|
||||
broken_face.location = (x + 670.0, y + 420.0)
|
||||
links.new(non_manifold_face.outputs["Value"], broken_face.inputs[0])
|
||||
links.new(boundary_face.outputs["Value"], broken_face.inputs[1])
|
||||
remove_broken = nodes.new("GeometryNodeSeparateGeometry")
|
||||
remove_broken.domain = "FACE"
|
||||
remove_broken.location = (x + 670.0, y)
|
||||
links.new(repeat_input.outputs["Geometry"], remove_broken.inputs["Geometry"])
|
||||
links.new(broken_face.outputs[0], remove_broken.inputs["Selection"])
|
||||
|
||||
remaining_neighbors = nodes.new("GeometryNodeInputMeshEdgeNeighbors")
|
||||
remaining_neighbors.location = (x + 650.0, y - 420.0)
|
||||
remaining_boundary = new_compare(
|
||||
nodes,
|
||||
"INT",
|
||||
"LESS_THAN",
|
||||
(x + 830.0, y - 420.0),
|
||||
2,
|
||||
)
|
||||
links.new(
|
||||
remaining_neighbors.outputs["Face Count"],
|
||||
input_socket(remaining_boundary, "A", 0),
|
||||
)
|
||||
boundary_face_2 = nodes.new("GeometryNodeFieldOnDomain")
|
||||
boundary_face_2.data_type = "BOOLEAN"
|
||||
boundary_face_2.domain = "FACE"
|
||||
boundary_face_2.location = (x + 1010.0, y - 420.0)
|
||||
boundary_edge_2 = nodes.new("GeometryNodeFieldOnDomain")
|
||||
boundary_edge_2.data_type = "BOOLEAN"
|
||||
boundary_edge_2.domain = "EDGE"
|
||||
boundary_edge_2.location = (x + 1190.0, y - 420.0)
|
||||
links.new(remaining_boundary.outputs["Result"], boundary_face_2.inputs["Value"])
|
||||
links.new(boundary_face_2.outputs["Value"], boundary_edge_2.inputs["Value"])
|
||||
face_groups = nodes.new("GeometryNodeEdgesToFaceGroups")
|
||||
face_groups.location = (x + 1010.0, y - 220.0)
|
||||
links.new(
|
||||
boundary_edge_2.outputs["Value"],
|
||||
face_groups.inputs["Boundary Edges"],
|
||||
)
|
||||
accumulate = nodes.new("GeometryNodeAccumulateField")
|
||||
accumulate.data_type = "INT"
|
||||
accumulate.domain = "FACE"
|
||||
accumulate.location = (x + 1190.0, y - 160.0)
|
||||
links.new(remaining_neighbors.outputs["Face Count"], accumulate.inputs["Value"])
|
||||
links.new(face_groups.outputs["Face Group ID"], accumulate.inputs["Group ID"])
|
||||
single_face = new_compare(
|
||||
nodes,
|
||||
"INT",
|
||||
"LESS_THAN",
|
||||
(x + 1370.0, y - 160.0),
|
||||
2,
|
||||
)
|
||||
links.new(accumulate.outputs["Total"], input_socket(single_face, "A", 0))
|
||||
remove_single = nodes.new("GeometryNodeSeparateGeometry")
|
||||
remove_single.domain = "FACE"
|
||||
remove_single.location = (x + 1250.0, y + 80.0)
|
||||
links.new(remove_broken.outputs["Inverted"], remove_single.inputs["Geometry"])
|
||||
links.new(single_face.outputs["Result"], remove_single.inputs["Selection"])
|
||||
links.new(remove_single.outputs["Inverted"], repeat_output.inputs["Geometry"])
|
||||
return repeat_output.outputs["Geometry"]
|
||||
|
||||
|
||||
def add_smooth_step(group, geometry, amount, location):
|
||||
nodes = group.nodes
|
||||
links = group.links
|
||||
x, y = location
|
||||
position = nodes.new("GeometryNodeInputPosition")
|
||||
position.location = (x + 100.0, y + 500.0)
|
||||
smooth_all = nodes.new("GeometryNodeBlurAttribute")
|
||||
smooth_all.data_type = "FLOAT_VECTOR"
|
||||
smooth_all.location = (x + 300.0, y + 500.0)
|
||||
smooth_all.inputs["Iterations"].default_value = 1
|
||||
smooth_all.inputs["Weight"].default_value = 1.0
|
||||
links.new(position.outputs["Position"], smooth_all.inputs["Value"])
|
||||
|
||||
boundary_field = add_edge_boundary_field(
|
||||
nodes,
|
||||
links,
|
||||
(x + 100.0, y - 350.0),
|
||||
)
|
||||
boundary_on_points = nodes.new("GeometryNodeFieldOnDomain")
|
||||
boundary_on_points.data_type = "BOOLEAN"
|
||||
boundary_on_points.domain = "POINT"
|
||||
boundary_on_points.location = (x + 700.0, y - 350.0)
|
||||
links.new(boundary_field, boundary_on_points.inputs["Value"])
|
||||
boundary_geometry = nodes.new("GeometryNodeSeparateGeometry")
|
||||
boundary_geometry.domain = "POINT"
|
||||
boundary_geometry.location = (x + 880.0, y - 220.0)
|
||||
links.new(geometry, boundary_geometry.inputs["Geometry"])
|
||||
links.new(boundary_on_points.outputs["Value"], boundary_geometry.inputs["Selection"])
|
||||
|
||||
smooth_boundary = nodes.new("GeometryNodeBlurAttribute")
|
||||
smooth_boundary.data_type = "FLOAT_VECTOR"
|
||||
smooth_boundary.location = (x + 500.0, y + 300.0)
|
||||
smooth_boundary.inputs["Iterations"].default_value = 1
|
||||
smooth_boundary.inputs["Weight"].default_value = 0.5
|
||||
links.new(position.outputs["Position"], smooth_boundary.inputs["Value"])
|
||||
nearest_boundary = nodes.new("GeometryNodeSampleNearest")
|
||||
nearest_boundary.domain = "POINT"
|
||||
nearest_boundary.location = (x + 1050.0, y - 120.0)
|
||||
links.new(boundary_geometry.outputs["Selection"], nearest_boundary.inputs["Geometry"])
|
||||
sample_boundary = nodes.new("GeometryNodeSampleIndex")
|
||||
sample_boundary.data_type = "FLOAT_VECTOR"
|
||||
sample_boundary.domain = "POINT"
|
||||
sample_boundary.clamp = False
|
||||
sample_boundary.location = (x + 1220.0, y + 20.0)
|
||||
links.new(boundary_geometry.outputs["Selection"], sample_boundary.inputs["Geometry"])
|
||||
links.new(nearest_boundary.outputs["Index"], sample_boundary.inputs["Index"])
|
||||
links.new(smooth_boundary.outputs["Value"], sample_boundary.inputs["Value"])
|
||||
|
||||
boundary_switch = nodes.new("GeometryNodeSwitch")
|
||||
boundary_switch.input_type = "VECTOR"
|
||||
boundary_switch.location = (x + 1050.0, y + 350.0)
|
||||
links.new(boundary_on_points.outputs["Value"], boundary_switch.inputs["Switch"])
|
||||
links.new(smooth_all.outputs["Value"], boundary_switch.inputs["False"])
|
||||
links.new(sample_boundary.outputs["Value"], boundary_switch.inputs["True"])
|
||||
|
||||
normal = nodes.new("GeometryNodeInputNormal")
|
||||
normal.location = (x + 300.0, y + 100.0)
|
||||
blur_normal = nodes.new("GeometryNodeBlurAttribute")
|
||||
blur_normal.data_type = "FLOAT_VECTOR"
|
||||
blur_normal.location = (x + 500.0, y + 100.0)
|
||||
blur_normal.inputs["Iterations"].default_value = 10
|
||||
links.new(normal.outputs["Normal"], blur_normal.inputs["Value"])
|
||||
normal_length = nodes.new("ShaderNodeVectorMath")
|
||||
normal_length.operation = "LENGTH"
|
||||
normal_length.location = (x + 700.0, y + 100.0)
|
||||
links.new(blur_normal.outputs["Value"], normal_length.inputs[0])
|
||||
weight = nodes.new("ShaderNodeMath")
|
||||
weight.operation = "MULTIPLY"
|
||||
weight.location = (x + 1050.0, y + 150.0)
|
||||
links.new(normal_length.outputs["Value"], weight.inputs[0])
|
||||
links.new(amount, weight.inputs[1])
|
||||
|
||||
mix = nodes.new("ShaderNodeMix")
|
||||
mix.data_type = "VECTOR"
|
||||
mix.location = (x + 1270.0, y + 300.0)
|
||||
links.new(weight.outputs[0], input_socket(mix, "Factor", 0))
|
||||
links.new(position.outputs["Position"], input_socket(mix, "A", 4))
|
||||
links.new(boundary_switch.outputs["Output"], input_socket(mix, "B", 5))
|
||||
set_position = nodes.new("GeometryNodeSetPosition")
|
||||
set_position.location = (x + 1450.0, y)
|
||||
links.new(geometry, set_position.inputs["Geometry"])
|
||||
links.new(mix.outputs["Result"], set_position.inputs["Position"])
|
||||
return set_position.outputs["Geometry"]
|
||||
|
||||
|
||||
def add_remesh_zone(
|
||||
group,
|
||||
geometry,
|
||||
resolution,
|
||||
iterations,
|
||||
fix_steps,
|
||||
smooth_amount,
|
||||
location,
|
||||
):
|
||||
nodes = group.nodes
|
||||
links = group.links
|
||||
x, y = location
|
||||
repeat_output = nodes.new("GeometryNodeRepeatOutput")
|
||||
repeat_output.location = (x + 3300.0, y)
|
||||
repeat_input = nodes.new("GeometryNodeRepeatInput")
|
||||
repeat_input.pair_with_output(repeat_output)
|
||||
repeat_input.location = (x, y)
|
||||
links.new(geometry, repeat_input.inputs["Geometry"])
|
||||
links.new(iterations, repeat_input.inputs["Iterations"])
|
||||
|
||||
half_resolution = nodes.new("ShaderNodeMath")
|
||||
half_resolution.operation = "MULTIPLY"
|
||||
half_resolution.inputs[1].default_value = 0.5
|
||||
half_resolution.location = (x + 100.0, y - 250.0)
|
||||
links.new(resolution, half_resolution.inputs[0])
|
||||
merge = nodes.new("GeometryNodeMergeByDistance")
|
||||
merge.mode = "CONNECTED"
|
||||
merge.location = (x + 300.0, y)
|
||||
links.new(repeat_input.outputs["Geometry"], merge.inputs["Geometry"])
|
||||
links.new(half_resolution.outputs[0], merge.inputs["Distance"])
|
||||
fixed = add_topology_fix_zone(
|
||||
group,
|
||||
merge.outputs["Geometry"],
|
||||
fix_steps,
|
||||
(x + 550.0, y),
|
||||
)
|
||||
smoothed = add_smooth_step(
|
||||
group,
|
||||
fixed,
|
||||
smooth_amount,
|
||||
(x + 2100.0, y),
|
||||
)
|
||||
links.new(smoothed, repeat_output.inputs["Geometry"])
|
||||
return repeat_output.outputs["Geometry"]
|
||||
|
||||
|
||||
def add_uv_reprojection(group, source, geometry, location):
|
||||
nodes = group.nodes
|
||||
links = group.links
|
||||
x, y = location
|
||||
position = nodes.new("GeometryNodeInputPosition")
|
||||
position.location = (x, y - 200.0)
|
||||
source_uv = nodes.new("GeometryNodeInputNamedAttribute")
|
||||
source_uv.data_type = "FLOAT_VECTOR"
|
||||
source_uv.inputs["Name"].default_value = "UVMap"
|
||||
source_uv.location = (x, y)
|
||||
sample_uv = nodes.new("GeometryNodeSampleNearestSurface")
|
||||
sample_uv.data_type = "FLOAT_VECTOR"
|
||||
sample_uv.location = (x + 430.0, y)
|
||||
links.new(source, sample_uv.inputs["Mesh"])
|
||||
links.new(source_uv.outputs["Attribute"], sample_uv.inputs["Value"])
|
||||
links.new(position.outputs["Position"], sample_uv.inputs["Sample Position"])
|
||||
store_uv = nodes.new("GeometryNodeStoreNamedAttribute")
|
||||
store_uv.data_type = "FLOAT2"
|
||||
store_uv.domain = "CORNER"
|
||||
store_uv.location = (x + 1050.0, y + 250.0)
|
||||
store_uv.inputs["Name"].default_value = "UVMap"
|
||||
links.new(geometry, store_uv.inputs["Geometry"])
|
||||
links.new(sample_uv.outputs["Value"], store_uv.inputs["Value"])
|
||||
return store_uv.outputs["Geometry"]
|
||||
|
||||
|
||||
def build_group():
|
||||
group = bpy.data.node_groups.new(GROUP_NAME, "GeometryNodeTree")
|
||||
group.is_modifier = True
|
||||
interface = group.interface
|
||||
new_socket(interface, "Geometry", "INPUT", "NodeSocketGeometry")
|
||||
|
||||
resolution = new_socket(
|
||||
interface,
|
||||
"Resolution",
|
||||
"INPUT",
|
||||
"NodeSocketFloat",
|
||||
)
|
||||
resolution.default_value = 0.01
|
||||
resolution.min_value = 0.000001
|
||||
resolution.subtype = "DISTANCE"
|
||||
remesh_iterations = new_socket(
|
||||
interface,
|
||||
"Relax Iterations",
|
||||
"INPUT",
|
||||
"NodeSocketInt",
|
||||
)
|
||||
remesh_iterations.default_value = 2
|
||||
remesh_iterations.min_value = 1
|
||||
remesh_iterations.max_value = 8
|
||||
smooth_amount = new_socket(
|
||||
interface,
|
||||
"Smooth",
|
||||
"INPUT",
|
||||
"NodeSocketFloat",
|
||||
)
|
||||
smooth_amount.default_value = 1.0
|
||||
smooth_amount.min_value = 0.0
|
||||
smooth_amount.max_value = 1.0
|
||||
smooth_amount.subtype = "FACTOR"
|
||||
|
||||
culling_panel = interface.new_panel(
|
||||
name="Culling",
|
||||
default_closed=False,
|
||||
)
|
||||
confidence = new_socket(
|
||||
interface,
|
||||
"Confidence",
|
||||
"INPUT",
|
||||
"NodeSocketFloat",
|
||||
parent=culling_panel,
|
||||
)
|
||||
confidence.default_value = 0.0
|
||||
confidence.min_value = 0.0
|
||||
confidence.max_value = 1.0
|
||||
confidence.subtype = "PERCENTAGE"
|
||||
edge = new_socket(
|
||||
interface,
|
||||
"Stretch",
|
||||
"INPUT",
|
||||
"NodeSocketFloat",
|
||||
parent=culling_panel,
|
||||
)
|
||||
edge.default_value = 0.1
|
||||
edge.min_value = 0.0
|
||||
edge.max_value = 1.0
|
||||
edge.subtype = "PERCENTAGE"
|
||||
background = new_socket(
|
||||
interface,
|
||||
"Background",
|
||||
"INPUT",
|
||||
"NodeSocketBool",
|
||||
parent=culling_panel,
|
||||
)
|
||||
background.default_value = True
|
||||
|
||||
options_panel = interface.new_panel(
|
||||
name="Options",
|
||||
default_closed=True,
|
||||
)
|
||||
fix_steps = new_socket(
|
||||
interface,
|
||||
"Fix Topology",
|
||||
"INPUT",
|
||||
"NodeSocketInt",
|
||||
parent=options_panel,
|
||||
)
|
||||
fix_steps.default_value = 3
|
||||
fix_steps.min_value = 1
|
||||
fix_steps.max_value = 8
|
||||
new_socket(interface, "Geometry", "OUTPUT", "NodeSocketGeometry")
|
||||
|
||||
nodes = group.nodes
|
||||
links = group.links
|
||||
geometry_input = new_group_input(nodes, (-650.0, 50.0), {"Geometry"})
|
||||
confidence_input = new_group_input(nodes, (-1800.0, 800.0), {"Confidence"})
|
||||
stretch_input = new_group_input(nodes, (-1800.0, 50.0), {"Stretch"})
|
||||
background_input = new_group_input(nodes, (-1800.0, -350.0), {"Background"})
|
||||
remesh_input = new_group_input(
|
||||
nodes,
|
||||
(-100.0, -500.0),
|
||||
{"Resolution", "Relax Iterations", "Smooth"},
|
||||
)
|
||||
topology_input = new_group_input(nodes, (350.0, -700.0), {"Fix Topology"})
|
||||
island_input = new_group_input(nodes, (3400.0, 100.0), {"Resolution"})
|
||||
uv_source_input = new_group_input(nodes, (4200.0, -300.0), {"Geometry"})
|
||||
group_output = nodes.new("NodeGroupOutput")
|
||||
group_output.location = (5650.0, 100.0)
|
||||
|
||||
confidence_attribute = nodes.new("GeometryNodeInputNamedAttribute")
|
||||
confidence_attribute.data_type = "FLOAT"
|
||||
confidence_attribute.inputs["Name"].default_value = "o_face_confidence"
|
||||
confidence_attribute.location = (-1800.0, 600.0)
|
||||
confidence_compare = new_compare(
|
||||
nodes,
|
||||
"FLOAT",
|
||||
"LESS_THAN",
|
||||
(-1550.0, 600.0),
|
||||
0.0,
|
||||
)
|
||||
links.new(
|
||||
confidence_attribute.outputs["Attribute"],
|
||||
input_socket(confidence_compare, "A", 0),
|
||||
)
|
||||
links.new(
|
||||
confidence_input.outputs["Confidence"],
|
||||
input_socket(confidence_compare, "B", 1),
|
||||
)
|
||||
|
||||
edge_attribute = nodes.new("GeometryNodeInputNamedAttribute")
|
||||
edge_attribute.data_type = "FLOAT"
|
||||
edge_attribute.inputs["Name"].default_value = "o_edge_stretch"
|
||||
edge_attribute.location = (-1800.0, 350.0)
|
||||
edge_maximum = nodes.new("ShaderNodeMath")
|
||||
edge_maximum.operation = "MAXIMUM"
|
||||
edge_maximum.inputs[1].default_value = 0.0001
|
||||
edge_maximum.location = (-1600.0, 100.0)
|
||||
edge_reciprocal = nodes.new("ShaderNodeMath")
|
||||
edge_reciprocal.operation = "DIVIDE"
|
||||
edge_reciprocal.inputs[0].default_value = 1.0
|
||||
edge_reciprocal.location = (-1400.0, 100.0)
|
||||
edge_compare = new_compare(
|
||||
nodes,
|
||||
"FLOAT",
|
||||
"GREATER_EQUAL",
|
||||
(-1200.0, 350.0),
|
||||
1.0,
|
||||
)
|
||||
links.new(stretch_input.outputs["Stretch"], edge_maximum.inputs[0])
|
||||
links.new(edge_maximum.outputs[0], edge_reciprocal.inputs[1])
|
||||
links.new(edge_attribute.outputs["Attribute"], input_socket(edge_compare, "A", 0))
|
||||
links.new(edge_reciprocal.outputs[0], input_socket(edge_compare, "B", 1))
|
||||
|
||||
background_attribute = nodes.new("GeometryNodeInputNamedAttribute")
|
||||
background_attribute.data_type = "BOOLEAN"
|
||||
background_attribute.inputs["Name"].default_value = "o_background_face"
|
||||
background_attribute.location = (-1800.0, -200.0)
|
||||
background_and = nodes.new("FunctionNodeBooleanMath")
|
||||
background_and.operation = "AND"
|
||||
background_and.location = (-1450.0, -200.0)
|
||||
links.new(background_attribute.outputs["Attribute"], background_and.inputs[0])
|
||||
links.new(background_input.outputs["Background"], background_and.inputs[1])
|
||||
first_or = nodes.new("FunctionNodeBooleanMath")
|
||||
first_or.operation = "OR"
|
||||
first_or.location = (-950.0, 450.0)
|
||||
links.new(confidence_compare.outputs["Result"], first_or.inputs[0])
|
||||
links.new(edge_compare.outputs["Result"], first_or.inputs[1])
|
||||
final_or = nodes.new("FunctionNodeBooleanMath")
|
||||
final_or.operation = "OR"
|
||||
final_or.location = (-750.0, 300.0)
|
||||
links.new(first_or.outputs[0], final_or.inputs[0])
|
||||
links.new(background_and.outputs[0], final_or.inputs[1])
|
||||
full_stretch = new_compare(
|
||||
nodes,
|
||||
"FLOAT",
|
||||
"GREATER_EQUAL",
|
||||
(-950.0, -20.0),
|
||||
0.999999,
|
||||
)
|
||||
links.new(
|
||||
stretch_input.outputs["Stretch"],
|
||||
input_socket(full_stretch, "A", 0),
|
||||
)
|
||||
culling_selection = nodes.new("FunctionNodeBooleanMath")
|
||||
culling_selection.operation = "OR"
|
||||
culling_selection.location = (-550.0, 300.0)
|
||||
links.new(final_or.outputs[0], culling_selection.inputs[0])
|
||||
links.new(full_stretch.outputs["Result"], culling_selection.inputs[1])
|
||||
delete_geometry = nodes.new("GeometryNodeDeleteGeometry")
|
||||
delete_geometry.domain = "FACE"
|
||||
delete_geometry.location = (-350.0, 100.0)
|
||||
links.new(geometry_input.outputs["Geometry"], delete_geometry.inputs["Geometry"])
|
||||
links.new(culling_selection.outputs[0], delete_geometry.inputs["Selection"])
|
||||
|
||||
vertex_neighbors = nodes.new("GeometryNodeInputMeshVertexNeighbors")
|
||||
vertex_neighbors.location = (-550.0, -320.0)
|
||||
orphan_vertex = new_compare(
|
||||
nodes,
|
||||
"INT",
|
||||
"EQUAL",
|
||||
(-350.0, -320.0),
|
||||
0,
|
||||
)
|
||||
links.new(
|
||||
vertex_neighbors.outputs["Face Count"],
|
||||
input_socket(orphan_vertex, "A", 0),
|
||||
)
|
||||
delete_orphans = nodes.new("GeometryNodeDeleteGeometry")
|
||||
delete_orphans.domain = "POINT"
|
||||
delete_orphans.location = (-300.0, 100.0)
|
||||
links.new(delete_geometry.outputs["Geometry"], delete_orphans.inputs["Geometry"])
|
||||
links.new(orphan_vertex.outputs["Result"], delete_orphans.inputs["Selection"])
|
||||
|
||||
remeshed = add_remesh_zone(
|
||||
group,
|
||||
delete_orphans.outputs["Geometry"],
|
||||
remesh_input.outputs["Resolution"],
|
||||
remesh_input.outputs["Relax Iterations"],
|
||||
topology_input.outputs["Fix Topology"],
|
||||
remesh_input.outputs["Smooth"],
|
||||
(-300.0, 100.0),
|
||||
)
|
||||
|
||||
mesh_island = nodes.new("GeometryNodeInputMeshIsland")
|
||||
mesh_island.location = (3400.0, 700.0)
|
||||
face_area = nodes.new("GeometryNodeInputMeshFaceArea")
|
||||
face_area.location = (3400.0, 550.0)
|
||||
island_area = nodes.new("GeometryNodeAccumulateField")
|
||||
island_area.data_type = "FLOAT"
|
||||
island_area.domain = "FACE"
|
||||
island_area.location = (3600.0, 630.0)
|
||||
links.new(face_area.outputs["Area"], island_area.inputs["Value"])
|
||||
links.new(mesh_island.outputs["Island Index"], island_area.inputs["Group ID"])
|
||||
resolution_squared = nodes.new("ShaderNodeMath")
|
||||
resolution_squared.operation = "POWER"
|
||||
resolution_squared.inputs[1].default_value = 2.0
|
||||
resolution_squared.location = (3600.0, 250.0)
|
||||
links.new(island_input.outputs["Resolution"], resolution_squared.inputs[0])
|
||||
tiny_island = new_compare(
|
||||
nodes,
|
||||
"FLOAT",
|
||||
"LESS_THAN",
|
||||
(3800.0, 630.0),
|
||||
0.0,
|
||||
)
|
||||
links.new(island_area.outputs["Total"], input_socket(tiny_island, "A", 0))
|
||||
links.new(resolution_squared.outputs[0], input_socket(tiny_island, "B", 1))
|
||||
remove_tiny = nodes.new("GeometryNodeDeleteGeometry")
|
||||
remove_tiny.domain = "FACE"
|
||||
remove_tiny.location = (4000.0, 100.0)
|
||||
links.new(remeshed, remove_tiny.inputs["Geometry"])
|
||||
links.new(tiny_island.outputs["Result"], remove_tiny.inputs["Selection"])
|
||||
|
||||
reprojected = add_uv_reprojection(
|
||||
group,
|
||||
uv_source_input.outputs["Geometry"],
|
||||
remove_tiny.outputs["Geometry"],
|
||||
(4100.0, 100.0),
|
||||
)
|
||||
shade_smooth = nodes.new("GeometryNodeSetShadeSmooth")
|
||||
shade_smooth.domain = "FACE"
|
||||
shade_smooth.location = (5400.0, 100.0)
|
||||
shade_smooth.inputs["Shade Smooth"].default_value = True
|
||||
links.new(reprojected, shade_smooth.inputs["Geometry"])
|
||||
links.new(shade_smooth.outputs["Geometry"], group_output.inputs["Geometry"])
|
||||
|
||||
group.color_tag = "GEOMETRY"
|
||||
group.description = (
|
||||
"Cull DA3/BEN2 faces, repair merged topology, preserve boundaries, "
|
||||
"and reproject UVs."
|
||||
)
|
||||
group.use_fake_user = True
|
||||
return group
|
||||
|
||||
|
||||
def main():
|
||||
bpy.ops.wm.read_factory_settings(use_empty=True)
|
||||
bpy.context.preferences.filepaths.save_version = 0
|
||||
build_group()
|
||||
OUTPUT_PATH.parent.mkdir(parents=True, exist_ok=True)
|
||||
bpy.ops.wm.save_as_mainfile(filepath=str(OUTPUT_PATH), check_existing=False)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
188
tools/validate_geometry_nodes_asset.py
Normal file
188
tools/validate_geometry_nodes_asset.py
Normal file
@@ -0,0 +1,188 @@
|
||||
from pathlib import Path
|
||||
import sys
|
||||
|
||||
import bpy
|
||||
import numpy as np
|
||||
|
||||
|
||||
sys.path.insert(0, str(Path(__file__).parents[1] / "src"))
|
||||
|
||||
from snapform.image_mesh import create_matching_camera, load_geometry_node_group
|
||||
|
||||
|
||||
def set_input(modifier, group, name, value):
|
||||
item = next(
|
||||
item
|
||||
for item in group.interface.items_tree
|
||||
if item.item_type == "SOCKET"
|
||||
and item.in_out == "INPUT"
|
||||
and item.name == name
|
||||
)
|
||||
modifier[item.identifier] = value
|
||||
|
||||
|
||||
def add_face_attributes(mesh, count, background_values=None):
|
||||
for name in ("o_face_confidence", "o_edge_stretch"):
|
||||
attribute = mesh.attributes.new(name, "FLOAT", "FACE")
|
||||
attribute.data.foreach_set("value", [1.0] * count)
|
||||
background = mesh.attributes.new("o_background_face", "BOOLEAN", "FACE")
|
||||
background.data.foreach_set(
|
||||
"value",
|
||||
background_values or [False] * count,
|
||||
)
|
||||
|
||||
|
||||
def main():
|
||||
bpy.ops.wm.read_factory_settings(use_empty=True)
|
||||
group = load_geometry_node_group()
|
||||
assert load_geometry_node_group() == group
|
||||
assert group.is_modifier
|
||||
assert group.asset_data is None
|
||||
assert not any(node.bl_idname == "GeometryNodeGroup" for node in group.nodes)
|
||||
assert not any(node.bl_idname == "GeometryNodeTriangulate" for node in group.nodes)
|
||||
group_inputs = [
|
||||
node for node in group.nodes if node.bl_idname == "NodeGroupInput"
|
||||
]
|
||||
assert len(group_inputs) >= 8
|
||||
for node in group_inputs:
|
||||
visible_outputs = [
|
||||
output for output in node.outputs if output.name and not output.hide
|
||||
]
|
||||
assert 1 <= len(visible_outputs) <= 3
|
||||
uv_stores = [
|
||||
node
|
||||
for node in group.nodes
|
||||
if node.bl_idname == "GeometryNodeStoreNamedAttribute"
|
||||
and node.inputs["Name"].default_value == "UVMap"
|
||||
]
|
||||
assert len(uv_stores) == 1
|
||||
assert uv_stores[0].data_type == "FLOAT2"
|
||||
panels = {
|
||||
item.name: item
|
||||
for item in group.interface.items_tree
|
||||
if item.item_type == "PANEL"
|
||||
}
|
||||
assert not panels["Culling"].default_closed
|
||||
assert panels["Options"].default_closed
|
||||
assert [item.name for item in panels["Culling"].interface_items] == [
|
||||
"Confidence",
|
||||
"Stretch",
|
||||
"Background",
|
||||
]
|
||||
assert [item.name for item in panels["Options"].interface_items] == [
|
||||
"Fix Topology",
|
||||
]
|
||||
if bpy.app.version >= (5, 0, 0):
|
||||
assert group.library is not None
|
||||
|
||||
mesh = bpy.data.meshes.new("Validation Mesh")
|
||||
mesh.from_pydata(
|
||||
[(0, 0, 0), (1, 0, 0), (0, 1, 0), (1, 1, 0)],
|
||||
[],
|
||||
[(0, 2, 1), (1, 2, 3)],
|
||||
)
|
||||
uv = mesh.uv_layers.new(name="UVMap")
|
||||
uv.data.foreach_set("uv", [0, 0, 0, 1, 1, 0, 1, 0, 0, 1, 1, 1])
|
||||
add_face_attributes(mesh, 2, [False, True])
|
||||
|
||||
obj = bpy.data.objects.new("Validation Object", mesh)
|
||||
bpy.context.scene.collection.objects.link(obj)
|
||||
modifier = obj.modifiers.new(name=group.name, type="NODES")
|
||||
modifier.node_group = group
|
||||
modifier.show_group_selector = False
|
||||
assert not modifier.show_group_selector
|
||||
set_input(modifier, group, "Resolution", 0.000001)
|
||||
set_input(modifier, group, "Relax Iterations", 1)
|
||||
set_input(modifier, group, "Fix Topology", 0)
|
||||
set_input(modifier, group, "Smooth", 0.0)
|
||||
evaluated = obj.evaluated_get(bpy.context.evaluated_depsgraph_get())
|
||||
result = evaluated.to_mesh()
|
||||
try:
|
||||
assert len(result.polygons) == 1
|
||||
assert "UVMap" in result.attributes
|
||||
assert result.attributes["UVMap"].data_type == "FLOAT2"
|
||||
finally:
|
||||
evaluated.to_mesh_clear()
|
||||
|
||||
set_input(modifier, group, "Stretch", 1.0)
|
||||
stretch_item = next(
|
||||
item
|
||||
for item in group.interface.items_tree
|
||||
if item.item_type == "SOCKET" and item.name == "Stretch"
|
||||
)
|
||||
assert modifier[stretch_item.identifier] == 1.0
|
||||
obj.update_tag(refresh={"DATA"})
|
||||
bpy.context.view_layer.update()
|
||||
culled = obj.evaluated_get(bpy.context.evaluated_depsgraph_get())
|
||||
culled_result = culled.to_mesh()
|
||||
try:
|
||||
assert len(culled_result.polygons) == 0
|
||||
assert len(culled_result.vertices) == 0
|
||||
finally:
|
||||
culled.to_mesh_clear()
|
||||
|
||||
size = 10
|
||||
dense_vertices = [
|
||||
(x * 0.1, y * 0.1, 0.0)
|
||||
for y in range(size)
|
||||
for x in range(size)
|
||||
]
|
||||
dense_faces = []
|
||||
for y in range(size - 1):
|
||||
for x in range(size - 1):
|
||||
a = y * size + x
|
||||
b = a + 1
|
||||
c = a + size
|
||||
d = c + 1
|
||||
dense_faces.extend(((a, c, b), (b, c, d)))
|
||||
dense_mesh = bpy.data.meshes.new("Dense Validation Mesh")
|
||||
dense_mesh.from_pydata(dense_vertices, [], dense_faces)
|
||||
dense_uv = dense_mesh.uv_layers.new(name="UVMap")
|
||||
dense_uv.data.foreach_set("uv", [0.0] * (len(dense_mesh.loops) * 2))
|
||||
add_face_attributes(dense_mesh, len(dense_faces))
|
||||
dense_object = bpy.data.objects.new("Dense Validation Object", dense_mesh)
|
||||
bpy.context.scene.collection.objects.link(dense_object)
|
||||
dense_modifier = dense_object.modifiers.new(name=group.name, type="NODES")
|
||||
dense_modifier.node_group = group
|
||||
set_input(dense_modifier, group, "Resolution", 0.22)
|
||||
dense_evaluated = dense_object.evaluated_get(
|
||||
bpy.context.evaluated_depsgraph_get()
|
||||
)
|
||||
dense_result = dense_evaluated.to_mesh()
|
||||
try:
|
||||
assert 4 < len(dense_result.polygons) < len(dense_faces)
|
||||
assert all(polygon.normal.z < -0.01 for polygon in dense_result.polygons)
|
||||
finally:
|
||||
dense_evaluated.to_mesh_clear()
|
||||
|
||||
source = bpy.data.objects.new("Validation Source", None)
|
||||
bpy.context.scene.collection.objects.link(source)
|
||||
source.location = (2.0, 3.0, 4.0)
|
||||
bpy.context.view_layer.update()
|
||||
obj.matrix_world = source.matrix_world.copy()
|
||||
bpy.context.view_layer.update()
|
||||
camera = create_matching_camera(
|
||||
bpy.context,
|
||||
source,
|
||||
obj,
|
||||
"Validation",
|
||||
np.asarray(
|
||||
((100.0, 0.0, 50.0), (0.0, 100.0, 50.0), (0.0, 0.0, 1.0))
|
||||
),
|
||||
(101, 101),
|
||||
{"scale_x": 1.0, "scale_z": 1.0, "location": (0.0, 0.0, 2.0)},
|
||||
)
|
||||
assert camera.parent == obj
|
||||
camera_location = tuple(
|
||||
round(value, 5) for value in camera.matrix_world.translation
|
||||
)
|
||||
assert camera_location == (
|
||||
2.0,
|
||||
3.0,
|
||||
6.0,
|
||||
)
|
||||
print(f"Validated Blender {bpy.app.version_string}: {group.name}")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
Reference in New Issue
Block a user