feat: Add MoGe-2 support and enhance depth plane functionality
- Introduced new MoGe-2 depth plane group with geometry and material displacement options. - Updated existing depth plane group to improve background depth handling and subdivision settings. - Enhanced tests to validate MoGe-2 models and ensure proper integration with existing systems. - Modified installation environment to include new dependencies for MoGe-2 and utils3d. - Updated vendor lock file to include new artifacts for MoGe and utils3d. - Refactored download and model handling to accommodate new MoGe-2 models. - Improved validation scripts for depth plane assets to ensure compliance with new specifications.
This commit is contained in:
@@ -19,7 +19,7 @@ SnapForm 是一个 Blender 扩展,可以把一张照片、一段视频或一
|
|||||||
最快的 CUDA/MPS,否则 CPU。下载超时会自动重试国内镜像,无需手动配置
|
最快的 CUDA/MPS,否则 CPU。下载超时会自动重试国内镜像,无需手动配置
|
||||||
5. 打开 3D 视图右侧的 `SnapForm` 面板,点击 `Install Environment`
|
5. 打开 3D 视图右侧的 `SnapForm` 面板,点击 `Install Environment`
|
||||||
|
|
||||||
DA3、BEN2 源码包和 uv 已内置在扩展里,安装时直接从本地解包,不需要访问 GitHub。首次使用仍会自动
|
DA3、MoGe-2、utils3d、BEN2 源码包和 uv 已内置在扩展里,安装时直接从本地解包,不需要访问 GitHub。首次使用仍会自动
|
||||||
下载运行环境依赖(PyTorch 等),随后自动下载 `DA3 Small` 和 `BEN2 Base`。其他 AI 模型可在
|
下载运行环境依赖(PyTorch 等),随后自动下载 `DA3 Small` 和 `BEN2 Base`。其他 AI 模型可在
|
||||||
`Models` panel 中按需下载,请保持网络畅通,首次安装需要一些时间。
|
`Models` panel 中按需下载,请保持网络畅通,首次安装需要一些时间。
|
||||||
|
|
||||||
@@ -30,7 +30,7 @@ DA3、BEN2 源码包和 uv 已内置在扩展里,安装时直接从本地解
|
|||||||
1. 在 3D 视图中选中一个 Image Empty(空物体,显示方式为 Image)
|
1. 在 3D 视图中选中一个 Image Empty(空物体,显示方式为 Image)
|
||||||
2. 右键打开 `SnapForm` 子菜单,选择 `Convert to Depth Plane`
|
2. 右键打开 `SnapForm` 子菜单,选择 `Convert to Depth Plane`
|
||||||
3. 在对话框中选择:
|
3. 在对话框中选择:
|
||||||
- `Depth from`:`Original` 直接用原图,`AI Generated` 让 AI 估算深度
|
- `Depth from`:`Original` 直接用原图,`DA3` 使用现有 DA3 流程,`MoGe-2` 生成 metric depth、normal 与有效区域 mask
|
||||||
- `Resolution Scale` 使用 `1–100%`,按原图长边计算 Process Res;结果会限制在 `252–1008` 并自动对齐到 14 的倍数
|
- `Resolution Scale` 使用 `1–100%`,按原图长边计算 Process Res;结果会限制在 `252–1008` 并自动对齐到 14 的倍数
|
||||||
- `Remove Background` 默认关闭;开启后在转换期间使用 BEN2 删除背景
|
- `Remove Background` 默认关闭;开启后在转换期间使用 BEN2 删除背景
|
||||||
- `Unshaded` 保留为直接显示贴图颜色的选项
|
- `Unshaded` 保留为直接显示贴图颜色的选项
|
||||||
|
|||||||
@@ -6,24 +6,24 @@
|
|||||||
`SnapForm` 是一套从图片快速构建空间结构的 Blender Extension。目前以
|
`SnapForm` 是一套从图片快速构建空间结构的 Blender Extension。目前以
|
||||||
[Depth Anything 3](https://github.com/ByteDance-Seed/Depth-Anything-3)
|
[Depth Anything 3](https://github.com/ByteDance-Seed/Depth-Anything-3)
|
||||||
作为首个 AI 能力,在独立 Environment 中运行 DA3,可将官方导出结果保存在独立 job 目录,
|
作为首个 AI 能力,在独立 Environment 中运行 DA3,可将官方导出结果保存在独立 job 目录,
|
||||||
也可把 Image Empty 转换成带几何与材质置换的立体面片。Debug 面板还可运行
|
也可把 Image Empty 转换成带几何与材质置换的立体面片。MoGe-2 作为独立深度来源,保留其 metric depth、normal、mask 与 intrinsics 输出,不复用 DA3 Prediction 后处理。Debug 面板还可运行
|
||||||
[BEN2](https://github.com/PramaLLC/BEN2) 背景剔除功能测试。
|
[BEN2](https://github.com/PramaLLC/BEN2) 背景剔除功能测试。
|
||||||
|
|
||||||
## 当前功能
|
## 当前功能
|
||||||
|
|
||||||
- 使用 Blender Extension manifest,可通过 `Install from Disk` 安装
|
- 使用 Blender Extension manifest,可通过 `Install from Disk` 安装
|
||||||
- 在独立 Environment 中通过常驻 Service 运行 Torch、DA3 与 BEN2,避免污染 Blender Python 进程
|
- 在独立 Environment 中通过常驻 Service 运行 Torch、DA3、MoGe-2 与 BEN2,避免污染 Blender Python 进程
|
||||||
- Blender 使用内置 `urllib.request` 访问仅监听 `127.0.0.1` 的 Service;请求带随机 Token
|
- Blender 使用内置 `urllib.request` 访问仅监听 `127.0.0.1` 的 Service;请求带随机 Token
|
||||||
- 常驻 Service 复用 Torch、CUDA、BEN2 与最近使用的 DA3 模型,连续执行时不再重复加载
|
- 常驻 Service 复用 Torch、CUDA、BEN2 与最近使用的 DA3 模型,连续执行时不再重复加载
|
||||||
- 多个 Blender 实例自动连接同一个共享 Service;`AI Service` 面板显示状态、端口、客户端与模型缓存,并提供安装、启动、清理、重启和停止控制
|
- 多个 Blender 实例自动连接同一个共享 Service;`AI Service` 面板显示状态、端口、客户端与模型缓存,并提供安装、启动、清理、重启和停止控制
|
||||||
- Environment、模型和任务结果默认保存在 `~/.snapform`,可由多个 Blender 版本共享
|
- Environment、模型和任务结果默认保存在 `~/.snapform`,可由多个 Blender 版本共享
|
||||||
- 支持 DA3 Small、Base、Large 1.1、Giant 1.1 和 Nested Giant Large 1.1
|
- 当前启用 DA3 Small 和 Base;Large 1.1、Giant 1.1 与 Nested Giant Large 1.1 暂停使用
|
||||||
- 支持 BEN2 Base 单图背景剔除测试,输出透明 PNG 与推理耗时
|
- 支持 BEN2 Base 单图背景剔除测试,输出透明 PNG 与推理耗时
|
||||||
- 支持单张 JPG、JPEG、PNG、WEBP 图片或常见视频文件输入
|
- 支持单张 JPG、JPEG、PNG、WEBP 图片或常见视频文件输入
|
||||||
- 视频按顺序截取前若干帧,可设置参与重建的最大帧数
|
- 视频按顺序截取前若干帧,可设置参与重建的最大帧数
|
||||||
- `Models` panel 默认折叠,展开后可逐项查看和下载所有可用模型
|
- `Models` panel 默认折叠,展开后可逐项查看和下载所有可用模型
|
||||||
- Environment 已安装时不额外占用面板空间;模型列表每个模型只占一行
|
- Environment 已安装时不额外占用面板空间;模型列表每个模型只占一行
|
||||||
- `Debug` 默认折叠,内部的 `DA3 Output Test` 与 `BEN2 Removal Test` 也分别默认折叠
|
- `Debug` 默认折叠,内部的 `DA3 Output Test`、`MoGe-2 API Test` 与 `BEN2 Removal Test` 也分别默认折叠
|
||||||
- 使用一个输出类型下拉菜单和一个 `Generate Output` 按钮
|
- 使用一个输出类型下拉菜单和一个 `Generate Output` 按钮
|
||||||
- 只显示当前格式的专属参数;GLB 与 PLY 设置不会同时出现
|
- 只显示当前格式的专属参数;GLB 与 PLY 设置不会同时出现
|
||||||
- Debug 的模型选项按格式过滤;PLY 只提供两个带 Gaussian head 的 Giant 模型
|
- Debug 的模型选项按格式过滤;PLY 只提供两个带 Gaussian head 的 Giant 模型
|
||||||
@@ -103,38 +103,41 @@ Cyclic 沿用原 Empty 的设置。序列帧无法被 Blender 整体 Pack,因
|
|||||||
|
|
||||||
1. 在 3D View 选中 `Object Type = Empty`、`Display As = Image` 的对象。
|
1. 在 3D View 选中 `Object Type = Empty`、`Display As = Image` 的对象。
|
||||||
2. 右键打开 `SnapForm` 子菜单并选择 `Convert to Depth Plane`,通过 `Depth from` 选择深度来源:
|
2. 右键打开 `SnapForm` 子菜单并选择 `Convert to Depth Plane`,通过 `Depth from` 选择深度来源:
|
||||||
- `AI Generated`:显示 Model、Use Ray Pose 和 Resolution Scale。Process Res 先以原图长边乘以
|
- `DA3`:显示 Model、Use Ray Pose 和 Resolution Scale。Process Res 先以原图长边乘以
|
||||||
Resolution Scale,再钳制到 `252–1008`,最后对齐到最接近的 `14` 倍数。Resolution Scale 以百分比
|
Resolution Scale,再钳制到 `252–1008`,最后对齐到最接近的 `14` 倍数。Resolution Scale 以百分比
|
||||||
显示,范围为 `1–100%`,默认 `100%`;界面只在标签中实时显示最终 Process Res。
|
显示,范围为 `1–100%`,默认 `100%`;界面只在标签中实时显示最终 Process Res。
|
||||||
转换导出 `mini_npz`,使用前需要在
|
转换导出 `mini_npz`,使用前需要在
|
||||||
`Models` panel 下载所选模型,并固定使用 `process_res_method="upper_bound_resize"`;设备和参考视图策略
|
`Models` panel 下载所选模型,并固定使用 `process_res_method="upper_bound_resize"`;设备和参考视图策略
|
||||||
继续使用场景设置。Model 下拉菜单只列出已经下载的 DA3 模型;若没有可用模型,对话框会引导到
|
继续使用场景设置。Model 下拉菜单只列出已经下载的 DA3 模型;若没有可用模型,对话框会引导到
|
||||||
`Models` panel 下载。
|
`Models` panel 下载。
|
||||||
|
- `MoGe-2`:使用 Base Normal 模型输出 metric depth、normal、validity mask 与 normalized intrinsics。正式转换的 `results.npz` 不保存体积较大的 points;Debug 测试会保留 points,并另外写出 depth、normal、mask 预览。
|
||||||
- `Original`:直接把原图作为深度贴图,不显示估算参数。仅在开启 `Remove Background` 时运行 BEN2
|
- `Original`:直接把原图作为深度贴图,不显示估算参数。仅在开启 `Remove Background` 时运行 BEN2
|
||||||
取得前景 Alpha;视频或序列帧按需同时抽取原始帧与前景序列,把原始帧作为 `.depth` 序列。
|
取得前景 Alpha;视频或序列帧按需同时抽取原始帧与前景序列,把原始帧作为 `.depth` 序列。
|
||||||
`Original` 是默认深度来源。选择 `AI Generated` 时,其 Model、Use Ray Pose 与 Resolution Scale
|
`Original` 是默认深度来源。选择 `DA3` 时,其 Model、Use Ray Pose 与 Resolution Scale
|
||||||
参数会集中显示在独立边框中。`Remove Background` 与 `Unshaded` 直接显示,不再使用 Options 折叠;
|
参数会集中显示在独立边框中。`Remove Background` 与 `Unshaded` 直接显示,不再使用 Options 折叠;
|
||||||
`Keep Original` 默认关闭,并在分割线下方保持为最后一项。仅当开启 `Remove Background` 时才需要
|
`Keep Original` 默认关闭,并在分割线下方保持为最后一项。仅当开启 `Remove Background` 时才需要
|
||||||
`BEN2 Base`,并把 `.foreground` 的 Alpha 作为独立
|
`BEN2 Base`,并把 `.foreground` 的 Alpha 作为独立
|
||||||
Mask,不在 Service 中改写深度值。保存 `.foreground` 时保留 BEN2 Alpha,但 RGB 使用未经修改的原始输入;
|
Mask,不在 Service 中改写深度值。保存 `.foreground` 时保留 BEN2 Alpha,但 RGB 使用未经修改的原始输入;
|
||||||
因此材质直接用 `.foreground` 同时提供颜色和 Mask,不再额外加载原图 Image Texture。
|
因此材质直接用 `.foreground` 同时提供颜色和 Mask,不再额外加载原图 Image Texture。
|
||||||
Image Empty 引用视频或序列帧时,AI Generated 会按 Empty 的 Image Frames 数值逐帧估计
|
Image Empty 引用视频或序列帧时,DA3 与 MoGe-2 会按 Empty 的 Image Frames 数值逐帧估计
|
||||||
深度(未设置时为完整帧数,显式设为 1 时只处理 1 帧),起始帧跟随 Empty 的 Offset,与背景蒙版范围一致;
|
深度(未设置时为完整帧数,显式设为 1 时只处理 1 帧),起始帧跟随 Empty 的 Offset,与背景蒙版范围一致;
|
||||||
Offset 超出视频长度时同样把窗口收进视频内。材质节点与置换贴图的 Start 保持与原 Empty
|
Offset 超出视频长度时同样把窗口收进视频内。材质节点与置换贴图的 Start 保持与原 Empty
|
||||||
一致;Original 模式的视频或序列帧使用 `frame.0001.jpg` 等原图序列作为颜色与深度,
|
一致;Original 模式的视频或序列帧使用 `frame.0001.jpg` 等原图序列作为颜色与深度,
|
||||||
`foreground.0001.png` 只提供前景 Alpha。
|
`foreground.0001.png` 只提供前景 Alpha。
|
||||||
AI Generated 的深度仍以 EXR 序列帧(`depth.0001.exr` 起)保存在 job 目录根下。
|
DA3 的深度仍以 EXR 序列帧(`depth.0001.exr` 起)保存在 job 目录根下;MoGe-2 分别使用 `moge_depth`、`moge_mask` 与 `moge_normal` EXR 序列。
|
||||||
`Unshaded`、`Remove Background` 与 `Keep Original` 均默认关闭,因此默认删除原对象。
|
`Unshaded`、`Remove Background` 与 `Keep Original` 均默认关闭,因此默认删除原对象。
|
||||||
3. AI Generated 完成后会自动创建面片。面片几何先由 `mini_npz` 的相机内参与深度确定,再通过对象
|
3. DA3 或 MoGe-2 完成后会自动创建面片。DA3 面片几何先由 `mini_npz` 的相机内参与深度确定,再通过对象
|
||||||
Transform 匹配原 Image Empty 的可见尺寸、位置、旋转和 Offset。Original 模式直接使用源 Empty
|
Transform 匹配原 Image Empty 的可见尺寸、位置、旋转和 Offset。Original 模式直接使用源 Empty
|
||||||
的可见尺寸、Offset 与 Transform。AI Generated 会把用于匹配画面尺寸的缩放预乘到 `.depth` 数值,
|
的可见尺寸、Offset 与 Transform。DA3 会把用于匹配画面尺寸的缩放预乘到 `.depth` 数值,
|
||||||
XY 缩放烘焙到 Mesh,Z 不参与平面匹配;最终对象 Scale 为 `(1, 1, 1)`,视觉深度与旧逻辑一致。
|
XY 缩放烘焙到 Mesh,Z 不参与平面匹配;最终对象 Scale 为 `(1, 1, 1)`,视觉深度与旧逻辑一致。
|
||||||
关闭 `Keep Original` 时,成功生成后删除源 Empty。生成面片 Object 使用源 Empty 的原名;开启
|
关闭 `Keep Original` 时,成功生成后删除源 Empty。生成面片 Object 使用源 Empty 的原名;开启
|
||||||
`Keep Original` 时不处理
|
`Keep Original` 时不处理
|
||||||
同名冲突,由 Blender 自动为生成面片追加编号。
|
同名冲突,由 Blender 自动为生成面片追加编号。
|
||||||
4. Blender 只创建保留图像纵横比的最低密度基础网格,再把 `O DA3 Depth Plane` 作为 Geometry Nodes
|
4. Blender 只创建保留图像纵横比的最低密度基础网格,再把 `O DA3 Depth Plane` 作为 Geometry Nodes
|
||||||
修改器添加到对象。节点组与 `O DA3 Surface` 一同保存在 `O_SnapForm.blend`,关闭
|
修改器添加到对象。节点组与 `O DA3 Surface` 一同保存在 `O_SnapForm.blend`,关闭
|
||||||
`show_group_selector`,不为每个对象复制节点组。修改器公开以下参数:
|
`show_group_selector`,不为每个对象复制节点组。MoGe-2 使用单独的 `O MoGe2 Depth Plane`,只保留
|
||||||
|
网格细分、metric depth 位移、mask 删点和材质设置;材质使用 `O MoGe2 Depth Plane Layer`,直接接入
|
||||||
|
模型 normal,不携带 DA3 的材质置换与背景深度分支。DA3 修改器公开以下参数:
|
||||||
- `Displacement`:位于 Geometry 之后的第一项,菜单包含 `Geometry`(默认)与 `Material`。`Geometry`
|
- `Displacement`:位于 Geometry 之后的第一项,菜单包含 `Geometry`(默认)与 `Material`。`Geometry`
|
||||||
在节点组内沿法线修改网格,`Material` 保持网格平面并启用材质置换;
|
在节点组内沿法线修改网格,`Material` 保持网格平面并启用材质置换;
|
||||||
- `Subdivision`:位于第二项,默认 `5`。节点读取基础平面的 Bounding Box,短边分段数为 `2^Subdivision`,
|
- `Subdivision`:位于第二项,默认 `5`。节点读取基础平面的 Bounding Box,短边分段数为 `2^Subdivision`,
|
||||||
@@ -259,7 +262,7 @@ Height = sqrt(Area / Aspect)
|
|||||||
参考深度和尺寸。
|
参考深度和尺寸。
|
||||||
|
|
||||||
材质始终导入深度 Image Texture,并连接到 Displacement 节点和 Material Output;材质 Surface
|
材质始终导入深度 Image Texture,并连接到 Displacement 节点和 Material Output;材质 Surface
|
||||||
Displacement 设置为 `Displacement Only`,AI Generated 与 Original 都使用 `Midlevel = 0`、`Scale = 1`。
|
Displacement 设置为 `Displacement Only`,DA3 与 Original 都使用 `Midlevel = 0`、`Scale = 1`。MoGe-2 的位移只在专用 Geometry Nodes 中处理。
|
||||||
`o_material_displacement` 为 `False` 时,材质把 Height 乘以零,置换只在几何节点中生效;为 `True` 时,
|
`o_material_displacement` 为 `False` 时,材质把 Height 乘以零,置换只在几何节点中生效;为 `True` 时,
|
||||||
几何节点关闭结构置换,材质 Height 恢复。色彩贴图的 Alpha 二值结果同时用于选择原始深度或有符号的
|
几何节点关闭结构置换,材质 Height 恢复。色彩贴图的 Alpha 二值结果同时用于选择原始深度或有符号的
|
||||||
`Background Depth`,使两种置换路径保持一致;未移除背景的普通不透明图片因此全部视为前景。
|
`Background Depth`,使两种置换路径保持一致;未移除背景的普通不透明图片因此全部视为前景。
|
||||||
@@ -408,10 +411,10 @@ Python 3.12 与全部依赖。`uv` 会缓存在扩展存储目录下,后续安
|
|||||||
`SNAPFORM_TORCH_INDEX`(PyTorch 索引)、`HF_ENDPOINT`(Hugging Face 镜像)、
|
`SNAPFORM_TORCH_INDEX`(PyTorch 索引)、`HF_ENDPOINT`(Hugging Face 镜像)、
|
||||||
`HTTP(S)_PROXY`/`ALL_PROXY`(代理)。未指定 PyPI 镜像时,
|
`HTTP(S)_PROXY`/`ALL_PROXY`(代理)。未指定 PyPI 镜像时,
|
||||||
pip 安装失败会自动依次重试镜像(清华、阿里云);模型下载不预探测,官方 Hugging Face
|
pip 安装失败会自动依次重试镜像(清华、阿里云);模型下载不预探测,官方 Hugging Face
|
||||||
失败后自动回退 ModelScope(`modelscope.cn`,仓库名与 Hugging Face 一致)。
|
失败后自动回退 OmooLab R2(`models.omoolab.xyz`),并校验文件大小和 SHA-256。
|
||||||
GitHub 不再参与安装:DA3、BEN2、uv 已内置,缺少内置文件会直接报错提示重新安装,不再下载兜底。
|
GitHub 不再参与安装:DA3、BEN2、uv 已内置,缺少内置文件会直接报错提示重新安装,不再下载兜底。
|
||||||
单独下载模型时遵循同样的顺序:Hugging Face(可用 `HF_ENDPOINT` 指定其他镜像)失败后自动切换到
|
单独下载模型时遵循同样的顺序:Hugging Face(可用 `HF_ENDPOINT` 指定其他入口)失败后自动切换到
|
||||||
ModelScope。
|
OmooLab R2,不再使用 ModelScope。
|
||||||
Windows 上代理若只在系统设置里而未写入环境变量,安装程序会尝试自动从系统代理注入子进程。
|
Windows 上代理若只在系统设置里而未写入环境变量,安装程序会尝试自动从系统代理注入子进程。
|
||||||
|
|
||||||
- PyTorch 2.9.1
|
- PyTorch 2.9.1
|
||||||
|
|||||||
22
patches/moge/pyproject.toml
Normal file
22
patches/moge/pyproject.toml
Normal file
@@ -0,0 +1,22 @@
|
|||||||
|
[build-system]
|
||||||
|
requires = ["setuptools>=61.0", "wheel"]
|
||||||
|
build-backend = "setuptools.build_meta"
|
||||||
|
|
||||||
|
[project]
|
||||||
|
name = "moge"
|
||||||
|
version = "2.0.0"
|
||||||
|
description = "MoGe monocular geometry estimation"
|
||||||
|
readme = "README.md"
|
||||||
|
license = {text = "MIT"}
|
||||||
|
dependencies = [
|
||||||
|
"huggingface_hub",
|
||||||
|
"numpy",
|
||||||
|
"pillow",
|
||||||
|
"torch>=2.0.0",
|
||||||
|
"utils3d",
|
||||||
|
]
|
||||||
|
requires-python = ">=3.9"
|
||||||
|
|
||||||
|
[tool.setuptools.packages.find]
|
||||||
|
where = ["."]
|
||||||
|
include = ["moge*"]
|
||||||
16
patches/utils3d/pyproject.toml
Normal file
16
patches/utils3d/pyproject.toml
Normal file
@@ -0,0 +1,16 @@
|
|||||||
|
[build-system]
|
||||||
|
requires = ["setuptools>=61.0", "wheel"]
|
||||||
|
build-backend = "setuptools.build_meta"
|
||||||
|
|
||||||
|
[project]
|
||||||
|
name = "utils3d"
|
||||||
|
version = "1.7"
|
||||||
|
description = "Utilities for 3D computer vision"
|
||||||
|
readme = "README.md"
|
||||||
|
license = {text = "MIT"}
|
||||||
|
dependencies = ["numpy", "scipy"]
|
||||||
|
requires-python = ">=3.8"
|
||||||
|
|
||||||
|
[tool.setuptools.packages.find]
|
||||||
|
where = ["."]
|
||||||
|
include = ["utils3d*"]
|
||||||
@@ -5,6 +5,7 @@ from .image_depth import (
|
|||||||
ApplyRemovedBackground,
|
ApplyRemovedBackground,
|
||||||
BuildDepthPlane,
|
BuildDepthPlane,
|
||||||
BuildOriginalDepthPlane,
|
BuildOriginalDepthPlane,
|
||||||
|
BuildMoge2DepthPlane,
|
||||||
ConvertImageToDepthPlane,
|
ConvertImageToDepthPlane,
|
||||||
RemoveImageBackground,
|
RemoveImageBackground,
|
||||||
SnapFormImageMenu,
|
SnapFormImageMenu,
|
||||||
@@ -24,6 +25,7 @@ from .panel import (
|
|||||||
DebugPanel,
|
DebugPanel,
|
||||||
AiServicePanel,
|
AiServicePanel,
|
||||||
ModelsPanel,
|
ModelsPanel,
|
||||||
|
Moge2ApiDebugPanel,
|
||||||
)
|
)
|
||||||
from .preferences import SnapFormPreferences
|
from .preferences import SnapFormPreferences
|
||||||
from .properties import SnapFormSettings
|
from .properties import SnapFormSettings
|
||||||
@@ -41,6 +43,7 @@ CLASSES = (
|
|||||||
RemoveImageBackground,
|
RemoveImageBackground,
|
||||||
BuildDepthPlane,
|
BuildDepthPlane,
|
||||||
BuildOriginalDepthPlane,
|
BuildOriginalDepthPlane,
|
||||||
|
BuildMoge2DepthPlane,
|
||||||
ConvertImageToDepthPlane,
|
ConvertImageToDepthPlane,
|
||||||
BuildImageMesh,
|
BuildImageMesh,
|
||||||
ConvertImageToMesh,
|
ConvertImageToMesh,
|
||||||
@@ -48,6 +51,7 @@ CLASSES = (
|
|||||||
ModelsPanel,
|
ModelsPanel,
|
||||||
DebugPanel,
|
DebugPanel,
|
||||||
Da3OutputDebugPanel,
|
Da3OutputDebugPanel,
|
||||||
|
Moge2ApiDebugPanel,
|
||||||
Ben2RemovalDebugPanel,
|
Ben2RemovalDebugPanel,
|
||||||
)
|
)
|
||||||
|
|
||||||
|
|||||||
Binary file not shown.
@@ -29,6 +29,7 @@ from .models import (
|
|||||||
DA3_COMMIT = "3d835ec1a5802d64a8b8b15f817a1ab54809bfe4"
|
DA3_COMMIT = "3d835ec1a5802d64a8b8b15f817a1ab54809bfe4"
|
||||||
BEN2_COMMIT = "2c99a5da477b5523585bfa5c893888a6e818a8f6"
|
BEN2_COMMIT = "2c99a5da477b5523585bfa5c893888a6e818a8f6"
|
||||||
MANIFEST_FILENAME = "manifest.json"
|
MANIFEST_FILENAME = "manifest.json"
|
||||||
|
MANIFEST_REVISION = 5
|
||||||
SERVICE_ADDRESS = "127.0.0.1"
|
SERVICE_ADDRESS = "127.0.0.1"
|
||||||
DEFAULT_STORAGE_ROOT = Path.home() / ".snapform"
|
DEFAULT_STORAGE_ROOT = Path.home() / ".snapform"
|
||||||
JOB_FOLDER_NAMES = {
|
JOB_FOLDER_NAMES = {
|
||||||
@@ -36,6 +37,8 @@ JOB_FOLDER_NAMES = {
|
|||||||
"DOWNLOAD_MODEL": "download-model",
|
"DOWNLOAD_MODEL": "download-model",
|
||||||
"RECONSTRUCT": "export-reconstruction",
|
"RECONSTRUCT": "export-reconstruction",
|
||||||
"IMAGE_DEPTH": "generate-depth-plane",
|
"IMAGE_DEPTH": "generate-depth-plane",
|
||||||
|
"MOGE2_DEPTH": "generate-moge2-depth-plane",
|
||||||
|
"MOGE2_DEBUG": "test-moge2-api",
|
||||||
"IMAGE_MESH": "generate-image-mesh",
|
"IMAGE_MESH": "generate-image-mesh",
|
||||||
"REMOVE_BACKGROUND": "remove-background",
|
"REMOVE_BACKGROUND": "remove-background",
|
||||||
"IMAGE_BACKGROUND": "remove-image-background",
|
"IMAGE_BACKGROUND": "remove-image-background",
|
||||||
@@ -109,7 +112,10 @@ def env_installed():
|
|||||||
manifest = json.loads(ready_path.read_text(encoding="utf-8"))
|
manifest = json.loads(ready_path.read_text(encoding="utf-8"))
|
||||||
except (json.JSONDecodeError, OSError):
|
except (json.JSONDecodeError, OSError):
|
||||||
return False
|
return False
|
||||||
return isinstance(manifest, dict)
|
return (
|
||||||
|
isinstance(manifest, dict)
|
||||||
|
and manifest.get("manifest_revision") == MANIFEST_REVISION
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
def models_directory():
|
def models_directory():
|
||||||
@@ -121,6 +127,13 @@ def models_directory():
|
|||||||
def model_directory(model_key):
|
def model_directory(model_key):
|
||||||
root = models_directory()
|
root = models_directory()
|
||||||
directory = root / model_directory_name(model_key)
|
directory = root / model_directory_name(model_key)
|
||||||
|
old_moge_directory = root / "moge2_base_normal"
|
||||||
|
if (
|
||||||
|
model_key == "MOGE2_VITB_NORMAL"
|
||||||
|
and not directory.is_dir()
|
||||||
|
and old_moge_directory.is_dir()
|
||||||
|
):
|
||||||
|
old_moge_directory.rename(directory)
|
||||||
legacy_directory = root / model_key.lower()
|
legacy_directory = root / model_key.lower()
|
||||||
if (
|
if (
|
||||||
not directory.is_dir()
|
not directory.is_dir()
|
||||||
@@ -132,9 +145,11 @@ def model_directory(model_key):
|
|||||||
|
|
||||||
|
|
||||||
def model_ready(model_key):
|
def model_ready(model_key):
|
||||||
|
model = get_downloadable_model(model_key)
|
||||||
path = model_directory(model_key)
|
path = model_directory(model_key)
|
||||||
return (path / "config.json").is_file() and any(
|
return all(
|
||||||
path.glob("*.safetensors")
|
any(path.glob(pattern))
|
||||||
|
for pattern in model.ready_patterns
|
||||||
)
|
)
|
||||||
|
|
||||||
|
|
||||||
@@ -743,6 +758,14 @@ def download_model_command(model_key, status_path):
|
|||||||
]
|
]
|
||||||
for pattern in model.download_patterns:
|
for pattern in model.download_patterns:
|
||||||
command.extend(["--allow-pattern", pattern])
|
command.extend(["--allow-pattern", pattern])
|
||||||
|
for pattern in model.ready_patterns:
|
||||||
|
command.extend(["--required-pattern", pattern])
|
||||||
|
if model.r2_directory:
|
||||||
|
command.extend(["--r2-directory", model.r2_directory])
|
||||||
|
for filename, size, checksum in model.r2_files:
|
||||||
|
command.extend(
|
||||||
|
["--r2-file", f"{filename}|{size}|{checksum}"]
|
||||||
|
)
|
||||||
return command
|
return command
|
||||||
|
|
||||||
|
|
||||||
@@ -875,6 +898,55 @@ def reconstruct_request(
|
|||||||
)
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def moge_request(
|
||||||
|
settings,
|
||||||
|
status_path,
|
||||||
|
result_directory,
|
||||||
|
input_path=None,
|
||||||
|
resolution_level=None,
|
||||||
|
include_points=False,
|
||||||
|
remove_background=False,
|
||||||
|
start_frame=1,
|
||||||
|
device=None,
|
||||||
|
model_key=None,
|
||||||
|
):
|
||||||
|
from .models import BEN2_MODEL, MOGE2_MODEL
|
||||||
|
|
||||||
|
selected_model_key = model_key or MOGE2_MODEL.key
|
||||||
|
|
||||||
|
parameters = {
|
||||||
|
"input": str(
|
||||||
|
Path(input_path)
|
||||||
|
if input_path is not None
|
||||||
|
else Path(bpy.path.abspath(settings.input_path))
|
||||||
|
),
|
||||||
|
"model_dir": str(model_directory(selected_model_key)),
|
||||||
|
"output_dir": str(result_directory),
|
||||||
|
"status": str(status_path),
|
||||||
|
"resolution_level": int(
|
||||||
|
settings.moge_resolution_level
|
||||||
|
if resolution_level is None
|
||||||
|
else resolution_level
|
||||||
|
),
|
||||||
|
"include_points": bool(include_points),
|
||||||
|
"video_frames": settings.video_frames,
|
||||||
|
"start_frame": start_frame,
|
||||||
|
"device": (device or settings.device).lower(),
|
||||||
|
"remove_background_model_dir": (
|
||||||
|
str(model_directory(BEN2_MODEL.key))
|
||||||
|
if remove_background
|
||||||
|
else ""
|
||||||
|
),
|
||||||
|
}
|
||||||
|
return JobRequest(
|
||||||
|
{
|
||||||
|
"job_id": job_id_from_status(status_path),
|
||||||
|
"command": "moge",
|
||||||
|
"parameters": parameters,
|
||||||
|
}
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
def creation_flags():
|
def creation_flags():
|
||||||
if os.name != "nt":
|
if os.name != "nt":
|
||||||
return 0
|
return 0
|
||||||
|
|||||||
@@ -13,12 +13,14 @@ ORIGINAL_DISPLACEMENT_STRENGTH = 1.0
|
|||||||
ORIGINAL_DISPLACEMENT_MIDLEVEL = 0.0
|
ORIGINAL_DISPLACEMENT_MIDLEVEL = 0.0
|
||||||
BUMP_BASE_DISTANCE = 0.001
|
BUMP_BASE_DISTANCE = 0.001
|
||||||
DEPTH_PLANE_NODE_GROUP_NAME = "O DA3 Depth Plane"
|
DEPTH_PLANE_NODE_GROUP_NAME = "O DA3 Depth Plane"
|
||||||
|
MOGE_DEPTH_PLANE_NODE_GROUP_NAME = "O MoGe2 Depth Plane"
|
||||||
NODE_ASSET_NAME = "O_SnapForm.blend"
|
NODE_ASSET_NAME = "O_SnapForm.blend"
|
||||||
MATERIAL_DISPLACEMENT_ATTRIBUTE = "o_material_displacement"
|
MATERIAL_DISPLACEMENT_ATTRIBUTE = "o_material_displacement"
|
||||||
BACKGROUND_DEPTH_ATTRIBUTE = "o_background_depth"
|
BACKGROUND_DEPTH_ATTRIBUTE = "o_background_depth"
|
||||||
DEPTH_SCALE_ATTRIBUTE = "o_depth_scale"
|
DEPTH_SCALE_ATTRIBUTE = "o_depth_scale"
|
||||||
SURFACE_MATERIAL_NODE_GROUP_NAME = "O DA3 Surface Layer"
|
SURFACE_MATERIAL_NODE_GROUP_NAME = "O DA3 Surface Layer"
|
||||||
DEPTH_PLANE_MATERIAL_NODE_GROUP_NAME = "O DA3 Depth Plane Layer"
|
DEPTH_PLANE_MATERIAL_NODE_GROUP_NAME = "O DA3 Depth Plane Layer"
|
||||||
|
MOGE_MATERIAL_NODE_GROUP_NAME = "O MoGe2 Depth Plane Layer"
|
||||||
DA3_PATCH_SIZE = 14
|
DA3_PATCH_SIZE = 14
|
||||||
DA3_MIN_PROCESS_RES = 252
|
DA3_MIN_PROCESS_RES = 252
|
||||||
DA3_MAX_PROCESS_RES = 1008
|
DA3_MAX_PROCESS_RES = 1008
|
||||||
@@ -34,6 +36,8 @@ IMAGE_NAME_SUFFIXES = (
|
|||||||
".bmp",
|
".bmp",
|
||||||
".tga",
|
".tga",
|
||||||
)
|
)
|
||||||
|
|
||||||
|
|
||||||
VIDEO_NAME_SUFFIXES = (
|
VIDEO_NAME_SUFFIXES = (
|
||||||
".mp4",
|
".mp4",
|
||||||
".mov",
|
".mov",
|
||||||
@@ -48,17 +52,12 @@ VIDEO_NAME_SUFFIXES = (
|
|||||||
|
|
||||||
|
|
||||||
def image_depth_model_items(owner, context):
|
def image_depth_model_items(owner, context):
|
||||||
from .models import model_enum_items
|
from .models import depth_plane_model_enum_items
|
||||||
from .client import model_ready
|
|
||||||
|
|
||||||
return [
|
return depth_plane_model_enum_items(owner, context)
|
||||||
item
|
|
||||||
for item in model_enum_items(owner, context)
|
|
||||||
if model_ready(item[0])
|
|
||||||
]
|
|
||||||
|
|
||||||
|
|
||||||
_last_depth_plane_model = "BASE"
|
_last_depth_plane_model = "MOGE2_VITB_NORMAL"
|
||||||
|
|
||||||
|
|
||||||
def is_image_empty(candidate):
|
def is_image_empty(candidate):
|
||||||
@@ -800,6 +799,264 @@ def create_depth_image(name, depth_values):
|
|||||||
return image
|
return image
|
||||||
|
|
||||||
|
|
||||||
|
def create_moge_mask_image(name, mask_values):
|
||||||
|
import numpy as np
|
||||||
|
|
||||||
|
height, width = mask_values.shape
|
||||||
|
image = bpy.data.images.new(
|
||||||
|
name,
|
||||||
|
width=width,
|
||||||
|
height=height,
|
||||||
|
alpha=True,
|
||||||
|
float_buffer=True,
|
||||||
|
)
|
||||||
|
image.colorspace_settings.name = "Non-Color"
|
||||||
|
values = np.flipud(mask_values.astype(np.float32))
|
||||||
|
rgba = np.ones((height, width, 4), dtype=np.float32)
|
||||||
|
rgba[..., :3] = values[..., None]
|
||||||
|
rgba[..., 3] = values
|
||||||
|
image.pixels.foreach_set(rgba.ravel())
|
||||||
|
image.pack()
|
||||||
|
return image
|
||||||
|
|
||||||
|
|
||||||
|
def create_moge_normal_image(name, normal_values):
|
||||||
|
import numpy as np
|
||||||
|
|
||||||
|
converted = normal_values.astype(np.float32, copy=True)
|
||||||
|
converted *= np.array([1.0, -1.0, -1.0], dtype=np.float32)
|
||||||
|
converted = np.clip(converted * 0.5 + 0.5, 0.0, 1.0)
|
||||||
|
height, width = converted.shape[:2]
|
||||||
|
image = bpy.data.images.new(
|
||||||
|
name,
|
||||||
|
width=width,
|
||||||
|
height=height,
|
||||||
|
alpha=False,
|
||||||
|
float_buffer=True,
|
||||||
|
)
|
||||||
|
image.colorspace_settings.name = "Non-Color"
|
||||||
|
rgba = np.ones((height, width, 4), dtype=np.float32)
|
||||||
|
rgba[..., :3] = np.flipud(converted)
|
||||||
|
image.pixels.foreach_set(rgba.ravel())
|
||||||
|
image.pack()
|
||||||
|
return image
|
||||||
|
|
||||||
|
|
||||||
|
def create_moge_sequence_image(name, frames, scene, directory, prefix, kind):
|
||||||
|
import numpy as np
|
||||||
|
|
||||||
|
directory = Path(directory)
|
||||||
|
directory.mkdir(parents=True, exist_ok=True)
|
||||||
|
height, width = frames[0].shape[:2]
|
||||||
|
image = bpy.data.images.new(
|
||||||
|
name,
|
||||||
|
width=width,
|
||||||
|
height=height,
|
||||||
|
alpha=True,
|
||||||
|
float_buffer=True,
|
||||||
|
)
|
||||||
|
image.colorspace_settings.name = "Non-Color"
|
||||||
|
image_settings = scene.render.image_settings
|
||||||
|
original_format = image_settings.file_format
|
||||||
|
paths = []
|
||||||
|
try:
|
||||||
|
image_settings.file_format = "OPEN_EXR"
|
||||||
|
for index, values in enumerate(frames):
|
||||||
|
rgba = np.ones((height, width, 4), dtype=np.float32)
|
||||||
|
if kind == "depth":
|
||||||
|
flipped = np.flipud(values.astype(np.float32))
|
||||||
|
rgba[..., :3] = flipped[..., None]
|
||||||
|
elif kind == "mask":
|
||||||
|
flipped = np.flipud(values.astype(np.float32))
|
||||||
|
rgba[..., :3] = flipped[..., None]
|
||||||
|
rgba[..., 3] = flipped
|
||||||
|
else:
|
||||||
|
converted = values.astype(np.float32, copy=True)
|
||||||
|
converted *= np.array([1.0, -1.0, -1.0], dtype=np.float32)
|
||||||
|
rgba[..., :3] = np.flipud(
|
||||||
|
np.clip(converted * 0.5 + 0.5, 0.0, 1.0)
|
||||||
|
)
|
||||||
|
image.pixels.foreach_set(rgba.ravel())
|
||||||
|
path = directory / f"{prefix}.{index + 1:04d}.exr"
|
||||||
|
image.save_render(str(path), scene=scene)
|
||||||
|
paths.append(path)
|
||||||
|
finally:
|
||||||
|
image_settings.file_format = original_format
|
||||||
|
bpy.data.images.remove(image, do_unlink=True)
|
||||||
|
sequence = bpy.data.images.load(str(paths[0]), check_existing=False)
|
||||||
|
force_sequence_source(sequence)
|
||||||
|
sequence.name = name
|
||||||
|
sequence.colorspace_settings.name = "Non-Color"
|
||||||
|
pack_sequence_if_possible(sequence, len(paths))
|
||||||
|
return sequence
|
||||||
|
|
||||||
|
|
||||||
|
def moge_material_node_group():
|
||||||
|
existing = bpy.data.node_groups.get(MOGE_MATERIAL_NODE_GROUP_NAME)
|
||||||
|
if existing is not None:
|
||||||
|
return existing
|
||||||
|
group = bpy.data.node_groups.new(
|
||||||
|
MOGE_MATERIAL_NODE_GROUP_NAME,
|
||||||
|
"ShaderNodeTree",
|
||||||
|
)
|
||||||
|
for name, socket_type in (
|
||||||
|
("Color", "NodeSocketColor"),
|
||||||
|
("Alpha", "NodeSocketFloat"),
|
||||||
|
("Depth", "NodeSocketFloat"),
|
||||||
|
("Bump Scale", "NodeSocketFloat"),
|
||||||
|
):
|
||||||
|
socket = group.interface.new_socket(
|
||||||
|
name=name,
|
||||||
|
in_out="INPUT",
|
||||||
|
socket_type=socket_type,
|
||||||
|
)
|
||||||
|
if name == "Alpha":
|
||||||
|
socket.default_value = 1.0
|
||||||
|
if name == "Bump Scale":
|
||||||
|
socket.default_value = 1.0
|
||||||
|
for name, socket_type in (
|
||||||
|
("Color", "NodeSocketColor"),
|
||||||
|
("Roughness", "NodeSocketFloat"),
|
||||||
|
("Alpha", "NodeSocketFloat"),
|
||||||
|
("Normal", "NodeSocketVector"),
|
||||||
|
("Displacement", "NodeSocketVector"),
|
||||||
|
):
|
||||||
|
group.interface.new_socket(
|
||||||
|
name=name,
|
||||||
|
in_out="OUTPUT",
|
||||||
|
socket_type=socket_type,
|
||||||
|
)
|
||||||
|
nodes = group.nodes
|
||||||
|
links = group.links
|
||||||
|
group_input = nodes.new("NodeGroupInput")
|
||||||
|
group_input.location = (-900.0, 100.0)
|
||||||
|
bump = nodes.new("ShaderNodeBump")
|
||||||
|
bump.location = (250.0, -100.0)
|
||||||
|
bump_distance = nodes.new("ShaderNodeMath")
|
||||||
|
bump_distance.operation = "MULTIPLY"
|
||||||
|
bump_distance.inputs[1].default_value = BUMP_BASE_DISTANCE
|
||||||
|
bump_distance.location = (0.0, -100.0)
|
||||||
|
background = nodes.new("ShaderNodeAttribute")
|
||||||
|
background.attribute_type = "GEOMETRY"
|
||||||
|
background.attribute_name = BACKGROUND_DEPTH_ATTRIBUTE
|
||||||
|
background.location = (-650.0, -400.0)
|
||||||
|
material_mode = nodes.new("ShaderNodeAttribute")
|
||||||
|
material_mode.attribute_type = "GEOMETRY"
|
||||||
|
material_mode.attribute_name = MATERIAL_DISPLACEMENT_ATTRIBUTE
|
||||||
|
material_mode.location = (-650.0, -600.0)
|
||||||
|
strength = nodes.new("ShaderNodeAttribute")
|
||||||
|
strength.attribute_type = "GEOMETRY"
|
||||||
|
strength.attribute_name = DEPTH_SCALE_ATTRIBUTE
|
||||||
|
strength.location = (-400.0, -600.0)
|
||||||
|
clamp_depth = nodes.new("ShaderNodeMath")
|
||||||
|
clamp_depth.operation = "MAXIMUM"
|
||||||
|
clamp_depth.location = (-350.0, -350.0)
|
||||||
|
enable_depth = nodes.new("ShaderNodeMath")
|
||||||
|
enable_depth.operation = "MULTIPLY"
|
||||||
|
enable_depth.location = (-100.0, -350.0)
|
||||||
|
scale_depth = nodes.new("ShaderNodeMath")
|
||||||
|
scale_depth.operation = "MULTIPLY"
|
||||||
|
scale_depth.location = (150.0, -350.0)
|
||||||
|
displacement = nodes.new("ShaderNodeDisplacement")
|
||||||
|
displacement.inputs["Midlevel"].default_value = 0.0
|
||||||
|
displacement.inputs["Scale"].default_value = 1.0
|
||||||
|
displacement.location = (400.0, -350.0)
|
||||||
|
group_output = nodes.new("NodeGroupOutput")
|
||||||
|
group_output.location = (300.0, 100.0)
|
||||||
|
links.new(group_input.outputs["Color"], group_output.inputs["Color"])
|
||||||
|
links.new(group_input.outputs["Alpha"], group_output.inputs["Alpha"])
|
||||||
|
links.new(group_input.outputs["Color"], bump.inputs["Height"])
|
||||||
|
links.new(group_input.outputs["Bump Scale"], bump_distance.inputs[0])
|
||||||
|
links.new(bump_distance.outputs[0], bump.inputs["Distance"])
|
||||||
|
links.new(bump.outputs["Normal"], group_output.inputs["Normal"])
|
||||||
|
links.new(group_input.outputs["Depth"], clamp_depth.inputs[0])
|
||||||
|
links.new(background.outputs["Fac"], clamp_depth.inputs[1])
|
||||||
|
links.new(clamp_depth.outputs[0], enable_depth.inputs[0])
|
||||||
|
links.new(material_mode.outputs["Fac"], enable_depth.inputs[1])
|
||||||
|
links.new(enable_depth.outputs[0], scale_depth.inputs[0])
|
||||||
|
links.new(strength.outputs["Fac"], scale_depth.inputs[1])
|
||||||
|
links.new(scale_depth.outputs[0], displacement.inputs["Height"])
|
||||||
|
links.new(displacement.outputs["Displacement"], group_output.inputs["Displacement"])
|
||||||
|
group_output.inputs["Roughness"].default_value = 0.5
|
||||||
|
group.color_tag = "SHADER"
|
||||||
|
group.description = "Apply MoGe-2 color and derive bump detail from it."
|
||||||
|
group.use_fake_user = True
|
||||||
|
return group
|
||||||
|
|
||||||
|
|
||||||
|
def create_moge_material(
|
||||||
|
name,
|
||||||
|
color_image,
|
||||||
|
depth_image,
|
||||||
|
depth_scale,
|
||||||
|
unshaded=False,
|
||||||
|
frame_count=1,
|
||||||
|
frame_start=1,
|
||||||
|
use_cyclic=False,
|
||||||
|
):
|
||||||
|
material = bpy.data.materials.new(name)
|
||||||
|
material.use_nodes = True
|
||||||
|
if hasattr(material, "surface_render_method"):
|
||||||
|
material.surface_render_method = "DITHERED"
|
||||||
|
nodes = material.node_tree.nodes
|
||||||
|
nodes.clear()
|
||||||
|
links = material.node_tree.links
|
||||||
|
uv = nodes.new("ShaderNodeUVMap")
|
||||||
|
uv.uv_map = "UVMap"
|
||||||
|
uv.location = (-900.0, 200.0)
|
||||||
|
color = nodes.new("ShaderNodeTexImage")
|
||||||
|
color.image = color_image
|
||||||
|
color.location = (-650.0, 300.0)
|
||||||
|
depth = nodes.new("ShaderNodeTexImage")
|
||||||
|
depth.image = depth_image
|
||||||
|
depth.interpolation = "Linear"
|
||||||
|
depth.location = (-650.0, -100.0)
|
||||||
|
if frame_count > 1:
|
||||||
|
configure_image_user(
|
||||||
|
color.image_user,
|
||||||
|
frame_count,
|
||||||
|
frame_start=frame_start,
|
||||||
|
use_cyclic=use_cyclic,
|
||||||
|
)
|
||||||
|
configure_image_user(
|
||||||
|
depth.image_user,
|
||||||
|
frame_count,
|
||||||
|
frame_start=frame_start,
|
||||||
|
use_cyclic=use_cyclic,
|
||||||
|
)
|
||||||
|
layer = nodes.new("ShaderNodeGroup")
|
||||||
|
layer.node_tree = moge_material_node_group()
|
||||||
|
layer.name = MOGE_MATERIAL_NODE_GROUP_NAME
|
||||||
|
layer.location = (-300.0, 200.0)
|
||||||
|
output = nodes.new("ShaderNodeOutputMaterial")
|
||||||
|
output.location = (500.0, 200.0)
|
||||||
|
links.new(uv.outputs["UV"], color.inputs["Vector"])
|
||||||
|
links.new(uv.outputs["UV"], depth.inputs["Vector"])
|
||||||
|
links.new(color.outputs["Color"], layer.inputs["Color"])
|
||||||
|
links.new(color.outputs["Alpha"], layer.inputs["Alpha"])
|
||||||
|
links.new(depth.outputs["Color"], layer.inputs["Depth"])
|
||||||
|
layer.inputs["Bump Scale"].default_value = depth_scale
|
||||||
|
if unshaded:
|
||||||
|
shader = nodes.new("ShaderNodeEmission")
|
||||||
|
transparent = nodes.new("ShaderNodeBsdfTransparent")
|
||||||
|
mix = nodes.new("ShaderNodeMixShader")
|
||||||
|
links.new(layer.outputs["Color"], shader.inputs["Color"])
|
||||||
|
links.new(layer.outputs["Alpha"], mix.inputs["Fac"])
|
||||||
|
links.new(transparent.outputs["BSDF"], mix.inputs[1])
|
||||||
|
links.new(shader.outputs["Emission"], mix.inputs[2])
|
||||||
|
links.new(mix.outputs["Shader"], output.inputs["Surface"])
|
||||||
|
return material
|
||||||
|
shader = nodes.new("ShaderNodeBsdfPrincipled")
|
||||||
|
links.new(layer.outputs["Color"], shader.inputs["Base Color"])
|
||||||
|
links.new(layer.outputs["Alpha"], shader.inputs["Alpha"])
|
||||||
|
links.new(layer.outputs["Roughness"], shader.inputs["Roughness"])
|
||||||
|
links.new(layer.outputs["Normal"], shader.inputs["Normal"])
|
||||||
|
links.new(shader.outputs["BSDF"], output.inputs["Surface"])
|
||||||
|
links.new(layer.outputs["Displacement"], output.inputs["Displacement"])
|
||||||
|
set_displacement_only(material)
|
||||||
|
return material
|
||||||
|
|
||||||
|
|
||||||
def grid_segments(bounds, subdivision_level):
|
def grid_segments(bounds, subdivision_level):
|
||||||
if subdivision_level < 0:
|
if subdivision_level < 0:
|
||||||
raise ValueError("Subdivision must not be negative")
|
raise ValueError("Subdivision must not be negative")
|
||||||
@@ -1193,8 +1450,8 @@ def node_asset_path():
|
|||||||
return Path(__file__).with_name("assets") / NODE_ASSET_NAME
|
return Path(__file__).with_name("assets") / NODE_ASSET_NAME
|
||||||
|
|
||||||
|
|
||||||
def load_depth_plane_node_group():
|
def load_depth_plane_node_group(group_name=DEPTH_PLANE_NODE_GROUP_NAME):
|
||||||
existing = bpy.data.node_groups.get(DEPTH_PLANE_NODE_GROUP_NAME)
|
existing = bpy.data.node_groups.get(group_name)
|
||||||
if existing is not None:
|
if existing is not None:
|
||||||
return existing
|
return existing
|
||||||
asset_path = node_asset_path()
|
asset_path = node_asset_path()
|
||||||
@@ -1204,26 +1461,26 @@ def load_depth_plane_node_group():
|
|||||||
if not use_pack:
|
if not use_pack:
|
||||||
directory = str(asset_path) + "/NodeTree/"
|
directory = str(asset_path) + "/NodeTree/"
|
||||||
result = bpy.ops.wm.append(
|
result = bpy.ops.wm.append(
|
||||||
filepath=directory + DEPTH_PLANE_NODE_GROUP_NAME,
|
filepath=directory + group_name,
|
||||||
directory=directory,
|
directory=directory,
|
||||||
filename=DEPTH_PLANE_NODE_GROUP_NAME,
|
filename=group_name,
|
||||||
link=False,
|
link=False,
|
||||||
do_reuse_local_id=True,
|
do_reuse_local_id=True,
|
||||||
)
|
)
|
||||||
if "FINISHED" not in result:
|
if "FINISHED" not in result:
|
||||||
raise RuntimeError("Unable to append the Geometry Nodes asset")
|
raise RuntimeError("Unable to append the Geometry Nodes asset")
|
||||||
node_group = bpy.data.node_groups.get(DEPTH_PLANE_NODE_GROUP_NAME)
|
node_group = bpy.data.node_groups.get(group_name)
|
||||||
if node_group is None:
|
if node_group is None:
|
||||||
raise RuntimeError("Appended Geometry Nodes asset is unavailable")
|
raise RuntimeError("Appended Geometry Nodes asset is unavailable")
|
||||||
node_group.use_fake_user = True
|
node_group.use_fake_user = True
|
||||||
return node_group
|
return node_group
|
||||||
with bpy.data.libraries.load(str(asset_path), link=True) as (source, target):
|
with bpy.data.libraries.load(str(asset_path), link=True) as (source, target):
|
||||||
if DEPTH_PLANE_NODE_GROUP_NAME not in source.node_groups:
|
if group_name not in source.node_groups:
|
||||||
raise RuntimeError(
|
raise RuntimeError(
|
||||||
"Geometry Nodes asset is missing "
|
"Geometry Nodes asset is missing "
|
||||||
f"{DEPTH_PLANE_NODE_GROUP_NAME}"
|
f"{group_name}"
|
||||||
)
|
)
|
||||||
target.node_groups = [DEPTH_PLANE_NODE_GROUP_NAME]
|
target.node_groups = [group_name]
|
||||||
linked_group = target.node_groups[0]
|
linked_group = target.node_groups[0]
|
||||||
node_group = bpy.data.pack_linked_ids_hierarchy(root_id=linked_group)
|
node_group = bpy.data.pack_linked_ids_hierarchy(root_id=linked_group)
|
||||||
if linked_group != node_group and linked_group.users == 0:
|
if linked_group != node_group and linked_group.users == 0:
|
||||||
@@ -1278,6 +1535,42 @@ def add_depth_plane_processing(
|
|||||||
return modifier
|
return modifier
|
||||||
|
|
||||||
|
|
||||||
|
def add_moge_depth_plane_processing(
|
||||||
|
plane_object,
|
||||||
|
depth_image,
|
||||||
|
material,
|
||||||
|
frame_start=1,
|
||||||
|
frame_duration=1,
|
||||||
|
use_cyclic=False,
|
||||||
|
animate_frame=False,
|
||||||
|
):
|
||||||
|
node_group = load_depth_plane_node_group(
|
||||||
|
MOGE_DEPTH_PLANE_NODE_GROUP_NAME
|
||||||
|
)
|
||||||
|
modifier = plane_object.modifiers.new(
|
||||||
|
name=MOGE_DEPTH_PLANE_NODE_GROUP_NAME,
|
||||||
|
type="NODES",
|
||||||
|
)
|
||||||
|
modifier.node_group = node_group
|
||||||
|
modifier.show_group_selector = False
|
||||||
|
for name, value in (
|
||||||
|
("Depth Image", depth_image),
|
||||||
|
("Material", material),
|
||||||
|
("Frame", 1),
|
||||||
|
("Frame Start", frame_start),
|
||||||
|
("Frames", max(int(frame_duration or 1), 1)),
|
||||||
|
("Cyclic", use_cyclic),
|
||||||
|
):
|
||||||
|
modifier[modifier_input_identifier(node_group, name)] = value
|
||||||
|
if animate_frame:
|
||||||
|
frame_identifier = modifier_input_identifier(node_group, "Frame")
|
||||||
|
driver = modifier.driver_add(f'["{frame_identifier}"]')
|
||||||
|
driver.driver.type = "SCRIPTED"
|
||||||
|
driver.driver.expression = "frame"
|
||||||
|
plane_object.update_tag(refresh={"DATA"})
|
||||||
|
return modifier
|
||||||
|
|
||||||
|
|
||||||
def finish_depth_plane(
|
def finish_depth_plane(
|
||||||
context,
|
context,
|
||||||
source_object,
|
source_object,
|
||||||
@@ -1573,6 +1866,152 @@ class BuildDepthPlane(bpy.types.Operator):
|
|||||||
return {"FINISHED"}
|
return {"FINISHED"}
|
||||||
|
|
||||||
|
|
||||||
|
def build_moge2_depth_plane(
|
||||||
|
context,
|
||||||
|
source_object,
|
||||||
|
npz_path,
|
||||||
|
delete_source,
|
||||||
|
unshaded=False,
|
||||||
|
color_image=None,
|
||||||
|
foreground_image=None,
|
||||||
|
data_frames_directory=None,
|
||||||
|
):
|
||||||
|
import numpy as np
|
||||||
|
|
||||||
|
try:
|
||||||
|
with np.load(npz_path, allow_pickle=False) as archive:
|
||||||
|
depth = np.asarray(archive["depth"], dtype=np.float32)
|
||||||
|
mask = np.asarray(archive["mask"], dtype=bool)
|
||||||
|
intrinsics = np.asarray(archive["intrinsics"], dtype=np.float32)
|
||||||
|
except (KeyError, OSError) as error:
|
||||||
|
raise ValueError(f"Invalid MoGe-2 NPZ: {npz_path}") from error
|
||||||
|
if depth.ndim == 2:
|
||||||
|
depth = depth[None, ...]
|
||||||
|
mask = mask[None, ...]
|
||||||
|
if intrinsics.ndim == 2:
|
||||||
|
intrinsics = intrinsics[None, ...]
|
||||||
|
if not (len(depth) == len(mask) == len(intrinsics)):
|
||||||
|
raise ValueError("MoGe-2 arrays have different frame counts")
|
||||||
|
|
||||||
|
first_valid = mask[0] & np.isfinite(depth[0]) & (depth[0] > 0.0)
|
||||||
|
if not first_valid.any():
|
||||||
|
raise ValueError("MoGe-2 mask contains no valid depth")
|
||||||
|
reference_depth = float(np.median(depth[0][first_valid]))
|
||||||
|
height, width = depth[0].shape
|
||||||
|
pixel_intrinsics = intrinsics[0].copy()
|
||||||
|
pixel_intrinsics[0] *= width
|
||||||
|
pixel_intrinsics[1] *= height
|
||||||
|
bounds, _ = camera_plane_bounds(
|
||||||
|
source_object,
|
||||||
|
depth[0],
|
||||||
|
pixel_intrinsics,
|
||||||
|
1.0,
|
||||||
|
reference_depth=reference_depth,
|
||||||
|
)
|
||||||
|
displacement = [reference_depth - frame for frame in depth]
|
||||||
|
source_frame_start, _, _, source_use_cyclic = image_user_settings(
|
||||||
|
source_object
|
||||||
|
)
|
||||||
|
mesh = create_plane_mesh(source_object, bounds, None)
|
||||||
|
plane_object = bpy.data.objects.new(source_object.name, mesh)
|
||||||
|
context.collection.objects.link(plane_object)
|
||||||
|
depth_scale = apply_plane_transform(plane_object, source_object, bounds)
|
||||||
|
apply_object_scale_to_mesh(plane_object)
|
||||||
|
depth_image = create_depth_image(
|
||||||
|
f"{depth_data_name(source_object)}.moge2",
|
||||||
|
displacement[0] * depth_scale,
|
||||||
|
)
|
||||||
|
if len(depth) > 1 and data_frames_directory is not None:
|
||||||
|
bpy.data.images.remove(depth_image, do_unlink=True)
|
||||||
|
depth_image = create_moge_sequence_image(
|
||||||
|
f"{depth_data_name(source_object)}.moge2",
|
||||||
|
[frame * depth_scale for frame in displacement],
|
||||||
|
context.scene,
|
||||||
|
data_frames_directory,
|
||||||
|
"moge_depth",
|
||||||
|
"depth",
|
||||||
|
)
|
||||||
|
material = create_moge_material(
|
||||||
|
source_object.name,
|
||||||
|
foreground_image or color_image or source_object.data,
|
||||||
|
depth_image,
|
||||||
|
depth_scale,
|
||||||
|
unshaded=unshaded,
|
||||||
|
frame_count=len(depth),
|
||||||
|
frame_start=source_frame_start,
|
||||||
|
use_cyclic=source_use_cyclic,
|
||||||
|
)
|
||||||
|
mesh.materials.append(material)
|
||||||
|
modifier = add_moge_depth_plane_processing(
|
||||||
|
plane_object,
|
||||||
|
depth_image,
|
||||||
|
material,
|
||||||
|
frame_start=source_frame_start,
|
||||||
|
frame_duration=len(depth),
|
||||||
|
use_cyclic=source_use_cyclic,
|
||||||
|
animate_frame=is_animated_image(source_object.data),
|
||||||
|
)
|
||||||
|
plane_object["snapform_reference_depth"] = reference_depth
|
||||||
|
plane_object["snapform_depth_scale"] = depth_scale
|
||||||
|
plane_object["snapform_frame_count"] = len(depth)
|
||||||
|
return finish_depth_plane(
|
||||||
|
context,
|
||||||
|
source_object,
|
||||||
|
plane_object,
|
||||||
|
delete_source,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
class BuildMoge2DepthPlane(bpy.types.Operator):
|
||||||
|
bl_idname = "snapform.build_moge2_depth_plane"
|
||||||
|
bl_label = "Build MoGe-2 Depth Plane"
|
||||||
|
bl_options = {"INTERNAL"}
|
||||||
|
|
||||||
|
snapform_source_object_name: bpy.props.StringProperty(options={"HIDDEN"})
|
||||||
|
snapform_npz_path: bpy.props.StringProperty(options={"HIDDEN"})
|
||||||
|
snapform_background_path: bpy.props.StringProperty(options={"HIDDEN"})
|
||||||
|
snapform_color_frames_dir: bpy.props.StringProperty(options={"HIDDEN"})
|
||||||
|
snapform_data_frames_dir: bpy.props.StringProperty(options={"HIDDEN"})
|
||||||
|
snapform_delete_source: bpy.props.BoolProperty(default=True, options={"HIDDEN"})
|
||||||
|
snapform_unshaded: bpy.props.BoolProperty(default=False, options={"HIDDEN"})
|
||||||
|
|
||||||
|
def execute(self, context):
|
||||||
|
source_object = bpy.data.objects.get(self.snapform_source_object_name)
|
||||||
|
if not is_image_empty(source_object):
|
||||||
|
self.report({"ERROR"}, "The source image empty is no longer available")
|
||||||
|
return {"CANCELLED"}
|
||||||
|
try:
|
||||||
|
foreground_image = None
|
||||||
|
if self.snapform_background_path:
|
||||||
|
foreground_image = create_removed_background_image(
|
||||||
|
source_object,
|
||||||
|
self.snapform_background_path,
|
||||||
|
assign_to_source=False,
|
||||||
|
)
|
||||||
|
color_image = None
|
||||||
|
if self.snapform_color_frames_dir:
|
||||||
|
color_image = create_sequence_image(
|
||||||
|
f"{image_base_name(source_object.data)}.frame",
|
||||||
|
self.snapform_color_frames_dir,
|
||||||
|
)
|
||||||
|
build_moge2_depth_plane(
|
||||||
|
context,
|
||||||
|
source_object,
|
||||||
|
Path(self.snapform_npz_path),
|
||||||
|
self.snapform_delete_source,
|
||||||
|
unshaded=self.snapform_unshaded,
|
||||||
|
color_image=color_image,
|
||||||
|
foreground_image=foreground_image,
|
||||||
|
data_frames_directory=(
|
||||||
|
self.snapform_data_frames_dir or None
|
||||||
|
),
|
||||||
|
)
|
||||||
|
except (OSError, RuntimeError, ValueError) as error:
|
||||||
|
self.report({"ERROR"}, str(error))
|
||||||
|
return {"CANCELLED"}
|
||||||
|
return {"FINISHED"}
|
||||||
|
|
||||||
|
|
||||||
class BuildOriginalDepthPlane(bpy.types.Operator):
|
class BuildOriginalDepthPlane(bpy.types.Operator):
|
||||||
bl_idname = "snapform.build_original_depth_plane"
|
bl_idname = "snapform.build_original_depth_plane"
|
||||||
bl_label = "Build Original Depth Plane"
|
bl_label = "Build Original Depth Plane"
|
||||||
@@ -1746,19 +2185,6 @@ class ConvertImageToDepthPlane(bpy.types.Operator):
|
|||||||
bl_description = "Convert an image to a displaced depth plane"
|
bl_description = "Convert an image to a displaced depth plane"
|
||||||
bl_options = {"REGISTER"}
|
bl_options = {"REGISTER"}
|
||||||
|
|
||||||
snapform_depth_source: bpy.props.EnumProperty(
|
|
||||||
name="Depth from",
|
|
||||||
description="Source used to displace the plane",
|
|
||||||
items=(
|
|
||||||
(
|
|
||||||
"ESTIMATE",
|
|
||||||
"AI Generated",
|
|
||||||
"Generate depth with an AI model",
|
|
||||||
),
|
|
||||||
("ORIGINAL", "Original", "Use the original image as depth"),
|
|
||||||
),
|
|
||||||
default="ORIGINAL",
|
|
||||||
)
|
|
||||||
snapform_unshaded: bpy.props.BoolProperty(
|
snapform_unshaded: bpy.props.BoolProperty(
|
||||||
name="Unshaded",
|
name="Unshaded",
|
||||||
description="Display the source color through an Emission shader",
|
description="Display the source color through an Emission shader",
|
||||||
@@ -1771,8 +2197,9 @@ class ConvertImageToDepthPlane(bpy.types.Operator):
|
|||||||
)
|
)
|
||||||
snapform_model: bpy.props.EnumProperty(
|
snapform_model: bpy.props.EnumProperty(
|
||||||
name="Model",
|
name="Model",
|
||||||
description="Model used to estimate image depth",
|
description="Model or original image used to build depth",
|
||||||
items=image_depth_model_items,
|
items=image_depth_model_items,
|
||||||
|
default=1,
|
||||||
)
|
)
|
||||||
snapform_use_ray_pose: bpy.props.BoolProperty(
|
snapform_use_ray_pose: bpy.props.BoolProperty(
|
||||||
name="Use Ray Pose",
|
name="Use Ray Pose",
|
||||||
@@ -1780,6 +2207,13 @@ class ConvertImageToDepthPlane(bpy.types.Operator):
|
|||||||
default=True,
|
default=True,
|
||||||
)
|
)
|
||||||
snapform_process_res_factor: process_resolution_scale_property()
|
snapform_process_res_factor: process_resolution_scale_property()
|
||||||
|
snapform_moge_resolution_level: bpy.props.IntProperty(
|
||||||
|
name="Resolution Level",
|
||||||
|
description="MoGe-2 inference resolution level; higher is finer and slower",
|
||||||
|
default=5,
|
||||||
|
min=0,
|
||||||
|
max=9,
|
||||||
|
)
|
||||||
snapform_keep_original: bpy.props.BoolProperty(
|
snapform_keep_original: bpy.props.BoolProperty(
|
||||||
name="Keep Original",
|
name="Keep Original",
|
||||||
description="Keep the source Image Empty after conversion",
|
description="Keep the source Image Empty after conversion",
|
||||||
@@ -1789,14 +2223,12 @@ class ConvertImageToDepthPlane(bpy.types.Operator):
|
|||||||
@classmethod
|
@classmethod
|
||||||
def poll(cls, context):
|
def poll(cls, context):
|
||||||
from .operators import active_job
|
from .operators import active_job
|
||||||
from .client import service_busy, model_ready
|
from .client import service_busy
|
||||||
from .models import MODELS
|
|
||||||
|
|
||||||
return (
|
return (
|
||||||
is_image_empty(context.object)
|
is_image_empty(context.object)
|
||||||
and active_job() is None
|
and active_job() is None
|
||||||
and not service_busy()
|
and not service_busy()
|
||||||
and any(model_ready(model_key) for model_key in MODELS)
|
|
||||||
)
|
)
|
||||||
|
|
||||||
def invoke(self, context, _event):
|
def invoke(self, context, _event):
|
||||||
@@ -1816,17 +2248,19 @@ class ConvertImageToDepthPlane(bpy.types.Operator):
|
|||||||
|
|
||||||
def draw(self, context):
|
def draw(self, context):
|
||||||
layout = self.layout
|
layout = self.layout
|
||||||
layout.prop(self, "snapform_depth_source")
|
layout.prop(self, "snapform_model")
|
||||||
if self.snapform_depth_source == "ESTIMATE":
|
if self.snapform_model in {"SMALL", "BASE"}:
|
||||||
estimate_box = layout.box()
|
estimate_box = layout.box()
|
||||||
available_models = image_depth_model_items(self, context)
|
|
||||||
if not available_models:
|
|
||||||
estimate_box.label(text="No DA3 models installed", icon="ERROR")
|
|
||||||
estimate_box.label(text="Download one in Models")
|
|
||||||
else:
|
|
||||||
estimate_box.prop(self, "snapform_model")
|
|
||||||
estimate_box.prop(self, "snapform_use_ray_pose")
|
estimate_box.prop(self, "snapform_use_ray_pose")
|
||||||
draw_process_resolution(estimate_box, self, context.object.data)
|
draw_process_resolution(estimate_box, self, context.object.data)
|
||||||
|
elif self.snapform_model.startswith("MOGE2_"):
|
||||||
|
from .client import model_ready
|
||||||
|
|
||||||
|
moge_box = layout.box()
|
||||||
|
if not model_ready(self.snapform_model):
|
||||||
|
moge_box.label(text="MoGe-2 model is not installed", icon="ERROR")
|
||||||
|
moge_box.label(text="Download it in Models")
|
||||||
|
moge_box.prop(self, "snapform_moge_resolution_level")
|
||||||
layout.prop(self, "snapform_remove_background")
|
layout.prop(self, "snapform_remove_background")
|
||||||
layout.prop(self, "snapform_unshaded")
|
layout.prop(self, "snapform_unshaded")
|
||||||
layout.separator(type="LINE")
|
layout.separator(type="LINE")
|
||||||
@@ -1835,6 +2269,7 @@ class ConvertImageToDepthPlane(bpy.types.Operator):
|
|||||||
def execute(self, context):
|
def execute(self, context):
|
||||||
from .operators import (
|
from .operators import (
|
||||||
IMAGE_DEPTH_JOB,
|
IMAGE_DEPTH_JOB,
|
||||||
|
MOGE2_DEPTH_JOB,
|
||||||
ORIGINAL_BACKGROUND_JOB,
|
ORIGINAL_BACKGROUND_JOB,
|
||||||
ORIGINAL_FRAMES_JOB,
|
ORIGINAL_FRAMES_JOB,
|
||||||
)
|
)
|
||||||
@@ -1844,7 +2279,7 @@ class ConvertImageToDepthPlane(bpy.types.Operator):
|
|||||||
context.scene.snapform_settings.video_frames = (
|
context.scene.snapform_settings.video_frames = (
|
||||||
default_extract_frames(source_object)
|
default_extract_frames(source_object)
|
||||||
)
|
)
|
||||||
if self.snapform_depth_source == "ORIGINAL":
|
if self.snapform_model == "ORIGINAL":
|
||||||
try:
|
try:
|
||||||
source_path, delete_input = prepare_image_input(
|
source_path, delete_input = prepare_image_input(
|
||||||
source_object.data,
|
source_object.data,
|
||||||
@@ -1871,6 +2306,34 @@ class ConvertImageToDepthPlane(bpy.types.Operator):
|
|||||||
remove_temporary_input(source_path)
|
remove_temporary_input(source_path)
|
||||||
return {"CANCELLED"}
|
return {"CANCELLED"}
|
||||||
|
|
||||||
|
if self.snapform_model.startswith("MOGE2_"):
|
||||||
|
try:
|
||||||
|
source_path, delete_input = prepare_image_input(
|
||||||
|
source_object.data,
|
||||||
|
context.scene,
|
||||||
|
)
|
||||||
|
except RuntimeError as error:
|
||||||
|
self.report({"ERROR"}, str(error))
|
||||||
|
return {"CANCELLED"}
|
||||||
|
result = bpy.ops.snapform.run_job(
|
||||||
|
job_type=MOGE2_DEPTH_JOB,
|
||||||
|
snapform_input_path=str(source_path),
|
||||||
|
snapform_source_object_name=source_object.name,
|
||||||
|
snapform_moge_resolution_level=(
|
||||||
|
self.snapform_moge_resolution_level
|
||||||
|
),
|
||||||
|
snapform_model_key=self.snapform_model,
|
||||||
|
snapform_delete_source=not self.snapform_keep_original,
|
||||||
|
snapform_delete_input=delete_input,
|
||||||
|
snapform_unshaded=self.snapform_unshaded,
|
||||||
|
snapform_remove_background=self.snapform_remove_background,
|
||||||
|
)
|
||||||
|
if "CANCELLED" not in result:
|
||||||
|
return {"FINISHED"}
|
||||||
|
if delete_input:
|
||||||
|
remove_temporary_input(source_path)
|
||||||
|
return {"CANCELLED"}
|
||||||
|
|
||||||
image = source_object.data
|
image = source_object.data
|
||||||
try:
|
try:
|
||||||
process_res = scaled_process_resolution(
|
process_res = scaled_process_resolution(
|
||||||
|
|||||||
@@ -20,7 +20,7 @@ SCRIPT_DIRECTORY = Path(__file__).resolve().parent
|
|||||||
if str(SCRIPT_DIRECTORY) not in sys.path:
|
if str(SCRIPT_DIRECTORY) not in sys.path:
|
||||||
sys.path.insert(0, str(SCRIPT_DIRECTORY))
|
sys.path.insert(0, str(SCRIPT_DIRECTORY))
|
||||||
|
|
||||||
from models import BEN2_MODEL, MODELS
|
from models import BEN2_MODEL, DOWNLOADABLE_MODELS, MOGE2_MODEL
|
||||||
|
|
||||||
|
|
||||||
PYPI_MIRRORS = (
|
PYPI_MIRRORS = (
|
||||||
@@ -34,12 +34,13 @@ XFORMERS_VERSION = "0.0.33.post2"
|
|||||||
ADDICT_VERSION = "2.4.0"
|
ADDICT_VERSION = "2.4.0"
|
||||||
FASTAPI_VERSION = "0.139.2"
|
FASTAPI_VERSION = "0.139.2"
|
||||||
UVICORN_VERSION = "0.51.0"
|
UVICORN_VERSION = "0.51.0"
|
||||||
|
MANIFEST_REVISION = 5
|
||||||
TORCH_INDEXES = {
|
TORCH_INDEXES = {
|
||||||
"cu128": "https://download.pytorch.org/whl/cu128",
|
"cu128": "https://download.pytorch.org/whl/cu128",
|
||||||
"cpu": "https://download.pytorch.org/whl/cpu",
|
"cpu": "https://download.pytorch.org/whl/cpu",
|
||||||
}
|
}
|
||||||
_resolved_pypi_index = None
|
_resolved_pypi_index = None
|
||||||
DEFAULT_MODEL_KEYS = ("SMALL", BEN2_MODEL.key)
|
DEFAULT_MODEL_KEYS = (MOGE2_MODEL.key, BEN2_MODEL.key)
|
||||||
RUNTIME_STAGE_COUNT = 3
|
RUNTIME_STAGE_COUNT = 3
|
||||||
ENVIRONMENT_SLICE = (0.02, 0.06)
|
ENVIRONMENT_SLICE = (0.02, 0.06)
|
||||||
PYTORCH_SLICE = (0.06, 0.48)
|
PYTORCH_SLICE = (0.06, 0.48)
|
||||||
@@ -61,6 +62,7 @@ CHECK_CODE = (
|
|||||||
"import json; import torch; "
|
"import json; import torch; "
|
||||||
"from depth_anything_3.api import DepthAnything3; "
|
"from depth_anything_3.api import DepthAnything3; "
|
||||||
"from ben2 import BEN_Base; "
|
"from ben2 import BEN_Base; "
|
||||||
|
"from moge.model.v2 import MoGeModel; "
|
||||||
"import fastapi, uvicorn; "
|
"import fastapi, uvicorn; "
|
||||||
"mps = getattr(torch.backends, 'mps', None); "
|
"mps = getattr(torch.backends, 'mps', None); "
|
||||||
"print('SNAPFORM_DEVICES=' + json.dumps({"
|
"print('SNAPFORM_DEVICES=' + json.dumps({"
|
||||||
@@ -70,6 +72,14 @@ CHECK_CODE = (
|
|||||||
)
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def configure_standard_streams():
|
||||||
|
"""Windows 图形进程可能继承 GBK stdout,安装日志统一写为 UTF-8。"""
|
||||||
|
for stream in (sys.stdout, sys.stderr):
|
||||||
|
reconfigure = getattr(stream, "reconfigure", None)
|
||||||
|
if callable(reconfigure):
|
||||||
|
reconfigure(encoding="utf-8", errors="backslashreplace")
|
||||||
|
|
||||||
|
|
||||||
def write_status(
|
def write_status(
|
||||||
path,
|
path,
|
||||||
progress,
|
progress,
|
||||||
@@ -98,6 +108,8 @@ def write_status(
|
|||||||
def child_environment(environment=None):
|
def child_environment(environment=None):
|
||||||
"""Inherit the current environment and inject system/registry proxy settings into child processes (uv only reads proxy env variables)."""
|
"""Inherit the current environment and inject system/registry proxy settings into child processes (uv only reads proxy env variables)."""
|
||||||
process_environment = os.environ.copy()
|
process_environment = os.environ.copy()
|
||||||
|
process_environment["PYTHONIOENCODING"] = "utf-8:backslashreplace"
|
||||||
|
process_environment["PYTHONUTF8"] = "1"
|
||||||
for key, value in urllib.request.getproxies().items():
|
for key, value in urllib.request.getproxies().items():
|
||||||
env_key = f"{key.upper()}_PROXY"
|
env_key = f"{key.upper()}_PROXY"
|
||||||
if env_key not in process_environment:
|
if env_key not in process_environment:
|
||||||
@@ -334,6 +346,14 @@ def install_stages(args, venv_dir, python, uv, vendor_dir=None):
|
|||||||
"ben2",
|
"ben2",
|
||||||
"ben2.tar.gz",
|
"ben2.tar.gz",
|
||||||
)
|
)
|
||||||
|
utils3d_requirement = archive_requirement(
|
||||||
|
"utils3d",
|
||||||
|
"utils3d.tar.gz",
|
||||||
|
)
|
||||||
|
moge_requirement = archive_requirement(
|
||||||
|
"moge",
|
||||||
|
"moge.tar.gz",
|
||||||
|
)
|
||||||
torch_command = [
|
torch_command = [
|
||||||
uv,
|
uv,
|
||||||
"pip",
|
"pip",
|
||||||
@@ -371,6 +391,8 @@ def install_stages(args, venv_dir, python, uv, vendor_dir=None):
|
|||||||
da3_requirement,
|
da3_requirement,
|
||||||
f"addict=={ADDICT_VERSION}",
|
f"addict=={ADDICT_VERSION}",
|
||||||
ben2_requirement,
|
ben2_requirement,
|
||||||
|
utils3d_requirement,
|
||||||
|
moge_requirement,
|
||||||
f"fastapi=={FASTAPI_VERSION}",
|
f"fastapi=={FASTAPI_VERSION}",
|
||||||
f"uvicorn=={UVICORN_VERSION}",
|
f"uvicorn=={UVICORN_VERSION}",
|
||||||
"setuptools",
|
"setuptools",
|
||||||
@@ -398,7 +420,7 @@ def install_stages(args, venv_dir, python, uv, vendor_dir=None):
|
|||||||
2,
|
2,
|
||||||
("AI components", xformers_command, None, "install"),
|
("AI components", xformers_command, None, "install"),
|
||||||
)
|
)
|
||||||
models = {**MODELS, BEN2_MODEL.key: BEN2_MODEL}
|
models = DOWNLOADABLE_MODELS
|
||||||
worker_script = Path(__file__).with_name("worker.py")
|
worker_script = Path(__file__).with_name("worker.py")
|
||||||
models_dir = Path(args.models_dir)
|
models_dir = Path(args.models_dir)
|
||||||
for model_key in DEFAULT_MODEL_KEYS:
|
for model_key in DEFAULT_MODEL_KEYS:
|
||||||
@@ -418,6 +440,16 @@ def install_stages(args, venv_dir, python, uv, vendor_dir=None):
|
|||||||
]
|
]
|
||||||
for pattern in model.download_patterns:
|
for pattern in model.download_patterns:
|
||||||
download_command.extend(["--allow-pattern", pattern])
|
download_command.extend(["--allow-pattern", pattern])
|
||||||
|
for pattern in model.ready_patterns:
|
||||||
|
download_command.extend(["--required-pattern", pattern])
|
||||||
|
if model.r2_directory:
|
||||||
|
download_command.extend(
|
||||||
|
["--r2-directory", model.r2_directory]
|
||||||
|
)
|
||||||
|
for filename, size, checksum in model.r2_files:
|
||||||
|
download_command.extend(
|
||||||
|
["--r2-file", f"{filename}|{size}|{checksum}"]
|
||||||
|
)
|
||||||
stages.append(
|
stages.append(
|
||||||
(
|
(
|
||||||
model.label,
|
model.label,
|
||||||
@@ -437,7 +469,10 @@ def env_installed(venv_dir, manifest_path=None):
|
|||||||
payload = json.loads(ready_path.read_text(encoding="utf-8"))
|
payload = json.loads(ready_path.read_text(encoding="utf-8"))
|
||||||
except (json.JSONDecodeError, OSError):
|
except (json.JSONDecodeError, OSError):
|
||||||
return False
|
return False
|
||||||
return isinstance(payload, dict)
|
return (
|
||||||
|
isinstance(payload, dict)
|
||||||
|
and payload.get("manifest_revision") == MANIFEST_REVISION
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
def installed_devices(args):
|
def installed_devices(args):
|
||||||
@@ -557,16 +592,66 @@ def bundled_uv_archive():
|
|||||||
return path if path.is_file() else None
|
return path if path.is_file() else None
|
||||||
|
|
||||||
|
|
||||||
|
def vendor_overlay_directory(filename):
|
||||||
|
names = {
|
||||||
|
"depth-anything-3.tar.gz": "depth-anything-3",
|
||||||
|
"moge.tar.gz": "moge",
|
||||||
|
"utils3d.tar.gz": "utils3d",
|
||||||
|
}
|
||||||
|
overlay_name = names.get(filename)
|
||||||
|
if overlay_name is None:
|
||||||
|
return None
|
||||||
|
project_root = Path(__file__).resolve().parents[2]
|
||||||
|
overlay = project_root / "patches" / overlay_name
|
||||||
|
return overlay if overlay.is_dir() else None
|
||||||
|
|
||||||
|
|
||||||
|
def copy_archive_with_overlay(source, target, overlay):
|
||||||
|
"""开发目录使用原始 vendor 时,应用与正式打包相同的源码覆盖。"""
|
||||||
|
with tarfile.open(source, "r:gz") as input_archive:
|
||||||
|
members = input_archive.getmembers()
|
||||||
|
top_level = members[0].name.split("/", 1)[0]
|
||||||
|
with tarfile.open(target, "w:gz") as output_archive:
|
||||||
|
for member in members:
|
||||||
|
if member.isdir() or member.islnk() or member.issym():
|
||||||
|
output_archive.addfile(member)
|
||||||
|
continue
|
||||||
|
relative = member.name[len(top_level) + 1 :]
|
||||||
|
replacement = overlay / relative
|
||||||
|
if replacement.is_file():
|
||||||
|
info = tarfile.TarInfo(member.name)
|
||||||
|
info.size = replacement.stat().st_size
|
||||||
|
info.mode = member.mode
|
||||||
|
with replacement.open("rb") as stream:
|
||||||
|
output_archive.addfile(info, stream)
|
||||||
|
continue
|
||||||
|
with input_archive.extractfile(member) as stream:
|
||||||
|
output_archive.addfile(member, stream)
|
||||||
|
|
||||||
|
|
||||||
def copy_vendor_inputs(bundled_dir, vendor_dir):
|
def copy_vendor_inputs(bundled_dir, vendor_dir):
|
||||||
"""Copy bundled installation inputs into a temporary directory."""
|
"""Copy bundled installation inputs into a temporary directory."""
|
||||||
for name in ("depth-anything-3.tar.gz", "ben2.tar.gz"):
|
for name in (
|
||||||
|
"depth-anything-3.tar.gz",
|
||||||
|
"ben2.tar.gz",
|
||||||
|
"utils3d.tar.gz",
|
||||||
|
"moge.tar.gz",
|
||||||
|
):
|
||||||
bundled = Path(bundled_dir) / name
|
bundled = Path(bundled_dir) / name
|
||||||
if not bundled.is_file():
|
if not bundled.is_file():
|
||||||
continue
|
continue
|
||||||
target = Path(vendor_dir) / name
|
target = Path(vendor_dir) / name
|
||||||
if target.is_file() and filecmp.cmp(bundled, target, shallow=False):
|
overlay = vendor_overlay_directory(name)
|
||||||
|
if overlay is None and target.is_file() and filecmp.cmp(
|
||||||
|
bundled,
|
||||||
|
target,
|
||||||
|
shallow=False,
|
||||||
|
):
|
||||||
continue
|
continue
|
||||||
Path(vendor_dir).mkdir(parents=True, exist_ok=True)
|
Path(vendor_dir).mkdir(parents=True, exist_ok=True)
|
||||||
|
if overlay is not None and tarfile.is_tarfile(bundled):
|
||||||
|
copy_archive_with_overlay(bundled, target, overlay)
|
||||||
|
else:
|
||||||
shutil.copy2(bundled, target)
|
shutil.copy2(bundled, target)
|
||||||
|
|
||||||
|
|
||||||
@@ -574,7 +659,12 @@ def remove_legacy_vendor(storage_root):
|
|||||||
"""Remove installation inputs persisted by older development builds."""
|
"""Remove installation inputs persisted by older development builds."""
|
||||||
root = Path(storage_root)
|
root = Path(storage_root)
|
||||||
shutil.rmtree(root / "vendor", ignore_errors=True)
|
shutil.rmtree(root / "vendor", ignore_errors=True)
|
||||||
for name in ("depth-anything-3.tar.gz", "ben2.tar.gz"):
|
for name in (
|
||||||
|
"depth-anything-3.tar.gz",
|
||||||
|
"ben2.tar.gz",
|
||||||
|
"utils3d.tar.gz",
|
||||||
|
"moge.tar.gz",
|
||||||
|
):
|
||||||
(root / name).unlink(missing_ok=True)
|
(root / name).unlink(missing_ok=True)
|
||||||
|
|
||||||
|
|
||||||
@@ -799,6 +889,7 @@ def install(args):
|
|||||||
ready_path.write_text(
|
ready_path.write_text(
|
||||||
json.dumps(
|
json.dumps(
|
||||||
{
|
{
|
||||||
|
"manifest_revision": MANIFEST_REVISION,
|
||||||
"python": "3.12",
|
"python": "3.12",
|
||||||
"torch": TORCH_VERSION,
|
"torch": TORCH_VERSION,
|
||||||
"xformers": XFORMERS_VERSION,
|
"xformers": XFORMERS_VERSION,
|
||||||
@@ -870,6 +961,7 @@ def parse_args():
|
|||||||
|
|
||||||
|
|
||||||
def main():
|
def main():
|
||||||
|
configure_standard_streams()
|
||||||
args = parse_args()
|
args = parse_args()
|
||||||
try:
|
try:
|
||||||
install(args)
|
install(args)
|
||||||
|
|||||||
@@ -11,6 +11,10 @@ class ModelSpec:
|
|||||||
description: str
|
description: str
|
||||||
supports_gs: bool
|
supports_gs: bool
|
||||||
download_patterns: tuple[str, ...] = ()
|
download_patterns: tuple[str, ...] = ()
|
||||||
|
ready_patterns: tuple[str, ...] = ("config.json", "*.safetensors")
|
||||||
|
enabled: bool = True
|
||||||
|
r2_directory: str = ""
|
||||||
|
r2_files: tuple[tuple[str, int, str], ...] = ()
|
||||||
|
|
||||||
|
|
||||||
MODELS = {
|
MODELS = {
|
||||||
@@ -22,6 +26,11 @@ MODELS = {
|
|||||||
license_name="Apache 2.0",
|
license_name="Apache 2.0",
|
||||||
description="Fastest any-view model with Apache 2.0 weights",
|
description="Fastest any-view model with Apache 2.0 weights",
|
||||||
supports_gs=False,
|
supports_gs=False,
|
||||||
|
r2_directory="DA3Small",
|
||||||
|
r2_files=(
|
||||||
|
("config.json", 1202, "a486e29e82b7ab4a7d4cefc1ea4526cfe2ae438a572c8ca98917cfbcde7447d2"),
|
||||||
|
("model.safetensors", 137248940, "364492e38a3a06d221ac75da7f6621ada3f2361cd24fde11ba79091e9f40efcf"),
|
||||||
|
),
|
||||||
),
|
),
|
||||||
"BASE": ModelSpec(
|
"BASE": ModelSpec(
|
||||||
key="BASE",
|
key="BASE",
|
||||||
@@ -31,6 +40,12 @@ MODELS = {
|
|||||||
license_name="Apache 2.0",
|
license_name="Apache 2.0",
|
||||||
description="Balanced any-view model with Apache 2.0 weights",
|
description="Balanced any-view model with Apache 2.0 weights",
|
||||||
supports_gs=False,
|
supports_gs=False,
|
||||||
|
r2_directory="DA3Base",
|
||||||
|
r2_files=(
|
||||||
|
("config.json", 1205, "5e34115ebc17bd2d8d43033c5f72e9446ac8833fd61d3fa160b7e67e0bb5b7b5"),
|
||||||
|
("configuration.json", 74, "b684d553257b2ca379fd5dfdf96a6e4a633eec3b969c97119c9875465a7c07a8"),
|
||||||
|
("model.safetensors", 541518028, "e01067dc1659613083d9145a9a2547ccdbe6ccbbf83c4fe7b3e8a4e2bdae78b5"),
|
||||||
|
),
|
||||||
),
|
),
|
||||||
"LARGE_1_1": ModelSpec(
|
"LARGE_1_1": ModelSpec(
|
||||||
key="LARGE_1_1",
|
key="LARGE_1_1",
|
||||||
@@ -40,6 +55,7 @@ MODELS = {
|
|||||||
license_name="CC BY-NC 4.0",
|
license_name="CC BY-NC 4.0",
|
||||||
description="Higher quality non-commercial model with larger VRAM use",
|
description="Higher quality non-commercial model with larger VRAM use",
|
||||||
supports_gs=False,
|
supports_gs=False,
|
||||||
|
enabled=False,
|
||||||
),
|
),
|
||||||
"GIANT_1_1": ModelSpec(
|
"GIANT_1_1": ModelSpec(
|
||||||
key="GIANT_1_1",
|
key="GIANT_1_1",
|
||||||
@@ -49,6 +65,7 @@ MODELS = {
|
|||||||
license_name="CC BY-NC 4.0",
|
license_name="CC BY-NC 4.0",
|
||||||
description="Non-commercial Giant model with Gaussian head",
|
description="Non-commercial Giant model with Gaussian head",
|
||||||
supports_gs=True,
|
supports_gs=True,
|
||||||
|
enabled=False,
|
||||||
),
|
),
|
||||||
"NESTED_GIANT_LARGE_1_1": ModelSpec(
|
"NESTED_GIANT_LARGE_1_1": ModelSpec(
|
||||||
key="NESTED_GIANT_LARGE_1_1",
|
key="NESTED_GIANT_LARGE_1_1",
|
||||||
@@ -58,6 +75,7 @@ MODELS = {
|
|||||||
license_name="CC BY-NC 4.0",
|
license_name="CC BY-NC 4.0",
|
||||||
description="Non-commercial nested model with metric depth and Gaussian head",
|
description="Non-commercial nested model with metric depth and Gaussian head",
|
||||||
supports_gs=True,
|
supports_gs=True,
|
||||||
|
enabled=False,
|
||||||
),
|
),
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -70,8 +88,51 @@ BEN2_MODEL = ModelSpec(
|
|||||||
description="Fast high-quality background removal model",
|
description="Fast high-quality background removal model",
|
||||||
supports_gs=False,
|
supports_gs=False,
|
||||||
download_patterns=("config.json", "model.safetensors"),
|
download_patterns=("config.json", "model.safetensors"),
|
||||||
|
r2_directory="BEN2Base",
|
||||||
|
r2_files=(
|
||||||
|
("config.json", 124, "01ac876925577b52fa7c6af3f5b1d1420bc9b40016b6c25e2de89b38d8fff865"),
|
||||||
|
("model.safetensors", 380577976, "ea8b7907176a09667c86343dc7d00de6a6d871076cb90bb5f753618fd6fb3ebb"),
|
||||||
|
),
|
||||||
)
|
)
|
||||||
DOWNLOADABLE_MODELS = {**MODELS, BEN2_MODEL.key: BEN2_MODEL}
|
def moge2_model(key, label, directory_name, repository, description, r2_directory, size, checksum):
|
||||||
|
return ModelSpec(
|
||||||
|
key=key,
|
||||||
|
label=label,
|
||||||
|
directory_name=directory_name,
|
||||||
|
repository=repository,
|
||||||
|
license_name="MIT",
|
||||||
|
description=description,
|
||||||
|
supports_gs=False,
|
||||||
|
download_patterns=("model.pt",),
|
||||||
|
ready_patterns=("model.pt",),
|
||||||
|
r2_directory=r2_directory,
|
||||||
|
r2_files=(("model.pt", size, checksum),),
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
MOGE2_MODELS = {
|
||||||
|
"MOGE2_VITS_NORMAL": moge2_model(
|
||||||
|
"MOGE2_VITS_NORMAL", "MoGe-2 ViT-S Normal", "moge2_vits_normal",
|
||||||
|
"Ruicheng/moge-2-vits-normal", "Fastest MoGe-2 model", "MoGe2ViTSNormal",
|
||||||
|
140550416, "79a16621928c2bf0ed04659218c55c01075e950507f40bb3332fb4c873d3e1dc",
|
||||||
|
),
|
||||||
|
"MOGE2_VITB_NORMAL": moge2_model(
|
||||||
|
"MOGE2_VITB_NORMAL", "MoGe-2 ViT-B Normal", "moge2_vitb_normal",
|
||||||
|
"Ruicheng/moge-2-vitb-normal", "Balanced MoGe-2 model", "MoGe2ViTBNormal",
|
||||||
|
419110160, "16b8110e86d5dc5a849db120ca96ef3a223fd30b0c9146d1d81db504073da5f6",
|
||||||
|
),
|
||||||
|
"MOGE2_VITL_NORMAL": moge2_model(
|
||||||
|
"MOGE2_VITL_NORMAL", "MoGe-2 ViT-L Normal", "moge2_vitl_normal",
|
||||||
|
"Ruicheng/moge-2-vitl-normal", "Highest-quality MoGe-2 model", "MoGe2ViTLNormal",
|
||||||
|
1323815904, "280741fd09bc3f403ccff9967784c2a391b52d2c0742ae3efdb21d9f90cc1a01",
|
||||||
|
),
|
||||||
|
}
|
||||||
|
MOGE2_MODEL = MOGE2_MODELS["MOGE2_VITB_NORMAL"]
|
||||||
|
DOWNLOADABLE_MODELS = {
|
||||||
|
**MODELS,
|
||||||
|
BEN2_MODEL.key: BEN2_MODEL,
|
||||||
|
**MOGE2_MODELS,
|
||||||
|
}
|
||||||
MODEL_NUMBERS = {
|
MODEL_NUMBERS = {
|
||||||
model_key: number for number, model_key in enumerate(MODELS)
|
model_key: number for number, model_key in enumerate(MODELS)
|
||||||
}
|
}
|
||||||
@@ -88,7 +149,8 @@ def enum_items(supports_gs=None, show_download_status=False):
|
|||||||
models = [
|
models = [
|
||||||
model
|
model
|
||||||
for model in MODELS.values()
|
for model in MODELS.values()
|
||||||
if supports_gs is None or model.supports_gs == supports_gs
|
if model.enabled
|
||||||
|
and (supports_gs is None or model.supports_gs == supports_gs)
|
||||||
]
|
]
|
||||||
download_status = (
|
download_status = (
|
||||||
tuple(model_ready(model.key) for model in models)
|
tuple(model_ready(model.key) for model in models)
|
||||||
@@ -130,6 +192,42 @@ def gs_model_enum_items(_owner, _context):
|
|||||||
return enum_items(supports_gs=True, show_download_status=True)
|
return enum_items(supports_gs=True, show_download_status=True)
|
||||||
|
|
||||||
|
|
||||||
|
def moge2_model_enum_items(_owner, _context):
|
||||||
|
from .client import model_ready
|
||||||
|
|
||||||
|
return [
|
||||||
|
(
|
||||||
|
model.key,
|
||||||
|
model.label if model_ready(model.key) else f"{model.label} (Missing)",
|
||||||
|
f"{model.description}; license: {model.license_name}",
|
||||||
|
index,
|
||||||
|
)
|
||||||
|
for index, model in enumerate(MOGE2_MODELS.values())
|
||||||
|
]
|
||||||
|
|
||||||
|
|
||||||
|
def depth_plane_model_enum_items(_owner, _context):
|
||||||
|
from .client import model_ready
|
||||||
|
|
||||||
|
models = [
|
||||||
|
*MOGE2_MODELS.values(),
|
||||||
|
*(model for model in MODELS.values() if model.enabled),
|
||||||
|
]
|
||||||
|
items = []
|
||||||
|
for index, model in enumerate(models):
|
||||||
|
label = model.label if model_ready(model.key) else f"{model.label} (Missing)"
|
||||||
|
items.append(
|
||||||
|
(
|
||||||
|
model.key,
|
||||||
|
label,
|
||||||
|
f"{model.description}; license: {model.license_name}",
|
||||||
|
index,
|
||||||
|
)
|
||||||
|
)
|
||||||
|
items.append(("ORIGINAL", "Original", "Use the original image as depth", len(items)))
|
||||||
|
return items
|
||||||
|
|
||||||
|
|
||||||
def selected_model_key(settings, export_format=None):
|
def selected_model_key(settings, export_format=None):
|
||||||
selected_format = export_format or settings.export_format
|
selected_format = export_format or settings.export_format
|
||||||
if selected_format == "gs_ply":
|
if selected_format == "gs_ply":
|
||||||
|
|||||||
@@ -1,3 +1,4 @@
|
|||||||
|
import os
|
||||||
import shutil
|
import shutil
|
||||||
import subprocess
|
import subprocess
|
||||||
import threading
|
import threading
|
||||||
@@ -9,6 +10,7 @@ import bpy
|
|||||||
from . import client
|
from . import client
|
||||||
from .models import (
|
from .models import (
|
||||||
BEN2_MODEL,
|
BEN2_MODEL,
|
||||||
|
MOGE2_MODEL,
|
||||||
get_downloadable_model,
|
get_downloadable_model,
|
||||||
get_model,
|
get_model,
|
||||||
selected_model_key,
|
selected_model_key,
|
||||||
@@ -17,6 +19,8 @@ from .models import (
|
|||||||
|
|
||||||
RECONSTRUCTION_JOB = "RECONSTRUCT"
|
RECONSTRUCTION_JOB = "RECONSTRUCT"
|
||||||
IMAGE_DEPTH_JOB = "IMAGE_DEPTH"
|
IMAGE_DEPTH_JOB = "IMAGE_DEPTH"
|
||||||
|
MOGE2_DEPTH_JOB = "MOGE2_DEPTH"
|
||||||
|
MOGE2_DEBUG_JOB = "MOGE2_DEBUG"
|
||||||
IMAGE_MESH_JOB = "IMAGE_MESH"
|
IMAGE_MESH_JOB = "IMAGE_MESH"
|
||||||
BACKGROUND_REMOVAL_JOB = "REMOVE_BACKGROUND"
|
BACKGROUND_REMOVAL_JOB = "REMOVE_BACKGROUND"
|
||||||
IMAGE_BACKGROUND_JOB = "IMAGE_BACKGROUND"
|
IMAGE_BACKGROUND_JOB = "IMAGE_BACKGROUND"
|
||||||
@@ -26,6 +30,7 @@ DA3_IMAGE_JOBS = frozenset({IMAGE_DEPTH_JOB, IMAGE_MESH_JOB})
|
|||||||
IMAGE_EMPTY_JOBS = frozenset(
|
IMAGE_EMPTY_JOBS = frozenset(
|
||||||
{
|
{
|
||||||
*DA3_IMAGE_JOBS,
|
*DA3_IMAGE_JOBS,
|
||||||
|
MOGE2_DEPTH_JOB,
|
||||||
IMAGE_BACKGROUND_JOB,
|
IMAGE_BACKGROUND_JOB,
|
||||||
ORIGINAL_BACKGROUND_JOB,
|
ORIGINAL_BACKGROUND_JOB,
|
||||||
ORIGINAL_FRAMES_JOB,
|
ORIGINAL_FRAMES_JOB,
|
||||||
@@ -34,6 +39,7 @@ IMAGE_EMPTY_JOBS = frozenset(
|
|||||||
PRODUCTION_AI_JOBS = frozenset(
|
PRODUCTION_AI_JOBS = frozenset(
|
||||||
{
|
{
|
||||||
*DA3_IMAGE_JOBS,
|
*DA3_IMAGE_JOBS,
|
||||||
|
MOGE2_DEPTH_JOB,
|
||||||
IMAGE_BACKGROUND_JOB,
|
IMAGE_BACKGROUND_JOB,
|
||||||
ORIGINAL_BACKGROUND_JOB,
|
ORIGINAL_BACKGROUND_JOB,
|
||||||
}
|
}
|
||||||
@@ -156,6 +162,8 @@ class RunJob(bpy.types.Operator):
|
|||||||
("DOWNLOAD_MODEL", "Download Model", ""),
|
("DOWNLOAD_MODEL", "Download Model", ""),
|
||||||
(RECONSTRUCTION_JOB, "Export DA3 Result", ""),
|
(RECONSTRUCTION_JOB, "Export DA3 Result", ""),
|
||||||
(IMAGE_DEPTH_JOB, "Convert Image to Depth Plane", ""),
|
(IMAGE_DEPTH_JOB, "Convert Image to Depth Plane", ""),
|
||||||
|
(MOGE2_DEPTH_JOB, "Convert Image with MoGe-2", ""),
|
||||||
|
(MOGE2_DEBUG_JOB, "Test MoGe-2 API", ""),
|
||||||
(IMAGE_MESH_JOB, "Convert Image to Surface", ""),
|
(IMAGE_MESH_JOB, "Convert Image to Surface", ""),
|
||||||
(BACKGROUND_REMOVAL_JOB, "Test BEN2 Background Removal", ""),
|
(BACKGROUND_REMOVAL_JOB, "Test BEN2 Background Removal", ""),
|
||||||
(IMAGE_BACKGROUND_JOB, "Remove Image Empty Background", ""),
|
(IMAGE_BACKGROUND_JOB, "Remove Image Empty Background", ""),
|
||||||
@@ -210,6 +218,12 @@ class RunJob(bpy.types.Operator):
|
|||||||
default=False,
|
default=False,
|
||||||
options={"HIDDEN", "SKIP_SAVE"},
|
options={"HIDDEN", "SKIP_SAVE"},
|
||||||
)
|
)
|
||||||
|
snapform_moge_resolution_level: bpy.props.IntProperty(
|
||||||
|
default=5,
|
||||||
|
min=0,
|
||||||
|
max=9,
|
||||||
|
options={"HIDDEN", "SKIP_SAVE"},
|
||||||
|
)
|
||||||
|
|
||||||
def execute(self, context):
|
def execute(self, context):
|
||||||
global _active_job
|
global _active_job
|
||||||
@@ -236,6 +250,8 @@ class RunJob(bpy.types.Operator):
|
|||||||
export_format = "mini_npz"
|
export_format = "mini_npz"
|
||||||
elif self.job_type == IMAGE_MESH_JOB:
|
elif self.job_type == IMAGE_MESH_JOB:
|
||||||
export_format = "mesh_data"
|
export_format = "mesh_data"
|
||||||
|
elif self.job_type in {MOGE2_DEPTH_JOB, MOGE2_DEBUG_JOB}:
|
||||||
|
export_format = "moge_npz"
|
||||||
|
|
||||||
job = ActiveJob(
|
job = ActiveJob(
|
||||||
kind=self.job_type,
|
kind=self.job_type,
|
||||||
@@ -300,12 +316,20 @@ class RunJob(bpy.types.Operator):
|
|||||||
if job.kind == "INSTALL_ENV":
|
if job.kind == "INSTALL_ENV":
|
||||||
client.stop_service()
|
client.stop_service()
|
||||||
job.log_file = job.log_path.open("w", encoding="utf-8")
|
job.log_file = job.log_path.open("w", encoding="utf-8")
|
||||||
|
process_environment = None
|
||||||
|
if job.kind == "INSTALL_ENV":
|
||||||
|
process_environment = os.environ.copy()
|
||||||
|
process_environment["PYTHONIOENCODING"] = (
|
||||||
|
"utf-8:backslashreplace"
|
||||||
|
)
|
||||||
|
process_environment["PYTHONUTF8"] = "1"
|
||||||
job.process = subprocess.Popen(
|
job.process = subprocess.Popen(
|
||||||
command,
|
command,
|
||||||
stdout=job.log_file,
|
stdout=job.log_file,
|
||||||
stderr=subprocess.STDOUT,
|
stderr=subprocess.STDOUT,
|
||||||
cwd=working_directory,
|
cwd=working_directory,
|
||||||
creationflags=creation_flags,
|
creationflags=creation_flags,
|
||||||
|
env=process_environment,
|
||||||
)
|
)
|
||||||
job.started = True
|
job.started = True
|
||||||
if job.cancelled:
|
if job.cancelled:
|
||||||
@@ -401,6 +425,35 @@ class RunJob(bpy.types.Operator):
|
|||||||
),
|
),
|
||||||
paths,
|
paths,
|
||||||
)
|
)
|
||||||
|
if self.job_type in {MOGE2_DEPTH_JOB, MOGE2_DEBUG_JOB}:
|
||||||
|
model_key = self.snapform_model_key or MOGE2_MODEL.key
|
||||||
|
model = get_downloadable_model(model_key)
|
||||||
|
if not client.model_ready(model_key):
|
||||||
|
raise RuntimeError(f"Download {model.label} first")
|
||||||
|
if remove_background and not client.model_ready(BEN2_MODEL.key):
|
||||||
|
raise RuntimeError(f"Download {BEN2_MODEL.label} first")
|
||||||
|
return (
|
||||||
|
client.moge_request(
|
||||||
|
settings,
|
||||||
|
paths["status"],
|
||||||
|
paths["result"],
|
||||||
|
input_path=input_path,
|
||||||
|
resolution_level=(
|
||||||
|
self.snapform_moge_resolution_level
|
||||||
|
if self.job_type == MOGE2_DEPTH_JOB
|
||||||
|
else settings.moge_resolution_level
|
||||||
|
),
|
||||||
|
include_points=self.job_type == MOGE2_DEBUG_JOB,
|
||||||
|
remove_background=(
|
||||||
|
self.job_type == MOGE2_DEPTH_JOB
|
||||||
|
and remove_background
|
||||||
|
),
|
||||||
|
start_frame=start_frame,
|
||||||
|
device=self._task_device(settings),
|
||||||
|
model_key=model_key,
|
||||||
|
),
|
||||||
|
paths,
|
||||||
|
)
|
||||||
if self.job_type not in {
|
if self.job_type not in {
|
||||||
RECONSTRUCTION_JOB,
|
RECONSTRUCTION_JOB,
|
||||||
IMAGE_DEPTH_JOB,
|
IMAGE_DEPTH_JOB,
|
||||||
@@ -478,6 +531,8 @@ class RunJob(bpy.types.Operator):
|
|||||||
return f"Starting {settings.export_format} export"
|
return f"Starting {settings.export_format} export"
|
||||||
if self.job_type == IMAGE_DEPTH_JOB:
|
if self.job_type == IMAGE_DEPTH_JOB:
|
||||||
return "Starting depth estimation"
|
return "Starting depth estimation"
|
||||||
|
if self.job_type in {MOGE2_DEPTH_JOB, MOGE2_DEBUG_JOB}:
|
||||||
|
return "Starting MoGe-2 inference"
|
||||||
if self.job_type == IMAGE_MESH_JOB:
|
if self.job_type == IMAGE_MESH_JOB:
|
||||||
return "Starting perspective surface generation"
|
return "Starting perspective surface generation"
|
||||||
if self.job_type == BACKGROUND_REMOVAL_JOB:
|
if self.job_type == BACKGROUND_REMOVAL_JOB:
|
||||||
@@ -626,6 +681,7 @@ class RunJob(bpy.types.Operator):
|
|||||||
if job.kind in {
|
if job.kind in {
|
||||||
RECONSTRUCTION_JOB,
|
RECONSTRUCTION_JOB,
|
||||||
BACKGROUND_REMOVAL_JOB,
|
BACKGROUND_REMOVAL_JOB,
|
||||||
|
MOGE2_DEBUG_JOB,
|
||||||
}:
|
}:
|
||||||
try:
|
try:
|
||||||
open_job_directory(job.result_directory)
|
open_job_directory(job.result_directory)
|
||||||
@@ -635,8 +691,10 @@ class RunJob(bpy.types.Operator):
|
|||||||
message = (
|
message = (
|
||||||
f"Exported {job.export_format} to {job.result_directory}"
|
f"Exported {job.export_format} to {job.result_directory}"
|
||||||
)
|
)
|
||||||
else:
|
elif job.kind == BACKGROUND_REMOVAL_JOB:
|
||||||
message = f"Removed background to {job.result_directory}"
|
message = f"Removed background to {job.result_directory}"
|
||||||
|
else:
|
||||||
|
message = f"Exported MoGe-2 data to {job.result_directory}"
|
||||||
self.report({"INFO"}, message)
|
self.report({"INFO"}, message)
|
||||||
if job.kind == IMAGE_DEPTH_JOB:
|
if job.kind == IMAGE_DEPTH_JOB:
|
||||||
try:
|
try:
|
||||||
@@ -653,6 +711,23 @@ class RunJob(bpy.types.Operator):
|
|||||||
self._redraw_ui(context, force=True)
|
self._redraw_ui(context, force=True)
|
||||||
return {"CANCELLED"}
|
return {"CANCELLED"}
|
||||||
status["message"] = "Depth plane is ready"
|
status["message"] = "Depth plane is ready"
|
||||||
|
if job.kind == MOGE2_DEPTH_JOB:
|
||||||
|
try:
|
||||||
|
settings.status = "Building MoGe-2 depth plane"
|
||||||
|
settings.progress = 0.96
|
||||||
|
self._redraw_ui(context, force=True)
|
||||||
|
self._build_moge2_depth_plane(job)
|
||||||
|
except (OSError, RuntimeError, ValueError) as import_error:
|
||||||
|
settings.status = "MoGe-2 depth import failed"
|
||||||
|
settings.progress = 0.0
|
||||||
|
self.report({"ERROR"}, str(import_error))
|
||||||
|
client.mark_service_job_complete(
|
||||||
|
client.job_id_from_status(job.status_path)
|
||||||
|
)
|
||||||
|
close_active_job()
|
||||||
|
self._redraw_ui(context, force=True)
|
||||||
|
return {"CANCELLED"}
|
||||||
|
status["message"] = "MoGe-2 depth plane is ready"
|
||||||
if job.kind == IMAGE_MESH_JOB:
|
if job.kind == IMAGE_MESH_JOB:
|
||||||
try:
|
try:
|
||||||
settings.status = "Building perspective surface"
|
settings.status = "Building perspective surface"
|
||||||
@@ -790,6 +865,35 @@ class RunJob(bpy.types.Operator):
|
|||||||
if "FINISHED" not in undo_result:
|
if "FINISHED" not in undo_result:
|
||||||
raise RuntimeError("Unable to create the depth plane Undo step")
|
raise RuntimeError("Unable to create the depth plane Undo step")
|
||||||
|
|
||||||
|
def _build_moge2_depth_plane(self, job):
|
||||||
|
result_directory = job.result_directory
|
||||||
|
is_sequence = bool(list(result_directory.glob("frame.*")))
|
||||||
|
foreground_sequence = bool(
|
||||||
|
list(result_directory.glob("foreground.*.png"))
|
||||||
|
)
|
||||||
|
result = bpy.ops.snapform.build_moge2_depth_plane(
|
||||||
|
snapform_source_object_name=job.source_object_name,
|
||||||
|
snapform_npz_path=str(result_directory / "results.npz"),
|
||||||
|
snapform_background_path=(
|
||||||
|
str(result_directory)
|
||||||
|
if job.remove_background and foreground_sequence
|
||||||
|
else str(result_directory / "foreground.png")
|
||||||
|
if job.remove_background
|
||||||
|
else ""
|
||||||
|
),
|
||||||
|
snapform_color_frames_dir=(
|
||||||
|
str(result_directory) if is_sequence else ""
|
||||||
|
),
|
||||||
|
snapform_data_frames_dir=str(result_directory),
|
||||||
|
snapform_delete_source=job.delete_source,
|
||||||
|
snapform_unshaded=job.unshaded,
|
||||||
|
)
|
||||||
|
if "FINISHED" not in result:
|
||||||
|
raise RuntimeError("Unable to build the MoGe-2 depth plane")
|
||||||
|
undo_result = bpy.ops.ed.undo_push(message="Convert to Depth Plane")
|
||||||
|
if "FINISHED" not in undo_result:
|
||||||
|
raise RuntimeError("Unable to create the depth plane Undo step")
|
||||||
|
|
||||||
def _apply_removed_background(self, job, push_undo=True):
|
def _apply_removed_background(self, job, push_undo=True):
|
||||||
result_directory = job.result_directory
|
result_directory = job.result_directory
|
||||||
foreground_paths = sorted(result_directory.glob("foreground.*.png"))
|
foreground_paths = sorted(result_directory.glob("foreground.*.png"))
|
||||||
|
|||||||
@@ -1,6 +1,6 @@
|
|||||||
import bpy
|
import bpy
|
||||||
|
|
||||||
from .models import BEN2_MODEL, MODELS, selected_model_key
|
from .models import BEN2_MODEL, MODELS, MOGE2_MODELS, selected_model_key
|
||||||
from .operators import active_job
|
from .operators import active_job
|
||||||
from . import client
|
from . import client
|
||||||
from .client import model_ready, env_installed
|
from .client import model_ready, env_installed
|
||||||
@@ -126,6 +126,9 @@ class ModelsPanel(bpy.types.Panel):
|
|||||||
running = active_job() is not None or client.service_busy()
|
running = active_job() is not None or client.service_busy()
|
||||||
|
|
||||||
for model in MODELS.values():
|
for model in MODELS.values():
|
||||||
|
if model.enabled:
|
||||||
|
self._draw_model(layout, model, running)
|
||||||
|
for model in MOGE2_MODELS.values():
|
||||||
self._draw_model(layout, model, running)
|
self._draw_model(layout, model, running)
|
||||||
self._draw_model(layout, BEN2_MODEL, running)
|
self._draw_model(layout, BEN2_MODEL, running)
|
||||||
|
|
||||||
@@ -236,6 +239,36 @@ class Da3OutputDebugPanel(bpy.types.Panel):
|
|||||||
operator.job_type = "RECONSTRUCT"
|
operator.job_type = "RECONSTRUCT"
|
||||||
|
|
||||||
|
|
||||||
|
class Moge2ApiDebugPanel(bpy.types.Panel):
|
||||||
|
bl_label = "MoGe-2 API Test"
|
||||||
|
bl_idname = "SNAPFORM_PT_debug_moge2_api"
|
||||||
|
bl_parent_id = DebugPanel.bl_idname
|
||||||
|
bl_space_type = "VIEW_3D"
|
||||||
|
bl_region_type = "UI"
|
||||||
|
bl_category = "SnapForm"
|
||||||
|
bl_options = {"DEFAULT_CLOSED"}
|
||||||
|
bl_order = 1
|
||||||
|
|
||||||
|
def draw(self, context):
|
||||||
|
layout = self.layout
|
||||||
|
settings = context.scene.snapform_settings
|
||||||
|
layout.prop(settings, "input_path")
|
||||||
|
layout.prop(settings, "moge_resolution_level")
|
||||||
|
layout.prop(settings, "moge_model")
|
||||||
|
run_row = layout.row()
|
||||||
|
run_row.enabled = (
|
||||||
|
active_job() is None
|
||||||
|
and not client.service_busy()
|
||||||
|
and model_ready(settings.moge_model)
|
||||||
|
)
|
||||||
|
operator = run_row.operator(
|
||||||
|
"snapform.run_job",
|
||||||
|
text="Generate MoGe-2 Output",
|
||||||
|
icon="EXPORT",
|
||||||
|
)
|
||||||
|
operator.job_type = "MOGE2_DEBUG"
|
||||||
|
|
||||||
|
|
||||||
class Ben2RemovalDebugPanel(bpy.types.Panel):
|
class Ben2RemovalDebugPanel(bpy.types.Panel):
|
||||||
bl_label = "BEN2 Removal Test"
|
bl_label = "BEN2 Removal Test"
|
||||||
bl_idname = "SNAPFORM_PT_debug_ben2_removal"
|
bl_idname = "SNAPFORM_PT_debug_ben2_removal"
|
||||||
|
|||||||
@@ -1,6 +1,6 @@
|
|||||||
import bpy
|
import bpy
|
||||||
|
|
||||||
from .models import gs_model_enum_items, model_enum_items
|
from .models import gs_model_enum_items, model_enum_items, moge2_model_enum_items
|
||||||
from .client import device_enum_items
|
from .client import device_enum_items
|
||||||
|
|
||||||
|
|
||||||
@@ -65,6 +65,19 @@ class SnapFormSettings(bpy.types.PropertyGroup):
|
|||||||
),
|
),
|
||||||
default="upper_bound_resize",
|
default="upper_bound_resize",
|
||||||
)
|
)
|
||||||
|
moge_resolution_level: bpy.props.IntProperty(
|
||||||
|
name="Resolution Level",
|
||||||
|
description="MoGe-2 inference resolution level; higher is finer and slower",
|
||||||
|
default=5,
|
||||||
|
min=0,
|
||||||
|
max=9,
|
||||||
|
)
|
||||||
|
moge_model: bpy.props.EnumProperty(
|
||||||
|
name="Model",
|
||||||
|
description="MoGe-2 model used for depth estimation",
|
||||||
|
items=moge2_model_enum_items,
|
||||||
|
default=1,
|
||||||
|
)
|
||||||
conf_thresh_percentile: bpy.props.FloatProperty(
|
conf_thresh_percentile: bpy.props.FloatProperty(
|
||||||
name="Confidence Threshold",
|
name="Confidence Threshold",
|
||||||
description="Remove the lowest-confidence percentile of points or surface faces",
|
description="Remove the lowest-confidence percentile of points or surface faces",
|
||||||
|
|||||||
@@ -20,6 +20,9 @@ import worker
|
|||||||
|
|
||||||
|
|
||||||
CLIENT_TIMEOUT_SECONDS = 10.0
|
CLIENT_TIMEOUT_SECONDS = 10.0
|
||||||
|
JOB_COMMANDS = frozenset(
|
||||||
|
{"remove-background", "reconstruct", "extract-frames", "moge"}
|
||||||
|
)
|
||||||
IDLE_SHUTDOWN_SECONDS = 20.0
|
IDLE_SHUTDOWN_SECONDS = 20.0
|
||||||
|
|
||||||
|
|
||||||
@@ -78,6 +81,8 @@ class ModelCache:
|
|||||||
self.ben2_key = None
|
self.ben2_key = None
|
||||||
self.da3_model = None
|
self.da3_model = None
|
||||||
self.da3_key = None
|
self.da3_key = None
|
||||||
|
self.moge_model = None
|
||||||
|
self.moge_key = None
|
||||||
|
|
||||||
def get_ben2(self, model_dir, device):
|
def get_ben2(self, model_dir, device):
|
||||||
key = (str(Path(model_dir).resolve()), device.type)
|
key = (str(Path(model_dir).resolve()), device.type)
|
||||||
@@ -88,6 +93,7 @@ class ModelCache:
|
|||||||
return self._load_ben2(model_dir, device, key)
|
return self._load_ben2(model_dir, device, key)
|
||||||
except self.torch.cuda.OutOfMemoryError:
|
except self.torch.cuda.OutOfMemoryError:
|
||||||
self.release_da3()
|
self.release_da3()
|
||||||
|
self.release_moge()
|
||||||
return self._load_ben2(model_dir, device, key)
|
return self._load_ben2(model_dir, device, key)
|
||||||
|
|
||||||
def _load_ben2(self, model_dir, device, key):
|
def _load_ben2(self, model_dir, device, key):
|
||||||
@@ -111,6 +117,7 @@ class ModelCache:
|
|||||||
return self._load_da3(model_dir, device, key)
|
return self._load_da3(model_dir, device, key)
|
||||||
except self.torch.cuda.OutOfMemoryError:
|
except self.torch.cuda.OutOfMemoryError:
|
||||||
self.release_ben2()
|
self.release_ben2()
|
||||||
|
self.release_moge()
|
||||||
return self._load_da3(model_dir, device, key)
|
return self._load_da3(model_dir, device, key)
|
||||||
|
|
||||||
def _load_da3(self, model_dir, device, key):
|
def _load_da3(self, model_dir, device, key):
|
||||||
@@ -123,6 +130,30 @@ class ModelCache:
|
|||||||
self.da3_key = key
|
self.da3_key = key
|
||||||
return model
|
return model
|
||||||
|
|
||||||
|
def get_moge(self, model_dir, device):
|
||||||
|
key = (str(Path(model_dir).resolve()), device.type)
|
||||||
|
if self.moge_model is not None and self.moge_key == key:
|
||||||
|
return self.moge_model
|
||||||
|
self.release_moge()
|
||||||
|
try:
|
||||||
|
return self._load_moge(model_dir, device, key)
|
||||||
|
except self.torch.cuda.OutOfMemoryError:
|
||||||
|
self.release_da3()
|
||||||
|
self.release_ben2()
|
||||||
|
return self._load_moge(model_dir, device, key)
|
||||||
|
|
||||||
|
def _load_moge(self, model_dir, device, key):
|
||||||
|
worker.prepare_moge_runtime()
|
||||||
|
from moge.model.v2 import MoGeModel
|
||||||
|
|
||||||
|
checkpoint = Path(model_dir) / "model.pt"
|
||||||
|
model = MoGeModel.from_pretrained(checkpoint)
|
||||||
|
model = model.to(device=device)
|
||||||
|
model.eval()
|
||||||
|
self.moge_model = model
|
||||||
|
self.moge_key = key
|
||||||
|
return model
|
||||||
|
|
||||||
def release_ben2(self):
|
def release_ben2(self):
|
||||||
self.ben2_model = None
|
self.ben2_model = None
|
||||||
self.ben2_key = None
|
self.ben2_key = None
|
||||||
@@ -133,11 +164,18 @@ class ModelCache:
|
|||||||
self.da3_key = None
|
self.da3_key = None
|
||||||
self._release_cuda_cache()
|
self._release_cuda_cache()
|
||||||
|
|
||||||
|
def release_moge(self):
|
||||||
|
self.moge_model = None
|
||||||
|
self.moge_key = None
|
||||||
|
self._release_cuda_cache()
|
||||||
|
|
||||||
def close(self):
|
def close(self):
|
||||||
self.ben2_model = None
|
self.ben2_model = None
|
||||||
self.da3_model = None
|
self.da3_model = None
|
||||||
|
self.moge_model = None
|
||||||
self.ben2_key = None
|
self.ben2_key = None
|
||||||
self.da3_key = None
|
self.da3_key = None
|
||||||
|
self.moge_key = None
|
||||||
self._release_cuda_cache()
|
self._release_cuda_cache()
|
||||||
|
|
||||||
def status(self):
|
def status(self):
|
||||||
@@ -150,6 +188,11 @@ class ModelCache:
|
|||||||
if self.da3_key is not None
|
if self.da3_key is not None
|
||||||
else ""
|
else ""
|
||||||
),
|
),
|
||||||
|
"moge2": (
|
||||||
|
Path(getattr(self, "moge_key", ("",))[0]).name
|
||||||
|
if getattr(self, "moge_key", None) is not None
|
||||||
|
else ""
|
||||||
|
),
|
||||||
}
|
}
|
||||||
|
|
||||||
def _release_cuda_cache(self):
|
def _release_cuda_cache(self):
|
||||||
@@ -281,6 +324,12 @@ class ServiceState:
|
|||||||
args,
|
args,
|
||||||
cancel_check=cancel_check,
|
cancel_check=cancel_check,
|
||||||
)
|
)
|
||||||
|
elif payload["command"] == "moge":
|
||||||
|
worker.run_moge(
|
||||||
|
args,
|
||||||
|
model_cache=self.models,
|
||||||
|
cancel_check=cancel_check,
|
||||||
|
)
|
||||||
else:
|
else:
|
||||||
worker.reconstruct(
|
worker.reconstruct(
|
||||||
args,
|
args,
|
||||||
@@ -429,11 +478,7 @@ def create_app(token, instance_id="test"):
|
|||||||
client_id = str(payload.get("client_id", ""))
|
client_id = str(payload.get("client_id", ""))
|
||||||
command = payload.get("command")
|
command = payload.get("command")
|
||||||
parameters = payload.get("parameters")
|
parameters = payload.get("parameters")
|
||||||
if not job_id or command not in {
|
if not job_id or command not in JOB_COMMANDS:
|
||||||
"remove-background",
|
|
||||||
"reconstruct",
|
|
||||||
"extract-frames",
|
|
||||||
}:
|
|
||||||
raise HTTPException(status_code=422, detail="Invalid job request")
|
raise HTTPException(status_code=422, detail="Invalid job request")
|
||||||
if not isinstance(parameters, dict) or not parameters.get("status"):
|
if not isinstance(parameters, dict) or not parameters.get("status"):
|
||||||
raise HTTPException(status_code=422, detail="Missing job parameters")
|
raise HTTPException(status_code=422, detail="Missing job parameters")
|
||||||
|
|||||||
@@ -1,5 +1,5 @@
|
|||||||
import argparse
|
import argparse
|
||||||
import fnmatch
|
import hashlib
|
||||||
import json
|
import json
|
||||||
import os
|
import os
|
||||||
import shutil
|
import shutil
|
||||||
@@ -7,7 +7,7 @@ import tempfile
|
|||||||
import threading
|
import threading
|
||||||
import time
|
import time
|
||||||
from pathlib import Path
|
from pathlib import Path
|
||||||
from urllib.parse import quote, urlencode
|
from urllib.parse import quote
|
||||||
from urllib.request import Request, urlopen
|
from urllib.request import Request, urlopen
|
||||||
|
|
||||||
|
|
||||||
@@ -33,9 +33,8 @@ EXPORT_FORMATS = (
|
|||||||
"mesh_data",
|
"mesh_data",
|
||||||
)
|
)
|
||||||
HF_ENDPOINT = "https://huggingface.co"
|
HF_ENDPOINT = "https://huggingface.co"
|
||||||
MODELSCOPE_ENDPOINT = "https://modelscope.cn"
|
R2_ENDPOINT = "https://models.omoolab.xyz"
|
||||||
MODELSCOPE_REVISION = "master"
|
DOWNLOAD_TIMEOUT = 60.0
|
||||||
MODELSCOPE_TIMEOUT = 60.0
|
|
||||||
|
|
||||||
|
|
||||||
class TaskCancelled(RuntimeError):
|
class TaskCancelled(RuntimeError):
|
||||||
@@ -925,8 +924,14 @@ def download_model(args):
|
|||||||
"stages": getattr(args, "stages", None),
|
"stages": getattr(args, "stages", None),
|
||||||
"stage_label": getattr(args, "stage_label", None),
|
"stage_label": getattr(args, "stage_label", None),
|
||||||
}
|
}
|
||||||
|
failures = []
|
||||||
|
try:
|
||||||
from huggingface_hub import snapshot_download
|
from huggingface_hub import snapshot_download
|
||||||
|
except Exception as error:
|
||||||
|
snapshot_download = None
|
||||||
|
failures.append((HF_ENDPOINT, error))
|
||||||
|
progress_class = None
|
||||||
|
if snapshot_download:
|
||||||
progress_class = create_download_progress_class(
|
progress_class = create_download_progress_class(
|
||||||
args.status,
|
args.status,
|
||||||
args.repository,
|
args.repository,
|
||||||
@@ -935,11 +940,7 @@ def download_model(args):
|
|||||||
label=label,
|
label=label,
|
||||||
**stage_kwargs,
|
**stage_kwargs,
|
||||||
)
|
)
|
||||||
failures = []
|
for endpoint in download_endpoints() if snapshot_download else ():
|
||||||
for endpoint in download_endpoints():
|
|
||||||
if not hf_endpoint_reachable(endpoint):
|
|
||||||
failures.append((endpoint, "endpoint unreachable"))
|
|
||||||
continue
|
|
||||||
os.environ["HF_ENDPOINT"] = endpoint
|
os.environ["HF_ENDPOINT"] = endpoint
|
||||||
message = f"Downloading {label}"
|
message = f"Downloading {label}"
|
||||||
if endpoint != HF_ENDPOINT:
|
if endpoint != HF_ENDPOINT:
|
||||||
@@ -953,7 +954,10 @@ def download_model(args):
|
|||||||
allow_patterns=args.allow_pattern or None,
|
allow_patterns=args.allow_pattern or None,
|
||||||
endpoint=endpoint,
|
endpoint=endpoint,
|
||||||
)
|
)
|
||||||
_verify_model_complete(model_dir)
|
_verify_model_complete(
|
||||||
|
model_dir,
|
||||||
|
getattr(args, "required_pattern", None),
|
||||||
|
)
|
||||||
except Exception as error:
|
except Exception as error:
|
||||||
failures.append((endpoint, error))
|
failures.append((endpoint, error))
|
||||||
continue
|
continue
|
||||||
@@ -964,15 +968,22 @@ def download_model(args):
|
|||||||
**stage_kwargs,
|
**stage_kwargs,
|
||||||
)
|
)
|
||||||
return
|
return
|
||||||
# Hugging Face 官方源与已配置镜像均失败时,回退 ModelScope(仓库名与 Hugging Face 一致)
|
# Hugging Face 失败后使用由项目维护的静态模型镜像。
|
||||||
|
r2_directory = getattr(args, "r2_directory", "")
|
||||||
|
r2_files = getattr(args, "r2_file", [])
|
||||||
|
if not r2_directory or not r2_files:
|
||||||
|
detail = "; ".join(
|
||||||
|
f"{source}: {error}" for source, error in failures
|
||||||
|
)
|
||||||
|
raise RuntimeError(f"All download sources failed: {detail}")
|
||||||
try:
|
try:
|
||||||
write_status(
|
write_status(
|
||||||
args.status,
|
args.status,
|
||||||
start,
|
start,
|
||||||
f"Downloading {label} | ModelScope mirror",
|
f"Downloading {label} | OmooLab mirror",
|
||||||
**stage_kwargs,
|
**stage_kwargs,
|
||||||
)
|
)
|
||||||
download_from_modelscope(
|
download_from_r2(
|
||||||
args,
|
args,
|
||||||
model_dir,
|
model_dir,
|
||||||
start=start,
|
start=start,
|
||||||
@@ -980,9 +991,12 @@ def download_model(args):
|
|||||||
label=label,
|
label=label,
|
||||||
stage_kwargs=stage_kwargs,
|
stage_kwargs=stage_kwargs,
|
||||||
)
|
)
|
||||||
_verify_model_complete(model_dir)
|
_verify_model_complete(
|
||||||
|
model_dir,
|
||||||
|
getattr(args, "required_pattern", None),
|
||||||
|
)
|
||||||
except Exception as error:
|
except Exception as error:
|
||||||
failures.append((MODELSCOPE_ENDPOINT, error))
|
failures.append((R2_ENDPOINT, error))
|
||||||
else:
|
else:
|
||||||
write_status(
|
write_status(
|
||||||
args.status,
|
args.status,
|
||||||
@@ -998,33 +1012,23 @@ def download_model(args):
|
|||||||
|
|
||||||
|
|
||||||
def download_endpoints():
|
def download_endpoints():
|
||||||
"""Hugging Face download candidates: configured mirror first, official endpoint otherwise."""
|
"""Return the configured Hugging Face endpoint or official source."""
|
||||||
endpoint = os.environ.get("HF_ENDPOINT") or HF_ENDPOINT
|
endpoint = os.environ.get("HF_ENDPOINT") or HF_ENDPOINT
|
||||||
return [endpoint.rstrip("/")]
|
return [endpoint.rstrip("/")]
|
||||||
|
|
||||||
|
|
||||||
def hf_endpoint_reachable(endpoint, timeout=10.0):
|
def _verify_model_complete(model_dir, required_patterns=None):
|
||||||
"""Quick connectivity probe so slow or blocked Hugging Face endpoints fail fast."""
|
|
||||||
try:
|
|
||||||
request = Request(
|
|
||||||
f"{endpoint.rstrip('/')}/api/models",
|
|
||||||
method="HEAD",
|
|
||||||
headers={"User-Agent": "SnapForm"},
|
|
||||||
)
|
|
||||||
with urlopen(request, timeout=timeout):
|
|
||||||
return True
|
|
||||||
except Exception:
|
|
||||||
return False
|
|
||||||
|
|
||||||
|
|
||||||
def _verify_model_complete(model_dir):
|
|
||||||
"""Refuse to report success when the required model files are missing."""
|
"""Refuse to report success when the required model files are missing."""
|
||||||
model_dir = Path(model_dir)
|
model_dir = Path(model_dir)
|
||||||
missing = []
|
required_patterns = required_patterns or (
|
||||||
if not (model_dir / "config.json").is_file():
|
"config.json",
|
||||||
missing.append("config.json")
|
"*.safetensors",
|
||||||
if not any(model_dir.glob("*.safetensors")):
|
)
|
||||||
missing.append("*.safetensors")
|
missing = [
|
||||||
|
pattern
|
||||||
|
for pattern in required_patterns
|
||||||
|
if not any(model_dir.glob(pattern))
|
||||||
|
]
|
||||||
if missing:
|
if missing:
|
||||||
raise RuntimeError(
|
raise RuntimeError(
|
||||||
"Model download reported success but files are missing: "
|
"Model download reported success but files are missing: "
|
||||||
@@ -1032,59 +1036,42 @@ def _verify_model_complete(model_dir):
|
|||||||
)
|
)
|
||||||
|
|
||||||
|
|
||||||
def _hidden_repo_path(path):
|
def _download_r2_file(
|
||||||
return any(part.startswith(".") for part in Path(path).parts)
|
directory,
|
||||||
|
filename,
|
||||||
|
|
||||||
def _model_file_allowed(path, patterns):
|
|
||||||
if not patterns:
|
|
||||||
return True
|
|
||||||
return any(fnmatch.fnmatch(path, pattern) for pattern in patterns)
|
|
||||||
|
|
||||||
|
|
||||||
def modelscope_file_list(repository, revision=MODELSCOPE_REVISION):
|
|
||||||
"""List (path, size) file entries of a ModelScope repository."""
|
|
||||||
url = (
|
|
||||||
f"{MODELSCOPE_ENDPOINT}/api/v1/models/{repository}/repo/files?"
|
|
||||||
+ urlencode({"Revision": revision, "Recursive": "true"})
|
|
||||||
)
|
|
||||||
request = Request(url, headers={"User-Agent": "SnapForm"})
|
|
||||||
with urlopen(request, timeout=MODELSCOPE_TIMEOUT) as response:
|
|
||||||
payload = json.loads(response.read().decode("utf-8"))
|
|
||||||
entries = []
|
|
||||||
for entry in payload.get("Data", {}).get("Files", []):
|
|
||||||
path = entry.get("Path")
|
|
||||||
if path and entry.get("Type") == "blob" and not _hidden_repo_path(path):
|
|
||||||
entries.append((path, int(entry.get("Size") or 0)))
|
|
||||||
return entries
|
|
||||||
|
|
||||||
|
|
||||||
def _download_modelscope_file(
|
|
||||||
repository,
|
|
||||||
path,
|
|
||||||
destination,
|
destination,
|
||||||
|
expected_size,
|
||||||
|
expected_checksum,
|
||||||
progress=None,
|
progress=None,
|
||||||
retries=2,
|
retries=2,
|
||||||
):
|
):
|
||||||
"""Download one ModelScope file atomically with a per-chunk progress callback."""
|
"""Download and verify one OmooLab R2 model file atomically."""
|
||||||
url = (
|
url = f"{R2_ENDPOINT}/{quote(directory)}/{quote(filename)}"
|
||||||
f"{MODELSCOPE_ENDPOINT}/models/{repository}/resolve/"
|
|
||||||
f"{MODELSCOPE_REVISION}/{quote(path)}"
|
|
||||||
)
|
|
||||||
request = Request(url, headers={"User-Agent": "SnapForm"})
|
request = Request(url, headers={"User-Agent": "SnapForm"})
|
||||||
last_error = None
|
last_error = None
|
||||||
for _attempt in range(retries):
|
for _attempt in range(retries):
|
||||||
temporary = destination.with_name(f"{destination.name}.part")
|
temporary = destination.with_name(f"{destination.name}.part")
|
||||||
try:
|
try:
|
||||||
with urlopen(request, timeout=MODELSCOPE_TIMEOUT) as response:
|
checksum = hashlib.sha256()
|
||||||
|
downloaded = 0
|
||||||
|
with urlopen(request, timeout=DOWNLOAD_TIMEOUT) as response:
|
||||||
with open(temporary, "wb") as output:
|
with open(temporary, "wb") as output:
|
||||||
while True:
|
while True:
|
||||||
chunk = response.read(1024 * 1024)
|
chunk = response.read(1024 * 1024)
|
||||||
if not chunk:
|
if not chunk:
|
||||||
break
|
break
|
||||||
output.write(chunk)
|
output.write(chunk)
|
||||||
|
checksum.update(chunk)
|
||||||
|
downloaded += len(chunk)
|
||||||
if progress is not None:
|
if progress is not None:
|
||||||
progress(len(chunk))
|
progress(len(chunk))
|
||||||
|
if downloaded != expected_size:
|
||||||
|
raise RuntimeError(
|
||||||
|
f"Size mismatch for {filename}: "
|
||||||
|
f"{downloaded} != {expected_size}"
|
||||||
|
)
|
||||||
|
if checksum.hexdigest().lower() != expected_checksum.lower():
|
||||||
|
raise RuntimeError(f"SHA-256 mismatch for {filename}")
|
||||||
temporary.replace(destination)
|
temporary.replace(destination)
|
||||||
return
|
return
|
||||||
except Exception as error:
|
except Exception as error:
|
||||||
@@ -1096,7 +1083,7 @@ def _download_modelscope_file(
|
|||||||
raise last_error
|
raise last_error
|
||||||
|
|
||||||
|
|
||||||
def download_from_modelscope(
|
def download_from_r2(
|
||||||
args,
|
args,
|
||||||
model_dir,
|
model_dir,
|
||||||
start=0.05,
|
start=0.05,
|
||||||
@@ -1104,19 +1091,12 @@ def download_from_modelscope(
|
|||||||
label=None,
|
label=None,
|
||||||
stage_kwargs=None,
|
stage_kwargs=None,
|
||||||
):
|
):
|
||||||
"""Download a model from ModelScope using only standard library requests."""
|
"""Download the declared model files from OmooLab R2."""
|
||||||
repository = args.repository
|
repository = args.repository
|
||||||
files = modelscope_file_list(repository)
|
selected = []
|
||||||
patterns = args.allow_pattern or None
|
for value in args.r2_file:
|
||||||
selected = [
|
filename, size, checksum = value.split("|", 2)
|
||||||
(path, size)
|
selected.append((filename, int(size), checksum))
|
||||||
for path, size in files
|
|
||||||
if _model_file_allowed(path, patterns)
|
|
||||||
]
|
|
||||||
if not selected:
|
|
||||||
raise RuntimeError(
|
|
||||||
f"No matching files in ModelScope repository: {repository}"
|
|
||||||
)
|
|
||||||
reporter = DownloadProgressReporter(
|
reporter = DownloadProgressReporter(
|
||||||
args.status,
|
args.status,
|
||||||
repository,
|
repository,
|
||||||
@@ -1125,7 +1105,7 @@ def download_from_modelscope(
|
|||||||
label=label or repository,
|
label=label or repository,
|
||||||
**(stage_kwargs or {}),
|
**(stage_kwargs or {}),
|
||||||
)
|
)
|
||||||
total_size = sum(size for _path, size in selected)
|
total_size = sum(size for _path, size, _checksum in selected)
|
||||||
downloaded = 0
|
downloaded = 0
|
||||||
|
|
||||||
def on_chunk(count):
|
def on_chunk(count):
|
||||||
@@ -1133,13 +1113,15 @@ def download_from_modelscope(
|
|||||||
downloaded += count
|
downloaded += count
|
||||||
reporter.report(downloaded, total_size)
|
reporter.report(downloaded, total_size)
|
||||||
|
|
||||||
for path, _size in selected:
|
for path, size, checksum in selected:
|
||||||
destination = Path(model_dir) / path
|
destination = Path(model_dir) / path
|
||||||
destination.parent.mkdir(parents=True, exist_ok=True)
|
destination.parent.mkdir(parents=True, exist_ok=True)
|
||||||
_download_modelscope_file(
|
_download_r2_file(
|
||||||
repository,
|
args.r2_directory,
|
||||||
path,
|
path,
|
||||||
destination,
|
destination,
|
||||||
|
size,
|
||||||
|
checksum,
|
||||||
progress=on_chunk,
|
progress=on_chunk,
|
||||||
)
|
)
|
||||||
|
|
||||||
@@ -1500,6 +1482,168 @@ def flatten_exported_paths(
|
|||||||
return flattened_paths
|
return flattened_paths
|
||||||
|
|
||||||
|
|
||||||
|
def _save_moge_preview(output_directory, index, depth, normal, mask):
|
||||||
|
import numpy as np
|
||||||
|
from PIL import Image
|
||||||
|
|
||||||
|
suffix = "" if index is None else f".{index + 1:04d}"
|
||||||
|
valid = mask & np.isfinite(depth) & (depth > 0)
|
||||||
|
preview = np.zeros(depth.shape, dtype=np.uint8)
|
||||||
|
if np.any(valid):
|
||||||
|
low, high = np.percentile(depth[valid], (2.0, 98.0))
|
||||||
|
span = max(float(high - low), 1e-6)
|
||||||
|
preview[valid] = np.clip(
|
||||||
|
(depth[valid] - low) / span * 255.0,
|
||||||
|
0.0,
|
||||||
|
255.0,
|
||||||
|
).astype(np.uint8)
|
||||||
|
Image.fromarray(preview, mode="L").save(
|
||||||
|
output_directory / f"depth{suffix}.png"
|
||||||
|
)
|
||||||
|
normal_rgb = np.clip(
|
||||||
|
(normal * np.array([1.0, -1.0, -1.0], dtype=np.float32) + 1.0)
|
||||||
|
* 127.5,
|
||||||
|
0.0,
|
||||||
|
255.0,
|
||||||
|
).astype(np.uint8)
|
||||||
|
Image.fromarray(normal_rgb, mode="RGB").save(
|
||||||
|
output_directory / f"normal{suffix}.png"
|
||||||
|
)
|
||||||
|
Image.fromarray(mask.astype(np.uint8) * 255, mode="L").save(
|
||||||
|
output_directory / f"mask{suffix}.png"
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def run_moge(args, model_cache=None, cancel_check=None):
|
||||||
|
import numpy as np
|
||||||
|
from PIL import Image
|
||||||
|
|
||||||
|
prepare_moge_runtime()
|
||||||
|
write_status(args.status, 0.02, "Starting MoGe-2")
|
||||||
|
if model_cache is None:
|
||||||
|
import torch
|
||||||
|
from moge.model.v2 import MoGeModel
|
||||||
|
else:
|
||||||
|
torch = model_cache.torch
|
||||||
|
output_directory = Path(args.output_dir)
|
||||||
|
output_directory.mkdir(parents=True, exist_ok=True)
|
||||||
|
images, temporary_directory = prepare_input(
|
||||||
|
args.input,
|
||||||
|
args.video_frames,
|
||||||
|
output_directory,
|
||||||
|
start_frame=getattr(args, "start_frame", 1),
|
||||||
|
cancel_check=cancel_check,
|
||||||
|
)
|
||||||
|
try:
|
||||||
|
device = select_device(args.device, torch)
|
||||||
|
write_status(args.status, 0.08, f"Loading MoGe-2 on {device.type}")
|
||||||
|
if model_cache is None:
|
||||||
|
checkpoint = Path(args.model_dir) / "model.pt"
|
||||||
|
model = MoGeModel.from_pretrained(checkpoint)
|
||||||
|
model = model.to(device=device)
|
||||||
|
model.eval()
|
||||||
|
else:
|
||||||
|
model = model_cache.get_moge(args.model_dir, device)
|
||||||
|
|
||||||
|
depths = []
|
||||||
|
normals = []
|
||||||
|
masks = []
|
||||||
|
points = []
|
||||||
|
intrinsics = []
|
||||||
|
for index, image_path in enumerate(images):
|
||||||
|
check_cancelled(cancel_check)
|
||||||
|
write_status(
|
||||||
|
args.status,
|
||||||
|
0.15 + 0.68 * index / len(images),
|
||||||
|
f"Estimating MoGe-2 geometry ({index + 1}/{len(images)})",
|
||||||
|
)
|
||||||
|
with Image.open(image_path) as opened:
|
||||||
|
image = np.asarray(opened.convert("RGB"), dtype=np.float32) / 255.0
|
||||||
|
image_tensor = torch.from_numpy(image).permute(2, 0, 1).to(device)
|
||||||
|
with torch.inference_mode():
|
||||||
|
prediction = model.infer(
|
||||||
|
image_tensor,
|
||||||
|
resolution_level=int(args.resolution_level),
|
||||||
|
force_projection=True,
|
||||||
|
apply_mask=False,
|
||||||
|
use_fp16=device.type == "cuda",
|
||||||
|
)
|
||||||
|
depth = prediction["depth"].detach().float().cpu().numpy()
|
||||||
|
normal = prediction["normal"].detach().float().cpu().numpy()
|
||||||
|
mask = prediction["mask"].detach().cpu().numpy().astype(bool)
|
||||||
|
point = prediction["points"].detach().float().cpu().numpy()
|
||||||
|
intrinsic = prediction["intrinsics"].detach().float().cpu().numpy()
|
||||||
|
depths.append(depth)
|
||||||
|
normals.append(normal)
|
||||||
|
masks.append(mask)
|
||||||
|
intrinsics.append(intrinsic)
|
||||||
|
if bool(getattr(args, "include_points", False)):
|
||||||
|
points.append(point)
|
||||||
|
_save_moge_preview(
|
||||||
|
output_directory,
|
||||||
|
None if len(images) == 1 else index,
|
||||||
|
depth,
|
||||||
|
normal,
|
||||||
|
mask,
|
||||||
|
)
|
||||||
|
|
||||||
|
shapes = {array.shape for array in depths}
|
||||||
|
if len(shapes) != 1:
|
||||||
|
raise RuntimeError("MoGe-2 frame sizes must match")
|
||||||
|
result = {
|
||||||
|
"depth": np.stack(depths).astype(np.float32),
|
||||||
|
"normal": np.stack(normals).astype(np.float32),
|
||||||
|
"mask": np.stack(masks),
|
||||||
|
"intrinsics": np.stack(intrinsics).astype(np.float32),
|
||||||
|
}
|
||||||
|
if points:
|
||||||
|
result["points"] = np.stack(points).astype(np.float32)
|
||||||
|
np.savez(output_directory / "results.npz", **result)
|
||||||
|
|
||||||
|
remove_background_model_dir = getattr(
|
||||||
|
args, "remove_background_model_dir", ""
|
||||||
|
)
|
||||||
|
if remove_background_model_dir:
|
||||||
|
write_status(args.status, 0.87, "Removing image background")
|
||||||
|
if len(images) == 1:
|
||||||
|
foreground, _, _ = infer_foreground(
|
||||||
|
images[0],
|
||||||
|
remove_background_model_dir,
|
||||||
|
device,
|
||||||
|
torch,
|
||||||
|
model_cache=model_cache,
|
||||||
|
cancel_check=cancel_check,
|
||||||
|
)
|
||||||
|
foreground.save(output_directory / "foreground.png")
|
||||||
|
else:
|
||||||
|
infer_foreground_frames(
|
||||||
|
images,
|
||||||
|
remove_background_model_dir,
|
||||||
|
device,
|
||||||
|
torch,
|
||||||
|
output_directory,
|
||||||
|
args.status,
|
||||||
|
0.87,
|
||||||
|
0.08,
|
||||||
|
model_cache=model_cache,
|
||||||
|
cancel_check=cancel_check,
|
||||||
|
)
|
||||||
|
copy_frames(images, output_directory)
|
||||||
|
write_status(
|
||||||
|
args.status,
|
||||||
|
1.0,
|
||||||
|
f"Exported MoGe-2 data for {len(images)} frame(s)",
|
||||||
|
)
|
||||||
|
finally:
|
||||||
|
if temporary_directory is not None:
|
||||||
|
shutil.rmtree(temporary_directory, ignore_errors=True)
|
||||||
|
|
||||||
|
|
||||||
|
def prepare_moge_runtime():
|
||||||
|
"""Use PyTorch SDPA because xFormers does not support every CUDA GPU."""
|
||||||
|
os.environ["XFORMERS_DISABLED"] = "1"
|
||||||
|
|
||||||
|
|
||||||
def reconstruct(args, model_cache=None, cancel_check=None):
|
def reconstruct(args, model_cache=None, cancel_check=None):
|
||||||
write_status(args.status, 0.02, "Starting Environment")
|
write_status(args.status, 0.02, "Starting Environment")
|
||||||
check_cancelled(cancel_check)
|
check_cancelled(cancel_check)
|
||||||
@@ -1686,6 +1830,9 @@ def parser():
|
|||||||
download.add_argument("--model-dir", required=True)
|
download.add_argument("--model-dir", required=True)
|
||||||
download.add_argument("--status", required=True)
|
download.add_argument("--status", required=True)
|
||||||
download.add_argument("--allow-pattern", action="append", default=[])
|
download.add_argument("--allow-pattern", action="append", default=[])
|
||||||
|
download.add_argument("--required-pattern", action="append", default=[])
|
||||||
|
download.add_argument("--r2-directory", default="")
|
||||||
|
download.add_argument("--r2-file", action="append", default=[])
|
||||||
download.add_argument("--stage", type=int, default=None)
|
download.add_argument("--stage", type=int, default=None)
|
||||||
download.add_argument("--stages", type=int, default=None)
|
download.add_argument("--stages", type=int, default=None)
|
||||||
download.add_argument("--stage-label", default=None)
|
download.add_argument("--stage-label", default=None)
|
||||||
|
|||||||
@@ -138,6 +138,15 @@ class AddonTest(unittest.TestCase):
|
|||||||
self.addon.BuildDepthPlane.bl_options,
|
self.addon.BuildDepthPlane.bl_options,
|
||||||
{"INTERNAL"},
|
{"INTERNAL"},
|
||||||
)
|
)
|
||||||
|
self.assertIn(
|
||||||
|
"snapform_color_frames_dir",
|
||||||
|
self.addon.BuildDepthPlane.__annotations__,
|
||||||
|
)
|
||||||
|
self.assertIn(
|
||||||
|
"snapform_depth_frames_dir",
|
||||||
|
self.addon.BuildDepthPlane.__annotations__,
|
||||||
|
)
|
||||||
|
self.assertIn("execute", self.addon.BuildDepthPlane.__dict__)
|
||||||
self.assertFalse(hasattr(self.addon, "ImportDepthPlane"))
|
self.assertFalse(hasattr(self.addon, "ImportDepthPlane"))
|
||||||
|
|
||||||
def test_registers_perspective_mesh_operators(self):
|
def test_registers_perspective_mesh_operators(self):
|
||||||
@@ -553,7 +562,7 @@ class AddonTest(unittest.TestCase):
|
|||||||
|
|
||||||
def test_original_depth_background_starts_independent_job(self):
|
def test_original_depth_background_starts_independent_job(self):
|
||||||
operator = self.addon.ConvertImageToDepthPlane()
|
operator = self.addon.ConvertImageToDepthPlane()
|
||||||
operator.snapform_depth_source = "ORIGINAL"
|
operator.snapform_model = "ORIGINAL"
|
||||||
operator.snapform_unshaded = False
|
operator.snapform_unshaded = False
|
||||||
operator.snapform_remove_background = True
|
operator.snapform_remove_background = True
|
||||||
operator.snapform_keep_original = True
|
operator.snapform_keep_original = True
|
||||||
@@ -586,7 +595,7 @@ class AddonTest(unittest.TestCase):
|
|||||||
|
|
||||||
def test_original_depth_animation_extracts_frames_and_background(self):
|
def test_original_depth_animation_extracts_frames_and_background(self):
|
||||||
operator = self.addon.ConvertImageToDepthPlane()
|
operator = self.addon.ConvertImageToDepthPlane()
|
||||||
operator.snapform_depth_source = "ORIGINAL"
|
operator.snapform_model = "ORIGINAL"
|
||||||
operator.snapform_unshaded = False
|
operator.snapform_unshaded = False
|
||||||
operator.snapform_remove_background = True
|
operator.snapform_remove_background = True
|
||||||
operator.snapform_keep_original = True
|
operator.snapform_keep_original = True
|
||||||
@@ -630,7 +639,7 @@ class AddonTest(unittest.TestCase):
|
|||||||
|
|
||||||
def test_original_depth_without_background_uses_frame_extraction(self):
|
def test_original_depth_without_background_uses_frame_extraction(self):
|
||||||
operator = self.addon.ConvertImageToDepthPlane()
|
operator = self.addon.ConvertImageToDepthPlane()
|
||||||
operator.snapform_depth_source = "ORIGINAL"
|
operator.snapform_model = "ORIGINAL"
|
||||||
operator.snapform_unshaded = False
|
operator.snapform_unshaded = False
|
||||||
operator.snapform_remove_background = False
|
operator.snapform_remove_background = False
|
||||||
operator.snapform_keep_original = False
|
operator.snapform_keep_original = False
|
||||||
@@ -664,10 +673,7 @@ class AddonTest(unittest.TestCase):
|
|||||||
def test_conversion_dialog_exposes_requested_settings(self):
|
def test_conversion_dialog_exposes_requested_settings(self):
|
||||||
operator = self.addon.ConvertImageToDepthPlane
|
operator = self.addon.ConvertImageToDepthPlane
|
||||||
|
|
||||||
self.assertNotIn(
|
self.assertEqual(operator.__annotations__["snapform_model"]["default"], 1)
|
||||||
"default",
|
|
||||||
operator.__annotations__["snapform_model"],
|
|
||||||
)
|
|
||||||
self.assertTrue(operator.__annotations__["snapform_use_ray_pose"]["default"])
|
self.assertTrue(operator.__annotations__["snapform_use_ray_pose"]["default"])
|
||||||
self.assertEqual(
|
self.assertEqual(
|
||||||
operator.__annotations__["snapform_process_res_factor"]["default"],
|
operator.__annotations__["snapform_process_res_factor"]["default"],
|
||||||
@@ -693,23 +699,16 @@ class AddonTest(unittest.TestCase):
|
|||||||
self.assertEqual(
|
self.assertEqual(
|
||||||
set(operator.__annotations__),
|
set(operator.__annotations__),
|
||||||
{
|
{
|
||||||
"snapform_depth_source",
|
|
||||||
"snapform_unshaded",
|
"snapform_unshaded",
|
||||||
"snapform_model",
|
"snapform_model",
|
||||||
"snapform_use_ray_pose",
|
"snapform_use_ray_pose",
|
||||||
"snapform_process_res_factor",
|
"snapform_process_res_factor",
|
||||||
|
"snapform_moge_resolution_level",
|
||||||
"snapform_remove_background",
|
"snapform_remove_background",
|
||||||
"snapform_keep_original",
|
"snapform_keep_original",
|
||||||
},
|
},
|
||||||
)
|
)
|
||||||
depth_sources = operator.__annotations__["snapform_depth_source"]
|
self.assertEqual(operator.__annotations__["snapform_model"]["name"], "Model")
|
||||||
self.assertEqual(depth_sources["default"], "ORIGINAL")
|
|
||||||
self.assertEqual(
|
|
||||||
[item[0] for item in depth_sources["items"]],
|
|
||||||
["ESTIMATE", "ORIGINAL"],
|
|
||||||
)
|
|
||||||
self.assertEqual(depth_sources["items"][0][1], "AI Generated")
|
|
||||||
self.assertEqual(depth_sources["name"], "Depth from")
|
|
||||||
self.assertEqual(
|
self.assertEqual(
|
||||||
operator.__annotations__["snapform_unshaded"]["name"],
|
operator.__annotations__["snapform_unshaded"]["name"],
|
||||||
"Unshaded",
|
"Unshaded",
|
||||||
@@ -724,11 +723,12 @@ class AddonTest(unittest.TestCase):
|
|||||||
self.assertEqual(
|
self.assertEqual(
|
||||||
[item[0] for item in model_items],
|
[item[0] for item in model_items],
|
||||||
[
|
[
|
||||||
|
"MOGE2_VITS_NORMAL",
|
||||||
|
"MOGE2_VITB_NORMAL",
|
||||||
|
"MOGE2_VITL_NORMAL",
|
||||||
"SMALL",
|
"SMALL",
|
||||||
"BASE",
|
"BASE",
|
||||||
"LARGE_1_1",
|
"ORIGINAL",
|
||||||
"GIANT_1_1",
|
|
||||||
"NESTED_GIANT_LARGE_1_1",
|
|
||||||
],
|
],
|
||||||
)
|
)
|
||||||
|
|
||||||
@@ -758,7 +758,7 @@ class AddonTest(unittest.TestCase):
|
|||||||
result = operator.invoke(context, None)
|
result = operator.invoke(context, None)
|
||||||
|
|
||||||
self.assertIs(result, operator)
|
self.assertIs(result, operator)
|
||||||
self.assertEqual(operator.snapform_model, "GIANT_1_1")
|
self.assertNotEqual(getattr(operator, "snapform_model", ""), "GIANT_1_1")
|
||||||
|
|
||||||
def test_remove_background_defaults_to_empty_image_frames(self):
|
def test_remove_background_defaults_to_empty_image_frames(self):
|
||||||
source_object = SimpleNamespace(
|
source_object = SimpleNamespace(
|
||||||
@@ -808,7 +808,6 @@ class AddonTest(unittest.TestCase):
|
|||||||
)
|
)
|
||||||
context = SimpleNamespace(object=source_object, scene=scene)
|
context = SimpleNamespace(object=source_object, scene=scene)
|
||||||
operator = self.addon.ConvertImageToDepthPlane()
|
operator = self.addon.ConvertImageToDepthPlane()
|
||||||
operator.snapform_depth_source = "ESTIMATE"
|
|
||||||
operator.snapform_model = "BASE"
|
operator.snapform_model = "BASE"
|
||||||
operator.snapform_use_ray_pose = True
|
operator.snapform_use_ray_pose = True
|
||||||
operator.snapform_process_res_factor = 50.0
|
operator.snapform_process_res_factor = 50.0
|
||||||
@@ -839,7 +838,7 @@ class AddonTest(unittest.TestCase):
|
|||||||
self.assertTrue(calls[0]["snapform_remove_background"])
|
self.assertTrue(calls[0]["snapform_remove_background"])
|
||||||
self.assertNotIn("snapform_background_depth_distance", calls[0])
|
self.assertNotIn("snapform_background_depth_distance", calls[0])
|
||||||
|
|
||||||
def test_conversion_dialog_only_lists_installed_models(self):
|
def test_conversion_dialog_marks_missing_models(self):
|
||||||
items_callback = self.addon.ConvertImageToDepthPlane.__annotations__[
|
items_callback = self.addon.ConvertImageToDepthPlane.__annotations__[
|
||||||
"snapform_model"
|
"snapform_model"
|
||||||
]["items"]
|
]["items"]
|
||||||
@@ -850,7 +849,20 @@ class AddonTest(unittest.TestCase):
|
|||||||
):
|
):
|
||||||
items = items_callback(None, None)
|
items = items_callback(None, None)
|
||||||
|
|
||||||
self.assertEqual([item[0] for item in items], ["SMALL", "BASE"])
|
self.assertEqual(
|
||||||
|
[item[0] for item in items],
|
||||||
|
[
|
||||||
|
"MOGE2_VITS_NORMAL",
|
||||||
|
"MOGE2_VITB_NORMAL",
|
||||||
|
"MOGE2_VITL_NORMAL",
|
||||||
|
"SMALL",
|
||||||
|
"BASE",
|
||||||
|
"ORIGINAL",
|
||||||
|
],
|
||||||
|
)
|
||||||
|
labels = {item[0]: item[1] for item in items}
|
||||||
|
self.assertEqual(labels["BASE"], "DA3 Base")
|
||||||
|
self.assertEqual(labels["MOGE2_VITB_NORMAL"], "MoGe-2 ViT-B Normal (Missing)")
|
||||||
|
|
||||||
def test_update_status_formats_stage_progress(self):
|
def test_update_status_formats_stage_progress(self):
|
||||||
operator = self.addon.RunJob()
|
operator = self.addon.RunJob()
|
||||||
@@ -936,7 +948,6 @@ class AddonTest(unittest.TestCase):
|
|||||||
|
|
||||||
def test_packed_image_is_exported_without_unpacking(self):
|
def test_packed_image_is_exported_without_unpacking(self):
|
||||||
operator = self.addon.ConvertImageToDepthPlane()
|
operator = self.addon.ConvertImageToDepthPlane()
|
||||||
operator.snapform_depth_source = "ESTIMATE"
|
|
||||||
operator.snapform_model = "GIANT_1_1"
|
operator.snapform_model = "GIANT_1_1"
|
||||||
operator.snapform_use_ray_pose = True
|
operator.snapform_use_ray_pose = True
|
||||||
operator.snapform_process_res_factor = 50.0
|
operator.snapform_process_res_factor = 50.0
|
||||||
@@ -1869,7 +1880,7 @@ class AddonTest(unittest.TestCase):
|
|||||||
|
|
||||||
self.assertEqual(
|
self.assertEqual(
|
||||||
identifiers,
|
identifiers,
|
||||||
["GIANT_1_1", "NESTED_GIANT_LARGE_1_1"],
|
[],
|
||||||
)
|
)
|
||||||
|
|
||||||
def test_device_enums_list_only_supported_devices(self):
|
def test_device_enums_list_only_supported_devices(self):
|
||||||
@@ -1966,7 +1977,7 @@ class AddonTest(unittest.TestCase):
|
|||||||
"model_ready",
|
"model_ready",
|
||||||
return_value=False,
|
return_value=False,
|
||||||
):
|
):
|
||||||
self.assertFalse(
|
self.assertTrue(
|
||||||
self.addon.ConvertImageToDepthPlane.poll(context)
|
self.addon.ConvertImageToDepthPlane.poll(context)
|
||||||
)
|
)
|
||||||
|
|
||||||
@@ -1977,6 +1988,7 @@ class AddonTest(unittest.TestCase):
|
|||||||
)
|
)
|
||||||
panels = (
|
panels = (
|
||||||
self.addon.Da3OutputDebugPanel,
|
self.addon.Da3OutputDebugPanel,
|
||||||
|
self.addon.Moge2ApiDebugPanel,
|
||||||
self.addon.Ben2RemovalDebugPanel,
|
self.addon.Ben2RemovalDebugPanel,
|
||||||
)
|
)
|
||||||
for panel in panels:
|
for panel in panels:
|
||||||
@@ -1984,7 +1996,7 @@ class AddonTest(unittest.TestCase):
|
|||||||
self.assertIn("DEFAULT_CLOSED", panel.bl_options)
|
self.assertIn("DEFAULT_CLOSED", panel.bl_options)
|
||||||
self.assertEqual(
|
self.assertEqual(
|
||||||
[panel.bl_label for panel in panels],
|
[panel.bl_label for panel in panels],
|
||||||
["DA3 Output Test", "BEN2 Removal Test"],
|
["DA3 Output Test", "MoGe-2 API Test", "BEN2 Removal Test"],
|
||||||
)
|
)
|
||||||
|
|
||||||
def test_service_panel_exposes_shared_service_state_and_controls(self):
|
def test_service_panel_exposes_shared_service_state_and_controls(self):
|
||||||
|
|||||||
@@ -168,7 +168,7 @@ class DownloadProgressTest(unittest.TestCase):
|
|||||||
|
|
||||||
self.assertFalse(status_path.exists())
|
self.assertFalse(status_path.exists())
|
||||||
|
|
||||||
def test_download_endpoints_keeps_direct_endpoint(self):
|
def test_download_endpoints_uses_direct_endpoint(self):
|
||||||
with patch.dict(os.environ, {"HF_ENDPOINT": ""}):
|
with patch.dict(os.environ, {"HF_ENDPOINT": ""}):
|
||||||
endpoints = self.worker.download_endpoints()
|
endpoints = self.worker.download_endpoints()
|
||||||
|
|
||||||
|
|||||||
@@ -10,7 +10,7 @@ import unittest
|
|||||||
import zipfile
|
import zipfile
|
||||||
from pathlib import Path
|
from pathlib import Path
|
||||||
from types import SimpleNamespace
|
from types import SimpleNamespace
|
||||||
from unittest.mock import patch
|
from unittest.mock import Mock, patch
|
||||||
|
|
||||||
|
|
||||||
PROJECT_ROOT = Path(__file__).resolve().parents[1]
|
PROJECT_ROOT = Path(__file__).resolve().parents[1]
|
||||||
@@ -32,6 +32,8 @@ class InstallEnvTest(unittest.TestCase):
|
|||||||
vendor.mkdir()
|
vendor.mkdir()
|
||||||
(vendor / "depth-anything-3.tar.gz").write_bytes(b"da3")
|
(vendor / "depth-anything-3.tar.gz").write_bytes(b"da3")
|
||||||
(vendor / "ben2.tar.gz").write_bytes(b"ben2")
|
(vendor / "ben2.tar.gz").write_bytes(b"ben2")
|
||||||
|
(vendor / "utils3d.tar.gz").write_bytes(b"utils3d")
|
||||||
|
(vendor / "moge.tar.gz").write_bytes(b"moge")
|
||||||
return vendor
|
return vendor
|
||||||
|
|
||||||
def test_installs_da3_from_archives_without_gsplat(self):
|
def test_installs_da3_from_archives_without_gsplat(self):
|
||||||
@@ -155,19 +157,13 @@ class InstallEnvTest(unittest.TestCase):
|
|||||||
if any(
|
if any(
|
||||||
repository in command
|
repository in command
|
||||||
for repository in (
|
for repository in (
|
||||||
"depth-anything/DA3-SMALL",
|
"Ruicheng/moge-2-vitb-normal",
|
||||||
"depth-anything/DA3-BASE",
|
|
||||||
"PramaLLC/BEN2",
|
"PramaLLC/BEN2",
|
||||||
)
|
)
|
||||||
)
|
)
|
||||||
]
|
]
|
||||||
self.assertEqual(len(default_downloads), 2)
|
self.assertEqual(len(default_downloads), 2)
|
||||||
self.assertFalse(
|
self.assertTrue(any("Ruicheng/moge-2-vitb-normal" in map(str, command) for command in default_downloads))
|
||||||
any(
|
|
||||||
"depth-anything/DA3-BASE" in map(str, command)
|
|
||||||
for command in default_downloads
|
|
||||||
)
|
|
||||||
)
|
|
||||||
ben2_download = next(
|
ben2_download = next(
|
||||||
command
|
command
|
||||||
for command in default_downloads
|
for command in default_downloads
|
||||||
@@ -663,6 +659,32 @@ class InstallEnvTest(unittest.TestCase):
|
|||||||
env["PIP_INDEX_URL"],
|
env["PIP_INDEX_URL"],
|
||||||
"https://pypi.tuna.tsinghua.edu.cn/simple",
|
"https://pypi.tuna.tsinghua.edu.cn/simple",
|
||||||
)
|
)
|
||||||
|
self.assertEqual(
|
||||||
|
env["PYTHONIOENCODING"],
|
||||||
|
"utf-8:backslashreplace",
|
||||||
|
)
|
||||||
|
self.assertEqual(env["PYTHONUTF8"], "1")
|
||||||
|
|
||||||
|
def test_standard_streams_are_reconfigured_for_unicode_logs(self):
|
||||||
|
bootstrap = load_install_env()
|
||||||
|
stdout = Mock()
|
||||||
|
stderr = Mock()
|
||||||
|
|
||||||
|
with patch.object(bootstrap.sys, "stdout", stdout), patch.object(
|
||||||
|
bootstrap.sys,
|
||||||
|
"stderr",
|
||||||
|
stderr,
|
||||||
|
):
|
||||||
|
bootstrap.configure_standard_streams()
|
||||||
|
|
||||||
|
stdout.reconfigure.assert_called_once_with(
|
||||||
|
encoding="utf-8",
|
||||||
|
errors="backslashreplace",
|
||||||
|
)
|
||||||
|
stderr.reconfigure.assert_called_once_with(
|
||||||
|
encoding="utf-8",
|
||||||
|
errors="backslashreplace",
|
||||||
|
)
|
||||||
|
|
||||||
def test_ready_json_records_installed_packages(self):
|
def test_ready_json_records_installed_packages(self):
|
||||||
bootstrap = load_install_env()
|
bootstrap = load_install_env()
|
||||||
@@ -706,6 +728,10 @@ class InstallEnvTest(unittest.TestCase):
|
|||||||
["numpy==1.26.4", "torch==2.9.1+cu128"],
|
["numpy==1.26.4", "torch==2.9.1+cu128"],
|
||||||
)
|
)
|
||||||
self.assertEqual(ready["devices"], ["cuda", "cpu"])
|
self.assertEqual(ready["devices"], ["cuda", "cpu"])
|
||||||
|
self.assertEqual(
|
||||||
|
ready["manifest_revision"],
|
||||||
|
bootstrap.MANIFEST_REVISION,
|
||||||
|
)
|
||||||
|
|
||||||
def test_broken_env_is_cleaned_before_reinstall(self):
|
def test_broken_env_is_cleaned_before_reinstall(self):
|
||||||
bootstrap = load_install_env()
|
bootstrap = load_install_env()
|
||||||
@@ -752,7 +778,9 @@ class InstallEnvTest(unittest.TestCase):
|
|||||||
(venv_dir / "Scripts").mkdir()
|
(venv_dir / "Scripts").mkdir()
|
||||||
(venv_dir / "Scripts" / "python.exe").write_text("python")
|
(venv_dir / "Scripts" / "python.exe").write_text("python")
|
||||||
(root / "manifest.json").write_text(
|
(root / "manifest.json").write_text(
|
||||||
json.dumps({}),
|
json.dumps(
|
||||||
|
{"manifest_revision": bootstrap.MANIFEST_REVISION}
|
||||||
|
),
|
||||||
encoding="utf-8",
|
encoding="utf-8",
|
||||||
)
|
)
|
||||||
(venv_dir / "marker.txt").write_text("keep")
|
(venv_dir / "marker.txt").write_text("keep")
|
||||||
@@ -786,7 +814,7 @@ class InstallEnvTest(unittest.TestCase):
|
|||||||
|
|
||||||
def fake_run(command, environment=None):
|
def fake_run(command, environment=None):
|
||||||
commands.append(command)
|
commands.append(command)
|
||||||
if any("DA3-SMALL" in str(value) for value in command):
|
if any("moge-2-vitb-normal" in str(value) for value in command):
|
||||||
raise subprocess.CalledProcessError(1, command)
|
raise subprocess.CalledProcessError(1, command)
|
||||||
|
|
||||||
with tempfile.TemporaryDirectory() as directory:
|
with tempfile.TemporaryDirectory() as directory:
|
||||||
@@ -880,6 +908,8 @@ class InstallEnvTest(unittest.TestCase):
|
|||||||
bundled.mkdir()
|
bundled.mkdir()
|
||||||
(bundled / "depth-anything-3.tar.gz").write_bytes(b"da3")
|
(bundled / "depth-anything-3.tar.gz").write_bytes(b"da3")
|
||||||
(bundled / "ben2.tar.gz").write_bytes(b"ben2")
|
(bundled / "ben2.tar.gz").write_bytes(b"ben2")
|
||||||
|
(bundled / "utils3d.tar.gz").write_bytes(b"utils3d")
|
||||||
|
(bundled / "moge.tar.gz").write_bytes(b"moge")
|
||||||
target, archive_name = bootstrap.uv_download_target()
|
target, archive_name = bootstrap.uv_download_target()
|
||||||
archive = bundled / f"uv-{target}.{archive_name}"
|
archive = bundled / f"uv-{target}.{archive_name}"
|
||||||
if archive_name.endswith("zip"):
|
if archive_name.endswith("zip"):
|
||||||
@@ -1025,6 +1055,8 @@ class InstallEnvTest(unittest.TestCase):
|
|||||||
bundled.mkdir()
|
bundled.mkdir()
|
||||||
(bundled / "depth-anything-3.tar.gz").write_bytes(b"da3")
|
(bundled / "depth-anything-3.tar.gz").write_bytes(b"da3")
|
||||||
(bundled / "ben2.tar.gz").write_bytes(b"ben2")
|
(bundled / "ben2.tar.gz").write_bytes(b"ben2")
|
||||||
|
(bundled / "utils3d.tar.gz").write_bytes(b"utils3d")
|
||||||
|
(bundled / "moge.tar.gz").write_bytes(b"moge")
|
||||||
vendor = root / "vendor"
|
vendor = root / "vendor"
|
||||||
|
|
||||||
bootstrap.copy_vendor_inputs(bundled, vendor)
|
bootstrap.copy_vendor_inputs(bundled, vendor)
|
||||||
@@ -1046,6 +1078,8 @@ class InstallEnvTest(unittest.TestCase):
|
|||||||
bundled.mkdir()
|
bundled.mkdir()
|
||||||
(bundled / "depth-anything-3.tar.gz").write_bytes(b"new")
|
(bundled / "depth-anything-3.tar.gz").write_bytes(b"new")
|
||||||
(bundled / "ben2.tar.gz").write_bytes(b"ben2")
|
(bundled / "ben2.tar.gz").write_bytes(b"ben2")
|
||||||
|
(bundled / "utils3d.tar.gz").write_bytes(b"utils3d")
|
||||||
|
(bundled / "moge.tar.gz").write_bytes(b"moge")
|
||||||
vendor = root / "vendor"
|
vendor = root / "vendor"
|
||||||
vendor.mkdir()
|
vendor.mkdir()
|
||||||
(vendor / "depth-anything-3.tar.gz").write_bytes(b"old")
|
(vendor / "depth-anything-3.tar.gz").write_bytes(b"old")
|
||||||
@@ -1086,6 +1120,8 @@ class InstallEnvTest(unittest.TestCase):
|
|||||||
vendor.mkdir()
|
vendor.mkdir()
|
||||||
(vendor / "depth-anything-3.tar.gz").write_bytes(b"da3")
|
(vendor / "depth-anything-3.tar.gz").write_bytes(b"da3")
|
||||||
(vendor / "ben2.tar.gz").write_bytes(b"ben2")
|
(vendor / "ben2.tar.gz").write_bytes(b"ben2")
|
||||||
|
(vendor / "utils3d.tar.gz").write_bytes(b"utils3d")
|
||||||
|
(vendor / "moge.tar.gz").write_bytes(b"moge")
|
||||||
uv_path = root / "uv"
|
uv_path = root / "uv"
|
||||||
uv_path.write_bytes(b"uv")
|
uv_path.write_bytes(b"uv")
|
||||||
args = SimpleNamespace(
|
args = SimpleNamespace(
|
||||||
@@ -1126,6 +1162,8 @@ class InstallEnvTest(unittest.TestCase):
|
|||||||
vendor.mkdir()
|
vendor.mkdir()
|
||||||
(vendor / "depth-anything-3.tar.gz").write_bytes(b"da3")
|
(vendor / "depth-anything-3.tar.gz").write_bytes(b"da3")
|
||||||
(vendor / "ben2.tar.gz").write_bytes(b"ben2")
|
(vendor / "ben2.tar.gz").write_bytes(b"ben2")
|
||||||
|
(vendor / "utils3d.tar.gz").write_bytes(b"utils3d")
|
||||||
|
(vendor / "moge.tar.gz").write_bytes(b"moge")
|
||||||
uv_path = root / "uv"
|
uv_path = root / "uv"
|
||||||
uv_path.write_bytes(b"uv")
|
uv_path.write_bytes(b"uv")
|
||||||
args = SimpleNamespace(
|
args = SimpleNamespace(
|
||||||
@@ -1150,9 +1188,15 @@ class InstallEnvTest(unittest.TestCase):
|
|||||||
for label, command, _environment, kind in stages
|
for label, command, _environment, kind in stages
|
||||||
if kind == "download"
|
if kind == "download"
|
||||||
}
|
}
|
||||||
self.assertIn("da3_small", " ".join(map(str, downloads["DA3 Small"])))
|
self.assertIn(
|
||||||
|
"moge2_vitb_normal",
|
||||||
|
" ".join(map(str, downloads["MoGe-2 ViT-B Normal"])),
|
||||||
|
)
|
||||||
self.assertIn("ben2_base", " ".join(map(str, downloads["BEN2 Base"])))
|
self.assertIn("ben2_base", " ".join(map(str, downloads["BEN2 Base"])))
|
||||||
self.assertNotIn("DA3 Base", downloads)
|
self.assertEqual(
|
||||||
|
set(downloads),
|
||||||
|
{"MoGe-2 ViT-B Normal", "BEN2 Base"},
|
||||||
|
)
|
||||||
|
|
||||||
def test_install_does_not_verify_env(self):
|
def test_install_does_not_verify_env(self):
|
||||||
bootstrap = load_install_env()
|
bootstrap = load_install_env()
|
||||||
|
|||||||
@@ -34,11 +34,18 @@ class ModelTest(unittest.TestCase):
|
|||||||
model = self.models.get_model("LARGE_1_1")
|
model = self.models.get_model("LARGE_1_1")
|
||||||
|
|
||||||
self.assertEqual(model.license_name, "CC BY-NC 4.0")
|
self.assertEqual(model.license_name, "CC BY-NC 4.0")
|
||||||
|
self.assertFalse(model.enabled)
|
||||||
|
|
||||||
def test_giant_models_expose_gaussian_head(self):
|
def test_giant_models_expose_gaussian_head(self):
|
||||||
for model_key in ("GIANT_1_1", "NESTED_GIANT_LARGE_1_1"):
|
for model_key in ("GIANT_1_1", "NESTED_GIANT_LARGE_1_1"):
|
||||||
with self.subTest(model_key=model_key):
|
with self.subTest(model_key=model_key):
|
||||||
self.assertTrue(self.models.get_model(model_key).supports_gs)
|
self.assertTrue(self.models.get_model(model_key).supports_gs)
|
||||||
|
self.assertFalse(self.models.get_model(model_key).enabled)
|
||||||
|
|
||||||
|
def test_only_uploaded_da3_models_are_enabled(self):
|
||||||
|
enabled = [key for key, model in self.models.MODELS.items() if model.enabled]
|
||||||
|
|
||||||
|
self.assertEqual(enabled, ["SMALL", "BASE"])
|
||||||
|
|
||||||
def test_unknown_model_is_rejected(self):
|
def test_unknown_model_is_rejected(self):
|
||||||
with self.assertRaisesRegex(ValueError, "Unknown DA3 model"):
|
with self.assertRaisesRegex(ValueError, "Unknown DA3 model"):
|
||||||
@@ -72,3 +79,22 @@ class ModelTest(unittest.TestCase):
|
|||||||
self.models.model_directory_name("BEN2_BASE"),
|
self.models.model_directory_name("BEN2_BASE"),
|
||||||
"ben2_base",
|
"ben2_base",
|
||||||
)
|
)
|
||||||
|
|
||||||
|
def test_moge2_uses_single_checkpoint_and_mit_license(self):
|
||||||
|
model = self.models.get_downloadable_model("MOGE2_VITB_NORMAL")
|
||||||
|
|
||||||
|
self.assertEqual(model.repository, "Ruicheng/moge-2-vitb-normal")
|
||||||
|
self.assertEqual(model.license_name, "MIT")
|
||||||
|
self.assertEqual(model.download_patterns, ("model.pt",))
|
||||||
|
self.assertEqual(model.ready_patterns, ("model.pt",))
|
||||||
|
self.assertFalse(model.supports_gs)
|
||||||
|
|
||||||
|
def test_moge2_models_match_uploaded_r2_variants(self):
|
||||||
|
self.assertEqual(
|
||||||
|
list(self.models.MOGE2_MODELS),
|
||||||
|
["MOGE2_VITS_NORMAL", "MOGE2_VITB_NORMAL", "MOGE2_VITL_NORMAL"],
|
||||||
|
)
|
||||||
|
self.assertEqual(
|
||||||
|
[model.r2_directory for model in self.models.MOGE2_MODELS.values()],
|
||||||
|
["MoGe2ViTSNormal", "MoGe2ViTBNormal", "MoGe2ViTLNormal"],
|
||||||
|
)
|
||||||
|
|||||||
@@ -131,6 +131,9 @@ class ServiceTest(unittest.TestCase):
|
|||||||
state.reserve("other", "second", "reconstruct", "other.json")
|
state.reserve("other", "second", "reconstruct", "other.json")
|
||||||
)
|
)
|
||||||
|
|
||||||
|
def test_service_accepts_moge_command(self):
|
||||||
|
self.assertIn("moge", self.service.JOB_COMMANDS)
|
||||||
|
|
||||||
def test_cancelling_job_keeps_service_and_model_cache_alive(self):
|
def test_cancelling_job_keeps_service_and_model_cache_alive(self):
|
||||||
fake_models = SimpleNamespace(
|
fake_models = SimpleNamespace(
|
||||||
status=lambda: {"ben2": "BEN2_BASE", "da3": ""},
|
status=lambda: {"ben2": "BEN2_BASE", "da3": ""},
|
||||||
|
|||||||
@@ -436,7 +436,13 @@ class WorkerTest(unittest.TestCase):
|
|||||||
with self.assertRaisesRegex(RuntimeError, "Unsupported input type"):
|
with self.assertRaisesRegex(RuntimeError, "Unsupported input type"):
|
||||||
self.worker.prepare_input(input_path, 16, directory)
|
self.worker.prepare_input(input_path, 16, directory)
|
||||||
|
|
||||||
def test_download_model_skips_unreachable_endpoint(self):
|
def test_moge_runtime_disables_unsupported_xformers_attention(self):
|
||||||
|
with patch.dict(os.environ, {}, clear=True):
|
||||||
|
self.worker.prepare_moge_runtime()
|
||||||
|
|
||||||
|
self.assertEqual(os.environ["XFORMERS_DISABLED"], "1")
|
||||||
|
|
||||||
|
def test_download_model_uses_actual_request_without_preflight(self):
|
||||||
calls = []
|
calls = []
|
||||||
fake_hugging_face = ModuleType("huggingface_hub")
|
fake_hugging_face = ModuleType("huggingface_hub")
|
||||||
fake_hugging_face.snapshot_download = lambda **options: calls.append(
|
fake_hugging_face.snapshot_download = lambda **options: calls.append(
|
||||||
@@ -457,13 +463,6 @@ class WorkerTest(unittest.TestCase):
|
|||||||
"create_download_progress_class",
|
"create_download_progress_class",
|
||||||
return_value=object,
|
return_value=object,
|
||||||
), patch.dict(os.environ, {"HF_ENDPOINT": ""}), patch.object(
|
), patch.dict(os.environ, {"HF_ENDPOINT": ""}), patch.object(
|
||||||
self.worker,
|
|
||||||
"hf_endpoint_reachable",
|
|
||||||
return_value=False,
|
|
||||||
), patch.object(
|
|
||||||
self.worker,
|
|
||||||
"download_from_modelscope",
|
|
||||||
) as mocked_modelscope, patch.object(
|
|
||||||
self.worker,
|
self.worker,
|
||||||
"_verify_model_complete",
|
"_verify_model_complete",
|
||||||
return_value=None,
|
return_value=None,
|
||||||
@@ -472,8 +471,7 @@ class WorkerTest(unittest.TestCase):
|
|||||||
args.status = str(Path(directory) / "status.json")
|
args.status = str(Path(directory) / "status.json")
|
||||||
self.worker.download_model(args)
|
self.worker.download_model(args)
|
||||||
|
|
||||||
self.assertEqual(calls, [])
|
self.assertEqual(len(calls), 1)
|
||||||
self.assertEqual(mocked_modelscope.call_count, 1)
|
|
||||||
|
|
||||||
def test_download_model_refuses_success_when_files_missing(self):
|
def test_download_model_refuses_success_when_files_missing(self):
|
||||||
fake_hugging_face = ModuleType("huggingface_hub")
|
fake_hugging_face = ModuleType("huggingface_hub")
|
||||||
@@ -492,10 +490,7 @@ class WorkerTest(unittest.TestCase):
|
|||||||
self.worker,
|
self.worker,
|
||||||
"create_download_progress_class",
|
"create_download_progress_class",
|
||||||
return_value=object,
|
return_value=object,
|
||||||
), patch.dict(os.environ, {"HF_ENDPOINT": ""}), patch.object(
|
), patch.dict(os.environ, {"HF_ENDPOINT": ""}):
|
||||||
self.worker,
|
|
||||||
"download_from_modelscope",
|
|
||||||
):
|
|
||||||
args.model_dir = str(Path(directory) / "model")
|
args.model_dir = str(Path(directory) / "model")
|
||||||
args.status = str(Path(directory) / "status.json")
|
args.status = str(Path(directory) / "status.json")
|
||||||
with self.assertRaisesRegex(
|
with self.assertRaisesRegex(
|
||||||
@@ -527,39 +522,35 @@ class WorkerTest(unittest.TestCase):
|
|||||||
return_value=object,
|
return_value=object,
|
||||||
), patch.dict(os.environ, {"HF_ENDPOINT": ""}), patch.object(
|
), patch.dict(os.environ, {"HF_ENDPOINT": ""}), patch.object(
|
||||||
self.worker,
|
self.worker,
|
||||||
"download_from_modelscope",
|
"download_from_r2",
|
||||||
side_effect=OSError("mirror down"),
|
side_effect=OSError("mirror down"),
|
||||||
):
|
):
|
||||||
|
args.r2_directory = "DA3Small"
|
||||||
|
args.r2_file = ["config.json|100|abc"]
|
||||||
args.model_dir = str(Path(directory) / "model")
|
args.model_dir = str(Path(directory) / "model")
|
||||||
args.status = str(Path(directory) / "status.json")
|
args.status = str(Path(directory) / "status.json")
|
||||||
with self.assertRaisesRegex(RuntimeError, "modelscope.cn"):
|
with self.assertRaisesRegex(RuntimeError, "models.omoolab.xyz"):
|
||||||
self.worker.download_model(args)
|
self.worker.download_model(args)
|
||||||
|
|
||||||
def test_download_from_modelscope_downloads_selected_files(self):
|
def test_download_from_r2_downloads_declared_files(self):
|
||||||
args = SimpleNamespace(
|
args = SimpleNamespace(
|
||||||
repository="depth-anything/DA3-SMALL",
|
repository="depth-anything/DA3-SMALL",
|
||||||
model_dir="model",
|
model_dir="model",
|
||||||
status="status.json",
|
status="status.json",
|
||||||
allow_pattern=["config.json"],
|
allow_pattern=["config.json"],
|
||||||
|
r2_directory="DA3Small",
|
||||||
|
r2_file=["config.json|100|abc"],
|
||||||
)
|
)
|
||||||
downloaded = []
|
downloaded = []
|
||||||
|
|
||||||
with tempfile.TemporaryDirectory() as directory, patch.object(
|
with tempfile.TemporaryDirectory() as directory, patch.object(
|
||||||
self.worker,
|
self.worker,
|
||||||
"modelscope_file_list",
|
"_download_r2_file",
|
||||||
return_value=[
|
|
||||||
("config.json", 100),
|
|
||||||
("model.safetensors", 200),
|
|
||||||
(".gitattributes", 10),
|
|
||||||
],
|
|
||||||
), patch.object(
|
|
||||||
self.worker,
|
|
||||||
"_download_modelscope_file",
|
|
||||||
side_effect=lambda *args, **kwargs: downloaded.append(args[1]),
|
side_effect=lambda *args, **kwargs: downloaded.append(args[1]),
|
||||||
), patch.object(self.worker, "write_status"):
|
), patch.object(self.worker, "write_status"):
|
||||||
args.model_dir = str(Path(directory) / "model")
|
args.model_dir = str(Path(directory) / "model")
|
||||||
args.status = str(Path(directory) / "status.json")
|
args.status = str(Path(directory) / "status.json")
|
||||||
self.worker.download_from_modelscope(args, args.model_dir)
|
self.worker.download_from_r2(args, args.model_dir)
|
||||||
|
|
||||||
self.assertEqual(downloaded, ["config.json"])
|
self.assertEqual(downloaded, ["config.json"])
|
||||||
|
|
||||||
|
|||||||
@@ -5,6 +5,7 @@ import bpy
|
|||||||
|
|
||||||
GROUP_NAME = "O DA3 Surface"
|
GROUP_NAME = "O DA3 Surface"
|
||||||
DEPTH_PLANE_GROUP_NAME = "O DA3 Depth Plane"
|
DEPTH_PLANE_GROUP_NAME = "O DA3 Depth Plane"
|
||||||
|
MOGE_DEPTH_PLANE_GROUP_NAME = "O MoGe2 Depth Plane"
|
||||||
MATERIAL_DISPLACEMENT_ATTRIBUTE = "o_material_displacement"
|
MATERIAL_DISPLACEMENT_ATTRIBUTE = "o_material_displacement"
|
||||||
BACKGROUND_DEPTH_ATTRIBUTE = "o_background_depth"
|
BACKGROUND_DEPTH_ATTRIBUTE = "o_background_depth"
|
||||||
DEPTH_SCALE_ATTRIBUTE = "o_depth_scale"
|
DEPTH_SCALE_ATTRIBUTE = "o_depth_scale"
|
||||||
@@ -689,7 +690,7 @@ def build_depth_plane_group():
|
|||||||
"INPUT",
|
"INPUT",
|
||||||
"NodeSocketInt",
|
"NodeSocketInt",
|
||||||
)
|
)
|
||||||
subdivision.default_value = 5
|
subdivision.default_value = 6
|
||||||
subdivision.min_value = 0
|
subdivision.min_value = 0
|
||||||
subdivision.max_value = 10
|
subdivision.max_value = 10
|
||||||
displacement_strength = new_socket(
|
displacement_strength = new_socket(
|
||||||
@@ -708,7 +709,7 @@ def build_depth_plane_group():
|
|||||||
"INPUT",
|
"INPUT",
|
||||||
"NodeSocketFloat",
|
"NodeSocketFloat",
|
||||||
)
|
)
|
||||||
background_depth.default_value = -1.0
|
background_depth.default_value = 1.0
|
||||||
background_depth.min_value = -10000.0
|
background_depth.min_value = -10000.0
|
||||||
background_depth.max_value = 10000.0
|
background_depth.max_value = 10000.0
|
||||||
background_depth.subtype = "DISTANCE"
|
background_depth.subtype = "DISTANCE"
|
||||||
@@ -786,6 +787,14 @@ def build_depth_plane_group():
|
|||||||
(350.0, 250.0),
|
(350.0, 250.0),
|
||||||
{"Background Depth"},
|
{"Background Depth"},
|
||||||
)
|
)
|
||||||
|
background_depth_value = nodes.new("ShaderNodeMath")
|
||||||
|
background_depth_value.operation = "MULTIPLY"
|
||||||
|
background_depth_value.inputs[1].default_value = -1.0
|
||||||
|
background_depth_value.location = (550.0, 250.0)
|
||||||
|
links.new(
|
||||||
|
background_input.outputs["Background Depth"],
|
||||||
|
background_depth_value.inputs[0],
|
||||||
|
)
|
||||||
displacement_input = new_group_input(
|
displacement_input = new_group_input(
|
||||||
nodes,
|
nodes,
|
||||||
(950.0, -150.0),
|
(950.0, -150.0),
|
||||||
@@ -906,7 +915,7 @@ def build_depth_plane_group():
|
|||||||
clamp_foreground.location = (600.0, 650.0)
|
clamp_foreground.location = (600.0, 650.0)
|
||||||
links.new(depth_texture.outputs["Color"], clamp_foreground.inputs[0])
|
links.new(depth_texture.outputs["Color"], clamp_foreground.inputs[0])
|
||||||
links.new(
|
links.new(
|
||||||
background_input.outputs["Background Depth"],
|
background_depth_value.outputs[0],
|
||||||
clamp_foreground.inputs[1],
|
clamp_foreground.inputs[1],
|
||||||
)
|
)
|
||||||
foreground_mask = new_compare(
|
foreground_mask = new_compare(
|
||||||
@@ -925,7 +934,7 @@ def build_depth_plane_group():
|
|||||||
masked_depth.location = (800.0, 650.0)
|
masked_depth.location = (800.0, 650.0)
|
||||||
links.new(foreground_mask.outputs["Result"], masked_depth.inputs[0])
|
links.new(foreground_mask.outputs["Result"], masked_depth.inputs[0])
|
||||||
links.new(
|
links.new(
|
||||||
background_input.outputs["Background Depth"],
|
background_depth_value.outputs[0],
|
||||||
masked_depth.inputs[2],
|
masked_depth.inputs[2],
|
||||||
)
|
)
|
||||||
links.new(clamp_foreground.outputs[0], masked_depth.inputs[3])
|
links.new(clamp_foreground.outputs[0], masked_depth.inputs[3])
|
||||||
@@ -989,7 +998,7 @@ def build_depth_plane_group():
|
|||||||
(
|
(
|
||||||
BACKGROUND_DEPTH_ATTRIBUTE,
|
BACKGROUND_DEPTH_ATTRIBUTE,
|
||||||
"FLOAT",
|
"FLOAT",
|
||||||
attributes_input.outputs["Background Depth"],
|
background_depth_value.outputs[0],
|
||||||
),
|
),
|
||||||
(
|
(
|
||||||
DEPTH_SCALE_ATTRIBUTE,
|
DEPTH_SCALE_ATTRIBUTE,
|
||||||
@@ -1020,11 +1029,225 @@ def build_depth_plane_group():
|
|||||||
return group
|
return group
|
||||||
|
|
||||||
|
|
||||||
|
def build_moge_depth_plane_group():
|
||||||
|
group = bpy.data.node_groups.new(
|
||||||
|
MOGE_DEPTH_PLANE_GROUP_NAME,
|
||||||
|
"GeometryNodeTree",
|
||||||
|
)
|
||||||
|
group.is_modifier = True
|
||||||
|
interface = group.interface
|
||||||
|
new_socket(interface, "Geometry", "INPUT", "NodeSocketGeometry")
|
||||||
|
displacement = new_socket(
|
||||||
|
interface, "Displacement", "INPUT", "NodeSocketMenu"
|
||||||
|
)
|
||||||
|
subdivision = new_socket(
|
||||||
|
interface, "Subdivision", "INPUT", "NodeSocketInt"
|
||||||
|
)
|
||||||
|
subdivision.default_value = 6
|
||||||
|
subdivision.min_value = 0
|
||||||
|
subdivision.max_value = 10
|
||||||
|
depth_scale = new_socket(
|
||||||
|
interface, "Depth Scale", "INPUT", "NodeSocketFloat"
|
||||||
|
)
|
||||||
|
depth_scale.default_value = 1.0
|
||||||
|
depth_scale.structure_type = "SINGLE"
|
||||||
|
background_depth = new_socket(
|
||||||
|
interface, "Background Depth", "INPUT", "NodeSocketFloat"
|
||||||
|
)
|
||||||
|
background_depth.default_value = 1.0
|
||||||
|
background_depth.min_value = -10000.0
|
||||||
|
background_depth.max_value = 10000.0
|
||||||
|
background_depth.subtype = "DISTANCE"
|
||||||
|
background_depth.structure_type = "SINGLE"
|
||||||
|
data_panel = interface.new_panel(name="Data", default_closed=True)
|
||||||
|
depth_image = new_socket(
|
||||||
|
interface, "Depth Image", "INPUT", "NodeSocketImage", data_panel
|
||||||
|
)
|
||||||
|
material = new_socket(
|
||||||
|
interface, "Material", "INPUT", "NodeSocketMaterial", data_panel
|
||||||
|
)
|
||||||
|
for socket in (depth_image, material):
|
||||||
|
socket.hide_in_modifier = True
|
||||||
|
frame_sockets = []
|
||||||
|
for name, socket_type, default in (
|
||||||
|
("Frame", "NodeSocketInt", 1),
|
||||||
|
("Frame Start", "NodeSocketInt", 1),
|
||||||
|
("Frames", "NodeSocketInt", 1),
|
||||||
|
("Cyclic", "NodeSocketBool", False),
|
||||||
|
):
|
||||||
|
socket = new_socket(interface, name, "INPUT", socket_type, data_panel)
|
||||||
|
socket.default_value = default
|
||||||
|
socket.hide_in_modifier = True
|
||||||
|
frame_sockets.append(socket)
|
||||||
|
new_socket(interface, "Geometry", "OUTPUT", "NodeSocketGeometry")
|
||||||
|
|
||||||
|
nodes = group.nodes
|
||||||
|
links = group.links
|
||||||
|
geometry_input = new_group_input(nodes, (-1600.0, 650.0), {"Geometry"})
|
||||||
|
controls = new_group_input(
|
||||||
|
nodes,
|
||||||
|
(-1600.0, 150.0),
|
||||||
|
{"Subdivision", "Displacement", "Depth Scale", "Background Depth"},
|
||||||
|
)
|
||||||
|
background_depth_value = nodes.new("ShaderNodeMath")
|
||||||
|
background_depth_value.operation = "MULTIPLY"
|
||||||
|
background_depth_value.inputs[1].default_value = -1.0
|
||||||
|
background_depth_value.location = (-1350.0, -150.0)
|
||||||
|
links.new(
|
||||||
|
controls.outputs["Background Depth"],
|
||||||
|
background_depth_value.inputs[0],
|
||||||
|
)
|
||||||
|
images = new_group_input(
|
||||||
|
nodes, (-500.0, 1100.0), {"Depth Image"}
|
||||||
|
)
|
||||||
|
frames = new_group_input(
|
||||||
|
nodes,
|
||||||
|
(-500.0, 1500.0),
|
||||||
|
{"Frame", "Frame Start", "Frames", "Cyclic"},
|
||||||
|
)
|
||||||
|
material_input = new_group_input(nodes, (900.0, 200.0), {"Material"})
|
||||||
|
output = nodes.new("NodeGroupOutput")
|
||||||
|
output.location = (1700.0, 650.0)
|
||||||
|
|
||||||
|
bounds = nodes.new("GeometryNodeBoundBox")
|
||||||
|
bounds.location = (-1400.0, 650.0)
|
||||||
|
links.new(geometry_input.outputs["Geometry"], bounds.inputs["Geometry"])
|
||||||
|
dimensions = nodes.new("ShaderNodeVectorMath")
|
||||||
|
dimensions.operation = "SUBTRACT"
|
||||||
|
dimensions.location = (-1200.0, 650.0)
|
||||||
|
links.new(bounds.outputs["Max"], dimensions.inputs[0])
|
||||||
|
links.new(bounds.outputs["Min"], dimensions.inputs[1])
|
||||||
|
separate = nodes.new("ShaderNodeSeparateXYZ")
|
||||||
|
separate.location = (-1000.0, 650.0)
|
||||||
|
links.new(dimensions.outputs["Vector"], separate.inputs[0])
|
||||||
|
segments = nodes.new("ShaderNodeMath")
|
||||||
|
segments.operation = "POWER"
|
||||||
|
segments.inputs[0].default_value = 2.0
|
||||||
|
segments.location = (-1200.0, 100.0)
|
||||||
|
links.new(controls.outputs["Subdivision"], segments.inputs[1])
|
||||||
|
vertices = nodes.new("ShaderNodeMath")
|
||||||
|
vertices.operation = "ADD"
|
||||||
|
vertices.inputs[1].default_value = 1.0
|
||||||
|
vertices.location = (-1000.0, 100.0)
|
||||||
|
links.new(segments.outputs[0], vertices.inputs[0])
|
||||||
|
grid = nodes.new("GeometryNodeMeshGrid")
|
||||||
|
grid.location = (-750.0, 650.0)
|
||||||
|
links.new(separate.outputs["X"], grid.inputs["Size X"])
|
||||||
|
links.new(separate.outputs["Y"], grid.inputs["Size Y"])
|
||||||
|
links.new(vertices.outputs[0], grid.inputs["Vertices X"])
|
||||||
|
links.new(vertices.outputs[0], grid.inputs["Vertices Y"])
|
||||||
|
store_uv = nodes.new("GeometryNodeStoreNamedAttribute")
|
||||||
|
store_uv.data_type = "FLOAT2"
|
||||||
|
store_uv.domain = "CORNER"
|
||||||
|
store_uv.inputs["Name"].default_value = "UVMap"
|
||||||
|
store_uv.location = (-500.0, 650.0)
|
||||||
|
links.new(grid.outputs["Mesh"], store_uv.inputs["Geometry"])
|
||||||
|
links.new(grid.outputs["UV Map"], store_uv.inputs["Value"])
|
||||||
|
|
||||||
|
center = nodes.new("ShaderNodeVectorMath")
|
||||||
|
center.operation = "ADD"
|
||||||
|
center.location = (-1000.0, 900.0)
|
||||||
|
links.new(bounds.outputs["Min"], center.inputs[0])
|
||||||
|
links.new(bounds.outputs["Max"], center.inputs[1])
|
||||||
|
half = nodes.new("ShaderNodeVectorMath")
|
||||||
|
half.operation = "SCALE"
|
||||||
|
half.inputs[3].default_value = 0.5
|
||||||
|
half.location = (-750.0, 900.0)
|
||||||
|
links.new(center.outputs["Vector"], half.inputs[0])
|
||||||
|
transform = nodes.new("GeometryNodeTransform")
|
||||||
|
transform.location = (-250.0, 650.0)
|
||||||
|
links.new(store_uv.outputs["Geometry"], transform.inputs["Geometry"])
|
||||||
|
links.new(half.outputs["Vector"], transform.inputs["Translation"])
|
||||||
|
|
||||||
|
depth_texture = nodes.new("GeometryNodeImageTexture")
|
||||||
|
depth_texture.interpolation = "Linear"
|
||||||
|
depth_texture.extension = "EXTEND"
|
||||||
|
depth_texture.location = (-250.0, 1100.0)
|
||||||
|
links.new(images.outputs["Depth Image"], depth_texture.inputs["Image"])
|
||||||
|
links.new(grid.outputs["UV Map"], depth_texture.inputs["Vector"])
|
||||||
|
frame = build_sequence_frame(
|
||||||
|
nodes, links, frames.outputs["Frame"], frames, 1800.0
|
||||||
|
)
|
||||||
|
links.new(frame, depth_texture.inputs["Frame"])
|
||||||
|
|
||||||
|
clamp_depth = nodes.new("ShaderNodeMath")
|
||||||
|
clamp_depth.operation = "MAXIMUM"
|
||||||
|
clamp_depth.location = (0.0, 750.0)
|
||||||
|
links.new(depth_texture.outputs["Color"], clamp_depth.inputs[0])
|
||||||
|
links.new(background_depth_value.outputs[0], clamp_depth.inputs[1])
|
||||||
|
displacement_menu = nodes.new("GeometryNodeMenuSwitch")
|
||||||
|
displacement_menu.data_type = "FLOAT"
|
||||||
|
displacement_menu.enum_items.clear()
|
||||||
|
displacement_menu.enum_items.new("Geometry")
|
||||||
|
displacement_menu.enum_items.new("Material")
|
||||||
|
displacement_menu.inputs["Geometry"].default_value = 1.0
|
||||||
|
displacement_menu.inputs["Material"].default_value = 0.0
|
||||||
|
displacement_menu.location = (0.0, 300.0)
|
||||||
|
links.new(controls.outputs["Displacement"], displacement_menu.inputs["Menu"])
|
||||||
|
displacement.default_value = "Geometry"
|
||||||
|
material_mode = nodes.new("ShaderNodeMath")
|
||||||
|
material_mode.operation = "SUBTRACT"
|
||||||
|
material_mode.inputs[0].default_value = 1.0
|
||||||
|
material_mode.location = (250.0, 250.0)
|
||||||
|
links.new(displacement_menu.outputs[0], material_mode.inputs[1])
|
||||||
|
geometry_strength = nodes.new("ShaderNodeMath")
|
||||||
|
geometry_strength.operation = "MULTIPLY"
|
||||||
|
geometry_strength.location = (250.0, 500.0)
|
||||||
|
links.new(displacement_menu.outputs[0], geometry_strength.inputs[0])
|
||||||
|
links.new(controls.outputs["Depth Scale"], geometry_strength.inputs[1])
|
||||||
|
scaled_depth = nodes.new("ShaderNodeMath")
|
||||||
|
scaled_depth.operation = "MULTIPLY"
|
||||||
|
scaled_depth.location = (250.0, 750.0)
|
||||||
|
links.new(clamp_depth.outputs[0], scaled_depth.inputs[0])
|
||||||
|
links.new(geometry_strength.outputs[0], scaled_depth.inputs[1])
|
||||||
|
normal = nodes.new("GeometryNodeInputNormal")
|
||||||
|
normal.location = (0.0, 500.0)
|
||||||
|
offset = nodes.new("ShaderNodeVectorMath")
|
||||||
|
offset.operation = "SCALE"
|
||||||
|
offset.location = (250.0, 700.0)
|
||||||
|
links.new(normal.outputs["Normal"], offset.inputs[0])
|
||||||
|
links.new(scaled_depth.outputs[0], offset.inputs[3])
|
||||||
|
set_position = nodes.new("GeometryNodeSetPosition")
|
||||||
|
set_position.location = (500.0, 650.0)
|
||||||
|
links.new(transform.outputs["Geometry"], set_position.inputs["Geometry"])
|
||||||
|
links.new(offset.outputs["Vector"], set_position.inputs["Offset"])
|
||||||
|
|
||||||
|
set_material = nodes.new("GeometryNodeSetMaterial")
|
||||||
|
set_material.location = (750.0, 650.0)
|
||||||
|
links.new(set_position.outputs["Geometry"], set_material.inputs["Geometry"])
|
||||||
|
links.new(material_input.outputs["Material"], set_material.inputs["Material"])
|
||||||
|
smooth = nodes.new("GeometryNodeSetShadeSmooth")
|
||||||
|
smooth.domain = "FACE"
|
||||||
|
smooth.inputs["Shade Smooth"].default_value = True
|
||||||
|
smooth.location = (1000.0, 650.0)
|
||||||
|
links.new(set_material.outputs["Geometry"], smooth.inputs["Geometry"])
|
||||||
|
geometry = smooth.outputs["Geometry"]
|
||||||
|
for index, (name, data_type, value) in enumerate((
|
||||||
|
(MATERIAL_DISPLACEMENT_ATTRIBUTE, "BOOLEAN", material_mode.outputs[0]),
|
||||||
|
(BACKGROUND_DEPTH_ATTRIBUTE, "FLOAT", background_depth_value.outputs[0]),
|
||||||
|
(DEPTH_SCALE_ATTRIBUTE, "FLOAT", controls.outputs["Depth Scale"]),
|
||||||
|
)):
|
||||||
|
store = nodes.new("GeometryNodeStoreNamedAttribute")
|
||||||
|
store.data_type = data_type
|
||||||
|
store.domain = "POINT"
|
||||||
|
store.inputs["Name"].default_value = name
|
||||||
|
store.location = (1200.0 + index * 200.0, 650.0)
|
||||||
|
links.new(geometry, store.inputs["Geometry"])
|
||||||
|
links.new(value, store.inputs["Value"])
|
||||||
|
geometry = store.outputs["Geometry"]
|
||||||
|
links.new(geometry, output.inputs["Geometry"])
|
||||||
|
group.color_tag = "GEOMETRY"
|
||||||
|
group.description = "Build a MoGe-2 depth plane with geometry or material displacement."
|
||||||
|
group.use_fake_user = True
|
||||||
|
return group
|
||||||
|
|
||||||
|
|
||||||
def main():
|
def main():
|
||||||
bpy.ops.wm.read_factory_settings(use_empty=True)
|
bpy.ops.wm.read_factory_settings(use_empty=True)
|
||||||
bpy.context.preferences.filepaths.save_version = 0
|
bpy.context.preferences.filepaths.save_version = 0
|
||||||
build_group()
|
build_group()
|
||||||
build_depth_plane_group()
|
build_depth_plane_group()
|
||||||
|
build_moge_depth_plane_group()
|
||||||
OUTPUT_PATH.parent.mkdir(parents=True, exist_ok=True)
|
OUTPUT_PATH.parent.mkdir(parents=True, exist_ok=True)
|
||||||
bpy.ops.wm.save_as_mainfile(filepath=str(OUTPUT_PATH), check_existing=False)
|
bpy.ops.wm.save_as_mainfile(filepath=str(OUTPUT_PATH), check_existing=False)
|
||||||
|
|
||||||
|
|||||||
@@ -29,7 +29,15 @@ def main():
|
|||||||
)
|
)
|
||||||
assert background_depth.subtype == "DISTANCE"
|
assert background_depth.subtype == "DISTANCE"
|
||||||
assert background_depth.structure_type == "SINGLE"
|
assert background_depth.structure_type == "SINGLE"
|
||||||
assert background_depth.default_value == -1.0
|
assert background_depth.default_value == 1.0
|
||||||
|
subdivision = next(
|
||||||
|
item
|
||||||
|
for item in group.interface.items_tree
|
||||||
|
if item.item_type == "SOCKET"
|
||||||
|
and item.in_out == "INPUT"
|
||||||
|
and item.name == "Subdivision"
|
||||||
|
)
|
||||||
|
assert subdivision.default_value == 6
|
||||||
frame_input = next(
|
frame_input = next(
|
||||||
item
|
item
|
||||||
for item in group.interface.items_tree
|
for item in group.interface.items_tree
|
||||||
|
|||||||
79
tools/validate_moge_depth_plane.py
Normal file
79
tools/validate_moge_depth_plane.py
Normal file
@@ -0,0 +1,79 @@
|
|||||||
|
import sys
|
||||||
|
import tempfile
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
import bpy
|
||||||
|
import numpy as np
|
||||||
|
|
||||||
|
|
||||||
|
PROJECT_ROOT = Path(__file__).parents[1]
|
||||||
|
sys.path.insert(0, str(PROJECT_ROOT / "src"))
|
||||||
|
|
||||||
|
from snapform.image_depth import (
|
||||||
|
MOGE_DEPTH_PLANE_NODE_GROUP_NAME,
|
||||||
|
MOGE_MATERIAL_NODE_GROUP_NAME,
|
||||||
|
build_moge2_depth_plane,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def main():
|
||||||
|
bpy.ops.wm.read_factory_settings(use_empty=True)
|
||||||
|
image = bpy.data.images.new("MoGe Test", width=64, height=48, alpha=True)
|
||||||
|
source = bpy.data.objects.new("MoGe Test", None)
|
||||||
|
source.empty_display_type = "IMAGE"
|
||||||
|
source.data = image
|
||||||
|
bpy.context.collection.objects.link(source)
|
||||||
|
depth = np.linspace(1.0, 2.0, 48 * 64, dtype=np.float32).reshape(1, 48, 64)
|
||||||
|
normal = np.zeros((1, 48, 64, 3), dtype=np.float32)
|
||||||
|
normal[..., 2] = -1.0
|
||||||
|
mask = np.ones((1, 48, 64), dtype=bool)
|
||||||
|
mask[:, :5] = False
|
||||||
|
intrinsics = np.array(
|
||||||
|
[[[1.0, 0.0, 0.5], [0.0, 1.0, 0.5], [0.0, 0.0, 1.0]]],
|
||||||
|
dtype=np.float32,
|
||||||
|
)
|
||||||
|
with tempfile.TemporaryDirectory() as temporary:
|
||||||
|
path = Path(temporary) / "results.npz"
|
||||||
|
np.savez(
|
||||||
|
path,
|
||||||
|
depth=depth,
|
||||||
|
normal=normal,
|
||||||
|
mask=mask,
|
||||||
|
intrinsics=intrinsics,
|
||||||
|
)
|
||||||
|
plane = build_moge2_depth_plane(
|
||||||
|
bpy.context,
|
||||||
|
source,
|
||||||
|
path,
|
||||||
|
delete_source=True,
|
||||||
|
)
|
||||||
|
assert plane.modifiers[0].node_group.name == MOGE_DEPTH_PLANE_NODE_GROUP_NAME
|
||||||
|
assert MOGE_MATERIAL_NODE_GROUP_NAME in bpy.data.node_groups
|
||||||
|
geometry_group = plane.modifiers[0].node_group
|
||||||
|
inputs = {
|
||||||
|
item.name
|
||||||
|
for item in geometry_group.interface.items_tree
|
||||||
|
if getattr(item, "in_out", None) == "INPUT"
|
||||||
|
}
|
||||||
|
assert "Displacement" in inputs
|
||||||
|
assert "Background Depth" in inputs
|
||||||
|
assert "Mask Image" not in inputs
|
||||||
|
assert "Use Mask" not in inputs
|
||||||
|
sockets = {
|
||||||
|
item.name: item
|
||||||
|
for item in geometry_group.interface.items_tree
|
||||||
|
if getattr(item, "in_out", None) == "INPUT"
|
||||||
|
}
|
||||||
|
assert sockets["Subdivision"].default_value == 6
|
||||||
|
assert sockets["Depth Scale"].structure_type == "SINGLE"
|
||||||
|
assert sockets["Background Depth"].structure_type == "SINGLE"
|
||||||
|
assert sockets["Background Depth"].subtype == "DISTANCE"
|
||||||
|
assert sockets["Background Depth"].default_value == 1.0
|
||||||
|
material_group = bpy.data.node_groups[MOGE_MATERIAL_NODE_GROUP_NAME]
|
||||||
|
assert not any(node.bl_idname == "ShaderNodeNormalMap" for node in material_group.nodes)
|
||||||
|
assert any(node.bl_idname == "ShaderNodeBump" for node in material_group.nodes)
|
||||||
|
print("MoGe-2 depth plane validation passed")
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
main()
|
||||||
@@ -13,6 +13,20 @@ filename = "ben2.tar.gz"
|
|||||||
url = "https://codeload.github.com/PramaLLC/BEN2/tar.gz/2c99a5da477b5523585bfa5c893888a6e818a8f6"
|
url = "https://codeload.github.com/PramaLLC/BEN2/tar.gz/2c99a5da477b5523585bfa5c893888a6e818a8f6"
|
||||||
sha256 = "01fd26716f3ff3a3f18ad2336c768af5b759edfc7c38bb3bf733b7c86eee6282"
|
sha256 = "01fd26716f3ff3a3f18ad2336c768af5b759edfc7c38bb3bf733b7c86eee6282"
|
||||||
|
|
||||||
|
[artifacts.moge]
|
||||||
|
revision = "925b8ed835a7a9cdb7578ba15c658a0afc969030"
|
||||||
|
filename = "moge.tar.gz"
|
||||||
|
url = "https://codeload.github.com/microsoft/MoGe/tar.gz/925b8ed835a7a9cdb7578ba15c658a0afc969030"
|
||||||
|
sha256 = "c6d90b2564d643f0f3ae65b5ffe163abaea3b55d194c6e2ba9976d67f552bd26"
|
||||||
|
patches = "moge"
|
||||||
|
|
||||||
|
[artifacts.utils3d]
|
||||||
|
revision = "3fab839f0be9931dac7c8488eb0e1600c236e183"
|
||||||
|
filename = "utils3d.tar.gz"
|
||||||
|
url = "https://codeload.github.com/EasternJournalist/utils3d/tar.gz/3fab839f0be9931dac7c8488eb0e1600c236e183"
|
||||||
|
sha256 = "f1f167b1f68809bea391a66dd0d2ad0e14f7382cbddf087b655181c31a285abb"
|
||||||
|
patches = "utils3d"
|
||||||
|
|
||||||
[uv]
|
[uv]
|
||||||
version = "0.11.3"
|
version = "0.11.3"
|
||||||
|
|
||||||
|
|||||||
Reference in New Issue
Block a user