Compare commits
10 Commits
a6370d678d
...
fb78a8068b
| Author | SHA1 | Date | |
|---|---|---|---|
| fb78a8068b | |||
| 1c98b21235 | |||
| c5b77c5c85 | |||
| b3a07cde32 | |||
| 0c8d3a8988 | |||
| 102e1bfa9b | |||
| 5255eb4e96 | |||
| 2ad30bf4a3 | |||
| 13fab9ce88 | |||
| caeb6d706c |
3
.github/workflows/build.yml
vendored
3
.github/workflows/build.yml
vendored
@@ -24,9 +24,6 @@ jobs:
|
||||
- name: 安装测试依赖
|
||||
run: python -m pip install numpy pytest
|
||||
|
||||
- name: 准备 Vendor
|
||||
run: python prepare_vendor.py
|
||||
|
||||
- name: 运行测试
|
||||
run: python -m pytest
|
||||
|
||||
|
||||
3
.gitignore
vendored
3
.gitignore
vendored
@@ -7,4 +7,5 @@ build/
|
||||
*.zip
|
||||
.pytest_cache/
|
||||
|
||||
/vendor/
|
||||
/models/
|
||||
wheels/
|
||||
97
AGENTS.md
97
AGENTS.md
@@ -1,32 +1,81 @@
|
||||
## 总规范
|
||||
# SnapForm 开发约定
|
||||
|
||||
- 说明性内容(例如提交说明、文档、注释、mermaid),以中文为主,专有名词、约定俗称可用英文
|
||||
## 总则
|
||||
|
||||
## Markdown 规范
|
||||
- 说明性内容(提交说明、文档、注释、Mermaid 等)以中文为主,专有名词和约定俗成的表达可使用英文
|
||||
- 动手前先读 `docs/index.md`;程序边界、调用流程和模块职责见 `docs/development.md#程序设计`
|
||||
- 修改代码、依赖或执行方式后,同步更新 `README.md` 与对应主题文档,保持文档、测试和实现一致
|
||||
- 优先选择依赖少、直接、易验证的实现;保留工作区中与当前任务无关的已有改动
|
||||
|
||||
- 不使用`---`分割器
|
||||
- `# Heading` 一级标题仅用于开头
|
||||
- mermaid 节点名用英文字符
|
||||
## 项目结构
|
||||
|
||||
```text
|
||||
.
|
||||
├── README.md # 用户入口与快速上手
|
||||
├── docs/ # 按功能拆分的设计与开发文档
|
||||
├── pack.py # 构建各平台 Blender Extension ZIP
|
||||
├── sync_models.py # 准备并同步 ONNX 模型
|
||||
├── src/snapform/
|
||||
│ ├── __init__.py # Blender 注册入口
|
||||
│ ├── operators.py # Job Operator 与结果导入
|
||||
│ ├── panel.py # 3D View 面板
|
||||
│ ├── preferences.py # Extension Preferences
|
||||
│ ├── properties.py # Scene 设置与动态枚举
|
||||
│ ├── job_runtime.py # BlendJob Runtime 配置
|
||||
│ ├── blender/ # Depth Plane、Surface 的 Blender 侧实现
|
||||
│ ├── server/ # Job、模型、ONNX 推理与输出实现
|
||||
│ ├── assets/O_SnapForm.blend # Geometry Nodes 与材质资产
|
||||
│ └── wheels/ # 随扩展分发的 BlendJob wheel
|
||||
├── tools/ # Blender 资产构建与验证脚本
|
||||
└── tests/ # Python 自动化测试
|
||||
```
|
||||
|
||||
文档按主题定位:功能与使用 `docs/features.md`、Depth Plane `docs/depth-plane.md`、Surface `docs/surface.md`、输出格式 `docs/outputs.md`、Job Server `docs/server.md`、程序设计与开发 `docs/development.md`。
|
||||
|
||||
## 常见命令
|
||||
|
||||
优先在项目根目录使用 Git Bash;Windows 路径在 Bash 命令中使用正斜杠。
|
||||
|
||||
```bash
|
||||
# 安装或同步开发依赖
|
||||
uv sync
|
||||
|
||||
# 运行全部测试
|
||||
uv run pytest
|
||||
|
||||
# 运行插件注册、Manifest 与打包相关的快速测试
|
||||
uv run pytest tests/test_addon.py tests/test_manifest.py tests/test_pack.py -v
|
||||
|
||||
# 构建 Windows、macOS、Linux 扩展包到 dist/
|
||||
uv run python pack.py
|
||||
|
||||
# 本地 BlendJob wheel 内容变化后刷新锁文件
|
||||
uv lock --refresh
|
||||
```
|
||||
|
||||
资产脚本必须由 Blender 执行;具体入口与验证范围见 `docs/development.md#资产构建与验证`,不要用普通 Python 解释器运行。
|
||||
|
||||
## 命名规范
|
||||
|
||||
- 变量、函数、Docstring 使用英文;同一概念在代码和文档中使用统一表达
|
||||
- Python 类使用大驼峰,不添加项目名前缀。Operator 使用动宾结构,如 `ConvertToDepthPlane`、`RemoveImageBackground`;Menu、Panel、AddonPreferences、PropertyGroup 分别使用实际类型后缀,如 `SnapFormImageMenu`、`ServerPanel`、`SnapFormPreferences`、`SnapFormSettings`
|
||||
- `bl_idname` 与挂载到公共 Blender RNA 类型的 Property 使用 `snapform` 项目前缀:Operator 如 `snapform.convert_to_depth_plane`,Menu 如 `SNAPFORM_MT_image_context`,Panel 如 `SNAPFORM_PT_server`,公共 Property 如 `Scene.snapform_settings`
|
||||
- Operator 自身 Property 不加项目前缀,使用业务名称,如 `input_path`、`process_res`、`resolution_level`
|
||||
- 扩展包名使用 Manifest 中的真实名称、版本和平台,例如 `SnapForm.v0.11.2.windows-x64.zip`;节点组名称沿用资产中的 `O DA3 Depth Plane`、`O MoGe2 Surface` 等既有名称
|
||||
|
||||
## 代码规范
|
||||
|
||||
- 避免嵌套结构
|
||||
- 特殊的、非常规的需要写注释
|
||||
- 变量、函数名、Docstring 用全英文
|
||||
- 同一事物用统一表达
|
||||
- 使用的依赖越少越好
|
||||
- 选用简单直接的方式实现
|
||||
- 代码、执行方式改变,调整已有的文档、计划,而非新建
|
||||
- 避免不必要的嵌套,优先使用提前返回和小而明确的函数
|
||||
- 只为特殊、非常规或存在兼容性原因的实现写注释,不复述代码本身
|
||||
- 删除功能时同时删除无效参数、分支、测试和文档,不保留没有实际消费者的兼容层
|
||||
- Blender 侧不载入 AI 依赖:UI、Operator 和结果导入放在 `src/snapform/` 或 `src/snapform/blender/`,模型下载、ONNX 推理和输出放在 `src/snapform/server/`
|
||||
- Job 参数使用可序列化的普通字典;Blender 数据只能在主线程的 Operator 响应阶段读写
|
||||
- 新增或修改 Blender 类型时同步检查 `src/snapform/__init__.py` 的注册与逆序注销,并补充对应测试
|
||||
- 依赖版本、wheel 或 Manifest 改变时同步检查 `pyproject.toml`、`uv.lock`、`blender_manifest.toml` 和打包测试
|
||||
|
||||
## Shell 规范
|
||||
## 文档与 Markdown 规范
|
||||
|
||||
- 优先使用、提供 Git Bash 执行所有 Shell 命令
|
||||
- Windows 路径在 Bash 命令中优先使用正斜杠形式
|
||||
|
||||
## Blender 开发规范
|
||||
|
||||
- 插件、资产库压缩包名用大驼峰,如果需要,用`b45`来表示适用blender版本。比如`OmooLab.v0.1.0.b45.zip`
|
||||
- Operator、Menu、Panel、AddonPreferences... Class 用大驼峰命名,不加任何前缀。Operator 动宾结构,比如`RenderImage`。Menu、Panel 以它们本身为后缀,比如`RenderPanel`
|
||||
- 根据项目名提炼项目缩写,作为 bl_idname 和 自定义 Property 的前缀以区分其他功能和属性,以`Omoo`为例
|
||||
- bl_idname 用下划线小写命名,Operator 的 bl_idname 以`omoo.*`为前缀,比如`omoo.render_panel`;Menu 的以`OMOO_MT_*`为前缀;Panel 的以`OMOO_PT_*`为前缀。比如`OMOO_PT_render_panel`
|
||||
- 自定义 Property 用下划线小写命名,以`omoo_*`为前缀。比如`omoo_progress_factor`
|
||||
- `README.md` 面向用户;实现原理放入 `docs/` 对应主题,不在 README 堆叠内部细节
|
||||
- 不使用 `---` 分隔线;一级标题只在文档开头使用一次
|
||||
- Mermaid 节点 ID 使用英文字符,节点显示文本可以使用中文
|
||||
- 文档描述当前已实现行为;限制、默认值、命令和文件名必须能从代码或测试中验证
|
||||
|
||||
52
README.md
52
README.md
@@ -16,12 +16,18 @@ SnapForm 是一个 Blender 扩展,可以把一张照片、一段视频或一
|
||||
2. 打开 Blender,进入 `Preferences > Extensions`
|
||||
3. 点击 `Install from Disk`,选择下载的压缩包
|
||||
4. 在扩展偏好设置中确认 Storage Root 与 Device(保持默认即可);Device 默认取本机
|
||||
最快的 CUDA/MPS,否则 CPU。下载超时会自动重试国内镜像,无需手动配置
|
||||
最快的 DirectML/CoreML provider,否则 CPU。下载超时会自动重试国内镜像,无需手动配置
|
||||
5. 打开 3D 视图右侧的 `SnapForm` 面板,点击 `Install Environment`
|
||||
|
||||
DA3、MoGe-2、utils3d、BEN2 源码包和 uv 已内置在扩展里,安装时直接从本地解包,不需要访问 GitHub。首次使用仍会自动
|
||||
下载运行环境依赖(PyTorch 等),随后自动下载 `DA3 Small` 和 `BEN2 Base`。其他 AI 模型可在
|
||||
扩展不内置 uv 或 AI 运行库。首次使用会先下载并校验固定版本的 uv,随后自动安装轻量的 ONNX Runtime 环境,再下载 `MoGe-2 ViT-S Normal` 和
|
||||
`BEN2 Base`;不安装 PyTorch,也不下载各模型的 Python 源码包。其他 AI 模型可在
|
||||
`Models` panel 中按需下载,请保持网络畅通,首次安装需要一些时间。
|
||||
安装时 Environment、MoGe-2 和 BEN2 各自使用独立的 0–100% 进度;Environment 这一轮内部依次完成 uv、Python 环境和 packages。进入后续模型时进度归零,但不显示 `2/2` 等阶段编号。模型源发生瞬时错误时会切换到另一来源继续下载。
|
||||
任一必需模型在两个来源都下载失败时,`Install Environment` 会明确报错;Environment 本身仍会保留。
|
||||
选中 Image Empty 后,右键 `SnapForm` 子菜单会在缺少 `MoGe-2 ViT-S Normal` 或 `BEN2 Base`
|
||||
时显示 `Download Required Models`,一次补齐缺失项,不会重复下载已经就绪的模型。
|
||||
必需模型补齐前,菜单中的 `Convert to Depth Plane` 会保持禁用。
|
||||
这些 UI 状态直接检查 Storage Root 的 `models/` 文件,不会仅因打开菜单或面板而启动 Job Server。
|
||||
|
||||
## 快速上手
|
||||
|
||||
@@ -44,40 +50,52 @@ DA3、MoGe-2、utils3d、BEN2 源码包和 uv 已内置在扩展里,安装时
|
||||
|
||||
### 转换为透视网格
|
||||
|
||||
选中单张图片的 Image Empty,右键打开 `SnapForm` 子菜单并选择 `Convert to Surface`。SnapForm 会按 DA3
|
||||
相机内参与深度逐像素反投影,并把生成网格匹配到原 Empty 的可见尺寸、Offset 与 Transform。DA3
|
||||
confidence 与边拉伸结果会保存为 Surface attributes,再由一个可复用的 Geometry Nodes
|
||||
选中单张图片的 Image Empty,右键打开 `SnapForm` 子菜单并选择 `Convert to Surface`。可选择 MoGe-2
|
||||
ViT-S、ViT-B、ViT-L 或 DA3 Small、Base,其中 MoGe-2 ViT-S 为默认模型。SnapForm 会按模型输出的
|
||||
相机内参与三维点建立透视网格,并匹配原 Empty 的可见尺寸、Offset 与 Transform。MoGe-2 使用内置 NumPy
|
||||
网格构建与深度断层 mask 生成干净拓扑;DA3 保留 confidence 与边拉伸剔除。两者再由独立的 Geometry Nodes
|
||||
修改器完成剔面、快速重拓扑、Relax 与 UV 重投。开启 Remove Background 时,背景三角面会在转换期间
|
||||
直接删除,不再交给 Geometry Nodes。匹配相机默认不创建;开启后会作为 Surface 的子级。
|
||||
MoGe-2 的像素级 normal 会作为 Object Space normal 贴图接入 `O MoGe2 Surface Layer`,并与原图
|
||||
Bump 合并;网格本身不写入 custom normal,因此重拓扑降低顶点数后仍能保留细腻的法线细节。
|
||||
当前此功能仅支持单张图片。
|
||||
|
||||
转换对话框提供模型与 `Resolution Scale`、`Create Matching Camera`、`Remove Background`、`Unshaded`,
|
||||
MoGe-2 ViT-S、ViT-B、ViT-L、DA3 Small、DA3 Base 与 BEN2 全部使用 ONNX。模型执行、相机与深度恢复、
|
||||
点云重投影和网格生成均不依赖 PyTorch。Windows 使用 DirectML 加速,provider 不可用时会回退到 ONNX Runtime CPU,
|
||||
并在任务状态中显示实际 provider。
|
||||
|
||||
转换对话框提供模型与 `Create Matching Camera`、`Remove Background`、`Unshaded`;DA3 显示
|
||||
`Resolution Scale`,MoGe-2 显示 `Resolution Level`,
|
||||
以及分割线下方的 `Keep Original`。
|
||||
界面会显示计算所得的实际 Process Res。剔除和重拓扑参数直接在
|
||||
Geometry Nodes 修改器中调整;`Unshaded` 可使用与 Depth Plane 相同的 Emission 材质。普通材质的
|
||||
Bump Scale 会在材质组内乘以 `0.001`,并按透视表面的深度空间缩放。Surface attributes 统一使用 `o_` 前缀;Service 准备过程显示
|
||||
Bump Scale 会在材质组内乘以 `0.001`,并按透视表面的深度空间缩放。Surface attributes 统一使用 `o_` 前缀;Server 准备过程显示
|
||||
`Preparing surface data n/6`,可看到当前阶段与总步骤。
|
||||
|
||||
### 视频与序列帧
|
||||
|
||||
SnapForm 支持视频(mp4、mkv、mov 等)和序列帧,会逐帧处理并生成逐帧对应的立体面片。处理范围默认
|
||||
跟随 Empty 上设置的 `Image Frames` 与 `Offset`,不需要额外配置。
|
||||
跟随 Empty 上设置的 `Image Frames` 与 `Offset`,不需要额外配置。DA3 的转换任务复用同一个 ONNX
|
||||
Session 逐帧推理,显存峰值不随视频帧数平方增长;Debug 多帧输出仍使用多视图联合推理。
|
||||
|
||||
## 任务结果保存在哪里
|
||||
|
||||
任务结果保存在 `~/.snapform/jobs/` 下,每个任务一个文件夹,名称统一为
|
||||
`动作-对象_日期-时间-微秒`。Packed Image 和序列帧的输入副本会直接归档到对应任务根目录,
|
||||
不再使用公共 `inputs/` 文件夹;导出产物也尽量平铺。视频或序列帧生成的面片会引用任务文件夹里的
|
||||
序列帧,请保留对应文件夹,不要随意删除。
|
||||
任务结果保存在 Runtime Storage Root 的 `jobs/` 下,默认是 `~/.snapform/jobs/`。Job Server 接受任务时自动生成 opaque Job ID,并创建同名文件夹;Blender 不预先申请 ID 或目录。Packed Image 和序列帧的临时输入由 Blender 在任务结束后清理,Server 产物直接写入该 Job 目录。视频或序列帧生成的面片会引用其中的序列帧,请保留对应文件夹,不要随意删除。
|
||||
|
||||
## 常见问题
|
||||
|
||||
- 首次使用需要下载模型,如果下载慢或失败,稍后在 `Models` panel 里重试即可
|
||||
- 安装 Environment 时提示下载超时:新版会自动探测并切换到镜像、失败时自动重试一次;如果仍然
|
||||
超时,在扩展偏好设置的 `PyPI Index` 中填入镜像地址(如清华)后重新安装即可
|
||||
- 从 0.12.0–0.12.2 升级到 0.12.3 后,请重新执行一次 `Install Environment`,让 Storage Root 中的 Environment 更新 BlendJob 运行依赖
|
||||
- 首次使用需要下载模型,如果下载慢或失败,可在 Image Empty 的 `SnapForm` 右键菜单点击
|
||||
`Download Required Models`,也可在 `Models` panel 里逐项重试
|
||||
- 安装 Environment 时 uv 下载失败会自动尝试 OmooLab 镜像;Python packages 需要指定镜像时设置
|
||||
`UV_INDEX_URL` 后重新安装
|
||||
- 默认模型下载会在 Hugging Face 与 OmooLab 镜像之间自动回退;某一来源暂时断线不会阻止后续模型尝试另一来源
|
||||
- 模型直链建连超过 10 秒,或连接后连续 60 秒没有收到数据,会切换到下一来源;不限制总下载时长
|
||||
- 找不到右键入口时,请确认选中了 Empty,且显示方式为 Image
|
||||
- 动画面片不动:检查 Empty 的 `Image Frames` 是否大于 1,并确认播放场景帧
|
||||
|
||||
## 技术文档
|
||||
|
||||
面向开发者的技术内容见 [docs/technical.md](docs/technical.md)。
|
||||
Blender 侧只负责界面、任务提交与结果导入,AI 与文件输出由独立 Job Server 执行。详细功能见
|
||||
[文档索引](docs/index.md),模块边界、调用流程、项目结构与开发命令见
|
||||
[程序设计、开发与许可证](docs/development.md)。
|
||||
|
||||
67
docs/depth-plane.md
Normal file
67
docs/depth-plane.md
Normal file
@@ -0,0 +1,67 @@
|
||||
# Depth Plane
|
||||
|
||||
## Image Empty 转深度面片
|
||||
|
||||
1. 在 3D View 选中 `Object Type = Empty`、`Display As = Image` 的对象。
|
||||
2. 右键打开 `SnapForm` 子菜单并选择 `Convert to Depth Plane`,通过 `Depth from` 选择深度来源:
|
||||
- `DA3`:显示 Model 和 Resolution Scale。ONNX 模型使用 camera decoder 恢复相机。Process Res 先以原图长边乘以
|
||||
Resolution Scale,再钳制到 `252–1008`,最后对齐到最接近的 `14` 倍数。Resolution Scale 以百分比
|
||||
显示,范围为 `1–100%`,默认 `100%`;界面只在标签中实时显示最终 Process Res。
|
||||
转换导出 `mini_npz`,使用前需要在
|
||||
`Models` panel 下载所选模型,并固定使用 `process_res_method="upper_bound_resize"`;设备和参考视图策略
|
||||
继续使用场景设置。Model 下拉菜单只列出已经下载的 DA3 模型;若没有可用模型,对话框会引导到
|
||||
`Models` panel 下载。
|
||||
- `MoGe-2`:使用所选模型输出 metric depth、normal、validity mask 与 normalized intrinsics。Depth Plane 不保存体积较大的 points;Surface 会在 AI 进程内用内置 NumPy 网格构建生成 `mesh.npz`。Debug 始终保留原始 `results.npz`,并按 Output Format 单独生成 Maps、纹理 GLB 或彩色 PLY。
|
||||
- `Original`:直接把原图作为深度贴图,不显示估算参数。仅在开启 `Remove Background` 时运行 BEN2
|
||||
取得前景 Alpha;视频或序列帧按需同时抽取原始帧与前景序列,把原始帧作为 `.depth` 序列。
|
||||
`MoGe-2 ViT-S` 是默认深度来源。选择 `DA3` 时,其 Model 与 Resolution Scale
|
||||
参数会集中显示在独立边框中。`Remove Background` 与 `Unshaded` 直接显示,不再使用 Options 折叠;
|
||||
`Keep Original` 默认关闭,并在分割线下方保持为最后一项。仅当开启 `Remove Background` 时才需要
|
||||
`BEN2 Base`,并把 `.foreground` 的 Alpha 作为独立
|
||||
Mask,不在 Server 中改写深度值。保存 `.foreground` 时保留 BEN2 Alpha,但 RGB 使用未经修改的原始输入;
|
||||
因此材质直接用 `.foreground` 同时提供颜色和 Mask,不再额外加载原图 Image Texture。
|
||||
Image Empty 引用视频或序列帧时,DA3 与 MoGe-2 会按 Empty 的 Image Frames 数值逐帧估计
|
||||
深度(未设置时为完整帧数,显式设为 1 时只处理 1 帧),起始帧跟随 Empty 的 Offset,与背景蒙版范围一致;
|
||||
DA3 会复用同一个 ONNX Session 对每帧独立推理,再合并为原有序列输出,避免多帧 Attention 的平方级显存占用。
|
||||
该行为只用于 Depth Plane 转换;Surface 仅支持单张图片,Debug 多帧输出仍联合推理。
|
||||
Offset 超出视频长度时同样把窗口收进视频内。材质节点与置换贴图的 Start 保持与原 Empty
|
||||
一致;Original 模式的视频或序列帧使用 `frame.0001.jpg` 等原图序列作为颜色与深度,
|
||||
`foreground.0001.png` 只提供前景 Alpha。
|
||||
DA3 的深度仍以 EXR 序列帧(`depth.0001.exr` 起)保存在 job 目录根下;MoGe-2 分别使用 `moge_depth`、`moge_mask` 与 `moge_normal` EXR 序列。
|
||||
`Unshaded`、`Remove Background` 与 `Keep Original` 均默认关闭,因此默认删除原对象。
|
||||
3. DA3 或 MoGe-2 完成后会自动创建面片。DA3 面片几何先由 `mini_npz` 的相机内参与深度确定,再通过对象
|
||||
Transform 匹配原 Image Empty 的可见尺寸、位置、旋转和 Offset。Original 模式直接使用源 Empty
|
||||
的可见尺寸、Offset 与 Transform。DA3 会把用于匹配画面尺寸的缩放预乘到 `.depth` 数值,
|
||||
XY 缩放烘焙到 Mesh,Z 不参与平面匹配;最终对象 Scale 为 `(1, 1, 1)`,视觉深度与旧逻辑一致。
|
||||
关闭 `Keep Original` 时,成功生成后删除源 Empty。生成面片 Object 使用源 Empty 的原名;开启
|
||||
`Keep Original` 时不处理
|
||||
同名冲突,由 Blender 自动为生成面片追加编号。
|
||||
4. Blender 只创建保留图像纵横比的最低密度基础网格,再把 `O DA3 Depth Plane` 作为 Geometry Nodes
|
||||
修改器添加到对象。节点组与 `O DA3 Surface` 一同保存在 `O_SnapForm.blend`,关闭
|
||||
`show_group_selector`,不为每个对象复制节点组。MoGe-2 使用单独的 `O MoGe2 Depth Plane`,只保留
|
||||
网格细分、metric depth 位移、mask 删点和材质设置;材质使用 `O MoGe2 Depth Plane Layer`,直接接入
|
||||
模型 normal,不携带 DA3 的材质置换与背景深度分支。DA3 修改器公开以下参数:
|
||||
- `Displacement`:位于 Geometry 之后的第一项,菜单包含 `Geometry`(默认)与 `Material`。`Geometry`
|
||||
在节点组内沿法线修改网格,`Material` 保持网格平面并启用材质置换;
|
||||
- `Subdivision`:位于第二项,默认 `5`。节点读取基础平面的 Bounding Box,短边分段数为 `2^Subdivision`,
|
||||
长边按宽高比计算并四舍五入;例如 16:9 图片在等级 `5` 时生成 `57×32` 个面;
|
||||
- `Background Depth`:默认 `-1`,允许正负值,Structure Type 为 `Single`,直接显示且不再放在 Panel 中。前景 Mask 外的结构或
|
||||
材质置换使用该值;是否存在背景只取决于转换所得色彩贴图的 `Alpha > 0.5` 二值结果;
|
||||
- `Depth Scale`:默认 `1`,允许正负值,Structure Type 为 `Single`,同时控制 Geometry 与 Material
|
||||
两条置换路径。
|
||||
Grid 重建后通过隐藏的 Material 输入重新赋予原图材质,并对所有面执行 Shade Smooth。
|
||||
材质使用显式指向 `UVMap` 的 UV Map 节点连接颜色、前景 Mask 与深度 Image Texture,不依赖活动 UV。
|
||||
前景深度先与 `Background Depth` 取 Maximum,再按色彩贴图 Alpha 与背景深度混合,
|
||||
深度分支在混合前使用 `Alpha > 0.5` 转为二值 Mask,避免半透明前景被背景深度拉扯;
|
||||
材质表面 Alpha 同样使用二值 Mask。材质置换使用相同的二值和钳制顺序。
|
||||
节点组还会把 `o_material_displacement` 与 `o_background_depth` 写入点域,材质通过
|
||||
`Geometry` 类型的 Attribute 节点读取。点域会为顶点保存重复值,但能在应用修改器后继续保留参数;
|
||||
相比自适应网格本身,其占用很小。Depth Image 与 Foreground Image 是修改器隐藏数据输入,
|
||||
由导入流程自动设置。
|
||||
视频与序列帧还会为 Depth 与 Foreground 共同写入隐藏的 `Frame`、`Frame Start`、`Frames` 与 `Cyclic`。
|
||||
节点组内部不再使用 Scene Time。只有原 Image Empty 是视频或序列时,导入流程才直接给修改器的 `Frame`
|
||||
socket 添加 Scripted Expression Driver `frame`;静态图保持默认值 `1`,不创建 Driver。导入内容已经
|
||||
是截取后的帧,因此 Offset 在节点组内恒定为 `0`。
|
||||
节点组根据 `Frame` 计算从 `1` 开始的 Image Texture Frame:场景帧与 Start 都为 `1` 时读取图片第 `1` 帧;非循环输入
|
||||
钳制到有效范围;循环输入使用 Floored Modulo,让 `Frame Start` 前后的负、正相对帧都映射到
|
||||
`1…Frames`,因此开始帧之前也会循环播放。Geometry 置换会像材质 Image Texture 一样随场景帧刷新。
|
||||
115
docs/development.md
Normal file
115
docs/development.md
Normal file
@@ -0,0 +1,115 @@
|
||||
# 程序设计、开发与许可证
|
||||
|
||||
## 程序设计
|
||||
|
||||
SnapForm 分成 Blender 主进程与独立 Job Server 两侧。Blender 侧只负责界面、任务参数和 `.blend` 数据修改;模型下载、媒体准备、ONNX 推理和文件输出都在 Python 3.12 Environment 中运行。两侧由随扩展分发的 BlendJob wheel 连接,避免把 FastAPI、ONNX Runtime 等计算依赖载入 Blender Python。
|
||||
|
||||
```mermaid
|
||||
flowchart LR
|
||||
User["Image Empty / Debug Panel"] --> Operator["Blender Operator"]
|
||||
Operator --> Runtime["JobRuntime"]
|
||||
Runtime --> Server["JobServer FIFO Queue"]
|
||||
Server --> Handler["SnapForm Job Handler"]
|
||||
Handler --> Model["ModelManager / ONNX"]
|
||||
Handler --> JobFiles["Job Directory"]
|
||||
JobFiles --> Response["Operator response"]
|
||||
Response --> Scene["Blender Scene"]
|
||||
```
|
||||
|
||||
核心边界如下:
|
||||
|
||||
- `src/snapform/__init__.py` 是注册入口,集中注册 Blender 类型、`Scene.snapform_settings`、右键菜单与 Runtime
|
||||
- `panel.py`、`preferences.py`、`properties.py` 只描述 UI 和 Blender 属性,不直接导入 Server 模型实现
|
||||
- `operators.py` 将 Blender 状态转换为普通字典参数;Job 完成后在主线程导入图片、网格、材质和相机
|
||||
- `blender/depth_plane.py` 与 `blender/surface.py` 实现 Image Empty 输入检查、几何构建和资产挂载
|
||||
- `job_runtime.py` 以入口字符串声明独立 Server,并配置 Environment、Storage Root 与首次安装后的推荐模型下载
|
||||
- `server/__init__.py` 创建 `JobServer`、注册公开 Job,并把 `ModelManager` 注册为 Resource
|
||||
- `server/media.py`、`background.py`、`da3.py`、`moge2.py` 负责业务流程;`onnx_*.py` 只处理模型输入、推理和输出恢复
|
||||
- 根目录 `model_catalog.py` 是 Blender UI 与独立 Server 共用的纯数据声明;`server/model_download.py`、`server/model_manager.py` 分别负责下载和 Session 生命周期
|
||||
- `server/geometry.py` 生成 NPZ、预览、点云和纹理网格;所有任务产物写入 Runtime 分配的 opaque-ID Job 目录
|
||||
|
||||
功能细节继续按主题维护:[Depth Plane](depth-plane.md)、[Surface](surface.md)、[输出格式](outputs.md) 和 [Job Server](server.md)。
|
||||
|
||||
## 项目结构
|
||||
|
||||
```text
|
||||
.
|
||||
├── AGENTS.md
|
||||
├── README.md
|
||||
├── pack.py
|
||||
├── sync_models.py
|
||||
├── docs/
|
||||
├── src/
|
||||
│ └── snapform/
|
||||
│ ├── __init__.py
|
||||
│ ├── blender_manifest.toml
|
||||
│ ├── job_runtime.py
|
||||
│ ├── operators.py
|
||||
│ ├── panel.py
|
||||
│ ├── preferences.py
|
||||
│ ├── properties.py
|
||||
│ ├── assets/
|
||||
│ │ └── O_SnapForm.blend
|
||||
│ ├── blender/
|
||||
│ │ ├── depth_plane.py
|
||||
│ │ └── surface.py
|
||||
│ ├── model_catalog.py
|
||||
│ ├── server/
|
||||
│ │ ├── __init__.py
|
||||
│ │ ├── background.py
|
||||
│ │ ├── da3.py
|
||||
│ │ ├── geometry.py
|
||||
│ │ ├── media.py
|
||||
│ │ ├── model_download.py
|
||||
│ │ ├── model_manager.py
|
||||
│ │ ├── moge2.py
|
||||
│ │ ├── onnx_ben2.py
|
||||
│ │ ├── onnx_da3.py
|
||||
│ │ ├── onnx_moge2.py
|
||||
│ │ └── onnx_runtime.py
|
||||
│ └── wheels/
|
||||
│ └── blendjob-0.1.6-py3-none-any.whl
|
||||
├── tests/
|
||||
└── tools/
|
||||
```
|
||||
|
||||
## 常见命令
|
||||
|
||||
在项目根目录使用 Git Bash:
|
||||
|
||||
```bash
|
||||
uv sync
|
||||
uv run pytest
|
||||
uv run pytest tests/test_addon.py tests/test_manifest.py tests/test_pack.py -v
|
||||
uv run python pack.py
|
||||
```
|
||||
|
||||
`pack.py` 根据 `blender_manifest.toml` 分别生成 Windows、macOS 和 Linux 扩展包,例如 `dist/SnapForm.v0.12.3.windows-x64.zip`。从 PyPI 下载并更换 `src/snapform/wheels/` 中随扩展分发的 wheel 后,先刷新锁文件再测试:
|
||||
|
||||
```bash
|
||||
uv lock --refresh
|
||||
uv run pytest
|
||||
```
|
||||
|
||||
## 资产构建与验证
|
||||
|
||||
`tools/` 脚本依赖 `bpy`,必须由 Blender 执行。设置本机 Blender 可执行文件后,可以重建或验证 `O_SnapForm.blend`:
|
||||
|
||||
```bash
|
||||
BLENDER_BIN="/c/Program Files/Blender Foundation/Blender 4.5/blender.exe"
|
||||
|
||||
"$BLENDER_BIN" --background --python tools/build_geometry_nodes_asset.py
|
||||
"$BLENDER_BIN" --background --python tools/validate_depth_plane_asset.py
|
||||
"$BLENDER_BIN" --background --python tools/validate_geometry_nodes_asset.py
|
||||
"$BLENDER_BIN" --background --python tools/validate_moge_depth_plane.py
|
||||
"$BLENDER_BIN" --background --python tools/validate_moge_surface.py
|
||||
```
|
||||
|
||||
只改 Python、文档或打包逻辑时不需要重建 `.blend` 资产;修改节点组、材质组或其接口时,应运行相应资产验证脚本,并再次运行完整测试。
|
||||
|
||||
## 上游与许可证
|
||||
|
||||
- Depth Anything 3 代码采用 Apache 2.0
|
||||
- BEN2 ONNX 模型采用其仓库声明的 Apache 2.0 License
|
||||
- SnapForm Extension 代码采用 GPL-3.0-or-later
|
||||
- 模型权重许可证由所选 Hugging Face 模型仓库决定,Extension 下载前会在界面显示
|
||||
110
docs/features.md
Normal file
110
docs/features.md
Normal file
@@ -0,0 +1,110 @@
|
||||
# 功能与使用
|
||||
|
||||
本文档面向开发者和需要了解内部实现细节的用户,从原 README 迁移而来。普通用户请阅读项目根目录的
|
||||
`README.md`。
|
||||
|
||||
`SnapForm` 是一套从图片快速构建空间结构的 Blender Extension。目前以
|
||||
[Depth Anything 3](https://github.com/ByteDance-Seed/Depth-Anything-3)
|
||||
作为首个 AI 能力,在独立 Environment 中运行 DA3,可将官方导出结果保存在独立 job 目录,
|
||||
也可把 Image Empty 转换成带几何与材质置换的立体面片。MoGe-2 作为独立深度来源,保留其 metric depth、normal、mask 与 intrinsics 输出,不复用 DA3 Prediction 后处理。Debug 面板还可运行
|
||||
[BEN2](https://github.com/PramaLLC/BEN2) 背景剔除功能测试。
|
||||
|
||||
## 当前功能
|
||||
|
||||
- 使用 Blender Extension manifest,可通过 `Install from Disk` 安装
|
||||
- 在独立 Environment 中通过常驻 Server 运行 ONNX Runtime;MoGe-2、DA3 与 BEN2 的模型执行及后处理完全不使用 PyTorch,避免污染 Blender Python 进程
|
||||
- Server 的深度模型缓存只保留 DA3 或 MoGe-2 其中一个,切换时主动释放前一个;BEN2 使用独立缓存,
|
||||
可与当前深度模型共存
|
||||
- Blender 使用内置 `urllib.request` 访问仅监听 `127.0.0.1` 的 Server;连接由随机端口与 Instance ID 归属当前 Runtime
|
||||
- 常驻 Server 复用 ONNX Runtime session、BEN2 与最近使用的深度模型,连续执行时不再重复加载
|
||||
- 每个 Blender 实例持有自己的 Job Server;`Job Server` 面板显示状态与模型缓存,并提供安装、启动、清理、重启和停止控制
|
||||
- Environment、模型和任务结果默认保存在 `~/.snapform`,可由多个 Blender 版本共享
|
||||
- 当前仅启用 FP32 DA3 Small 和 Base;它们不随首次安装下载,Giant 1.1 与 Nested Giant Large 1.1 暂停使用
|
||||
- 支持 BEN2 Base 单图背景剔除测试,输出透明 PNG 与推理耗时
|
||||
- 支持单张 JPG、JPEG、PNG、WEBP 图片或常见视频文件输入
|
||||
- 视频按顺序截取前若干帧,可设置参与重建的最大帧数
|
||||
- `Models` panel 默认折叠,展开后可逐项查看和下载所有可用模型
|
||||
- Environment 已安装时不额外占用面板空间;模型列表每个模型只占一行
|
||||
- `Debug` 默认折叠,内部的 `DA3`、`MoGe-2` 与 `BEN2` 也分别默认折叠
|
||||
- 使用一个输出类型下拉菜单和一个 `Generate Output` 按钮
|
||||
- DA3 与 MoGe-2 Debug 都先选择单一 Output Format,只显示当前格式的专属参数;GLB 与 PLY 不会在一次测试中同时生成
|
||||
- Debug 的模型选项按格式过滤;PLY 只提供两个带 Gaussian head 的 Giant 模型
|
||||
- Debug 与 Convert to Depth Plane 的模型选项直接显示下载状态;未下载模型带
|
||||
`(Missing)`,已下载模型保持原名
|
||||
- 分别调用 DA3 的 `glb`、`gs_ply`、`depth_vis` 和 `mini_npz` exporter
|
||||
- `Textured Surface (GLB)` 直接使用 DA3 Prediction 中的深度、置信度和相机参数,把相邻像素
|
||||
组成三角面并写入 `mesh.glb`;每个输入帧生成一张独立纹理网格,不做多视图融合
|
||||
- Image Empty 右键菜单可直接运行 DA3,根据 `mini_npz` 深度生成面片
|
||||
- Image Empty 右键菜单可直接运行 BEN2,保留原图并换用新建的透明背景图片
|
||||
- 缺少必需的 `MoGe-2 ViT-S Normal` 或 `BEN2 Base` 时,Image Empty 的 `SnapForm`
|
||||
右键子菜单显示 `Download Required Models`;单次任务只下载缺失项,并在补齐前禁用
|
||||
`Convert to Depth Plane`
|
||||
- 菜单、模型面板和转换对话框直接检查 Storage Root 的 `models/` 必需文件,显示模型状态时不会启动 Job Server
|
||||
- Image Empty 支持视频与序列帧输入:`Convert to Depth Plane` 逐帧估计深度,
|
||||
`Remove Background` 逐帧抠图,入口保持不变。DA3 转换时每帧独立执行 ONNX 推理并复用模型 Session;
|
||||
Debug 多帧任务继续联合推理,以保留跨视图相机与深度关系
|
||||
- 视频生成的序列帧以外部文件引用保留在 job 目录(Blender 无法整体 Pack 序列帧);
|
||||
Geometry Nodes 与材质 Image Texture 均可按场景帧播放序列帧
|
||||
- 使用浮点深度贴图、`O DA3 Depth Plane` Geometry Nodes 和原图 Bump 构建立体面片
|
||||
- 转换对话框选择推理模式、处理分辨率与背景移除;细分和置换方式在 Geometry Nodes 修改器中调整
|
||||
- Debug 的 DA3 输出或 BEN2 背景剔除完成后只打开对应 job 文件夹,不修改当前场景
|
||||
- 完整保留每次任务生成的所有官方文件
|
||||
- 安装和模型下载使用后台子进程;推理提交给常驻 Server,状态栏实时显示阶段、百分比进度条与取消按钮
|
||||
- 后台任务运行时,Install、Download 和 Export 按钮保持原位置并变灰,任务完成后恢复
|
||||
|
||||
## 使用
|
||||
|
||||
1. 运行 `uv run python pack.py`,分别构建 `windows-x64`、`macos-arm64`、
|
||||
`linux-x64` 三个平台的 Extension ZIP(如 `SnapForm.v0.12.3.macos-arm64.zip`)。
|
||||
安装包不携带 uv 或 AI 项目源码;首次安装 Environment 时下载固定版本 uv,官方源建连超过 3 秒后回退 OmooLab 镜像,并校验固定 SHA-256。
|
||||
2. 在 Blender 的 `Preferences > Extensions` 中选择 `Install from Disk`。
|
||||
3. 在 Extension Preferences 中确认 Storage Root 与 Device。Device 是正式功能
|
||||
(Image Empty 右键菜单)使用的推理设备,默认取本机最快的 DirectML/CoreML provider,否则 CPU,且只列出
|
||||
本机支持的设备。
|
||||
4. 在 `Job Server` 点击 `Install Environment`。首次安装会下载 Python、ONNX Runtime、FastAPI、Uvicorn;
|
||||
uv 由 Job Runtime 的 Environment 安装器下载并校验,DA3、BEN2 与 MoGe 不安装源码包,
|
||||
并在 Environment 安装后自动下载 `MoGe-2 ViT-S Normal` 和 FP16 `BEN2 Base` 推荐模型。首次安装
|
||||
依次显示 `Environment`、`MoGe-2 ViT-S Normal` 与 `BEN2 Base`;每项都独立从 0 走到 100%,
|
||||
进入下一项时进度条归零,模型 post-install 不显示 `2/2` 等阶段编号。uv、Python Environment 与 packages 合并在第一轮,
|
||||
两个默认模型由普通的 `post_install(runtime)` 发起正式 Server Job。
|
||||
安装完成后直接写入 Environment Manifest,不再额外导入全部组件进行验证;`DA3 Base` 等其他模型可稍后在 `Models` 面板按需下载。
|
||||
依赖安装显示当前安装输出;模型下载显示速度,例如
|
||||
`Downloading DA3 Small 14.2 MB/s`;镜像来源显示为
|
||||
`Downloading DA3 Small (mirror) 14.2 MB/s`。下载按字节连续推进,重试或切换镜像时进度不会倒退。
|
||||
Environment 安装阶段与 `Models` 面板单独下载使用相同的模型名称格式,不会改写为通用的 `Downloading packages`。
|
||||
Environment 安装后即使模型下载中断也仍会保留,但 `Install Environment` 会以失败结束并显示模型下载错误。
|
||||
可稍后在 Image Empty 的 `SnapForm` 右键菜单用 `Download Required Models` 一次补齐缺失的
|
||||
MoGe-2 ViT-S 与 BEN2,也可在 `Models` 面板单独补下。
|
||||
安装不做启动网络预检;每个模型都先尝试 Hugging Face,失败后尝试 OmooLab R2,两者都失败时当前 Job 失败。
|
||||
下载源不写入持久状态;需要手动指定依赖镜像时可通过
|
||||
`UV_INDEX_URL` 环境变量覆盖。
|
||||
5. 在 3D View 按 `N`,打开 `SnapForm` 页签。
|
||||
6. 如需其他模型,展开 `Models` panel,找到模型并点击该行的 `Download`。
|
||||
7. 展开默认收起的 `Debug`,选择 `Output Type` 和本次执行使用的模型。
|
||||
8. 选择一张图片或一个视频文件。视频可设置从头截取的帧数,然后点击 `Generate Output`。
|
||||
9. 测试背景剔除时,确认 `Models` panel 中的 `BEN2 Base` 已就绪,然后在 `Debug` 的
|
||||
`BEN2` 区域选择单张图片并点击 `Remove Background`。
|
||||
完成后会打开 job 文件夹,其中的 `foreground.png` 带透明 Alpha;状态栏显示
|
||||
BEN2 推理、模型加载与任务总耗时。当前测试固定关闭较慢的 foreground refinement。
|
||||
|
||||
## Image Empty 去背景
|
||||
|
||||
1. 确认 `Models` panel 中的 `BEN2 Base` 已就绪;首次安装 Environment 时会默认下载。
|
||||
2. 在 3D View 选中 Image Empty,右键打开 `SnapForm` 子菜单并选择 `Remove Background`。
|
||||
3. 完成后以 Fake User 保留原 Image datablock,新建并 Pack 一张透明 PNG 到当前 `.blend`,然后只把
|
||||
当前 Empty 的 `data` 换成新图;Empty 的 Transform、显示尺寸、Offset 及其他设置保持不变。
|
||||
4. 新 Image 使用原图去掉格式后再追加 `.foreground` 的名称。例如 `img.png` 生成
|
||||
`img.foreground`,`img.png.001` 生成 `img.001.foreground`。已有同名 datablock 时由
|
||||
Blender 自动追加编号。
|
||||
|
||||
原图已 Pack 时会先写出临时 PNG 交给 BEN2,任务结束后删除临时输入;原 Image 仍保持 Pack
|
||||
状态和原始内容。BEN2 的 `foreground.png` 也会保留在对应 job 目录中。
|
||||
|
||||
Image Empty 引用视频或序列帧时,Remove Background 直接执行,默认取 Empty 的 Image Frames
|
||||
数值(未设置时为完整帧数,显式设为 1 时只处理 1 帧),起始帧跟随 Empty 的 Offset;若 Offset 加 Frames 超出视频长度,
|
||||
会把窗口整体收进视频内(保证实际处理 Frames 帧数),Offset 过小时从第 1 帧开始。输出
|
||||
`foreground.0001.png` 等透明序列帧(帧号从 0001 开始,与 Blender 场景帧一一对应),并把
|
||||
源帧以 `frame.0001.jpg` 等命名复制到 job 目录根下供材质颜色导入。替换 Empty 的 Image 为
|
||||
加载这些帧的 SEQUENCE Image,从原 Empty 的 Start 位置开始播放(Start 原样保留),
|
||||
Cyclic 沿用原 Empty 的设置。序列帧无法被 Blender 整体 Pack,因此保持外部引用 job 目录;
|
||||
请保留对应 job 目录,否则序列会失效。
|
||||
10
docs/index.md
Normal file
10
docs/index.md
Normal file
@@ -0,0 +1,10 @@
|
||||
# SnapForm 文档
|
||||
|
||||
技术内容已按主题拆分:
|
||||
|
||||
- [功能与使用](features.md):能力概览、安装使用和 Image Empty 去背景
|
||||
- [Depth Plane](depth-plane.md):深度面片的数据、节点与材质流程
|
||||
- [Surface](surface.md):透视网格、Geometry Nodes 后处理与材质法线
|
||||
- [输出格式](outputs.md):Debug 输出类型、参数和任务产物
|
||||
- [Job Server](server.md):Environment、模型缓存、任务目录与运行机制
|
||||
- [程序设计、开发与许可证](development.md):模块边界、调用流程、项目结构、常用命令与许可证
|
||||
29
docs/outputs.md
Normal file
29
docs/outputs.md
Normal file
@@ -0,0 +1,29 @@
|
||||
# 输出格式
|
||||
|
||||
## DA3 ONNX
|
||||
|
||||
`Output Type` 提供以下结果:
|
||||
|
||||
- Point Cloud (GLB):由 depth、confidence、相机参数和原图生成彩色点云
|
||||
- Depth Preview (Images):保存深度着色预览
|
||||
- Depth Data (NPZ):保存 depth、confidence、extrinsics、intrinsics 与处理后的图像
|
||||
- Textured Surface (GLB):SnapForm 根据 Prediction 构建纹理网格
|
||||
|
||||
DA3 Small 与 Base 的 ONNX 图不包含 Gaussian head,因此不提供 Gaussian Splats。默认推理参数为:
|
||||
|
||||
- `process_res=504`
|
||||
- `process_res_method="upper_bound_resize"`
|
||||
- `ref_view_strategy="saddle_balanced"`
|
||||
- `conf_thresh_percentile=40.0`
|
||||
- `num_max_points=1_000_000`
|
||||
- `show_cameras=True`
|
||||
|
||||
相机外参与内参由模型的 camera decoder 输出恢复,不运行 ray-pose 求解器。
|
||||
|
||||
## MoGe-2 ONNX
|
||||
|
||||
MoGe-2 Debug 可保存原始 NPZ、depth/normal/mask maps、纹理 GLB 或彩色 PLY。Surface 输出由模型 points、mask 与 depth edge 在内部构建三角网格。
|
||||
|
||||
## BEN2 ONNX
|
||||
|
||||
BEN2 输出 `foreground.png`,RGB 保留原始输入,Alpha 来自 ONNX mask 并恢复到输入分辨率。
|
||||
123
docs/server.md
Normal file
123
docs/server.md
Normal file
@@ -0,0 +1,123 @@
|
||||
# Job Server
|
||||
|
||||
## 运行架构
|
||||
|
||||
SnapForm 在独立的 Python 3.12 Environment 中运行 HTTP Job Server,Blender 只通过标准库 HTTP Client 提交和轮询任务,不把计算依赖载入 Blender Python。Server 只监听 `127.0.0.1`,由外置 BlendJob wheel 提供的 `ServerController` 保存随机端口与 Instance ID;专属本地进程协议不使用额外 Token 鉴权。
|
||||
|
||||
每个 Blender 实例启动并持有一个 SnapForm 专属 Server,不发现或复用其它 Blender、其它插件启动的本地 Server。多个 Blender 可以对应多个进程,但模型、Environment 与任务产物仍可共享同一个 Storage Root。
|
||||
|
||||
Environment、模型和任务结果默认保存在 `~/.snapform`:
|
||||
|
||||
```text
|
||||
~/.snapform/
|
||||
├── .venv/
|
||||
├── environment.json
|
||||
├── models/
|
||||
├── jobs/
|
||||
├── manifest.json
|
||||
├── install.log
|
||||
├── install-status.json # 仅在安装期间存在
|
||||
└── server.log
|
||||
```
|
||||
|
||||
不同 Blender 版本可共享此目录。也可以在 Extension Preferences 的 `Storage Root` 中修改位置。连接端口与 Instance ID 只保存在启动它们的 Blender 进程内存中。
|
||||
|
||||
Blender 侧的通用 HTTP Runtime 由 `src/snapform/wheels/` 中随扩展分发的 BlendJob package 提供;独立 Environment 由 Runtime 从 PyPI 自动安装相同版本。`JobServer` 包装 FastAPI 并提供 Job、FIFO Queue 与 Resource API,`JobClient` 是无第三方依赖的 Blender 侧访问器。Blender 侧只把 `server/__init__.py:server` 入口字符串交给 Runtime,不导入 `server` package;独立进程启动后才取得其中的 `server` 实例、注册全部公开 Job,并创建 `model_manager` Resource。
|
||||
|
||||
SnapForm 根 package 不再有 `client.py`、`environment.py`、`storage.py` 或 `models.py`。这些文件会制造并不存在的业务边界:
|
||||
|
||||
- Storage Root、`.venv`、Manifest、安装状态与日志路径由 `JobRuntime` 唯一管理;
|
||||
- `InstallEnvironment` 由 `runtime.operator_classes()` 提供,Python 与 Job 自身的额外 packages 直接声明在传给 Runtime 的 `environment` 字典中;BlendJob 不需要重复声明;
|
||||
- 模型 catalog 由 `model_catalog.py` 统一声明;下载和 ONNX Session 属于 Server,分别位于 `model_download.py` 与 `model_manager.py`;
|
||||
- Blender UI 复用同一 catalog,直接检查 Storage Root 下的模型必需文件,不为显示下载状态启动 Server;已加载 Session 状态仍读取现有 Server Snapshot。
|
||||
|
||||
BlendJob 的 `JobClient` 只是无依赖 HTTP transport,不是 SnapForm 业务抽象,也不向 Operator 暴露 `JobRequest`。
|
||||
|
||||
SnapForm 的业务 Operator 直接提交参数字典,Job Type 一一对应:
|
||||
|
||||
- `download_model` → `download-model`
|
||||
- `download_required_models` → `download-required-models`
|
||||
- `remove_background` → `remove-background`
|
||||
- `extract_frames` → `extract-frames`
|
||||
- `generate_da3_depth` → `generate-da3-depth`
|
||||
- `generate_moge2_depth` → `generate-moge2-depth`
|
||||
- `generate_da3_surface` → `generate-da3-surface`
|
||||
- `generate_moge2_surface` → `generate-moge2-surface`
|
||||
- `debug_da3` → `debug-da3`
|
||||
- `debug_moge2` → `debug-moge2`
|
||||
- `debug_ben2` → `debug-ben2`
|
||||
|
||||
DA3 与 MoGe-2 的生产任务不再合并成带 Backend 参数的通用请求。每个 Operator 的 `request(context)` 直接构造自己的有效参数,Debug 任务也与生产任务完全分开。
|
||||
|
||||
Server 端没有 `execute_*`、`*_result` 或 CLI worker 中转。全部 `@server.job(...)` 都集中在 `server/__init__.py`:简单任务直接实现,复杂任务调用按领域拆分的 `media.py`、`background.py`、`moge2.py` 或 `da3.py`;共享的纯几何计算位于 `geometry.py`。模型下载由 `ModelManager` 单向调用 `model_download.py`,不存在循环 import。
|
||||
|
||||
## ONNX-only Environment
|
||||
|
||||
扩展不内置 `uv` 可执行文件或 AI 运行库。`Install Environment` 首先下载固定版本的 uv,校验 SHA-256 后解压到共享 Storage Root,再用它创建环境并安装:
|
||||
|
||||
- ONNX Runtime 1.24.4;Windows 使用 `onnxruntime-directml`
|
||||
- NumPy 2.4.2、Pillow 12.1.1、PyAV 17.0.0、trimesh 4.11.2
|
||||
- BlendJob 0.1.6、FastAPI 0.139.2、Uvicorn 0.51.0、huggingface-hub 1.4.1
|
||||
|
||||
运行环境不安装 PyTorch、TorchVision、xFormers,也不安装 DA3、MoGe、BEN2 或 utils3d 的 Python 源码包。模型推理和 SnapForm 所需的相机、点云、网格后处理均由 ONNX Runtime、NumPy 与内部实现完成。
|
||||
|
||||
Windows 当前发行环境提供 DirectML 与 CPU;`Auto` 优先使用 `DmlExecutionProvider`。显式指定不可用设备时会安全回退到 CPU。其他平台安装标准 ONNX Runtime,并按可用 provider 选择 CoreML、OpenVINO 或 CPU。
|
||||
|
||||
`environment.json` 保存规范化后的声明,`manifest.json` 保存该声明、BlendJob 版本与基础 Server packages 的 Hash。Python、packages、平台 packages、BlendJob 版本或 Environment Schema 改变后需要重新执行 `Install Environment`,不再维护手工 Manifest Revision。
|
||||
|
||||
## 模型
|
||||
|
||||
所有正式功能都使用 ONNX:
|
||||
|
||||
- MoGe-2 ViT-S、ViT-B、ViT-L Normal
|
||||
- DA3 Small、DA3 Base
|
||||
- BEN2 Base
|
||||
|
||||
首次安装在 `post_install(runtime)` 中自动下载 `MoGe-2 ViT-S Normal` 与 FP16 `BEN2 Base`。它们和 `Models` panel 的后续下载一样,都是 `download-model` Job,与推理共用队列、HTTP 进度和取消协议。模型按声明的目录名保存在 `~/.snapform/models/`,并保留仓库内部结构。声明了文件清单与校验值的模型先走 Hugging Face `resolve` 直链流式下载,失败后尝试 `https://models.omoolab.xyz/<repository-name>/<relative-path>`;两者都失败时当前 Job 直接失败,不记录或猜测后续下载源。模型直链的 TCP、代理 CONNECT 与 TLS 建连最长等待 10 秒;连接后连续 60 秒没有收到响应数据则视为当前来源失败并切换下一来源。该空闲超时不限制模型总下载时长。R2 文件会校验大小与 SHA-256。未声明清单的仓库仍使用 `huggingface_hub` 快照下载。可用 `HF_ENDPOINT` 指定 Hugging Face 镜像。
|
||||
|
||||
Image Empty 的右键菜单会读取 `model_manager` 状态;缺少 MoGe-2 ViT-S 或 BEN2 时显示
|
||||
`Download Required Models`。对应的 `download-required-models` Job 在 Server 端再次检查就绪状态,
|
||||
只下载缺失项,并把多个下载的进度合并为单次 0–100%。必需模型补齐前,右键菜单中的
|
||||
`Convert to Depth Plane` 保持禁用。
|
||||
|
||||
`model_manager` Resource 在 Server 内持有模型 catalog、磁盘就绪状态和内存 Session。Blender UI 使用 catalog 声明的目录名与 `ready_patterns` 直接检查 `<storage-root>/models/`,因此展开模型面板、打开转换对话框或 Image Empty 右键菜单都不会仅为状态检测启动 Server。真正下载或推理时,Server 仍通过 `ModelManager.ready()` 重新验证文件。`ModelSessionCache` 保留 BEN2,并在 DA3 与 MoGe-2 之间只保留最近使用的一种 Depth Session。`GET /resources/model_manager` 继续返回 catalog、下载状态和当前加载状态;`POST /resources/model_manager/clear` 只允许在没有 Active 或 Queued Job 时执行。
|
||||
|
||||
DA3 ONNX 使用模型的 camera decoder 恢复外参与内参,不提供 ray-pose 参数。MoGe-2 的 metric depth、normal、mask、normalized intrinsics,以及 BEN2 的 Alpha,均在 NumPy 后处理中恢复。
|
||||
|
||||
|
||||
## 安装网络与镜像
|
||||
|
||||
安装器直接开始工作,不做阻塞式网络预检。packages 由 uv 安装,可用 `UV_INDEX_URL` 显式指定镜像。Windows 系统代理会注入子进程;同时支持 `HTTP_PROXY`、`HTTPS_PROXY` 与 `ALL_PROXY`。
|
||||
|
||||
uv 默认从 Astral 的 GitHub Release 下载,失败后回退到 `https://mirror.omoolab.xyz/uv/<version>/`。两个来源使用相同的连续字节进度并共用固定 SHA-256 校验。下载压缩包只作临时文件,提取完成后删除。模型下载不经过 GitHub。
|
||||
|
||||
首次安装依次显示 Environment、MoGe-2 下载与 BEN2 下载,每项分别使用 0–100%。Environment 由 Job Runtime 依次完成 uv 下载、Python Environment 创建与 packages 安装,uv 不再占用独立一轮进度。两个模型下载由 SnapForm 的 `post_install(runtime)` 通过 `runtime.request()` 顺序发起;切换模型时进度归零,但不显示 `2/2` 等阶段编号。
|
||||
任一 post-install 模型失败时,Environment 文件仍然保留,但安装 Operator 返回失败并显示聚合后的模型错误,
|
||||
不再把带 `post_install_error` 的终态报告为成功。
|
||||
|
||||
## 生命周期
|
||||
|
||||
扩展启用后不启动 Server;第一次执行 Job 或点击 `Start` 时才为当前 Blender 启动专属实例。正常关闭或禁用扩展时直接停止自己启动的进程;Server 也监视 Blender 父进程 PID,父进程异常退出后会自行停止。
|
||||
|
||||
AI、帧提取和 Models panel 下载的 Job 进度保存在 Server 内存中。Handler 只使用 `context.progress(value, message)` 发布进度,终态由 JobServer 内部管理;SnapForm 不再维护独立的 `server/progress.py` 或 Job 状态 JSON。Blender 每 0.25 秒轮询 `GET /jobs/{job_id}`,任务目录不写 `status.json`。Operator 直接提交参数字典;Job 被接受时由 `JobServer` 自动生成 ID 并创建 `jobs/<opaque-job-id>/`,提交响应同时返回 `job_id` 与 `directory`。路径通过 `JobContext.directory` 传给 Handler,SnapForm Server 强制将输出写入该目录。
|
||||
|
||||
`job_runtime.py` 创建唯一的 `runtime = JobRuntime(...)`,以自身目录为 `entrypoint_root`,传入 `server:server` 相对入口字符串与显式 `snapform` namespace,并导出 `runtime.JobOperatorBase`。Runtime 独占 Storage Root、Environment 路径、安装进程、Server 控制和 Status Bar。Operator 未覆盖 `request(context)` 时会提交全部 Property;覆盖后返回值纯替换默认参数。可选的 `response(context, result)` 在 Blender 主线程直接完成结果导入。SnapForm 不再提供 Client 请求层、Job Mixin、Status Bar 或 Build 中转 Operator。
|
||||
|
||||
Environment 安装是 Server 启动前唯一的 Bootstrap 流程:`environment.json` 记录期望安装的环境声明,`install-status.json` 在安装期间回传进度并在任务收尾后删除,`install.log` 持久记录安装输出。Bootstrap 不创建 Job 目录。Environment 成功后执行普通的 `post_install(runtime)`;默认模型下载此时已经是正式 Server Job,会创建自己的 Job 目录,并使用与其它任务相同的进度和取消协议。
|
||||
|
||||
Server 使用标准库单 Worker FIFO Queue,全局同时只执行一个 Job,包括模型下载。Server 忙碌时新 Job 进入 `queued`,不再返回 409;排队任务可以在执行前取消。取消运行中任务不会重启 Server;流式模型下载会在分块检查点结束,ONNX Runtime 当前调用则在返回后的最近检查点结束。JobServer 最多保留 100 个已完成 Job 的内存状态,后续提交时清理更早记录。
|
||||
|
||||
Server 由 Environment 通过 `python -m blendjob.runner` 启动,避免 BlendJob 内部模块名遮蔽 Python 标准库。启动与崩溃信息写入 `~/.snapform/server.log`。
|
||||
|
||||
## 已知限制
|
||||
|
||||
DA3 ONNX 的注意力是朴素实现(`MatMul → Softmax → MatMul`),峰值显存与 `(帧数 × 每帧 patch 数)²` 成正比。迁移到 ONNX 前的 PyTorch 版使用 `scaled_dot_product_attention`(`XFORMERS_DISABLED=1`),按块计算注意力分数、不落地完整注意力矩阵,显存近似线性,因此视频能处理几十上百帧;ONNX 导出把 SDPA 回退成朴素注意力,多帧在中高分辨率下会显存爆炸(约 12 帧 @ 1008 即需 ~90 GB)。
|
||||
|
||||
Depth Plane 的视频、序列帧转换会复用同一个 ONNX Session 逐帧独立推理,再把结果合并到现有序列输出,因此峰值显存只取决于单帧分辨率。逐帧结果不保留 DA3 的跨视图尺度与相机位姿一致性,适用于当前逐帧动画结构。Surface 只支持单张图片;Debug 多帧输出仍使用联合推理,以保留多视图关系。
|
||||
|
||||
联合推理时,`onnx_da3.infer` 会在执行前按 `views × patches` 预估注意力张量,超过 4 GiB 直接报错并提示减少帧数或 Resolution Scale,避免 onnxruntime 报出难以定位的分配失败或 `UnicodeDecodeError`(DirectML 在中文 Windows 下报错信息为 GBK 编码,被 UTF-8 解码掩盖)。
|
||||
|
||||
待解决方向:
|
||||
|
||||
- 重新导出 DA3 ONNX,使用 ORT 的 `Attention` / `MultiHeadAttention` 贡献算子,或注入内存高效的注意力算子;
|
||||
- 若未来需要统一坐标系的视频重建,可按小窗口分批推理,再对齐相机位姿与深度尺度。
|
||||
190
docs/surface.md
Normal file
190
docs/surface.md
Normal file
@@ -0,0 +1,190 @@
|
||||
# Surface
|
||||
|
||||
## Image Empty 转透视网格
|
||||
|
||||
1. 在 3D View 选中仅包含一帧的 Image Empty,右键打开 `SnapForm` 子菜单并选择
|
||||
`Convert to Surface`。该入口独立于
|
||||
`Convert to Depth Plane`,当前不处理视频或序列帧。
|
||||
模型列表包含三个 MoGe-2 Normal 模型和两个 DA3 模型,默认使用 MoGe-2 ViT-S Normal。DA3 的
|
||||
Resolution Scale 与 Depth Plane 使用相同设计;MoGe-2 改为显示共享的 Resolution Level。
|
||||
2. Server 请求 DA3 的 `depth`、`conf` 与 `intrinsics`。仅在开启 `Remove Background` 时运行 BEN2 取得
|
||||
前景 Alpha,并在输出前删除包含背景顶点的三角面;关闭时不加载 BEN2。Depth 不做 smoothing,Server
|
||||
反投影深度网格,建立源三角面,计算 confidence percentile 与 3×3 相对深度不连续度,并写入
|
||||
基于原图像素坐标的源 `UVMap`。这些步骤只准备修改器的输入网格,不执行 Relax、拓扑修复、平滑、
|
||||
小岛清理或最终 UV 重投影。运行时显示的六个 `Preparing surface data` 步骤均指这段 Server 端源数据准备。
|
||||
3. Blender 把勾选 `Usage > Modifier` 的 `O DA3 Surface` 节点组直接作为 Geometry Nodes 修改器
|
||||
添加到 Surface,不另设修改器名称。节点组内没有子节点组,依次完成:
|
||||
- 修改器顶层依次显示 `Resolution`、`Relax Iterations`、`Smooth`;默认展开的 `Culling` Panel
|
||||
依次包含 `Depth Continuity` 与 `Confidence`。前者直接读取面域 3×3 相对深度差,并使用指数映射:
|
||||
默认 `50%` 对应官方 `rtol=0.04`,数值越大,内部阈值越小,剔除的断层面越多;
|
||||
- 每轮 Remesh Iteration 依次执行 Connected Merge、循环拓扑修复与一次 Smooth,使新拓扑在进入下一轮
|
||||
Merge 前先稳定;拓扑修复删除非流形面、单面拓扑和小面积 mesh islands;
|
||||
- 默认 `Resolution` 为原始有效三角边长中位数乘以 `sqrt(10) * 2`。`sqrt(10)` 来自原先 90% 面数缩减
|
||||
的二维边距估计,再乘以 2 为迭代合并留出更平滑、稳定的默认尺度;节点内部 Connected Merge 的
|
||||
Distance 为 `Resolution * 0.5`;
|
||||
- Smooth 采用 `O Smooth` 的边界检测和模糊法线权重,固定开放边界与薄面,只保留一个
|
||||
`Smooth` 程度系数;节点组不再额外执行 Flip Faces;
|
||||
- 从修改前的原始表面以 nearest-surface 重新采样 `UVMap`,最终明确写为 Corner 域 `2D Vector`。
|
||||
MoGe-2 使用独立的 `O MoGe2 Surface`。Server 由内置 NumPy 网格构建函数
|
||||
生成有限深度范围内的紧凑三角网格,并保留模型 mask、逐点 depth 及相邻点相对深度差。Geometry Nodes
|
||||
按官方 `mask & ~depth_map_edge(depth, rtol=0.04)` 的含义,先由 `Mask` 删除 mask 外的面,再由
|
||||
`Depth Continuity` 按同一份 3×3 邻域相对深度差删除断层面。DA3 与 MoGe-2 使用相同的属性、
|
||||
名称、指数滑块和 FACE 删除逻辑,不再额外删除孤立点。之后才执行拓扑修复、Relax、孤岛清理与 UV 重投影。
|
||||
节点组不提供 Confidence 或 Remesh 开关;Remesh 仍是节点处理流程的一部分。
|
||||
4. 节点组随插件放在 `O_SnapForm.blend`。Blender 4.5 首次 Append 后复用同一 datablock;
|
||||
Blender 5.0 以 linked ID 载入后调用 Pack,后续 Surface 继续复用。插件不会为每个 Surface 新建节点组,
|
||||
也不会在添加后重命名 Geometry Nodes 修改器。节点组只启用 `Usage > Modifier`,不 Mark as Asset;
|
||||
修改器关闭 `show_group_selector`。`Fix Topology` 位于默认折叠的 `Options` Panel。
|
||||
节点图按 Culling、Remesh、Topology Fix、Island Cleanup 与 UV 分区摆放;各区就近使用独立的
|
||||
Group Input,并隐藏没有连线的接口,避免跨越整个节点图的输入连线。
|
||||
`O DA3 Surface` 与 `O MoGe2 Surface` 的 `Resolution` 默认值均为 `0.1`,所有输入 socket 的
|
||||
Structure Type 均固定为 `Single`。
|
||||
5. Server 准备过程固定显示 6 个编号阶段:验证 DA3 数据、反投影、准备 DA3 与可选 BEN2 数据、构建原始
|
||||
源三角面、源 UV/attributes、写出网格。状态栏显示 `Preparing surface data n/6: ...`。
|
||||
6. Server 在 job 根目录写出 `mesh.npz` 与 `texture.png`。NPZ 保存 camera-space `vertices`、
|
||||
`faces`、`uv`、`intrinsics`、`image_size` 与有效深度中位数 `reference_depth`。Blender 读取这些
|
||||
数据后直接创建 Surface 与原图材质,不需要 Open3D 或 OpenCV 进入 Blender Python 环境。
|
||||
DA3 Surface 只保留 Geometry Nodes 后处理实际使用的 attributes:点域 `o_depth`、面域
|
||||
`o_face_confidence` 与面域 `o_depth_discontinuity`。Remove Background 已在 Server
|
||||
建面前直接删除背景三角面,不写入任何 background attribute。MoGe-2 的导入网格保留点域 `o_depth`,
|
||||
以及面域 `o_mask`、`o_depth_discontinuity`,不伪造 confidence、stretch 或 background 属性,
|
||||
也不写入 custom normal。完整分辨率的 `normal_map` 保存在 NPZ 中并转换为材质贴图。
|
||||
所有 MoGe-2 Image datablock 都以源图片名而非对象名为基名:depth、normal 与 mask 分别使用
|
||||
`图片名.depth`、`图片名.normal`、`图片名.mask`,不附加 `.moge2`。
|
||||
7. 网格以 `reference_depth` 截面匹配 Image Empty 的显示矩形:该截面的投影四角与 Empty 的可见
|
||||
边界重合,主点同时匹配 Empty Offset。网格对象直接沿用源 Empty 的 `matrix_world`,因此位置、旋转、
|
||||
缩放及父级关系产生的世界变换保持一致。更近的顶点沿 Empty 局部 `+Z` 方向凸出,三角面正面朝向
|
||||
观察相机,启用 Backface Culling 时可从正面看到。
|
||||
8. 转换对话框不再提供后处理参数,只保留模型与 Resolution Scale 设置、`Create Matching Camera`、
|
||||
`Remove Background`、`Unshaded`,以及分割线下方的 `Keep Original`。背景移除默认关闭;开启后 Server
|
||||
使用 BEN2,并在写出 Surface 前直接删除背景三角面,不再由 Geometry Nodes 的 Background 开关处理。
|
||||
该预删除同时适用于 DA3 与 MoGe-2;因此 MoGe-2 转换后即使关闭 `Mask`,BEN2 已删除的背景也不会恢复。
|
||||
`Unshaded` 使用与 Depth Plane 相同的 Emission/Transparent 材质路径;普通材质的
|
||||
Bump Scale 按 `abs(object.scale.z) × camera scale_z` 设置,并在材质组内乘以固定基准 `0.001`,
|
||||
与透视表面实际深度尺度一致;它允许负数以反转凹凸方向。
|
||||
相机默认关闭;开启后会在 Empty 局部 `+Z` 侧创建一台朝向局部 `-Z` 的透视相机,并成为生成 Surface
|
||||
的子级,使二者移动时保持投影关系。关闭 `Keep Original` 时,网格成功创建后删除源 Empty。
|
||||
表面、材质与相机会归入同一个 `Convert to Surface` Undo 步骤。
|
||||
|
||||
Process Res 的设计依据 DA3 官方实现:API 与 CLI 默认使用 `process_res=504` 和
|
||||
`process_res_method="upper_bound_resize"`;输入预处理的 `PATCH_SIZE=14`,会把宽高分别调整到最接近的
|
||||
`14` 倍数。DA3 论文以 `504×504` 为 base resolution,公开的训练尺寸长边为 `504`、`672`、`756` 或
|
||||
`896`。官方 CLI 虽给出 `1024` 示例,但没有声明更高分辨率会提升质量,也没有规定 `1000` 为硬上限。
|
||||
因此转换入口使用相对原图的 Resolution Scale,同时通过上下限避免 Process Res 过低或过高,并把训练范围
|
||||
之外的值明确视为实验设置。参考
|
||||
[DA3 API](https://github.com/ByteDance-Seed/Depth-Anything-3/blob/main/docs/API.md)、
|
||||
[输入预处理实现](https://github.com/ByteDance-Seed/Depth-Anything-3/blob/main/src/depth_anything_3/utils/io/input_processor.py)
|
||||
与[技术报告](https://depth-anything-3.github.io/assets/da3_tech_report_2025.pdf)。
|
||||
|
||||
Debug 的 Textured Surface GLB 会把 depth、confidence 与 confidence percentile 写入自定义顶点
|
||||
attributes。其他 DA3 Debug 三维输出会同时写出未压缩的 `prediction.attributes.npz`,保存完整
|
||||
depth、confidence、intrinsics、extrinsics 与 confidence percentile。当前 trimesh 的 glTF 点图元
|
||||
导出器只写 Position/Color,无法可靠把任意 attribute 嵌入点云 GLB,因此点云先以 companion NPZ
|
||||
保留完整结构;后续若改为 SnapForm 自有 glTF writer,可再把这些字段直接嵌入点图元。
|
||||
|
||||
所有任务使用统一的状态栏进度显示。变化中的阶段说明显示在右侧的进度条内部,取消按钮
|
||||
位于进度条右侧。状态字段支持阶段、当前操作与预计剩余秒数,界面统一组合为紧凑文案并保证
|
||||
进度单调递增。任务进程与 Server 请求在后台启动,按钮点击后立即进入进度状态。Image Depth 的阶段包括 Environment 启动、输入准备、模型加载、深度估算、缓存
|
||||
写出和面片构建。进度更新会强制刷新状态栏与窗口,无需移动鼠标即可实时显示。安装中断后残留的
|
||||
半成品 Environment 会在下次安装前自动清理,不会因为中断而无法再次安装。
|
||||
|
||||
导入过程精确保留 NPZ 内深度值之间的差值。例如深度 `2.0` 和 `5.5` 在
|
||||
`Displacement Strength = -1` 时,沿局部 Z 轴相差 `3.5 Blender Unit`。它不会把数值压缩到
|
||||
`0..1`,也不再提供视觉归一化模式。
|
||||
|
||||
面片尺寸以有效深度的中位数 `Zref` 作为参考截面。先用 `mini_npz` 中的相机内参计算原始
|
||||
投影面积,再恢复源图纵横比:
|
||||
|
||||
```text
|
||||
RawWidth = depth_width × Zref × abs(Displacement Strength) / fx
|
||||
RawHeight = depth_height × Zref × abs(Displacement Strength) / fy
|
||||
Area = RawWidth × RawHeight
|
||||
Aspect = source_image_width / source_image_height
|
||||
Width = sqrt(Area × Aspect)
|
||||
Height = sqrt(Area / Aspect)
|
||||
```
|
||||
|
||||
这样即使 DA3 估计出的 `fx/fy` 不一致,也不会把正方形原图拉成长方形。主点 `cx/cy` 仍用于
|
||||
确定原始相机面片,随后对象 XY Transform 将它匹配到源 Image Empty。生成对象会记录
|
||||
`snapform_reference_depth`、`snapform_camera_plane_width` 和
|
||||
`snapform_camera_plane_height`,便于检查实际采用的
|
||||
参考深度和尺寸。
|
||||
|
||||
材质始终导入深度 Image Texture,并连接到 Displacement 节点和 Material Output;材质 Surface
|
||||
Displacement 设置为 `Displacement Only`,DA3 与 Original 都使用 `Midlevel = 0`、`Scale = 1`。MoGe-2 的位移只在专用 Geometry Nodes 中处理。
|
||||
`o_material_displacement` 为 `False` 时,材质把 Height 乘以零,置换只在几何节点中生效;为 `True` 时,
|
||||
几何节点关闭结构置换,材质 Height 恢复。色彩贴图的 Alpha 二值结果同时用于选择原始深度或有符号的
|
||||
`Background Depth`,使两种置换路径保持一致;未移除背景的普通不透明图片因此全部视为前景。
|
||||
视频与序列帧的颜色、深度与前景 Image Texture
|
||||
通过 image_user 的 Frames / Offset / Start 把第 i 帧映射到场景 `Start+i` 帧,并启用 Auto Refresh。
|
||||
颜色贴图到 Principled BSDF 之间使用独立的 Shader Node Group。DA3 Surface 使用 `O DA3 Surface Layer`,
|
||||
MoGe-2 Surface 使用独立的 `O MoGe2 Surface Layer`;它额外接收模型输出的 `Normal` 贴图,使用
|
||||
`Normal Map` 节点按 `Object Space` 解码后连接到 Bump 的 `Normal` 输入,使像素级模型法线与原图
|
||||
Bump 合并,不依赖重拓扑后的顶点密度;`Normal Strength` 默认值为 `1.0`,只调节模型 normal map,
|
||||
原图 Bump 仍由独立的 `Bump Scale` 控制;
|
||||
Depth Plane 使用 `O DA3 Depth Plane Layer` 材质组,并把背景剔除、背景深度与材质置换 Attribute
|
||||
读取全部收进组内。两者都接收 `Color`、`Alpha` 与 `Bump Scale`,只有 `O DA3 Depth Plane Layer`
|
||||
额外接收 Float 类型的 `Depth`。
|
||||
`O DA3 Surface Layer` 只输出 `Color`、`Roughness`、`Alpha` 与 `Normal`,以原图 Color 保留 Bump 效果,
|
||||
不创建 Displacement 节点或输出;`O DA3 Depth Plane Layer` 另外输出 `Displacement`。外层材质只保留贴图、
|
||||
材质组、Principled BSDF 与 Material Output,方便继续调整。
|
||||
Depth Plane 的 Bump Height 直接使用材质组的 `Color` 输入,只受独立的 `Bump Scale` 影响,不使用深度贴图,
|
||||
也不经过 `Displacement` 模式开关、`Depth Scale` 或 `Background Depth`;材质 Displacement 仍使用这些参数处理后的深度。
|
||||
无论选择哪种置换方式,原始彩色贴图都会连接
|
||||
Principled BSDF 的 `Base Color`,并通过
|
||||
`Bump.Height → Bump.Normal → Principled.Normal` 提供表面细节。Bump 的基础 Distance 为
|
||||
`0.001`,会乘以面片 Transform 的统一缩放倍率。原图 Color 还会经过 `Float Curve` 连接到
|
||||
Principled BSDF 的 `Roughness`,曲线两端输出分别为 `0.5` 和 `0.1`;材质 IOR 保持
|
||||
Principled BSDF 默认值 `1.5`。
|
||||
开启 `Unshaded` 后,色彩贴图 Color 连接 Emission,二值 Alpha 作为 Mix Shader 系数,在
|
||||
Transparent BSDF 与 Emission 之间混合后连接 Material Output;不创建 Principled BSDF、Bump 或
|
||||
Float Curve。开启 `Remove Background` 时色彩贴图使用 BEN2 输出;关闭时使用原图。
|
||||
|
||||
这里的“原始距离”是 DA3 模型输出的尺度。普通 Small、Base、Large 和 Giant 模型的深度不应
|
||||
被解释为现实世界的米;需要现实尺度时应选择带 metric depth 的
|
||||
`DA3 Nested Giant Large 1.1`。此外,当前结果仍是固定 XY 平面上的深度位移面,不是使用
|
||||
相机内参逐像素反投影得到的完整 camera-space 网格;中位深度截面为 Geometry Nodes
|
||||
深度面片提供正确的整体高宽尺度。
|
||||
|
||||
生成完成后会直接创建面片,不提供缓存重建入口或单独的导入参数。重新选择
|
||||
`Convert to Depth Plane` 会按当前 Depth from 再次转换。
|
||||
面片、深度 Image、前景 Image、Material 与 Geometry Nodes 修改器会在同一个 Blender Undo 步骤中创建;转换完成后执行
|
||||
`Ctrl+Z` 会完整撤销本次转换,不会越过转换去回退更早的 Image Empty 操作。
|
||||
|
||||
Image Empty 使用 Packed Image 时,SnapForm 会通过 Blender 写出临时 PNG 交给 DA3,
|
||||
不会 Unpack 或修改源 Image;任务结束后自动删除临时文件。普通图片仍直接读取源文件。DA3 的原始
|
||||
`results.npz` 保留在对应 job 文件夹根下,生成的浮点深度 Image 会 Pack 到
|
||||
当前 `.blend`。Depth Image 与 Displace Texture 根据源 Image datablock 命名。例如 Empty Object
|
||||
名为 `obj_img`、引用的 Image 名为 `img.png` 时,Depth Image 与 Texture 都命名为 `img.depth`;
|
||||
生成的 Object、Mesh 与 Material 仍命名为 `obj_img`。Pack 使用 OpenEXR,因而不会把大于 `1.0`
|
||||
的原始深度截断。已有同名 datablock 时由 Blender 自动追加 `.001` 等编号。
|
||||
视频输入时 Blender 无法整体 Pack 序列帧,因此 `.depth` 与颜色序列改为引用 job 目录根下的
|
||||
`depth.0001.exr`、`frame.0001.jpg`(`.frame`)或 `foreground.0001.png`,并在每帧保留原始
|
||||
浮点深度;保存 `.blend` 后再次打开仍需对应 job 目录存在。
|
||||
|
||||
选择 PLY 后,模型菜单只会提供 `DA3 Giant 1.1` 和
|
||||
`DA3 Nested Giant Large 1.1`。Small 和 Base 没有 Gaussian head,无法由
|
||||
DA3 生成 `gs_ply`。
|
||||
|
||||
每个 Server Job 被接受时,Job Runtime 在 Runtime Storage Root 的 `jobs/` 下创建一个独立文件夹。文件夹名是 32 位 opaque Job ID,不包含任务类型或对象名称。执行逻辑从 `JobContext.directory` 获取路径,输出产物直接写入该目录。
|
||||
|
||||
Blender 为 Packed Image 或序列帧生成的输入副本先保存在系统临时目录,Server 读取完成后由 Blender 清理,不归档到 Job 目录。普通磁盘文件和视频仍直接读取原文件。Server 日志统一写入 `server.log`,进度只保存在 Server 内存中。Environment Bootstrap 使用 Storage Root 下的 `install.log` 与临时的 `install-status.json`,不创建 Job 目录。
|
||||
|
||||
与几何输出相关的 Job Type 为 `generate-da3-depth`、`generate-moge2-depth`、`generate-da3-surface`、`generate-moge2-surface`、`debug-da3` 与 `debug-moge2`。每种 Job 只接收对应模型与输出所需的参数。
|
||||
|
||||
DA3 ONNX exporter 产生的 `depth_vis/`、`exports/mini_npz/` 会在完成后自动平铺,
|
||||
不会保留只含一层产物的子目录:
|
||||
|
||||
```text
|
||||
<opaque-job-id>/
|
||||
├── scene.glb、*.ply、*.jpg 或 results.npz
|
||||
└── 其它对应 Debug 格式的产物
|
||||
|
||||
<opaque-job-id>/
|
||||
├── foreground.png 单图结果
|
||||
├── foreground.0001.png… 视频/序列帧结果
|
||||
├── frame.0001.jpg… 视频/序列帧的源帧副本
|
||||
└── depth.0001.exr… AI 深度序列帧
|
||||
```
|
||||
|
||||
建议从 `DA3 Base` 开始。它使用 Apache 2.0 模型权重。
|
||||
@@ -1,534 +0,0 @@
|
||||
# SnapForm 技术文档
|
||||
|
||||
本文档面向开发者和需要了解内部实现细节的用户,从原 README 迁移而来。普通用户请阅读项目根目录的
|
||||
`README.md`。
|
||||
|
||||
`SnapForm` 是一套从图片快速构建空间结构的 Blender Extension。目前以
|
||||
[Depth Anything 3](https://github.com/ByteDance-Seed/Depth-Anything-3)
|
||||
作为首个 AI 能力,在独立 Environment 中运行 DA3,可将官方导出结果保存在独立 job 目录,
|
||||
也可把 Image Empty 转换成带几何与材质置换的立体面片。MoGe-2 作为独立深度来源,保留其 metric depth、normal、mask 与 intrinsics 输出,不复用 DA3 Prediction 后处理。Debug 面板还可运行
|
||||
[BEN2](https://github.com/PramaLLC/BEN2) 背景剔除功能测试。
|
||||
|
||||
## 当前功能
|
||||
|
||||
- 使用 Blender Extension manifest,可通过 `Install from Disk` 安装
|
||||
- 在独立 Environment 中通过常驻 Service 运行 Torch、DA3、MoGe-2 与 BEN2,避免污染 Blender Python 进程
|
||||
- Blender 使用内置 `urllib.request` 访问仅监听 `127.0.0.1` 的 Service;请求带随机 Token
|
||||
- 常驻 Service 复用 Torch、CUDA、BEN2 与最近使用的 DA3 模型,连续执行时不再重复加载
|
||||
- 多个 Blender 实例自动连接同一个共享 Service;`AI Service` 面板显示状态、端口、客户端与模型缓存,并提供安装、启动、清理、重启和停止控制
|
||||
- Environment、模型和任务结果默认保存在 `~/.snapform`,可由多个 Blender 版本共享
|
||||
- 当前启用 DA3 Small 和 Base;Large 1.1、Giant 1.1 与 Nested Giant Large 1.1 暂停使用
|
||||
- 支持 BEN2 Base 单图背景剔除测试,输出透明 PNG 与推理耗时
|
||||
- 支持单张 JPG、JPEG、PNG、WEBP 图片或常见视频文件输入
|
||||
- 视频按顺序截取前若干帧,可设置参与重建的最大帧数
|
||||
- `Models` panel 默认折叠,展开后可逐项查看和下载所有可用模型
|
||||
- Environment 已安装时不额外占用面板空间;模型列表每个模型只占一行
|
||||
- `Debug` 默认折叠,内部的 `DA3 Output Test`、`MoGe-2 API Test` 与 `BEN2 Removal Test` 也分别默认折叠
|
||||
- 使用一个输出类型下拉菜单和一个 `Generate Output` 按钮
|
||||
- 只显示当前格式的专属参数;GLB 与 PLY 设置不会同时出现
|
||||
- Debug 的模型选项按格式过滤;PLY 只提供两个带 Gaussian head 的 Giant 模型
|
||||
- Debug 与 Convert to Depth Plane 的模型选项直接显示下载状态;未下载模型带
|
||||
`(Missing)`,已下载模型保持原名
|
||||
- 分别调用 DA3 的 `glb`、`gs_ply`、`depth_vis` 和 `mini_npz` exporter
|
||||
- `Textured Surface (GLB)` 直接使用 DA3 Prediction 中的深度、置信度和相机参数,把相邻像素
|
||||
组成三角面并写入 `mesh.glb`;每个输入帧生成一张独立纹理网格,不做多视图融合
|
||||
- Image Empty 右键菜单可直接运行 DA3,根据 `mini_npz` 深度生成面片
|
||||
- Image Empty 右键菜单可直接运行 BEN2,保留原图并换用新建的透明背景图片
|
||||
- Image Empty 支持视频与序列帧输入:`Convert to Depth Plane` 逐帧估计深度,
|
||||
`Remove Background` 逐帧抠图,入口保持不变
|
||||
- 视频生成的序列帧以外部文件引用保留在 job 目录(Blender 无法整体 Pack 序列帧);
|
||||
Geometry Nodes 与材质 Image Texture 均可按场景帧播放序列帧
|
||||
- 使用浮点深度贴图、`O DA3 Depth Plane` Geometry Nodes 和原图 Bump 构建立体面片
|
||||
- 转换对话框选择推理模式、处理分辨率与背景移除;细分和置换方式在 Geometry Nodes 修改器中调整
|
||||
- Debug 的 DA3 输出或 BEN2 背景剔除完成后只打开对应 job 文件夹,不修改当前场景
|
||||
- 完整保留每次任务生成的所有官方文件
|
||||
- 安装和模型下载使用后台子进程;推理提交给常驻 Service,状态栏实时显示阶段、百分比进度条与取消按钮
|
||||
- 后台任务运行时,Install、Download 和 Export 按钮保持原位置并变灰,任务完成后恢复
|
||||
|
||||
## 使用
|
||||
|
||||
1. 先运行 `uv run python prepare_vendor.py`,按 `vendor.lock.toml` 下载并校验固定版本的
|
||||
第三方构建输入;再运行 `uv run python pack.py`,分别构建 `windows-x64`、`macos-arm64`、
|
||||
`linux-x64` 三个平台的 Extension ZIP(如 `SnapForm.v0.10.11.macos-arm64.zip`)。
|
||||
每个包只含对应平台的 uv 二进制,并内置 DA3、BEN2 源码包。
|
||||
2. 在 Blender 的 `Preferences > Extensions` 中选择 `Install from Disk`。
|
||||
3. 在 Extension Preferences 中确认 Storage Root 与 Device。Device 是正式功能
|
||||
(Image Empty 右键菜单)使用的推理设备,默认取本机最快的 CUDA/MPS,否则 CPU,且只列出
|
||||
本机支持的设备。
|
||||
4. 在 `AI Service` 点击 `Install Environment`。首次安装会下载 Python、PyTorch、FastAPI、Uvicorn;
|
||||
DA3、BEN2 源码包与 uv 已内置在扩展中,安装时从 安装任务的临时目录中使用,安装结束后自动清理,
|
||||
并在 Environment 安装后自动下载 `DA3 Small` 和 `BEN2 Base` 推荐模型。Environment 安装
|
||||
固定显示 `Environment`、`PyTorch`、`AI components` 三个用户阶段;xFormers(macOS 跳过)、
|
||||
FastAPI 等内部安装步骤合并在 `AI components` 中。耗时最长的 PyTorch 与 AI 组件占主要进度
|
||||
区间。安装完成后直接写入 Environment Manifest,不再额外导入全部组件进行验证。推荐模型属于安装
|
||||
后的自动步骤,不计入阶段编号;`DA3 Base` 等其他模型可稍后在 `Models` 面板按需下载。
|
||||
状态文字使用 `阶段/总数 阶段名 | 当前操作 | 预计剩余时间`,依赖安装会结合预设耗时与实际
|
||||
进度修正预期;模型下载根据字节数和速度计算剩余时间,例如
|
||||
`Downloading DA3 Small | 14.2 MB/s | About 2 min left`。重试或切换镜像时进度不会倒退。
|
||||
Environment 安装后即使模型下载中断也不影响 Service 使用,可稍后在 `Models` 面板单独补下。
|
||||
安装不做启动网络预检:pip 安装失败会自动依次重试清华、阿里镜像,模型下载由 worker
|
||||
自行探测回退到 HF 镜像;需要手动指定镜像时可通过 `PIP_INDEX_URL` 环境变量覆盖。
|
||||
5. 在 3D View 按 `N`,打开 `SnapForm` 页签。
|
||||
6. 如需其他模型,展开 `Models` panel,找到模型并点击该行的 `Download`。
|
||||
7. 展开默认收起的 `Debug`,选择 `Output Type` 和本次执行使用的模型。
|
||||
8. 选择一张图片或一个视频文件。视频可设置从头截取的帧数,然后点击 `Generate Output`。
|
||||
9. 测试背景剔除时,确认 `Models` panel 中的 `BEN2 Base` 已就绪,然后在 `Debug` 的
|
||||
`BEN2 Removal Test` 区域选择单张图片并点击 `Remove Background`。
|
||||
完成后会打开 job 文件夹,其中的 `foreground.png` 带透明 Alpha;状态栏显示
|
||||
BEN2 推理、模型加载与任务总耗时。当前测试固定关闭较慢的 foreground refinement。
|
||||
|
||||
## Image Empty 去背景
|
||||
|
||||
1. 确认 `Models` panel 中的 `BEN2 Base` 已就绪;首次安装 Environment 时会默认下载。
|
||||
2. 在 3D View 选中 Image Empty,右键打开 `SnapForm` 子菜单并选择 `Remove Background`。
|
||||
3. 完成后以 Fake User 保留原 Image datablock,新建并 Pack 一张透明 PNG 到当前 `.blend`,然后只把
|
||||
当前 Empty 的 `data` 换成新图;Empty 的 Transform、显示尺寸、Offset 及其他设置保持不变。
|
||||
4. 新 Image 使用原图去掉格式后再追加 `.foreground` 的名称。例如 `img.png` 生成
|
||||
`img.foreground`,`img.png.001` 生成 `img.001.foreground`。已有同名 datablock 时由
|
||||
Blender 自动追加编号。
|
||||
|
||||
原图已 Pack 时会先写出临时 PNG 交给 BEN2,任务结束后删除临时输入;原 Image 仍保持 Pack
|
||||
状态和原始内容。BEN2 的 `foreground.png` 也会保留在对应 job 目录中。
|
||||
|
||||
Image Empty 引用视频或序列帧时,Remove Background 直接执行,默认取 Empty 的 Image Frames
|
||||
数值(未设置时为完整帧数,显式设为 1 时只处理 1 帧),起始帧跟随 Empty 的 Offset;若 Offset 加 Frames 超出视频长度,
|
||||
会把窗口整体收进视频内(保证实际处理 Frames 帧数),Offset 过小时从第 1 帧开始。输出
|
||||
`foreground.0001.png` 等透明序列帧(帧号从 0001 开始,与 Blender 场景帧一一对应),并把
|
||||
源帧以 `frame.0001.jpg` 等命名复制到 job 目录根下供材质颜色导入。替换 Empty 的 Image 为
|
||||
加载这些帧的 SEQUENCE Image,从原 Empty 的 Start 位置开始播放(Start 原样保留),
|
||||
Cyclic 沿用原 Empty 的设置。序列帧无法被 Blender 整体 Pack,因此保持外部引用 job 目录;
|
||||
请保留对应 job 目录,否则序列会失效。
|
||||
|
||||
## Image Empty 转深度面片
|
||||
|
||||
1. 在 3D View 选中 `Object Type = Empty`、`Display As = Image` 的对象。
|
||||
2. 右键打开 `SnapForm` 子菜单并选择 `Convert to Depth Plane`,通过 `Depth from` 选择深度来源:
|
||||
- `DA3`:显示 Model、Use Ray Pose 和 Resolution Scale。Process Res 先以原图长边乘以
|
||||
Resolution Scale,再钳制到 `252–1008`,最后对齐到最接近的 `14` 倍数。Resolution Scale 以百分比
|
||||
显示,范围为 `1–100%`,默认 `100%`;界面只在标签中实时显示最终 Process Res。
|
||||
转换导出 `mini_npz`,使用前需要在
|
||||
`Models` panel 下载所选模型,并固定使用 `process_res_method="upper_bound_resize"`;设备和参考视图策略
|
||||
继续使用场景设置。Model 下拉菜单只列出已经下载的 DA3 模型;若没有可用模型,对话框会引导到
|
||||
`Models` panel 下载。
|
||||
- `MoGe-2`:使用 Base Normal 模型输出 metric depth、normal、validity mask 与 normalized intrinsics。正式转换的 `results.npz` 不保存体积较大的 points;Debug 测试会保留 points,并另外写出 depth、normal、mask 预览。
|
||||
- `Original`:直接把原图作为深度贴图,不显示估算参数。仅在开启 `Remove Background` 时运行 BEN2
|
||||
取得前景 Alpha;视频或序列帧按需同时抽取原始帧与前景序列,把原始帧作为 `.depth` 序列。
|
||||
`Original` 是默认深度来源。选择 `DA3` 时,其 Model、Use Ray Pose 与 Resolution Scale
|
||||
参数会集中显示在独立边框中。`Remove Background` 与 `Unshaded` 直接显示,不再使用 Options 折叠;
|
||||
`Keep Original` 默认关闭,并在分割线下方保持为最后一项。仅当开启 `Remove Background` 时才需要
|
||||
`BEN2 Base`,并把 `.foreground` 的 Alpha 作为独立
|
||||
Mask,不在 Service 中改写深度值。保存 `.foreground` 时保留 BEN2 Alpha,但 RGB 使用未经修改的原始输入;
|
||||
因此材质直接用 `.foreground` 同时提供颜色和 Mask,不再额外加载原图 Image Texture。
|
||||
Image Empty 引用视频或序列帧时,DA3 与 MoGe-2 会按 Empty 的 Image Frames 数值逐帧估计
|
||||
深度(未设置时为完整帧数,显式设为 1 时只处理 1 帧),起始帧跟随 Empty 的 Offset,与背景蒙版范围一致;
|
||||
Offset 超出视频长度时同样把窗口收进视频内。材质节点与置换贴图的 Start 保持与原 Empty
|
||||
一致;Original 模式的视频或序列帧使用 `frame.0001.jpg` 等原图序列作为颜色与深度,
|
||||
`foreground.0001.png` 只提供前景 Alpha。
|
||||
DA3 的深度仍以 EXR 序列帧(`depth.0001.exr` 起)保存在 job 目录根下;MoGe-2 分别使用 `moge_depth`、`moge_mask` 与 `moge_normal` EXR 序列。
|
||||
`Unshaded`、`Remove Background` 与 `Keep Original` 均默认关闭,因此默认删除原对象。
|
||||
3. DA3 或 MoGe-2 完成后会自动创建面片。DA3 面片几何先由 `mini_npz` 的相机内参与深度确定,再通过对象
|
||||
Transform 匹配原 Image Empty 的可见尺寸、位置、旋转和 Offset。Original 模式直接使用源 Empty
|
||||
的可见尺寸、Offset 与 Transform。DA3 会把用于匹配画面尺寸的缩放预乘到 `.depth` 数值,
|
||||
XY 缩放烘焙到 Mesh,Z 不参与平面匹配;最终对象 Scale 为 `(1, 1, 1)`,视觉深度与旧逻辑一致。
|
||||
关闭 `Keep Original` 时,成功生成后删除源 Empty。生成面片 Object 使用源 Empty 的原名;开启
|
||||
`Keep Original` 时不处理
|
||||
同名冲突,由 Blender 自动为生成面片追加编号。
|
||||
4. Blender 只创建保留图像纵横比的最低密度基础网格,再把 `O DA3 Depth Plane` 作为 Geometry Nodes
|
||||
修改器添加到对象。节点组与 `O DA3 Surface` 一同保存在 `O_SnapForm.blend`,关闭
|
||||
`show_group_selector`,不为每个对象复制节点组。MoGe-2 使用单独的 `O MoGe2 Depth Plane`,只保留
|
||||
网格细分、metric depth 位移、mask 删点和材质设置;材质使用 `O MoGe2 Depth Plane Layer`,直接接入
|
||||
模型 normal,不携带 DA3 的材质置换与背景深度分支。DA3 修改器公开以下参数:
|
||||
- `Displacement`:位于 Geometry 之后的第一项,菜单包含 `Geometry`(默认)与 `Material`。`Geometry`
|
||||
在节点组内沿法线修改网格,`Material` 保持网格平面并启用材质置换;
|
||||
- `Subdivision`:位于第二项,默认 `5`。节点读取基础平面的 Bounding Box,短边分段数为 `2^Subdivision`,
|
||||
长边按宽高比计算并四舍五入;例如 16:9 图片在等级 `5` 时生成 `57×32` 个面;
|
||||
- `Background Depth`:默认 `-1`,允许正负值,Structure Type 为 `Single`,直接显示且不再放在 Panel 中。前景 Mask 外的结构或
|
||||
材质置换使用该值;是否存在背景只取决于转换所得色彩贴图的 `Alpha > 0.5` 二值结果;
|
||||
- `Depth Scale`:默认 `1`,允许正负值,Structure Type 为 `Single`,同时控制 Geometry 与 Material
|
||||
两条置换路径。
|
||||
Grid 重建后通过隐藏的 Material 输入重新赋予原图材质,并对所有面执行 Shade Smooth。
|
||||
材质使用显式指向 `UVMap` 的 UV Map 节点连接颜色、前景 Mask 与深度 Image Texture,不依赖活动 UV。
|
||||
前景深度先与 `Background Depth` 取 Maximum,再按色彩贴图 Alpha 与背景深度混合,
|
||||
深度分支在混合前使用 `Alpha > 0.5` 转为二值 Mask,避免半透明前景被背景深度拉扯;
|
||||
材质表面 Alpha 同样使用二值 Mask。材质置换使用相同的二值和钳制顺序。
|
||||
节点组还会把 `o_material_displacement` 与 `o_background_depth` 写入点域,材质通过
|
||||
`Geometry` 类型的 Attribute 节点读取。点域会为顶点保存重复值,但能在应用修改器后继续保留参数;
|
||||
相比自适应网格本身,其占用很小。Depth Image 与 Foreground Image 是修改器隐藏数据输入,
|
||||
由导入流程自动设置。
|
||||
视频与序列帧还会为 Depth 与 Foreground 共同写入隐藏的 `Frame`、`Frame Start`、`Frames` 与 `Cyclic`。
|
||||
节点组内部不再使用 Scene Time。只有原 Image Empty 是视频或序列时,导入流程才直接给修改器的 `Frame`
|
||||
socket 添加 Scripted Expression Driver `frame`;静态图保持默认值 `1`,不创建 Driver。导入内容已经
|
||||
是截取后的帧,因此 Offset 在节点组内恒定为 `0`。
|
||||
节点组根据 `Frame` 计算从 `1` 开始的 Image Texture Frame:场景帧与 Start 都为 `1` 时读取图片第 `1` 帧;非循环输入
|
||||
钳制到有效范围;循环输入使用 Floored Modulo,让 `Frame Start` 前后的负、正相对帧都映射到
|
||||
`1…Frames`,因此开始帧之前也会循环播放。Geometry 置换会像材质 Image Texture 一样随场景帧刷新。
|
||||
|
||||
## Image Empty 转透视网格
|
||||
|
||||
1. 在 3D View 选中仅包含一帧的 Image Empty,右键打开 `SnapForm` 子菜单并选择
|
||||
`Convert to Surface`。该入口独立于
|
||||
`Convert to Depth Plane`,当前不处理视频或序列帧。
|
||||
对话框中的 Resolution Scale 与 Depth Plane 使用相同设计:原图长边乘以 `1–100%` 的比例后,
|
||||
将实际 Process Res 钳制在 `252–1008` 并按 `14` 对齐。
|
||||
2. Service 请求 DA3 的 `depth`、`conf` 与 `intrinsics`。仅在开启 `Remove Background` 时运行 BEN2 取得
|
||||
前景 Alpha,并在输出前删除包含背景顶点的三角面;关闭时不加载 BEN2。Depth 不做 smoothing,Service
|
||||
反投影深度网格,建立源三角面,计算 confidence percentile 与 camera-space edge stretch,并写入
|
||||
基于原图像素坐标的源 `UVMap`。这些步骤只准备修改器的输入网格,不执行 Relax、拓扑修复、平滑、
|
||||
小岛清理或最终 UV 重投影。运行时显示的六个 `Preparing surface data` 步骤均指这段 Service 端源数据准备。
|
||||
3. Blender 把勾选 `Usage > Modifier` 的 `O DA3 Surface` 节点组直接作为 Geometry Nodes 修改器
|
||||
添加到 Surface,不另设修改器名称。节点组内没有子节点组,依次完成:
|
||||
- 修改器顶层依次显示 `Resolution`、`Relax Iterations`、`Smooth`;默认展开的 `Culling` Panel
|
||||
只包含 `Confidence` 与 `Stretch`。两个百分比输入都使用 `0–100` 的节点值,并在组内
|
||||
转换为 `0–1` 阈值;`Stretch = 100%` 明确选择全部面,剔面后同时删除
|
||||
Face Count 为零的孤立顶点;
|
||||
- 每轮 Remesh Iteration 依次执行 Connected Merge、循环拓扑修复与一次 Smooth,使新拓扑在进入下一轮
|
||||
Merge 前先稳定;拓扑修复删除非流形面、单面拓扑和小面积 mesh islands;
|
||||
- 默认 `Resolution` 为原始有效三角边长中位数乘以 `sqrt(10) * 2`。`sqrt(10)` 来自原先 90% 面数缩减
|
||||
的二维边距估计,再乘以 2 为迭代合并留出更平滑、稳定的默认尺度;节点内部 Connected Merge 的
|
||||
Distance 为 `Resolution * 0.5`;
|
||||
- Smooth 采用 `O Smooth` 的边界检测和模糊法线权重,固定开放边界与薄面,只保留一个
|
||||
`Smooth` 程度系数;节点组不再额外执行 Flip Faces;
|
||||
- 从修改前的原始表面以 nearest-surface 重新采样 `UVMap`,最终明确写为 Corner 域 `2D Vector`。
|
||||
4. 节点组随插件放在 `O_SnapForm.blend`。Blender 4.5 首次 Append 后复用同一 datablock;
|
||||
Blender 5.0 以 linked ID 载入后调用 Pack,后续 Surface 继续复用。插件不会为每个 Surface 新建节点组,
|
||||
也不会在添加后重命名 Geometry Nodes 修改器。节点组只启用 `Usage > Modifier`,不 Mark as Asset;
|
||||
修改器关闭 `show_group_selector`。`Fix Topology` 位于默认折叠的 `Options` Panel。
|
||||
节点图按 Culling、Remesh、Topology Fix、Island Cleanup 与 UV 分区摆放;各区就近使用独立的
|
||||
Group Input,并隐藏没有连线的接口,避免跨越整个节点图的输入连线。
|
||||
5. Service 准备过程固定显示 6 个编号阶段:验证 DA3 数据、反投影、准备 DA3 与可选 BEN2 数据、构建原始
|
||||
源三角面、源 UV/attributes、写出网格。状态栏显示 `Preparing surface data n/6: ...`。
|
||||
6. Service 在 job 根目录写出 `mesh.npz` 与 `texture.png`。NPZ 保存 camera-space `vertices`、
|
||||
`faces`、`uv`、`intrinsics`、`image_size` 与有效深度中位数 `reference_depth`。Blender 读取这些
|
||||
数据后直接创建 Surface 与原图材质,不需要 Open3D 或 OpenCV 进入 Blender Python 环境。
|
||||
Surface 保留下列结构化 attributes:
|
||||
- 点域:`o_depth`、`o_confidence`、`o_confidence_percentile`、`o_background`;
|
||||
- 面域:`o_face_confidence`、`o_edge_stretch`、`o_background_face`。
|
||||
7. 网格以 `reference_depth` 截面匹配 Image Empty 的显示矩形:该截面的投影四角与 Empty 的可见
|
||||
边界重合,主点同时匹配 Empty Offset。网格对象直接沿用源 Empty 的 `matrix_world`,因此位置、旋转、
|
||||
缩放及父级关系产生的世界变换保持一致。更近的顶点沿 Empty 局部 `+Z` 方向凸出,三角面正面朝向
|
||||
观察相机,启用 Backface Culling 时可从正面看到。
|
||||
8. 转换对话框不再提供后处理参数,只保留模型与 Resolution Scale 设置、`Create Matching Camera`、
|
||||
`Remove Background`、`Unshaded`,以及分割线下方的 `Keep Original`。背景移除默认关闭;开启后 Service
|
||||
使用 BEN2,并在写出 Surface 前直接删除背景三角面,不再由 Geometry Nodes 的 Background 开关处理。
|
||||
`Unshaded` 使用与 Depth Plane 相同的 Emission/Transparent 材质路径;普通材质的
|
||||
Bump Scale 按 `abs(object.scale.z) × camera scale_z` 设置,并在材质组内乘以固定基准 `0.001`,
|
||||
与透视表面实际深度尺度一致;它允许负数以反转凹凸方向。
|
||||
相机默认关闭;开启后会在 Empty 局部 `+Z` 侧创建一台朝向局部 `-Z` 的透视相机,并成为生成 Surface
|
||||
的子级,使二者移动时保持投影关系。关闭 `Keep Original` 时,网格成功创建后删除源 Empty。
|
||||
表面、材质与相机会归入同一个 `Convert to Surface` Undo 步骤。
|
||||
|
||||
Process Res 的设计依据 DA3 官方实现:API 与 CLI 默认使用 `process_res=504` 和
|
||||
`process_res_method="upper_bound_resize"`;输入预处理的 `PATCH_SIZE=14`,会把宽高分别调整到最接近的
|
||||
`14` 倍数。DA3 论文以 `504×504` 为 base resolution,公开的训练尺寸长边为 `504`、`672`、`756` 或
|
||||
`896`。官方 CLI 虽给出 `1024` 示例,但没有声明更高分辨率会提升质量,也没有规定 `1000` 为硬上限。
|
||||
因此转换入口使用相对原图的 Resolution Scale,同时通过上下限避免 Process Res 过低或过高,并把训练范围
|
||||
之外的值明确视为实验设置。参考
|
||||
[DA3 API](https://github.com/ByteDance-Seed/Depth-Anything-3/blob/main/docs/API.md)、
|
||||
[输入预处理实现](https://github.com/ByteDance-Seed/Depth-Anything-3/blob/main/src/depth_anything_3/utils/io/input_processor.py)
|
||||
与[技术报告](https://depth-anything-3.github.io/assets/da3_tech_report_2025.pdf)。
|
||||
|
||||
Debug 的 Textured Surface GLB 会把 depth、confidence 与 confidence percentile 写入自定义顶点
|
||||
attributes。其他 DA3 Debug 三维输出会同时写出未压缩的 `prediction.attributes.npz`,保存完整
|
||||
depth、confidence、intrinsics、extrinsics 与 confidence percentile。当前 trimesh 的 glTF 点图元
|
||||
导出器只写 Position/Color,无法可靠把任意 attribute 嵌入点云 GLB,因此点云先以 companion NPZ
|
||||
保留完整结构;后续若改为 SnapForm 自有 glTF writer,可再把这些字段直接嵌入点图元。
|
||||
|
||||
所有任务使用统一的状态栏进度显示。变化中的阶段说明显示在右侧的进度条内部,取消按钮
|
||||
位于进度条右侧。状态字段支持阶段、当前操作与预计剩余秒数,界面统一组合为紧凑文案并保证
|
||||
进度单调递增。任务进程与 Service 请求在后台启动,按钮点击后立即进入进度状态。Image Depth 的阶段包括 Environment 启动、输入准备、模型加载、深度估算、缓存
|
||||
写出和面片构建。进度更新会强制刷新状态栏与窗口,无需移动鼠标即可实时显示。安装中断后残留的
|
||||
半成品 Environment 会在下次安装前自动清理,不会因为中断而无法再次安装。
|
||||
|
||||
导入过程精确保留 NPZ 内深度值之间的差值。例如深度 `2.0` 和 `5.5` 在
|
||||
`Displacement Strength = -1` 时,沿局部 Z 轴相差 `3.5 Blender Unit`。它不会把数值压缩到
|
||||
`0..1`,也不再提供视觉归一化模式。
|
||||
|
||||
面片尺寸以有效深度的中位数 `Zref` 作为参考截面。先用 `mini_npz` 中的相机内参计算原始
|
||||
投影面积,再恢复源图纵横比:
|
||||
|
||||
```text
|
||||
RawWidth = depth_width × Zref × abs(Displacement Strength) / fx
|
||||
RawHeight = depth_height × Zref × abs(Displacement Strength) / fy
|
||||
Area = RawWidth × RawHeight
|
||||
Aspect = source_image_width / source_image_height
|
||||
Width = sqrt(Area × Aspect)
|
||||
Height = sqrt(Area / Aspect)
|
||||
```
|
||||
|
||||
这样即使 DA3 估计出的 `fx/fy` 不一致,也不会把正方形原图拉成长方形。主点 `cx/cy` 仍用于
|
||||
确定原始相机面片,随后对象 XY Transform 将它匹配到源 Image Empty。生成对象会记录
|
||||
`snapform_reference_depth`、`snapform_camera_plane_width` 和
|
||||
`snapform_camera_plane_height`,便于检查实际采用的
|
||||
参考深度和尺寸。
|
||||
|
||||
材质始终导入深度 Image Texture,并连接到 Displacement 节点和 Material Output;材质 Surface
|
||||
Displacement 设置为 `Displacement Only`,DA3 与 Original 都使用 `Midlevel = 0`、`Scale = 1`。MoGe-2 的位移只在专用 Geometry Nodes 中处理。
|
||||
`o_material_displacement` 为 `False` 时,材质把 Height 乘以零,置换只在几何节点中生效;为 `True` 时,
|
||||
几何节点关闭结构置换,材质 Height 恢复。色彩贴图的 Alpha 二值结果同时用于选择原始深度或有符号的
|
||||
`Background Depth`,使两种置换路径保持一致;未移除背景的普通不透明图片因此全部视为前景。
|
||||
视频与序列帧的颜色、深度与前景 Image Texture
|
||||
通过 image_user 的 Frames / Offset / Start 把第 i 帧映射到场景 `Start+i` 帧,并启用 Auto Refresh。
|
||||
颜色贴图到 Principled BSDF 之间使用独立的 Shader Node Group。Surface 使用 `O DA3 Surface Layer`;
|
||||
Depth Plane 使用 `O DA3 Depth Plane Layer` 材质组,并把背景剔除、背景深度与材质置换 Attribute
|
||||
读取全部收进组内。两者都接收 `Color`、`Alpha` 与 `Bump Scale`,只有 `O DA3 Depth Plane Layer`
|
||||
额外接收 Float 类型的 `Depth`。
|
||||
`O DA3 Surface Layer` 只输出 `Color`、`Roughness`、`Alpha` 与 `Normal`,以原图 Color 保留 Bump 效果,
|
||||
不创建 Displacement 节点或输出;`O DA3 Depth Plane Layer` 另外输出 `Displacement`。外层材质只保留贴图、
|
||||
材质组、Principled BSDF 与 Material Output,方便继续调整。
|
||||
Depth Plane 的 Bump Height 直接使用材质组的 `Color` 输入,只受独立的 `Bump Scale` 影响,不使用深度贴图,
|
||||
也不经过 `Displacement` 模式开关、`Depth Scale` 或 `Background Depth`;材质 Displacement 仍使用这些参数处理后的深度。
|
||||
无论选择哪种置换方式,原始彩色贴图都会连接
|
||||
Principled BSDF 的 `Base Color`,并通过
|
||||
`Bump.Height → Bump.Normal → Principled.Normal` 提供表面细节。Bump 的基础 Distance 为
|
||||
`0.001`,会乘以面片 Transform 的统一缩放倍率。原图 Color 还会经过 `Float Curve` 连接到
|
||||
Principled BSDF 的 `Roughness`,曲线两端输出分别为 `0.5` 和 `0.1`;材质 IOR 保持
|
||||
Principled BSDF 默认值 `1.5`。
|
||||
开启 `Unshaded` 后,色彩贴图 Color 连接 Emission,二值 Alpha 作为 Mix Shader 系数,在
|
||||
Transparent BSDF 与 Emission 之间混合后连接 Material Output;不创建 Principled BSDF、Bump 或
|
||||
Float Curve。开启 `Remove Background` 时色彩贴图使用 BEN2 输出;关闭时使用原图。
|
||||
|
||||
这里的“原始距离”是 DA3 模型输出的尺度。普通 Small、Base、Large 和 Giant 模型的深度不应
|
||||
被解释为现实世界的米;需要现实尺度时应选择带 metric depth 的
|
||||
`DA3 Nested Giant Large 1.1`。此外,当前结果仍是固定 XY 平面上的深度位移面,不是使用
|
||||
相机内参逐像素反投影得到的完整 camera-space 网格;中位深度截面为 Geometry Nodes
|
||||
深度面片提供正确的整体高宽尺度。
|
||||
|
||||
生成完成后会直接创建面片,不提供缓存重建入口或单独的导入参数。重新选择
|
||||
`Convert to Depth Plane` 会按当前 Depth from 再次转换。
|
||||
面片、深度 Image、前景 Image、Material 与 Geometry Nodes 修改器会在同一个 Blender Undo 步骤中创建;转换完成后执行
|
||||
`Ctrl+Z` 会完整撤销本次转换,不会越过转换去回退更早的 Image Empty 操作。
|
||||
|
||||
Image Empty 使用 Packed Image 时,SnapForm 会通过 Blender 写出临时 PNG 交给 DA3,
|
||||
不会 Unpack 或修改源 Image;任务结束后自动删除临时文件。普通图片仍直接读取源文件。DA3 的原始
|
||||
`results.npz` 保留在对应 job 文件夹根下,生成的浮点深度 Image 会 Pack 到
|
||||
当前 `.blend`。Depth Image 与 Displace Texture 根据源 Image datablock 命名。例如 Empty Object
|
||||
名为 `obj_img`、引用的 Image 名为 `img.png` 时,Depth Image 与 Texture 都命名为 `img.depth`;
|
||||
生成的 Object、Mesh 与 Material 仍命名为 `obj_img`。Pack 使用 OpenEXR,因而不会把大于 `1.0`
|
||||
的原始深度截断。已有同名 datablock 时由 Blender 自动追加 `.001` 等编号。
|
||||
视频输入时 Blender 无法整体 Pack 序列帧,因此 `.depth` 与颜色序列改为引用 job 目录根下的
|
||||
`depth.0001.exr`、`frame.0001.jpg`(`.frame`)或 `foreground.0001.png`,并在每帧保留原始
|
||||
浮点深度;保存 `.blend` 后再次打开仍需对应 job 目录存在。
|
||||
|
||||
选择 PLY 后,模型菜单只会提供 `DA3 Giant 1.1` 和
|
||||
`DA3 Nested Giant Large 1.1`。Small、Base 和 Large 没有 Gaussian head,无法由
|
||||
DA3 生成 `gs_ply`。
|
||||
|
||||
每个任务在 `~/.snapform/jobs/` 下用独立文件夹包裹。文件夹统一采用
|
||||
`动作-对象_日期-时间-微秒`,每种任务都有唯一名称,例如
|
||||
`generate-depth-plane_20260807-153000-123456`。微秒用于避免同一秒启动的任务共用目录。
|
||||
|
||||
Blender 为 Packed Image 或序列帧生成的输入副本也归档到对应 job 根目录:单图命名为
|
||||
`input.png`,序列帧命名为 `frame.0001.*`。它们以前暂存在 `jobs/inputs/`,现在不再创建这个
|
||||
公共中转文件夹。普通磁盘文件和视频仍直接读取原文件,不额外复制。
|
||||
|
||||
日志、运行中的状态文件、归档输入与输出产物均直接放在 job 根目录。各类任务名称为:
|
||||
|
||||
- `install-environment`
|
||||
- `download-model`
|
||||
- `export-reconstruction`
|
||||
- `generate-depth-plane`
|
||||
- `generate-image-mesh`
|
||||
- `remove-background`
|
||||
- `remove-image-background`
|
||||
- `generate-original-depth-plane`
|
||||
|
||||
DA3 官方 exporter 产生的 `gs_ply/`、`depth_vis/`、`exports/mini_npz/` 会在完成后自动平铺,
|
||||
不会保留只含一层产物的子目录:
|
||||
|
||||
```text
|
||||
export-reconstruction_20260807-153000-123456/
|
||||
├── job.log
|
||||
├── status.json
|
||||
└── scene.glb
|
||||
|
||||
export-reconstruction_20260807-153000-123456/
|
||||
├── job.log
|
||||
├── status.json
|
||||
└── *.ply
|
||||
|
||||
export-reconstruction_20260807-153000-123456/
|
||||
├── job.log
|
||||
├── status.json
|
||||
└── *.jpg
|
||||
|
||||
export-reconstruction_20260807-153000-123456/
|
||||
├── job.log
|
||||
├── status.json
|
||||
└── results.npz
|
||||
|
||||
remove-background_20260807-153000-123456/
|
||||
├── job.log
|
||||
├── status.json
|
||||
├── foreground.png 单图结果
|
||||
├── foreground.0001.png… 视频/序列帧结果
|
||||
├── frame.0001.jpg… 视频/序列帧的源帧副本
|
||||
└── depth.0001.exr… AI 深度序列帧
|
||||
```
|
||||
|
||||
建议从 `DA3 Base` 开始。它使用 Apache 2.0 模型权重;`DA3 Large 1.1` 权重采用
|
||||
CC BY-NC 4.0,仅限非商业用途。
|
||||
|
||||
## 输出格式说明
|
||||
|
||||
`Output Type` 提供以下结果:
|
||||
|
||||
- Point Cloud (GLB):DA3 `glb`
|
||||
- Gaussian Splats (PLY):DA3 `gs_ply`,同时传入 `infer_gs=True`
|
||||
- Depth Preview (Images):DA3 `depth_vis`
|
||||
- Depth Data (NPZ):DA3 `mini_npz`
|
||||
- Textured Surface (GLB):SnapForm 根据 DA3 Prediction 构建并输出 `mesh.glb`
|
||||
|
||||
Point Cloud 专属设置仅在对应类型下显示,`gs_views_interval` 仅用于 Gaussian Splats。
|
||||
Textured Surface 使用 0–100% 的 Confidence Threshold 删除置信度最低的一定比例:
|
||||
|
||||
- `process_res=504`
|
||||
- `process_res_method="upper_bound_resize"`
|
||||
- `use_ray_pose=False`
|
||||
- `ref_view_strategy="saddle_balanced"`
|
||||
- `conf_thresh_percentile=40.0`
|
||||
- `num_max_points=1_000_000`
|
||||
- `show_cameras=True`
|
||||
- `export_kwargs["gs_ply"]["gs_views_interval"]=1`
|
||||
|
||||
DA3 官方 `glb` 是彩色点云与可选相机线框;`gs_ply` 是带 Gaussian 属性的 PLY;
|
||||
`depth_vis` 是原图与着色深度图的并排预览;`mini_npz` 保存深度、置信度和可用的相机参数。
|
||||
Point Cloud GLB 任务会关闭其默认附带的深度预览,使五种输出产物保持分离。
|
||||
|
||||
官方格式与限制可参阅
|
||||
[DA3 Python API 文档](https://github.com/ByteDance-Seed/Depth-Anything-3/blob/main/docs/API.md)。
|
||||
|
||||
## AI Service
|
||||
|
||||
Environment 指磁盘上已安装的 Python、依赖及模型执行组件;Service 是由 Environment 启动、接收 Blender 任务的
|
||||
常驻 HTTP 进程。`manifest.json` 记录当前 Environment 的安装信息;Service 的 `/health` 返回进程状态和
|
||||
实例标识。项目发布前不维护安装清单或任务协议的兼容版本号。
|
||||
|
||||
Extension 本身不包含 Torch。`Install Environment` 使用 `uv` 创建独立 Python 3.12
|
||||
环境,并安装。机器上即使没有安装 `uv` 或 Python 也没关系:安装程序由 Blender
|
||||
自带的 Python 启动,缺 `uv` 时会解包扩展内置的对应平台二进制,再用它下载
|
||||
Python 3.12 与全部依赖。`uv` 会缓存在扩展存储目录下,后续安装直接复用。
|
||||
安装不做网络预检,直接开始;环境变量仍可切换镜像与代理:`PIP_INDEX_URL`(PyPI 镜像)、
|
||||
`SNAPFORM_TORCH_INDEX`(PyTorch 索引)、`HF_ENDPOINT`(Hugging Face 镜像)、
|
||||
`HTTP(S)_PROXY`/`ALL_PROXY`(代理)。未指定 PyPI 镜像时,
|
||||
pip 安装失败会自动依次重试镜像(清华、阿里云);模型下载不预探测,官方 Hugging Face
|
||||
失败后自动回退 OmooLab R2(`models.omoolab.xyz`),并校验文件大小和 SHA-256。
|
||||
GitHub 不再参与安装:DA3、BEN2、uv 已内置,缺少内置文件会直接报错提示重新安装,不再下载兜底。
|
||||
单独下载模型时遵循同样的顺序:Hugging Face(可用 `HF_ENDPOINT` 指定其他入口)失败后自动切换到
|
||||
OmooLab R2,不再使用 ModelScope。
|
||||
Windows 上代理若只在系统设置里而未写入环境变量,安装程序会尝试自动从系统代理注入子进程。
|
||||
|
||||
- PyTorch 2.9.1
|
||||
- TorchVision 0.24.1
|
||||
- xFormers 0.0.33.post2(与 PyTorch 2.9.1 对应,仅 Windows/Linux;macOS 无 wheel 自动跳过)
|
||||
- 固定到已验证 commit 的 Depth Anything 3
|
||||
- 固定到已验证 commit 的 BEN2
|
||||
- FastAPI 0.139.2 与 Uvicorn 0.51.0
|
||||
- DA3 与 BEN2 源码包随扩展打包,安装时复制到临时目录后由 uv 本地安装,完成后立即清理
|
||||
(无需访问 GitHub);SnapForm 直接调用 DA3
|
||||
底层 export 接口,`gs_ply` 导出不会触发 `gs_video` 渲染,因此不需要 gsplat
|
||||
与 C++ 编译工具链
|
||||
- `vendor/` 是不进入 Git 的本地缓存;`vendor.lock.toml` 固定下载 URL、commit/version 与
|
||||
SHA-256,`patches/<包名>/` 进入 Git 并存放 SnapForm 对源码的覆盖文件(移除 xformers
|
||||
硬依赖、export 入口改懒加载、
|
||||
射线位姿的 3×3 QR 分解在 MPS 上改走 CPU),
|
||||
`pack.py` 打包时自动套入官方包,扩展 zip 内才是修补后的版本;更新上游只需替换官方包
|
||||
并核对覆盖文件是否仍适用
|
||||
- `addict` 2.4.0(补充 DA3 上游包元数据暂未声明的运行依赖)
|
||||
|
||||
安装 Environment 时会自动选择 PyTorch 后端:检测到 NVIDIA 驱动则安装 CUDA 12.8 版本,否则安装
|
||||
CPU 版;macOS 始终安装 CPU 版 PyTorch(MPS 包含其中)。CPU 可以用于验证,但实际推理通常很慢。
|
||||
Extension Preferences 的 `Device` 决定正式功能的推理设备,默认取本机最快的 CUDA/MPS,否则
|
||||
CPU;Debug 面板的 `Debug Device` 只影响 Debug 任务。两处设备选项都只列出本机支持的设备:
|
||||
Windows 不显示 MPS,机器上没有可用 NVIDIA 驱动时不显示 CUDA。Environment 安装完成时会检测真实
|
||||
可用的设备并写入 `manifest.json`,设备列表以该检测结果为准。
|
||||
CUDA 12.8 的 PyTorch wheel 自带 CUDA 运行库,无需安装 CUDA Toolkit;只需要
|
||||
NVIDIA 驱动 ≥ 570(数据中心显卡可用更老驱动)。驱动不满足或没有 NVIDIA 显卡时,
|
||||
设备会自动回退到 CPU(macOS 为 MPS),只有显式选择 CUDA 才会报错。
|
||||
|
||||
macOS(Apple Silicon)会自动从 PyPI 安装 CPU 版 PyTorch,推理设备默认使用 MPS
|
||||
(Apple Metal)加速,也可以在设备选项里切换 CPU。PyTorch 自 2.3 起不再提供 Intel
|
||||
Mac(x86_64)轮子,Intel Mac 暂不支持。DA3 射线位姿估计中的 3×3 QR 分解在 MPS
|
||||
上尚未实现,SnapForm 的覆盖文件会单独把该分解放到 CPU 执行,其余推理仍走 MPS。
|
||||
|
||||
Extension 启用后只会查找已有的共享 Service,不会主动启动新进程。第一次执行 AI 操作或在侧边栏
|
||||
手动点击 `Start` 时才会启动 Service。Service 只绑定回环地址,并让操作系统分配动态端口;第一个实例把
|
||||
端口、进程和随机 Token 写入 `service.json`,其余 Blender 通过标准库
|
||||
`urllib.request` 连接同一个 Service。`service.lock` 保证同一 Storage Root 同时只有一个 Service,Token
|
||||
则阻止同机其他进程直接调用接口。Service 全局同时只执行一个 AI 任务。
|
||||
|
||||
每个 Blender 会注册独立客户端并每 5 秒发送心跳。正常关闭 Blender 时先注销自身;如果当前 Service
|
||||
就是由这个 Blender 启动的,会直接终止它,无需等待。异常退出的客户端会在 10 秒后过期;Service
|
||||
在最后一个客户端离开后仍会保留模型缓存等待 20 秒再退出,作为 Service 由其他进程启动等场景的兜底。
|
||||
取消任务不会重启 Service,并会保留已经加载的模型;
|
||||
如果取消发生在一次不可中断的 Torch 调用中,任务会在该调用返回后的最近检查点结束。
|
||||
|
||||
侧边栏 `AI Service` 面板以 `Running (Clients: n)` 表示空闲服务,以 `Running (Busy)` 表示正在执行任务,
|
||||
并显示缓存模型和活动任务。Blender 完成结果导入时会立即释放当前任务对应的本地 Busy 快照,无需等待
|
||||
下一次 Service 轮询。空闲且只有当前 Blender 连接时,可以清理模型缓存、重启或停止 Service;
|
||||
多客户端或任务执行期间不会允许破坏共享服务。
|
||||
|
||||
BEN2 在首次使用后保持加载;DA3 保留最近使用的一个模型。切换 DA3 模型时会释放旧模型;加载时如果
|
||||
显存不足,会先释放另一类缓存模型并重试。
|
||||
|
||||
Environment Manifest Revision 变化时,旧 Environment 需要重新执行 `Install Environment`。
|
||||
Service 的启动和崩溃日志写入 `~/.snapform/service.log`。
|
||||
|
||||
Environment、模型和任务结果默认保存在用户主目录的 `~/.snapform`:
|
||||
|
||||
```text
|
||||
~/.snapform/
|
||||
├── .venv/
|
||||
├── models/
|
||||
├── jobs/
|
||||
├── service.log
|
||||
├── service.json # Service 运行时存在
|
||||
└── service.lock # Service 运行时持有
|
||||
```
|
||||
|
||||
`models/` 下每个模型一个文件夹,命名统一带前缀:DA3 为 `da3_*`(如 `da3_small`、`da3_base`),
|
||||
BEN2 为 `ben2_*`(如 `ben2_base`)。旧版无前缀的 DA3 文件夹会在首次使用时自动改名为新名称。
|
||||
|
||||
不同 Blender 版本安装相同 Extension 后会共用这个目录。也可以在 Extension
|
||||
Preferences 的 `Storage Root` 中修改共享位置。不要把模型写入 Extension 安装目录,
|
||||
因为该目录可能只读,并且升级时会被替换。不要同时从多个 Blender 实例执行
|
||||
`Install Environment`;Service、任务队列和模型读取可以共享。
|
||||
|
||||
## 开发
|
||||
|
||||
```bash
|
||||
uv run python prepare_vendor.py
|
||||
uv run pytest tests -v
|
||||
uv run python pack.py
|
||||
```
|
||||
|
||||
项目结构:
|
||||
|
||||
```text
|
||||
.
|
||||
├── vendor.lock.toml 第三方构建输入的版本、URL 与 SHA-256
|
||||
├── prepare_vendor.py 下载并校验第三方构建输入
|
||||
├── pack.py
|
||||
├── patches/ 进入 Git 的 DA3 源码覆盖文件
|
||||
├── vendor/ 不进入 Git 的下载缓存;本地调试也直接读取这里
|
||||
├── src/
|
||||
│ └── snapform/
|
||||
│ ├── blender_manifest.toml
|
||||
│ ├── service.py
|
||||
│ ├── install_env.py
|
||||
│ ├── models.py
|
||||
│ ├── image_depth.py
|
||||
│ ├── operators.py
|
||||
│ ├── panel.py
|
||||
│ ├── preferences.py
|
||||
│ ├── properties.py
|
||||
│ ├── client.py
|
||||
│ └── worker.py
|
||||
└── tests/
|
||||
```
|
||||
|
||||
## 上游与许可证
|
||||
|
||||
- Depth Anything 3 代码采用 Apache 2.0。
|
||||
- BEN2 代码与模型权重采用 MIT License。
|
||||
- 本 Extension 代码采用 GPL-3.0-or-later。
|
||||
- 模型权重许可证由所选 Hugging Face 模型仓库决定,Extension 下载前会在界面显示。
|
||||
124
pack.py
124
pack.py
@@ -1,6 +1,3 @@
|
||||
import hashlib
|
||||
import io
|
||||
import tarfile
|
||||
import tomllib
|
||||
from pathlib import Path
|
||||
from zipfile import ZIP_DEFLATED, ZipFile
|
||||
@@ -8,93 +5,8 @@ from zipfile import ZIP_DEFLATED, ZipFile
|
||||
|
||||
PROJECT_ROOT = Path(__file__).resolve().parent
|
||||
SOURCE_DIR = PROJECT_ROOT / "src" / "snapform"
|
||||
VENDOR_DIR = PROJECT_ROOT / "vendor"
|
||||
DIST_DIR = PROJECT_ROOT / "dist"
|
||||
PATCHES_DIR = PROJECT_ROOT / "patches"
|
||||
LOCK_PATH = PROJECT_ROOT / "vendor.lock.toml"
|
||||
|
||||
|
||||
def load_vendor_lock():
|
||||
"""Return the locked third-party artifact metadata."""
|
||||
with LOCK_PATH.open("rb") as stream:
|
||||
lock = tomllib.load(stream)
|
||||
if lock.get("schema_version") != 1:
|
||||
raise RuntimeError("Unsupported vendor.lock.toml schema")
|
||||
return lock
|
||||
|
||||
|
||||
VENDOR_LOCK = load_vendor_lock()
|
||||
PLATFORMS = {
|
||||
platform: artifact["filename"]
|
||||
for platform, artifact in VENDOR_LOCK["platforms"].items()
|
||||
}
|
||||
BUNDLED_ARCHIVES = {
|
||||
f"vendor/{artifact['filename']}": artifact["filename"]
|
||||
for artifact in VENDOR_LOCK["artifacts"].values()
|
||||
}
|
||||
OVERLAYS = {
|
||||
artifact["filename"]: artifact["patches"]
|
||||
for artifact in VENDOR_LOCK["artifacts"].values()
|
||||
if "patches" in artifact
|
||||
}
|
||||
|
||||
|
||||
def file_sha256(path):
|
||||
"""Return the SHA-256 digest for a file."""
|
||||
digest = hashlib.sha256()
|
||||
with Path(path).open("rb") as stream:
|
||||
while chunk := stream.read(1024 * 1024):
|
||||
digest.update(chunk)
|
||||
return digest.hexdigest()
|
||||
|
||||
|
||||
def validate_artifact(path, expected_sha256):
|
||||
"""Require an artifact to exist and match its locked digest."""
|
||||
if not path.is_file():
|
||||
raise RuntimeError(
|
||||
f"Missing vendor artifact: {path}. "
|
||||
"Run: uv run python prepare_vendor.py"
|
||||
)
|
||||
actual = file_sha256(path)
|
||||
if actual != expected_sha256:
|
||||
raise RuntimeError(
|
||||
f"Vendor artifact hash mismatch: {path}. "
|
||||
"Run: uv run python prepare_vendor.py"
|
||||
)
|
||||
|
||||
|
||||
def bundled_artifact_bytes(filename):
|
||||
"""Return a bundled artifact with its configured overlay applied."""
|
||||
archive_path = VENDOR_DIR / filename
|
||||
overlay_dir = PATCHES_DIR / OVERLAYS.get(filename, filename)
|
||||
if not overlay_dir.is_dir():
|
||||
return archive_path.read_bytes()
|
||||
if not filename.endswith(".tar.gz"):
|
||||
raise ValueError(f"Unsupported overlay archive: {filename}")
|
||||
return _apply_tar_overlay(archive_path, overlay_dir)
|
||||
|
||||
|
||||
def _apply_tar_overlay(archive_path, overlay_dir):
|
||||
with tarfile.open(archive_path, "r:gz") as source:
|
||||
members = source.getmembers()
|
||||
top_level = members[0].name.split("/", 1)[0]
|
||||
buffer = io.BytesIO()
|
||||
with tarfile.open(fileobj=buffer, mode="w:gz") as output:
|
||||
for member in members:
|
||||
if member.isdir() or member.islnk() or member.issym():
|
||||
output.addfile(member)
|
||||
continue
|
||||
relative = member.name[len(top_level) + 1:]
|
||||
overlay_file = overlay_dir / relative
|
||||
if overlay_file.is_file():
|
||||
info = tarfile.TarInfo(member.name)
|
||||
info.size = overlay_file.stat().st_size
|
||||
info.mode = member.mode
|
||||
output.addfile(info, overlay_file.open("rb"))
|
||||
else:
|
||||
with source.extractfile(member) as extracted:
|
||||
output.addfile(member, extracted)
|
||||
return buffer.getvalue()
|
||||
PLATFORMS = ("windows-x64", "macos-arm64", "linux-x64")
|
||||
|
||||
|
||||
def manifest_for_platform(platform):
|
||||
@@ -119,31 +31,6 @@ def manifest_for_platform(platform):
|
||||
return "\n".join(lines) + "\n"
|
||||
|
||||
|
||||
def bundled_vendor_files(platform):
|
||||
entries = {
|
||||
name: VENDOR_DIR / filename
|
||||
for name, filename in BUNDLED_ARCHIVES.items()
|
||||
}
|
||||
entries[f"vendor/{PLATFORMS[platform]}"] = (
|
||||
VENDOR_DIR / PLATFORMS[platform]
|
||||
)
|
||||
return entries
|
||||
|
||||
|
||||
def validate_vendor_files(platform):
|
||||
"""Validate common archives and the selected uv archive."""
|
||||
for artifact in VENDOR_LOCK["artifacts"].values():
|
||||
validate_artifact(
|
||||
VENDOR_DIR / artifact["filename"],
|
||||
artifact["sha256"],
|
||||
)
|
||||
uv_artifact = VENDOR_LOCK["platforms"][platform]
|
||||
validate_artifact(
|
||||
VENDOR_DIR / uv_artifact["filename"],
|
||||
uv_artifact["sha256"],
|
||||
)
|
||||
|
||||
|
||||
def build_extension(platform):
|
||||
if platform not in PLATFORMS:
|
||||
raise ValueError(f"Unsupported platform: {platform}")
|
||||
@@ -152,8 +39,6 @@ def build_extension(platform):
|
||||
f"{manifest['name']}.v{manifest['version']}.{platform}.zip"
|
||||
)
|
||||
DIST_DIR.mkdir(exist_ok=True)
|
||||
vendor_files = bundled_vendor_files(platform)
|
||||
validate_vendor_files(platform)
|
||||
with ZipFile(output_path, "w", ZIP_DEFLATED) as archive:
|
||||
for source_path in sorted(SOURCE_DIR.rglob("*")):
|
||||
if not source_path.is_file():
|
||||
@@ -161,8 +46,6 @@ def build_extension(platform):
|
||||
if "__pycache__" in source_path.parts:
|
||||
continue
|
||||
relative_path = source_path.relative_to(SOURCE_DIR)
|
||||
if relative_path.parts[0] == "vendor":
|
||||
continue
|
||||
if source_path.name == "blender_manifest.toml":
|
||||
continue
|
||||
archive.write(source_path, relative_path)
|
||||
@@ -170,11 +53,6 @@ def build_extension(platform):
|
||||
"blender_manifest.toml",
|
||||
manifest_for_platform(platform),
|
||||
)
|
||||
for archive_name, source_path in vendor_files.items():
|
||||
archive.writestr(
|
||||
archive_name,
|
||||
bundled_artifact_bytes(source_path.name),
|
||||
)
|
||||
return output_path
|
||||
|
||||
|
||||
|
||||
@@ -1,93 +0,0 @@
|
||||
[build-system]
|
||||
requires = ["hatchling>=1.25", "hatch-vcs>=0.4"]
|
||||
build-backend = "hatchling.build"
|
||||
|
||||
[project]
|
||||
name = "depth-anything-3"
|
||||
version = "0.0.0"
|
||||
description = "Depth Anything 3"
|
||||
readme = "README.md"
|
||||
requires-python = ">=3.9, <=3.13"
|
||||
license = { text = "Apache-2.0" }
|
||||
authors = [{ name = "Your Name" }]
|
||||
|
||||
dependencies = [
|
||||
"pre-commit",
|
||||
"trimesh",
|
||||
"torch>=2",
|
||||
"torchvision",
|
||||
"einops",
|
||||
"huggingface_hub",
|
||||
"imageio",
|
||||
"numpy<2",
|
||||
"opencv-python",
|
||||
"open3d",
|
||||
"fastapi",
|
||||
"uvicorn",
|
||||
"requests",
|
||||
"typer",
|
||||
"pillow",
|
||||
"omegaconf",
|
||||
"evo",
|
||||
"e3nn",
|
||||
"moviepy",
|
||||
"plyfile",
|
||||
"pillow_heif",
|
||||
"safetensors",
|
||||
"uvicorn",
|
||||
"moviepy==1.0.3",
|
||||
"typer>=0.9.0",
|
||||
"pycolmap",
|
||||
]
|
||||
|
||||
[project.optional-dependencies]
|
||||
app = ["gradio>=5", "pillow>=9.0"] # requires that python3>=3.10
|
||||
gs = ["gsplat @ git+https://github.com/nerfstudio-project/gsplat.git@0b4dddf04cb687367602c01196913cde6a743d70"]
|
||||
all = ["depth-anything-3[app,gs]"]
|
||||
|
||||
|
||||
[project.scripts]
|
||||
da3 = "depth_anything_3.cli:app"
|
||||
|
||||
[project.urls]
|
||||
Homepage = "https://github.com/ByteDance-Seed/Depth-Anything-3"
|
||||
|
||||
[tool.hatch.version]
|
||||
source = "vcs"
|
||||
|
||||
[tool.hatch.build.targets.wheel]
|
||||
packages = ["src/depth_anything_3"]
|
||||
|
||||
[tool.hatch.build.targets.sdist]
|
||||
include = [
|
||||
"/README.md",
|
||||
"/pyproject.toml",
|
||||
"/src/depth_anything_3",
|
||||
]
|
||||
|
||||
[tool.hatch.metadata]
|
||||
allow-direct-references = true
|
||||
|
||||
[tool.mypy]
|
||||
plugins = ["jaxtyping.mypy_plugin"]
|
||||
|
||||
[tool.black]
|
||||
line-length = 99
|
||||
target-version = ['py37', 'py38', 'py39', 'py310', 'py311']
|
||||
include = '\.pyi?$'
|
||||
exclude = '''
|
||||
/(
|
||||
| \.git
|
||||
)/
|
||||
'''
|
||||
|
||||
[tool.isort]
|
||||
profile = "black"
|
||||
multi_line_output = 3
|
||||
include_trailing_comma = true
|
||||
known_third_party = ["bson","cruise","cv2","dataloader","diffusers","omegaconf","tensorflow","torch","torchvision","transformers","gsplat"]
|
||||
known_first_party = ["common", "data", "models", "projects"]
|
||||
sections = ["FUTURE","STDLIB","THIRDPARTY","FIRSTPARTY","LOCALFOLDER"]
|
||||
skip_gitignore = true
|
||||
line_length = 99
|
||||
no_lines_before="THIRDPARTY"
|
||||
@@ -1,23 +0,0 @@
|
||||
pre-commit
|
||||
trimesh
|
||||
torch>=2
|
||||
torchvision
|
||||
einops
|
||||
huggingface_hub
|
||||
imageio
|
||||
numpy<2
|
||||
opencv-python
|
||||
open3d
|
||||
fastapi
|
||||
uvicorn
|
||||
requests
|
||||
typer
|
||||
pillow
|
||||
omegaconf
|
||||
evo
|
||||
e3nn
|
||||
moviepy
|
||||
plyfile
|
||||
pillow_heif
|
||||
safetensors
|
||||
pycolmap
|
||||
@@ -1,84 +0,0 @@
|
||||
# Copyright (c) 2025 ByteDance Ltd. and/or its affiliates
|
||||
#
|
||||
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||
# you may not use this file except in compliance with the License.
|
||||
# You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
|
||||
from depth_anything_3.specs import Prediction
|
||||
|
||||
# Canonical set of export formats this dispatcher understands. Treat this as the
|
||||
# server-trusted enum for any request-facing validation (e.g. the HTTP backend) instead
|
||||
# of letting a client-supplied export_format string reach the dispatcher unchecked.
|
||||
SUPPORTED_EXPORT_FORMATS = frozenset(
|
||||
{
|
||||
"glb",
|
||||
"mini_npz",
|
||||
"npz",
|
||||
"feat_vis",
|
||||
"depth_vis",
|
||||
"gs_ply",
|
||||
"gs_video",
|
||||
"colmap",
|
||||
}
|
||||
)
|
||||
|
||||
|
||||
def export(
|
||||
prediction: Prediction,
|
||||
export_format: str,
|
||||
export_dir: str,
|
||||
**kwargs,
|
||||
):
|
||||
if "-" in export_format:
|
||||
export_formats = export_format.split("-")
|
||||
for export_format in export_formats:
|
||||
export(prediction, export_format, export_dir, **kwargs)
|
||||
return # Prevent falling through to single-format handling
|
||||
|
||||
if export_format == "glb":
|
||||
from .glb import export_to_glb
|
||||
|
||||
export_to_glb(prediction, export_dir, **kwargs.get(export_format, {}))
|
||||
elif export_format == "mini_npz":
|
||||
from .npz import export_to_mini_npz
|
||||
|
||||
export_to_mini_npz(prediction, export_dir)
|
||||
elif export_format == "npz":
|
||||
from .npz import export_to_npz
|
||||
|
||||
export_to_npz(prediction, export_dir)
|
||||
elif export_format == "feat_vis":
|
||||
from .feat_vis import export_to_feat_vis
|
||||
|
||||
export_to_feat_vis(prediction, export_dir, **kwargs.get(export_format, {}))
|
||||
elif export_format == "depth_vis":
|
||||
from .depth_vis import export_to_depth_vis
|
||||
|
||||
export_to_depth_vis(prediction, export_dir)
|
||||
elif export_format == "gs_ply":
|
||||
from .gs import export_to_gs_ply
|
||||
|
||||
export_to_gs_ply(prediction, export_dir, **kwargs.get(export_format, {}))
|
||||
elif export_format == "gs_video":
|
||||
from .gs import export_to_gs_video
|
||||
|
||||
export_to_gs_video(prediction, export_dir, **kwargs.get(export_format, {}))
|
||||
elif export_format == "colmap":
|
||||
from .colmap import export_to_colmap
|
||||
|
||||
export_to_colmap(prediction, export_dir, **kwargs.get(export_format, {}))
|
||||
else:
|
||||
raise ValueError(f"Unsupported export format: {export_format}")
|
||||
|
||||
|
||||
__all__ = [
|
||||
export,
|
||||
]
|
||||
@@ -1,531 +0,0 @@
|
||||
# Copyright (c) 2025 ByteDance Ltd. and/or its affiliates
|
||||
#
|
||||
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||
# you may not use this file except in compliance with the License.
|
||||
# You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
|
||||
import torch
|
||||
from einops import repeat
|
||||
from .geometry import unproject_depth
|
||||
|
||||
|
||||
def compute_optimal_rotation_intrinsics_batch(
|
||||
rays_origin, rays_target, z_threshold=1e-4, reproj_threshold=0.2, weights=None,
|
||||
n_sample = None,
|
||||
n_iter=100,
|
||||
num_sample_for_ransac=8,
|
||||
rand_sample_iters_idx=None,
|
||||
):
|
||||
"""
|
||||
Args:
|
||||
rays_origin (torch.Tensor): (B, N, 3)
|
||||
rays_target (torch.Tensor): (B, N, 3)
|
||||
z_threshold (float): Threshold for z value to be considered valid.
|
||||
|
||||
Returns:
|
||||
R (torch.tensor): (3, 3)
|
||||
focal_length (torch.tensor): (2,)
|
||||
principal_point (torch.tensor): (2,)
|
||||
"""
|
||||
device = rays_origin.device
|
||||
B, N, _ = rays_origin.shape
|
||||
z_mask = torch.logical_and(
|
||||
torch.abs(rays_target[:, :, 2]) > z_threshold, torch.abs(rays_origin[:, :, 2]) > z_threshold
|
||||
) # (B, N, 1)
|
||||
rays_origin = rays_origin.clone()
|
||||
rays_target = rays_target.clone()
|
||||
rays_origin[:, :, 0][z_mask] /= rays_origin[:, :, 2][z_mask]
|
||||
rays_origin[:, :, 1][z_mask] /= rays_origin[:, :, 2][z_mask]
|
||||
rays_target[:, :, 0][z_mask] /= rays_target[:, :, 2][z_mask]
|
||||
rays_target[:, :, 1][z_mask] /= rays_target[:, :, 2][z_mask]
|
||||
|
||||
rays_origin = rays_origin[:, :, :2]
|
||||
rays_target = rays_target[:, :, :2]
|
||||
assert weights is not None, "weights must be provided"
|
||||
weights[~z_mask] = 0
|
||||
|
||||
A_list = []
|
||||
max_chunk_size = 2
|
||||
for i in range(0, rays_origin.shape[0], max_chunk_size):
|
||||
A = ransac_find_homography_weighted_fast_batch(
|
||||
rays_origin[i:i+max_chunk_size],
|
||||
rays_target[i:i+max_chunk_size],
|
||||
weights[i:i+max_chunk_size],
|
||||
n_iter=n_iter,
|
||||
n_sample = n_sample,
|
||||
num_sample_for_ransac=num_sample_for_ransac,
|
||||
reproj_threshold=reproj_threshold,
|
||||
rand_sample_iters_idx=rand_sample_iters_idx,
|
||||
max_inlier_num=8000,
|
||||
)
|
||||
A = A.to(device)
|
||||
A_need_inv_mask = torch.linalg.det(A) < 0
|
||||
A[A_need_inv_mask] = -A[A_need_inv_mask]
|
||||
A_list.append(A)
|
||||
|
||||
A = torch.cat(A_list, dim=0)
|
||||
|
||||
R_list = []
|
||||
f_list = []
|
||||
pp_list = []
|
||||
for i in range(A.shape[0]):
|
||||
R, L = ql_decomposition(A[i])
|
||||
L = L / L[2][2]
|
||||
|
||||
f = torch.stack((L[0][0], L[1][1]))
|
||||
pp = torch.stack((L[2][0], L[2][1]))
|
||||
R_list.append(R)
|
||||
f_list.append(f)
|
||||
pp_list.append(pp)
|
||||
|
||||
R = torch.stack(R_list)
|
||||
f = torch.stack(f_list)
|
||||
pp = torch.stack(pp_list)
|
||||
|
||||
return R, f, pp
|
||||
|
||||
|
||||
# https://www.reddit.com/r/learnmath/comments/v1crd7/linear_algebra_qr_to_ql_decomposition/
|
||||
def ql_decomposition(A):
|
||||
device = A.device
|
||||
P = torch.tensor([[0, 0, 1], [0, 1, 0], [1, 0, 0]], device=device).float()
|
||||
A_tilde = torch.matmul(A, P)
|
||||
# MPS 尚未实现 torch.linalg.qr,仅把 3x3 分解放到 CPU 执行后移回
|
||||
use_cpu = device.type == "mps"
|
||||
if use_cpu:
|
||||
A_tilde = A_tilde.cpu()
|
||||
Q_tilde, R_tilde = torch.linalg.qr(A_tilde)
|
||||
if use_cpu:
|
||||
Q_tilde = Q_tilde.to(device=device)
|
||||
R_tilde = R_tilde.to(device=device)
|
||||
Q = torch.matmul(Q_tilde, P)
|
||||
L = torch.matmul(torch.matmul(P, R_tilde), P)
|
||||
d = torch.diag(L)
|
||||
Q[:, 0] *= torch.sign(d[0])
|
||||
Q[:, 1] *= torch.sign(d[1])
|
||||
Q[:, 2] *= torch.sign(d[2])
|
||||
L[0] *= torch.sign(d[0])
|
||||
L[1] *= torch.sign(d[1])
|
||||
L[2] *= torch.sign(d[2])
|
||||
return Q, L
|
||||
|
||||
def find_homography_least_squares_weighted_torch(src_pts, dst_pts, confident_weight):
|
||||
"""
|
||||
src_pts: (N,2) source points (torch.Tensor, float32/float64)
|
||||
dst_pts: (N,2) target points (torch.Tensor, float32/float64)
|
||||
confident_weight: (N,) weights (torch.Tensor)
|
||||
Returns: (3,3) homography matrix H (torch.Tensor)
|
||||
"""
|
||||
assert src_pts.shape == dst_pts.shape
|
||||
N = src_pts.shape[0]
|
||||
if N < 4:
|
||||
raise ValueError("At least 4 points are required to compute homography.")
|
||||
assert confident_weight.shape == (N,)
|
||||
|
||||
w = confident_weight.sqrt().unsqueeze(1) # (N,1)
|
||||
|
||||
x = src_pts[:, 0:1] # (N,1)
|
||||
y = src_pts[:, 1:2] # (N,1)
|
||||
u = dst_pts[:, 0:1]
|
||||
v = dst_pts[:, 1:2]
|
||||
|
||||
zeros = torch.zeros_like(x)
|
||||
|
||||
# Construct A matrix (2N, 9)
|
||||
A1 = torch.cat([-x * w, -y * w, -w, zeros, zeros, zeros, x * u * w, y * u * w, u * w], dim=1)
|
||||
A2 = torch.cat([zeros, zeros, zeros, -x * w, -y * w, -w, x * v * w, y * v * w, v * w], dim=1)
|
||||
A = torch.cat([A1, A2], dim=0) # (2N, 9)
|
||||
|
||||
# SVD
|
||||
# Note: torch.linalg.svd returns U, S, Vh, where Vh is the transpose of V
|
||||
_, _, Vh = torch.linalg.svd(A)
|
||||
H = Vh[-1].reshape(3, 3)
|
||||
H = H / H[-1, -1]
|
||||
return H
|
||||
|
||||
|
||||
def ransac_find_homography_weighted(
|
||||
src_pts,
|
||||
dst_pts,
|
||||
confident_weight,
|
||||
n_iter=100,
|
||||
sample_ratio=0.2,
|
||||
reproj_threshold=3.0,
|
||||
num_sample_for_ransac=16,
|
||||
random_seed=None,
|
||||
):
|
||||
"""
|
||||
RANSAC version of weighted Homography estimation.
|
||||
Sample 4 points from the top 50% weighted points each time.
|
||||
reproj_threshold: points with reprojection error less than this value are inliers
|
||||
Returns: best_H
|
||||
"""
|
||||
if random_seed is not None:
|
||||
torch.manual_seed(random_seed)
|
||||
N = src_pts.shape[0]
|
||||
assert N >= 4
|
||||
# 1. Select top 50% weighted points
|
||||
sorted_idx = torch.argsort(confident_weight, descending=True)
|
||||
n_sample = max(num_sample_for_ransac, int(N * sample_ratio))
|
||||
candidate_idx = sorted_idx[:n_sample]
|
||||
best_inlier_mask = None
|
||||
best_score = 0
|
||||
for _ in range(n_iter):
|
||||
# 2. Randomly sample 4 points
|
||||
idx = candidate_idx[torch.randperm(n_sample)[:num_sample_for_ransac]]
|
||||
# 3. Compute Homography
|
||||
try:
|
||||
H = find_homography_least_squares_weighted_torch(
|
||||
src_pts[idx], dst_pts[idx], confident_weight[idx]
|
||||
)
|
||||
except Exception:
|
||||
H = torch.eye(3, dtype=src_pts.dtype, device=src_pts.device)
|
||||
# 4. Compute reprojection error for all points
|
||||
src_homo = torch.cat(
|
||||
[src_pts, torch.ones(N, 1, dtype=src_pts.dtype, device=src_pts.device)], dim=1
|
||||
)
|
||||
proj = (H @ src_homo.T).T
|
||||
proj = proj[:, :2] / proj[:, 2:3]
|
||||
error = ((proj - dst_pts) ** 2).sum(dim=1).sqrt() # Euclidean distance
|
||||
inlier_mask = error < reproj_threshold
|
||||
total_score = (inlier_mask * confident_weight).sum().item()
|
||||
n_inlier = inlier_mask.sum().item()
|
||||
if n_inlier < 4:
|
||||
continue # At least 4 inliers required for fitting
|
||||
|
||||
if total_score > best_score:
|
||||
best_score = total_score
|
||||
best_inlier_mask = inlier_mask
|
||||
|
||||
# 5. Refit Homography using inliers
|
||||
H_inlier = find_homography_least_squares_weighted_torch(
|
||||
src_pts[best_inlier_mask], dst_pts[best_inlier_mask], confident_weight[best_inlier_mask]
|
||||
)
|
||||
|
||||
return H_inlier
|
||||
|
||||
|
||||
def find_homography_least_squares_weighted_torch_batch(
|
||||
src_pts_batch, dst_pts_batch, confident_weight_batch
|
||||
):
|
||||
"""
|
||||
Batch version of weighted least squares Homography
|
||||
src_pts_batch: (B, K, 2)
|
||||
dst_pts_batch: (B, K, 2)
|
||||
confident_weight_batch: (B, K)
|
||||
Returns: (B, 3, 3)
|
||||
"""
|
||||
B, K, _ = src_pts_batch.shape
|
||||
w = confident_weight_batch.sqrt().unsqueeze(2) # (B,K,1)
|
||||
x = src_pts_batch[:, :, 0:1]
|
||||
y = src_pts_batch[:, :, 1:2]
|
||||
u = dst_pts_batch[:, :, 0:1]
|
||||
v = dst_pts_batch[:, :, 1:2]
|
||||
zeros = torch.zeros_like(x)
|
||||
A1 = torch.cat([-x * w, -y * w, -w, zeros, zeros, zeros, x * u * w, y * u * w, u * w], dim=2)
|
||||
A2 = torch.cat([zeros, zeros, zeros, -x * w, -y * w, -w, x * v * w, y * v * w, v * w], dim=2)
|
||||
A = torch.cat([A1, A2], dim=1) # (B, 2K, 9)
|
||||
# SVD: torch.linalg.svd supports batch
|
||||
_, _, Vh = torch.linalg.svd(A)
|
||||
H = Vh[:, -1].reshape(B, 3, 3)
|
||||
H = H / H[:, 2:3, 2:3]
|
||||
return H
|
||||
|
||||
|
||||
def ransac_find_homography_weighted_fast(
|
||||
src_pts,
|
||||
dst_pts,
|
||||
confident_weight,
|
||||
n_sample,
|
||||
n_iter=100,
|
||||
reproj_threshold=3.0,
|
||||
num_sample_for_ransac=8,
|
||||
random_seed=None,
|
||||
rand_sample_iters_idx=None,
|
||||
):
|
||||
"""
|
||||
Batch version of RANSAC weighted Homography estimation.
|
||||
Returns: H_inlier
|
||||
"""
|
||||
if random_seed is not None:
|
||||
torch.manual_seed(random_seed)
|
||||
N = src_pts.shape[0]
|
||||
device = src_pts.device
|
||||
assert N >= 4
|
||||
# 1. Select top weighted points by sample_ratio
|
||||
sorted_idx = torch.argsort(confident_weight, descending=True)
|
||||
candidate_idx = sorted_idx[:n_sample] # (n_sample,)
|
||||
if rand_sample_iters_idx is None:
|
||||
rand_sample_iters_idx = torch.stack(
|
||||
[torch.randperm(n_sample, device=device)[:num_sample_for_ransac] for _ in range(n_iter)],
|
||||
dim=0,
|
||||
) # (n_iter, num_sample_for_ransac)
|
||||
# 2. Generate all sampling groups at once
|
||||
# shape: (n_iter, num_sample_for_ransac)
|
||||
rand_idx = candidate_idx[rand_sample_iters_idx] # (n_iter, num_sample_for_ransac)
|
||||
# 3. Construct batch input
|
||||
src_pts_batch = src_pts[rand_idx] # (n_iter, num_sample_for_ransac, 2)
|
||||
dst_pts_batch = dst_pts[rand_idx] # (n_iter, num_sample_for_ransac, 2)
|
||||
confident_weight_batch = confident_weight[rand_idx] # (n_iter, num_sample_for_ransac)
|
||||
# 4. Batch fit Homography
|
||||
H_batch = find_homography_least_squares_weighted_torch_batch(
|
||||
src_pts_batch, dst_pts_batch, confident_weight_batch
|
||||
) # (n_iter, 3, 3)
|
||||
# 5. Batch evaluate inliers for all H
|
||||
src_homo = torch.cat(
|
||||
[src_pts, torch.ones(N, 1, dtype=src_pts.dtype, device=src_pts.device)], dim=1
|
||||
) # (N,3)
|
||||
src_homo_expand = src_homo.unsqueeze(0).expand(n_iter, N, 3) # (n_iter, N, 3)
|
||||
dst_pts_expand = dst_pts.unsqueeze(0).expand(n_iter, N, 2) # (n_iter, N, 2)
|
||||
confident_weight_expand = confident_weight.unsqueeze(0).expand(n_iter, N) # (n_iter, N)
|
||||
# H_batch: (n_iter, 3, 3)
|
||||
proj = torch.bmm(src_homo_expand, H_batch.transpose(1, 2)) # (n_iter, N, 3)
|
||||
proj_xy = proj[:, :, :2] / proj[:, :, 2:3] # (n_iter, N, 2)
|
||||
error = ((proj_xy - dst_pts_expand) ** 2).sum(dim=2).sqrt() # (n_iter, N)
|
||||
inlier_mask = error < reproj_threshold # (n_iter, N)
|
||||
total_score = (inlier_mask * confident_weight_expand).sum(dim=1) # (n_iter,)
|
||||
# 6. Select the sampling group with the highest score
|
||||
best_idx = torch.argmax(total_score)
|
||||
best_inlier_mask = inlier_mask[best_idx] # (N,)
|
||||
inlier_src_pts = src_pts[best_inlier_mask]
|
||||
inlier_dst_pts = dst_pts[best_inlier_mask]
|
||||
inlier_confident_weight = confident_weight[best_inlier_mask]
|
||||
|
||||
max_inlier_num = 10000
|
||||
sorted_idx = torch.argsort(inlier_confident_weight, descending=True)
|
||||
|
||||
# method 1: sort according to confident_weight, and only keep max_inlier_num pts
|
||||
# sorted_idx = sorted_idx[:max_inlier_num]
|
||||
|
||||
# method 2: random choose max_inlier_num pts
|
||||
sorted_idx = sorted_idx[torch.randperm(len(sorted_idx))[:max_inlier_num]]
|
||||
|
||||
inlier_src_pts = inlier_src_pts[sorted_idx]
|
||||
inlier_dst_pts = inlier_dst_pts[sorted_idx]
|
||||
inlier_confident_weight = inlier_confident_weight[sorted_idx]
|
||||
# 7. Refit Homography using inliers
|
||||
H_inlier = find_homography_least_squares_weighted_torch(
|
||||
inlier_src_pts, inlier_dst_pts, inlier_confident_weight
|
||||
)
|
||||
return H_inlier
|
||||
|
||||
|
||||
def ransac_find_homography_weighted_fast_batch(
|
||||
src_pts, # (B, N, 3)
|
||||
dst_pts, # (B, N, 2)
|
||||
confident_weight, # (B, N)
|
||||
n_sample,
|
||||
n_iter=100,
|
||||
reproj_threshold=3.0,
|
||||
num_sample_for_ransac=8,
|
||||
max_inlier_num=10000,
|
||||
random_seed=None,
|
||||
rand_sample_iters_idx=None,
|
||||
):
|
||||
"""
|
||||
Batch version of RANSAC weighted Homography estimation (supports batch).
|
||||
Input:
|
||||
src_pts: (B, N, 2)
|
||||
dst_pts: (B, N, 2)
|
||||
confident_weight: (B, N)
|
||||
Returns:
|
||||
H_inlier: (B, 3, 3)
|
||||
"""
|
||||
if random_seed is not None:
|
||||
torch.manual_seed(random_seed)
|
||||
B, N, _ = src_pts.shape
|
||||
assert N >= 4
|
||||
|
||||
device = src_pts.device
|
||||
|
||||
# 1. Select top weighted points by sample_ratio
|
||||
sorted_idx = torch.argsort(confident_weight, descending=True, dim=1) # (B, N)
|
||||
candidate_idx = sorted_idx[:, :n_sample] # (B, n_sample)
|
||||
|
||||
# 2. Generate all sampling groups at once
|
||||
# rand_idx: (B, n_iter, num_sample_for_ransac)
|
||||
if rand_sample_iters_idx is None:
|
||||
rand_sample_iters_idx = torch.stack(
|
||||
[torch.randperm(n_sample, device=device)[:num_sample_for_ransac] for _ in range(n_iter)],
|
||||
dim=0,
|
||||
) # (n_iter, num_sample_for_ransac)
|
||||
|
||||
rand_idx = candidate_idx[:, rand_sample_iters_idx] # (B, n_iter, num_sample_for_ransac)
|
||||
|
||||
# 3. Construct batch input
|
||||
# Indexing method below: (B, n_iter, num_sample_for_ransac, ...)
|
||||
b_idx = torch.arange(B, device=device).view(B, 1, 1).expand(B, n_iter, num_sample_for_ransac)
|
||||
src_pts_batch = src_pts[b_idx, rand_idx] # (B, n_iter, num_sample_for_ransac, 2)
|
||||
dst_pts_batch = dst_pts[b_idx, rand_idx] # (B, n_iter, num_sample_for_ransac, 2)
|
||||
confident_weight_batch = confident_weight[b_idx, rand_idx] # (B, n_iter, num_sample_for_ransac)
|
||||
|
||||
# 4. Batch fit Homography
|
||||
# Need to implement batch version that supports (B, n_iter, num_sample_for_ransac, ...) input
|
||||
# Output H_batch: (B, n_iter, 3, 3)
|
||||
cB, cN = src_pts_batch.shape[:2]
|
||||
H_batch = find_homography_least_squares_weighted_torch_batch(
|
||||
src_pts_batch.flatten(0, 1), dst_pts_batch.flatten(0, 1), confident_weight_batch.flatten(0, 1)
|
||||
) # (B, n_iter, 3, 3)
|
||||
H_batch = H_batch.unflatten(0, (cB, cN))
|
||||
|
||||
# 5. Batch evaluate inliers for all H
|
||||
src_homo = torch.cat(
|
||||
[src_pts, torch.ones(B, N, 1, dtype=src_pts.dtype, device=src_pts.device)], dim=2
|
||||
) # (B, N, 3)
|
||||
src_homo_expand = src_homo.unsqueeze(1).expand(B, n_iter, N, 3) # (B, n_iter, N, 3)
|
||||
dst_pts_expand = dst_pts.unsqueeze(1).expand(B, n_iter, N, 2) # (B, n_iter, N, 2)
|
||||
confident_weight_expand = confident_weight.unsqueeze(1).expand(B, n_iter, N) # (B, n_iter, N)
|
||||
|
||||
# H_batch: (B, n_iter, 3, 3)
|
||||
# Need to reshape H_batch to (B*n_iter, 3, 3), src_homo_expand to (B*n_iter, N, 3)
|
||||
H_batch_flat = H_batch.reshape(-1, 3, 3)
|
||||
src_homo_expand_flat = src_homo_expand.reshape(-1, N, 3)
|
||||
proj = torch.bmm(src_homo_expand_flat, H_batch_flat.transpose(1, 2)) # (B*n_iter, N, 3)
|
||||
proj_xy = proj[:, :, :2] / proj[:, :, 2:3] # (B*n_iter, N, 2)
|
||||
proj_xy = proj_xy.reshape(B, n_iter, N, 2)
|
||||
error = ((proj_xy - dst_pts_expand) ** 2).sum(dim=3).sqrt() # (B, n_iter, N)
|
||||
inlier_mask = error < reproj_threshold # (B, n_iter, N)
|
||||
total_score = (inlier_mask * confident_weight_expand).sum(dim=2) # (B, n_iter)
|
||||
|
||||
# 6. Select the sampling group with the highest score
|
||||
best_idx = torch.argmax(total_score, dim=1) # (B,)
|
||||
best_inlier_mask = inlier_mask[torch.arange(B, device=device), best_idx] # (B, N)
|
||||
|
||||
# 7. Refit Homography using inliers
|
||||
H_inlier_list = []
|
||||
for b in range(B):
|
||||
mask = best_inlier_mask[b]
|
||||
inlier_src_pts = src_pts[b][mask] # (?, 3)
|
||||
inlier_dst_pts = dst_pts[b][mask] # (?, 2)
|
||||
inlier_confident_weight = confident_weight[b][mask] # (?)
|
||||
|
||||
sorted_idx = torch.argsort(inlier_confident_weight, descending=True)
|
||||
# # method 1: sort according to confident_weight, and only keep max_inlier_num pts
|
||||
# sorted_idx = sorted_idx[:max_inlier_num]
|
||||
# method 2: random choose max_inlier_num pts
|
||||
if len(sorted_idx) > max_inlier_num:
|
||||
# random choose from first 95% confident pts
|
||||
keep_len = max(int(len(sorted_idx) * 0.95), max_inlier_num)
|
||||
sorted_idx = sorted_idx[:keep_len]
|
||||
perm = torch.randperm(len(sorted_idx), device=device)[:max_inlier_num]
|
||||
sorted_idx = sorted_idx[perm]
|
||||
inlier_src_pts = inlier_src_pts[sorted_idx]
|
||||
inlier_dst_pts = inlier_dst_pts[sorted_idx]
|
||||
inlier_confident_weight = inlier_confident_weight[sorted_idx]
|
||||
|
||||
H_inlier = find_homography_least_squares_weighted_torch(
|
||||
inlier_src_pts, inlier_dst_pts, inlier_confident_weight
|
||||
) # (3, 3)
|
||||
H_inlier_list.append(H_inlier)
|
||||
H_inlier = torch.stack(H_inlier_list, dim=0) # (B, 3, 3)
|
||||
return H_inlier
|
||||
|
||||
def get_params_for_ransac(N, device):
|
||||
n_iter=100
|
||||
sample_ratio=0.3
|
||||
num_sample_for_ransac=8
|
||||
n_sample = max(num_sample_for_ransac, int(N * sample_ratio))
|
||||
rand_sample_iters_idx = torch.stack(
|
||||
[torch.randperm(n_sample, device=device)[:num_sample_for_ransac] for _ in range(n_iter)],
|
||||
dim=0,
|
||||
) # (n_iter, num_sample_for_ransac)
|
||||
return n_iter, num_sample_for_ransac, n_sample, rand_sample_iters_idx
|
||||
|
||||
|
||||
def camray_to_caminfo(camray, confidence=None, reproj_threshold=0.2, training=False):
|
||||
"""
|
||||
Args:
|
||||
camray: (B, S, num_patches_y, num_patches_x, 6)
|
||||
confidence: (B, S, num_patches_y, num_patches_x)
|
||||
Returns:
|
||||
R: (B, S, 3, 3)
|
||||
T: (B, S, 3)
|
||||
focal_lengths: (B, S, 2)
|
||||
principal_points: (B, S, 2)
|
||||
"""
|
||||
if confidence is None:
|
||||
confidence = torch.ones_like(camray[:, :, :, :, 0])
|
||||
B, S, num_patches_y, num_patches_x, _ = camray.shape
|
||||
# identity K, assume imw=imh=2.0
|
||||
I_K = torch.eye(3, dtype=camray.dtype, device=camray.device)
|
||||
I_K[0, 2] = 1.0
|
||||
I_K[1, 2] = 1.0
|
||||
# repeat I_K to match camray
|
||||
I_K = I_K.unsqueeze(0).unsqueeze(0).expand(B, S, -1, -1)
|
||||
|
||||
cam_plane_depth = torch.ones(
|
||||
B, S, num_patches_y, num_patches_x, 1, dtype=camray.dtype, device=camray.device
|
||||
)
|
||||
I_cam_plane_unproj = unproject_depth(
|
||||
cam_plane_depth,
|
||||
I_K,
|
||||
c2w=None,
|
||||
ixt_normalized=True,
|
||||
num_patches_x=num_patches_x,
|
||||
num_patches_y=num_patches_y,
|
||||
) # (B, S, num_patches_y, num_patches_x, 3)
|
||||
|
||||
camray = camray.flatten(0, 1).flatten(1, 2) # (B*S, num_patches_y*num_patches_x, 6)
|
||||
I_cam_plane_unproj = I_cam_plane_unproj.flatten(0, 1).flatten(
|
||||
1, 2
|
||||
) # (B*S, num_patches_y*num_patches_x, 3)
|
||||
confidence = confidence.flatten(0, 1).flatten(1, 2) # (B*S, num_patches_y*num_patches_x)
|
||||
|
||||
# Compute optimal rotation to align rays
|
||||
N = camray.shape[-2]
|
||||
device = camray.device
|
||||
n_iter, num_sample_for_ransac, n_sample, rand_sample_iters_idx = get_params_for_ransac(N, device)
|
||||
|
||||
# Use batch processing (confidence is guaranteed to be not None at this point)
|
||||
if training:
|
||||
camray = camray.clone().detach()
|
||||
I_cam_plane_unproj = I_cam_plane_unproj.clone().detach()
|
||||
confidence = confidence.clone().detach()
|
||||
R, focal_lengths, principal_points = compute_optimal_rotation_intrinsics_batch(
|
||||
I_cam_plane_unproj,
|
||||
camray[:, :, :3],
|
||||
reproj_threshold=reproj_threshold,
|
||||
weights=confidence,
|
||||
n_sample = n_sample,
|
||||
n_iter=n_iter,
|
||||
num_sample_for_ransac=num_sample_for_ransac,
|
||||
rand_sample_iters_idx=rand_sample_iters_idx,
|
||||
)
|
||||
|
||||
T = torch.sum(camray[:, :, 3:] * confidence.unsqueeze(-1), dim=1) / torch.sum(
|
||||
confidence, dim=-1, keepdim=True
|
||||
)
|
||||
|
||||
R = R.reshape(B, S, 3, 3)
|
||||
T = T.reshape(B, S, 3)
|
||||
focal_lengths = focal_lengths.reshape(B, S, 2)
|
||||
principal_points = principal_points.reshape(B, S, 2)
|
||||
|
||||
return R, T, 1.0 / focal_lengths, principal_points + 1.0
|
||||
|
||||
def get_extrinsic_from_camray(camray, conf, patch_size_y, patch_size_x, training=False):
|
||||
pred_R, pred_T, pred_focal_lengths, pred_principal_points = camray_to_caminfo(
|
||||
camray, confidence=conf.squeeze(-1), training=training
|
||||
)
|
||||
|
||||
pred_extrinsic = torch.cat(
|
||||
[
|
||||
torch.cat([pred_R, pred_T.unsqueeze(-1)], dim=-1),
|
||||
repeat(
|
||||
torch.tensor([0, 0, 0, 1], dtype=pred_R.dtype, device=pred_R.device),
|
||||
"c -> b s 1 c",
|
||||
b=pred_R.shape[0],
|
||||
s=pred_R.shape[1],
|
||||
),
|
||||
],
|
||||
dim=-2,
|
||||
) # B, S, 4, 4
|
||||
return pred_extrinsic, pred_focal_lengths, pred_principal_points
|
||||
@@ -1,22 +0,0 @@
|
||||
[build-system]
|
||||
requires = ["setuptools>=61.0", "wheel"]
|
||||
build-backend = "setuptools.build_meta"
|
||||
|
||||
[project]
|
||||
name = "moge"
|
||||
version = "2.0.0"
|
||||
description = "MoGe monocular geometry estimation"
|
||||
readme = "README.md"
|
||||
license = {text = "MIT"}
|
||||
dependencies = [
|
||||
"huggingface_hub",
|
||||
"numpy",
|
||||
"pillow",
|
||||
"torch>=2.0.0",
|
||||
"utils3d",
|
||||
]
|
||||
requires-python = ">=3.9"
|
||||
|
||||
[tool.setuptools.packages.find]
|
||||
where = ["."]
|
||||
include = ["moge*"]
|
||||
@@ -1,16 +0,0 @@
|
||||
[build-system]
|
||||
requires = ["setuptools>=61.0", "wheel"]
|
||||
build-backend = "setuptools.build_meta"
|
||||
|
||||
[project]
|
||||
name = "utils3d"
|
||||
version = "1.7"
|
||||
description = "Utilities for 3D computer vision"
|
||||
readme = "README.md"
|
||||
license = {text = "MIT"}
|
||||
dependencies = ["numpy", "scipy"]
|
||||
requires-python = ">=3.8"
|
||||
|
||||
[tool.setuptools.packages.find]
|
||||
where = ["."]
|
||||
include = ["utils3d*"]
|
||||
@@ -1,111 +0,0 @@
|
||||
import argparse
|
||||
import hashlib
|
||||
import sys
|
||||
import tomllib
|
||||
import urllib.request
|
||||
from pathlib import Path
|
||||
|
||||
|
||||
PROJECT_ROOT = Path(__file__).resolve().parent
|
||||
LOCK_PATH = PROJECT_ROOT / "vendor.lock.toml"
|
||||
VENDOR_DIR = PROJECT_ROOT / "vendor"
|
||||
CHUNK_SIZE = 1024 * 1024
|
||||
|
||||
|
||||
def load_lock(path=LOCK_PATH):
|
||||
"""Load and validate the vendor lock file."""
|
||||
with Path(path).open("rb") as stream:
|
||||
lock = tomllib.load(stream)
|
||||
if lock.get("schema_version") != 1:
|
||||
raise RuntimeError("Unsupported vendor.lock.toml schema")
|
||||
return lock
|
||||
|
||||
|
||||
def file_sha256(path):
|
||||
"""Return the SHA-256 digest for a file."""
|
||||
digest = hashlib.sha256()
|
||||
with Path(path).open("rb") as stream:
|
||||
while chunk := stream.read(CHUNK_SIZE):
|
||||
digest.update(chunk)
|
||||
return digest.hexdigest()
|
||||
|
||||
|
||||
def download_artifact(artifact, destination_dir=VENDOR_DIR):
|
||||
"""Download one locked artifact and verify its content."""
|
||||
destination_dir = Path(destination_dir)
|
||||
destination = destination_dir / artifact["filename"]
|
||||
expected = artifact["sha256"]
|
||||
if destination.is_file() and file_sha256(destination) == expected:
|
||||
print(f"Ready: {destination.name}")
|
||||
return destination
|
||||
|
||||
destination_dir.mkdir(parents=True, exist_ok=True)
|
||||
temporary = destination.with_name(f"{destination.name}.part")
|
||||
temporary.unlink(missing_ok=True)
|
||||
print(f"Downloading: {destination.name}")
|
||||
request = urllib.request.Request(
|
||||
artifact["url"],
|
||||
headers={"User-Agent": "SnapForm vendor preparation"},
|
||||
)
|
||||
try:
|
||||
with urllib.request.urlopen(request) as response, temporary.open("wb") as output:
|
||||
while chunk := response.read(CHUNK_SIZE):
|
||||
output.write(chunk)
|
||||
actual = file_sha256(temporary)
|
||||
if actual != expected:
|
||||
raise RuntimeError(
|
||||
f"SHA-256 mismatch for {destination.name}: "
|
||||
f"expected {expected}, got {actual}"
|
||||
)
|
||||
temporary.replace(destination)
|
||||
except Exception:
|
||||
temporary.unlink(missing_ok=True)
|
||||
raise
|
||||
return destination
|
||||
|
||||
|
||||
def prepare(platforms=None, lock_path=LOCK_PATH, destination_dir=VENDOR_DIR):
|
||||
"""Prepare common third-party archives for selected platforms."""
|
||||
lock = load_lock(lock_path)
|
||||
available = lock["platforms"]
|
||||
selected = list(available) if platforms is None else platforms
|
||||
unknown = sorted(set(selected) - set(available))
|
||||
if unknown:
|
||||
raise ValueError(f"Unsupported platform: {', '.join(unknown)}")
|
||||
|
||||
artifacts = list(lock["artifacts"].values())
|
||||
artifacts.extend(available[platform] for platform in selected)
|
||||
return [
|
||||
download_artifact(artifact, destination_dir)
|
||||
for artifact in artifacts
|
||||
]
|
||||
|
||||
|
||||
def parse_args(argv=None):
|
||||
"""Parse command-line options."""
|
||||
parser = argparse.ArgumentParser(
|
||||
description="Download and verify third-party files for SnapForm.",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--platform",
|
||||
action="append",
|
||||
dest="platforms",
|
||||
help="Prepare one platform; repeat as needed. Default: all platforms.",
|
||||
)
|
||||
return parser.parse_args(argv)
|
||||
|
||||
|
||||
def main(argv=None):
|
||||
"""Run vendor preparation from the command line."""
|
||||
args = parse_args(argv)
|
||||
try:
|
||||
paths = prepare(args.platforms)
|
||||
except (OSError, RuntimeError, ValueError) as error:
|
||||
print(f"Preparation failed: {error}", file=sys.stderr)
|
||||
return 1
|
||||
print(f"Prepared {len(paths)} files")
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
raise SystemExit(main())
|
||||
@@ -1,14 +1,17 @@
|
||||
[project]
|
||||
name = "snapform"
|
||||
version = "0.10.11"
|
||||
version = "0.12.3"
|
||||
description = "Fast image-guided structure building tools for Blender"
|
||||
readme = "README.md"
|
||||
requires-python = ">=3.11"
|
||||
license = "GPL-3.0-or-later"
|
||||
dependencies = []
|
||||
dependencies = ["blendjob"]
|
||||
|
||||
[dependency-groups]
|
||||
dev = ["numpy", "pytest"]
|
||||
dev = ["numpy", "pillow", "pytest"]
|
||||
|
||||
[tool.uv]
|
||||
package = false
|
||||
|
||||
[tool.uv.sources]
|
||||
blendjob = { path = "src/snapform/wheels/blendjob-0.1.6-py3-none-any.whl" }
|
||||
|
||||
@@ -1,57 +1,43 @@
|
||||
import bpy
|
||||
|
||||
from . import client
|
||||
from .image_depth import (
|
||||
ApplyRemovedBackground,
|
||||
BuildDepthPlane,
|
||||
BuildOriginalDepthPlane,
|
||||
BuildMoge2DepthPlane,
|
||||
ConvertImageToDepthPlane,
|
||||
from .blender import depth_plane as image_depth
|
||||
from .blender import surface as image_mesh
|
||||
from .blender.depth_plane import (
|
||||
ConvertToDepthPlane,
|
||||
RemoveImageBackground,
|
||||
SnapFormImageMenu,
|
||||
draw_image_context_menu,
|
||||
)
|
||||
from .image_mesh import BuildImageMesh, ConvertImageToMesh
|
||||
from .blender.surface import ConvertToSurface
|
||||
from .operators import (
|
||||
CancelJob,
|
||||
ControlService,
|
||||
RunJob,
|
||||
cancel_active_job,
|
||||
close_active_job,
|
||||
JOB_OPERATOR_CLASSES,
|
||||
)
|
||||
from .job_runtime import runtime
|
||||
from .panel import (
|
||||
Ben2RemovalDebugPanel,
|
||||
ServerPanel,
|
||||
Da3OutputDebugPanel,
|
||||
DebugPanel,
|
||||
AiServicePanel,
|
||||
ModelsPanel,
|
||||
Moge2ApiDebugPanel,
|
||||
)
|
||||
from .preferences import SnapFormPreferences
|
||||
from .properties import SnapFormSettings
|
||||
from .status_bar import draw_status_bar
|
||||
|
||||
|
||||
CLASSES = (
|
||||
SnapFormPreferences,
|
||||
SnapFormSettings,
|
||||
RunJob,
|
||||
CancelJob,
|
||||
ControlService,
|
||||
*JOB_OPERATOR_CLASSES,
|
||||
SnapFormImageMenu,
|
||||
ApplyRemovedBackground,
|
||||
RemoveImageBackground,
|
||||
BuildDepthPlane,
|
||||
BuildOriginalDepthPlane,
|
||||
BuildMoge2DepthPlane,
|
||||
ConvertImageToDepthPlane,
|
||||
BuildImageMesh,
|
||||
ConvertImageToMesh,
|
||||
AiServicePanel,
|
||||
ConvertToDepthPlane,
|
||||
ConvertToSurface,
|
||||
ServerPanel,
|
||||
ModelsPanel,
|
||||
DebugPanel,
|
||||
Da3OutputDebugPanel,
|
||||
Moge2ApiDebugPanel,
|
||||
Da3OutputDebugPanel,
|
||||
Ben2RemovalDebugPanel,
|
||||
)
|
||||
|
||||
@@ -62,20 +48,15 @@ def register():
|
||||
bpy.types.Scene.snapform_settings = bpy.props.PointerProperty(
|
||||
type=SnapFormSettings,
|
||||
)
|
||||
bpy.types.STATUSBAR_HT_header.append(draw_status_bar)
|
||||
bpy.types.VIEW3D_MT_object_context_menu.append(draw_image_context_menu)
|
||||
bpy.types.OUTLINER_MT_object.append(draw_image_context_menu)
|
||||
client.schedule_service_start()
|
||||
runtime.register()
|
||||
|
||||
|
||||
def unregister():
|
||||
cancel_active_job()
|
||||
close_active_job()
|
||||
client.unschedule_service()
|
||||
client.detach_service_client()
|
||||
runtime.unregister()
|
||||
bpy.types.VIEW3D_MT_object_context_menu.remove(draw_image_context_menu)
|
||||
bpy.types.OUTLINER_MT_object.remove(draw_image_context_menu)
|
||||
bpy.types.STATUSBAR_HT_header.remove(draw_status_bar)
|
||||
if hasattr(bpy.types.Scene, "snapform_settings"):
|
||||
del bpy.types.Scene.snapform_settings
|
||||
for class_type in reversed(CLASSES):
|
||||
|
||||
Binary file not shown.
1
src/snapform/blender/__init__.py
Normal file
1
src/snapform/blender/__init__.py
Normal file
@@ -0,0 +1 @@
|
||||
"""Blender-process geometry construction and import logic."""
|
||||
@@ -1,5 +1,6 @@
|
||||
import re
|
||||
import shutil
|
||||
import tempfile
|
||||
from pathlib import Path
|
||||
|
||||
import bpy
|
||||
@@ -19,6 +20,7 @@ MATERIAL_DISPLACEMENT_ATTRIBUTE = "o_material_displacement"
|
||||
BACKGROUND_DEPTH_ATTRIBUTE = "o_background_depth"
|
||||
DEPTH_SCALE_ATTRIBUTE = "o_depth_scale"
|
||||
SURFACE_MATERIAL_NODE_GROUP_NAME = "O DA3 Surface Layer"
|
||||
MOGE_SURFACE_MATERIAL_NODE_GROUP_NAME = "O MoGe2 Surface Layer"
|
||||
DEPTH_PLANE_MATERIAL_NODE_GROUP_NAME = "O DA3 Depth Plane Layer"
|
||||
MOGE_MATERIAL_NODE_GROUP_NAME = "O MoGe2 Depth Plane Layer"
|
||||
DA3_PATCH_SIZE = 14
|
||||
@@ -52,14 +54,11 @@ VIDEO_NAME_SUFFIXES = (
|
||||
|
||||
|
||||
def image_depth_model_items(owner, context):
|
||||
from .models import depth_plane_model_enum_items
|
||||
from ..properties import depth_plane_model_enum_items
|
||||
|
||||
return depth_plane_model_enum_items(owner, context)
|
||||
|
||||
|
||||
_last_depth_plane_model = "MOGE2_VITB_NORMAL"
|
||||
|
||||
|
||||
def is_image_empty(candidate):
|
||||
return bool(
|
||||
candidate
|
||||
@@ -169,6 +168,14 @@ def depth_data_name(source_object):
|
||||
return f"{image_base_name(source_object.data)}.depth"
|
||||
|
||||
|
||||
def normal_data_name(source_object):
|
||||
return f"{image_base_name(source_object.data)}.normal"
|
||||
|
||||
|
||||
def mask_data_name(source_object):
|
||||
return f"{image_base_name(source_object.data)}.mask"
|
||||
|
||||
|
||||
def copy_depth_image(source_object):
|
||||
"""Return a renamed copy of the source image for use as the depth map."""
|
||||
depth_image = source_object.data.copy()
|
||||
@@ -212,19 +219,17 @@ def process_resolution_scale_property():
|
||||
def draw_process_resolution(layout, owner, image):
|
||||
process_res = scaled_process_resolution(
|
||||
image,
|
||||
owner.snapform_process_res_factor,
|
||||
owner.process_res_factor,
|
||||
)
|
||||
layout.prop(
|
||||
owner,
|
||||
"snapform_process_res_factor",
|
||||
"process_res_factor",
|
||||
text=f"Resolution Scale ({process_res} px)",
|
||||
)
|
||||
|
||||
|
||||
def save_packed_image(image, scene):
|
||||
from .client import temporary_input_path
|
||||
|
||||
output_path = temporary_input_path()
|
||||
output_path = Path(tempfile.mkdtemp(prefix="snapform-input-")) / "input.png"
|
||||
image_settings = scene.render.image_settings
|
||||
original_file_format = image_settings.file_format
|
||||
try:
|
||||
@@ -280,9 +285,7 @@ def prepare_image_input(image, scene):
|
||||
return source_path, False
|
||||
sequence_frames = collect_sequence_frames(image)
|
||||
if sequence_frames:
|
||||
from .client import temporary_input_directory
|
||||
|
||||
temporary_directory = Path(temporary_input_directory())
|
||||
temporary_directory = Path(tempfile.mkdtemp(prefix="snapform-input-"))
|
||||
try:
|
||||
for index, frame_path in enumerate(sequence_frames):
|
||||
suffix = frame_path.suffix.lower() or ".jpg"
|
||||
@@ -820,11 +823,12 @@ def create_moge_mask_image(name, mask_values):
|
||||
return image
|
||||
|
||||
|
||||
def create_moge_normal_image(name, normal_values):
|
||||
def create_moge_normal_image(name, normal_values, convert_coordinates=True):
|
||||
import numpy as np
|
||||
|
||||
converted = normal_values.astype(np.float32, copy=True)
|
||||
converted *= np.array([1.0, -1.0, -1.0], dtype=np.float32)
|
||||
if convert_coordinates:
|
||||
converted *= np.array([1.0, -1.0, -1.0], dtype=np.float32)
|
||||
converted = np.clip(converted * 0.5 + 0.5, 0.0, 1.0)
|
||||
height, width = converted.shape[:2]
|
||||
image = bpy.data.images.new(
|
||||
@@ -1006,10 +1010,13 @@ def create_moge_material(
|
||||
uv.location = (-900.0, 200.0)
|
||||
color = nodes.new("ShaderNodeTexImage")
|
||||
color.image = color_image
|
||||
color.interpolation = "Linear"
|
||||
color.extension = "EXTEND"
|
||||
color.location = (-650.0, 300.0)
|
||||
depth = nodes.new("ShaderNodeTexImage")
|
||||
depth.image = depth_image
|
||||
depth.interpolation = "Linear"
|
||||
depth.extension = "EXTEND"
|
||||
depth.location = (-650.0, -100.0)
|
||||
if frame_count > 1:
|
||||
configure_image_user(
|
||||
@@ -1127,11 +1134,11 @@ def create_plane_mesh(source_object, bounds=None, subdivision_level=5):
|
||||
return mesh
|
||||
|
||||
|
||||
def material_node_group(depth_plane=False):
|
||||
def material_node_group(depth_plane=False, surface_group_name=None):
|
||||
group_name = (
|
||||
DEPTH_PLANE_MATERIAL_NODE_GROUP_NAME
|
||||
if depth_plane
|
||||
else SURFACE_MATERIAL_NODE_GROUP_NAME
|
||||
else surface_group_name or SURFACE_MATERIAL_NODE_GROUP_NAME
|
||||
)
|
||||
existing = bpy.data.node_groups.get(group_name)
|
||||
if existing is not None:
|
||||
@@ -1143,6 +1150,9 @@ def material_node_group(depth_plane=False):
|
||||
("Alpha", "NodeSocketFloat"),
|
||||
("Bump Scale", "NodeSocketFloat"),
|
||||
]
|
||||
if group_name == MOGE_SURFACE_MATERIAL_NODE_GROUP_NAME:
|
||||
input_sockets.insert(2, ("Normal", "NodeSocketColor"))
|
||||
input_sockets.insert(3, ("Normal Strength", "NodeSocketFloat"))
|
||||
if depth_plane:
|
||||
input_sockets.insert(2, ("Depth", "NodeSocketFloat"))
|
||||
for name, socket_type in input_sockets:
|
||||
@@ -1160,6 +1170,10 @@ def material_node_group(depth_plane=False):
|
||||
socket.default_value = 1.0
|
||||
socket.min_value = -10000.0
|
||||
socket.max_value = 10000.0
|
||||
elif name == "Normal Strength":
|
||||
socket.default_value = 1.0
|
||||
socket.min_value = 0.0
|
||||
socket.max_value = 10.0
|
||||
output_sockets = [
|
||||
("Color", "NodeSocketColor"),
|
||||
("Roughness", "NodeSocketFloat"),
|
||||
@@ -1190,6 +1204,16 @@ def material_node_group(depth_plane=False):
|
||||
roughness_curve.location = (-250.0, 350.0)
|
||||
bump = nodes.new("ShaderNodeBump")
|
||||
bump.location = (-250.0, 0.0)
|
||||
if group_name == MOGE_SURFACE_MATERIAL_NODE_GROUP_NAME:
|
||||
normal_map = nodes.new("ShaderNodeNormalMap")
|
||||
normal_map.space = "OBJECT"
|
||||
normal_map.location = (-500.0, -150.0)
|
||||
links.new(group_input.outputs["Normal"], normal_map.inputs["Color"])
|
||||
links.new(
|
||||
group_input.outputs["Normal Strength"],
|
||||
normal_map.inputs["Strength"],
|
||||
)
|
||||
links.new(normal_map.outputs["Normal"], bump.inputs["Normal"])
|
||||
displacement = None
|
||||
|
||||
processed_alpha = group_input.outputs["Alpha"]
|
||||
@@ -1269,7 +1293,7 @@ def material_node_group(depth_plane=False):
|
||||
group.description = (
|
||||
"Prepare DA3 depth-plane material outputs."
|
||||
if depth_plane
|
||||
else "Prepare DA3 surface material outputs."
|
||||
else f"Prepare {group_name} material outputs."
|
||||
)
|
||||
group.use_fake_user = True
|
||||
return group
|
||||
@@ -1293,6 +1317,8 @@ def create_material(
|
||||
depth_frame_offset=0,
|
||||
depth_frame_duration=None,
|
||||
use_cyclic=None,
|
||||
surface_group_name=None,
|
||||
normal_image=None,
|
||||
):
|
||||
material = bpy.data.materials.new(name)
|
||||
material.use_nodes = True
|
||||
@@ -1347,13 +1373,24 @@ def create_material(
|
||||
alpha = color_texture.outputs["Alpha"]
|
||||
material_inputs = nodes.new("ShaderNodeGroup")
|
||||
is_depth_plane = displacement_image is not None
|
||||
material_inputs.node_tree = material_node_group(is_depth_plane)
|
||||
material_inputs.node_tree = material_node_group(
|
||||
is_depth_plane,
|
||||
surface_group_name=surface_group_name,
|
||||
)
|
||||
material_inputs.name = material_inputs.node_tree.name
|
||||
material_inputs.label = material_inputs.name
|
||||
material_inputs.location = (-300.0, 350.0)
|
||||
material_inputs.inputs["Bump Scale"].default_value = depth_scale
|
||||
links.new(color_texture.outputs["Color"], material_inputs.inputs["Color"])
|
||||
links.new(alpha, material_inputs.inputs["Alpha"])
|
||||
if normal_image is not None and "Normal" in material_inputs.inputs:
|
||||
normal_texture = nodes.new("ShaderNodeTexImage")
|
||||
normal_texture.image = normal_image
|
||||
normal_texture.interpolation = "Linear"
|
||||
normal_texture.extension = "EXTEND"
|
||||
normal_texture.location = (-1000.0, -50.0)
|
||||
links.new(uv_map.outputs["UV"], normal_texture.inputs["Vector"])
|
||||
links.new(normal_texture.outputs["Color"], material_inputs.inputs["Normal"])
|
||||
if unshaded:
|
||||
emission = nodes.new("ShaderNodeEmission")
|
||||
transparent = nodes.new("ShaderNodeBsdfTransparent")
|
||||
@@ -1447,7 +1484,7 @@ def set_displacement_only(material):
|
||||
|
||||
|
||||
def node_asset_path():
|
||||
return Path(__file__).with_name("assets") / NODE_ASSET_NAME
|
||||
return Path(__file__).parent.parent / "assets" / NODE_ASSET_NAME
|
||||
|
||||
|
||||
def load_depth_plane_node_group(group_name=DEPTH_PLANE_NODE_GROUP_NAME):
|
||||
@@ -1498,6 +1535,53 @@ def modifier_input_identifier(node_group, name):
|
||||
raise RuntimeError(f"Geometry Nodes input is missing: {name}")
|
||||
|
||||
|
||||
def modifier_input_slots(modifier):
|
||||
"""Return the runtime input collection, or None for the legacy API."""
|
||||
properties = getattr(modifier, "properties", None)
|
||||
if properties is None:
|
||||
return None
|
||||
return getattr(properties, "inputs", None)
|
||||
|
||||
|
||||
def set_modifier_input(modifier, identifier, value):
|
||||
"""Assign a value to a Geometry Nodes modifier input socket."""
|
||||
inputs = modifier_input_slots(modifier)
|
||||
if inputs is not None:
|
||||
# Blender 5.2+ exposes inputs as runtime RNA keyed by socket identifier.
|
||||
getattr(inputs, identifier).value = value
|
||||
else:
|
||||
# Blender 4.x exposes inputs as modifier ID properties.
|
||||
modifier[identifier] = value
|
||||
|
||||
|
||||
def get_modifier_input(modifier, identifier):
|
||||
"""Read a Geometry Nodes modifier input socket value."""
|
||||
inputs = modifier_input_slots(modifier)
|
||||
if inputs is not None:
|
||||
return getattr(inputs, identifier).value
|
||||
return modifier[identifier]
|
||||
|
||||
|
||||
def modifier_input_driver_path(modifier, identifier):
|
||||
"""Return the driver data path for a modifier input socket."""
|
||||
if modifier_input_slots(modifier) is not None:
|
||||
return (
|
||||
f'modifiers["{modifier.name}"]'
|
||||
f".properties.inputs.{identifier}.value"
|
||||
)
|
||||
return f'modifiers["{modifier.name}"]["{identifier}"]'
|
||||
|
||||
|
||||
def drive_modifier_input(owner_object, modifier, identifier, expression="frame"):
|
||||
"""Add a scripted driver to a modifier input so it follows the scene frame."""
|
||||
driver = owner_object.driver_add(
|
||||
modifier_input_driver_path(modifier, identifier)
|
||||
)
|
||||
driver.driver.type = "SCRIPTED"
|
||||
driver.driver.expression = expression
|
||||
return driver
|
||||
|
||||
|
||||
def add_depth_plane_processing(
|
||||
plane_object,
|
||||
depth_image,
|
||||
@@ -1525,12 +1609,10 @@ def add_depth_plane_processing(
|
||||
("Cyclic", use_cyclic),
|
||||
):
|
||||
identifier = modifier_input_identifier(node_group, name)
|
||||
modifier[identifier] = value
|
||||
set_modifier_input(modifier, identifier, value)
|
||||
if animate_frame:
|
||||
frame_identifier = modifier_input_identifier(node_group, "Frame")
|
||||
driver = modifier.driver_add(f'["{frame_identifier}"]')
|
||||
driver.driver.type = "SCRIPTED"
|
||||
driver.driver.expression = "frame"
|
||||
drive_modifier_input(plane_object, modifier, frame_identifier)
|
||||
plane_object.update_tag(refresh={"DATA"})
|
||||
return modifier
|
||||
|
||||
@@ -1561,12 +1643,11 @@ def add_moge_depth_plane_processing(
|
||||
("Frames", max(int(frame_duration or 1), 1)),
|
||||
("Cyclic", use_cyclic),
|
||||
):
|
||||
modifier[modifier_input_identifier(node_group, name)] = value
|
||||
identifier = modifier_input_identifier(node_group, name)
|
||||
set_modifier_input(modifier, identifier, value)
|
||||
if animate_frame:
|
||||
frame_identifier = modifier_input_identifier(node_group, "Frame")
|
||||
driver = modifier.driver_add(f'["{frame_identifier}"]')
|
||||
driver.driver.type = "SCRIPTED"
|
||||
driver.driver.expression = "frame"
|
||||
drive_modifier_input(plane_object, modifier, frame_identifier)
|
||||
plane_object.update_tag(refresh={"DATA"})
|
||||
return modifier
|
||||
|
||||
@@ -1784,88 +1865,6 @@ def build_depth_plane(
|
||||
)
|
||||
|
||||
|
||||
class BuildDepthPlane(bpy.types.Operator):
|
||||
bl_idname = "snapform.build_depth_plane"
|
||||
bl_label = "Build Depth Plane"
|
||||
bl_options = {"INTERNAL"}
|
||||
|
||||
snapform_source_object_name: bpy.props.StringProperty(
|
||||
options={"HIDDEN", "SKIP_SAVE"},
|
||||
)
|
||||
snapform_npz_path: bpy.props.StringProperty(
|
||||
options={"HIDDEN", "SKIP_SAVE"},
|
||||
)
|
||||
snapform_depth_data_name: bpy.props.StringProperty(
|
||||
options={"HIDDEN", "SKIP_SAVE"},
|
||||
)
|
||||
snapform_background_path: bpy.props.StringProperty(
|
||||
options={"HIDDEN", "SKIP_SAVE"},
|
||||
)
|
||||
snapform_color_frames_dir: bpy.props.StringProperty(
|
||||
options={"HIDDEN", "SKIP_SAVE"},
|
||||
)
|
||||
snapform_depth_frames_dir: bpy.props.StringProperty(
|
||||
options={"HIDDEN", "SKIP_SAVE"},
|
||||
)
|
||||
snapform_delete_source: bpy.props.BoolProperty(
|
||||
default=True,
|
||||
options={"HIDDEN", "SKIP_SAVE"},
|
||||
)
|
||||
snapform_unshaded: bpy.props.BoolProperty(
|
||||
default=False,
|
||||
options={"HIDDEN", "SKIP_SAVE"},
|
||||
)
|
||||
|
||||
def execute(self, context):
|
||||
source_object = bpy.data.objects.get(self.snapform_source_object_name)
|
||||
if not is_image_empty(source_object):
|
||||
self.report(
|
||||
{"ERROR"},
|
||||
"The source image empty is no longer available",
|
||||
)
|
||||
return {"CANCELLED"}
|
||||
|
||||
npz_path = Path(self.snapform_npz_path)
|
||||
if not npz_path.is_file():
|
||||
self.report(
|
||||
{"ERROR"},
|
||||
f"Depth result does not exist: {npz_path}",
|
||||
)
|
||||
return {"CANCELLED"}
|
||||
|
||||
try:
|
||||
foreground_image = None
|
||||
if self.snapform_background_path:
|
||||
foreground_image = create_removed_background_image(
|
||||
source_object,
|
||||
self.snapform_background_path,
|
||||
assign_to_source=False,
|
||||
)
|
||||
color_image = None
|
||||
if self.snapform_color_frames_dir:
|
||||
color_image = create_sequence_image(
|
||||
f"{image_base_name(source_object.data)}.frame",
|
||||
self.snapform_color_frames_dir,
|
||||
)
|
||||
build_depth_plane(
|
||||
context,
|
||||
source_object,
|
||||
npz_path,
|
||||
self.snapform_delete_source,
|
||||
unshaded=self.snapform_unshaded,
|
||||
depth_name=self.snapform_depth_data_name,
|
||||
color_image=color_image,
|
||||
foreground_image=foreground_image,
|
||||
depth_frames_directory=(
|
||||
self.snapform_depth_frames_dir or None
|
||||
),
|
||||
)
|
||||
except (OSError, RuntimeError, ValueError) as error:
|
||||
self.report({"ERROR"}, str(error))
|
||||
return {"CANCELLED"}
|
||||
return {"FINISHED"}
|
||||
|
||||
|
||||
def build_moge2_depth_plane(
|
||||
context,
|
||||
source_object,
|
||||
@@ -1918,13 +1917,13 @@ def build_moge2_depth_plane(
|
||||
depth_scale = apply_plane_transform(plane_object, source_object, bounds)
|
||||
apply_object_scale_to_mesh(plane_object)
|
||||
depth_image = create_depth_image(
|
||||
f"{depth_data_name(source_object)}.moge2",
|
||||
depth_data_name(source_object),
|
||||
displacement[0] * depth_scale,
|
||||
)
|
||||
if len(depth) > 1 and data_frames_directory is not None:
|
||||
bpy.data.images.remove(depth_image, do_unlink=True)
|
||||
depth_image = create_moge_sequence_image(
|
||||
f"{depth_data_name(source_object)}.moge2",
|
||||
depth_data_name(source_object),
|
||||
[frame * depth_scale for frame in displacement],
|
||||
context.scene,
|
||||
data_frames_directory,
|
||||
@@ -1962,173 +1961,6 @@ def build_moge2_depth_plane(
|
||||
)
|
||||
|
||||
|
||||
class BuildMoge2DepthPlane(bpy.types.Operator):
|
||||
bl_idname = "snapform.build_moge2_depth_plane"
|
||||
bl_label = "Build MoGe-2 Depth Plane"
|
||||
bl_options = {"INTERNAL"}
|
||||
|
||||
snapform_source_object_name: bpy.props.StringProperty(options={"HIDDEN"})
|
||||
snapform_npz_path: bpy.props.StringProperty(options={"HIDDEN"})
|
||||
snapform_background_path: bpy.props.StringProperty(options={"HIDDEN"})
|
||||
snapform_color_frames_dir: bpy.props.StringProperty(options={"HIDDEN"})
|
||||
snapform_data_frames_dir: bpy.props.StringProperty(options={"HIDDEN"})
|
||||
snapform_delete_source: bpy.props.BoolProperty(default=True, options={"HIDDEN"})
|
||||
snapform_unshaded: bpy.props.BoolProperty(default=False, options={"HIDDEN"})
|
||||
|
||||
def execute(self, context):
|
||||
source_object = bpy.data.objects.get(self.snapform_source_object_name)
|
||||
if not is_image_empty(source_object):
|
||||
self.report({"ERROR"}, "The source image empty is no longer available")
|
||||
return {"CANCELLED"}
|
||||
try:
|
||||
foreground_image = None
|
||||
if self.snapform_background_path:
|
||||
foreground_image = create_removed_background_image(
|
||||
source_object,
|
||||
self.snapform_background_path,
|
||||
assign_to_source=False,
|
||||
)
|
||||
color_image = None
|
||||
if self.snapform_color_frames_dir:
|
||||
color_image = create_sequence_image(
|
||||
f"{image_base_name(source_object.data)}.frame",
|
||||
self.snapform_color_frames_dir,
|
||||
)
|
||||
build_moge2_depth_plane(
|
||||
context,
|
||||
source_object,
|
||||
Path(self.snapform_npz_path),
|
||||
self.snapform_delete_source,
|
||||
unshaded=self.snapform_unshaded,
|
||||
color_image=color_image,
|
||||
foreground_image=foreground_image,
|
||||
data_frames_directory=(
|
||||
self.snapform_data_frames_dir or None
|
||||
),
|
||||
)
|
||||
except (OSError, RuntimeError, ValueError) as error:
|
||||
self.report({"ERROR"}, str(error))
|
||||
return {"CANCELLED"}
|
||||
return {"FINISHED"}
|
||||
|
||||
|
||||
class BuildOriginalDepthPlane(bpy.types.Operator):
|
||||
bl_idname = "snapform.build_original_depth_plane"
|
||||
bl_label = "Build Original Depth Plane"
|
||||
bl_options = {"INTERNAL"}
|
||||
|
||||
snapform_source_object_name: bpy.props.StringProperty(
|
||||
options={"HIDDEN", "SKIP_SAVE"},
|
||||
)
|
||||
snapform_background_path: bpy.props.StringProperty(
|
||||
options={"HIDDEN", "SKIP_SAVE"},
|
||||
)
|
||||
snapform_color_frames_dir: bpy.props.StringProperty(
|
||||
options={"HIDDEN", "SKIP_SAVE"},
|
||||
)
|
||||
snapform_depth_frames_dir: bpy.props.StringProperty(
|
||||
options={"HIDDEN", "SKIP_SAVE"},
|
||||
)
|
||||
snapform_depth_npz_path: bpy.props.StringProperty(
|
||||
options={"HIDDEN", "SKIP_SAVE"},
|
||||
)
|
||||
snapform_delete_source: bpy.props.BoolProperty(
|
||||
default=True,
|
||||
options={"HIDDEN", "SKIP_SAVE"},
|
||||
)
|
||||
snapform_unshaded: bpy.props.BoolProperty(
|
||||
default=False,
|
||||
options={"HIDDEN", "SKIP_SAVE"},
|
||||
)
|
||||
|
||||
def execute(self, context):
|
||||
source_object = bpy.data.objects.get(self.snapform_source_object_name)
|
||||
if not is_image_empty(source_object):
|
||||
self.report(
|
||||
{"ERROR"},
|
||||
"The source image empty is no longer available",
|
||||
)
|
||||
return {"CANCELLED"}
|
||||
try:
|
||||
foreground_image = None
|
||||
if self.snapform_background_path:
|
||||
foreground_image = create_removed_background_image(
|
||||
source_object,
|
||||
self.snapform_background_path,
|
||||
assign_to_source=False,
|
||||
)
|
||||
color_image = None
|
||||
if self.snapform_color_frames_dir:
|
||||
color_image = create_sequence_image(
|
||||
f"{image_base_name(source_object.data)}.frame",
|
||||
self.snapform_color_frames_dir,
|
||||
)
|
||||
depth_image = None
|
||||
if self.snapform_depth_npz_path:
|
||||
depth_arrays = extract_depth_arrays(
|
||||
self.snapform_depth_npz_path
|
||||
)
|
||||
if len(depth_arrays) == 1:
|
||||
depth_image = create_depth_image(
|
||||
depth_data_name(source_object),
|
||||
depth_arrays[0],
|
||||
)
|
||||
else:
|
||||
depth_image = create_depth_sequence_image(
|
||||
depth_data_name(source_object),
|
||||
depth_arrays,
|
||||
context.scene,
|
||||
self.snapform_depth_frames_dir,
|
||||
)
|
||||
build_image_depth_plane(
|
||||
context,
|
||||
source_object,
|
||||
self.snapform_delete_source,
|
||||
unshaded=self.snapform_unshaded,
|
||||
color_image=color_image,
|
||||
foreground_image=foreground_image,
|
||||
depth_frames_directory=(
|
||||
self.snapform_depth_frames_dir or None
|
||||
),
|
||||
depth_image=depth_image,
|
||||
)
|
||||
except (OSError, RuntimeError, ValueError) as error:
|
||||
self.report({"ERROR"}, str(error))
|
||||
return {"CANCELLED"}
|
||||
return {"FINISHED"}
|
||||
|
||||
|
||||
class ApplyRemovedBackground(bpy.types.Operator):
|
||||
bl_idname = "snapform.apply_removed_background"
|
||||
bl_label = "Apply Removed Background"
|
||||
bl_options = {"INTERNAL"}
|
||||
|
||||
snapform_source_object_name: bpy.props.StringProperty(
|
||||
options={"HIDDEN", "SKIP_SAVE"},
|
||||
)
|
||||
snapform_output_path: bpy.props.StringProperty(
|
||||
options={"HIDDEN", "SKIP_SAVE"},
|
||||
)
|
||||
|
||||
def execute(self, _context):
|
||||
source_object = bpy.data.objects.get(self.snapform_source_object_name)
|
||||
if not is_image_empty(source_object):
|
||||
self.report(
|
||||
{"ERROR"},
|
||||
"The source image empty is no longer available",
|
||||
)
|
||||
return {"CANCELLED"}
|
||||
try:
|
||||
create_removed_background_image(
|
||||
source_object,
|
||||
self.snapform_output_path,
|
||||
)
|
||||
except RuntimeError as error:
|
||||
self.report({"ERROR"}, str(error))
|
||||
return {"CANCELLED"}
|
||||
return {"FINISHED"}
|
||||
|
||||
|
||||
class RemoveImageBackground(bpy.types.Operator):
|
||||
bl_idname = "snapform.remove_image_background"
|
||||
bl_label = "Remove Background"
|
||||
@@ -2137,20 +1969,16 @@ class RemoveImageBackground(bpy.types.Operator):
|
||||
|
||||
@classmethod
|
||||
def poll(cls, context):
|
||||
from .operators import active_job
|
||||
from .client import service_busy, model_ready
|
||||
from .models import BEN2_MODEL
|
||||
from ..job_runtime import runtime
|
||||
from ..properties import BEN2_MODEL_KEY, model_ready
|
||||
|
||||
return (
|
||||
is_image_empty(context.object)
|
||||
and active_job() is None
|
||||
and not service_busy()
|
||||
and model_ready(BEN2_MODEL.key)
|
||||
and not runtime.server_busy()
|
||||
and model_ready(BEN2_MODEL_KEY)
|
||||
)
|
||||
|
||||
def execute(self, context):
|
||||
from .operators import IMAGE_BACKGROUND_JOB
|
||||
|
||||
source_object = context.object
|
||||
total_frames = video_total_frames(source_object.data)
|
||||
if total_frames > 1:
|
||||
@@ -2166,11 +1994,10 @@ class RemoveImageBackground(bpy.types.Operator):
|
||||
self.report({"ERROR"}, str(error))
|
||||
return {"CANCELLED"}
|
||||
|
||||
result = bpy.ops.snapform.run_job(
|
||||
job_type=IMAGE_BACKGROUND_JOB,
|
||||
snapform_input_path=str(source_path),
|
||||
snapform_source_object_name=source_object.name,
|
||||
snapform_delete_input=delete_input,
|
||||
result = bpy.ops.snapform.process_image_background(
|
||||
input_path=str(source_path),
|
||||
source_object_name=source_object.name,
|
||||
delete_input=delete_input,
|
||||
)
|
||||
if "CANCELLED" not in result:
|
||||
return {"FINISHED"}
|
||||
@@ -2179,42 +2006,37 @@ class RemoveImageBackground(bpy.types.Operator):
|
||||
return {"CANCELLED"}
|
||||
|
||||
|
||||
class ConvertImageToDepthPlane(bpy.types.Operator):
|
||||
bl_idname = "snapform.convert_image_to_depth_plane"
|
||||
class ConvertToDepthPlane(bpy.types.Operator):
|
||||
bl_idname = "snapform.convert_to_depth_plane"
|
||||
bl_label = "Convert to Depth Plane"
|
||||
bl_description = "Convert an image to a displaced depth plane"
|
||||
bl_options = {"REGISTER"}
|
||||
|
||||
snapform_unshaded: bpy.props.BoolProperty(
|
||||
unshaded: bpy.props.BoolProperty(
|
||||
name="Unshaded",
|
||||
description="Display the source color through an Emission shader",
|
||||
default=False,
|
||||
)
|
||||
snapform_remove_background: bpy.props.BoolProperty(
|
||||
remove_background: bpy.props.BoolProperty(
|
||||
name="Remove Background",
|
||||
description="Remove the background during conversion with BEN2",
|
||||
default=False,
|
||||
)
|
||||
snapform_model: bpy.props.EnumProperty(
|
||||
name="Model",
|
||||
model: bpy.props.EnumProperty(
|
||||
name="Depth From",
|
||||
description="Model or original image used to build depth",
|
||||
items=image_depth_model_items,
|
||||
default=1,
|
||||
default=0,
|
||||
)
|
||||
snapform_use_ray_pose: bpy.props.BoolProperty(
|
||||
name="Use Ray Pose",
|
||||
description="Use ray-based camera pose estimation",
|
||||
default=True,
|
||||
)
|
||||
snapform_process_res_factor: process_resolution_scale_property()
|
||||
snapform_moge_resolution_level: bpy.props.IntProperty(
|
||||
process_res_factor: process_resolution_scale_property()
|
||||
resolution_level: bpy.props.IntProperty(
|
||||
name="Resolution Level",
|
||||
description="MoGe-2 inference resolution level; higher is finer and slower",
|
||||
default=5,
|
||||
min=0,
|
||||
max=9,
|
||||
)
|
||||
snapform_keep_original: bpy.props.BoolProperty(
|
||||
keep_original: bpy.props.BoolProperty(
|
||||
name="Keep Original",
|
||||
description="Keep the source Image Empty after conversion",
|
||||
default=False,
|
||||
@@ -2222,64 +2044,35 @@ class ConvertImageToDepthPlane(bpy.types.Operator):
|
||||
|
||||
@classmethod
|
||||
def poll(cls, context):
|
||||
from .operators import active_job
|
||||
from .client import service_busy
|
||||
from ..job_runtime import runtime
|
||||
|
||||
return (
|
||||
is_image_empty(context.object)
|
||||
and active_job() is None
|
||||
and not service_busy()
|
||||
and not runtime.server_busy()
|
||||
)
|
||||
|
||||
def invoke(self, context, _event):
|
||||
global _last_depth_plane_model
|
||||
|
||||
if not self.properties.is_property_set("snapform_model"):
|
||||
available_keys = [
|
||||
item[0] for item in image_depth_model_items(self, context)
|
||||
]
|
||||
if available_keys:
|
||||
self.snapform_model = (
|
||||
_last_depth_plane_model
|
||||
if _last_depth_plane_model in available_keys
|
||||
else available_keys[0]
|
||||
)
|
||||
return context.window_manager.invoke_props_dialog(self, width=340)
|
||||
|
||||
def draw(self, context):
|
||||
layout = self.layout
|
||||
layout.prop(self, "snapform_model")
|
||||
if self.snapform_model in {"SMALL", "BASE"}:
|
||||
estimate_box = layout.box()
|
||||
estimate_box.prop(self, "snapform_use_ray_pose")
|
||||
draw_process_resolution(estimate_box, self, context.object.data)
|
||||
elif self.snapform_model.startswith("MOGE2_"):
|
||||
from .client import model_ready
|
||||
|
||||
moge_box = layout.box()
|
||||
if not model_ready(self.snapform_model):
|
||||
moge_box.label(text="MoGe-2 model is not installed", icon="ERROR")
|
||||
moge_box.label(text="Download it in Models")
|
||||
moge_box.prop(self, "snapform_moge_resolution_level")
|
||||
layout.prop(self, "snapform_remove_background")
|
||||
layout.prop(self, "snapform_unshaded")
|
||||
layout.prop(self, "model")
|
||||
if self.model in {"SMALL", "BASE"}:
|
||||
draw_process_resolution(layout, self, context.object.data)
|
||||
elif self.model.startswith("MOGE2_"):
|
||||
layout.prop(self, "resolution_level")
|
||||
layout.prop(self, "remove_background")
|
||||
layout.prop(self, "unshaded")
|
||||
layout.separator(type="LINE")
|
||||
layout.prop(self, "snapform_keep_original")
|
||||
layout.prop(self, "keep_original")
|
||||
|
||||
def execute(self, context):
|
||||
from .operators import (
|
||||
IMAGE_DEPTH_JOB,
|
||||
MOGE2_DEPTH_JOB,
|
||||
ORIGINAL_BACKGROUND_JOB,
|
||||
ORIGINAL_FRAMES_JOB,
|
||||
)
|
||||
|
||||
source_object = context.object
|
||||
if video_total_frames(source_object.data) > 1:
|
||||
context.scene.snapform_settings.video_frames = (
|
||||
default_extract_frames(source_object)
|
||||
)
|
||||
if self.snapform_model == "ORIGINAL":
|
||||
if self.model == "ORIGINAL":
|
||||
try:
|
||||
source_path, delete_input = prepare_image_input(
|
||||
source_object.data,
|
||||
@@ -2288,17 +2081,17 @@ class ConvertImageToDepthPlane(bpy.types.Operator):
|
||||
except RuntimeError as error:
|
||||
self.report({"ERROR"}, str(error))
|
||||
return {"CANCELLED"}
|
||||
result = bpy.ops.snapform.run_job(
|
||||
job_type=(
|
||||
ORIGINAL_BACKGROUND_JOB
|
||||
if self.snapform_remove_background
|
||||
else ORIGINAL_FRAMES_JOB
|
||||
),
|
||||
snapform_input_path=str(source_path),
|
||||
snapform_source_object_name=source_object.name,
|
||||
snapform_delete_source=not self.snapform_keep_original,
|
||||
snapform_delete_input=delete_input,
|
||||
snapform_unshaded=self.snapform_unshaded,
|
||||
run_original = (
|
||||
bpy.ops.snapform.generate_original_background
|
||||
if self.remove_background
|
||||
else bpy.ops.snapform.extract_original_frames
|
||||
)
|
||||
result = run_original(
|
||||
input_path=str(source_path),
|
||||
source_object_name=source_object.name,
|
||||
delete_source=not self.keep_original,
|
||||
delete_input=delete_input,
|
||||
unshaded=self.unshaded,
|
||||
)
|
||||
if "CANCELLED" not in result:
|
||||
return {"FINISHED"}
|
||||
@@ -2306,7 +2099,7 @@ class ConvertImageToDepthPlane(bpy.types.Operator):
|
||||
remove_temporary_input(source_path)
|
||||
return {"CANCELLED"}
|
||||
|
||||
if self.snapform_model.startswith("MOGE2_"):
|
||||
if self.model.startswith("MOGE2_"):
|
||||
try:
|
||||
source_path, delete_input = prepare_image_input(
|
||||
source_object.data,
|
||||
@@ -2315,18 +2108,15 @@ class ConvertImageToDepthPlane(bpy.types.Operator):
|
||||
except RuntimeError as error:
|
||||
self.report({"ERROR"}, str(error))
|
||||
return {"CANCELLED"}
|
||||
result = bpy.ops.snapform.run_job(
|
||||
job_type=MOGE2_DEPTH_JOB,
|
||||
snapform_input_path=str(source_path),
|
||||
snapform_source_object_name=source_object.name,
|
||||
snapform_moge_resolution_level=(
|
||||
self.snapform_moge_resolution_level
|
||||
),
|
||||
snapform_model_key=self.snapform_model,
|
||||
snapform_delete_source=not self.snapform_keep_original,
|
||||
snapform_delete_input=delete_input,
|
||||
snapform_unshaded=self.snapform_unshaded,
|
||||
snapform_remove_background=self.snapform_remove_background,
|
||||
result = bpy.ops.snapform.generate_moge2_depth(
|
||||
input_path=str(source_path),
|
||||
source_object_name=source_object.name,
|
||||
resolution_level=self.resolution_level,
|
||||
model=self.model,
|
||||
delete_source=not self.keep_original,
|
||||
delete_input=delete_input,
|
||||
unshaded=self.unshaded,
|
||||
remove_background=self.remove_background,
|
||||
)
|
||||
if "CANCELLED" not in result:
|
||||
return {"FINISHED"}
|
||||
@@ -2338,7 +2128,7 @@ class ConvertImageToDepthPlane(bpy.types.Operator):
|
||||
try:
|
||||
process_res = scaled_process_resolution(
|
||||
image,
|
||||
self.snapform_process_res_factor,
|
||||
self.process_res_factor,
|
||||
)
|
||||
except ValueError as error:
|
||||
self.report({"ERROR"}, str(error))
|
||||
@@ -2352,25 +2142,21 @@ class ConvertImageToDepthPlane(bpy.types.Operator):
|
||||
self.report({"ERROR"}, str(error))
|
||||
return {"CANCELLED"}
|
||||
|
||||
result = bpy.ops.snapform.run_job(
|
||||
job_type=IMAGE_DEPTH_JOB,
|
||||
snapform_input_path=str(source_path),
|
||||
snapform_source_object_name=source_object.name,
|
||||
snapform_depth_data_name=depth_data_name(source_object),
|
||||
snapform_model_key=self.snapform_model,
|
||||
snapform_use_ray_pose=self.snapform_use_ray_pose,
|
||||
snapform_process_res=process_res,
|
||||
snapform_delete_source=not self.snapform_keep_original,
|
||||
snapform_delete_input=delete_input,
|
||||
snapform_unshaded=self.snapform_unshaded,
|
||||
snapform_remove_background=self.snapform_remove_background,
|
||||
result = bpy.ops.snapform.generate_da3_depth(
|
||||
input_path=str(source_path),
|
||||
source_object_name=source_object.name,
|
||||
depth_data_name=depth_data_name(source_object),
|
||||
model=self.model,
|
||||
process_res=process_res,
|
||||
delete_source=not self.keep_original,
|
||||
delete_input=delete_input,
|
||||
unshaded=self.unshaded,
|
||||
remove_background=self.remove_background,
|
||||
)
|
||||
if "CANCELLED" in result:
|
||||
if delete_input:
|
||||
remove_temporary_input(source_path)
|
||||
return {"CANCELLED"}
|
||||
global _last_depth_plane_model
|
||||
_last_depth_plane_model = self.snapform_model
|
||||
return {"FINISHED"}
|
||||
|
||||
|
||||
@@ -2379,25 +2165,35 @@ class SnapFormImageMenu(bpy.types.Menu):
|
||||
bl_label = "SnapForm"
|
||||
|
||||
def draw(self, _context):
|
||||
from .client import env_installed
|
||||
from ..job_runtime import runtime
|
||||
from ..properties import required_models_ready
|
||||
|
||||
layout = self.layout
|
||||
layout.operator_context = "INVOKE_DEFAULT"
|
||||
if not env_installed():
|
||||
operator = layout.operator(
|
||||
"snapform.run_job",
|
||||
if not runtime.environment_ready():
|
||||
layout.operator(
|
||||
"snapform.install_environment",
|
||||
text="Install Environment",
|
||||
icon="IMPORT",
|
||||
)
|
||||
operator.job_type = "INSTALL_ENV"
|
||||
return
|
||||
layout.operator(
|
||||
"snapform.convert_image_to_depth_plane",
|
||||
models_ready = required_models_ready()
|
||||
if not models_ready:
|
||||
layout.operator(
|
||||
"snapform.download_required_models",
|
||||
text="Download Required Models",
|
||||
icon="IMPORT",
|
||||
)
|
||||
layout.separator()
|
||||
depth_plane_row = layout.row()
|
||||
depth_plane_row.enabled = models_ready
|
||||
depth_plane_row.operator(
|
||||
"snapform.convert_to_depth_plane",
|
||||
text="Convert to Depth Plane",
|
||||
icon="MOD_DISPLACE",
|
||||
)
|
||||
layout.operator(
|
||||
"snapform.convert_image_to_mesh",
|
||||
"snapform.convert_to_surface",
|
||||
text="Convert to Surface",
|
||||
icon="MESH_GRID",
|
||||
)
|
||||
@@ -2,49 +2,52 @@ from pathlib import Path
|
||||
|
||||
import bpy
|
||||
|
||||
from .image_depth import (
|
||||
from .depth_plane import (
|
||||
MOGE_SURFACE_MATERIAL_NODE_GROUP_NAME,
|
||||
create_material,
|
||||
draw_process_resolution,
|
||||
image_depth_model_items,
|
||||
is_image_empty,
|
||||
normal_data_name,
|
||||
plane_bounds,
|
||||
prepare_image_input,
|
||||
process_resolution_scale_property,
|
||||
remove_temporary_input,
|
||||
scaled_process_resolution,
|
||||
set_modifier_input,
|
||||
video_total_frames,
|
||||
)
|
||||
|
||||
|
||||
_last_mesh_model = "BASE"
|
||||
|
||||
POINT_ATTRIBUTES = {
|
||||
"o_depth": ("FLOAT", "point_depth"),
|
||||
"o_confidence": ("FLOAT", "point_confidence"),
|
||||
"o_confidence_percentile": (
|
||||
"FLOAT",
|
||||
"point_confidence_percentile",
|
||||
),
|
||||
"o_background": ("BOOLEAN", "point_background"),
|
||||
}
|
||||
FACE_ATTRIBUTES = {
|
||||
"o_face_confidence": (
|
||||
"FLOAT",
|
||||
"face_confidence_percentile",
|
||||
),
|
||||
"o_edge_stretch": ("FLOAT", "face_edge_stretch"),
|
||||
"o_background_face": ("BOOLEAN", "face_background"),
|
||||
}
|
||||
OPTIONAL_FACE_ATTRIBUTES = {
|
||||
"o_mask": ("BOOLEAN", "face_mask"),
|
||||
"o_depth_discontinuity": ("FLOAT", "face_depth_discontinuity"),
|
||||
}
|
||||
NODE_GROUP_NAME = "O DA3 Surface"
|
||||
MOGE_NODE_GROUP_NAME = "O MoGe2 Surface"
|
||||
NODE_ASSET_NAME = "O_SnapForm.blend"
|
||||
|
||||
|
||||
def load_image_mesh_data(npz_path):
|
||||
def image_surface_model_items(owner, context):
|
||||
from ..properties import surface_model_enum_items
|
||||
|
||||
return surface_model_enum_items(owner, context)
|
||||
|
||||
|
||||
def load_image_mesh_data(npz_path, moge2=False):
|
||||
import numpy as np
|
||||
|
||||
try:
|
||||
with np.load(npz_path, allow_pickle=False) as archive:
|
||||
required = (
|
||||
required = [
|
||||
"vertices",
|
||||
"faces",
|
||||
"uv",
|
||||
@@ -52,13 +55,13 @@ def load_image_mesh_data(npz_path):
|
||||
"image_size",
|
||||
"reference_depth",
|
||||
"point_depth",
|
||||
"point_confidence",
|
||||
"point_confidence_percentile",
|
||||
"point_background",
|
||||
"face_confidence_percentile",
|
||||
"face_edge_stretch",
|
||||
"face_background",
|
||||
)
|
||||
]
|
||||
if moge2:
|
||||
required.extend(("face_mask", "face_depth_discontinuity", "normal_map"))
|
||||
else:
|
||||
required.extend(
|
||||
("face_confidence_percentile", "face_depth_discontinuity")
|
||||
)
|
||||
missing = [key for key in required if key not in archive.files]
|
||||
if missing:
|
||||
raise ValueError(
|
||||
@@ -81,7 +84,16 @@ def load_image_mesh_data(npz_path):
|
||||
**POINT_ATTRIBUTES,
|
||||
**FACE_ATTRIBUTES,
|
||||
}.items()
|
||||
if key in archive.files
|
||||
}
|
||||
for _name, (_data_type, key) in OPTIONAL_FACE_ATTRIBUTES.items():
|
||||
if key in archive.files:
|
||||
attributes[key] = np.asarray(archive[key])
|
||||
if "normal_map" in archive.files:
|
||||
attributes["normal_map"] = np.asarray(
|
||||
archive["normal_map"],
|
||||
dtype=np.float32,
|
||||
)
|
||||
except OSError as error:
|
||||
raise ValueError(f"Unable to read surface data: {npz_path}") from error
|
||||
if vertices.ndim != 2 or vertices.shape[1] != 3:
|
||||
@@ -96,11 +108,22 @@ def load_image_mesh_data(npz_path):
|
||||
raise ValueError(f"Unsupported image size: {image_size}")
|
||||
if not reference_depth > 0.0:
|
||||
raise ValueError("Surface reference depth must be positive")
|
||||
if moge2 and attributes["normal_map"].shape != (
|
||||
image_size[1],
|
||||
image_size[0],
|
||||
3,
|
||||
):
|
||||
raise ValueError("Surface normal map and image sizes do not match")
|
||||
if moge2 and not np.isfinite(attributes["normal_map"]).all():
|
||||
raise ValueError("Surface normal map contains invalid values")
|
||||
for _name, (_data_type, key) in POINT_ATTRIBUTES.items():
|
||||
if len(attributes[key]) != len(vertices):
|
||||
if key in attributes and len(attributes[key]) != len(vertices):
|
||||
raise ValueError(f"Point attribute has the wrong length: {key}")
|
||||
for _name, (_data_type, key) in FACE_ATTRIBUTES.items():
|
||||
if len(attributes[key]) != len(faces):
|
||||
if key in attributes and len(attributes[key]) != len(faces):
|
||||
raise ValueError(f"Face attribute has the wrong length: {key}")
|
||||
for _name, (_data_type, key) in OPTIONAL_FACE_ATTRIBUTES.items():
|
||||
if key in attributes and len(attributes[key]) != len(faces):
|
||||
raise ValueError(f"Face attribute has the wrong length: {key}")
|
||||
return (
|
||||
vertices,
|
||||
@@ -185,7 +208,18 @@ def assign_mesh_attribute(mesh, name, data_type, domain, values):
|
||||
attribute.data.foreach_set("value", array.reshape(-1))
|
||||
|
||||
|
||||
def assign_snapform_attributes(mesh, attributes):
|
||||
def assign_snapform_attributes(mesh, attributes, moge2=False):
|
||||
if moge2:
|
||||
assign_mesh_attribute(mesh, "o_depth", "FLOAT", "POINT", attributes["point_depth"])
|
||||
assign_mesh_attribute(mesh, "o_mask", "BOOLEAN", "FACE", attributes["face_mask"])
|
||||
assign_mesh_attribute(
|
||||
mesh,
|
||||
"o_depth_discontinuity",
|
||||
"FLOAT",
|
||||
"FACE",
|
||||
attributes["face_depth_discontinuity"],
|
||||
)
|
||||
return
|
||||
for name, (data_type, key) in POINT_ATTRIBUTES.items():
|
||||
assign_mesh_attribute(
|
||||
mesh,
|
||||
@@ -202,38 +236,104 @@ def assign_snapform_attributes(mesh, attributes):
|
||||
"FACE",
|
||||
attributes[key],
|
||||
)
|
||||
|
||||
|
||||
def remove_surface_background(vertices, faces, uv, attributes):
|
||||
"""Remove background faces and compact matching point/face data."""
|
||||
import numpy as np
|
||||
|
||||
background_faces = np.asarray(attributes["face_background"], dtype=bool)
|
||||
kept_faces = faces[~background_faces]
|
||||
if not len(kept_faces):
|
||||
raise ValueError("Background removal deleted every surface face")
|
||||
used_vertices = np.unique(kept_faces)
|
||||
index_map = np.full(len(vertices), -1, dtype=np.int32)
|
||||
index_map[used_vertices] = np.arange(len(used_vertices), dtype=np.int32)
|
||||
filtered_attributes = {}
|
||||
for _name, (_data_type, key) in POINT_ATTRIBUTES.items():
|
||||
filtered_attributes[key] = attributes[key][used_vertices]
|
||||
for _name, (_data_type, key) in FACE_ATTRIBUTES.items():
|
||||
filtered_attributes[key] = attributes[key][~background_faces]
|
||||
return (
|
||||
vertices[used_vertices],
|
||||
index_map[kept_faces],
|
||||
uv[used_vertices],
|
||||
filtered_attributes,
|
||||
for name, (data_type, key) in OPTIONAL_FACE_ATTRIBUTES.items():
|
||||
if key in attributes:
|
||||
assign_mesh_attribute(
|
||||
mesh,
|
||||
name,
|
||||
data_type,
|
||||
"FACE",
|
||||
attributes[key],
|
||||
)
|
||||
assign_mesh_attribute(
|
||||
mesh,
|
||||
"o_depth_discontinuity",
|
||||
"FLOAT",
|
||||
"FACE",
|
||||
attributes["face_depth_discontinuity"],
|
||||
)
|
||||
|
||||
|
||||
def prepare_moge_surface_archive(results_path, output_path):
|
||||
import numpy as np
|
||||
|
||||
with np.load(results_path, allow_pickle=False) as archive:
|
||||
required = {"depth", "mask", "normal", "points", "intrinsics"}
|
||||
missing = required.difference(archive.files)
|
||||
if missing:
|
||||
raise ValueError(f"MoGe-2 surface data is missing: {', '.join(sorted(missing))}")
|
||||
depth = np.asarray(archive["depth"], dtype=np.float32)
|
||||
mask = np.asarray(archive["mask"], dtype=bool)
|
||||
points = np.asarray(archive["points"], dtype=np.float32)
|
||||
intrinsics = np.asarray(archive["intrinsics"], dtype=np.float32)
|
||||
normal = np.asarray(archive["normal"], dtype=np.float32)
|
||||
if depth.shape[0] != 1 or points.shape[:3] != (*depth.shape,):
|
||||
raise ValueError("Convert to Surface requires one matching MoGe-2 point grid")
|
||||
raw_depth = depth[0]
|
||||
finite = np.isfinite(raw_depth) & (raw_depth > 0.0)
|
||||
finite &= np.isfinite(points[0]).all(axis=-1)
|
||||
height, width = raw_depth.shape
|
||||
indices = np.arange(height * width, dtype=np.int32).reshape(height, width)
|
||||
quads = np.stack(
|
||||
(indices[:-1, :-1], indices[1:, :-1], indices[1:, 1:], indices[:-1, 1:]),
|
||||
axis=-1,
|
||||
).reshape(-1, 4)
|
||||
faces = np.concatenate((quads[:, (0, 1, 2)], quads[:, (0, 2, 3)]), axis=0)
|
||||
faces = faces[finite.reshape(-1)[faces].all(axis=1)]
|
||||
if not len(faces):
|
||||
raise ValueError("MoGe-2 output does not contain a valid surface")
|
||||
flat_points = points[0].reshape(-1, 3)
|
||||
face_mask = mask[0].reshape(-1)[faces].all(axis=1)
|
||||
depth_edge_ratio = depth_discontinuity_ratio(raw_depth)
|
||||
face_depth_discontinuity = depth_edge_ratio.reshape(-1)[faces].max(axis=1)
|
||||
used = np.unique(faces)
|
||||
index_map = np.full(height * width, -1, dtype=np.int32)
|
||||
index_map[used] = np.arange(len(used), dtype=np.int32)
|
||||
u = np.linspace(0.0, 1.0, width, dtype=np.float32)
|
||||
v = np.linspace(1.0, 0.0, height, dtype=np.float32)
|
||||
uv = np.stack(np.meshgrid(u, v), axis=-1).reshape(-1, 2)[used]
|
||||
camera_intrinsics = intrinsics[0].copy()
|
||||
principal_x = float(camera_intrinsics[0, 2])
|
||||
principal_y = float(camera_intrinsics[1, 2])
|
||||
if 0.0 <= principal_x <= 1.0 and 0.0 <= principal_y <= 1.0:
|
||||
camera_intrinsics[0] *= width
|
||||
camera_intrinsics[1] *= height
|
||||
camera_intrinsics[2] = (0.0, 0.0, 1.0)
|
||||
point_depth = raw_depth.reshape(-1)[used]
|
||||
masked_depth = raw_depth[finite & mask[0]]
|
||||
reference_depth = float(np.median(masked_depth if len(masked_depth) else point_depth))
|
||||
np.savez(
|
||||
output_path,
|
||||
vertices=flat_points[used],
|
||||
faces=index_map[faces],
|
||||
uv=uv,
|
||||
intrinsics=camera_intrinsics,
|
||||
image_size=np.array((width, height), dtype=np.int32),
|
||||
reference_depth=np.float32(reference_depth),
|
||||
point_depth=point_depth,
|
||||
normal_map=normal[0],
|
||||
face_mask=face_mask,
|
||||
face_depth_discontinuity=face_depth_discontinuity,
|
||||
)
|
||||
return output_path
|
||||
|
||||
|
||||
def depth_discontinuity_ratio(depth):
|
||||
import numpy as np
|
||||
|
||||
padded = np.pad(depth, 1, mode="edge")
|
||||
windows = np.lib.stride_tricks.sliding_window_view(padded, (3, 3))
|
||||
difference = np.nanmax(windows, axis=(-2, -1))
|
||||
difference -= np.nanmin(windows, axis=(-2, -1))
|
||||
return difference / np.maximum(depth, 1e-6)
|
||||
|
||||
|
||||
def node_asset_path():
|
||||
return Path(__file__).with_name("assets") / NODE_ASSET_NAME
|
||||
return Path(__file__).parent.parent / "assets" / NODE_ASSET_NAME
|
||||
|
||||
|
||||
def load_geometry_node_group():
|
||||
existing = bpy.data.node_groups.get(NODE_GROUP_NAME)
|
||||
def load_geometry_node_group(group_name=NODE_GROUP_NAME):
|
||||
existing = bpy.data.node_groups.get(group_name)
|
||||
if existing is not None:
|
||||
return existing
|
||||
asset_path = node_asset_path()
|
||||
@@ -243,15 +343,15 @@ def load_geometry_node_group():
|
||||
if not use_pack:
|
||||
directory = str(asset_path) + "/NodeTree/"
|
||||
result = bpy.ops.wm.append(
|
||||
filepath=directory + NODE_GROUP_NAME,
|
||||
filepath=directory + group_name,
|
||||
directory=directory,
|
||||
filename=NODE_GROUP_NAME,
|
||||
filename=group_name,
|
||||
link=False,
|
||||
do_reuse_local_id=True,
|
||||
)
|
||||
if "FINISHED" not in result:
|
||||
raise RuntimeError("Unable to append the Geometry Nodes asset")
|
||||
node_group = bpy.data.node_groups.get(NODE_GROUP_NAME)
|
||||
node_group = bpy.data.node_groups.get(group_name)
|
||||
if node_group is None:
|
||||
raise RuntimeError("Appended Geometry Nodes asset is unavailable")
|
||||
node_group.use_fake_user = True
|
||||
@@ -260,11 +360,11 @@ def load_geometry_node_group():
|
||||
source,
|
||||
target,
|
||||
):
|
||||
if NODE_GROUP_NAME not in source.node_groups:
|
||||
if group_name not in source.node_groups:
|
||||
raise RuntimeError(
|
||||
f"Geometry Nodes asset is missing {NODE_GROUP_NAME}"
|
||||
f"Geometry Nodes asset is missing {group_name}"
|
||||
)
|
||||
target.node_groups = [NODE_GROUP_NAME]
|
||||
target.node_groups = [group_name]
|
||||
node_group = target.node_groups[0]
|
||||
if use_pack:
|
||||
linked_group = node_group
|
||||
@@ -307,13 +407,22 @@ def material_depth_scale(mesh_object, camera_parameters):
|
||||
)
|
||||
|
||||
|
||||
def add_geometry_nodes_processing(mesh_object, vertices, faces):
|
||||
node_group = load_geometry_node_group()
|
||||
def add_geometry_nodes_processing(
|
||||
mesh_object,
|
||||
vertices,
|
||||
faces,
|
||||
group_name=NODE_GROUP_NAME,
|
||||
):
|
||||
node_group = load_geometry_node_group(group_name)
|
||||
modifier = mesh_object.modifiers.new(name=node_group.name, type="NODES")
|
||||
modifier.node_group = node_group
|
||||
modifier.show_group_selector = False
|
||||
identifier = modifier_input_identifier(node_group, "Resolution")
|
||||
modifier[identifier] = default_remesh_resolution(vertices, faces)
|
||||
set_modifier_input(
|
||||
modifier,
|
||||
identifier,
|
||||
default_remesh_resolution(vertices, faces),
|
||||
)
|
||||
|
||||
|
||||
def create_matching_camera(
|
||||
@@ -359,8 +468,10 @@ def build_image_mesh(
|
||||
delete_source,
|
||||
create_camera,
|
||||
unshaded,
|
||||
remove_background=False,
|
||||
moge2=False,
|
||||
):
|
||||
import numpy as np
|
||||
|
||||
(
|
||||
camera_vertices,
|
||||
faces,
|
||||
@@ -369,14 +480,7 @@ def build_image_mesh(
|
||||
image_size,
|
||||
reference_depth,
|
||||
attributes,
|
||||
) = load_image_mesh_data(npz_path)
|
||||
if remove_background:
|
||||
camera_vertices, faces, uv, attributes = remove_surface_background(
|
||||
camera_vertices,
|
||||
faces,
|
||||
uv,
|
||||
attributes,
|
||||
)
|
||||
) = load_image_mesh_data(npz_path, moge2=moge2)
|
||||
vertices, camera_parameters = empty_anchored_vertices(
|
||||
source_object,
|
||||
camera_vertices,
|
||||
@@ -388,7 +492,7 @@ def build_image_mesh(
|
||||
mesh = bpy.data.meshes.new(source_name)
|
||||
mesh.from_pydata(vertices.tolist(), [], faces.tolist())
|
||||
assign_vertex_uv(mesh, uv)
|
||||
assign_snapform_attributes(mesh, attributes)
|
||||
assign_snapform_attributes(mesh, attributes, moge2=moge2)
|
||||
mesh.update()
|
||||
for polygon in mesh.polygons:
|
||||
polygon.use_smooth = True
|
||||
@@ -396,16 +500,40 @@ def build_image_mesh(
|
||||
context.collection.objects.link(mesh_object)
|
||||
mesh_object.matrix_world = source_object.matrix_world.copy()
|
||||
depth_scale = material_depth_scale(mesh_object, camera_parameters)
|
||||
normal_image = None
|
||||
if moge2 and attributes.get("normal_map", np.empty(0)).size:
|
||||
from .depth_plane import create_moge_normal_image
|
||||
|
||||
normal_map = attributes["normal_map"].copy()
|
||||
normal_map[..., 0] /= camera_parameters["scale_x"]
|
||||
normal_map[..., 1] /= -camera_parameters["scale_y"]
|
||||
normal_map[..., 2] /= -camera_parameters["scale_z"]
|
||||
lengths = np.linalg.norm(normal_map, axis=-1, keepdims=True)
|
||||
normal_map /= np.maximum(lengths, 1e-6)
|
||||
normal_image = create_moge_normal_image(
|
||||
normal_data_name(source_object),
|
||||
normal_map,
|
||||
convert_coordinates=False,
|
||||
)
|
||||
material = create_material(
|
||||
source_name,
|
||||
source_object.data,
|
||||
depth_scale,
|
||||
unshaded=unshaded,
|
||||
surface_group_name=(
|
||||
MOGE_SURFACE_MATERIAL_NODE_GROUP_NAME if moge2 else None
|
||||
),
|
||||
normal_image=normal_image,
|
||||
)
|
||||
mesh.materials.append(material)
|
||||
mesh_object["snapform_reference_depth"] = reference_depth
|
||||
mesh_object["snapform_source_type"] = "IMAGE_MESH"
|
||||
add_geometry_nodes_processing(mesh_object, vertices, faces)
|
||||
add_geometry_nodes_processing(
|
||||
mesh_object,
|
||||
vertices,
|
||||
faces,
|
||||
MOGE_NODE_GROUP_NAME if moge2 else NODE_GROUP_NAME,
|
||||
)
|
||||
if create_camera:
|
||||
create_matching_camera(
|
||||
context,
|
||||
@@ -426,88 +554,42 @@ def build_image_mesh(
|
||||
return mesh_object
|
||||
|
||||
|
||||
class BuildImageMesh(bpy.types.Operator):
|
||||
bl_idname = "snapform.build_image_mesh"
|
||||
bl_label = "Build Image Surface"
|
||||
bl_options = {"INTERNAL"}
|
||||
|
||||
snapform_source_object_name: bpy.props.StringProperty(
|
||||
options={"HIDDEN", "SKIP_SAVE"},
|
||||
)
|
||||
snapform_npz_path: bpy.props.StringProperty(
|
||||
options={"HIDDEN", "SKIP_SAVE"},
|
||||
)
|
||||
snapform_delete_source: bpy.props.BoolProperty(
|
||||
default=True,
|
||||
options={"HIDDEN", "SKIP_SAVE"},
|
||||
)
|
||||
snapform_create_camera: bpy.props.BoolProperty(
|
||||
default=False,
|
||||
options={"HIDDEN", "SKIP_SAVE"},
|
||||
)
|
||||
snapform_unshaded: bpy.props.BoolProperty(
|
||||
default=False,
|
||||
options={"HIDDEN", "SKIP_SAVE"},
|
||||
)
|
||||
snapform_remove_background: bpy.props.BoolProperty(
|
||||
default=False,
|
||||
options={"HIDDEN", "SKIP_SAVE"},
|
||||
)
|
||||
|
||||
def execute(self, context):
|
||||
source_object = bpy.data.objects.get(self.snapform_source_object_name)
|
||||
if not is_image_empty(source_object):
|
||||
self.report({"ERROR"}, "The source Image Empty no longer exists")
|
||||
return {"CANCELLED"}
|
||||
try:
|
||||
build_image_mesh(
|
||||
context,
|
||||
source_object,
|
||||
self.snapform_npz_path,
|
||||
self.snapform_delete_source,
|
||||
self.snapform_create_camera,
|
||||
self.snapform_unshaded,
|
||||
self.snapform_remove_background,
|
||||
)
|
||||
except (OSError, RuntimeError, ValueError) as error:
|
||||
self.report({"ERROR"}, str(error))
|
||||
return {"CANCELLED"}
|
||||
return {"FINISHED"}
|
||||
|
||||
|
||||
class ConvertImageToMesh(bpy.types.Operator):
|
||||
bl_idname = "snapform.convert_image_to_mesh"
|
||||
class ConvertToSurface(bpy.types.Operator):
|
||||
bl_idname = "snapform.convert_to_surface"
|
||||
bl_label = "Convert to Surface"
|
||||
bl_description = "Convert an Image Empty into a perspective textured surface"
|
||||
bl_options = {"REGISTER"}
|
||||
|
||||
snapform_model: bpy.props.EnumProperty(
|
||||
name="Model",
|
||||
model: bpy.props.EnumProperty(
|
||||
name="Depth From",
|
||||
description="Model used to estimate image depth",
|
||||
items=image_depth_model_items,
|
||||
items=image_surface_model_items,
|
||||
default=0,
|
||||
)
|
||||
snapform_use_ray_pose: bpy.props.BoolProperty(
|
||||
name="Use Ray Pose",
|
||||
description="Use ray-based camera pose estimation",
|
||||
default=True,
|
||||
process_res_factor: process_resolution_scale_property()
|
||||
resolution_level: bpy.props.IntProperty(
|
||||
name="Resolution Level",
|
||||
description="MoGe-2 inference resolution level; higher is finer and slower",
|
||||
default=5,
|
||||
min=0,
|
||||
max=9,
|
||||
)
|
||||
snapform_process_res_factor: process_resolution_scale_property()
|
||||
snapform_create_camera: bpy.props.BoolProperty(
|
||||
create_camera: bpy.props.BoolProperty(
|
||||
name="Create Matching Camera",
|
||||
description="Create a perspective camera matching the source projection",
|
||||
default=False,
|
||||
)
|
||||
snapform_unshaded: bpy.props.BoolProperty(
|
||||
unshaded: bpy.props.BoolProperty(
|
||||
name="Unshaded",
|
||||
description="Display the source color through an Emission shader",
|
||||
default=False,
|
||||
)
|
||||
snapform_remove_background: bpy.props.BoolProperty(
|
||||
remove_background: bpy.props.BoolProperty(
|
||||
name="Remove Background",
|
||||
description="Remove the background during conversion with BEN2",
|
||||
default=False,
|
||||
)
|
||||
snapform_keep_original: bpy.props.BoolProperty(
|
||||
keep_original: bpy.props.BoolProperty(
|
||||
name="Keep Original",
|
||||
description="Keep the source Image Empty after conversion",
|
||||
default=False,
|
||||
@@ -515,56 +597,49 @@ class ConvertImageToMesh(bpy.types.Operator):
|
||||
|
||||
@classmethod
|
||||
def poll(cls, context):
|
||||
from .models import MODELS
|
||||
from .operators import active_job
|
||||
from .client import service_busy, model_ready
|
||||
from ..job_runtime import runtime
|
||||
from ..properties import model_catalog
|
||||
|
||||
return (
|
||||
is_image_empty(context.object)
|
||||
and video_total_frames(context.object.data) == 1
|
||||
and active_job() is None
|
||||
and not service_busy()
|
||||
and any(model_ready(model_key) for model_key in MODELS)
|
||||
and not runtime.server_busy()
|
||||
and any(
|
||||
model.get("ready")
|
||||
and model.get("enabled")
|
||||
and model.get("family") in {"moge2", "da3"}
|
||||
for model in model_catalog()
|
||||
)
|
||||
)
|
||||
|
||||
def invoke(self, context, _event):
|
||||
global _last_mesh_model
|
||||
|
||||
if not self.properties.is_property_set("snapform_model"):
|
||||
available_keys = [
|
||||
item[0] for item in image_depth_model_items(self, context)
|
||||
]
|
||||
if available_keys:
|
||||
self.snapform_model = (
|
||||
_last_mesh_model
|
||||
if _last_mesh_model in available_keys
|
||||
else available_keys[0]
|
||||
)
|
||||
return context.window_manager.invoke_props_dialog(self, width=360)
|
||||
|
||||
def draw(self, context):
|
||||
layout = self.layout
|
||||
layout.prop(self, "snapform_model")
|
||||
layout.prop(self, "snapform_use_ray_pose")
|
||||
draw_process_resolution(layout, self, context.object.data)
|
||||
layout.prop(self, "snapform_create_camera")
|
||||
layout.prop(self, "snapform_remove_background")
|
||||
layout.prop(self, "snapform_unshaded")
|
||||
layout.prop(self, "model")
|
||||
if self.model.startswith("MOGE2_"):
|
||||
layout.prop(self, "resolution_level")
|
||||
else:
|
||||
draw_process_resolution(layout, self, context.object.data)
|
||||
layout.prop(self, "create_camera")
|
||||
layout.prop(self, "remove_background")
|
||||
layout.prop(self, "unshaded")
|
||||
layout.separator(type="LINE")
|
||||
layout.prop(self, "snapform_keep_original")
|
||||
layout.prop(self, "keep_original")
|
||||
|
||||
def execute(self, context):
|
||||
from .operators import IMAGE_MESH_JOB
|
||||
|
||||
source_object = context.object
|
||||
if video_total_frames(source_object.data) != 1:
|
||||
self.report({"ERROR"}, "Convert to Surface currently supports still images")
|
||||
return {"CANCELLED"}
|
||||
try:
|
||||
process_res = scaled_process_resolution(
|
||||
source_object.data,
|
||||
self.snapform_process_res_factor,
|
||||
)
|
||||
process_res = 1
|
||||
if not self.model.startswith("MOGE2_"):
|
||||
process_res = scaled_process_resolution(
|
||||
source_object.data,
|
||||
self.process_res_factor,
|
||||
)
|
||||
source_path, delete_input = prepare_image_input(
|
||||
source_object.data,
|
||||
context.scene,
|
||||
@@ -572,23 +647,28 @@ class ConvertImageToMesh(bpy.types.Operator):
|
||||
except (RuntimeError, ValueError) as error:
|
||||
self.report({"ERROR"}, str(error))
|
||||
return {"CANCELLED"}
|
||||
result = bpy.ops.snapform.run_job(
|
||||
job_type=IMAGE_MESH_JOB,
|
||||
snapform_input_path=str(source_path),
|
||||
snapform_source_object_name=source_object.name,
|
||||
snapform_model_key=self.snapform_model,
|
||||
snapform_use_ray_pose=self.snapform_use_ray_pose,
|
||||
snapform_process_res=process_res,
|
||||
snapform_delete_source=not self.snapform_keep_original,
|
||||
snapform_delete_input=delete_input,
|
||||
snapform_mesh_create_camera=self.snapform_create_camera,
|
||||
snapform_unshaded=self.snapform_unshaded,
|
||||
snapform_remove_background=self.snapform_remove_background,
|
||||
)
|
||||
common = {
|
||||
"input_path": str(source_path),
|
||||
"source_object_name": source_object.name,
|
||||
"model": self.model,
|
||||
"delete_source": not self.keep_original,
|
||||
"delete_input": delete_input,
|
||||
"create_camera": self.create_camera,
|
||||
"unshaded": self.unshaded,
|
||||
"remove_background": self.remove_background,
|
||||
}
|
||||
if self.model.startswith("MOGE2_"):
|
||||
result = bpy.ops.snapform.generate_moge2_surface(
|
||||
**common,
|
||||
resolution_level=self.resolution_level,
|
||||
)
|
||||
else:
|
||||
result = bpy.ops.snapform.generate_da3_surface(
|
||||
**common,
|
||||
process_res=process_res,
|
||||
)
|
||||
if "CANCELLED" in result:
|
||||
if delete_input:
|
||||
remove_temporary_input(source_path)
|
||||
return {"CANCELLED"}
|
||||
global _last_mesh_model
|
||||
_last_mesh_model = self.snapform_model
|
||||
return {"FINISHED"}
|
||||
@@ -1,7 +1,7 @@
|
||||
schema_version = "1.0.0"
|
||||
|
||||
id = "snapform"
|
||||
version = "0.10.11"
|
||||
version = "0.12.3"
|
||||
name = "SnapForm"
|
||||
tagline = "Build spatial structures from images, fast"
|
||||
maintainer = "OmooLab"
|
||||
@@ -10,9 +10,10 @@ type = "add-on"
|
||||
blender_version_min = "4.5.0"
|
||||
platforms = ["windows-x64", "macos-arm64", "linux-x64"]
|
||||
license = ["SPDX:GPL-3.0-or-later"]
|
||||
wheels = ["./wheels/blendjob-0.1.6-py3-none-any.whl"]
|
||||
|
||||
[permissions]
|
||||
network = "Access the local Service and download environment dependencies and model weights"
|
||||
network = "Access the local Job Server and download environment dependencies and model weights"
|
||||
files = "Read source images and store environment data"
|
||||
|
||||
[build]
|
||||
|
||||
@@ -1,960 +0,0 @@
|
||||
import json
|
||||
import os
|
||||
import platform
|
||||
import secrets
|
||||
import shutil
|
||||
import socket
|
||||
import subprocess
|
||||
import sys
|
||||
import tempfile
|
||||
import threading
|
||||
import time
|
||||
import uuid
|
||||
from dataclasses import dataclass
|
||||
from datetime import datetime
|
||||
from pathlib import Path
|
||||
from urllib.error import HTTPError, URLError
|
||||
from urllib.request import ProxyHandler, Request, build_opener
|
||||
|
||||
import bpy
|
||||
|
||||
from .models import (
|
||||
BEN2_MODEL,
|
||||
get_downloadable_model,
|
||||
model_directory_name,
|
||||
selected_model_key,
|
||||
)
|
||||
|
||||
|
||||
DA3_COMMIT = "3d835ec1a5802d64a8b8b15f817a1ab54809bfe4"
|
||||
BEN2_COMMIT = "2c99a5da477b5523585bfa5c893888a6e818a8f6"
|
||||
MANIFEST_FILENAME = "manifest.json"
|
||||
MANIFEST_REVISION = 5
|
||||
SERVICE_ADDRESS = "127.0.0.1"
|
||||
DEFAULT_STORAGE_ROOT = Path.home() / ".snapform"
|
||||
JOB_FOLDER_NAMES = {
|
||||
"INSTALL_ENV": "install-environment",
|
||||
"DOWNLOAD_MODEL": "download-model",
|
||||
"RECONSTRUCT": "export-reconstruction",
|
||||
"IMAGE_DEPTH": "generate-depth-plane",
|
||||
"MOGE2_DEPTH": "generate-moge2-depth-plane",
|
||||
"MOGE2_DEBUG": "test-moge2-api",
|
||||
"IMAGE_MESH": "generate-image-mesh",
|
||||
"REMOVE_BACKGROUND": "remove-background",
|
||||
"IMAGE_BACKGROUND": "remove-image-background",
|
||||
"ORIGINAL_BACKGROUND": "generate-original-depth-plane",
|
||||
"ORIGINAL_FRAMES": "extract-original-frames",
|
||||
}
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class JobRequest:
|
||||
payload: dict
|
||||
|
||||
|
||||
@dataclass
|
||||
class Connection:
|
||||
port: int
|
||||
token: str
|
||||
pid: int
|
||||
instance_id: str
|
||||
|
||||
|
||||
@dataclass
|
||||
class ServiceProcess:
|
||||
process: subprocess.Popen
|
||||
log_file: object
|
||||
|
||||
|
||||
_service_connection = None
|
||||
_launched_service = None
|
||||
_registered_instance_id = None
|
||||
_client_id = uuid.uuid4().hex
|
||||
_service_auto_start = False
|
||||
_service_snapshot = {"state": "STOPPED", "message": "Stopped"}
|
||||
_local_opener = build_opener(ProxyHandler({}))
|
||||
_client_label = f"Blender {getattr(bpy.app, 'version_string', '')}".strip()
|
||||
|
||||
|
||||
def preferences():
|
||||
addon = bpy.context.preferences.addons.get(__package__)
|
||||
return addon.preferences if addon is not None else None
|
||||
|
||||
|
||||
def storage_root():
|
||||
addon_preferences = preferences()
|
||||
configured = getattr(addon_preferences, "storage_root", "")
|
||||
root = (
|
||||
Path(bpy.path.abspath(configured)).expanduser()
|
||||
if configured
|
||||
else DEFAULT_STORAGE_ROOT
|
||||
)
|
||||
root.mkdir(parents=True, exist_ok=True)
|
||||
return root
|
||||
|
||||
|
||||
def venv_directory():
|
||||
return storage_root() / ".venv"
|
||||
|
||||
|
||||
def venv_python():
|
||||
directory = venv_directory()
|
||||
if os.name == "nt":
|
||||
return directory / "Scripts" / "python.exe"
|
||||
return directory / "bin" / "python"
|
||||
|
||||
|
||||
def env_installed():
|
||||
ready_path = storage_root() / MANIFEST_FILENAME
|
||||
if not venv_python().is_file() or not ready_path.is_file():
|
||||
return False
|
||||
try:
|
||||
manifest = json.loads(ready_path.read_text(encoding="utf-8"))
|
||||
except (json.JSONDecodeError, OSError):
|
||||
return False
|
||||
return (
|
||||
isinstance(manifest, dict)
|
||||
and manifest.get("manifest_revision") == MANIFEST_REVISION
|
||||
)
|
||||
|
||||
|
||||
def models_directory():
|
||||
path = storage_root() / "models"
|
||||
path.mkdir(parents=True, exist_ok=True)
|
||||
return path
|
||||
|
||||
|
||||
def model_directory(model_key):
|
||||
root = models_directory()
|
||||
directory = root / model_directory_name(model_key)
|
||||
old_moge_directory = root / "moge2_base_normal"
|
||||
if (
|
||||
model_key == "MOGE2_VITB_NORMAL"
|
||||
and not directory.is_dir()
|
||||
and old_moge_directory.is_dir()
|
||||
):
|
||||
old_moge_directory.rename(directory)
|
||||
legacy_directory = root / model_key.lower()
|
||||
if (
|
||||
not directory.is_dir()
|
||||
and legacy_directory.is_dir()
|
||||
and legacy_directory != directory
|
||||
):
|
||||
legacy_directory.rename(directory)
|
||||
return directory
|
||||
|
||||
|
||||
def model_ready(model_key):
|
||||
model = get_downloadable_model(model_key)
|
||||
path = model_directory(model_key)
|
||||
return all(
|
||||
any(path.glob(pattern))
|
||||
for pattern in model.ready_patterns
|
||||
)
|
||||
|
||||
|
||||
def env_devices():
|
||||
"""Devices detected by the installed environment, or None when unknown."""
|
||||
ready_path = storage_root() / MANIFEST_FILENAME
|
||||
if not ready_path.is_file():
|
||||
return None
|
||||
try:
|
||||
payload = json.loads(ready_path.read_text(encoding="utf-8"))
|
||||
except (json.JSONDecodeError, OSError):
|
||||
return None
|
||||
devices = payload.get("devices")
|
||||
if not isinstance(devices, list) or not devices:
|
||||
return None
|
||||
return [str(name).lower() for name in devices]
|
||||
|
||||
|
||||
def machine_devices():
|
||||
"""Devices the machine can run before environment installation."""
|
||||
devices = ["cpu"]
|
||||
if shutil.which("nvidia-smi"):
|
||||
devices.insert(0, "cuda")
|
||||
elif sys.platform == "darwin" and platform.machine().lower() in {
|
||||
"arm64",
|
||||
"aarch64",
|
||||
}:
|
||||
devices.insert(0, "mps")
|
||||
return devices
|
||||
|
||||
|
||||
def supported_devices():
|
||||
"""Inference devices available on this machine, fastest first."""
|
||||
return env_devices() or machine_devices()
|
||||
|
||||
|
||||
DEVICE_LABELS = {
|
||||
"cuda": ("CUDA", "NVIDIA CUDA"),
|
||||
"mps": ("MPS", "Apple Metal (MPS)"),
|
||||
"cpu": ("CPU", "CPU"),
|
||||
}
|
||||
|
||||
|
||||
def device_enum_items(_owner, _context):
|
||||
"""Blender EnumProperty items listing only devices supported on this machine."""
|
||||
return [
|
||||
(name.upper(), *DEVICE_LABELS[name], index)
|
||||
for index, name in enumerate(supported_devices())
|
||||
]
|
||||
|
||||
|
||||
def production_device():
|
||||
"""Device configured for production tasks, falling back to Auto."""
|
||||
addon_preferences = preferences()
|
||||
configured = getattr(addon_preferences, "device", "")
|
||||
return configured or "AUTO"
|
||||
|
||||
|
||||
def jobs_directory():
|
||||
path = storage_root() / "jobs"
|
||||
path.mkdir(parents=True, exist_ok=True)
|
||||
return path
|
||||
|
||||
|
||||
def temporary_input_path():
|
||||
directory = Path(tempfile.mkdtemp(prefix="snapform-input-"))
|
||||
return directory / "input.png"
|
||||
|
||||
|
||||
def temporary_input_directory():
|
||||
return Path(tempfile.mkdtemp(prefix="snapform-input-"))
|
||||
|
||||
|
||||
def move_input_into_job(input_path, job_directory):
|
||||
"""Move a Blender-generated input copy into the flat job directory."""
|
||||
input_path = Path(input_path)
|
||||
job_directory = Path(job_directory)
|
||||
if input_path.is_file():
|
||||
destination = job_directory / "input.png"
|
||||
input_path.replace(destination)
|
||||
try:
|
||||
input_path.parent.rmdir()
|
||||
except OSError:
|
||||
pass
|
||||
return destination
|
||||
if not input_path.is_dir():
|
||||
raise ValueError(f"Temporary input does not exist: {input_path}")
|
||||
for source in input_path.iterdir():
|
||||
if not source.is_file():
|
||||
raise ValueError(f"Temporary input contains a directory: {source}")
|
||||
destination = job_directory / source.name
|
||||
if destination.exists():
|
||||
raise ValueError(f"Job input filename already exists: {source.name}")
|
||||
source.replace(destination)
|
||||
input_path.rmdir()
|
||||
return job_directory
|
||||
|
||||
|
||||
def job_folder_name(job_type):
|
||||
"""Return the folder name segment for a job type."""
|
||||
return JOB_FOLDER_NAMES.get(job_type, job_type.lower())
|
||||
|
||||
|
||||
def new_job_paths(job_type):
|
||||
timestamp = datetime.now().strftime("%Y%m%d-%H%M%S-%f")
|
||||
stem = f"{job_folder_name(job_type)}_{timestamp}"
|
||||
directory = jobs_directory() / stem
|
||||
directory.mkdir(parents=True, exist_ok=True)
|
||||
return {
|
||||
"log": directory / "job.log",
|
||||
"status": directory / "status.json",
|
||||
"result": directory,
|
||||
}
|
||||
|
||||
|
||||
def job_id_from_status(status_path):
|
||||
return Path(status_path).parent.name
|
||||
|
||||
|
||||
def uv_executable():
|
||||
"""Return the bundled uv path inside the shared storage."""
|
||||
return str(uv_cache_path())
|
||||
|
||||
|
||||
def uv_cache_path():
|
||||
if os.name == "nt":
|
||||
return storage_root() / "tools" / "uv.exe"
|
||||
return storage_root() / "tools" / "uv"
|
||||
|
||||
|
||||
def install_env_command(status_path):
|
||||
executable = uv_executable()
|
||||
torch_variant = "cu128" if "cuda" in machine_devices() else "cpu"
|
||||
script = Path(__file__).with_name("install_env.py")
|
||||
command = [
|
||||
sys.executable,
|
||||
str(script),
|
||||
"--uv",
|
||||
executable,
|
||||
"--venv-dir",
|
||||
str(venv_directory()),
|
||||
"--manifest",
|
||||
str(storage_root() / MANIFEST_FILENAME),
|
||||
"--models-dir",
|
||||
str(models_directory()),
|
||||
"--torch-variant",
|
||||
torch_variant,
|
||||
"--da3-commit",
|
||||
DA3_COMMIT,
|
||||
"--ben2-commit",
|
||||
BEN2_COMMIT,
|
||||
"--status",
|
||||
str(status_path),
|
||||
]
|
||||
return command
|
||||
|
||||
|
||||
def service_log_path():
|
||||
return storage_root() / "service.log"
|
||||
|
||||
|
||||
def service_discovery_path():
|
||||
return storage_root() / "service.json"
|
||||
|
||||
|
||||
def service_lock_path():
|
||||
return storage_root() / "service.lock"
|
||||
|
||||
|
||||
def _available_service_port():
|
||||
with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as listener:
|
||||
listener.bind((SERVICE_ADDRESS, 0))
|
||||
return listener.getsockname()[1]
|
||||
|
||||
|
||||
def _service_url(connection, path):
|
||||
return f"http://{SERVICE_ADDRESS}:{connection.port}{path}"
|
||||
|
||||
|
||||
def _service_http_request(connection, method, path, payload=None, timeout=2.0):
|
||||
data = None
|
||||
headers = {"X-SnapForm-Token": connection.token}
|
||||
if payload is not None:
|
||||
data = json.dumps(payload).encode("utf-8")
|
||||
headers["Content-Type"] = "application/json"
|
||||
request = Request(
|
||||
_service_url(connection, path),
|
||||
data=data,
|
||||
headers=headers,
|
||||
method=method,
|
||||
)
|
||||
try:
|
||||
with _local_opener.open(request, timeout=timeout) as response:
|
||||
return json.loads(response.read().decode("utf-8"))
|
||||
except HTTPError as error:
|
||||
try:
|
||||
detail = json.loads(error.read().decode("utf-8")).get("detail")
|
||||
except (json.JSONDecodeError, UnicodeDecodeError):
|
||||
detail = None
|
||||
raise RuntimeError(
|
||||
detail or f"Service returned HTTP {error.code}"
|
||||
)
|
||||
except (OSError, URLError) as error:
|
||||
raise RuntimeError(f"Unable to reach Service: {error}")
|
||||
|
||||
|
||||
def _read_service_discovery():
|
||||
try:
|
||||
payload = json.loads(service_discovery_path().read_text(encoding="utf-8"))
|
||||
return Connection(
|
||||
port=int(payload["port"]),
|
||||
token=str(payload["token"]),
|
||||
pid=int(payload["pid"]),
|
||||
instance_id=str(payload["instance_id"]),
|
||||
)
|
||||
except (KeyError, TypeError, ValueError, OSError, json.JSONDecodeError):
|
||||
return None
|
||||
|
||||
|
||||
def _check_service(connection, timeout=0.3):
|
||||
health = _service_http_request(
|
||||
connection,
|
||||
"GET",
|
||||
"/health",
|
||||
timeout=timeout,
|
||||
)
|
||||
if health.get("instance_id") != connection.instance_id:
|
||||
raise RuntimeError("Service discovery is stale")
|
||||
return health
|
||||
|
||||
|
||||
def _discover_live_service(timeout=0.3):
|
||||
connection = _read_service_discovery()
|
||||
if connection is None:
|
||||
return None, None
|
||||
try:
|
||||
return connection, _check_service(connection, timeout=timeout)
|
||||
except RuntimeError:
|
||||
return None, None
|
||||
|
||||
|
||||
def start_service():
|
||||
global _launched_service
|
||||
|
||||
if _launched_service is not None and _launched_service.process.poll() is None:
|
||||
return
|
||||
_close_launched_service()
|
||||
port = _available_service_port()
|
||||
token = secrets.token_urlsafe(32)
|
||||
instance_id = uuid.uuid4().hex
|
||||
log_file = service_log_path().open("a", encoding="utf-8")
|
||||
command = [
|
||||
str(venv_python()),
|
||||
"-u",
|
||||
str(Path(__file__).with_name("service.py")),
|
||||
"--host",
|
||||
SERVICE_ADDRESS,
|
||||
"--port",
|
||||
str(port),
|
||||
"--token",
|
||||
token,
|
||||
"--instance-id",
|
||||
instance_id,
|
||||
"--discovery",
|
||||
str(service_discovery_path()),
|
||||
"--lock",
|
||||
str(service_lock_path()),
|
||||
]
|
||||
try:
|
||||
process = subprocess.Popen(
|
||||
command,
|
||||
stdout=log_file,
|
||||
stderr=subprocess.STDOUT,
|
||||
cwd=storage_root(),
|
||||
creationflags=creation_flags(),
|
||||
)
|
||||
except OSError:
|
||||
log_file.close()
|
||||
raise
|
||||
_launched_service = ServiceProcess(process, log_file)
|
||||
|
||||
|
||||
def ensure_service(timeout=60.0):
|
||||
global _service_auto_start, _service_connection, _service_snapshot
|
||||
|
||||
_service_auto_start = True
|
||||
deadline = time.monotonic() + timeout
|
||||
last_launch = 0.0
|
||||
starting = False
|
||||
while time.monotonic() < deadline:
|
||||
connection, health = _discover_live_service(timeout=0.5)
|
||||
if connection is not None:
|
||||
_service_connection = connection
|
||||
_register_service_client(connection)
|
||||
_service_snapshot = _format_service_snapshot(connection, health)
|
||||
_redraw_service_ui()
|
||||
return connection
|
||||
if not starting:
|
||||
_service_snapshot = {"state": "STARTING", "message": "Starting"}
|
||||
_redraw_service_ui()
|
||||
starting = True
|
||||
_close_finished_launched_service()
|
||||
if (
|
||||
_launched_service is None
|
||||
and time.monotonic() - last_launch >= 1.0
|
||||
):
|
||||
start_service()
|
||||
last_launch = time.monotonic()
|
||||
time.sleep(0.05)
|
||||
raise RuntimeError(
|
||||
f"Service did not become ready; log: {service_log_path()}"
|
||||
)
|
||||
|
||||
|
||||
def submit_service_job(request):
|
||||
connection = _service_connection or ensure_service()
|
||||
payload = dict(request.payload)
|
||||
payload["client_id"] = _client_id
|
||||
_service_http_request(
|
||||
connection,
|
||||
"POST",
|
||||
"/jobs",
|
||||
payload=payload,
|
||||
timeout=5.0,
|
||||
)
|
||||
return connection
|
||||
|
||||
|
||||
def stop_service():
|
||||
global _service_auto_start, _service_connection, _service_snapshot
|
||||
global _registered_instance_id
|
||||
|
||||
connection = _service_connection
|
||||
if connection is None:
|
||||
connection, _health = _discover_live_service(timeout=0.5)
|
||||
if connection is None:
|
||||
_close_launched_service(terminate=True)
|
||||
_service_auto_start = False
|
||||
_service_snapshot = {"state": "STOPPED", "message": "Stopped"}
|
||||
return
|
||||
_register_service_client(connection)
|
||||
_service_http_request(
|
||||
connection,
|
||||
"POST",
|
||||
"/shutdown",
|
||||
payload={"client_id": _client_id},
|
||||
timeout=2.0,
|
||||
)
|
||||
_service_auto_start = False
|
||||
deadline = time.monotonic() + 10.0
|
||||
while time.monotonic() < deadline:
|
||||
try:
|
||||
_check_service(connection, timeout=0.2)
|
||||
except RuntimeError:
|
||||
break
|
||||
time.sleep(0.05)
|
||||
else:
|
||||
raise RuntimeError("Service did not stop")
|
||||
_service_connection = None
|
||||
_registered_instance_id = None
|
||||
_close_finished_launched_service()
|
||||
_service_snapshot = {"state": "STOPPED", "message": "Stopped"}
|
||||
|
||||
|
||||
def restart_service():
|
||||
global _service_auto_start, _service_connection, _registered_instance_id
|
||||
global _service_snapshot
|
||||
|
||||
connection = ensure_service()
|
||||
_service_http_request(
|
||||
connection,
|
||||
"POST",
|
||||
"/shutdown",
|
||||
payload={"client_id": _client_id},
|
||||
timeout=2.0,
|
||||
)
|
||||
_service_auto_start = True
|
||||
_service_connection = None
|
||||
_registered_instance_id = None
|
||||
_service_snapshot = {"state": "STARTING", "message": "Restarting"}
|
||||
|
||||
|
||||
def enable_service():
|
||||
global _service_auto_start, _service_connection, _service_snapshot
|
||||
|
||||
_service_auto_start = True
|
||||
connection, health = _discover_live_service(timeout=0.2)
|
||||
if connection is not None:
|
||||
_service_connection = connection
|
||||
_register_service_client(connection)
|
||||
_service_snapshot = _format_service_snapshot(connection, health)
|
||||
_redraw_service_ui()
|
||||
return
|
||||
start_service()
|
||||
_service_snapshot = {"state": "STARTING", "message": "Starting"}
|
||||
_redraw_service_ui()
|
||||
|
||||
|
||||
def clear_service_models():
|
||||
connection = ensure_service()
|
||||
return _service_http_request(
|
||||
connection,
|
||||
"POST",
|
||||
"/models/clear",
|
||||
timeout=5.0,
|
||||
)
|
||||
|
||||
|
||||
def cancel_service_job(job_id):
|
||||
connection = ensure_service()
|
||||
return _service_http_request(
|
||||
connection,
|
||||
"DELETE",
|
||||
f"/jobs/{job_id}",
|
||||
payload={"client_id": _client_id},
|
||||
timeout=2.0,
|
||||
)
|
||||
|
||||
|
||||
def detach_service_client():
|
||||
global _service_connection, _registered_instance_id
|
||||
|
||||
connection = _service_connection
|
||||
if connection is not None and _registered_instance_id is not None:
|
||||
try:
|
||||
_service_http_request(
|
||||
connection,
|
||||
"DELETE",
|
||||
f"/clients/{_client_id}",
|
||||
timeout=0.5,
|
||||
)
|
||||
except RuntimeError:
|
||||
pass
|
||||
_service_connection = None
|
||||
_registered_instance_id = None
|
||||
# 当前 Blender 启动的 Service 随 Blender 关闭,不等待空闲超时。
|
||||
_close_launched_service(terminate=True)
|
||||
|
||||
|
||||
def schedule_service_start():
|
||||
timers = getattr(bpy.app, "timers", None)
|
||||
if timers is None:
|
||||
return
|
||||
if timers.is_registered(_service_timer):
|
||||
return
|
||||
timers.register(_service_timer, first_interval=1.0, persistent=True)
|
||||
|
||||
|
||||
def unschedule_service():
|
||||
timers = getattr(bpy.app, "timers", None)
|
||||
if timers is None:
|
||||
return
|
||||
if timers.is_registered(_service_timer):
|
||||
timers.unregister(_service_timer)
|
||||
|
||||
|
||||
def service_status():
|
||||
return dict(_service_snapshot)
|
||||
|
||||
|
||||
def service_busy():
|
||||
return _service_snapshot.get("state") == "BUSY"
|
||||
|
||||
|
||||
def mark_service_job_complete(job_id):
|
||||
"""Release a stale local busy snapshot after its terminal status is observed."""
|
||||
global _service_snapshot
|
||||
|
||||
active = _service_snapshot.get("active_job")
|
||||
if active and active.get("job_id") != job_id:
|
||||
return
|
||||
if _service_snapshot.get("state") != "BUSY":
|
||||
return
|
||||
_service_snapshot = {
|
||||
**_service_snapshot,
|
||||
"busy": False,
|
||||
"active_job": None,
|
||||
"state": "READY",
|
||||
"message": "Ready",
|
||||
}
|
||||
|
||||
|
||||
def _service_timer():
|
||||
global _service_connection, _service_snapshot, _registered_instance_id
|
||||
|
||||
if not env_installed():
|
||||
_service_snapshot = {
|
||||
"state": "UNAVAILABLE",
|
||||
"message": "Environment is not installed",
|
||||
}
|
||||
return 2.0
|
||||
connection, health = _discover_live_service(timeout=0.2)
|
||||
if connection is None:
|
||||
_service_connection = None
|
||||
_registered_instance_id = None
|
||||
_close_finished_launched_service()
|
||||
if not _service_auto_start:
|
||||
if _service_snapshot.get("state") != "STOPPED":
|
||||
_service_snapshot = {"state": "STOPPED", "message": "Stopped"}
|
||||
_redraw_service_ui()
|
||||
return 2.0
|
||||
try:
|
||||
start_service()
|
||||
except OSError as error:
|
||||
_service_snapshot = {"state": "ERROR", "message": str(error)}
|
||||
return 2.0
|
||||
_service_snapshot = {"state": "STARTING", "message": "Starting"}
|
||||
_redraw_service_ui()
|
||||
return 1.0
|
||||
try:
|
||||
_service_connection = connection
|
||||
if _registered_instance_id == connection.instance_id:
|
||||
try:
|
||||
_service_http_request(
|
||||
connection,
|
||||
"POST",
|
||||
f"/clients/{_client_id}/heartbeat",
|
||||
timeout=0.5,
|
||||
)
|
||||
except RuntimeError:
|
||||
_registered_instance_id = None
|
||||
_register_service_client(connection)
|
||||
health = _check_service(connection, timeout=0.5)
|
||||
_service_snapshot = _format_service_snapshot(connection, health)
|
||||
except RuntimeError as error:
|
||||
_service_connection = None
|
||||
_registered_instance_id = None
|
||||
_service_snapshot = {"state": "ERROR", "message": str(error)}
|
||||
_redraw_service_ui()
|
||||
return 5.0
|
||||
|
||||
|
||||
def _register_service_client(connection):
|
||||
global _registered_instance_id
|
||||
|
||||
if _registered_instance_id == connection.instance_id:
|
||||
return
|
||||
_service_http_request(
|
||||
connection,
|
||||
"POST",
|
||||
"/clients",
|
||||
payload={"client_id": _client_id, "label": _client_label},
|
||||
timeout=2.0,
|
||||
)
|
||||
_registered_instance_id = connection.instance_id
|
||||
|
||||
|
||||
def _format_service_snapshot(connection, health):
|
||||
state = "BUSY" if health.get("busy") else "READY"
|
||||
return {
|
||||
**health,
|
||||
"state": state,
|
||||
"message": "Busy" if state == "BUSY" else "Ready",
|
||||
"port": connection.port,
|
||||
}
|
||||
|
||||
|
||||
def _redraw_service_ui():
|
||||
if threading.current_thread() is not threading.main_thread():
|
||||
return
|
||||
try:
|
||||
from .status_bar import redraw_ui
|
||||
|
||||
redraw_ui(bpy.context, force=True)
|
||||
except (AttributeError, RuntimeError):
|
||||
pass
|
||||
|
||||
|
||||
def _close_finished_launched_service():
|
||||
if _launched_service is None or _launched_service.process.poll() is None:
|
||||
return
|
||||
_close_launched_service()
|
||||
|
||||
|
||||
def _close_launched_service(terminate=False):
|
||||
global _launched_service
|
||||
|
||||
launched = _launched_service
|
||||
_launched_service = None
|
||||
if launched is None:
|
||||
return
|
||||
if terminate and launched.process.poll() is None:
|
||||
launched.process.terminate()
|
||||
try:
|
||||
launched.process.wait(timeout=3.0)
|
||||
except subprocess.TimeoutExpired:
|
||||
launched.process.kill()
|
||||
launched.process.wait(timeout=3.0)
|
||||
launched.log_file.close()
|
||||
|
||||
|
||||
def download_model_command(model_key, status_path):
|
||||
model = get_downloadable_model(model_key)
|
||||
command = [
|
||||
str(venv_python()),
|
||||
str(Path(__file__).with_name("worker.py")),
|
||||
"download",
|
||||
"--repository",
|
||||
model.repository,
|
||||
"--model-dir",
|
||||
str(model_directory(model_key)),
|
||||
"--status",
|
||||
str(status_path),
|
||||
"--label",
|
||||
model.label,
|
||||
]
|
||||
for pattern in model.download_patterns:
|
||||
command.extend(["--allow-pattern", pattern])
|
||||
for pattern in model.ready_patterns:
|
||||
command.extend(["--required-pattern", pattern])
|
||||
if model.r2_directory:
|
||||
command.extend(["--r2-directory", model.r2_directory])
|
||||
for filename, size, checksum in model.r2_files:
|
||||
command.extend(
|
||||
["--r2-file", f"{filename}|{size}|{checksum}"]
|
||||
)
|
||||
return command
|
||||
|
||||
|
||||
def remove_background_request(
|
||||
settings,
|
||||
input_path,
|
||||
status_path,
|
||||
result_directory,
|
||||
model_key,
|
||||
video_frames=None,
|
||||
start_frame=1,
|
||||
device=None,
|
||||
):
|
||||
return JobRequest(
|
||||
{
|
||||
"job_id": job_id_from_status(status_path),
|
||||
"command": "remove-background",
|
||||
"parameters": {
|
||||
"input": str(input_path),
|
||||
"model_dir": str(model_directory(model_key)),
|
||||
"output_dir": str(result_directory),
|
||||
"status": str(status_path),
|
||||
"device": (device or settings.device).lower(),
|
||||
"video_frames": (
|
||||
settings.video_frames
|
||||
if video_frames is None
|
||||
else video_frames
|
||||
),
|
||||
"start_frame": start_frame,
|
||||
},
|
||||
}
|
||||
)
|
||||
|
||||
|
||||
def extract_frames_request(
|
||||
settings,
|
||||
input_path,
|
||||
status_path,
|
||||
result_directory,
|
||||
video_frames=None,
|
||||
start_frame=1,
|
||||
):
|
||||
return JobRequest(
|
||||
{
|
||||
"job_id": job_id_from_status(status_path),
|
||||
"command": "extract-frames",
|
||||
"parameters": {
|
||||
"input": str(input_path),
|
||||
"output_dir": str(result_directory),
|
||||
"status": str(status_path),
|
||||
"video_frames": (
|
||||
settings.video_frames
|
||||
if video_frames is None
|
||||
else video_frames
|
||||
),
|
||||
"start_frame": start_frame,
|
||||
},
|
||||
}
|
||||
)
|
||||
|
||||
|
||||
def reconstruct_request(
|
||||
settings,
|
||||
export_format,
|
||||
status_path,
|
||||
result_directory,
|
||||
input_path=None,
|
||||
model_key=None,
|
||||
process_res=None,
|
||||
process_res_method=None,
|
||||
use_ray_pose=None,
|
||||
remove_background=False,
|
||||
mesh_filter_edges=None,
|
||||
start_frame=1,
|
||||
device=None,
|
||||
):
|
||||
snapform_selected_key = model_key or selected_model_key(settings, export_format)
|
||||
snapform_selected_process_res = (
|
||||
settings.process_res if process_res is None else process_res
|
||||
)
|
||||
snapform_selected_process_res_method = (
|
||||
settings.process_res_method
|
||||
if process_res_method is None
|
||||
else process_res_method
|
||||
)
|
||||
snapform_selected_ray_pose = (
|
||||
settings.use_ray_pose if use_ray_pose is None else use_ray_pose
|
||||
)
|
||||
parameters = {
|
||||
"input": str(
|
||||
Path(input_path)
|
||||
if input_path is not None
|
||||
else Path(bpy.path.abspath(settings.input_path))
|
||||
),
|
||||
"model_dir": str(model_directory(snapform_selected_key)),
|
||||
"output_dir": str(result_directory),
|
||||
"export_format": export_format,
|
||||
"status": str(status_path),
|
||||
"process_res": snapform_selected_process_res,
|
||||
"process_res_method": snapform_selected_process_res_method,
|
||||
"conf_thresh_percentile": settings.conf_thresh_percentile,
|
||||
"num_max_points": settings.num_max_points,
|
||||
"show_cameras": 1 if settings.show_cameras else 0,
|
||||
"gs_views_interval": settings.gs_views_interval,
|
||||
"video_frames": settings.video_frames,
|
||||
"start_frame": start_frame,
|
||||
"device": (device or settings.device).lower(),
|
||||
"use_ray_pose": 1 if snapform_selected_ray_pose else 0,
|
||||
"ref_view_strategy": settings.ref_view_strategy,
|
||||
"remove_background_model_dir": "",
|
||||
"mesh_filter_edges": bool(
|
||||
getattr(settings, "mesh_filter_edges", True)
|
||||
if mesh_filter_edges is None
|
||||
else mesh_filter_edges
|
||||
),
|
||||
"mesh_max_edge_length": float(
|
||||
getattr(settings, "mesh_max_edge_length", 0.1)
|
||||
),
|
||||
}
|
||||
if remove_background:
|
||||
parameters["remove_background_model_dir"] = str(
|
||||
model_directory(BEN2_MODEL.key)
|
||||
)
|
||||
return JobRequest(
|
||||
{
|
||||
"job_id": job_id_from_status(status_path),
|
||||
"command": "reconstruct",
|
||||
"parameters": parameters,
|
||||
}
|
||||
)
|
||||
|
||||
|
||||
def moge_request(
|
||||
settings,
|
||||
status_path,
|
||||
result_directory,
|
||||
input_path=None,
|
||||
resolution_level=None,
|
||||
include_points=False,
|
||||
remove_background=False,
|
||||
start_frame=1,
|
||||
device=None,
|
||||
model_key=None,
|
||||
):
|
||||
from .models import BEN2_MODEL, MOGE2_MODEL
|
||||
|
||||
selected_model_key = model_key or MOGE2_MODEL.key
|
||||
|
||||
parameters = {
|
||||
"input": str(
|
||||
Path(input_path)
|
||||
if input_path is not None
|
||||
else Path(bpy.path.abspath(settings.input_path))
|
||||
),
|
||||
"model_dir": str(model_directory(selected_model_key)),
|
||||
"output_dir": str(result_directory),
|
||||
"status": str(status_path),
|
||||
"resolution_level": int(
|
||||
settings.moge_resolution_level
|
||||
if resolution_level is None
|
||||
else resolution_level
|
||||
),
|
||||
"include_points": bool(include_points),
|
||||
"video_frames": settings.video_frames,
|
||||
"start_frame": start_frame,
|
||||
"device": (device or settings.device).lower(),
|
||||
"remove_background_model_dir": (
|
||||
str(model_directory(BEN2_MODEL.key))
|
||||
if remove_background
|
||||
else ""
|
||||
),
|
||||
}
|
||||
return JobRequest(
|
||||
{
|
||||
"job_id": job_id_from_status(status_path),
|
||||
"command": "moge",
|
||||
"parameters": parameters,
|
||||
}
|
||||
)
|
||||
|
||||
|
||||
def creation_flags():
|
||||
if os.name != "nt":
|
||||
return 0
|
||||
return subprocess.CREATE_NO_WINDOW
|
||||
|
||||
|
||||
def read_status(path):
|
||||
try:
|
||||
return json.loads(Path(path).read_text(encoding="utf-8"))
|
||||
except (FileNotFoundError, json.JSONDecodeError, OSError):
|
||||
return None
|
||||
@@ -1,979 +0,0 @@
|
||||
import argparse
|
||||
import filecmp
|
||||
import json
|
||||
import os
|
||||
import platform
|
||||
import queue
|
||||
import shutil
|
||||
import subprocess
|
||||
import sys
|
||||
import tarfile
|
||||
import tempfile
|
||||
import threading
|
||||
import time
|
||||
import urllib.request
|
||||
import zipfile
|
||||
from pathlib import Path
|
||||
|
||||
|
||||
SCRIPT_DIRECTORY = Path(__file__).resolve().parent
|
||||
if str(SCRIPT_DIRECTORY) not in sys.path:
|
||||
sys.path.insert(0, str(SCRIPT_DIRECTORY))
|
||||
|
||||
from models import BEN2_MODEL, DOWNLOADABLE_MODELS, MOGE2_MODEL
|
||||
|
||||
|
||||
PYPI_MIRRORS = (
|
||||
"https://pypi.tuna.tsinghua.edu.cn/simple",
|
||||
"https://mirrors.aliyun.com/pypi/simple/",
|
||||
)
|
||||
PYPI_DEFAULT_INDEX = "https://pypi.org/simple/"
|
||||
TORCH_VERSION = "2.9.1"
|
||||
TORCHVISION_VERSION = "0.24.1"
|
||||
XFORMERS_VERSION = "0.0.33.post2"
|
||||
ADDICT_VERSION = "2.4.0"
|
||||
FASTAPI_VERSION = "0.139.2"
|
||||
UVICORN_VERSION = "0.51.0"
|
||||
MANIFEST_REVISION = 5
|
||||
TORCH_INDEXES = {
|
||||
"cu128": "https://download.pytorch.org/whl/cu128",
|
||||
"cpu": "https://download.pytorch.org/whl/cpu",
|
||||
}
|
||||
_resolved_pypi_index = None
|
||||
DEFAULT_MODEL_KEYS = (MOGE2_MODEL.key, BEN2_MODEL.key)
|
||||
RUNTIME_STAGE_COUNT = 3
|
||||
ENVIRONMENT_SLICE = (0.02, 0.06)
|
||||
PYTORCH_SLICE = (0.06, 0.48)
|
||||
XFORMERS_SLICE = (0.48, 0.62)
|
||||
AI_COMPONENTS_SLICE = (0.62, 0.85)
|
||||
MODELS_SLICE_START = 0.85
|
||||
MODELS_SLICE_END = 1.0
|
||||
RAMP_RATE = 0.02
|
||||
CREEP_CAP = 0.60
|
||||
INSTALL_ESTIMATES = {
|
||||
"PyTorch": 8 * 60,
|
||||
"AI components": 8 * 60,
|
||||
}
|
||||
INSTALL_ESTIMATE_RANGES = {
|
||||
"PyTorch": PYTORCH_SLICE,
|
||||
"AI components": (XFORMERS_SLICE[0], AI_COMPONENTS_SLICE[1]),
|
||||
}
|
||||
CHECK_CODE = (
|
||||
"import json; import torch; "
|
||||
"from depth_anything_3.api import DepthAnything3; "
|
||||
"from ben2 import BEN_Base; "
|
||||
"from moge.model.v2 import MoGeModel; "
|
||||
"import fastapi, uvicorn; "
|
||||
"mps = getattr(torch.backends, 'mps', None); "
|
||||
"print('SNAPFORM_DEVICES=' + json.dumps({"
|
||||
"'torch': torch.__version__, "
|
||||
"'cuda': torch.cuda.is_available(), "
|
||||
"'mps': bool(mps is not None and mps.is_available())}))"
|
||||
)
|
||||
|
||||
|
||||
def configure_standard_streams():
|
||||
"""Windows 图形进程可能继承 GBK stdout,安装日志统一写为 UTF-8。"""
|
||||
for stream in (sys.stdout, sys.stderr):
|
||||
reconfigure = getattr(stream, "reconfigure", None)
|
||||
if callable(reconfigure):
|
||||
reconfigure(encoding="utf-8", errors="backslashreplace")
|
||||
|
||||
|
||||
def write_status(
|
||||
path,
|
||||
progress,
|
||||
message,
|
||||
error=None,
|
||||
stage=None,
|
||||
stages=None,
|
||||
stage_label=None,
|
||||
eta_seconds=None,
|
||||
):
|
||||
payload = {"progress": progress, "message": message}
|
||||
if error:
|
||||
payload["error"] = error
|
||||
if stage is not None:
|
||||
payload["stage"] = stage
|
||||
payload["stages"] = stages
|
||||
payload["stage_label"] = stage_label
|
||||
if eta_seconds is not None:
|
||||
payload["eta_seconds"] = max(int(eta_seconds), 0)
|
||||
Path(path).write_text(
|
||||
json.dumps(payload, ensure_ascii=False),
|
||||
encoding="utf-8",
|
||||
)
|
||||
|
||||
|
||||
def child_environment(environment=None):
|
||||
"""Inherit the current environment and inject system/registry proxy settings into child processes (uv only reads proxy env variables)."""
|
||||
process_environment = os.environ.copy()
|
||||
process_environment["PYTHONIOENCODING"] = "utf-8:backslashreplace"
|
||||
process_environment["PYTHONUTF8"] = "1"
|
||||
for key, value in urllib.request.getproxies().items():
|
||||
env_key = f"{key.upper()}_PROXY"
|
||||
if env_key not in process_environment:
|
||||
process_environment[env_key] = value
|
||||
if _resolved_pypi_index:
|
||||
process_environment["PIP_INDEX_URL"] = _resolved_pypi_index
|
||||
if environment:
|
||||
process_environment.update(environment)
|
||||
return process_environment
|
||||
|
||||
|
||||
def run(command, environment=None):
|
||||
print("Running:", subprocess.list2cmdline([str(value) for value in command]))
|
||||
subprocess.check_call(
|
||||
[str(value) for value in command],
|
||||
env=child_environment(environment),
|
||||
)
|
||||
|
||||
|
||||
def format_elapsed(seconds):
|
||||
seconds = int(seconds)
|
||||
if seconds < 60:
|
||||
return f"{seconds}s"
|
||||
return f"{seconds // 60}m {seconds % 60:02d}s"
|
||||
|
||||
|
||||
def parse_package_count(line):
|
||||
try:
|
||||
return int(line.split()[1])
|
||||
except (IndexError, ValueError):
|
||||
return 0
|
||||
|
||||
|
||||
def estimated_remaining(label, progress, elapsed=0.0):
|
||||
expected_seconds = INSTALL_ESTIMATES.get(label)
|
||||
progress_range = INSTALL_ESTIMATE_RANGES.get(label)
|
||||
if expected_seconds is None or progress_range is None:
|
||||
return None
|
||||
start, end = progress_range
|
||||
if end <= start:
|
||||
return 0.0
|
||||
remaining_fraction = 1.0 - min(
|
||||
max((progress - start) / (end - start), 0.0),
|
||||
1.0,
|
||||
)
|
||||
return max(expected_seconds * remaining_fraction - elapsed, 0.0)
|
||||
|
||||
|
||||
def effective_torch_index(torch_variant):
|
||||
return os.environ.get("SNAPFORM_TORCH_INDEX") or TORCH_INDEXES.get(torch_variant)
|
||||
|
||||
|
||||
def resolve_indexes(args):
|
||||
"""???????????????????????????????????"""
|
||||
global _resolved_pypi_index
|
||||
_resolved_pypi_index = (
|
||||
getattr(args, "pypi_index", None)
|
||||
or os.environ.get("PIP_INDEX_URL")
|
||||
)
|
||||
|
||||
|
||||
def _run_install_once(
|
||||
status_path,
|
||||
stage,
|
||||
stages,
|
||||
label,
|
||||
command,
|
||||
environment,
|
||||
start,
|
||||
end,
|
||||
clock,
|
||||
):
|
||||
"""Run a uv install command once and map package-count events onto the stage progress range."""
|
||||
print("Running:", subprocess.list2cmdline([str(value) for value in command]))
|
||||
process_environment = child_environment(environment)
|
||||
process_environment["UV_LOG"] = "debug"
|
||||
process = subprocess.Popen(
|
||||
[str(value) for value in command],
|
||||
env=process_environment,
|
||||
stdout=subprocess.PIPE,
|
||||
stderr=subprocess.STDOUT,
|
||||
text=True,
|
||||
encoding="utf-8",
|
||||
errors="replace",
|
||||
)
|
||||
output_queue = queue.Queue()
|
||||
|
||||
def read_output():
|
||||
for raw_line in process.stdout:
|
||||
output_queue.put(raw_line)
|
||||
output_queue.put(None)
|
||||
|
||||
threading.Thread(target=read_output, daemon=True).start()
|
||||
|
||||
def eased(fraction):
|
||||
return start + min(max(fraction, 0.0), 1.0) * (end - start)
|
||||
|
||||
installed = []
|
||||
total = 0
|
||||
displayed = start
|
||||
target = start
|
||||
started = clock()
|
||||
last_tick = started
|
||||
last_heartbeat = 0.0
|
||||
def detail(message):
|
||||
write_status(
|
||||
status_path,
|
||||
displayed,
|
||||
message,
|
||||
stage=stage,
|
||||
stages=stages,
|
||||
stage_label=label,
|
||||
eta_seconds=estimated_remaining(
|
||||
label,
|
||||
displayed,
|
||||
clock() - started,
|
||||
),
|
||||
)
|
||||
|
||||
while True:
|
||||
try:
|
||||
raw_line = output_queue.get(timeout=0.2)
|
||||
empty = False
|
||||
except queue.Empty:
|
||||
raw_line = None
|
||||
empty = True
|
||||
now = clock()
|
||||
elapsed = now - started
|
||||
dt = now - last_tick
|
||||
last_tick = now
|
||||
if total:
|
||||
effective = max(target, eased(CREEP_CAP))
|
||||
if effective > displayed:
|
||||
displayed = min(effective, displayed + dt * RAMP_RATE)
|
||||
if raw_line is None and not empty:
|
||||
break
|
||||
if empty:
|
||||
if elapsed - last_heartbeat >= 5.0:
|
||||
last_heartbeat = elapsed
|
||||
detail("Downloading packages")
|
||||
continue
|
||||
line = raw_line.rstrip()
|
||||
print(line)
|
||||
if line.startswith("Resolved ") and "package" in line:
|
||||
total = parse_package_count(line)
|
||||
resolved_progress = eased(0.02)
|
||||
displayed = max(displayed, resolved_progress)
|
||||
target = max(target, resolved_progress)
|
||||
detail(f"Resolving {total} packages")
|
||||
elif line.startswith("Prepared ") and "package" in line:
|
||||
done = parse_package_count(line)
|
||||
if total:
|
||||
target = max(target, eased(0.10 + 0.60 * done / total))
|
||||
detail(f"Downloading packages {done}/{total}")
|
||||
elif line.startswith("Installed ") and "package" in line:
|
||||
done = parse_package_count(line)
|
||||
if total:
|
||||
target = max(target, eased(0.75 + 0.25 * done / total))
|
||||
detail(f"Installing packages {done}/{total}")
|
||||
elif line.startswith("+"):
|
||||
entry = line[1:].strip()
|
||||
installed.append(entry)
|
||||
return_code = process.wait()
|
||||
if return_code != 0:
|
||||
raise subprocess.CalledProcessError(return_code, command)
|
||||
return installed
|
||||
|
||||
|
||||
def run_install(
|
||||
status_path,
|
||||
stage,
|
||||
stages,
|
||||
label,
|
||||
command,
|
||||
environment=None,
|
||||
start=0.0,
|
||||
end=1.0,
|
||||
clock=time.monotonic,
|
||||
):
|
||||
"""Run a uv install command, retrying with mirrors when the default index fails."""
|
||||
attempts = [environment]
|
||||
if _resolved_pypi_index in (None, PYPI_DEFAULT_INDEX):
|
||||
attempts.extend(
|
||||
{"PIP_INDEX_URL": mirror} for mirror in PYPI_MIRRORS
|
||||
)
|
||||
for index, attempt_environment in enumerate(attempts):
|
||||
try:
|
||||
return _run_install_once(
|
||||
status_path,
|
||||
stage,
|
||||
stages,
|
||||
label,
|
||||
command,
|
||||
attempt_environment,
|
||||
start,
|
||||
end,
|
||||
clock,
|
||||
)
|
||||
except subprocess.CalledProcessError:
|
||||
if index + 1 == len(attempts):
|
||||
raise
|
||||
print(f"Retrying with PyPI mirror: {PYPI_MIRRORS[0]}")
|
||||
|
||||
|
||||
def venv_python(venv_dir):
|
||||
if sys.platform == "win32":
|
||||
return Path(venv_dir) / "Scripts" / "python.exe"
|
||||
return Path(venv_dir) / "bin" / "python"
|
||||
|
||||
|
||||
def install_stages(args, venv_dir, python, uv, vendor_dir=None):
|
||||
# macOS 的 PyTorch wheel 只在 PyPI 上,download.pytorch.org 没有 macosx 构建
|
||||
torch_index = (
|
||||
None
|
||||
if sys.platform == "darwin"
|
||||
else effective_torch_index(args.torch_variant)
|
||||
)
|
||||
vendor_dir = Path(vendor_dir or Path(venv_dir).parent / "vendor")
|
||||
|
||||
def archive_requirement(project_name, filename):
|
||||
local = Path(vendor_dir) / filename
|
||||
if not local.is_file():
|
||||
raise RuntimeError(
|
||||
"Bundled vendor artifact is missing; reinstall the extension: "
|
||||
f"{local}"
|
||||
)
|
||||
return f"{project_name} @ {local.as_uri()}"
|
||||
|
||||
da3_requirement = archive_requirement(
|
||||
"depth-anything-3",
|
||||
"depth-anything-3.tar.gz",
|
||||
)
|
||||
ben2_requirement = archive_requirement(
|
||||
"ben2",
|
||||
"ben2.tar.gz",
|
||||
)
|
||||
utils3d_requirement = archive_requirement(
|
||||
"utils3d",
|
||||
"utils3d.tar.gz",
|
||||
)
|
||||
moge_requirement = archive_requirement(
|
||||
"moge",
|
||||
"moge.tar.gz",
|
||||
)
|
||||
torch_command = [
|
||||
uv,
|
||||
"pip",
|
||||
"install",
|
||||
"-v",
|
||||
"--python",
|
||||
python,
|
||||
f"torch=={TORCH_VERSION}",
|
||||
f"torchvision=={TORCHVISION_VERSION}",
|
||||
]
|
||||
if torch_index is not None:
|
||||
torch_command.extend(["--index-url", torch_index])
|
||||
stages = [
|
||||
(
|
||||
"Environment",
|
||||
[uv, "venv", "--clear", "--python", "3.12", venv_dir],
|
||||
None,
|
||||
"plain",
|
||||
),
|
||||
(
|
||||
"PyTorch",
|
||||
torch_command,
|
||||
None,
|
||||
"install",
|
||||
),
|
||||
(
|
||||
"AI components",
|
||||
[
|
||||
uv,
|
||||
"pip",
|
||||
"install",
|
||||
"-v",
|
||||
"--python",
|
||||
python,
|
||||
da3_requirement,
|
||||
f"addict=={ADDICT_VERSION}",
|
||||
ben2_requirement,
|
||||
utils3d_requirement,
|
||||
moge_requirement,
|
||||
f"fastapi=={FASTAPI_VERSION}",
|
||||
f"uvicorn=={UVICORN_VERSION}",
|
||||
"setuptools",
|
||||
"wheel",
|
||||
],
|
||||
None,
|
||||
"install",
|
||||
),
|
||||
]
|
||||
if sys.platform != "darwin":
|
||||
xformers_command = [
|
||||
uv,
|
||||
"pip",
|
||||
"install",
|
||||
"-v",
|
||||
"--python",
|
||||
python,
|
||||
f"xformers=={XFORMERS_VERSION}",
|
||||
]
|
||||
if args.torch_variant == "cu128":
|
||||
xformers_command.extend(["--index-url", torch_index])
|
||||
else:
|
||||
xformers_command.append("--no-deps")
|
||||
stages.insert(
|
||||
2,
|
||||
("AI components", xformers_command, None, "install"),
|
||||
)
|
||||
models = DOWNLOADABLE_MODELS
|
||||
worker_script = Path(__file__).with_name("worker.py")
|
||||
models_dir = Path(args.models_dir)
|
||||
for model_key in DEFAULT_MODEL_KEYS:
|
||||
model = models[model_key]
|
||||
download_command = [
|
||||
python,
|
||||
worker_script,
|
||||
"download",
|
||||
"--repository",
|
||||
model.repository,
|
||||
"--model-dir",
|
||||
models_dir / model.directory_name,
|
||||
"--status",
|
||||
args.status,
|
||||
"--label",
|
||||
model.label,
|
||||
]
|
||||
for pattern in model.download_patterns:
|
||||
download_command.extend(["--allow-pattern", pattern])
|
||||
for pattern in model.ready_patterns:
|
||||
download_command.extend(["--required-pattern", pattern])
|
||||
if model.r2_directory:
|
||||
download_command.extend(
|
||||
["--r2-directory", model.r2_directory]
|
||||
)
|
||||
for filename, size, checksum in model.r2_files:
|
||||
download_command.extend(
|
||||
["--r2-file", f"{filename}|{size}|{checksum}"]
|
||||
)
|
||||
stages.append(
|
||||
(
|
||||
model.label,
|
||||
download_command,
|
||||
None,
|
||||
"download",
|
||||
)
|
||||
)
|
||||
return stages
|
||||
|
||||
|
||||
def env_installed(venv_dir, manifest_path=None):
|
||||
ready_path = Path(manifest_path or Path(venv_dir).parent / "manifest.json")
|
||||
if not venv_python(venv_dir).is_file() or not ready_path.is_file():
|
||||
return False
|
||||
try:
|
||||
payload = json.loads(ready_path.read_text(encoding="utf-8"))
|
||||
except (json.JSONDecodeError, OSError):
|
||||
return False
|
||||
return (
|
||||
isinstance(payload, dict)
|
||||
and payload.get("manifest_revision") == MANIFEST_REVISION
|
||||
)
|
||||
|
||||
|
||||
def installed_devices(args):
|
||||
devices = []
|
||||
machine = platform.machine().lower()
|
||||
if sys.platform == "darwin" and machine in {"arm64", "aarch64"}:
|
||||
devices.append("mps")
|
||||
elif args.torch_variant.startswith("cu"):
|
||||
devices.append("cuda")
|
||||
devices.append("cpu")
|
||||
return devices
|
||||
|
||||
|
||||
def verify_command(python):
|
||||
return [python, "-c", CHECK_CODE]
|
||||
|
||||
|
||||
def run_verify(
|
||||
python,
|
||||
status_path,
|
||||
start=0.0,
|
||||
end=1.0,
|
||||
clock=time.monotonic,
|
||||
stage=None,
|
||||
stages=None,
|
||||
stage_label=None,
|
||||
):
|
||||
"""Run the environment import check with heartbeat progress; return the device report."""
|
||||
command = verify_command(python)
|
||||
print("Running:", subprocess.list2cmdline([str(value) for value in command]))
|
||||
process = subprocess.Popen(
|
||||
[str(value) for value in command],
|
||||
stdout=subprocess.PIPE,
|
||||
stderr=subprocess.STDOUT,
|
||||
text=True,
|
||||
encoding="utf-8",
|
||||
errors="replace",
|
||||
)
|
||||
output_queue = queue.Queue()
|
||||
|
||||
def read_output():
|
||||
for raw_line in process.stdout:
|
||||
output_queue.put(raw_line)
|
||||
output_queue.put(None)
|
||||
|
||||
threading.Thread(target=read_output, daemon=True).start()
|
||||
|
||||
report = {}
|
||||
started = clock()
|
||||
last_heartbeat = 0.0
|
||||
while True:
|
||||
try:
|
||||
raw_line = output_queue.get(timeout=0.2)
|
||||
empty = False
|
||||
except queue.Empty:
|
||||
raw_line = None
|
||||
empty = True
|
||||
if raw_line is None and not empty:
|
||||
break
|
||||
if empty:
|
||||
elapsed = clock() - started
|
||||
if elapsed - last_heartbeat >= 5.0:
|
||||
last_heartbeat = elapsed
|
||||
creep = min(elapsed / 60.0, CREEP_CAP)
|
||||
progress = start + creep * (end - start)
|
||||
write_status(
|
||||
status_path,
|
||||
progress,
|
||||
"Checking Environment",
|
||||
stage=stage,
|
||||
stages=stages,
|
||||
stage_label=stage_label,
|
||||
eta_seconds=max(60.0 - elapsed, 0.0),
|
||||
)
|
||||
continue
|
||||
line = raw_line.rstrip()
|
||||
print(line)
|
||||
if line.startswith("SNAPFORM_DEVICES="):
|
||||
try:
|
||||
report = json.loads(line[len("SNAPFORM_DEVICES=") :])
|
||||
except json.JSONDecodeError:
|
||||
report = {}
|
||||
return_code = process.wait()
|
||||
if return_code != 0:
|
||||
raise subprocess.CalledProcessError(return_code, command)
|
||||
return report
|
||||
|
||||
|
||||
def uv_download_target():
|
||||
machine = platform.machine().lower()
|
||||
if sys.platform == "win32":
|
||||
if machine in {"arm64", "aarch64"}:
|
||||
return "aarch64-pc-windows-msvc", "zip"
|
||||
return "x86_64-pc-windows-msvc", "zip"
|
||||
if sys.platform == "darwin":
|
||||
arch = "aarch64" if machine in {"arm64", "aarch64"} else "x86_64"
|
||||
return f"{arch}-apple-darwin", "tar.gz"
|
||||
if sys.platform.startswith("linux"):
|
||||
arch = "aarch64" if machine in {"arm64", "aarch64"} else "x86_64"
|
||||
return f"{arch}-unknown-linux-gnu", "tar.gz"
|
||||
raise RuntimeError(f"Unsupported platform for uv download: {sys.platform}")
|
||||
|
||||
|
||||
def bundled_vendor_directory():
|
||||
packaged = Path(__file__).with_name("vendor")
|
||||
if packaged.is_dir():
|
||||
return packaged
|
||||
project_root = Path(__file__).resolve().parents[2]
|
||||
if (project_root / "pyproject.toml").is_file():
|
||||
return project_root / "vendor"
|
||||
return packaged
|
||||
|
||||
|
||||
def bundled_uv_archive():
|
||||
target, archive = uv_download_target()
|
||||
path = bundled_vendor_directory() / f"uv-{target}.{archive}"
|
||||
return path if path.is_file() else None
|
||||
|
||||
|
||||
def vendor_overlay_directory(filename):
|
||||
names = {
|
||||
"depth-anything-3.tar.gz": "depth-anything-3",
|
||||
"moge.tar.gz": "moge",
|
||||
"utils3d.tar.gz": "utils3d",
|
||||
}
|
||||
overlay_name = names.get(filename)
|
||||
if overlay_name is None:
|
||||
return None
|
||||
project_root = Path(__file__).resolve().parents[2]
|
||||
overlay = project_root / "patches" / overlay_name
|
||||
return overlay if overlay.is_dir() else None
|
||||
|
||||
|
||||
def copy_archive_with_overlay(source, target, overlay):
|
||||
"""开发目录使用原始 vendor 时,应用与正式打包相同的源码覆盖。"""
|
||||
with tarfile.open(source, "r:gz") as input_archive:
|
||||
members = input_archive.getmembers()
|
||||
top_level = members[0].name.split("/", 1)[0]
|
||||
with tarfile.open(target, "w:gz") as output_archive:
|
||||
for member in members:
|
||||
if member.isdir() or member.islnk() or member.issym():
|
||||
output_archive.addfile(member)
|
||||
continue
|
||||
relative = member.name[len(top_level) + 1 :]
|
||||
replacement = overlay / relative
|
||||
if replacement.is_file():
|
||||
info = tarfile.TarInfo(member.name)
|
||||
info.size = replacement.stat().st_size
|
||||
info.mode = member.mode
|
||||
with replacement.open("rb") as stream:
|
||||
output_archive.addfile(info, stream)
|
||||
continue
|
||||
with input_archive.extractfile(member) as stream:
|
||||
output_archive.addfile(member, stream)
|
||||
|
||||
|
||||
def copy_vendor_inputs(bundled_dir, vendor_dir):
|
||||
"""Copy bundled installation inputs into a temporary directory."""
|
||||
for name in (
|
||||
"depth-anything-3.tar.gz",
|
||||
"ben2.tar.gz",
|
||||
"utils3d.tar.gz",
|
||||
"moge.tar.gz",
|
||||
):
|
||||
bundled = Path(bundled_dir) / name
|
||||
if not bundled.is_file():
|
||||
continue
|
||||
target = Path(vendor_dir) / name
|
||||
overlay = vendor_overlay_directory(name)
|
||||
if overlay is None and target.is_file() and filecmp.cmp(
|
||||
bundled,
|
||||
target,
|
||||
shallow=False,
|
||||
):
|
||||
continue
|
||||
Path(vendor_dir).mkdir(parents=True, exist_ok=True)
|
||||
if overlay is not None and tarfile.is_tarfile(bundled):
|
||||
copy_archive_with_overlay(bundled, target, overlay)
|
||||
else:
|
||||
shutil.copy2(bundled, target)
|
||||
|
||||
|
||||
def remove_legacy_vendor(storage_root):
|
||||
"""Remove installation inputs persisted by older development builds."""
|
||||
root = Path(storage_root)
|
||||
shutil.rmtree(root / "vendor", ignore_errors=True)
|
||||
for name in (
|
||||
"depth-anything-3.tar.gz",
|
||||
"ben2.tar.gz",
|
||||
"utils3d.tar.gz",
|
||||
"moge.tar.gz",
|
||||
):
|
||||
(root / name).unlink(missing_ok=True)
|
||||
|
||||
|
||||
def archive_is_valid(path):
|
||||
if zipfile.is_zipfile(path):
|
||||
return True
|
||||
try:
|
||||
with tarfile.open(path) as archive:
|
||||
archive.getmembers()
|
||||
return True
|
||||
except (OSError, tarfile.TarError):
|
||||
return False
|
||||
|
||||
|
||||
def extract_uv(archive_path, uv_path):
|
||||
executable_name = "uv.exe" if sys.platform == "win32" else "uv"
|
||||
with tempfile.TemporaryDirectory() as directory:
|
||||
if zipfile.is_zipfile(archive_path):
|
||||
with zipfile.ZipFile(archive_path) as archive:
|
||||
archive.extractall(directory)
|
||||
else:
|
||||
with tarfile.open(archive_path) as archive:
|
||||
archive.extractall(directory)
|
||||
source = next(Path(directory).rglob(executable_name))
|
||||
shutil.copy2(source, uv_path)
|
||||
if sys.platform != "win32":
|
||||
os.chmod(uv_path, 0o755)
|
||||
|
||||
|
||||
def install_uv(
|
||||
uv,
|
||||
status_path,
|
||||
stage=None,
|
||||
stages=None,
|
||||
stage_label=None,
|
||||
start=0.0,
|
||||
end=1.0,
|
||||
):
|
||||
"""Extract the bundled uv binary; fail clearly when it is missing."""
|
||||
stage_fields = {
|
||||
"stage": stage,
|
||||
"stages": stages,
|
||||
"stage_label": stage_label,
|
||||
}
|
||||
bundled_uv = bundled_uv_archive()
|
||||
if bundled_uv is None:
|
||||
raise RuntimeError(
|
||||
"Bundled uv is missing; reinstall the extension matching your "
|
||||
f"platform (searched {bundled_vendor_directory()})"
|
||||
)
|
||||
if not archive_is_valid(bundled_uv):
|
||||
raise RuntimeError(f"Bundled uv archive is invalid: {bundled_uv}")
|
||||
print(f"Extracting bundled uv from {bundled_uv}")
|
||||
write_status(
|
||||
status_path,
|
||||
start,
|
||||
"extracting bundled uv",
|
||||
**stage_fields,
|
||||
)
|
||||
Path(uv).parent.mkdir(parents=True, exist_ok=True)
|
||||
extract_uv(bundled_uv, uv)
|
||||
|
||||
|
||||
def run_stage(
|
||||
status_path,
|
||||
stage,
|
||||
stages,
|
||||
label,
|
||||
command,
|
||||
environment,
|
||||
kind,
|
||||
start=0.0,
|
||||
end=1.0,
|
||||
):
|
||||
if stage is None:
|
||||
opening_message = f"Downloading {label}"
|
||||
elif kind == "install":
|
||||
opening_message = "Preparing packages"
|
||||
else:
|
||||
opening_message = ""
|
||||
write_status(
|
||||
status_path,
|
||||
start,
|
||||
opening_message,
|
||||
stage=stage,
|
||||
stages=stages,
|
||||
stage_label=label,
|
||||
eta_seconds=estimated_remaining(label, start),
|
||||
)
|
||||
installed = []
|
||||
if kind == "install":
|
||||
installed = run_install(
|
||||
status_path,
|
||||
stage,
|
||||
stages,
|
||||
label,
|
||||
command,
|
||||
environment,
|
||||
start=start,
|
||||
end=end,
|
||||
)
|
||||
elif kind == "download":
|
||||
progress_command = [
|
||||
*command,
|
||||
"--progress-start",
|
||||
str(start),
|
||||
"--progress-end",
|
||||
str(end),
|
||||
]
|
||||
if stage is not None:
|
||||
progress_command.extend(
|
||||
[
|
||||
"--stage",
|
||||
str(stage),
|
||||
"--stages",
|
||||
str(stages),
|
||||
"--stage-label",
|
||||
label,
|
||||
]
|
||||
)
|
||||
run(progress_command, environment)
|
||||
else:
|
||||
run(command, environment)
|
||||
write_status(
|
||||
status_path,
|
||||
end,
|
||||
f"{label} ready" if stage is None else "",
|
||||
stage=stage,
|
||||
stages=stages,
|
||||
stage_label=label,
|
||||
)
|
||||
return installed
|
||||
|
||||
|
||||
def install(args):
|
||||
if sys.platform == "darwin" and platform.machine().lower() not in {
|
||||
"arm64",
|
||||
"aarch64",
|
||||
}:
|
||||
raise RuntimeError(
|
||||
"SnapForm requires Apple Silicon macOS; PyTorch stopped "
|
||||
"shipping Intel Mac (x86_64) wheels after 2.2.2"
|
||||
)
|
||||
resolve_indexes(args)
|
||||
venv_dir = Path(args.venv_dir)
|
||||
manifest_path = Path(
|
||||
getattr(args, "manifest", venv_dir.parent / "manifest.json")
|
||||
)
|
||||
venv_dir.parent.mkdir(parents=True, exist_ok=True)
|
||||
if not env_installed(venv_dir, manifest_path):
|
||||
shutil.rmtree(venv_dir, ignore_errors=True)
|
||||
manifest_path.unlink(missing_ok=True)
|
||||
uv = Path(args.uv)
|
||||
if not uv.is_file():
|
||||
resolved = shutil.which(args.uv)
|
||||
uv = Path(resolved) if resolved else uv
|
||||
uv_missing = not uv.is_file()
|
||||
python = venv_python(venv_dir)
|
||||
vendor_temp = tempfile.TemporaryDirectory(prefix="snapform_vendor_")
|
||||
vendor_dir = Path(vendor_temp.name)
|
||||
copy_vendor_inputs(
|
||||
bundled_vendor_directory(),
|
||||
vendor_dir,
|
||||
)
|
||||
stages = install_stages(args, venv_dir, python, uv, vendor_dir)
|
||||
env_stages = [stage for stage in stages if stage[3] != "download"]
|
||||
model_stages = [stage for stage in stages if stage[3] == "download"]
|
||||
model_span = (
|
||||
(MODELS_SLICE_END - MODELS_SLICE_START) / len(model_stages)
|
||||
if model_stages
|
||||
else 0.0
|
||||
)
|
||||
installed_packages = []
|
||||
component_stages = env_stages[2:]
|
||||
component_ranges = (
|
||||
[XFORMERS_SLICE, AI_COMPONENTS_SLICE]
|
||||
if len(component_stages) == 2
|
||||
else [(XFORMERS_SLICE[0], AI_COMPONENTS_SLICE[1])]
|
||||
)
|
||||
for index, (label, command, environment, kind) in enumerate(env_stages):
|
||||
if index == 0:
|
||||
stage = 1
|
||||
stage_start, stage_end = ENVIRONMENT_SLICE
|
||||
elif index == 1:
|
||||
stage = 2
|
||||
stage_start, stage_end = PYTORCH_SLICE
|
||||
else:
|
||||
stage = 3
|
||||
component_index = index - 2
|
||||
stage_start, stage_end = component_ranges[component_index]
|
||||
if uv_missing and index == 0:
|
||||
midpoint = stage_start + (stage_end - stage_start) * 0.5
|
||||
install_uv(
|
||||
uv,
|
||||
args.status,
|
||||
stage=stage,
|
||||
stages=RUNTIME_STAGE_COUNT,
|
||||
stage_label=label,
|
||||
start=stage_start,
|
||||
end=midpoint,
|
||||
)
|
||||
stage_start = midpoint
|
||||
stage_installed = run_stage(
|
||||
args.status,
|
||||
stage,
|
||||
RUNTIME_STAGE_COUNT,
|
||||
label,
|
||||
command,
|
||||
environment,
|
||||
kind,
|
||||
start=stage_start,
|
||||
end=stage_end,
|
||||
)
|
||||
if isinstance(stage_installed, list):
|
||||
installed_packages.extend(stage_installed)
|
||||
vendor_temp.cleanup()
|
||||
remove_legacy_vendor(venv_dir.parent)
|
||||
devices = installed_devices(args)
|
||||
venv_dir.mkdir(parents=True, exist_ok=True)
|
||||
manifest_path.parent.mkdir(parents=True, exist_ok=True)
|
||||
ready_path = manifest_path
|
||||
ready_path.write_text(
|
||||
json.dumps(
|
||||
{
|
||||
"manifest_revision": MANIFEST_REVISION,
|
||||
"python": "3.12",
|
||||
"torch": TORCH_VERSION,
|
||||
"xformers": XFORMERS_VERSION,
|
||||
"fastapi": FASTAPI_VERSION,
|
||||
"uvicorn": UVICORN_VERSION,
|
||||
"torch_variant": args.torch_variant,
|
||||
"devices": devices,
|
||||
"da3_commit": args.da3_commit,
|
||||
"ben2_commit": args.ben2_commit,
|
||||
"packages": sorted(set(installed_packages)),
|
||||
},
|
||||
indent=2,
|
||||
),
|
||||
encoding="utf-8",
|
||||
)
|
||||
model_offset = MODELS_SLICE_START
|
||||
failed_models = []
|
||||
for label, command, environment, kind in model_stages:
|
||||
stage_start = model_offset
|
||||
stage_end = stage_start + model_span
|
||||
try:
|
||||
run_stage(
|
||||
args.status,
|
||||
None,
|
||||
None,
|
||||
label,
|
||||
command,
|
||||
environment,
|
||||
kind,
|
||||
start=stage_start,
|
||||
end=stage_end,
|
||||
)
|
||||
except Exception as error:
|
||||
print(f"Model download failed: {label}: {error}")
|
||||
failed_models.append(label)
|
||||
write_status(
|
||||
args.status,
|
||||
stage_end,
|
||||
f"Skipping {label}; continuing",
|
||||
)
|
||||
model_offset = stage_end
|
||||
if failed_models:
|
||||
labels = ", ".join(failed_models)
|
||||
write_status(
|
||||
args.status,
|
||||
1.0,
|
||||
f"Environment is installed; retry {labels} from Models",
|
||||
)
|
||||
return
|
||||
write_status(
|
||||
args.status,
|
||||
1.0,
|
||||
"Environment and recommended models ready",
|
||||
)
|
||||
|
||||
|
||||
def parse_args():
|
||||
parser = argparse.ArgumentParser()
|
||||
parser.add_argument("--uv", required=True)
|
||||
parser.add_argument("--venv-dir", required=True)
|
||||
parser.add_argument("--manifest", required=True)
|
||||
parser.add_argument("--models-dir", required=True)
|
||||
parser.add_argument("--torch-variant", choices=tuple(TORCH_INDEXES), required=True)
|
||||
parser.add_argument("--da3-commit", required=True)
|
||||
parser.add_argument("--ben2-commit", required=True)
|
||||
parser.add_argument("--status", required=True)
|
||||
parser.add_argument("--pypi-index", default=None)
|
||||
return parser.parse_args()
|
||||
|
||||
|
||||
def main():
|
||||
configure_standard_streams()
|
||||
args = parse_args()
|
||||
try:
|
||||
install(args)
|
||||
except Exception as error:
|
||||
write_status(
|
||||
args.status,
|
||||
1.0,
|
||||
"Environment installation failed",
|
||||
str(error),
|
||||
)
|
||||
raise
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
59
src/snapform/job_runtime.py
Normal file
59
src/snapform/job_runtime.py
Normal file
@@ -0,0 +1,59 @@
|
||||
from pathlib import Path
|
||||
|
||||
import bpy
|
||||
|
||||
from blendjob import JobRuntime
|
||||
|
||||
|
||||
DEFAULT_STORAGE_ROOT = Path.home() / ".snapform"
|
||||
|
||||
|
||||
def configured_storage_root():
|
||||
addon = bpy.context.preferences.addons.get(__package__)
|
||||
configured = getattr(getattr(addon, "preferences", None), "storage_root", "")
|
||||
if configured:
|
||||
return Path(bpy.path.abspath(configured))
|
||||
return DEFAULT_STORAGE_ROOT
|
||||
|
||||
|
||||
ENVIRONMENT = {
|
||||
"python": "3.12",
|
||||
"packages": [
|
||||
"numpy==2.4.2",
|
||||
"pillow==12.1.1",
|
||||
"av==17.0.0",
|
||||
"trimesh==4.11.2",
|
||||
"huggingface-hub==1.4.1",
|
||||
],
|
||||
"platform_packages": {
|
||||
"windows": ["onnxruntime-directml==1.24.4"],
|
||||
"default": ["onnxruntime==1.24.4"],
|
||||
},
|
||||
}
|
||||
|
||||
|
||||
def post_install(job_runtime):
|
||||
errors = []
|
||||
models = (
|
||||
("MOGE2_VITS_NORMAL", "MoGe-2 ViT-S Normal"),
|
||||
("BEN2_BASE", "BEN2 Base"),
|
||||
)
|
||||
for model, label in models:
|
||||
try:
|
||||
job_runtime.request("download-model", {"model": model})
|
||||
except RuntimeError as error:
|
||||
errors.append(f"{label}: {error}")
|
||||
if errors:
|
||||
raise RuntimeError("; ".join(errors))
|
||||
|
||||
|
||||
runtime = JobRuntime(
|
||||
"server:server",
|
||||
entrypoint_root=Path(__file__).parent,
|
||||
storage_root=configured_storage_root,
|
||||
environment=ENVIRONMENT,
|
||||
post_install=post_install,
|
||||
namespace="snapform",
|
||||
)
|
||||
|
||||
JobOperatorBase = runtime.JobOperatorBase
|
||||
188
src/snapform/model_catalog.py
Normal file
188
src/snapform/model_catalog.py
Normal file
@@ -0,0 +1,188 @@
|
||||
from dataclasses import dataclass
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class ModelSpec:
|
||||
key: str
|
||||
label: str
|
||||
directory_name: str
|
||||
repository: str
|
||||
license_name: str
|
||||
description: str
|
||||
supports_gs: bool
|
||||
download_patterns: tuple[str, ...] = ()
|
||||
ready_patterns: tuple[str, ...] = ("config.json", "*.safetensors")
|
||||
enabled: bool = True
|
||||
r2_directory: str = ""
|
||||
r2_files: tuple[tuple[str, int, str], ...] = ()
|
||||
|
||||
|
||||
MODELS = {
|
||||
"SMALL": ModelSpec(
|
||||
key="SMALL",
|
||||
label="DA3 Small",
|
||||
directory_name="DA3-SMALL",
|
||||
repository="depth-anything/DA3-SMALL",
|
||||
license_name="Apache 2.0",
|
||||
description="Fastest any-view model using ONNX Runtime",
|
||||
supports_gs=False,
|
||||
download_patterns=("model.onnx",),
|
||||
ready_patterns=("model.onnx",),
|
||||
r2_directory="DA3-SMALL",
|
||||
r2_files=(
|
||||
("model.onnx", 105484029, "7a9d81475b3e2334ace0d6d2c2a816b3c93081437e1866cc434ce1789125f631"),
|
||||
),
|
||||
),
|
||||
"BASE": ModelSpec(
|
||||
key="BASE",
|
||||
label="DA3 Base",
|
||||
directory_name="DA3-BASE",
|
||||
repository="depth-anything/DA3-BASE",
|
||||
license_name="Apache 2.0",
|
||||
description="Balanced any-view model using ONNX Runtime",
|
||||
supports_gs=False,
|
||||
download_patterns=("model.onnx",),
|
||||
ready_patterns=("model.onnx",),
|
||||
r2_directory="DA3-BASE",
|
||||
r2_files=(
|
||||
("model.onnx", 412856697, "31f38fc4990539d7902ba267924b9731db1acb00d840c248a60b1865810ff9e6"),
|
||||
),
|
||||
),
|
||||
"GIANT_1_1": ModelSpec(
|
||||
key="GIANT_1_1",
|
||||
label="DA3 Giant 1.1",
|
||||
directory_name="da3_giant_1_1",
|
||||
repository="depth-anything/DA3-GIANT-1.1",
|
||||
license_name="CC BY-NC 4.0",
|
||||
description="Non-commercial Giant model with Gaussian head",
|
||||
supports_gs=True,
|
||||
enabled=False,
|
||||
),
|
||||
"NESTED_GIANT_LARGE_1_1": ModelSpec(
|
||||
key="NESTED_GIANT_LARGE_1_1",
|
||||
label="DA3 Nested Giant Large 1.1",
|
||||
directory_name="da3_nested_giant_large_1_1",
|
||||
repository="depth-anything/DA3NESTED-GIANT-LARGE-1.1",
|
||||
license_name="CC BY-NC 4.0",
|
||||
description="Non-commercial nested model with metric depth and Gaussian head",
|
||||
supports_gs=True,
|
||||
enabled=False,
|
||||
),
|
||||
}
|
||||
|
||||
BEN2_MODEL = ModelSpec(
|
||||
key="BEN2_BASE",
|
||||
label="BEN2 Base",
|
||||
directory_name="BEN2-ONNX",
|
||||
repository="onnx-community/BEN2-ONNX",
|
||||
license_name="Apache 2.0",
|
||||
description="Fast high-quality FP16 ONNX background removal model",
|
||||
supports_gs=False,
|
||||
download_patterns=("onnx/model_fp16.onnx",),
|
||||
ready_patterns=("onnx/model_fp16.onnx",),
|
||||
r2_directory="BEN2-ONNX",
|
||||
r2_files=((
|
||||
"onnx/model_fp16.onnx",
|
||||
219121675,
|
||||
"dfdc25f421f32a0d1268e0f2ff2153d340e8f1d52d3dd16f5dc33c1ce85cedf1",
|
||||
),),
|
||||
)
|
||||
|
||||
|
||||
MOGE2_MODELS = {
|
||||
"MOGE2_VITS_NORMAL": ModelSpec(
|
||||
key="MOGE2_VITS_NORMAL",
|
||||
label="MoGe-2 ViT-S Normal",
|
||||
directory_name="moge-2-vits-normal-onnx",
|
||||
repository="Ruicheng/moge-2-vits-normal-onnx",
|
||||
license_name="MIT",
|
||||
description="Fastest MoGe-2 model using ONNX Runtime",
|
||||
supports_gs=False,
|
||||
download_patterns=("model.onnx",),
|
||||
ready_patterns=("model.onnx",),
|
||||
r2_directory="moge-2-vits-normal-onnx",
|
||||
r2_files=((
|
||||
"model.onnx",
|
||||
140852051,
|
||||
"24eacb5dc7a2c54c7bc98f7de085ffbed79ad006ea5b664c2c2cdc02ff3a52f0",
|
||||
),),
|
||||
),
|
||||
"MOGE2_VITB_NORMAL": ModelSpec(
|
||||
key="MOGE2_VITB_NORMAL",
|
||||
label="MoGe-2 ViT-B Normal",
|
||||
directory_name="moge-2-vitb-normal-onnx",
|
||||
repository="Ruicheng/moge-2-vitb-normal-onnx",
|
||||
license_name="MIT",
|
||||
description="Balanced MoGe-2 model using ONNX Runtime",
|
||||
supports_gs=False,
|
||||
download_patterns=("model.onnx",),
|
||||
ready_patterns=("model.onnx",),
|
||||
r2_directory="moge-2-vitb-normal-onnx",
|
||||
r2_files=((
|
||||
"model.onnx",
|
||||
419411850,
|
||||
"bbf14e07a30f11e69d36ab861590123f5598ababcbc8946a063eb4a966f35a21",
|
||||
),),
|
||||
),
|
||||
"MOGE2_VITL_NORMAL": ModelSpec(
|
||||
key="MOGE2_VITL_NORMAL",
|
||||
label="MoGe-2 ViT-L Normal",
|
||||
directory_name="moge-2-vitl-normal-onnx",
|
||||
repository="Ruicheng/moge-2-vitl-normal-onnx",
|
||||
license_name="MIT",
|
||||
description="Highest-quality MoGe-2 model using ONNX Runtime",
|
||||
supports_gs=False,
|
||||
download_patterns=("model.onnx",),
|
||||
ready_patterns=("model.onnx",),
|
||||
r2_directory="moge-2-vitl-normal-onnx",
|
||||
r2_files=((
|
||||
"model.onnx",
|
||||
1324265014,
|
||||
"afbc4ccc3450298f3afb35b90f015f4c4f552dea21dc6470d5f7b78b77e2d751",
|
||||
),),
|
||||
),
|
||||
}
|
||||
MOGE2_MODEL = MOGE2_MODELS["MOGE2_VITS_NORMAL"]
|
||||
DOWNLOADABLE_MODELS = {
|
||||
**MODELS,
|
||||
BEN2_MODEL.key: BEN2_MODEL,
|
||||
**MOGE2_MODELS,
|
||||
}
|
||||
MODEL_NUMBERS = {
|
||||
model_key: number for number, model_key in enumerate(MODELS)
|
||||
}
|
||||
|
||||
|
||||
def get_model(model_key):
|
||||
try:
|
||||
return MODELS[model_key]
|
||||
except KeyError as error:
|
||||
raise ValueError(f"Unknown DA3 model: {model_key}") from error
|
||||
|
||||
|
||||
def get_downloadable_model(model_key):
|
||||
try:
|
||||
return DOWNLOADABLE_MODELS[model_key]
|
||||
except KeyError as error:
|
||||
raise ValueError(f"Unknown model: {model_key}") from error
|
||||
|
||||
|
||||
def model_directory_name(model_key):
|
||||
"""Return the model's folder name inside the shared models directory."""
|
||||
return get_downloadable_model(model_key).directory_name
|
||||
|
||||
|
||||
def model_record(model, ready=False):
|
||||
"""Return public model metadata exposed by the Models Resource."""
|
||||
return {
|
||||
"key": model.key,
|
||||
"label": model.label,
|
||||
"description": model.description,
|
||||
"license": model.license_name,
|
||||
"supports_gs": model.supports_gs,
|
||||
"enabled": model.enabled,
|
||||
"family": "moge2" if model.key.startswith("MOGE2_") else (
|
||||
"ben2" if model.key == BEN2_MODEL.key else "da3"
|
||||
),
|
||||
"ready": bool(ready),
|
||||
}
|
||||
@@ -1,254 +0,0 @@
|
||||
from dataclasses import dataclass
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class ModelSpec:
|
||||
key: str
|
||||
label: str
|
||||
directory_name: str
|
||||
repository: str
|
||||
license_name: str
|
||||
description: str
|
||||
supports_gs: bool
|
||||
download_patterns: tuple[str, ...] = ()
|
||||
ready_patterns: tuple[str, ...] = ("config.json", "*.safetensors")
|
||||
enabled: bool = True
|
||||
r2_directory: str = ""
|
||||
r2_files: tuple[tuple[str, int, str], ...] = ()
|
||||
|
||||
|
||||
MODELS = {
|
||||
"SMALL": ModelSpec(
|
||||
key="SMALL",
|
||||
label="DA3 Small",
|
||||
directory_name="da3_small",
|
||||
repository="depth-anything/DA3-SMALL",
|
||||
license_name="Apache 2.0",
|
||||
description="Fastest any-view model with Apache 2.0 weights",
|
||||
supports_gs=False,
|
||||
r2_directory="DA3Small",
|
||||
r2_files=(
|
||||
("config.json", 1202, "a486e29e82b7ab4a7d4cefc1ea4526cfe2ae438a572c8ca98917cfbcde7447d2"),
|
||||
("model.safetensors", 137248940, "364492e38a3a06d221ac75da7f6621ada3f2361cd24fde11ba79091e9f40efcf"),
|
||||
),
|
||||
),
|
||||
"BASE": ModelSpec(
|
||||
key="BASE",
|
||||
label="DA3 Base",
|
||||
directory_name="da3_base",
|
||||
repository="depth-anything/DA3-BASE",
|
||||
license_name="Apache 2.0",
|
||||
description="Balanced any-view model with Apache 2.0 weights",
|
||||
supports_gs=False,
|
||||
r2_directory="DA3Base",
|
||||
r2_files=(
|
||||
("config.json", 1205, "5e34115ebc17bd2d8d43033c5f72e9446ac8833fd61d3fa160b7e67e0bb5b7b5"),
|
||||
("configuration.json", 74, "b684d553257b2ca379fd5dfdf96a6e4a633eec3b969c97119c9875465a7c07a8"),
|
||||
("model.safetensors", 541518028, "e01067dc1659613083d9145a9a2547ccdbe6ccbbf83c4fe7b3e8a4e2bdae78b5"),
|
||||
),
|
||||
),
|
||||
"LARGE_1_1": ModelSpec(
|
||||
key="LARGE_1_1",
|
||||
label="DA3 Large 1.1",
|
||||
directory_name="da3_large_1_1",
|
||||
repository="depth-anything/DA3-LARGE-1.1",
|
||||
license_name="CC BY-NC 4.0",
|
||||
description="Higher quality non-commercial model with larger VRAM use",
|
||||
supports_gs=False,
|
||||
enabled=False,
|
||||
),
|
||||
"GIANT_1_1": ModelSpec(
|
||||
key="GIANT_1_1",
|
||||
label="DA3 Giant 1.1",
|
||||
directory_name="da3_giant_1_1",
|
||||
repository="depth-anything/DA3-GIANT-1.1",
|
||||
license_name="CC BY-NC 4.0",
|
||||
description="Non-commercial Giant model with Gaussian head",
|
||||
supports_gs=True,
|
||||
enabled=False,
|
||||
),
|
||||
"NESTED_GIANT_LARGE_1_1": ModelSpec(
|
||||
key="NESTED_GIANT_LARGE_1_1",
|
||||
label="DA3 Nested Giant Large 1.1",
|
||||
directory_name="da3_nested_giant_large_1_1",
|
||||
repository="depth-anything/DA3NESTED-GIANT-LARGE-1.1",
|
||||
license_name="CC BY-NC 4.0",
|
||||
description="Non-commercial nested model with metric depth and Gaussian head",
|
||||
supports_gs=True,
|
||||
enabled=False,
|
||||
),
|
||||
}
|
||||
|
||||
BEN2_MODEL = ModelSpec(
|
||||
key="BEN2_BASE",
|
||||
label="BEN2 Base",
|
||||
directory_name="ben2_base",
|
||||
repository="PramaLLC/BEN2",
|
||||
license_name="MIT",
|
||||
description="Fast high-quality background removal model",
|
||||
supports_gs=False,
|
||||
download_patterns=("config.json", "model.safetensors"),
|
||||
r2_directory="BEN2Base",
|
||||
r2_files=(
|
||||
("config.json", 124, "01ac876925577b52fa7c6af3f5b1d1420bc9b40016b6c25e2de89b38d8fff865"),
|
||||
("model.safetensors", 380577976, "ea8b7907176a09667c86343dc7d00de6a6d871076cb90bb5f753618fd6fb3ebb"),
|
||||
),
|
||||
)
|
||||
def moge2_model(key, label, directory_name, repository, description, r2_directory, size, checksum):
|
||||
return ModelSpec(
|
||||
key=key,
|
||||
label=label,
|
||||
directory_name=directory_name,
|
||||
repository=repository,
|
||||
license_name="MIT",
|
||||
description=description,
|
||||
supports_gs=False,
|
||||
download_patterns=("model.pt",),
|
||||
ready_patterns=("model.pt",),
|
||||
r2_directory=r2_directory,
|
||||
r2_files=(("model.pt", size, checksum),),
|
||||
)
|
||||
|
||||
|
||||
MOGE2_MODELS = {
|
||||
"MOGE2_VITS_NORMAL": moge2_model(
|
||||
"MOGE2_VITS_NORMAL", "MoGe-2 ViT-S Normal", "moge2_vits_normal",
|
||||
"Ruicheng/moge-2-vits-normal", "Fastest MoGe-2 model", "MoGe2ViTSNormal",
|
||||
140550416, "79a16621928c2bf0ed04659218c55c01075e950507f40bb3332fb4c873d3e1dc",
|
||||
),
|
||||
"MOGE2_VITB_NORMAL": moge2_model(
|
||||
"MOGE2_VITB_NORMAL", "MoGe-2 ViT-B Normal", "moge2_vitb_normal",
|
||||
"Ruicheng/moge-2-vitb-normal", "Balanced MoGe-2 model", "MoGe2ViTBNormal",
|
||||
419110160, "16b8110e86d5dc5a849db120ca96ef3a223fd30b0c9146d1d81db504073da5f6",
|
||||
),
|
||||
"MOGE2_VITL_NORMAL": moge2_model(
|
||||
"MOGE2_VITL_NORMAL", "MoGe-2 ViT-L Normal", "moge2_vitl_normal",
|
||||
"Ruicheng/moge-2-vitl-normal", "Highest-quality MoGe-2 model", "MoGe2ViTLNormal",
|
||||
1323815904, "280741fd09bc3f403ccff9967784c2a391b52d2c0742ae3efdb21d9f90cc1a01",
|
||||
),
|
||||
}
|
||||
MOGE2_MODEL = MOGE2_MODELS["MOGE2_VITB_NORMAL"]
|
||||
DOWNLOADABLE_MODELS = {
|
||||
**MODELS,
|
||||
BEN2_MODEL.key: BEN2_MODEL,
|
||||
**MOGE2_MODELS,
|
||||
}
|
||||
MODEL_NUMBERS = {
|
||||
model_key: number for number, model_key in enumerate(MODELS)
|
||||
}
|
||||
|
||||
|
||||
_ENUM_ITEMS_CACHE = {}
|
||||
|
||||
|
||||
def enum_items(supports_gs=None, show_download_status=False):
|
||||
model_ready = None
|
||||
if show_download_status:
|
||||
from .client import model_ready
|
||||
|
||||
models = [
|
||||
model
|
||||
for model in MODELS.values()
|
||||
if model.enabled
|
||||
and (supports_gs is None or model.supports_gs == supports_gs)
|
||||
]
|
||||
download_status = (
|
||||
tuple(model_ready(model.key) for model in models)
|
||||
if model_ready is not None
|
||||
else ()
|
||||
)
|
||||
cache_key = (
|
||||
supports_gs,
|
||||
show_download_status,
|
||||
download_status,
|
||||
)
|
||||
cached_items = _ENUM_ITEMS_CACHE.get(cache_key)
|
||||
if cached_items is not None:
|
||||
return cached_items
|
||||
|
||||
items = []
|
||||
for index, model in enumerate(models):
|
||||
label = model.label
|
||||
if download_status and not download_status[index]:
|
||||
label = f"{label} (Missing)"
|
||||
items.append(
|
||||
(
|
||||
model.key,
|
||||
label,
|
||||
f"{model.description}; license: {model.license_name}",
|
||||
MODEL_NUMBERS[model.key],
|
||||
)
|
||||
)
|
||||
|
||||
_ENUM_ITEMS_CACHE[cache_key] = items
|
||||
return items
|
||||
|
||||
|
||||
def model_enum_items(_owner, _context):
|
||||
return enum_items(show_download_status=True)
|
||||
|
||||
|
||||
def gs_model_enum_items(_owner, _context):
|
||||
return enum_items(supports_gs=True, show_download_status=True)
|
||||
|
||||
|
||||
def moge2_model_enum_items(_owner, _context):
|
||||
from .client import model_ready
|
||||
|
||||
return [
|
||||
(
|
||||
model.key,
|
||||
model.label if model_ready(model.key) else f"{model.label} (Missing)",
|
||||
f"{model.description}; license: {model.license_name}",
|
||||
index,
|
||||
)
|
||||
for index, model in enumerate(MOGE2_MODELS.values())
|
||||
]
|
||||
|
||||
|
||||
def depth_plane_model_enum_items(_owner, _context):
|
||||
from .client import model_ready
|
||||
|
||||
models = [
|
||||
*MOGE2_MODELS.values(),
|
||||
*(model for model in MODELS.values() if model.enabled),
|
||||
]
|
||||
items = []
|
||||
for index, model in enumerate(models):
|
||||
label = model.label if model_ready(model.key) else f"{model.label} (Missing)"
|
||||
items.append(
|
||||
(
|
||||
model.key,
|
||||
label,
|
||||
f"{model.description}; license: {model.license_name}",
|
||||
index,
|
||||
)
|
||||
)
|
||||
items.append(("ORIGINAL", "Original", "Use the original image as depth", len(items)))
|
||||
return items
|
||||
|
||||
|
||||
def selected_model_key(settings, export_format=None):
|
||||
selected_format = export_format or settings.export_format
|
||||
if selected_format == "gs_ply":
|
||||
return settings.gs_model
|
||||
return settings.model
|
||||
|
||||
|
||||
def get_model(model_key):
|
||||
try:
|
||||
return MODELS[model_key]
|
||||
except KeyError as error:
|
||||
raise ValueError(f"Unknown DA3 model: {model_key}") from error
|
||||
|
||||
|
||||
def get_downloadable_model(model_key):
|
||||
try:
|
||||
return DOWNLOADABLE_MODELS[model_key]
|
||||
except KeyError as error:
|
||||
raise ValueError(f"Unknown model: {model_key}") from error
|
||||
|
||||
|
||||
def model_directory_name(model_key):
|
||||
"""Return the model's folder name inside the shared models directory."""
|
||||
return get_downloadable_model(model_key).directory_name
|
||||
File diff suppressed because it is too large
Load Diff
@@ -1,14 +1,16 @@
|
||||
import bpy
|
||||
|
||||
from .models import BEN2_MODEL, MODELS, MOGE2_MODELS, selected_model_key
|
||||
from .operators import active_job
|
||||
from . import client
|
||||
from .client import model_ready, env_installed
|
||||
from .job_runtime import runtime
|
||||
from .properties import BEN2_MODEL_KEY, model_catalog, model_ready
|
||||
|
||||
|
||||
class AiServicePanel(bpy.types.Panel):
|
||||
bl_label = "AI Service"
|
||||
bl_idname = "SNAPFORM_PT_service"
|
||||
def selected_model_key(settings):
|
||||
return settings.gs_model if settings.export_format == "gs_ply" else settings.model
|
||||
|
||||
|
||||
class ServerPanel(bpy.types.Panel):
|
||||
bl_label = "Job Server"
|
||||
bl_idname = "SNAPFORM_PT_server"
|
||||
bl_space_type = "VIEW_3D"
|
||||
bl_region_type = "UI"
|
||||
bl_category = "SnapForm"
|
||||
@@ -16,23 +18,22 @@ class AiServicePanel(bpy.types.Panel):
|
||||
|
||||
def draw(self, _context):
|
||||
layout = self.layout
|
||||
running = active_job() is not None or client.service_busy()
|
||||
if not env_installed():
|
||||
running = runtime.server_busy()
|
||||
if not runtime.environment_ready():
|
||||
layout.label(
|
||||
text="Environment is not installed",
|
||||
icon="ERROR",
|
||||
)
|
||||
install_row = layout.row()
|
||||
install_row.enabled = not running
|
||||
operator = install_row.operator(
|
||||
"snapform.run_job",
|
||||
install_row.operator(
|
||||
"snapform.install_environment",
|
||||
text="Install Environment",
|
||||
icon="IMPORT",
|
||||
)
|
||||
operator.job_type = "INSTALL_ENV"
|
||||
return
|
||||
|
||||
status = client.service_status()
|
||||
status = runtime.server_status()
|
||||
state = status.get("state", "STOPPED")
|
||||
icons = {
|
||||
"READY": "RADIOBUT_ON",
|
||||
@@ -46,7 +47,7 @@ class AiServicePanel(bpy.types.Panel):
|
||||
if state in {"READY", "BUSY"}:
|
||||
row.label(
|
||||
text=(
|
||||
f"Running (Clients: {status.get('clients', 0)})"
|
||||
"Running"
|
||||
if state == "READY"
|
||||
else "Running (Busy)"
|
||||
),
|
||||
@@ -54,41 +55,40 @@ class AiServicePanel(bpy.types.Panel):
|
||||
)
|
||||
if state == "READY":
|
||||
controls = row.row(align=True)
|
||||
controls.enabled = status.get("clients", 0) <= 1
|
||||
operator = controls.operator(
|
||||
"snapform.control_service",
|
||||
controls.operator(
|
||||
"snapform.restart_server",
|
||||
text="",
|
||||
icon="FILE_REFRESH",
|
||||
)
|
||||
operator.action = "RESTART"
|
||||
operator = controls.operator(
|
||||
"snapform.control_service",
|
||||
controls.operator(
|
||||
"snapform.stop_server",
|
||||
text="",
|
||||
icon="PAUSE",
|
||||
)
|
||||
operator.action = "STOP"
|
||||
elif state == "STOPPED":
|
||||
operator = row.operator(
|
||||
"snapform.control_service",
|
||||
row.operator(
|
||||
"snapform.start_server",
|
||||
text="Start",
|
||||
icon="PLAY",
|
||||
)
|
||||
operator.action = "START"
|
||||
else:
|
||||
row.label(
|
||||
text=status.get("message", state.title()),
|
||||
icon=icons.get(state, "QUESTION"),
|
||||
)
|
||||
if state == "ERROR":
|
||||
operator = row.operator(
|
||||
"snapform.control_service",
|
||||
row.operator(
|
||||
"snapform.start_server",
|
||||
text="Start",
|
||||
icon="PLAY",
|
||||
)
|
||||
operator.action = "START"
|
||||
|
||||
if state in {"READY", "BUSY"}:
|
||||
cached = status.get("cached_models", {})
|
||||
cached = (
|
||||
status.get("resources", {})
|
||||
.get("model_manager", {})
|
||||
.get("loaded", {})
|
||||
)
|
||||
cached_names = [name for name in cached.values() if name]
|
||||
cached_row = layout.row(align=True)
|
||||
cached_row.label(
|
||||
@@ -100,12 +100,11 @@ class AiServicePanel(bpy.types.Panel):
|
||||
)
|
||||
clear = cached_row.row(align=True)
|
||||
clear.enabled = state == "READY"
|
||||
operator = clear.operator(
|
||||
"snapform.control_service",
|
||||
clear.operator(
|
||||
"snapform.clear_models",
|
||||
text="",
|
||||
icon="TRASH",
|
||||
)
|
||||
operator.action = "CLEAR"
|
||||
|
||||
|
||||
class ModelsPanel(bpy.types.Panel):
|
||||
@@ -119,34 +118,41 @@ class ModelsPanel(bpy.types.Panel):
|
||||
|
||||
@classmethod
|
||||
def poll(cls, _context):
|
||||
return env_installed()
|
||||
return runtime.environment_ready()
|
||||
|
||||
def draw(self, _context):
|
||||
layout = self.layout
|
||||
running = active_job() is not None or client.service_busy()
|
||||
running = runtime.server_busy()
|
||||
|
||||
for model in MODELS.values():
|
||||
if model.enabled:
|
||||
models = model_catalog()
|
||||
for model in models:
|
||||
if model.get("family") == "moge2":
|
||||
self._draw_model(layout, model, running)
|
||||
for model in MOGE2_MODELS.values():
|
||||
self._draw_model(layout, model, running)
|
||||
self._draw_model(layout, BEN2_MODEL, running)
|
||||
for model in models:
|
||||
if model.get("family") == "da3" and model.get("enabled"):
|
||||
self._draw_model(layout, model, running)
|
||||
layout.separator(type="LINE")
|
||||
ben2 = next(
|
||||
(model for model in models if model.get("key") == BEN2_MODEL_KEY),
|
||||
None,
|
||||
)
|
||||
if ben2:
|
||||
self._draw_model(layout, ben2, running)
|
||||
|
||||
def _draw_model(self, layout, model, running):
|
||||
row = layout.row(align=True)
|
||||
row.label(text=model.label)
|
||||
if model_ready(model.key):
|
||||
row.label(text=model["label"])
|
||||
if model.get("ready"):
|
||||
row.label(text="Ready", icon="CHECKMARK")
|
||||
else:
|
||||
download_row = row.row(align=True)
|
||||
download_row.enabled = env_installed() and not running
|
||||
download_row.enabled = runtime.environment_ready() and not running
|
||||
operator = download_row.operator(
|
||||
"snapform.run_job",
|
||||
"snapform.download_model",
|
||||
text="Download",
|
||||
icon="IMPORT",
|
||||
)
|
||||
operator.job_type = "DOWNLOAD_MODEL"
|
||||
operator.model_key = model.key
|
||||
operator.model = model["key"]
|
||||
|
||||
|
||||
class DebugPanel(bpy.types.Panel):
|
||||
@@ -185,19 +191,18 @@ def draw_da3_input_settings(layout, settings, model_property):
|
||||
layout.prop(settings, "video_frames")
|
||||
layout.prop(settings, "process_res")
|
||||
layout.prop(settings, "process_res_method")
|
||||
layout.prop(settings, "use_ray_pose")
|
||||
layout.prop(settings, "ref_view_strategy")
|
||||
|
||||
|
||||
class Da3OutputDebugPanel(bpy.types.Panel):
|
||||
bl_label = "DA3 Output Test"
|
||||
bl_label = "DA3"
|
||||
bl_idname = "SNAPFORM_PT_debug_da3_output"
|
||||
bl_parent_id = DebugPanel.bl_idname
|
||||
bl_space_type = "VIEW_3D"
|
||||
bl_region_type = "UI"
|
||||
bl_category = "SnapForm"
|
||||
bl_options = {"DEFAULT_CLOSED"}
|
||||
bl_order = 0
|
||||
bl_order = 1
|
||||
|
||||
def draw(self, context):
|
||||
layout = self.layout
|
||||
@@ -227,50 +232,47 @@ class Da3OutputDebugPanel(bpy.types.Panel):
|
||||
mesh_box.prop(settings, "mesh_max_edge_length")
|
||||
export_row = layout.row()
|
||||
export_row.enabled = (
|
||||
active_job() is None
|
||||
and not client.service_busy()
|
||||
not runtime.server_busy()
|
||||
and model_ready(selected_model_key(settings))
|
||||
)
|
||||
operator = export_row.operator(
|
||||
"snapform.run_job",
|
||||
export_row.operator(
|
||||
"snapform.debug_da3",
|
||||
text="Generate Output",
|
||||
icon="EXPORT",
|
||||
)
|
||||
operator.job_type = "RECONSTRUCT"
|
||||
|
||||
|
||||
class Moge2ApiDebugPanel(bpy.types.Panel):
|
||||
bl_label = "MoGe-2 API Test"
|
||||
bl_label = "MoGe-2"
|
||||
bl_idname = "SNAPFORM_PT_debug_moge2_api"
|
||||
bl_parent_id = DebugPanel.bl_idname
|
||||
bl_space_type = "VIEW_3D"
|
||||
bl_region_type = "UI"
|
||||
bl_category = "SnapForm"
|
||||
bl_options = {"DEFAULT_CLOSED"}
|
||||
bl_order = 1
|
||||
bl_order = 0
|
||||
|
||||
def draw(self, context):
|
||||
layout = self.layout
|
||||
settings = context.scene.snapform_settings
|
||||
layout.prop(settings, "moge_export_format")
|
||||
layout.prop(settings, "input_path")
|
||||
layout.prop(settings, "moge_resolution_level")
|
||||
layout.prop(settings, "moge_model")
|
||||
layout.prop(settings, "moge_resolution_level")
|
||||
run_row = layout.row()
|
||||
run_row.enabled = (
|
||||
active_job() is None
|
||||
and not client.service_busy()
|
||||
not runtime.server_busy()
|
||||
and model_ready(settings.moge_model)
|
||||
)
|
||||
operator = run_row.operator(
|
||||
"snapform.run_job",
|
||||
run_row.operator(
|
||||
"snapform.debug_moge2",
|
||||
text="Generate MoGe-2 Output",
|
||||
icon="EXPORT",
|
||||
)
|
||||
operator.job_type = "MOGE2_DEBUG"
|
||||
|
||||
|
||||
class Ben2RemovalDebugPanel(bpy.types.Panel):
|
||||
bl_label = "BEN2 Removal Test"
|
||||
bl_label = "BEN2"
|
||||
bl_idname = "SNAPFORM_PT_debug_ben2_removal"
|
||||
bl_parent_id = DebugPanel.bl_idname
|
||||
bl_space_type = "VIEW_3D"
|
||||
@@ -286,13 +288,11 @@ class Ben2RemovalDebugPanel(bpy.types.Panel):
|
||||
layout.prop(settings, "background_input_path")
|
||||
background_row = layout.row()
|
||||
background_row.enabled = (
|
||||
active_job() is None
|
||||
and not client.service_busy()
|
||||
and model_ready(BEN2_MODEL.key)
|
||||
not runtime.server_busy()
|
||||
and model_ready(BEN2_MODEL_KEY)
|
||||
)
|
||||
operator = background_row.operator(
|
||||
"snapform.run_job",
|
||||
background_row.operator(
|
||||
"snapform.debug_ben2",
|
||||
text="Remove Background",
|
||||
icon="IMAGE_ALPHA",
|
||||
)
|
||||
operator.job_type = "REMOVE_BACKGROUND"
|
||||
|
||||
@@ -1,8 +1,7 @@
|
||||
from pathlib import Path
|
||||
|
||||
import bpy
|
||||
|
||||
from .client import device_enum_items
|
||||
from .job_runtime import DEFAULT_STORAGE_ROOT, runtime
|
||||
from .properties import device_enum_items
|
||||
|
||||
|
||||
class SnapFormPreferences(bpy.types.AddonPreferences):
|
||||
@@ -15,7 +14,7 @@ class SnapFormPreferences(bpy.types.AddonPreferences):
|
||||
"all Blender versions can use the same directory"
|
||||
),
|
||||
subtype="DIR_PATH",
|
||||
default=str(Path.home() / ".snapform"),
|
||||
default=str(DEFAULT_STORAGE_ROOT),
|
||||
)
|
||||
device: bpy.props.EnumProperty(
|
||||
name="Device",
|
||||
@@ -28,29 +27,24 @@ class SnapFormPreferences(bpy.types.AddonPreferences):
|
||||
)
|
||||
|
||||
def draw(self, _context):
|
||||
from .operators import active_job
|
||||
from .client import venv_directory, env_installed
|
||||
|
||||
layout = self.layout
|
||||
layout.prop(self, "storage_root")
|
||||
layout.prop(self, "device")
|
||||
layout.separator()
|
||||
layout.label(text=f"Environment: {venv_directory()}")
|
||||
if env_installed():
|
||||
layout.label(text=f"Environment: {runtime.environment_directory()}")
|
||||
if runtime.environment_ready():
|
||||
layout.label(text="Environment is installed", icon="CHECKMARK")
|
||||
else:
|
||||
install_row = layout.row()
|
||||
install_row.enabled = active_job() is None
|
||||
operator = install_row.operator(
|
||||
"snapform.run_job",
|
||||
install_row.enabled = runtime.active_job is None
|
||||
install_row.operator(
|
||||
"snapform.install_environment",
|
||||
text="Install Environment",
|
||||
icon="IMPORT",
|
||||
)
|
||||
operator.job_type = "INSTALL_ENV"
|
||||
layout.separator()
|
||||
operator = layout.operator(
|
||||
"snapform.control_service",
|
||||
text="Open Service Log",
|
||||
layout.operator(
|
||||
"snapform.open_server_log",
|
||||
text="Open Server Log",
|
||||
icon="TEXT",
|
||||
)
|
||||
operator.action = "LOG"
|
||||
|
||||
@@ -1,7 +1,150 @@
|
||||
import platform
|
||||
import sys
|
||||
|
||||
import bpy
|
||||
|
||||
from .models import gs_model_enum_items, model_enum_items, moge2_model_enum_items
|
||||
from .client import device_enum_items
|
||||
from .job_runtime import runtime
|
||||
from . import model_catalog as shared_model_catalog
|
||||
|
||||
|
||||
BEN2_MODEL_KEY = "BEN2_BASE"
|
||||
DEFAULT_DA3_MODEL_KEY = "BASE"
|
||||
DEFAULT_MOGE2_MODEL_KEY = "MOGE2_VITS_NORMAL"
|
||||
REQUIRED_MODEL_KEYS = (DEFAULT_MOGE2_MODEL_KEY, BEN2_MODEL_KEY)
|
||||
|
||||
|
||||
DEVICE_LABELS = {
|
||||
"cuda": ("CUDA", "NVIDIA CUDA"),
|
||||
"directml": ("DirectML", "Windows GPU (DirectML)"),
|
||||
"mps": ("MPS", "Apple Metal (MPS)"),
|
||||
"cpu": ("CPU", "CPU"),
|
||||
}
|
||||
|
||||
|
||||
def supported_devices():
|
||||
devices = runtime.environment_manifest().get("devices")
|
||||
if isinstance(devices, list) and devices:
|
||||
return [str(device).lower() for device in devices]
|
||||
if sys.platform == "win32":
|
||||
return ["directml", "cpu"]
|
||||
if sys.platform == "darwin" and platform.machine().lower() in {
|
||||
"arm64",
|
||||
"aarch64",
|
||||
}:
|
||||
return ["mps", "cpu"]
|
||||
return ["cpu"]
|
||||
|
||||
|
||||
def device_enum_items(_owner, _context):
|
||||
return [
|
||||
(name.upper(), *DEVICE_LABELS[name], index)
|
||||
for index, name in enumerate(supported_devices())
|
||||
]
|
||||
|
||||
|
||||
def models_status(refresh=False):
|
||||
status = runtime.server_status().get("resources", {}).get("model_manager", {})
|
||||
return {
|
||||
"catalog": tuple(
|
||||
shared_model_catalog.model_record(
|
||||
model,
|
||||
_model_files_ready(model),
|
||||
)
|
||||
for model in shared_model_catalog.DOWNLOADABLE_MODELS.values()
|
||||
),
|
||||
"loaded": status.get("loaded", {}),
|
||||
}
|
||||
|
||||
|
||||
def _model_files_ready(model):
|
||||
directory = runtime.storage_root() / "models" / model.directory_name
|
||||
return all(
|
||||
any(directory.glob(pattern))
|
||||
for pattern in model.ready_patterns
|
||||
)
|
||||
|
||||
|
||||
def model_ready(model_key):
|
||||
return any(
|
||||
model.get("key") == model_key and model.get("ready")
|
||||
for model in models_status().get("catalog", ())
|
||||
)
|
||||
|
||||
|
||||
def required_models_ready():
|
||||
return all(model_ready(model_key) for model_key in REQUIRED_MODEL_KEYS)
|
||||
|
||||
|
||||
def model_catalog():
|
||||
return models_status().get("catalog", ())
|
||||
|
||||
|
||||
def model_record(model_key):
|
||||
return next(
|
||||
(model for model in model_catalog() if model.get("key") == model_key),
|
||||
None,
|
||||
)
|
||||
|
||||
|
||||
def _model_items(models):
|
||||
return [
|
||||
(
|
||||
model["key"],
|
||||
model["label"] if model.get("ready") else f"{model['label']} (Missing)",
|
||||
f"{model['description']}; license: {model['license']}",
|
||||
index,
|
||||
)
|
||||
for index, model in enumerate(models)
|
||||
]
|
||||
|
||||
|
||||
def model_enum_items(_owner, _context):
|
||||
return _model_items(
|
||||
model
|
||||
for model in model_catalog()
|
||||
if model.get("family") == "da3" and model.get("enabled")
|
||||
)
|
||||
|
||||
|
||||
def gs_model_enum_items(_owner, _context):
|
||||
return _model_items(
|
||||
model
|
||||
for model in model_catalog()
|
||||
if model.get("family") == "da3"
|
||||
and model.get("enabled")
|
||||
and model.get("supports_gs")
|
||||
)
|
||||
|
||||
|
||||
def moge2_model_enum_items(_owner, _context):
|
||||
return _model_items(
|
||||
model for model in model_catalog() if model.get("family") == "moge2"
|
||||
)
|
||||
|
||||
|
||||
def depth_plane_model_enum_items(_owner, _context):
|
||||
models = [
|
||||
model
|
||||
for model in model_catalog()
|
||||
if model.get("family") in {"moge2", "da3"} and model.get("enabled")
|
||||
]
|
||||
models.sort(key=lambda model: model.get("key") != DEFAULT_MOGE2_MODEL_KEY)
|
||||
items = _model_items(models)
|
||||
items.append(
|
||||
("ORIGINAL", "Original", "Use the original image as depth", len(items))
|
||||
)
|
||||
return items
|
||||
|
||||
|
||||
def surface_model_enum_items(_owner, _context):
|
||||
models = [
|
||||
model
|
||||
for model in model_catalog()
|
||||
if model.get("family") in {"moge2", "da3"}
|
||||
and model.get("enabled")
|
||||
]
|
||||
models.sort(key=lambda model: model.get("key") != DEFAULT_MOGE2_MODEL_KEY)
|
||||
return _model_items(models)
|
||||
|
||||
|
||||
class SnapFormSettings(bpy.types.PropertyGroup):
|
||||
@@ -76,7 +219,17 @@ class SnapFormSettings(bpy.types.PropertyGroup):
|
||||
name="Model",
|
||||
description="MoGe-2 model used for depth estimation",
|
||||
items=moge2_model_enum_items,
|
||||
default=1,
|
||||
default=0,
|
||||
)
|
||||
moge_export_format: bpy.props.EnumProperty(
|
||||
name="Output Format",
|
||||
description="Additional MoGe-2 debug output generated with results.npz",
|
||||
items=(
|
||||
("maps", "Maps", "Export depth, normal, and mask previews"),
|
||||
("glb", "Textured Surface (GLB)", "Export the official-style textured mesh"),
|
||||
("ply", "Point Cloud (PLY)", "Export the official-style colored point cloud"),
|
||||
),
|
||||
default="maps",
|
||||
)
|
||||
conf_thresh_percentile: bpy.props.FloatProperty(
|
||||
name="Confidence Threshold",
|
||||
@@ -98,11 +251,6 @@ class SnapFormSettings(bpy.types.PropertyGroup):
|
||||
description="Include DA3 camera wireframes in the GLB export",
|
||||
default=True,
|
||||
)
|
||||
use_ray_pose: bpy.props.BoolProperty(
|
||||
name="Use Ray Pose",
|
||||
description="Use the slower and more accurate ray-based pose estimation",
|
||||
default=False,
|
||||
)
|
||||
ref_view_strategy: bpy.props.EnumProperty(
|
||||
name="Ref View Strategy",
|
||||
description="DA3 reference view selection strategy",
|
||||
@@ -175,16 +323,3 @@ class SnapFormSettings(bpy.types.PropertyGroup):
|
||||
default=1,
|
||||
min=1,
|
||||
)
|
||||
status: bpy.props.StringProperty(
|
||||
name="Status",
|
||||
default="Ready",
|
||||
options={"SKIP_SAVE"},
|
||||
)
|
||||
progress: bpy.props.FloatProperty(
|
||||
name="Progress",
|
||||
default=0.0,
|
||||
min=0.0,
|
||||
max=1.0,
|
||||
subtype="FACTOR",
|
||||
options={"SKIP_SAVE"},
|
||||
)
|
||||
|
||||
57
src/snapform/server/__init__.py
Normal file
57
src/snapform/server/__init__.py
Normal file
@@ -0,0 +1,57 @@
|
||||
"""SnapForm Job Server without Blender dependencies."""
|
||||
|
||||
from blendjob import JobServer
|
||||
|
||||
from .model_manager import ModelManager, ModelSessionCache
|
||||
from .background import remove_background as _remove_background
|
||||
from .da3 import generate_da3 as _generate_da3
|
||||
from .media import extract_frames as _extract_frames
|
||||
from .moge2 import generate_moge2 as _generate_moge2
|
||||
|
||||
|
||||
server = JobServer("SnapForm Job Server")
|
||||
|
||||
|
||||
@server.resource("model_manager")
|
||||
def create_model_manager(job_server):
|
||||
return ModelManager(job_server.storage_root / "models")
|
||||
|
||||
|
||||
@server.job("download-model")
|
||||
def download_model(context, parameters):
|
||||
manager = context.resource("model_manager")
|
||||
return manager.download(context, parameters["model"])
|
||||
|
||||
|
||||
@server.job("download-required-models")
|
||||
def download_required_models(context, parameters):
|
||||
manager = context.resource("model_manager")
|
||||
return manager.download_missing(context, parameters["models"])
|
||||
|
||||
|
||||
@server.job("debug-ben2")
|
||||
@server.job("remove-background")
|
||||
def remove_background(context, parameters):
|
||||
return _remove_background(context, parameters)
|
||||
|
||||
|
||||
@server.job("extract-frames")
|
||||
def extract_frames(context, parameters):
|
||||
return _extract_frames(context, parameters)
|
||||
|
||||
|
||||
@server.job("debug-moge2")
|
||||
@server.job("generate-moge2-surface")
|
||||
@server.job("generate-moge2-depth")
|
||||
def generate_moge2(context, parameters):
|
||||
return _generate_moge2(context, parameters)
|
||||
|
||||
|
||||
@server.job("debug-da3")
|
||||
@server.job("generate-da3-surface")
|
||||
@server.job("generate-da3-depth")
|
||||
def generate_da3(context, parameters):
|
||||
return _generate_da3(context, parameters)
|
||||
|
||||
|
||||
__all__ = ("ModelManager", "ModelSessionCache", "server")
|
||||
170
src/snapform/server/background.py
Normal file
170
src/snapform/server/background.py
Normal file
@@ -0,0 +1,170 @@
|
||||
import shutil
|
||||
import time
|
||||
from pathlib import Path
|
||||
|
||||
from .media import (
|
||||
IMAGE_SUFFIXES,
|
||||
check_cancelled,
|
||||
copy_frames,
|
||||
prepare_background_input,
|
||||
)
|
||||
|
||||
|
||||
def infer_foreground(
|
||||
input_path,
|
||||
model_dir,
|
||||
device,
|
||||
model_cache=None,
|
||||
cancel_check=None,
|
||||
):
|
||||
from PIL import Image
|
||||
|
||||
input_path = Path(input_path)
|
||||
check_cancelled(cancel_check)
|
||||
if input_path.suffix.lower() not in IMAGE_SUFFIXES:
|
||||
raise RuntimeError("BEN2 requires a JPG, JPEG, PNG, or WEBP image")
|
||||
from .onnx_ben2 import create_session, infer
|
||||
|
||||
owns_model = model_cache is None
|
||||
requested_device = getattr(device, "type", device)
|
||||
if owns_model:
|
||||
load_started = time.perf_counter()
|
||||
model = create_session(model_dir, requested_device)
|
||||
load_ms = (time.perf_counter() - load_started) * 1000.0
|
||||
else:
|
||||
model, load_ms = model_cache.get_ben2(model_dir, device)
|
||||
|
||||
check_cancelled(cancel_check)
|
||||
with Image.open(input_path) as opened_image:
|
||||
image = opened_image.convert("RGB")
|
||||
inference_started = time.perf_counter()
|
||||
foreground = infer(model, image)
|
||||
foreground = preserve_source_rgb(image, foreground)
|
||||
inference_ms = (time.perf_counter() - inference_started) * 1000.0
|
||||
check_cancelled(cancel_check)
|
||||
return foreground, load_ms, inference_ms
|
||||
|
||||
|
||||
def preserve_source_rgb(source, foreground):
|
||||
"""Combine the untouched source RGB with BEN2's foreground alpha."""
|
||||
source_rgba = source.convert("RGBA")
|
||||
alpha = foreground.convert("RGBA").getchannel("A")
|
||||
if alpha.size != source_rgba.size:
|
||||
from PIL import Image
|
||||
|
||||
alpha = alpha.resize(source_rgba.size, Image.Resampling.LANCZOS)
|
||||
source_rgba.putalpha(alpha)
|
||||
return source_rgba
|
||||
|
||||
|
||||
def infer_foreground_frames(
|
||||
images,
|
||||
model_dir,
|
||||
device,
|
||||
output_directory,
|
||||
progress_callback,
|
||||
start_progress,
|
||||
progress_span,
|
||||
model_cache=None,
|
||||
cancel_check=None,
|
||||
):
|
||||
"""Run BEN2 on every frame and write foreground.0001.png etc."""
|
||||
total_load_ms = 0.0
|
||||
total_inference_ms = 0.0
|
||||
for index, image_path in enumerate(images):
|
||||
check_cancelled(cancel_check)
|
||||
progress_callback(
|
||||
start_progress + progress_span * index / len(images),
|
||||
f"Removing background ({index + 1}/{len(images)})",
|
||||
)
|
||||
foreground, load_ms, inference_ms = infer_foreground(
|
||||
image_path,
|
||||
model_dir,
|
||||
device,
|
||||
model_cache=model_cache,
|
||||
cancel_check=cancel_check,
|
||||
)
|
||||
output_path = output_directory / f"foreground.{index + 1:04d}.png"
|
||||
foreground.convert("RGBA").save(
|
||||
output_path,
|
||||
format="PNG",
|
||||
compress_level=1,
|
||||
)
|
||||
if not output_path.is_file():
|
||||
raise RuntimeError(f"BEN2 did not write: {output_path}")
|
||||
total_load_ms += load_ms
|
||||
total_inference_ms += inference_ms
|
||||
return total_load_ms, total_inference_ms
|
||||
|
||||
|
||||
def remove_background(context, parameters):
|
||||
"""Run BEN2 and write transparent foreground images into the Job directory."""
|
||||
total_started = time.perf_counter()
|
||||
context.progress(0.02, "Starting Environment")
|
||||
context.check_cancelled()
|
||||
|
||||
output_directory = context.directory
|
||||
output_directory.mkdir(parents=True, exist_ok=True)
|
||||
device = parameters["device"]
|
||||
model_manager = context.resource("model_manager")
|
||||
model_directory = model_manager.directory(parameters["model"])
|
||||
|
||||
video_frames = parameters.get("video_frames", 16)
|
||||
start_frame = parameters.get("start_frame", 1)
|
||||
images, temporary_directory = prepare_background_input(
|
||||
parameters["input"],
|
||||
video_frames,
|
||||
output_directory,
|
||||
start_frame=start_frame,
|
||||
cancel_check=context.check_cancelled,
|
||||
)
|
||||
try:
|
||||
context.progress(0.12, f"Loading BEN2 for {device.upper()}")
|
||||
if len(images) == 1:
|
||||
context.progress(0.35, "Removing background")
|
||||
foreground, load_ms, inference_ms = infer_foreground(
|
||||
images[0],
|
||||
model_directory,
|
||||
device,
|
||||
model_cache=model_manager,
|
||||
cancel_check=context.check_cancelled,
|
||||
)
|
||||
context.check_cancelled()
|
||||
context.progress(0.92, "Writing transparent PNG")
|
||||
output_path = output_directory / "foreground.png"
|
||||
foreground.convert("RGBA").save(
|
||||
output_path,
|
||||
format="PNG",
|
||||
compress_level=1,
|
||||
)
|
||||
if not output_path.is_file():
|
||||
raise RuntimeError("BEN2 did not write a foreground image")
|
||||
else:
|
||||
load_ms, inference_ms = infer_foreground_frames(
|
||||
images,
|
||||
model_directory,
|
||||
device,
|
||||
output_directory,
|
||||
context.progress,
|
||||
0.12,
|
||||
0.7,
|
||||
model_cache=model_manager,
|
||||
cancel_check=context.check_cancelled,
|
||||
)
|
||||
copy_frames(images, output_directory)
|
||||
finally:
|
||||
if temporary_directory is not None:
|
||||
shutil.rmtree(temporary_directory, ignore_errors=True)
|
||||
|
||||
context.check_cancelled()
|
||||
total_ms = (time.perf_counter() - total_started) * 1000.0
|
||||
frame_text = "" if len(images) == 1 else f" from {len(images)} frame(s)"
|
||||
message = (
|
||||
f"Background removed{frame_text} in {inference_ms:.0f} ms "
|
||||
f"(model load {load_ms:.0f} ms, total {total_ms:.0f} ms)"
|
||||
)
|
||||
print(message)
|
||||
context.progress(1.0, message)
|
||||
if tuple(output_directory.glob("foreground.*.png")):
|
||||
return {"foreground": "."}
|
||||
return {"foreground": "foreground.png"}
|
||||
232
src/snapform/server/da3.py
Normal file
232
src/snapform/server/da3.py
Normal file
@@ -0,0 +1,232 @@
|
||||
import shutil
|
||||
|
||||
from .background import infer_foreground, infer_foreground_frames
|
||||
from .geometry import (
|
||||
export_da3_depth_preview,
|
||||
export_da3_mini_npz,
|
||||
export_da3_point_cloud,
|
||||
export_image_mesh_data,
|
||||
export_prediction_attributes,
|
||||
export_textured_mesh_glb,
|
||||
)
|
||||
from .media import (
|
||||
copy_frames,
|
||||
flatten_exported_paths,
|
||||
prepare_input,
|
||||
wait_for_exported_paths,
|
||||
)
|
||||
|
||||
|
||||
def merge_da3_frame_predictions(predictions):
|
||||
"""Combine independent single-frame DA3 results for existing exporters."""
|
||||
import numpy as np
|
||||
|
||||
if not predictions:
|
||||
raise ValueError("DA3 returned no frame predictions")
|
||||
merged = predictions[0]
|
||||
for name in (
|
||||
"depth",
|
||||
"conf",
|
||||
"extrinsics",
|
||||
"intrinsics",
|
||||
"processed_images",
|
||||
):
|
||||
values = [getattr(prediction, name) for prediction in predictions]
|
||||
setattr(merged, name, np.concatenate(values, axis=0))
|
||||
return merged
|
||||
|
||||
|
||||
def generate_da3(context, parameters):
|
||||
"""Run DA3 inference and return the artifact used by the requesting Job."""
|
||||
context.progress(0.02, "Starting Environment")
|
||||
context.check_cancelled()
|
||||
from .onnx_da3 import infer
|
||||
from .onnx_runtime import provider_label
|
||||
|
||||
output_directory = context.directory
|
||||
output_directory.mkdir(parents=True, exist_ok=True)
|
||||
context.progress( 0.08, "Preparing input")
|
||||
model_manager = context.resource("model_manager")
|
||||
model_directory = model_manager.directory(parameters["model"])
|
||||
device = parameters["device"]
|
||||
export_format = parameters["export_format"]
|
||||
images, temporary_directory = prepare_input(
|
||||
parameters["input"],
|
||||
parameters.get("video_frames", 16),
|
||||
output_directory,
|
||||
start_frame=parameters.get("start_frame", 1),
|
||||
cancel_check=context.check_cancelled,
|
||||
)
|
||||
|
||||
try:
|
||||
context.check_cancelled()
|
||||
remove_background_model = parameters.get("remove_background_model", "")
|
||||
remove_background_model_dir = (
|
||||
model_manager.directory(remove_background_model)
|
||||
if remove_background_model
|
||||
else ""
|
||||
)
|
||||
foreground_images = []
|
||||
if remove_background_model_dir:
|
||||
context.progress( 0.12, "Removing image background")
|
||||
if len(images) == 1:
|
||||
foreground, load_ms, inference_ms = infer_foreground(
|
||||
images[0],
|
||||
remove_background_model_dir,
|
||||
device,
|
||||
model_cache=model_manager,
|
||||
cancel_check=context.check_cancelled,
|
||||
)
|
||||
context.check_cancelled()
|
||||
foreground.convert("RGBA").save(
|
||||
output_directory / "foreground.png",
|
||||
format="PNG",
|
||||
compress_level=1,
|
||||
)
|
||||
if not (output_directory / "foreground.png").is_file():
|
||||
raise RuntimeError("BEN2 did not write a foreground image")
|
||||
foreground_images = [output_directory / "foreground.png"]
|
||||
else:
|
||||
load_ms, inference_ms = infer_foreground_frames(
|
||||
images,
|
||||
remove_background_model_dir,
|
||||
device,
|
||||
output_directory,
|
||||
context.progress,
|
||||
0.12,
|
||||
0.05,
|
||||
model_cache=model_manager,
|
||||
cancel_check=context.check_cancelled,
|
||||
)
|
||||
foreground_images = sorted(
|
||||
output_directory.glob("foreground.*.png")
|
||||
)
|
||||
if remove_background_model_dir:
|
||||
print(
|
||||
f"BEN2 inference {inference_ms:.0f} ms; "
|
||||
f"model load {load_ms:.0f} ms"
|
||||
)
|
||||
context.progress( 0.15, "Loading DA3 ONNX model")
|
||||
model = model_manager.get_da3(model_directory, device)
|
||||
context.progress(
|
||||
0.20,
|
||||
f"Loaded DA3 on {provider_label(model)}",
|
||||
)
|
||||
|
||||
context.check_cancelled()
|
||||
context.progress(
|
||||
0.25,
|
||||
f"Estimating depth for {len(images)} frame(s)",
|
||||
)
|
||||
inference_options = {
|
||||
"process_res": parameters["process_res"],
|
||||
"process_res_method": parameters["process_res_method"],
|
||||
"ref_view_strategy": parameters["ref_view_strategy"],
|
||||
}
|
||||
try:
|
||||
if parameters.get("inference_mode", "joint") == "per_frame":
|
||||
predictions = []
|
||||
for index, image in enumerate(images):
|
||||
context.check_cancelled()
|
||||
context.progress(
|
||||
0.25 + 0.60 * index / len(images),
|
||||
f"Estimating DA3 depth ({index + 1}/{len(images)})",
|
||||
)
|
||||
predictions.append(
|
||||
infer(model, [str(image)], **inference_options)
|
||||
)
|
||||
prediction = merge_da3_frame_predictions(predictions)
|
||||
else:
|
||||
prediction = infer(
|
||||
model,
|
||||
[str(path) for path in images],
|
||||
**inference_options,
|
||||
)
|
||||
except UnicodeDecodeError as error:
|
||||
# DirectML reports failures with a locale-specific message that
|
||||
# ONNX Runtime cannot always decode, masking the real error.
|
||||
if provider_label(model) == "DirectML":
|
||||
raise RuntimeError(
|
||||
"DA3 inference failed on DirectML. Switch the device to "
|
||||
"CPU in SnapForm Preferences and retry. "
|
||||
f"Original error: {error}"
|
||||
) from error
|
||||
raise
|
||||
context.check_cancelled()
|
||||
context.progress( 0.88, f"Exporting {export_format}")
|
||||
if export_format == "mesh":
|
||||
export_textured_mesh_glb(
|
||||
prediction,
|
||||
images,
|
||||
output_directory,
|
||||
parameters.get("conf_thresh_percentile", 40.0),
|
||||
bool(parameters.get("mesh_filter_edges", True)),
|
||||
float(parameters.get("mesh_max_edge_length", 0.1)),
|
||||
)
|
||||
elif export_format == "mesh_data":
|
||||
def mesh_progress(step, total, label):
|
||||
context.progress(
|
||||
0.88 + 0.09 * step / total,
|
||||
f"Preparing surface data {step}/{total}: {label}",
|
||||
)
|
||||
|
||||
export_image_mesh_data(
|
||||
prediction,
|
||||
images[0],
|
||||
output_directory,
|
||||
foreground_path=(
|
||||
foreground_images[0] if foreground_images else None
|
||||
),
|
||||
progress_callback=mesh_progress,
|
||||
)
|
||||
elif export_format == "mini_npz":
|
||||
export_da3_mini_npz(prediction, output_directory)
|
||||
elif export_format == "depth_vis":
|
||||
export_da3_depth_preview(prediction, output_directory)
|
||||
elif export_format == "glb":
|
||||
export_da3_point_cloud(
|
||||
prediction,
|
||||
output_directory,
|
||||
parameters["conf_thresh_percentile"],
|
||||
parameters["num_max_points"],
|
||||
)
|
||||
else:
|
||||
raise ValueError(
|
||||
f"DA3 ONNX does not support export format: {export_format}"
|
||||
)
|
||||
if export_format in {"glb", "gs_ply", "depth_vis", "mesh"}:
|
||||
context.progress(
|
||||
0.97,
|
||||
"Writing structured debug attributes",
|
||||
)
|
||||
export_prediction_attributes(prediction, output_directory)
|
||||
context.check_cancelled()
|
||||
|
||||
result_paths = wait_for_exported_paths(
|
||||
output_directory,
|
||||
export_format,
|
||||
cancel_check=context.check_cancelled,
|
||||
)
|
||||
if not result_paths:
|
||||
raise RuntimeError(
|
||||
f"No {export_format} files were exported"
|
||||
)
|
||||
result_paths = flatten_exported_paths(
|
||||
output_directory,
|
||||
result_paths,
|
||||
cancel_check=context.check_cancelled,
|
||||
)
|
||||
context.check_cancelled()
|
||||
copy_frames(images, output_directory)
|
||||
context.progress(
|
||||
1.0,
|
||||
f"Exported {len(result_paths)} {export_format} file(s)",
|
||||
)
|
||||
if context.job_type == "generate-da3-surface":
|
||||
return {"surface": "mesh.npz"}
|
||||
if context.job_type == "generate-da3-depth":
|
||||
return {"depth": "results.npz"}
|
||||
return {"directory": "."}
|
||||
finally:
|
||||
if temporary_directory is not None:
|
||||
shutil.rmtree(temporary_directory, ignore_errors=True)
|
||||
616
src/snapform/server/geometry.py
Normal file
616
src/snapform/server/geometry.py
Normal file
@@ -0,0 +1,616 @@
|
||||
from pathlib import Path
|
||||
|
||||
|
||||
SCENE_FILENAME = "scene.glb"
|
||||
|
||||
|
||||
def unproject_depth_frame(depth, intrinsics, extrinsics):
|
||||
import numpy as np
|
||||
|
||||
height, width = depth.shape
|
||||
x, y = np.meshgrid(
|
||||
np.arange(width, dtype=np.float32),
|
||||
np.arange(height, dtype=np.float32),
|
||||
)
|
||||
pixels = np.stack((x, y, np.ones_like(x)), axis=-1).reshape(-1, 3)
|
||||
rays = (np.linalg.inv(intrinsics) @ pixels.T).T
|
||||
camera_points = rays * depth.reshape(-1, 1)
|
||||
camera_points = np.column_stack(
|
||||
(camera_points, np.ones(camera_points.shape[0], dtype=np.float32))
|
||||
)
|
||||
world_to_camera = np.eye(4, dtype=np.float32)
|
||||
world_to_camera[: extrinsics.shape[0], :4] = extrinsics
|
||||
world_points = (np.linalg.inv(world_to_camera) @ camera_points.T).T
|
||||
return world_points[:, :3].reshape(height, width, 3)
|
||||
|
||||
|
||||
def unproject_camera_depth(depth, intrinsics):
|
||||
import numpy as np
|
||||
|
||||
height, width = depth.shape
|
||||
x, y = np.meshgrid(
|
||||
np.arange(width, dtype=np.float32),
|
||||
np.arange(height, dtype=np.float32),
|
||||
)
|
||||
pixels = np.stack((x, y, np.ones_like(x)), axis=-1).reshape(-1, 3)
|
||||
rays = (np.linalg.inv(intrinsics) @ pixels.T).T
|
||||
return (rays * depth.reshape(-1, 1)).reshape(height, width, 3)
|
||||
|
||||
|
||||
def compact_mesh(vertices, faces):
|
||||
import numpy as np
|
||||
|
||||
vertices = np.asarray(vertices, dtype=np.float32)
|
||||
faces = np.asarray(faces, dtype=np.int32)
|
||||
used_vertices = np.unique(faces)
|
||||
index_map = np.full(len(vertices), -1, dtype=np.int32)
|
||||
index_map[used_vertices] = np.arange(len(used_vertices), dtype=np.int32)
|
||||
vertices = vertices[used_vertices]
|
||||
faces = index_map[faces]
|
||||
return vertices, faces
|
||||
|
||||
|
||||
def project_camera_uv(vertices, intrinsics, image_size):
|
||||
import numpy as np
|
||||
|
||||
width, height = image_size
|
||||
z = vertices[:, 2]
|
||||
if np.any(~np.isfinite(z)) or np.any(z <= 0.0):
|
||||
raise ValueError("Simplified mesh contains invalid camera depth")
|
||||
u = intrinsics[0, 0] * vertices[:, 0] / z + intrinsics[0, 2]
|
||||
v = intrinsics[1, 1] * vertices[:, 1] / z + intrinsics[1, 2]
|
||||
denominator_x = max(width - 1, 1)
|
||||
denominator_y = max(height - 1, 1)
|
||||
return np.column_stack(
|
||||
(
|
||||
np.clip(u / denominator_x, 0.0, 1.0),
|
||||
np.clip(1.0 - v / denominator_y, 0.0, 1.0),
|
||||
)
|
||||
).astype(np.float32)
|
||||
|
||||
|
||||
def project_camera_pixels(vertices, intrinsics):
|
||||
import numpy as np
|
||||
|
||||
z = vertices[:, 2]
|
||||
if np.any(~np.isfinite(z)) or np.any(z <= 0.0):
|
||||
raise ValueError("Mesh contains invalid camera depth")
|
||||
return np.column_stack(
|
||||
(
|
||||
intrinsics[0, 0] * vertices[:, 0] / z + intrinsics[0, 2],
|
||||
intrinsics[1, 1] * vertices[:, 1] / z + intrinsics[1, 2],
|
||||
)
|
||||
)
|
||||
|
||||
|
||||
def sample_image_nearest(values, pixels):
|
||||
import numpy as np
|
||||
|
||||
height, width = values.shape
|
||||
x = np.clip(np.rint(pixels[:, 0]).astype(np.int32), 0, width - 1)
|
||||
y = np.clip(np.rint(pixels[:, 1]).astype(np.int32), 0, height - 1)
|
||||
return np.asarray(values)[y, x]
|
||||
|
||||
|
||||
def confidence_percentile_map(depth, confidence):
|
||||
import numpy as np
|
||||
|
||||
valid = np.isfinite(depth) & (depth > 0.0) & np.isfinite(confidence)
|
||||
result = np.zeros(depth.shape, dtype=np.float32)
|
||||
if not valid.any():
|
||||
return result
|
||||
sorted_values = np.sort(np.asarray(confidence)[valid])
|
||||
ranks = np.searchsorted(
|
||||
sorted_values,
|
||||
np.asarray(confidence)[valid],
|
||||
side="right",
|
||||
)
|
||||
result[valid] = ranks.astype(np.float32) / len(sorted_values)
|
||||
return result
|
||||
|
||||
|
||||
def depth_discontinuity_ratio(depth):
|
||||
import numpy as np
|
||||
|
||||
padded = np.pad(depth, 1, mode="edge")
|
||||
windows = np.lib.stride_tricks.sliding_window_view(padded, (3, 3))
|
||||
difference = np.nanmax(windows, axis=(-2, -1))
|
||||
difference -= np.nanmin(windows, axis=(-2, -1))
|
||||
return difference / np.maximum(depth, 1e-6)
|
||||
|
||||
|
||||
def triangle_edge_stretch_values(points, triangles):
|
||||
import numpy as np
|
||||
|
||||
flat_points = np.asarray(points, dtype=np.float32).reshape(-1, 3)
|
||||
triangle_points = flat_points[triangles]
|
||||
triangle_depth = triangle_points[..., 2]
|
||||
safe_depth = np.where(
|
||||
np.abs(triangle_depth) > 1e-8,
|
||||
triangle_depth,
|
||||
np.nan,
|
||||
)
|
||||
triangle_rays = triangle_points / safe_depth[..., None]
|
||||
next_points = np.roll(triangle_points, -1, axis=1)
|
||||
next_rays = np.roll(triangle_rays, -1, axis=1)
|
||||
next_depth = np.roll(triangle_depth, -1, axis=1)
|
||||
local_depth = (triangle_depth + next_depth) * 0.5
|
||||
flat_edges = (
|
||||
triangle_rays * local_depth[..., None]
|
||||
- next_rays * local_depth[..., None]
|
||||
)
|
||||
flat_lengths = np.linalg.norm(flat_edges, axis=-1)
|
||||
actual_lengths = np.linalg.norm(
|
||||
triangle_points - next_points,
|
||||
axis=-1,
|
||||
)
|
||||
stretch = actual_lengths / np.maximum(flat_lengths, 1e-8)
|
||||
stretch[np.isclose(stretch, 1.0, rtol=1e-5, atol=1e-6)] = 1.0
|
||||
return stretch
|
||||
|
||||
|
||||
def image_mesh_attributes(
|
||||
vertices,
|
||||
faces,
|
||||
intrinsics,
|
||||
confidence_percentiles,
|
||||
face_depth_discontinuity,
|
||||
):
|
||||
import numpy as np
|
||||
|
||||
point_percentile = sample_image_nearest(
|
||||
confidence_percentiles,
|
||||
project_camera_pixels(vertices, intrinsics),
|
||||
).astype(np.float32)
|
||||
face_confidence = point_percentile[faces].min(axis=1)
|
||||
return {
|
||||
"point_depth": vertices[:, 2].astype(np.float32),
|
||||
"face_confidence_percentile": face_confidence.astype(np.float32),
|
||||
"face_depth_discontinuity": np.asarray(
|
||||
face_depth_discontinuity,
|
||||
dtype=np.float32,
|
||||
),
|
||||
}
|
||||
|
||||
|
||||
def export_prediction_attributes(prediction, output_directory):
|
||||
import numpy as np
|
||||
|
||||
values = {}
|
||||
for name in ("depth", "conf", "intrinsics", "extrinsics"):
|
||||
value = getattr(prediction, name, None)
|
||||
if value is not None:
|
||||
values[name] = np.asarray(value)
|
||||
if "depth" in values and "conf" in values:
|
||||
values["confidence_percentile"] = np.stack(
|
||||
[
|
||||
confidence_percentile_map(depth, confidence)
|
||||
for depth, confidence in zip(
|
||||
values["depth"],
|
||||
values["conf"],
|
||||
)
|
||||
]
|
||||
)
|
||||
output_path = Path(output_directory) / "prediction.attributes.npz"
|
||||
np.savez(output_path, **values)
|
||||
return output_path
|
||||
|
||||
|
||||
def export_da3_mini_npz(prediction, output_directory):
|
||||
import numpy as np
|
||||
|
||||
values = {"depth": np.round(prediction.depth, 8)}
|
||||
for name in ("conf", "extrinsics", "intrinsics"):
|
||||
value = getattr(prediction, name, None)
|
||||
if value is not None:
|
||||
values[name] = np.round(value, 2) if name == "conf" else value
|
||||
output_path = Path(output_directory) / "results.npz"
|
||||
np.savez_compressed(output_path, **values)
|
||||
return output_path
|
||||
|
||||
|
||||
def export_da3_depth_preview(prediction, output_directory):
|
||||
import numpy as np
|
||||
from PIL import Image
|
||||
|
||||
target = Path(output_directory) / "depth_vis"
|
||||
target.mkdir(parents=True, exist_ok=True)
|
||||
for index, depth in enumerate(prediction.depth):
|
||||
valid = np.isfinite(depth) & (depth > 0.0)
|
||||
normalized = np.zeros(depth.shape, dtype=np.float32)
|
||||
if valid.any():
|
||||
low, high = np.percentile(depth[valid], (2.0, 98.0))
|
||||
normalized[valid] = np.clip(
|
||||
(depth[valid] - low) / max(float(high - low), 1e-6),
|
||||
0.0,
|
||||
1.0,
|
||||
)
|
||||
color = np.stack(
|
||||
(
|
||||
normalized,
|
||||
1.0 - np.abs(normalized * 2.0 - 1.0),
|
||||
1.0 - normalized,
|
||||
),
|
||||
axis=-1,
|
||||
)
|
||||
color = np.asarray(color * 255.0, dtype=np.uint8)
|
||||
source = np.asarray(prediction.processed_images[index], dtype=np.uint8)
|
||||
combined = np.concatenate((source, color), axis=1)
|
||||
Image.fromarray(combined, mode="RGB").save(
|
||||
target / f"{index:04d}.jpg",
|
||||
quality=95,
|
||||
)
|
||||
|
||||
|
||||
def export_da3_point_cloud(
|
||||
prediction,
|
||||
output_directory,
|
||||
confidence_percentile=40.0,
|
||||
num_max_points=1_000_000,
|
||||
):
|
||||
import numpy as np
|
||||
import trimesh
|
||||
|
||||
depth = np.asarray(prediction.depth, dtype=np.float32)
|
||||
confidence = np.asarray(prediction.conf, dtype=np.float32)
|
||||
threshold = confidence_percentile_threshold(
|
||||
depth,
|
||||
confidence,
|
||||
confidence_percentile,
|
||||
)
|
||||
point_sets = []
|
||||
color_sets = []
|
||||
for index in range(depth.shape[0]):
|
||||
points = unproject_depth_frame(
|
||||
depth[index],
|
||||
prediction.intrinsics[index],
|
||||
prediction.extrinsics[index],
|
||||
)
|
||||
valid = np.isfinite(points).all(axis=-1) & (depth[index] > 0.0)
|
||||
if threshold is not None:
|
||||
valid &= confidence[index] >= threshold
|
||||
point_sets.append(points[valid])
|
||||
color_sets.append(prediction.processed_images[index][valid])
|
||||
points = np.concatenate(point_sets, axis=0)
|
||||
colors = np.concatenate(color_sets, axis=0)
|
||||
maximum = max(int(num_max_points), 1)
|
||||
if len(points) > maximum:
|
||||
indices = np.linspace(0, len(points) - 1, maximum, dtype=np.int64)
|
||||
points = points[indices]
|
||||
colors = colors[indices]
|
||||
scene = trimesh.Scene(trimesh.points.PointCloud(points, colors=colors))
|
||||
scene.export(Path(output_directory) / SCENE_FILENAME)
|
||||
|
||||
|
||||
def remove_background_faces(faces, foreground_mask):
|
||||
"""Return only triangles whose vertices all belong to the foreground."""
|
||||
foreground_points = foreground_mask.reshape(-1)
|
||||
return faces[foreground_points[faces].all(axis=1)]
|
||||
|
||||
|
||||
def export_image_mesh_data(
|
||||
prediction,
|
||||
image_path,
|
||||
output_directory,
|
||||
foreground_path=None,
|
||||
progress_callback=None,
|
||||
):
|
||||
import numpy as np
|
||||
from PIL import Image
|
||||
|
||||
total_steps = 6
|
||||
|
||||
def progress(step, label):
|
||||
if progress_callback is not None:
|
||||
progress_callback(step, total_steps, label)
|
||||
|
||||
progress(1, "Validating DA3 data")
|
||||
required = ("depth", "conf", "intrinsics")
|
||||
missing = [
|
||||
name
|
||||
for name in required
|
||||
if getattr(prediction, name, None) is None
|
||||
]
|
||||
if missing:
|
||||
raise ValueError(f"DA3 mesh output is missing: {', '.join(missing)}")
|
||||
depth_frames = np.asarray(prediction.depth, dtype=np.float32)
|
||||
confidence_frames = np.asarray(prediction.conf, dtype=np.float32)
|
||||
if depth_frames.shape[0] != 1:
|
||||
raise ValueError("Convert to Surface currently supports one image")
|
||||
raw_depth = depth_frames[0]
|
||||
confidence = confidence_frames[0]
|
||||
intrinsics = np.asarray(prediction.intrinsics[0], dtype=np.float32)
|
||||
progress(2, "Unprojecting the depth grid")
|
||||
points = unproject_camera_depth(raw_depth, intrinsics)
|
||||
progress(3, "Preparing confidence and background data")
|
||||
foreground_mask = np.ones(raw_depth.shape, dtype=bool)
|
||||
if foreground_path is not None:
|
||||
foreground_mask = foreground_mesh_mask(
|
||||
foreground_path,
|
||||
raw_depth.shape,
|
||||
)
|
||||
confidence_percentiles = confidence_percentile_map(
|
||||
raw_depth,
|
||||
confidence,
|
||||
)
|
||||
progress(4, "Building source triangles")
|
||||
faces = mesh_triangle_faces(
|
||||
points,
|
||||
raw_depth,
|
||||
confidence,
|
||||
None,
|
||||
False,
|
||||
0.0,
|
||||
)
|
||||
if foreground_path is not None:
|
||||
faces = remove_background_faces(faces, foreground_mask)
|
||||
if not len(faces):
|
||||
raise ValueError(
|
||||
"DA3 output does not contain any valid surface faces"
|
||||
)
|
||||
progress(5, "Preparing source UV and attributes")
|
||||
depth_discontinuity = depth_discontinuity_ratio(raw_depth)
|
||||
face_depth_discontinuity = depth_discontinuity.reshape(-1)[faces].max(axis=1)
|
||||
vertices, faces = compact_mesh(
|
||||
points.reshape(-1, 3),
|
||||
faces,
|
||||
)
|
||||
height, width = raw_depth.shape
|
||||
uv = project_camera_uv(vertices, intrinsics, (width, height))
|
||||
attributes = image_mesh_attributes(
|
||||
vertices,
|
||||
faces,
|
||||
intrinsics,
|
||||
confidence_percentiles,
|
||||
face_depth_discontinuity,
|
||||
)
|
||||
reference_depth = float(np.median(vertices[:, 2]))
|
||||
output_directory = Path(output_directory)
|
||||
output_directory.mkdir(parents=True, exist_ok=True)
|
||||
progress(6, "Writing source surface data")
|
||||
result_path = output_directory / "mesh.npz"
|
||||
np.savez_compressed(
|
||||
result_path,
|
||||
vertices=vertices,
|
||||
faces=faces,
|
||||
uv=uv,
|
||||
intrinsics=intrinsics,
|
||||
image_size=np.asarray((width, height), dtype=np.int32),
|
||||
reference_depth=np.asarray(reference_depth, dtype=np.float32),
|
||||
**attributes,
|
||||
)
|
||||
texture_path = output_directory / "texture.png"
|
||||
texture_source = foreground_path or image_path
|
||||
with Image.open(texture_source) as image:
|
||||
image.convert("RGBA").save(texture_path, format="PNG", compress_level=1)
|
||||
return result_path, texture_path
|
||||
|
||||
|
||||
def depth_edge_mask(depth, threshold=12.0 / 255.0):
|
||||
import numpy as np
|
||||
|
||||
values = np.asarray(depth, dtype=np.float32)
|
||||
edges = np.zeros(values.shape, dtype=bool)
|
||||
for axis in (0, 1):
|
||||
first = np.take(values, range(values.shape[axis] - 1), axis=axis)
|
||||
second = np.take(values, range(1, values.shape[axis]), axis=axis)
|
||||
valid = (
|
||||
np.isfinite(first)
|
||||
& np.isfinite(second)
|
||||
& (first > 0.0)
|
||||
& (second > 0.0)
|
||||
)
|
||||
scale = np.maximum(np.minimum(np.abs(first), np.abs(second)), 1e-6)
|
||||
discontinuity = valid & (np.abs(first - second) / scale > threshold)
|
||||
lower = [slice(None), slice(None)]
|
||||
upper = [slice(None), slice(None)]
|
||||
lower[axis] = slice(0, -1)
|
||||
upper[axis] = slice(1, None)
|
||||
edges[tuple(lower)] |= discontinuity
|
||||
edges[tuple(upper)] |= discontinuity
|
||||
return edges
|
||||
|
||||
|
||||
def edge_culling_stretch_limit(culling):
|
||||
factor = min(max(float(culling) / 100.0, 0.0001), 1.0)
|
||||
return 1.0 / factor
|
||||
|
||||
|
||||
def foreground_mesh_mask(foreground_path, depth_shape):
|
||||
import numpy as np
|
||||
from PIL import Image, ImageFilter
|
||||
|
||||
height, width = depth_shape
|
||||
with Image.open(foreground_path) as source:
|
||||
alpha = source.convert("RGBA").getchannel("A")
|
||||
alpha = alpha.resize(
|
||||
(width, height),
|
||||
resample=Image.Resampling.BILINEAR,
|
||||
)
|
||||
alpha = alpha.filter(ImageFilter.GaussianBlur(radius=0.75))
|
||||
return np.asarray(alpha) >= 128
|
||||
|
||||
|
||||
def confidence_percentile_threshold(depth, confidence, percentile):
|
||||
import numpy as np
|
||||
|
||||
if float(percentile) <= 0.0:
|
||||
return None
|
||||
valid = np.isfinite(depth) & (depth > 0.0) & np.isfinite(confidence)
|
||||
if not valid.any():
|
||||
raise ValueError("DA3 returned no valid depth confidence values")
|
||||
return float(np.percentile(confidence[valid], percentile))
|
||||
|
||||
|
||||
def mesh_triangle_faces(
|
||||
points,
|
||||
depth,
|
||||
confidence,
|
||||
confidence_threshold,
|
||||
filter_edges,
|
||||
max_edge_length,
|
||||
edge_threshold=12.0 / 255.0,
|
||||
edge_stretch_limit=None,
|
||||
vertex_mask=None,
|
||||
):
|
||||
import numpy as np
|
||||
|
||||
height, width = depth.shape
|
||||
indices = np.arange(height * width, dtype=np.int32).reshape(height, width)
|
||||
quads = np.stack(
|
||||
(
|
||||
indices[:-1, :-1],
|
||||
indices[1:, :-1],
|
||||
indices[1:, 1:],
|
||||
indices[:-1, 1:],
|
||||
),
|
||||
axis=-1,
|
||||
).reshape(-1, 4)
|
||||
valid = (
|
||||
np.isfinite(points).all(axis=-1)
|
||||
& np.isfinite(depth)
|
||||
& (depth > 0.0)
|
||||
)
|
||||
if confidence_threshold is not None:
|
||||
valid &= np.isfinite(confidence) & (
|
||||
confidence >= confidence_threshold
|
||||
)
|
||||
if filter_edges:
|
||||
valid &= ~depth_edge_mask(depth, edge_threshold)
|
||||
if vertex_mask is not None:
|
||||
if np.shape(vertex_mask) != depth.shape:
|
||||
raise ValueError("Foreground mask and depth dimensions do not match")
|
||||
valid &= np.asarray(vertex_mask, dtype=bool)
|
||||
triangles = np.concatenate(
|
||||
(quads[:, (0, 1, 2)], quads[:, (0, 2, 3)]),
|
||||
axis=0,
|
||||
)
|
||||
face_valid = valid.reshape(-1)[triangles].all(axis=1)
|
||||
if edge_stretch_limit is not None:
|
||||
face_valid &= triangle_edge_stretch_mask(
|
||||
points,
|
||||
triangles,
|
||||
edge_stretch_limit,
|
||||
)
|
||||
if max_edge_length > 0.0:
|
||||
flat_points = points.reshape(-1, 3)
|
||||
triangle_points = flat_points[triangles]
|
||||
next_points = np.roll(triangle_points, -1, axis=1)
|
||||
edge_lengths = np.linalg.norm(
|
||||
triangle_points - next_points,
|
||||
axis=-1,
|
||||
)
|
||||
face_valid &= (edge_lengths <= max_edge_length).all(axis=1)
|
||||
triangles = triangles[face_valid]
|
||||
if not len(triangles):
|
||||
return np.empty((0, 3), dtype=np.int32)
|
||||
return triangles
|
||||
|
||||
|
||||
def triangle_edge_stretch_mask(points, triangles, stretch_limit):
|
||||
stretch = triangle_edge_stretch_values(points, triangles)
|
||||
tolerance = float(stretch_limit) * (1.0 + 1e-5)
|
||||
return (stretch <= tolerance).all(axis=1)
|
||||
|
||||
|
||||
def export_textured_mesh_glb(
|
||||
prediction,
|
||||
image_paths,
|
||||
output_directory,
|
||||
confidence_percentile,
|
||||
filter_edges,
|
||||
max_edge_length,
|
||||
):
|
||||
import numpy as np
|
||||
import trimesh
|
||||
from PIL import Image
|
||||
|
||||
required = ("depth", "conf", "intrinsics", "extrinsics")
|
||||
missing = [name for name in required if getattr(prediction, name) is None]
|
||||
if missing:
|
||||
raise ValueError(f"DA3 mesh output is missing: {', '.join(missing)}")
|
||||
depth = np.asarray(prediction.depth, dtype=np.float32)
|
||||
confidence = np.asarray(prediction.conf, dtype=np.float32)
|
||||
threshold = confidence_percentile_threshold(
|
||||
depth,
|
||||
confidence,
|
||||
confidence_percentile,
|
||||
)
|
||||
meshes = []
|
||||
referenced_points = []
|
||||
for index in range(depth.shape[0]):
|
||||
points = unproject_depth_frame(
|
||||
depth[index],
|
||||
prediction.intrinsics[index],
|
||||
prediction.extrinsics[index],
|
||||
)
|
||||
faces = mesh_triangle_faces(
|
||||
points,
|
||||
depth[index],
|
||||
confidence[index],
|
||||
threshold,
|
||||
filter_edges,
|
||||
max_edge_length,
|
||||
)
|
||||
if not len(faces):
|
||||
continue
|
||||
height, width = depth[index].shape
|
||||
u = np.linspace(0.0, 1.0, width, dtype=np.float32)
|
||||
v = np.linspace(1.0, 0.0, height, dtype=np.float32)
|
||||
uu, vv = np.meshgrid(u, v)
|
||||
uv = np.stack((uu, vv), axis=-1).reshape(-1, 2)
|
||||
with Image.open(image_paths[index]) as source_image:
|
||||
texture_image = source_image.convert("RGB").copy()
|
||||
visual = trimesh.visual.texture.TextureVisuals(
|
||||
uv=uv,
|
||||
image=texture_image,
|
||||
)
|
||||
mesh = trimesh.Trimesh(
|
||||
vertices=points.reshape(-1, 3),
|
||||
faces=faces,
|
||||
visual=visual,
|
||||
process=False,
|
||||
)
|
||||
mesh.vertex_attributes["o_depth"] = depth[index].reshape(-1)
|
||||
mesh.vertex_attributes["o_confidence"] = confidence[
|
||||
index
|
||||
].reshape(-1)
|
||||
mesh.vertex_attributes["o_confidence_percentile"] = (
|
||||
confidence_percentile_map(
|
||||
depth[index],
|
||||
confidence[index],
|
||||
).reshape(-1)
|
||||
)
|
||||
mesh.remove_unreferenced_vertices()
|
||||
meshes.append(mesh)
|
||||
referenced_points.append(mesh.vertices)
|
||||
if not meshes:
|
||||
raise ValueError(
|
||||
"Surface filters removed every face; lower Confidence Threshold or disable filters"
|
||||
)
|
||||
|
||||
world_to_camera = np.asarray(prediction.extrinsics[0], dtype=np.float64)
|
||||
if world_to_camera.shape == (3, 4):
|
||||
homogeneous = np.eye(4, dtype=np.float64)
|
||||
homogeneous[:3, :4] = world_to_camera
|
||||
world_to_camera = homogeneous
|
||||
cv_to_gltf = np.eye(4, dtype=np.float64)
|
||||
cv_to_gltf[1, 1] = -1.0
|
||||
cv_to_gltf[2, 2] = -1.0
|
||||
alignment = cv_to_gltf @ world_to_camera
|
||||
aligned_points = trimesh.transform_points(
|
||||
np.concatenate(referenced_points, axis=0),
|
||||
alignment,
|
||||
)
|
||||
center = np.median(aligned_points, axis=0)
|
||||
alignment[:3, 3] -= center
|
||||
|
||||
scene = trimesh.Scene()
|
||||
for index, mesh in enumerate(meshes):
|
||||
mesh.apply_transform(alignment)
|
||||
scene.add_geometry(mesh, node_name=f"DepthMesh.{index + 1:04d}")
|
||||
output_directory = Path(output_directory)
|
||||
output_directory.mkdir(parents=True, exist_ok=True)
|
||||
output_path = output_directory / "mesh.glb"
|
||||
scene.export(output_path)
|
||||
return output_path
|
||||
306
src/snapform/server/media.py
Normal file
306
src/snapform/server/media.py
Normal file
@@ -0,0 +1,306 @@
|
||||
import shutil
|
||||
import tempfile
|
||||
import time
|
||||
from pathlib import Path
|
||||
|
||||
|
||||
IMAGE_SUFFIXES = {".jpg", ".jpeg", ".png", ".webp"}
|
||||
VIDEO_SUFFIXES = {
|
||||
".mp4",
|
||||
".mov",
|
||||
".avi",
|
||||
".mkv",
|
||||
".webm",
|
||||
".m4v",
|
||||
".mts",
|
||||
".mpg",
|
||||
".mpeg",
|
||||
}
|
||||
SCENE_FILENAME = "scene.glb"
|
||||
|
||||
|
||||
class TaskCancelled(RuntimeError):
|
||||
pass
|
||||
|
||||
|
||||
def check_cancelled(cancel_check):
|
||||
if cancel_check is not None and cancel_check():
|
||||
raise TaskCancelled("Task cancelled")
|
||||
|
||||
|
||||
def collect_images(directory):
|
||||
directory = Path(directory)
|
||||
return sorted(
|
||||
path
|
||||
for path in directory.iterdir()
|
||||
if path.is_file() and path.suffix.lower() in IMAGE_SUFFIXES
|
||||
)
|
||||
|
||||
|
||||
def first_frames(images, maximum_frames, start_frame=1):
|
||||
"""Return up to maximum_frames image paths in order from start_frame."""
|
||||
if maximum_frames <= 0:
|
||||
return images
|
||||
start_index = max(start_frame - 1, 0)
|
||||
return images[start_index:start_index + maximum_frames]
|
||||
|
||||
|
||||
def extract_video_frames(
|
||||
video_path,
|
||||
output_directory,
|
||||
maximum_frames,
|
||||
av,
|
||||
start_frame=1,
|
||||
cancel_check=None,
|
||||
):
|
||||
output_directory = Path(output_directory)
|
||||
output_directory.mkdir(parents=True, exist_ok=True)
|
||||
try:
|
||||
container = av.open(str(video_path))
|
||||
except Exception as error:
|
||||
raise RuntimeError(f"Unable to open video: {video_path}") from error
|
||||
image_paths = []
|
||||
skipped = 0
|
||||
try:
|
||||
stream = container.streams.video[0]
|
||||
for frame in container.decode(stream):
|
||||
check_cancelled(cancel_check)
|
||||
if skipped < start_frame - 1:
|
||||
skipped += 1
|
||||
continue
|
||||
image_path = output_directory / f"frame.{len(image_paths) + 1:04d}.jpg"
|
||||
frame.to_image().save(image_path, format="JPEG")
|
||||
image_paths.append(image_path)
|
||||
if len(image_paths) >= maximum_frames:
|
||||
break
|
||||
finally:
|
||||
container.close()
|
||||
|
||||
if not image_paths:
|
||||
raise RuntimeError(f"Unable to read any video frame from: {video_path}")
|
||||
return image_paths
|
||||
|
||||
|
||||
def prepare_input(
|
||||
input_path,
|
||||
video_frames,
|
||||
temporary_root,
|
||||
start_frame=1,
|
||||
cancel_check=None,
|
||||
):
|
||||
check_cancelled(cancel_check)
|
||||
input_path = Path(input_path)
|
||||
if input_path.is_dir():
|
||||
images = collect_images(input_path)
|
||||
if not images:
|
||||
raise RuntimeError(f"No image frames found in: {input_path}")
|
||||
return first_frames(images, video_frames, start_frame), None
|
||||
suffix = input_path.suffix.lower()
|
||||
if suffix in IMAGE_SUFFIXES:
|
||||
return [input_path], None
|
||||
if suffix not in VIDEO_SUFFIXES:
|
||||
supported = ", ".join(sorted(IMAGE_SUFFIXES | VIDEO_SUFFIXES))
|
||||
raise RuntimeError(f"Unsupported input type; expected one of: {supported}")
|
||||
|
||||
import av
|
||||
|
||||
temporary_directory = Path(
|
||||
tempfile.mkdtemp(prefix="snapform-video-", dir=temporary_root)
|
||||
)
|
||||
try:
|
||||
images = extract_video_frames(
|
||||
input_path,
|
||||
temporary_directory,
|
||||
video_frames,
|
||||
av,
|
||||
start_frame=start_frame,
|
||||
cancel_check=cancel_check,
|
||||
)
|
||||
except Exception:
|
||||
shutil.rmtree(temporary_directory, ignore_errors=True)
|
||||
raise
|
||||
return images, temporary_directory
|
||||
|
||||
|
||||
def prepare_background_input(
|
||||
input_path,
|
||||
video_frames,
|
||||
temporary_root,
|
||||
start_frame=1,
|
||||
cancel_check=None,
|
||||
):
|
||||
"""Resolve a BEN2 input to frame images; returns (images, temporary_directory)."""
|
||||
input_path = Path(input_path)
|
||||
if input_path.is_dir():
|
||||
images = collect_images(input_path)
|
||||
if not images:
|
||||
raise RuntimeError(f"No image frames found in: {input_path}")
|
||||
return first_frames(images, video_frames, start_frame), None
|
||||
if input_path.suffix.lower() in IMAGE_SUFFIXES:
|
||||
return [input_path], None
|
||||
return prepare_input(
|
||||
input_path,
|
||||
video_frames,
|
||||
temporary_root,
|
||||
start_frame=start_frame,
|
||||
cancel_check=cancel_check,
|
||||
)
|
||||
|
||||
|
||||
def copy_frames(images, output_directory):
|
||||
"""Copy a multi-frame input as numbered frame files for the Blender sequence."""
|
||||
if len(images) <= 1:
|
||||
return
|
||||
for index, image_path in enumerate(images):
|
||||
suffix = Path(image_path).suffix.lower() or ".jpg"
|
||||
destination = (
|
||||
Path(output_directory) / f"frame.{index + 1:04d}{suffix}"
|
||||
)
|
||||
if Path(image_path).resolve() == destination.resolve():
|
||||
continue
|
||||
shutil.copy2(image_path, destination)
|
||||
|
||||
|
||||
def extract_frames(context, parameters):
|
||||
"""Copy selected source frames into the current job directory."""
|
||||
context.directory.mkdir(parents=True, exist_ok=True)
|
||||
images, temporary_directory = prepare_input(
|
||||
parameters["input"],
|
||||
parameters.get("video_frames", 16),
|
||||
context.directory,
|
||||
start_frame=parameters.get("start_frame", 1),
|
||||
cancel_check=context.check_cancelled,
|
||||
)
|
||||
try:
|
||||
copy_frames(images, context.directory)
|
||||
finally:
|
||||
if temporary_directory is not None:
|
||||
shutil.rmtree(temporary_directory, ignore_errors=True)
|
||||
message = f"Extracted {len(images)} frame(s)"
|
||||
context.progress(1.0, message)
|
||||
return {"frames": len(images)}
|
||||
|
||||
|
||||
def exported_paths(output_directory, export_format):
|
||||
output_directory = Path(output_directory)
|
||||
if export_format == "glb":
|
||||
scene_path = output_directory / SCENE_FILENAME
|
||||
return [scene_path] if scene_path.is_file() else []
|
||||
if export_format == "gs_ply":
|
||||
return sorted((output_directory / "gs_ply").glob("*.ply"))
|
||||
if export_format == "depth_vis":
|
||||
return sorted((output_directory / "depth_vis").glob("*.jpg"))
|
||||
if export_format == "mesh":
|
||||
result_path = output_directory / "mesh.glb"
|
||||
return [result_path] if result_path.is_file() else []
|
||||
if export_format == "mesh_data":
|
||||
result_path = output_directory / "mesh.npz"
|
||||
return [result_path] if result_path.is_file() else []
|
||||
if export_format == "mini_npz":
|
||||
result_path = output_directory / "results.npz"
|
||||
if result_path.is_file():
|
||||
return [result_path]
|
||||
result_path = (
|
||||
output_directory / "exports" / "mini_npz" / "results.npz"
|
||||
)
|
||||
return [result_path] if result_path.is_file() else []
|
||||
return []
|
||||
|
||||
|
||||
def npz_is_complete(path):
|
||||
"""Return True only when the npz archive can be fully read."""
|
||||
try:
|
||||
import numpy as np
|
||||
|
||||
with np.load(path, allow_pickle=False) as archive:
|
||||
return "depth" in archive.files
|
||||
except Exception:
|
||||
return False
|
||||
|
||||
|
||||
def wait_for_exported_paths(
|
||||
output_directory,
|
||||
export_format,
|
||||
timeout=30.0,
|
||||
cancel_check=None,
|
||||
):
|
||||
# DA3 的 NPZ exporter 在后台线程写入,文件出现时可能尚未写完整,
|
||||
# 直接返回会让下游读到半截 zip 而报 EOFError,因此需等文件可完整读取
|
||||
deadline = time.monotonic() + timeout
|
||||
while time.monotonic() < deadline:
|
||||
check_cancelled(cancel_check)
|
||||
result_paths = exported_paths(output_directory, export_format)
|
||||
if result_paths:
|
||||
if (
|
||||
export_format in ("mini_npz", "npz")
|
||||
and not npz_is_complete(result_paths[0])
|
||||
):
|
||||
time.sleep(0.1)
|
||||
continue
|
||||
return result_paths
|
||||
time.sleep(0.05)
|
||||
return []
|
||||
|
||||
|
||||
def move_exported_file(
|
||||
source,
|
||||
destination,
|
||||
timeout=10.0,
|
||||
cancel_check=None,
|
||||
):
|
||||
"""Move an async export after Windows releases its file handle."""
|
||||
source = Path(source)
|
||||
destination = Path(destination)
|
||||
deadline = time.monotonic() + timeout
|
||||
while True:
|
||||
check_cancelled(cancel_check)
|
||||
try:
|
||||
source.replace(destination)
|
||||
return destination
|
||||
except OSError as error:
|
||||
is_file_lock = isinstance(error, PermissionError) or getattr(
|
||||
error,
|
||||
"winerror",
|
||||
None,
|
||||
) in {32, 33}
|
||||
if not is_file_lock or time.monotonic() >= deadline:
|
||||
raise
|
||||
time.sleep(0.1)
|
||||
|
||||
|
||||
def flatten_exported_paths(
|
||||
output_directory,
|
||||
result_paths,
|
||||
cancel_check=None,
|
||||
):
|
||||
"""Move files produced in exporter subdirectories to the job root."""
|
||||
output_directory = Path(output_directory)
|
||||
flattened_paths = []
|
||||
for source in result_paths:
|
||||
source = Path(source)
|
||||
check_cancelled(cancel_check)
|
||||
if source.parent == output_directory:
|
||||
flattened_paths.append(source)
|
||||
continue
|
||||
destination = output_directory / source.name
|
||||
if destination.exists():
|
||||
raise RuntimeError(f"Export filename already exists: {destination.name}")
|
||||
flattened_paths.append(
|
||||
move_exported_file(
|
||||
source,
|
||||
destination,
|
||||
cancel_check=cancel_check,
|
||||
)
|
||||
)
|
||||
for directory in sorted(
|
||||
{path.parent for path in map(Path, result_paths)},
|
||||
key=lambda path: len(path.parts),
|
||||
reverse=True,
|
||||
):
|
||||
while directory != output_directory:
|
||||
try:
|
||||
directory.rmdir()
|
||||
except OSError:
|
||||
break
|
||||
directory = directory.parent
|
||||
return flattened_paths
|
||||
445
src/snapform/server/model_download.py
Normal file
445
src/snapform/server/model_download.py
Normal file
@@ -0,0 +1,445 @@
|
||||
import hashlib
|
||||
import http.client
|
||||
import os
|
||||
import threading
|
||||
import time
|
||||
from pathlib import Path
|
||||
from urllib.parse import quote
|
||||
from urllib.request import (
|
||||
HTTPSHandler,
|
||||
ProxyHandler,
|
||||
Request,
|
||||
build_opener,
|
||||
urlopen,
|
||||
)
|
||||
|
||||
HF_ENDPOINT = "https://huggingface.co"
|
||||
R2_ENDPOINT = "https://models.omoolab.xyz"
|
||||
CONNECT_TIMEOUT = 10.0
|
||||
READ_TIMEOUT = 60.0
|
||||
DOWNLOAD_CHUNK_SIZE = 256 * 1024
|
||||
|
||||
|
||||
class _ShortConnectHTTPSConnection(http.client.HTTPSConnection):
|
||||
"""Bound connection and idle reads without limiting total download time."""
|
||||
|
||||
def connect(self):
|
||||
try:
|
||||
self.timeout = CONNECT_TIMEOUT
|
||||
super().connect()
|
||||
finally:
|
||||
self.timeout = None
|
||||
self.sock.settimeout(READ_TIMEOUT)
|
||||
|
||||
|
||||
class _ShortConnectHTTPSHandler(HTTPSHandler):
|
||||
def https_open(self, request):
|
||||
return self.do_open(_ShortConnectHTTPSConnection, request)
|
||||
|
||||
|
||||
_download_opener = build_opener(ProxyHandler(), _ShortConnectHTTPSHandler())
|
||||
|
||||
|
||||
class DownloadProgressReporter:
|
||||
def __init__(
|
||||
self,
|
||||
progress_callback,
|
||||
repository,
|
||||
clock=time.monotonic,
|
||||
start=0.05,
|
||||
span=0.90,
|
||||
label=None,
|
||||
):
|
||||
self.progress_callback = progress_callback
|
||||
self.repository = repository
|
||||
self.clock = clock
|
||||
self.lock = threading.Lock()
|
||||
self.start = start
|
||||
self.span = span
|
||||
self.label = label
|
||||
self.progress = start
|
||||
self.last_write_time = 0.0
|
||||
self.started_at = self.clock()
|
||||
|
||||
def report(self, current, total, rate=None):
|
||||
if not total or current <= 0:
|
||||
return
|
||||
|
||||
factor = min(max(current / total, 0.0), 1.0)
|
||||
progress = self.start + factor * self.span
|
||||
now = self.clock()
|
||||
with self.lock:
|
||||
if progress <= self.progress:
|
||||
return
|
||||
if now - self.last_write_time < 0.20 and factor < 1.0:
|
||||
return
|
||||
self.progress = progress
|
||||
self.last_write_time = now
|
||||
effective_rate = rate
|
||||
elapsed = now - self.started_at
|
||||
if not effective_rate and elapsed > 0:
|
||||
effective_rate = current / elapsed
|
||||
speed = format_rate(effective_rate) if effective_rate else ""
|
||||
prefix = self.label or self.repository
|
||||
message = f"Downloading {prefix}"
|
||||
if speed:
|
||||
message = f"{message} {speed}"
|
||||
self.progress_callback(
|
||||
progress,
|
||||
message,
|
||||
)
|
||||
|
||||
|
||||
def create_download_progress_class(
|
||||
progress_callback,
|
||||
repository,
|
||||
tqdm_type=None,
|
||||
clock=time.monotonic,
|
||||
start=0.05,
|
||||
span=0.90,
|
||||
label=None,
|
||||
):
|
||||
if tqdm_type is None:
|
||||
from tqdm.auto import tqdm as tqdm_type
|
||||
|
||||
reporter = DownloadProgressReporter(
|
||||
progress_callback,
|
||||
repository,
|
||||
clock,
|
||||
start,
|
||||
span,
|
||||
label,
|
||||
)
|
||||
|
||||
class DownloadProgress(tqdm_type):
|
||||
def __init__(self, *args, **kwargs):
|
||||
description = str(kwargs.get("desc", ""))
|
||||
# huggingface_hub 1.x 的字节进度条描述是 "Downloading (incomplete total...)",
|
||||
# 用前缀匹配而非精确匹配,否则真实下载进度会被忽略。
|
||||
self.snapform_reports_progress = (
|
||||
description.startswith("Downloading")
|
||||
or description.startswith("Fetching ")
|
||||
)
|
||||
super().__init__(*args, **kwargs)
|
||||
|
||||
def update(self, amount=1):
|
||||
result = super().update(amount)
|
||||
if self.snapform_reports_progress:
|
||||
reporter.report(self.n, self.total, self._download_rate())
|
||||
return result
|
||||
|
||||
def refresh(self, *args, **kwargs):
|
||||
result = super().refresh(*args, **kwargs)
|
||||
if self.snapform_reports_progress:
|
||||
reporter.report(self.n, self.total, self._download_rate())
|
||||
return result
|
||||
|
||||
def _download_rate(self):
|
||||
try:
|
||||
return getattr(self, "format_dict", {}).get("rate")
|
||||
except Exception:
|
||||
return None
|
||||
|
||||
return DownloadProgress
|
||||
|
||||
|
||||
def progress_range(args):
|
||||
start = getattr(args, "progress_start", None)
|
||||
end = getattr(args, "progress_end", None)
|
||||
if start is not None and end is not None:
|
||||
return float(start), float(end) - float(start)
|
||||
return 0.05, 0.90
|
||||
|
||||
|
||||
def download_model(args):
|
||||
label = getattr(args, "label", None) or args.repository
|
||||
model_dir = Path(args.model_dir)
|
||||
model_dir.mkdir(parents=True, exist_ok=True)
|
||||
start, span = progress_range(args)
|
||||
r2_directory = getattr(args, "r2_directory", "")
|
||||
r2_files = getattr(args, "r2_file", [])
|
||||
if r2_directory and r2_files:
|
||||
_download_declared_or_fail(
|
||||
args,
|
||||
model_dir,
|
||||
label,
|
||||
start,
|
||||
span,
|
||||
)
|
||||
return
|
||||
_download_snapshot_or_fail(args, model_dir, label, start, span)
|
||||
|
||||
|
||||
def _download_declared_or_fail(args, model_dir, label, start, span):
|
||||
"""Download declared files from Hugging Face, then the R2 mirror."""
|
||||
sources = (
|
||||
(HF_ENDPOINT, download_from_hf, f"Downloading {label}"),
|
||||
(
|
||||
R2_ENDPOINT,
|
||||
download_from_r2,
|
||||
f"Downloading {label} (mirror)",
|
||||
),
|
||||
)
|
||||
failures = []
|
||||
for source, downloader, message in sources:
|
||||
args.progress(start, message)
|
||||
try:
|
||||
downloader(
|
||||
args,
|
||||
model_dir,
|
||||
start=start,
|
||||
span=span,
|
||||
label=label,
|
||||
)
|
||||
_verify_model_complete(
|
||||
model_dir,
|
||||
getattr(args, "required_pattern", None),
|
||||
)
|
||||
except Exception as error:
|
||||
cancel_check = getattr(args, "cancel_check", None)
|
||||
if cancel_check is not None:
|
||||
cancel_check()
|
||||
failures.append((source, error))
|
||||
continue
|
||||
args.progress(start + span, f"{label} ready")
|
||||
return
|
||||
detail = "; ".join(
|
||||
f"{source}: {error}" for source, error in failures
|
||||
)
|
||||
raise RuntimeError(f"All download sources failed: {detail}")
|
||||
|
||||
|
||||
def _download_snapshot_or_fail(args, model_dir, label, start, span):
|
||||
"""Fallback download for repositories without declared files, via huggingface_hub."""
|
||||
failures = []
|
||||
try:
|
||||
from huggingface_hub import snapshot_download
|
||||
except Exception as error:
|
||||
snapshot_download = None
|
||||
failures.append((HF_ENDPOINT, error))
|
||||
progress_class = None
|
||||
if snapshot_download:
|
||||
progress_class = create_download_progress_class(
|
||||
args.progress,
|
||||
args.repository,
|
||||
start=start,
|
||||
span=span,
|
||||
label=label,
|
||||
)
|
||||
for endpoint in download_endpoints() if snapshot_download else ():
|
||||
os.environ["HF_ENDPOINT"] = endpoint
|
||||
# snapshot_download 先拉取文件清单并校验 ETag,还没有字节下载,
|
||||
# 用 Checking 提示当前阶段,避免进度条看起来卡住。
|
||||
message = f"Checking {label} files"
|
||||
if endpoint != HF_ENDPOINT:
|
||||
message = f"{message} (mirror)"
|
||||
args.progress(start, message)
|
||||
try:
|
||||
snapshot_download(
|
||||
repo_id=args.repository,
|
||||
local_dir=model_dir,
|
||||
tqdm_class=progress_class,
|
||||
allow_patterns=args.allow_pattern or None,
|
||||
endpoint=endpoint,
|
||||
)
|
||||
_verify_model_complete(
|
||||
model_dir,
|
||||
getattr(args, "required_pattern", None),
|
||||
)
|
||||
except Exception as error:
|
||||
cancel_check = getattr(args, "cancel_check", None)
|
||||
if cancel_check is not None:
|
||||
cancel_check()
|
||||
failures.append((endpoint, error))
|
||||
continue
|
||||
args.progress(start + span, f"{label} ready")
|
||||
return
|
||||
detail = "; ".join(
|
||||
f"{source}: {error}" for source, error in failures
|
||||
)
|
||||
raise RuntimeError(f"All download sources failed: {detail}")
|
||||
|
||||
|
||||
def download_endpoints():
|
||||
"""Return the configured Hugging Face endpoint or official source."""
|
||||
endpoint = os.environ.get("HF_ENDPOINT") or HF_ENDPOINT
|
||||
return [endpoint.rstrip("/")]
|
||||
|
||||
|
||||
def _verify_model_complete(model_dir, required_patterns=None):
|
||||
"""Refuse to report success when the required model files are missing."""
|
||||
model_dir = Path(model_dir)
|
||||
required_patterns = required_patterns or (
|
||||
"config.json",
|
||||
"*.safetensors",
|
||||
)
|
||||
missing = [
|
||||
pattern
|
||||
for pattern in required_patterns
|
||||
if not any(model_dir.glob(pattern))
|
||||
]
|
||||
if missing:
|
||||
raise RuntimeError(
|
||||
"Model download reported success but files are missing: "
|
||||
+ ", ".join(missing)
|
||||
)
|
||||
|
||||
|
||||
def _download_file_from_url(
|
||||
url,
|
||||
destination,
|
||||
expected_size,
|
||||
expected_checksum,
|
||||
progress=None,
|
||||
retries=1,
|
||||
cancel_check=None,
|
||||
):
|
||||
"""Download and verify one model file atomically from a direct URL."""
|
||||
request = Request(url, headers={"User-Agent": "SnapForm"})
|
||||
opener = _download_opener.open if request.type == "https" else urlopen
|
||||
last_error = None
|
||||
for _attempt in range(retries):
|
||||
temporary = destination.with_name(f"{destination.name}.part")
|
||||
try:
|
||||
checksum = hashlib.sha256()
|
||||
downloaded = 0
|
||||
with opener(request) as response:
|
||||
with open(temporary, "wb") as output:
|
||||
while True:
|
||||
if cancel_check is not None:
|
||||
cancel_check()
|
||||
chunk = response.read(DOWNLOAD_CHUNK_SIZE)
|
||||
if not chunk:
|
||||
break
|
||||
output.write(chunk)
|
||||
checksum.update(chunk)
|
||||
downloaded += len(chunk)
|
||||
if progress is not None:
|
||||
progress(len(chunk))
|
||||
if downloaded != expected_size:
|
||||
raise RuntimeError(
|
||||
f"Size mismatch for {destination.name}: "
|
||||
f"{downloaded} != {expected_size}"
|
||||
)
|
||||
if checksum.hexdigest().lower() != expected_checksum.lower():
|
||||
raise RuntimeError(f"SHA-256 mismatch for {destination.name}")
|
||||
temporary.replace(destination)
|
||||
return
|
||||
except Exception as error:
|
||||
if cancel_check is not None:
|
||||
cancel_check()
|
||||
last_error = error
|
||||
try:
|
||||
temporary.unlink()
|
||||
except OSError:
|
||||
pass
|
||||
raise last_error
|
||||
|
||||
|
||||
def r2_file_url(directory, filename):
|
||||
return f"{R2_ENDPOINT}/{quote(directory)}/{quote(filename)}"
|
||||
|
||||
|
||||
def hf_file_url(repository, filename):
|
||||
"""Hugging Face resolve URL for a direct, metadata-free download."""
|
||||
endpoint = (os.environ.get("HF_ENDPOINT") or HF_ENDPOINT).rstrip("/")
|
||||
return f"{endpoint}/{repository}/resolve/main/{quote(filename)}"
|
||||
|
||||
|
||||
def _parse_declared_files(args):
|
||||
selected = []
|
||||
for value in args.r2_file:
|
||||
filename, size, checksum = value.split("|", 2)
|
||||
selected.append((filename, int(size), checksum))
|
||||
return selected
|
||||
|
||||
|
||||
def _download_declared_files(
|
||||
args,
|
||||
model_dir,
|
||||
url_builder,
|
||||
start,
|
||||
span,
|
||||
label,
|
||||
retries=1,
|
||||
):
|
||||
"""Download each declared file through url_builder with aggregated progress."""
|
||||
repository = args.repository
|
||||
selected = _parse_declared_files(args)
|
||||
reporter = DownloadProgressReporter(
|
||||
args.progress,
|
||||
repository,
|
||||
start=start,
|
||||
span=span,
|
||||
label=label or repository,
|
||||
)
|
||||
total_size = sum(size for _path, size, _checksum in selected)
|
||||
downloaded = 0
|
||||
|
||||
def on_chunk(count):
|
||||
nonlocal downloaded
|
||||
downloaded += count
|
||||
reporter.report(downloaded, total_size)
|
||||
|
||||
for path, size, checksum in selected:
|
||||
destination = Path(model_dir) / path
|
||||
destination.parent.mkdir(parents=True, exist_ok=True)
|
||||
_download_file_from_url(
|
||||
url_builder(path),
|
||||
destination,
|
||||
size,
|
||||
checksum,
|
||||
progress=on_chunk,
|
||||
retries=retries,
|
||||
cancel_check=getattr(args, "cancel_check", None),
|
||||
)
|
||||
|
||||
|
||||
def download_from_r2(
|
||||
args,
|
||||
model_dir,
|
||||
start=0.05,
|
||||
span=0.90,
|
||||
label=None,
|
||||
):
|
||||
"""Download the declared model files from OmooLab R2."""
|
||||
_download_declared_files(
|
||||
args,
|
||||
model_dir,
|
||||
lambda filename: r2_file_url(args.r2_directory, filename),
|
||||
start,
|
||||
span,
|
||||
f"{label} (mirror)" if label else "mirror",
|
||||
)
|
||||
|
||||
|
||||
def download_from_hf(
|
||||
args,
|
||||
model_dir,
|
||||
start=0.05,
|
||||
span=0.90,
|
||||
label=None,
|
||||
):
|
||||
"""Download the declared model files directly from Hugging Face resolve URLs."""
|
||||
_download_declared_files(
|
||||
args,
|
||||
model_dir,
|
||||
lambda filename: hf_file_url(args.repository, filename),
|
||||
start,
|
||||
span,
|
||||
label,
|
||||
)
|
||||
|
||||
|
||||
def format_rate(rate):
|
||||
size = float(rate)
|
||||
if size <= 0:
|
||||
return ""
|
||||
units = ("B/s", "KB/s", "MB/s", "GB/s")
|
||||
unit_index = 0
|
||||
while size >= 1024 and unit_index < len(units) - 1:
|
||||
size /= 1024
|
||||
unit_index += 1
|
||||
if unit_index == 0:
|
||||
return f"{size:.0f} {units[unit_index]}"
|
||||
return f"{size:.1f} {units[unit_index]}"
|
||||
194
src/snapform/server/model_manager.py
Normal file
194
src/snapform/server/model_manager.py
Normal file
@@ -0,0 +1,194 @@
|
||||
import gc
|
||||
import time
|
||||
from pathlib import Path
|
||||
from types import SimpleNamespace
|
||||
|
||||
import model_catalog
|
||||
from .model_download import download_model
|
||||
|
||||
|
||||
class ModelSessionCache:
|
||||
"""Cache ONNX sessions shared by Jobs in one Server process."""
|
||||
|
||||
def __init__(self):
|
||||
self.ben2_model = None
|
||||
self.ben2_key = None
|
||||
self.da3_model = None
|
||||
self.da3_key = None
|
||||
self.moge_model = None
|
||||
self.moge_key = None
|
||||
|
||||
def get_ben2(self, model_dir, device):
|
||||
requested_device = getattr(device, "type", device)
|
||||
key = (str(Path(model_dir).resolve()), requested_device, "onnx")
|
||||
if self.ben2_model is not None and self.ben2_key == key:
|
||||
return self.ben2_model, 0.0
|
||||
self.release_ben2()
|
||||
return self._load_ben2(model_dir, requested_device, key)
|
||||
|
||||
def _load_ben2(self, model_dir, requested_device, key):
|
||||
from .onnx_ben2 import create_session
|
||||
|
||||
started = time.perf_counter()
|
||||
model = create_session(model_dir, requested_device)
|
||||
self.ben2_model = model
|
||||
self.ben2_key = key
|
||||
elapsed_ms = (time.perf_counter() - started) * 1000.0
|
||||
return model, elapsed_ms
|
||||
|
||||
def get_da3(self, model_dir, device):
|
||||
requested_device = getattr(device, "type", device)
|
||||
key = (str(Path(model_dir).resolve()), requested_device, "onnx")
|
||||
if self.moge_model is not None:
|
||||
self.release_moge()
|
||||
if self.da3_model is not None and self.da3_key == key:
|
||||
return self.da3_model
|
||||
self.release_da3()
|
||||
return self._load_da3(model_dir, requested_device, key)
|
||||
|
||||
def _load_da3(self, model_dir, requested_device, key):
|
||||
from .onnx_da3 import create_session
|
||||
|
||||
model = create_session(model_dir, requested_device)
|
||||
self.da3_model = model
|
||||
self.da3_key = key
|
||||
return model
|
||||
|
||||
def get_moge2(self, model_dir, requested_device):
|
||||
from .onnx_moge2 import create_session
|
||||
|
||||
key = (str(Path(model_dir).resolve()), requested_device, "onnx")
|
||||
if self.da3_model is not None:
|
||||
self.release_da3()
|
||||
if self.moge_model is not None and self.moge_key == key:
|
||||
return self.moge_model
|
||||
self.release_moge()
|
||||
model = create_session(model_dir, requested_device)
|
||||
self.moge_model = model
|
||||
self.moge_key = key
|
||||
return model
|
||||
|
||||
def release_ben2(self):
|
||||
self.ben2_model = None
|
||||
self.ben2_key = None
|
||||
self._release_runtime_cache()
|
||||
|
||||
def release_da3(self):
|
||||
self.da3_model = None
|
||||
self.da3_key = None
|
||||
self._release_runtime_cache()
|
||||
|
||||
def release_moge(self):
|
||||
self.moge_model = None
|
||||
self.moge_key = None
|
||||
self._release_runtime_cache()
|
||||
|
||||
def close(self):
|
||||
self.ben2_model = None
|
||||
self.da3_model = None
|
||||
self.moge_model = None
|
||||
self.ben2_key = None
|
||||
self.da3_key = None
|
||||
self.moge_key = None
|
||||
self._release_runtime_cache()
|
||||
|
||||
def snapshot(self):
|
||||
return {
|
||||
"ben2": Path(self.ben2_key[0]).name if self.ben2_key else "",
|
||||
"da3": Path(self.da3_key[0]).name if self.da3_key else "",
|
||||
"moge2": Path(self.moge_key[0]).name if self.moge_key else "",
|
||||
}
|
||||
|
||||
def _release_runtime_cache(self):
|
||||
gc.collect()
|
||||
|
||||
|
||||
class ModelManager:
|
||||
"""Own downloaded model files and loaded inference sessions."""
|
||||
|
||||
def __init__(self, models_directory):
|
||||
self.models_directory = Path(models_directory)
|
||||
self.sessions = ModelSessionCache()
|
||||
|
||||
def directory(self, model_key):
|
||||
model = model_catalog.get_downloadable_model(model_key)
|
||||
directory = self.models_directory / model.directory_name
|
||||
old_moge_directory = self.models_directory / "moge2_base_normal"
|
||||
if (
|
||||
model_key == "MOGE2_VITB_NORMAL"
|
||||
and not directory.is_dir()
|
||||
and old_moge_directory.is_dir()
|
||||
):
|
||||
old_moge_directory.rename(directory)
|
||||
legacy_directory = self.models_directory / model_key.lower()
|
||||
if (
|
||||
not directory.is_dir()
|
||||
and legacy_directory.is_dir()
|
||||
and legacy_directory != directory
|
||||
):
|
||||
legacy_directory.rename(directory)
|
||||
return directory
|
||||
|
||||
def ready(self, model_key):
|
||||
model = model_catalog.get_downloadable_model(model_key)
|
||||
directory = self.directory(model_key)
|
||||
return all(any(directory.glob(pattern)) for pattern in model.ready_patterns)
|
||||
|
||||
def get_ben2(self, model_dir, device):
|
||||
return self.sessions.get_ben2(model_dir, device)
|
||||
|
||||
def get_da3(self, model_dir, device):
|
||||
return self.sessions.get_da3(model_dir, device)
|
||||
|
||||
def get_moge2(self, model_dir, device):
|
||||
return self.sessions.get_moge2(model_dir, device)
|
||||
|
||||
def snapshot(self):
|
||||
catalog = [
|
||||
model_catalog.model_record(model, self.ready(model.key))
|
||||
for model in model_catalog.DOWNLOADABLE_MODELS.values()
|
||||
]
|
||||
return {"loaded": self.sessions.snapshot(), "catalog": catalog}
|
||||
|
||||
def clear(self):
|
||||
self.sessions.close()
|
||||
|
||||
def close(self):
|
||||
self.sessions.close()
|
||||
|
||||
def download(self, context, model_key):
|
||||
return self._download(context, model_key, context.progress)
|
||||
|
||||
def download_missing(self, context, model_keys):
|
||||
missing = [key for key in model_keys if not self.ready(key)]
|
||||
if not missing:
|
||||
context.progress(1.0, "Required models are ready")
|
||||
return {"downloaded": []}
|
||||
|
||||
total = len(missing)
|
||||
for index, model_key in enumerate(missing):
|
||||
def report(value, message, *, offset=index):
|
||||
context.progress((offset + value) / total, message)
|
||||
|
||||
self._download(context, model_key, report)
|
||||
return {"downloaded": missing}
|
||||
|
||||
def _download(self, context, model_key, progress):
|
||||
model = model_catalog.get_downloadable_model(model_key)
|
||||
args = SimpleNamespace(
|
||||
repository=model.repository,
|
||||
model_dir=str(self.directory(model_key)),
|
||||
progress=progress,
|
||||
cancel_check=context.check_cancelled,
|
||||
label=model.label,
|
||||
allow_pattern=list(model.download_patterns),
|
||||
required_pattern=list(model.ready_patterns),
|
||||
r2_directory=model.r2_directory,
|
||||
r2_file=[
|
||||
f"{filename}|{size}|{checksum}"
|
||||
for filename, size, checksum in model.r2_files
|
||||
],
|
||||
)
|
||||
context.check_cancelled()
|
||||
download_model(args)
|
||||
context.check_cancelled()
|
||||
304
src/snapform/server/moge2.py
Normal file
304
src/snapform/server/moge2.py
Normal file
@@ -0,0 +1,304 @@
|
||||
import shutil
|
||||
from pathlib import Path
|
||||
|
||||
from .background import infer_foreground, infer_foreground_frames
|
||||
from .geometry import (
|
||||
depth_discontinuity_ratio,
|
||||
depth_edge_mask,
|
||||
foreground_mesh_mask,
|
||||
mesh_triangle_faces,
|
||||
)
|
||||
from .media import copy_frames, prepare_input
|
||||
|
||||
|
||||
def _save_moge_preview(output_directory, index, depth, normal, mask):
|
||||
import numpy as np
|
||||
from PIL import Image
|
||||
|
||||
suffix = "" if index is None else f".{index + 1:04d}"
|
||||
valid = mask & np.isfinite(depth) & (depth > 0)
|
||||
preview = np.zeros(depth.shape, dtype=np.uint8)
|
||||
if np.any(valid):
|
||||
low, high = np.percentile(depth[valid], (2.0, 98.0))
|
||||
span = max(float(high - low), 1e-6)
|
||||
preview[valid] = np.clip(
|
||||
(depth[valid] - low) / span * 255.0,
|
||||
0.0,
|
||||
255.0,
|
||||
).astype(np.uint8)
|
||||
Image.fromarray(preview, mode="L").save(
|
||||
output_directory / f"depth{suffix}.png"
|
||||
)
|
||||
normal_rgb = np.clip(
|
||||
(normal * np.array([1.0, -1.0, -1.0], dtype=np.float32) + 1.0)
|
||||
* 127.5,
|
||||
0.0,
|
||||
255.0,
|
||||
).astype(np.uint8)
|
||||
Image.fromarray(normal_rgb, mode="RGB").save(
|
||||
output_directory / f"normal{suffix}.png"
|
||||
)
|
||||
Image.fromarray(mask.astype(np.uint8) * 255, mode="L").save(
|
||||
output_directory / f"mask{suffix}.png"
|
||||
)
|
||||
|
||||
|
||||
def export_moge_debug_mesh(
|
||||
output_directory,
|
||||
points,
|
||||
depth,
|
||||
mask,
|
||||
normal,
|
||||
image,
|
||||
export_format,
|
||||
):
|
||||
import numpy as np
|
||||
import trimesh
|
||||
|
||||
clean_mask = mask & ~depth_edge_mask(depth, threshold=0.04)
|
||||
faces = mesh_triangle_faces(
|
||||
points,
|
||||
depth,
|
||||
np.ones(depth.shape, dtype=np.float32),
|
||||
None,
|
||||
False,
|
||||
0.0,
|
||||
vertex_mask=clean_mask,
|
||||
)
|
||||
vertices = points.reshape(-1, 3)
|
||||
colors = np.asarray(image.reshape(-1, 3) * 255.0, dtype=np.uint8)
|
||||
vertex_normals = normal.reshape(-1, 3)
|
||||
vertices = vertices * np.array([1.0, -1.0, -1.0], dtype=np.float32)
|
||||
vertex_normals = vertex_normals * np.array([1.0, -1.0, -1.0], dtype=np.float32)
|
||||
mesh = trimesh.Trimesh(
|
||||
vertices=vertices,
|
||||
faces=faces,
|
||||
vertex_colors=colors,
|
||||
vertex_normals=vertex_normals,
|
||||
process=False,
|
||||
)
|
||||
mesh.remove_unreferenced_vertices()
|
||||
filename = "mesh.glb" if export_format == "glb" else "pointcloud.ply"
|
||||
mesh.export(Path(output_directory) / filename)
|
||||
|
||||
|
||||
def export_moge_surface_archive(
|
||||
output_path,
|
||||
points,
|
||||
depth,
|
||||
mask,
|
||||
normal,
|
||||
intrinsics,
|
||||
foreground_mask=None,
|
||||
):
|
||||
import numpy as np
|
||||
|
||||
height, width = depth.shape
|
||||
finite_mask = np.isfinite(depth) & (depth > 0.0)
|
||||
finite_mask &= np.isfinite(points).all(axis=-1)
|
||||
if foreground_mask is not None:
|
||||
finite_mask &= foreground_mask
|
||||
depth_discontinuity = depth_discontinuity_ratio(depth)
|
||||
faces = mesh_triangle_faces(
|
||||
points,
|
||||
depth,
|
||||
np.ones(depth.shape, dtype=np.float32),
|
||||
None,
|
||||
False,
|
||||
0.0,
|
||||
vertex_mask=finite_mask,
|
||||
)
|
||||
vertices = points.reshape(-1, 3)
|
||||
u = np.linspace(0.0, 1.0, width, dtype=np.float32)
|
||||
v = np.linspace(1.0, 0.0, height, dtype=np.float32)
|
||||
uu, vv = np.meshgrid(u, v)
|
||||
vertex_uvs = np.stack((uu, vv), axis=-1).reshape(-1, 2)
|
||||
pixel_intrinsics = intrinsics.copy()
|
||||
pixel_intrinsics[0] *= width
|
||||
pixel_intrinsics[1] *= height
|
||||
pixel_intrinsics[2] = (0.0, 0.0, 1.0)
|
||||
point_depth = depth.reshape(-1).astype(np.float32)
|
||||
face_mask = mask.reshape(-1)[faces].all(axis=1)
|
||||
face_depth_discontinuity = depth_discontinuity.reshape(-1)[faces].max(axis=1)
|
||||
masked_depth = point_depth[mask.reshape(-1).astype(bool)]
|
||||
reference_depth = np.median(masked_depth if len(masked_depth) else point_depth)
|
||||
np.savez(
|
||||
output_path,
|
||||
vertices=vertices.astype(np.float32),
|
||||
faces=faces.astype(np.int32),
|
||||
uv=vertex_uvs.astype(np.float32),
|
||||
intrinsics=pixel_intrinsics.astype(np.float32),
|
||||
image_size=np.array((width, height), dtype=np.int32),
|
||||
reference_depth=np.float32(reference_depth),
|
||||
point_depth=point_depth,
|
||||
normal_map=normal.astype(np.float32),
|
||||
face_mask=face_mask,
|
||||
face_depth_discontinuity=face_depth_discontinuity,
|
||||
)
|
||||
|
||||
|
||||
def generate_moge2(context, parameters):
|
||||
"""Run MoGe-2 inference and return the artifact used by the requesting Job."""
|
||||
import numpy as np
|
||||
from PIL import Image
|
||||
from .onnx_moge2 import (
|
||||
infer as infer_moge2,
|
||||
provider_label,
|
||||
)
|
||||
|
||||
context.progress(0.02, "Starting MoGe-2")
|
||||
output_directory = context.directory
|
||||
output_directory.mkdir(parents=True, exist_ok=True)
|
||||
model_manager = context.resource("model_manager")
|
||||
model_directory = model_manager.directory(parameters["model"])
|
||||
device = parameters["device"]
|
||||
images, temporary_directory = prepare_input(
|
||||
parameters["input"],
|
||||
parameters.get("video_frames", 16),
|
||||
output_directory,
|
||||
start_frame=parameters.get("start_frame", 1),
|
||||
cancel_check=context.check_cancelled,
|
||||
)
|
||||
try:
|
||||
model = model_manager.get_moge2(model_directory, device)
|
||||
context.progress(
|
||||
0.08,
|
||||
f"Loading MoGe-2 on {provider_label(model)}",
|
||||
)
|
||||
|
||||
depths = []
|
||||
normals = []
|
||||
masks = []
|
||||
points = []
|
||||
intrinsics = []
|
||||
source_images = []
|
||||
for index, image_path in enumerate(images):
|
||||
context.check_cancelled()
|
||||
context.progress(
|
||||
0.15 + 0.68 * index / len(images),
|
||||
f"Estimating MoGe-2 geometry ({index + 1}/{len(images)})",
|
||||
)
|
||||
with Image.open(image_path) as opened:
|
||||
image = np.asarray(opened.convert("RGB"), dtype=np.float32) / 255.0
|
||||
source_images.append(image)
|
||||
prediction = infer_moge2(
|
||||
model,
|
||||
image,
|
||||
int(parameters["resolution_level"]),
|
||||
)
|
||||
depth = prediction["depth"]
|
||||
normal = prediction["normal"]
|
||||
mask = prediction["mask"].astype(bool)
|
||||
point = prediction["points"]
|
||||
intrinsic = prediction["intrinsics"]
|
||||
depths.append(depth)
|
||||
normals.append(normal)
|
||||
masks.append(mask)
|
||||
intrinsics.append(intrinsic)
|
||||
if bool(
|
||||
parameters.get("include_points", False)
|
||||
or parameters.get("surface_mesh", False)
|
||||
):
|
||||
points.append(point)
|
||||
debug_format = parameters.get("export_format", "")
|
||||
if not debug_format or debug_format == "maps":
|
||||
_save_moge_preview(
|
||||
output_directory,
|
||||
None if len(images) == 1 else index,
|
||||
depth,
|
||||
normal,
|
||||
mask,
|
||||
)
|
||||
|
||||
shapes = {array.shape for array in depths}
|
||||
if len(shapes) != 1:
|
||||
raise RuntimeError("MoGe-2 frame sizes must match")
|
||||
result = {
|
||||
"depth": np.stack(depths).astype(np.float32),
|
||||
"normal": np.stack(normals).astype(np.float32),
|
||||
"mask": np.stack(masks),
|
||||
"intrinsics": np.stack(intrinsics).astype(np.float32),
|
||||
}
|
||||
if points:
|
||||
result["points"] = np.stack(points).astype(np.float32)
|
||||
debug_format = parameters.get("export_format", "")
|
||||
if debug_format in {"glb", "ply"} and len(images) == 1:
|
||||
context.progress(
|
||||
0.84,
|
||||
f"Exporting MoGe-2 {debug_format.upper()}",
|
||||
)
|
||||
export_moge_debug_mesh(
|
||||
output_directory,
|
||||
points[0],
|
||||
depths[0],
|
||||
masks[0],
|
||||
normals[0],
|
||||
source_images[0],
|
||||
debug_format,
|
||||
)
|
||||
|
||||
foreground_mask = None
|
||||
remove_background_model = parameters.get("remove_background_model", "")
|
||||
if remove_background_model:
|
||||
remove_background_model_dir = model_manager.directory(
|
||||
remove_background_model
|
||||
)
|
||||
context.progress( 0.87, "Removing image background")
|
||||
if len(images) == 1:
|
||||
foreground, _, _ = infer_foreground(
|
||||
images[0],
|
||||
remove_background_model_dir,
|
||||
device,
|
||||
model_cache=model_manager,
|
||||
cancel_check=context.check_cancelled,
|
||||
)
|
||||
foreground_path = output_directory / "foreground.png"
|
||||
foreground.save(foreground_path)
|
||||
if points:
|
||||
foreground_mask = foreground_mesh_mask(
|
||||
foreground_path,
|
||||
result["depth"][0].shape,
|
||||
)
|
||||
else:
|
||||
infer_foreground_frames(
|
||||
images,
|
||||
remove_background_model_dir,
|
||||
device,
|
||||
output_directory,
|
||||
context.progress,
|
||||
0.87,
|
||||
0.08,
|
||||
model_cache=model_manager,
|
||||
cancel_check=context.check_cancelled,
|
||||
)
|
||||
if bool(parameters.get("surface_mesh", False)):
|
||||
if len(images) != 1 or not points:
|
||||
raise RuntimeError("MoGe-2 Surface requires one point map")
|
||||
context.progress( 0.94, "Building official MoGe-2 surface")
|
||||
export_moge_surface_archive(
|
||||
output_directory / "mesh.npz",
|
||||
points[0],
|
||||
result["depth"][0],
|
||||
result["mask"][0],
|
||||
result["normal"][0],
|
||||
result["intrinsics"][0],
|
||||
foreground_mask=(
|
||||
foreground_mask
|
||||
if remove_background_model
|
||||
else None
|
||||
),
|
||||
)
|
||||
np.savez(output_directory / "results.npz", **result)
|
||||
copy_frames(images, output_directory)
|
||||
context.progress(
|
||||
1.0,
|
||||
f"Exported MoGe-2 data for {len(images)} frame(s)",
|
||||
)
|
||||
if context.job_type == "generate-moge2-surface":
|
||||
return {"surface": "mesh.npz"}
|
||||
if context.job_type == "generate-moge2-depth":
|
||||
return {"depth": "results.npz"}
|
||||
return {"directory": "."}
|
||||
finally:
|
||||
if temporary_directory is not None:
|
||||
shutil.rmtree(temporary_directory, ignore_errors=True)
|
||||
51
src/snapform/server/onnx_ben2.py
Normal file
51
src/snapform/server/onnx_ben2.py
Normal file
@@ -0,0 +1,51 @@
|
||||
"""BEN2 ONNX inference with Pillow and NumPy preprocessing."""
|
||||
|
||||
from pathlib import Path
|
||||
|
||||
try:
|
||||
from .onnx_runtime import create_session as create_runtime_session
|
||||
except ImportError:
|
||||
from onnx_runtime import create_session as create_runtime_session
|
||||
|
||||
|
||||
ONNX_PATH = Path("onnx/model_fp16.onnx")
|
||||
INPUT_SIZE = (1024, 1024)
|
||||
|
||||
|
||||
def model_path(model_directory):
|
||||
return Path(model_directory) / ONNX_PATH
|
||||
|
||||
|
||||
def create_session(model_directory, requested_device="auto"):
|
||||
return create_runtime_session(model_path(model_directory), requested_device)
|
||||
|
||||
|
||||
def preprocess(image):
|
||||
import numpy as np
|
||||
from PIL import Image
|
||||
|
||||
resized = image.convert("RGB").resize(INPUT_SIZE, Image.Resampling.LANCZOS)
|
||||
array = np.asarray(resized, dtype=np.float32) / 255.0
|
||||
mean = np.array((0.485, 0.456, 0.406), dtype=np.float32)
|
||||
std = np.array((0.229, 0.224, 0.225), dtype=np.float32)
|
||||
array = (array - mean) / std
|
||||
return np.ascontiguousarray(array.transpose(2, 0, 1)[None])
|
||||
|
||||
|
||||
def infer(session, image):
|
||||
import numpy as np
|
||||
from PIL import Image
|
||||
|
||||
input_name = session.get_inputs()[0].name
|
||||
output_name = session.get_outputs()[0].name
|
||||
alpha = session.run([output_name], {input_name: preprocess(image)})[0]
|
||||
alpha = np.asarray(alpha, dtype=np.float32).squeeze()
|
||||
minimum = float(alpha.min())
|
||||
span = max(float(alpha.max()) - minimum, np.finfo(np.float32).eps)
|
||||
alpha = np.clip((alpha - minimum) / span * 255.0, 0.0, 255.0)
|
||||
mask = Image.fromarray(alpha.astype(np.uint8), mode="L")
|
||||
mask = mask.resize(image.size, Image.Resampling.LANCZOS)
|
||||
foreground = image.convert("RGBA")
|
||||
foreground.putalpha(mask)
|
||||
return foreground
|
||||
|
||||
240
src/snapform/server/onnx_da3.py
Normal file
240
src/snapform/server/onnx_da3.py
Normal file
@@ -0,0 +1,240 @@
|
||||
"""Depth Anything 3 inference with ONNX Runtime and NumPy."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from dataclasses import dataclass
|
||||
from pathlib import Path
|
||||
|
||||
import numpy as np
|
||||
|
||||
try:
|
||||
from .onnx_runtime import create_session as create_onnx_session
|
||||
except ImportError:
|
||||
from onnx_runtime import create_session as create_onnx_session
|
||||
|
||||
|
||||
ONNX_PATH = "model.onnx"
|
||||
PATCH_SIZE = 14
|
||||
# DA3 cross-view attention is quadratic in (views x patches); the peak
|
||||
# attention tensor alone can exhaust memory long before ONNX Runtime reports
|
||||
# a generic allocation failure.
|
||||
ATTENTION_BUDGET_BYTES = 4 * 1024 * 1024 * 1024
|
||||
STRATEGY_CODES = {
|
||||
"first": 0,
|
||||
"middle": 1,
|
||||
"saddle_balanced": 2,
|
||||
"saddle_sim_range": 3,
|
||||
}
|
||||
IMAGE_MEAN = np.asarray((0.485, 0.456, 0.406), dtype=np.float32)
|
||||
IMAGE_STD = np.asarray((0.229, 0.224, 0.225), dtype=np.float32)
|
||||
|
||||
|
||||
@dataclass
|
||||
class Prediction:
|
||||
depth: np.ndarray
|
||||
conf: np.ndarray
|
||||
extrinsics: np.ndarray
|
||||
intrinsics: np.ndarray
|
||||
processed_images: np.ndarray
|
||||
is_metric: int = 0
|
||||
sky: np.ndarray | None = None
|
||||
gaussians: object | None = None
|
||||
aux: dict | None = None
|
||||
scale_factor: float | None = None
|
||||
|
||||
|
||||
def create_session(model_dir, requested_device="auto"):
|
||||
path = Path(model_dir) / ONNX_PATH
|
||||
return create_onnx_session(path, requested_device)
|
||||
|
||||
|
||||
def nearest_multiple(value, multiple=PATCH_SIZE):
|
||||
lower = value // multiple * multiple
|
||||
upper = lower + multiple
|
||||
return upper if abs(upper - value) <= abs(value - lower) else lower
|
||||
|
||||
|
||||
def target_size(size, process_res, method):
|
||||
width, height = size
|
||||
boundary = max(width, height)
|
||||
if method.startswith("lower_bound"):
|
||||
boundary = min(width, height)
|
||||
scale = float(process_res) / boundary
|
||||
width = max(1, int(round(width * scale)))
|
||||
height = max(1, int(round(height * scale)))
|
||||
if method.endswith("resize"):
|
||||
return (
|
||||
max(PATCH_SIZE, nearest_multiple(width)),
|
||||
max(PATCH_SIZE, nearest_multiple(height)),
|
||||
)
|
||||
if method.endswith("crop"):
|
||||
return (
|
||||
max(PATCH_SIZE, width // PATCH_SIZE * PATCH_SIZE),
|
||||
max(PATCH_SIZE, height // PATCH_SIZE * PATCH_SIZE),
|
||||
)
|
||||
raise ValueError(f"Unsupported process_res_method: {method}")
|
||||
|
||||
|
||||
def resize_and_crop(image, process_res, method):
|
||||
from PIL import Image
|
||||
width, height = image.size
|
||||
boundary = max(width, height)
|
||||
if method.startswith("lower_bound"):
|
||||
boundary = min(width, height)
|
||||
scale = float(process_res) / boundary
|
||||
resized_size = (
|
||||
max(1, int(round(width * scale))),
|
||||
max(1, int(round(height * scale))),
|
||||
)
|
||||
resampling = (
|
||||
Image.Resampling.BICUBIC
|
||||
if scale > 1.0
|
||||
else Image.Resampling.BOX
|
||||
)
|
||||
image = image.resize(resized_size, resample=resampling)
|
||||
final_size = target_size((width, height), process_res, method)
|
||||
if method.endswith("resize"):
|
||||
if image.size != final_size:
|
||||
upscale = final_size[0] > image.width or final_size[1] > image.height
|
||||
resampling = (
|
||||
Image.Resampling.BICUBIC
|
||||
if upscale
|
||||
else Image.Resampling.BOX
|
||||
)
|
||||
image = image.resize(final_size, resample=resampling)
|
||||
return image
|
||||
left = (image.width - final_size[0]) // 2
|
||||
top = (image.height - final_size[1]) // 2
|
||||
return image.crop((left, top, left + final_size[0], top + final_size[1]))
|
||||
|
||||
|
||||
def preprocess(image_paths, process_res=504, process_res_method="upper_bound_resize"):
|
||||
from PIL import Image
|
||||
processed = []
|
||||
for path in image_paths:
|
||||
with Image.open(path) as source:
|
||||
image = resize_and_crop(
|
||||
source.convert("RGB"),
|
||||
process_res,
|
||||
process_res_method,
|
||||
)
|
||||
processed.append(np.asarray(image, dtype=np.uint8))
|
||||
min_height = min(image.shape[0] for image in processed)
|
||||
min_width = min(image.shape[1] for image in processed)
|
||||
cropped = []
|
||||
for image in processed:
|
||||
top = (image.shape[0] - min_height) // 2
|
||||
left = (image.shape[1] - min_width) // 2
|
||||
cropped.append(
|
||||
image[top : top + min_height, left : left + min_width]
|
||||
)
|
||||
images = np.stack(cropped)
|
||||
normalized = images.astype(np.float32) / 255.0
|
||||
normalized = (normalized - IMAGE_MEAN) / IMAGE_STD
|
||||
model_input = normalized.transpose(0, 3, 1, 2)[None]
|
||||
return np.ascontiguousarray(model_input), images
|
||||
|
||||
|
||||
def quaternion_to_matrix(quaternion):
|
||||
i, j, k, real = np.moveaxis(quaternion, -1, 0)
|
||||
two_scale = 2.0 / np.sum(quaternion * quaternion, axis=-1)
|
||||
values = (
|
||||
1 - two_scale * (j * j + k * k),
|
||||
two_scale * (i * j - k * real),
|
||||
two_scale * (i * k + j * real),
|
||||
two_scale * (i * j + k * real),
|
||||
1 - two_scale * (i * i + k * k),
|
||||
two_scale * (j * k - i * real),
|
||||
two_scale * (i * k - j * real),
|
||||
two_scale * (j * k + i * real),
|
||||
1 - two_scale * (i * i + j * j),
|
||||
)
|
||||
return np.stack(values, axis=-1).reshape(quaternion.shape[:-1] + (3, 3))
|
||||
|
||||
|
||||
def camera_from_pose(pose_encoding, height, width):
|
||||
translation = pose_encoding[..., :3]
|
||||
rotation = quaternion_to_matrix(pose_encoding[..., 3:7])
|
||||
rotation_inverse = np.swapaxes(rotation, -1, -2)
|
||||
translation_inverse = -rotation_inverse @ translation[..., None]
|
||||
extrinsics = np.concatenate(
|
||||
(rotation_inverse, translation_inverse),
|
||||
axis=-1,
|
||||
).astype(np.float32)
|
||||
|
||||
field_of_view_height = pose_encoding[..., 7]
|
||||
field_of_view_width = pose_encoding[..., 8]
|
||||
focal_y = (height / 2.0) / np.maximum(
|
||||
np.tan(field_of_view_height / 2.0),
|
||||
1e-6,
|
||||
)
|
||||
focal_x = (width / 2.0) / np.maximum(
|
||||
np.tan(field_of_view_width / 2.0),
|
||||
1e-6,
|
||||
)
|
||||
intrinsics = np.zeros(pose_encoding.shape[:-1] + (3, 3), dtype=np.float32)
|
||||
intrinsics[..., 0, 0] = focal_x
|
||||
intrinsics[..., 1, 1] = focal_y
|
||||
intrinsics[..., 0, 2] = width / 2.0
|
||||
intrinsics[..., 1, 2] = height / 2.0
|
||||
intrinsics[..., 2, 2] = 1.0
|
||||
return extrinsics, intrinsics
|
||||
|
||||
|
||||
def estimate_attention_bytes(views, height, width):
|
||||
"""Estimate the peak cross-view attention tensor size in bytes."""
|
||||
patches = (height // PATCH_SIZE) * (width // PATCH_SIZE)
|
||||
tokens = views * patches
|
||||
# 12 attention heads in DA3 Base; float32 activations.
|
||||
return tokens * tokens * 12 * 4
|
||||
|
||||
|
||||
def infer(
|
||||
session,
|
||||
image_paths,
|
||||
process_res=504,
|
||||
process_res_method="upper_bound_resize",
|
||||
ref_view_strategy="saddle_balanced",
|
||||
):
|
||||
inputs = session.get_inputs()
|
||||
model_input, processed_images = preprocess(
|
||||
image_paths,
|
||||
process_res,
|
||||
process_res_method,
|
||||
)
|
||||
_, views, _, height, width = model_input.shape
|
||||
attention_bytes = estimate_attention_bytes(views, height, width)
|
||||
if attention_bytes > ATTENTION_BUDGET_BYTES:
|
||||
raise RuntimeError(
|
||||
f"DA3 cannot process {views} frame(s) at {width}x{height}: "
|
||||
"the cross-view attention would need about "
|
||||
f"{attention_bytes // 2**30} GB. Reduce the frame count or "
|
||||
"the Resolution Scale."
|
||||
)
|
||||
strategy_code = np.asarray(
|
||||
[STRATEGY_CODES.get(ref_view_strategy, 2)],
|
||||
dtype=np.int64,
|
||||
)
|
||||
input_names = [value.name for value in inputs]
|
||||
outputs = session.run(
|
||||
None,
|
||||
{
|
||||
input_names[0]: model_input,
|
||||
input_names[1]: strategy_code,
|
||||
},
|
||||
)
|
||||
depth, confidence, pose_encoding = outputs[:3]
|
||||
height, width = depth.shape[-2:]
|
||||
extrinsics, intrinsics = camera_from_pose(
|
||||
pose_encoding[0],
|
||||
height,
|
||||
width,
|
||||
)
|
||||
return Prediction(
|
||||
depth=np.asarray(depth[0], dtype=np.float32),
|
||||
conf=np.asarray(confidence[0], dtype=np.float32),
|
||||
extrinsics=extrinsics,
|
||||
intrinsics=intrinsics,
|
||||
processed_images=processed_images,
|
||||
aux={},
|
||||
)
|
||||
155
src/snapform/server/onnx_moge2.py
Normal file
155
src/snapform/server/onnx_moge2.py
Normal file
@@ -0,0 +1,155 @@
|
||||
"""MoGe-2 ONNX inference and NumPy post-processing."""
|
||||
|
||||
from pathlib import Path
|
||||
|
||||
from .onnx_runtime import create_session as create_runtime_session
|
||||
from .onnx_runtime import provider_label, select_providers
|
||||
|
||||
|
||||
ONNX_FILENAME = "model.onnx"
|
||||
MIN_TOKENS = 1200
|
||||
MAX_TOKENS = 3600
|
||||
|
||||
|
||||
def is_onnx_model(model_directory):
|
||||
return (Path(model_directory) / ONNX_FILENAME).is_file()
|
||||
|
||||
|
||||
def create_session(model_directory, requested_device="auto"):
|
||||
path = Path(model_directory) / ONNX_FILENAME
|
||||
return create_runtime_session(path, requested_device)
|
||||
|
||||
|
||||
def normalized_view_plane_uv(width, height, dtype):
|
||||
import numpy as np
|
||||
|
||||
aspect_ratio = width / height
|
||||
span_x = aspect_ratio / (1.0 + aspect_ratio**2) ** 0.5
|
||||
span_y = 1.0 / (1.0 + aspect_ratio**2) ** 0.5
|
||||
u = np.linspace(
|
||||
-span_x * (width - 1) / width,
|
||||
span_x * (width - 1) / width,
|
||||
width,
|
||||
dtype=dtype,
|
||||
)
|
||||
v = np.linspace(
|
||||
-span_y * (height - 1) / height,
|
||||
span_y * (height - 1) / height,
|
||||
height,
|
||||
dtype=dtype,
|
||||
)
|
||||
u, v = np.meshgrid(u, v, indexing="xy")
|
||||
return np.stack((u, v), axis=-1)
|
||||
|
||||
|
||||
def solve_focal_shift(uv, points):
|
||||
import numpy as np
|
||||
|
||||
uv = uv.reshape(-1, 2)
|
||||
xy = points[..., :2].reshape(-1, 2)
|
||||
z = points[..., 2].reshape(-1)
|
||||
matrix = np.stack((xy, -uv), axis=-1).reshape(-1, 2)
|
||||
target = (uv * z[:, None]).reshape(-1)
|
||||
focal, shift = np.linalg.lstsq(matrix, target, rcond=None)[0]
|
||||
epsilon = np.finfo(np.float32).eps
|
||||
for _iteration in range(20):
|
||||
denominator = z + shift
|
||||
denominator = np.where(
|
||||
np.abs(denominator) < epsilon,
|
||||
np.copysign(epsilon, denominator),
|
||||
denominator,
|
||||
)
|
||||
projected = xy / denominator[:, None]
|
||||
residual = (focal * projected - uv).reshape(-1)
|
||||
jacobian = np.stack(
|
||||
(
|
||||
projected,
|
||||
-focal * xy / np.square(denominator)[:, None],
|
||||
),
|
||||
axis=-1,
|
||||
).reshape(-1, 2)
|
||||
delta = np.linalg.lstsq(jacobian, -residual, rcond=None)[0]
|
||||
focal += delta[0]
|
||||
shift += delta[1]
|
||||
if np.max(np.abs(delta)) < 1e-6:
|
||||
break
|
||||
return np.float32(focal), shift
|
||||
|
||||
|
||||
def recover_focal_shift(points, mask, downsample_size=(64, 64)):
|
||||
import numpy as np
|
||||
|
||||
height, width = points.shape[:2]
|
||||
uv = normalized_view_plane_uv(width, height, points.dtype)
|
||||
target_height, target_width = downsample_size
|
||||
y = np.floor(np.arange(target_height) * height / target_height).astype(int)
|
||||
x = np.floor(np.arange(target_width) * width / target_width).astype(int)
|
||||
points_low = points[y[:, None], x[None, :]]
|
||||
uv_low = uv[y[:, None], x[None, :]]
|
||||
mask_low = mask[y[:, None], x[None, :]]
|
||||
points_low = points_low[mask_low]
|
||||
uv_low = uv_low[mask_low]
|
||||
if len(points_low) < 2:
|
||||
return np.float32(1.0), np.float32(0.0)
|
||||
return solve_focal_shift(uv_low, points_low)
|
||||
|
||||
|
||||
def depth_to_points(depth, intrinsics):
|
||||
import numpy as np
|
||||
|
||||
height, width = depth.shape
|
||||
u = np.linspace(0.5 / width, 1.0 - 0.5 / width, width, dtype=depth.dtype)
|
||||
v = np.linspace(0.5 / height, 1.0 - 0.5 / height, height, dtype=depth.dtype)
|
||||
u, v = np.meshgrid(u, v, indexing="xy")
|
||||
x = (u - intrinsics[0, 2]) / intrinsics[0, 0] * depth
|
||||
y = (v - intrinsics[1, 2]) / intrinsics[1, 1] * depth
|
||||
return np.stack((x, y, depth), axis=-1)
|
||||
|
||||
|
||||
def postprocess(raw_output):
|
||||
import numpy as np
|
||||
|
||||
points = np.asarray(raw_output["points"][0], dtype=np.float32)
|
||||
normal = np.asarray(raw_output["normal"][0], dtype=np.float32)
|
||||
mask = np.asarray(raw_output["mask"][0] > 0.5)
|
||||
metric_scale = np.float32(np.asarray(raw_output["metric_scale"]).reshape(-1)[0])
|
||||
height, width = mask.shape
|
||||
aspect_ratio = width / height
|
||||
focal, shift = recover_focal_shift(points, mask)
|
||||
diagonal_factor = (1.0 + aspect_ratio**2) ** 0.5
|
||||
fx = focal / 2.0 * diagonal_factor / aspect_ratio
|
||||
fy = focal / 2.0 * diagonal_factor
|
||||
intrinsics = np.array(
|
||||
((fx, 0.0, 0.5), (0.0, fy, 0.5), (0.0, 0.0, 1.0)),
|
||||
dtype=np.float32,
|
||||
)
|
||||
depth = points[..., 2] + shift
|
||||
mask &= depth > 0.0
|
||||
depth = depth * metric_scale
|
||||
points = depth_to_points(depth, intrinsics)
|
||||
return {
|
||||
"points": points.astype(np.float32),
|
||||
"depth": depth.astype(np.float32),
|
||||
"normal": normal,
|
||||
"mask": mask,
|
||||
"intrinsics": intrinsics,
|
||||
}
|
||||
|
||||
|
||||
def infer(session, image, resolution_level):
|
||||
import numpy as np
|
||||
|
||||
level = min(max(int(resolution_level), 0), 9)
|
||||
num_tokens = int(MIN_TOKENS + level / 9.0 * (MAX_TOKENS - MIN_TOKENS))
|
||||
image_input = np.ascontiguousarray(image.transpose(2, 0, 1)[None], dtype=np.float32)
|
||||
input_feed = {
|
||||
"image": image_input,
|
||||
"num_tokens": np.asarray(num_tokens, dtype=np.int64),
|
||||
}
|
||||
output_names = {output.name for output in session.get_outputs()}
|
||||
scale_name = "metric_scale" if "metric_scale" in output_names else "scale"
|
||||
names = ("points", "normal", "mask", scale_name)
|
||||
values = session.run(list(names), input_feed)
|
||||
raw_output = dict(zip(names, values))
|
||||
raw_output["metric_scale"] = raw_output.pop(scale_name)
|
||||
return postprocess(raw_output)
|
||||
55
src/snapform/server/onnx_runtime.py
Normal file
55
src/snapform/server/onnx_runtime.py
Normal file
@@ -0,0 +1,55 @@
|
||||
"""Shared ONNX Runtime session helpers."""
|
||||
|
||||
|
||||
def select_providers(requested, available):
|
||||
available = set(available)
|
||||
providers_for_device = {
|
||||
"cuda": ("CUDAExecutionProvider", "DmlExecutionProvider"),
|
||||
"directml": ("DmlExecutionProvider",),
|
||||
"mps": ("CoreMLExecutionProvider",),
|
||||
"cpu": ("CPUExecutionProvider",),
|
||||
}
|
||||
if requested != "auto":
|
||||
candidates = providers_for_device[requested]
|
||||
for provider in candidates:
|
||||
if provider in available:
|
||||
return [provider]
|
||||
if "CPUExecutionProvider" in available:
|
||||
return ["CPUExecutionProvider"]
|
||||
raise RuntimeError(
|
||||
f"{requested.upper()} was requested but the ONNX Runtime "
|
||||
f"providers {', '.join(candidates)} are not available"
|
||||
)
|
||||
priorities = (
|
||||
"CUDAExecutionProvider",
|
||||
"CoreMLExecutionProvider",
|
||||
"DmlExecutionProvider",
|
||||
"OpenVINOExecutionProvider",
|
||||
"CPUExecutionProvider",
|
||||
)
|
||||
providers = [provider for provider in priorities if provider in available]
|
||||
if not providers:
|
||||
raise RuntimeError("ONNX Runtime has no usable execution provider")
|
||||
return providers
|
||||
|
||||
|
||||
def create_session(path, requested_device="auto"):
|
||||
import onnxruntime
|
||||
|
||||
providers = select_providers(
|
||||
requested_device,
|
||||
onnxruntime.get_available_providers(),
|
||||
)
|
||||
return onnxruntime.InferenceSession(str(path), providers=providers)
|
||||
|
||||
|
||||
def provider_label(session):
|
||||
provider = session.get_providers()[0]
|
||||
labels = {
|
||||
"CUDAExecutionProvider": "CUDA",
|
||||
"CoreMLExecutionProvider": "CoreML",
|
||||
"DmlExecutionProvider": "DirectML",
|
||||
"OpenVINOExecutionProvider": "OpenVINO",
|
||||
"CPUExecutionProvider": "CPU",
|
||||
}
|
||||
return labels.get(provider, provider)
|
||||
@@ -1,623 +0,0 @@
|
||||
import argparse
|
||||
import gc
|
||||
import hmac
|
||||
import json
|
||||
import os
|
||||
import sys
|
||||
import threading
|
||||
import time
|
||||
import traceback
|
||||
from contextlib import asynccontextmanager
|
||||
from pathlib import Path
|
||||
from types import SimpleNamespace
|
||||
|
||||
|
||||
SCRIPT_DIRECTORY = Path(__file__).resolve().parent
|
||||
if str(SCRIPT_DIRECTORY) not in sys.path:
|
||||
sys.path.insert(0, str(SCRIPT_DIRECTORY))
|
||||
|
||||
import worker
|
||||
|
||||
|
||||
CLIENT_TIMEOUT_SECONDS = 10.0
|
||||
JOB_COMMANDS = frozenset(
|
||||
{"remove-background", "reconstruct", "extract-frames", "moge"}
|
||||
)
|
||||
IDLE_SHUTDOWN_SECONDS = 20.0
|
||||
|
||||
|
||||
class ServiceLock:
|
||||
def __init__(self, path):
|
||||
self.path = Path(path)
|
||||
self.file = None
|
||||
|
||||
def acquire(self):
|
||||
self.path.parent.mkdir(parents=True, exist_ok=True)
|
||||
lock_file = self.path.open("a+b")
|
||||
lock_file.seek(0, os.SEEK_END)
|
||||
if lock_file.tell() == 0:
|
||||
lock_file.write(b"0")
|
||||
lock_file.flush()
|
||||
lock_file.seek(0)
|
||||
try:
|
||||
if sys.platform == "win32":
|
||||
import msvcrt
|
||||
|
||||
msvcrt.locking(lock_file.fileno(), msvcrt.LK_NBLCK, 1)
|
||||
else:
|
||||
import fcntl
|
||||
|
||||
fcntl.flock(lock_file.fileno(), fcntl.LOCK_EX | fcntl.LOCK_NB)
|
||||
except OSError:
|
||||
lock_file.close()
|
||||
return False
|
||||
self.file = lock_file
|
||||
return True
|
||||
|
||||
def release(self):
|
||||
if self.file is None:
|
||||
return
|
||||
try:
|
||||
self.file.seek(0)
|
||||
if sys.platform == "win32":
|
||||
import msvcrt
|
||||
|
||||
msvcrt.locking(self.file.fileno(), msvcrt.LK_UNLCK, 1)
|
||||
else:
|
||||
import fcntl
|
||||
|
||||
fcntl.flock(self.file.fileno(), fcntl.LOCK_UN)
|
||||
finally:
|
||||
self.file.close()
|
||||
self.file = None
|
||||
|
||||
|
||||
class ModelCache:
|
||||
def __init__(self):
|
||||
import torch
|
||||
|
||||
self.torch = torch
|
||||
self.ben2_model = None
|
||||
self.ben2_key = None
|
||||
self.da3_model = None
|
||||
self.da3_key = None
|
||||
self.moge_model = None
|
||||
self.moge_key = None
|
||||
|
||||
def get_ben2(self, model_dir, device):
|
||||
key = (str(Path(model_dir).resolve()), device.type)
|
||||
if self.ben2_model is not None and self.ben2_key == key:
|
||||
return self.ben2_model, 0.0
|
||||
self.release_ben2()
|
||||
try:
|
||||
return self._load_ben2(model_dir, device, key)
|
||||
except self.torch.cuda.OutOfMemoryError:
|
||||
self.release_da3()
|
||||
self.release_moge()
|
||||
return self._load_ben2(model_dir, device, key)
|
||||
|
||||
def _load_ben2(self, model_dir, device, key):
|
||||
from ben2 import BEN_Base
|
||||
|
||||
started = time.perf_counter()
|
||||
model = BEN_Base.from_pretrained(model_dir)
|
||||
model = model.to(device=device)
|
||||
model.eval()
|
||||
self.ben2_model = model
|
||||
self.ben2_key = key
|
||||
elapsed_ms = (time.perf_counter() - started) * 1000.0
|
||||
return model, elapsed_ms
|
||||
|
||||
def get_da3(self, model_dir, device):
|
||||
key = (str(Path(model_dir).resolve()), device.type)
|
||||
if self.da3_model is not None and self.da3_key == key:
|
||||
return self.da3_model
|
||||
self.release_da3()
|
||||
try:
|
||||
return self._load_da3(model_dir, device, key)
|
||||
except self.torch.cuda.OutOfMemoryError:
|
||||
self.release_ben2()
|
||||
self.release_moge()
|
||||
return self._load_da3(model_dir, device, key)
|
||||
|
||||
def _load_da3(self, model_dir, device, key):
|
||||
from depth_anything_3.api import DepthAnything3
|
||||
|
||||
model = DepthAnything3.from_pretrained(model_dir)
|
||||
model = model.to(device=device)
|
||||
model.eval()
|
||||
self.da3_model = model
|
||||
self.da3_key = key
|
||||
return model
|
||||
|
||||
def get_moge(self, model_dir, device):
|
||||
key = (str(Path(model_dir).resolve()), device.type)
|
||||
if self.moge_model is not None and self.moge_key == key:
|
||||
return self.moge_model
|
||||
self.release_moge()
|
||||
try:
|
||||
return self._load_moge(model_dir, device, key)
|
||||
except self.torch.cuda.OutOfMemoryError:
|
||||
self.release_da3()
|
||||
self.release_ben2()
|
||||
return self._load_moge(model_dir, device, key)
|
||||
|
||||
def _load_moge(self, model_dir, device, key):
|
||||
worker.prepare_moge_runtime()
|
||||
from moge.model.v2 import MoGeModel
|
||||
|
||||
checkpoint = Path(model_dir) / "model.pt"
|
||||
model = MoGeModel.from_pretrained(checkpoint)
|
||||
model = model.to(device=device)
|
||||
model.eval()
|
||||
self.moge_model = model
|
||||
self.moge_key = key
|
||||
return model
|
||||
|
||||
def release_ben2(self):
|
||||
self.ben2_model = None
|
||||
self.ben2_key = None
|
||||
self._release_cuda_cache()
|
||||
|
||||
def release_da3(self):
|
||||
self.da3_model = None
|
||||
self.da3_key = None
|
||||
self._release_cuda_cache()
|
||||
|
||||
def release_moge(self):
|
||||
self.moge_model = None
|
||||
self.moge_key = None
|
||||
self._release_cuda_cache()
|
||||
|
||||
def close(self):
|
||||
self.ben2_model = None
|
||||
self.da3_model = None
|
||||
self.moge_model = None
|
||||
self.ben2_key = None
|
||||
self.da3_key = None
|
||||
self.moge_key = None
|
||||
self._release_cuda_cache()
|
||||
|
||||
def status(self):
|
||||
return {
|
||||
"ben2": (
|
||||
Path(self.ben2_key[0]).name if self.ben2_key is not None else ""
|
||||
),
|
||||
"da3": (
|
||||
Path(self.da3_key[0]).name
|
||||
if self.da3_key is not None
|
||||
else ""
|
||||
),
|
||||
"moge2": (
|
||||
Path(getattr(self, "moge_key", ("",))[0]).name
|
||||
if getattr(self, "moge_key", None) is not None
|
||||
else ""
|
||||
),
|
||||
}
|
||||
|
||||
def _release_cuda_cache(self):
|
||||
gc.collect()
|
||||
if self.torch.cuda.is_available():
|
||||
self.torch.cuda.empty_cache()
|
||||
|
||||
|
||||
class ServiceState:
|
||||
def __init__(self, token, instance_id):
|
||||
self.token = token
|
||||
self.instance_id = instance_id
|
||||
self.models = ModelCache()
|
||||
self.lock = threading.Lock()
|
||||
self.clients = {}
|
||||
self.active_job = None
|
||||
self.started_at = time.time()
|
||||
self.last_without_clients = time.monotonic()
|
||||
self.shutdown_event = threading.Event()
|
||||
self.monitor_stop = threading.Event()
|
||||
self.monitor_thread = None
|
||||
|
||||
def register_client(self, client_id, label):
|
||||
with self.lock:
|
||||
self.clients[client_id] = {
|
||||
"label": label,
|
||||
"last_seen": time.monotonic(),
|
||||
}
|
||||
self.last_without_clients = None
|
||||
|
||||
def heartbeat(self, client_id):
|
||||
with self.lock:
|
||||
client = self.clients.get(client_id)
|
||||
if client is None:
|
||||
return False
|
||||
client["last_seen"] = time.monotonic()
|
||||
return True
|
||||
|
||||
def unregister_client(self, client_id):
|
||||
with self.lock:
|
||||
self.clients.pop(client_id, None)
|
||||
self._mark_empty_clients()
|
||||
|
||||
def reserve(self, job_id, client_id, command, status_path):
|
||||
with self.lock:
|
||||
if client_id not in self.clients or self.active_job is not None:
|
||||
return False
|
||||
self.active_job = {
|
||||
"job_id": job_id,
|
||||
"client_id": client_id,
|
||||
"command": command,
|
||||
"status": status_path,
|
||||
"cancel_requested": False,
|
||||
}
|
||||
return True
|
||||
|
||||
def snapshot(self):
|
||||
self.expire_clients()
|
||||
with self.lock:
|
||||
return {
|
||||
"ready": True,
|
||||
"instance_id": self.instance_id,
|
||||
"busy": self.active_job is not None,
|
||||
"active_job": dict(self.active_job) if self.active_job else None,
|
||||
"clients": len(self.clients),
|
||||
"cached_models": self.models.status(),
|
||||
"started_at": self.started_at,
|
||||
}
|
||||
|
||||
def clear_models(self):
|
||||
with self.lock:
|
||||
if self.active_job is not None:
|
||||
return False
|
||||
self.models.close()
|
||||
return True
|
||||
|
||||
def request_shutdown(self, client_id):
|
||||
with self.lock:
|
||||
if client_id not in self.clients:
|
||||
return "missing"
|
||||
if len(self.clients) > 1:
|
||||
return "shared"
|
||||
if self.active_job is not None:
|
||||
return "busy"
|
||||
self.shutdown_event.set()
|
||||
return "accepted"
|
||||
|
||||
def cancel_job(self, job_id, client_id):
|
||||
with self.lock:
|
||||
active = self.active_job
|
||||
if active is None or active["job_id"] != job_id:
|
||||
return None
|
||||
if active["client_id"] != client_id:
|
||||
return None
|
||||
active["cancel_requested"] = True
|
||||
return dict(active)
|
||||
|
||||
def is_job_cancelled(self, job_id):
|
||||
with self.lock:
|
||||
active = self.active_job
|
||||
return bool(
|
||||
active is not None
|
||||
and active["job_id"] == job_id
|
||||
and active["cancel_requested"]
|
||||
)
|
||||
|
||||
def run_job(self, payload):
|
||||
job_id = payload["job_id"]
|
||||
parameters = payload["parameters"]
|
||||
status_path = parameters["status"]
|
||||
worker.write_status(
|
||||
status_path,
|
||||
0.0,
|
||||
"Service accepted the task",
|
||||
state="running",
|
||||
)
|
||||
try:
|
||||
cancel_check = lambda: self.is_job_cancelled(job_id)
|
||||
worker.check_cancelled(cancel_check)
|
||||
args = SimpleNamespace(**parameters)
|
||||
if payload["command"] == "remove-background":
|
||||
worker.remove_background(
|
||||
args,
|
||||
model_cache=self.models,
|
||||
cancel_check=cancel_check,
|
||||
)
|
||||
elif payload["command"] == "extract-frames":
|
||||
worker.extract_frames(
|
||||
args,
|
||||
cancel_check=cancel_check,
|
||||
)
|
||||
elif payload["command"] == "moge":
|
||||
worker.run_moge(
|
||||
args,
|
||||
model_cache=self.models,
|
||||
cancel_check=cancel_check,
|
||||
)
|
||||
else:
|
||||
worker.reconstruct(
|
||||
args,
|
||||
model_cache=self.models,
|
||||
cancel_check=cancel_check,
|
||||
)
|
||||
worker.check_cancelled(cancel_check)
|
||||
except worker.TaskCancelled:
|
||||
worker.write_status(
|
||||
status_path,
|
||||
1.0,
|
||||
"Task cancelled",
|
||||
state="cancelled",
|
||||
)
|
||||
except Exception as error:
|
||||
traceback.print_exc()
|
||||
worker.write_status(
|
||||
status_path,
|
||||
1.0,
|
||||
"Task failed",
|
||||
str(error),
|
||||
state="failed",
|
||||
)
|
||||
finally:
|
||||
with self.lock:
|
||||
if (
|
||||
self.active_job is not None
|
||||
and self.active_job["job_id"] == job_id
|
||||
):
|
||||
self.active_job = None
|
||||
self._mark_empty_clients()
|
||||
|
||||
def start_monitor(self):
|
||||
self.monitor_thread = threading.Thread(
|
||||
target=self._monitor,
|
||||
name="SnapFormServiceMonitor",
|
||||
daemon=True,
|
||||
)
|
||||
self.monitor_thread.start()
|
||||
|
||||
def close(self):
|
||||
self.monitor_stop.set()
|
||||
if self.monitor_thread is not None:
|
||||
self.monitor_thread.join(timeout=1.0)
|
||||
self.models.close()
|
||||
|
||||
def expire_clients(self):
|
||||
deadline = time.monotonic() - CLIENT_TIMEOUT_SECONDS
|
||||
with self.lock:
|
||||
expired = [
|
||||
client_id
|
||||
for client_id, client in self.clients.items()
|
||||
if client["last_seen"] < deadline
|
||||
]
|
||||
for client_id in expired:
|
||||
self.clients.pop(client_id, None)
|
||||
self._mark_empty_clients()
|
||||
|
||||
def _mark_empty_clients(self):
|
||||
if self.clients:
|
||||
self.last_without_clients = None
|
||||
elif self.last_without_clients is None:
|
||||
self.last_without_clients = time.monotonic()
|
||||
|
||||
def _monitor(self):
|
||||
while not self.monitor_stop.wait(1.0):
|
||||
self.expire_clients()
|
||||
with self.lock:
|
||||
idle = self.active_job is None
|
||||
empty_since = self.last_without_clients
|
||||
if not idle or empty_since is None:
|
||||
continue
|
||||
if time.monotonic() - empty_since >= IDLE_SHUTDOWN_SECONDS:
|
||||
self.shutdown_event.set()
|
||||
return
|
||||
|
||||
|
||||
def create_app(token, instance_id="test"):
|
||||
from fastapi import BackgroundTasks, FastAPI, Header, HTTPException
|
||||
|
||||
state = ServiceState(token, instance_id)
|
||||
|
||||
@asynccontextmanager
|
||||
async def lifespan(_app):
|
||||
state.start_monitor()
|
||||
yield
|
||||
state.close()
|
||||
|
||||
app = FastAPI(
|
||||
title="SnapForm Service",
|
||||
docs_url=None,
|
||||
redoc_url=None,
|
||||
openapi_url=None,
|
||||
lifespan=lifespan,
|
||||
)
|
||||
app.state.snapform_service_state = state
|
||||
|
||||
def authorize(x_snapform_token: str = Header(default="")):
|
||||
if not hmac.compare_digest(x_snapform_token, state.token):
|
||||
raise HTTPException(status_code=401, detail="Invalid service token")
|
||||
|
||||
@app.get("/health")
|
||||
def health(x_snapform_token: str = Header(default="")):
|
||||
authorize(x_snapform_token)
|
||||
return state.snapshot()
|
||||
|
||||
@app.post("/clients", status_code=201)
|
||||
def register_client(
|
||||
payload: dict,
|
||||
x_snapform_token: str = Header(default=""),
|
||||
):
|
||||
authorize(x_snapform_token)
|
||||
client_id = str(payload.get("client_id", ""))
|
||||
if not client_id:
|
||||
raise HTTPException(status_code=422, detail="Missing client id")
|
||||
state.register_client(client_id, str(payload.get("label", "Blender")))
|
||||
return {"client_id": client_id, "lease_seconds": CLIENT_TIMEOUT_SECONDS}
|
||||
|
||||
@app.post("/clients/{client_id}/heartbeat")
|
||||
def heartbeat(
|
||||
client_id: str,
|
||||
x_snapform_token: str = Header(default=""),
|
||||
):
|
||||
authorize(x_snapform_token)
|
||||
if not state.heartbeat(client_id):
|
||||
raise HTTPException(status_code=404, detail="Client is not registered")
|
||||
return {"client_id": client_id, "state": "active"}
|
||||
|
||||
@app.delete("/clients/{client_id}")
|
||||
def unregister_client(
|
||||
client_id: str,
|
||||
x_snapform_token: str = Header(default=""),
|
||||
):
|
||||
authorize(x_snapform_token)
|
||||
state.unregister_client(client_id)
|
||||
return {"client_id": client_id, "state": "removed"}
|
||||
|
||||
@app.post("/jobs", status_code=202)
|
||||
def submit_job(
|
||||
payload: dict,
|
||||
background_tasks: BackgroundTasks,
|
||||
x_snapform_token: str = Header(default=""),
|
||||
):
|
||||
authorize(x_snapform_token)
|
||||
job_id = str(payload.get("job_id", ""))
|
||||
client_id = str(payload.get("client_id", ""))
|
||||
command = payload.get("command")
|
||||
parameters = payload.get("parameters")
|
||||
if not job_id or command not in JOB_COMMANDS:
|
||||
raise HTTPException(status_code=422, detail="Invalid job request")
|
||||
if not isinstance(parameters, dict) or not parameters.get("status"):
|
||||
raise HTTPException(status_code=422, detail="Missing job parameters")
|
||||
if not state.reserve(
|
||||
job_id,
|
||||
client_id,
|
||||
command,
|
||||
parameters["status"],
|
||||
):
|
||||
raise HTTPException(status_code=409, detail="Service is busy")
|
||||
worker.write_status(
|
||||
parameters["status"],
|
||||
0.0,
|
||||
"Task queued",
|
||||
state="queued",
|
||||
)
|
||||
background_tasks.add_task(state.run_job, payload)
|
||||
return {"job_id": job_id, "state": "queued"}
|
||||
|
||||
@app.delete("/jobs/{job_id}", status_code=202)
|
||||
def cancel_job(
|
||||
job_id: str,
|
||||
payload: dict,
|
||||
x_snapform_token: str = Header(default=""),
|
||||
):
|
||||
authorize(x_snapform_token)
|
||||
client_id = str(payload.get("client_id", ""))
|
||||
active = state.cancel_job(job_id, client_id)
|
||||
if active is None:
|
||||
raise HTTPException(status_code=404, detail="Active job was not found")
|
||||
try:
|
||||
current = json.loads(
|
||||
Path(active["status"]).read_text(encoding="utf-8")
|
||||
)
|
||||
progress = float(current.get("progress", 0.0))
|
||||
except (OSError, ValueError, TypeError, json.JSONDecodeError):
|
||||
progress = 0.0
|
||||
worker.write_status(
|
||||
active["status"],
|
||||
progress,
|
||||
"Cancelling task",
|
||||
state="cancelling",
|
||||
)
|
||||
return {"job_id": job_id, "state": "cancelling"}
|
||||
|
||||
@app.post("/models/clear")
|
||||
def clear_models(x_snapform_token: str = Header(default="")):
|
||||
authorize(x_snapform_token)
|
||||
if not state.clear_models():
|
||||
raise HTTPException(status_code=409, detail="Service is busy")
|
||||
return {"state": "cleared"}
|
||||
|
||||
@app.post("/shutdown", status_code=202)
|
||||
def shutdown_service(
|
||||
payload: dict,
|
||||
x_snapform_token: str = Header(default=""),
|
||||
):
|
||||
authorize(x_snapform_token)
|
||||
result = state.request_shutdown(str(payload.get("client_id", "")))
|
||||
if result == "shared":
|
||||
raise HTTPException(status_code=409, detail="Other Blender clients are connected")
|
||||
if result == "busy":
|
||||
raise HTTPException(status_code=409, detail="Service is busy")
|
||||
if result != "accepted":
|
||||
raise HTTPException(status_code=404, detail="Client is not registered")
|
||||
return {"state": "stopping"}
|
||||
|
||||
return app
|
||||
|
||||
|
||||
def parse_args():
|
||||
parser = argparse.ArgumentParser()
|
||||
parser.add_argument("--host", default="127.0.0.1")
|
||||
parser.add_argument("--port", type=int, required=True)
|
||||
parser.add_argument("--token", required=True)
|
||||
parser.add_argument("--instance-id", required=True)
|
||||
parser.add_argument("--discovery", required=True)
|
||||
parser.add_argument("--lock", required=True)
|
||||
return parser.parse_args()
|
||||
|
||||
|
||||
def write_discovery(args):
|
||||
path = Path(args.discovery)
|
||||
path.parent.mkdir(parents=True, exist_ok=True)
|
||||
temporary = path.with_name(f"{path.name}.tmp")
|
||||
temporary.write_text(
|
||||
json.dumps(
|
||||
{
|
||||
"port": args.port,
|
||||
"token": args.token,
|
||||
"pid": os.getpid(),
|
||||
"instance_id": args.instance_id,
|
||||
}
|
||||
),
|
||||
encoding="utf-8",
|
||||
)
|
||||
temporary.replace(path)
|
||||
|
||||
|
||||
def remove_discovery(path, instance_id):
|
||||
path = Path(path)
|
||||
try:
|
||||
payload = json.loads(path.read_text(encoding="utf-8"))
|
||||
except (FileNotFoundError, json.JSONDecodeError, OSError):
|
||||
return
|
||||
if payload.get("instance_id") != instance_id:
|
||||
return
|
||||
path.unlink(missing_ok=True)
|
||||
|
||||
|
||||
def main():
|
||||
import uvicorn
|
||||
|
||||
args = parse_args()
|
||||
service_lock = ServiceLock(args.lock)
|
||||
if not service_lock.acquire():
|
||||
return
|
||||
try:
|
||||
app = create_app(args.token, args.instance_id)
|
||||
config = uvicorn.Config(
|
||||
app,
|
||||
host=args.host,
|
||||
port=args.port,
|
||||
access_log=False,
|
||||
log_level="warning",
|
||||
)
|
||||
server = uvicorn.Server(config)
|
||||
|
||||
def watch_shutdown():
|
||||
app.state.snapform_service_state.shutdown_event.wait()
|
||||
server.should_exit = True
|
||||
|
||||
threading.Thread(target=watch_shutdown, daemon=True).start()
|
||||
write_discovery(args)
|
||||
server.run()
|
||||
finally:
|
||||
remove_discovery(args.discovery, args.instance_id)
|
||||
service_lock.release()
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -1,55 +0,0 @@
|
||||
from .operators import active_job
|
||||
|
||||
|
||||
def draw_status_bar(self, context):
|
||||
scene = getattr(context, "scene", None)
|
||||
job = active_job()
|
||||
if job is None or scene is None:
|
||||
return
|
||||
|
||||
settings = getattr(scene, "snapform_settings", None)
|
||||
if settings is None:
|
||||
return
|
||||
|
||||
progress = min(max(settings.progress, 0.0), 1.0)
|
||||
row = self.layout.row(align=True)
|
||||
progress_row = row.row(align=True)
|
||||
progress_row.ui_units_x = 18.0
|
||||
progress_row.progress(
|
||||
factor=progress,
|
||||
type="BAR",
|
||||
text=settings.status,
|
||||
)
|
||||
row.operator(
|
||||
"snapform.cancel_job",
|
||||
text="",
|
||||
icon="X",
|
||||
)
|
||||
|
||||
|
||||
def tag_ui_regions(context):
|
||||
window_manager = getattr(context, "window_manager", None)
|
||||
if window_manager is None:
|
||||
return
|
||||
|
||||
for window in window_manager.windows:
|
||||
screen = getattr(window, "screen", None)
|
||||
if screen is None:
|
||||
continue
|
||||
for area in screen.areas:
|
||||
if area.type in {"STATUSBAR", "VIEW_3D"}:
|
||||
area.tag_redraw()
|
||||
for region in getattr(area, "regions", ()):
|
||||
region.tag_redraw()
|
||||
|
||||
|
||||
def redraw_ui(context, force=False):
|
||||
tag_ui_regions(context)
|
||||
if not force:
|
||||
return
|
||||
workspace = getattr(context, "workspace", None)
|
||||
if workspace is not None:
|
||||
try:
|
||||
workspace.status_text_set_internal(None)
|
||||
except (AttributeError, RuntimeError, TypeError):
|
||||
pass
|
||||
File diff suppressed because it is too large
Load Diff
194
sync_models.py
Normal file
194
sync_models.py
Normal file
@@ -0,0 +1,194 @@
|
||||
"""Prepare SnapForm ONNX models and sync them to OmooLab R2."""
|
||||
|
||||
import argparse
|
||||
import hashlib
|
||||
import shutil
|
||||
import subprocess
|
||||
import sys
|
||||
from dataclasses import dataclass
|
||||
from pathlib import Path
|
||||
from urllib.request import Request, urlopen
|
||||
|
||||
|
||||
PROJECT_ROOT = Path(__file__).resolve().parent
|
||||
MODELS_DIR = PROJECT_ROOT / "models"
|
||||
R2_ENDPOINT = "https://models.omoolab.xyz"
|
||||
CHUNK_SIZE = 1024 * 1024
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class ModelFile:
|
||||
folder: str
|
||||
size: int
|
||||
sha256: str
|
||||
sources: tuple[str, ...]
|
||||
|
||||
@property
|
||||
def destination(self):
|
||||
filename = "model_fp16.onnx" if self.folder.endswith("/onnx") else "model.onnx"
|
||||
return MODELS_DIR / self.folder / filename
|
||||
|
||||
|
||||
def hf_url(repository, filename):
|
||||
return f"https://huggingface.co/{repository}/resolve/main/{filename}"
|
||||
|
||||
|
||||
def r2_url(folder, filename="model.onnx"):
|
||||
return f"{R2_ENDPOINT}/{folder}/{filename}"
|
||||
|
||||
|
||||
MODEL_FILES = (
|
||||
ModelFile(
|
||||
"DA3-SMALL",
|
||||
105484029,
|
||||
"7a9d81475b3e2334ace0d6d2c2a816b3c93081437e1866cc434ce1789125f631",
|
||||
(r2_url("DA3-SMALL"),),
|
||||
),
|
||||
ModelFile(
|
||||
"DA3-BASE",
|
||||
412856697,
|
||||
"31f38fc4990539d7902ba267924b9731db1acb00d840c248a60b1865810ff9e6",
|
||||
(r2_url("DA3-BASE"),),
|
||||
),
|
||||
ModelFile(
|
||||
"BEN2-ONNX/onnx",
|
||||
219121675,
|
||||
"dfdc25f421f32a0d1268e0f2ff2153d340e8f1d52d3dd16f5dc33c1ce85cedf1",
|
||||
(
|
||||
hf_url("onnx-community/BEN2-ONNX", "onnx/model_fp16.onnx"),
|
||||
r2_url("BEN2-ONNX", "onnx/model_fp16.onnx"),
|
||||
),
|
||||
),
|
||||
ModelFile(
|
||||
"moge-2-vits-normal-onnx",
|
||||
140852051,
|
||||
"24eacb5dc7a2c54c7bc98f7de085ffbed79ad006ea5b664c2c2cdc02ff3a52f0",
|
||||
(
|
||||
hf_url("Ruicheng/moge-2-vits-normal-onnx", "model.onnx"),
|
||||
r2_url("moge-2-vits-normal-onnx"),
|
||||
),
|
||||
),
|
||||
ModelFile(
|
||||
"moge-2-vitb-normal-onnx",
|
||||
419411850,
|
||||
"bbf14e07a30f11e69d36ab861590123f5598ababcbc8946a063eb4a966f35a21",
|
||||
(
|
||||
hf_url("Ruicheng/moge-2-vitb-normal-onnx", "model.onnx"),
|
||||
r2_url("moge-2-vitb-normal-onnx"),
|
||||
),
|
||||
),
|
||||
ModelFile(
|
||||
"moge-2-vitl-normal-onnx",
|
||||
1324265014,
|
||||
"afbc4ccc3450298f3afb35b90f015f4c4f552dea21dc6470d5f7b78b77e2d751",
|
||||
(
|
||||
hf_url("Ruicheng/moge-2-vitl-normal-onnx", "model.onnx"),
|
||||
r2_url("moge-2-vitl-normal-onnx"),
|
||||
),
|
||||
),
|
||||
)
|
||||
|
||||
|
||||
def file_sha256(path):
|
||||
digest = hashlib.sha256()
|
||||
with Path(path).open("rb") as stream:
|
||||
while chunk := stream.read(CHUNK_SIZE):
|
||||
digest.update(chunk)
|
||||
return digest.hexdigest()
|
||||
|
||||
|
||||
def verify_existing(model):
|
||||
path = model.destination
|
||||
if not path.is_file():
|
||||
return False
|
||||
if path.stat().st_size != model.size:
|
||||
raise RuntimeError(f"Existing model has an unexpected size: {path}")
|
||||
if file_sha256(path) != model.sha256:
|
||||
raise RuntimeError(f"Existing model has an unexpected SHA-256: {path}")
|
||||
print(f"Skip existing: {path.relative_to(PROJECT_ROOT)}")
|
||||
return True
|
||||
|
||||
|
||||
def download_once(model, url):
|
||||
destination = model.destination
|
||||
destination.parent.mkdir(parents=True, exist_ok=True)
|
||||
temporary = destination.with_suffix(".onnx.part")
|
||||
temporary.unlink(missing_ok=True)
|
||||
request = Request(url, headers={"User-Agent": "SnapForm model sync"})
|
||||
print(f"Download: {url}")
|
||||
downloaded = 0
|
||||
digest = hashlib.sha256()
|
||||
try:
|
||||
with urlopen(request, timeout=60) as response, temporary.open("wb") as output:
|
||||
while chunk := response.read(CHUNK_SIZE):
|
||||
output.write(chunk)
|
||||
digest.update(chunk)
|
||||
downloaded += len(chunk)
|
||||
print(
|
||||
f"\r {downloaded / 1024**2:.1f} / "
|
||||
f"{model.size / 1024**2:.1f} MiB",
|
||||
end="",
|
||||
flush=True,
|
||||
)
|
||||
print()
|
||||
if downloaded != model.size:
|
||||
raise RuntimeError(f"size mismatch: {downloaded} != {model.size}")
|
||||
if digest.hexdigest() != model.sha256:
|
||||
raise RuntimeError("SHA-256 mismatch")
|
||||
temporary.replace(destination)
|
||||
except Exception:
|
||||
temporary.unlink(missing_ok=True)
|
||||
raise
|
||||
|
||||
|
||||
def prepare_model(model):
|
||||
if verify_existing(model):
|
||||
return
|
||||
failures = []
|
||||
for url in model.sources:
|
||||
try:
|
||||
download_once(model, url)
|
||||
return
|
||||
except Exception as error:
|
||||
failures.append(f"{url}: {error}")
|
||||
print(f"Source failed: {error}", file=sys.stderr)
|
||||
raise RuntimeError("All sources failed:\n" + "\n".join(failures))
|
||||
|
||||
|
||||
def sync_models():
|
||||
executable = shutil.which("rclone")
|
||||
if executable is None:
|
||||
raise RuntimeError("rclone is not available on PATH")
|
||||
subprocess.run(
|
||||
[
|
||||
executable,
|
||||
"sync",
|
||||
"models",
|
||||
"omoolab-r2:ai-models",
|
||||
"--progress",
|
||||
],
|
||||
cwd=PROJECT_ROOT,
|
||||
check=True,
|
||||
)
|
||||
|
||||
|
||||
def parse_args():
|
||||
parser = argparse.ArgumentParser()
|
||||
parser.add_argument(
|
||||
"--download-only",
|
||||
action="store_true",
|
||||
help="Prepare and verify models without running rclone",
|
||||
)
|
||||
return parser.parse_args()
|
||||
|
||||
|
||||
def main():
|
||||
args = parse_args()
|
||||
for model in MODEL_FILES:
|
||||
prepare_model(model)
|
||||
if not args.download_only:
|
||||
sync_models()
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
3114
tests/test_addon.py
3114
tests/test_addon.py
File diff suppressed because it is too large
Load Diff
58
tests/test_ben2_onnx.py
Normal file
58
tests/test_ben2_onnx.py
Normal file
@@ -0,0 +1,58 @@
|
||||
import sys
|
||||
import unittest
|
||||
from pathlib import Path
|
||||
from types import SimpleNamespace
|
||||
|
||||
import numpy as np
|
||||
from PIL import Image
|
||||
|
||||
|
||||
SOURCE_DIR = (
|
||||
Path(__file__).resolve().parents[1]
|
||||
/ "src"
|
||||
/ "snapform"
|
||||
/ "server"
|
||||
)
|
||||
sys.path.insert(0, str(SOURCE_DIR))
|
||||
|
||||
import onnx_ben2
|
||||
|
||||
|
||||
class FakeSession:
|
||||
def get_inputs(self):
|
||||
return [SimpleNamespace(name="input")]
|
||||
|
||||
def get_outputs(self):
|
||||
return [SimpleNamespace(name="alpha")]
|
||||
|
||||
def run(self, output_names, feeds):
|
||||
self.output_names = output_names
|
||||
self.feeds = feeds
|
||||
gradient = np.linspace(0.0, 1.0, 1024, dtype=np.float32)
|
||||
return [np.tile(gradient, (1024, 1))[None, None]]
|
||||
|
||||
|
||||
class Ben2OnnxTest(unittest.TestCase):
|
||||
def test_preprocess_matches_model_layout(self):
|
||||
image = Image.new("RGB", (40, 20), (64, 128, 192))
|
||||
result = onnx_ben2.preprocess(image)
|
||||
|
||||
self.assertEqual(result.shape, (1, 3, 1024, 1024))
|
||||
self.assertEqual(result.dtype, np.float32)
|
||||
self.assertTrue(result.flags.c_contiguous)
|
||||
|
||||
def test_infer_restores_original_size_and_alpha(self):
|
||||
session = FakeSession()
|
||||
image = Image.new("RGB", (40, 20), (10, 20, 30))
|
||||
|
||||
result = onnx_ben2.infer(session, image)
|
||||
|
||||
self.assertEqual(result.mode, "RGBA")
|
||||
self.assertEqual(result.size, image.size)
|
||||
self.assertEqual(session.feeds["input"].shape, (1, 3, 1024, 1024))
|
||||
alpha = np.asarray(result.getchannel("A"))
|
||||
self.assertLess(int(alpha[:, 0].max()), int(alpha[:, -1].min()))
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
66
tests/test_da3_onnx.py
Normal file
66
tests/test_da3_onnx.py
Normal file
@@ -0,0 +1,66 @@
|
||||
import sys
|
||||
import tempfile
|
||||
import unittest
|
||||
from pathlib import Path
|
||||
from types import SimpleNamespace
|
||||
|
||||
import numpy as np
|
||||
from PIL import Image
|
||||
|
||||
|
||||
SOURCE_DIR = (
|
||||
Path(__file__).resolve().parents[1]
|
||||
/ "src"
|
||||
/ "snapform"
|
||||
/ "server"
|
||||
)
|
||||
sys.path.insert(0, str(SOURCE_DIR))
|
||||
|
||||
import onnx_da3
|
||||
|
||||
|
||||
class FakeSession:
|
||||
def get_inputs(self):
|
||||
return [SimpleNamespace(name="images"), SimpleNamespace(name="strategy")]
|
||||
|
||||
def run(self, output_names, feeds):
|
||||
self.feeds = feeds
|
||||
_, views, _, height, width = feeds["images"].shape
|
||||
depth = np.ones((1, views, height, width), dtype=np.float32)
|
||||
confidence = np.full_like(depth, 0.75)
|
||||
pose = np.zeros((1, views, 9), dtype=np.float32)
|
||||
pose[..., 6] = 1.0
|
||||
pose[..., 7:] = np.pi / 2.0
|
||||
return depth, confidence, pose
|
||||
|
||||
|
||||
class Da3OnnxTest(unittest.TestCase):
|
||||
def test_target_size_uses_patch_multiple(self):
|
||||
self.assertEqual(
|
||||
onnx_da3.target_size((800, 600), 504, "upper_bound_resize"),
|
||||
(504, 378),
|
||||
)
|
||||
|
||||
def test_infer_returns_camera_and_image_arrays(self):
|
||||
with tempfile.TemporaryDirectory() as directory:
|
||||
image_path = Path(directory) / "input.png"
|
||||
Image.new("RGB", (80, 60), (10, 20, 30)).save(image_path)
|
||||
session = FakeSession()
|
||||
|
||||
result = onnx_da3.infer(
|
||||
session,
|
||||
[image_path],
|
||||
process_res=56,
|
||||
ref_view_strategy="middle",
|
||||
)
|
||||
|
||||
self.assertEqual(result.depth.shape, (1, 42, 56))
|
||||
self.assertEqual(result.conf.shape, result.depth.shape)
|
||||
self.assertEqual(result.processed_images.shape, (1, 42, 56, 3))
|
||||
self.assertEqual(result.extrinsics.shape, (1, 3, 4))
|
||||
self.assertEqual(result.intrinsics.shape, (1, 3, 3))
|
||||
self.assertEqual(int(session.feeds["strategy"][0]), 1)
|
||||
np.testing.assert_allclose(result.extrinsics[0, :3, :3], np.eye(3))
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
@@ -1,23 +1,24 @@
|
||||
import importlib.util
|
||||
import json
|
||||
import os
|
||||
import tempfile
|
||||
import sys
|
||||
import unittest
|
||||
from pathlib import Path
|
||||
from types import SimpleNamespace
|
||||
from unittest.mock import patch
|
||||
|
||||
|
||||
PROJECT_ROOT = Path(__file__).resolve().parents[1]
|
||||
|
||||
|
||||
def load_worker():
|
||||
specification = importlib.util.spec_from_file_location(
|
||||
"snapform_download_worker",
|
||||
PROJECT_ROOT / "src" / "snapform" / "worker.py",
|
||||
)
|
||||
module = importlib.util.module_from_spec(specification)
|
||||
specification.loader.exec_module(module)
|
||||
return module
|
||||
def load_download_module():
|
||||
addon_directory = PROJECT_ROOT / "src" / "snapform"
|
||||
sys.path.insert(0, str(addon_directory))
|
||||
try:
|
||||
return __import__(
|
||||
"server.model_download",
|
||||
fromlist=["model_download"],
|
||||
)
|
||||
finally:
|
||||
sys.path.remove(str(addon_directory))
|
||||
|
||||
|
||||
class FakeTqdm:
|
||||
@@ -36,29 +37,63 @@ class FakeTqdm:
|
||||
class DownloadProgressTest(unittest.TestCase):
|
||||
@classmethod
|
||||
def setUpClass(cls):
|
||||
cls.worker = load_worker()
|
||||
cls.download = load_download_module()
|
||||
|
||||
def create_progress(self, description="Downloading bytes", **options):
|
||||
reports = []
|
||||
progress_class = self.download.create_download_progress_class(
|
||||
lambda value, message: reports.append((value, message)),
|
||||
"owner/model",
|
||||
tqdm_type=options.pop("tqdm_type", FakeTqdm),
|
||||
clock=options.pop("clock", lambda: 1.0),
|
||||
**options,
|
||||
)
|
||||
progress = progress_class(
|
||||
desc=description,
|
||||
total=100,
|
||||
initial=0,
|
||||
unit="B",
|
||||
)
|
||||
return progress, reports
|
||||
|
||||
def test_https_connection_bounds_connect_and_idle_read_time(self):
|
||||
connect_timeouts = []
|
||||
read_timeouts = []
|
||||
|
||||
def connect(connection):
|
||||
connect_timeouts.append(connection.timeout)
|
||||
connection.sock = SimpleNamespace(
|
||||
settimeout=read_timeouts.append,
|
||||
)
|
||||
|
||||
connection = self.download._ShortConnectHTTPSConnection("example.com")
|
||||
with patch.object(
|
||||
self.download.http.client.HTTPSConnection,
|
||||
"connect",
|
||||
new=connect,
|
||||
):
|
||||
connection.connect()
|
||||
|
||||
self.assertEqual(connect_timeouts, [10.0])
|
||||
self.assertEqual(read_timeouts, [60.0])
|
||||
self.assertIsNone(connection.timeout)
|
||||
|
||||
def test_reports_aggregated_download_bytes(self):
|
||||
with tempfile.TemporaryDirectory() as directory:
|
||||
status_path = Path(directory) / "status.json"
|
||||
progress_class = self.worker.create_download_progress_class(
|
||||
status_path,
|
||||
"owner/model",
|
||||
tqdm_type=FakeTqdm,
|
||||
clock=lambda: 1.0,
|
||||
)
|
||||
progress = progress_class(
|
||||
desc="Downloading bytes",
|
||||
total=100,
|
||||
initial=0,
|
||||
unit="B",
|
||||
)
|
||||
progress, reports = self.create_progress()
|
||||
|
||||
progress.update(25)
|
||||
progress.update(25)
|
||||
|
||||
status = json.loads(status_path.read_text(encoding="utf-8"))
|
||||
self.assertAlmostEqual(status["progress"], 0.275)
|
||||
self.assertEqual(status["message"], "Downloading owner/model")
|
||||
self.assertAlmostEqual(reports[-1][0], 0.275)
|
||||
self.assertEqual(reports[-1][1], "Downloading owner/model")
|
||||
|
||||
def test_reports_hugging_face_aggregated_bar(self):
|
||||
progress, reports = self.create_progress(
|
||||
"Downloading (incomplete total...)"
|
||||
)
|
||||
|
||||
progress.update(25)
|
||||
|
||||
self.assertAlmostEqual(reports[-1][0], 0.275)
|
||||
|
||||
def test_reports_download_speed(self):
|
||||
class FakeTqdmWithRate(FakeTqdm):
|
||||
@@ -66,129 +101,48 @@ class DownloadProgressTest(unittest.TestCase):
|
||||
def format_dict(self):
|
||||
return {"rate": 14_000_000}
|
||||
|
||||
with tempfile.TemporaryDirectory() as directory:
|
||||
status_path = Path(directory) / "status.json"
|
||||
progress_class = self.worker.create_download_progress_class(
|
||||
status_path,
|
||||
"owner/model",
|
||||
tqdm_type=FakeTqdmWithRate,
|
||||
clock=lambda: 1.0,
|
||||
label="da3 small",
|
||||
)
|
||||
progress = progress_class(
|
||||
desc="Downloading bytes",
|
||||
total=100,
|
||||
initial=0,
|
||||
unit="B",
|
||||
)
|
||||
progress, reports = self.create_progress(
|
||||
tqdm_type=FakeTqdmWithRate,
|
||||
label="da3 small",
|
||||
)
|
||||
|
||||
progress.update(50)
|
||||
progress.update(50)
|
||||
|
||||
status = json.loads(status_path.read_text(encoding="utf-8"))
|
||||
self.assertEqual(
|
||||
status["message"],
|
||||
"Downloading da3 small | 13.4 MB/s",
|
||||
)
|
||||
self.assertEqual(
|
||||
reports[-1][1],
|
||||
"Downloading da3 small 13.4 MB/s",
|
||||
)
|
||||
|
||||
def test_reports_estimated_time_remaining(self):
|
||||
def test_progress_callback_has_no_eta_or_state_protocol(self):
|
||||
times = iter((0.0, 10.0))
|
||||
with tempfile.TemporaryDirectory() as directory:
|
||||
status_path = Path(directory) / "status.json"
|
||||
progress_class = self.worker.create_download_progress_class(
|
||||
status_path,
|
||||
"owner/model",
|
||||
tqdm_type=FakeTqdm,
|
||||
clock=lambda: next(times),
|
||||
)
|
||||
progress = progress_class(
|
||||
desc="Downloading bytes",
|
||||
total=100,
|
||||
initial=0,
|
||||
unit="B",
|
||||
)
|
||||
progress, reports = self.create_progress(clock=lambda: next(times))
|
||||
|
||||
progress.update(50)
|
||||
progress.update(50)
|
||||
|
||||
status = json.loads(status_path.read_text(encoding="utf-8"))
|
||||
self.assertEqual(status["eta_seconds"], 10)
|
||||
|
||||
def test_staged_download_prefixes_speed_with_downloading(self):
|
||||
class FakeTqdmWithRate(FakeTqdm):
|
||||
@property
|
||||
def format_dict(self):
|
||||
return {"rate": 14_000_000}
|
||||
|
||||
with tempfile.TemporaryDirectory() as directory:
|
||||
status_path = Path(directory) / "status.json"
|
||||
progress_class = self.worker.create_download_progress_class(
|
||||
status_path,
|
||||
"owner/model",
|
||||
tqdm_type=FakeTqdmWithRate,
|
||||
clock=lambda: 1.0,
|
||||
label="da3 small",
|
||||
stage=6,
|
||||
stages=8,
|
||||
stage_label="da3 small",
|
||||
)
|
||||
progress = progress_class(
|
||||
desc="Downloading bytes",
|
||||
total=100,
|
||||
initial=0,
|
||||
unit="B",
|
||||
)
|
||||
|
||||
progress.update(50)
|
||||
|
||||
status = json.loads(status_path.read_text(encoding="utf-8"))
|
||||
self.assertEqual(
|
||||
status["message"],
|
||||
"Downloading packages | 13.4 MB/s",
|
||||
)
|
||||
self.assertEqual(status["stage"], 6)
|
||||
self.assertEqual(status["stages"], 8)
|
||||
self.assertEqual(status["stage_label"], "da3 small")
|
||||
self.assertEqual(len(reports[-1]), 2)
|
||||
|
||||
def test_ignores_unrelated_progress_bars(self):
|
||||
with tempfile.TemporaryDirectory() as directory:
|
||||
status_path = Path(directory) / "status.json"
|
||||
progress_class = self.worker.create_download_progress_class(
|
||||
status_path,
|
||||
"owner/model",
|
||||
tqdm_type=FakeTqdm,
|
||||
clock=lambda: 1.0,
|
||||
)
|
||||
progress = progress_class(
|
||||
desc="Reconstructing",
|
||||
total=100,
|
||||
initial=0,
|
||||
unit="B",
|
||||
)
|
||||
progress, reports = self.create_progress("Reconstructing")
|
||||
|
||||
progress.update(25)
|
||||
progress.update(25)
|
||||
|
||||
self.assertFalse(status_path.exists())
|
||||
self.assertEqual(reports, [])
|
||||
|
||||
def test_download_endpoints_uses_direct_endpoint(self):
|
||||
with patch.dict(os.environ, {"HF_ENDPOINT": ""}):
|
||||
endpoints = self.worker.download_endpoints()
|
||||
endpoints = self.download.download_endpoints()
|
||||
|
||||
self.assertEqual(
|
||||
endpoints,
|
||||
["https://huggingface.co"],
|
||||
)
|
||||
self.assertEqual(endpoints, ["https://huggingface.co"])
|
||||
|
||||
def test_download_endpoints_respects_environment_override(self):
|
||||
with patch.dict(os.environ, {"HF_ENDPOINT": "https://hf.example"}):
|
||||
endpoints = self.worker.download_endpoints()
|
||||
endpoints = self.download.download_endpoints()
|
||||
|
||||
self.assertEqual(
|
||||
endpoints,
|
||||
["https://hf.example"],
|
||||
)
|
||||
self.assertEqual(endpoints, ["https://hf.example"])
|
||||
|
||||
def test_download_endpoints_strips_trailing_slash(self):
|
||||
with patch.dict(os.environ, {"HF_ENDPOINT": "https://hf.example/"}):
|
||||
endpoints = self.worker.download_endpoints()
|
||||
endpoints = self.download.download_endpoints()
|
||||
|
||||
self.assertEqual(endpoints, ["https://hf.example"])
|
||||
|
||||
|
||||
@@ -26,7 +26,11 @@ def load_image_depth():
|
||||
with patch.dict(sys.modules, {"bpy": fake_bpy}):
|
||||
specification = importlib.util.spec_from_file_location(
|
||||
"snapform_image_depth_test",
|
||||
PROJECT_ROOT / "src" / "snapform" / "image_depth.py",
|
||||
PROJECT_ROOT
|
||||
/ "src"
|
||||
/ "snapform"
|
||||
/ "blender"
|
||||
/ "depth_plane.py",
|
||||
)
|
||||
module = importlib.util.module_from_spec(specification)
|
||||
specification.loader.exec_module(module)
|
||||
@@ -601,6 +605,21 @@ class ImageDepthTest(unittest.TestCase):
|
||||
"img.001.depth",
|
||||
)
|
||||
|
||||
def test_moge_map_names_use_source_image_name(self):
|
||||
source_object = SimpleNamespace(
|
||||
name="obj_img",
|
||||
data=SimpleNamespace(name="img.png"),
|
||||
)
|
||||
|
||||
self.assertEqual(
|
||||
self.image_depth.normal_data_name(source_object),
|
||||
"img.normal",
|
||||
)
|
||||
self.assertEqual(
|
||||
self.image_depth.mask_data_name(source_object),
|
||||
"img.mask",
|
||||
)
|
||||
|
||||
def test_copy_depth_image_renames_still_source(self):
|
||||
class FakeImage:
|
||||
def __init__(self, name):
|
||||
|
||||
File diff suppressed because it is too large
Load Diff
@@ -21,7 +21,7 @@ class ModelTest(unittest.TestCase):
|
||||
def setUpClass(cls):
|
||||
cls.models = load_module(
|
||||
"snapform_models",
|
||||
"src/snapform/models.py",
|
||||
"src/snapform/model_catalog.py",
|
||||
)
|
||||
|
||||
def test_base_model_is_default_friendly_apache_model(self):
|
||||
@@ -30,12 +30,6 @@ class ModelTest(unittest.TestCase):
|
||||
self.assertEqual(model.repository, "depth-anything/DA3-BASE")
|
||||
self.assertEqual(model.license_name, "Apache 2.0")
|
||||
|
||||
def test_large_model_exposes_noncommercial_license(self):
|
||||
model = self.models.get_model("LARGE_1_1")
|
||||
|
||||
self.assertEqual(model.license_name, "CC BY-NC 4.0")
|
||||
self.assertFalse(model.enabled)
|
||||
|
||||
def test_giant_models_expose_gaussian_head(self):
|
||||
for model_key in ("GIANT_1_1", "NESTED_GIANT_LARGE_1_1"):
|
||||
with self.subTest(model_key=model_key):
|
||||
@@ -54,19 +48,20 @@ class ModelTest(unittest.TestCase):
|
||||
def test_ben2_download_only_includes_required_hugging_face_files(self):
|
||||
model = self.models.get_downloadable_model("BEN2_BASE")
|
||||
|
||||
self.assertEqual(model.repository, "PramaLLC/BEN2")
|
||||
self.assertEqual(model.license_name, "MIT")
|
||||
self.assertEqual(
|
||||
model.download_patterns,
|
||||
("config.json", "model.safetensors"),
|
||||
)
|
||||
self.assertEqual(model.repository, "onnx-community/BEN2-ONNX")
|
||||
self.assertEqual(model.license_name, "Apache 2.0")
|
||||
self.assertEqual(model.download_patterns, ("onnx/model_fp16.onnx",))
|
||||
self.assertEqual(model.ready_patterns, ("onnx/model_fp16.onnx",))
|
||||
self.assertEqual(model.r2_files[0][0], "onnx/model_fp16.onnx")
|
||||
self.assertNotIn("BEN2_BASE", self.models.MODELS)
|
||||
|
||||
def test_da3_model_directories_share_the_da3_prefix(self):
|
||||
def test_da3_model_directories_use_repository_names(self):
|
||||
for model_key, model in self.models.MODELS.items():
|
||||
if not model.enabled:
|
||||
continue
|
||||
with self.subTest(model_key=model_key):
|
||||
self.assertTrue(
|
||||
model.directory_name.startswith("da3_"),
|
||||
model.directory_name.startswith("DA3-"),
|
||||
model.directory_name,
|
||||
)
|
||||
self.assertEqual(
|
||||
@@ -74,20 +69,37 @@ class ModelTest(unittest.TestCase):
|
||||
model.directory_name,
|
||||
)
|
||||
|
||||
def test_ben2_model_directory_uses_ben2_prefix(self):
|
||||
def test_ben2_model_directory_uses_repository_name(self):
|
||||
self.assertEqual(
|
||||
self.models.model_directory_name("BEN2_BASE"),
|
||||
"ben2_base",
|
||||
"BEN2-ONNX",
|
||||
)
|
||||
|
||||
def test_moge2_uses_single_checkpoint_and_mit_license(self):
|
||||
model = self.models.get_downloadable_model("MOGE2_VITB_NORMAL")
|
||||
def test_enabled_da3_models_use_onnx_and_r2_checksums(self):
|
||||
for model in self.models.MODELS.values():
|
||||
if not model.enabled:
|
||||
continue
|
||||
with self.subTest(model=model.key):
|
||||
self.assertTrue(model.download_patterns[0].endswith(".onnx"))
|
||||
self.assertEqual(model.ready_patterns, ("model.onnx",))
|
||||
self.assertEqual(model.r2_files[0][0], "model.onnx")
|
||||
|
||||
self.assertEqual(model.repository, "Ruicheng/moge-2-vitb-normal")
|
||||
self.assertEqual(model.license_name, "MIT")
|
||||
self.assertEqual(model.download_patterns, ("model.pt",))
|
||||
self.assertEqual(model.ready_patterns, ("model.pt",))
|
||||
self.assertFalse(model.supports_gs)
|
||||
def test_moge2_models_use_official_onnx_models(self):
|
||||
for key, variant in (("MOGE2_VITS_NORMAL", "vits"),
|
||||
("MOGE2_VITB_NORMAL", "vitb"),
|
||||
("MOGE2_VITL_NORMAL", "vitl")):
|
||||
with self.subTest(key=key):
|
||||
model = self.models.get_downloadable_model(key)
|
||||
self.assertEqual(
|
||||
model.repository,
|
||||
f"Ruicheng/moge-2-{variant}-normal-onnx",
|
||||
)
|
||||
self.assertEqual(model.license_name, "MIT")
|
||||
self.assertEqual(model.download_patterns, ("model.onnx",))
|
||||
self.assertEqual(model.ready_patterns, ("model.onnx",))
|
||||
self.assertEqual(model.r2_directory, model.repository.split("/")[-1])
|
||||
self.assertEqual(model.r2_files[0][0], "model.onnx")
|
||||
self.assertFalse(model.supports_gs)
|
||||
|
||||
def test_moge2_models_match_uploaded_r2_variants(self):
|
||||
self.assertEqual(
|
||||
@@ -96,5 +108,9 @@ class ModelTest(unittest.TestCase):
|
||||
)
|
||||
self.assertEqual(
|
||||
[model.r2_directory for model in self.models.MOGE2_MODELS.values()],
|
||||
["MoGe2ViTSNormal", "MoGe2ViTBNormal", "MoGe2ViTLNormal"],
|
||||
[
|
||||
"moge-2-vits-normal-onnx",
|
||||
"moge-2-vitb-normal-onnx",
|
||||
"moge-2-vitl-normal-onnx",
|
||||
],
|
||||
)
|
||||
|
||||
136
tests/test_moge_onnx.py
Normal file
136
tests/test_moge_onnx.py
Normal file
@@ -0,0 +1,136 @@
|
||||
import importlib.util
|
||||
import sys
|
||||
import unittest
|
||||
from pathlib import Path
|
||||
from types import ModuleType
|
||||
from unittest.mock import patch
|
||||
|
||||
import numpy as np
|
||||
|
||||
|
||||
PROJECT_ROOT = Path(__file__).resolve().parents[1]
|
||||
|
||||
|
||||
def load_module():
|
||||
snapform = ModuleType("snapform")
|
||||
snapform.__path__ = [str(PROJECT_ROOT / "src" / "snapform")]
|
||||
server = ModuleType("snapform.server")
|
||||
server.__path__ = [str(PROJECT_ROOT / "src" / "snapform" / "server")]
|
||||
specification = importlib.util.spec_from_file_location(
|
||||
"snapform.server.onnx_moge2",
|
||||
PROJECT_ROOT / "src" / "snapform" / "server" / "onnx_moge2.py",
|
||||
)
|
||||
module = importlib.util.module_from_spec(specification)
|
||||
with patch.dict(sys.modules, {"snapform": snapform, "snapform.server": server}):
|
||||
specification.loader.exec_module(module)
|
||||
return module
|
||||
|
||||
|
||||
class FakeInput:
|
||||
def __init__(self, name):
|
||||
self.name = name
|
||||
|
||||
|
||||
class FakeSession:
|
||||
def __init__(self, outputs):
|
||||
self.outputs = outputs
|
||||
self.calls = []
|
||||
|
||||
def run(self, names, inputs):
|
||||
self.calls.append((names, inputs))
|
||||
return [self.outputs[name] for name in names]
|
||||
|
||||
def get_outputs(self):
|
||||
return [FakeInput(name) for name in self.outputs]
|
||||
|
||||
|
||||
class MogeOnnxTest(unittest.TestCase):
|
||||
@classmethod
|
||||
def setUpClass(cls):
|
||||
cls.moge = load_module()
|
||||
|
||||
def test_auto_provider_prefers_accelerated_runtime(self):
|
||||
providers = self.moge.select_providers(
|
||||
"auto",
|
||||
["CPUExecutionProvider", "DmlExecutionProvider"],
|
||||
)
|
||||
|
||||
self.assertEqual(
|
||||
providers,
|
||||
["DmlExecutionProvider", "CPUExecutionProvider"],
|
||||
)
|
||||
|
||||
def test_explicit_unavailable_provider_falls_back_to_cpu(self):
|
||||
providers = self.moge.select_providers(
|
||||
"cuda",
|
||||
["CPUExecutionProvider"],
|
||||
)
|
||||
|
||||
self.assertEqual(providers, ["CPUExecutionProvider"])
|
||||
|
||||
def test_infer_uses_dynamic_token_input_and_numpy_postprocess(self):
|
||||
height, width = 8, 12
|
||||
raw = {
|
||||
"points": np.zeros((1, height, width, 3), dtype=np.float32),
|
||||
"normal": np.zeros((1, height, width, 3), dtype=np.float32),
|
||||
"mask": np.ones((1, height, width), dtype=np.float32),
|
||||
"scale": np.ones((1,), dtype=np.float32),
|
||||
}
|
||||
raw["points"][..., 2] = 1.0
|
||||
raw["normal"][..., 2] = 1.0
|
||||
session = FakeSession(raw)
|
||||
expected = {
|
||||
"points": np.zeros((height, width, 3), dtype=np.float32),
|
||||
"depth": np.ones((height, width), dtype=np.float32),
|
||||
"normal": raw["normal"][0],
|
||||
"mask": np.ones((height, width), dtype=bool),
|
||||
"intrinsics": np.eye(3, dtype=np.float32),
|
||||
}
|
||||
|
||||
with patch.object(self.moge, "postprocess", return_value=expected):
|
||||
result = self.moge.infer(
|
||||
session,
|
||||
np.zeros((height, width, 3), dtype=np.float32),
|
||||
resolution_level=9,
|
||||
)
|
||||
|
||||
self.assertIs(result, expected)
|
||||
names, inputs = session.calls[0]
|
||||
self.assertEqual(names, ["points", "normal", "mask", "scale"])
|
||||
self.assertEqual(inputs["image"].shape, (1, 3, height, width))
|
||||
self.assertEqual(inputs["num_tokens"].dtype, np.int64)
|
||||
self.assertEqual(inputs["num_tokens"], 3600)
|
||||
|
||||
def test_postprocess_recovers_camera_projection_and_metric_scale(self):
|
||||
height, width = 64, 80
|
||||
focal = np.float32(0.9)
|
||||
shift = np.float32(0.3)
|
||||
metric_scale = np.float32(2.0)
|
||||
uv = self.moge.normalized_view_plane_uv(width, height, np.float32)
|
||||
depth = np.linspace(1.0, 2.4, height * width, dtype=np.float32).reshape(
|
||||
height,
|
||||
width,
|
||||
)
|
||||
raw_points = np.concatenate(
|
||||
(uv * depth[..., None] / focal, (depth - shift)[..., None]),
|
||||
axis=-1,
|
||||
)
|
||||
raw = {
|
||||
"points": raw_points[None],
|
||||
"normal": np.broadcast_to(
|
||||
np.array((0.0, 0.0, 1.0), dtype=np.float32),
|
||||
(1, height, width, 3),
|
||||
),
|
||||
"mask": np.ones((1, height, width), dtype=np.float32),
|
||||
"metric_scale": np.array((metric_scale,), dtype=np.float32),
|
||||
}
|
||||
|
||||
result = self.moge.postprocess(raw)
|
||||
|
||||
np.testing.assert_allclose(result["depth"], depth * metric_scale, rtol=1e-3)
|
||||
np.testing.assert_allclose(result["points"][..., 2], result["depth"], rtol=1e-5)
|
||||
self.assertTrue(result["mask"].all())
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
@@ -1,7 +1,4 @@
|
||||
import ast
|
||||
import importlib.util
|
||||
import io
|
||||
import tarfile
|
||||
import tempfile
|
||||
import tomllib
|
||||
import unittest
|
||||
@@ -55,32 +52,41 @@ class PackTest(unittest.TestCase):
|
||||
|
||||
self.assertIn("__init__.py", names)
|
||||
self.assertIn("blender_manifest.toml", names)
|
||||
self.assertIn("image_depth.py", names)
|
||||
self.assertIn("blender/depth_plane.py", names)
|
||||
self.assertIn("blender/surface.py", names)
|
||||
self.assertIn(
|
||||
"assets/O_SnapForm.blend",
|
||||
names,
|
||||
)
|
||||
self.assertIn("worker.py", names)
|
||||
self.assertIn("service.py", names)
|
||||
self.assertNotIn("snapform/__init__.py", names)
|
||||
self.assertNotIn("importer.py", names)
|
||||
self.assertIn("vendor/depth-anything-3.tar.gz", names)
|
||||
self.assertIn("vendor/ben2.tar.gz", names)
|
||||
self.assertIn("server/media.py", names)
|
||||
self.assertIn("server/geometry.py", names)
|
||||
self.assertIn("server/background.py", names)
|
||||
self.assertIn("server/moge2.py", names)
|
||||
self.assertIn("server/da3.py", names)
|
||||
self.assertNotIn("server/processing.py", names)
|
||||
self.assertNotIn("server/progress.py", names)
|
||||
self.assertIn("model_catalog.py", names)
|
||||
self.assertIn("server/model_download.py", names)
|
||||
self.assertIn("server/onnx_ben2.py", names)
|
||||
self.assertIn("server/onnx_da3.py", names)
|
||||
self.assertIn("server/onnx_moge2.py", names)
|
||||
self.assertNotIn("server/jobs.py", names)
|
||||
self.assertIn("server/__init__.py", names)
|
||||
self.assertIn(
|
||||
f"vendor/{pack.PLATFORMS[platform]}",
|
||||
"wheels/blendjob-0.1.6-py3-none-any.whl",
|
||||
names,
|
||||
)
|
||||
self.assertFalse(
|
||||
any(name.startswith("blendjob/") for name in names)
|
||||
)
|
||||
self.assertNotIn("install_env.py", names)
|
||||
self.assertNotIn("snapform/__init__.py", names)
|
||||
self.assertNotIn("importer.py", names)
|
||||
self.assertFalse(
|
||||
any(name.startswith("sources/") for name in names)
|
||||
)
|
||||
bundled_uv = {
|
||||
name
|
||||
for name in names
|
||||
if name.startswith("vendor/uv-")
|
||||
}
|
||||
self.assertEqual(
|
||||
bundled_uv,
|
||||
{f"vendor/{pack.PLATFORMS[platform]}"},
|
||||
self.assertFalse(
|
||||
any(name.startswith("vendor/") for name in names)
|
||||
)
|
||||
self.assertIn(f'platforms = ["{platform}"]', manifest)
|
||||
self.assertNotIn(
|
||||
@@ -108,95 +114,5 @@ class PackTest(unittest.TestCase):
|
||||
{"windows-x64", "macos-arm64", "linux-x64"},
|
||||
)
|
||||
|
||||
def test_bundled_depth_anything_3_has_no_xformers_dependency(self):
|
||||
pack = load_pack()
|
||||
with tarfile.open(
|
||||
fileobj=io.BytesIO(
|
||||
pack.bundled_artifact_bytes("depth-anything-3.tar.gz")
|
||||
),
|
||||
mode="r:gz",
|
||||
) as archive:
|
||||
raw_pyproject = archive.extractfile(
|
||||
"Depth-Anything-3-3d835ec1a5802d64a8b8b15f817a1ab54809bfe4/"
|
||||
"pyproject.toml"
|
||||
).read()
|
||||
pyproject = raw_pyproject.decode("utf-8")
|
||||
self.assertNotIn("xformers", pyproject)
|
||||
self.assertFalse(raw_pyproject.startswith(b"\xef\xbb\xbf"))
|
||||
|
||||
def test_bundled_depth_anything_3_exports_are_lazy(self):
|
||||
pack = load_pack()
|
||||
with tarfile.open(
|
||||
fileobj=io.BytesIO(
|
||||
pack.bundled_artifact_bytes("depth-anything-3.tar.gz")
|
||||
),
|
||||
mode="r:gz",
|
||||
) as archive:
|
||||
source = archive.extractfile(
|
||||
"Depth-Anything-3-3d835ec1a5802d64a8b8b15f817a1ab54809bfe4/"
|
||||
"src/depth_anything_3/utils/export/__init__.py"
|
||||
).read().decode("utf-8")
|
||||
tree = ast.parse(source)
|
||||
module_imports = [
|
||||
ast.get_source_segment(source, node)
|
||||
for node in tree.body
|
||||
if isinstance(node, (ast.Import, ast.ImportFrom))
|
||||
]
|
||||
self.assertEqual(
|
||||
module_imports,
|
||||
["from depth_anything_3.specs import Prediction"],
|
||||
)
|
||||
|
||||
def test_official_depth_anything_3_baseline_is_pristine(self):
|
||||
pack = load_pack()
|
||||
archive_path = pack.VENDOR_DIR / "depth-anything-3.tar.gz"
|
||||
with tarfile.open(archive_path, "r:gz") as archive:
|
||||
raw_pyproject = archive.extractfile(
|
||||
"Depth-Anything-3-3d835ec1a5802d64a8b8b15f817a1ab54809bfe4/"
|
||||
"pyproject.toml"
|
||||
).read()
|
||||
export_source = archive.extractfile(
|
||||
"Depth-Anything-3-3d835ec1a5802d64a8b8b15f817a1ab54809bfe4/"
|
||||
"src/depth_anything_3/utils/export/__init__.py"
|
||||
).read().decode("utf-8")
|
||||
pyproject = raw_pyproject.decode("utf-8")
|
||||
self.assertIn("xformers", pyproject)
|
||||
self.assertFalse(raw_pyproject.startswith(b"\xef\xbb\xbf"))
|
||||
self.assertIn(
|
||||
"from depth_anything_3.utils.export.gs import",
|
||||
export_source,
|
||||
)
|
||||
|
||||
def test_bundled_depth_anything_3_qr_falls_back_on_mps(self):
|
||||
pack = load_pack()
|
||||
with tarfile.open(
|
||||
fileobj=io.BytesIO(
|
||||
pack.bundled_artifact_bytes("depth-anything-3.tar.gz")
|
||||
),
|
||||
mode="r:gz",
|
||||
) as archive:
|
||||
source = archive.extractfile(
|
||||
"Depth-Anything-3-3d835ec1a5802d64a8b8b15f817a1ab54809bfe4/"
|
||||
"src/depth_anything_3/utils/ray_utils.py"
|
||||
).read().decode("utf-8")
|
||||
self.assertIn('use_cpu = device.type == "mps"', source)
|
||||
self.assertIn("A_tilde = A_tilde.cpu()", source)
|
||||
self.assertIn("Q_tilde = Q_tilde.to(device=device)", source)
|
||||
|
||||
def test_official_depth_anything_3_ray_utils_is_pristine(self):
|
||||
pack = load_pack()
|
||||
archive_path = pack.VENDOR_DIR / "depth-anything-3.tar.gz"
|
||||
with tarfile.open(archive_path, "r:gz") as archive:
|
||||
source = archive.extractfile(
|
||||
"Depth-Anything-3-3d835ec1a5802d64a8b8b15f817a1ab54809bfe4/"
|
||||
"src/depth_anything_3/utils/ray_utils.py"
|
||||
).read().decode("utf-8")
|
||||
self.assertIn(
|
||||
"Q_tilde, R_tilde = torch.linalg.qr(A_tilde)",
|
||||
source,
|
||||
)
|
||||
self.assertNotIn("use_cpu", source)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
|
||||
@@ -1,67 +0,0 @@
|
||||
import hashlib
|
||||
import importlib.util
|
||||
import io
|
||||
import tempfile
|
||||
import unittest
|
||||
from pathlib import Path
|
||||
from unittest.mock import patch
|
||||
|
||||
|
||||
PROJECT_ROOT = Path(__file__).resolve().parents[1]
|
||||
|
||||
|
||||
def load_prepare_vendor():
|
||||
specification = importlib.util.spec_from_file_location(
|
||||
"snapform_prepare_vendor",
|
||||
PROJECT_ROOT / "prepare_vendor.py",
|
||||
)
|
||||
module = importlib.util.module_from_spec(specification)
|
||||
specification.loader.exec_module(module)
|
||||
return module
|
||||
|
||||
|
||||
class PrepareVendorTest(unittest.TestCase):
|
||||
def test_downloads_and_verifies_artifact(self):
|
||||
prepare = load_prepare_vendor()
|
||||
content = b"locked artifact"
|
||||
artifact = {
|
||||
"filename": "artifact.tar.gz",
|
||||
"url": "https://example.invalid/artifact.tar.gz",
|
||||
"sha256": hashlib.sha256(content).hexdigest(),
|
||||
}
|
||||
with tempfile.TemporaryDirectory() as directory, patch.object(
|
||||
prepare.urllib.request,
|
||||
"urlopen",
|
||||
return_value=io.BytesIO(content),
|
||||
):
|
||||
path = prepare.download_artifact(artifact, Path(directory))
|
||||
self.assertEqual(path.read_bytes(), content)
|
||||
|
||||
def test_rejects_hash_mismatch_without_leaving_partial_file(self):
|
||||
prepare = load_prepare_vendor()
|
||||
artifact = {
|
||||
"filename": "artifact.tar.gz",
|
||||
"url": "https://example.invalid/artifact.tar.gz",
|
||||
"sha256": hashlib.sha256(b"expected").hexdigest(),
|
||||
}
|
||||
with tempfile.TemporaryDirectory() as directory, patch.object(
|
||||
prepare.urllib.request,
|
||||
"urlopen",
|
||||
return_value=io.BytesIO(b"unexpected"),
|
||||
):
|
||||
destination = Path(directory)
|
||||
with self.assertRaisesRegex(RuntimeError, "SHA-256 mismatch"):
|
||||
prepare.download_artifact(artifact, destination)
|
||||
self.assertFalse((destination / artifact["filename"]).exists())
|
||||
self.assertFalse(
|
||||
(destination / f"{artifact['filename']}.part").exists()
|
||||
)
|
||||
|
||||
def test_rejects_unknown_platform(self):
|
||||
prepare = load_prepare_vendor()
|
||||
with self.assertRaisesRegex(ValueError, "Unsupported platform"):
|
||||
prepare.prepare(["windows-arm64"])
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
371
tests/test_server.py
Normal file
371
tests/test_server.py
Normal file
@@ -0,0 +1,371 @@
|
||||
import importlib
|
||||
import sys
|
||||
import tempfile
|
||||
import threading
|
||||
import time
|
||||
import unittest
|
||||
from pathlib import Path
|
||||
from types import ModuleType, SimpleNamespace
|
||||
from unittest.mock import patch
|
||||
|
||||
|
||||
PROJECT_ROOT = Path(__file__).resolve().parents[1]
|
||||
|
||||
|
||||
def load_server_module():
|
||||
addon_directory = PROJECT_ROOT / "src" / "snapform"
|
||||
sys.path.insert(0, str(addon_directory))
|
||||
try:
|
||||
return importlib.import_module("server")
|
||||
finally:
|
||||
sys.path.remove(str(addon_directory))
|
||||
|
||||
|
||||
class FakeCuda:
|
||||
class OutOfMemoryError(RuntimeError):
|
||||
pass
|
||||
|
||||
@staticmethod
|
||||
def is_available():
|
||||
return False
|
||||
|
||||
@staticmethod
|
||||
def empty_cache():
|
||||
return None
|
||||
|
||||
|
||||
class ServerTest(unittest.TestCase):
|
||||
@classmethod
|
||||
def setUpClass(cls):
|
||||
cls.server_module = load_server_module()
|
||||
|
||||
def setUp(self):
|
||||
self.jobs_temp = tempfile.TemporaryDirectory()
|
||||
self.addCleanup(self.jobs_temp.cleanup)
|
||||
self.storage_root = Path(self.jobs_temp.name)
|
||||
|
||||
def test_required_model_download_skips_ready_model(self):
|
||||
manager = self.server_module.ModelManager(Path("."))
|
||||
downloads = []
|
||||
context = SimpleNamespace(progress=lambda *_args: None)
|
||||
with patch.object(
|
||||
manager,
|
||||
"ready",
|
||||
side_effect=lambda key: key == "BEN2_BASE",
|
||||
), patch.object(
|
||||
manager,
|
||||
"_download",
|
||||
side_effect=lambda _context, key, _progress: downloads.append(key),
|
||||
):
|
||||
result = manager.download_missing(
|
||||
context,
|
||||
("MOGE2_VITS_NORMAL", "BEN2_BASE"),
|
||||
)
|
||||
|
||||
self.assertEqual(downloads, ["MOGE2_VITS_NORMAL"])
|
||||
self.assertEqual(result, {"downloaded": ["MOGE2_VITS_NORMAL"]})
|
||||
|
||||
def test_models_are_owned_by_the_server_resource(self):
|
||||
self.assertFalse((PROJECT_ROOT / "src" / "snapform" / "models.py").exists())
|
||||
with tempfile.TemporaryDirectory() as directory:
|
||||
manager = self.server_module.ModelManager(Path(directory))
|
||||
snapshot = manager.snapshot()
|
||||
self.assertIn("catalog", snapshot)
|
||||
self.assertTrue(snapshot["catalog"])
|
||||
|
||||
def create_cache(self):
|
||||
cache = self.server_module.ModelSessionCache.__new__(
|
||||
self.server_module.ModelSessionCache
|
||||
)
|
||||
cache.ben2_model = None
|
||||
cache.ben2_key = None
|
||||
cache.da3_model = None
|
||||
cache.da3_key = None
|
||||
cache.moge_model = None
|
||||
cache.moge_key = None
|
||||
return cache
|
||||
|
||||
def create_server(self):
|
||||
source = self.server_module.server
|
||||
server = self.server_module.JobServer(
|
||||
"SnapForm Job Server", storage_root=self.storage_root
|
||||
)
|
||||
server.handlers.update(source.handlers)
|
||||
server.resource_factories.update(source.resource_factories)
|
||||
server.bind(self.storage_root)
|
||||
return server
|
||||
|
||||
def wait_for_job(self, context, timeout=2.0):
|
||||
deadline = time.monotonic() + timeout
|
||||
while time.monotonic() < deadline:
|
||||
if context.snapshot()["state"] in {"succeeded", "failed", "cancelled"}:
|
||||
return context.snapshot()
|
||||
time.sleep(0.01)
|
||||
self.fail(f"Job did not finish: {context.snapshot()}")
|
||||
|
||||
def test_ben2_model_is_reused_for_matching_runtime_key(self):
|
||||
loads = []
|
||||
|
||||
ben2 = ModuleType("server.onnx_ben2")
|
||||
ben2.create_session = lambda model_dir, device: (
|
||||
loads.append((model_dir, device)) or object()
|
||||
)
|
||||
cache = self.create_cache()
|
||||
device = SimpleNamespace(type="cuda")
|
||||
with patch.dict(sys.modules, {"server.onnx_ben2": ben2}):
|
||||
first, _load_ms = cache.get_ben2("model", device)
|
||||
second, cached_load_ms = cache.get_ben2("model", device)
|
||||
|
||||
self.assertIs(first, second)
|
||||
self.assertEqual(loads, [("model", "cuda")])
|
||||
self.assertEqual(cache.snapshot()["ben2"], "model")
|
||||
self.assertEqual(cached_load_ms, 0.0)
|
||||
|
||||
def test_da3_model_is_reused_for_matching_runtime_key(self):
|
||||
loads = []
|
||||
|
||||
da3 = ModuleType("server.onnx_da3")
|
||||
da3.create_session = lambda model_dir, device: (
|
||||
loads.append((model_dir, device)) or object()
|
||||
)
|
||||
cache = self.create_cache()
|
||||
device = SimpleNamespace(type="cuda")
|
||||
with patch.dict(
|
||||
sys.modules,
|
||||
{
|
||||
"server.onnx_da3": da3,
|
||||
},
|
||||
):
|
||||
first = cache.get_da3("model", device)
|
||||
second = cache.get_da3("model", device)
|
||||
|
||||
self.assertIs(first, second)
|
||||
self.assertEqual(loads, [("model", "cuda")])
|
||||
self.assertEqual(cache.snapshot()["da3"], "model")
|
||||
|
||||
def test_depth_model_cache_keeps_only_da3_or_moge(self):
|
||||
cache = self.create_cache()
|
||||
cache.ben2_model = object()
|
||||
cache.ben2_key = ("ben2", "cuda")
|
||||
cache.moge_model = object()
|
||||
cache.moge_key = ("moge", "cuda")
|
||||
device = SimpleNamespace(type="cuda")
|
||||
da3 = object()
|
||||
|
||||
def load_da3(_model_dir, _device, key):
|
||||
cache.da3_model = da3
|
||||
cache.da3_key = key
|
||||
return da3
|
||||
|
||||
with patch.object(cache, "_load_da3", side_effect=load_da3):
|
||||
self.assertIs(cache.get_da3("da3", device), da3)
|
||||
|
||||
self.assertIsNone(cache.moge_model)
|
||||
self.assertIsNotNone(cache.ben2_model)
|
||||
|
||||
moge = object()
|
||||
|
||||
fake_moge = ModuleType("server.onnx_moge2")
|
||||
fake_moge.create_session = lambda _model_dir, _device: moge
|
||||
with patch.dict(sys.modules, {"server.onnx_moge2": fake_moge}):
|
||||
self.assertIs(cache.get_moge2("moge", "cuda"), moge)
|
||||
|
||||
self.assertIsNone(cache.da3_model)
|
||||
self.assertIsNotNone(cache.ben2_model)
|
||||
|
||||
def test_dedicated_server_runs_only_one_job_at_a_time(self):
|
||||
server = self.server_module.JobServer("Test", storage_root=self.storage_root)
|
||||
first_started = threading.Event()
|
||||
release_first = threading.Event()
|
||||
order = []
|
||||
|
||||
@server.job("example")
|
||||
def example(_context, parameters):
|
||||
order.append(parameters["name"])
|
||||
if parameters["name"] == "first":
|
||||
first_started.set()
|
||||
release_first.wait(1.0)
|
||||
|
||||
first = server.submit("example", {"name": "first"}, job_id="job")
|
||||
self.assertTrue(first_started.wait(1.0))
|
||||
second = server.submit("example", {"name": "second"}, job_id="other")
|
||||
|
||||
self.assertIsNotNone(first)
|
||||
self.assertIsNotNone(second)
|
||||
self.assertTrue(server.snapshot("instance")["busy"])
|
||||
self.assertEqual(server.snapshot("instance")["queued_jobs"], 1)
|
||||
self.assertNotIn("clients", server.snapshot("instance"))
|
||||
release_first.set()
|
||||
self.wait_for_job(first)
|
||||
self.wait_for_job(second)
|
||||
self.assertEqual(order, ["first", "second"])
|
||||
server.close()
|
||||
|
||||
def test_server_registers_tasks_instead_of_model_names(self):
|
||||
server = self.create_server()
|
||||
self.assertEqual(
|
||||
set(server.handlers),
|
||||
{
|
||||
"download-model",
|
||||
"download-required-models",
|
||||
"remove-background",
|
||||
"extract-frames",
|
||||
"generate-da3-depth",
|
||||
"generate-moge2-depth",
|
||||
"generate-da3-surface",
|
||||
"generate-moge2-surface",
|
||||
"debug-da3",
|
||||
"debug-moge2",
|
||||
"debug-ben2",
|
||||
},
|
||||
)
|
||||
|
||||
def test_queued_job_can_be_cancelled_before_it_runs(self):
|
||||
server = self.server_module.JobServer("Test", storage_root=self.storage_root)
|
||||
first_started = threading.Event()
|
||||
release_first = threading.Event()
|
||||
executed = []
|
||||
|
||||
@server.job("example")
|
||||
def example(_context, parameters):
|
||||
executed.append(parameters["name"])
|
||||
if parameters["name"] == "first":
|
||||
first_started.set()
|
||||
release_first.wait(1.0)
|
||||
|
||||
first = server.submit("example", {"name": "first"})
|
||||
self.assertTrue(first_started.wait(1.0))
|
||||
second = server.submit("example", {"name": "second"})
|
||||
|
||||
self.assertIs(server.cancel(second.job_id), second)
|
||||
release_first.set()
|
||||
self.wait_for_job(first)
|
||||
second_status = self.wait_for_job(second)
|
||||
|
||||
self.assertEqual(second_status["state"], "cancelled")
|
||||
self.assertEqual(executed, ["first"])
|
||||
server.close()
|
||||
|
||||
def test_cancelling_job_keeps_server_and_model_cache_alive(self):
|
||||
fake_models = SimpleNamespace(
|
||||
snapshot=lambda: {
|
||||
"loaded": {"ben2": "BEN2_BASE", "da3": ""}
|
||||
},
|
||||
clear=lambda: None,
|
||||
close=lambda: None,
|
||||
)
|
||||
with patch.object(
|
||||
self.server_module, "ModelManager", return_value=fake_models
|
||||
):
|
||||
server = self.create_server()
|
||||
context = server.submit("remove-background", {}, job_id="job")
|
||||
self.assertIsNotNone(server.cancel("job"))
|
||||
self.assertTrue(context.is_cancelled())
|
||||
status = self.wait_for_job(context)
|
||||
|
||||
self.assertEqual(status["state"], "cancelled")
|
||||
self.assertFalse(server.shutdown_event.is_set())
|
||||
self.assertEqual(
|
||||
server.snapshot("instance")["resources"]["model_manager"]["loaded"]["ben2"],
|
||||
"BEN2_BASE",
|
||||
)
|
||||
|
||||
def test_job_progress_is_kept_in_server_memory(self):
|
||||
server = self.server_module.JobServer(
|
||||
"Test", storage_root=self.storage_root
|
||||
)
|
||||
|
||||
@server.job("example")
|
||||
def example(context, _parameters):
|
||||
context.progress(0.5, "Half way")
|
||||
|
||||
context = server.submit("example", {})
|
||||
self.wait_for_job(context)
|
||||
|
||||
self.assertRegex(context.job_id, r"^[0-9a-f]{32}$")
|
||||
self.assertEqual(context.directory.name, context.job_id)
|
||||
self.assertEqual(context.snapshot()["state"], "succeeded")
|
||||
self.assertEqual(context.snapshot()["progress"], 1.0)
|
||||
|
||||
def test_runtime_storage_root_owns_the_jobs_directory(self):
|
||||
storage_root = self.storage_root
|
||||
server = self.server_module.JobServer("Test")
|
||||
server.bind(storage_root)
|
||||
|
||||
@server.job("example")
|
||||
def example(_context, _parameters):
|
||||
return {"value": 42}
|
||||
|
||||
context = server.submit("example", {})
|
||||
self.wait_for_job(context)
|
||||
|
||||
self.assertEqual(server.jobs_directory, storage_root / "jobs")
|
||||
self.assertEqual(context.storage_root, storage_root.resolve())
|
||||
self.assertEqual(context.snapshot()["result"], {"value": 42})
|
||||
|
||||
def test_job_server_owns_and_clears_shared_resources(self):
|
||||
events = []
|
||||
resource = SimpleNamespace(
|
||||
snapshot=lambda: {"loaded": "model"},
|
||||
clear=lambda: events.append("clear"),
|
||||
close=lambda: events.append("close"),
|
||||
)
|
||||
server = self.server_module.JobServer(
|
||||
"Test", storage_root=self.storage_root
|
||||
)
|
||||
server.add_resource("model_manager", resource)
|
||||
|
||||
@server.job("example")
|
||||
def example(context, _parameters):
|
||||
self.assertIs(context.resource("model_manager"), resource)
|
||||
release.wait(1.0)
|
||||
|
||||
release = threading.Event()
|
||||
context = server.submit("example", {}, job_id="job")
|
||||
self.assertFalse(server.clear_resource("model_manager"))
|
||||
release.set()
|
||||
self.wait_for_job(context)
|
||||
self.assertTrue(server.clear_resource("model_manager"))
|
||||
server.close()
|
||||
|
||||
self.assertEqual(
|
||||
server.snapshot("instance")["resources"]["model_manager"]["loaded"],
|
||||
"model",
|
||||
)
|
||||
self.assertEqual(events, ["clear", "close"])
|
||||
|
||||
def test_server_dispatches_frame_extraction_without_models(self):
|
||||
fake_models = SimpleNamespace(
|
||||
snapshot=lambda: {"loaded": {"ben2": "", "da3": ""}},
|
||||
clear=lambda: None,
|
||||
close=lambda: None,
|
||||
)
|
||||
with patch.object(
|
||||
self.server_module, "ModelManager", return_value=fake_models
|
||||
):
|
||||
server = self.create_server()
|
||||
with tempfile.TemporaryDirectory() as directory:
|
||||
root = Path(directory)
|
||||
input_directory = root / "frames"
|
||||
input_directory.mkdir()
|
||||
(input_directory / "frame-0001.jpg").touch()
|
||||
(input_directory / "frame-0002.jpg").touch()
|
||||
context = server.submit(
|
||||
"extract-frames",
|
||||
{
|
||||
"input": str(input_directory),
|
||||
"video_frames": 16,
|
||||
"start_frame": 1,
|
||||
},
|
||||
job_id="job",
|
||||
)
|
||||
status = self.wait_for_job(context)
|
||||
self.assertTrue(
|
||||
(context.directory / "frame.0001.jpg").is_file()
|
||||
)
|
||||
|
||||
self.assertEqual(status["state"], "succeeded")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
@@ -1,255 +0,0 @@
|
||||
import importlib.util
|
||||
import sys
|
||||
import tempfile
|
||||
import unittest
|
||||
from pathlib import Path
|
||||
from types import ModuleType, SimpleNamespace
|
||||
from unittest.mock import patch
|
||||
|
||||
|
||||
PROJECT_ROOT = Path(__file__).resolve().parents[1]
|
||||
|
||||
|
||||
def load_service():
|
||||
specification = importlib.util.spec_from_file_location(
|
||||
"snapform_service",
|
||||
PROJECT_ROOT / "src" / "snapform" / "service.py",
|
||||
)
|
||||
module = importlib.util.module_from_spec(specification)
|
||||
specification.loader.exec_module(module)
|
||||
return module
|
||||
|
||||
|
||||
class FakeCuda:
|
||||
class OutOfMemoryError(RuntimeError):
|
||||
pass
|
||||
|
||||
@staticmethod
|
||||
def is_available():
|
||||
return False
|
||||
|
||||
@staticmethod
|
||||
def empty_cache():
|
||||
return None
|
||||
|
||||
|
||||
class ServiceTest(unittest.TestCase):
|
||||
@classmethod
|
||||
def setUpClass(cls):
|
||||
cls.service = load_service()
|
||||
|
||||
def create_cache(self):
|
||||
cache = self.service.ModelCache.__new__(self.service.ModelCache)
|
||||
cache.torch = SimpleNamespace(cuda=FakeCuda())
|
||||
cache.ben2_model = None
|
||||
cache.ben2_key = None
|
||||
cache.da3_model = None
|
||||
cache.da3_key = None
|
||||
return cache
|
||||
|
||||
def test_ben2_model_is_reused_for_matching_runtime_key(self):
|
||||
loads = []
|
||||
|
||||
class FakeModel:
|
||||
@classmethod
|
||||
def from_pretrained(cls, model_dir):
|
||||
loads.append(model_dir)
|
||||
return cls()
|
||||
|
||||
def to(self, device):
|
||||
self.device = device
|
||||
return self
|
||||
|
||||
def eval(self):
|
||||
return None
|
||||
|
||||
ben2 = ModuleType("ben2")
|
||||
ben2.BEN_Base = FakeModel
|
||||
cache = self.create_cache()
|
||||
device = SimpleNamespace(type="cuda")
|
||||
with patch.dict(sys.modules, {"ben2": ben2}):
|
||||
first, _load_ms = cache.get_ben2("model", device)
|
||||
second, cached_load_ms = cache.get_ben2("model", device)
|
||||
|
||||
self.assertIs(first, second)
|
||||
self.assertEqual(loads, ["model"])
|
||||
self.assertEqual(cache.status()["ben2"], "model")
|
||||
self.assertEqual(cached_load_ms, 0.0)
|
||||
|
||||
def test_da3_model_is_reused_for_matching_runtime_key(self):
|
||||
loads = []
|
||||
|
||||
class FakeModel:
|
||||
@classmethod
|
||||
def from_pretrained(cls, model_dir):
|
||||
loads.append(model_dir)
|
||||
return cls()
|
||||
|
||||
def to(self, device):
|
||||
self.device = device
|
||||
return self
|
||||
|
||||
def eval(self):
|
||||
return None
|
||||
|
||||
package = ModuleType("depth_anything_3")
|
||||
api = ModuleType("depth_anything_3.api")
|
||||
api.DepthAnything3 = FakeModel
|
||||
cache = self.create_cache()
|
||||
device = SimpleNamespace(type="cuda")
|
||||
with patch.dict(
|
||||
sys.modules,
|
||||
{
|
||||
"depth_anything_3": package,
|
||||
"depth_anything_3.api": api,
|
||||
},
|
||||
):
|
||||
first = cache.get_da3("model", device)
|
||||
second = cache.get_da3("model", device)
|
||||
|
||||
self.assertIs(first, second)
|
||||
self.assertEqual(loads, ["model"])
|
||||
self.assertEqual(cache.status()["da3"], "model")
|
||||
|
||||
def test_service_tracks_clients_and_rejects_shared_shutdown(self):
|
||||
fake_models = SimpleNamespace(
|
||||
status=lambda: {"ben2": "", "da3": ""},
|
||||
close=lambda: None,
|
||||
)
|
||||
with patch.object(self.service, "ModelCache", return_value=fake_models):
|
||||
state = self.service.ServiceState("token", "instance")
|
||||
|
||||
state.register_client("first", "Blender A")
|
||||
state.register_client("second", "Blender B")
|
||||
|
||||
self.assertEqual(state.snapshot()["clients"], 2)
|
||||
self.assertEqual(state.request_shutdown("first"), "shared")
|
||||
self.assertTrue(
|
||||
state.reserve("job", "first", "reconstruct", "status.json")
|
||||
)
|
||||
self.assertFalse(
|
||||
state.reserve("other", "second", "reconstruct", "other.json")
|
||||
)
|
||||
|
||||
def test_service_accepts_moge_command(self):
|
||||
self.assertIn("moge", self.service.JOB_COMMANDS)
|
||||
|
||||
def test_cancelling_job_keeps_service_and_model_cache_alive(self):
|
||||
fake_models = SimpleNamespace(
|
||||
status=lambda: {"ben2": "BEN2_BASE", "da3": ""},
|
||||
close=lambda: None,
|
||||
)
|
||||
with patch.object(self.service, "ModelCache", return_value=fake_models):
|
||||
state = self.service.ServiceState("token", "instance")
|
||||
state.register_client("client", "Blender")
|
||||
with tempfile.TemporaryDirectory() as directory:
|
||||
status_path = Path(directory) / "status.json"
|
||||
state.reserve(
|
||||
"job",
|
||||
"client",
|
||||
"remove-background",
|
||||
str(status_path),
|
||||
)
|
||||
|
||||
active = state.cancel_job("job", "client")
|
||||
self.assertTrue(active["cancel_requested"])
|
||||
self.assertTrue(state.is_job_cancelled("job"))
|
||||
state.run_job(
|
||||
{
|
||||
"job_id": "job",
|
||||
"command": "remove-background",
|
||||
"parameters": {"status": str(status_path)},
|
||||
}
|
||||
)
|
||||
status = self.service.json.loads(
|
||||
status_path.read_text(encoding="utf-8")
|
||||
)
|
||||
|
||||
self.assertEqual(status["state"], "cancelled")
|
||||
self.assertFalse(state.shutdown_event.is_set())
|
||||
self.assertEqual(state.snapshot()["cached_models"]["ben2"], "BEN2_BASE")
|
||||
|
||||
def test_expired_clients_are_removed(self):
|
||||
fake_models = SimpleNamespace(
|
||||
status=lambda: {"ben2": "", "da3": ""},
|
||||
close=lambda: None,
|
||||
)
|
||||
with patch.object(self.service, "ModelCache", return_value=fake_models):
|
||||
state = self.service.ServiceState("token", "instance")
|
||||
state.register_client("client", "Blender")
|
||||
state.clients["client"]["last_seen"] -= (
|
||||
self.service.CLIENT_TIMEOUT_SECONDS + 1.0
|
||||
)
|
||||
|
||||
state.expire_clients()
|
||||
|
||||
self.assertEqual(state.snapshot()["clients"], 0)
|
||||
|
||||
def test_discovery_is_only_removed_by_matching_service(self):
|
||||
with tempfile.TemporaryDirectory() as directory:
|
||||
path = Path(directory) / "service.json"
|
||||
args = SimpleNamespace(
|
||||
discovery=str(path),
|
||||
port=51234,
|
||||
token="token",
|
||||
instance_id="owner",
|
||||
)
|
||||
with patch.object(self.service.os, "getpid", return_value=42):
|
||||
self.service.write_discovery(args)
|
||||
|
||||
payload = self.service.json.loads(path.read_text(encoding="utf-8"))
|
||||
self.assertEqual(payload["port"], 51234)
|
||||
self.assertEqual(payload["pid"], 42)
|
||||
self.service.remove_discovery(path, "other")
|
||||
self.assertTrue(path.is_file())
|
||||
self.service.remove_discovery(path, "owner")
|
||||
self.assertFalse(path.exists())
|
||||
|
||||
def test_service_dispatches_frame_extraction_without_models(self):
|
||||
fake_models = SimpleNamespace(
|
||||
status=lambda: {"ben2": "", "da3": ""},
|
||||
close=lambda: None,
|
||||
)
|
||||
with patch.object(self.service, "ModelCache", return_value=fake_models):
|
||||
state = self.service.ServiceState("token", "instance")
|
||||
state.register_client("client", "Blender")
|
||||
with tempfile.TemporaryDirectory() as directory:
|
||||
root = Path(directory)
|
||||
input_directory = root / "frames"
|
||||
input_directory.mkdir()
|
||||
(input_directory / "frame-0001.jpg").touch()
|
||||
(input_directory / "frame-0002.jpg").touch()
|
||||
result_directory = root / "result"
|
||||
status_path = root / "status.json"
|
||||
state.reserve(
|
||||
"job",
|
||||
"client",
|
||||
"extract-frames",
|
||||
str(status_path),
|
||||
)
|
||||
|
||||
state.run_job(
|
||||
{
|
||||
"job_id": "job",
|
||||
"command": "extract-frames",
|
||||
"parameters": {
|
||||
"input": str(input_directory),
|
||||
"output_dir": str(result_directory),
|
||||
"status": str(status_path),
|
||||
"video_frames": 16,
|
||||
"start_frame": 1,
|
||||
},
|
||||
}
|
||||
)
|
||||
status = self.service.json.loads(
|
||||
status_path.read_text(encoding="utf-8")
|
||||
)
|
||||
self.assertTrue(
|
||||
(result_directory / "frame.0001.jpg").is_file()
|
||||
)
|
||||
|
||||
self.assertEqual(status["state"], "succeeded")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
File diff suppressed because it is too large
Load Diff
@@ -4,6 +4,7 @@ import bpy
|
||||
|
||||
|
||||
GROUP_NAME = "O DA3 Surface"
|
||||
MOGE_SURFACE_GROUP_NAME = "O MoGe2 Surface"
|
||||
DEPTH_PLANE_GROUP_NAME = "O DA3 Depth Plane"
|
||||
MOGE_DEPTH_PLANE_GROUP_NAME = "O MoGe2 Depth Plane"
|
||||
MATERIAL_DISPLACEMENT_ATTRIBUTE = "o_material_displacement"
|
||||
@@ -24,12 +25,15 @@ def input_socket(node, name, fallback):
|
||||
|
||||
|
||||
def new_socket(interface, name, in_out, socket_type, parent=None):
|
||||
return interface.new_socket(
|
||||
socket = interface.new_socket(
|
||||
name=name,
|
||||
in_out=in_out,
|
||||
socket_type=socket_type,
|
||||
parent=parent,
|
||||
)
|
||||
if in_out == "INPUT" and hasattr(socket, "structure_type"):
|
||||
socket.structure_type = "SINGLE"
|
||||
return socket
|
||||
|
||||
|
||||
def new_compare(nodes, data_type, operation, location, value):
|
||||
@@ -357,8 +361,16 @@ def add_uv_reprojection(group, source, geometry, location):
|
||||
return store_uv.outputs["Geometry"]
|
||||
|
||||
|
||||
def build_group():
|
||||
group = bpy.data.node_groups.new(GROUP_NAME, "GeometryNodeTree")
|
||||
def build_group(
|
||||
group_name=GROUP_NAME,
|
||||
include_confidence=True,
|
||||
include_discontinuity=True,
|
||||
include_mask=False,
|
||||
discontinuity_name="Depth Continuity",
|
||||
discontinuity_attribute="o_depth_discontinuity",
|
||||
discontinuity_default=50.0,
|
||||
):
|
||||
group = bpy.data.node_groups.new(group_name, "GeometryNodeTree")
|
||||
group.is_modifier = True
|
||||
interface = group.interface
|
||||
geometry_input_socket = new_socket(
|
||||
@@ -375,7 +387,7 @@ def build_group():
|
||||
"INPUT",
|
||||
"NodeSocketFloat",
|
||||
)
|
||||
resolution.default_value = 0.01
|
||||
resolution.default_value = 0.1
|
||||
resolution.min_value = 0.000001
|
||||
resolution.subtype = "DISTANCE"
|
||||
remesh_iterations = new_socket(
|
||||
@@ -393,37 +405,49 @@ def build_group():
|
||||
"INPUT",
|
||||
"NodeSocketFloat",
|
||||
)
|
||||
smooth_amount.default_value = 1.0
|
||||
smooth_amount.default_value = 0.5
|
||||
smooth_amount.min_value = 0.0
|
||||
smooth_amount.max_value = 1.0
|
||||
smooth_amount.subtype = "FACTOR"
|
||||
|
||||
culling_panel = interface.new_panel(
|
||||
name="Culling",
|
||||
default_closed=False,
|
||||
)
|
||||
confidence = new_socket(
|
||||
interface,
|
||||
"Confidence",
|
||||
"INPUT",
|
||||
"NodeSocketFloat",
|
||||
parent=culling_panel,
|
||||
)
|
||||
confidence.default_value = 0.0
|
||||
confidence.min_value = 0.0
|
||||
confidence.max_value = 100.0
|
||||
confidence.subtype = "PERCENTAGE"
|
||||
edge = new_socket(
|
||||
interface,
|
||||
"Stretch",
|
||||
"INPUT",
|
||||
"NodeSocketFloat",
|
||||
parent=culling_panel,
|
||||
)
|
||||
edge.default_value = 10.0
|
||||
edge.min_value = 0.0
|
||||
edge.max_value = 100.0
|
||||
edge.subtype = "PERCENTAGE"
|
||||
culling_panel = None
|
||||
if include_confidence or include_discontinuity or include_mask:
|
||||
culling_panel = interface.new_panel(
|
||||
name="Culling",
|
||||
default_closed=False,
|
||||
)
|
||||
if include_discontinuity:
|
||||
discontinuity = new_socket(
|
||||
interface,
|
||||
discontinuity_name,
|
||||
"INPUT",
|
||||
"NodeSocketFloat",
|
||||
parent=culling_panel,
|
||||
)
|
||||
discontinuity.default_value = discontinuity_default
|
||||
discontinuity.min_value = 0.0
|
||||
discontinuity.max_value = 100.0
|
||||
discontinuity.subtype = "PERCENTAGE"
|
||||
if include_confidence:
|
||||
confidence = new_socket(
|
||||
interface,
|
||||
"Confidence",
|
||||
"INPUT",
|
||||
"NodeSocketFloat",
|
||||
parent=culling_panel,
|
||||
)
|
||||
confidence.default_value = 0.0
|
||||
confidence.min_value = 0.0
|
||||
confidence.max_value = 100.0
|
||||
confidence.subtype = "PERCENTAGE"
|
||||
if include_mask:
|
||||
mask_socket = new_socket(
|
||||
interface,
|
||||
"Mask",
|
||||
"INPUT",
|
||||
"NodeSocketBool",
|
||||
parent=culling_panel,
|
||||
)
|
||||
mask_socket.default_value = False
|
||||
|
||||
options_panel = interface.new_panel(
|
||||
name="Options",
|
||||
@@ -444,8 +468,20 @@ def build_group():
|
||||
nodes = group.nodes
|
||||
links = group.links
|
||||
geometry_input = new_group_input(nodes, (-650.0, 50.0), {"Geometry"})
|
||||
confidence_input = new_group_input(nodes, (-1800.0, 800.0), {"Confidence"})
|
||||
stretch_input = new_group_input(nodes, (-1800.0, 50.0), {"Stretch"})
|
||||
confidence_input = None
|
||||
if include_confidence:
|
||||
confidence_input = new_group_input(
|
||||
nodes,
|
||||
(-1800.0, 800.0),
|
||||
{"Confidence"},
|
||||
)
|
||||
discontinuity_input = None
|
||||
if include_discontinuity:
|
||||
discontinuity_input = new_group_input(
|
||||
nodes,
|
||||
(-1800.0, 50.0),
|
||||
{discontinuity_name},
|
||||
)
|
||||
remesh_input = new_group_input(
|
||||
nodes,
|
||||
(-100.0, -500.0),
|
||||
@@ -457,112 +493,107 @@ def build_group():
|
||||
group_output = nodes.new("NodeGroupOutput")
|
||||
group_output.location = (5650.0, 100.0)
|
||||
|
||||
confidence_attribute = nodes.new("GeometryNodeInputNamedAttribute")
|
||||
confidence_attribute.data_type = "FLOAT"
|
||||
confidence_attribute.inputs["Name"].default_value = "o_face_confidence"
|
||||
confidence_attribute.location = (-1800.0, 600.0)
|
||||
confidence_scale = nodes.new("ShaderNodeMath")
|
||||
confidence_scale.operation = "MULTIPLY"
|
||||
confidence_scale.inputs[1].default_value = 0.01
|
||||
confidence_scale.location = (-1600.0, 800.0)
|
||||
confidence_compare = new_compare(
|
||||
nodes,
|
||||
"FLOAT",
|
||||
"LESS_THAN",
|
||||
(-1550.0, 600.0),
|
||||
0.0,
|
||||
)
|
||||
links.new(
|
||||
confidence_attribute.outputs["Attribute"],
|
||||
input_socket(confidence_compare, "A", 0),
|
||||
)
|
||||
links.new(
|
||||
confidence_input.outputs["Confidence"],
|
||||
confidence_scale.inputs[0],
|
||||
)
|
||||
links.new(
|
||||
confidence_scale.outputs[0],
|
||||
input_socket(confidence_compare, "B", 1),
|
||||
)
|
||||
geometry = geometry_input.outputs["Geometry"]
|
||||
if include_confidence:
|
||||
confidence_attribute = nodes.new("GeometryNodeInputNamedAttribute")
|
||||
confidence_attribute.data_type = "FLOAT"
|
||||
confidence_attribute.inputs["Name"].default_value = "o_face_confidence"
|
||||
confidence_attribute.location = (-1800.0, 600.0)
|
||||
confidence_scale = nodes.new("ShaderNodeMath")
|
||||
confidence_scale.operation = "MULTIPLY"
|
||||
confidence_scale.inputs[1].default_value = 0.01
|
||||
confidence_scale.location = (-1600.0, 800.0)
|
||||
confidence_compare = new_compare(
|
||||
nodes,
|
||||
"FLOAT",
|
||||
"LESS_THAN",
|
||||
(-1550.0, 600.0),
|
||||
0.0,
|
||||
)
|
||||
links.new(
|
||||
confidence_attribute.outputs["Attribute"],
|
||||
input_socket(confidence_compare, "A", 0),
|
||||
)
|
||||
links.new(
|
||||
confidence_input.outputs["Confidence"],
|
||||
confidence_scale.inputs[0],
|
||||
)
|
||||
links.new(
|
||||
confidence_scale.outputs[0],
|
||||
input_socket(confidence_compare, "B", 1),
|
||||
)
|
||||
delete_confidence = nodes.new("GeometryNodeDeleteGeometry")
|
||||
delete_confidence.domain = "FACE"
|
||||
delete_confidence.location = (-1050.0, 600.0)
|
||||
links.new(geometry, delete_confidence.inputs["Geometry"])
|
||||
links.new(
|
||||
confidence_compare.outputs["Result"],
|
||||
delete_confidence.inputs["Selection"],
|
||||
)
|
||||
geometry = delete_confidence.outputs["Geometry"]
|
||||
|
||||
edge_attribute = nodes.new("GeometryNodeInputNamedAttribute")
|
||||
edge_attribute.data_type = "FLOAT"
|
||||
edge_attribute.inputs["Name"].default_value = "o_edge_stretch"
|
||||
edge_attribute.location = (-1800.0, 350.0)
|
||||
edge_scale = nodes.new("ShaderNodeMath")
|
||||
edge_scale.operation = "MULTIPLY"
|
||||
edge_scale.inputs[1].default_value = 0.01
|
||||
edge_scale.location = (-1600.0, -50.0)
|
||||
edge_maximum = nodes.new("ShaderNodeMath")
|
||||
edge_maximum.operation = "MAXIMUM"
|
||||
edge_maximum.inputs[1].default_value = 0.0001
|
||||
edge_maximum.location = (-1600.0, 100.0)
|
||||
edge_reciprocal = nodes.new("ShaderNodeMath")
|
||||
edge_reciprocal.operation = "DIVIDE"
|
||||
edge_reciprocal.inputs[0].default_value = 1.0
|
||||
edge_reciprocal.location = (-1400.0, 100.0)
|
||||
edge_compare = new_compare(
|
||||
nodes,
|
||||
"FLOAT",
|
||||
"GREATER_EQUAL",
|
||||
(-1200.0, 350.0),
|
||||
1.0,
|
||||
)
|
||||
links.new(stretch_input.outputs["Stretch"], edge_scale.inputs[0])
|
||||
links.new(edge_scale.outputs[0], edge_maximum.inputs[0])
|
||||
links.new(edge_maximum.outputs[0], edge_reciprocal.inputs[1])
|
||||
links.new(edge_attribute.outputs["Attribute"], input_socket(edge_compare, "A", 0))
|
||||
links.new(edge_reciprocal.outputs[0], input_socket(edge_compare, "B", 1))
|
||||
|
||||
first_or = nodes.new("FunctionNodeBooleanMath")
|
||||
first_or.operation = "OR"
|
||||
first_or.location = (-950.0, 450.0)
|
||||
links.new(confidence_compare.outputs["Result"], first_or.inputs[0])
|
||||
links.new(edge_compare.outputs["Result"], first_or.inputs[1])
|
||||
full_stretch = new_compare(
|
||||
nodes,
|
||||
"FLOAT",
|
||||
"GREATER_EQUAL",
|
||||
(-950.0, -20.0),
|
||||
0.999999,
|
||||
)
|
||||
links.new(
|
||||
edge_scale.outputs[0],
|
||||
input_socket(full_stretch, "A", 0),
|
||||
)
|
||||
culling_selection = nodes.new("FunctionNodeBooleanMath")
|
||||
culling_selection.operation = "OR"
|
||||
culling_selection.location = (-550.0, 300.0)
|
||||
links.new(first_or.outputs[0], culling_selection.inputs[0])
|
||||
links.new(full_stretch.outputs["Result"], culling_selection.inputs[1])
|
||||
delete_geometry = nodes.new("GeometryNodeDeleteGeometry")
|
||||
delete_geometry.domain = "FACE"
|
||||
delete_geometry.location = (-350.0, 100.0)
|
||||
links.new(geometry_input.outputs["Geometry"], delete_geometry.inputs["Geometry"])
|
||||
links.new(culling_selection.outputs[0], delete_geometry.inputs["Selection"])
|
||||
|
||||
vertex_neighbors = nodes.new("GeometryNodeInputMeshVertexNeighbors")
|
||||
vertex_neighbors.location = (-550.0, -320.0)
|
||||
orphan_vertex = new_compare(
|
||||
nodes,
|
||||
"INT",
|
||||
"EQUAL",
|
||||
(-350.0, -320.0),
|
||||
0,
|
||||
)
|
||||
links.new(
|
||||
vertex_neighbors.outputs["Face Count"],
|
||||
input_socket(orphan_vertex, "A", 0),
|
||||
)
|
||||
delete_orphans = nodes.new("GeometryNodeDeleteGeometry")
|
||||
delete_orphans.domain = "POINT"
|
||||
delete_orphans.location = (-300.0, 100.0)
|
||||
links.new(delete_geometry.outputs["Geometry"], delete_orphans.inputs["Geometry"])
|
||||
links.new(orphan_vertex.outputs["Result"], delete_orphans.inputs["Selection"])
|
||||
culled_geometry = geometry
|
||||
if include_discontinuity:
|
||||
discontinuity_data = nodes.new("GeometryNodeInputNamedAttribute")
|
||||
discontinuity_data.data_type = "FLOAT"
|
||||
discontinuity_data.inputs["Name"].default_value = discontinuity_attribute
|
||||
discontinuity_data.location = (-1800.0, 350.0)
|
||||
culling_scale = nodes.new("ShaderNodeMath")
|
||||
culling_scale.operation = "MULTIPLY"
|
||||
culling_scale.inputs[1].default_value = 0.01
|
||||
culling_scale.location = (-1650.0, -50.0)
|
||||
inverse_culling = nodes.new("ShaderNodeMath")
|
||||
inverse_culling.operation = "SUBTRACT"
|
||||
inverse_culling.inputs[0].default_value = 0.5
|
||||
inverse_culling.location = (-1450.0, -50.0)
|
||||
exponent_scale = nodes.new("ShaderNodeMath")
|
||||
exponent_scale.operation = "MULTIPLY"
|
||||
exponent_scale.inputs[1].default_value = 10.0
|
||||
exponent_scale.location = (-1250.0, -50.0)
|
||||
threshold_power = nodes.new("ShaderNodeMath")
|
||||
threshold_power.operation = "POWER"
|
||||
threshold_power.inputs[0].default_value = 2.0
|
||||
threshold_power.location = (-1050.0, -50.0)
|
||||
threshold_scale = nodes.new("ShaderNodeMath")
|
||||
threshold_scale.operation = "MULTIPLY"
|
||||
threshold_scale.inputs[1].default_value = 0.04
|
||||
threshold_scale.location = (-850.0, -50.0)
|
||||
discontinuity_compare = new_compare(
|
||||
nodes,
|
||||
"FLOAT",
|
||||
"GREATER_THAN",
|
||||
(-650.0, 350.0),
|
||||
0.0,
|
||||
)
|
||||
links.new(
|
||||
discontinuity_input.outputs[discontinuity_name],
|
||||
culling_scale.inputs[0],
|
||||
)
|
||||
links.new(culling_scale.outputs[0], inverse_culling.inputs[1])
|
||||
links.new(inverse_culling.outputs[0], exponent_scale.inputs[0])
|
||||
links.new(exponent_scale.outputs[0], threshold_power.inputs[1])
|
||||
links.new(threshold_power.outputs[0], threshold_scale.inputs[0])
|
||||
links.new(
|
||||
discontinuity_data.outputs["Attribute"],
|
||||
input_socket(discontinuity_compare, "A", 0),
|
||||
)
|
||||
links.new(
|
||||
threshold_scale.outputs[0],
|
||||
input_socket(discontinuity_compare, "B", 1),
|
||||
)
|
||||
delete_geometry = nodes.new("GeometryNodeDeleteGeometry")
|
||||
delete_geometry.domain = "FACE"
|
||||
delete_geometry.location = (-350.0, 100.0)
|
||||
links.new(geometry, delete_geometry.inputs["Geometry"])
|
||||
links.new(
|
||||
discontinuity_compare.outputs["Result"],
|
||||
delete_geometry.inputs["Selection"],
|
||||
)
|
||||
culled_geometry = delete_geometry.outputs["Geometry"]
|
||||
|
||||
remeshed = add_remesh_zone(
|
||||
group,
|
||||
delete_orphans.outputs["Geometry"],
|
||||
culled_geometry,
|
||||
remesh_input.outputs["Resolution"],
|
||||
remesh_input.outputs["Relax Iterations"],
|
||||
topology_input.outputs["Fix Topology"],
|
||||
@@ -622,6 +653,53 @@ def build_group():
|
||||
return group
|
||||
|
||||
|
||||
def configure_moge_surface_group(group):
|
||||
nodes = group.nodes
|
||||
links = group.links
|
||||
controls = new_group_input(
|
||||
nodes,
|
||||
(-1800.0, 1050.0),
|
||||
{"Mask"},
|
||||
)
|
||||
mask_attribute = nodes.new("GeometryNodeInputNamedAttribute")
|
||||
mask_attribute.data_type = "BOOLEAN"
|
||||
mask_attribute.inputs["Name"].default_value = "o_mask"
|
||||
mask_attribute.location = (-1550.0, 1050.0)
|
||||
invert_mask = nodes.new("FunctionNodeBooleanMath")
|
||||
invert_mask.operation = "NOT"
|
||||
invert_mask.location = (-1350.0, 1050.0)
|
||||
use_mask = nodes.new("FunctionNodeBooleanMath")
|
||||
use_mask.operation = "AND"
|
||||
use_mask.location = (-1150.0, 1050.0)
|
||||
links.new(mask_attribute.outputs["Attribute"], invert_mask.inputs[0])
|
||||
links.new(controls.outputs["Mask"], use_mask.inputs[0])
|
||||
links.new(invert_mask.outputs[0], use_mask.inputs[1])
|
||||
first_processing = min(
|
||||
(
|
||||
node
|
||||
for node in nodes
|
||||
if node.bl_idname == "GeometryNodeDeleteGeometry"
|
||||
),
|
||||
key=lambda node: node.location.x,
|
||||
)
|
||||
geometry_input = first_processing.inputs["Geometry"]
|
||||
previous_link = geometry_input.links[0]
|
||||
previous_geometry = previous_link.from_socket
|
||||
links.remove(previous_link)
|
||||
delete_mask = nodes.new("GeometryNodeDeleteGeometry")
|
||||
delete_mask.domain = "FACE"
|
||||
delete_mask.location = (-750.0, 650.0)
|
||||
links.new(previous_geometry, delete_mask.inputs["Geometry"])
|
||||
links.new(use_mask.outputs[0], delete_mask.inputs["Selection"])
|
||||
links.new(delete_mask.outputs["Geometry"], geometry_input)
|
||||
group.description = (
|
||||
"Cull MoGe-2 mask and depth-edge faces, repair topology, preserve boundaries, "
|
||||
"and reproject UVs."
|
||||
)
|
||||
group.use_fake_user = True
|
||||
return group
|
||||
|
||||
|
||||
def build_sequence_frame(nodes, links, scene_frame, frames_input, y):
|
||||
relative_frame = nodes.new("ShaderNodeMath")
|
||||
relative_frame.operation = "SUBTRACT"
|
||||
@@ -1246,6 +1324,16 @@ def main():
|
||||
bpy.ops.wm.read_factory_settings(use_empty=True)
|
||||
bpy.context.preferences.filepaths.save_version = 0
|
||||
build_group()
|
||||
moge_surface_group = build_group(
|
||||
MOGE_SURFACE_GROUP_NAME,
|
||||
include_confidence=False,
|
||||
include_discontinuity=True,
|
||||
include_mask=True,
|
||||
discontinuity_name="Depth Continuity",
|
||||
discontinuity_attribute="o_depth_discontinuity",
|
||||
discontinuity_default=50.0,
|
||||
)
|
||||
configure_moge_surface_group(moge_surface_group)
|
||||
build_depth_plane_group()
|
||||
build_moge_depth_plane_group()
|
||||
OUTPUT_PATH.parent.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
@@ -6,20 +6,30 @@ import bpy
|
||||
|
||||
sys.path.insert(0, str(Path(__file__).parents[1] / "src"))
|
||||
|
||||
from snapform.image_depth import (
|
||||
from snapform.blender.depth_plane import (
|
||||
DEPTH_PLANE_MATERIAL_NODE_GROUP_NAME,
|
||||
SURFACE_MATERIAL_NODE_GROUP_NAME,
|
||||
add_depth_plane_processing,
|
||||
create_material,
|
||||
get_modifier_input,
|
||||
load_depth_plane_node_group,
|
||||
modifier_input_driver_path,
|
||||
modifier_input_identifier,
|
||||
set_modifier_input,
|
||||
)
|
||||
from snapform.image_mesh import load_geometry_node_group
|
||||
from snapform.blender.surface import load_geometry_node_group
|
||||
|
||||
|
||||
def main():
|
||||
bpy.ops.wm.read_factory_settings(use_empty=True)
|
||||
group = load_depth_plane_node_group()
|
||||
geometry_textures = [
|
||||
node
|
||||
for node in group.nodes
|
||||
if node.bl_idname == "GeometryNodeImageTexture"
|
||||
]
|
||||
assert geometry_textures
|
||||
assert all(node.extension == "EXTEND" for node in geometry_textures)
|
||||
background_depth = next(
|
||||
item
|
||||
for item in group.interface.items_tree
|
||||
@@ -212,6 +222,11 @@ def main():
|
||||
node.bl_idname in forbidden_outer_nodes
|
||||
for node in material.node_tree.nodes
|
||||
)
|
||||
assert all(
|
||||
node.extension == "EXTEND"
|
||||
for node in material.node_tree.nodes
|
||||
if node.bl_idname == "ShaderNodeTexImage"
|
||||
)
|
||||
surface_material = create_material("Surface", foreground, 1.0)
|
||||
surface_group = bpy.data.node_groups[SURFACE_MATERIAL_NODE_GROUP_NAME]
|
||||
surface_group_nodes = [
|
||||
@@ -251,6 +266,11 @@ def main():
|
||||
node.bl_idname in forbidden_outer_nodes
|
||||
for node in surface_material.node_tree.nodes
|
||||
)
|
||||
assert all(
|
||||
node.extension == "EXTEND"
|
||||
for node in surface_material.node_tree.nodes
|
||||
if node.bl_idname == "ShaderNodeTexImage"
|
||||
)
|
||||
modifier = add_depth_plane_processing(
|
||||
obj,
|
||||
depth,
|
||||
@@ -258,24 +278,28 @@ def main():
|
||||
material,
|
||||
animate_frame=True,
|
||||
)
|
||||
modifier[modifier_input_identifier(group, "Depth Scale")] = -2.0
|
||||
set_modifier_input(
|
||||
modifier,
|
||||
modifier_input_identifier(group, "Depth Scale"),
|
||||
-2.0,
|
||||
)
|
||||
for name, expected in (
|
||||
("Frame Start", 1),
|
||||
("Frames", 1),
|
||||
("Cyclic", False),
|
||||
):
|
||||
assert modifier[modifier_input_identifier(group, name)] == expected
|
||||
identifier = modifier_input_identifier(group, name)
|
||||
assert get_modifier_input(modifier, identifier) == expected
|
||||
assert not any(
|
||||
node.bl_idname == "GeometryNodeInputSceneTime"
|
||||
for node in group.nodes
|
||||
)
|
||||
frame_identifier = modifier_input_identifier(group, "Frame")
|
||||
frame_path = modifier_input_driver_path(modifier, frame_identifier)
|
||||
frame_driver = next(
|
||||
fcurve
|
||||
for fcurve in obj.animation_data.drivers
|
||||
if fcurve.data_path == (
|
||||
f'modifiers["{modifier.name}"]["{frame_identifier}"]'
|
||||
)
|
||||
if fcurve.data_path == frame_path
|
||||
)
|
||||
assert frame_driver.driver.type == "SCRIPTED"
|
||||
assert frame_driver.driver.expression == "frame"
|
||||
|
||||
@@ -7,7 +7,11 @@ import numpy as np
|
||||
|
||||
sys.path.insert(0, str(Path(__file__).parents[1] / "src"))
|
||||
|
||||
from snapform.image_mesh import create_matching_camera, load_geometry_node_group
|
||||
from snapform.blender.depth_plane import (
|
||||
get_modifier_input,
|
||||
set_modifier_input,
|
||||
)
|
||||
from snapform.blender.surface import create_matching_camera, load_geometry_node_group
|
||||
|
||||
|
||||
def set_input(modifier, group, name, value):
|
||||
@@ -18,18 +22,12 @@ def set_input(modifier, group, name, value):
|
||||
and item.in_out == "INPUT"
|
||||
and item.name == name
|
||||
)
|
||||
modifier[item.identifier] = value
|
||||
set_modifier_input(modifier, item.identifier, value)
|
||||
|
||||
|
||||
def add_face_attributes(mesh, count, background_values=None):
|
||||
for name in ("o_face_confidence", "o_edge_stretch"):
|
||||
attribute = mesh.attributes.new(name, "FLOAT", "FACE")
|
||||
attribute.data.foreach_set("value", [1.0] * count)
|
||||
background = mesh.attributes.new("o_background_face", "BOOLEAN", "FACE")
|
||||
background.data.foreach_set(
|
||||
"value",
|
||||
background_values or [False] * count,
|
||||
)
|
||||
def add_face_attributes(mesh, count):
|
||||
confidence = mesh.attributes.new("o_face_confidence", "FLOAT", "FACE")
|
||||
confidence.data.foreach_set("value", [1.0] * count)
|
||||
|
||||
|
||||
def main():
|
||||
@@ -45,6 +43,13 @@ def main():
|
||||
and item.name == "Geometry"
|
||||
)
|
||||
assert geometry_input.structure_type == "SINGLE"
|
||||
input_sockets = [
|
||||
item for item in group.interface.items_tree
|
||||
if item.item_type == "SOCKET" and item.in_out == "INPUT"
|
||||
]
|
||||
assert all(item.structure_type == "SINGLE" for item in input_sockets)
|
||||
resolution = next(item for item in input_sockets if item.name == "Resolution")
|
||||
assert abs(resolution.default_value - 0.1) < 1e-6
|
||||
assert group.asset_data is None
|
||||
assert not any(node.bl_idname == "GeometryNodeGroup" for node in group.nodes)
|
||||
assert not any(node.bl_idname == "GeometryNodeTriangulate" for node in group.nodes)
|
||||
@@ -56,7 +61,7 @@ def main():
|
||||
visible_outputs = [
|
||||
output for output in node.outputs if output.name and not output.hide
|
||||
]
|
||||
assert 1 <= len(visible_outputs) <= 3
|
||||
assert 1 <= len(visible_outputs) <= 4
|
||||
uv_stores = [
|
||||
node
|
||||
for node in group.nodes
|
||||
@@ -73,13 +78,24 @@ def main():
|
||||
assert not panels["Culling"].default_closed
|
||||
assert panels["Options"].default_closed
|
||||
assert [item.name for item in panels["Culling"].interface_items] == [
|
||||
"Depth Continuity",
|
||||
"Confidence",
|
||||
"Stretch",
|
||||
]
|
||||
assert [item.name for item in panels["Options"].interface_items] == [
|
||||
"Fix Topology",
|
||||
]
|
||||
for name, default_value in (("Confidence", 0.0), ("Stretch", 10.0)):
|
||||
smooth = next(
|
||||
item
|
||||
for item in group.interface.items_tree
|
||||
if item.item_type == "SOCKET"
|
||||
and item.in_out == "INPUT"
|
||||
and item.name == "Smooth"
|
||||
)
|
||||
assert smooth.default_value == 0.5
|
||||
for name, default_value in (
|
||||
("Confidence", 0.0),
|
||||
("Depth Continuity", 50.0),
|
||||
):
|
||||
item = next(
|
||||
item
|
||||
for item in panels["Culling"].interface_items
|
||||
@@ -98,9 +114,21 @@ def main():
|
||||
[],
|
||||
[(0, 2, 1), (1, 2, 3)],
|
||||
)
|
||||
discontinuity = mesh.attributes.new(
|
||||
"o_depth_discontinuity",
|
||||
"FLOAT",
|
||||
"FACE",
|
||||
)
|
||||
discontinuity.data.foreach_set("value", [0.03] * len(mesh.polygons))
|
||||
uv = mesh.uv_layers.new(name="UVMap")
|
||||
uv.data.foreach_set("uv", [0, 0, 0, 1, 1, 0, 1, 0, 0, 1, 1, 1])
|
||||
add_face_attributes(mesh, 2, [False, True])
|
||||
add_face_attributes(mesh, 2)
|
||||
assert not {
|
||||
"o_background",
|
||||
"o_background_face",
|
||||
"o_confidence",
|
||||
"o_confidence_percentile",
|
||||
}.intersection(mesh.attributes.keys())
|
||||
|
||||
obj = bpy.data.objects.new("Validation Object", mesh)
|
||||
bpy.context.scene.collection.objects.link(obj)
|
||||
@@ -121,13 +149,13 @@ def main():
|
||||
finally:
|
||||
evaluated.to_mesh_clear()
|
||||
|
||||
set_input(modifier, group, "Stretch", 100.0)
|
||||
stretch_item = next(
|
||||
set_input(modifier, group, "Depth Continuity", 100.0)
|
||||
discontinuity_item = next(
|
||||
item
|
||||
for item in group.interface.items_tree
|
||||
if item.item_type == "SOCKET" and item.name == "Stretch"
|
||||
if item.item_type == "SOCKET" and item.name == "Depth Continuity"
|
||||
)
|
||||
assert modifier[stretch_item.identifier] == 100.0
|
||||
assert get_modifier_input(modifier, discontinuity_item.identifier) == 100.0
|
||||
obj.update_tag(refresh={"DATA"})
|
||||
bpy.context.view_layer.update()
|
||||
culled = obj.evaluated_get(bpy.context.evaluated_depsgraph_get())
|
||||
|
||||
@@ -9,7 +9,7 @@ import numpy as np
|
||||
PROJECT_ROOT = Path(__file__).parents[1]
|
||||
sys.path.insert(0, str(PROJECT_ROOT / "src"))
|
||||
|
||||
from snapform.image_depth import (
|
||||
from snapform.blender.depth_plane import (
|
||||
MOGE_DEPTH_PLANE_NODE_GROUP_NAME,
|
||||
MOGE_MATERIAL_NODE_GROUP_NAME,
|
||||
build_moge2_depth_plane,
|
||||
@@ -72,6 +72,22 @@ def main():
|
||||
material_group = bpy.data.node_groups[MOGE_MATERIAL_NODE_GROUP_NAME]
|
||||
assert not any(node.bl_idname == "ShaderNodeNormalMap" for node in material_group.nodes)
|
||||
assert any(node.bl_idname == "ShaderNodeBump" for node in material_group.nodes)
|
||||
material = plane.data.materials[0]
|
||||
image_textures = [
|
||||
node
|
||||
for node in material.node_tree.nodes
|
||||
if node.bl_idname == "ShaderNodeTexImage"
|
||||
]
|
||||
assert image_textures
|
||||
assert all(node.extension == "EXTEND" for node in image_textures)
|
||||
assert any(node.image.name == "MoGe Test.depth" for node in image_textures)
|
||||
geometry_textures = [
|
||||
node
|
||||
for node in geometry_group.nodes
|
||||
if node.bl_idname == "GeometryNodeImageTexture"
|
||||
]
|
||||
assert geometry_textures
|
||||
assert all(node.extension == "EXTEND" for node in geometry_textures)
|
||||
print("MoGe-2 depth plane validation passed")
|
||||
|
||||
|
||||
|
||||
226
tools/validate_moge_surface.py
Normal file
226
tools/validate_moge_surface.py
Normal file
@@ -0,0 +1,226 @@
|
||||
import sys
|
||||
import tempfile
|
||||
from pathlib import Path
|
||||
|
||||
import bpy
|
||||
import numpy as np
|
||||
|
||||
|
||||
PROJECT_ROOT = Path(__file__).parents[1]
|
||||
sys.path.insert(0, str(PROJECT_ROOT / "src"))
|
||||
|
||||
from snapform.blender.depth_plane import MOGE_SURFACE_MATERIAL_NODE_GROUP_NAME
|
||||
from snapform.blender.surface import (
|
||||
MOGE_NODE_GROUP_NAME,
|
||||
build_image_mesh,
|
||||
prepare_moge_surface_archive,
|
||||
)
|
||||
|
||||
|
||||
def main():
|
||||
bpy.ops.wm.read_factory_settings(use_empty=True)
|
||||
arguments = sys.argv[sys.argv.index("--") + 1:] if "--" in sys.argv else []
|
||||
if arguments:
|
||||
validate_real_result(Path(arguments[0]), Path(arguments[1]))
|
||||
return
|
||||
height, width = 48, 64
|
||||
u, v = np.meshgrid(
|
||||
np.linspace(-0.5, 0.5, width, dtype=np.float32),
|
||||
np.linspace(-0.375, 0.375, height, dtype=np.float32),
|
||||
)
|
||||
depth = np.full((1, height, width), 2.0, dtype=np.float32)
|
||||
points = np.stack((u * 2.0, v * 2.0, depth[0]), axis=-1)[None]
|
||||
mask = np.ones((1, height, width), dtype=bool)
|
||||
mask[:, :2] = False
|
||||
intrinsics = np.array(
|
||||
[[[1.0, 0.0, 0.5], [0.0, 1.0, 0.5], [0.0, 0.0, 1.0]]],
|
||||
dtype=np.float32,
|
||||
)
|
||||
image = bpy.data.images.new("MoGe Surface Test", width=width, height=height)
|
||||
source = bpy.data.objects.new("MoGe Surface Test", None)
|
||||
source.empty_display_type = "IMAGE"
|
||||
source.data = image
|
||||
bpy.context.collection.objects.link(source)
|
||||
with tempfile.TemporaryDirectory() as temporary:
|
||||
directory = Path(temporary)
|
||||
results_path = directory / "results.npz"
|
||||
mesh_path = directory / "mesh.npz"
|
||||
np.savez(
|
||||
results_path,
|
||||
depth=depth,
|
||||
mask=mask,
|
||||
points=points,
|
||||
intrinsics=intrinsics,
|
||||
normal=np.broadcast_to(
|
||||
np.array([0.0, 0.0, -1.0], dtype=np.float32),
|
||||
(1, height, width, 3),
|
||||
),
|
||||
)
|
||||
prepare_moge_surface_archive(results_path, mesh_path)
|
||||
surface = build_image_mesh(
|
||||
bpy.context,
|
||||
source,
|
||||
mesh_path,
|
||||
False,
|
||||
True,
|
||||
False,
|
||||
True,
|
||||
)
|
||||
assert len(surface.data.vertices) > 0
|
||||
assert surface.modifiers[0].node_group.name == MOGE_NODE_GROUP_NAME
|
||||
inputs = {
|
||||
item.name
|
||||
for item in surface.modifiers[0].node_group.interface.items_tree
|
||||
if getattr(item, "in_out", None) == "INPUT"
|
||||
}
|
||||
assert "Stretch" not in inputs
|
||||
assert "Depth Continuity" in inputs
|
||||
assert "Mask" in inputs
|
||||
assert "Confidence" not in inputs
|
||||
assert "o_mask" in surface.data.attributes
|
||||
assert surface.data.attributes["o_mask"].domain == "FACE"
|
||||
assert not all(item.value for item in surface.data.attributes["o_mask"].data)
|
||||
attribute_names = set(surface.data.attributes.keys())
|
||||
assert attribute_names >= {"o_depth", "o_mask"}
|
||||
assert "o_depth_discontinuity" in attribute_names
|
||||
assert surface.data.attributes["o_depth_discontinuity"].data_type == "FLOAT"
|
||||
assert "o_edge_stretch" not in attribute_names
|
||||
assert not {
|
||||
"o_confidence",
|
||||
"o_confidence_percentile",
|
||||
"o_face_confidence",
|
||||
"o_background",
|
||||
"o_background_face",
|
||||
}.intersection(attribute_names)
|
||||
mask_socket = next(
|
||||
item
|
||||
for item in surface.modifiers[0].node_group.interface.items_tree
|
||||
if getattr(item, "in_out", None) == "INPUT" and item.name == "Mask"
|
||||
)
|
||||
assert mask_socket.default_value is False
|
||||
assert mask_socket.structure_type == "SINGLE"
|
||||
assert mask_socket.parent.name == "Culling"
|
||||
depth_edge_socket = next(
|
||||
item
|
||||
for item in surface.modifiers[0].node_group.interface.items_tree
|
||||
if getattr(item, "in_out", None) == "INPUT"
|
||||
and item.name == "Depth Continuity"
|
||||
)
|
||||
assert depth_edge_socket.default_value == 50.0
|
||||
assert depth_edge_socket.subtype == "PERCENTAGE"
|
||||
assert depth_edge_socket.structure_type == "SINGLE"
|
||||
culling_panel = next(
|
||||
item for item in surface.modifiers[0].node_group.interface.items_tree
|
||||
if item.item_type == "PANEL" and item.name == "Culling"
|
||||
)
|
||||
assert [item.name for item in culling_panel.interface_items] == [
|
||||
"Depth Continuity",
|
||||
"Mask",
|
||||
]
|
||||
input_sockets = [
|
||||
item for item in surface.modifiers[0].node_group.interface.items_tree
|
||||
if item.item_type == "SOCKET" and item.in_out == "INPUT"
|
||||
]
|
||||
assert all(item.structure_type == "SINGLE" for item in input_sockets)
|
||||
resolution_socket = next(
|
||||
item for item in input_sockets if item.name == "Resolution"
|
||||
)
|
||||
assert abs(resolution_socket.default_value - 0.1) < 1e-6
|
||||
smooth_socket = next(
|
||||
item
|
||||
for item in surface.modifiers[0].node_group.interface.items_tree
|
||||
if getattr(item, "in_out", None) == "INPUT" and item.name == "Smooth"
|
||||
)
|
||||
assert smooth_socket.default_value == 0.5
|
||||
assert not any(
|
||||
node.bl_idname == "GeometryNodeInputNamedAttribute"
|
||||
and node.inputs["Name"].default_value == "o_face_confidence"
|
||||
for node in surface.modifiers[0].node_group.nodes
|
||||
)
|
||||
assert MOGE_SURFACE_MATERIAL_NODE_GROUP_NAME in bpy.data.node_groups
|
||||
material_group = bpy.data.node_groups[MOGE_SURFACE_MATERIAL_NODE_GROUP_NAME]
|
||||
assert any(node.bl_idname == "ShaderNodeBump" for node in material_group.nodes)
|
||||
normal_map = next(
|
||||
node for node in material_group.nodes
|
||||
if node.bl_idname == "ShaderNodeNormalMap"
|
||||
)
|
||||
assert normal_map.space == "OBJECT"
|
||||
normal_strength = next(
|
||||
item for item in material_group.interface.items_tree
|
||||
if getattr(item, "in_out", None) == "INPUT"
|
||||
and item.name == "Normal Strength"
|
||||
)
|
||||
assert normal_strength.default_value == 1.0
|
||||
assert normal_strength.min_value == 0.0
|
||||
assert normal_map.inputs["Strength"].is_linked
|
||||
assert not surface.data.has_custom_normals
|
||||
normal_texture = next(
|
||||
node for node in surface.data.materials[0].node_tree.nodes
|
||||
if node.bl_idname == "ShaderNodeTexImage"
|
||||
and node.image.colorspace_settings.name == "Non-Color"
|
||||
)
|
||||
assert normal_texture.image.name == "MoGe Surface Test.normal"
|
||||
assert any(child.type == "CAMERA" for child in surface.children)
|
||||
print("MoGe-2 surface validation passed")
|
||||
|
||||
|
||||
def validate_real_result(results_path, image_path):
|
||||
image = bpy.data.images.load(str(image_path), check_existing=False)
|
||||
source = bpy.data.objects.new("MoGe Real Surface", None)
|
||||
source.empty_display_type = "IMAGE"
|
||||
source.data = image
|
||||
bpy.context.collection.objects.link(source)
|
||||
mesh_path = results_path.with_name("mesh.npz")
|
||||
if not mesh_path.is_file():
|
||||
prepare_moge_surface_archive(results_path, mesh_path)
|
||||
surface = build_image_mesh(
|
||||
bpy.context,
|
||||
source,
|
||||
mesh_path,
|
||||
False,
|
||||
True,
|
||||
False,
|
||||
True,
|
||||
)
|
||||
assert len(surface.data.vertices) > 100
|
||||
assert len(surface.data.polygons) > 100
|
||||
assert surface.modifiers[0].node_group.name == MOGE_NODE_GROUP_NAME
|
||||
assert MOGE_SURFACE_MATERIAL_NODE_GROUP_NAME in bpy.data.node_groups
|
||||
assert not surface.data.has_custom_normals
|
||||
material_group = bpy.data.node_groups[MOGE_SURFACE_MATERIAL_NODE_GROUP_NAME]
|
||||
assert any(node.bl_idname == "ShaderNodeBump" for node in material_group.nodes)
|
||||
normal_map = next(
|
||||
node for node in material_group.nodes
|
||||
if node.bl_idname == "ShaderNodeNormalMap"
|
||||
)
|
||||
assert normal_map.space == "OBJECT"
|
||||
assert normal_map.inputs["Strength"].is_linked
|
||||
attribute_names = set(surface.data.attributes.keys())
|
||||
assert {"o_depth", "o_mask"}.issubset(attribute_names)
|
||||
assert "o_depth_discontinuity" in attribute_names
|
||||
assert "o_edge_stretch" not in attribute_names
|
||||
assert not {
|
||||
"o_confidence",
|
||||
"o_confidence_percentile",
|
||||
"o_face_confidence",
|
||||
"o_background",
|
||||
"o_background_face",
|
||||
}.intersection(attribute_names)
|
||||
camera = next(child for child in surface.children if child.type == "CAMERA")
|
||||
dimensions = surface.dimensions
|
||||
assert max(dimensions) < 10.0
|
||||
assert min(dimensions.x, dimensions.y) > 0.1
|
||||
assert 1.0 <= camera.data.lens <= 500.0
|
||||
assert abs(camera.data.shift_x) <= 1.0
|
||||
assert abs(camera.data.shift_y) <= 1.0
|
||||
print(
|
||||
f"Real MoGe-2 surface validation passed: "
|
||||
f"{len(surface.data.vertices)} vertices, {len(surface.data.polygons)} faces, "
|
||||
f"dimensions={tuple(round(value, 4) for value in dimensions)}, "
|
||||
f"lens={camera.data.lens:.3f}, shift=({camera.data.shift_x:.4f}, "
|
||||
f"{camera.data.shift_y:.4f})"
|
||||
)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
108
uv.lock
generated
108
uv.lock
generated
@@ -6,6 +6,21 @@ resolution-markers = [
|
||||
"python_full_version < '3.12'",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "blendjob"
|
||||
version = "0.1.6"
|
||||
source = { path = "src/snapform/wheels/blendjob-0.1.6-py3-none-any.whl" }
|
||||
wheels = [
|
||||
{ filename = "blendjob-0.1.6-py3-none-any.whl", hash = "sha256:ed7c0ee4a0c952c25a7b6606bcd860b5c782ecb04ae6f9e7b1c475a03ab9c6b1" },
|
||||
]
|
||||
|
||||
[package.metadata]
|
||||
requires-dist = [
|
||||
{ name = "fastapi", marker = "extra == 'server'", specifier = "==0.139.2" },
|
||||
{ name = "uvicorn", marker = "extra == 'server'", specifier = "==0.51.0" },
|
||||
]
|
||||
provides-extras = ["server"]
|
||||
|
||||
[[package]]
|
||||
name = "colorama"
|
||||
version = "0.4.6"
|
||||
@@ -169,6 +184,91 @@ wheels = [
|
||||
{ url = "https://files.pythonhosted.org/packages/63/34/ba1c580383c9eada3711951fef0795c80b829a078d72188184bcab9dd527/packaging-26.3-py3-none-any.whl", hash = "sha256:d7193f7c8e4e93f444fde0262bf90af30e16fa0ad0ad44cb553c87339b23cd1c", size = 129956, upload-time = "2026-08-04T18:15:27.159Z" },
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "pillow"
|
||||
version = "12.3.0"
|
||||
source = { registry = "https://pypi.org/simple" }
|
||||
sdist = { url = "https://files.pythonhosted.org/packages/1c/3d/bb7fca845737cf9d7dbde16ed1843984665ff2e0a518f5db43e77ec540b9/pillow-12.3.0.tar.gz", hash = "sha256:3b8182a766685eaa002637e28b4ec8d6b18819a0c71f579bf0dbaa5830297cce", size = 47025035, upload-time = "2026-07-01T11:56:38.965Z" }
|
||||
wheels = [
|
||||
{ url = "https://files.pythonhosted.org/packages/fb/c8/0a78b0e02d7ac54bc03e5321c9220da52f0c2ea83b21f7c40e7f3169c502/pillow-12.3.0-cp311-cp311-macosx_10_10_x86_64.whl", hash = "sha256:00808c5e14ef63ac5161091d242999076604ff74b883423a11e5d7bbb38bf756", size = 5392415, upload-time = "2026-07-01T11:53:47.162Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/b2/5b/a02d30018abd97ced9f5a6c63d28597694a00d066516b9c1c6de45859fc9/pillow-12.3.0-cp311-cp311-macosx_11_0_arm64.whl", hash = "sha256:37d6d0a00072fd2948eb22bce7e1475f34569d90c87c59f7a2ec59541b77f7a6", size = 4785266, upload-time = "2026-07-01T11:53:49.079Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/c8/98/766667a4be768150a202836acd9fad19c06824ca86c4286d3cf6b274964e/pillow-12.3.0-cp311-cp311-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:bcb46e2f9feff8d06323983bd83ed00c201fdcab3d74973e7072a889b3979fcd", size = 6263814, upload-time = "2026-07-01T11:53:51.32Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/3b/2d/ede717bc1144f63886c21fd349bb95860b0d1a21149ff16f2bb362b612b6/pillow-12.3.0-cp311-cp311-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:23d27a3e0307ec2244cc51e7287b919aa68d097504ebe19df4e76a98a3eea5bd", size = 6934408, upload-time = "2026-07-01T11:53:53.487Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/a3/48/9c58b685e69d49c31af6c8eb9012055fab7e665785165c84796e2c73ce72/pillow-12.3.0-cp311-cp311-musllinux_1_2_aarch64.whl", hash = "sha256:4f883547d4b7f0495ebe7056b0cc2aea76094e7a4abc8e933540f3271df27d9c", size = 6337160, upload-time = "2026-07-01T11:53:55.457Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/ff/fa/dc2a5c0ba6df93f67c31d34b808b7ce440b40cdbf96f0b81cde1d1e6fa93/pillow-12.3.0-cp311-cp311-musllinux_1_2_x86_64.whl", hash = "sha256:236ff70b9312fb68943c703aa842ca6a758abfa45ac187a5e7c1452e96ef72b5", size = 7045172, upload-time = "2026-07-01T11:53:57.736Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/86/a5/444817a4d4c4c2417df00513086ca196f388d8f9ef40c2e4ccd1ad1af54b/pillow-12.3.0-cp311-cp311-win32.whl", hash = "sha256:10e41f0fbf1eec8cfd234b8fe17a4caac7c9d0db4c204d3c173a8f9f6ef3232b", size = 6472232, upload-time = "2026-07-01T11:53:59.767Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/63/c6/4bad1b18d132a50b27e1365e1ab163616f7a5bb56d330f66f9d1d9d4f9d4/pillow-12.3.0-cp311-cp311-win_amd64.whl", hash = "sha256:8e95e1385e4998ae9694eeaa4730ba5457ff61185b3a55e2e7bea0880aef452a", size = 7233653, upload-time = "2026-07-01T11:54:02.066Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/fd/16/00f91ab7760dc842f5aad55217e80fc4a7067a0604535249bc8a2d6d9870/pillow-12.3.0-cp311-cp311-win_arm64.whl", hash = "sha256:ebaea975e03d3141d9d3a507df75c9b3ec90fa9d2ffd07567b3a978d9d790b26", size = 2568195, upload-time = "2026-07-01T11:54:04.622Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/37/bf/fb3ebff8ddcb76aac5a01389251bbbb9519922a9b520d8247c1ca864a25d/pillow-12.3.0-cp312-cp312-macosx_10_13_x86_64.whl", hash = "sha256:ba09209fbe443b4acccebe845d8a138b89a8f4fbaeedd44953490b5315d5e965", size = 5345969, upload-time = "2026-07-01T11:54:06.397Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/d8/66/9a386a92561f402389a4fc70c18838bf6d35eb5eb5c6850b4b2dc64f5048/pillow-12.3.0-cp312-cp312-macosx_11_0_arm64.whl", hash = "sha256:ffd0c5368496f41b0944be820fcb7a838aa6e623d250b01acf2643939c3f99d7", size = 4780323, upload-time = "2026-07-01T11:54:09.351Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/25/27/ac8f99618ffd3dde21db0f4d4b1d2ab00c0880595bfd17df103f7f39fd0c/pillow-12.3.0-cp312-cp312-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:d9c7f76c0673154f044e9d78c8655fb4213f6ca31a836df48b40fe5d187717b9", size = 6266838, upload-time = "2026-07-01T11:54:11.71Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/84/21/a35af28dcc61f37ed850a2d64c65c701321dfbf25085e469d5559360cbbf/pillow-12.3.0-cp312-cp312-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:78cb2c6865a35ab8ff8b75fd122f6033b92a62c82801110e48ddd6c936a45d91", size = 6940830, upload-time = "2026-07-01T11:54:13.732Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/eb/51/8b08617af3ad95e33ce6d7dd2c99ed6c8298f7fb131636303956be022e25/pillow-12.3.0-cp312-cp312-musllinux_1_2_aarch64.whl", hash = "sha256:e491916b378fba47242221bb9ead245211b70d504f495d105d17b14a24b4907c", size = 6344383, upload-time = "2026-07-01T11:54:15.756Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/1d/72/cf78ac9780bb93c28328f408973845a309d4d145041665f734572ced1b52/pillow-12.3.0-cp312-cp312-musllinux_1_2_x86_64.whl", hash = "sha256:0dd2064cbc55aaec028ef5fbb60fa47bb6c3e7918e07ff17935284b227a9d2df", size = 7052934, upload-time = "2026-07-01T11:54:17.721Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/20/20/25e0f4dc178a6bc0696793720055519a0de89e7661dae886992decbd2f81/pillow-12.3.0-cp312-cp312-win32.whl", hash = "sha256:dbce0b29841537a2fa4a214c2bbf14de3587c9680caa9b4e217568472490b28f", size = 6472684, upload-time = "2026-07-01T11:54:19.839Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/45/89/da2f7971a317f83d807fdd4065c0af40208e59e692cc43d315a71a0e96d1/pillow-12.3.0-cp312-cp312-win_amd64.whl", hash = "sha256:a2b55dd6b2a4c4b7d87ffa56bdb33fdc5fdb9a462173861a7bc097f17d91cb09", size = 7227137, upload-time = "2026-07-01T11:54:22.025Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/de/47/4845a0a6c0dbf1db8456bd9fc791f13c5ced7ced20606d08a0aacfd25b49/pillow-12.3.0-cp312-cp312-win_arm64.whl", hash = "sha256:331b624368d4f1d069149002f25f44bc61c8919ce8ddb3c45bdad8f6e2d89510", size = 2568267, upload-time = "2026-07-01T11:54:24.051Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/9d/ac/31fb64e1e7efb5a4b50cd3d92049ba89ac6e4d8d3bb6a74e15048ca3353e/pillow-12.3.0-cp313-cp313-ios_13_0_arm64_iphoneos.whl", hash = "sha256:21900ce7ba264168cd50defae43cd75d25c833ad4ad6e73ffc5596d12e25ac89", size = 4161684, upload-time = "2026-07-01T11:54:25.934Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/87/b4/9805e23d2b4d77842b468513841fda254ee42f0289d25088340e4ff46e2d/pillow-12.3.0-cp313-cp313-ios_13_0_arm64_iphonesimulator.whl", hash = "sha256:4e8c2a84d977f50b9daed6eeaf3baef67d00d5d74d932288f02cb94518ee3ace", size = 4255487, upload-time = "2026-07-01T11:54:27.935Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/df/39/ecf519435a200c693fe053a6ee4d835b41cf963a4dfc2551c4e637cb2a71/pillow-12.3.0-cp313-cp313-ios_13_0_x86_64_iphonesimulator.whl", hash = "sha256:ae26d61dfa7a47befdc7572b521024e8745f3d809bd95ca9505a7bba9ef849ec", size = 3696433, upload-time = "2026-07-01T11:54:29.813Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/42/92/2fc3ffad878ae8dd5469ec1bc8eb83b71f48e13efdf68f02709003982a32/pillow-12.3.0-cp313-cp313-macosx_10_13_x86_64.whl", hash = "sha256:7a743ff716f746fc19a9557f60dab1600d4613255f8a7aeb3cdde4db7eb15a66", size = 5345889, upload-time = "2026-07-01T11:54:31.97Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/10/76/8803c13605b763d33d156c4678fc77f8443389c0c51c8aef707bb02015f4/pillow-12.3.0-cp313-cp313-macosx_11_0_arm64.whl", hash = "sha256:d69141514cc30b774ceea5e3ed3a6635c8d8a96edf664689b890f4089111fb35", size = 4780109, upload-time = "2026-07-01T11:54:34.026Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/1f/01/e18aff37cb0b4aac47ac90f016d347a49aca667ef97f190b06ac2aabc928/pillow-12.3.0-cp313-cp313-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:f7401aebd7f581d7f83a439d87d474999317ee099218e5ad25d125290990ba65", size = 6263736, upload-time = "2026-07-01T11:54:36.131Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/f7/62/de5bdd77d935331f4f802edc11e4d82950f642caad6cb2f949837b8560e2/pillow-12.3.0-cp313-cp313-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:0847a763afefb695bc912d7c131e7e0632d4edc1d8698f58ddabec8e46b8b6d3", size = 6937129, upload-time = "2026-07-01T11:54:38.216Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/70/4d/105627a13300c5e0df1d174230b32fd1273062c96f7745fd552b945d1e1d/pillow-12.3.0-cp313-cp313-musllinux_1_2_aarch64.whl", hash = "sha256:571b9fcb07b97ef3a492028fb3d2dc0993ca23a06138b0315286566d29ef718a", size = 6339562, upload-time = "2026-07-01T11:54:40.354Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/6b/1d/f13de01a553988ab895ba1c722e06cf3144d4f57656fd5b81b6d881f1179/pillow-12.3.0-cp313-cp313-musllinux_1_2_x86_64.whl", hash = "sha256:756c768d0c9c2955feb7a56c37ea24aea2e369f8d36a88da270b6a9f19e62b5e", size = 7049439, upload-time = "2026-07-01T11:54:42.489Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/c9/f9/066794cca041b969964f779ee5fa66a9498bbf34248ac39c5d7954e4198f/pillow-12.3.0-cp313-cp313-win32.whl", hash = "sha256:a876864214e136f0eb367788dbd7df045f4806801518e2cfe9e13229cfe06d8f", size = 6473287, upload-time = "2026-07-01T11:54:44.9Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/a6/9b/7a58e61d62be561da3a356fe2384d4059a6345fc130e23ef1c36a5b81d24/pillow-12.3.0-cp313-cp313-win_amd64.whl", hash = "sha256:1cca606cd25738df4ed873d5ad46bbdb3d83b5cbca291f6b4ff13a4df6b0bbe8", size = 7239691, upload-time = "2026-07-01T11:54:47.141Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/aa/b0/c4ed4f0ef8f8fa5ee8351537db6650bb8189f7e118842978dd6589065692/pillow-12.3.0-cp313-cp313-win_arm64.whl", hash = "sha256:b629de27fda84b42cde7edef0d85f13b958b47f6e9bbcbba9b673c562a89bd8b", size = 2568185, upload-time = "2026-07-01T11:54:49.137Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/dc/01/001f65b68192f0228cc1dbbc8d2530ab5d58b61037ba0587f946fea607cd/pillow-12.3.0-cp314-cp314-ios_13_0_arm64_iphoneos.whl", hash = "sha256:9cf95fe4d0f84c82d282745d9bb08ad9f926efa00be4697e767b814ce40d4330", size = 4161736, upload-time = "2026-07-01T11:54:51.156Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/1a/d2/0219746d0fd16fc8a84498e79452375be3797d3ce4044596ce565164b84f/pillow-12.3.0-cp314-cp314-ios_13_0_arm64_iphonesimulator.whl", hash = "sha256:8728f216dcdb6e6d555cf971cb34076139ad74b31fc2c14da4fafc741c5f6217", size = 4255435, upload-time = "2026-07-01T11:54:53.414Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/c8/02/8d0bc62ef0302318c46ff2a512822d2610e81c7aa46c9b3abe6cbaca5ad0/pillow-12.3.0-cp314-cp314-ios_13_0_x86_64_iphonesimulator.whl", hash = "sha256:a45650e8ce7fafffd731db8550230db6b0d306d181a90b67d3e6bca2f1990930", size = 3696262, upload-time = "2026-07-01T11:54:55.739Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/85/e2/73c77d218410b14f5f2d565e8a998d5317b7b9c75368d29985139f7a46f0/pillow-12.3.0-cp314-cp314-macosx_10_15_x86_64.whl", hash = "sha256:ba54cfebe86920a559a7c4d6b9050791c20513650a1952ebe3368c7dc70306f8", size = 5350344, upload-time = "2026-07-01T11:54:57.657Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/c7/da/32c752228ae345f489e3a42499d817b6c3996da7e8a3bc7a04fc806b243b/pillow-12.3.0-cp314-cp314-macosx_11_0_arm64.whl", hash = "sha256:e158cb00350dc278f3b91551101aa7d12415a66ebf2c91d8d5ac14e56ddd3ad0", size = 4780131, upload-time = "2026-07-01T11:54:59.713Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/b1/9d/8b2c807dbef61a5197c047afe99823787eb66f63daf9fb2432f91d6f0462/pillow-12.3.0-cp314-cp314-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:e9aeb04d6aef139de265b29683e119b638208f88cf73cdd1658aa07221165321", size = 6263757, upload-time = "2026-07-01T11:55:01.778Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/5c/44/c85361f65dbe00eea8576ee467c768d25129989efb76e94f205e9ca9bb46/pillow-12.3.0-cp314-cp314-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:251bf95b67017e27b13d82f5b326234ca62d70f9cf4c2b9032de2358a3b12c7b", size = 6936962, upload-time = "2026-07-01T11:55:03.93Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/18/7e/e483414b35800b86b6f08dbbc7803fb5cd52c4d6f897f47d53ea2c7e6f65/pillow-12.3.0-cp314-cp314-musllinux_1_2_aarch64.whl", hash = "sha256:fe3cca2e4e8a592be0f269a1ca4835c25199d9f3ce815c8491048f785b0a0198", size = 6339171, upload-time = "2026-07-01T11:55:05.989Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/f0/f4/68c491844841ede6bed70189546b3ee9731cf9f2cbad396faff5e1ccba45/pillow-12.3.0-cp314-cp314-musllinux_1_2_x86_64.whl", hash = "sha256:23aceaa007d6172b02c277f0cd359c79492bbb14f7072b4ede9fbcaf20648130", size = 7048116, upload-time = "2026-07-01T11:55:08.131Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/a3/34/77f3f793fed8efc7d243f21b33c5a3f0d1c97ee70346d3db855587e155ff/pillow-12.3.0-cp314-cp314-win32.whl", hash = "sha256:af8d94b0db561cf68b88a267c5c44b49e134f525d0dc2cb7ed413a66bc23559a", size = 6467209, upload-time = "2026-07-01T11:55:10.408Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/f1/e0/492879f69d94f91f60fc8cd05ba03650e9520afebb2fb7aa12777d7c7f38/pillow-12.3.0-cp314-cp314-win_amd64.whl", hash = "sha256:fdafc9cce40277e0f7a0feabce0ee50dd2fa1800f3b38015e51296b5e814048d", size = 7237707, upload-time = "2026-07-01T11:55:12.745Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/c9/ac/6b11f2875f1c2ac040d84e1bbf9cf22a88038f901ca1037898b280b38365/pillow-12.3.0-cp314-cp314-win_arm64.whl", hash = "sha256:e91206ee562682b51b98ef4b26a6ef48fd84e15fd4c4bc5ec768eb641d206838", size = 2565995, upload-time = "2026-07-01T11:55:14.736Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/52/69/c2208e56af9bfc1913afb24020297a691eb1d4ef688474c8a04913f65e04/pillow-12.3.0-cp314-cp314t-macosx_10_15_x86_64.whl", hash = "sha256:164b31cd1a0490ab6efae01aa5df49da7061be0af1b30e035b6e9a1bfe34ee6e", size = 5352503, upload-time = "2026-07-01T11:55:17.076Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/07/70/e5686d753e898a45d778ff1718dba8516ead6ab6b95d85fc8c4b70650cf2/pillow-12.3.0-cp314-cp314t-macosx_11_0_arm64.whl", hash = "sha256:5afb51d599ea772b8365ae807ae557f18bccfe46ab261fd1c2a9ed700fc6eb17", size = 4782956, upload-time = "2026-07-01T11:55:19.448Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/d5/37/25c6692f06927ee973ff18c8d9ee98ad0b4d84ee67a09610c2dd1447958e/pillow-12.3.0-cp314-cp314t-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:3edce1d53195db527e0191f84b71d02022de0540bf43a16ed734ed7537b07385", size = 6322855, upload-time = "2026-07-01T11:55:21.613Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/cc/91/420637fcb8f1bc11029e403b4538e6694744428d8246118e45719f944556/pillow-12.3.0-cp314-cp314t-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:bf16ba1b4d0b6b7c8e534936632270cf70eb00dbe09005bc345b2677b726855c", size = 6989642, upload-time = "2026-07-01T11:55:24.006Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/10/08/b94d7811281ccf0d143a1cf768d1c49e1e54af63e7b708ab2ee3eb87face/pillow-12.3.0-cp314-cp314t-musllinux_1_2_aarch64.whl", hash = "sha256:24870b09b224f7ae3c39ed07d10e819d06f8720bc551847b1d623832b5b0e28d", size = 6391281, upload-time = "2026-07-01T11:55:26.252Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/d2/87/24233f785f55474dc02ce3e739c5528a77e3a862e9333d1dd7a25cc31f70/pillow-12.3.0-cp314-cp314t-musllinux_1_2_x86_64.whl", hash = "sha256:30f2aa603c41533cc25c05acd0da21636e84a315768feb631c937177db558931", size = 7096716, upload-time = "2026-07-01T11:55:28.318Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/23/26/fcb2f6e37175b04f53570b59937867e2b80ee1685e744023153028fc14f9/pillow-12.3.0-cp314-cp314t-win32.whl", hash = "sha256:4b0a7fe987b14c31ebda6083f74f22b561fd3739bc0ac51e019622e3d72668c7", size = 6474125, upload-time = "2026-07-01T11:55:30.956Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/90/de/3634abee5f1c9e13c56787b7d5517b0ba8d6de51700b95578cf338349c9f/pillow-12.3.0-cp314-cp314t-win_amd64.whl", hash = "sha256:962864dc93511324d51ddbb5b9f8731bf71675b93ca612a07441896f4688fb8c", size = 7242939, upload-time = "2026-07-01T11:55:34.044Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/ce/2a/fd13f8eb24de5714a6eb444a3d67e2842c6c576e159a43793adf23051351/pillow-12.3.0-cp314-cp314t-win_arm64.whl", hash = "sha256:0740a512dc522224c77d9aa5a8d70d8b7d73fb91f2c21125d8d025d3b8990e45", size = 2567506, upload-time = "2026-07-01T11:55:35.988Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/5d/dc/8fdce34ec725a33c81c6ba122b904d6b9024e50ea9ac7bede62fab54506c/pillow-12.3.0-cp315-cp315-ios_13_0_arm64_iphoneos.whl", hash = "sha256:0feb2e9d6ad6c9e3c06effe9d00f3f1e618a6643273576b016f591e9315a7139", size = 4162063, upload-time = "2026-07-01T11:55:37.941Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/76/66/2044b9a63d3b84ff048228dfcb7cd9bf0df983e8470971bf7d4c57b693de/pillow-12.3.0-cp315-cp315-ios_13_0_arm64_iphonesimulator.whl", hash = "sha256:9e881fca225083806662a5c43d627d215f258ff43c890f831966c7d7ba9c7402", size = 4255549, upload-time = "2026-07-01T11:55:40.022Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/52/7e/1f67e6f4ece6b582ee4b539decbcc9f848dc245a93ed8cd7338bafef72f1/pillow-12.3.0-cp315-cp315-ios_13_0_x86_64_iphonesimulator.whl", hash = "sha256:4998562bf62a445225f22e07c896bb04b35b1b1f2eb6d760584c9c51d7a5f78c", size = 3696331, upload-time = "2026-07-01T11:55:41.98Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/12/40/d306fc2c8e4d45d7f175c77edca7063be7b86fe7fe6e68f4353bf71d808c/pillow-12.3.0-cp315-cp315-macosx_10_15_x86_64.whl", hash = "sha256:dc624f6bc473dacdf7ef7eb8678d0d08edf15cd94fad6ae5c7d6cc67a4e4902f", size = 5350370, upload-time = "2026-07-01T11:55:44.028Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/dd/44/668fb1437e8ce420f62d6106eb66e44a5971602a4d794615bdf79315d82d/pillow-12.3.0-cp315-cp315-macosx_11_0_arm64.whl", hash = "sha256:71d6097b330eea8fd15097780c8e89cb1a8ce7838669f48c5bacd6f663dd4701", size = 4780147, upload-time = "2026-07-01T11:55:46.073Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/0c/08/93fa2e70e30a2d81547e481b6ee2bb9522117221fb1e0ce4b5df70967677/pillow-12.3.0-cp315-cp315-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:28ce87c5ab450a9dd970b52e5aca5fe63ed432d18a2eaddd1979a00a1ba24ace", size = 6273659, upload-time = "2026-07-01T11:55:48.264Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/f8/6d/043e96ff814fc31a33077e4cba86082167db520c93632afdf2042febbb0c/pillow-12.3.0-cp315-cp315-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:6b02afb9b97f65fbca5f31db6a2a3ba21aa93030225f150fa3f249717e938fb4", size = 6947439, upload-time = "2026-07-01T11:55:50.503Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/af/92/ba71d2ee2ac0edf3fa33bd9d5ee9ee080da70b1766f3ca3934f9938ddac9/pillow-12.3.0-cp315-cp315-musllinux_1_2_aarch64.whl", hash = "sha256:1182d52bc2d5e5d7d0949503aa7e36d12f42205dc287e4883f407b1988820d39", size = 6353577, upload-time = "2026-07-01T11:55:52.697Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/0f/ce/e63064e2122923ff687c8ad792d0d736a7b3920a56a46982e81a7fdd25d6/pillow-12.3.0-cp315-cp315-musllinux_1_2_x86_64.whl", hash = "sha256:e795b7eb908249c4e43c7c99fac7c2c75dab0c43566e37db472a355f63693d71", size = 7060394, upload-time = "2026-07-01T11:55:55.149Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/54/76/a09cc3ccc8d773a7283d34c38bec1708f9e3cc932093cbc4c5e71ac4060b/pillow-12.3.0-cp315-cp315-win32.whl", hash = "sha256:57b3d78c95ba9059768b10e28b813002261d3f3dfc55cc48b0c988f625175827", size = 6467375, upload-time = "2026-07-01T11:55:57.769Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/3e/03/1846c49ba3b1d5550392a4bbd06d6fb4578e1cd91a803198b5c90f5f7d53/pillow-12.3.0-cp315-cp315-win_amd64.whl", hash = "sha256:fa4ecea169a355be7a3ade2c783e2ed12f0e40d2c5621cda8b3297faf7fbb9f5", size = 7237048, upload-time = "2026-07-01T11:55:59.975Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/fb/bb/89f35dcc79610423f9f195504d7def7f0d1416a711541b42867e25fe3412/pillow-12.3.0-cp315-cp315-win_arm64.whl", hash = "sha256:877c3f311ff35410f690861c4409e7ccbf0cd2f878e50628a28e5a0bb689e658", size = 2566006, upload-time = "2026-07-01T11:56:02.143Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/30/88/707027ba09942dfa2c28759b5c222d769290a41c6d20ea60ec250801941f/pillow-12.3.0-cp315-cp315t-macosx_10_15_x86_64.whl", hash = "sha256:e9871b1ffbfa9656b60aeee92ed5136a5742696006fa322b29ea3d8da0ecc9cf", size = 5352509, upload-time = "2026-07-01T11:56:04.2Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/b0/6d/00352fa25332c2569cd387851f568cc5a4b75a9adbfb37ac4fbce4c02eec/pillow-12.3.0-cp315-cp315t-macosx_11_0_arm64.whl", hash = "sha256:53aa02d20d10c3d814d536aa4e5ac9b84ca0ff5a88377963b085ad6822f93e64", size = 4783167, upload-time = "2026-07-01T11:56:06.631Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/13/4f/9e049dfa21af7c22427275720e2490267ba8138120add5c4c574deb69782/pillow-12.3.0-cp315-cp315t-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:446c34dcc4324b084a53b705127dc15717b22c5e140ae0a3c38349d4efec071e", size = 6329237, upload-time = "2026-07-01T11:56:08.868Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/36/16/cf6eeaae8d0fce8dd390a33437cf68c5d5bd73834a2bc6e2f14efda0ab45/pillow-12.3.0-cp315-cp315t-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:cf1845d02ad822a369a49f2bb9345b1614744267682e7a03527dc3bf6eea1777", size = 6997047, upload-time = "2026-07-01T11:56:11.379Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/1e/69/dbf769bdd55f48bf5733cac28edc6364ffaa072ec9ba336266e4fe66be55/pillow-12.3.0-cp315-cp315t-musllinux_1_2_aarch64.whl", hash = "sha256:186941b6aef820ad110fb01fb06eb925374dc3a21b17e37ec9a53b250c6fe2d1", size = 6400440, upload-time = "2026-07-01T11:56:13.908Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/a0/e1/ffc9cfc2eea0d178da8018e18e959301ad9d6bc9f3edb7181e748a474b97/pillow-12.3.0-cp315-cp315t-musllinux_1_2_x86_64.whl", hash = "sha256:f13c32a3abd6079a66d9526e18dad9b6d280384d49d7c54040cd57b6424041d9", size = 7105895, upload-time = "2026-07-01T11:56:16.575Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/18/f0/a5595c1e8c3ae44b9828cb2f0fa8155e5095ef04d6327b8f61cf44a3df85/pillow-12.3.0-cp315-cp315t-win32.whl", hash = "sha256:1657923d2d45afb66526e5b933e5b3052e6bdea196c90d3abb2424e18c77dae8", size = 6474384, upload-time = "2026-07-01T11:56:18.855Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/e4/04/62bcd9f844984c5938d3b05264a61d797a29d3e0812341a8204af70bbdee/pillow-12.3.0-cp315-cp315t-win_amd64.whl", hash = "sha256:8cd2f7bdda092d99c9fc2fb7391354f306d01443d22785d0cbfafa2e2c8bb418", size = 7243537, upload-time = "2026-07-01T11:56:21.214Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/3d/68/1f3066acedf37673694a7141381d8f811ae97f30d34413d236abe7d489f1/pillow-12.3.0-cp315-cp315t-win_arm64.whl", hash = "sha256:06ff022112bc9cbf83b60f8e028d94ad87b60621706487e65f673de61610ab59", size = 2567491, upload-time = "2026-07-01T11:56:23.506Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/75/18/2e8b40223153ccbc60df07f9e8928dc0c76202aa4e55ae9f53962b6510d6/pillow-12.3.0-pp311-pypy311_pp73-macosx_10_15_x86_64.whl", hash = "sha256:b3c777e849237620b022f7f297dd67705f9f5cf1685f09f02e46f93e92725468", size = 5302510, upload-time = "2026-07-01T11:56:25.736Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/46/3e/51fabf59d5ab801ceab709453d3ab6b180083496579549de4c45ced6528a/pillow-12.3.0-pp311-pypy311_pp73-macosx_11_0_arm64.whl", hash = "sha256:b343699e8308bdc51978310e1c959c584e7869cc8c40780058c87da7781a1e94", size = 4736058, upload-time = "2026-07-01T11:56:28.041Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/bf/20/22fe9384b7949e25fb1293bcfc84fb82590ff4ea6b37c95b24d26d793d86/pillow-12.3.0-pp311-pypy311_pp73-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:fbd139c8447d25dd750ab79ee274cc5e1fe80fc56340ab10b18a195e1b6eca3e", size = 5237776, upload-time = "2026-07-01T11:56:30.263Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/08/14/f6ba68107680ffa74b39985f3f30884e41318fbc4250caa423c79b4788bb/pillow-12.3.0-pp311-pypy311_pp73-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:e7e480451b9fa137494bccd3a7d69adbe8ac65a87d97be61e11f1b1050a5bac3", size = 5860358, upload-time = "2026-07-01T11:56:32.68Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/36/54/0169bc772ec491108b62f644f8ecf1fe5d8ae5ebafde2ee2142210166903/pillow-12.3.0-pp311-pypy311_pp73-win_amd64.whl", hash = "sha256:04f01d28a6aaff387bf842a13be313df23ba0597a44f1a976c9feb3c6ff4711a", size = 7231786, upload-time = "2026-07-01T11:56:35.046Z" },
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "pluggy"
|
||||
version = "1.6.0"
|
||||
@@ -205,20 +305,26 @@ wheels = [
|
||||
|
||||
[[package]]
|
||||
name = "snapform"
|
||||
version = "0.10.11"
|
||||
version = "0.12.3"
|
||||
source = { virtual = "." }
|
||||
dependencies = [
|
||||
{ name = "blendjob" },
|
||||
]
|
||||
|
||||
[package.dev-dependencies]
|
||||
dev = [
|
||||
{ name = "numpy", version = "2.4.6", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version < '3.12'" },
|
||||
{ name = "numpy", version = "2.5.1", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version >= '3.12'" },
|
||||
{ name = "pillow" },
|
||||
{ name = "pytest" },
|
||||
]
|
||||
|
||||
[package.metadata]
|
||||
requires-dist = [{ name = "blendjob", path = "src/snapform/wheels/blendjob-0.1.6-py3-none-any.whl" }]
|
||||
|
||||
[package.metadata.requires-dev]
|
||||
dev = [
|
||||
{ name = "numpy" },
|
||||
{ name = "pillow" },
|
||||
{ name = "pytest" },
|
||||
]
|
||||
|
||||
@@ -1,46 +0,0 @@
|
||||
schema_version = 1
|
||||
|
||||
[artifacts.depth-anything-3]
|
||||
revision = "3d835ec1a5802d64a8b8b15f817a1ab54809bfe4"
|
||||
filename = "depth-anything-3.tar.gz"
|
||||
url = "https://codeload.github.com/ByteDance-Seed/Depth-Anything-3/tar.gz/3d835ec1a5802d64a8b8b15f817a1ab54809bfe4"
|
||||
sha256 = "98aa2dd53ab44b96cef5190ae4841c6ae51797d099b784e08e12a1a55bd3a69b"
|
||||
patches = "depth-anything-3"
|
||||
|
||||
[artifacts.ben2]
|
||||
revision = "2c99a5da477b5523585bfa5c893888a6e818a8f6"
|
||||
filename = "ben2.tar.gz"
|
||||
url = "https://codeload.github.com/PramaLLC/BEN2/tar.gz/2c99a5da477b5523585bfa5c893888a6e818a8f6"
|
||||
sha256 = "01fd26716f3ff3a3f18ad2336c768af5b759edfc7c38bb3bf733b7c86eee6282"
|
||||
|
||||
[artifacts.moge]
|
||||
revision = "925b8ed835a7a9cdb7578ba15c658a0afc969030"
|
||||
filename = "moge.tar.gz"
|
||||
url = "https://codeload.github.com/microsoft/MoGe/tar.gz/925b8ed835a7a9cdb7578ba15c658a0afc969030"
|
||||
sha256 = "c6d90b2564d643f0f3ae65b5ffe163abaea3b55d194c6e2ba9976d67f552bd26"
|
||||
patches = "moge"
|
||||
|
||||
[artifacts.utils3d]
|
||||
revision = "3fab839f0be9931dac7c8488eb0e1600c236e183"
|
||||
filename = "utils3d.tar.gz"
|
||||
url = "https://codeload.github.com/EasternJournalist/utils3d/tar.gz/3fab839f0be9931dac7c8488eb0e1600c236e183"
|
||||
sha256 = "f1f167b1f68809bea391a66dd0d2ad0e14f7382cbddf087b655181c31a285abb"
|
||||
patches = "utils3d"
|
||||
|
||||
[uv]
|
||||
version = "0.11.3"
|
||||
|
||||
[platforms.windows-x64]
|
||||
filename = "uv-x86_64-pc-windows-msvc.zip"
|
||||
url = "https://github.com/astral-sh/uv/releases/download/0.11.3/uv-x86_64-pc-windows-msvc.zip"
|
||||
sha256 = "ae681c0aaec7cc96af184648cb88d73f8393ed60fa5880abdd6bdb910f9b227c"
|
||||
|
||||
[platforms.macos-arm64]
|
||||
filename = "uv-aarch64-apple-darwin.tar.gz"
|
||||
url = "https://github.com/astral-sh/uv/releases/download/0.11.3/uv-aarch64-apple-darwin.tar.gz"
|
||||
sha256 = "2bc3d0c7bf2bd08325b1e170abac6f7e5b3346e1d4eab3370d17cefec934996f"
|
||||
|
||||
[platforms.linux-x64]
|
||||
filename = "uv-x86_64-unknown-linux-gnu.tar.gz"
|
||||
url = "https://github.com/astral-sh/uv/releases/download/0.11.3/uv-x86_64-unknown-linux-gnu.tar.gz"
|
||||
sha256 = "c0f3236f146e55472663cfbcc9be3042a9f1092275bbe3fe2a56a6cbfd3da5ce"
|
||||
Reference in New Issue
Block a user