feat(cutout_tool): Implement cutout region to shape functionality

- Added interaction helpers for cutout region validation and shape menu interaction.
- Created main operator for converting image regions into cutout shapes, including AI artifact generation.
- Introduced object creation logic for cutout shapes, handling depth and normal textures.
- Defined cutout shape identifiers and node groups for various shape types.
- Implemented depth texture writing and prediction artifact generation for MoGe-2.
- Developed server jobs for generating cutout artifacts, depth planes, and refining image regions.
- Added region refinement logic to enhance gesture masks using BEN2.
This commit is contained in:
2026-08-31 11:36:27 +08:00
parent 08309f719d
commit 87d7721128
80 changed files with 4823 additions and 4297 deletions

View File

@@ -0,0 +1,156 @@
---
name: openspec-apply-change
description: Implement tasks from an OpenSpec change. Use when the user wants to start implementing, continue implementation, or work through tasks.
license: MIT
compatibility: Requires openspec CLI.
metadata:
author: openspec
version: "1.0"
generatedBy: "1.3.1"
---
Implement tasks from an OpenSpec change.
**Input**: Optionally specify a change name. If omitted, check if it can be inferred from conversation context. If vague or ambiguous you MUST prompt for available changes.
**Steps**
1. **Select the change**
If a name is provided, use it. Otherwise:
- Infer from conversation context if the user mentioned a change
- Auto-select if only one active change exists
- If ambiguous, run `openspec list --json` to get available changes and use the **AskUserQuestion tool** to let the user select
Always announce: "Using change: <name>" and how to override (e.g., `/opsx:apply <other>`).
2. **Check status to understand the schema**
```bash
openspec status --change "<name>" --json
```
Parse the JSON to understand:
- `schemaName`: The workflow being used (e.g., "spec-driven")
- Which artifact contains the tasks (typically "tasks" for spec-driven, check status for others)
3. **Get apply instructions**
```bash
openspec instructions apply --change "<name>" --json
```
This returns:
- `contextFiles`: artifact ID -> array of concrete file paths (varies by schema - could be proposal/specs/design/tasks or spec/tests/implementation/docs)
- Progress (total, complete, remaining)
- Task list with status
- Dynamic instruction based on current state
**Handle states:**
- If `state: "blocked"` (missing artifacts): show message, suggest using openspec-continue-change
- If `state: "all_done"`: congratulate, suggest archive
- Otherwise: proceed to implementation
4. **Read context files**
Read every file path listed under `contextFiles` from the apply instructions output.
The files depend on the schema being used:
- **spec-driven**: proposal, specs, design, tasks
- Other schemas: follow the contextFiles from CLI output
5. **Show current progress**
Display:
- Schema being used
- Progress: "N/M tasks complete"
- Remaining tasks overview
- Dynamic instruction from CLI
6. **Implement tasks (loop until done or blocked)**
For each pending task:
- Show which task is being worked on
- Make the code changes required
- Keep changes minimal and focused
- Mark task complete in the tasks file: `- [ ]` → `- [x]`
- Continue to next task
**Pause if:**
- Task is unclear → ask for clarification
- Implementation reveals a design issue → suggest updating artifacts
- Error or blocker encountered → report and wait for guidance
- User interrupts
7. **On completion or pause, show status**
Display:
- Tasks completed this session
- Overall progress: "N/M tasks complete"
- If all done: suggest archive
- If paused: explain why and wait for guidance
**Output During Implementation**
```
## Implementing: <change-name> (schema: <schema-name>)
Working on task 3/7: <task description>
[...implementation happening...]
✓ Task complete
Working on task 4/7: <task description>
[...implementation happening...]
✓ Task complete
```
**Output On Completion**
```
## Implementation Complete
**Change:** <change-name>
**Schema:** <schema-name>
**Progress:** 7/7 tasks complete ✓
### Completed This Session
- [x] Task 1
- [x] Task 2
...
All tasks complete! Ready to archive this change.
```
**Output On Pause (Issue Encountered)**
```
## Implementation Paused
**Change:** <change-name>
**Schema:** <schema-name>
**Progress:** 4/7 tasks complete
### Issue Encountered
<description of the issue>
**Options:**
1. <option 1>
2. <option 2>
3. Other approach
What would you like to do?
```
**Guardrails**
- Keep going through tasks until done or blocked
- Always read context files before starting (from the apply instructions output)
- If task is ambiguous, pause and ask before implementing
- If implementation reveals issues, pause and suggest artifact updates
- Keep code changes minimal and scoped to each task
- Update task checkbox immediately after completing each task
- Pause on errors, blockers, or unclear requirements - don't guess
- Use contextFiles from CLI output, don't assume specific file names
**Fluid Workflow Integration**
This skill supports the "actions on a change" model:
- **Can be invoked anytime**: Before all artifacts are done (if tasks exist), after partial implementation, interleaved with other actions
- **Allows artifact updates**: If implementation reveals design issues, suggest updating artifacts - not phase-locked, work fluidly

View File

@@ -0,0 +1,114 @@
---
name: openspec-archive-change
description: Archive a completed change in the experimental workflow. Use when the user wants to finalize and archive a change after implementation is complete.
license: MIT
compatibility: Requires openspec CLI.
metadata:
author: openspec
version: "1.0"
generatedBy: "1.3.1"
---
Archive a completed change in the experimental workflow.
**Input**: Optionally specify a change name. If omitted, check if it can be inferred from conversation context. If vague or ambiguous you MUST prompt for available changes.
**Steps**
1. **If no change name provided, prompt for selection**
Run `openspec list --json` to get available changes. Use the **AskUserQuestion tool** to let the user select.
Show only active changes (not already archived).
Include the schema used for each change if available.
**IMPORTANT**: Do NOT guess or auto-select a change. Always let the user choose.
2. **Check artifact completion status**
Run `openspec status --change "<name>" --json` to check artifact completion.
Parse the JSON to understand:
- `schemaName`: The workflow being used
- `artifacts`: List of artifacts with their status (`done` or other)
**If any artifacts are not `done`:**
- Display warning listing incomplete artifacts
- Use **AskUserQuestion tool** to confirm user wants to proceed
- Proceed if user confirms
3. **Check task completion status**
Read the tasks file (typically `tasks.md`) to check for incomplete tasks.
Count tasks marked with `- [ ]` (incomplete) vs `- [x]` (complete).
**If incomplete tasks found:**
- Display warning showing count of incomplete tasks
- Use **AskUserQuestion tool** to confirm user wants to proceed
- Proceed if user confirms
**If no tasks file exists:** Proceed without task-related warning.
4. **Assess delta spec sync state**
Check for delta specs at `openspec/changes/<name>/specs/`. If none exist, proceed without sync prompt.
**If delta specs exist:**
- Compare each delta spec with its corresponding main spec at `openspec/specs/<capability>/spec.md`
- Determine what changes would be applied (adds, modifications, removals, renames)
- Show a combined summary before prompting
**Prompt options:**
- If changes needed: "Sync now (recommended)", "Archive without syncing"
- If already synced: "Archive now", "Sync anyway", "Cancel"
If user chooses sync, use Task tool (subagent_type: "general-purpose", prompt: "Use Skill tool to invoke openspec-sync-specs for change '<name>'. Delta spec analysis: <include the analyzed delta spec summary>"). Proceed to archive regardless of choice.
5. **Perform the archive**
Create the archive directory if it doesn't exist:
```bash
mkdir -p openspec/changes/archive
```
Generate target name using current date: `YYYY-MM-DD-<change-name>`
**Check if target already exists:**
- If yes: Fail with error, suggest renaming existing archive or using different date
- If no: Move the change directory to archive
```bash
mv openspec/changes/<name> openspec/changes/archive/YYYY-MM-DD-<name>
```
6. **Display summary**
Show archive completion summary including:
- Change name
- Schema that was used
- Archive location
- Whether specs were synced (if applicable)
- Note about any warnings (incomplete artifacts/tasks)
**Output On Success**
```
## Archive Complete
**Change:** <change-name>
**Schema:** <schema-name>
**Archived to:** openspec/changes/archive/YYYY-MM-DD-<name>/
**Specs:** ✓ Synced to main specs (or "No delta specs" or "Sync skipped")
All artifacts complete. All tasks complete.
```
**Guardrails**
- Always prompt for change selection if not provided
- Use artifact graph (openspec status --json) for completion checking
- Don't block archive on warnings - just inform and confirm
- Preserve .openspec.yaml when moving to archive (it moves with the directory)
- Show clear summary of what happened
- If sync is requested, use openspec-sync-specs approach (agent-driven)
- If delta specs exist, always run the sync assessment and show the combined summary before prompting

View File

@@ -0,0 +1,288 @@
---
name: openspec-explore
description: Enter explore mode - a thinking partner for exploring ideas, investigating problems, and clarifying requirements. Use when the user wants to think through something before or during a change.
license: MIT
compatibility: Requires openspec CLI.
metadata:
author: openspec
version: "1.0"
generatedBy: "1.3.1"
---
Enter explore mode. Think deeply. Visualize freely. Follow the conversation wherever it goes.
**IMPORTANT: Explore mode is for thinking, not implementing.** You may read files, search code, and investigate the codebase, but you must NEVER write code or implement features. If the user asks you to implement something, remind them to exit explore mode first and create a change proposal. You MAY create OpenSpec artifacts (proposals, designs, specs) if the user asks—that's capturing thinking, not implementing.
**This is a stance, not a workflow.** There are no fixed steps, no required sequence, no mandatory outputs. You're a thinking partner helping the user explore.
---
## The Stance
- **Curious, not prescriptive** - Ask questions that emerge naturally, don't follow a script
- **Open threads, not interrogations** - Surface multiple interesting directions and let the user follow what resonates. Don't funnel them through a single path of questions.
- **Visual** - Use ASCII diagrams liberally when they'd help clarify thinking
- **Adaptive** - Follow interesting threads, pivot when new information emerges
- **Patient** - Don't rush to conclusions, let the shape of the problem emerge
- **Grounded** - Explore the actual codebase when relevant, don't just theorize
---
## What You Might Do
Depending on what the user brings, you might:
**Explore the problem space**
- Ask clarifying questions that emerge from what they said
- Challenge assumptions
- Reframe the problem
- Find analogies
**Investigate the codebase**
- Map existing architecture relevant to the discussion
- Find integration points
- Identify patterns already in use
- Surface hidden complexity
**Compare options**
- Brainstorm multiple approaches
- Build comparison tables
- Sketch tradeoffs
- Recommend a path (if asked)
**Visualize**
```
┌─────────────────────────────────────────┐
│ Use ASCII diagrams liberally │
├─────────────────────────────────────────┤
│ │
│ ┌────────┐ ┌────────┐ │
│ │ State │────────▶│ State │ │
│ │ A │ │ B │ │
│ └────────┘ └────────┘ │
│ │
│ System diagrams, state machines, │
│ data flows, architecture sketches, │
│ dependency graphs, comparison tables │
│ │
└─────────────────────────────────────────┘
```
**Surface risks and unknowns**
- Identify what could go wrong
- Find gaps in understanding
- Suggest spikes or investigations
---
## OpenSpec Awareness
You have full context of the OpenSpec system. Use it naturally, don't force it.
### Check for context
At the start, quickly check what exists:
```bash
openspec list --json
```
This tells you:
- If there are active changes
- Their names, schemas, and status
- What the user might be working on
### When no change exists
Think freely. When insights crystallize, you might offer:
- "This feels solid enough to start a change. Want me to create a proposal?"
- Or keep exploring - no pressure to formalize
### When a change exists
If the user mentions a change or you detect one is relevant:
1. **Read existing artifacts for context**
- `openspec/changes/<name>/proposal.md`
- `openspec/changes/<name>/design.md`
- `openspec/changes/<name>/tasks.md`
- etc.
2. **Reference them naturally in conversation**
- "Your design mentions using Redis, but we just realized SQLite fits better..."
- "The proposal scopes this to premium users, but we're now thinking everyone..."
3. **Offer to capture when decisions are made**
| Insight Type | Where to Capture |
|----------------------------|--------------------------------|
| New requirement discovered | `specs/<capability>/spec.md` |
| Requirement changed | `specs/<capability>/spec.md` |
| Design decision made | `design.md` |
| Scope changed | `proposal.md` |
| New work identified | `tasks.md` |
| Assumption invalidated | Relevant artifact |
Example offers:
- "That's a design decision. Capture it in design.md?"
- "This is a new requirement. Add it to specs?"
- "This changes scope. Update the proposal?"
4. **The user decides** - Offer and move on. Don't pressure. Don't auto-capture.
---
## What You Don't Have To Do
- Follow a script
- Ask the same questions every time
- Produce a specific artifact
- Reach a conclusion
- Stay on topic if a tangent is valuable
- Be brief (this is thinking time)
---
## Handling Different Entry Points
**User brings a vague idea:**
```
User: I'm thinking about adding real-time collaboration
You: Real-time collab is a big space. Let me think about this...
COLLABORATION SPECTRUM
════════════════════════════════════════════
Awareness Coordination Sync
│ │ │
▼ ▼ ▼
┌────────┐ ┌────────┐ ┌────────┐
│Presence│ │Cursors │ │ CRDT │
│ "3 │ │ Multi │ │Conflict│
│online" │ │ select │ │ free │
└────────┘ └────────┘ └────────┘
│ │ │
trivial moderate complex
Where's your head at?
```
**User brings a specific problem:**
```
User: The auth system is a mess
You: [reads codebase]
┌─────────────────────────────────────────────┐
│ CURRENT AUTH FLOW │
└─────────────────────────────────────────────┘
│
┌─────────────┼─────────────┐
▼ ▼ ▼
┌─────────┐ ┌─────────┐ ┌─────────┐
│ Google │ │ GitHub │ │ Email │
│ OAuth │ │ OAuth │ │ Magic │
└────┬────┘ └────┬────┘ └────┬────┘
│ │ │
└─────────────┼─────────────┘
▼
┌───────────┐
│ Session │
└─────┬─────┘
│
▼
┌───────────┐
│ Perms │
└───────────┘
I see three tangles. Which one's burning?
```
**User is stuck mid-implementation:**
```
User: /opsx:explore add-auth-system
The OAuth integration is more complex than expected
You: [reads change artifacts]
You're on task 4: "Implement OAuth flow"
Let me trace what's involved...
[draws diagram, explores options, suggests paths]
Want to update the design to reflect this?
Or add a spike task to investigate?
```
**User wants to compare options:**
```
User: Should we use Postgres or SQLite?
You: Generic answer is boring. What's the context?
User: A CLI tool that tracks local dev environments
You: That changes everything.
┌─────────────────────────────────────────────────┐
│ CLI TOOL DATA STORAGE │
└─────────────────────────────────────────────────┘
Key constraints:
• No daemon running
• Must work offline
• Single user
SQLite Postgres
Deployment embedded ✓ needs server ✗
Offline yes ✓ no ✗
Single file yes ✓ no ✗
SQLite. Not even close.
Unless... is there a sync component?
```
---
## Ending Discovery
There's no required ending. Discovery might:
- **Flow into a proposal**: "Ready to start? I can create a change proposal."
- **Result in artifact updates**: "Updated design.md with these decisions"
- **Just provide clarity**: User has what they need, moves on
- **Continue later**: "We can pick this up anytime"
When it feels like things are crystallizing, you might summarize:
```
## What We Figured Out
**The problem**: [crystallized understanding]
**The approach**: [if one emerged]
**Open questions**: [if any remain]
**Next steps** (if ready):
- Create a change proposal
- Keep exploring: just keep talking
```
But this summary is optional. Sometimes the thinking IS the value.
---
## Guardrails
- **Don't implement** - Never write code or implement features. Creating OpenSpec artifacts is fine, writing application code is not.
- **Don't fake understanding** - If something is unclear, dig deeper
- **Don't rush** - Discovery is thinking time, not task time
- **Don't force structure** - Let patterns emerge naturally
- **Don't auto-capture** - Offer to save insights, don't just do it
- **Do visualize** - A good diagram is worth many paragraphs
- **Do explore the codebase** - Ground discussions in reality
- **Do question assumptions** - Including the user's and your own

View File

@@ -0,0 +1,110 @@
---
name: openspec-propose
description: Propose a new change with all artifacts generated in one step. Use when the user wants to quickly describe what they want to build and get a complete proposal with design, specs, and tasks ready for implementation.
license: MIT
compatibility: Requires openspec CLI.
metadata:
author: openspec
version: "1.0"
generatedBy: "1.3.1"
---
Propose a new change - create the change and generate all artifacts in one step.
I'll create a change with artifacts:
- proposal.md (what & why)
- design.md (how)
- tasks.md (implementation steps)
When ready to implement, run /opsx:apply
---
**Input**: The user's request should include a change name (kebab-case) OR a description of what they want to build.
**Steps**
1. **If no clear input provided, ask what they want to build**
Use the **AskUserQuestion tool** (open-ended, no preset options) to ask:
> "What change do you want to work on? Describe what you want to build or fix."
From their description, derive a kebab-case name (e.g., "add user authentication" → `add-user-auth`).
**IMPORTANT**: Do NOT proceed without understanding what the user wants to build.
2. **Create the change directory**
```bash
openspec new change "<name>"
```
This creates a scaffolded change at `openspec/changes/<name>/` with `.openspec.yaml`.
3. **Get the artifact build order**
```bash
openspec status --change "<name>" --json
```
Parse the JSON to get:
- `applyRequires`: array of artifact IDs needed before implementation (e.g., `["tasks"]`)
- `artifacts`: list of all artifacts with their status and dependencies
4. **Create artifacts in sequence until apply-ready**
Use the **TodoWrite tool** to track progress through the artifacts.
Loop through artifacts in dependency order (artifacts with no pending dependencies first):
a. **For each artifact that is `ready` (dependencies satisfied)**:
- Get instructions:
```bash
openspec instructions <artifact-id> --change "<name>" --json
```
- The instructions JSON includes:
- `context`: Project background (constraints for you - do NOT include in output)
- `rules`: Artifact-specific rules (constraints for you - do NOT include in output)
- `template`: The structure to use for your output file
- `instruction`: Schema-specific guidance for this artifact type
- `outputPath`: Where to write the artifact
- `dependencies`: Completed artifacts to read for context
- Read any completed dependency files for context
- Create the artifact file using `template` as the structure
- Apply `context` and `rules` as constraints - but do NOT copy them into the file
- Show brief progress: "Created <artifact-id>"
b. **Continue until all `applyRequires` artifacts are complete**
- After creating each artifact, re-run `openspec status --change "<name>" --json`
- Check if every artifact ID in `applyRequires` has `status: "done"` in the artifacts array
- Stop when all `applyRequires` artifacts are done
c. **If an artifact requires user input** (unclear context):
- Use **AskUserQuestion tool** to clarify
- Then continue with creation
5. **Show final status**
```bash
openspec status --change "<name>"
```
**Output**
After completing all artifacts, summarize:
- Change name and location
- List of artifacts created with brief descriptions
- What's ready: "All artifacts created! Ready for implementation."
- Prompt: "Run `/opsx:apply` or ask me to implement to start working on the tasks."
**Artifact Creation Guidelines**
- Follow the `instruction` field from `openspec instructions` for each artifact type
- The schema defines what each artifact should contain - follow it
- Read dependency artifacts for context before creating new ones
- Use `template` as the structure for your output file - fill in its sections
- **IMPORTANT**: `context` and `rules` are constraints for YOU, not content for the file
- Do NOT copy `<context>`, `<rules>`, `<project_context>` blocks into the artifact
- These guide what you write, but should never appear in the output
**Guardrails**
- Create ALL artifacts needed for implementation (as defined by schema's `apply.requires`)
- Always read dependency artifacts before creating a new one
- If context is critically unclear, ask the user - but prefer making reasonable decisions to keep momentum
- If a change with that name already exists, ask if user wants to continue it or create a new one
- Verify each artifact file exists after writing before proceeding to next

View File

@@ -54,9 +54,9 @@ uv lock --refresh
## 命名规范 ## 命名规范
- 变量、函数、Docstring 使用英文;同一概念在代码和文档中使用统一表达 - 变量、函数、Docstring 使用英文;同一概念在代码和文档中使用统一表达
- 功能代码的动作前缀应表达产物边界:`generate_*` 表示 AI 生成,`build_*` 只在内存中构建值或几何,`create_*` 表示创建 Blender Image、Mesh、Material、Object 等真实数据 - 函数的动作前缀应表达产物边界:`generate_*` 表示 AI 生成,`build_*` 只在内存中构建值或几何,`create_*` 表示创建 Blender Image、Mesh、Material、Object 等真实数据
- 名称应说明业务职责,并与用户概念、公开协议和界面术语保持一致;不要使用脱离上下文后无法判断用途的宽泛名称 - 名称应说明业务职责,并与用户概念、公开协议和界面术语保持一致;不要使用脱离上下文后无法判断用途的宽泛名称
- 文件和目录名应反映职责层级:集合层使用复数,单个功能或动作使用单数或动宾结构;同一层级保持一致 - 文件和目录名应表达稳定的业务分类和模块职责,不直接沿用函数名,也不按临时执行步骤命名;集合层使用复数,单项分类使用单数,同一层级保持一致
- Python 类使用大驼峰,不添加项目名前缀。Operator 使用动宾结构,如 `ConvertToDepthPlane`、`RemoveImageBackground`;Menu、Panel、AddonPreferences、PropertyGroup 分别使用实际类型后缀,如 `AnyImageImageMenu`、`ServerPanel`、`AnyImagePreferences`、`AnyImageSettings` - Python 类使用大驼峰,不添加项目名前缀。Operator 使用动宾结构,如 `ConvertToDepthPlane`、`RemoveImageBackground`;Menu、Panel、AddonPreferences、PropertyGroup 分别使用实际类型后缀,如 `AnyImageImageMenu`、`ServerPanel`、`AnyImagePreferences`、`AnyImageSettings`
- `bl_idname` 与挂载到公共 Blender RNA 类型的 Property 使用 `anyimage` 项目前缀:Operator 如 `anyimage.convert_to_depth_plane`,Menu 如 `ANYIMAGE_MT_image_context`,Panel 如 `ANYIMAGE_PT_server`,公共 Property 如 `Scene.anyimage_settings` - `bl_idname` 与挂载到公共 Blender RNA 类型的 Property 使用 `anyimage` 项目前缀:Operator 如 `anyimage.convert_to_depth_plane`,Menu 如 `ANYIMAGE_MT_image_context`,Panel 如 `ANYIMAGE_PT_server`,公共 Property 如 `Scene.anyimage_settings`
- Operator 自身 Property 不加项目前缀,使用业务名称,如 `input_path`、`process_res`、`resolution_level` - Operator 自身 Property 不加项目前缀,使用业务名称,如 `input_path`、`process_res`、`resolution_level`

View File

@@ -8,15 +8,15 @@ AnyImage 是一款 Blender Extension,用于把图片带入可编辑的 2.5D
- 在 3D View、Node Editor 与绘画模式中用 `Ctrl+V`(macOS 为 `Cmd+V`)粘贴并 Pack 剪贴板图片 - 在 3D View、Node Editor 与绘画模式中用 `Ctrl+V`(macOS 为 `Cmd+V`)粘贴并 Pack 剪贴板图片
- 将 Image Empty 转换为带 `O Mesh Plane` Geometry Nodes 的普通 Plane - 将 Image Empty 转换为带 `O Mesh Plane` Geometry Nodes 的普通 Plane
- 使用 MoGe-2 或 DA3 将静态图转换为可调 Depth Plane - 使用 MoGe-2 将静态图转换为可调 Depth Plane
- 用 Cutout 套索在图片可见 Alpha 内生成 Surface、Balloon、Depth Balloon 或 Depth Surface,并通过统一的 `O Image Cutout` 修改器切换 Shape、按创建分辨率清理碎岛、根据透明选区校准浮雕基准和调整表面 - 用 Cutout 套索在图片可见 Alpha 内生成 Surface、Balloon、Depth Balloon 或 Depth Surface,并通过统一的 `O Image Cutout` 修改器切换 Shape、按创建分辨率清理碎岛、根据 Region Alpha 校准浮雕基准和调整表面
- 用 Image Box、Lasso、Line、Polyline 和 Perspective 清理或矫正静态图片 - 用 Image Box、Lasso、Line、Polyline 和 Perspective 清理或矫正静态图片
- 用 BEN2 为单图、视频或序列帧去除背景 - 用 BEN2 为单图、视频或序列帧去除背景
- 用三档 ONNX 模型将单图、视频或序列帧放大 4×,并独立重采样 Alpha - 用三档 ONNX 模型将单图、视频或序列帧放大 4×,并独立重采样 Alpha
AI 推理运行在 Blender 之外的独立 Python 3.12 Environment 中。Blender 进程只负责 UI、任务提交和结果导入,不加载 ONNX Runtime 或模型权重。 AI 推理运行在 Blender 之外的独立 Python 3.12 Environment 中。Blender 进程只负责 UI、任务提交和结果导入,不加载 ONNX Runtime 或模型权重。
全局 **Maximum AI Input Size** 默认设为 2048 px。当前图片长边达到该值时 Upscale 不可用;Depth 与 Normal 的分析输入超过该值时等比缩小,但不会降低颜色贴图分辨率。Cutout 在套索裁切后应用该限制,并以同一分析尺度计算网格密度。 **Maximum AI Input Size** 默认设为 2048 px,只限制 Upscale 可执行图片长边 以及 Depth、Normal Map、Fit to Foreground 的 AI 分析输入,不降低颜色贴图分辨率。Cutout 网格密度由默认 `1024` 的独立 **Maximum Cutout Mesh Resolution** 限制,因此本地与 AI Shape 使用同一套明确的网格上限。
## 安装 ## 安装
@@ -28,7 +28,7 @@ AI 推理运行在 Blender 之外的独立 Python 3.12 Environment 中。Blender
3. 选中 Image Empty,在右键 **AnyImage** 菜单中点击 **Set Up AI Environment…**。 3. 选中 Image Empty,在右键 **AnyImage** 菜单中点击 **Set Up AI Environment…**。
4. 安装完成后,在扩展 Preferences 中确认默认 MoGe-2、BEN2 与 Real-ESRGAN WDN 模型显示 Ready。 4. 安装完成后,在扩展 Preferences 中确认默认 MoGe-2、BEN2 与 Real-ESRGAN WDN 模型显示 Ready。
普通 Plane、基础 Image 编辑,以及不带 Refine / Normal 的 Cutout Surface 与 Balloon 无需 AI Environment。Cutout 的 Depth 与 Normal 统一使用 MoGe-2;两者同时启用时共享同一次 Prediction,不会重复运行模型。Cutout Normal 可选择 None、基于完整图片平面的 Tangent Space 浮雕,或直接编码到对象局部轴的 Object Space。 普通 Plane、基础 Image 编辑,以及未启用 **Fit to Foreground** / **Generate Normal Map** 的 Cutout Surface 与 Balloon 无需 AI Environment。Cutout 的 Depth 与 Normal Map 统一使用 MoGe-2;两者同时启用时共享同一次 Prediction,不会重复运行模型。Cutout 的 **Generate Normal Map** 是启用开关;开启后,Depth Surface 自动使用 Object Space,其余 Shape 自动使用 Tangent Space。
## 快速使用 ## 快速使用
@@ -46,13 +46,13 @@ AI 推理运行在 Blender 之外的独立 Python 3.12 Environment 中。Blender
```text ```text
src/anyimage/ src/anyimage/
├── operators/ # Blender 交互、任务提交与结果导入 ├── operators/ # Blender 交互、任务提交与结果导入
├── server/ # 无 Blender 依赖的 Job、媒体、选区、几何与模型能力 ├── server/ # 无 Blender 依赖的 Job、媒体、Region、几何与模型能力
├── assets/ # Geometry Nodes 与 Shader 资产 ├── assets/ # Geometry Nodes 与 Shader 资产
├── runtime.py # BlendJob Runtime 配置 ├── runtime.py # BlendJob Runtime 配置
└── __init__.py # Blender 注册入口 └── __init__.py # Blender 注册入口
``` ```
程序边界、功能文件结构和调用流程见[内部实现总览](docs/internals/index.md);开发新功能前先查看[Common 公共函数](docs/internals/common.md),复用已经存在的 Image、ImageGeometry、Material 和对象替换入口。文档按开始使用、功能指南、参考和内部实现分层,不在 README 重复内部算法细节。 程序边界、功能文件结构和调用流程见[内部实现总览](docs/internals/index.md);开发新功能前先查看[Common 公共函数](docs/internals/common.md),复用已经存在的 Image、Depth 产物、Material 和对象替换入口。文档按开始使用、功能指南、参考和内部实现分层,不在 README 重复内部算法细节。
## 开发 ## 开发

View File

@@ -25,9 +25,10 @@ AnyImage 的扩展偏好设置按功能分组。Environment 区包含:
Environment 检测后显示 Diagnostics。Depth & Normal、Remove Background、Upscale 与 Cutout 是默认展开的可折叠功能区,各自包含对应设置和模型状态。其中: Environment 检测后显示 Diagnostics。Depth & Normal、Remove Background、Upscale 与 Cutout 是默认展开的可折叠功能区,各自包含对应设置和模型状态。其中:
- **Default Upscale Model**:Upscale 使用的模型,默认是 General WDN x4v3。 - **Default Upscale Model**:Upscale 使用的模型,默认是 General WDN x4v3。
- **Maximum AI Input Size**:Upscale、Depth 与 Normal 分析输入的长边上限,默认 2048 px;只缩放分析输入,不降低颜色贴图分辨率。 - **Maximum AI Input Size**:Upscale、Depth、Normal Map 与 Fit to Foreground 的 AI 输入长边上限,默认 2048 px;只缩放分析输入,不降低颜色贴图分辨率。
- **Maximum Cutout Mesh Resolution**:所有 Cutout Shape 的网格采样长边上限,默认 1024,独立于 AI 输入尺寸。
Depth Plane 的 Depth Model 依次列出三档 MoGe-2 与两档 DA3;选择 DA3 时会提示该模型不能生成 normal。Cutout 固定使用 MoGe-2:全局选择 MoGe-2 时沿用对应档位,选择 DA3 时回退默认 MoGe-2。可选模型的 **Ready** 状态或 **Download** 按钮直接位于模型选择项下方;BEN2 等固定模型则显示在所属功能区。Cutout 区只保留 Balloon Profile。 Depth & Normal 只列出三档 MoGe-2,Depth Plane 与 Cutout Depth / Normal Map 使用同一个显式选择,不存在生产回退。可选模型的 **Ready** 状态或 **Download** 按钮位于选择项下方;Cutout 区提供独立网格上限与 Balloon Profile。
如果准备把 Storage Root 放到其他磁盘,建议在首次安装 Environment 之前修改。更换目录后,旧目录中的环境和模型不会自动移动。 如果准备把 Storage Root 放到其他磁盘,建议在首次安装 Environment 之前修改。更换目录后,旧目录中的环境和模型不会自动移动。
@@ -54,7 +55,7 @@ Depth Plane 的 Depth Model 依次列出三档 MoGe-2 与两档 DA3;选择 DA3
- AnyImage 扩展偏好设置中的 MoGe-2 ViT-S Normal、BEN2 Base 和 Real-ESRGAN General WDN x4v3 显示 **Ready**。 - AnyImage 扩展偏好设置中的 MoGe-2 ViT-S Normal、BEN2 Base 和 Real-ESRGAN General WDN x4v3 显示 **Ready**。
- 选中 Image Empty 后,右键菜单底部显示 **AI Environment Settings…**。 - 选中 Image Empty 后,右键菜单底部显示 **AI Environment Settings…**。
**Convert to Depth Plane**、**Remove Background** 和 **Upscale** 在 AI 未就绪时仍可点击,点击后会根据状态转入 **Set Up AI Environment…** 或 **Download Required Models…**。普通 Plane、Image 编辑,以及未启用 Refine / Normal 的 Cutout Surface 与 Balloon 不要求安装 AI Environment。 **Convert to Depth Plane**、**Remove Background** 和 **Upscale** 在 AI 未就绪时仍可点击,点击后会根据状态转入 **Set Up AI Environment…** 或 **Download Required Models…**。普通 Plane、Image 编辑,以及未启用 **Fit to Foreground** / **Generate Normal Map** 的 Cutout Surface 与 Balloon 不要求安装 AI Environment。
## 下一步 ## 下一步

View File

@@ -21,36 +21,38 @@ Cutout 用套索从一张 Image Empty 中选择区域,再生成 Surface、Ball
四种结果都保留源 Image Empty,并在同一位置创建新网格。它们共享选区轮廓、UV、材质和基础三角网格,因此可以从同一张图多次生成不同 Shape 进行比较。 四种结果都保留源 Image Empty,并在同一位置创建新网格。它们共享选区轮廓、UV、材质和基础三角网格,因此可以从同一张图多次生成不同 Shape 进行比较。
无论是否启用 Refine Selection,套索超出图片的部分都不会生成网格,源图中 `Alpha <= 0.1` 的像素也始终从 Region 中排除。双击使用完整画面时同样只为 `Alpha > 0.1` 的 Region 生成网格。 无论是否启用 Fit to Foreground,套索超出图片的部分都不会生成网格,源图中 `Alpha <= 0.1` 的像素也始终从 Region 中排除。双击使用完整画面时同样只为 `Alpha > 0.1` 的 Region 生成网格。
## 工具设置 ## 工具设置
- **Point Spacing**:相邻网格点大约对应多少源图像素。数值越小,网格越密,轮廓和变形细节越多。 选项按执行意图排列:
- **Normal**:使用 MoGe-2 生成法线贴图。默认关闭。
- **Refine Selection**:用 BEN2 把几何套索收紧到其中的前景内容。默认关闭。
Surface 与 Balloon 在 Normal、Refine 都关闭时完全在 Blender 中生成,不需要 AI Environment。开启任一 AI 选项,或选择 Depth Shape 时,会提交后台任务。 - **Fit to Foreground**:用 BEN2 把几何套索贴合到其中的前景主体,默认关闭。
- **Mesh Sampling Spacing**:相邻网格采样点大约对应多少源图像素;数值越小,三角网格越密。
- **Generate Normal Map**:是否使用 MoGe-2 生成法线贴图,默认关闭;空间由 Shape 自动决定。
Point Spacing 先按完整源图换算基础密度;选区裁切后的长边若超过全局 **Maximum AI Input Size**,网格密度会按分析图的等比缩放比例降低。颜色贴图始终保留原裁切分辨率,不会使用限制后的临时分析图。建议先用默认 `32` 观察形状;轮廓需要更细时再逐步减小。 Surface 与 Balloon 在 **Generate Normal Map**、**Fit to Foreground** 都关闭时完全在 Blender 中生成,不需要 AI Environment。开启任一 AI 选项,或选择 Depth Shape 时,会提交后台任务。
Mesh Sampling Spacing 先按完整源图换算期望密度,再由 Cutout 区的 **Maximum Cutout Mesh Resolution** 限制裁切区域的最长采样边。这个网格上限对 Surface、Balloon 与 Depth Shape 完全相同。**Maximum AI Input Size** 只限制 Fit to Foreground、Depth 和 Normal Map 的模型分析图;颜色贴图始终保留原裁切分辨率。建议先用默认 `32` 观察形状,轮廓需要更细时再逐步减小。
## 调整修改器 ## 调整修改器
四个 Shape 都使用同一个 `O Image Cutout` Geometry Nodes 修改器。创建时 Shape 菜单已经设为饼菜单所选类型;切换 Shape 后,Blender 只显示该类型实际使用的输入。常用参数包括: 四个 Shape 都使用同一个 `O Image Cutout` Geometry Nodes 修改器。创建时 Shape 菜单已经设为饼菜单所选类型;切换 Shape 后,Blender 只显示该类型实际使用的输入。常用参数包括:
- **Thickness**:Surface / Depth Surface 使用场景距离;Balloon / Depth Balloon 使用鼓起高度倍率。 - **Thickness**:Surface / Depth Surface 使用场景距离;Balloon / Depth Balloon 使用鼓起高度倍率。
- **Cleanup**:碎岛面积阈值;创建时根据 Point Spacing 和图片平面尺寸自动写入。为便于在修改器中调节,显示值放大了 `100` 倍,节点内部会还原为实际面积,因此这次显示缩放不会改变默认清理效果。 - **Cleanup**:碎岛面积阈值;创建时根据 Mesh Sampling Spacing 和图片平面尺寸自动写入。为便于在修改器中调节,显示值放大了 `100` 倍,节点内部会还原为实际面积,因此这次显示缩放不会改变默认清理效果。
- **Smooth Iterations**:位于外层,控制平滑次数,默认 `0`;**Smooth** 位于 Options 中,控制每次平滑的混合强度。四种 Shape 都可使用。 - **Smooth Iterations**:位于外层,控制平滑次数,默认 `0`;**Smooth** 位于 Options 中,控制每次平滑的混合强度。四种 Shape 都可使用。
- **Depth Scale**:Depth Shape 的深度强度;`0` 回到基础平面或 Balloon。 - **Depth Scale**:Depth Shape 的深度强度;`0` 回到基础平面或 Balloon。
两个 Depth Shape 共用 **Base Plane Depth**;Depth Balloon 还提供 **Depth Direction**。生成前会检查最终 Region 输入的 Alpha:`Alpha < 0.1` 的像素达到 `5%` 时,根据有效 Region 自适应 Base Plane Depth,并为 Depth Balloon 拟合 Direction;不足 `5%` 时使用稳定的固定基准。这个判断不依赖双击全图或 Refine Selection。插件自动写入的 Depth Image 与 Uniform Scale 位于折叠的 Data 面板,一般无需手动修改。 两个 Depth Shape 共用 **Base Plane Depth**;Depth Balloon 还提供 **Depth Direction**。生成前会检查最终 Region 输入的 Alpha:`Alpha < 0.1` 的像素达到 `5%` 时,根据有效 Region 自适应 Base Plane Depth,并为 Depth Balloon 拟合 Direction;不足 `5%` 时使用稳定的固定基准。这个判断不依赖双击全图或 Fit to Foreground。插件自动写入的 Depth Image 与 Uniform Scale 位于折叠的 Data 面板,一般无需手动修改。
Balloon 的轮廓高度算法由扩展 Preferences 中的 **Balloon Profile** 控制:Poisson 更平滑,Teddy 更强调轮廓骨架。修改默认值只影响之后创建的 Shape。 Balloon 的轮廓高度算法由扩展 Preferences 中的 **Balloon Profile** 控制:Poisson 更平滑,Teddy 更强调轮廓骨架。修改默认值只影响之后创建的 Shape。
## Refine 与 Normal ## Fit to Foreground 与 Normal Map
Refine Selection 只在“套索与源图可见 Alpha 的交集”内收紧最终 Region,不把 AI 生成的颜色写回原图。材质继续使用当前 Image Empty 的源颜色和原 Alpha。 Fit to Foreground 只在“套索与源图可见 Alpha 的交集”内收紧最终 Region,不把 AI 生成的颜色写回原图。材质使用按最终 Region 包围矩形裁切的当前 Image Empty 源图,保留源颜色和原 Alpha,不把 RegionMask 写入 Alpha。
Normal 设为 **None** 时使用几何法线;**Tangent** 把 MoGe-2 结果作为基于完整图片平面的浮雕细节,**Object** 则直接编码到 Cutout 对象局部轴。两种模式都不根据 Geometry Nodes 的成品网格再次烘焙。Normal 保持模型分析分辨率,Color Image 仍保留原始裁切分辨率;Depth 与 Normal 同时需要时共享同一次 Prediction,不会重复运行模型。 Normal Map 关闭时使用几何法线。开启后,Depth Surface 自动使用 Object Space,把模型法线编码到 Cutout 对象局部轴;Surface、Balloon 与 Depth Balloon 自动使用 Tangent Space,作为相对图片平面的浮雕细节。用户不直接选择 Normal Space,两种路径都不根据 Geometry Nodes 的成品网格再次烘焙。Normal Map 保持模型分析分辨率,Color Image 仍保留原始裁切分辨率;Depth 与 Normal Map 同时需要时共享同一次 Prediction,不会重复运行模型。
## 当前范围 ## 当前范围

View File

@@ -16,12 +16,6 @@ Depth Plane 将一张静态图片转换为带深度位移的网格,适合快
默认推荐。MoGe-2 估算尺度一致的深度与相机信息,使用 **MoGe Resolution Level** 控制处理精度。第一次使用建议保持 `5`。 默认推荐。MoGe-2 估算尺度一致的深度与相机信息,使用 **MoGe Resolution Level** 控制处理精度。第一次使用建议保持 `5`。
### DA3
DA3 提供另一种深度结果,使用 **DA3 Resolution Scale** 按原图长边计算处理尺寸。比例越高,通常会保留更多细节,也会增加耗时和内存占用。
切换模型后,如果对应文件尚未下载,先在同一设置区点击 **Download**。
## 调整结果 ## 调整结果
生成对象使用运行时创建的 `O Image Depth Plane` Geometry Nodes 修改器: 生成对象使用运行时创建的 `O Image Depth Plane` Geometry Nodes 修改器:

View File

@@ -39,7 +39,7 @@ Environment 或默认模型缺失不会让四个 AI 功能变灰;点击功能
## Cutout Depth Surface 边缘破碎或细节太多 ## Cutout Depth Surface 边缘破碎或细节太多
先增大 **Point Spacing** 降低基础网格密度,再调整 **Cleanup** 清理小块,并用 **Smooth** 稳定表面。必要时降低模型分辨率。 先增大 **Mesh Sampling Spacing** 降低基础网格密度,再调整 **Cleanup** 清理小块,并用 **Smooth** 稳定表面。必要时降低模型分辨率。
## 动画面片不播放 ## 动画面片不播放

View File

@@ -12,12 +12,12 @@ Image Box、Lasso、Line、Polyline 与 Perspective 直接编辑静态 Image Emp
| **Image Polyline** | 逐点构建精确多边形 | | **Image Polyline** | 逐点构建精确多边形 |
| **Image Perspective** | 将任意顺序输入的四个角矫正为矩形 | | **Image Perspective** | 将任意顺序输入的四个角矫正为矩形 |
在 3D Viewport 左侧工具栏长按 Image Mask 图标,可以在 Box、Lasso、Line 与 Polyline 之间切换。**Refine Selection** 只显示在 Box / Lasso;Line / Polyline 保持几何直线边界。Perspective 使用独立图标。 在 3D Viewport 左侧工具栏长按 Image Mask 图标,可以在 Box、Lasso、Line 与 Polyline 之间切换。**Fit to Foreground** 只显示在 Box / Lasso;Line / Polyline 保持几何直线边界。Perspective 使用独立图标。
<div class="screenshot-placeholder" markdown> <div class="screenshot-placeholder" markdown>
**截图占位:Image 工具组与 Tool Settings** **截图占位:Image 工具组与 Tool Settings**
显示包含四个工具的 Image Mask 组、独立 Perspective,以及 Box / Lasso 的 Refine Selection。 显示包含四个工具的 Image Mask 组、独立 Perspective,以及 Box / Lasso 的 Fit to Foreground。
目标文件:`images/image-tools/tool-group.png` 目标文件:`images/image-tools/tool-group.png`
</div> </div>
@@ -48,7 +48,7 @@ Image Box、Lasso、Line、Polyline 与 Perspective 直接编辑静态 Image Emp
- 关闭:保留选区内; - 关闭:保留选区内;
- 开启:保留选区外。 - 开启:保留选区外。
Cutout、Box 与 Lasso 分别保存各自的 **Refine Selection** 设置;在一个工具中切换不会改变另外两个工具。启用后,BEN2 会先把几何选区收紧到其中的前景,再应用反选。例如框住一个苹果并开启反选,最终会清理苹果本身、保留其余内容。Line / Polyline 不提供 Refine,以保持直线边界和明确的裁切方向。 Cutout、Box 与 Lasso 分别保存各自的 **Fit to Foreground** 设置;在一个工具中切换不会改变另外两个工具。启用后,BEN2 会先把几何 Region 收紧到其中的前景,再应用反选。例如框住一个苹果并开启反选,最终会清理苹果本身、保留其余内容。Line / Polyline 不提供 Fit to Foreground,以保持直线边界和明确的裁切方向。
结果会直接更新当前 Empty 的图片,并把画布紧裁切到最终可见 Alpha。只有一个 Image Empty 使用该图片时,工具更新原 Image;两个或更多 Image Empty 共用它时,工具为当前 Empty 创建独立结果,其他 Empty 保持不变。对象 Transform 不变,显示尺寸与 Image Offset 会更新,使剩余像素保持在原图片平面上的位置。透明区只修改 Alpha,保留源 RGB 供纹理过滤使用。未形成有效区域的单击会静默结束;有效选区没有可见像素时只显示警告,不修改图片。 结果会直接更新当前 Empty 的图片,并把画布紧裁切到最终可见 Alpha。只有一个 Image Empty 使用该图片时,工具更新原 Image;两个或更多 Image Empty 共用它时,工具为当前 Empty 创建独立结果,其他 Empty 保持不变。对象 Transform 不变,显示尺寸与 Image Offset 会更新,使剩余像素保持在原图片平面上的位置。透明区只修改 Alpha,保留源 RGB 供纹理过滤使用。未形成有效区域的单击会静默结束;有效选区没有可见像素时只显示警告,不修改图片。
@@ -69,7 +69,7 @@ Cutout、Box 与 Lasso 分别保存各自的 **Refine Selection** 设置;在
`Backspace` 返回上一步,右键或 `Esc` 取消。四点会自动排序,点击顺序不会改变输出旋转。 `Backspace` 返回上一步,右键或 `Esc` 取消。四点会自动排序,点击顺序不会改变输出旋转。
Perspective 只执行图片 Homography 与紧裁切,不做 Refine,也不尝试从当前 View 推导真实相机。结果保持源 Empty 的 Transform,并按原四边形覆盖范围近似更新显示位置与尺寸。选区完全位于图片外或没有可见像素时不会执行。 Perspective 只执行图片 Homography 与紧裁切,不提供 Fit to Foreground,也不尝试从当前 View 推导真实相机。结果保持源 Empty 的 Transform,并按原四边形覆盖范围近似更新显示位置与尺寸。选区完全位于图片外或没有可见像素时不会执行。
<div class="screenshot-placeholder" markdown> <div class="screenshot-placeholder" markdown>
**截图占位:Perspective 比例确认** **截图占位:Perspective 比例确认**

View File

@@ -9,18 +9,18 @@ AnyImage 把运行环境、模型和任务结果放在 Blender 之外,多个 B
- **Ready** 表示模型文件已经就绪。 - **Ready** 表示模型文件已经就绪。
- **Download** 用于按需下载模型。 - **Download** 用于按需下载模型。
- Depth & Normal 与 Upscale 的下载按钮位于当前模型选择项下方;切换选择后,可直接下载对应模型。 - Depth & Normal 与 Upscale 的下载按钮位于当前模型选择项下方;切换选择后,可直接下载对应模型。
- Depth Model 先列出三档 MoGe-2,再列出两档 DA3;DA3 不输出 normal,选择后会在设置下方提示。 - Depth & Normal 只列出三档 MoGe-2;Depth Plane 与 Cutout 共用这个显式选择。
- Remove Background 使用的 BEN2 固定模型显示在对应功能区;Cutout 区只包含 Balloon Profile。 - Remove Background 使用的 BEN2 固定模型显示在对应功能区;Cutout 区只包含 Balloon Profile。
- 默认安装会准备 MoGe-2 ViT-S Normal、BEN2 Base 和 Real-ESRGAN General WDN x4v3。 - 默认安装会准备 MoGe-2 ViT-S Normal、BEN2 Base 和 Real-ESRGAN General WDN x4v3。
- DA3、更大的 MoGe-2,以及可选的 SAFMN Real x4、Real-ESRGAN x4plus 需要时再下载即可。 - 更大的 MoGe-2,以及可选的 SAFMN Real x4、Real-ESRGAN x4plus 需要时再下载即可。
- 可公开取得的 ONNX 制品使用经过大小与 SHA-256 校验的原址,并以 OmooLab R2 为镜像;不会把源码仓库名称推测成 Hugging Face 下载地址。 - 可公开取得的 ONNX 制品使用经过大小与 SHA-256 校验的原址,并以 OmooLab R2 为镜像;不会把源码仓库名称推测成 Hugging Face 下载地址。
- Real-ESRGAN General WDN x4v3 与 SAFMN Real x4 的上游只发布 PyTorch 权重,当前 ONNX 均从官方权重导出并仅从 R2 下载。 - Real-ESRGAN General WDN x4v3 与 SAFMN Real x4 的上游只发布 PyTorch 权重,当前 ONNX 均从官方权重导出并仅从 R2 下载。
较大的模型不一定适合所有图片。建议先用默认模型完成构图,再针对细节与性能需求比较其他模型。 较大的模型不一定适合所有图片。建议先用默认模型完成构图,再针对细节与性能需求比较其他模型。
## 全局 AI 输入尺寸 ## AI 输入与 Cutout 网格尺寸
Environment 区的 **Maximum AI Input Size** 默认 `2048 px`,可在 `256–8192 px` 之间调整。它既是允许执行 Upscale 的当前图片长边上限,也是 Depth 与 Normal 分析输入的最大长边;Upscale 在达到上限时置灰,Depth 与 Normal 则保持宽高比缩小分析输入。颜色贴图仍使用当前图片的原始分辨率。Cutout 在完成套索裁切后再应用上限,因此整张图片很大但选区较小时不会被无谓缩小。 Environment 区的 **Maximum AI Input Size** 默认 `2048 px`,可在 `256–8192 px` 之间调整。它是 Upscale 的可执行源图长边上限,也是 Depth、Normal Map 与 Fit to Foreground 模型分析输入的最大长边;颜色贴图仍使用原始分辨率。Cutout 区的 **Maximum Cutout Mesh Resolution** 默认 `1024`,独立限制所有 Shape 的网格采样密度,不会因为某个 Shape 是否使用 AI 而改变含义。
## Device ## Device
扩展偏好设置中的 **Device** 控制 Depth Plane、Cutout 的 AI 增强、Remove Background 和 Upscale 等正式任务。 扩展偏好设置中的 **Device** 控制 Depth Plane、Cutout 的 AI 增强、Remove Background 和 Upscale 等正式任务。

View File

@@ -6,11 +6,11 @@
| --- | --- | --- | --- | | --- | --- | --- | --- |
| Clipboard Image | 剪贴板静态图 | Reference、Plane、节点或笔刷纹理 | 不需要 | | Clipboard Image | 剪贴板静态图 | Reference、Plane、节点或笔刷纹理 | 不需要 |
| Convert to Plane | 单图、Movie、Sequence | 带材质的普通网格 | 不需要 | | Convert to Plane | 单图、Movie、Sequence | 带材质的普通网格 | 不需要 |
| Convert to Depth Plane | 静态图 | 带深度位移的整图面片 | DA3 或 MoGe-2 | | Convert to Depth Plane | 静态图 | 带深度位移的整图面片 | MoGe-2 |
| Cutout | 静态图局部选区 | 四种局部 Shape | 按 Shape 与选项决定 | | Cutout | 静态图局部选区 | 四种局部 Shape | 按 Shape 与选项决定 |
| Remove Background | 单图、Movie、Sequence | 透明图片或序列 | BEN2 | | Remove Background | 单图、Movie、Sequence | 透明图片或序列 | BEN2 |
| Upscale | 单图、Movie、Sequence | 4× 图片或序列 | Upscale 模型 | | Upscale | 单图、Movie、Sequence | 4× 图片或序列 | Upscale 模型 |
| Image Mask / Perspective | 静态图 | 原位更新的裁切或矫正图 | Refine 时使用 BEN2 | | Image Mask / Perspective | 静态图 | 原位更新的裁切或矫正图 | Fit to Foreground 时使用 BEN2 |
## 普通 Plane ## 普通 Plane
@@ -26,7 +26,7 @@
## Cutout ## Cutout
只需要图片局部,或希望按轮廓生成 Balloon / Depth Surface 时,使用工具栏中的 **Cutout**。普通 Surface 与 Balloon 可以完全在 Blender 内完成,Refine、Normal 与 Depth Shape 按需调用 AI。 只需要图片局部,或希望按轮廓生成 Balloon / Depth Surface 时,使用工具栏中的 **Cutout**。普通 Surface 与 Balloon 可以完全在 Blender 内完成,Fit to Foreground、Generate Normal Map 与 Depth Shape 按需调用 AI。
[阅读 Cutout 指南](cutout.md) [阅读 Cutout 指南](cutout.md)

View File

@@ -37,11 +37,11 @@ flowchart TD
Menu --> Runtime["注册 JobRuntime"] Menu --> Runtime["注册 JobRuntime"]
``` ```
`__init__.py` 按依赖顺序注册,并在注销时逆序清理。扩展级设置统一由 `preferences.addon_preferences()` 读取;`runtime.py` 只消费 `configured_storage_root` 回调,不持有 Blender 偏好设置的定位逻辑。交互中的临时状态和 Debug 参数保存在 `Scene.anyimage_settings`。全局 `max_ai_input_size` 在 `preferences.py` 定义与读取,由 Operator 冻结到每次 Job 请求;Upscale 还用它判断当前 Image 是否允许执行。 `__init__.py` 按依赖顺序注册,并在注销时逆序清理。扩展级设置统一由 `preferences.addon_preferences()` 读取;`runtime.py` 只消费 `configured_storage_root` 回调,不持有 Blender 偏好设置的定位逻辑。交互中的临时状态和 Debug 参数保存在 `Scene.anyimage_settings`。`max_ai_input_size` 与 `max_cutout_mesh_resolution` 都在 `preferences.py` 定义与读取:前者由 Operator 冻结到 AI Job 请求,后者在 Blender 侧统一限制所有 Cutout Shape 的网格采样密度。Upscale 还用 AI 输入上限判断当前 Image 是否允许执行。
## 同步与异步路径 ## 同步与异步路径
无需 AI 或媒体处理的功能直接在 Blender 中完成,例如 Plane、普通 Image Mask、Perspective,以及未启用 Refine/Normal 的 Surface 和 Balloon。需要模型或独立文件处理时,功能 Operator 派生自 `JobOperatorBase`: 无需 AI 或媒体处理的功能直接在 Blender 中完成,例如 Plane、普通 Image Mask、Perspective,以及未启用 Fit to Foreground / Generate Normal Map 的 Surface 和 Balloon。需要模型或独立文件处理时,功能 Operator 派生自 `JobOperatorBase`:
```mermaid ```mermaid
sequenceDiagram sequenceDiagram

View File

@@ -6,7 +6,7 @@
## Image 数据块 ## Image 数据块
`image_data.py` 处理 Blender `Image` 数据块及其在 Image Empty 上的使用方式,包括像素读取、动画帧、Job 输入、结果 Image 创建和 Empty 引用替换。它不处理 Viewport 手势或选区算法。 `image_data.py` 处理 Blender `Image` 数据块及其在 Image Empty 上的使用方式,包括像素读取、动画帧、Job 输入、结果 Image 创建和 Empty 引用替换。它不处理 Viewport 手势或 Region 算法。
| 函数 | 职责 | | 函数 | 职责 |
| --- | --- | | --- | --- |
@@ -25,7 +25,7 @@
| `image_data.depth_data_name()` | 生成 Depth 结果数据块文件名。 | | `image_data.depth_data_name()` | 生成 Depth 结果数据块文件名。 |
| `image_data.color_data_name()` | 生成 Color 结果数据块文件名。 | | `image_data.color_data_name()` | 生成 Color 结果数据块文件名。 |
| `image_data.load_color_result_image()` | 加载、命名、Pack 并配置 Job 的 Color 或 Sequence 结果;不修改 Empty。 | | `image_data.load_color_result_image()` | 加载、命名、Pack 并配置 Job 的 Color 或 Sequence 结果;不修改 Empty。 |
| `image_data.load_normal_result_image()` | 加载、命名、设为 Non-Color 并 Pack Normal 结果。 | | `image_data.load_normal_result_image()` | 加载、命名、设为 Non-Color 并 Pack Normal Map。 |
| `image_data.create_pixel_result_image()` | 从本地编辑后的 RGBA 像素创建 Packed Image;不修改 Empty。 | | `image_data.create_pixel_result_image()` | 从本地编辑后的 RGBA 像素创建 Packed Image;不修改 Empty。 |
| `image_data.replace_empty_image()` | 执行 Image Empty 结果替换、共享引用分流、旧数据清理、帧设置继承和可选画布对齐。 | | `image_data.replace_empty_image()` | 执行 Image Empty 结果替换、共享引用分流、旧数据清理、帧设置继承和可选画布对齐。 |
| `image_data.image_empty_bounds()` | 计算 Image Empty 在对象局部坐标中的实际显示边界。 | | `image_data.image_empty_bounds()` | 计算 Image Empty 在对象局部坐标中的实际显示边界。 |
@@ -39,7 +39,7 @@
- Region 是用户在图片空间圈定的业务范围;`RegionPath` 保存路径点、反选状态和填充规则。 - Region 是用户在图片空间圈定的业务范围;`RegionPath` 保存路径点、反选状态和填充规则。
- `RegionMask` 是 Region 的栅格化结果,只包含局部像素权重 `values` 和源图 `bounds`。 - `RegionMask` 是 Region 的栅格化结果,只包含局部像素权重 `values` 和源图 `bounds`。
- Mask 只表示二维像素权重或 Alpha 运算,不作为手势、协议或业务范围的别名。 - Mask 只表示二维像素权重或 Alpha 运算,不作为手势、协议或业务范围的别名。
- Selection 只用于 Blender 对象选择和界面动作 Refine Selection,不表示中间数据。 - Selection 只用于 Blender 对象选择;图片业务范围统一称为 Region,细化动作称为 Fit to Foreground。
`image_interaction.py` 处理 3D Viewport 中的点击、坐标投影、`ImageRegionGesture`、工具设置和预览绘制;`image_region.py` 负责图片空间 Region 的值、栅格化和 Alpha 裁切。前者负责用户如何圈定 Region,后者负责 Region 如何变成 Mask。 `image_interaction.py` 处理 3D Viewport 中的点击、坐标投影、`ImageRegionGesture`、工具设置和预览绘制;`image_region.py` 负责图片空间 Region 的值、栅格化和 Alpha 裁切。前者负责用户如何圈定 Region,后者负责 Region 如何变成 Mask。
@@ -51,8 +51,8 @@
| `image_interaction.image_edit_poll()` | 判断当前上下文是否允许启动 Image 编辑工具。 | | `image_interaction.image_edit_poll()` | 判断当前上下文是否允许启动 Image 编辑工具。 |
| `image_interaction.resolve_image_edit_click()` | 解析点击到 Image Empty,并保持或恢复选择状态。 | | `image_interaction.resolve_image_edit_click()` | 解析点击到 Image Empty,并保持或恢复选择状态。 |
| `image_interaction.active_view3d_tool_id()` | 取得当前 View3D Workspace Tool ID。 | | `image_interaction.active_view3d_tool_id()` | 取得当前 View3D Workspace Tool ID。 |
| `image_interaction.apply_region_settings()` | 将当前 Region 工具的 Refine 设置同步到手势 Operator。 | | `image_interaction.apply_region_settings()` | 将当前 Region 工具的 Fit to Foreground 设置同步到手势 Operator。 |
| `image_interaction.draw_region_settings()` | 绘制 Region 工具的公共 Refine 设置。 | | `image_interaction.draw_region_settings()` | 绘制 Region 工具的公共 Fit to Foreground 设置。 |
| `image_interaction.draw_dashed_line()` | 绘制手势或透视框虚线。 | | `image_interaction.draw_dashed_line()` | 绘制手势或透视框虚线。 |
| `image_interaction.draw_polygon_fill()` | 绘制手势或透视框填充预览。 | | `image_interaction.draw_polygon_fill()` | 绘制手势或透视框填充预览。 |
| `image_interaction.gesture_property()` | 声明统一的 Box、Lasso、Line 和 Polyline RNA 属性。 | | `image_interaction.gesture_property()` | 声明统一的 Box、Lasso、Line 和 Polyline RNA 属性。 |
@@ -69,16 +69,24 @@
| `image_region.rasterize_region_path()` | 把 `RegionPath` 栅格化为 `RegionMask`,并可与源 Alpha 相交。 | | `image_region.rasterize_region_path()` | 把 `RegionPath` 栅格化为 `RegionMask`,并可与源 Alpha 相交。 |
| `image_region.crop_pixels_to_alpha()` | 按 Alpha 非零区域紧裁切 RGBA 像素。 | | `image_region.crop_pixels_to_alpha()` | 按 Alpha 非零区域紧裁切 RGBA 像素。 |
## Mesh 结果构建 ## Depth 产物
`mesh_result.py` 集中 Convert to Plane 与 Cutout 共同使用的结果构建能力:读取图片几何产物、创建材质、加载节点资产、设置修改器输入,并完成结果对象替换。 `depth_data.py` 负责 Blender 侧共享的 Depth 文件加载和相机尺度换算;Server 已经写好 EXR,Blender 不再从点场创建第二张 Depth Image。
`ImageGeometry` 表示按图片像素组织的三维点、相机内参、有效区域、参考深度和选区统计;`load_image_geometry()` 从 `image_geometry.npz` 读取并校验这组数据。
| 函数 | 职责 | | 函数 | 职责 |
| --- | --- | | --- | --- |
| `mesh_result.load_image_geometry()` | 读取并校验 `image_geometry.npz`,返回用于构建 Mesh 的 `ImageGeometry`。 | | `depth_data.load_depth_metadata()` | 读取并校验 `depth.json` 中的图片尺寸、相机内参与参考深度。 |
| `mesh_result.create_image_material()` | 从 Image Empty 创建包含颜色、位移、Normal、Alpha 和动画设置的完整材质。 | | `depth_data.load_depth_result_image()` | 直接加载、命名、设为 Non-Color 并 Pack `vector-depth.exr` 或 `z-depth.exr`。 |
| `depth_data.camera_depth_values()` | 从已加载 Depth Image 读取用于本地标定的 Z 与有效区。 |
| `depth_data.depth_uniform_scale()` | 根据相机内参、参考深度和 Image Empty 范围计算统一尺度。 |
## Mesh 结果构建
`mesh_result.py` 集中 Convert to Plane 与 Cutout 共同使用的结果构建能力:创建材质、加载节点资产、设置修改器输入,并完成结果对象替换。
| 函数 | 职责 |
| --- | --- |
| `mesh_result.create_image_material()` | 从 Image Empty 创建包含颜色、位移、Normal Map、Alpha 和动画设置的完整材质。 |
| `mesh_result.load_asset_node_group()` | 从随扩展分发的 Blend 资产加载指定 Node Group。 | | `mesh_result.load_asset_node_group()` | 从随扩展分发的 Blend 资产加载指定 Node Group。 |
| `mesh_result.modifier_input_identifier()` | 按接口名称解析 Geometry Nodes Modifier 输入 Identifier。 | | `mesh_result.modifier_input_identifier()` | 按接口名称解析 Geometry Nodes Modifier 输入 Identifier。 |
| `mesh_result.set_modifier_input()` | 跨受支持 Blender RNA 形式设置 Modifier 输入。 | | `mesh_result.set_modifier_input()` | 跨受支持 Blender RNA 形式设置 Modifier 输入。 |
@@ -88,7 +96,7 @@
## AI 任务准备 ## AI 任务准备
`ai_job.py` 处理多个 AI 功能共同使用的任务准备,包括输入路径校验、Environment 与模型检查、Device 选择,以及 DA3 和 MoGe-2 请求参数组装。它不包含业务 Operator、结果导入或 Undo 操作。 `ai_job.py` 处理多个 AI 功能共同使用的任务准备,包括输入路径校验、Environment 与模型检查、Device 选择和模型请求参数组装。DA3 参数只供 Debug Operator 使用;正式几何流程只组装 MoGe-2 参数。它不包含业务 Operator、结果导入或 Undo 操作。
| 函数 | 职责 | | 函数 | 职责 |
| --- | --- | | --- | --- |
@@ -98,5 +106,5 @@
| `ai_job.require_environment()` | 要求 AnyImage AI Environment 已就绪。 | | `ai_job.require_environment()` | 要求 AnyImage AI Environment 已就绪。 |
| `ai_job.invoke_ai_setup_if_needed()` | Environment 未就绪时启动安装交互。 | | `ai_job.invoke_ai_setup_if_needed()` | Environment 未就绪时启动安装交互。 |
| `ai_job.require_model()` | 校验模型标识和下载状态,并返回模型记录。 | | `ai_job.require_model()` | 校验模型标识和下载状态,并返回模型记录。 |
| `ai_job.da3_parameters()` | 组装 DA3 正式任务与 Debug 共用参数。 | | `ai_job.da3_parameters()` | 组装 `debug-da3` 参数。 |
| `ai_job.moge2_parameters()` | 组装 MoGe-2 正式任务与 Debug 共用参数。 | | `ai_job.moge2_parameters()` | 组装 MoGe-2 正式任务与 Debug 共用参数。 |

View File

@@ -1,23 +1,8 @@
# AI 环境管理 # AI 环境管理
本页对应 `src/anyimage/operators/ai_setup.py`。该模块把环境安装、必需模型下载和模型缓存清理组织成 Blender 可调用的 Operator。 AI 环境管理从当前安装状态出发,补齐独立运行环境和必需模型,并提供设置入口与模型缓存释放操作。
## 对应文件 ## 执行过程
```text
src/anyimage/
├── operators/ai_setup.py # 环境与模型管理入口
├── preferences.py # 环境位置和扩展级设置
├── properties.py # 环境状态与模型目录
├── runtime.py # BlendJob Runtime 访问入口
└── server/
├── app.py # 下载 Job 注册
├── model_catalog.py # 模型元数据
├── model_download.py # 模型文件下载
└── model_manager.py # 模型资源与 Session 缓存
```
## 主流程
```mermaid ```mermaid
flowchart TD flowchart TD
@@ -27,10 +12,31 @@ flowchart TD
Ready -- "否" --> Environment{"运行环境是否就绪?"} Ready -- "否" --> Environment{"运行环境是否就绪?"}
Environment -- "否" --> Install["安装隔离 Python 环境"] Environment -- "否" --> Install["安装隔离 Python 环境"]
Environment -- "是" --> Download["下载缺失的必需模型"] Environment -- "是" --> Download["下载缺失的必需模型"]
Install --> Download Install --> Download --> Done
Download --> Done
``` ```
`SetupAIEnvironment` 是统一入口:首次使用时选择普通源或镜像安装隔离环境;环境已存在时,只补齐必需模型。单模型下载和必需模型批量下载都通过 Job Server 执行,因此状态、错误与进度沿用同一套 Runtime 通道。 `SetupAIEnvironment` 是统一入口。首次使用时先安装运行环境,再下载默认模型;环境已经存在时,只补齐缺失模型。环境安装由 Blender Operator 启动,模型下载通过 Job Server 执行。
`OpenAIEnvironmentSettings` 将用户带到扩展首选项。`ClearModels` 清除 Runtime 中的 `model_manager` 资源,使已加载的模型 Session 释放;它不负责删除磁盘上的模型文件。 ## 节点说明
### 读取状态,选择下一步
入口位于 `src/anyimage/operators/ai_setup.py`。`SetupAIEnvironment` 通过 `properties.py` 中的 `ai_status()` 读取环境和模型状态,并根据结果结束、安装环境或调用 `DownloadRequiredModels`。对话框同时显示预计下载量和缺失模型名称。
`runtime.py` 提供当前 Environment 与 Server 状态;Server 正忙时不会启动新的安装流程。`OpenAIEnvironmentSettings` 也在 `ai_setup.py` 中,它只负责打开 AnyImage 的扩展首选项。
### 安装独立环境
环境尚未就绪时,`SetupAIEnvironment` 调用 Runtime 注册的安装 Operator。普通源与中国镜像使用同一条安装流程,`use_mirror` 只决定 uv、Python 和 Python 包的下载源,不改变模型来源。
安装完成后继续进入模型下载,不在 Blender Python 中加载 ONNX Runtime 或模型依赖。
### 下载模型
`ai_setup.py` 中的 `DownloadModel` 处理单个模型,`DownloadRequiredModels` 处理默认必需模型集合。两者先检查 Environment、Blender Online Access 和模型记录,再提交 `download-model` 或 `download-required-models` Job。
Job 在 `server/app.py` 注册;`server/model_catalog.py` 保存模型名称、文件和来源,`server/model_download.py` 下载并校验文件,`server/model_manager.py` 管理模型目录与已加载 Session。下载进度和错误通过同一套 BlendJob Runtime 返回 Blender。
### 释放模型缓存
`ai_setup.py` 中的 `ClearModels` 调用 `runtime.clear_resource("model_manager")`,释放 Server 中已加载的模型 Session。它不会删除磁盘上的模型文件,下次推理仍可从现有文件重新加载。

View File

@@ -1,34 +1,41 @@
# 剪贴板图片 # 剪贴板图片
剪贴板功能把操作系统中的图片转换为 Packed Blender Image,再根据当前编辑器创建对象、节点或画笔纹理。它完全运行在 Blender 进程中,不使用 Job Server。 剪贴板图片从 `Ctrl/Cmd + V` 开始,把操作系统中的图片变成 Packed Blender Image,并放到当前编辑器支持的位置。
## 对应文件 ## 执行过程
```text
operators/clipboard_image/
├── __init__.py # Operator、原生复制优先级与快捷键
├── clipboard.py # Windows、macOS、Linux 剪贴板读取
└── actions.py # Image 复用、Pack 与各编辑器落点
```
## 处理流程
```mermaid ```mermaid
flowchart TD flowchart TD
Paste["Ctrl/Cmd + V"] --> Priority{"刚执行过 Blender Copy?"} Paste["Ctrl/Cmd + V"] --> Priority{"刚执行过 Blender Copy?"}
Priority -- "是" --> Native["交回原生 Paste"] Priority -- "是" --> Native["交回原生 Paste"]
Priority -- "否" --> Read["读取系统图片"] Priority -- "否" --> Read["读取系统图片"]
Read --> Image["按内容哈希复用或创建 Packed Image"] Read --> Image["复用或创建 Packed Image"]
Image --> Target{"当前编辑器"} Image --> Target{"当前编辑器"}
Target --> View["Reference / Plane"] Target --> View["Reference / Plane"]
Target --> Node["Image Node"] Target --> Node["Image Node"]
Target --> Paint["Brush Texture"] Target --> Paint["Brush Texture"]
``` ```
`TrackNativeCopy` 在相同快捷键下记录系统剪贴板变化标记。如果剪贴板没有在 Blender Copy 之后变化,Paste Operator 返回 `PASS_THROUGH`,让 Blender 原生对象或节点粘贴优先执行。 整条流程运行在 Blender 进程中,不使用 Job Server。快捷键会先尊重 Blender 自己的 Copy/Paste;只有系统剪贴板出现新的图片内容时,AnyImage 才接管粘贴。
`clipboard.py` 封装平台差异:Windows 读取 DIB/DIBV5,macOS 与 Linux 调用平台提供的图片导出命令。模块最终都返回编码后的图片字节和扩展名,后续流程不再关心来源平台。 ## 节点说明
`actions.py` 以 SHA-256 标记 Packed Image,相同内容可复用已有数据块。3D View 中的 Reference 和 Plane 使用鼠标位置或视图落点;Plane 复用 `convert_to_plane/mesh_plane.py` 的基础网格与 `O Mesh Plane` 资产。Node Editor 根据树类型选择对应 Image 节点,绘制模式则为当前画笔配置 Texture。 ### 判断是否交给 Blender
快捷键只注册到 3D View 与 Node Editor;编辑器内部的具体支持范围由 `target_for_context()` 决定。创建成功后图片始终 Pack 到当前 `.blend`,避免依赖临时剪贴板文件。 `operators/clipboard_image/__init__.py` 注册 `TrackNativeCopy`、`PasteClipboardImage` 和 3D View、Node Editor 快捷键。`TrackNativeCopy` 记录 Blender Copy 时的系统剪贴板变化标记;粘贴时若标记没有变化,`PasteClipboardImage` 返回 `PASS_THROUGH`,让 Blender 继续处理对象或节点粘贴。
同一文件随后调用 `actions.target_for_context()` 判断当前编辑器是否支持图片落点。
### 读取系统图片
`clipboard.py` 负责平台差异。Windows 读取 DIB 或 DIBV5 并补成 BMP;macOS 和 Linux 调用系统提供的图片导出命令。无论来源平台,输出都收敛为图片字节和扩展名,后续步骤不再判断操作系统。
### 创建或复用 Packed Image
`actions.py` 对图片内容计算 SHA-256。相同内容已经粘贴过时会复用现有 Packed Image,否则创建新 Image 并 Pack 到当前 `.blend`,避免依赖临时剪贴板文件。
### 放到当前编辑器
落点仍在 `actions.py` 中完成:3D View 创建 Reference 或 Plane,Node Editor 创建与当前节点树匹配的 Image 节点,绘制模式把 Image 配置为当前画笔的 Texture。鼠标位置和视图位置也在这里换算。
Plane 路径调用 `convert_to_plane/mesh_plane.py` 创建基础网格,并使用 `convert_to_plane/nodes.py` 提供的 `O Mesh Plane` 节点组。`clipboard_image/__init__.py` 只负责入口和快捷键,具体 Blender 数据始终由 `actions.py` 创建。

View File

@@ -1,61 +1,42 @@
# Plane 转换 # Plane 转换
Plane 转换包含两条路径:普通 Plane 在 Blender 内同步完成;Depth Plane 先通过 `generate-depth-plane` Job 取得深度场,再回到 Blender 构建面片。两者共享图片命名、材质与基础四边形能力,但使用不同的 Geometry Nodes。 Plane 转换从当前 Image Empty 创建普通 Plane 或 Depth Plane:普通 Plane 同步完成,Depth Plane 先取得 MoGe-2 Z Depth 文件再回到 Blender 创建对象。
## 对应文件 ## 执行过程
```text
operators/convert_to_plane/
├── __init__.py # 对外导出
├── conversion.py # 两个入口 Operator 与 Depth Job
├── build.py # 颜色、深度场、材质和对象导入
├── mesh_plane.py # 基础四边形与 O Mesh Plane 修改器
└── nodes.py # 运行时 O Image Depth Plane 节点图
server/
├── jobs/generate_depth_plane.py
├── geometry/generate_depth.py
├── models/da3.py
├── models/moge2.py
└── geometry/projection.py
```
## 总体流程
```mermaid ```mermaid
flowchart TD flowchart TD
Empty["Image Empty"] --> Kind{"转换类型"} Empty["Image Empty"] --> Kind{"转换类型"}
Kind -- "Plane" --> Copy["复制 Image 与材质"] Kind -- "Plane" --> Color["复制 Color Image"]
Copy --> Quad["创建四边形 + O Mesh Plane"] Color --> Plane["创建 Plane Object"]
Kind -- "Depth Plane" --> Input["准备颜色与分析输入"] Kind -- "Depth Plane" --> Job["generate-depth-plane-geometry"]
Input --> Job["generate-depth-plane"] Job --> Result["Color Image + z-depth.exr + depth.json"]
Job --> Field["image.0001 + image_geometry.npz"] Result --> Import["直接加载并 Pack Depth Image"]
Field --> Anchor["按 Empty 范围锚定相机点"] Import --> Depth["创建 Depth Plane Object"]
Anchor --> Depth["创建深度 Image + 材质 + 节点修改器"]
Quad --> Replace["替换源 Empty"]
Depth --> Replace
``` ```
## 普通 Plane 两条路径共用 Image Empty 的显示范围、颜色材质和基础 Plane。Depth Plane 多出一次 Server Job,并由节点根据相机元数据把 Z Depth 转换为局部位移。正式流程只使用 MoGe-2;DA3 只保留 Debug 入口。
`ConvertToPlane` 复制当前 Image 数据,保留 Movie / Sequence 的 Image User 设置,创建与 Image Empty 显示范围一致的四边形,再加载内置 `O Mesh Plane` 节点组。原始 Mesh 始终只有一个四边形;`Subdivide`、`Thickness` 和材质赋值由修改器执行。 ## 节点说明
对象继承源 Empty 的 `matrix_world`。转换完成后删除源 Empty,并让新 Plane 沿用对象名称。剪贴板的 Plane 导入复用同一构建函数,因此两种入口得到一致的基础网格与修改器。 ### 选择 Plane 或 Depth Plane
## Depth Plane `operators/convert_to_plane/__init__.py` 导出注册类型,实际入口在 `conversion.py`。`ConvertToPlane` 进入本地路径,`ConvertToDepthPlane` 检查静态图片和 AI 环境后启动 `GenerateDepthPlane`。
`ConvertToDepthPlane` 当前只接受静态图片。深度模型和分辨率来自扩展 Preferences:MoGe-2 使用 Resolution Level,DA3 使用按原图尺寸换算并对齐到模型 Patch 的处理分辨率。 ### 创建普通 Plane
`GenerateDepthPlane` 把当前 Image 随模型参数提交给 `generate-depth-plane`。`jobs/generate_depth_plane.py` 负责单图协议,`geometry/generate_depth.py` 负责 AI 深度生成;Server 将模型结果统一为 `GeometryFrame`,写出包含相机点、深度、内参、有效区域和参考深度的 `image_geometry.npz`。Blender 侧通过公共 `load_image_geometry()` 读取为 `ImageGeometry`,与 Cutout 使用同一协议校验。这一步不创建网格。`max_input_size` 同时传入 Job;Server 通过 `media/resolution.py` 在模型推理前限制分析图长边,默认 `2048`。颜色输入仍按当前 Image 的未缩放分辨率复制,受限分析图不会成为材质 Color。 `conversion.py` 先调用 `object.create_color_image()` 复制当前 Image,再由 `object.create_plane_object()` 创建对象。`object.py` 使用 `mesh_plane.py` 生成与 Image Empty 显示范围一致的基础四边形,并通过 `nodes.py` 加载 `O Mesh Plane` 修改器。
Blender 响应阶段执行三件事: Movie 或 Sequence 的 Image User 设置会保留。转换完成后,新 Plane 替换源 Empty,并沿用原对象名称。
1. 根据 Image Empty 的平面范围、相机内参和参考深度,把相机点等比锚定到对象局部空间。 ### 生成 Z Depth
2. 将锚定点的局部 Z 写入 Packed 浮点 `_depth.exr` Image,并创建颜色材质。
3. 创建基础四边形和运行时 `O Image Depth Plane` 节点组,以 UV 采样深度并移动正面。
Depth 节点组先复用 `O Mesh Plane` 完成细分与材质,再生成背面厚度、限制正面向内穿透,最后在 Thickness 为零时删除背面和侧面。`Depth Amount` 控制位移强度;`Depth Image` 与 `Material` 是由插件填入的内部输入。 Depth 路径中的 `GenerateDepthPlane` 位于 `conversion.py`。它从 Preferences 读取 **Depth & Normal Model**、**MoGe Resolution Level** 和 **Maximum AI Input Size**,提交 `generate-depth-plane-geometry` Job。
与 Cutout 的 Depth Surface 不同,Depth Plane 只使用锚定后的 Z 高度,不在 Geometry Nodes 中重建完整相机空间 XYZ。它强调一张图上的可编辑浮雕,而不是局部透视表面。 `server/jobs/depth_plane.py` 编排任务,`server/geometry/prediction_artifacts.py` 调用 MoGe-2,写出 `z-depth.exr` 与 `depth.json`。EXR 的 RGB 重复保存相机 Z,Alpha 保存有效区;JSON 保存图片尺寸、相机内参与参考深度。AI 输入上限只影响模型分析图,颜色副本仍保留原始分辨率。
颜色材质、Normal 导入和结果对象激活分别复用 `create_image_material()`、`load_normal_result_image()` 与 `finalize_object_result()`;普通 Plane 和 Depth Plane 不再各自维护对象替换步骤。 ### 创建 Depth Plane
Job 返回后,`object.create_depth_plane_from_result()` 直接加载并 Pack `z-depth.exr`,数据块仍由 `depth_data_name()` 命名。`depth.json` 用于计算统一尺度,`O Image Depth Plane` 根据参考深度、统一尺度和用户的 Depth Amount 把相机 Z 转换为局部位移。
随后 `object.py` 创建颜色材质和基础四边形,`nodes.py` 加载 `O Image Depth Plane` 修改器。`conversion.py` 最后用一个 Undo 步骤替换源 Empty。

View File

@@ -1,76 +1,91 @@
# Cutout Tool # Cutout Tool
Cutout 由两个核心 Operator 组成: Cutout 从图片上的一个 Region 开始,可按需执行 Fit to Foreground、生成 Depth 或 Normal Map,最后在 Blender 中创建带材质与 Modifier 的 Mesh Object。
- `SelectCutoutRegion`(`select_region.py`)把 Viewport 手势转换为图片空间 Region,并弹出 Shape 菜单。 ## 执行过程
- `CutoutRegionToShape`(`region_to_shape.py`)准备所需数据,再构建并创建 Shape。
路径约定:Blender 侧文件相对 `src/anyimage/operators/cutout_tool/`,Server 侧文件相对 `src/anyimage/server/`。
## 执行路径
```mermaid ```mermaid
flowchart TD flowchart TD
Select["圈定 Cutout Region<br/>选择 Shape"] Select["圈定 Region,选择 Shape"]
Raster["栅格化 Region"] Raster["栅格化 Region"]
Route{"需要 AI 数据?"} Route{"需要后台生成数据?"}
Local["使用本地 Region 数据"] Local["沿用本地 RegionMask"]
AI["生成 Refine / Depth / Normal"] Job["执行 Fit to Foreground,生成 Depth / Normal 产物"]
Ready["汇总 Shape 输入"] Ready["汇总 Shape 输入"]
Build["构建基础 Shape"] Build["构建 BaseShape"]
Create["创建 Blender 对象"] Create["创建 Blender 对象"]
Select --> Raster Select --> Raster --> Route
Raster --> Route
Route -- "否" --> Local --> Ready Route -- "否" --> Local --> Ready
Route -- "是" --> AI --> Ready Route -- "是" --> Job --> Ready
Ready --> Build --> Create Ready --> Build --> Create
``` ```
Surface 与 Balloon 可以完全在 Blender 中完成;启用 Fit to Foreground、生成 Depth Shape 或 Normal Map 时,中间会经过一次 Server Job。
两条路径的区别只在中间是否需要 Server 产物。回到 Blender 后,它们都使用最终的 `RegionMask` 构建同一种 `BaseShape`,再由同一入口创建对象。Server 不创建 Blender Mesh,也不访问 `bpy`。
下文中的 Blender 侧文件相对 `src/anyimage/operators/cutout_tool/`,Server 侧文件相对 `src/anyimage/server/`。
## 节点说明 ## 节点说明
### 圈定 Cutout Region ### 圈定 Region,选择 Shape
`SelectCutoutRegion` 将 Lasso 路径投射到图片像素坐标,形成 `RegionPath`,然后把 Region 和用户选择的 Shape 交给 `CutoutRegionToShape`。双击图片时,Region 使用完整图片范围。 Cutout 工具由 `__init__.py` 注册,交互从 `main.py` 中的 `SelectCutoutRegion` 开始。它接收 Lasso 或双击整图手势,把图片空间路径保存为 `RegionPath`,然后调用 `interaction.py` 打开 Shape 饼菜单。
`RegionPath` 定义在 `operators/common/image_region.py`,包含图片空间路径点、反选状态和填充规则。它既供 Cutout 使用,也供 Image Tool 使用。 `interaction.py` 在这里完成 Region 有效性检查、读取三个工具选项,并根据 **Mesh Sampling Spacing** 换算期望网格分辨率;它不定义 Operator。菜单中的 Surface、Balloon、Depth Balloon 和 Depth Surface,以及它们对应的节点分支索引,统一来自 `shape.py`。
### 栅格化 Region ### 栅格化 Region
`CutoutRegionToShape` 把 `RegionPath` 栅格化为 `RegionMask`。`RegionMask` 只包含局部像素权重 `values` 和它在源图中的 `bounds`,后续几何、颜色裁切和 AI 输入都以它为范围。 用户选定 Shape 后,`main.py` 中的 `CutoutRegionToShape` 接过主线。它使用 `operators/common/image_region.py` 把 `RegionPath` 栅格化为 `RegionMask`:`values` 保存局部像素权重,`bounds` 保存它在源图中的位置。套索外和源图透明区域不会进入最终 Region。
Surface 或 Balloon 在未启用 Refine 与 Normal 时直接使用本地 Region 数据;选择 Depth Shape,或启用 Refine、Normal 时进入 AI 路径。两条路径最终都调用 `region_to_shape()`。 此时也会应用 **Maximum Cutout Mesh Resolution**。它限制裁切区域的最长采样边,对四种 Shape 一视同仁,与是否使用 AI 无关。
### 生成 AI 数据 ### 需要后台生成数据?
`generate-region-data` Job(`jobs/generate_region_data.py`)根据请求生成所需数据: 只有以下情况需要提交 Job:
- `generate_selection()`(`region/generate_selection.py`)使用 BEN2 收紧 Region; - 开启 **Fit to Foreground**;
- `generate_depth(..., normal_mode=...)`(`geometry/generate_depth.py`)使用 MoGe-2 生成 Depth,并可从同一次 Prediction 派生 Tangent 或 Object Normal。 - 选择 Depth Balloon 或 Depth Surface;
- 开启 **Generate Normal Map**。
Blender 传入的已是带 Region Alpha 的局部 RGBA。Depth 与 Normal 同时启用时只运行一次 MoGe-2,颜色贴图仍使用 Region 的原始裁切分辨率。 没有开启两个 AI 选项的 Surface 和 Balloon 会直接沿用本地 `RegionMask`。
### 构建基础 Shape ### 执行 Fit to Foreground,生成 Depth / Normal 产物
`region_to_shape()` 汇总 Region、Depth 和 Normal 输入,再依次执行几何构建与 Blender 数据创建。 `main.py` 把请求提交给 `server/jobs/cutout.py`。这个 Job 只生成本次 Shape 实际需要的产物:
`build_base_shape()` 从 `RegionMask` 构建纯内存 `BaseShape`:先由 `region_contours.py` 提取包含外轮廓与孔洞的 `RegionContours`,再由 `triangulation.py` 按 Point Spacing 采样并约束三角化,最后使用 `balloon.py` 计算 `o_balloon`。`depth_calibration.py` 负责把 AI Depth 标定为 Blender 中使用的深度基准与方向。四种 Shape 共用这张单面基础网格。 - **Fit to Foreground** 调用 `server/region/refine_region.py`,写出收紧后的 `region.png` 和 `region_bounds.json`;
- 需要 Vector Depth 时写出相机空间 XYZ 组成的 `vector-depth.exr`,需要 Z Depth 时写出 `z-depth.exr`,两者都配套 `depth.json`;
- **Generate Normal Map** 根据既定空间写出 `tangent-normal.png` 或 `object-normal.png`。
Depth 与 Normal Map 同时需要时,共享同一次 MoGe-2 Prediction。Prediction 只存在于 Server 内存,Job 目录只保留 Blender 可直接加载的图片和 JSON。Depth Surface 的 Normal Map 编码为 Cutout 对象空间,其正面是对象局部 `-X`;其余 Shape 使用 Tangent Space。Normal Space 由 `main.normal_mode_for_shape()` 根据 Shape 决定,不是单独的用户选项。
**Maximum AI Input Size** 只限制 BEN2 和 MoGe-2 看到的分析图长边。它不改变网格密度,也不降低最终颜色图分辨率。DA3 不进入这条正式路径,只保留 Debug 功能。
### 构建 BaseShape
Job 完成后,`main.py` 在 Blender 主线程读取可选的 RegionMask,直接加载并 Pack Depth 与 Normal 图片;本地路径则直接使用已有数据。两者最终都进入 `main.create_cutout_shape()`,再调用 `geometry.py` 中的 `build_base_shape()`:
1. 从最终 `RegionMask` 提取外轮廓和孔洞;
2. 按 Mesh Sampling Spacing 生成轮廓与内部采样点;
3. 进行约束三角化;
4. 计算四种 Shape 共用的 `o_balloon`;
5. 汇总顶点、三角面、UV 和 Balloon 权重,得到内存中的 `BaseShape`。
`geometry.py` 还提供 Depth 标定、采样和网格尺度计算,供后面的对象创建阶段使用。
### 创建 Blender 对象 ### 创建 Blender 对象
`create_shape_object()` 创建裁切后的 Color Image、Material、Mesh Object 和统一的 `O Image Cutout` Modifier。Shape 的差异由 Modifier 中对应的几何分支表达: `main.create_cutout_shape()` 把 `BaseShape` 和可选产物交给 `object.py` 的 `create_shape_object()`。这里创建裁切后的 Color Image、Material、Mesh Object,并挂上统一的 `O Image Cutout` Modifier。材质通过 `operators/common/mesh_result.py` 创建,Cutout 使用 `1.2` 的 IOR。Color Image 是按最终 Region 包围矩形裁切的源图 RGBA;RegionMask 不写入颜色图 Alpha,源图原有 Alpha 会原样保留。
| Shape | 形成方式 | Shape 自身所需 AI 数据 | | Shape | Modifier 如何形成结果 | 必需后台产物 |
| --- | --- | --- | | --- | --- | --- |
| Surface | 保持基础 Shape 的平面轮廓,并可增加真实厚度 | 无 | | Surface | 保持基础平面轮廓,可增加真实厚度 | 无 |
| Balloon | 使用 `o_balloon` 从轮廓向内部平滑鼓起 | 无 | | Balloon | 使用 `o_balloon` 从边缘向内部鼓起 | 无 |
| Depth Balloon | 在 Balloon 上叠加相对基准面的图片深度,并沿 Depth Direction 形成浮雕 | MoGe-2 Depth | | Depth Balloon | 在 Balloon 上叠加相对基准面的深度 | `z-depth.exr`、`depth.json` |
| Depth Surface | 使用完整相机点 XYZ 重建带透视位移的表面,并可增加厚度 | MoGe-2 Camera Points | | Depth Surface | 使用相机点 XYZ 重建透视表面 | `vector-depth.exr`、`depth.json` |
四种 Shape 共用 Region、基础三角面、UV、材质、Cleanup 和 Smooth。Cleanup 的 UI 数值放大 `100` 倍,节点内部乘以 `0.01` 还原,因此显示尺度不会改变实际清理阈值。 Depth Shape 的 Base Plane Depth,以及 Depth Balloon 的 Direction,都由 `object.py` 调用 `geometry.py` 的标定函数,根据最终 Region Alpha 计算。透明像素达到 `5%` 时使用有效 Region 自适应标定,否则使用稳定的固定基准。
### Depth 与 Normal Normal Map 是材质细节,不改变上述四种 Shape 的几何分类。关闭时材质使用几何法线;开启时,`object.py` 把生成的图片和已经确定的 Normal Space 交给材质节点。最后 `main.py` 提交 Undo,整条 Cutout 流程结束。
Depth Shape 根据最终 Region Alpha 选择基准:透明像素比例达到 `5%` 时从有效内容自适应计算 Base Plane Depth;不足 `5%` 时使用固定基准。Depth Balloon 还会在自适应路径中拟合 Depth Direction。
Normal 是材质层的浮雕细节,与四种 Shape 都可组合。None 使用几何法线;Tangent 以完整图片平面为切线基准;Object 直接编码到 Cutout 对象局部轴。Geometry Nodes 负责大轮廓和深度,Normal 负责表面细节。

View File

@@ -1,27 +1,32 @@
# Debug Operator # Debug Operator
本页对应 `src/anyimage/operators/debug.py`。这些 Operator 为开发者直接验证 DA3、MoGe-2、BEN2 和 Upscale 的 Server 接口,不参与面向用户的对象创建流程。 Debug Operator 从面板中的模型参数发起一次原始推理,把结果文件留在 Job 目录中供开发者检查,不创建用户场景对象。
## 对应文件 ## 执行过程
```text
src/anyimage/
├── operators/debug.py # 调试请求与结果目录打开
├── panel.py # Debug 面板入口
└── server/
├── jobs/debug_predictions.py # 四种公开 Debug Job 的共享编排
└── outputs/ # 调试产物写出
```
## 主流程
```mermaid ```mermaid
flowchart TD flowchart TD
Panel["Debug 面板"] --> Operator["选择模型调试 Operator"] Panel["Debug 面板"] --> Operator["选择模型 Debug Operator"]
Operator --> Request["从 Scene 设置组装请求"] Operator --> Request["读取 Scene 参数"]
Request --> Job["提交对应 debug Job"] Request --> Job["提交对应 Debug Job"]
Job --> Output["写出原始预测产物"] Job --> Output["写出原始预测产物"]
Output --> Directory["打开 Job 目录"] Output --> Directory["打开 Job 目录"]
``` ```
调试入口读取 Scene 中的输入路径、设备与模型参数,直接调用对应 Server Job。任务完成后只打开 Job 目录供检查,不把结果导入为 Blender Image、Mesh 或节点资产。因此它适合验证模型、设备和输出格式,不代表正式功能的用户流程。 DA3、MoGe-2、BEN2 和 Upscale 都走这条调试路径。它验证模型、设备、请求参数和输出格式,但不代表正式功能的对象创建流程。
## 节点说明
### 从 Debug 面板进入
入口显示在 `src/anyimage/panel.py`,四个 Operator 定义在 `operators/debug.py`:`DebugDa3`、`DebugMoge2`、`DebugBen2` 和 `DebugUpscale`。它们读取 `Scene.anyimage_settings` 中的输入路径、设备、模型和分辨率参数,并检查 Environment 与模型是否可用。
### 提交原始推理
`debug.py` 分别提交 `debug-da3`、`debug-moge2`、`debug-ben2` 和 `debug-upscale` Job。Server 侧由 `server/jobs/debug_predictions.py` 接收请求,再调用相应模型适配器。
DA3 和 MoGe-2 的可视化与数组输出由 `server/outputs/debug_da3.py`、`server/outputs/debug_moge2.py` 写出;BEN2 与 Upscale 保留模型返回的图片结果。这里允许 DA3 继续存在,但它不会被 Plane、Depth Plane 或 Cutout 的正式 Operator 调用。
### 查看结果
任务完成后,`debug.py` 的 `_open_job_directory()` 调用 Blender 的路径打开操作,直接显示该 Job 的目录。响应阶段不导入 Image,不创建 Mesh、Material 或 Modifier,也不修改当前场景。

View File

@@ -1,48 +1,45 @@
# Image Tool # Image Tool
`operators/image_tool/` 对应工具栏中的 Box、Lasso、Line 与 Polyline。四个 WorkspaceTool 共用一个 `EditImageMask` Operator:手势先形成图片空间 Region,Region 栅格化为 Mask 后才修改图片 Alpha;只有 Box 与 Lasso 可以用 BEN2 执行 Refine Selection。 Image Tool 从 Box、Lasso、Line 或 Polyline 手势得到一个图片 Region,再用它修改当前 Image Empty 的 Alpha。
## 对应文件 ## 执行过程
```text
operators/image_tool/
├── __init__.py # 类型清单与对外导出
├── box.py # Box WorkspaceTool
├── lasso.py # Lasso WorkspaceTool
├── line.py # Line WorkspaceTool
├── polyline.py # Polyline WorkspaceTool
└── edit_mask.py # 本地编辑与 Refine Job 响应
```
手势、坐标和像素能力由 `operators/common/` 提供,四点矫正则属于独立的 [`operators/perspective_tool/`](perspective-tool.md)。
## 编辑流程
```mermaid ```mermaid
flowchart TD flowchart TD
Tool["Box / Lasso / Line / Polyline"] --> Gesture["ImageRegionGesture"] Tool["Box / Lasso / Line / Polyline"] --> Gesture["采集图片空间手势"]
Gesture --> Region["RegionPath"] Gesture --> Region["RegionPath → RegionMask"]
Region --> Raster["栅格化 Region"] Region --> Refine{"Fit to Foreground?"}
Raster --> Data["RegionMask"] Refine -- "否" --> Mask["使用本地 RegionMask"]
Data --> Refine{"Refine Selection?"} Refine -- "是" --> Job["refine-image-region"]
Refine -- "否" --> Mask["Region Mask"]
Refine -- "是" --> Input["带 Region Alpha 的裁切 RGBA"]
Input --> Job["refine-image-mask"]
Job --> Mask Job --> Mask
Mask --> Alpha["源 Alpha × Mask"] Mask --> Alpha["源 Alpha × RegionMask"]
Alpha --> Crop["紧裁切并写回 Empty"] Alpha --> Crop["紧裁切并写回 Empty"]
``` ```
`EditImageMask` 只接受静态图片。`ImageRegionGesture` 把屏幕路径投射为 `RegionPath`;`rasterize_region_path()` 再生成 `RegionMask`。未启用 Refine 时直接把 Region Mask 乘入源 Alpha;启用后,Operator 将 `RegionMask` 写成局部 RGBA 并提交 Job,响应阶段读取细化后的 Alpha 与 `bounds`,再映射回完整画布。 四个工具共用 `EditImageMask`。Box 和 Lasso 可以执行 Fit to Foreground;Line 与 Polyline 始终保留几何边界,直接使用本地 `RegionMask`。
Box 和 Lasso 分别保存自己的 Refine Selection 状态。Line 与 Polyline 强制关闭 Refine,只使用本地 Region Mask;Line 将有向线扩展为半平面 Path,Polyline 显式闭合用户确认的 Path。 ## 节点说明
## 手势与预览 ### 选择工具并采集手势
四个工具使用相同的白色虚线轮廓和半透明填充。Lasso 按移动距离采点,并在线合并近似共线点;普通凹多边形使用三角剖分,自交路径使用 EVEN_ODD 扫描填充。预览缓存只在轨迹变化时更新。 `operators/image_tool/__init__.py` 汇总注册类型;`box.py`、`lasso.py`、`line.py` 和 `polyline.py` 分别定义四个 WorkspaceTool。它们都把操作交给 `edit_mask.py` 中的 `EditImageMask`。
首次点击通过公共选择逻辑决定是切换 Active Image Empty,还是开始编辑当前图片。这样工具不需要替代 Blender 的普通对象选择。 屏幕坐标投射、点击选择、轨迹预览和状态切换来自 `operators/common/image_interaction.py`。Lasso 按移动距离采点并合并近似共线点;Line 扩展为有向半平面;Polyline 在确认时闭合路径。
## 写回结果 ### RegionPath → RegionMask
最终像素通过公共 `image_pixels()` / `image_rgba()` 读取,保留源 RGB,只修改 Alpha,再按非零 Alpha 紧裁切。公共像素写回函数创建结果后,通过统一的 Image Empty 替换入口处理共享引用,并更新 `empty_display_size` 与 `empty_image_offset`,因此剩余像素继续对齐原图片平面,对象 `matrix_world` 不变。 `EditImageMask` 通过 `operators/common/image_region.py` 把手势整理为 `RegionPath`,再栅格化为 `RegionMask`。凹多边形使用三角剖分,自交路径使用 `EVEN_ODD` 填充。预览缓存只在轨迹变化时更新。
Box 和 Lasso 分别读取自己的 **Fit to Foreground** 设置。Line 与 Polyline 强制关闭该选项,因此不会改变直线边界。
### Fit to Foreground
未开启 Fit to Foreground 时,`edit_mask.py` 直接使用本地 `RegionMask`。开启后,`RefineImageRegion` 把带 Region Alpha 的局部 RGBA 提交给 `server/jobs/refine_image_region.py`;Job 调用 `server/region/refine_region.py` 运行 BEN2,并返回收紧后的图片和 `bounds`。
响应阶段仍由 `edit_mask.py` 把局部结果映射回完整画布。
### 源 Alpha × RegionMask
`edit_mask.py` 保留源 RGB,只把最终 `RegionMask` 乘入源 Alpha,再按非零 Alpha 紧裁切。像素读取、结果 Image 创建和共享引用处理使用 `operators/common/image_data.py`。
公共 Image Empty 替换入口会同步更新 `empty_display_size` 和 `empty_image_offset`,因此裁切后的内容仍对齐原图片平面,对象 `matrix_world` 不变。Perspective 使用相同的像素写回能力,但属于独立工具和状态。

View File

@@ -1,32 +1,40 @@
# Perspective Tool # Perspective Tool
`operators/perspective_tool/` 对应独立的 Image Perspective WorkspaceTool。它采集四个图片角点,把透视四边形重采样为指定宽高比的矩形,不使用 Job Server。 Perspective Tool 从图片上的四个角点建立透视四边形,把其中内容重采样为指定宽高比的矩形并写回当前 Image Empty。
## 对应文件 ## 执行过程
```text
operators/perspective_tool/
├── __init__.py # 类型清单与对外导出
└── edit_perspective.py # 四点交互、预览、Homography 与写回
```
图片坐标投射、Alpha 裁切和 Empty 写回复用 `operators/common/` 的基础能力;四点数据始终称为 Quad,不与 Image Region 或 Image Mask Operator 共享业务状态。
## 矫正流程
```mermaid ```mermaid
flowchart TD flowchart TD
Click["任意顺序的四个角点"] --> Quad["规范为凸四边形"] Click["点击四个角点"] --> Quad["规范为凸四边形"]
Quad --> Preview["生成一次 512 × 512 预览"] Quad --> Preview["生成 512 × 512 预览"]
Preview --> Ratio["鼠标调整目标宽高比"] Preview --> Ratio["调整目标宽高比"]
Ratio --> Size["计算受限输出尺寸"] Ratio --> Size["计算受限输出尺寸"]
Size --> Warp["全分辨率 Homography"] Size --> Warp["全分辨率 Homography"]
Warp --> Crop["按可见 Alpha 紧裁切"] Warp --> Crop["按可见 Alpha 紧裁切"]
Crop --> Write["Pack 并写回当前 Empty"] Crop --> Write["Pack 并写回 Empty"]
``` ```
四点输入顺序不决定输出方向;规范化阶段依据源图 X/Y 方向得到不交错的凸四边形。第四点之后只创建一次低分辨率纹理,交互阶段改变 UI 宽高比,确认时才执行全分辨率重采样。 预览只在第四点确认后生成一次;之后拖动鼠标只改变目标宽高比。用户确认时才执行全分辨率 Warp。整个流程在 Blender 进程内完成,不使用 Job Server。
Homography 使用预乘 Alpha 插值,避免透明边缘混入黑色。输出最长边限制为 8192 px,总像素限制为 16,777,216,并按 256 行分块计算,控制中间数组大小。 ## 节点说明
Perspective 是 deform 工具,不提供 Refine 或反选。它通过公共 `image_pixels()` 读取供预览与最终 Warp 共用的源像素,并与 Image Mask 共用像素结果创建和 Image Empty 替换入口。结果比例可能不同于原四边形的轴对齐范围,因此 Empty 写回保持中心和覆盖尺度近似一致,而不是假设当前 3D View 相机等于图片拍摄相机。 ### 采集并规范四个角点
`operators/perspective_tool/__init__.py` 导出注册类型,交互和计算集中在 `edit_perspective.py`。`ImagePerspectiveTool` 启动 `EditImagePerspective`,后者接受任意顺序的四个图片角点,再根据源图 X/Y 方向整理成不交错的凸 Quad。
图片坐标投射和首次点击选择复用 `operators/common/image_interaction.py`。Perspective 始终把四点称为 Quad,不使用 Image Tool 的 Region 或 Mask 状态。
### 预览并调整比例
第四点确定后,`edit_perspective.py` 创建一次 512 × 512 预览纹理。鼠标移动只更新目标宽高比和 HUD,可吸附到常用比例,不会重复重采样源图。
输出尺寸根据 Quad 边长和目标比例计算,最长边不超过 8192 px,总像素不超过 16,777,216。
### 执行 Homography
确认后,`edit_perspective.py` 计算 Homography,并按 256 行分块处理全分辨率像素。插值使用预乘 Alpha,避免透明边缘混入黑色。Perspective 只做形变,不提供 Fit to Foreground 或反选。
### 裁切并写回
Warp 结果按可见 Alpha 紧裁切,再通过 `operators/common/image_data.py` 创建 Packed Image 并替换当前 Empty。结果宽高比可能不同于原 Quad 的轴对齐范围,因此写回时保持中心和覆盖尺度近似一致,而不是复用原图片像素范围。

View File

@@ -1,31 +1,37 @@
# Remove Background # Remove Background
本页对应 `src/anyimage/operators/remove_background.py`。该模块负责从 Image Empty 准备媒体输入、提交 BEN2 去背任务,并用结果 Image 替换当前图像;模型推理留在 Server 进程。 Remove Background 从当前 Image Empty 准备单图或多帧输入,运行 BEN2 生成前景 Alpha,再用结果 Image 替换原图。
## 对应文件 ## 执行过程
```text
src/anyimage/
├── operators/
│ ├── remove_background.py # Blender 入口、任务参数与结果替换
│ └── common/image_data.py # 媒体输入准备与结果 Image 创建
└── server/
├── jobs/remove_background.py # 单图与多帧任务编排
└── models/ben2.py # BEN2 前景适配
```
## 主流程
```mermaid ```mermaid
flowchart TD flowchart TD
Empty["Image Empty"] --> Prepare["准备图片、视频或序列输入"] Empty["Image Empty"] --> Prepare["准备图片、视频或序列输入"]
Prepare --> Operator["RunBackgroundRemoval"] Prepare --> Job["remove-background Job"]
Operator --> Job["remove-background Job"] Job --> Frames["逐帧运行 BEN2"]
Job --> Frames["生成 PNG 结果序列"] Frames --> Result["写出 PNG 结果序列"]
Frames --> Import["创建结果 Image"] Result --> Import["创建结果 Image"]
Import --> Replace["替换 Empty.data"] Import --> Replace["替换 Empty.data"]
``` ```
Packed 图片或内存中修改过的单图会先导出为临时输入;文件图片、视频和序列则沿用可读取的源路径。多帧输入由统一媒体层展开,结果保持连续帧命名,导入后继续沿用源 Image User 的帧起点、偏移与循环设置。 Blender 负责媒体输入准备和结果导入,BEN2 推理留在 Server。Packed 图片和内存中修改过的单图会先导出临时文件;可直接读取的文件图片、视频和序列沿用原路径。
BEN2 推理前使用输入 Alpha 屏蔽透明区域,模型输出的前景 Alpha 再与源 Alpha 合成,RGB 保持为源图内容。响应阶段通过公共 Image Empty 替换入口导入结果;当旧 Image 没有其他用户和 Fake User 时,旧数据块会被清理并由结果继承名称。 ## 节点说明
### 准备媒体输入
入口是 `operators/remove_background.py` 中的 `RemoveImageBackground`。它检查当前对象、AI Environment 和帧数,再调用 `operators/common/image_data.py` 的 `prepare_image_input()`。多帧输入会先写入 Scene 中的抽帧设置。
准备完成后,内部 Operator `RunBackgroundRemoval` 保存输入路径、源对象名和临时文件清理标记。
### 运行 BEN2
`RunBackgroundRemoval` 提交 `remove-background` Job。`server/jobs/remove_background.py` 使用公共媒体输入层展开单图、视频或序列,并逐帧调用 `server/models/ben2.py`。
BEN2 推理前先用源 Alpha 屏蔽透明区域,模型输出的前景 Alpha 再与源 Alpha 合成,RGB 保持源图内容。多帧结果按连续文件名写入 Job 目录。
### 导入并替换结果
Job 返回后,`remove_background.py` 的 `apply_background_response()` 调用 `operators/common/image_data.py` 创建单图或序列 Image,并恢复源 Image User 的帧起点、偏移和循环设置。
同一公共入口随后替换 `Empty.data`。旧 Image 没有其他用户且未启用 Fake User 时会被清理,结果继承原名;否则保留旧数据块。最后提交一个 Remove Background Undo 步骤,并清理本次创建的临时输入。

View File

@@ -1,37 +1,44 @@
# Upscale # Upscale
本页对应 `src/anyimage/operators/upscale.py`。该模块读取扩展首选项中的模型与全局 AI 输入长边上限,检查当前 Image 是否可处理,再提交 4× 高清任务。 Upscale 从当前 Image Empty 读取单图或多帧媒体,在输入长边允许时生成 4× RGBA 结果并替换原图。
## 对应文件 ## 执行过程
```text
src/anyimage/
├── operators/
│ ├── upscale.py # Blender 入口、任务参数与结果替换
│ └── common/image_data.py # 媒体输入准备与结果 Image 创建
└── server/
├── jobs/upscale.py # 单图与多帧任务编排
├── media/resolution.py # 共用长边限制与等比缩放
├── models/upscale.py # 输入约束与 Alpha 合并
└── models/onnx_upscale.py # 分块 4× ONNX 推理
```
## 主流程
```mermaid ```mermaid
flowchart TD flowchart TD
Empty["Image Empty"] --> Source["读取当前 Image"] Empty["Image Empty"] --> Gate{"长边小于输入上限?"}
Source --> Gate{"当前长边小于全局上限?"} Gate -- "否" --> Disabled["禁用或报告错误"]
Gate -- "否" --> Disabled["Upscale 置灰"]
Gate -- "是" --> Prepare["准备媒体输入"] Gate -- "是" --> Prepare["准备媒体输入"]
Prepare --> Job["upscale-image Job"] Prepare --> Job["upscale-image Job"]
Job --> RGB["分块执行 4× RGB 推理"] Job --> RGB["分块执行 4× RGB 推理"]
Job --> Alpha["独立缩放 Alpha"] Job --> Alpha["独立缩放 Alpha"]
RGB --> Merge["合并 RGBA 结果"] RGB --> Merge["合并 RGBA"]
Alpha --> Merge Alpha --> Merge
Merge --> Replace["导入并替换当前 Image"] Merge --> Replace["导入并替换当前 Image"]
``` ```
模型以 256×256 分块处理 RGB,并带入相邻上下文以减弱块边界;Alpha 不进入模型,使用 Lanczos 独立放大后合并为 RGBA。`max_input_size` 来自全局 **Maximum AI Input Size**,默认 `2048`。Operator 只允许当前 Image 长边小于该值时执行,因此正式流程不会预缩放输入,4× 输出长边小于上限的四倍。Server 仍保留相同上限作为任务边界。 模型只处理 RGB,Alpha 单独放大后再合并。Blender 侧在提交前检查 **Maximum AI Input Size**,Server 模型层也应用同一个输入上限。
菜单根据当前尺寸只显示预计的 4× 结果尺寸。结果通过与 Remove Background、Image Mask 和 Perspective 相同的 Image Empty 替换入口处理:原 Image 没有其他用户且未启用 Fake User 时,移除旧数据并让结果沿用原名;存在其他用户时保留原 Image,并只让当前 Empty 使用独立结果。结果不增加 Upscale 专属后缀,也不保存来源引用。后续 Upscale 与 Depth Plane 都读取当前 Image。模型与全局输入上限在每次执行时从 Preferences 读取,再写入可序列化的隐藏属性交给 Job。Provider 资源错误会释放当前 Upscale Session,并提示降低全局输入上限或改用 CPU。 ## 节点说明
### 检查尺寸并准备输入
入口位于 `operators/upscale.py`。`UpscaleImage.poll()` 读取 `preferences.py` 中的 **Maximum AI Input Size**;当前 Image 长边必须严格小于该值。菜单显示的预计尺寸来自同一文件的 `upscale_output_size()`。
执行时,`upscale.py` 冻结当前模型和输入上限,并通过 `operators/common/image_data.py` 准备单图、视频或序列输入。Packed Image 与内存修改结果会导出临时文件,多帧输入保留帧起点和抽帧设置。
### 分块放大 RGB
`UpscaleImage` 提交 `upscale-image` Job。`server/jobs/upscale.py` 展开媒体帧并逐帧调用模型;`server/models/upscale.py` 负责输入约束、RGB/Alpha 分离和结果合并,`server/models/onnx_upscale.py` 以 256 × 256 分块执行 4× ONNX 推理并带入相邻上下文。
`server/media/resolution.py` 统一解释输入长边上限。正式 Upscale 在 Blender 侧直接禁用超限图片,因此不会进入预缩放路径,输出长边小于上限的四倍;模型层仍保留同一限制作为任务边界。
### 合并 Alpha
Alpha 不进入模型,由 `server/models/upscale.py` 使用 Lanczos 放大到目标尺寸,再与 4× RGB 合并为 RGBA。Provider 资源不足时,Server 释放当前 Upscale Session,并返回降低输入上限或改用 CPU 的提示。
### 导入并替换结果
响应阶段回到 `operators/upscale.py`,通过 `operators/common/image_data.py` 读取 `upscale.*` 结果序列并替换当前 Empty。源 Image 没有其他用户且未启用 Fake User 时会被清理并由结果继承名称;存在其他用户时只替换当前 Empty 的引用。
结果不添加 Upscale 专属数据块后缀,也不保留来源引用。后续再次 Upscale 或转换 Depth Plane 时,读取的都是当前结果 Image。

View File

@@ -1,72 +1,51 @@
# Job 产物 # Job 产物
Job 目录是 Server 与 Blender Operator 的交换边界。Server 写入普通图片、JSON 或 NPZ;Blender 只根据 Job 返回值读取需要的文件,并创建最终数据块。 Job 目录是 Server 与 Blender Operator 的交换边界。Server 写入普通图片、JSON 或 Debug NPZ;Blender 只读取 Job 返回值明确声明的文件,再创建最终 Image、Mesh、Material 与 Object。
## 产物流向
```mermaid
flowchart TD
Input["媒体输入"] --> Job["Server Job"]
Job --> Color["PNG 颜色 / Alpha"]
Job --> Field["NPZ 深度场"]
Job --> Meta["JSON bounds"]
Color --> Import["Blender response()"]
Field --> Import
Meta --> Import
Import --> Data["Image / Mesh / Material"]
```
## 正式 Job ## 正式 Job
| Job | 主要文件 | 消费方 | | Job | 主要文件 | Blender 消费方式 |
| --- | --- | --- | | | | |
| `remove-background` | `image.0001.png` 起 | 替换 Image Empty 的颜色数据 | | `remove-background` | `image.0001.png` 起 | 替换 Image Empty 的颜色数据 |
| `upscale-image` | `upscale.0001.png` 起 | 按共享引用情况替换当前 Image Empty 的颜色数据,并沿用原 Image 命名语义 | | `upscale-image` | `upscale.0001.png` 起 | 替换或新建高清颜色数据 |
| `refine-image-mask` | `image.png`、`bounds.json` | Image Box / Lasso 读取细化 Alpha 与原图坐标 | | `refine-image-region` | `image.png`、`bounds.json` | Image Box / Lasso 读取 Fit to Foreground 结果 |
| `generate-depth-plane` | `image.0001.*`、`image_geometry.npz` | 创建颜色图、深度图、材质和 Plane | | `generate-depth-plane-geometry` | `image.0001.*`、`z-depth.exr`、`depth.json` | 创建 Depth Plane |
| `generate-region-data` | 可选的 `region.png`、`region_bounds.json`、`image_geometry.npz`、`normal.png` | Blender 主线程构建 Cutout Shape 与贴图 | | `generate-cutout-artifacts` | 可选 Region、Depth 与 Normal 文件 | 创建 Cutout Shape 与材质 |
两个几何 Job 都不输出 Blender 网格。`generate-depth-plane` 生成完整图片的颜色副本与深度场;`generate-region-data` 只生成请求的 AI 产物。轮廓提取、约束三角化以及 Mesh、Material 和 Geometry Nodes 创建都在 Blender 主线程完成。 两个 Depth Job 都不输出 Blender 网格。Server 保存可由 Blender 直接加载的模型空间 EXR;轮廓提取、三角化、Depth 标定以及 Blender 数据创建仍在主线程完成。
## 深度场 ## Depth 与 Normal 文件
`image_geometry.npz` 是 Depth Plane、Depth Balloon 与 Depth Surface 共用的模型中间结果。Blender 侧统一通过 `operators/common/mesh_result.py` 的 `load_image_geometry()` 读取为 `ImageGeometry`,功能模块不分别解释字段: MoGe-2 Prediction 只存在于 Server 内存。`geometry/prediction_artifacts.py` 共享一次推理,并按请求写出以下文件:
| 字段 | 含义 | | 文件 | 内容 |
| --- | --- | | --- | --- |
| `points` | 每个模型像素的相机空间 XYZ | | `vector-depth.exr` | RGB 保存相机空间 XYZ,Alpha 保存有效区 |
| `depth` | 相机深度 | | `z-depth.exr` | RGB 重复保存相机 Z,Alpha 保存有效区 |
| `intrinsics` | 像素坐标下的 3×3 相机内参 | | `depth.json` | 保存 `image_size`、3×3 `intrinsics` 与 `reference_depth` |
| `valid` | 模型与数值检查后的有效区域 | | `tangent-normal.png` | Tangent Space Normal Map |
| `calibration_valid` | 用于结果定位或深度基准的区域;Cutout 时与 Region 相交 | | `object-normal.png` | Object Space Normal Map |
| `image_size` | 深度场宽高 |
| `reference_depth` | 校准区域中的参考深度 |
Server 保留模型空间数据,不提前换算 Blender 单位。Depth Plane 在导入时根据 Image Empty 平面范围和相机内参锚定点场;Cutout 深度形状则计算 `Uniform Scale` 和局部坐标,再把相机点写入 Packed 浮点 Image 供 Geometry Nodes 采样。 Blender 通过 `operators/common/depth_data.py` 直接加载 EXR,设为 Non-Color 并 Pack;不会从 NumPy 点场再创建一张 Depth Image。Job 文件名不改变 Blender 数据块命名:Depth 与 Normal 仍分别使用 `depth_data_name()` 和 `normal_data_name()`。Blender 读取 Z 与 Alpha 只用于最终 Region 的本地标定,Geometry Nodes 直接采样已加载的 EXR。
## Cutout Region 产物 ## Cutout Region 产物
Blender 侧先把 `RegionPath` 栅格化为 `RegionMask`,再将 Region 裁切为带有效 Alpha 的临时 RGBA 输入。Server 不接收路径或 Mask 协议,只按需产生 AI 数据: Blender 先把 `RegionPath` 栅格化为 `RegionMask`,再把局部 RGBA 作为 Server 输入。Server 不接收 Path 或 Mask 对象协议:
- 启用 Refine 时,`region.png` 保存细化后的局部 RGBA,`region_bounds.json` 保存它在输入裁切图中的范围; - 开启 Fit to Foreground 时,`region.png` 保存 Fit to Foreground 后的局部 RGBA,`region_bounds.json` 保存它相对输入裁切图的范围;
- Depth Shape 使用 `image_geometry.npz`; - 需要 Vector Depth 时请求 `vector-depth.exr` 与 `depth.json`;
- Tangent 或 Object Normal 使用 `normal.png`。 - 需要 Z Depth 时请求 `z-depth.exr` 与 `depth.json`;
- Normal Map 按空间请求 `tangent-normal.png` 或 `object-normal.png`。
Blender 将局部范围与原始 `RegionMask.bounds` 合成回源图坐标,并直接从源 Image 创建最终 Color Image。未启用 Refine、Depth 或 Normal 时不提交 Job。 未启用 Fit to Foreground,且不需要 Depth 或 Normal Map 时,Surface 与 Balloon 不提交 Job。最终颜色图仍由 Blender 从源 Image 裁切,并把 Alpha 乘以最终 `RegionMask`。
## Debug 产物 ## Debug 产物
Debug Job 保留模型适配层的直接输出,不执行正式功能的 Blender 导入:
| Job | 文件 | | Job | 文件 |
| --- | --- | | | |
| `debug-da3` | `prediction.npz` | | `debug-da3` | `prediction.npz` |
| `debug-moge2` | `prediction*.npz` | | `debug-moge2` | `prediction*.npz` |
| `debug-ben2` | `foreground*.png` | | `debug-ben2` | `foreground*.png` |
| `debug-upscale` | `prediction*.png` | | `debug-upscale` | `prediction*.png` |
这些文件用于验证模型、Device 和适配层。正式 Job 会继续派生 Region、Alpha、深度场或可导入图片,因此两者的文件结构有意不同。 Debug Job 只用于直接检查模型适配层,不参与正式对象创建流程。
## 单图与序列
正式颜色序列从 `image.0001.*` 开始,Upscale 使用独立的 `upscale.0001.png` 前缀。Remove Background 与 Upscale 的动画结果继续引用 Job 目录;单图结果通常在 Blender 导入后 Pack 到 `.blend`。不要清理仍被工程引用的序列目录。

View File

@@ -1,26 +1,28 @@
# Job Runtime 与 Server # Job Runtime 与 Server
AnyImage 使用 BlendJob 管理独立 Python Environment、Server 进程和任务队列。Blender 侧只配置 Runtime 与提交参数;`server/` 是不依赖 `bpy` 的计算包,可以在独立解释器中导入。 AnyImage 使用 BlendJob 管理独立 Python Environment、Server 进程与单 Worker 队列。Blender 侧只准备输入、提交普通字典参数并导入结果;`server/` 不依赖 `bpy`。
## 对应文件 ## 目录职责
```text ```text
src/anyimage/ src/anyimage/
├── runtime.py # Environment 与 JobRuntime 配置 ├── runtime.py # Environment 与 JobRuntime 配置
└── server/ └── server/
├── app.py # Job 注册和共享资源 ├── app.py # 公开 Job 注册
├── model_catalog.py # 模型声明 ├── model_catalog.py # 模型声明
├── model_download.py # 下载、校验与镜像回退 ├── model_download.py # 下载与校验
├── model_manager.py # 模型目录与 Session 缓存 ├── model_manager.py # 模型目录与 Session 生命周期
├── media/input.py # 图片、视频和序列输入 ├── jobs/ # 公开协议与流程编排
├── jobs/ # app.py 可提交的公开任务 ├── media/ # 图片、视频、序列帧和尺寸处理
├── region/ # BEN2 Region 细化 ├── region/ # 通用 BEN2 Region 细化
├── geometry/ # Depth / Normal 文件产物与相机元数据
├── models/ # 模型适配与 ONNX 推理 ├── models/ # 模型适配与 ONNX 推理
├── geometry/ # 深度反投影与法线纹理编码
└── outputs/ # Debug Prediction 序列化 └── outputs/ # Debug Prediction 序列化
``` ```
## 任务生命周期 公开 Job 不互相导入实现。`jobs/` 只编排,稳定的原子能力放在对应领域目录中。
## 生命周期
```mermaid ```mermaid
flowchart TD flowchart TD
@@ -32,56 +34,51 @@ flowchart TD
Directory --> Response["Operator response()"] Directory --> Response["Operator response()"]
``` ```
Runtime 在首次任务或显式启动时拉起 Server。任务在队列中串行执行,进度、取消和日志由 BlendJob 提供;AnyImage Job 在阶段边界调用 `context.check_cancelled()`,并把业务产物写入当前 Job 目录。Operator 完成响应后再清理 Blender 侧为 Packed Image 准备的临时输入。 Job 在阶段边界调用 `context.check_cancelled()`,并把稳定产物写入当前 Job 目录。Blender 响应完成后,再由输入所有者清理临时 Packed Image 文件。
`server/app.py` 是任务清单的唯一入口: ## 公开 Job
| Job | 主要用途 | | Job | 主要用途 |
| --- | --- | | --- | --- |
| `download-model`、`download-required-models` | 准备模型文件 | | `download-model`、`download-required-models` | 准备模型文件 |
| `remove-background` | BEN2 透明前景 | | `remove-background` | 生成 BEN2 透明前景 |
| `upscale-image` | 4× 图片或帧序列 | | `upscale-image` | 4× 图片或帧序列 |
| `refine-image-mask` | 为 Image Box / Lasso 细化 Region | | `refine-image-region` | 为 Image Box / Lasso 执行 Fit to Foreground |
| `generate-depth-plane` | 为 Depth Plane 生成颜色副本与深度场 | | `generate-depth-plane-geometry` | 为 Depth Plane 复制颜色并生成 Z Depth 与相机元数据 |
| `generate-region-data` | 按需为 Cutout Region 生成 Refine、Depth 或 Normal | | `generate-cutout-artifacts` | 按需执行 Fit to Foreground,并生成 Depth 或 Normal 文件产物 |
| `debug-*` | 保存模型适配层的直接输出 | | `debug-*` | 保存模型适配层的直接输出 |
## 媒体输入 `generate-depth-plane-geometry` 对应 `jobs/depth_plane.py`;`generate-cutout-artifacts` 对应 `jobs/cutout.py`。共享的 MoGe-2 推理编排位于 `geometry/prediction_artifacts.py`,Depth EXR 与元数据写入位于 `geometry/depth_texture.py`,Normal 编码位于 `geometry/normal_texture.py`,Fit to Foreground 的通用实现位于 `region/refine_region.py`。
`server/media/input.py` 把单图、视频和图片目录统一为有序帧路径,并用上下文管理器负责临时帧目录的生命周期。帧展开是媒体输入的内部能力,不再暴露没有业务消费者的独立 Job。`server/media/resolution.py` 提供默认 `2048` 的共用输入长边限制;Upscale Operator 只允许长边小于该值的当前 Image 进入正式任务,Depth Plane 与 Cutout 则只限制模型分析输入,不降低颜色贴图分辨率。 ## 尺寸约束
`generate-depth-plane` 与 `generate-region-data` 只接受一帧,因此 Depth Plane 与 Cutout 都是静态图工作流。Remove Background 与 Upscale 通过同一媒体入口处理单图、视频和序列帧。 `server/media/resolution.py` 提供 **Maximum AI Input Size**,默认 `2048 px`。它限制 Upscale 的可提交源图长边,以及 Depth、Normal Map、Fit to Foreground 的模型分析图长边;不会降低材质颜色图分辨率。
公开 Job 按稳定的计算边界划分,不按界面入口机械映射。Depth Plane 使用 `jobs/generate_depth_plane.py`,Cutout 使用 `jobs/generate_region_data.py`,四个 Debug Job 共享 `jobs/debug_predictions.py`。Job 只负责协议和编排;可复用的输入、AI Region、模型与几何数据能力分别位于 `media/`、`region/`、`models/` 和 `geometry/`,Job 之间不互相导入实现。 Cutout 另有 Blender 侧的 **Maximum Cutout Mesh Resolution**。它限制所有 Cutout Shape 的网格采样密度,包括完全不使用 AI 的 Surface 与 Balloon。两个设置没有隐式绑定。
## 模型层 `generate-depth-plane-geometry` 与 `generate-cutout-artifacts` 只接受一帧,因此 Depth Plane 与 Cutout 是静态图工作流。Remove Background 与 Upscale 继续通过媒体入口处理单图、视频和序列帧。
Job 只编排流程,模型适配层把不同推理 API 统一为内存对象: ## 模型边界
- DA3 适配为深度、置信度、相机内外参与相机点。 正式 Depth Plane、Cutout Depth 与 Normal Map 都只使用 MoGe-2。BEN2 用于 Remove Background 与 Fit to Foreground,Upscale 使用所选高清模型。DA3 只保留 `debug-da3` 及其 Debug 参数、模型下载与适配代码,不进入任何生产 Operator、Job 或自动回退。
- MoGe-2 适配为深度、法线、有效区域、相机内参与可选相机点。
- BEN2 返回带 Alpha 的前景图。
- Upscale 返回保留源 Alpha 的 4× RGBA 图。
`ModelManager` 管理下载目录和 ONNX Session。缓存按 BEN2、几何模型和 Upscale 三个族分开;每个族只保留最近一次使用的模型与 Device,切换同族模型时释放旧 Session。这样连续任务可以复用模型,同时避免同一类模型重复占用资源。 `ModelManager` 按模型族管理下载目录与 ONNX Session,同一族只保留最近使用的模型和 Device,避免同类模型重复占用资源。
## Environment 与 Storage Root ## Environment 与 Storage Root
`runtime.py` 声明 Python 3.12 环境和固定版本的 NumPy、Pillow、PyAV、trimesh、Hugging Face Hub 与 ONNX Runtime。Windows 安装 DirectML 版本,其他平台安装标准 ONNX Runtime。首次安装后依次下载默认 MoGe-2、BEN2 和 Real-ESRGAN WDN 模型。 `runtime.py` 声明独立 Python 3.12 环境和固定依赖。首次安装会准备默认 MoGe-2、BEN2 与 Real-ESRGAN WDN 模型。Storage Root 默认是 `~/.anyimage`:
Storage Root 默认是 `~/.anyimage`:
```text ```text
~/.anyimage/ ~/.anyimage/
├── .venv/ # 独立 Environment ├── .venv/
├── models/ # 模型目录 ├── models/
├── jobs/ # Job 产物与 job.log ├── jobs/
├── environment.json ├── environment.json
├── manifest.json ├── manifest.json
├── install.log ├── install.log
└── server.log └── server.log
``` ```
Storage Root 由扩展偏好设置提供给 Runtime。切换路径等同于切换整套 Environment、模型和任务数据,不会自动迁移旧目录。 切换 Storage Root 等同于切换整套 Environment、模型与任务数据,不自动迁移旧目录。
各 Job 的稳定产物见 [Job 产物](outputs.md)。 各 Job 的稳定产物见 [Job 产物](outputs.md)。

View File

@@ -6,7 +6,7 @@
| 入口 | 位置 | 输入 | | 入口 | 位置 | 输入 |
| --- | --- | --- | | --- | --- | --- |
| Paste Clipboard Image | 3D View / Node Editor 的 `Ctrl+V` 或 `Cmd+V` | 剪贴板静态图 | | Paste Clipboard Image | 3D View / oode Editor 的 `Ctrl+V` 或 `Cmd+V` | 剪贴板静态图 |
| Convert to Plane | Image Empty 右键菜单 | 单图、Movie、Sequence | | Convert to Plane | Image Empty 右键菜单 | 单图、Movie、Sequence |
| Convert to Depth Plane | Image Empty 右键菜单 | 静态图 | | Convert to Depth Plane | Image Empty 右键菜单 | 静态图 |
| Remove Background | Image Empty 右键菜单 | 单图、Movie、Sequence | | Remove Background | Image Empty 右键菜单 | 单图、Movie、Sequence |
@@ -15,7 +15,7 @@
| Image Perspective | 3D Viewport 工具栏 | 静态 Image Empty | | Image Perspective | 3D Viewport 工具栏 | 静态 Image Empty |
| Cutout | 3D Viewport 工具栏 | 静态 Image Empty | | Cutout | 3D Viewport 工具栏 | 静态 Image Empty |
右键菜单只在当前对象是 Image Empty 时出现。AI 功能会在 Environment 或默认模型未就绪时转入安装流程;普通 Plane、非 Refine 的 Image Tools 与基础 Cutout Surface / Balloon 不依赖 Environment。 右键菜单只在当前对象是 Image Empty 时出现。AI 功能会在 Environment 或默认模型未就绪时转入安装流程;普通 Plane、未启用 Fit to Foreground 的 Image Tools 与基础 Cutout Surface / Balloon 不依赖 Environment。
## 结果类型 ## 结果类型
@@ -34,7 +34,7 @@ Cutout 提供四种结果:
| Depth Balloon | `O Image Depth Balloon` | MoGe-2 高度 | | Depth Balloon | `O Image Depth Balloon` | MoGe-2 高度 |
| Depth Surface | `O Image Depth Surface` | MoGe-2 相机点 | | Depth Surface | `O Image Depth Surface` | MoGe-2 相机点 |
四种 Shape 共用 `O Image Cutout` 修改器、规则三角基础面、UV 和 `o_balloon` 属性。Shape 菜单选择内部几何分支,Cleanup 在分支前执行且使用便于调节的百倍显示值,Smooth 在分支后执行;Depth Shape 根据 Region 透明比例选择固定或自适应 Base Plane Depth,Normal 可选 None、Tangent 或 Object,Refine Selection 按需启用 BEN2。 四种 Shape 共用 `O Image Cutout` 修改器、规则三角基础面、UV 和 `o_balloon` 属性。Shape 菜单选择内部几何分支,Cleanup 在分支前执行且使用便于调节的百倍显示值,Smooth 在分支后执行;Depth Shape 根据 Region 透明比例选择固定或自适应 Base Plane Depth,Generate Normal Map 是启用开关:Depth Surface 自动使用 Object Space,其余 Shape 自动使用 Tangent Space;Fit to Foreground 按需启用 BEN2。
### 图片结果 ### 图片结果
@@ -43,7 +43,6 @@ Image Mask 和 Perspective 直接更新当前 Empty 的图片并 Pack。Remove B
## 模型 ## 模型
- MoGe-2 ViT-S / ViT-B / ViT-L Normal - MoGe-2 ViT-S / ViT-B / ViT-L Normal
- DA3 Small / Base
- BEN2 Base - BEN2 Base
- Real-ESRGAN General WDN x4v3 - Real-ESRGAN General WDN x4v3
- SAFMN Real x4 - SAFMN Real x4

20
openspec/config.yaml Normal file
View File

@@ -0,0 +1,20 @@
schema: spec-driven
# Project context (optional)
# This is shown to AI when creating artifacts.
# Add your tech stack, conventions, style guides, domain knowledge, etc.
# Example:
# context: |
# Tech stack: TypeScript, React, Node.js
# We use conventional commits
# Domain: e-commerce platform
# Per-artifact rules (optional)
# Add custom rules for specific artifacts.
# Example:
# rules:
# proposal:
# - Keep proposals under 500 words
# - Always include a "Non-goals" section
# tasks:
# - Break tasks into chunks of max 2 hours

View File

@@ -0,0 +1,80 @@
"""Load and scale Depth file artifacts shared by Blender workflows."""
import json
from pathlib import Path
import bpy
from .image_data import depth_data_name, image_empty_bounds
def load_depth_metadata(path):
"""Read and validate the compact camera metadata beside a Depth texture."""
metadata = json.loads(Path(path).read_text(encoding="utf-8"))
try:
image_size = tuple(int(value) for value in metadata["image_size"])
intrinsics = tuple(
tuple(float(value) for value in row)
for row in metadata["intrinsics"]
)
reference_depth = float(metadata["reference_depth"])
except (KeyError, TypeError, ValueError) as error:
raise ValueError("Depth metadata is invalid") from error
if (
len(image_size) != 2
or min(image_size) <= 0
or len(intrinsics) != 3
or any(len(row) != 3 for row in intrinsics)
or intrinsics[0][0] <= 0.0
or intrinsics[1][1] <= 0.0
or reference_depth <= 0.0
):
raise ValueError("Depth metadata is invalid")
return {
"image_size": image_size,
"intrinsics": intrinsics,
"reference_depth": reference_depth,
}
def load_depth_result_image(path, source_object, metadata):
"""Load, validate, name, configure, and pack a generated Depth texture."""
image = bpy.data.images.load(str(path), check_existing=False)
try:
if tuple(int(value) for value in image.size) != metadata["image_size"]:
raise ValueError("Depth texture dimensions do not match its metadata")
image.name = depth_data_name(source_object)
image.colorspace_settings.name = "Non-Color"
image.alpha_mode = "CHANNEL_PACKED"
image.pack()
return image
except Exception:
bpy.data.images.remove(image, do_unlink=True)
raise
def camera_depth_values(image):
"""Return top-down camera Z and validity from a loaded Depth texture."""
import numpy as np
width, height = (int(value) for value in image.size)
pixels = np.empty(width * height * 4, dtype=np.float32)
image.pixels.foreach_get(pixels)
pixels = np.flipud(pixels.reshape((height, width, 4)))
return pixels[..., 2], pixels[..., 3] > 0.5
def depth_uniform_scale(source_object, metadata, bounds=None):
"""Convert model camera units uniformly into Image Empty local distance."""
import numpy as np
intrinsics = metadata["intrinsics"]
width, height = metadata["image_size"]
reference_depth = metadata["reference_depth"]
x_min, x_max, y_min, y_max = bounds or image_empty_bounds(source_object)
plane_width = x_max - x_min
plane_height = y_max - y_min
fx, fy = float(intrinsics[0][0]), float(intrinsics[1][1])
scale_x = plane_width * fx / (max(width - 1, 1) * reference_depth)
scale_y = plane_height * fy / (max(height - 1, 1) * reference_depth)
return float(np.sqrt(scale_x * scale_y))

View File

@@ -371,7 +371,7 @@ def force_sequence_source(image):
def load_normal_result_image(path, source_object): def load_normal_result_image(path, source_object):
"""Load, name, configure, and pack a tangent normal image.""" """Load, name, configure, and pack a generated normal image."""
image = bpy.data.images.load(str(path), check_existing=False) image = bpy.data.images.load(str(path), check_existing=False)
image.name = normal_data_name(source_object) image.name = normal_data_name(source_object)
image.colorspace_settings.name = "Non-Color" image.colorspace_settings.name = "Non-Color"

View File

@@ -148,22 +148,16 @@ def active_view3d_tool_id(context):
return getattr(tool, "idname", None) return getattr(tool, "idname", None)
def refine_selection_setting_name(operator): def refine_region_setting_name(operator):
if getattr(operator, "active_tool_id", None) == "anyimage.cutout_lasso": setting_names = getattr(operator, "refine_region_settings", {})
return "cutout_refine_selection" return setting_names.get(operator.gesture)
return {
"BOX": "image_box_refine_selection",
"LASSO": "image_lasso_refine_selection",
}.get(operator.gesture)
def apply_region_settings(operator, context): def apply_region_settings(operator, context):
settings = context.scene.anyimage_settings settings = context.scene.anyimage_settings
setting_name = refine_selection_setting_name(operator) setting_name = refine_region_setting_name(operator)
operator.refine_selection = ( operator.refine_region = (
setting_name is not None setting_name is not None and ai_ready() and getattr(settings, setting_name)
and ai_ready()
and getattr(settings, setting_name)
) )
@@ -174,7 +168,7 @@ def draw_region_operator_settings(layout, operator):
status = ai_status() status = ai_status()
row = layout.row() row = layout.row()
row.enabled = status["ready"] row.enabled = status["ready"]
row.prop(operator, "refine_selection") row.prop(operator, "refine_region")
layout.prop(operator, "invert") layout.prop(operator, "invert")
if not status["ready"]: if not status["ready"]:
layout.operator( layout.operator(
@@ -200,7 +194,9 @@ def draw_region_settings(context, layout, setting_name):
def dashed_line_segments(vertices, dash_length=4.0, gap_length=4.0): def dashed_line_segments(vertices, dash_length=4.0, gap_length=4.0):
if dash_length <= 0.0 or gap_length < 0.0: if dash_length <= 0.0 or gap_length < 0.0:
raise ValueError("Dash length must be positive and gap length cannot be negative") raise ValueError(
"Dash length must be positive and gap length cannot be negative"
)
pattern_length = dash_length + gap_length pattern_length = dash_length + gap_length
segments = [] segments = []
distance_offset = 0.0 distance_offset = 0.0
@@ -259,16 +255,19 @@ def draw_dashed_line(
def _signed_area(vertices): def _signed_area(vertices):
return sum( return (
start[0] * end[1] - end[0] * start[1] sum(
for start, end in zip(vertices, (*vertices[1:], vertices[0])) start[0] * end[1] - end[0] * start[1]
) * 0.5 for start, end in zip(vertices, (*vertices[1:], vertices[0]))
)
* 0.5
)
def _triangle_cross(start, middle, end): def _triangle_cross(start, middle, end):
return (middle[0] - start[0]) * (end[1] - start[1]) - ( return (middle[0] - start[0]) * (end[1] - start[1]) - (middle[1] - start[1]) * (
middle[1] - start[1] end[0] - start[0]
) * (end[0] - start[0]) )
def _point_in_triangle(point, start, middle, end, orientation, epsilon): def _point_in_triangle(point, start, middle, end, orientation, epsilon):
@@ -329,9 +328,7 @@ def triangulated_polygon_vertices(vertices):
try: try:
import mathutils import mathutils
points = [ points = [mathutils.Vector((vertex[0], vertex[1], 0.0)) for vertex in vertices]
mathutils.Vector((vertex[0], vertex[1], 0.0)) for vertex in vertices
]
triangles = mathutils.geometry.tessellate_polygon([points]) triangles = mathutils.geometry.tessellate_polygon([points])
return tuple( return tuple(
(float(point[0]), float(point[1])) (float(point[0]), float(point[1]))
@@ -342,12 +339,12 @@ def triangulated_polygon_vertices(vertices):
return _triangulated_polygon_vertices_python(vertices) return _triangulated_polygon_vertices_python(vertices)
def scanline_fill_triangles(vertices, fill_rule='EVEN_ODD', step=2.0): def scanline_fill_triangles(vertices, fill_rule="EVEN_ODD", step=2.0):
vertices = tuple(tuple(float(value) for value in vertex) for vertex in vertices) vertices = tuple(tuple(float(value) for value in vertex) for vertex in vertices)
if len(vertices) < 3: if len(vertices) < 3:
return () return ()
if fill_rule not in {'EVEN_ODD', 'NONZERO'}: if fill_rule not in {"EVEN_ODD", "NONZERO"}:
raise ValueError(f'Unknown polygon fill rule: {fill_rule}') raise ValueError(f"Unknown polygon fill rule: {fill_rule}")
minimum_y = min(vertex[1] for vertex in vertices) minimum_y = min(vertex[1] for vertex in vertices)
maximum_y = max(vertex[1] for vertex in vertices) maximum_y = max(vertex[1] for vertex in vertices)
band_bottom = minimum_y band_bottom = minimum_y
@@ -358,9 +355,7 @@ def scanline_fill_triangles(vertices, fill_rule='EVEN_ODD', step=2.0):
sample_y = (band_bottom + band_top) * 0.5 sample_y = (band_bottom + band_top) * 0.5
events = [] events = []
for start, end in edges: for start, end in edges:
if (start[1] <= sample_y < end[1]) or ( if (start[1] <= sample_y < end[1]) or (end[1] <= sample_y < start[1]):
end[1] <= sample_y < start[1]
):
factor = (sample_y - start[1]) / (end[1] - start[1]) factor = (sample_y - start[1]) / (end[1] - start[1])
events.append( events.append(
( (
@@ -382,7 +377,7 @@ def scanline_fill_triangles(vertices, fill_rule='EVEN_ODD', step=2.0):
count += 1 count += 1
index += 1 index += 1
was_inside = winding != 0 was_inside = winding != 0
if fill_rule == 'EVEN_ODD': if fill_rule == "EVEN_ODD":
if count % 2: if count % 2:
winding = 0 if was_inside else 1 winding = 0 if was_inside else 1
else: else:
@@ -412,9 +407,7 @@ def scanline_fill_triangles(vertices, fill_rule='EVEN_ODD', step=2.0):
def _point_on_segment(point, start, end, epsilon=1e-8): def _point_on_segment(point, start, end, epsilon=1e-8):
return ( return (
min(start[0], end[0]) - epsilon min(start[0], end[0]) - epsilon <= point[0] <= max(start[0], end[0]) + epsilon
<= point[0]
<= max(start[0], end[0]) + epsilon
and min(start[1], end[1]) - epsilon and min(start[1], end[1]) - epsilon
<= point[1] <= point[1]
<= max(start[1], end[1]) + epsilon <= max(start[1], end[1]) + epsilon
@@ -428,9 +421,9 @@ def _segments_intersect(first_start, first_end, second_start, second_end):
_triangle_cross(second_start, second_end, first_start), _triangle_cross(second_start, second_end, first_start),
_triangle_cross(second_start, second_end, first_end), _triangle_cross(second_start, second_end, first_end),
) )
if (crosses[0] > 1e-8) != (crosses[1] > 1e-8) and ( if (crosses[0] > 1e-8) != (crosses[1] > 1e-8) and (crosses[2] > 1e-8) != (
crosses[2] > 1e-8 crosses[3] > 1e-8
) != (crosses[3] > 1e-8): ):
return True return True
return any( return any(
abs(cross) <= 1e-8 and _point_on_segment(point, start, end) abs(cross) <= 1e-8 and _point_on_segment(point, start, end)
@@ -456,14 +449,10 @@ def polygon_self_intersects(vertices, cell_size=32.0):
bottom = math.floor(min(start[1], end[1]) / cell_size) bottom = math.floor(min(start[1], end[1]) / cell_size)
top = math.floor(max(start[1], end[1]) / cell_size) top = math.floor(max(start[1], end[1]) / cell_size)
edge_cells = tuple( edge_cells = tuple(
(x, y) (x, y) for x in range(left, right + 1) for y in range(bottom, top + 1)
for x in range(left, right + 1)
for y in range(bottom, top + 1)
) )
candidates = { candidates = {
candidate candidate for cell in edge_cells for candidate in cells.get(cell, ())
for cell in edge_cells
for candidate in cells.get(cell, ())
} }
for candidate in candidates: for candidate in candidates:
if abs(edge_index - candidate) <= 1 or { if abs(edge_index - candidate) <= 1 or {
@@ -478,7 +467,7 @@ def polygon_self_intersects(vertices, cell_size=32.0):
return False return False
def preview_fill_triangles(vertices, fill_rule='EVEN_ODD'): def preview_fill_triangles(vertices, fill_rule="EVEN_ODD"):
if not polygon_self_intersects(vertices): if not polygon_self_intersects(vertices):
triangles = triangulated_polygon_vertices(vertices) triangles = triangulated_polygon_vertices(vertices)
if triangles: if triangles:
@@ -541,10 +530,9 @@ def append_lasso_point(path, point):
anchor[0] + segment_x * factor, anchor[0] + segment_x * factor,
anchor[1] + segment_y * factor, anchor[1] + segment_y * factor,
) )
forward = ( forward = (previous[0] - anchor[0]) * (point[0] - previous[0]) + (
(previous[0] - anchor[0]) * (point[0] - previous[0]) previous[1] - anchor[1]
+ (previous[1] - anchor[1]) * (point[1] - previous[1]) ) * (point[1] - previous[1])
)
if ( if (
0.0 <= factor <= 1.0 0.0 <= factor <= 1.0
and forward >= 0.0 and forward >= 0.0
@@ -634,7 +622,7 @@ def image_line_half_plane_polygon(line, image_size):
class ImageRegionGesture: class ImageRegionGesture:
active_tool_id = None active_tool_id = None
preview_fill_rule = 'EVEN_ODD' preview_fill_rule = "EVEN_ODD"
_path = None _path = None
_cursor = None _cursor = None
_handle = None _handle = None
@@ -675,7 +663,7 @@ class ImageRegionGesture:
polygon = self._screen_path() polygon = self._screen_path()
if polygon != self._outline_polygon: if polygon != self._outline_polygon:
self._outline_polygon = polygon self._outline_polygon = polygon
if self.gesture == 'LINE': if self.gesture == "LINE":
outline = (self._path[0], self._cursor) outline = (self._path[0], self._cursor)
else: else:
outline = (*polygon, polygon[0]) if len(polygon) > 1 else () outline = (*polygon, polygon[0]) if len(polygon) > 1 else ()

View File

@@ -1,4 +1,3 @@
from dataclasses import dataclass
from pathlib import Path from pathlib import Path
import bpy import bpy
@@ -11,49 +10,6 @@ from .image_data import (
) )
@dataclass(frozen=True)
class ImageGeometry:
points: object
intrinsics: object
valid: object
calibration_valid: object
image_size: tuple
reference_depth: float
region_transparent_fraction: float
def load_image_geometry(path):
"""Load and validate the shared image geometry Job artifact."""
import numpy as np
with np.load(path, allow_pickle=False) as archive:
geometry = ImageGeometry(
points=np.asarray(archive["points"], dtype=np.float32),
intrinsics=np.asarray(archive["intrinsics"], dtype=np.float32),
valid=np.asarray(archive["valid"], dtype=bool),
calibration_valid=np.asarray(archive["calibration_valid"], dtype=bool),
image_size=tuple(int(value) for value in archive["image_size"]),
reference_depth=float(archive["reference_depth"]),
region_transparent_fraction=float(
archive["region_transparent_fraction"]
),
)
if (
geometry.points.ndim != 3
or geometry.points.shape[2] != 3
or geometry.valid.shape != geometry.points.shape[:2]
or geometry.calibration_valid.shape != geometry.points.shape[:2]
):
raise ValueError("The mesh point field is invalid")
if (
geometry.intrinsics.shape != (3, 3)
or not geometry.reference_depth > 0.0
or not 0.0 <= geometry.region_transparent_fraction <= 1.0
):
raise ValueError("The mesh camera field is invalid")
return geometry
NODE_ASSET_NAME = "O_AnyImage.blend" NODE_ASSET_NAME = "O_AnyImage.blend"
@@ -161,12 +117,9 @@ def set_material_color_space(image, scene=None):
preferred = material_color_space_name(scene) preferred = material_color_space_name(scene)
original = settings.name original = settings.name
pixels = None pixels = None
if ( if preferred != original and (
preferred != original getattr(image, "source", None) == "GENERATED"
and ( or getattr(image, "is_dirty", False)
getattr(image, "source", None) == "GENERATED"
or getattr(image, "is_dirty", False)
)
): ):
pixels = image_pixels(image) pixels = image_pixels(image)
for color_space in dict.fromkeys((preferred, "sRGB", original)): for color_space in dict.fromkeys((preferred, "sRGB", original)):
@@ -389,6 +342,7 @@ def create_image_material(
use_cyclic=None, use_cyclic=None,
normal_image=None, normal_image=None,
normal_space="TANGENT", normal_space="TANGENT",
ior=None,
scene=None, scene=None,
): ):
material = bpy.data.materials.new(image_base_name(source_object.data)) material = bpy.data.materials.new(image_base_name(source_object.data))
@@ -483,6 +437,8 @@ def create_image_material(
else: else:
shader = nodes.new("ShaderNodeBsdfPrincipled") shader = nodes.new("ShaderNodeBsdfPrincipled")
shader.location = (250.0, 400.0) shader.location = (250.0, 400.0)
if ior is not None:
shader.inputs["IOR"].default_value = float(ior)
links.new( links.new(
material_inputs.outputs["Color"], material_inputs.outputs["Color"],

View File

@@ -1,162 +0,0 @@
import bpy
from ..common.mesh_result import load_image_geometry
from ..common.image_data import (
color_data_name,
depth_data_name,
image_pixels,
load_color_result_image,
load_normal_result_image,
image_empty_bounds,
require_image_empty,
)
from ..common.mesh_result import create_image_material
from ..common.mesh_result import modifier_input_identifier, set_modifier_input
from ..common.mesh_result import finalize_object_result
from .mesh_plane import create_mesh_plane_mesh, create_mesh_plane_object
from .normal_map import bake_tangent_normal_image
from .nodes import depth_plane_node_group
def copy_color_image(source_object):
source = source_object.data
image = source.copy()
if getattr(source, "source", None) == "GENERATED" or getattr(
source, "is_dirty", False
):
pixels = image_pixels(source)
image.pixels.foreach_set(pixels)
image.update()
image.name = color_data_name(source_object)
return image
def build_mesh_plane(context, source_object, subdivisions, material):
source_name = source_object.name
plane = create_mesh_plane_object(
context,
source_name,
image_empty_bounds(source_object),
material,
matrix_world=source_object.matrix_world,
subdivisions=subdivisions,
)
for polygon in plane.data.polygons:
polygon.use_smooth = True
plane["anyimage_mesh_shape"] = "PLANE"
return finalize_object_result(context, source_object, plane)
def anchored_point_field(source_object, field):
import numpy as np
points = field.points
intrinsics = field.intrinsics
valid = field.valid
width, height = field.image_size
x_min, x_max, y_min, y_max = image_empty_bounds(source_object)
plane_width = x_max - x_min
plane_height = y_max - y_min
fx, fy = float(intrinsics[0, 0]), float(intrinsics[1, 1])
cx, cy = float(intrinsics[0, 2]), float(intrinsics[1, 2])
if fx <= 0.0 or fy <= 0.0:
raise ValueError("Camera intrinsics contain an invalid focal length")
scale_x = plane_width * fx / (max(width - 1, 1) * field.reference_depth)
scale_y = plane_height * fy / (max(height - 1, 1) * field.reference_depth)
scale_z = float(np.sqrt(scale_x * scale_y))
camera_x = x_min + plane_width * cx / max(width - 1, 1)
camera_y = y_max - plane_height * cy / max(height - 1, 1)
anchored = np.empty_like(points, dtype=np.float32)
anchored[..., 0] = camera_x + points[..., 0] * scale_x
anchored[..., 1] = camera_y - points[..., 1] * scale_y
anchored[..., 2] = (field.reference_depth - points[..., 2]) * scale_z
return anchored, valid, points[..., 2]
def create_depth_image(source_object, anchored_points):
import numpy as np
values = np.asarray(anchored_points, dtype=np.float32)[..., 2]
height, width = values.shape
pixels = np.empty((height, width, 4), dtype=np.float32)
pixels[:, :, :3] = values[:, :, None]
pixels[:, :, 3] = 1.0
image = bpy.data.images.new(
depth_data_name(source_object),
width=width,
height=height,
alpha=True,
float_buffer=True,
)
image.colorspace_settings.name = "Non-Color"
image.pixels.foreach_set(np.flipud(pixels).ravel())
image.pack()
return image
def build_depth_plane(context, source_object, subdivisions, material, field):
source_name = source_object.name
mesh = create_mesh_plane_mesh(source_name, image_empty_bounds(source_object))
for polygon in mesh.polygons:
polygon.use_smooth = True
mesh.materials.append(material)
plane = bpy.data.objects.new(source_name, mesh)
context.collection.objects.link(plane)
plane.matrix_world = source_object.matrix_world.copy()
plane["anyimage_mesh_shape"] = "DEPTH_PLANE"
node_group = depth_plane_node_group()
modifier = plane.modifiers.new(name=node_group.name, type="NODES")
modifier.node_group = node_group
modifier.show_group_selector = False
depth_image = create_depth_image(
source_object, anchored_point_field(source_object, field)[0]
)
for name, value in (
("Subdivide", max(0, min(10, int(subdivisions)))),
("Material", material),
("Thickness", 0.1),
("Depth Amount", 1.0),
("Depth Image", depth_image),
):
set_modifier_input(
modifier,
modifier_input_identifier(node_group, name),
value,
)
return finalize_object_result(context, source_object, plane)
def build_depth_plane_response(context, result, operator):
source_object = require_image_empty(operator.source_object_name)
color_image = load_color_result_image(source_object, result.directory)
color_image.name = color_data_name(source_object)
normal_path = result.directory / "normal.png"
normal_image = (
load_normal_result_image(normal_path, source_object)
if normal_path.is_file()
else None
)
material = create_image_material(
source_object,
color_image,
0.0,
normal_image=normal_image,
scene=context.scene,
)
plane = build_depth_plane(
context,
source_object,
operator.mesh_detail,
material,
load_image_geometry(result.file("image_geometry")),
)
if normal_image is not None:
bake_tangent_normal_image(
context,
source_object,
plane,
normal_image,
)
if "FINISHED" not in bpy.ops.ed.undo_push(message="Convert to Depth Plane"):
raise RuntimeError("Unable to create the Convert to Depth Plane Undo step")

View File

@@ -8,6 +8,14 @@ from ...server.media.resolution import (
MAX_MAX_AI_INPUT_SIZE, MAX_MAX_AI_INPUT_SIZE,
MIN_MAX_AI_INPUT_SIZE, MIN_MAX_AI_INPUT_SIZE,
) )
from ..common.ai_job import (
invoke_ai_setup_if_needed,
moge2_parameters,
production_device,
require_environment,
require_input_path,
require_model,
)
from ..common.image_data import ( from ..common.image_data import (
cleanup_image_input, cleanup_image_input,
image_user_settings, image_user_settings,
@@ -17,18 +25,11 @@ from ..common.image_data import (
video_total_frames, video_total_frames,
) )
from ..common.mesh_result import create_image_material from ..common.mesh_result import create_image_material
from ..common.ai_job import ( from .object import (
da3_parameters, create_color_image,
require_input_path, create_depth_plane_from_result,
invoke_ai_setup_if_needed, create_plane_object,
moge2_parameters,
production_device,
require_environment,
require_model,
) )
from .build import build_depth_plane_response, build_mesh_plane, copy_color_image
from .resolution import scaled_process_resolution
_HIDDEN = {"HIDDEN", "SKIP_SAVE"} _HIDDEN = {"HIDDEN", "SKIP_SAVE"}
@@ -36,23 +37,21 @@ _HIDDEN = {"HIDDEN", "SKIP_SAVE"}
def depth_plane_settings(context): def depth_plane_settings(context):
preferences = addon_preferences(context) preferences = addon_preferences(context)
return ( return (
getattr(preferences, "mesh_depth_model", DEFAULT_MOGE2_MODEL_KEY), getattr(preferences, "geometry_model", DEFAULT_MOGE2_MODEL_KEY),
float(getattr(preferences, "mesh_process_res_factor", 100.0)), int(getattr(preferences, "geometry_resolution_level", 5)),
int(getattr(preferences, "mesh_resolution_level", 5)),
) )
class GenerateDepthPlane(JobOperatorBase, bpy.types.Operator): class GenerateDepthPlane(JobOperatorBase, bpy.types.Operator):
bl_idname = "anyimage.generate_depth_plane" bl_idname = "anyimage.generate_depth_plane"
bl_label = "Generate Depth Plane" bl_label = "Generate Depth Plane"
job_type = "generate-depth-plane" job_type = "generate-depth-plane-geometry"
starting_message = "Preparing Depth Plane" starting_message = "Preparing Depth Plane"
input_path: bpy.props.StringProperty(options=_HIDDEN) input_path: bpy.props.StringProperty(options=_HIDDEN)
source_object_name: bpy.props.StringProperty(options=_HIDDEN) source_object_name: bpy.props.StringProperty(options=_HIDDEN)
mesh_detail: bpy.props.IntProperty(default=0, min=0, max=10, options=_HIDDEN) mesh_detail: bpy.props.IntProperty(default=0, min=0, max=10, options=_HIDDEN)
model: bpy.props.StringProperty(default=DEFAULT_MOGE2_MODEL_KEY, options=_HIDDEN) model: bpy.props.StringProperty(default=DEFAULT_MOGE2_MODEL_KEY, options=_HIDDEN)
process_res: bpy.props.IntProperty(default=1008, min=1, options=_HIDDEN)
resolution_level: bpy.props.IntProperty(default=5, min=0, max=9, options=_HIDDEN) resolution_level: bpy.props.IntProperty(default=5, min=0, max=9, options=_HIDDEN)
max_input_size: bpy.props.IntProperty( max_input_size: bpy.props.IntProperty(
default=DEFAULT_MAX_AI_INPUT_SIZE, default=DEFAULT_MAX_AI_INPUT_SIZE,
@@ -65,38 +64,27 @@ class GenerateDepthPlane(JobOperatorBase, bpy.types.Operator):
def request(self, context): def request(self, context):
require_environment() require_environment()
require_model(self.model) require_model(self.model)
input_path = require_input_path(self.input_path)
parameters = { parameters = {
"input": str(require_input_path(self.input_path)), "input": str(input_path),
"device": production_device().lower(), "device": production_device().lower(),
"video_frames": 1, "video_frames": 1,
"start_frame": source_start_frame(self.source_object_name), "start_frame": source_start_frame(self.source_object_name),
"max_input_size": int(self.max_input_size), "max_input_size": int(self.max_input_size),
} }
settings = context.scene.anyimage_settings parameters.update(
if self.model.startswith("MOGE2_"): moge2_parameters(
parameters.update( context.scene.anyimage_settings,
moge2_parameters( input_path,
settings, self.model,
require_input_path(self.input_path), self.resolution_level,
self.model, source_start_frame(self.source_object_name),
self.resolution_level,
source_start_frame(self.source_object_name),
)
)
else:
parameters.update(
da3_parameters(
settings,
require_input_path(self.input_path),
self.model,
self.process_res,
source_start_frame(self.source_object_name),
)
) )
)
return parameters return parameters
def response(self, context, result): def response(self, context, result):
build_depth_plane_response(context, result, self) create_depth_plane_from_result(context, result, self)
return "Depth Plane is ready" return "Depth Plane is ready"
def cleanup(self): def cleanup(self):
@@ -116,12 +104,7 @@ def start_depth_plane_job(operator, context):
source_object.data, source_object.data,
context.scene, context.scene,
) )
model, process_factor, resolution_level = depth_plane_settings(context) model, resolution_level = depth_plane_settings(context)
process_res = (
scaled_process_resolution(source_object.data, process_factor)
if not model.startswith("MOGE2_")
else 1
)
except (RuntimeError, ValueError) as error: except (RuntimeError, ValueError) as error:
operator.report({"ERROR"}, str(error)) operator.report({"ERROR"}, str(error))
return {"CANCELLED"} return {"CANCELLED"}
@@ -130,14 +113,12 @@ def start_depth_plane_job(operator, context):
source_object_name=source_object.name, source_object_name=source_object.name,
mesh_detail=operator.mesh_detail, mesh_detail=operator.mesh_detail,
model=model, model=model,
process_res=process_res,
resolution_level=resolution_level, resolution_level=resolution_level,
max_input_size=configured_max_ai_input_size(context), max_input_size=configured_max_ai_input_size(context),
delete_input=delete_input, delete_input=delete_input,
) )
if "CANCELLED" in result: if "CANCELLED" in result and delete_input:
if delete_input: cleanup_image_input(source_path, True)
cleanup_image_input(source_path, True)
return {"CANCELLED"} if "CANCELLED" in result else {"FINISHED"} return {"CANCELLED"} if "CANCELLED" in result else {"FINISHED"}
@@ -155,7 +136,7 @@ class ConvertToPlane(bpy.types.Operator):
def execute(self, context): def execute(self, context):
source_object = context.object source_object = context.object
color_image = copy_color_image(source_object) color_image = create_color_image(source_object)
frame_start, frame_offset, frame_duration, use_cyclic = image_user_settings( frame_start, frame_offset, frame_duration, use_cyclic = image_user_settings(
source_object source_object
) )
@@ -169,7 +150,7 @@ class ConvertToPlane(bpy.types.Operator):
use_cyclic=use_cyclic, use_cyclic=use_cyclic,
scene=context.scene, scene=context.scene,
) )
build_mesh_plane(context, source_object, self.mesh_detail, material) create_plane_object(context, source_object, self.mesh_detail, material)
return {"FINISHED"} return {"FINISHED"}

View File

@@ -53,6 +53,14 @@ def depth_plane_node_group():
depth_amount = _interface_socket(group, "Depth Amount", "INPUT", "NodeSocketFloat") depth_amount = _interface_socket(group, "Depth Amount", "INPUT", "NodeSocketFloat")
depth_amount.default_value = 1.0 depth_amount.default_value = 1.0
depth_amount.min_value = 0.0 depth_amount.min_value = 0.0
reference_depth = _interface_socket(
group, "Reference Depth", "INPUT", "NodeSocketFloat"
)
reference_depth.hide_in_modifier = True
uniform_scale = _interface_socket(
group, "Uniform Scale", "INPUT", "NodeSocketFloat"
)
uniform_scale.hide_in_modifier = True
depth_image = _interface_socket(group, "Depth Image", "INPUT", "NodeSocketImage") depth_image = _interface_socket(group, "Depth Image", "INPUT", "NodeSocketImage")
depth_image.hide_in_modifier = True depth_image.hide_in_modifier = True
material = _interface_socket(group, "Material", "INPUT", "NodeSocketMaterial") material = _interface_socket(group, "Material", "INPUT", "NodeSocketMaterial")
@@ -71,10 +79,14 @@ def depth_plane_node_group():
(-1250.0, 850.0), (-1250.0, 850.0),
{"Thickness", "Depth Amount"}, {"Thickness", "Depth Amount"},
) )
image_input = _group_input(nodes, (-1050.0, 50.0), {"Depth Image"}) image_input = _group_input(
nodes,
(-1050.0, 50.0),
{"Depth Image", "Reference Depth", "Uniform Scale"},
)
cleanup_input = _group_input(nodes, (650.0, -200.0), {"Thickness"}) cleanup_input = _group_input(nodes, (650.0, -200.0), {"Thickness"})
group_output = nodes.new("NodeGroupOutput") group_output = nodes.new("NodeGroupOutput")
group_output.location = (1600.0, 450.0) group_output.location = (1850.0, 450.0)
mesh_plane = nodes.new("GeometryNodeGroup") mesh_plane = nodes.new("GeometryNodeGroup")
mesh_plane.node_tree = mesh_plane_node_group() mesh_plane.node_tree = mesh_plane_node_group()
mesh_plane.inputs["Thickness"].default_value = 0.0 mesh_plane.inputs["Thickness"].default_value = 0.0
@@ -103,41 +115,51 @@ def depth_plane_node_group():
separate = nodes.new("ShaderNodeSeparateXYZ") separate = nodes.new("ShaderNodeSeparateXYZ")
separate.location = (-600.0, 50.0) separate.location = (-600.0, 50.0)
links.new(texture.outputs["Color"], separate.inputs["Vector"]) links.new(texture.outputs["Color"], separate.inputs["Vector"])
relative_depth = nodes.new("ShaderNodeMath")
relative_depth.operation = "SUBTRACT"
relative_depth.location = (-600.0, -150.0)
links.new(image_input.outputs["Reference Depth"], relative_depth.inputs[0])
links.new(separate.outputs["X"], relative_depth.inputs[1])
local_depth = nodes.new("ShaderNodeMath")
local_depth.operation = "MULTIPLY"
local_depth.location = (-350.0, -150.0)
links.new(relative_depth.outputs[0], local_depth.inputs[0])
links.new(image_input.outputs["Uniform Scale"], local_depth.inputs[1])
raw_depth = nodes.new("ShaderNodeMath") raw_depth = nodes.new("ShaderNodeMath")
raw_depth.operation = "MULTIPLY" raw_depth.operation = "MULTIPLY"
raw_depth.location = (-350.0, 150.0) raw_depth.location = (-100.0, 150.0)
links.new(separate.outputs["X"], raw_depth.inputs[0]) links.new(local_depth.outputs[0], raw_depth.inputs[0])
links.new(shape_input.outputs["Depth Amount"], raw_depth.inputs[1]) links.new(shape_input.outputs["Depth Amount"], raw_depth.inputs[1])
inward_limit = nodes.new("ShaderNodeMath") inward_limit = nodes.new("ShaderNodeMath")
inward_limit.operation = "MULTIPLY" inward_limit.operation = "MULTIPLY"
inward_limit.inputs[1].default_value = -0.98 inward_limit.inputs[1].default_value = -0.98
inward_limit.location = (-350.0, -50.0) inward_limit.location = (-100.0, -50.0)
links.new(shape_input.outputs["Thickness"], inward_limit.inputs[0]) links.new(shape_input.outputs["Thickness"], inward_limit.inputs[0])
clamp = nodes.new("ShaderNodeMath") clamp = nodes.new("ShaderNodeMath")
clamp.operation = "MAXIMUM" clamp.operation = "MAXIMUM"
clamp.location = (-100.0, 50.0) clamp.location = (150.0, 50.0)
links.new(raw_depth.outputs[0], clamp.inputs[0]) links.new(raw_depth.outputs[0], clamp.inputs[0])
links.new(inward_limit.outputs[0], clamp.inputs[1]) links.new(inward_limit.outputs[0], clamp.inputs[1])
has_thickness = nodes.new("ShaderNodeMath") has_thickness = nodes.new("ShaderNodeMath")
has_thickness.operation = "GREATER_THAN" has_thickness.operation = "GREATER_THAN"
has_thickness.inputs[1].default_value = 1e-6 has_thickness.inputs[1].default_value = 1e-6
has_thickness.location = (-350.0, -250.0) has_thickness.location = (-100.0, -350.0)
links.new(shape_input.outputs["Thickness"], has_thickness.inputs[0]) links.new(shape_input.outputs["Thickness"], has_thickness.inputs[0])
choose_depth = nodes.new("GeometryNodeSwitch") choose_depth = nodes.new("GeometryNodeSwitch")
choose_depth.input_type = "FLOAT" choose_depth.input_type = "FLOAT"
choose_depth.location = (150.0, 50.0) choose_depth.location = (400.0, 50.0)
links.new(has_thickness.outputs[0], choose_depth.inputs["Switch"]) links.new(has_thickness.outputs[0], choose_depth.inputs["Switch"])
links.new(raw_depth.outputs[0], choose_depth.inputs["False"]) links.new(raw_depth.outputs[0], choose_depth.inputs["False"])
links.new(clamp.outputs[0], choose_depth.inputs["True"]) links.new(clamp.outputs[0], choose_depth.inputs["True"])
combine = nodes.new("ShaderNodeCombineXYZ") combine = nodes.new("ShaderNodeCombineXYZ")
combine.location = (350.0, 50.0) combine.location = (600.0, 50.0)
links.new(choose_depth.outputs["Output"], combine.inputs["Z"]) links.new(choose_depth.outputs["Output"], combine.inputs["Z"])
front = nodes.new("FunctionNodeBooleanMath") front = nodes.new("FunctionNodeBooleanMath")
front.operation = "NOT" front.operation = "NOT"
front.location = (150.0, 300.0) front.location = (400.0, 300.0)
links.new(extrude.outputs["Top"], front.inputs[0]) links.new(extrude.outputs["Top"], front.inputs[0])
set_position = nodes.new("GeometryNodeSetPosition") set_position = nodes.new("GeometryNodeSetPosition")
set_position.location = (550.0, 450.0) set_position.location = (800.0, 450.0)
links.new(geometry, set_position.inputs["Geometry"]) links.new(geometry, set_position.inputs["Geometry"])
links.new(front.outputs[0], set_position.inputs["Selection"]) links.new(front.outputs[0], set_position.inputs["Selection"])
links.new(combine.outputs["Vector"], set_position.inputs["Offset"]) links.new(combine.outputs["Vector"], set_position.inputs["Offset"])
@@ -146,27 +168,27 @@ def depth_plane_node_group():
zero_thickness = nodes.new("ShaderNodeMath") zero_thickness = nodes.new("ShaderNodeMath")
zero_thickness.operation = "LESS_THAN" zero_thickness.operation = "LESS_THAN"
zero_thickness.inputs[1].default_value = 1e-6 zero_thickness.inputs[1].default_value = 1e-6
zero_thickness.location = (750.0, -150.0) zero_thickness.location = (1000.0, -150.0)
links.new(cleanup_input.outputs["Thickness"], zero_thickness.inputs[0]) links.new(cleanup_input.outputs["Thickness"], zero_thickness.inputs[0])
back_or_side = nodes.new("FunctionNodeBooleanMath") back_or_side = nodes.new("FunctionNodeBooleanMath")
back_or_side.operation = "OR" back_or_side.operation = "OR"
back_or_side.location = (750.0, 150.0) back_or_side.location = (1000.0, 150.0)
links.new(extrude.outputs["Top"], back_or_side.inputs[0]) links.new(extrude.outputs["Top"], back_or_side.inputs[0])
links.new(extrude.outputs["Side"], back_or_side.inputs[1]) links.new(extrude.outputs["Side"], back_or_side.inputs[1])
delete_selection = nodes.new("FunctionNodeBooleanMath") delete_selection = nodes.new("FunctionNodeBooleanMath")
delete_selection.operation = "AND" delete_selection.operation = "AND"
delete_selection.location = (950.0, 50.0) delete_selection.location = (1200.0, 50.0)
links.new(zero_thickness.outputs[0], delete_selection.inputs[0]) links.new(zero_thickness.outputs[0], delete_selection.inputs[0])
links.new(back_or_side.outputs[0], delete_selection.inputs[1]) links.new(back_or_side.outputs[0], delete_selection.inputs[1])
delete_geometry = nodes.new("GeometryNodeDeleteGeometry") delete_geometry = nodes.new("GeometryNodeDeleteGeometry")
delete_geometry.domain = "FACE" delete_geometry.domain = "FACE"
delete_geometry.location = (1100.0, 450.0) delete_geometry.location = (1350.0, 450.0)
links.new(geometry, delete_geometry.inputs["Geometry"]) links.new(geometry, delete_geometry.inputs["Geometry"])
links.new(delete_selection.outputs[0], delete_geometry.inputs["Selection"]) links.new(delete_selection.outputs[0], delete_geometry.inputs["Selection"])
shade_smooth = nodes.new("GeometryNodeSetShadeSmooth") shade_smooth = nodes.new("GeometryNodeSetShadeSmooth")
shade_smooth.domain = "FACE" shade_smooth.domain = "FACE"
shade_smooth.inputs["Shade Smooth"].default_value = True shade_smooth.inputs["Shade Smooth"].default_value = True
shade_smooth.location = (1350.0, 450.0) shade_smooth.location = (1600.0, 450.0)
links.new(delete_geometry.outputs["Geometry"], shade_smooth.inputs["Geometry"]) links.new(delete_geometry.outputs["Geometry"], shade_smooth.inputs["Geometry"])
links.new(shade_smooth.outputs["Geometry"], group_output.inputs["Geometry"]) links.new(shade_smooth.outputs["Geometry"], group_output.inputs["Geometry"])
group.use_fake_user = True group.use_fake_user = True

View File

@@ -1,159 +0,0 @@
from ..common.image_data import image_pixels
def _normalized(values, fallback):
import numpy as np
values = np.asarray(values, dtype=np.float64)
lengths = np.linalg.norm(values, axis=-1)
result = np.empty_like(values)
usable = lengths > 1e-10
result[usable] = values[usable] / lengths[usable, None]
result[~usable] = fallback
return result
def rasterize_tangent_normal(
object_normal,
triangle_uv,
triangle_normal,
triangle_tangent,
triangle_sign,
return_coverage=False,
):
"""Convert object-space pixels through a smooth per-corner UV tangent frame."""
import numpy as np
object_normal = _normalized(object_normal, (0.0, 0.0, 1.0))
height, width = object_normal.shape[:2]
output = np.zeros((height, width, 3), dtype=np.float32)
output[..., 2] = 1.0
coverage = np.zeros((height, width), dtype=bool)
scale = np.asarray((max(width - 1, 1), max(height - 1, 1)))
for uv, corner_normal, corner_tangent, corner_sign in zip(
triangle_uv,
triangle_normal,
triangle_tangent,
triangle_sign,
strict=True,
):
pixel = np.asarray(uv, dtype=np.float64) * scale
x_min = max(int(np.ceil(np.min(pixel[:, 0]))), 0)
x_max = min(int(np.floor(np.max(pixel[:, 0]))), width - 1)
y_min = max(int(np.ceil(np.min(pixel[:, 1]))), 0)
y_max = min(int(np.floor(np.max(pixel[:, 1]))), height - 1)
if x_min > x_max or y_min > y_max:
continue
a, b, c = pixel
denominator = (b[1] - c[1]) * (a[0] - c[0]) + (
c[0] - b[0]
) * (a[1] - c[1])
if abs(denominator) <= 1e-12:
continue
grid_x, grid_y = np.meshgrid(
np.arange(x_min, x_max + 1, dtype=np.float64),
np.arange(y_min, y_max + 1, dtype=np.float64),
)
first = (
(b[1] - c[1]) * (grid_x - c[0])
+ (c[0] - b[0]) * (grid_y - c[1])
) / denominator
second = (
(c[1] - a[1]) * (grid_x - c[0])
+ (a[0] - c[0]) * (grid_y - c[1])
) / denominator
third = 1.0 - first - second
inside = (first >= -1e-6) & (second >= -1e-6) & (third >= -1e-6)
if not inside.any():
continue
weights = np.stack((first[inside], second[inside], third[inside]), axis=-1)
normal = _normalized(weights @ corner_normal, (0.0, 0.0, 1.0))
tangent = weights @ corner_tangent
tangent -= normal * np.sum(tangent * normal, axis=-1, keepdims=True)
tangent = _normalized(tangent, (1.0, 0.0, 0.0))
sign = np.where(weights @ corner_sign >= 0.0, 1.0, -1.0)
bitangent = np.cross(normal, tangent) * sign[:, None]
rows = grid_y[inside].astype(np.int32)
columns = grid_x[inside].astype(np.int32)
source = object_normal[rows, columns]
output[rows, columns, 0] = np.sum(source * tangent, axis=-1)
output[rows, columns, 1] = np.sum(source * bitangent, axis=-1)
output[rows, columns, 2] = np.sum(source * normal, axis=-1)
coverage[rows, columns] = True
output[coverage] = _normalized(output[coverage], (0.0, 0.0, 1.0))
return (output, coverage) if return_coverage else output
def bake_tangent_normal_image(context, source_object, target_object, image):
"""Bake a MoGe object-space image against the evaluated target mesh TBN."""
import numpy as np
width, height = (int(value) for value in image.size)
rgba = image_pixels(image).reshape((height, width, 4))
source_normal = rgba[..., :3].astype(np.float64) * 2.0 - 1.0
source_to_target = (
target_object.matrix_world.inverted().to_3x3()
@ source_object.matrix_world.to_3x3()
)
normal_matrix = source_to_target.inverted().transposed()
transform = np.asarray(normal_matrix, dtype=np.float64)
target_normal = source_normal @ transform.T
target_normal = _normalized(target_normal, (0.0, 0.0, 1.0))
front = _normalized(
np.asarray((0.0, 0.0, 1.0), dtype=np.float64)[None, :] @ transform.T,
(0.0, 0.0, 1.0),
)[0]
context.view_layer.update()
depsgraph = context.evaluated_depsgraph_get()
evaluated = target_object.evaluated_get(depsgraph)
mesh = evaluated.to_mesh(preserve_all_data_layers=True, depsgraph=depsgraph)
try:
uv_layer = mesh.uv_layers.active
if uv_layer is None:
raise ValueError("The target mesh has no active UV map")
mesh.calc_loop_triangles()
mesh.calc_tangents(uvmap=uv_layer.name)
loop_count = len(mesh.loops)
uv = np.empty((loop_count, 2), dtype=np.float64)
uv_layer.data.foreach_get("uv", uv.ravel())
tangent = np.empty((loop_count, 3), dtype=np.float64)
sign = np.empty(loop_count, dtype=np.float64)
mesh.loops.foreach_get("tangent", tangent.ravel())
mesh.loops.foreach_get("bitangent_sign", sign)
normal = np.asarray(
[tuple(loop.normal) for loop in mesh.loops],
dtype=np.float64,
)
triangles = np.asarray(
[tuple(triangle.loops) for triangle in mesh.loop_triangles],
dtype=np.int32,
)
triangle_normal = np.asarray(
[tuple(triangle.normal) for triangle in mesh.loop_triangles],
dtype=np.float64,
)
front_facing = triangle_normal @ front > 1e-6
triangles = triangles[front_facing]
tangent_normal, coverage = rasterize_tangent_normal(
target_normal,
uv[triangles],
normal[triangles],
tangent[triangles],
sign[triangles],
return_coverage=True,
)
finally:
evaluated.to_mesh_clear()
rgba[coverage, :3] = tangent_normal[coverage] * 0.5 + 0.5
image.pixels.foreach_set(rgba.ravel())
image.update()
image.pack()
return image

View File

@@ -0,0 +1,125 @@
"""Create Blender Image, Material, Mesh, and Object data for Plane conversion."""
import bpy
from ..common.image_data import (
color_data_name,
image_empty_bounds,
image_pixels,
load_color_result_image,
require_image_empty,
)
from ..common.depth_data import (
depth_uniform_scale,
load_depth_metadata,
load_depth_result_image,
)
from ..common.mesh_result import (
create_image_material,
finalize_object_result,
modifier_input_identifier,
set_modifier_input,
)
from .mesh_plane import create_mesh_plane_mesh, create_mesh_plane_object
from .nodes import depth_plane_node_group
def create_color_image(source_object):
source = source_object.data
image = source.copy()
if getattr(source, "source", None) == "GENERATED" or getattr(
source, "is_dirty", False
):
pixels = image_pixels(source)
image.pixels.foreach_set(pixels)
image.update()
image.name = color_data_name(source_object)
return image
def create_plane_object(context, source_object, subdivisions, material):
source_name = source_object.name
plane = create_mesh_plane_object(
context,
source_name,
image_empty_bounds(source_object),
material,
matrix_world=source_object.matrix_world,
subdivisions=subdivisions,
)
for polygon in plane.data.polygons:
polygon.use_smooth = True
plane["anyimage_mesh_shape"] = "PLANE"
return finalize_object_result(context, source_object, plane)
def create_depth_plane_object(
context,
source_object,
subdivisions,
material,
depth_image,
depth_metadata,
):
source_name = source_object.name
mesh = create_mesh_plane_mesh(source_name, image_empty_bounds(source_object))
for polygon in mesh.polygons:
polygon.use_smooth = True
mesh.materials.append(material)
plane = bpy.data.objects.new(source_name, mesh)
context.collection.objects.link(plane)
plane.matrix_world = source_object.matrix_world.copy()
plane["anyimage_mesh_shape"] = "DEPTH_PLANE"
node_group = depth_plane_node_group()
modifier = plane.modifiers.new(name=node_group.name, type="NODES")
modifier.node_group = node_group
modifier.show_group_selector = False
for name, value in (
("Subdivide", max(0, min(10, int(subdivisions)))),
("Material", material),
("Thickness", 0.1),
("Depth Amount", 1.0),
("Reference Depth", depth_metadata["reference_depth"]),
("Uniform Scale", depth_uniform_scale(source_object, depth_metadata)),
("Depth Image", depth_image),
):
set_modifier_input(
modifier,
modifier_input_identifier(node_group, name),
value,
)
return finalize_object_result(context, source_object, plane)
def create_depth_plane_from_result(context, result, operator):
source_object = require_image_empty(operator.source_object_name)
color_image = load_color_result_image(source_object, result.directory)
color_image.name = color_data_name(source_object)
material = create_image_material(
source_object,
color_image,
0.0,
scene=context.scene,
)
depth_metadata = load_depth_metadata(result.file("depth_metadata"))
depth_image = load_depth_result_image(
result.file("z_depth"),
source_object,
depth_metadata,
)
try:
create_depth_plane_object(
context,
source_object,
operator.mesh_detail,
material,
depth_image,
depth_metadata,
)
except Exception:
if depth_image.users == 0:
bpy.data.images.remove(depth_image, do_unlink=True)
raise
if "FINISHED" not in bpy.ops.ed.undo_push(message="Convert to Depth Plane"):
raise RuntimeError("Unable to create the Convert to Depth Plane Undo step")

View File

@@ -1,16 +0,0 @@
DA3_PATCH_SIZE = 14
DA3_MIN_PROCESS_RES = 252
DA3_MAX_PROCESS_RES = 1008
def scaled_process_resolution(image, scale_percent):
"""Return the scaled long edge clamped and aligned for DA3 inference."""
width, height = image.size
if width <= 0 or height <= 0:
raise ValueError("The source image has no readable dimensions")
if scale_percent < 1.0 or scale_percent > 100.0:
raise ValueError("Resolution Scale must be between 1% and 100%")
process_res = max(width, height) * scale_percent / 100.0
process_res = max(DA3_MIN_PROCESS_RES, process_res)
process_res = min(DA3_MAX_PROCESS_RES, process_res)
return int(process_res / DA3_PATCH_SIZE + 0.5) * DA3_PATCH_SIZE

View File

@@ -1,5 +1,4 @@
from .region_to_shape import CutoutRegionToShape from .main import CutoutRegionToShape, CutoutTool, SelectCutoutRegion
from .select_region import CutoutTool, SelectCutoutRegion
__all__ = ( __all__ = (
"CutoutRegionToShape", "CutoutRegionToShape",

File diff suppressed because it is too large Load Diff

View File

@@ -1 +0,0 @@
"""Cutout 的内存几何算法。"""

View File

@@ -1,402 +0,0 @@
import math
DEFAULT_INFLATION = 0.45
def distances_to_contour(points, contour):
import numpy as np
points = np.asarray(points, dtype=np.float64)
contour = np.asarray(contour, dtype=np.float64)
squared = np.full(len(points), np.inf, dtype=np.float64)
for start, end in zip(contour, np.roll(contour, -1, axis=0)):
segment = end - start
length_squared = float(segment.dot(segment))
if length_squared <= 1e-16:
continue
factor = np.clip(
(points - start) @ segment / length_squared,
0.0,
1.0,
)
nearest = start + factor[:, None] * segment
squared = np.minimum(
squared,
np.sum((points - nearest) ** 2, axis=1),
)
return np.sqrt(squared)
def chordal_axis(contour, epsilon=1e-9):
import numpy as np
from mathutils import Vector
from mathutils.geometry import delaunay_2d_cdt
contour = np.asarray(contour, dtype=np.float64)
boundary_count = len(contour)
boundary_edges = [
(index, (index + 1) % boundary_count)
for index in range(boundary_count)
]
result = delaunay_2d_cdt(
[Vector(point) for point in contour],
boundary_edges,
[list(range(boundary_count))],
1,
epsilon,
)
points = np.asarray(
[tuple(point) for point in result[0]],
dtype=np.float64,
)
triangles = [tuple(face) for face in result[2] if len(face) == 3]
boundary = {
tuple(sorted(edge))
for edge in boundary_edges
}
node_indices = {}
nodes = []
segments = set()
def edge_node(edge):
key = ("edge", *tuple(sorted(edge)))
if key not in node_indices:
node_indices[key] = len(nodes)
nodes.append((points[edge[0]] + points[edge[1]]) * 0.5)
return node_indices[key]
for face_index, face in enumerate(triangles):
edges = [
(face[0], face[1]),
(face[1], face[2]),
(face[2], face[0]),
]
internal = [
edge
for edge in edges
if tuple(sorted(edge)) not in boundary
]
internal_nodes = [edge_node(edge) for edge in internal]
if len(internal_nodes) == 2:
segments.add(tuple(sorted(internal_nodes)))
elif len(internal_nodes) == 3:
key = ("face", face_index)
center_index = len(nodes)
node_indices[key] = center_index
nodes.append(points[list(face)].mean(axis=0))
for midpoint_index in internal_nodes:
segments.add(tuple(sorted((center_index, midpoint_index))))
if not nodes:
center = contour.mean(axis=0)
return np.asarray((center,), dtype=np.float64), [], np.asarray(
(distances_to_contour((center,), contour)[0],),
dtype=np.float64,
)
nodes = np.asarray(nodes, dtype=np.float64)
radii = distances_to_contour(nodes, contour)
active = set(segments)
while len(active) > 1:
degree = [0] * len(nodes)
for start, end in active:
degree[start] += 1
degree[end] += 1
removable = []
for edge in active:
start, end = edge
leaf = start if degree[start] == 1 else end if degree[end] == 1 else None
if leaf is None:
continue
length = float(np.linalg.norm(nodes[start] - nodes[end]))
if length < max(radii[leaf] * 0.5, epsilon):
removable.append(edge)
if not removable:
break
active.difference_update(removable)
if active:
used = sorted({index for edge in active for index in edge})
remap = {old: new for new, old in enumerate(used)}
nodes = nodes[used]
radii = radii[used]
active = {
tuple(sorted((remap[start], remap[end])))
for start, end in active
}
else:
widest = int(np.argmax(radii))
nodes = nodes[[widest]]
radii = radii[[widest]]
return nodes, sorted(active), radii
def smooth_interior_heights(
heights,
planar_faces,
boundary_mask,
iterations=2,
factor=0.3,
):
import numpy as np
values = np.asarray(heights, dtype=np.float64).copy()
boundary_mask = np.asarray(boundary_mask, dtype=bool)
neighbors = [set() for _index in range(len(values))]
for face in planar_faces:
for start, end in zip(face, face[1:] + face[:1]):
neighbors[start].add(end)
neighbors[end].add(start)
for _iteration in range(iterations):
updated = values.copy()
for index in np.flatnonzero(~boundary_mask):
adjacent = [
neighbor
for neighbor in neighbors[index]
if not boundary_mask[neighbor]
]
if adjacent:
average = float(values[adjacent].mean())
updated[index] = values[index] * (1.0 - factor) + average * factor
updated[boundary_mask] = 0.0
values = updated
return values
def teddy_balloon_heights(
planar_points,
planar_faces,
contour,
boundary_mask,
inflation,
):
import numpy as np
points = np.asarray(planar_points, dtype=np.float64)
boundary_mask = np.asarray(boundary_mask, dtype=bool)
nodes, segments, radii = chordal_axis(contour)
squared_heights = np.zeros(len(points), dtype=np.float64)
for start, end in segments:
axis_start = nodes[start]
axis_end = nodes[end]
axis = axis_end - axis_start
length_squared = float(axis.dot(axis))
if length_squared <= 1e-16:
continue
factors = np.clip(
(points - axis_start) @ axis / length_squared,
0.0,
1.0,
)
centers = axis_start + factors[:, None] * axis
local_radii = radii[start] + factors * (radii[end] - radii[start])
distance_squared = np.sum((points - centers) ** 2, axis=1)
squared_heights = np.maximum(
squared_heights,
local_radii * local_radii - distance_squared,
)
degree = np.zeros(len(nodes), dtype=np.int64)
for start, end in segments:
degree[start] += 1
degree[end] += 1
for index in np.flatnonzero(degree != 2):
distance_squared = np.sum((points - nodes[index]) ** 2, axis=1)
squared_heights = np.maximum(
squared_heights,
radii[index] * radii[index] - distance_squared,
)
missing = (~boundary_mask) & (squared_heights <= 0.0)
if np.any(missing):
fallback = distances_to_contour(points[missing], contour)
squared_heights[missing] = fallback * fallback
heights = inflation * np.sqrt(np.maximum(squared_heights, 0.0))
heights[boundary_mask] = 0.0
return smooth_interior_heights(
heights,
planar_faces,
boundary_mask,
)
DEFAULT_ALGORITHM = "POISSON"
def cotangent_poisson_field(
planar_points,
planar_faces,
boundary_mask,
tolerance=1e-8,
):
import numpy as np
points = np.asarray(planar_points, dtype=np.float64)
faces = np.asarray(planar_faces, dtype=np.int64)
if faces.size == 0:
faces = np.empty((0, 3), dtype=np.int64)
if faces.ndim != 2 or faces.shape[1] != 3:
raise ValueError("Balloon profile faces must be triangles")
boundary_mask = np.asarray(boundary_mask, dtype=bool)
triangles = points[faces]
a, b, c = (triangles[:, index] for index in range(3))
cross = np.abs(
(b[:, 0] - a[:, 0]) * (c[:, 1] - a[:, 1])
- (b[:, 1] - a[:, 1]) * (c[:, 0] - a[:, 0])
)
usable = cross > 1e-14
faces = faces[usable]
a, b, c = a[usable], b[usable], c[usable]
cross = cross[usable]
loads = np.bincount(
faces.ravel(),
weights=np.repeat(cross / 6.0, 3),
minlength=len(points),
)
edges = np.concatenate(
(
faces[:, (1, 2)],
faces[:, (0, 2)],
faces[:, (0, 1)],
),
axis=0,
)
edge_weights = np.concatenate(
(
np.sum((b - a) * (c - a), axis=1),
np.sum((a - b) * (c - b), axis=1),
np.sum((a - c) * (b - c), axis=1),
)
) / np.tile(cross * 2.0, 3)
edges.sort(axis=1)
edges, inverse = np.unique(edges, axis=0, return_inverse=True)
edge_weights = np.bincount(
inverse,
weights=edge_weights,
minlength=len(edges),
)
interior = np.flatnonzero(~boundary_mask)
field = np.zeros(len(points), dtype=np.float64)
if not len(interior):
return field
compact = np.full(len(points), -1, dtype=np.int64)
compact[interior] = np.arange(len(interior), dtype=np.int64)
# Constrained triangulation can create negative cotangent weights.
# Clamping them keeps the Dirichlet system positive definite.
edge_weights = np.maximum(edge_weights, 0.0)
usable = edge_weights > 1e-14
compact_edges = compact[edges[usable]]
edge_weights = edge_weights[usable]
start_compact = compact_edges[:, 0]
end_compact = compact_edges[:, 1]
start_usable = start_compact >= 0
end_usable = end_compact >= 0
diagonal = np.zeros(len(interior), dtype=np.float64)
diagonal += np.bincount(
start_compact[start_usable],
weights=edge_weights[start_usable],
minlength=len(interior),
)
diagonal += np.bincount(
end_compact[end_usable],
weights=edge_weights[end_usable],
minlength=len(interior),
)
if np.any(diagonal <= 1e-14):
raise ValueError("Unable to build a stable Balloon profile")
interior_edges = start_usable & end_usable
edge_starts = start_compact[interior_edges]
edge_ends = end_compact[interior_edges]
weights = edge_weights[interior_edges]
directed_starts = np.concatenate((edge_starts, edge_ends))
directed_ends = np.concatenate((edge_ends, edge_starts))
directed_weights = np.concatenate((weights, weights))
def multiply(values):
result = diagonal * values
if len(directed_weights):
result -= np.bincount(
directed_starts,
weights=directed_weights * values[directed_ends],
minlength=len(values),
)
return result
right_hand_side = loads[interior]
solution = np.zeros(len(interior), dtype=np.float64)
residual = right_hand_side.copy()
preconditioned = residual / diagonal
direction = preconditioned.copy()
residual_product = float(residual.dot(preconditioned))
target = max(float(np.linalg.norm(right_hand_side)) * tolerance, 1e-12)
max_iterations = max(64, min(len(interior) * 4, 4000))
for _iteration in range(max_iterations):
if float(np.linalg.norm(residual)) <= target:
break
multiplied = multiply(direction)
denominator = float(direction.dot(multiplied))
if denominator <= 0.0 or not math.isfinite(denominator):
raise ValueError("Unable to solve the Balloon profile")
step = residual_product / denominator
solution += step * direction
residual -= step * multiplied
preconditioned = residual / diagonal
next_product = float(residual.dot(preconditioned))
if not math.isfinite(next_product):
raise ValueError("Unable to solve the Balloon profile")
if abs(residual_product) <= 1e-30:
break
direction = preconditioned + next_product / residual_product * direction
residual_product = next_product
field[interior] = np.maximum(solution, 0.0)
return field
def poisson_balloon_profile(
planar_points,
planar_faces,
boundary_mask,
inflation=DEFAULT_INFLATION,
):
import numpy as np
field = cotangent_poisson_field(
planar_points,
planar_faces,
boundary_mask,
)
return 2.0 * float(inflation) * np.sqrt(field)
def balloon_profile(
algorithm,
planar_points,
planar_faces,
boundary_mask,
components,
inflation=DEFAULT_INFLATION,
):
import numpy as np
algorithm = str(algorithm).upper()
if algorithm == "POISSON":
return poisson_balloon_profile(
planar_points,
planar_faces,
boundary_mask,
inflation,
)
if algorithm != "TEDDY":
raise ValueError(f"Unknown Balloon profile algorithm: {algorithm}")
result = np.zeros(len(planar_points), dtype=np.float64)
for start, end, component_faces, outer_contour in components:
result[start:end] = teddy_balloon_heights(
np.asarray(planar_points)[start:end],
component_faces,
outer_contour,
np.asarray(boundary_mask)[start:end],
inflation,
)
return result

View File

@@ -1,175 +0,0 @@
"""Calibrate metric depth for Cutout Depth Balloon and Depth Surface."""
PLANE_VARIANCE_FLOOR_RATIO = 0.025
DEPTH_REFERENCE_PERCENTILE = 95.0
ADAPTIVE_DEPTH_TRANSPARENCY_RATIO = 0.05
DEFAULT_DEPTH_BASELINE = 8.0
DEFAULT_DEPTH_DIRECTION = (0.0, 0.0, 1.0)
def use_adaptive_depth_calibration(region_transparent_fraction):
"""Return whether the Region input has enough transparent context."""
return (
float(region_transparent_fraction)
>= ADAPTIVE_DEPTH_TRANSPARENCY_RATIO
)
def metric_depth_reference(camera_depth, valid, uniform_scale):
"""Return the shared upper-percentile scene depth for both Depth shapes."""
import numpy as np
camera_depth = np.asarray(camera_depth, dtype=np.float32)
valid = np.asarray(valid, dtype=bool)
if valid.shape != camera_depth.shape:
raise ValueError("Depth values and validity mask dimensions do not match")
usable = valid & np.isfinite(camera_depth) & (camera_depth > 0.0)
if not usable.any():
raise ValueError("The depth field contains no valid metric depth")
if not uniform_scale > 0.0:
raise ValueError("Depth Uniform Scale must be positive")
reference = np.percentile(
camera_depth[usable],
DEPTH_REFERENCE_PERCENTILE,
)
return float(reference) * float(uniform_scale)
def _sample_valid_bilinear(values, valid, x, y):
import numpy as np
height, width = values.shape
x = np.clip(x, 0.0, max(width - 1, 0))
y = np.clip(y, 0.0, max(height - 1, 0))
x0 = np.floor(x).astype(np.int64)
y0 = np.floor(y).astype(np.int64)
x1 = np.minimum(x0 + 1, width - 1)
y1 = np.minimum(y0 + 1, height - 1)
fx = x - x0
fy = y - y0
samples = np.stack(
(
values[y0, x0],
values[y0, x1],
values[y1, x0],
values[y1, x1],
),
axis=1,
)
weights = np.stack(
(
(1.0 - fx) * (1.0 - fy),
fx * (1.0 - fy),
(1.0 - fx) * fy,
fx * fy,
),
axis=1,
)
sample_valid = np.stack(
(
valid[y0, x0],
valid[y0, x1],
valid[y1, x0],
valid[y1, x1],
),
axis=1,
)
weights *= sample_valid
totals = weights.sum(axis=1)
result = np.zeros(len(x), dtype=np.float32)
usable = totals > 1e-8
result[usable] = (samples[usable] * weights[usable]).sum(axis=1) / totals[usable]
return result, usable
def _weighted_plane(coordinates, values, weights):
import numpy as np
total = float(np.sum(weights))
if total <= 1e-12:
return np.zeros(3, dtype=np.float32)
center = np.sum(coordinates * weights[:, None], axis=0) / total
value_center = float(np.sum(values * weights) / total)
centered = coordinates - center
covariance = (centered * weights[:, None]).T @ centered / total
cross = centered.T @ (weights * (values - value_center)) / total
eigenvalues, eigenvectors = np.linalg.eigh(covariance)
largest = max(float(eigenvalues[-1]), 0.0)
if largest <= 1e-12:
slopes = np.zeros(2, dtype=np.float64)
else:
supported = np.maximum(eigenvalues, largest * PLANE_VARIANCE_FLOOR_RATIO)
slopes = eigenvectors @ ((eigenvectors.T @ cross) / supported)
intercept = value_center - float(center @ slopes)
return np.asarray((*slopes, intercept), dtype=np.float32)
def _stable_plane(coordinates, values):
import numpy as np
coordinates = np.asarray(coordinates, dtype=np.float64)
values = np.asarray(values, dtype=np.float64)
if len(coordinates) < 2:
return np.asarray((0.0, 0.0, np.median(values)), dtype=np.float32)
weights = np.ones(len(values), dtype=np.float64)
plane = np.zeros(3, dtype=np.float32)
for _iteration in range(4):
plane = _weighted_plane(coordinates, values, weights)
residual = values - coordinates @ plane[:2] - plane[2]
centered = residual - np.median(residual)
scale = max(float(np.median(np.abs(centered))) * 1.4826, 1e-6)
weights = np.minimum(
1.0,
(2.5 * scale) / np.maximum(np.abs(centered), 1e-12),
)
return plane
def depth_balloon_projection_parameters(
vertices,
vertex_uv,
camera_depth,
valid,
uniform_scale,
):
"""Fit a metric base plane and return its center depth and direction."""
import numpy as np
vertices = np.asarray(vertices, dtype=np.float32)
uv = np.asarray(vertex_uv, dtype=np.float32)
if vertices.ndim != 2 or vertices.shape[1] != 3:
raise ValueError("Depth Balloon vertices must have three components")
if uv.shape != (len(vertices), 2):
raise ValueError("Depth Balloon UV coordinates do not match its vertices")
camera_depth = np.asarray(camera_depth, dtype=np.float32)
valid = np.asarray(valid, dtype=bool)
if valid.shape != camera_depth.shape:
raise ValueError("Depth Balloon depth and mask dimensions do not match")
if not uniform_scale > 0.0:
raise ValueError("Depth Balloon Uniform Scale must be positive")
height, width = camera_depth.shape
sample_x = uv[:, 0] * max(width - 1, 0)
sample_y = (1.0 - uv[:, 1]) * max(height - 1, 0)
calibration_valid = valid & np.isfinite(camera_depth) & (camera_depth > 0.0)
sampled, usable = _sample_valid_bilinear(
camera_depth,
calibration_valid,
sample_x,
sample_y,
)
if not usable.any():
raise ValueError("MoGe-2 found no valid Depth Balloon samples")
local_coordinates = vertices[:, 1:]
metric_depth = sampled[usable] * float(uniform_scale)
plane = _stable_plane(local_coordinates[usable], metric_depth)
reference = metric_depth_reference(
camera_depth,
calibration_valid,
uniform_scale,
)
direction = np.asarray((-plane[0], plane[1], 1.0), dtype=np.float64)
direction /= np.linalg.norm(direction)
return reference, tuple(float(value) for value in direction)

View File

@@ -1,241 +0,0 @@
"""Extract Cutout Region contours from a Region Mask."""
from dataclasses import dataclass
@dataclass(frozen=True)
class RegionContours:
components: tuple
image_size: tuple[int, int]
def __post_init__(self):
components = tuple(
tuple(
tuple((float(x), float(y)) for x, y in contour)
for contour in component
)
for component in self.components
)
image_size = tuple(int(value) for value in self.image_size)
if not components or any(not component for component in components):
raise ValueError("The Region has no usable contour components")
if any(
len(contour) < 3
for component in components
for contour in component
):
raise ValueError("A Region contour has fewer than three points")
if len(image_size) != 2 or min(image_size) <= 0:
raise ValueError("The Region contour image size is invalid")
object.__setattr__(self, "components", components)
object.__setattr__(self, "image_size", image_size)
def _choose_boundary_edge(start, direction, candidates):
right = (-direction[1], direction[0])
left = (direction[1], -direction[0])
back = (-direction[0], -direction[1])
preference = {right: 0, direction: 1, left: 2, back: 3}
return min(
candidates,
key=lambda end: preference.get(
(end[0] - start[0], end[1] - start[1]),
4,
),
)
def _simplify_axis_aligned_contour(points):
simplified = []
for index, current in enumerate(points):
previous = points[index - 1]
following = points[(index + 1) % len(points)]
first = (current[0] - previous[0], current[1] - previous[1])
second = (following[0] - current[0], following[1] - current[1])
if first[0] * second[1] - first[1] * second[0] != 0:
simplified.append(current)
return simplified
def _trace_region_contours(mask):
import numpy as np
mask = np.asarray(mask, dtype=bool)
if mask.ndim != 2:
raise ValueError("Cutout Alpha must be two-dimensional")
height, width = mask.shape
edges = {}
def add_edge(start, end):
edges.setdefault(start, []).append(end)
top = mask.copy()
top[1:] &= ~mask[:-1]
for y, x in np.argwhere(top):
add_edge((int(x), int(y)), (int(x) + 1, int(y)))
right = mask.copy()
right[:, :-1] &= ~mask[:, 1:]
for y, x in np.argwhere(right):
add_edge((int(x) + 1, int(y)), (int(x) + 1, int(y) + 1))
bottom = mask.copy()
bottom[:-1] &= ~mask[1:]
for y, x in np.argwhere(bottom):
add_edge((int(x) + 1, int(y) + 1), (int(x), int(y) + 1))
left = mask.copy()
left[:, 1:] &= ~mask[:, :-1]
for y, x in np.argwhere(left):
add_edge((int(x), int(y) + 1), (int(x), int(y)))
if not edges:
raise ValueError("The Cutout region has no usable boundary")
contours = []
edge_count = sum(len(values) for values in edges.values())
while edges:
start = min(edges, key=lambda point: (point[1], point[0]))
current = start
direction = (1, 0)
contour = []
for _index in range(edge_count + 1):
contour.append(current)
candidates = edges.get(current)
if not candidates:
raise ValueError("The Cutout boundary is open")
following = _choose_boundary_edge(current, direction, candidates)
candidates.remove(following)
if not candidates:
del edges[current]
direction = (following[0] - current[0], following[1] - current[1])
current = following
if current == start:
break
else:
raise ValueError("The Cutout boundary could not be closed")
contour = _simplify_axis_aligned_contour(contour)
if len(contour) >= 3:
contours.append(contour)
if not contours:
raise ValueError("The Cutout region is too small")
return contours
def _signed_contour_area(contour):
return 0.5 * sum(
start[0] * end[1] - end[0] * start[1]
for start, end in zip(contour, contour[1:] + contour[:1])
)
def _point_inside_contour(point, contour):
inside = False
previous = contour[-1]
for current in contour:
if (current[1] > point[1]) != (previous[1] > point[1]):
crossing_x = current[0] + (
(point[1] - current[1])
* (previous[0] - current[0])
/ (previous[1] - current[1])
)
inside ^= point[0] < crossing_x
previous = current
return inside
def _group_region_contours(mask):
contours = _trace_region_contours(mask)
outer = [contour for contour in contours if _signed_contour_area(contour) > 0.0]
holes = [contour for contour in contours if _signed_contour_area(contour) < 0.0]
if not outer:
raise ValueError("The Cutout region has no outer boundary")
regions = [[contour] for contour in outer]
outer_areas = [abs(_signed_contour_area(contour)) for contour in outer]
for hole in holes:
start = hole[0]
end = hole[1]
edge_x = end[0] - start[0]
edge_y = end[1] - start[1]
length = max((edge_x * edge_x + edge_y * edge_y) ** 0.5, 1e-12)
sample = (
(start[0] + end[0]) * 0.5 - edge_y * 0.25 / length,
(start[1] + end[1]) * 0.5 + edge_x * 0.25 / length,
)
owners = [
index
for index, contour in enumerate(outer)
if _point_inside_contour(sample, contour)
]
if not owners:
raise ValueError("The Cutout hole has no enclosing region")
owner = min(owners, key=outer_areas.__getitem__)
regions[owner].append(hole)
return regions
def _resize_values(values, size):
import numpy as np
target_width, target_height = (int(value) for value in size)
source_height, source_width = values.shape
if (target_width, target_height) == (source_width, source_height):
return values
x = (np.arange(target_width, dtype=np.float32) + 0.5) * (
source_width / target_width
) - 0.5
y = (np.arange(target_height, dtype=np.float32) + 0.5) * (
source_height / target_height
) - 0.5
x0_unclipped = np.floor(x).astype(np.int32)
y0_unclipped = np.floor(y).astype(np.int32)
x_weight = x - x0_unclipped
y_weight = y - y0_unclipped
x0 = np.clip(x0_unclipped, 0, source_width - 1)
y0 = np.clip(y0_unclipped, 0, source_height - 1)
x1 = np.clip(x0_unclipped + 1, 0, source_width - 1)
y1 = np.clip(y0_unclipped + 1, 0, source_height - 1)
horizontal = (
values[:, x0] * (1.0 - x_weight)[None, :]
+ values[:, x1] * x_weight[None, :]
)
return (
horizontal[y0] * (1.0 - y_weight)[:, None]
+ horizontal[y1] * y_weight[:, None]
)
def extract_region_contours(region_mask, max_resolution=512, threshold=0.5):
import numpy as np
values = np.asarray(region_mask.values)
if values.ndim != 2:
raise ValueError("Cutout Alpha must be two-dimensional")
if np.issubdtype(values.dtype, np.floating):
values = np.clip(values, 0.0, 1.0) * 255.0
else:
values = np.clip(values, 0, 255).astype(np.float32)
source_size = (values.shape[1], values.shape[0])
scale = min(1.0, float(max_resolution) / max(source_size))
sampled_size = tuple(max(1, round(value * scale)) for value in source_size)
if sampled_size != source_size:
values = _resize_values(values, sampled_size)
mask = values >= 255.0 * threshold
components = _group_region_contours(mask)
scale_x = source_size[0] / sampled_size[0]
scale_y = source_size[1] / sampled_size[1]
return RegionContours(
components=tuple(
tuple(
tuple(
(
min(max(float(x) * scale_x, 0.0), source_size[0]),
min(max(float(y) * scale_y, 0.0), source_size[1]),
)
for x, y in contour
)
for contour in component
)
for component in components
),
image_size=source_size,
)

View File

@@ -1,436 +0,0 @@
import math
POINT_SPACING = 0.1
def contour_area(points):
import numpy as np
points = np.asarray(points, dtype=np.float64)
following = np.roll(points, -1, axis=0)
return 0.5 * float(
np.sum(
points[:, 0] * following[:, 1]
- following[:, 0] * points[:, 1]
)
)
def resample_closed_contour(contour, spacing):
import numpy as np
points = np.asarray(contour, dtype=np.float64)
if len(points) < 3:
raise ValueError("The lasso does not define a valid image region")
segments = np.roll(points, -1, axis=0) - points
lengths = np.linalg.norm(segments, axis=1)
points = points[lengths > 1e-8]
if len(points) < 3:
raise ValueError("The lasso region is too small to create a Cutout")
segments = np.roll(points, -1, axis=0) - points
lengths = np.linalg.norm(segments, axis=1)
perimeter = float(lengths.sum())
if perimeter <= spacing * 3.0:
raise ValueError("The lasso region is too small to create a Cutout")
count = max(3, int(round(perimeter / spacing)))
distances = np.arange(count, dtype=np.float64) * perimeter / count
cumulative = np.concatenate(([0.0], np.cumsum(lengths)))
indices = np.searchsorted(cumulative[1:], distances, side="right")
factors = (distances - cumulative[indices]) / lengths[indices]
return points[indices] + segments[indices] * factors[:, None]
def smooth_region_contour(contour, spacing, iterations=16):
contour = resample_closed_contour(contour, spacing * 0.25)
original_area = abs(contour_area(contour))
for _index in range(iterations):
import numpy as np
contour = (
np.roll(contour, 1, axis=0) * 0.125
+ contour * 0.75
+ np.roll(contour, -1, axis=0) * 0.125
)
smooth_area = abs(contour_area(contour))
if original_area > 0.0 and smooth_area > 0.0:
center = contour.mean(axis=0)
contour = center + (contour - center) * math.sqrt(
original_area / smooth_area
)
contour = resample_closed_contour(contour, spacing)
if contour_area(contour) < 0.0:
contour = contour[::-1].copy()
return contour
def _contour_edges(contours):
import numpy as np
starts = []
ends = []
for contour in contours:
contour = np.asarray(contour, dtype=np.float64)
following = np.roll(contour, -1, axis=0)
usable = np.abs(following[:, 1] - contour[:, 1]) > 1e-12
starts.append(contour[usable])
ends.append(following[usable])
if not starts:
empty = np.empty((0, 2), dtype=np.float64)
return empty, empty
return np.concatenate(starts), np.concatenate(ends)
def _edge_bucket_indices(starts, ends, minimum_y, bucket_height, bucket_count):
import numpy as np
lower = np.minimum(starts[:, 1], ends[:, 1])
upper = np.maximum(starts[:, 1], ends[:, 1])
first = np.floor((lower - minimum_y) / bucket_height).astype(np.int64)
last = np.floor((np.nextafter(upper, lower) - minimum_y) / bucket_height).astype(
np.int64
)
first = np.clip(first, 0, bucket_count - 1)
last = np.clip(last, 0, bucket_count - 1)
buckets = [[] for _index in range(bucket_count)]
for edge_index, (start, stop) in enumerate(zip(first, last)):
for bucket_index in range(int(start), int(stop) + 1):
buckets[bucket_index].append(edge_index)
return buckets
def points_inside_contours(points, contours):
import numpy as np
points = np.asarray(points, dtype=np.float64)
inside = np.zeros(len(points), dtype=bool)
if not len(points):
return inside
starts, ends = _contour_edges(contours)
if not len(starts):
return inside
minimum_y = min(float(points[:, 1].min()), float(starts[:, 1].min()))
maximum_y = max(float(points[:, 1].max()), float(starts[:, 1].max()))
typical_span = float(np.median(np.abs(ends[:, 1] - starts[:, 1])))
bucket_height = max(typical_span, (maximum_y - minimum_y) / 4096.0, 1e-9)
bucket_count = max(1, int(math.ceil((maximum_y - minimum_y) / bucket_height)))
edge_buckets = _edge_bucket_indices(
starts, ends, minimum_y, bucket_height, bucket_count
)
point_buckets = np.floor(
(points[:, 1] - minimum_y) / bucket_height
).astype(np.int64)
point_buckets = np.clip(point_buckets, 0, bucket_count - 1)
order = np.argsort(point_buckets, kind="stable")
ordered_buckets = point_buckets[order]
split = np.flatnonzero(np.diff(ordered_buckets)) + 1
for indices in np.split(order, split):
edge_indices = edge_buckets[int(point_buckets[indices[0]])]
if not edge_indices:
continue
selected_starts = starts[edge_indices]
selected_ends = ends[edge_indices]
sample_y = points[indices, 1, None]
crosses = (selected_starts[None, :, 1] > sample_y) != (
selected_ends[None, :, 1] > sample_y
)
crossing_x = selected_starts[None, :, 0] + (
(sample_y - selected_starts[None, :, 1])
* (selected_ends[None, :, 0] - selected_starts[None, :, 0])
/ (selected_ends[None, :, 1] - selected_starts[None, :, 1])
)
inside[indices] = np.count_nonzero(
crosses & (points[indices, 0, None] < crossing_x), axis=1
) % 2 == 1
return inside
def uniform_interior_points_for_contours(contours, spacing):
import numpy as np
contours = [np.asarray(contour, dtype=np.float64) for contour in contours]
boundary_points = np.concatenate(contours, axis=0)
minimum = boundary_points.min(axis=0)
maximum = boundary_points.max(axis=0)
row_spacing = spacing * math.sqrt(3.0) * 0.5
row_count = int(math.ceil((maximum[1] - minimum[1]) / row_spacing)) + 1
column_count = int(math.ceil((maximum[0] - minimum[0]) / spacing)) + 1
rows = np.arange(row_count, dtype=np.float64)
columns = np.arange(column_count, dtype=np.float64)
y = minimum[1] + rows * row_spacing
x = (
minimum[0]
+ columns[None, :] * spacing
+ (rows.astype(np.int64) % 2)[:, None] * spacing * 0.5
)
x = np.broadcast_to(x, (row_count, column_count))
points = np.column_stack((x.ravel(), np.repeat(y, column_count)))
inside = points_inside_contours(points, contours).reshape(
row_count, column_count
)
clearance = spacing * 0.4
near_boundary = np.zeros_like(inside)
for contour in contours:
for start, end in zip(contour, np.roll(contour, -1, axis=0)):
segment = end - start
length_squared = float(segment.dot(segment))
if length_squared <= 1e-16:
continue
first_row = max(
0,
int(
math.ceil(
(min(start[1], end[1]) - clearance - minimum[1])
/ row_spacing
)
),
)
last_row = min(
row_count - 1,
int(
math.floor(
(max(start[1], end[1]) + clearance - minimum[1])
/ row_spacing
)
),
)
for row in range(first_row, last_row + 1):
offset = spacing * 0.5 if row % 2 else 0.0
first_column = max(
0,
int(
math.ceil(
(min(start[0], end[0]) - clearance - minimum[0] - offset)
/ spacing
)
),
)
last_column = min(
column_count - 1,
int(
math.floor(
(max(start[0], end[0]) + clearance - minimum[0] - offset)
/ spacing
)
),
)
if last_column < first_column:
continue
selected = np.arange(first_column, last_column + 1)
row_points = np.column_stack(
(x[row, selected], np.full(len(selected), y[row]))
)
factor = np.clip(
(row_points - start) @ segment / length_squared, 0.0, 1.0
)
nearest = start + factor[:, None] * segment
close = np.sum((row_points - nearest) ** 2, axis=1) < clearance**2
near_boundary[row, selected[close]] = True
usable = inside & ~near_boundary
return np.column_stack(
(x[usable], np.broadcast_to(y[:, None], x.shape)[usable])
)
def metric_transform(world_metric=None):
import numpy as np
metric = (
np.eye(2, dtype=np.float64)
if world_metric is None
else np.asarray(world_metric, dtype=np.float64)
)
if metric.shape != (2, 2):
raise ValueError("The Image Empty has an invalid world transform")
try:
return np.linalg.cholesky(metric)
except np.linalg.LinAlgError as error:
raise ValueError(
"The Image Empty has a degenerate world transform"
) from error
def prepare_planar_contours(
contours,
image_size,
local_bounds,
spacing=POINT_SPACING,
world_metric=None,
):
import numpy as np
if spacing <= 0.0:
raise ValueError("Cutout point spacing must be greater than zero")
image_width, image_height = image_size
x_min, x_max, y_min, y_max = local_bounds
local_width = x_max - x_min
local_height = y_max - y_min
transform = metric_transform(world_metric)
prepared = []
for contour in contours:
pixels = np.asarray(contour, dtype=np.float64)
if pixels.ndim != 2 or pixels.shape[0] < 3 or pixels.shape[1] != 2:
continue
local = np.empty_like(pixels)
local[:, 0] = x_min + local_width * pixels[:, 0] / image_width
local[:, 1] = y_max - local_height * pixels[:, 1] / image_height
planar = local @ transform
perimeter = float(
np.linalg.norm(np.roll(planar, -1, axis=0) - planar, axis=1).sum()
)
if perimeter <= 1e-12:
continue
contour_spacing = min(spacing, perimeter / 8.0)
smoothed = smooth_region_contour(planar, contour_spacing)
smoothed_local = smoothed @ np.linalg.inv(transform)
smoothed_local[:, 0] = np.clip(smoothed_local[:, 0], x_min, x_max)
smoothed_local[:, 1] = np.clip(smoothed_local[:, 1], y_min, y_max)
prepared.append(smoothed_local @ transform)
if not prepared:
raise ValueError("The Cutout has no region large enough for this resolution")
interior = uniform_interior_points_for_contours(prepared, spacing)
return prepared, interior, spacing
def _prepare_region_contours(
region_contours,
local_bounds,
spacing=POINT_SPACING,
world_metric=None,
):
prepared = []
errors = []
for component in region_contours.components:
try:
contours, interior, _spacing = prepare_planar_contours(
component,
region_contours.image_size,
local_bounds,
spacing=spacing,
world_metric=world_metric,
)
except ValueError as error:
errors.append(error)
continue
prepared.append((contours, interior))
if not prepared:
if errors:
raise errors[0]
raise ValueError("The Cutout has no usable region")
return prepared, spacing
def _triangulate_contour_component(contours, interior, spacing):
import numpy as np
from mathutils import Vector
from mathutils.geometry import delaunay_2d_cdt
input_points = np.concatenate((*contours, interior), axis=0)
boundary_edges = []
offset = 0
for contour in contours:
count = len(contour)
boundary_edges.extend(
(offset + index, offset + (index + 1) % count)
for index in range(count)
)
offset += count
result = delaunay_2d_cdt(
[Vector(point) for point in input_points],
boundary_edges,
[],
0,
max(spacing * 1e-6, 1e-9),
False,
)
planar_points = np.asarray(
[tuple(point) for point in result[0]],
dtype=np.float64,
)
faces = np.asarray(
[tuple(face) for face in result[2] if len(face) == 3],
dtype=np.int32,
)
if len(faces):
centers = planar_points[faces].mean(axis=1)
faces = faces[points_inside_contours(centers, contours)]
if not len(faces):
raise ValueError("Unable to triangulate the Cutout regions")
used = np.unique(faces)
if len(used) != len(planar_points):
remap = np.full(len(planar_points), -1, dtype=np.int32)
remap[used] = np.arange(len(used), dtype=np.int32)
planar_points = planar_points[used]
faces = remap[faces]
boundary = boundary_vertices_from_faces(len(planar_points), faces)
planar_faces = [tuple(int(value) for value in face) for face in faces]
return planar_points, planar_faces, boundary
def boundary_vertices_from_faces(vertex_count, faces):
import numpy as np
faces = np.asarray(faces, dtype=np.int32)
boundary = np.zeros(vertex_count, dtype=bool)
if not len(faces):
return boundary
edges = np.concatenate(
[
faces[:, (index, (index + 1) % faces.shape[1])]
for index in range(faces.shape[1])
],
axis=0,
)
edges.sort(axis=1)
unique_edges, users = np.unique(edges, axis=0, return_counts=True)
boundary[unique_edges[users == 1].ravel()] = True
return boundary
def triangulate_region_contours(
region_contours,
local_bounds,
spacing=POINT_SPACING,
world_metric=None,
return_components=False,
):
import numpy as np
prepared, spacing = _prepare_region_contours(
region_contours,
local_bounds,
spacing=spacing,
world_metric=world_metric,
)
points = []
faces = []
boundaries = []
components = []
offset = 0
for contours, interior in prepared:
region_points, region_faces, region_boundary = _triangulate_contour_component(
contours,
interior,
spacing,
)
components.append(
(
offset,
offset + len(region_points),
region_faces,
contours[0],
)
)
points.append(region_points)
faces.extend(
tuple(offset + value for value in face) for face in region_faces
)
boundaries.append(region_boundary)
offset += len(region_points)
if not points:
raise ValueError("Unable to triangulate the Cutout regions")
result = (
np.concatenate(points, axis=0),
faces,
np.concatenate(boundaries),
)
return (*result, components) if return_components else result

View File

@@ -0,0 +1,126 @@
"""Cutout Region validation and Shape-menu interaction helpers."""
from ...preferences import (
MAX_CUTOUT_MESH_RESOLUTION,
MIN_CUTOUT_MESH_RESOLUTION,
)
from ..common.image_data import image_rgba
from ..common.image_region import rasterize_region_path
from .geometry import CUTOUT_ALPHA_THRESHOLD
def validate_cutout_region(
source_object,
region_path,
):
image = source_object.data
rgba = image_rgba(image)
rasterize_region_path(
tuple(image.size),
region_path,
source_alpha=rgba[:, :, 3],
alpha_threshold=CUTOUT_ALPHA_THRESHOLD,
antialias=True,
)
def cutout_mesh_resolution(image_size, point_spacing):
longest_edge = max(int(value) for value in image_size)
detail = round(longest_edge / max(int(point_spacing), 1))
return max(
MIN_CUTOUT_MESH_RESOLUTION,
min(MAX_CUTOUT_MESH_RESOLUTION, detail),
)
def _pie_button(
pie,
shape,
label,
icon,
source_name,
region_path_json,
refine_region,
generate_normal,
mesh_resolution,
):
operator = pie.operator(
"anyimage.cutout_region_to_shape",
text=label,
icon=icon,
)
operator.source_object_name = source_name
operator.region_path_json = region_path_json
operator.shape = shape
operator.refine_region = refine_region
operator.generate_normal = generate_normal
operator.mesh_resolution = mesh_resolution
def open_shape_pie(
context,
event,
source_object,
region_path,
refine_region=False,
):
source_name = source_object.name
region_path_json = region_path.to_json()
settings = context.scene.anyimage_settings
point_spacing = settings.cutout_point_spacing
generate_normal = bool(getattr(settings, "cutout_generate_normal", False))
mesh_resolution = cutout_mesh_resolution(source_object.data.size, point_spacing)
def draw(menu, _context):
pie = menu.layout.menu_pie()
_pie_button(
pie,
"SURFACE",
"Surface",
"MESH_GRID",
source_name,
region_path_json,
refine_region,
generate_normal,
mesh_resolution,
)
_pie_button(
pie,
"BALLOON",
"Balloon",
"MESH_UVSPHERE",
source_name,
region_path_json,
refine_region,
generate_normal,
mesh_resolution,
)
_pie_button(
pie,
"DEPTH_BALLOON",
"Depth Balloon",
"META_BALL",
source_name,
region_path_json,
refine_region,
generate_normal,
mesh_resolution,
)
_pie_button(
pie,
"DEPTH_SURFACE",
"Depth Surface",
"SURFACE_NCURVE",
source_name,
region_path_json,
refine_region,
generate_normal,
mesh_resolution,
)
context.window_manager.popup_menu_pie(
event,
draw,
title="Cutout Shape",
icon="MESH_DATA",
)

View File

@@ -0,0 +1,455 @@
"""Convert one image Region into a Cutout Shape."""
import json
import bpy
from ...preferences import (
MAX_CUTOUT_MESH_RESOLUTION,
MIN_CUTOUT_MESH_RESOLUTION,
addon_preferences,
configured_max_ai_input_size,
configured_max_cutout_mesh_resolution,
)
from ...properties import (
BEN2_MODEL_KEY,
DEFAULT_MOGE2_MODEL_KEY,
refine_region_property,
)
from ...runtime import JobOperatorBase
from ...server.media.resolution import (
DEFAULT_MAX_AI_INPUT_SIZE,
MAX_MAX_AI_INPUT_SIZE,
MIN_MAX_AI_INPUT_SIZE,
)
from ..common.ai_job import (
invoke_ai_setup_if_needed,
moge2_parameters,
production_device,
require_environment,
require_input_path,
require_model,
)
from ..common.image_data import (
cleanup_image_input,
image_rgba,
load_normal_result_image,
prepare_region_input,
require_image_empty,
source_start_frame,
video_total_frames,
)
from ..common.depth_data import load_depth_metadata, load_depth_result_image
from ..common.image_interaction import (
ImageRegionGesture,
apply_region_settings,
draw_region_settings,
gesture_property,
report_image_edit_exception,
resolve_image_edit_click,
serialize_matrix,
)
from ..common.image_region import (
ImageEditWarning,
RegionMask,
RegionPath,
rasterize_region_path,
)
from .geometry import (
CUTOUT_ALPHA_THRESHOLD,
build_base_shape,
limit_cutout_mesh_resolution,
)
from .interaction import open_shape_pie, validate_cutout_region
from .object import NormalMode, create_shape_object
from .shape import CUTOUT_SHAPES, DEPTH_CUTOUT_SHAPES
_HIDDEN = {"HIDDEN", "SKIP_SAVE"}
def normal_mode_for_shape(shape, generate_normal):
if not generate_normal:
return NormalMode.NONE
if shape == "DEPTH_SURFACE":
return NormalMode.OBJECT
return NormalMode.TANGENT
def depth_mode_for_shape(shape):
if shape == "DEPTH_SURFACE":
return "VECTOR"
if shape == "DEPTH_BALLOON":
return "Z"
return "NONE"
def create_cutout_shape(
context,
source_object,
shape,
mesh_resolution,
region_mask,
*,
depth_image=None,
depth_metadata=None,
normal_image=None,
normal_mode=NormalMode.NONE,
source_rgba=None,
):
base_shape = build_base_shape(
context,
source_object,
mesh_resolution,
region_mask,
)
result = create_shape_object(
context,
source_object,
shape,
mesh_resolution,
region_mask,
base_shape,
depth_image=depth_image,
depth_metadata=depth_metadata,
normal_image=normal_image,
normal_mode=normal_mode,
source_rgba=source_rgba,
)
undo_message = f"Convert to {shape.replace('_', ' ').title()}"
if "FINISHED" not in bpy.ops.ed.undo_push(message=undo_message):
raise RuntimeError(f"Unable to create the {undo_message} Undo step")
return result
def _region_mask_from_image(path, bounds):
image = bpy.data.images.load(str(path), check_existing=False)
try:
return RegionMask(
image_rgba(image)[:, :, 3],
tuple(int(value) for value in bounds),
)
finally:
bpy.data.images.remove(image, do_unlink=True)
def _generated_region_mask(result, input_path, source_bounds):
if not isinstance(result.value, dict) or "region_image" not in result.value:
return _region_mask_from_image(input_path, source_bounds)
local_bounds = tuple(
int(value)
for value in json.loads(
result.file("region_bounds").read_text(encoding="utf-8")
)
)
bounds = (
source_bounds[0] + local_bounds[0],
source_bounds[1] + local_bounds[1],
source_bounds[0] + local_bounds[2],
source_bounds[1] + local_bounds[3],
)
return _region_mask_from_image(result.file("region_image"), bounds)
def cutout_geometry_settings(context):
preferences = addon_preferences(context)
model = getattr(preferences, "geometry_model", DEFAULT_MOGE2_MODEL_KEY)
return (
model,
int(getattr(preferences, "geometry_resolution_level", 5)),
)
class CutoutRegionToShape(JobOperatorBase, bpy.types.Operator):
bl_idname = "anyimage.cutout_region_to_shape"
bl_label = "Cutout Region to Shape"
bl_description = "Build the selected image Region as a Cutout shape"
bl_options = {"INTERNAL"}
job_type = "generate-cutout-artifacts"
starting_message = "Generating Cutout artifacts"
input_path: bpy.props.StringProperty(options=_HIDDEN)
source_object_name: bpy.props.StringProperty(options=_HIDDEN)
region_path_json: bpy.props.StringProperty(options=_HIDDEN)
region_bounds_json: bpy.props.StringProperty(options=_HIDDEN)
shape: bpy.props.StringProperty(options=_HIDDEN)
mesh_resolution: bpy.props.IntProperty(
default=128,
min=MIN_CUTOUT_MESH_RESOLUTION,
max=MAX_CUTOUT_MESH_RESOLUTION,
options=_HIDDEN,
)
model: bpy.props.StringProperty(
default=DEFAULT_MOGE2_MODEL_KEY,
options=_HIDDEN,
)
resolution_level: bpy.props.IntProperty(
default=5,
min=0,
max=9,
options=_HIDDEN,
)
max_ai_input_size: bpy.props.IntProperty(
default=DEFAULT_MAX_AI_INPUT_SIZE,
min=MIN_MAX_AI_INPUT_SIZE,
max=MAX_MAX_AI_INPUT_SIZE,
options=_HIDDEN,
)
generate_normal: bpy.props.BoolProperty(default=False, options=_HIDDEN)
refine_region: bpy.props.BoolProperty(default=False, options=_HIDDEN)
def _needs_generation(self):
return (
self.refine_region
or self.shape in DEPTH_CUTOUT_SHAPES
or self.generate_normal
)
def execute(self, context):
try:
source_object = require_image_empty(self.source_object_name)
if video_total_frames(source_object.data) != 1:
raise ValueError("Cutout currently supports still images")
if self.shape not in CUTOUT_SHAPES:
raise ValueError(f"Unsupported Cutout shape: {self.shape}")
region_path = RegionPath.from_json(self.region_path_json)
source_rgba = image_rgba(source_object.data)
region_mask = rasterize_region_path(
tuple(source_object.data.size),
region_path,
source_alpha=source_rgba[:, :, 3],
alpha_threshold=CUTOUT_ALPHA_THRESHOLD,
antialias=True,
)
context.view_layer.objects.active = source_object
source_object.select_set(True)
crop_width = region_mask.bounds[2] - region_mask.bounds[0]
crop_height = region_mask.bounds[3] - region_mask.bounds[1]
self.mesh_resolution = limit_cutout_mesh_resolution(
self.mesh_resolution,
(crop_width, crop_height),
configured_max_cutout_mesh_resolution(context),
)
self.max_ai_input_size = configured_max_ai_input_size(context)
if not self._needs_generation():
create_cutout_shape(
context,
source_object,
self.shape,
self.mesh_resolution,
region_mask,
source_rgba=source_rgba,
)
return {"FINISHED"}
setup_result = invoke_ai_setup_if_needed()
if setup_result is not None:
return setup_result
self.model, self.resolution_level = cutout_geometry_settings(context)
self.region_bounds_json = json.dumps(
region_mask.bounds,
separators=(",", ":"),
)
self.input_path = str(prepare_region_input(source_object.data, region_mask))
except (
ImageEditWarning,
OSError,
RuntimeError,
TypeError,
ValueError,
) as error:
report_image_edit_exception(self, error)
return {"CANCELLED"}
result = super().execute(context)
if "CANCELLED" in result:
cleanup_image_input(self.input_path, True)
return result
def request(self, context):
require_environment()
source_path = require_input_path(self.input_path)
depth_mode = depth_mode_for_shape(self.shape)
normal_mode = normal_mode_for_shape(self.shape, self.generate_normal)
if self.refine_region:
require_model(BEN2_MODEL_KEY)
if depth_mode != "NONE" or normal_mode != NormalMode.NONE:
require_model(self.model)
parameters = {
"input": str(source_path),
"device": production_device().lower(),
"refine_region": self.refine_region,
"depth_mode": depth_mode,
"normal_mode": normal_mode.value,
"max_input_size": int(self.max_ai_input_size),
}
if self.refine_region:
parameters["refine_model"] = BEN2_MODEL_KEY
if depth_mode != "NONE" or normal_mode != NormalMode.NONE:
start_frame = source_start_frame(self.source_object_name)
parameters.update(
model=self.model,
resolution_level=self.resolution_level,
)
parameters.update(
moge2_parameters(
context.scene.anyimage_settings,
source_path,
self.model,
self.resolution_level,
start_frame,
)
)
return parameters
def response(self, context, result):
source_object = require_image_empty(self.source_object_name)
source_bounds = tuple(json.loads(self.region_bounds_json))
region_mask = _generated_region_mask(
result,
self.input_path,
source_bounds,
)
value = result.value if isinstance(result.value, dict) else {}
normal_image = None
depth_image = None
try:
normal_mode = normal_mode_for_shape(self.shape, self.generate_normal)
normal_key = f"{normal_mode.value.lower()}_normal"
if normal_key in value:
normal_image = load_normal_result_image(
result.file(normal_key),
source_object,
)
depth_metadata = None
depth_mode = depth_mode_for_shape(self.shape)
if depth_mode != "NONE":
depth_metadata = load_depth_metadata(result.file("depth_metadata"))
depth_image = load_depth_result_image(
result.file(f"{depth_mode.lower()}_depth"),
source_object,
depth_metadata,
)
create_cutout_shape(
context,
source_object,
self.shape,
self.mesh_resolution,
region_mask,
depth_image=depth_image,
depth_metadata=depth_metadata,
normal_image=normal_image,
normal_mode=normal_mode,
)
except Exception:
if normal_image is not None and normal_image.users == 0:
bpy.data.images.remove(normal_image, do_unlink=True)
if depth_image is not None and depth_image.users == 0:
bpy.data.images.remove(depth_image, do_unlink=True)
raise
return f"{self.shape.replace('_', ' ').title()} is ready"
def cleanup(self):
cleanup_image_input(self.input_path, True)
class SelectCutoutRegion(ImageRegionGesture, bpy.types.Operator):
bl_idname = "anyimage.select_cutout_region"
bl_label = "Select Cutout Region"
bl_description = "Lasso an Image Empty region, then choose its mesh shape"
bl_options = {"INTERNAL"}
active_tool_id = "anyimage.cutout_lasso"
refine_region_settings = {"LASSO": "cutout_refine_region"}
source_object_name: bpy.props.StringProperty(options={"HIDDEN", "SKIP_SAVE"})
source_matrix_data: bpy.props.StringProperty(options={"HIDDEN", "SKIP_SAVE"})
region_path_json: bpy.props.StringProperty(options={"HIDDEN", "SKIP_SAVE"})
refine_region: refine_region_property()
invert: bpy.props.BoolProperty(default=False, options={"SKIP_SAVE"})
gesture: gesture_property()
full_image: bpy.props.BoolProperty(default=False, options={"HIDDEN", "SKIP_SAVE"})
preview_fill_rule = "NONZERO"
_pie_event = None
def _set_status(self, context):
invert = "On" if self.invert else "Off"
context.workspace.status_text_set(
f"Cutout Lasso: drag and release for shapes; "
f"F toggles Invert ({invert}); RMB or Esc cancels"
)
def invoke(self, context, event):
self._pie_event = event
if not self.full_image and event.value != "DOUBLE_CLICK":
return super().invoke(context, event)
source_object, should_edit = resolve_image_edit_click(
context, event, self.report
)
if not should_edit:
return {"FINISHED"} if source_object is not None else {"CANCELLED"}
if source_object is None:
return {"CANCELLED"}
if video_total_frames(source_object.data) > 1:
self.report({"WARNING"}, "Image tools only support a still image")
return {"CANCELLED"}
apply_region_settings(self, context)
self.invert = False
self.source_object_name = source_object.name
self.source_matrix_data = serialize_matrix(source_object.matrix_world)
width, height = (int(value) for value in source_object.data.size)
self.region_path_json = RegionPath(
points=((0, 0), (width, 0), (width, height), (0, height)),
fill_rule="NONZERO",
).to_json()
return self.execute(context)
def modal(self, context, event):
self._pie_event = event
return super().modal(context, event)
def execute(self, context):
try:
source_object = require_image_empty(self.source_object_name)
except RuntimeError as error:
self.report({"ERROR"}, str(error))
return {"CANCELLED"}
try:
region_path = RegionPath.from_json(self.region_path_json)
validate_cutout_region(source_object, region_path)
except (ImageEditWarning, TypeError, ValueError) as error:
report_image_edit_exception(self, error)
return {"CANCELLED"}
open_shape_pie(
context,
self._pie_event,
source_object,
region_path,
self.refine_region,
)
return {"FINISHED"}
class CutoutTool(bpy.types.WorkSpaceTool):
bl_space_type = "VIEW_3D"
bl_context_mode = "OBJECT"
bl_idname = "anyimage.cutout_lasso"
bl_label = "Cutout"
bl_description = "Lasso an Image Empty and choose a mesh shape"
bl_icon = "ops.sculpt.lasso_mask"
bl_operator = SelectCutoutRegion.bl_idname
bl_keymap = (
(
SelectCutoutRegion.bl_idname,
{"type": "LEFTMOUSE", "value": "DOUBLE_CLICK"},
{"properties": [("gesture", "LASSO"), ("full_image", True)]},
),
(
SelectCutoutRegion.bl_idname,
{"type": "LEFTMOUSE", "value": "PRESS"},
{"properties": [("gesture", "LASSO")]},
),
)
def draw_settings(context, layout, _tool):
settings = context.scene.anyimage_settings
draw_region_settings(context, layout, "cutout_refine_region")
layout.prop(settings, "cutout_point_spacing")
layout.prop(settings, "cutout_generate_normal")

View File

@@ -0,0 +1,292 @@
"""Create Blender data for a Cutout Shape."""
from enum import StrEnum
import bpy
from ..common.image_data import (
color_data_name,
image_rgba,
)
from ..common.depth_data import camera_depth_values, depth_uniform_scale
from ..common.mesh_result import (
create_image_material,
finalize_object_result,
load_asset_node_group,
modifier_input_identifier,
set_modifier_input,
)
from .geometry import (
DEFAULT_DEPTH_BASELINE,
DEFAULT_DEPTH_DIRECTION,
crop_plane_bounds,
cutout_cleanup_value,
cutout_point_spacing,
depth_balloon_projection_parameters,
metric_depth_reference,
region_calibration,
use_adaptive_depth_calibration,
world_plane_metric,
)
from .shape import (
BALLOON_ATTRIBUTE_NAME,
CUTOUT_NODE_GROUP_NAME,
CUTOUT_SHAPE_INDICES,
DEPTH_CUTOUT_SHAPES,
)
class NormalMode(StrEnum):
NONE = "NONE"
TANGENT = "TANGENT"
OBJECT = "OBJECT"
CUTOUT_AXIS_TO_SOURCE = (
(0.0, -1.0, 0.0, 0.0),
(0.0, 0.0, 1.0, 0.0),
(-1.0, 0.0, 0.0, 0.0),
(0.0, 0.0, 0.0, 1.0),
)
def _cutout_world_matrix(source_matrix, crop_center):
from mathutils import Matrix
center_x, center_y = (float(value) for value in crop_center)
return (
source_matrix
@ Matrix.Translation((center_x, center_y, 0.0))
@ Matrix(CUTOUT_AXIS_TO_SOURCE)
)
def _assign_uv(mesh, uv):
import numpy as np
for layer in list(mesh.uv_layers):
mesh.uv_layers.remove(layer)
uv_layer = mesh.uv_layers.new(name="UVMap")
loop_vertices = np.empty(len(mesh.loops), dtype=np.int32)
mesh.loops.foreach_get("vertex_index", loop_vertices)
loop_uv = np.asarray(uv, dtype=np.float32)[loop_vertices]
uv_layer.data.foreach_set("uv", loop_uv.ravel())
def _assign_float_point_attribute(mesh, name, values):
import numpy as np
attribute = mesh.attributes.new(name=name, type="FLOAT", domain="POINT")
attribute.data.foreach_set("value", np.asarray(values, dtype=np.float32))
def _cropped_color_image(source_object, region_mask, source_rgba):
crop_bounds = region_mask.bounds
import numpy as np
source = source_object.data
rgba = np.asarray(source_rgba)
left, top, right, bottom = crop_bounds
width = right - left
height = bottom - top
if width <= 0 or height <= 0:
raise ValueError("The Cutout crop dimensions must be positive")
cropped = np.zeros((height, width, 4), dtype=np.float32)
source_height, source_width = rgba.shape[:2]
source_left = max(0, left)
source_top = max(0, top)
source_right = min(source_width, right)
source_bottom = min(source_height, bottom)
if source_right > source_left and source_bottom > source_top:
cropped[
source_top - top : source_bottom - top,
source_left - left : source_right - left,
] = rgba[source_top:source_bottom, source_left:source_right]
image = bpy.data.images.new(
color_data_name(source_object),
width=width,
height=height,
alpha=True,
)
try:
image.colorspace_settings.name = source.colorspace_settings.name
image.alpha_mode = source.alpha_mode
image.pixels.foreach_set(np.flipud(cropped).ravel())
image.pack()
image.update()
except Exception:
bpy.data.images.remove(image, do_unlink=True)
raise
return image
def load_cutout_node_group():
return load_asset_node_group(CUTOUT_NODE_GROUP_NAME)
def create_shape_object(
context,
source_object,
shape,
mesh_resolution,
region_mask,
base_shape,
*,
depth_image=None,
depth_metadata=None,
normal_image=None,
normal_mode=NormalMode.NONE,
source_rgba=None,
):
import numpy as np
normal_mode = NormalMode(normal_mode)
bounds = crop_plane_bounds(
source_object,
region_mask.bounds,
tuple(source_object.data.size),
)
crop_center = (
(bounds[0] + bounds[1]) * 0.5,
(bounds[2] + bounds[3]) * 0.5,
)
world_metric, _world_z_scale = world_plane_metric(source_object.matrix_world)
cleanup_value = cutout_cleanup_value(
cutout_point_spacing(source_object, mesh_resolution),
world_metric,
)
color_image = _cropped_color_image(
source_object,
region_mask,
image_rgba(source_object.data) if source_rgba is None else source_rgba,
)
material = None
mesh = None
mesh_object = None
try:
material = create_image_material(
source_object,
color_image,
0.0,
normal_image=normal_image,
normal_space=(
NormalMode.TANGENT.value
if normal_mode == NormalMode.NONE
else normal_mode.value
),
ior=1.2,
scene=context.scene,
)
depth_reference = None
uniform_scale = None
depth_direction = None
if shape in DEPTH_CUTOUT_SHAPES:
if depth_image is None or depth_metadata is None:
raise ValueError("Depth Shape requires a Depth texture and metadata")
camera_depth, valid = camera_depth_values(depth_image)
calibration_valid, transparent_fraction = region_calibration(
region_mask, valid.shape
)
calibration_valid &= valid
uniform_scale = depth_uniform_scale(
source_object,
depth_metadata,
bounds,
)
if not use_adaptive_depth_calibration(transparent_fraction):
depth_reference = DEFAULT_DEPTH_BASELINE
if shape == "DEPTH_BALLOON":
depth_direction = DEFAULT_DEPTH_DIRECTION
elif shape == "DEPTH_SURFACE":
depth_reference = metric_depth_reference(
camera_depth,
calibration_valid,
uniform_scale,
)
else:
depth_reference, depth_direction = depth_balloon_projection_parameters(
base_shape.vertices,
base_shape.uv,
camera_depth,
calibration_valid,
uniform_scale,
)
source_name = source_object.name
mesh = bpy.data.meshes.new(source_name)
mesh.from_pydata(
np.asarray(base_shape.vertices).tolist(),
[],
base_shape.faces,
)
for face in mesh.polygons:
face.use_smooth = True
_assign_uv(mesh, base_shape.uv)
_assign_float_point_attribute(
mesh,
BALLOON_ATTRIBUTE_NAME,
base_shape.balloon,
)
mesh.materials.append(material)
mesh.update()
mesh_object = bpy.data.objects.new(source_name, mesh)
context.collection.objects.link(mesh_object)
mesh_object.matrix_world = _cutout_world_matrix(
source_object.matrix_world,
crop_center,
)
mesh_object["anyimage_mesh_shape"] = shape
node_group = load_cutout_node_group()
modifier = mesh_object.modifiers.new(name=node_group.name, type="NODES")
modifier.node_group = node_group
modifier.show_group_selector = False
for name, value in (
("Shape", CUTOUT_SHAPE_INDICES[shape]),
("Cleanup", cleanup_value),
("Smooth Iterations", 0),
("Smooth", 1.0),
(
"Thickness",
1.0 if shape in {"BALLOON", "DEPTH_BALLOON"} else 0.0,
),
):
set_modifier_input(
modifier,
modifier_input_identifier(node_group, name),
value,
)
if shape in DEPTH_CUTOUT_SHAPES:
for name, value in (
("Depth Scale", 1.0),
("Uniform Scale", uniform_scale),
("Base Plane Depth", depth_reference),
("Depth Image", depth_image),
):
set_modifier_input(
modifier,
modifier_input_identifier(node_group, name),
value,
)
if shape == "DEPTH_BALLOON":
set_modifier_input(
modifier,
modifier_input_identifier(node_group, "Depth Direction"),
depth_direction,
)
return finalize_object_result(
context,
source_object,
mesh_object,
keep_source=True,
)
except Exception:
if mesh_object is not None:
bpy.data.objects.remove(mesh_object, do_unlink=True)
if mesh is not None and mesh.users == 0:
bpy.data.meshes.remove(mesh)
if material is not None and material.users == 0:
bpy.data.materials.remove(material)
if color_image.users == 0:
bpy.data.images.remove(color_image, do_unlink=True)
raise

View File

@@ -1,842 +0,0 @@
import json
from dataclasses import dataclass
from enum import StrEnum
import bpy
from ...preferences import addon_preferences, configured_max_ai_input_size
from ...properties import BEN2_MODEL_KEY, DEFAULT_MOGE2_MODEL_KEY
from ...runtime import JobOperatorBase
from ...server.media.resolution import limited_image_size
from ...server.media.resolution import (
DEFAULT_MAX_AI_INPUT_SIZE,
MAX_MAX_AI_INPUT_SIZE,
MIN_MAX_AI_INPUT_SIZE,
)
from ..common.image_data import (
cleanup_image_input,
color_data_name,
depth_data_name,
image_rgba,
load_normal_result_image,
image_empty_bounds,
prepare_region_input,
require_image_empty,
source_start_frame,
video_total_frames,
)
from ..common.image_interaction import report_image_edit_exception
from ..common.image_region import (
ImageEditWarning,
RegionMask,
RegionPath,
rasterize_region_path,
)
from ..common.mesh_result import (
create_image_material,
finalize_object_result,
load_asset_node_group,
load_image_geometry,
modifier_input_identifier,
set_modifier_input,
)
from ..common.ai_job import (
invoke_ai_setup_if_needed,
moge2_parameters,
production_device,
require_environment,
require_input_path,
require_model,
)
from .geometry.depth_calibration import (
DEFAULT_DEPTH_BASELINE,
DEFAULT_DEPTH_DIRECTION,
depth_balloon_projection_parameters,
metric_depth_reference,
use_adaptive_depth_calibration,
)
from .geometry.triangulation import (
metric_transform,
triangulate_region_contours,
)
from .geometry.balloon import balloon_profile as calculate_balloon_profile
from .geometry.region_contours import extract_region_contours
CUTOUT_ALPHA_THRESHOLD = 0.1
CUTOUT_CLEANUP_UI_SCALE = 100.0
BALLOON_ATTRIBUTE_NAME = "o_balloon"
CUTOUT_NODE_GROUP_NAME = "O Image Cutout"
CUTOUT_NODE_GROUP_NAMES = {
"SURFACE": "O Image Surface",
"BALLOON": "O Image Balloon",
"DEPTH_BALLOON": "O Image Depth Balloon",
"DEPTH_SURFACE": "O Image Depth Surface",
}
CUTOUT_SHAPE_INDICES = {
shape: index for index, shape in enumerate(CUTOUT_NODE_GROUP_NAMES)
}
CUTOUT_SHAPES = frozenset(CUTOUT_NODE_GROUP_NAMES)
DEPTH_CUTOUT_SHAPES = frozenset({"DEPTH_BALLOON", "DEPTH_SURFACE"})
_HIDDEN = {"HIDDEN", "SKIP_SAVE"}
class NormalMode(StrEnum):
NONE = "NONE"
TANGENT = "TANGENT"
OBJECT = "OBJECT"
@dataclass(frozen=True)
class BaseShape:
vertices: object
faces: object
uv: object
balloon: object
CUTOUT_AXIS_TO_SOURCE = (
(0.0, -1.0, 0.0, 0.0),
(0.0, 0.0, 1.0, 0.0),
(-1.0, 0.0, 0.0, 0.0),
(0.0, 0.0, 0.0, 1.0),
)
def _cutout_depth_image(source_object, points, valid, camera_depth):
import numpy as np
points = np.asarray(points, dtype=np.float32)
valid = np.asarray(valid, dtype=bool)
height, width = valid.shape
if points.shape != (height, width, 3):
raise ValueError("Depth Surface point dimensions do not match")
discontinuity = _depth_discontinuity_map(camera_depth, valid)
discontinuity[~valid] = -1.0
pixels = np.empty((height, width, 4), dtype=np.float32)
pixels[:, :, :3] = points
pixels[:, :, 3] = discontinuity
image = bpy.data.images.new(
depth_data_name(source_object),
width=width,
height=height,
alpha=True,
float_buffer=True,
)
image.colorspace_settings.name = "Non-Color"
image.alpha_mode = "CHANNEL_PACKED"
image.pixels.foreach_set(np.flipud(pixels).ravel())
image.pack()
return image
def _cutout_world_matrix(source_matrix, crop_center):
from mathutils import Matrix
center_x, center_y = (float(value) for value in crop_center)
return (
source_matrix
@ Matrix.Translation((center_x, center_y, 0.0))
@ Matrix(CUTOUT_AXIS_TO_SOURCE)
)
def _orient_cutout_vertices(vertices):
import numpy as np
vertices = np.asarray(vertices, dtype=np.float64)
return np.column_stack(
(
-vertices[:, 2],
-vertices[:, 0],
vertices[:, 1],
)
)
def _cutout_domain(
region_contours,
bounds,
spacing,
world_metric,
balloon_algorithm="POISSON",
):
import numpy as np
x_min, x_max, y_min, y_max = bounds
planar, planar_faces, boundary, components = triangulate_region_contours(
region_contours,
bounds,
spacing=spacing,
world_metric=world_metric,
return_components=True,
)
profile = calculate_balloon_profile(
balloon_algorithm, planar, planar_faces, boundary, components
)
faces = np.asarray(planar_faces, dtype=np.int32)
local = planar @ np.linalg.inv(metric_transform(world_metric))
uv = np.column_stack(
(
(local[:, 0] - x_min) / (x_max - x_min),
(local[:, 1] - y_min) / (y_max - y_min),
)
).astype(np.float32)
crop_center = np.asarray(
((x_min + x_max) * 0.5, (y_min + y_max) * 0.5),
dtype=np.float64,
)
local -= crop_center
vertices = np.column_stack((local, np.zeros(len(local), dtype=np.float64)))
return vertices, faces, uv, boundary, planar, profile
def _world_plane_metric(matrix_world):
from mathutils import Vector
linear = matrix_world.to_3x3()
x = linear @ Vector((1.0, 0.0, 0.0))
y = linear @ Vector((0.0, 1.0, 0.0))
z = linear @ Vector((0.0, 0.0, 1.0))
return ((x.dot(x), x.dot(y)), (x.dot(y), y.dot(y))), z.length
def _cutout_cleanup_value(spacing, world_metric):
import numpy as np
if spacing <= 0.0:
raise ValueError("Cutout point spacing must be greater than zero")
area_scale = abs(float(np.linalg.det(metric_transform(world_metric))))
if area_scale <= 1e-12:
raise ValueError("The Image Empty has a degenerate world transform")
return CUTOUT_CLEANUP_UI_SCALE * float(spacing) ** 2 / area_scale
def _depth_uniform_scale(
source_object,
field,
bounds=None,
):
import numpy as np
intrinsics = field.intrinsics
width, height = field.image_size
x_min, x_max, y_min, y_max = bounds or image_empty_bounds(source_object)
plane_width = x_max - x_min
plane_height = y_max - y_min
fx, fy = float(intrinsics[0, 0]), float(intrinsics[1, 1])
if fx <= 0.0 or fy <= 0.0:
raise ValueError("Camera intrinsics contain an invalid focal length")
scale_x = plane_width * fx / (max(width - 1, 1) * field.reference_depth)
scale_y = plane_height * fy / (max(height - 1, 1) * field.reference_depth)
return float(np.sqrt(scale_x * scale_y))
def _depth_discontinuity_map(depth, valid):
import numpy as np
depth = np.asarray(depth, dtype=np.float32)
valid = np.asarray(valid, dtype=bool) & np.isfinite(depth) & (depth > 0.0)
padded_depth = np.pad(depth, 1, mode="edge")
padded_valid = np.pad(valid, 1, mode="edge")
windows = np.lib.stride_tricks.sliding_window_view(padded_depth, (3, 3))
usable = np.lib.stride_tricks.sliding_window_view(padded_valid, (3, 3))
minimum = np.min(np.where(usable, windows, np.inf), axis=(-2, -1))
maximum = np.max(np.where(usable, windows, -np.inf), axis=(-2, -1))
result = (maximum - minimum) / np.maximum(np.abs(depth), 1e-6)
result[~usable.all(axis=(-2, -1))] = 1.0
return result.astype(np.float32)
def _counter_clockwise_faces(points, faces):
import numpy as np
points = np.asarray(points, dtype=np.float64)
result = np.asarray(faces, dtype=np.int32).copy()
polygons = points[result]
following = np.roll(polygons, -1, axis=1)
signed_area = np.sum(
polygons[..., 0] * following[..., 1]
- polygons[..., 1] * following[..., 0],
axis=1,
)
result[signed_area < 0.0] = result[signed_area < 0.0, ::-1]
return result
def _assign_uv(mesh, uv):
import numpy as np
for layer in list(mesh.uv_layers):
mesh.uv_layers.remove(layer)
uv_layer = mesh.uv_layers.new(name="UVMap")
loop_vertices = np.empty(len(mesh.loops), dtype=np.int32)
mesh.loops.foreach_get("vertex_index", loop_vertices)
loop_uv = np.asarray(uv, dtype=np.float32)[loop_vertices]
uv_layer.data.foreach_set("uv", loop_uv.ravel())
def _assign_float_point_attribute(mesh, name, values):
import numpy as np
attribute = mesh.attributes.new(name=name, type="FLOAT", domain="POINT")
attribute.data.foreach_set("value", np.asarray(values, dtype=np.float32))
def _crop_plane_bounds(source_object, crop_bounds, source_size):
source_width, source_height = (float(value) for value in source_size)
left, top, right, bottom = (float(value) for value in crop_bounds)
x_min, x_max, y_min, y_max = image_empty_bounds(source_object)
width = x_max - x_min
height = y_max - y_min
return (
x_min + left * width / source_width,
x_min + right * width / source_width,
y_max - bottom * height / source_height,
y_max - top * height / source_height,
)
def _cropped_color_image(source_object, crop_bounds, source_rgba):
import numpy as np
source = source_object.data
rgba = np.asarray(source_rgba)
left, top, right, bottom = crop_bounds
width = right - left
height = bottom - top
if width <= 0 or height <= 0:
raise ValueError("The Cutout crop dimensions must be positive")
cropped = np.zeros((height, width, 4), dtype=np.float32)
source_height, source_width = rgba.shape[:2]
source_left = max(0, left)
source_top = max(0, top)
source_right = min(source_width, right)
source_bottom = min(source_height, bottom)
if source_right > source_left and source_bottom > source_top:
cropped[
source_top - top : source_bottom - top,
source_left - left : source_right - left,
] = rgba[source_top:source_bottom, source_left:source_right]
image = bpy.data.images.new(
color_data_name(source_object),
width=width,
height=height,
alpha=True,
)
try:
image.colorspace_settings.name = source.colorspace_settings.name
image.alpha_mode = source.alpha_mode
image.pixels.foreach_set(np.flipud(cropped).ravel())
image.pack()
image.update()
except Exception:
bpy.data.images.remove(image, do_unlink=True)
raise
return image
def _extract_cutout_contours(
region_mask,
mesh_detail,
threshold=CUTOUT_ALPHA_THRESHOLD,
):
contour_resolution = max(512, min(int(mesh_detail) * 8, 1024))
return extract_region_contours(
region_mask,
max_resolution=contour_resolution,
threshold=threshold,
)
def cutout_node_group(shape):
if shape not in CUTOUT_SHAPES:
raise ValueError(f"Unsupported Cutout shape: {shape}")
return load_asset_node_group(CUTOUT_NODE_GROUP_NAME)
def limited_cutout_mesh_detail(mesh_detail, crop_size, max_input_size):
crop_width, crop_height = (int(value) for value in crop_size)
limited_width, limited_height = limited_image_size(
(crop_width, crop_height),
max_input_size,
)
scale = max(limited_width, limited_height) / max(crop_width, crop_height)
return max(4, min(1024, round(int(mesh_detail) * scale)))
def _cutout_point_spacing(source_object, detail):
import math
x_min, x_max, y_min, y_max = image_empty_bounds(source_object)
metric, _world_z_scale = _world_plane_metric(source_object.matrix_world)
width = (x_max - x_min) * math.sqrt(max(metric[0][0], 0.0))
height = (y_max - y_min) * math.sqrt(max(metric[1][1], 0.0))
return max(width, height) / max(int(detail), 8)
def build_base_shape(context, source_object, detail, region_mask):
import numpy as np
bounds = _crop_plane_bounds(
source_object,
region_mask.bounds,
tuple(source_object.data.size),
)
region_contours = _extract_cutout_contours(
region_mask,
detail,
threshold=CUTOUT_ALPHA_THRESHOLD,
)
world_metric, world_z_scale = _world_plane_metric(source_object.matrix_world)
if world_z_scale <= 1e-10:
raise ValueError("The Image Empty has a degenerate world transform")
preferences = addon_preferences(context)
balloon_algorithm = getattr(
preferences, "balloon_profile_algorithm", "POISSON"
)
point_spacing = _cutout_point_spacing(source_object, detail)
domain = _cutout_domain(
region_contours,
bounds,
point_spacing,
world_metric,
balloon_algorithm,
)
source_vertices, faces, uv, _boundary, planar, balloon = domain
return BaseShape(
vertices=_orient_cutout_vertices(source_vertices),
faces=_counter_clockwise_faces(planar, faces),
uv=np.asarray(uv, dtype=np.float32),
balloon=np.asarray(balloon, dtype=np.float64) / world_z_scale,
)
def create_shape_object(
context,
source_object,
shape,
detail,
region_mask,
base_shape,
*,
field=None,
normal_image=None,
normal_mode=NormalMode.NONE,
source_rgba=None,
):
import numpy as np
normal_mode = NormalMode(normal_mode)
bounds = _crop_plane_bounds(
source_object,
region_mask.bounds,
tuple(source_object.data.size),
)
crop_center = (
(bounds[0] + bounds[1]) * 0.5,
(bounds[2] + bounds[3]) * 0.5,
)
world_metric, _world_z_scale = _world_plane_metric(source_object.matrix_world)
cleanup_value = _cutout_cleanup_value(
_cutout_point_spacing(source_object, detail),
world_metric,
)
color_image = _cropped_color_image(
source_object,
region_mask.bounds,
image_rgba(source_object.data) if source_rgba is None else source_rgba,
)
material = None
depth_image = None
mesh = None
mesh_object = None
try:
material = create_image_material(
source_object,
color_image,
0.0,
normal_image=normal_image,
normal_space=(
NormalMode.TANGENT.value
if normal_mode == NormalMode.NONE
else normal_mode.value
),
scene=context.scene,
)
depth_reference = None
depth_uniform_scale = None
depth_direction = None
if shape in DEPTH_CUTOUT_SHAPES:
points = field.points
valid = field.valid
calibration_valid = field.calibration_valid
depth_image = _cutout_depth_image(
source_object, points, valid, points[..., 2]
)
depth_uniform_scale = _depth_uniform_scale(
source_object,
field,
bounds,
)
if not use_adaptive_depth_calibration(
field.region_transparent_fraction
):
depth_reference = DEFAULT_DEPTH_BASELINE
if shape == "DEPTH_BALLOON":
depth_direction = DEFAULT_DEPTH_DIRECTION
elif shape == "DEPTH_SURFACE":
depth_reference = metric_depth_reference(
points[..., 2],
calibration_valid,
depth_uniform_scale,
)
else:
depth_reference, depth_direction = (
depth_balloon_projection_parameters(
base_shape.vertices,
base_shape.uv,
points[..., 2],
calibration_valid,
depth_uniform_scale,
)
)
source_name = source_object.name
mesh = bpy.data.meshes.new(source_name)
mesh.from_pydata(
np.asarray(base_shape.vertices).tolist(),
[],
base_shape.faces,
)
for face in mesh.polygons:
face.use_smooth = True
_assign_uv(mesh, base_shape.uv)
_assign_float_point_attribute(
mesh,
BALLOON_ATTRIBUTE_NAME,
base_shape.balloon,
)
mesh.materials.append(material)
mesh.update()
mesh_object = bpy.data.objects.new(source_name, mesh)
context.collection.objects.link(mesh_object)
mesh_object.matrix_world = _cutout_world_matrix(
source_object.matrix_world,
crop_center,
)
mesh_object["anyimage_mesh_shape"] = shape
node_group = cutout_node_group(shape)
modifier = mesh_object.modifiers.new(name=node_group.name, type="NODES")
modifier.node_group = node_group
modifier.show_group_selector = False
for name, value in (
("Shape", CUTOUT_SHAPE_INDICES[shape]),
("Cleanup", cleanup_value),
("Smooth Iterations", 0),
("Smooth", 1.0),
(
"Thickness",
1.0 if shape in {"BALLOON", "DEPTH_BALLOON"} else 0.0,
),
):
set_modifier_input(
modifier,
modifier_input_identifier(node_group, name),
value,
)
if shape in DEPTH_CUTOUT_SHAPES:
for name, value in (
("Depth Scale", 1.0),
("Uniform Scale", depth_uniform_scale),
("Base Plane Depth", depth_reference),
("Depth Image", depth_image),
):
set_modifier_input(
modifier,
modifier_input_identifier(node_group, name),
value,
)
if shape == "DEPTH_BALLOON":
set_modifier_input(
modifier,
modifier_input_identifier(node_group, "Depth Direction"),
depth_direction,
)
return finalize_object_result(
context,
source_object,
mesh_object,
keep_source=True,
)
except Exception:
if mesh_object is not None:
bpy.data.objects.remove(mesh_object, do_unlink=True)
if mesh is not None and mesh.users == 0:
bpy.data.meshes.remove(mesh)
if material is not None and material.users == 0:
bpy.data.materials.remove(material)
if color_image.users == 0:
bpy.data.images.remove(color_image, do_unlink=True)
if normal_image is not None and normal_image.users == 0:
bpy.data.images.remove(normal_image, do_unlink=True)
if depth_image is not None and depth_image.users == 0:
bpy.data.images.remove(depth_image, do_unlink=True)
raise
def region_to_shape(
context,
source_object,
shape,
detail,
region_mask,
*,
field=None,
normal_image=None,
normal_mode=NormalMode.NONE,
source_rgba=None,
):
crop_width = region_mask.bounds[2] - region_mask.bounds[0]
crop_height = region_mask.bounds[3] - region_mask.bounds[1]
detail = limited_cutout_mesh_detail(
detail,
(crop_width, crop_height),
configured_max_ai_input_size(context),
)
base_shape = build_base_shape(
context,
source_object,
detail,
region_mask,
)
result = create_shape_object(
context,
source_object,
shape,
detail,
region_mask,
base_shape,
field=field,
normal_image=normal_image,
normal_mode=normal_mode,
source_rgba=source_rgba,
)
undo_message = f"Convert to {shape.replace('_', ' ').title()}"
if "FINISHED" not in bpy.ops.ed.undo_push(message=undo_message):
raise RuntimeError(f"Unable to create the {undo_message} Undo step")
return result
def _region_mask_from_image(path, bounds):
image = bpy.data.images.load(str(path), check_existing=False)
try:
return RegionMask(
image_rgba(image)[:, :, 3],
tuple(int(value) for value in bounds),
)
finally:
bpy.data.images.remove(image, do_unlink=True)
def _generated_region_mask(result, input_path, source_bounds):
if not isinstance(result.value, dict) or "region" not in result.value:
return _region_mask_from_image(input_path, source_bounds)
local_bounds = tuple(
int(value)
for value in json.loads(
result.file("region_bounds").read_text(encoding="utf-8")
)
)
bounds = (
source_bounds[0] + local_bounds[0],
source_bounds[1] + local_bounds[1],
source_bounds[0] + local_bounds[2],
source_bounds[1] + local_bounds[3],
)
return _region_mask_from_image(result.file("region"), bounds)
def cutout_moge_settings(context):
preferences = addon_preferences(context)
model = getattr(preferences, "mesh_depth_model", DEFAULT_MOGE2_MODEL_KEY)
if not model.startswith("MOGE2_"):
model = DEFAULT_MOGE2_MODEL_KEY
return (
model,
int(getattr(preferences, "mesh_resolution_level", 5)),
)
class CutoutRegionToShape(JobOperatorBase, bpy.types.Operator):
bl_idname = "anyimage.cutout_region_to_shape"
bl_label = "Cutout Region to Shape"
bl_description = "Build the selected image Region as a Cutout shape"
bl_options = {"INTERNAL"}
job_type = "generate-region-data"
starting_message = "Generating Cutout Region data"
input_path: bpy.props.StringProperty(options=_HIDDEN)
source_object_name: bpy.props.StringProperty(options=_HIDDEN)
region_path_json: bpy.props.StringProperty(options=_HIDDEN)
region_bounds_json: bpy.props.StringProperty(options=_HIDDEN)
shape: bpy.props.StringProperty(options=_HIDDEN)
mesh_detail: bpy.props.IntProperty(
default=128,
min=4,
max=1024,
options=_HIDDEN,
)
model: bpy.props.StringProperty(
default=DEFAULT_MOGE2_MODEL_KEY,
options=_HIDDEN,
)
resolution_level: bpy.props.IntProperty(
default=5,
min=0,
max=9,
options=_HIDDEN,
)
max_input_size: bpy.props.IntProperty(
default=DEFAULT_MAX_AI_INPUT_SIZE,
min=MIN_MAX_AI_INPUT_SIZE,
max=MAX_MAX_AI_INPUT_SIZE,
options=_HIDDEN,
)
normal_mode: bpy.props.StringProperty(
default=NormalMode.NONE.value,
options=_HIDDEN,
)
refine_selection: bpy.props.BoolProperty(default=False, options=_HIDDEN)
def _needs_generation(self):
return (
self.refine_selection
or self.shape in DEPTH_CUTOUT_SHAPES
or NormalMode(self.normal_mode) != NormalMode.NONE
)
def execute(self, context):
try:
source_object = require_image_empty(self.source_object_name)
if video_total_frames(source_object.data) != 1:
raise ValueError("Cutout currently supports still images")
if self.shape not in CUTOUT_SHAPES:
raise ValueError(f"Unsupported Cutout shape: {self.shape}")
self.normal_mode = NormalMode(self.normal_mode).value
region_path = RegionPath.from_json(self.region_path_json)
source_rgba = image_rgba(source_object.data)
region_mask = rasterize_region_path(
tuple(source_object.data.size),
region_path,
source_alpha=source_rgba[:, :, 3],
alpha_threshold=CUTOUT_ALPHA_THRESHOLD,
antialias=True,
)
context.view_layer.objects.active = source_object
source_object.select_set(True)
self.max_input_size = configured_max_ai_input_size(context)
if not self._needs_generation():
region_to_shape(
context,
source_object,
self.shape,
self.mesh_detail,
region_mask,
source_rgba=source_rgba,
)
return {"FINISHED"}
setup_result = invoke_ai_setup_if_needed()
if setup_result is not None:
return setup_result
self.model, self.resolution_level = cutout_moge_settings(context)
self.region_bounds_json = json.dumps(
region_mask.bounds,
separators=(",", ":"),
)
self.input_path = str(
prepare_region_input(source_object.data, region_mask)
)
except (
ImageEditWarning,
OSError,
RuntimeError,
TypeError,
ValueError,
) as error:
report_image_edit_exception(self, error)
return {"CANCELLED"}
result = super().execute(context)
if "CANCELLED" in result:
cleanup_image_input(self.input_path, True)
return result
def request(self, context):
require_environment()
source_path = require_input_path(self.input_path)
output_depth = self.shape in DEPTH_CUTOUT_SHAPES
normal_mode = NormalMode(self.normal_mode)
if self.refine_selection:
require_model(BEN2_MODEL_KEY)
if output_depth or normal_mode != NormalMode.NONE:
require_model(self.model)
parameters = {
"input": str(source_path),
"device": production_device().lower(),
"refine_region": self.refine_selection,
"output_depth": output_depth,
"normal_mode": normal_mode.value,
"max_input_size": int(self.max_input_size),
}
if self.refine_selection:
parameters["refine_model"] = BEN2_MODEL_KEY
if output_depth or normal_mode != NormalMode.NONE:
start_frame = source_start_frame(self.source_object_name)
parameters.update(
model=self.model,
resolution_level=self.resolution_level,
)
parameters.update(
moge2_parameters(
context.scene.anyimage_settings,
source_path,
self.model,
self.resolution_level,
start_frame,
)
)
return parameters
def response(self, context, result):
source_object = require_image_empty(self.source_object_name)
source_bounds = tuple(json.loads(self.region_bounds_json))
region_mask = _generated_region_mask(
result,
self.input_path,
source_bounds,
)
value = result.value if isinstance(result.value, dict) else {}
normal_image = (
load_normal_result_image(result.file("normal"), source_object)
if "normal" in value
else None
)
field = (
load_image_geometry(result.file("image_geometry"))
if self.shape in DEPTH_CUTOUT_SHAPES
else None
)
region_to_shape(
context,
source_object,
self.shape,
self.mesh_detail,
region_mask,
field=field,
normal_image=normal_image,
normal_mode=NormalMode(self.normal_mode),
)
return f"{self.shape.replace('_', ' ').title()} is ready"
def cleanup(self):
cleanup_image_input(self.input_path, True)

View File

@@ -1,240 +0,0 @@
import bpy
from ...properties import (
refine_selection_property,
)
from ..common.image_data import image_rgba, require_image_empty, video_total_frames
from ..common.image_interaction import (
apply_region_settings,
draw_region_settings,
report_image_edit_exception,
resolve_image_edit_click,
)
from ..common.image_interaction import serialize_matrix
from ..common.image_interaction import ImageRegionGesture, gesture_property
from ..common.image_region import (
ImageEditWarning,
RegionPath,
rasterize_region_path,
)
CUTOUT_DETAIL_MIN = 4
CUTOUT_DETAIL_MAX = 1024
def _validate_cutout_region(
source_object,
region_path,
):
image = source_object.data
rgba = image_rgba(image)
rasterize_region_path(
tuple(image.size),
region_path,
source_alpha=rgba[:, :, 3],
alpha_threshold=0.1,
antialias=True,
)
def cutout_mesh_detail(image_size, point_spacing):
longest_edge = max(int(value) for value in image_size)
detail = round(longest_edge / max(int(point_spacing), 1))
return max(CUTOUT_DETAIL_MIN, min(CUTOUT_DETAIL_MAX, detail))
def _pie_button(
pie,
shape,
label,
icon,
source_name,
region_path_json,
refine_selection,
normal_mode,
mesh_detail,
):
operator = pie.operator(
"anyimage.cutout_region_to_shape",
text=label,
icon=icon,
)
operator.source_object_name = source_name
operator.region_path_json = region_path_json
operator.shape = shape
operator.refine_selection = refine_selection
operator.normal_mode = normal_mode
operator.mesh_detail = mesh_detail
def _open_shape_pie(
context,
event,
source_object,
region_path,
refine_selection=False,
):
source_name = source_object.name
region_path_json = region_path.to_json()
settings = context.scene.anyimage_settings
point_spacing = settings.cutout_point_spacing
normal_mode = str(getattr(settings, "cutout_normal_mode", "NONE"))
mesh_detail = cutout_mesh_detail(source_object.data.size, point_spacing)
def draw(menu, _context):
pie = menu.layout.menu_pie()
_pie_button(
pie,
"SURFACE",
"Surface",
"MESH_GRID",
source_name,
region_path_json,
refine_selection,
normal_mode,
mesh_detail,
)
_pie_button(
pie,
"BALLOON",
"Balloon",
"MESH_UVSPHERE",
source_name,
region_path_json,
refine_selection,
normal_mode,
mesh_detail,
)
_pie_button(
pie,
"DEPTH_BALLOON",
"Depth Balloon",
"META_BALL",
source_name,
region_path_json,
refine_selection,
normal_mode,
mesh_detail,
)
_pie_button(
pie,
"DEPTH_SURFACE",
"Depth Surface",
"SURFACE_NCURVE",
source_name,
region_path_json,
refine_selection,
normal_mode,
mesh_detail,
)
context.window_manager.popup_menu_pie(
event,
draw,
title="Cutout Shape",
icon="MESH_DATA",
)
class SelectCutoutRegion(ImageRegionGesture, bpy.types.Operator):
bl_idname = "anyimage.select_cutout_region"
bl_label = "Select Cutout Region"
bl_description = "Lasso an Image Empty region, then choose its mesh shape"
bl_options = {"INTERNAL"}
active_tool_id = "anyimage.cutout_lasso"
source_object_name: bpy.props.StringProperty(options={"HIDDEN", "SKIP_SAVE"})
source_matrix_data: bpy.props.StringProperty(options={"HIDDEN", "SKIP_SAVE"})
region_path_json: bpy.props.StringProperty(options={"HIDDEN", "SKIP_SAVE"})
refine_selection: refine_selection_property()
invert: bpy.props.BoolProperty(default=False, options={"SKIP_SAVE"})
gesture: gesture_property()
full_image: bpy.props.BoolProperty(default=False, options={"HIDDEN", "SKIP_SAVE"})
preview_fill_rule = 'NONZERO'
_pie_event = None
def _set_status(self, context):
invert = "On" if self.invert else "Off"
context.workspace.status_text_set(
f"Cutout Lasso: drag and release for shapes; "
f"F toggles Invert ({invert}); RMB or Esc cancels"
)
def invoke(self, context, event):
self._pie_event = event
if not self.full_image and event.value != "DOUBLE_CLICK":
return super().invoke(context, event)
source_object, should_edit = resolve_image_edit_click(
context, event, self.report
)
if not should_edit:
return {"FINISHED"} if source_object is not None else {"CANCELLED"}
if source_object is None:
return {"CANCELLED"}
if video_total_frames(source_object.data) > 1:
self.report({"WARNING"}, "Image tools only support a still image")
return {"CANCELLED"}
apply_region_settings(self, context)
self.invert = False
self.source_object_name = source_object.name
self.source_matrix_data = serialize_matrix(source_object.matrix_world)
width, height = (int(value) for value in source_object.data.size)
self.region_path_json = RegionPath(
points=((0, 0), (width, 0), (width, height), (0, height)),
fill_rule="NONZERO",
).to_json()
return self.execute(context)
def modal(self, context, event):
self._pie_event = event
return super().modal(context, event)
def execute(self, context):
try:
source_object = require_image_empty(self.source_object_name)
except RuntimeError as error:
self.report({"ERROR"}, str(error))
return {"CANCELLED"}
try:
region_path = RegionPath.from_json(self.region_path_json)
_validate_cutout_region(source_object, region_path)
except (ImageEditWarning, TypeError, ValueError) as error:
report_image_edit_exception(self, error)
return {"CANCELLED"}
_open_shape_pie(
context,
self._pie_event,
source_object,
region_path,
self.refine_selection,
)
return {"FINISHED"}
class CutoutTool(bpy.types.WorkSpaceTool):
bl_space_type = "VIEW_3D"
bl_context_mode = "OBJECT"
bl_idname = "anyimage.cutout_lasso"
bl_label = "Cutout"
bl_description = "Lasso an Image Empty and choose a mesh shape"
bl_icon = "ops.sculpt.lasso_mask"
bl_operator = SelectCutoutRegion.bl_idname
bl_keymap = (
(
SelectCutoutRegion.bl_idname,
{"type": "LEFTMOUSE", "value": "DOUBLE_CLICK"},
{"properties": [("gesture", "LASSO"), ("full_image", True)]},
),
(
SelectCutoutRegion.bl_idname,
{"type": "LEFTMOUSE", "value": "PRESS"},
{"properties": [("gesture", "LASSO")]},
),
)
def draw_settings(context, layout, _tool):
settings = context.scene.anyimage_settings
layout.prop(settings, "cutout_point_spacing")
layout.prop(settings, "cutout_normal_mode")
draw_region_settings(context, layout, "cutout_refine_selection")

View File

@@ -0,0 +1,21 @@
"""Cutout Shape identifiers shared by runtime and node-asset tools."""
BALLOON_ATTRIBUTE_NAME = "o_balloon"
CUTOUT_NODE_GROUP_NAME = "O Image Cutout"
CUTOUT_NODE_GROUP_NAMES = {
"SURFACE": "O Image Surface",
"BALLOON": "O Image Balloon",
"DEPTH_BALLOON": "O Image Depth Balloon",
"DEPTH_SURFACE": "O Image Depth Surface",
}
CUTOUT_SHAPE_LABELS = {
"SURFACE": "Surface",
"BALLOON": "Balloon",
"DEPTH_BALLOON": "Depth Balloon",
"DEPTH_SURFACE": "Depth Surface",
}
CUTOUT_SHAPE_INDICES = {
shape: index for index, shape in enumerate(CUTOUT_NODE_GROUP_NAMES)
}
CUTOUT_SHAPES = frozenset(CUTOUT_NODE_GROUP_NAMES)
DEPTH_CUTOUT_SHAPES = frozenset({"DEPTH_BALLOON", "DEPTH_SURFACE"})

View File

@@ -1,13 +1,12 @@
from .box import ImageBoxTool from .box import ImageBoxTool
from .edit_mask import ActivateImageTool, EditImageMask, RefineImageRegion
from .lasso import ImageLassoTool from .lasso import ImageLassoTool
from .line import ImageLineTool from .line import ImageLineTool
from .edit_mask import ActivateImageTool, EditImageMask, RefineImageMask
from .polyline import ImagePolylineTool from .polyline import ImagePolylineTool
CLASSES = ( CLASSES = (
EditImageMask, EditImageMask,
RefineImageMask, RefineImageRegion,
ActivateImageTool, ActivateImageTool,
) )

View File

@@ -21,4 +21,4 @@ class ImageBoxTool(bpy.types.WorkSpaceTool):
) )
def draw_settings(context, layout, _tool): def draw_settings(context, layout, _tool):
draw_region_settings(context, layout, "image_box_refine_selection") draw_region_settings(context, layout, "image_box_refine_region")

View File

@@ -2,35 +2,35 @@ import json
import bpy import bpy
from ...properties import BEN2_MODEL_KEY, refine_selection_property from ...properties import BEN2_MODEL_KEY, refine_region_property
from ...runtime import JobOperatorBase from ...runtime import JobOperatorBase
from ..common.ai_job import (
production_device,
require_environment,
require_model,
)
from ..common.image_data import ( from ..common.image_data import (
cleanup_image_input, cleanup_image_input,
create_pixel_result_image, create_pixel_result_image,
image_rgba,
image_pixels, image_pixels,
image_rgba,
prepare_region_input, prepare_region_input,
replace_empty_image, replace_empty_image,
require_image_empty, require_image_empty,
video_total_frames, video_total_frames,
) )
from ..common.image_interaction import ( from ..common.image_interaction import (
ImageRegionGesture,
gesture_property,
image_edit_poll, image_edit_poll,
report_image_edit_exception, report_image_edit_exception,
) )
from ..common.image_interaction import ImageRegionGesture, gesture_property
from ..common.image_region import ( from ..common.image_region import (
ImageEditWarning, ImageEditWarning,
RegionPath, RegionPath,
crop_pixels_to_alpha, crop_pixels_to_alpha,
rasterize_region_path, rasterize_region_path,
) )
from ..common.ai_job import (
production_device,
require_environment,
require_model,
)
_HIDDEN = {"HIDDEN", "SKIP_SAVE"} _HIDDEN = {"HIDDEN", "SKIP_SAVE"}
@@ -41,9 +41,7 @@ def _refined_mask_from_pixels(mask_pixels, mask_size, bounds, source_size):
mask_width, mask_height = (int(value) for value in mask_size) mask_width, mask_height = (int(value) for value in mask_size)
source_width, source_height = (int(value) for value in source_size) source_width, source_height = (int(value) for value in source_size)
local = np.flipud( local = np.flipud(
np.asarray(mask_pixels, dtype=np.float32).reshape( np.asarray(mask_pixels, dtype=np.float32).reshape((mask_height, mask_width, 4))
(mask_height, mask_width, 4)
)
)[:, :, 3] )[:, :, 3]
result = np.zeros((source_height, source_width), dtype=np.float32) result = np.zeros((source_height, source_width), dtype=np.float32)
left, top, right, bottom = (int(value) for value in bounds) left, top, right, bottom = (int(value) for value in bounds)
@@ -107,6 +105,10 @@ def _edit_image_object(source_object, region_mask, invert=False):
class EditImageMask(ImageRegionGesture, bpy.types.Operator): class EditImageMask(ImageRegionGesture, bpy.types.Operator):
refine_region_settings = {
"BOX": "image_box_refine_region",
"LASSO": "image_lasso_refine_region",
}
bl_idname = "anyimage.edit_image_mask" bl_idname = "anyimage.edit_image_mask"
bl_label = "Edit Image Mask" bl_label = "Edit Image Mask"
bl_description = "Edit and tightly crop an Image Empty with a gesture" bl_description = "Edit and tightly crop an Image Empty with a gesture"
@@ -115,7 +117,7 @@ class EditImageMask(ImageRegionGesture, bpy.types.Operator):
source_object_name: bpy.props.StringProperty(options={"HIDDEN", "SKIP_SAVE"}) source_object_name: bpy.props.StringProperty(options={"HIDDEN", "SKIP_SAVE"})
source_matrix_data: bpy.props.StringProperty(options={"HIDDEN", "SKIP_SAVE"}) source_matrix_data: bpy.props.StringProperty(options={"HIDDEN", "SKIP_SAVE"})
region_path_json: bpy.props.StringProperty(options={"HIDDEN", "SKIP_SAVE"}) region_path_json: bpy.props.StringProperty(options={"HIDDEN", "SKIP_SAVE"})
refine_selection: refine_selection_property() refine_region: refine_region_property()
invert: bpy.props.BoolProperty( invert: bpy.props.BoolProperty(
name="Invert", name="Invert",
description="Keep the opposite side of the refined gesture mask", description="Keep the opposite side of the refined gesture mask",
@@ -124,7 +126,6 @@ class EditImageMask(ImageRegionGesture, bpy.types.Operator):
) )
gesture: gesture_property() gesture: gesture_property()
def execute(self, context): def execute(self, context):
try: try:
source_object = require_image_empty(self.source_object_name) source_object = require_image_empty(self.source_object_name)
@@ -140,12 +141,12 @@ class EditImageMask(ImageRegionGesture, bpy.types.Operator):
region_path, region_path,
source_alpha=image_rgba(source_object.data)[:, :, 3], source_alpha=image_rgba(source_object.data)[:, :, 3],
) )
if self.refine_selection: if self.refine_region:
source_path = prepare_region_input( source_path = prepare_region_input(
source_object.data, source_object.data,
region_mask, region_mask,
) )
result = bpy.ops.anyimage.refine_image_mask( result = bpy.ops.anyimage.refine_image_region(
input_path=str(source_path), input_path=str(source_path),
source_object_name=self.source_object_name, source_object_name=self.source_object_name,
region_bounds_json=json.dumps(region_mask.bounds), region_bounds_json=json.dumps(region_mask.bounds),
@@ -164,10 +165,10 @@ class EditImageMask(ImageRegionGesture, bpy.types.Operator):
return {"FINISHED"} return {"FINISHED"}
class RefineImageMask(JobOperatorBase, bpy.types.Operator): class RefineImageRegion(JobOperatorBase, bpy.types.Operator):
bl_idname = "anyimage.refine_image_mask" bl_idname = "anyimage.refine_image_region"
bl_label = "Refine Image Mask" bl_label = "Refine Image Region"
job_type = "refine-image-mask" job_type = "refine-image-region"
starting_message = "Refining image selection" starting_message = "Refining image selection"
input_path: bpy.props.StringProperty(options=_HIDDEN) input_path: bpy.props.StringProperty(options=_HIDDEN)
@@ -185,11 +186,10 @@ class RefineImageMask(JobOperatorBase, bpy.types.Operator):
def response(self, _context, result): def response(self, _context, result):
source_object = require_image_empty(self.source_object_name) source_object = require_image_empty(self.source_object_name)
import numpy as np
source_bounds = tuple(json.loads(self.region_bounds_json)) source_bounds = tuple(json.loads(self.region_bounds_json))
local_bounds = tuple( local_bounds = tuple(
json.loads(result.file("bounds").read_text(encoding="utf-8")) json.loads(result.file("region_bounds").read_text(encoding="utf-8"))
) )
bounds = ( bounds = (
source_bounds[0] + local_bounds[0], source_bounds[0] + local_bounds[0],
@@ -198,7 +198,7 @@ class RefineImageMask(JobOperatorBase, bpy.types.Operator):
source_bounds[1] + local_bounds[3], source_bounds[1] + local_bounds[3],
) )
refined_image = bpy.data.images.load( refined_image = bpy.data.images.load(
str(result.file("image")), check_existing=False str(result.file("region_image")), check_existing=False
) )
try: try:
refined_pixels = image_pixels(refined_image) refined_pixels = image_pixels(refined_image)
@@ -217,7 +217,7 @@ class RefineImageMask(JobOperatorBase, bpy.types.Operator):
return str(error) return str(error)
if "FINISHED" not in bpy.ops.ed.undo_push(message="Edit Image Mask"): if "FINISHED" not in bpy.ops.ed.undo_push(message="Edit Image Mask"):
raise RuntimeError("Unable to create the Edit Image Mask Undo step") raise RuntimeError("Unable to create the Edit Image Mask Undo step")
return "Image selection refined" return "Image Region refined"
def cleanup(self): def cleanup(self):
cleanup_image_input(self.input_path, True) cleanup_image_input(self.input_path, True)

View File

@@ -21,4 +21,4 @@ class ImageLassoTool(bpy.types.WorkSpaceTool):
) )
def draw_settings(context, layout, _tool): def draw_settings(context, layout, _tool):
draw_region_settings(context, layout, "image_lasso_refine_selection") draw_region_settings(context, layout, "image_lasso_refine_region")

View File

@@ -13,6 +13,9 @@ from .server.model_catalog import (
) )
DEFAULT_MOGE2_MODEL_KEY = "MOGE2_VITS_NORMAL" DEFAULT_MOGE2_MODEL_KEY = "MOGE2_VITS_NORMAL"
DEFAULT_MAX_CUTOUT_MESH_RESOLUTION = 1024
MIN_CUTOUT_MESH_RESOLUTION = 4
MAX_CUTOUT_MESH_RESOLUTION = 1024
ADDON_PACKAGE = __package__ ADDON_PACKAGE = __package__
DEFAULT_STORAGE_ROOT = Path.home() / ".anyimage" DEFAULT_STORAGE_ROOT = Path.home() / ".anyimage"
@@ -46,11 +49,24 @@ def configured_storage_root():
return DEFAULT_STORAGE_ROOT return DEFAULT_STORAGE_ROOT
def configured_max_ai_input_size(context=None): def configured_max_cutout_mesh_resolution(context=None):
preferences = addon_preferences(context) preferences = addon_preferences(context)
value = int( value = int(
getattr(preferences, "max_ai_input_size", DEFAULT_MAX_AI_INPUT_SIZE) getattr(
preferences,
"max_cutout_mesh_resolution",
DEFAULT_MAX_CUTOUT_MESH_RESOLUTION,
)
) )
return min(
max(value, MIN_CUTOUT_MESH_RESOLUTION),
MAX_CUTOUT_MESH_RESOLUTION,
)
def configured_max_ai_input_size(context=None):
preferences = addon_preferences(context)
value = int(getattr(preferences, "max_ai_input_size", DEFAULT_MAX_AI_INPUT_SIZE))
return min(max(value, MIN_MAX_AI_INPUT_SIZE), MAX_MAX_AI_INPUT_SIZE) return min(max(value, MIN_MAX_AI_INPUT_SIZE), MAX_MAX_AI_INPUT_SIZE)
@@ -60,10 +76,10 @@ def device_items(owner, context):
return device_enum_items(owner, context) return device_enum_items(owner, context)
def mesh_depth_model_items(owner, context): def geometry_model_items(owner, context):
from .properties import mesh_depth_model_enum_items from .properties import geometry_model_enum_items
return mesh_depth_model_enum_items(owner, context) return geometry_model_enum_items(owner, context)
class AnyImagePreferences(bpy.types.AddonPreferences): class AnyImagePreferences(bpy.types.AddonPreferences):
@@ -97,7 +113,7 @@ class AnyImagePreferences(bpy.types.AddonPreferences):
name="Maximum AI Input Size", name="Maximum AI Input Size",
description=( description=(
"Maximum source long edge eligible for Upscale and maximum long edge " "Maximum source long edge eligible for Upscale and maximum long edge "
"used by Depth and Normal analysis" "used by Depth, Normal and Region refinement"
), ),
default=DEFAULT_MAX_AI_INPUT_SIZE, default=DEFAULT_MAX_AI_INPUT_SIZE,
min=MIN_MAX_AI_INPUT_SIZE, min=MIN_MAX_AI_INPUT_SIZE,
@@ -105,31 +121,30 @@ class AnyImagePreferences(bpy.types.AddonPreferences):
step=256, step=256,
subtype="PIXEL", subtype="PIXEL",
) )
max_cutout_mesh_resolution: bpy.props.IntProperty(
name="Maximum Cutout Mesh Resolution",
description="Maximum sampling resolution along the longest Cutout edge",
default=DEFAULT_MAX_CUTOUT_MESH_RESOLUTION,
min=MIN_CUTOUT_MESH_RESOLUTION,
max=MAX_CUTOUT_MESH_RESOLUTION,
step=64,
subtype="PIXEL",
)
balloon_profile_algorithm: bpy.props.EnumProperty( balloon_profile_algorithm: bpy.props.EnumProperty(
name="Balloon Profile", name="Balloon Profile",
description="Algorithm used to generate o_balloon for every Cutout shape", description="Algorithm used to generate o_balloon for every Cutout shape",
items=BALLOON_PROFILE_ALGORITHM_ITEMS, items=BALLOON_PROFILE_ALGORITHM_ITEMS,
default="POISSON", default="POISSON",
) )
mesh_depth_model: bpy.props.EnumProperty( geometry_model: bpy.props.EnumProperty(
name="Mesh Depth Model", name="Depth & Normal Model",
description="Depth model used by Depth Plane, Depth Balloon and Depth Surface", description="MoGe-2 model used for geometry and normal generation",
items=mesh_depth_model_items, items=geometry_model_items,
default=0, default=0,
) )
mesh_process_res_factor: bpy.props.FloatProperty( geometry_resolution_level: bpy.props.IntProperty(
name="DA3 Resolution Scale",
description="Source image scale used for Depth Anything 3 mesh conversion",
default=100.0,
min=1.0,
max=100.0,
step=100,
precision=0,
subtype="PERCENTAGE",
)
mesh_resolution_level: bpy.props.IntProperty(
name="MoGe Resolution Level", name="MoGe Resolution Level",
description="MoGe-2 inference resolution used for mesh conversion", description="MoGe-2 inference resolution used for geometry and normal generation",
default=5, default=5,
min=0, min=0,
max=9, max=9,
@@ -205,34 +220,24 @@ class AnyImagePreferences(bpy.types.AddonPreferences):
section = self._draw_panel(layout, "cutout", "Cutout") section = self._draw_panel(layout, "cutout", "Cutout")
if section is not None: if section is not None:
section.prop(self, "max_cutout_mesh_resolution")
section.prop(self, "balloon_profile_algorithm") section.prop(self, "balloon_profile_algorithm")
def _draw_depth_normal(self, layout, models, environment_ready): def _draw_depth_normal(self, layout, models, environment_ready):
layout.prop(self, "mesh_depth_model") layout.prop(self, "geometry_model")
mesh_depth_model = getattr( geometry_model = getattr(self, "geometry_model", DEFAULT_MOGE2_MODEL_KEY)
self, "mesh_depth_model", DEFAULT_MOGE2_MODEL_KEY if geometry_model not in models:
) geometry_model = DEFAULT_MOGE2_MODEL_KEY
if mesh_depth_model not in models:
mesh_depth_model = DEFAULT_MOGE2_MODEL_KEY
self._draw_model( self._draw_model(
layout, layout,
models.get(mesh_depth_model), models.get(geometry_model),
environment_ready, environment_ready,
) )
if mesh_depth_model.startswith("MOGE2_"): layout.prop(self, "geometry_resolution_level")
layout.prop(self, "mesh_resolution_level")
else:
layout.prop(self, "mesh_process_res_factor")
layout.label(
text="DA3 models cannot generate normals",
icon="INFO",
)
def _draw_upscale(self, layout, models, environment_ready): def _draw_upscale(self, layout, models, environment_ready):
layout.prop(self, "upscale_model") layout.prop(self, "upscale_model")
upscale_model = getattr( upscale_model = getattr(self, "upscale_model", DEFAULT_UPSCALE_MODEL_KEY)
self, "upscale_model", DEFAULT_UPSCALE_MODEL_KEY
)
if upscale_model not in models: if upscale_model not in models:
upscale_model = DEFAULT_UPSCALE_MODEL_KEY upscale_model = DEFAULT_UPSCALE_MODEL_KEY
self._draw_model( self._draw_model(

View File

@@ -184,11 +184,7 @@ def moge2_model_enum_items(_owner, _context):
def depth_plane_model_enum_items(_owner, _context): def depth_plane_model_enum_items(_owner, _context):
models = [ models = [model for model in model_catalog() if model.get("family") == "moge2"]
model
for model in model_catalog()
if model.get("family") in {"moge2", "da3"} and model.get("enabled")
]
models.sort(key=lambda model: model.get("family") != "moge2") models.sort(key=lambda model: model.get("family") != "moge2")
items = _model_items(models) items = _model_items(models)
items.append( items.append(
@@ -197,12 +193,8 @@ def depth_plane_model_enum_items(_owner, _context):
return items return items
def mesh_depth_model_enum_items(_owner, _context): def geometry_model_enum_items(_owner, _context):
models = [ models = [model for model in model_catalog() if model.get("family") == "moge2"]
model
for model in model_catalog()
if model.get("family") in {"moge2", "da3"} and model.get("enabled")
]
models.sort(key=lambda model: model.get("family") != "moge2") models.sort(key=lambda model: model.get("family") != "moge2")
return _model_items(models) return _model_items(models)
@@ -224,38 +216,28 @@ def upscale_model_label(model_key):
return model.label if model is not None else model_key return model.label if model is not None else model_key
def refine_selection_property(): def refine_region_property():
return bpy.props.BoolProperty( return bpy.props.BoolProperty(
name="Refine Selection", name="Fit to Foreground",
description="Use BEN2 to fit the gesture mask to foreground content", description="Use BEN2 to fit the gesture mask to foreground content",
default=False, default=False,
) )
def cutout_normal_mode_property(): def cutout_generate_normal_property():
return bpy.props.EnumProperty( return bpy.props.BoolProperty(
name="Normal", name="Generate Normal Map",
description="Normal texture generated with MoGe-2", description=(
items=( "Generate a MoGe-2 normal texture; Depth Surface uses Object Space "
("NONE", "None", "Do not generate a normal texture"), "and other shapes use Tangent Space"
(
"TANGENT",
"Tangent",
"Generate a normal texture relative to the image plane",
),
(
"OBJECT",
"Object",
"Generate a normal texture in Cutout object-local axes",
),
), ),
default="NONE", default=False,
) )
def cutout_point_spacing_property(): def cutout_point_spacing_property():
return bpy.props.IntProperty( return bpy.props.IntProperty(
name="Point Spacing", name="Mesh Sampling Spacing",
description=( description=(
"Source image pixels between neighboring mesh points; smaller values " "Source image pixels between neighboring mesh points; smaller values "
"create denser geometry" "create denser geometry"
@@ -268,10 +250,10 @@ def cutout_point_spacing_property():
class AnyImageSettings(bpy.types.PropertyGroup): class AnyImageSettings(bpy.types.PropertyGroup):
cutout_point_spacing: cutout_point_spacing_property() cutout_point_spacing: cutout_point_spacing_property()
cutout_normal_mode: cutout_normal_mode_property() cutout_generate_normal: cutout_generate_normal_property()
cutout_refine_selection: refine_selection_property() cutout_refine_region: refine_region_property()
image_box_refine_selection: refine_selection_property() image_box_refine_region: refine_region_property()
image_lasso_refine_selection: refine_selection_property() image_lasso_refine_region: refine_region_property()
input_path: bpy.props.StringProperty( input_path: bpy.props.StringProperty(
name="Image or Video", name="Image or Video",
description="A single image or video file used for reconstruction", description="A single image or video file used for reconstruction",

View File

@@ -2,20 +2,25 @@
from blendjob import JobServer from blendjob import JobServer
from .jobs.remove_background import run as _remove_background
from .jobs.debug_predictions import ( from .jobs.debug_predictions import (
run_ben2 as _debug_ben2, run_ben2 as _debug_ben2,
)
from .jobs.debug_predictions import (
run_da3 as _debug_da3, run_da3 as _debug_da3,
)
from .jobs.debug_predictions import (
run_moge2 as _debug_moge2, run_moge2 as _debug_moge2,
)
from .jobs.debug_predictions import (
run_upscale as _debug_upscale, run_upscale as _debug_upscale,
) )
from .jobs.refine_image_mask import run as _refine_image_mask from .jobs.cutout import run as _generate_cutout_artifacts
from .jobs.generate_depth_plane import run as _generate_depth_plane from .jobs.depth_plane import run as _generate_depth_plane_geometry
from .jobs.generate_region_data import run as _generate_region_data from .jobs.refine_image_region import run as _refine_image_region
from .jobs.remove_background import run as _remove_background
from .jobs.upscale import run as _upscale_image from .jobs.upscale import run as _upscale_image
from .model_manager import ModelManager from .model_manager import ModelManager
server = JobServer("AnyImage Job Server") server = JobServer("AnyImage Job Server")
@@ -36,9 +41,9 @@ def download_required_models(context, parameters):
return manager.download_missing(context, parameters["models"]) return manager.download_missing(context, parameters["models"])
@server.job("refine-image-mask") @server.job("refine-image-region")
def refine_image_mask(context, parameters): def refine_image_region(context, parameters):
return _refine_image_mask(context, parameters) return _refine_image_region(context, parameters)
@server.job("remove-background") @server.job("remove-background")
@@ -51,14 +56,14 @@ def upscale_image(context, parameters):
return _upscale_image(context, parameters) return _upscale_image(context, parameters)
@server.job("generate-depth-plane") @server.job("generate-depth-plane-geometry")
def generate_depth_plane(context, parameters): def generate_depth_plane_geometry(context, parameters):
return _generate_depth_plane(context, parameters) return _generate_depth_plane_geometry(context, parameters)
@server.job("generate-region-data") @server.job("generate-cutout-artifacts")
def generate_region_data(context, parameters): def generate_cutout_artifacts(context, parameters):
return _generate_region_data(context, parameters) return _generate_cutout_artifacts(context, parameters)
@server.job("debug-da3") @server.job("debug-da3")
@@ -79,4 +84,3 @@ def debug_ben2(context, parameters):
@server.job("debug-upscale") @server.job("debug-upscale")
def debug_upscale(context, parameters): def debug_upscale(context, parameters):
return _debug_upscale(context, parameters) return _debug_upscale(context, parameters)

View File

@@ -0,0 +1,128 @@
"""Write model-space depth textures and their compact metadata."""
import json
import struct
from pathlib import Path
def _exr_attribute(name, kind, value):
return (
name.encode("ascii")
+ b"\0"
+ kind.encode("ascii")
+ b"\0"
+ struct.pack("<I", len(value))
+ value
)
def _exr_channels():
return b"".join(
name.encode("ascii")
+ b"\0"
+ struct.pack("<iB3xii", 2, 0, 1, 1)
for name in ("A", "B", "G", "R")
) + b"\0"
def _write_float_exr(rgba, output_path):
"""Write one uncompressed scanline RGBA OpenEXR without extra dependencies."""
import numpy as np
rgba = np.asarray(rgba, dtype=np.float32)
if rgba.ndim != 3 or rgba.shape[2] != 4:
raise ValueError("Depth texture must contain RGBA float pixels")
height, width = rgba.shape[:2]
if width <= 0 or height <= 0:
raise ValueError("Depth texture dimensions must be positive")
box = struct.pack("<iiii", 0, 0, width - 1, height - 1)
header = b"".join(
(
struct.pack("<II", 20000630, 2),
_exr_attribute("channels", "chlist", _exr_channels()),
_exr_attribute("compression", "compression", b"\0"),
_exr_attribute("dataWindow", "box2i", box),
_exr_attribute("displayWindow", "box2i", box),
_exr_attribute("lineOrder", "lineOrder", b"\0"),
_exr_attribute("pixelAspectRatio", "float", struct.pack("<f", 1.0)),
_exr_attribute(
"screenWindowCenter",
"v2f",
struct.pack("<ff", 0.0, 0.0),
),
_exr_attribute("screenWindowWidth", "float", struct.pack("<f", 1.0)),
b"\0",
)
)
row_size = width * 4 * 4
block_size = 8 + row_size
first_block = len(header) + height * 8
output_path = Path(output_path)
with output_path.open("wb") as output:
output.write(header)
for row in range(height):
output.write(struct.pack("<Q", first_block + row * block_size))
for row in range(height):
output.write(struct.pack("<iI", row, row_size))
for channel in (3, 2, 1, 0):
output.write(
np.asarray(rgba[row, :, channel], dtype="<f4").tobytes()
)
return output_path
def write_vector_depth_texture(points, valid, output_path):
"""Write camera-space XYZ and validity as a float RGBA texture."""
import numpy as np
points = np.asarray(points, dtype=np.float32)
valid = np.asarray(valid, dtype=bool)
if points.ndim != 3 or points.shape[2] != 3 or valid.shape != points.shape[:2]:
raise ValueError("Vector depth dimensions do not match")
rgba = np.zeros((*valid.shape, 4), dtype=np.float32)
rgba[valid, :3] = points[valid]
rgba[..., 3] = valid
return _write_float_exr(rgba, output_path)
def write_z_depth_texture(depth, valid, output_path):
"""Write camera Z and validity as a float RGBA texture."""
import numpy as np
depth = np.asarray(depth, dtype=np.float32)
valid = np.asarray(valid, dtype=bool)
if depth.ndim != 2 or valid.shape != depth.shape:
raise ValueError("Z depth dimensions do not match")
rgba = np.zeros((*depth.shape, 4), dtype=np.float32)
rgba[..., :3] = np.where(valid, depth, 0.0)[..., None]
rgba[..., 3] = valid
return _write_float_exr(rgba, output_path)
def write_depth_metadata(frame, reference_valid, output_path):
"""Write camera values needed to scale a model-space depth texture."""
import numpy as np
depth = np.asarray(frame.depth, dtype=np.float32)
valid = np.asarray(frame.valid_mask, dtype=bool)
intrinsics = np.asarray(frame.intrinsics, dtype=np.float32)
reference_valid = np.asarray(reference_valid, dtype=bool) & valid
if depth.ndim != 2 or valid.shape != depth.shape:
raise ValueError("Depth metadata dimensions do not match")
if intrinsics.shape != (3, 3):
raise ValueError("Depth metadata requires 3x3 camera intrinsics")
if not valid.any():
raise ValueError("MoGe-2 returned no valid depth pixels")
if not reference_valid.any():
raise ValueError("The image contains no valid depth pixels")
metadata = {
"image_size": [int(depth.shape[1]), int(depth.shape[0])],
"intrinsics": intrinsics.tolist(),
"reference_depth": float(np.median(depth[reference_valid])),
}
output_path = Path(output_path)
output_path.write_text(
json.dumps(metadata, separators=(",", ":")),
encoding="utf-8",
)
return output_path

View File

@@ -1,181 +0,0 @@
"""Generate model-derived depth and normal data for one raster image."""
from ..media import DEFAULT_MAX_AI_INPUT_SIZE, limit_image
from ..models import geometry_adapter, moge2
from .normal_texture import write_camera_normal_texture
def _limited_model_input(source_path, output_path, max_input_size):
from PIL import Image
with Image.open(source_path) as opened:
limited = limit_image(opened, max_input_size)
if limited is opened:
return source_path
limited.save(output_path, format="PNG", compress_level=1)
return output_path
def _alpha_mask(image_path, shape):
import numpy as np
from PIL import Image
with Image.open(image_path) as opened:
alpha = opened.convert("RGBA").getchannel("A")
alpha = alpha.resize(
(int(shape[1]), int(shape[0])),
Image.Resampling.NEAREST,
)
return np.asarray(alpha, dtype=np.uint8) >= 26
def transparent_fraction(image_path, alpha_threshold=26):
import numpy as np
from PIL import Image
with Image.open(image_path) as opened:
alpha = np.asarray(
opened.convert("RGBA").getchannel("A"),
dtype=np.uint8,
)
return float(np.count_nonzero(alpha < alpha_threshold) / alpha.size)
def _write_image_geometry_archive(
frame,
output_path,
calibration_valid,
region_transparent_fraction,
):
import numpy as np
depth = np.asarray(frame.depth, dtype=np.float32)
points = np.asarray(frame.points, dtype=np.float32)
intrinsics = np.asarray(frame.intrinsics, dtype=np.float32)
valid = np.asarray(frame.valid_mask, dtype=bool)
if depth.ndim != 2 or points.shape != (*depth.shape, 3):
raise ValueError("The depth model returned an invalid point field")
valid &= np.isfinite(depth) & (depth > 0.0)
valid &= np.isfinite(points).all(axis=-1)
if not valid.any():
raise ValueError("The depth model returned no valid geometry")
calibration_valid = np.asarray(calibration_valid, dtype=bool) & valid
if not calibration_valid.any():
raise ValueError("The depth field contains no valid Region pixels")
reference_values = depth[calibration_valid]
if frame.model_mask is not None:
model_mask = np.asarray(frame.model_mask, dtype=bool)
masked = depth[calibration_valid & model_mask]
if len(masked):
reference_values = masked
np.savez_compressed(
output_path,
points=points,
depth=depth,
intrinsics=intrinsics,
valid=valid,
calibration_valid=calibration_valid,
image_size=np.asarray((depth.shape[1], depth.shape[0]), dtype=np.int32),
reference_depth=np.asarray(np.median(reference_values), dtype=np.float32),
region_transparent_fraction=np.asarray(
region_transparent_fraction,
dtype=np.float32,
),
)
def _write_normal(
prediction,
color_path,
output_path,
normal_mode,
):
import numpy as np
from PIL import Image
normal = np.asarray(prediction.normal[0], dtype=np.float32)
valid = _alpha_mask(color_path, normal.shape[:2])
with Image.open(color_path) as source:
output_size = source.size
return write_camera_normal_texture(
normal,
valid,
(0, 0),
output_size,
output_size,
output_path,
normal_mode=normal_mode,
)
def generate_depth(
context,
parameters,
input_path,
*,
output_depth,
normal_mode="NONE",
):
"""Generate requested AI data from an already prepared image Region."""
normal_mode = str(normal_mode).upper()
if normal_mode not in {"NONE", "TANGENT", "OBJECT"}:
raise ValueError(f"Unsupported normal mode: {normal_mode}")
model_path = _limited_model_input(
input_path,
context.directory / "model-input.png",
parameters.get("max_input_size", DEFAULT_MAX_AI_INPUT_SIZE),
)
try:
model_manager = context.resource("model_manager")
adapter = geometry_adapter(parameters["model"])
prediction = None
if output_depth:
context.progress(0.45, "Generating depth")
if adapter is moge2:
prediction = adapter.infer(
model_manager,
parameters,
[model_path],
include_points=True,
cancel_check=context.check_cancelled,
)
else:
prediction = adapter.infer(
model_manager,
parameters,
[model_path],
cancel_check=context.check_cancelled,
)
frame = adapter.geometry_frame(prediction)
calibration_valid = _alpha_mask(input_path, frame.valid_mask.shape)
archive_path = context.directory / "image_geometry.npz"
_write_image_geometry_archive(
frame,
archive_path,
calibration_valid,
transparent_fraction(input_path),
)
else:
archive_path = None
normal_path = None
if normal_mode != "NONE":
context.progress(0.72, "Generating surface normal")
if prediction is None or adapter is not moge2:
prediction = moge2.infer(
model_manager,
parameters,
[model_path],
include_points=False,
cancel_check=context.check_cancelled,
)
normal_path = _write_normal(
prediction,
input_path,
context.directory / "normal.png",
normal_mode,
)
return archive_path, normal_path
finally:
if model_path != input_path:
model_path.unlink(missing_ok=True)

View File

@@ -43,10 +43,10 @@ def write_camera_normal_texture(
empty_normal = (128, 128, 255) empty_normal = (128, 128, 255)
elif normal_mode == "OBJECT": elif normal_mode == "OBJECT":
converted = normal[..., (2, 0, 1)] * np.asarray( converted = normal[..., (2, 0, 1)] * np.asarray(
(-1.0, -1.0, -1.0), (1.0, -1.0, -1.0),
dtype=np.float32, dtype=np.float32,
) )
empty_normal = (255, 128, 128) empty_normal = (0, 128, 128)
else: else:
raise ValueError(f"Unsupported normal mode: {normal_mode}") raise ValueError(f"Unsupported normal mode: {normal_mode}")
encoded = np.clip( encoded = np.clip(

View File

@@ -0,0 +1,122 @@
"""Export requested depth and normal artifacts from one MoGe-2 prediction."""
from ..media import DEFAULT_MAX_AI_INPUT_SIZE, limit_image
from ..models import moge2
from .depth_texture import (
write_depth_metadata,
write_vector_depth_texture,
write_z_depth_texture,
)
from .normal_texture import write_camera_normal_texture
def _limited_model_input(source_path, output_path, max_input_size):
from PIL import Image
with Image.open(source_path) as opened:
limited = limit_image(opened, max_input_size)
if limited is opened:
return source_path
limited.save(output_path, format="PNG", compress_level=1)
return output_path
def _alpha_mask(image_path, shape):
import numpy as np
from PIL import Image
with Image.open(image_path) as opened:
alpha = opened.convert("RGBA").getchannel("A")
alpha = alpha.resize(
(int(shape[1]), int(shape[0])),
Image.Resampling.NEAREST,
)
return np.asarray(alpha, dtype=np.uint8) >= 26
def _write_normal(prediction, color_path, output_path, normal_mode):
import numpy as np
from PIL import Image
normal = np.asarray(prediction.normal[0], dtype=np.float32)
valid = _alpha_mask(color_path, normal.shape[:2])
with Image.open(color_path) as source:
output_size = source.size
return write_camera_normal_texture(
normal,
valid,
(0, 0),
output_size,
output_size,
output_path,
normal_mode=normal_mode,
)
def generate_moge2_artifacts(
context,
parameters,
input_path,
*,
depth_mode="NONE",
normal_mode="NONE",
):
"""Run MoGe-2 once and write only the requested file artifacts."""
depth_mode = str(depth_mode).upper()
normal_mode = str(normal_mode).upper()
if depth_mode not in {"NONE", "VECTOR", "Z"}:
raise ValueError(f"Unsupported depth mode: {depth_mode}")
if normal_mode not in {"NONE", "TANGENT", "OBJECT"}:
raise ValueError(f"Unsupported normal mode: {normal_mode}")
model_path = _limited_model_input(
input_path,
context.directory / "model-input.png",
parameters.get("max_input_size", DEFAULT_MAX_AI_INPUT_SIZE),
)
try:
prediction = moge2.infer(
context.resource("model_manager"),
parameters,
[model_path],
include_points=depth_mode == "VECTOR",
cancel_check=context.check_cancelled,
)
result = {}
if depth_mode != "NONE":
context.progress(0.45, "Generating depth texture")
frame = moge2.geometry_frame(prediction)
if depth_mode == "VECTOR":
if frame.points is None:
raise ValueError("Vector depth requires a MoGe-2 point field")
depth_path = write_vector_depth_texture(
frame.points,
frame.valid_mask,
context.directory / "vector-depth.exr",
)
result["vector_depth"] = depth_path
else:
depth_path = write_z_depth_texture(
frame.depth,
frame.valid_mask,
context.directory / "z-depth.exr",
)
result["z_depth"] = depth_path
result["depth_metadata"] = write_depth_metadata(
frame,
_alpha_mask(input_path, frame.valid_mask.shape),
context.directory / "depth.json",
)
if normal_mode != "NONE":
context.progress(0.72, "Generating surface normal")
key = f"{normal_mode.lower()}_normal"
result[key] = _write_normal(
prediction,
input_path,
context.directory / f"{normal_mode.lower()}-normal.png",
normal_mode,
)
return result
finally:
if model_path != input_path:
model_path.unlink(missing_ok=True)

View File

@@ -1,21 +1,21 @@
"""Generate optional AI data for one Blender-rasterized Cutout Region.""" """Generate optional AI files for one Blender-rasterized Cutout Region."""
import json import json
from pathlib import Path from pathlib import Path
from ..geometry.generate_depth import generate_depth from ..geometry.prediction_artifacts import generate_moge2_artifacts
from ..region import generate_selection from ..region import refine_region
def run(context, parameters): def run(context, parameters):
context.progress(0.05, "Preparing Cutout Region") context.progress(0.05, "Preparing Cutout Region")
context.directory.mkdir(parents=True, exist_ok=True) context.directory.mkdir(parents=True, exist_ok=True)
input_path = Path(parameters["input"]) input_path = Path(parameters["input"])
region_path = input_path region_image_path = input_path
result = {} result = {}
if parameters.get("refine_region"): if parameters.get("refine_region"):
context.progress(0.2, "Generating refined Region") context.progress(0.2, "Refining Region")
region_path, bounds = generate_selection( region_image_path, bounds = refine_region(
context, context,
input_path, input_path,
parameters["refine_model"], parameters["refine_model"],
@@ -28,24 +28,21 @@ def run(context, parameters):
encoding="utf-8", encoding="utf-8",
) )
result.update( result.update(
region=region_path.name, region_image=region_image_path.name,
region_bounds=bounds_path.name, region_bounds=bounds_path.name,
) )
output_depth = bool(parameters.get("output_depth")) depth_mode = str(parameters.get("depth_mode", "NONE"))
normal_mode = str(parameters.get("normal_mode", "NONE")) normal_mode = str(parameters.get("normal_mode", "NONE"))
if output_depth or normal_mode != "NONE": if depth_mode != "NONE" or normal_mode != "NONE":
archive_path, normal_path = generate_depth( artifacts = generate_moge2_artifacts(
context, context,
parameters, parameters,
region_path, region_image_path,
output_depth=output_depth, depth_mode=depth_mode,
normal_mode=normal_mode, normal_mode=normal_mode,
) )
if archive_path is not None: result.update({key: path.name for key, path in artifacts.items()})
result["image_geometry"] = archive_path.name
if normal_path is not None:
result["normal"] = normal_path.name
context.check_cancelled() context.check_cancelled()
context.progress(1.0, "Cutout Region data is ready") context.progress(1.0, "Cutout artifacts are ready")
return result return result

View File

@@ -1,6 +1,6 @@
"""Generate depth data and preserve color for one complete image.""" """Generate model-space Z Depth and preserve color for one complete image."""
from ..geometry.generate_depth import generate_depth from ..geometry.prediction_artifacts import generate_moge2_artifacts
from ..media import copy_color_images, input_frames from ..media import copy_color_images, input_frames
@@ -11,12 +11,12 @@ def run(context, parameters):
if len(images) != 1: if len(images) != 1:
raise ValueError("Depth Plane currently supports one image") raise ValueError("Depth Plane currently supports one image")
copy_color_images(images, context.directory) copy_color_images(images, context.directory)
archive_path, _normal_path = generate_depth( artifacts = generate_moge2_artifacts(
context, context,
parameters, parameters,
images[0], images[0],
output_depth=True, depth_mode="Z",
) )
context.check_cancelled() context.check_cancelled()
context.progress(1.0, "Depth Plane data is ready") context.progress(1.0, "Depth Plane data is ready")
return {"image_geometry": archive_path.name} return {key: path.name for key, path in artifacts.items()}

View File

@@ -1,18 +1,18 @@
import json import json
import time import time
from ..region import generate_selection from ..region import refine_region
def run(context, parameters): def run(context, parameters):
"""Refine a gesture mask with BEN2 and return its source-image bounds.""" """Refine a gesture mask with BEN2 and return its source-image bounds."""
started = time.perf_counter() started = time.perf_counter()
context.progress(0.05, "Preparing image selection") context.progress(0.05, "Preparing image Region")
context.check_cancelled() context.check_cancelled()
output_directory = context.directory output_directory = context.directory
output_directory.mkdir(parents=True, exist_ok=True) output_directory.mkdir(parents=True, exist_ok=True)
context.progress(0.2, f"Loading BEN2 for {parameters['device'].upper()}") context.progress(0.2, f"Loading BEN2 for {parameters['device'].upper()}")
output_path, bounds = generate_selection( output_path, bounds = refine_region(
context, context,
parameters["input"], parameters["input"],
parameters["model"], parameters["model"],
@@ -22,6 +22,6 @@ def run(context, parameters):
bounds_path = output_directory / "bounds.json" bounds_path = output_directory / "bounds.json"
bounds_path.write_text(json.dumps(bounds, separators=(",", ":")), encoding="utf-8") bounds_path.write_text(json.dumps(bounds, separators=(",", ":")), encoding="utf-8")
elapsed_ms = (time.perf_counter() - started) * 1000.0 elapsed_ms = (time.perf_counter() - started) * 1000.0
message = f"Image selection refined in {elapsed_ms:.0f} ms" message = f"Image Region refined in {elapsed_ms:.0f} ms"
context.progress(1.0, message) context.progress(1.0, message)
return {"image": output_path.name, "bounds": bounds_path.name} return {"region_image": output_path.name, "region_bounds": bounds_path.name}

View File

@@ -1,14 +1,8 @@
from . import ben2, da3, moge2, upscale from . import ben2, da3, moge2, upscale
def geometry_adapter(model_key):
return moge2 if model_key.startswith("MOGE2_") else da3
__all__ = ( __all__ = (
"ben2", "ben2",
"da3", "da3",
"moge2", "moge2",
"upscale", "upscale",
"geometry_adapter",
) )

View File

@@ -4,7 +4,7 @@ from dataclasses import dataclass
@dataclass @dataclass
class GeometryFrame: class GeometryFrame:
depth: object depth: object
points: object points: object | None
intrinsics: object intrinsics: object
valid_mask: object valid_mask: object
confidence: object | None = None confidence: object | None = None

View File

@@ -92,10 +92,10 @@ def infer_one(model_manager, parameters, image_path):
def geometry_frame(prediction): def geometry_frame(prediction):
import numpy as np import numpy as np
if prediction.depth.shape[0] != 1 or prediction.points is None: if prediction.depth.shape[0] != 1:
raise ValueError("Image mesh generation requires one MoGe-2 point grid") raise ValueError("Image depth generation requires one MoGe-2 frame")
depth = prediction.depth[0] depth = prediction.depth[0]
points = prediction.points[0] points = prediction.points[0] if prediction.points is not None else None
intrinsics = prediction.intrinsics[0].copy() intrinsics = prediction.intrinsics[0].copy()
height, width = depth.shape height, width = depth.shape
if ( if (
@@ -109,8 +109,9 @@ def geometry_frame(prediction):
prediction.mask[0] prediction.mask[0]
& np.isfinite(depth) & np.isfinite(depth)
& (depth > 0.0) & (depth > 0.0)
& np.isfinite(points).all(axis=-1)
) )
if points is not None:
valid &= np.isfinite(points).all(axis=-1)
return GeometryFrame( return GeometryFrame(
depth=depth, depth=depth,
points=points, points=points,

View File

@@ -1,5 +1,5 @@
"""AI generation for rasterized image Regions.""" """AI generation for rasterized image Regions."""
from .generate_selection import generate_selection from .refine_region import refine_region
__all__ = ("generate_selection",) __all__ = ("refine_region",)

View File

@@ -17,14 +17,14 @@ def _crop_to_alpha(image):
return image.crop(bounds), bounds return image.crop(bounds), bounds
def generate_selection( def refine_region(
context, context,
input_path, input_path,
model, model,
device, device,
output_path, output_path,
): ):
"""Generate a refined selection from an already rasterized Region.""" """Refine an image Region from an already rasterized Region."""
model_manager = context.resource("model_manager") model_manager = context.resource("model_manager")
foreground, _load_ms, _inference_ms = infer_foreground( foreground, _load_ms, _inference_ms = infer_foreground(
input_path, input_path,

View File

@@ -84,21 +84,27 @@ class AddonTest(unittest.TestCase):
cls.addon = importlib.import_module("anyimage") cls.addon = importlib.import_module("anyimage")
cls.job_runtime = importlib.import_module("anyimage.runtime") cls.job_runtime = importlib.import_module("anyimage.runtime")
cls.model_operators = importlib.import_module("anyimage.operators.ai_setup") cls.model_operators = importlib.import_module("anyimage.operators.ai_setup")
cls.image_depth = importlib.import_module("anyimage.operators.common.mesh_result") cls.image_depth = importlib.import_module(
"anyimage.operators.common.mesh_result"
)
cls.image_data = importlib.import_module("anyimage.operators.common.image_data") cls.image_data = importlib.import_module("anyimage.operators.common.image_data")
cls.object_data = importlib.import_module("anyimage.operators.common.mesh_result") cls.object_data = importlib.import_module(
"anyimage.operators.common.mesh_result"
)
cls.debug = importlib.import_module("anyimage.operators.debug") cls.debug = importlib.import_module("anyimage.operators.debug")
cls.mesh_plane = importlib.import_module("anyimage.operators.convert_to_plane.mesh_plane") cls.mesh_plane = importlib.import_module(
cls.mesh = importlib.import_module("anyimage.operators.convert_to_plane.conversion") "anyimage.operators.convert_to_plane.mesh_plane"
)
cls.mesh = importlib.import_module(
"anyimage.operators.convert_to_plane.conversion"
)
cls.cutout_conversion = importlib.import_module( cls.cutout_conversion = importlib.import_module(
"anyimage.operators.cutout_tool.region_to_shape" "anyimage.operators.cutout_tool.main"
) )
cls.cutout_build = importlib.import_module( cls.cutout_build = importlib.import_module(
"anyimage.operators.cutout_tool.region_to_shape" "anyimage.operators.cutout_tool.main"
)
cls.mesh_shapes = importlib.import_module(
"anyimage.operators.cutout_tool.region_to_shape"
) )
cls.mesh_shapes = importlib.import_module("anyimage.operators.cutout_tool.main")
cls.image_background = importlib.import_module( cls.image_background = importlib.import_module(
"anyimage.operators.remove_background" "anyimage.operators.remove_background"
) )
@@ -116,9 +122,7 @@ class AddonTest(unittest.TestCase):
"anyimage.operators.common.image_interaction" "anyimage.operators.common.image_interaction"
) )
cls.image_box = importlib.import_module("anyimage.operators.image_tool.box") cls.image_box = importlib.import_module("anyimage.operators.image_tool.box")
cls.image_lasso = importlib.import_module( cls.image_lasso = importlib.import_module("anyimage.operators.image_tool.lasso")
"anyimage.operators.image_tool.lasso"
)
cls.image_line = importlib.import_module("anyimage.operators.image_tool.line") cls.image_line = importlib.import_module("anyimage.operators.image_tool.line")
cls.image_polyline = importlib.import_module( cls.image_polyline = importlib.import_module(
"anyimage.operators.image_tool.polyline" "anyimage.operators.image_tool.polyline"
@@ -127,10 +131,10 @@ class AddonTest(unittest.TestCase):
"anyimage.operators.perspective_tool.edit_perspective" "anyimage.operators.perspective_tool.edit_perspective"
) )
cls.cutout_geometry = importlib.import_module( cls.cutout_geometry = importlib.import_module(
"anyimage.operators.cutout_tool.geometry.triangulation" "anyimage.operators.cutout_tool.geometry"
) )
cls.cutout_depth = importlib.import_module( cls.cutout_depth = importlib.import_module(
"anyimage.operators.cutout_tool.geometry.depth_calibration" "anyimage.operators.cutout_tool.geometry"
) )
cls.image_interaction = importlib.import_module( cls.image_interaction = importlib.import_module(
"anyimage.operators.common.image_interaction" "anyimage.operators.common.image_interaction"
@@ -525,7 +529,7 @@ class AddonTest(unittest.TestCase):
self.assertNotIn("stage", controller.post_status) self.assertNotIn("stage", controller.post_status)
self.assertNotIn("stages", controller.post_status) self.assertNotIn("stages", controller.post_status)
def test_conversion_model_lists_moge2_before_da3(self): def test_production_geometry_models_exclude_da3(self):
properties = importlib.import_module("anyimage.properties") properties = importlib.import_module("anyimage.properties")
catalog = ( catalog = (
{ {
@@ -571,14 +575,12 @@ class AddonTest(unittest.TestCase):
) )
with patch.object(properties, "model_catalog", return_value=catalog): with patch.object(properties, "model_catalog", return_value=catalog):
depth_items = properties.depth_plane_model_enum_items(None, None) depth_items = properties.depth_plane_model_enum_items(None, None)
mesh_depth_items = properties.mesh_depth_model_enum_items(None, None) mesh_depth_items = properties.geometry_model_enum_items(None, None)
expected = ( expected = (
"MOGE2_VITS_NORMAL", "MOGE2_VITS_NORMAL",
"MOGE2_VITB_NORMAL", "MOGE2_VITB_NORMAL",
"MOGE2_VITL_NORMAL", "MOGE2_VITL_NORMAL",
"SMALL",
"BASE",
) )
self.assertEqual(tuple(item[0] for item in depth_items[:-1]), expected) self.assertEqual(tuple(item[0] for item in depth_items[:-1]), expected)
self.assertEqual(tuple(item[0] for item in mesh_depth_items), expected) self.assertEqual(tuple(item[0] for item in mesh_depth_items), expected)
@@ -687,11 +689,10 @@ class AddonTest(unittest.TestCase):
self.model_operators.OpenAIEnvironmentSettings, self.model_operators.OpenAIEnvironmentSettings,
self.addon.CLASSES, self.addon.CLASSES,
) )
def test_global_ai_input_size_defaults_and_clamps(self): def test_global_ai_input_size_defaults_and_clamps(self):
preference_module = self.addon.preferences preference_module = self.addon.preferences
preferences = self.fake_bpy.context.preferences.addons[ preferences = self.fake_bpy.context.preferences.addons["anyimage"].preferences
"anyimage"
].preferences
self.assertEqual( self.assertEqual(
preference_module.configured_max_ai_input_size(), preference_module.configured_max_ai_input_size(),
@@ -708,6 +709,25 @@ class AddonTest(unittest.TestCase):
256, 256,
) )
def test_cutout_mesh_resolution_defaults_and_clamps_independently(self):
preference_module = self.addon.preferences
preferences = self.fake_bpy.context.preferences.addons["anyimage"].preferences
self.assertEqual(
preference_module.configured_max_cutout_mesh_resolution(),
1024,
)
preferences.max_cutout_mesh_resolution = 2048
self.assertEqual(
preference_module.configured_max_cutout_mesh_resolution(),
1024,
)
preferences.max_cutout_mesh_resolution = 1
self.assertEqual(
preference_module.configured_max_cutout_mesh_resolution(),
4,
)
def test_preferences_draws_upscale_model_download(self): def test_preferences_draws_upscale_model_download(self):
preference_module = importlib.import_module("anyimage.preferences") preference_module = importlib.import_module("anyimage.preferences")
downloads = [] downloads = []
@@ -742,7 +762,7 @@ class AddonTest(unittest.TestCase):
preferences = preference_module.AnyImagePreferences() preferences = preference_module.AnyImagePreferences()
preferences.layout = Layout() preferences.layout = Layout()
preferences.mesh_depth_model = "BASE" preferences.geometry_model = "MOGE2_VITS_NORMAL"
preferences.upscale_model = "SAFMN_REAL_X4" preferences.upscale_model = "SAFMN_REAL_X4"
catalog = tuple( catalog = tuple(
{ {
@@ -781,7 +801,7 @@ class AddonTest(unittest.TestCase):
self.assertEqual( self.assertEqual(
[download.model for download in downloads], [download.model for download in downloads],
["BASE", "BEN2_BASE", "SAFMN_REAL_X4"], ["MOGE2_VITS_NORMAL", "BEN2_BASE", "SAFMN_REAL_X4"],
) )
self.assertEqual( self.assertEqual(
[ [
@@ -815,10 +835,10 @@ class AddonTest(unittest.TestCase):
("anyimage_preferences_cutout", False), ("anyimage_preferences_cutout", False),
], ],
) )
self.assertIn("DA3 models cannot generate normals", labels)
self.assertNotIn("Normal Model", labels) self.assertNotIn("Normal Model", labels)
self.assertIn("upscale_model", properties) self.assertIn("upscale_model", properties)
self.assertIn("max_ai_input_size", properties) self.assertIn("max_ai_input_size", properties)
self.assertIn("max_cutout_mesh_resolution", properties)
self.assertIn("balloon_profile_algorithm", properties) self.assertIn("balloon_profile_algorithm", properties)
balloon_profile = preference_module.AnyImagePreferences.__annotations__[ balloon_profile = preference_module.AnyImagePreferences.__annotations__[
"balloon_profile_algorithm" "balloon_profile_algorithm"
@@ -887,8 +907,8 @@ class AddonTest(unittest.TestCase):
return_value=(1, 0, 24, True), return_value=(1, 0, 24, True),
), ),
patch.object(self.mesh, "create_image_material", return_value="Material"), patch.object(self.mesh, "create_image_material", return_value="Material"),
patch.object(self.mesh, "copy_color_image", return_value="Color"), patch.object(self.mesh, "create_color_image", return_value="Color"),
patch.object(self.mesh, "build_mesh_plane") as build_mesh, patch.object(self.mesh, "create_plane_object") as build_mesh,
): ):
result = self.mesh.ConvertToPlane.execute(operator, context) result = self.mesh.ConvertToPlane.execute(operator, context)
@@ -905,9 +925,7 @@ class AddonTest(unittest.TestCase):
operator.resolution_level = 5 operator.resolution_level = 5
operator.max_input_size = 1536 operator.max_input_size = 1536
context = SimpleNamespace( context = SimpleNamespace(
scene=SimpleNamespace( scene=SimpleNamespace(anyimage_settings=SimpleNamespace(video_frames=1))
anyimage_settings=SimpleNamespace(video_frames=1)
)
) )
with ( with (
patch.object(self.mesh, "require_environment"), patch.object(self.mesh, "require_environment"),
@@ -924,8 +942,12 @@ class AddonTest(unittest.TestCase):
def test_surface_and_balloon_without_ai_build_locally(self): def test_surface_and_balloon_without_ai_build_locally(self):
import numpy as np import numpy as np
source = SimpleNamespace(data=SimpleNamespace(size=(10, 10)), select_set=lambda _value: None) source = SimpleNamespace(
context = SimpleNamespace(view_layer=SimpleNamespace(objects=SimpleNamespace(active=None))) data=SimpleNamespace(size=(10, 10)), select_set=lambda _value: None
)
context = SimpleNamespace(
view_layer=SimpleNamespace(objects=SimpleNamespace(active=None))
)
region_mask = self.cutout_conversion.RegionMask( region_mask = self.cutout_conversion.RegionMask(
np.ones((8, 8), dtype=np.float32), (1, 1, 9, 9) np.ones((8, 8), dtype=np.float32), (1, 1, 9, 9)
) )
@@ -938,25 +960,46 @@ class AddonTest(unittest.TestCase):
{"points": [[1, 1], [9, 1], [9, 9], [1, 9]]} {"points": [[1, 1], [9, 1], [9, 9], [1, 9]]}
), ),
shape=shape, shape=shape,
mesh_detail=64, mesh_resolution=64,
normal_mode="NONE", generate_normal=False,
refine_selection=False, refine_region=False,
report=lambda *_args: None, report=lambda *_args: None,
_needs_generation=lambda: False, _needs_generation=lambda: False,
) )
with ( with (
patch.object(self.cutout_conversion, "require_image_empty", return_value=source), patch.object(
patch.object(self.cutout_conversion, "video_total_frames", return_value=1), self.cutout_conversion,
patch.object(self.cutout_conversion, "image_rgba", return_value=source_rgba), "require_image_empty",
return_value=source,
),
patch.object(
self.cutout_conversion, "video_total_frames", return_value=1
),
patch.object(
self.cutout_conversion, "image_rgba", return_value=source_rgba
),
patch.object( patch.object(
self.cutout_conversion, self.cutout_conversion,
"rasterize_region_path", "rasterize_region_path",
return_value=region_mask, return_value=region_mask,
), ),
patch.object(self.cutout_conversion, "configured_max_ai_input_size", return_value=2048), patch.object(
patch.object(self.cutout_conversion, "region_to_shape") as convert, self.cutout_conversion,
"configured_max_ai_input_size",
return_value=2048,
),
patch.object(
self.cutout_conversion,
"configured_max_cutout_mesh_resolution",
return_value=1024,
),
patch.object(
self.cutout_conversion, "create_cutout_shape"
) as convert,
): ):
result = self.cutout_conversion.CutoutRegionToShape.execute(operator, context) result = self.cutout_conversion.CutoutRegionToShape.execute(
operator, context
)
self.assertEqual(result, {"FINISHED"}) self.assertEqual(result, {"FINISHED"})
convert.assert_called_once_with( convert.assert_called_once_with(
@@ -975,16 +1018,24 @@ class AddonTest(unittest.TestCase):
operator.shape = "DEPTH_SURFACE" operator.shape = "DEPTH_SURFACE"
operator.model = self.addon.properties.DEFAULT_MOGE2_MODEL_KEY operator.model = self.addon.properties.DEFAULT_MOGE2_MODEL_KEY
operator.resolution_level = 5 operator.resolution_level = 5
operator.max_input_size = 2048 operator.max_ai_input_size = 2048
operator.normal_mode = "OBJECT" operator.generate_normal = True
operator.refine_selection = True operator.refine_region = True
context = SimpleNamespace(scene=SimpleNamespace(anyimage_settings=SimpleNamespace())) context = SimpleNamespace(
scene=SimpleNamespace(anyimage_settings=SimpleNamespace())
)
with ( with (
patch.object(self.cutout_conversion, "require_environment"), patch.object(self.cutout_conversion, "require_environment"),
patch.object(self.cutout_conversion, "require_model") as require_model, patch.object(self.cutout_conversion, "require_model") as require_model,
patch.object(self.cutout_conversion, "production_device", return_value="CPU"), patch.object(
self.cutout_conversion, "production_device", return_value="CPU"
),
patch.object(self.cutout_conversion, "source_start_frame", return_value=1), patch.object(self.cutout_conversion, "source_start_frame", return_value=1),
patch.object(self.cutout_conversion, "moge2_parameters", return_value={"model": operator.model, "resolution_level": 5}), patch.object(
self.cutout_conversion,
"moge2_parameters",
return_value={"model": operator.model, "resolution_level": 5},
),
): ):
parameters = operator.request(context) parameters = operator.request(context)
@@ -993,27 +1044,27 @@ class AddonTest(unittest.TestCase):
[self.addon.properties.BEN2_MODEL_KEY, operator.model], [self.addon.properties.BEN2_MODEL_KEY, operator.model],
) )
self.assertTrue(parameters["refine_region"]) self.assertTrue(parameters["refine_region"])
self.assertTrue(parameters["output_depth"]) self.assertEqual(parameters["depth_mode"], "VECTOR")
self.assertEqual(parameters["normal_mode"], "OBJECT") self.assertEqual(parameters["normal_mode"], "OBJECT")
self.assertNotIn("polygon", parameters) self.assertNotIn("polygon", parameters)
self.assertNotIn("shape", parameters) self.assertNotIn("shape", parameters)
self.assertNotIn("normal_analysis", parameters) self.assertNotIn("normal_analysis", parameters)
def test_cutout_moge_settings_falls_back_from_da3(self): def test_cutout_geometry_settings_use_configured_moge(self):
preferences = SimpleNamespace( preferences = SimpleNamespace(
mesh_depth_model="DA3_BASE", geometry_model="MOGE2_VITB_NORMAL",
mesh_resolution_level=7, geometry_resolution_level=7,
) )
with patch.object( with patch.object(
self.cutout_conversion, self.cutout_conversion,
"addon_preferences", "addon_preferences",
return_value=preferences, return_value=preferences,
): ):
settings = self.cutout_conversion.cutout_moge_settings(None) settings = self.cutout_conversion.cutout_geometry_settings(None)
self.assertEqual( self.assertEqual(
settings, settings,
(self.addon.properties.DEFAULT_MOGE2_MODEL_KEY, 7), ("MOGE2_VITB_NORMAL", 7),
) )
def test_image_menu_keeps_ai_features_clickable_and_offers_models(self): def test_image_menu_keeps_ai_features_clickable_and_offers_models(self):
@@ -1321,9 +1372,7 @@ class AddonTest(unittest.TestCase):
self.assertTrue(nonzero[3, 5]) self.assertTrue(nonzero[3, 5])
def preview_area(fill_rule): def preview_area(fill_rule):
triangles = self.image_edit_common.scanline_fill_triangles( triangles = self.image_edit_common.scanline_fill_triangles(path, fill_rule)
path, fill_rule
)
return sum( return sum(
abs( abs(
self.image_edit_common._triangle_cross( self.image_edit_common._triangle_cross(
@@ -1336,8 +1385,8 @@ class AddonTest(unittest.TestCase):
for index in range(0, len(triangles), 3) for index in range(0, len(triangles), 3)
) )
self.assertAlmostEqual(preview_area('EVEN_ODD'), 36.0) self.assertAlmostEqual(preview_area("EVEN_ODD"), 36.0)
self.assertAlmostEqual(preview_area('NONZERO'), 54.0) self.assertAlmostEqual(preview_area("NONZERO"), 54.0)
def test_expected_empty_image_edits_are_reported_as_warnings(self): def test_expected_empty_image_edits_are_reported_as_warnings(self):
reports = [] reports = []
@@ -1370,7 +1419,7 @@ class AddonTest(unittest.TestCase):
def test_image_edit_tools_and_operators_are_registered(self): def test_image_edit_tools_and_operators_are_registered(self):
operator = self.image_mask.EditImageMask operator = self.image_mask.EditImageMask
self.assertIn(operator, self.addon.CLASSES) self.assertIn(operator, self.addon.CLASSES)
self.assertIn(self.image_mask.RefineImageMask, self.addon.CLASSES) self.assertIn(self.image_mask.RefineImageRegion, self.addon.CLASSES)
self.assertIn(self.image_mask.ActivateImageTool, self.addon.CLASSES) self.assertIn(self.image_mask.ActivateImageTool, self.addon.CLASSES)
tool_cases = ( tool_cases = (
(self.image_box.ImageBoxTool, "BOX", "ops.sculpt.border_mask"), (self.image_box.ImageBoxTool, "BOX", "ops.sculpt.border_mask"),
@@ -1391,24 +1440,36 @@ class AddonTest(unittest.TestCase):
self.assertIn("draw_settings", self.image_box.ImageBoxTool.__dict__) self.assertIn("draw_settings", self.image_box.ImageBoxTool.__dict__)
self.assertIn("draw_settings", self.image_lasso.ImageLassoTool.__dict__) self.assertIn("draw_settings", self.image_lasso.ImageLassoTool.__dict__)
context = SimpleNamespace(scene=SimpleNamespace(anyimage_settings=object())) context = SimpleNamespace(scene=SimpleNamespace(anyimage_settings=object()))
layout = SimpleNamespace(prop=lambda *_args: None) drawn_settings = []
layout = SimpleNamespace(
prop=lambda _settings, name: drawn_settings.append(name)
)
with patch.object(self.image_box, "draw_region_settings") as draw: with patch.object(self.image_box, "draw_region_settings") as draw:
self.image_box.ImageBoxTool.draw_settings(context, layout, None) self.image_box.ImageBoxTool.draw_settings(context, layout, None)
draw.assert_called_once_with( draw.assert_called_once_with(context, layout, "image_box_refine_region")
context, layout, "image_box_refine_selection"
)
with patch.object(self.image_lasso, "draw_region_settings") as draw: with patch.object(self.image_lasso, "draw_region_settings") as draw:
self.image_lasso.ImageLassoTool.draw_settings(context, layout, None) self.image_lasso.ImageLassoTool.draw_settings(context, layout, None)
draw.assert_called_once_with( draw.assert_called_once_with(context, layout, "image_lasso_refine_region")
context, layout, "image_lasso_refine_selection"
)
cutout_tool = importlib.import_module( cutout_tool = importlib.import_module(
"anyimage.operators.cutout_tool" "anyimage.operators.cutout_tool"
).CutoutTool ).CutoutTool
cutout_module = sys.modules[cutout_tool.__module__] cutout_module = sys.modules[cutout_tool.__module__]
with patch.object(cutout_module, "draw_region_settings") as draw: drawn_settings.clear()
with patch.object(
cutout_module,
"draw_region_settings",
side_effect=lambda _context, _layout, name: drawn_settings.append(name),
) as draw:
cutout_tool.draw_settings(context, layout, None) cutout_tool.draw_settings(context, layout, None)
draw.assert_called_once_with(context, layout, "cutout_refine_selection") draw.assert_called_once_with(context, layout, "cutout_refine_region")
self.assertEqual(
drawn_settings,
[
"cutout_refine_region",
"cutout_point_spacing",
"cutout_generate_normal",
],
)
self.assertNotIn("draw_settings", self.image_line.ImageLineTool.__dict__) self.assertNotIn("draw_settings", self.image_line.ImageLineTool.__dict__)
self.assertNotIn( self.assertNotIn(
"draw_settings", self.image_polyline.ImagePolylineTool.__dict__ "draw_settings", self.image_polyline.ImagePolylineTool.__dict__
@@ -1419,51 +1480,69 @@ class AddonTest(unittest.TestCase):
self.assertNotIn("padding", operator.__annotations__) self.assertNotIn("padding", operator.__annotations__)
settings = self.addon.properties.AnyImageSettings.__annotations__ settings = self.addon.properties.AnyImageSettings.__annotations__
for name in ( for name in (
"cutout_refine_selection", "cutout_refine_region",
"image_box_refine_selection", "image_box_refine_region",
"image_lasso_refine_selection", "image_lasso_refine_region",
): ):
self.assertFalse(settings[name]["default"]) self.assertFalse(settings[name]["default"])
self.assertNotIn("refine_selection", settings) self.assertNotIn("refine_region", settings)
self.assertNotIn("image_edit_refine_selection", settings) self.assertNotIn("image_edit_refine_region", settings)
self.assertEqual(settings["cutout_point_spacing"]["default"], 32) self.assertEqual(settings["cutout_point_spacing"]["default"], 32)
self.assertEqual(settings["cutout_point_spacing"]["min"], 4) self.assertEqual(settings["cutout_point_spacing"]["min"], 4)
self.assertEqual(settings["cutout_point_spacing"]["max"], 32) self.assertEqual(settings["cutout_point_spacing"]["max"], 32)
self.assertEqual(settings["cutout_normal_mode"]["default"], "NONE") self.assertFalse(settings["cutout_generate_normal"]["default"])
self.assertEqual(settings["cutout_refine_region"]["name"], "Fit to Foreground")
self.assertEqual(
settings["cutout_point_spacing"]["name"], "Mesh Sampling Spacing"
)
self.assertEqual(
settings["cutout_generate_normal"]["name"],
"Generate Normal Map",
)
self.assertNotIn("image_extract_continuous", settings) self.assertNotIn("image_extract_continuous", settings)
self.assertNotIn("image_extract_padding", settings) self.assertNotIn("image_extract_padding", settings)
self.assertNotIn("image_extract_remove_background", settings) self.assertNotIn("image_extract_remove_background", settings)
def test_refine_selection_is_independent_for_each_area_tool(self):
def test_refine_region_is_independent_for_each_area_tool(self):
settings = SimpleNamespace( settings = SimpleNamespace(
cutout_refine_selection=True, cutout_refine_region=True,
image_box_refine_selection=False, image_box_refine_region=False,
image_lasso_refine_selection=True, image_lasso_refine_region=True,
) )
context = SimpleNamespace(scene=SimpleNamespace(anyimage_settings=settings)) context = SimpleNamespace(scene=SimpleNamespace(anyimage_settings=settings))
with patch.object(self.image_edit_common, "ai_ready", return_value=True): with patch.object(self.image_edit_common, "ai_ready", return_value=True):
for gesture in ("LINE", "POLYLINE"): for gesture in ("LINE", "POLYLINE"):
operator = SimpleNamespace( operator = SimpleNamespace(
gesture=gesture, gesture=gesture,
refine_selection=True, refine_region=True,
) )
self.image_edit_common.apply_region_settings(operator, context) self.image_edit_common.apply_region_settings(operator, context)
self.assertFalse(operator.refine_selection) self.assertFalse(operator.refine_region)
box = SimpleNamespace(gesture="BOX", refine_selection=True) box = SimpleNamespace(
gesture="BOX",
refine_region=True,
refine_region_settings={"BOX": "image_box_refine_region"},
)
self.image_edit_common.apply_region_settings(box, context) self.image_edit_common.apply_region_settings(box, context)
self.assertFalse(box.refine_selection) self.assertFalse(box.refine_region)
lasso = SimpleNamespace(gesture="LASSO", refine_selection=False) lasso = SimpleNamespace(
gesture="LASSO",
refine_region=False,
refine_region_settings={"LASSO": "image_lasso_refine_region"},
)
self.image_edit_common.apply_region_settings(lasso, context) self.image_edit_common.apply_region_settings(lasso, context)
self.assertTrue(lasso.refine_selection) self.assertTrue(lasso.refine_region)
cutout = SimpleNamespace( cutout = SimpleNamespace(
gesture="LASSO", gesture="LASSO",
active_tool_id="anyimage.cutout_lasso", active_tool_id="anyimage.cutout_lasso",
refine_selection=False, refine_region=False,
refine_region_settings={"LASSO": "cutout_refine_region"},
) )
self.image_edit_common.apply_region_settings(cutout, context) self.image_edit_common.apply_region_settings(cutout, context)
self.assertTrue(cutout.refine_selection) self.assertTrue(cutout.refine_region)
def test_image_edit_first_click_selects_another_image_without_editing(self): def test_image_edit_first_click_selects_another_image_without_editing(self):
selected = [] selected = []
@@ -1652,14 +1731,14 @@ class AddonTest(unittest.TestCase):
self.assertAlmostEqual(area, 10.0) self.assertAlmostEqual(area, 10.0)
def test_image_mask_fill_uses_blender_geometry_without_stopping_the_draw(self): def test_image_mask_fill_uses_blender_geometry_without_stopping_the_draw(self):
mathutils = ModuleType('mathutils') mathutils = ModuleType("mathutils")
mathutils.Vector = lambda values: values mathutils.Vector = lambda values: values
mathutils.geometry = SimpleNamespace( mathutils.geometry = SimpleNamespace(
tessellate_polygon=lambda _loops: ( tessellate_polygon=lambda _loops: (
((0.0, 0.0, 0.0), (2.0, 0.0, 0.0), (0.0, 2.0, 0.0)), ((0.0, 0.0, 0.0), (2.0, 0.0, 0.0), (0.0, 2.0, 0.0)),
) )
) )
with patch.dict(sys.modules, {'mathutils': mathutils}): with patch.dict(sys.modules, {"mathutils": mathutils}):
triangles = self.image_edit_common.triangulated_polygon_vertices( triangles = self.image_edit_common.triangulated_polygon_vertices(
((0, 0), (2, 0), (0, 2)) ((0, 0), (2, 0), (0, 2))
) )
@@ -1670,12 +1749,12 @@ class AddonTest(unittest.TestCase):
) )
def test_image_mask_fill_falls_back_when_blender_tessellation_rejects_path(self): def test_image_mask_fill_falls_back_when_blender_tessellation_rejects_path(self):
mathutils = ModuleType('mathutils') mathutils = ModuleType("mathutils")
mathutils.Vector = lambda values: values mathutils.Vector = lambda values: values
mathutils.geometry = SimpleNamespace( mathutils.geometry = SimpleNamespace(
tessellate_polygon=lambda _loops: (_ for _ in ()).throw(TypeError()) tessellate_polygon=lambda _loops: (_ for _ in ()).throw(TypeError())
) )
with patch.dict(sys.modules, {'mathutils': mathutils}): with patch.dict(sys.modules, {"mathutils": mathutils}):
triangles = self.image_edit_common.triangulated_polygon_vertices( triangles = self.image_edit_common.triangulated_polygon_vertices(
((0, 0), (2, 0), (0, 2)) ((0, 0), (2, 0), (0, 2))
) )
@@ -1684,9 +1763,7 @@ class AddonTest(unittest.TestCase):
def test_image_mask_preview_fills_a_self_intersecting_lasso(self): def test_image_mask_preview_fills_a_self_intersecting_lasso(self):
polygon = ((0, 0), (4, 4), (0, 4), (4, 0)) polygon = ((0, 0), (4, 4), (0, 4), (4, 0))
triangles = self.image_edit_common.preview_fill_triangles( triangles = self.image_edit_common.preview_fill_triangles(polygon, "NONZERO")
polygon, 'NONZERO'
)
area = sum( area = sum(
abs( abs(
self.image_edit_common._triangle_cross( self.image_edit_common._triangle_cross(
@@ -1788,9 +1865,10 @@ class AddonTest(unittest.TestCase):
self.assertNotIn("image_perspective_aspect_ratio", settings) self.assertNotIn("image_perspective_aspect_ratio", settings)
self.assertNotIn("aspect_mode", operator.__annotations__) self.assertNotIn("aspect_mode", operator.__annotations__)
self.assertIn("aspect_ratio", operator.__annotations__) self.assertIn("aspect_ratio", operator.__annotations__)
self.assertNotIn("refine_selection", operator.__annotations__) self.assertNotIn("refine_region", operator.__annotations__)
self.assertNotIn("continuous", operator.__annotations__) self.assertNotIn("continuous", operator.__annotations__)
self.assertNotIn("padding", operator.__annotations__) self.assertNotIn("padding", operator.__annotations__)
def test_image_box_builds_a_rectangular_screen_path(self): def test_image_box_builds_a_rectangular_screen_path(self):
self.assertEqual( self.assertEqual(
self.image_gesture.box_screen_path((2, 3), (8, 9)), self.image_gesture.box_screen_path((2, 3), (8, 9)),
@@ -1830,9 +1908,7 @@ class AddonTest(unittest.TestCase):
return_value=polygon, return_value=polygon,
), ),
): ):
result = self.image_gesture.ImageRegionGesture._complete( result = self.image_gesture.ImageRegionGesture._complete(operator, context)
operator, context
)
self.assertEqual(result, {"FINISHED"}) self.assertEqual(result, {"FINISHED"})
self.assertEqual( self.assertEqual(
@@ -1854,12 +1930,12 @@ class AddonTest(unittest.TestCase):
def test_image_lasso_preview_uses_fill_and_a_closed_dashed_outline(self): def test_image_lasso_preview_uses_fill_and_a_closed_dashed_outline(self):
polygon = ((1.0, 2.0), (8.0, 3.0), (4.0, 9.0)) polygon = ((1.0, 2.0), (8.0, 3.0), (4.0, 9.0))
operator = SimpleNamespace( operator = SimpleNamespace(
gesture='LASSO', gesture="LASSO",
invert=False, invert=False,
_path=[polygon[0]], _path=[polygon[0]],
_area_pointer=1, _area_pointer=1,
_region_pointer=2, _region_pointer=2,
preview_fill_rule='NONZERO', preview_fill_rule="NONZERO",
_fill_polygon=None, _fill_polygon=None,
_fill_triangles=(), _fill_triangles=(),
_outline_polygon=None, _outline_polygon=None,
@@ -1867,33 +1943,33 @@ class AddonTest(unittest.TestCase):
_screen_path=lambda: polygon, _screen_path=lambda: polygon,
) )
with ( with (
patch.object(self.image_gesture, 'drawing_in_region', return_value=True), patch.object(self.image_gesture, "drawing_in_region", return_value=True),
patch.object( patch.object(
self.image_gesture, self.image_gesture,
'dashed_line_segments', "dashed_line_segments",
return_value=('segments',), return_value=("segments",),
) as build_outline, ) as build_outline,
patch.object( patch.object(
self.image_gesture, self.image_gesture,
'preview_fill_triangles', "preview_fill_triangles",
return_value=('triangles',), return_value=("triangles",),
) as build_fill, ) as build_fill,
patch.object(self.image_gesture, 'draw_polygon_fill') as draw_fill, patch.object(self.image_gesture, "draw_polygon_fill") as draw_fill,
patch.object(self.image_gesture, 'draw_dashed_line') as draw_outline, patch.object(self.image_gesture, "draw_dashed_line") as draw_outline,
): ):
self.image_gesture.ImageRegionGesture._draw_region(operator) self.image_gesture.ImageRegionGesture._draw_region(operator)
self.image_gesture.ImageRegionGesture._draw_region(operator) self.image_gesture.ImageRegionGesture._draw_region(operator)
build_fill.assert_called_once_with(polygon, 'NONZERO') build_fill.assert_called_once_with(polygon, "NONZERO")
build_outline.assert_called_once_with((*polygon, polygon[0])) build_outline.assert_called_once_with((*polygon, polygon[0]))
self.assertEqual(draw_fill.call_count, 2) self.assertEqual(draw_fill.call_count, 2)
draw_fill.assert_called_with( draw_fill.assert_called_with(
polygon, polygon,
('triangles',), ("triangles",),
(1.0, 1.0, 1.0, 0.16), (1.0, 1.0, 1.0, 0.16),
) )
self.assertEqual(draw_outline.call_count, 2) self.assertEqual(draw_outline.call_count, 2)
draw_outline.assert_called_with((), segments=('segments',)) draw_outline.assert_called_with((), segments=("segments",))
def test_image_lasso_does_not_add_stationary_or_nearly_collinear_points(self): def test_image_lasso_does_not_add_stationary_or_nearly_collinear_points(self):
path = [(0.0, 0.0)] path = [(0.0, 0.0)]
@@ -1912,8 +1988,7 @@ class AddonTest(unittest.TestCase):
previous_ops = getattr(self.fake_bpy, "ops", None) previous_ops = getattr(self.fake_bpy, "ops", None)
self.fake_bpy.ops = SimpleNamespace( self.fake_bpy.ops = SimpleNamespace(
wm=SimpleNamespace( wm=SimpleNamespace(
tool_set_by_id=lambda **options: calls.append(options) tool_set_by_id=lambda **options: calls.append(options) or {"FINISHED"}
or {"FINISHED"}
) )
) )
try: try:
@@ -1935,9 +2010,7 @@ class AddonTest(unittest.TestCase):
endpoint = self.image_gesture.snapped_line_endpoint((0, 0), (10, 1), True) endpoint = self.image_gesture.snapped_line_endpoint((0, 0), (10, 1), True)
self.assertAlmostEqual(math.degrees(math.atan2(endpoint[1], endpoint[0])), 5.0) self.assertAlmostEqual(math.degrees(math.atan2(endpoint[1], endpoint[0])), 5.0)
preview = self.image_gesture.line_half_plane_polygon( preview = self.image_gesture.line_half_plane_polygon((0, 5), (10, 5), (10, 10))
(0, 5), (10, 5), (10, 10)
)
image = self.image_gesture.image_line_half_plane_polygon( image = self.image_gesture.image_line_half_plane_polygon(
((0, 5), (10, 5)), (10, 10) ((0, 5), (10, 5)), (10, 10)
) )
@@ -2191,5 +2264,6 @@ class AddonTest(unittest.TestCase):
self.assertEqual(calls[0][1]["text"], "Working") self.assertEqual(calls[0][1]["text"], "Working")
self.assertEqual(calls[1], ("operator", "anyimage.cancel_job")) self.assertEqual(calls[1], ("operator", "anyimage.cancel_job"))
if __name__ == "__main__": if __name__ == "__main__":
unittest.main() unittest.main()

View File

@@ -1,4 +1,3 @@
import json
import sys import sys
from pathlib import Path from pathlib import Path
from types import MethodType, SimpleNamespace from types import MethodType, SimpleNamespace
@@ -9,8 +8,9 @@ SOURCE_ROOT = Path(__file__).parents[1] / "src"
sys.path.insert(0, str(SOURCE_ROOT)) sys.path.insert(0, str(SOURCE_ROOT))
from anyimage.operators.common.image_interaction import ImageRegionGesture from anyimage.operators.common.image_interaction import ImageRegionGesture
from anyimage.operators.cutout_tool import select_region as cutout
from anyimage.operators.common.image_region import RegionPath from anyimage.operators.common.image_region import RegionPath
from anyimage.operators.cutout_tool import interaction
from anyimage.operators.cutout_tool import main as cutout
class Pie: class Pie:
@@ -32,10 +32,10 @@ class WindowManager:
draw(menu, None) draw(menu, None)
def cutout_context(window_manager, point_spacing=32, normal_mode="NONE"): def cutout_context(window_manager, point_spacing=32, generate_normal=False):
settings = SimpleNamespace( settings = SimpleNamespace(
cutout_point_spacing=point_spacing, cutout_point_spacing=point_spacing,
cutout_normal_mode=normal_mode, cutout_generate_normal=generate_normal,
) )
return SimpleNamespace( return SimpleNamespace(
window_manager=window_manager, window_manager=window_manager,
@@ -52,7 +52,7 @@ def test_cutout_pie_uses_four_semantic_shapes_in_cardinal_order():
context = cutout_context(window_manager) context = cutout_context(window_manager)
source = cutout_source() source = cutout_source()
cutout._open_shape_pie( interaction.open_shape_pie(
context, context,
object(), object(),
source, source,
@@ -75,19 +75,17 @@ def test_cutout_pie_uses_four_semantic_shapes_in_cardinal_order():
call[0] == "anyimage.cutout_region_to_shape" call[0] == "anyimage.cutout_region_to_shape"
for call in window_manager.pie.calls for call in window_manager.pie.calls
) )
assert not any(call[2].refine_selection for call in window_manager.pie.calls) assert not any(call[2].refine_region for call in window_manager.pie.calls)
assert all(call[2].normal_mode == "NONE" for call in window_manager.pie.calls) assert not any(call[2].generate_normal for call in window_manager.pie.calls)
assert all(call[2].mesh_detail == 64 for call in window_manager.pie.calls) assert all(call[2].mesh_resolution == 64 for call in window_manager.pie.calls)
def test_cutout_pie_passes_image_lasso_options_to_every_shape(): def test_cutout_pie_passes_image_lasso_options_to_every_shape():
window_manager = WindowManager() window_manager = WindowManager()
context = cutout_context( context = cutout_context(window_manager, point_spacing=16, generate_normal=True)
window_manager, point_spacing=16, normal_mode="OBJECT"
)
source = cutout_source(size=(4096, 2048)) source = cutout_source(size=(4096, 2048))
cutout._open_shape_pie( interaction.open_shape_pie(
context, context,
object(), object(),
source, source,
@@ -95,22 +93,30 @@ def test_cutout_pie_passes_image_lasso_options_to_every_shape():
points=((0, 0), (10, 0), (0, 10)), points=((0, 0), (10, 0), (0, 10)),
invert=True, invert=True,
), ),
refine_selection=True, refine_region=True,
) )
assert all(call[2].refine_selection for call in window_manager.pie.calls) assert all(call[2].refine_region for call in window_manager.pie.calls)
assert all(call[2].normal_mode == "OBJECT" for call in window_manager.pie.calls) assert all(call[2].generate_normal for call in window_manager.pie.calls)
assert all( assert all(
RegionPath.from_json(call[2].region_path_json).invert RegionPath.from_json(call[2].region_path_json).invert
for call in window_manager.pie.calls for call in window_manager.pie.calls
) )
assert all(call[2].mesh_detail == 256 for call in window_manager.pie.calls) assert all(call[2].mesh_resolution == 256 for call in window_manager.pie.calls)
def test_cutout_normal_space_is_derived_from_shape():
assert cutout.normal_mode_for_shape("DEPTH_SURFACE", False) == "NONE"
assert cutout.normal_mode_for_shape("SURFACE", True) == "TANGENT"
assert cutout.normal_mode_for_shape("BALLOON", True) == "TANGENT"
assert cutout.normal_mode_for_shape("DEPTH_BALLOON", True) == "TANGENT"
assert cutout.normal_mode_for_shape("DEPTH_SURFACE", True) == "OBJECT"
def test_cutout_mesh_detail_tracks_the_source_long_edge_and_clamps(): def test_cutout_mesh_detail_tracks_the_source_long_edge_and_clamps():
assert cutout.cutout_mesh_detail((4096, 2048), 32) == 128 assert interaction.cutout_mesh_resolution((4096, 2048), 32) == 128
assert cutout.cutout_mesh_detail((16, 8), 4) == 4 assert interaction.cutout_mesh_resolution((16, 8), 4) == 4
assert cutout.cutout_mesh_detail((65536, 32768), 4) == 1024 assert interaction.cutout_mesh_resolution((65536, 32768), 4) == 1024
def test_cutout_reuses_image_lasso_gesture_and_double_click_selects_all(monkeypatch): def test_cutout_reuses_image_lasso_gesture_and_double_click_selects_all(monkeypatch):
@@ -128,11 +134,11 @@ def test_cutout_reuses_image_lasso_gesture_and_double_click_selects_all(monkeypa
) )
monkeypatch.setattr(cutout, "video_total_frames", lambda _image: 1) monkeypatch.setattr(cutout, "video_total_frames", lambda _image: 1)
monkeypatch.setattr(cutout, "apply_region_settings", lambda *_args: None) monkeypatch.setattr(cutout, "apply_region_settings", lambda *_args: None)
monkeypatch.setattr(cutout, "_validate_cutout_region", lambda *_args: None) monkeypatch.setattr(cutout, "validate_cutout_region", lambda *_args: None)
operator = SimpleNamespace( operator = SimpleNamespace(
full_image=True, full_image=True,
refine_selection=False, refine_region=False,
report=lambda *_args: None, report=lambda *_args: None,
) )
operator.execute = MethodType(cutout.SelectCutoutRegion.execute, operator) operator.execute = MethodType(cutout.SelectCutoutRegion.execute, operator)
@@ -142,11 +148,9 @@ def test_cutout_reuses_image_lasso_gesture_and_double_click_selects_all(monkeypa
assert issubclass(cutout.SelectCutoutRegion, ImageRegionGesture) assert issubclass(cutout.SelectCutoutRegion, ImageRegionGesture)
assert result == {"FINISHED"} assert result == {"FINISHED"}
region_path = RegionPath.from_json( region_path = RegionPath.from_json(window_manager.pie.calls[0][2].region_path_json)
window_manager.pie.calls[0][2].region_path_json
)
assert region_path.points == ((0, 0), (320, 0), (320, 180), (0, 180)) assert region_path.points == ((0, 0), (320, 0), (320, 180), (0, 180))
assert window_manager.pie.calls[0][2].mesh_detail == 10 assert window_manager.pie.calls[0][2].mesh_resolution == 10
assert not hasattr(cutout, "pending_selection") assert not hasattr(cutout, "pending_selection")

View File

@@ -6,9 +6,6 @@ from pathlib import Path
from types import ModuleType, SimpleNamespace from types import ModuleType, SimpleNamespace
from unittest.mock import patch from unittest.mock import patch
import numpy as np
PROJECT_ROOT = Path(__file__).resolve().parents[1] PROJECT_ROOT = Path(__file__).resolve().parents[1]
@@ -44,15 +41,6 @@ def load_image_modules():
/ "common" / "common"
/ "mesh_result.py", / "mesh_result.py",
), ),
(
"anyimage.operators.convert_to_plane.resolution",
PROJECT_ROOT
/ "src"
/ "anyimage"
/ "operators"
/ "convert_to_plane"
/ "resolution.py",
),
) )
packages = { packages = {
name: ModuleType(name) name: ModuleType(name)
@@ -85,6 +73,7 @@ def load_image_modules():
setattr(namespace, name, value) setattr(namespace, name, value)
return namespace return namespace
class ImageTest(unittest.TestCase): class ImageTest(unittest.TestCase):
@classmethod @classmethod
def setUpClass(cls): def setUpClass(cls):
@@ -95,6 +84,7 @@ class ImageTest(unittest.TestCase):
self.image_depth.node_asset_path(), self.image_depth.node_asset_path(),
PROJECT_ROOT / "src" / "anyimage" / "assets" / "O_AnyImage.blend", PROJECT_ROOT / "src" / "anyimage" / "assets" / "O_AnyImage.blend",
) )
def test_material_color_space_matches_scene_view_transform(self): def test_material_color_space_matches_scene_view_transform(self):
mappings = { mappings = {
"ACES 1.3": "ACES 1.3 sRGB", "ACES 1.3": "ACES 1.3 sRGB",
@@ -143,19 +133,13 @@ class ImageTest(unittest.TestCase):
def test_is_animated_image_accepts_movie_and_sequence(self): def test_is_animated_image_accepts_movie_and_sequence(self):
self.assertTrue( self.assertTrue(
self.image_depth.is_animated_image( self.image_depth.is_animated_image(SimpleNamespace(source="MOVIE"))
SimpleNamespace(source="MOVIE")
)
) )
self.assertTrue( self.assertTrue(
self.image_depth.is_animated_image( self.image_depth.is_animated_image(SimpleNamespace(source="SEQUENCE"))
SimpleNamespace(source="SEQUENCE")
)
) )
self.assertFalse( self.assertFalse(
self.image_depth.is_animated_image( self.image_depth.is_animated_image(SimpleNamespace(source="FILE"))
SimpleNamespace(source="FILE")
)
) )
def test_video_total_frames_reads_movie_duration(self): def test_video_total_frames_reads_movie_duration(self):
@@ -327,9 +311,7 @@ class ImageTest(unittest.TestCase):
def test_image_user_settings_defaults_without_user(self): def test_image_user_settings_defaults_without_user(self):
self.assertEqual( self.assertEqual(
self.image_depth.image_user_settings( self.image_depth.image_user_settings(SimpleNamespace(image_user=None)),
SimpleNamespace(image_user=None)
),
(1, 0, 0, False), (1, 0, 0, False),
) )
@@ -411,31 +393,6 @@ class ImageTest(unittest.TestCase):
["frame-1.jpg", "frame-2.jpg", "frame-10.jpg"], ["frame-1.jpg", "frame-2.jpg", "frame-10.jpg"],
) )
def test_process_resolution_scales_long_edge_and_aligns_to_patch_size(self):
image = SimpleNamespace(size=(1080, 1920))
self.assertEqual(
self.image_depth.scaled_process_resolution(image, 25.0),
476,
)
self.assertEqual(
self.image_depth.scaled_process_resolution(image, 50.0),
966,
)
def test_process_resolution_clamps_to_supported_range(self):
large_image = SimpleNamespace(size=(4000, 3000))
small_image = SimpleNamespace(size=(200, 100))
self.assertEqual(
self.image_depth.scaled_process_resolution(large_image, 100.0),
1008,
)
self.assertEqual(
self.image_depth.scaled_process_resolution(small_image, 1.0),
252,
)
def test_depth_data_uses_source_image_name(self): def test_depth_data_uses_source_image_name(self):
source_object = SimpleNamespace( source_object = SimpleNamespace(
name="obj_img", name="obj_img",

View File

@@ -1,3 +1,4 @@
import json
import builtins import builtins
import sys import sys
from pathlib import Path from pathlib import Path
@@ -9,53 +10,53 @@ import numpy as np
SOURCE_ROOT = Path(__file__).parents[1] / "src" SOURCE_ROOT = Path(__file__).parents[1] / "src"
sys.path.insert(0, str(SOURCE_ROOT)) sys.path.insert(0, str(SOURCE_ROOT))
from anyimage.operators.cutout_tool import region_to_shape as mesh
from anyimage.operators.cutout_tool import region_to_shape as cutout_nodes
from anyimage.operators.cutout_tool import region_to_shape as shapes
from anyimage.operators.cutout_tool.geometry import balloon as cutout_profile
from anyimage.operators.cutout_tool.geometry import depth_calibration as cutout_depth
from anyimage.operators.cutout_tool.geometry.balloon import poisson_balloon_profile
from anyimage.operators.common.mesh_result import ImageGeometry, load_image_geometry
from anyimage.operators.common.image_region import ( from anyimage.operators.common.image_region import (
RegionMask, RegionMask,
RegionPath, RegionPath,
rasterize_region_path, rasterize_region_path,
) )
from anyimage.operators.convert_to_plane.normal_map import rasterize_tangent_normal from anyimage.operators.common.depth_data import (
from anyimage.operators.cutout_tool.geometry.region_contours import ( depth_uniform_scale,
load_depth_metadata,
)
from anyimage.operators.cutout_tool import geometry as cutout_depth
from anyimage.operators.cutout_tool import geometry as cutout_profile
from anyimage.operators.cutout_tool import geometry as mesh
from anyimage.operators.cutout_tool import object as cutout_nodes
from anyimage.operators.cutout_tool import shape as shapes
from anyimage.operators.cutout_tool.geometry import (
RegionContours, RegionContours,
extract_region_contours, extract_region_contours,
poisson_balloon_profile,
) )
from anyimage.server.models import moge2 as moge2_model from anyimage.server.models import moge2 as moge2_model
def test_cutout_mesh_detail_uses_the_cropped_analysis_scale(): def test_cutout_mesh_detail_uses_the_cropped_analysis_scale():
assert mesh.limited_cutout_mesh_detail(256, (4096, 1024), 2048) == 128 assert mesh.limit_cutout_mesh_resolution(256, (4096, 1024), 2048) == 128
assert mesh.limited_cutout_mesh_detail(256, (1024, 512), 2048) == 256 assert mesh.limit_cutout_mesh_resolution(256, (1024, 512), 2048) == 256
assert mesh.limited_cutout_mesh_detail(8, (8192, 4096), 2048) == 4 assert mesh.limit_cutout_mesh_resolution(8, (8192, 4096), 2048) == 4
def test_image_geometry_loader_uses_one_schema_for_plane_and_cutout(tmp_path): def test_depth_metadata_loader_uses_one_schema_for_plane_and_cutout(tmp_path):
path = tmp_path / "image_geometry.npz" path = tmp_path / "depth.json"
points = np.ones((2, 3, 3), dtype=np.float32) path.write_text(
valid = np.ones((2, 3), dtype=bool) json.dumps(
np.savez_compressed( {
path, "image_size": [3, 2],
points=points, "intrinsics": np.eye(3).tolist(),
intrinsics=np.eye(3, dtype=np.float32), "reference_depth": 1.0,
valid=valid, }
calibration_valid=valid, ),
image_size=np.asarray((3, 2), dtype=np.int32), encoding="utf-8",
reference_depth=np.asarray(1.0, dtype=np.float32),
region_transparent_fraction=np.asarray(0.25, dtype=np.float32),
) )
field = load_image_geometry(path) metadata = load_depth_metadata(path)
assert metadata["image_size"] == (3, 2)
assert metadata["intrinsics"] == ((1.0, 0.0, 0.0), (0.0, 1.0, 0.0), (0.0, 0.0, 1.0))
assert metadata["reference_depth"] == 1.0
assert field.points.shape == (2, 3, 3)
assert field.image_size == (3, 2)
assert field.reference_depth == 1.0
assert field.region_transparent_fraction == 0.25
def test_cutout_node_groups_are_loaded_from_the_asset(): def test_cutout_node_groups_are_loaded_from_the_asset():
loaded = object() loaded = object()
@@ -63,7 +64,7 @@ def test_cutout_node_groups_are_loaded_from_the_asset():
cutout_nodes, "load_asset_node_group", return_value=loaded cutout_nodes, "load_asset_node_group", return_value=loaded
) as load: ) as load:
for shape in shapes.CUTOUT_NODE_GROUP_NAMES: for shape in shapes.CUTOUT_NODE_GROUP_NAMES:
assert cutout_nodes.cutout_node_group(shape) is loaded assert cutout_nodes.load_cutout_node_group() is loaded
load.assert_called_with(shapes.CUTOUT_NODE_GROUP_NAME) load.assert_called_with(shapes.CUTOUT_NODE_GROUP_NAME)
assert load.call_count == 4 assert load.call_count == 4
@@ -81,9 +82,7 @@ def test_base_plane_fit_is_stable_for_a_slender_diagonal_domain():
plane = cutout_depth._stable_plane(coordinates, values) plane = cutout_depth._stable_plane(coordinates, values)
selected_plane = coordinates @ plane[:2] + plane[2] selected_plane = coordinates @ plane[:2] + plane[2]
corners = np.asarray( corners = np.asarray(((-1.0, -1.0), (-1.0, 1.0), (1.0, -1.0), (1.0, 1.0)))
((-1.0, -1.0), (-1.0, 1.0), (1.0, -1.0), (1.0, 1.0))
)
corner_plane = corners @ plane[:2] + plane[2] corner_plane = corners @ plane[:2] + plane[2]
assert np.ptp(corner_plane) <= np.ptp(selected_plane) * 1.25 assert np.ptp(corner_plane) <= np.ptp(selected_plane) * 1.25
@@ -166,63 +165,29 @@ def mesh_component_count(faces):
return count return count
def test_tangent_normal_pixels_use_the_final_geometry_frame():
object_normal = np.zeros((3, 3, 3), dtype=np.float32)
object_normal[..., 2] = 1.0
triangle_uv = np.asarray((((0.0, 0.0), (1.0, 0.0), (0.0, 1.0)),))
triangle_normal = np.asarray((((1.0, 0.0, 0.0),) * 3,))
triangle_tangent = np.asarray((((0.0, 1.0, 0.0),) * 3,))
triangle_sign = np.ones((1, 3), dtype=np.float32)
tangent = rasterize_tangent_normal(
object_normal,
triangle_uv,
triangle_normal,
triangle_tangent,
triangle_sign,
)
assert np.allclose(tangent[0, 0], (0.0, 1.0, 0.0))
assert not np.allclose(tangent[0, 0], object_normal[0, 0])
assert np.allclose(tangent[2, 2], (0.0, 0.0, 1.0))
def test_depth_surface_discontinuity_map_uses_local_three_by_three_range():
depth = np.ones((5, 5), dtype=np.float32)
depth[:, 3:] = 2.0
valid = np.ones_like(depth, dtype=bool)
discontinuity = mesh._depth_discontinuity_map(depth, valid)
assert discontinuity[2, 0] == 0.0
assert discontinuity[2, 2] == 1.0
def test_depth_surface_projection_uses_one_scale_for_all_camera_axes(): def test_depth_surface_projection_uses_one_scale_for_all_camera_axes():
points = np.zeros((51, 101, 3), dtype=np.float32) points = np.zeros((51, 101, 3), dtype=np.float32)
points[..., 2] = np.linspace(1.0, 3.0, 51 * 101).reshape(51, 101) points[..., 2] = np.linspace(1.0, 3.0, 51 * 101).reshape(51, 101)
field = ImageGeometry( metadata = {
points=points, "intrinsics": (
intrinsics=np.asarray( (100.0, 0.0, 50.0),
((100.0, 0.0, 50.0), (0.0, 100.0, 25.0), (0.0, 0.0, 1.0)) (0.0, 100.0, 25.0),
(0.0, 0.0, 1.0),
), ),
valid=np.ones((51, 101), dtype=bool), "image_size": (101, 51),
calibration_valid=np.ones((51, 101), dtype=bool), "reference_depth": 2.0,
image_size=(101, 51), }
reference_depth=2.0,
region_transparent_fraction=0.0,
)
uniform_scale = mesh._depth_uniform_scale( uniform_scale = depth_uniform_scale(
SimpleNamespace(), SimpleNamespace(),
field, metadata,
bounds=(-2.0, 2.0, -1.0, 1.0), bounds=(-2.0, 2.0, -1.0, 1.0),
) )
assert uniform_scale == 2.0 assert uniform_scale == 2.0
reference_depth = cutout_depth.metric_depth_reference( reference_depth = cutout_depth.metric_depth_reference(
points[..., 2], points[..., 2],
field.calibration_valid, np.ones((51, 101), dtype=bool),
uniform_scale, uniform_scale,
) )
assert np.isclose(reference_depth, 5.8, atol=1e-6) assert np.isclose(reference_depth, 5.8, atol=1e-6)
@@ -245,6 +210,21 @@ def test_moge_surface_field_uses_the_model_validity_mask():
assert np.array_equal(frame.valid_mask, prediction.mask[0]) assert np.array_equal(frame.valid_mask, prediction.mask[0])
def test_moge_z_depth_frame_does_not_require_points():
prediction = moge2_model.Prediction(
depth=np.ones((1, 2, 2), dtype=np.float32),
normal=np.zeros((1, 2, 2, 3), dtype=np.float32),
mask=np.ones((1, 2, 2), dtype=bool),
intrinsics=np.eye(3, dtype=np.float32)[None],
points=None,
)
frame = moge2_model.geometry_frame(prediction)
assert frame.points is None
assert frame.valid_mask.all()
def test_region_contours_group_holes_with_their_connected_component(): def test_region_contours_group_holes_with_their_connected_component():
alpha = np.zeros((120, 200), dtype=np.uint8) alpha = np.zeros((120, 200), dtype=np.uint8)
alpha[20:100, 50:160] = 255 alpha[20:100, 50:160] = 255
@@ -259,13 +239,10 @@ def test_region_contours_group_holes_with_their_connected_component():
assert region_contours.image_size == (200, 120) assert region_contours.image_size == (200, 120)
assert sorted(len(component) for component in region_contours.components) == [1, 2] assert sorted(len(component) for component in region_contours.components) == [1, 2]
contours = [ contours = [
contour contour for component in region_contours.components for contour in component
for component in region_contours.components
for contour in component
] ]
bounds = [ bounds = [
(*np.min(contour, axis=0), *np.max(contour, axis=0)) (*np.min(contour, axis=0), *np.max(contour, axis=0)) for contour in contours
for contour in contours
] ]
assert (50.0, 20.0, 160.0, 100.0) in bounds assert (50.0, 20.0, 160.0, 100.0) in bounds
assert (80.0, 45.0, 120.0, 75.0) in bounds assert (80.0, 45.0, 120.0, 75.0) in bounds
@@ -294,7 +271,7 @@ def test_cutout_crop_bounds_map_to_the_source_plane():
empty_image_offset=(-0.5, -0.5), empty_image_offset=(-0.5, -0.5),
) )
bounds = mesh._crop_plane_bounds( bounds = mesh.crop_plane_bounds(
source, source,
(50, 25, 150, 75), (50, 25, 150, 75),
(200, 100), (200, 100),
@@ -308,7 +285,7 @@ def test_depth_surface_cutout_uses_half_alpha_boundary():
alpha[2:10, 2:14] = 0.49 alpha[2:10, 2:14] = 0.49
alpha[4:8, 5:11] = 0.51 alpha[4:8, 5:11] = 0.51
region_contours = mesh._extract_cutout_contours( region_contours = mesh.extract_cutout_contours(
RegionMask(alpha, (0, 0, 16, 12)), RegionMask(alpha, (0, 0, 16, 12)),
64, 64,
threshold=0.5, threshold=0.5,
@@ -319,6 +296,7 @@ def test_depth_surface_cutout_uses_half_alpha_boundary():
np.testing.assert_allclose(contour.min(axis=0), (5.0, 4.0)) np.testing.assert_allclose(contour.min(axis=0), (5.0, 4.0))
np.testing.assert_allclose(contour.max(axis=0), (11.0, 8.0)) np.testing.assert_allclose(contour.max(axis=0), (11.0, 8.0))
def test_blender_cutout_alpha_pipeline_does_not_import_pillow(): def test_blender_cutout_alpha_pipeline_does_not_import_pillow():
original_import = builtins.__import__ original_import = builtins.__import__
@@ -343,6 +321,7 @@ def test_blender_cutout_alpha_pipeline_does_not_import_pillow():
assert region_contours.components assert region_contours.components
assert np.any((antialiased > 0.0) & (antialiased < 1.0)) assert np.any((antialiased > 0.0) & (antialiased < 1.0))
def test_blender_cutout_contour_resolution_tracks_mesh_detail(): def test_blender_cutout_contour_resolution_tracks_mesh_detail():
expected = RegionContours( expected = RegionContours(
components=(((((0, 0), (1, 0), (0, 1))),),), components=(((((0, 0), (1, 0), (0, 1))),),),
@@ -350,37 +329,38 @@ def test_blender_cutout_contour_resolution_tracks_mesh_detail():
) )
region_mask = RegionMask(np.ones((2, 2), dtype=np.float32), (0, 0, 2, 2)) region_mask = RegionMask(np.ones((2, 2), dtype=np.float32), (0, 0, 2, 2))
with patch.object(mesh, "extract_region_contours", return_value=expected) as trace: with patch.object(mesh, "extract_region_contours", return_value=expected) as trace:
assert mesh._extract_cutout_contours(region_mask, 32) == expected assert mesh.extract_cutout_contours(region_mask, 32) == expected
assert mesh._extract_cutout_contours(region_mask, 256) == expected assert mesh.extract_cutout_contours(region_mask, 256) == expected
assert trace.call_args_list[0].kwargs["max_resolution"] == 512 assert trace.call_args_list[0].kwargs["max_resolution"] == 512
assert trace.call_args_list[1].kwargs["max_resolution"] == 1024 assert trace.call_args_list[1].kwargs["max_resolution"] == 1024
def test_cutout_point_spacing_uses_the_complete_source_plane(): def test_cutout_point_spacing_uses_the_complete_source_plane():
source = SimpleNamespace(matrix_world=object()) source = SimpleNamespace(matrix_world=object())
with ( with (
patch.object(mesh, "image_empty_bounds", return_value=(-2.0, 2.0, -1.0, 1.0)), patch.object(mesh, "image_empty_bounds", return_value=(-2.0, 2.0, -1.0, 1.0)),
patch.object( patch.object(
mesh, mesh,
"_world_plane_metric", "world_plane_metric",
return_value=(((1.0, 0.0), (0.0, 1.0)), 1.0), return_value=(((1.0, 0.0), (0.0, 1.0)), 1.0),
), ),
): ):
spacing = mesh._cutout_point_spacing(source, 200) spacing = mesh.cutout_point_spacing(source, 200)
assert spacing == 0.02 assert spacing == 0.02
def test_cutout_cleanup_value_scales_the_ui_without_changing_node_area(): def test_cutout_cleanup_value_scales_the_ui_without_changing_node_area():
assert np.isclose( assert np.isclose(
mesh._cutout_cleanup_value( mesh.cutout_cleanup_value(
0.25, 0.25,
((1.0, 0.0), (0.0, 1.0)), ((1.0, 0.0), (0.0, 1.0)),
), ),
6.25, 6.25,
) )
assert np.isclose( assert np.isclose(
mesh._cutout_cleanup_value( mesh.cutout_cleanup_value(
0.25, 0.25,
((4.0, 0.0), (0.0, 9.0)), ((4.0, 0.0), (0.0, 9.0)),
), ),
@@ -437,6 +417,8 @@ def test_poisson_balloon_profile_makes_wide_regions_thicker():
narrow_profile = poisson_balloon_profile(narrow, faces, boundary) narrow_profile = poisson_balloon_profile(narrow, faces, boundary)
assert wide_profile[4] > narrow_profile[4] assert wide_profile[4] > narrow_profile[4]
def test_teddy_balloon_profile_uses_chordal_axis_radius(): def test_teddy_balloon_profile_uses_chordal_axis_radius():
points = np.asarray( points = np.asarray(
((0, 0), (4, 0), (4, 2), (0, 2), (2, 1)), ((0, 0), (4, 0), (4, 2), (0, 2), (2, 1)),
@@ -463,14 +445,10 @@ def test_teddy_balloon_profile_uses_chordal_axis_radius():
assert np.isclose(profile[4], 0.45, atol=1e-6) assert np.isclose(profile[4], 0.45, atol=1e-6)
def test_cutout_domain_adapts_boundary_spacing_for_a_small_valid_region(): def test_cutout_domain_adapts_boundary_spacing_for_a_small_valid_region():
regions = ( regions = ((((2046, 2046), (2050, 2046), (2050, 2050), (2046, 2050)),),)
(((2046, 2046), (2050, 2046), (2050, 2050), (2046, 2050)),),
)
domain = mesh._cutout_domain( domain = mesh.cutout_domain(
RegionContours(regions, (4096, 4096)), RegionContours(regions, (4096, 4096)),
(-1.0, 1.0, -1.0, 1.0), (-1.0, 1.0, -1.0, 1.0),
0.5, 0.5,
@@ -484,7 +462,7 @@ def test_cutout_domain_adapts_boundary_spacing_for_a_small_valid_region():
def test_cutout_domain_builds_a_full_image_at_minimum_detail(): def test_cutout_domain_builds_a_full_image_at_minimum_detail():
regions = ((((0, 0), (4096, 0), (4096, 4096), (0, 4096)),),) regions = ((((0, 0), (4096, 0), (4096, 4096), (0, 4096)),),)
domain = mesh._cutout_domain( domain = mesh.cutout_domain(
RegionContours(regions, (4096, 4096)), RegionContours(regions, (4096, 4096)),
(-1.0, 1.0, -1.0, 1.0), (-1.0, 1.0, -1.0, 1.0),
0.5, 0.5,
@@ -494,10 +472,11 @@ def test_cutout_domain_builds_a_full_image_at_minimum_detail():
assert len(domain[0]) >= 4 assert len(domain[0]) >= 4
assert len(domain[1]) >= 2 assert len(domain[1]) >= 2
def test_cutout_domain_centers_vertices_on_crop_bounds(): def test_cutout_domain_centers_vertices_on_crop_bounds():
regions = ((((0, 0), (100, 0), (100, 50), (0, 50)),),) regions = ((((0, 0), (100, 0), (100, 50), (0, 50)),),)
vertices, _faces, uv, _boundary, _planar, _profile = mesh._cutout_domain( vertices, _faces, uv, _boundary, _planar, _profile = mesh.cutout_domain(
RegionContours(regions, (100, 50)), RegionContours(regions, (100, 50)),
(2.0, 6.0, -3.0, 1.0), (2.0, 6.0, -3.0, 1.0),
0.5, 0.5,
@@ -516,11 +495,9 @@ def test_cutout_domain_keeps_overlapping_regions_disconnected():
((20, 0), (100, 0), (100, 80), (20, 80)), ((20, 0), (100, 0), (100, 80), (20, 80)),
((50, 20), (50, 60), (70, 60), (70, 20)), ((50, 20), (50, 60), (70, 60), (70, 20)),
), ),
( (((0, 20), (40, 20), (40, 50), (0, 50)),),
((0, 20), (40, 20), (40, 50), (0, 50)),
),
) )
domain = mesh._cutout_domain( domain = mesh.cutout_domain(
RegionContours(regions, (120, 80)), RegionContours(regions, (120, 80)),
(0.0, 1.0, 0.0, 1.0), (0.0, 1.0, 0.0, 1.0),
0.025, 0.025,
@@ -544,3 +521,15 @@ def test_cutout_domain_keeps_overlapping_regions_disconnected():
assert not in_hole.any() assert not in_hole.any()
assert in_left_region.any() assert in_left_region.any()
assert mesh_component_count(domain[1]) == 2 assert mesh_component_count(domain[1]) == 2
def test_region_calibration_uses_the_final_region_mask():
region_mask = RegionMask(
np.asarray(((1.0, 1.0), (1.0, 0.0)), dtype=np.float32),
(0, 0, 2, 2),
)
valid, transparent_fraction = mesh.region_calibration(region_mask, (2, 2))
assert np.array_equal(valid, ((True, True), (True, False)))
assert transparent_fraction == 0.25

View File

@@ -5,27 +5,26 @@ from unittest.mock import patch
import bpy import bpy
import numpy as np import numpy as np
from mathutils import Matrix, Vector from mathutils import Vector
SOURCE_ROOT = Path(__file__).parents[1] / "src" SOURCE_ROOT = Path(__file__).parents[1] / "src"
sys.path.insert(0, str(SOURCE_ROOT)) sys.path.insert(0, str(SOURCE_ROOT))
from anyimage.operators.common import image_data as common_image from anyimage.operators.common import image_data as common_image
from anyimage.operators.common import depth_data as common_depth
from anyimage.operators.common import mesh_result as material from anyimage.operators.common import mesh_result as material
from anyimage.operators.common.mesh_result import ImageGeometry
from anyimage.operators.common.image_region import ( from anyimage.operators.common.image_region import (
RegionMask, RegionMask,
RegionPath, RegionPath,
rasterize_region_path, rasterize_region_path,
) )
from anyimage.operators.cutout_tool import region_to_shape as mesh
from anyimage.operators.cutout_tool import region_to_shape as cutout_nodes
from anyimage.operators.cutout_tool import region_to_shape as shapes
from anyimage.operators.convert_to_plane import nodes as plane_nodes from anyimage.operators.convert_to_plane import nodes as plane_nodes
from anyimage.operators.convert_to_plane import build as plane_build from anyimage.operators.convert_to_plane import object as plane_object
from anyimage.operators.convert_to_plane.normal_map import ( from anyimage.operators.cutout_tool import geometry as cutout_geometry
bake_tangent_normal_image, from anyimage.operators.cutout_tool import main as cutout_main
) from anyimage.operators.cutout_tool import object as cutout_object
from anyimage.operators.cutout_tool import shape as shapes
from anyimage.server.geometry.depth_texture import write_vector_depth_texture
def _inputs(group): def _inputs(group):
@@ -44,93 +43,101 @@ def _group_node(group, node_tree):
) )
def _create_cutout_shape(source, shape, detail, *, bounds=None, field=None): def _create_cutout_shape(
source,
shape,
detail,
*,
bounds=None,
depth_image=None,
depth_metadata=None,
region_values=None,
):
if bounds is None: if bounds is None:
bounds = (0, 0, *tuple(int(value) for value in source.data.size)) bounds = (0, 0, *tuple(int(value) for value in source.data.size))
left, top, right, bottom = bounds left, top, right, bottom = bounds
region_mask = RegionMask( if region_values is None:
np.ones((bottom - top, right - left), dtype=np.float32), region_values = np.ones((bottom - top, right - left), dtype=np.float32)
bounds, region_mask = RegionMask(region_values, bounds)
) base_shape = cutout_geometry.build_base_shape(
base_shape = mesh.build_base_shape(
bpy.context, bpy.context,
source, source,
detail, detail,
region_mask, region_mask,
) )
return mesh.create_shape_object( return cutout_object.create_shape_object(
bpy.context, bpy.context,
source, source,
shape, shape,
detail, detail,
region_mask, region_mask,
base_shape, base_shape,
field=field, depth_image=depth_image,
depth_metadata=depth_metadata,
) )
def test_depth_surface_image_packs_metric_points_and_edge_validity(): def _depth_image(points, valid, name="Depth"):
points = np.asarray(points, dtype=np.float32)
valid = np.asarray(valid, dtype=bool)
height, width = valid.shape
pixels = np.zeros((height, width, 4), dtype=np.float32)
pixels[valid, :3] = points[valid]
pixels[..., 3] = valid
image = bpy.data.images.new(
name,
width=width,
height=height,
alpha=True,
float_buffer=True,
)
image.colorspace_settings.name = "Non-Color"
image.alpha_mode = "CHANNEL_PACKED"
image.pixels.foreach_set(np.flipud(pixels).ravel())
image.pack()
return image
def _depth_metadata(intrinsics, image_size, reference_depth):
return {
"intrinsics": tuple(tuple(float(value) for value in row) for row in intrinsics),
"image_size": tuple(int(value) for value in image_size),
"reference_depth": float(reference_depth),
}
def test_depth_result_image_loads_server_vector_exr_and_keeps_data_name(tmp_path):
bpy.ops.wm.read_factory_settings(use_empty=True) bpy.ops.wm.read_factory_settings(use_empty=True)
source_image = bpy.data.images.new("Source.png", width=2, height=2) source_image = bpy.data.images.new("Source.png", width=2, height=2)
source = bpy.data.objects.new("Source", None) source = bpy.data.objects.new("Source", None)
source.data = source_image source.data = source_image
points = np.arange(12, dtype=np.float32).reshape((2, 2, 3)) points = np.arange(12, dtype=np.float32).reshape((2, 2, 3))
depth = np.asarray(((1.0, 1.0), (1.0, 2.0)), dtype=np.float32)
valid = np.asarray(((True, False), (True, True))) valid = np.asarray(((True, False), (True, True)))
path = tmp_path / "vector-depth.exr"
write_vector_depth_texture(points, valid, path)
metadata = _depth_metadata(np.eye(3), (2, 2), 1.0)
image = mesh._cutout_depth_image(source, points, valid, depth) image = common_depth.load_depth_result_image(path, source, metadata)
packed = np.empty(16, dtype=np.float32) packed = np.empty(16, dtype=np.float32)
image.pixels.foreach_get(packed) image.pixels.foreach_get(packed)
packed = np.flipud(packed.reshape((2, 2, 4))) packed = np.flipud(packed.reshape((2, 2, 4)))
assert np.array_equal(packed[:, :, :3], points) assert np.array_equal(packed[valid, :3], points[valid])
assert packed[0, 1, 3] == -1.0 assert np.array_equal(packed[..., 3] > 0.5, valid)
assert np.all(packed[valid, 3] >= 0.0)
assert image.alpha_mode == "CHANNEL_PACKED" assert image.alpha_mode == "CHANNEL_PACKED"
assert image.name == "Source_depth.exr" assert image.name == "Source_depth.exr"
def test_tangent_bake_changes_pixels_using_evaluated_mesh_tbn():
bpy.ops.wm.read_factory_settings(use_empty=True)
source = bpy.data.objects.new("Source", None)
bpy.context.collection.objects.link(source)
target_mesh = bpy.data.meshes.new("Target")
target_mesh.from_pydata(
((0.0, 0.0, 0.0), (0.0, 1.0, 0.0), (0.0, 0.0, 1.0)),
(),
((0, 1, 2),),
)
target_mesh.polygons[0].use_smooth = True
mesh._assign_uv(
target_mesh,
np.asarray(((0.0, 0.0), (1.0, 0.0), (0.0, 1.0))),
)
target = bpy.data.objects.new("Target", target_mesh)
bpy.context.collection.objects.link(target)
target.matrix_world = Matrix.Rotation(np.radians(-90.0), 4, "Y")
image = bpy.data.images.new("Object Normal", width=3, height=3, alpha=True)
pixels = np.ones((3, 3, 4), dtype=np.float32)
pixels[..., :3] = (1.0, 0.5, 0.5)
image.pixels.foreach_set(pixels.ravel())
bake_tangent_normal_image(bpy.context, source, target, image)
baked = np.empty(3 * 3 * 4, dtype=np.float32)
image.pixels.foreach_get(baked)
baked = baked.reshape((3, 3, 4))
assert not np.allclose(baked[0, 0, :3], pixels[0, 0, :3])
def test_cutout_modifier_wraps_four_menu_controlled_shape_groups(): def test_cutout_modifier_wraps_four_menu_controlled_shape_groups():
bpy.ops.wm.read_factory_settings(use_empty=True) bpy.ops.wm.read_factory_settings(use_empty=True)
plane = plane_nodes.plane_node_group("PLANE") plane = plane_nodes.plane_node_group("PLANE")
depth_plane = plane_nodes.plane_node_group("DEPTH_PLANE") depth_plane = plane_nodes.plane_node_group("DEPTH_PLANE")
cutout = cutout_nodes.cutout_node_group("SURFACE") cutout = cutout_object.load_cutout_node_group()
assert plane.name == "O Mesh Plane" assert plane.name == "O Mesh Plane"
assert cutout.name == shapes.CUTOUT_NODE_GROUP_NAME assert cutout.name == shapes.CUTOUT_NODE_GROUP_NAME
assert all( assert all(
cutout_nodes.cutout_node_group(shape) == cutout cutout_object.load_cutout_node_group() == cutout
for shape in shapes.CUTOUT_SHAPES for shape in shapes.CUTOUT_SHAPES
) )
assert _inputs(plane) == ["Geometry", "Subdivide", "Thickness", "Material"] assert _inputs(plane) == ["Geometry", "Subdivide", "Thickness", "Material"]
@@ -139,6 +146,8 @@ def test_cutout_modifier_wraps_four_menu_controlled_shape_groups():
"Subdivide", "Subdivide",
"Thickness", "Thickness",
"Depth Amount", "Depth Amount",
"Reference Depth",
"Uniform Scale",
"Depth Image", "Depth Image",
"Material", "Material",
] ]
@@ -156,9 +165,7 @@ def test_cutout_modifier_wraps_four_menu_controlled_shape_groups():
"Uniform Scale", "Uniform Scale",
"Depth Image", "Depth Image",
] ]
panels = [ panels = [item for item in cutout.interface.items_tree if item.item_type == "PANEL"]
item for item in cutout.interface.items_tree if item.item_type == "PANEL"
]
assert [panel.name for panel in panels] == ["Options", "Data"] assert [panel.name for panel in panels] == ["Options", "Data"]
assert [item.name for item in panels[0].interface_items] == [ assert [item.name for item in panels[0].interface_items] == [
"Cleanup", "Cleanup",
@@ -258,33 +265,24 @@ def test_cutout_modifier_wraps_four_menu_controlled_shape_groups():
"Depth Image": {"DEPTH_BALLOON", "DEPTH_SURFACE"}, "Depth Image": {"DEPTH_BALLOON", "DEPTH_SURFACE"},
} }
assert sum( assert (
node.bl_idname == "GeometryNodeInputMeshIsland" for node in cutout.nodes sum(node.bl_idname == "GeometryNodeInputMeshIsland" for node in cutout.nodes)
) == 1 == 1
)
assert not any( assert not any(
node.bl_idname == "GeometryNodeAttributeStatistic" node.bl_idname == "GeometryNodeAttributeStatistic" for node in cutout.nodes
for node in cutout.nodes
) )
assert all( assert all(
not any( not any(node.bl_idname == "GeometryNodeInputMeshIsland" for node in group.nodes)
node.bl_idname == "GeometryNodeInputMeshIsland"
for node in group.nodes
)
for group in child_groups.values() for group in child_groups.values()
) )
assert any( assert any(node.bl_idname == "GeometryNodeBlurAttribute" for node in cutout.nodes)
node.bl_idname == "GeometryNodeBlurAttribute" for node in cutout.nodes
)
assert all( assert all(
not any( not any(node.bl_idname == "GeometryNodeBlurAttribute" for node in group.nodes)
node.bl_idname == "GeometryNodeBlurAttribute" for node in group.nodes
)
for group in child_groups.values() for group in child_groups.values()
) )
repeat_counts = { repeat_counts = {
shape: sum( shape: sum(node.bl_idname == "GeometryNodeRepeatInput" for node in group.nodes)
node.bl_idname == "GeometryNodeRepeatInput" for node in group.nodes
)
for shape, group in child_groups.items() for shape, group in child_groups.items()
} }
assert repeat_counts == { assert repeat_counts == {
@@ -293,9 +291,9 @@ def test_cutout_modifier_wraps_four_menu_controlled_shape_groups():
"DEPTH_BALLOON": 0, "DEPTH_BALLOON": 0,
"DEPTH_SURFACE": 1, "DEPTH_SURFACE": 1,
} }
assert sum( assert (
node.bl_idname == "GeometryNodeRepeatInput" for node in cutout.nodes sum(node.bl_idname == "GeometryNodeRepeatInput" for node in cutout.nodes) == 1
) == 1 )
smooth = next( smooth = next(
item item
@@ -338,15 +336,12 @@ def test_cutout_modifier_wraps_four_menu_controlled_shape_groups():
for group in child_groups.values(): for group in child_groups.values():
extrudes = [ extrudes = [
node node for node in group.nodes if node.bl_idname == "GeometryNodeExtrudeMesh"
for node in group.nodes
if node.bl_idname == "GeometryNodeExtrudeMesh"
] ]
assert extrudes assert extrudes
assert all(not node.inputs["Individual"].default_value for node in extrudes) assert all(not node.inputs["Individual"].default_value for node in extrudes)
assert all( assert all(
node.inputs["Offset Scale"].default_value == 1.0 node.inputs["Offset Scale"].default_value == 1.0 for node in extrudes
for node in extrudes
) )
for group in ( for group in (
child_groups["SURFACE"], child_groups["SURFACE"],
@@ -371,6 +366,7 @@ def test_cutout_modifier_wraps_four_menu_controlled_shape_groups():
assert thickness.subtype == "NONE" assert thickness.subtype == "NONE"
assert thickness.default_value == 1.0 assert thickness.default_value == 1.0
def test_image_layer_switches_between_tangent_and_object_space_normal(): def test_image_layer_switches_between_tangent_and_object_space_normal():
bpy.ops.wm.read_factory_settings(use_empty=True) bpy.ops.wm.read_factory_settings(use_empty=True)
@@ -409,9 +405,7 @@ def test_image_layer_switches_between_tangent_and_object_space_normal():
) )
assert object_space.default_value is False assert object_space.default_value is False
normal_maps = { normal_maps = {
node.space node.space for node in group.nodes if node.bl_idname == "ShaderNodeNormalMap"
for node in group.nodes
if node.bl_idname == "ShaderNodeNormalMap"
} }
assert normal_maps == {"TANGENT", "OBJECT"} assert normal_maps == {"TANGENT", "OBJECT"}
assert all( assert all(
@@ -442,9 +436,7 @@ def test_image_layer_switches_between_tangent_and_object_space_normal():
assert layer.inputs["Object Space"].default_value is False assert layer.inputs["Object Space"].default_value is False
reused = material.material_node_group() reused = material.material_node_group()
assert { assert {
node.space node.space for node in reused.nodes if node.bl_idname == "ShaderNodeNormalMap"
for node in reused.nodes
if node.bl_idname == "ShaderNodeNormalMap"
} == {"TANGENT", "OBJECT"} } == {"TANGENT", "OBJECT"}
@@ -458,7 +450,7 @@ def test_build_plane_keeps_one_quad_and_generates_topology_in_geometry_nodes():
bpy.context.collection.objects.link(source) bpy.context.collection.objects.link(source)
material = bpy.data.materials.new("Source.material") material = bpy.data.materials.new("Source.material")
result = plane_build.build_mesh_plane( result = plane_object.create_plane_object(
bpy.context, bpy.context,
source, source,
2, 2,
@@ -517,7 +509,7 @@ def test_teddy_preference_writes_the_shared_balloon_attribute():
source.empty_display_size = 2.0 source.empty_display_size = 2.0
bpy.context.collection.objects.link(source) bpy.context.collection.objects.link(source)
with patch.object( with patch.object(
mesh, cutout_geometry,
"addon_preferences", "addon_preferences",
return_value=SimpleNamespace(balloon_profile_algorithm="TEDDY"), return_value=SimpleNamespace(balloon_profile_algorithm="TEDDY"),
): ):
@@ -533,6 +525,24 @@ def test_teddy_preference_writes_the_shared_balloon_attribute():
assert profile.min() == 0.0 and profile.max() > 0.0 assert profile.min() == 0.0 and profile.max() > 0.0
def test_cutout_material_uses_ior_1_2():
bpy.ops.wm.read_factory_settings(use_empty=True)
image = bpy.data.images.new("Cutout", width=16, height=16, alpha=True)
source = bpy.data.objects.new("Cutout", None)
source.empty_display_type = "IMAGE"
source.data = image
source.empty_display_size = 2.0
bpy.context.collection.objects.link(source)
result = _create_cutout_shape(source, "SURFACE", 8)
shader = next(
node
for node in result.data.materials[0].node_tree.nodes
if node.bl_idname == "ShaderNodeBsdfPrincipled"
)
assert np.isclose(shader.inputs["IOR"].default_value, 1.2)
def test_cutout_material_uses_flat_tangent_normal(): def test_cutout_material_uses_flat_tangent_normal():
bpy.ops.wm.read_factory_settings(use_empty=True) bpy.ops.wm.read_factory_settings(use_empty=True)
source = bpy.data.objects.new("Source", None) source = bpy.data.objects.new("Source", None)
@@ -553,18 +563,6 @@ def test_cutout_material_uses_flat_tangent_normal():
assert layer.inputs["Object Space"].default_value is False assert layer.inputs["Object Space"].default_value is False
def test_internal_depth_plane_image_uses_standard_data_name():
bpy.ops.wm.read_factory_settings(use_empty=True)
plane_source = bpy.data.objects.new("Plane Source", None)
plane_source.data = bpy.data.images.new(
"Plane Source.png", width=2, height=2, alpha=True
)
anchored = np.zeros((2, 2, 3), dtype=np.float32)
plane_depth = plane_build.create_depth_image(plane_source, anchored)
assert plane_depth.name == "Plane Source_depth.exr"
def test_depth_surface_protected_smooth_evaluates(): def test_depth_surface_protected_smooth_evaluates():
bpy.ops.wm.read_factory_settings(use_empty=True) bpy.ops.wm.read_factory_settings(use_empty=True)
image = bpy.data.images.new("Surface", width=64, height=64) image = bpy.data.images.new("Surface", width=64, height=64)
@@ -584,54 +582,52 @@ def test_depth_surface_protected_smooth_evaluates():
points[..., 0] = (x - intrinsics[0, 2]) / intrinsics[0, 0] * depth points[..., 0] = (x - intrinsics[0, 2]) / intrinsics[0, 0] * depth
points[..., 1] = (y - intrinsics[1, 2]) / intrinsics[1, 1] * depth points[..., 1] = (y - intrinsics[1, 2]) / intrinsics[1, 1] * depth
points[..., 2] = depth points[..., 2] = depth
field = ImageGeometry( valid = np.ones((size, size), dtype=bool)
points=points, depth_image = _depth_image(points, valid)
intrinsics=intrinsics, depth_metadata = _depth_metadata(intrinsics, (size, size), 1.0)
valid=np.ones((size, size), dtype=bool),
calibration_valid=np.ones((size, size), dtype=bool),
image_size=(size, size),
reference_depth=1.0,
region_transparent_fraction=0.0,
)
result = _create_cutout_shape( result = _create_cutout_shape(
source, source,
"DEPTH_SURFACE", "DEPTH_SURFACE",
8, 8,
field=field, depth_image=depth_image,
depth_metadata=depth_metadata,
) )
assert all(abs(vertex.co.x) < 1e-8 for vertex in result.data.vertices) assert all(abs(vertex.co.x) < 1e-8 for vertex in result.data.vertices)
assert shapes.BALLOON_ATTRIBUTE_NAME in result.data.attributes assert shapes.BALLOON_ATTRIBUTE_NAME in result.data.attributes
modifier = result.modifiers[0] modifier = result.modifiers[0]
assert modifier.node_group.name == shapes.CUTOUT_NODE_GROUP_NAME assert modifier.node_group.name == shapes.CUTOUT_NODE_GROUP_NAME
assert modifier[ assert (
mesh.modifier_input_identifier(modifier.node_group, "Shape") modifier[material.modifier_input_identifier(modifier.node_group, "Shape")]
] == shapes.CUTOUT_SHAPE_INDICES["DEPTH_SURFACE"] == shapes.CUTOUT_SHAPE_INDICES["DEPTH_SURFACE"]
expected_cleanup = mesh._cutout_cleanup_value( )
mesh._cutout_point_spacing(source, 8), expected_cleanup = cutout_geometry.cutout_cleanup_value(
mesh._world_plane_metric(source.matrix_world)[0], cutout_geometry.cutout_point_spacing(source, 8),
cutout_geometry.world_plane_metric(source.matrix_world)[0],
) )
assert np.isclose( assert np.isclose(
modifier[mesh.modifier_input_identifier(modifier.node_group, "Cleanup")], modifier[material.modifier_input_identifier(modifier.node_group, "Cleanup")],
expected_cleanup, expected_cleanup,
) )
assert modifier[ assert (
mesh.modifier_input_identifier(modifier.node_group, "Smooth") modifier[material.modifier_input_identifier(modifier.node_group, "Smooth")]
] == 1.0 == 1.0
assert modifier[ )
mesh.modifier_input_identifier( assert (
modifier.node_group, "Smooth Iterations" modifier[
) material.modifier_input_identifier(modifier.node_group, "Smooth Iterations")
] == 0 ]
reference_id = mesh.modifier_input_identifier( == 0
)
reference_id = material.modifier_input_identifier(
modifier.node_group, modifier.node_group,
"Base Plane Depth", "Base Plane Depth",
) )
assert modifier[reference_id] == 8.0 assert modifier[reference_id] == 8.0
mesh.set_modifier_input( material.set_modifier_input(
modifier, modifier,
mesh.modifier_input_identifier(modifier.node_group, "Smooth"), material.modifier_input_identifier(modifier.node_group, "Smooth"),
0.0, 0.0,
) )
evaluated = result.evaluated_get(bpy.context.evaluated_depsgraph_get()) evaluated = result.evaluated_get(bpy.context.evaluated_depsgraph_get())
@@ -643,9 +639,9 @@ def test_depth_surface_protected_smooth_evaluates():
finally: finally:
evaluated.to_mesh_clear() evaluated.to_mesh_clear()
mesh.set_modifier_input( material.set_modifier_input(
modifier, modifier,
mesh.modifier_input_identifier(modifier.node_group, "Thickness"), material.modifier_input_identifier(modifier.node_group, "Thickness"),
1.0, 1.0,
) )
result.update_tag(refresh={"DATA"}) result.update_tag(refresh={"DATA"})
@@ -660,16 +656,14 @@ def test_depth_surface_protected_smooth_evaluates():
finally: finally:
evaluated.to_mesh_clear() evaluated.to_mesh_clear()
mesh.set_modifier_input( material.set_modifier_input(
modifier, modifier,
mesh.modifier_input_identifier( material.modifier_input_identifier(modifier.node_group, "Smooth Iterations"),
modifier.node_group, "Smooth Iterations"
),
4, 4,
) )
mesh.set_modifier_input( material.set_modifier_input(
modifier, modifier,
mesh.modifier_input_identifier(modifier.node_group, "Smooth"), material.modifier_input_identifier(modifier.node_group, "Smooth"),
1.0, 1.0,
) )
result.update_tag(refresh={"DATA"}) result.update_tag(refresh={"DATA"})
@@ -688,7 +682,6 @@ def test_depth_surface_protected_smooth_evaluates():
assert np.isfinite(internal_positions).all() assert np.isfinite(internal_positions).all()
def _evaluated_positions(obj): def _evaluated_positions(obj):
evaluated = obj.evaluated_get(bpy.context.evaluated_depsgraph_get()) evaluated = obj.evaluated_get(bpy.context.evaluated_depsgraph_get())
result = evaluated.to_mesh() result = evaluated.to_mesh()
@@ -699,38 +692,35 @@ def _evaluated_positions(obj):
def _add_cutout_modifier(obj, shape): def _add_cutout_modifier(obj, shape):
group = cutout_nodes.cutout_node_group(shape) group = cutout_object.load_cutout_node_group()
modifier = obj.modifiers.new(group.name, "NODES") modifier = obj.modifiers.new(group.name, "NODES")
modifier.node_group = group modifier.node_group = group
mesh.set_modifier_input( material.set_modifier_input(
modifier, modifier,
mesh.modifier_input_identifier(group, "Shape"), material.modifier_input_identifier(group, "Shape"),
shapes.CUTOUT_SHAPE_INDICES[shape], shapes.CUTOUT_SHAPE_INDICES[shape],
) )
mesh.set_modifier_input( material.set_modifier_input(
modifier, modifier,
mesh.modifier_input_identifier(group, "Cleanup"), material.modifier_input_identifier(group, "Cleanup"),
0.0, 0.0,
) )
mesh.set_modifier_input( material.set_modifier_input(
modifier, modifier,
mesh.modifier_input_identifier(group, "Smooth"), material.modifier_input_identifier(group, "Smooth"),
0.0, 0.0,
) )
mesh.set_modifier_input( material.set_modifier_input(
modifier, modifier,
mesh.modifier_input_identifier(group, "Smooth Iterations"), material.modifier_input_identifier(group, "Smooth Iterations"),
1, 1,
) )
return modifier return modifier
def test_cutout_thickness_keeps_one_connected_shell(): def test_cutout_thickness_keeps_one_connected_shell():
bpy.ops.wm.read_factory_settings(use_empty=True) bpy.ops.wm.read_factory_settings(use_empty=True)
vertices = [ vertices = [(0.0, float(y), float(z)) for z in range(3) for y in range(3)]
(0.0, float(y), float(z))
for z in range(3)
for y in range(3)
]
faces = [] faces = []
for z in range(2): for z in range(2):
for y in range(2): for y in range(2):
@@ -742,14 +732,14 @@ def test_cutout_thickness_keeps_one_connected_shell():
bpy.context.collection.objects.link(obj) bpy.context.collection.objects.link(obj)
modifier = _add_cutout_modifier(obj, "SURFACE") modifier = _add_cutout_modifier(obj, "SURFACE")
mesh.set_modifier_input( material.set_modifier_input(
modifier, modifier,
mesh.modifier_input_identifier(modifier.node_group, "Cleanup"), material.modifier_input_identifier(modifier.node_group, "Cleanup"),
0.0, 0.0,
) )
mesh.set_modifier_input( material.set_modifier_input(
modifier, modifier,
mesh.modifier_input_identifier(modifier.node_group, "Thickness"), material.modifier_input_identifier(modifier.node_group, "Thickness"),
1.0, 1.0,
) )
@@ -764,9 +754,7 @@ def test_cutout_thickness_keeps_one_connected_shell():
front_faces = [ front_faces = [
polygon for polygon in result.polygons if polygon.normal.x < -0.9 polygon for polygon in result.polygons if polygon.normal.x < -0.9
] ]
back_faces = [ back_faces = [polygon for polygon in result.polygons if polygon.normal.x > 0.9]
polygon for polygon in result.polygons if polygon.normal.x > 0.9
]
side_faces = [ side_faces = [
polygon for polygon in result.polygons if abs(polygon.normal.x) < 0.1 polygon for polygon in result.polygons if abs(polygon.normal.x) < 0.1
] ]
@@ -780,9 +768,9 @@ def test_cutout_thickness_keeps_one_connected_shell():
finally: finally:
evaluated.to_mesh_clear() evaluated.to_mesh_clear()
mesh.set_modifier_input( material.set_modifier_input(
modifier, modifier,
mesh.modifier_input_identifier(modifier.node_group, "Thickness"), material.modifier_input_identifier(modifier.node_group, "Thickness"),
2.4, 2.4,
) )
obj.update_tag(refresh={"DATA"}) obj.update_tag(refresh={"DATA"})
@@ -809,11 +797,7 @@ def test_cutout_thickness_keeps_one_connected_shell():
def test_balloon_uses_injected_profile_at_default_height(): def test_balloon_uses_injected_profile_at_default_height():
bpy.ops.wm.read_factory_settings(use_empty=True) bpy.ops.wm.read_factory_settings(use_empty=True)
vertices = [ vertices = [(0.0, float(y), float(z)) for z in range(3) for y in range(3)]
(0.0, float(y), float(z))
for z in range(3)
for y in range(3)
]
faces = [] faces = []
for z in range(2): for z in range(2):
for y in range(2): for y in range(2):
@@ -821,7 +805,7 @@ def test_balloon_uses_injected_profile_at_default_height():
faces.extend(((a, a + 1, a + 4), (a, a + 4, a + 3))) faces.extend(((a, a + 1, a + 4), (a, a + 4, a + 3)))
data = bpy.data.meshes.new("Balloon Profile") data = bpy.data.meshes.new("Balloon Profile")
data.from_pydata(vertices, (), faces) data.from_pydata(vertices, (), faces)
mesh._assign_float_point_attribute( cutout_object._assign_float_point_attribute(
data, data,
shapes.BALLOON_ATTRIBUTE_NAME, shapes.BALLOON_ATTRIBUTE_NAME,
(0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0), (0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0),
@@ -830,14 +814,14 @@ def test_balloon_uses_injected_profile_at_default_height():
bpy.context.collection.objects.link(obj) bpy.context.collection.objects.link(obj)
modifier = _add_cutout_modifier(obj, "BALLOON") modifier = _add_cutout_modifier(obj, "BALLOON")
mesh.set_modifier_input( material.set_modifier_input(
modifier, modifier,
mesh.modifier_input_identifier(modifier.node_group, "Cleanup"), material.modifier_input_identifier(modifier.node_group, "Cleanup"),
0.0, 0.0,
) )
mesh.set_modifier_input( material.set_modifier_input(
modifier, modifier,
mesh.modifier_input_identifier(modifier.node_group, "Thickness"), material.modifier_input_identifier(modifier.node_group, "Thickness"),
1.0, 1.0,
) )
@@ -856,9 +840,9 @@ def test_balloon_uses_injected_profile_at_default_height():
baseline_face_count = len(result.polygons) baseline_face_count = len(result.polygons)
finally: finally:
evaluated.to_mesh_clear() evaluated.to_mesh_clear()
mesh.set_modifier_input( material.set_modifier_input(
modifier, modifier,
mesh.modifier_input_identifier(modifier.node_group, "Thickness"), material.modifier_input_identifier(modifier.node_group, "Thickness"),
2.0, 2.0,
) )
obj.update_tag(refresh={"DATA"}) obj.update_tag(refresh={"DATA"})
@@ -872,9 +856,9 @@ def test_balloon_uses_injected_profile_at_default_height():
assert np.isclose(x.max(), 2.0, atol=1e-6) assert np.isclose(x.max(), 2.0, atol=1e-6)
finally: finally:
evaluated.to_mesh_clear() evaluated.to_mesh_clear()
mesh.set_modifier_input( material.set_modifier_input(
modifier, modifier,
mesh.modifier_input_identifier(modifier.node_group, "Thickness"), material.modifier_input_identifier(modifier.node_group, "Thickness"),
0.0, 0.0,
) )
obj.update_tag(refresh={"DATA"}) obj.update_tag(refresh={"DATA"})
@@ -886,11 +870,7 @@ def test_balloon_uses_injected_profile_at_default_height():
def test_cutout_cleanup_uses_absolute_area_with_many_small_islands(): def test_cutout_cleanup_uses_absolute_area_with_many_small_islands():
bpy.ops.wm.read_factory_settings(use_empty=True) bpy.ops.wm.read_factory_settings(use_empty=True)
vertices = [ vertices = [(0.0, float(y), float(z)) for z in range(3) for y in range(3)]
(0.0, float(y), float(z))
for z in range(3)
for y in range(3)
]
faces = [] faces = []
for z in range(2): for z in range(2):
for y in range(2): for y in range(2):
@@ -899,9 +879,7 @@ def test_cutout_cleanup_uses_absolute_area_with_many_small_islands():
for island in range(12): for island in range(12):
start = len(vertices) start = len(vertices)
y = 5.0 + island * 0.1 y = 5.0 + island * 0.1
vertices.extend( vertices.extend(((0.0, y, 5.0), (0.0, y + 0.01, 5.0), (0.0, y, 5.01)))
((0.0, y, 5.0), (0.0, y + 0.01, 5.0), (0.0, y, 5.01))
)
faces.append((start, start + 1, start + 2)) faces.append((start, start + 1, start + 2))
data = bpy.data.meshes.new("Cleanup") data = bpy.data.meshes.new("Cleanup")
data.from_pydata(vertices, (), faces) data.from_pydata(vertices, (), faces)
@@ -909,13 +887,13 @@ def test_cutout_cleanup_uses_absolute_area_with_many_small_islands():
bpy.context.collection.objects.link(obj) bpy.context.collection.objects.link(obj)
modifier = _add_cutout_modifier(obj, "SURFACE") modifier = _add_cutout_modifier(obj, "SURFACE")
cleanup_id = mesh.modifier_input_identifier(modifier.node_group, "Cleanup") cleanup_id = material.modifier_input_identifier(modifier.node_group, "Cleanup")
mesh.set_modifier_input(modifier, cleanup_id, 0.0) material.set_modifier_input(modifier, cleanup_id, 0.0)
bpy.context.view_layer.update() bpy.context.view_layer.update()
assert len(_evaluated_positions(obj)) == 45 assert len(_evaluated_positions(obj)) == 45
mesh.set_modifier_input(modifier, cleanup_id, 0.1) material.set_modifier_input(modifier, cleanup_id, 0.1)
obj.update_tag(refresh={"DATA"}) obj.update_tag(refresh={"DATA"})
bpy.context.view_layer.update() bpy.context.view_layer.update()
assert len(_evaluated_positions(obj)) == 9 assert len(_evaluated_positions(obj)) == 9
@@ -930,8 +908,7 @@ def test_depth_balloon_uses_metric_plane_inputs_without_controller():
source.empty_display_size = 2.0 source.empty_display_size = 2.0
bpy.context.collection.objects.link(source) bpy.context.collection.objects.link(source)
crop_bounds = (50, 20, 90, 80) crop_bounds = (50, 20, 90, 80)
crop = ((0, 0), (40, 0), (40, 60), (0, 60)) local_bounds = cutout_geometry.crop_plane_bounds(
local_bounds = mesh._crop_plane_bounds(
source, source,
crop_bounds, crop_bounds,
(100, 100), (100, 100),
@@ -948,22 +925,21 @@ def test_depth_balloon_uses_metric_plane_inputs_without_controller():
((50.0, 0.0, 20.0), (0.0, 50.0, 30.0), (0.0, 0.0, 1.0)), ((50.0, 0.0, 20.0), (0.0, 50.0, 30.0), (0.0, 0.0, 1.0)),
dtype=np.float32, dtype=np.float32,
) )
field = ImageGeometry( valid = np.ones((61, 41), dtype=bool)
points=points, z_depth = np.repeat(points[..., 2, None], 3, axis=2)
intrinsics=intrinsics, depth_image = _depth_image(z_depth, valid)
valid=np.ones((61, 41), dtype=bool), depth_metadata = _depth_metadata(intrinsics, (41, 61), 2.0)
calibration_valid=np.ones((61, 41), dtype=bool),
image_size=(41, 61),
reference_depth=2.0,
region_transparent_fraction=0.05,
)
region_values = np.ones((60, 40), dtype=np.float32)
region_values[:, :2] = 0.0
result = _create_cutout_shape( result = _create_cutout_shape(
source, source,
"DEPTH_BALLOON", "DEPTH_BALLOON",
8, 8,
field=field, depth_image=depth_image,
depth_metadata=depth_metadata,
bounds=crop_bounds, bounds=crop_bounds,
region_values=region_values,
) )
assert all(abs(vertex.co.x) < 1e-8 for vertex in result.data.vertices) assert all(abs(vertex.co.x) < 1e-8 for vertex in result.data.vertices)
@@ -976,7 +952,7 @@ def test_depth_balloon_uses_metric_plane_inputs_without_controller():
result.name, result.name,
} }
modifier = result.modifiers[0] modifier = result.modifiers[0]
depth_image_id = mesh.modifier_input_identifier( depth_image_id = material.modifier_input_identifier(
modifier.node_group, modifier.node_group,
"Depth Image", "Depth Image",
) )
@@ -996,39 +972,39 @@ def test_depth_balloon_uses_metric_plane_inputs_without_controller():
atol=1e-5, atol=1e-5,
) )
depth_reference_id = mesh.modifier_input_identifier( depth_reference_id = material.modifier_input_identifier(
modifier.node_group, modifier.node_group,
"Base Plane Depth", "Base Plane Depth",
) )
assert not np.isclose(modifier[depth_reference_id], 8.0) assert not np.isclose(modifier[depth_reference_id], 8.0)
depth_direction_id = mesh.modifier_input_identifier( depth_direction_id = material.modifier_input_identifier(
modifier.node_group, modifier.node_group,
"Depth Direction", "Depth Direction",
) )
direction = np.asarray(modifier[depth_direction_id], dtype=np.float64) direction = np.asarray(modifier[depth_direction_id], dtype=np.float64)
assert not np.allclose(direction, (0.0, 0.0, 1.0)) assert not np.allclose(direction, (0.0, 0.0, 1.0))
uniform_scale_id = mesh.modifier_input_identifier( uniform_scale_id = material.modifier_input_identifier(
modifier.node_group, modifier.node_group,
"Uniform Scale", "Uniform Scale",
) )
assert modifier[uniform_scale_id] > 0.0 assert modifier[uniform_scale_id] > 0.0
thickness_id = mesh.modifier_input_identifier( thickness_id = material.modifier_input_identifier(
modifier.node_group, modifier.node_group,
"Thickness", "Thickness",
) )
smooth_id = mesh.modifier_input_identifier( smooth_id = material.modifier_input_identifier(
modifier.node_group, modifier.node_group,
"Smooth", "Smooth",
) )
assert modifier[smooth_id] == 1.0 assert modifier[smooth_id] == 1.0
mesh.set_modifier_input(modifier, smooth_id, 0.0) material.set_modifier_input(modifier, smooth_id, 0.0)
result.update_tag(refresh={"DATA"}) result.update_tag(refresh={"DATA"})
bpy.context.view_layer.update() bpy.context.view_layer.update()
positions = _evaluated_positions(result) positions = _evaluated_positions(result)
center_distance = np.linalg.norm(positions[:, 1:] - expected_center, axis=1) center_distance = np.linalg.norm(positions[:, 1:] - expected_center, axis=1)
nearest_center = center_distance <= center_distance.min() + 1e-6 nearest_center = center_distance <= center_distance.min() + 1e-6
depth_at_thickness_one = positions[nearest_center, 0].min() depth_at_thickness_one = positions[nearest_center, 0].min()
mesh.set_modifier_input(modifier, thickness_id, 2.0) material.set_modifier_input(modifier, thickness_id, 2.0)
result.update_tag(refresh={"DATA"}) result.update_tag(refresh={"DATA"})
bpy.context.view_layer.update() bpy.context.view_layer.update()
positions = _evaluated_positions(result) positions = _evaluated_positions(result)
@@ -1040,12 +1016,12 @@ def test_depth_balloon_uses_metric_plane_inputs_without_controller():
depth_at_thickness_two, depth_at_thickness_two,
atol=1e-5, atol=1e-5,
) )
mesh.set_modifier_input(modifier, thickness_id, 1.0) material.set_modifier_input(modifier, thickness_id, 1.0)
result.update_tag(refresh={"DATA"}) result.update_tag(refresh={"DATA"})
bpy.context.view_layer.update() bpy.context.view_layer.update()
before_positions = _evaluated_positions(result) before_positions = _evaluated_positions(result)
mesh.set_modifier_input( material.set_modifier_input(
modifier, modifier,
depth_reference_id, depth_reference_id,
modifier[depth_reference_id] + 0.2, modifier[depth_reference_id] + 0.2,
@@ -1059,31 +1035,31 @@ def test_depth_balloon_uses_metric_plane_inputs_without_controller():
atol=1e-5, atol=1e-5,
) )
depth_scale_id = mesh.modifier_input_identifier( depth_scale_id = material.modifier_input_identifier(
modifier.node_group, modifier.node_group,
"Depth Scale", "Depth Scale",
) )
mesh.set_modifier_input(modifier, depth_scale_id, 0.0) material.set_modifier_input(modifier, depth_scale_id, 0.0)
result.update_tag(refresh={"DATA"}) result.update_tag(refresh={"DATA"})
bpy.context.view_layer.update() bpy.context.view_layer.update()
zero_scale_before = _evaluated_positions(result) zero_scale_before = _evaluated_positions(result)
mesh.set_modifier_input( material.set_modifier_input(
modifier, modifier,
depth_reference_id, depth_reference_id,
modifier[depth_reference_id] + 0.15, modifier[depth_reference_id] + 0.15,
) )
mesh.set_modifier_input(modifier, depth_direction_id, (0.2, 0.1, 1.0)) material.set_modifier_input(modifier, depth_direction_id, (0.2, 0.1, 1.0))
result.update_tag(refresh={"DATA"}) result.update_tag(refresh={"DATA"})
bpy.context.view_layer.update() bpy.context.view_layer.update()
zero_scale_after = _evaluated_positions(result) zero_scale_after = _evaluated_positions(result)
assert np.allclose(zero_scale_after, zero_scale_before, atol=1e-5) assert np.allclose(zero_scale_after, zero_scale_before, atol=1e-5)
mesh.set_modifier_input(modifier, depth_scale_id, 1.0) material.set_modifier_input(modifier, depth_scale_id, 1.0)
double_sided_id = mesh.modifier_input_identifier( double_sided_id = material.modifier_input_identifier(
modifier.node_group, modifier.node_group,
"Double Sided", "Double Sided",
) )
mesh.set_modifier_input(modifier, double_sided_id, True) material.set_modifier_input(modifier, double_sided_id, True)
result.update_tag(refresh={"DATA"}) result.update_tag(refresh={"DATA"})
bpy.context.view_layer.update() bpy.context.view_layer.update()
mirrored = _evaluated_positions(result) mirrored = _evaluated_positions(result)
@@ -1115,6 +1091,7 @@ def test_loaded_normal_image_uses_normal_texture_name(tmp_path):
assert image.colorspace_settings.name == "Non-Color" assert image.colorspace_settings.name == "Non-Color"
assert image.packed_file is not None assert image.packed_file is not None
def test_converted_color_images_are_new_and_use_blender_duplicate_suffix(): def test_converted_color_images_are_new_and_use_blender_duplicate_suffix():
bpy.ops.wm.read_factory_settings(use_empty=True) bpy.ops.wm.read_factory_settings(use_empty=True)
image = bpy.data.images.new("Poster", width=16, height=16) image = bpy.data.images.new("Poster", width=16, height=16)
@@ -1123,8 +1100,8 @@ def test_converted_color_images_are_new_and_use_blender_duplicate_suffix():
source.data = image source.data = image
bpy.context.collection.objects.link(source) bpy.context.collection.objects.link(source)
first = plane_build.copy_color_image(source) first = plane_object.create_color_image(source)
second = plane_build.copy_color_image(source) second = plane_object.create_color_image(source)
assert first is not image assert first is not image
assert second is not image assert second is not image
@@ -1151,11 +1128,13 @@ def test_local_cutout_selection_matches_uint8_tenth_alpha_boundary():
assert selection[1, 1] == 0.0 assert selection[1, 1] == 0.0
assert selection[1, 2] > 0.1 assert selection[1, 2] > 0.1
def test_local_cutout_builds_from_blender_image_pixels_without_a_job(): def test_local_cutout_builds_from_blender_image_pixels_without_a_job():
bpy.ops.wm.read_factory_settings(use_empty=True) bpy.ops.wm.read_factory_settings(use_empty=True)
image = bpy.data.images.new("Local Cutout", width=64, height=48, alpha=True) image = bpy.data.images.new("Local Cutout", width=64, height=48, alpha=True)
pixels = np.ones((48, 64, 4), dtype=np.float32) source_rgba = np.ones((48, 64, 4), dtype=np.float32)
image.pixels.foreach_set(pixels.ravel()) source_rgba[:, :, 3] = np.linspace(0.25, 1.0, 48, dtype=np.float32)[:, None]
image.pixels.foreach_set(np.flipud(source_rgba).ravel())
source = bpy.data.objects.new("Local Cutout", None) source = bpy.data.objects.new("Local Cutout", None)
source.empty_display_type = "IMAGE" source.empty_display_type = "IMAGE"
source.data = image source.data = image
@@ -1166,30 +1145,29 @@ def test_local_cutout_builds_from_blender_image_pixels_without_a_job():
points=((-8, 8), (72, 8), (72, 40), (-8, 40)), points=((-8, 8), (72, 8), (72, 40), (-8, 40)),
fill_rule="NONZERO", fill_rule="NONZERO",
) )
source_bounds = mesh.image_empty_bounds(source) source_bounds = common_image.image_empty_bounds(source)
with patch.object( with patch.object(
mesh, common_image,
"image_rgba", "image_rgba",
wraps=mesh.image_rgba, wraps=common_image.image_rgba,
) as read_rgba: ) as read_rgba:
rgba = mesh.image_rgba(image) rgba = common_image.image_rgba(image)
region_mask = rasterize_region_path( region_mask = rasterize_region_path(
tuple(image.size), tuple(image.size),
region_path, region_path,
source_alpha=rgba[:, :, 3], source_alpha=rgba[:, :, 3],
alpha_threshold=mesh.CUTOUT_ALPHA_THRESHOLD, alpha_threshold=cutout_geometry.CUTOUT_ALPHA_THRESHOLD,
antialias=True, antialias=True,
) )
with patch.object(mesh, "configured_max_ai_input_size", return_value=2048): cutout_main.create_cutout_shape(
mesh.region_to_shape( bpy.context,
bpy.context, source,
source, "SURFACE",
"SURFACE", 16,
16, region_mask,
region_mask, source_rgba=rgba,
source_rgba=rgba, )
)
read_rgba.assert_called_once_with(image) read_rgba.assert_called_once_with(image)
@@ -1199,25 +1177,28 @@ def test_local_cutout_builds_from_blender_image_pixels_without_a_job():
assert result["anyimage_mesh_shape"] == "SURFACE" assert result["anyimage_mesh_shape"] == "SURFACE"
modifier = result.modifiers[0] modifier = result.modifiers[0]
assert modifier.node_group.name == shapes.CUTOUT_NODE_GROUP_NAME assert modifier.node_group.name == shapes.CUTOUT_NODE_GROUP_NAME
assert modifier[ assert (
mesh.modifier_input_identifier(modifier.node_group, "Shape") modifier[material.modifier_input_identifier(modifier.node_group, "Shape")]
] == shapes.CUTOUT_SHAPE_INDICES["SURFACE"] == shapes.CUTOUT_SHAPE_INDICES["SURFACE"]
expected_cleanup = mesh._cutout_cleanup_value( )
mesh._cutout_point_spacing(source, 16), expected_cleanup = cutout_geometry.cutout_cleanup_value(
mesh._world_plane_metric(source.matrix_world)[0], cutout_geometry.cutout_point_spacing(source, 16),
cutout_geometry.world_plane_metric(source.matrix_world)[0],
) )
assert np.isclose( assert np.isclose(
modifier[mesh.modifier_input_identifier(modifier.node_group, "Cleanup")], modifier[material.modifier_input_identifier(modifier.node_group, "Cleanup")],
expected_cleanup, expected_cleanup,
) )
assert modifier[ assert (
mesh.modifier_input_identifier(modifier.node_group, "Smooth") modifier[material.modifier_input_identifier(modifier.node_group, "Smooth")]
] == 1.0 == 1.0
assert modifier[ )
mesh.modifier_input_identifier( assert (
modifier.node_group, "Smooth Iterations" modifier[
) material.modifier_input_identifier(modifier.node_group, "Smooth Iterations")
] == 0 ]
== 0
)
assert len(result.data.vertices) > 4 assert len(result.data.vertices) > 4
assert len(result.data.polygons) > 1 assert len(result.data.polygons) > 1
assert all(polygon.use_smooth for polygon in result.data.polygons) assert all(polygon.use_smooth for polygon in result.data.polygons)
@@ -1242,4 +1223,8 @@ def test_local_cutout_builds_from_blender_image_pixels_without_a_job():
copied = np.empty(34 * 64 * 4, dtype=np.float32) copied = np.empty(34 * 64 * 4, dtype=np.float32)
color_texture.image.pixels.foreach_get(copied) color_texture.image.pixels.foreach_get(copied)
copied = copied.reshape((34, 64, 4)) copied = copied.reshape((34, 64, 4))
assert np.allclose(copied, 1.0) assert np.allclose(copied[:, :, :3], 1.0)
left, top, right, bottom = region_mask.bounds
assert np.allclose(
copied[:, :, 3], np.flipud(rgba[top:bottom, left:right, 3])
)

View File

@@ -6,7 +6,6 @@ from pathlib import Path
from unittest.mock import patch from unittest.mock import patch
from zipfile import ZipFile from zipfile import ZipFile
PROJECT_ROOT = Path(__file__).resolve().parents[1] PROJECT_ROOT = Path(__file__).resolve().parents[1]
@@ -40,24 +39,17 @@ class PackTest(unittest.TestCase):
pack.DIST_DIR = original_dist pack.DIST_DIR = original_dist
version = tomllib.loads( version = tomllib.loads(
(pack.SOURCE_DIR / "blender_manifest.toml").read_text( (pack.SOURCE_DIR / "blender_manifest.toml").read_text(encoding="utf-8")
encoding="utf-8"
)
)["version"] )["version"]
self.assertEqual( self.assertEqual(
{path.name for path in output_paths}, {path.name for path in output_paths},
{ {f"AnyImage.v{version}.{platform}.zip" for platform in pack.PLATFORMS},
f"AnyImage.v{version}.{platform}.zip"
for platform in pack.PLATFORMS
},
) )
for output_path in output_paths: for output_path in output_paths:
platform = output_path.stem.split(".")[-1] platform = output_path.stem.split(".")[-1]
with ZipFile(output_path) as archive: with ZipFile(output_path) as archive:
names = set(archive.namelist()) names = set(archive.namelist())
manifest = archive.read("blender_manifest.toml").decode( manifest = archive.read("blender_manifest.toml").decode("utf-8")
"utf-8"
)
self.assertIn("__init__.py", names) self.assertIn("__init__.py", names)
self.assertIn("operators/clipboard_image/__init__.py", names) self.assertIn("operators/clipboard_image/__init__.py", names)
@@ -72,18 +64,14 @@ class PackTest(unittest.TestCase):
"operators/common/image_interaction.py", "operators/common/image_interaction.py",
"operators/common/image_region.py", "operators/common/image_region.py",
"operators/common/mesh_result.py", "operators/common/mesh_result.py",
"operators/convert_to_plane/normal_map.py",
"operators/convert_to_plane/resolution.py",
"operators/cutout_tool/__init__.py", "operators/cutout_tool/__init__.py",
"operators/cutout_tool/select_region.py", "operators/cutout_tool/main.py",
"operators/cutout_tool/region_to_shape.py", "operators/cutout_tool/interaction.py",
"operators/cutout_tool/geometry/__init__.py", "operators/cutout_tool/geometry.py",
"operators/cutout_tool/geometry/balloon.py", "operators/cutout_tool/object.py",
"operators/cutout_tool/geometry/depth_calibration.py", "operators/cutout_tool/shape.py",
"operators/cutout_tool/geometry/region_contours.py",
"operators/cutout_tool/geometry/triangulation.py",
"operators/convert_to_plane/__init__.py", "operators/convert_to_plane/__init__.py",
"operators/convert_to_plane/build.py", "operators/convert_to_plane/object.py",
"operators/convert_to_plane/conversion.py", "operators/convert_to_plane/conversion.py",
"operators/convert_to_plane/mesh_plane.py", "operators/convert_to_plane/mesh_plane.py",
"operators/convert_to_plane/nodes.py", "operators/convert_to_plane/nodes.py",
@@ -109,15 +97,20 @@ class PackTest(unittest.TestCase):
names, names,
) )
self.assertIn("server/media/input.py", names) self.assertIn("server/media/input.py", names)
self.assertIn("operators/common/depth_data.py", names)
self.assertIn("server/geometry/projection.py", names) self.assertIn("server/geometry/projection.py", names)
self.assertIn("server/models/geometry.py", names) self.assertIn("server/models/geometry.py", names)
self.assertIn("server/jobs/remove_background.py", names) self.assertIn("server/jobs/remove_background.py", names)
self.assertIn("server/jobs/refine_image_mask.py", names) self.assertIn("server/jobs/refine_image_region.py", names)
self.assertIn("server/region/generate_selection.py", names) self.assertIn("server/region/refine_region.py", names)
self.assertIn("server/jobs/generate_region_data.py", names) self.assertIn("server/jobs/cutout.py", names)
self.assertIn("server/jobs/generate_depth_plane.py", names) self.assertIn("server/jobs/depth_plane.py", names)
self.assertIn("server/geometry/generate_depth.py", names) self.assertIn("server/geometry/depth_texture.py", names)
self.assertIn("server/geometry/prediction_artifacts.py", names)
self.assertIn("server/geometry/normal_texture.py", names) self.assertIn("server/geometry/normal_texture.py", names)
self.assertNotIn(
"server/geometry/generate_geometry_and_normal.py", names
)
self.assertNotIn("operators/common/image_selection.py", names) self.assertNotIn("operators/common/image_selection.py", names)
self.assertNotIn("server/selection/cutout_images.py", names) self.assertNotIn("server/selection/cutout_images.py", names)
self.assertNotIn("server/selection/polygon_mask.py", names) self.assertNotIn("server/selection/polygon_mask.py", names)
@@ -139,21 +132,13 @@ class PackTest(unittest.TestCase):
"wheels/blendjob-0.1.14-py3-none-any.whl", "wheels/blendjob-0.1.14-py3-none-any.whl",
names, names,
) )
self.assertFalse( self.assertFalse(any(name.startswith("blendjob/") for name in names))
any(name.startswith("blendjob/") for name in names)
)
self.assertNotIn("anyimage/__init__.py", names) self.assertNotIn("anyimage/__init__.py", names)
self.assertFalse( self.assertFalse(any(name.startswith("sources/") for name in names))
any(name.startswith("sources/") for name in names) self.assertFalse(any(name.startswith("vendor/") for name in names))
)
self.assertFalse(
any(name.startswith("vendor/") for name in names)
)
self.assertIn(f'platforms = ["{platform}"]', manifest) self.assertIn(f'platforms = ["{platform}"]', manifest)
self.assertNotIn( self.assertNotIn(
"windows-x64" "windows-x64" if platform != "windows-x64" else "macos-arm64",
if platform != "windows-x64"
else "macos-arm64",
manifest.split("platforms =", 1)[-1], manifest.split("platforms =", 1)[-1],
) )
@@ -168,9 +153,7 @@ class PackTest(unittest.TestCase):
def fake_run(command, check): def fake_run(command, check):
self.assertTrue(check) self.assertTrue(check)
(destination / "blendjob-0.1.14-py3-none-any.whl").write_bytes( (destination / "blendjob-0.1.14-py3-none-any.whl").write_bytes(b"wheel")
b"wheel"
)
with patch.object(pack.subprocess, "run", side_effect=fake_run) as run: with patch.object(pack.subprocess, "run", side_effect=fake_run) as run:
wheels = pack.download_wheels("linux-x64", destination) wheels = pack.download_wheels("linux-x64", destination)
@@ -185,6 +168,7 @@ class PackTest(unittest.TestCase):
[wheel.name for wheel in wheels], [wheel.name for wheel in wheels],
["blendjob-0.1.14-py3-none-any.whl"], ["blendjob-0.1.14-py3-none-any.whl"],
) )
def test_rejects_unknown_platform(self): def test_rejects_unknown_platform(self):
pack = load_pack() pack = load_pack()
with self.assertRaisesRegex(ValueError, "Unsupported platform"): with self.assertRaisesRegex(ValueError, "Unsupported platform"):
@@ -193,9 +177,7 @@ class PackTest(unittest.TestCase):
def test_platforms_match_manifest(self): def test_platforms_match_manifest(self):
pack = load_pack() pack = load_pack()
manifest = tomllib.loads( manifest = tomllib.loads(
(pack.SOURCE_DIR / "blender_manifest.toml").read_text( (pack.SOURCE_DIR / "blender_manifest.toml").read_text(encoding="utf-8")
encoding="utf-8"
)
) )
self.assertEqual(set(pack.PLATFORMS), set(manifest["platforms"])) self.assertEqual(set(pack.PLATFORMS), set(manifest["platforms"]))
self.assertEqual( self.assertEqual(
@@ -203,5 +185,6 @@ class PackTest(unittest.TestCase):
{"windows-x64", "macos-arm64", "linux-x64"}, {"windows-x64", "macos-arm64", "linux-x64"},
) )
if __name__ == "__main__": if __name__ == "__main__":
unittest.main() unittest.main()

View File

@@ -1,4 +1,5 @@
import importlib.util import importlib.util
import json
import sys import sys
import tempfile import tempfile
import threading import threading
@@ -41,8 +42,8 @@ class ServerTest(unittest.TestCase):
@classmethod @classmethod
def setUpClass(cls): def setUpClass(cls):
cls.server_module = load_server_module() cls.server_module = load_server_module()
importlib.import_module("server.geometry.generate_depth") importlib.import_module("server.geometry.prediction_artifacts")
importlib.import_module("server.jobs.generate_region_data") importlib.import_module("server.jobs.cutout")
from blendjob import JobServer from blendjob import JobServer
from server.model_manager import ModelManager, ModelSessionCache from server.model_manager import ModelManager, ModelSessionCache
@@ -258,11 +259,11 @@ class ServerTest(unittest.TestCase):
{ {
"download-model", "download-model",
"download-required-models", "download-required-models",
"refine-image-mask", "refine-image-region",
"remove-background", "remove-background",
"upscale-image", "upscale-image",
"generate-depth-plane", "generate-depth-plane-geometry",
"generate-region-data", "generate-cutout-artifacts",
"debug-da3", "debug-da3",
"debug-moge2", "debug-moge2",
"debug-ben2", "debug-ben2",
@@ -386,23 +387,12 @@ class ServerTest(unittest.TestCase):
) )
self.assertEqual(events, ["clear", "close"]) self.assertEqual(events, ["clear", "close"])
def test_region_transparent_fraction_uses_visible_alpha_threshold(self): def test_depth_artifacts_store_texture_and_compact_metadata(self):
import numpy as np
from PIL import Image
depth = sys.modules["server.geometry.generate_depth"]
output_path = self.storage_root / "region.png"
pixels = np.full((10, 10, 4), 255, dtype=np.uint8)
pixels.reshape(-1, 4)[:5, 3] = 25
Image.fromarray(pixels, mode="RGBA").save(output_path)
self.assertEqual(depth.transparent_fraction(output_path), 0.05)
def test_image_geometry_keeps_model_and_region_validity_separate(self):
import numpy as np import numpy as np
depth = sys.modules["server.geometry.generate_depth"] depth = importlib.import_module("server.geometry.depth_texture")
output_path = self.storage_root / "image_geometry.npz" texture_path = self.storage_root / "z-depth.exr"
metadata_path = self.storage_root / "depth.json"
valid = np.asarray(((True, True, False), (True, True, True))) valid = np.asarray(((True, True, False), (True, True, True)))
calibration = np.asarray(((False, True, False), (False, False, True))) calibration = np.asarray(((False, True, False), (False, False, True)))
frame = SimpleNamespace( frame = SimpleNamespace(
@@ -413,19 +403,60 @@ class ServerTest(unittest.TestCase):
model_mask=None, model_mask=None,
) )
depth._write_image_geometry_archive(frame, output_path, calibration, 0.05) depth.write_z_depth_texture(frame.depth, frame.valid_mask, texture_path)
depth.write_depth_metadata(frame, calibration, metadata_path)
with np.load(output_path) as archive: self.assertTrue(texture_path.is_file())
np.testing.assert_array_equal(archive["valid"], valid) self.assertEqual(texture_path.read_bytes()[:4], b"v/1\x01")
np.testing.assert_array_equal(archive["calibration_valid"], calibration) metadata = json.loads(metadata_path.read_text(encoding="utf-8"))
self.assertAlmostEqual( self.assertEqual(metadata["image_size"], [3, 2])
float(archive["region_transparent_fraction"]), self.assertEqual(metadata["intrinsics"], np.eye(3).tolist())
0.05, self.assertAlmostEqual(metadata["reference_depth"], 4.0)
places=6,
)
def test_generate_region_data_only_returns_requested_outputs(self): def test_object_normal_texture_matches_depth_surface_axes(self):
job = sys.modules["server.jobs.generate_region_data"] import numpy as np
from PIL import Image
normal_texture = importlib.import_module("server.geometry.normal_texture")
output_path = self.storage_root / "object-normal.png"
camera_normals = np.asarray(
[[
(0.0, 0.0, -1.0),
(1.0, 0.0, 0.0),
(0.0, 1.0, 0.0),
(0.0, 0.0, -1.0),
]],
dtype=np.float32,
)
valid = np.asarray([[True, True, True, False]], dtype=bool)
normal_texture.write_camera_normal_texture(
camera_normals,
valid,
(0, 0),
(4, 1),
(4, 1),
output_path,
normal_mode="OBJECT",
)
with Image.open(output_path) as opened:
encoded = np.asarray(opened, dtype=np.uint8)
np.testing.assert_array_equal(
encoded,
np.asarray(
[[
(0, 127, 127),
(127, 0, 127),
(127, 127, 0),
(0, 128, 128),
]],
dtype=np.uint8,
),
)
def test_generate_cutout_artifacts_only_returns_requested_outputs(self):
job = sys.modules["server.jobs.cutout"]
input_path = self.storage_root / "input.png" input_path = self.storage_root / "input.png"
input_path.write_bytes(b"region") input_path.write_bytes(b"region")
context = SimpleNamespace( context = SimpleNamespace(
@@ -433,40 +464,49 @@ class ServerTest(unittest.TestCase):
progress=lambda *_args: None, progress=lambda *_args: None,
check_cancelled=lambda: None, check_cancelled=lambda: None,
) )
archive = context.directory / "image_geometry.npz" vector_depth = context.directory / "vector-depth.exr"
normal = context.directory / "normal.png" metadata = context.directory / "depth.json"
normal = context.directory / "object-normal.png"
with patch.object( with patch.object(
job, job,
"generate_depth", "generate_moge2_artifacts",
return_value=(archive, normal), return_value={
"vector_depth": vector_depth,
"depth_metadata": metadata,
"object_normal": normal,
},
) as generate: ) as generate:
result = job.run( result = job.run(
context, context,
{ {
"input": str(input_path), "input": str(input_path),
"output_depth": True, "depth_mode": "VECTOR",
"normal_mode": "OBJECT", "normal_mode": "OBJECT",
}, },
) )
self.assertEqual( self.assertEqual(
result, result,
{"image_geometry": "image_geometry.npz", "normal": "normal.png"}, {
"vector_depth": "vector-depth.exr",
"depth_metadata": "depth.json",
"object_normal": "object-normal.png",
},
) )
generate.assert_called_once_with( generate.assert_called_once_with(
context, context,
{ {
"input": str(input_path), "input": str(input_path),
"output_depth": True, "depth_mode": "VECTOR",
"normal_mode": "OBJECT", "normal_mode": "OBJECT",
}, },
input_path, input_path,
output_depth=True, depth_mode="VECTOR",
normal_mode="OBJECT", normal_mode="OBJECT",
) )
def test_generate_region_data_refines_an_already_rasterized_region(self): def test_generate_cutout_artifacts_refine_an_already_rasterized_region(self):
job = sys.modules["server.jobs.generate_region_data"] job = sys.modules["server.jobs.cutout"]
input_path = self.storage_root / "input.png" input_path = self.storage_root / "input.png"
input_path.write_bytes(b"region") input_path.write_bytes(b"region")
directory = self.storage_root / "job" directory = self.storage_root / "job"
@@ -480,7 +520,7 @@ class ServerTest(unittest.TestCase):
output.write_bytes(b"refined") output.write_bytes(b"refined")
return output, (2, 3, 8, 9) return output, (2, 3, 8, 9)
with patch.object(job, "generate_selection", side_effect=generate): with patch.object(job, "refine_region", side_effect=generate):
result = job.run( result = job.run(
context, context,
{ {
@@ -493,7 +533,7 @@ class ServerTest(unittest.TestCase):
self.assertEqual( self.assertEqual(
result, result,
{"region": "region.png", "region_bounds": "region_bounds.json"}, {"region_image": "region.png", "region_bounds": "region_bounds.json"},
) )
self.assertEqual( self.assertEqual(
(directory / "region_bounds.json").read_text(encoding="utf-8"), (directory / "region_bounds.json").read_text(encoding="utf-8"),

View File

@@ -1,12 +1,10 @@
"""Build the four Cutout Geometry Nodes groups stored in O_AnyImage.blend.""" """Build the four Cutout Geometry Nodes groups stored in O_AnyImage.blend."""
import math
import bpy import bpy
from node_graph import expand_smooth from node_graph import expand_smooth
from anyimage.operators.cutout_tool.shape_types import ( from anyimage.operators.cutout_tool.shape import (
BALLOON_ATTRIBUTE_NAME, BALLOON_ATTRIBUTE_NAME,
CUTOUT_NODE_GROUP_NAME, CUTOUT_NODE_GROUP_NAME,
CUTOUT_NODE_GROUP_NAMES, CUTOUT_NODE_GROUP_NAMES,
@@ -51,9 +49,7 @@ def _localize_group_inputs(group, controls):
for link in ordered: for link in ordered:
if ( if (
not clusters not clusters
or link.to_node.location.x or link.to_node.location.x - clusters[-1][-1].to_node.location.x > 900.0
- clusters[-1][-1].to_node.location.x
> 900.0
): ):
clusters.append([]) clusters.append([])
clusters[-1].append(link) clusters[-1].append(link)
@@ -356,9 +352,7 @@ def _set_segmented_layer_x(
flip_side.location = (x + 2850.0, y + 350.0) flip_side.location = (x + 2850.0, y + 350.0)
links.new(positioned_side, flip_side.inputs["Mesh"]) links.new(positioned_side, flip_side.inputs["Mesh"])
back = _offset_geometry_x( back = _offset_geometry_x(group, geometry, back_x, (x + 2450.0, y - 400.0))
group, geometry, back_x, (x + 2450.0, y - 400.0)
)
flip_back = nodes.new("GeometryNodeFlipFaces") flip_back = nodes.new("GeometryNodeFlipFaces")
flip_back.location = (x + 2850.0, y - 400.0) flip_back.location = (x + 2850.0, y - 400.0)
links.new(back, flip_back.inputs["Mesh"]) links.new(back, flip_back.inputs["Mesh"])
@@ -503,17 +497,13 @@ def _build_cutout(group, geometry, thickness):
thickness, thickness,
(250.0, 300.0), (250.0, 300.0),
) )
return _remove_zero_thickness( return _remove_zero_thickness(group, geometry, shaped, thickness, (3750.0, 300.0))
group, geometry, shaped, thickness, (3750.0, 300.0)
)
def _build_balloon(group, geometry, thickness): def _build_balloon(group, geometry, thickness):
nodes = group.nodes nodes = group.nodes
links = group.links links = group.links
profile, _profile_factor = _profile_field( profile, _profile_factor = _profile_field(group, geometry, (-200.0, 0.0))
group, geometry, (-200.0, 0.0)
)
front = _math(nodes, "MULTIPLY", (2400.0, 500.0)) front = _math(nodes, "MULTIPLY", (2400.0, 500.0))
front.inputs[1].default_value = -1.0 front.inputs[1].default_value = -1.0
links.new(profile, front.inputs[0]) links.new(profile, front.inputs[0])
@@ -532,9 +522,7 @@ def _build_balloon(group, geometry, thickness):
back.outputs[0], back.outputs[0],
(2850.0, 650.0), (2850.0, 650.0),
) )
return _remove_zero_thickness( return _remove_zero_thickness(group, geometry, shaped, thickness, (4400.0, 650.0))
group, geometry, shaped, thickness, (4400.0, 650.0)
)
def _build_depth_balloon( def _build_depth_balloon(
@@ -550,9 +538,7 @@ def _build_depth_balloon(
): ):
nodes = group.nodes nodes = group.nodes
links = group.links links = group.links
profile, profile_factor = _profile_field( profile, profile_factor = _profile_field(group, geometry, (-200.0, 250.0))
group, geometry, (-200.0, 250.0)
)
camera_position = _position_field(group, depth_image, (200.0, -250.0)) camera_position = _position_field(group, depth_image, (200.0, -250.0))
separate_camera = nodes.new("ShaderNodeSeparateXYZ") separate_camera = nodes.new("ShaderNodeSeparateXYZ")
separate_camera.location = (650.0, -250.0) separate_camera.location = (650.0, -250.0)
@@ -615,9 +601,7 @@ def _build_depth_balloon(
front_only = _set_geometry_x( front_only = _set_geometry_x(
group, geometry, front_mix.outputs["Result"], (3500.0, 100.0) group, geometry, front_mix.outputs["Result"], (3500.0, 100.0)
) )
return _remove_zero_thickness( return _remove_zero_thickness(group, front_only, shaped, thickness, (4850.0, 600.0))
group, front_only, shaped, thickness, (4850.0, 600.0)
)
def _build_depth_surface( def _build_depth_surface(
@@ -709,9 +693,7 @@ def build_node_group(shape):
"the source sheet" "the source sheet"
) )
else: else:
thickness = _float_input( thickness = _float_input(group, "Thickness", 0.0, 0.0, None, subtype="DISTANCE")
group, "Thickness", 0.0, 0.0, None, subtype="DISTANCE"
)
thickness.description = "Final thickness in scene distance" thickness.description = "Final thickness in scene distance"
depth_image = None depth_image = None
@@ -766,17 +748,15 @@ def build_node_group(shape):
if shape in {"DEPTH_BALLOON", "DEPTH_SURFACE"}: if shape in {"DEPTH_BALLOON", "DEPTH_SURFACE"}:
data = group.interface.new_panel(name="Data", default_closed=True) data = group.interface.new_panel(name="Data", default_closed=True)
uniform_scale = _float_input( uniform_scale = _float_input(group, "Uniform Scale", 1.0, 0.0, parent=data)
group, "Uniform Scale", 1.0, 0.0, parent=data
)
uniform_scale.description = ( uniform_scale.description = (
"Internal uniform conversion from MoGe meters to local distance" "Internal uniform conversion from MoGe meters to local distance"
) )
uniform_scale.hide_in_modifier = True uniform_scale.hide_in_modifier = True
depth_image = _socket( depth_image = _socket(group, "Depth Image", "INPUT", "NodeSocketImage", data)
group, "Depth Image", "INPUT", "NodeSocketImage", data depth_image.description = (
f"Internally generated {CUTOUT_SHAPE_LABELS[shape]} data"
) )
depth_image.description = f"Internally generated {CUTOUT_SHAPE_LABELS[shape]} data"
depth_image.hide_in_modifier = True depth_image.hide_in_modifier = True
_socket(group, "Geometry", "OUTPUT", "NodeSocketGeometry") _socket(group, "Geometry", "OUTPUT", "NodeSocketGeometry")
@@ -795,7 +775,7 @@ def build_node_group(shape):
}, },
) )
geometry = controls.outputs["Geometry"] geometry = controls.outputs["Geometry"]
if shape == "CUTOUT": if shape == "SURFACE":
geometry = _build_cutout(group, geometry, controls.outputs["Thickness"]) geometry = _build_cutout(group, geometry, controls.outputs["Thickness"])
output_location = (4050.0, 300.0) output_location = (4050.0, 300.0)
elif shape == "BALLOON": elif shape == "BALLOON":
@@ -833,7 +813,7 @@ def build_node_group(shape):
_shade_output(group, geometry, output_location) _shade_output(group, geometry, output_location)
group.color_tag = "GEOMETRY" group.color_tag = "GEOMETRY"
group.description = { group.description = {
"CUTOUT": "Symmetrically thicken a Surface sheet.", "SURFACE": "Symmetrically thicken a Surface sheet.",
"BALLOON": "Inflate a Surface sheet from its boundary profile.", "BALLOON": "Inflate a Surface sheet from its boundary profile.",
"DEPTH_BALLOON": ( "DEPTH_BALLOON": (
"Mix Balloon profile and metric image depth around a fitted plane." "Mix Balloon profile and metric image depth around a fitted plane."
@@ -866,9 +846,7 @@ def build_cutout_modifier_group(shape_groups):
"Mirror the complete Depth Balloon front surface across the local YZ plane" "Mirror the complete Depth Balloon front surface across the local YZ plane"
) )
iterations = _socket( iterations = _socket(group, "Smooth Iterations", "INPUT", "NodeSocketInt")
group, "Smooth Iterations", "INPUT", "NodeSocketInt"
)
iterations.default_value = 0 iterations.default_value = 0
iterations.min_value = 0 iterations.min_value = 0
iterations.max_value = 20 iterations.max_value = 20
@@ -882,9 +860,7 @@ def build_cutout_modifier_group(shape_groups):
None, None,
parent=options, parent=options,
) )
cleanup.description = ( cleanup.description = "Delete mesh islands below this absolute local-area threshold"
"Delete mesh islands below this absolute local-area threshold"
)
smooth = _float_input( smooth = _float_input(
group, group,
"Smooth", "Smooth",
@@ -915,18 +891,13 @@ def build_cutout_modifier_group(shape_groups):
) )
depth_direction.description = "Normal direction of the fitted metric depth plane" depth_direction.description = "Normal direction of the fitted metric depth plane"
data = group.interface.new_panel(name="Data", default_closed=True) data = group.interface.new_panel(name="Data", default_closed=True)
uniform_scale = _float_input( uniform_scale = _float_input(group, "Uniform Scale", 1.0, 0.0, parent=data)
group, "Uniform Scale", 1.0, 0.0, parent=data
)
uniform_scale.description = ( uniform_scale.description = (
"Internal uniform conversion from MoGe meters to local distance" "Internal uniform conversion from MoGe meters to local distance"
) )
uniform_scale.hide_in_modifier = True uniform_scale.hide_in_modifier = True
depth_image = _socket( depth_image = _socket(group, "Depth Image", "INPUT", "NodeSocketImage", data)
group, "Depth Image", "INPUT", "NodeSocketImage", data
)
depth_image.description = "Internally generated Cutout depth data" depth_image.description = "Internally generated Cutout depth data"
depth_image.hide_in_modifier = True depth_image.hide_in_modifier = True
_socket(group, "Geometry", "OUTPUT", "NodeSocketGeometry") _socket(group, "Geometry", "OUTPUT", "NodeSocketGeometry")
@@ -947,7 +918,7 @@ def build_cutout_modifier_group(shape_groups):
) )
branch_y = { branch_y = {
"CUTOUT": 1150.0, "SURFACE": 1150.0,
"BALLOON": 550.0, "BALLOON": 550.0,
"DEPTH_BALLOON": -50.0, "DEPTH_BALLOON": -50.0,
"DEPTH_SURFACE": -650.0, "DEPTH_SURFACE": -650.0,

View File

@@ -1,9 +1,8 @@
from pathlib import Path
import sys import sys
from pathlib import Path
import bpy import bpy
PROJECT_ROOT = Path(__file__).parents[2] PROJECT_ROOT = Path(__file__).parents[2]
sys.path.insert(0, str(PROJECT_ROOT / "src")) sys.path.insert(0, str(PROJECT_ROOT / "src"))
sys.path.insert( sys.path.insert(
@@ -14,7 +13,7 @@ sys.path.insert(
from anyimage.operators.common.mesh_result import ( from anyimage.operators.common.mesh_result import (
load_asset_node_group, load_asset_node_group,
) )
from anyimage.operators.cutout_tool.shape_types import ( from anyimage.operators.cutout_tool.shape import (
CUTOUT_NODE_GROUP_NAME, CUTOUT_NODE_GROUP_NAME,
CUTOUT_NODE_GROUP_NAMES, CUTOUT_NODE_GROUP_NAMES,
) )
@@ -34,8 +33,7 @@ def main():
shadeless = load_asset_node_group("O Shadeless") shadeless = load_asset_node_group("O Shadeless")
cutout = load_asset_node_group(CUTOUT_NODE_GROUP_NAME) cutout = load_asset_node_group(CUTOUT_NODE_GROUP_NAME)
cutout_groups = { cutout_groups = {
name: bpy.data.node_groups[name] name: bpy.data.node_groups[name] for name in CUTOUT_NODE_GROUP_NAMES.values()
for name in CUTOUT_NODE_GROUP_NAMES.values()
} }
assert mesh_plane.name == "O Mesh Plane" assert mesh_plane.name == "O Mesh Plane"

View File

@@ -1,9 +1,8 @@
from pathlib import Path
import sys import sys
from pathlib import Path
import bpy import bpy
PROJECT_ROOT = Path(__file__).parents[2] PROJECT_ROOT = Path(__file__).parents[2]
sys.path.insert(0, str(PROJECT_ROOT / "src")) sys.path.insert(0, str(PROJECT_ROOT / "src"))
sys.path.insert( sys.path.insert(
@@ -11,12 +10,12 @@ sys.path.insert(
str(next((PROJECT_ROOT / "src" / "anyimage" / "wheels").glob("blendjob-*.whl"))), str(next((PROJECT_ROOT / "src" / "anyimage" / "wheels").glob("blendjob-*.whl"))),
) )
from anyimage.operators.cutout_tool import node_assets as cutout_nodes
from anyimage.operators.cutout_tool import shape_types as shapes
from anyimage.operators.common.mesh_result import ( from anyimage.operators.common.mesh_result import (
modifier_input_identifier, modifier_input_identifier,
set_modifier_input, set_modifier_input,
) )
from anyimage.operators.cutout_tool import object as cutout_objects
from anyimage.operators.cutout_tool import shape as shapes
def evaluated_mesh(obj): def evaluated_mesh(obj):
@@ -34,11 +33,7 @@ def input_names(group):
def grid_object(name): def grid_object(name):
vertices = [ vertices = [(0.0, float(y), float(z)) for z in range(3) for y in range(3)]
(0.0, float(y), float(z))
for z in range(3)
for y in range(3)
]
faces = [] faces = []
for z in range(2): for z in range(2):
for y in range(2): for y in range(2):
@@ -56,7 +51,7 @@ def grid_object(name):
def add_modifier(obj, shape, thickness): def add_modifier(obj, shape, thickness):
group = cutout_nodes.cutout_node_group(shape) group = cutout_objects.load_cutout_node_group()
modifier = obj.modifiers.new(group.name, "NODES") modifier = obj.modifiers.new(group.name, "NODES")
modifier.node_group = group modifier.node_group = group
for name, value in ( for name, value in (
@@ -76,9 +71,9 @@ def add_modifier(obj, shape, thickness):
def main(): def main():
bpy.ops.wm.read_factory_settings(use_empty=True) bpy.ops.wm.read_factory_settings(use_empty=True)
wrapper = cutout_nodes.cutout_node_group("CUTOUT") wrapper = cutout_objects.load_cutout_node_group()
assert all( assert all(
cutout_nodes.cutout_node_group(shape) == wrapper cutout_objects.load_cutout_node_group() == wrapper
for shape in shapes.CUTOUT_SHAPES for shape in shapes.CUTOUT_SHAPES
) )
assert wrapper.name == shapes.CUTOUT_NODE_GROUP_NAME assert wrapper.name == shapes.CUTOUT_NODE_GROUP_NAME
@@ -90,8 +85,7 @@ def main():
assert all(not group.is_modifier for group in groups.values()) assert all(not group.is_modifier for group in groups.values())
menu_switch = next( menu_switch = next(
node for node in wrapper.nodes node for node in wrapper.nodes if node.bl_idname == "GeometryNodeMenuSwitch"
if node.bl_idname == "GeometryNodeMenuSwitch"
) )
assert [item.name for item in menu_switch.enum_items] == list( assert [item.name for item in menu_switch.enum_items] == list(
shapes.CUTOUT_SHAPE_LABELS.values() shapes.CUTOUT_SHAPE_LABELS.values()
@@ -131,7 +125,7 @@ def main():
"Depth Image", "Depth Image",
] ]
expected_inputs = { expected_inputs = {
"CUTOUT": ["Geometry", "Thickness"], "SURFACE": ["Geometry", "Thickness"],
"BALLOON": ["Geometry", "Thickness"], "BALLOON": ["Geometry", "Thickness"],
"DEPTH_BALLOON": [ "DEPTH_BALLOON": [
"Geometry", "Geometry",
@@ -162,9 +156,7 @@ def main():
if link.to_node.bl_idname != "GeometryNodeGroup": if link.to_node.bl_idname != "GeometryNodeGroup":
continue continue
shape = next( shape = next(
shape shape for shape, group in groups.items() if group == link.to_node.node_tree
for shape, group in groups.items()
if group == link.to_node.node_tree
) )
dependencies.setdefault(link.from_socket.name, set()).add(shape) dependencies.setdefault(link.from_socket.name, set()).add(shape)
assert dependencies == { assert dependencies == {
@@ -178,41 +170,30 @@ def main():
} }
repeat_counts = { repeat_counts = {
shape: sum( shape: sum(node.bl_idname == "GeometryNodeRepeatInput" for node in group.nodes)
node.bl_idname == "GeometryNodeRepeatInput"
for node in group.nodes
)
for shape, group in groups.items() for shape, group in groups.items()
} }
assert repeat_counts == { assert repeat_counts == {
"CUTOUT": 1, "SURFACE": 1,
"BALLOON": 0, "BALLOON": 0,
"DEPTH_BALLOON": 0, "DEPTH_BALLOON": 0,
"DEPTH_SURFACE": 1, "DEPTH_SURFACE": 1,
} }
assert sum( assert (
node.bl_idname == "GeometryNodeRepeatInput" sum(node.bl_idname == "GeometryNodeRepeatInput" for node in wrapper.nodes) == 1
for node in wrapper.nodes )
) == 1 assert (
assert sum( sum(node.bl_idname == "GeometryNodeInputMeshIsland" for node in wrapper.nodes)
node.bl_idname == "GeometryNodeInputMeshIsland" == 1
for node in wrapper.nodes )
) == 1
assert not any( assert not any(
node.bl_idname == "GeometryNodeAttributeStatistic" node.bl_idname == "GeometryNodeAttributeStatistic" for node in wrapper.nodes
for node in wrapper.nodes
) )
assert all( assert all(
not any( not any(node.bl_idname == "GeometryNodeInputMeshIsland" for node in group.nodes)
node.bl_idname == "GeometryNodeInputMeshIsland"
for node in group.nodes
)
for group in groups.values() for group in groups.values()
) )
assert any( assert any(node.bl_idname == "GeometryNodeBlurAttribute" for node in wrapper.nodes)
node.bl_idname == "GeometryNodeBlurAttribute"
for node in wrapper.nodes
)
smooth = next( smooth = next(
item item
for item in wrapper.interface.items_tree for item in wrapper.interface.items_tree
@@ -244,23 +225,19 @@ def main():
assert iterations.min_value == 0 assert iterations.min_value == 0
assert getattr(iterations.parent, "name", "") == "" assert getattr(iterations.parent, "name", "") == ""
assert all( assert all(
not any( not any(node.bl_idname == "GeometryNodeBlurAttribute" for node in group.nodes)
node.bl_idname == "GeometryNodeBlurAttribute"
for node in group.nodes
)
for group in groups.values() for group in groups.values()
) )
for group in (wrapper, *groups.values()): for group in (wrapper, *groups.values()):
assert all(not node.label for node in group.nodes) assert all(not node.label for node in group.nodes)
assert all( assert all(
link.from_node.location.x <= link.to_node.location.x link.from_node.location.x <= link.to_node.location.x for link in group.links
for link in group.links
) )
group_nodes = list(group.nodes) group_nodes = list(group.nodes)
assert len({tuple(node.location) for node in group_nodes}) == len(group_nodes) assert len({tuple(node.location) for node in group_nodes}) == len(group_nodes)
for index, node in enumerate(group_nodes): for index, node in enumerate(group_nodes):
for other in group_nodes[index + 1:]: for other in group_nodes[index + 1 :]:
assert not ( assert not (
abs(node.location.x - other.location.x) < 150.0 abs(node.location.x - other.location.x) < 150.0
and abs(node.location.y - other.location.y) < 100.0 and abs(node.location.y - other.location.y) < 100.0
@@ -277,14 +254,13 @@ def main():
if item.item_type == "SOCKET" and item.in_out == "INPUT": if item.item_type == "SOCKET" and item.in_out == "INPUT":
assert item.structure_type == "SINGLE" assert item.structure_type == "SINGLE"
surface = grid_object("Surface Validation") surface = grid_object("Surface Validation")
add_modifier(surface, "CUTOUT", 2.4) add_modifier(surface, "SURFACE", 2.4)
evaluated, result = evaluated_mesh(surface) evaluated, result = evaluated_mesh(surface)
try: try:
assert len(result.vertices) == 34 assert len(result.vertices) == 34
assert len(result.polygons) == 40 assert len(result.polygons) == 40
side_faces = [ side_faces = [
polygon for polygon in result.polygons polygon for polygon in result.polygons if abs(polygon.normal.x) < 0.1
if abs(polygon.normal.x) < 0.1
] ]
assert len(side_faces) == 24 assert len(side_faces) == 24
finally: finally:
@@ -415,14 +391,12 @@ def main():
try: try:
coordinates = [tuple(vertex.co) for vertex in result.vertices] coordinates = [tuple(vertex.co) for vertex in result.vertices]
assert all( assert all(
all(abs(actual - expected) < 1e-5 for actual, expected in zip(point, target)) all(
abs(actual - expected) < 1e-5 for actual, expected in zip(point, target)
)
for point, target in zip( for point, target in zip(
coordinates, coordinates,
( ((y * 0.25, y * 0.5, z * 0.5) for z in range(3) for y in range(3)),
(y * 0.25, y * 0.5, z * 0.5)
for z in range(3)
for y in range(3)
),
) )
) )
finally: finally: