feat: unify reference depth calculation by using median-based range factor and baseline
This commit is contained in:
@@ -39,11 +39,11 @@ Alpha 已经包含源图 Alpha 与模型有效性两个条件，因此不再需
|
||||
|
||||
备选是统一取中位数：实现同样简单，但 Depth Solid 与 Depth Symmetry 会有一半内容被钳制在深度零点，厚度归一化也随之改变。
|
||||
|
||||
### 参考深度丢弃超出米制上限的样本
|
||||
### 参考深度丢弃超出中位数倍数的样本
|
||||
|
||||
参考深度同时是 `Uniform Scale` 的锚点，`Uniform Scale` 与参考深度成反比，所以远景会把整个重建结果缩小并整体推向基准之前。风景图没有主体，中位数与上分位都落在远景上，这个效应最明显。因此统计前丢弃距离超过 `REFERENCE_DEPTH_LIMIT`（米）的样本，让基准由主要内容的距离决定；距离上限之外的内容仍保留在深度纹理里，只影响几何投影，不影响标定。
|
||||
参考深度同时是 `Uniform Scale` 的锚点，`Uniform Scale` 与参考深度成反比，所以远景会把整个重建结果缩小并整体推向基准之前。风景图没有主体，中位数与上分位都落在远景上，这个效应最明显。因此统计前丢弃距离超过 `REFERENCE_DEPTH_RANGE_FACTOR`（`1.1`）倍中位数的样本，让基准由主要内容的距离决定；被丢弃的内容仍保留在深度纹理里，只影响几何投影，不影响标定。
|
||||
|
||||
上限「选不出样本」和「没有有效像素」是同一个问题的两面——有效像素的选择多了一条距离条件——所以两者用同一条规则：没有样本剩下时，参考深度取上限本身。这样参考深度的取值域始终是 `(0, REFERENCE_DEPTH_LIMIT]`，同一张图不会因为少量近景像素出现而跳到另一个尺度，也不会因为深度不可信而失败或弹出提示。上限取固定米数而不是相对中位数的比例，因为米制深度是这个工具与 Blender 场景的既有接口，固定上限可以直接说明「主要内容在多少米以内」。
|
||||
上限取中位数的倍数而不是固定米数，因为固定米数假设「主要内容在多少米以内」，一旦主体整体在固定上限之外，基准会落到主体之前，`reference − depth` 全为负而被钳制，几何直接消失。倍数上限随内容尺度变化：中位数永远落在上限之内，所以只要存在有效像素，统计集合一定非空，也不会因为题材远近而失效。代价是风景图的上限随内容放大，远景被推后的力度比固定小上限时轻。
|
||||
|
||||
### 参考域可选，Cutout 传入选区
|
||||
|
||||
|
||||
@@ -6,7 +6,7 @@
|
||||
|
||||
- **BREAKING**：服务端不再计算参考深度，`depth.json` 只保留 `image_size` 与 `intrinsics`。
|
||||
- 在 `common/depth.py` 提供唯一的参考深度计算：以 `depth.exr` 的 Alpha 大于 `0.95`、深度有限且为正为有效域，取相机 Z 的 95 百分位。
|
||||
- 统计前丢弃距离超过 `REFERENCE_DEPTH_LIMIT`（`8` 米）的样本，避免远景把参考深度与整个重建尺度拖到远处；选择后没有样本时参考深度取该上限，不报错也不提示。
|
||||
- 统计前丢弃距离超过中位数 `REFERENCE_DEPTH_RANGE_FACTOR`（`1.1`）倍的样本，避免远景把参考深度与整个重建尺度拖到远处；完全没有有效像素时参考深度取 `REFERENCE_DEPTH_BASELINE`（`8` 米），不报错也不提示。
|
||||
- 深度方向拟合没有可用样本时返回无倾斜方向，Relief Plane 与 Depth Symmetry 的 `Direction` 一律回退为 `(0, 0, 1)`，不再报错，也不在调用点各自兜底。
|
||||
- Depth Plane、Relief Plane、Depth Solid、Depth Symmetry 共用同一实现与同一统计量，不再区分「服务端基准」与「标定基准」；Cutout 把选区作为可选参考域传入，Plane 转换使用整幅深度图。
|
||||
- **BREAKING**：参考深度从内容中位数改为上分位，基准平面移动到场景后端，Depth Plane 与 Relief Plane 的几何结果随之变化。
|
||||
|
||||
@@ -26,20 +26,20 @@
|
||||
|
||||
#### Scenario: Exclude everything
|
||||
- **WHEN** 没有任何像素满足有效条件
|
||||
- **THEN** 参考深度取 `REFERENCE_DEPTH_LIMIT`
|
||||
- **THEN** 参考深度取 `REFERENCE_DEPTH_BASELINE`
|
||||
|
||||
### Requirement: Reference depth uses the upper depth percentile
|
||||
参考深度 SHALL 取有效像素相机 Z 的 95 百分位，并以模型单位返回。所有深度几何 SHALL 使用同一判据，MUST NOT 按目标区分统计量。
|
||||
|
||||
#### Scenario: Distant background is dropped before the percentile
|
||||
- **WHEN** 部分有效像素的距离超过 `REFERENCE_DEPTH_LIMIT` 米
|
||||
- **WHEN** 部分有效像素的距离超过有效像素中位数的 `REFERENCE_DEPTH_RANGE_FACTOR` 倍
|
||||
- **THEN** 这些像素不参与百分位统计
|
||||
- **AND** 参考深度只由上限之内的像素决定
|
||||
- **AND** 参考深度只由该范围内的像素决定
|
||||
|
||||
#### Scenario: No sample survives the selection
|
||||
- **WHEN** 有效像素的距离全部超过 `REFERENCE_DEPTH_LIMIT` 米
|
||||
- **THEN** 参考深度取 `REFERENCE_DEPTH_LIMIT`
|
||||
- **AND** 系统不报告错误，也不提示基准来源
|
||||
#### Scenario: Keep a distant subject
|
||||
- **WHEN** 全部有效像素都远于 `REFERENCE_DEPTH_BASELINE`，但仍在自身中位数的 `REFERENCE_DEPTH_RANGE_FACTOR` 倍以内
|
||||
- **THEN** 参考深度按这些像素的 95 百分位计算
|
||||
- **AND** 主体不会因为基准落在自身之前而被钳制
|
||||
|
||||
#### Scenario: Depth distribution is skewed
|
||||
- **WHEN** 有效深度在前后景之间分布不均
|
||||
|
||||
@@ -4,7 +4,7 @@
|
||||
- [x] 1.2 Drop `reference_depth` from `load_depth_metadata()` and change `depth_uniform_scale()` to take the computed reference depth instead of reading it from metadata.
|
||||
- [x] 1.3 Move the base-shape direction fit into `common/depth.py` as `fit_symmetry_depth_direction()` beside `fit_depth_direction()`, then delete `src/anyimage/operators/cutout_tool/depth_calibration.py`.
|
||||
- [x] 1.4 Delete `selection_calibration()` from `src/anyimage/operators/cutout_tool/geometry.py` and remove its test.
|
||||
- [x] 1.5 Drop reference samples farther than `REFERENCE_DEPTH_LIMIT` meters before the percentile and use that limit as the reference when the selection leaves no sample.
|
||||
- [x] 1.5 Drop reference samples farther than `REFERENCE_DEPTH_RANGE_FACTOR` times the median before the percentile, and return `REFERENCE_DEPTH_BASELINE` when there is no valid pixel at all.
|
||||
- [x] 1.6 Return the flat canonical direction from `fit_depth_direction()` when no sample is usable, and delete the caller-side `try/except` in plane conversion.
|
||||
|
||||
## 2. Server artifacts
|
||||
|
||||
@@ -11,7 +11,8 @@ from .image import depth_data_name, image_empty_bounds
|
||||
PLANE_VARIANCE_FLOOR_RATIO = 0.025
|
||||
REFERENCE_DEPTH_PERCENTILE = 95.0
|
||||
DEPTH_ALPHA_THRESHOLD = 0.95
|
||||
REFERENCE_DEPTH_LIMIT = 8.0
|
||||
REFERENCE_DEPTH_RANGE_FACTOR = 1.1
|
||||
REFERENCE_DEPTH_BASELINE = 8.0
|
||||
FLAT_DEPTH_DIRECTION = (0.0, 1.0, 0.0)
|
||||
|
||||
|
||||
@@ -72,10 +73,10 @@ def reference_depth(image, mask=None):
|
||||
"""Return the upper-percentile camera depth of a loaded Depth texture.
|
||||
|
||||
Distances are in model meters. Samples farther than
|
||||
``REFERENCE_DEPTH_LIMIT`` are dropped so a distant background cannot push
|
||||
the reference plane, and with it the whole reconstruction scale, out to
|
||||
the far field. The limit itself is the reference when the selection leaves
|
||||
no sample.
|
||||
``REFERENCE_DEPTH_RANGE_FACTOR`` times the median are dropped so a distant
|
||||
background cannot push the reference plane, and with it the whole
|
||||
reconstruction scale, out to the far field. A selection without usable
|
||||
samples returns ``REFERENCE_DEPTH_BASELINE``.
|
||||
"""
|
||||
import numpy as np
|
||||
|
||||
@@ -87,10 +88,12 @@ def reference_depth(image, mask=None):
|
||||
raise ValueError("Depth reference mask dimensions do not match")
|
||||
usable &= mask
|
||||
values = depth[usable]
|
||||
values = values[values <= REFERENCE_DEPTH_LIMIT]
|
||||
if not values.size:
|
||||
return REFERENCE_DEPTH_LIMIT
|
||||
return float(np.percentile(values, REFERENCE_DEPTH_PERCENTILE))
|
||||
return REFERENCE_DEPTH_BASELINE
|
||||
near = values[
|
||||
values <= REFERENCE_DEPTH_RANGE_FACTOR * float(np.median(values))
|
||||
]
|
||||
return float(np.percentile(near, REFERENCE_DEPTH_PERCENTILE))
|
||||
|
||||
|
||||
def _weighted_depth_plane(coordinates, values, weights):
|
||||
|
||||
@@ -9,7 +9,7 @@ import pytest
|
||||
from anyimage.common.coordinate import canonical_direction_to_symmetry_legacy
|
||||
from anyimage.common.depth import (
|
||||
FLAT_DEPTH_DIRECTION,
|
||||
REFERENCE_DEPTH_LIMIT,
|
||||
REFERENCE_DEPTH_BASELINE,
|
||||
depth_uniform_scale,
|
||||
fit_depth_direction,
|
||||
fit_symmetry_depth_direction,
|
||||
@@ -70,7 +70,7 @@ def test_base_plane_fit_is_stable_for_a_slender_diagonal_domain():
|
||||
|
||||
|
||||
def test_reference_depth_uses_the_upper_depth_percentile():
|
||||
depth = np.linspace(1.0, 2.0, 400, dtype=np.float32).reshape(20, 20)
|
||||
depth = np.linspace(1.0, 1.2, 400, dtype=np.float32).reshape(20, 20)
|
||||
|
||||
reference = reference_depth(_camera_depth_image(depth))
|
||||
|
||||
@@ -80,7 +80,7 @@ def test_reference_depth_uses_the_upper_depth_percentile():
|
||||
|
||||
def test_reference_depth_ignores_transparent_invalid_and_non_positive_pixels():
|
||||
depth = np.asarray(
|
||||
((1.0, 2.0, 3.0, 4.0), (5.0, 0.0, np.nan, 6.0)),
|
||||
((1.0, 1.05, 1.1, 9.0), (1.15, 0.0, np.nan, 1.2)),
|
||||
dtype=np.float32,
|
||||
)
|
||||
alpha = np.asarray(
|
||||
@@ -90,12 +90,12 @@ def test_reference_depth_ignores_transparent_invalid_and_non_positive_pixels():
|
||||
reference = reference_depth(_camera_depth_image(depth, alpha))
|
||||
|
||||
assert reference == pytest.approx(
|
||||
float(np.percentile(np.asarray((1.0, 2.0, 3.0, 5.0, 6.0)), 95.0))
|
||||
float(np.percentile(np.asarray((1.0, 1.05, 1.1, 1.15, 1.2)), 95.0))
|
||||
)
|
||||
|
||||
|
||||
def test_reference_depth_narrows_its_domain_to_the_mask():
|
||||
depth = np.asarray(((4.0, 5.0, 1.0, 2.0),), dtype=np.float32)
|
||||
depth = np.asarray(((1.0, 1.05, 1.1, 1.15),), dtype=np.float32)
|
||||
image = _camera_depth_image(depth)
|
||||
mask = np.asarray(((False, False, True, True),))
|
||||
|
||||
@@ -103,40 +103,46 @@ def test_reference_depth_narrows_its_domain_to_the_mask():
|
||||
float(np.percentile(depth, 95.0))
|
||||
)
|
||||
assert reference_depth(image, mask) == pytest.approx(
|
||||
float(np.percentile(np.asarray((1.0, 2.0)), 95.0))
|
||||
float(np.percentile(np.asarray((1.1, 1.15)), 95.0))
|
||||
)
|
||||
|
||||
|
||||
def test_reference_depth_ignores_soft_depth_alpha():
|
||||
depth = np.asarray(((1.0, 2.0, 3.0, 4.0),), dtype=np.float32)
|
||||
depth = np.asarray(((1.0, 1.05, 1.1, 1.15),), dtype=np.float32)
|
||||
image = _camera_depth_image(depth, ((1.0, 1.0, 0.9, 0.6),))
|
||||
|
||||
assert reference_depth(image) == pytest.approx(
|
||||
float(np.percentile(np.asarray((1.0, 2.0)), 95.0))
|
||||
float(np.percentile(np.asarray((1.0, 1.05)), 95.0))
|
||||
)
|
||||
|
||||
|
||||
def test_reference_depth_ignores_samples_beyond_the_metric_limit():
|
||||
def test_reference_depth_ignores_a_distant_tail():
|
||||
depth = np.asarray(((2.0, 4.0, 6.0, 1000.0),), dtype=np.float32)
|
||||
|
||||
reference = reference_depth(_camera_depth_image(depth))
|
||||
|
||||
assert reference == pytest.approx(
|
||||
float(np.percentile(np.asarray((2.0, 4.0, 6.0)), 95.0))
|
||||
float(np.percentile(np.asarray((2.0, 4.0)), 95.0))
|
||||
)
|
||||
|
||||
|
||||
def test_reference_depth_falls_back_to_the_limit_without_a_selection():
|
||||
def test_reference_depth_follows_the_content_scale_when_everything_is_far():
|
||||
depth = np.asarray(((9.4, 10.6, 11.6, 13.0),), dtype=np.float32)
|
||||
|
||||
reference = reference_depth(_camera_depth_image(depth))
|
||||
|
||||
assert reference == pytest.approx(
|
||||
float(np.percentile(np.asarray((9.4, 10.6, 11.6)), 95.0))
|
||||
)
|
||||
|
||||
|
||||
def test_reference_depth_falls_back_to_the_baseline_without_a_selection():
|
||||
transparent = _camera_depth_image(
|
||||
np.zeros((2, 2), dtype=np.float32),
|
||||
np.zeros((2, 2), dtype=np.float32),
|
||||
)
|
||||
far = _camera_depth_image(
|
||||
np.asarray(((1000.0, 2000.0), (4000.0, 8000.0)), dtype=np.float32),
|
||||
)
|
||||
|
||||
assert reference_depth(transparent) == pytest.approx(REFERENCE_DEPTH_LIMIT)
|
||||
assert reference_depth(far) == pytest.approx(REFERENCE_DEPTH_LIMIT)
|
||||
assert reference_depth(transparent) == pytest.approx(REFERENCE_DEPTH_BASELINE)
|
||||
|
||||
|
||||
def test_symmetry_depth_direction_fits_a_sloped_base_plane():
|
||||
|
||||
Reference in New Issue
Block a user