Changes

Page history
docs: 2026-05-10 wiki refresh (current code + open audit) authored by Hani Alhabboub's avatar Hani Alhabboub 
Sync the wiki against `main` as of commit db98e165.  The previous
revision was authored ~March 2026 and predated nine modules and
the entire eval / baseline subsystem.

Pages updated:

- Home: refresh CLI flag table (smart log-level default; --baseline
  / --uniform-cell-size / --unit-codes / --no-boxel-viz / --show-free /
  --n-objects; stack scene preset; goal kinds), add eval-sweep
  quickstart, refresh tech-stack table, retire "as of March 2026".

- Architecture_Overview: add belief.py, execution.py,
  uniform_grid.py, reboxelize.py, smart_filter.py, eval_runner.py,
  eval_plotter.py to the module dependency diagram, module summary
  table, and file-structure tree.  Replace stale archive listing
  with the actual current archive contents (RESOLVED + DEFERRED
  audit splits).  Drop SESSION_HANDOFF.md (gone).

- Known_Issues_and_Roadmap: full rewrite against
  CODEBASE_AUDIT.txt SUMMARY block (last sync 2026-05-10).  Drop
  the stale #76/#55/#80-94 numbering scheme; reflect current
  TIER 0/1/2/3 layout (#50, #57, #59, #61, #62, #63, #65, #66
  Plan C, with #67 just closed); add a "Recently Resolved" table
  for the 2026-05-07 -> 2026-05-10 closure burst (#9, #10, #36,
  #41, #48, #49, #58, #60, #67, etc.); add a roadmap-to-submission
  ordering ending at #63 doc sweep.

- Spatial_Reasoning: list all five contributing modules in the
  overview; add a UniformGridGenerator section (audit #10); note
  that semantic baseline's min_resolution is now clamped at
  max_object_AABB + 1 cm so semantic vs uniform compare fairly
  (commit db98e165); mention reboxelize.py.

- Scene_Environment: add stack_scene preset (audit #30); document
  scalability_scene's n_hidden guarantee (audit #29); add tray-entity
  section (audit #49); drop "(not yet implemented)" claim on
  scalability_scene (eval framework landed 2026-05-08).

- Design_Decisions: add semantic-vs-uniform baseline rationale
  inline with the existing semantic-boxels-vs-voxels section; add
  Action Costs (stack=2 vs others=1) and Post-Action Lift
  Workaround sections.

- Planning_System: update planner.plan() signature with held_obj
  (audit #58) and stackable_objects (--goal stack /
  find-and-tray-stack); add compute-stack-kin to the stream map.

- Execution_Pipeline: split orchestration (test_full_pipeline.py)
  from action handlers (execution.py); document the three --goal
  modes; replace the stale "AABB-shadow-containment" hidden-target
  description with camera-ray occlusion (audit #67); add stack
  action handler section; document _apply_post_action_lift,
  _verify_cube_on, reboxelize_after_relocation in the place
  postlude; add timing_summary.json + run_config.json + smart-filter
  notes to the Logging section.

- PDDL_Domain_Reference: add :conditional-effects + :action-costs
  to requirements; add boxel_fits, on_surface, stack_kin,
  on, on_table, clear, is_tray to the predicate tables; refresh
  pick/place pre/eff against current PDDL (clear/on_table/forall
  conditional effect on pick; on_surface + boxel_fits on place);
  add stack action; add compute-stack-kin stream; add PF-6 entry
  for stack alignment.

- Core_Data_Structures: relocate BeliefState to belief.py (audit
  #26 extraction) with full method table; mark the old
  boxel_types.Boxel dataclass as removed in audit #35 (BoxelData
  is now the sole representation); add ObjectInfo.is_tray and
  SceneConfig.enable_tray fields; add stack_scene factory.
Hide whitespace changes
Inline Side-by-side
Known_Issues_and_Roadmap.md
View page @ de218bbc
......@@ -4,172 +4,120 @@
## Overview
This page tracks the current state of the codebase audit, summarizes resolved issues, lists open issues by priority tier, documents proposal alignment gaps, and outlines future work. The canonical source for issue details is `CODEBASE_AUDIT.txt` in the repository root; this page provides a navigable overview with cross-references to the relevant wiki pages.
---
## Resolved Issues
The following issues have been fixed and archived to `archive/CODEBASE_AUDIT_RESOLVED.txt`. They are listed here for completeness.
### Tier 1 -- Quick Wins (all completed 2026-03-14)
| Issue | Description | Resolution |
|-------|-------------|------------|
| #80 | Fake IK fallback (`_heuristic_ik`) allows garbage data | Removed entirely. `BoxelStreams` raises `ValueError` if `robot_id` is `None`. |
| #83 | Raycasting blind spot -- rays hitting robot arm counted as empty | `sense_shadow_raycasting()` now accepts `robot_id`; robot-arm hits produce `still_blocked`. |
| #84 | Incomplete magic number cleanup in execution code | Added justification comments for approach/lift heights. |
| #85 | Dead/commented push code polluting codebase | Removed `sample_push_config`, push action, push stream. |
### Tier 2 -- High Value Fixes (all completed 2026-03-14)
| Issue | Description | Resolution |
|-------|-------------|------------|
| #60 | Simulation-wide flickering during planning | Reference-counted `RenderingLock` in `robot_utils.py`. |
| #67 | `_build_init()` hardcodes targets-only-in-shadows assumption | Added optional `observed_clear_regions` parameter. |
| #69 | ~30 magic numbers without justification | Justification comments added to all constants. |
| #78 | Incomplete view-blocking model -- pipeline loops forever | `compute_shadow_blockers()` finds ALL objects blocking each shadow. |
| #79 | Robot arm blocks sensing during execution | Sense handler moves arm to home before raycasting. |
| #82 | Silent IK failures in execution | `execute_pick()`/`execute_place()` abort and trigger replan on IK failure. |
| #86 | PDDL state drift after pick/place | Execution reads actual PyBullet joint state after every action. |
### Earlier Resolved Issues
| Issue | Description |
|-------|-------------|
| #1 | Oracle hiding replaced with AABB containment check |
| #2 | String-comparison sensing replaced with PyBullet raycasting |
| #5 | Linear interpolation replaced with RRT-Connect motion planning |
| #7 | Pick/place now use plan grasp + execution-time IK |
| #20 | `BoxelStreams` wired into `PDDLStreamPlanner` stream map |
| #49 | Stream IK failures on replan fixed (seed reset + null-space) |
| #52 | Move execution uses plan trajectory waypoints |
| #53 | Push replaced with pick-and-place |
| #54 | Oracle visibility uses 8-ray AABB check |
| #57 | Pick configs collide with grasped object -- added `ignored_body_ids` |
| #58 | Pipeline logging with `RunLogger` |
This page tracks the current state of the codebase audit, summarizes recent
resolutions, lists open issues by priority tier, and outlines remaining work
ahead of thesis submission. The canonical source for issue details is
`CODEBASE_AUDIT.txt` in the repository root; this page is a periodically
refreshed view of its `SUMMARY` block (last sync: 2026-05-10) with cross-
references to the relevant wiki pages.
The pre-evaluation correctness chain ended on 2026-05-08 with audit #60 [DONE]
(post-action lift fixes the move-after-pick collision). The eval framework
(audit #9) and the uniform-voxelization baseline (audit #10) were both
promoted from TIER 3 to TIER 1 in the 2026-05-08 restructure and have since
landed.
---
## Open Issues by Priority Tier
### Tier 3 -- Thesis-Critical
These issues directly block the thesis evaluation and results sections.
### TIER 0 — Top Priority
| Issue | Description | Effort | Documented In |
|-------|-------------|--------|---------------|
| #55 | **Hardcoded scene -- no scenario variation.** Scene parameterization infrastructure exists (`SceneConfig`, presets, CLI flags) but no batch evaluation runner uses it. | Partially done | [Scene Environment](Scene_Environment) |
| #76 | **No evaluation framework (PA-1).** No success rate measurement, no planning time collection, no baselines, no experiment runner. The thesis cannot have a Results section without this. | ~1-2 weeks | See PA-1 below |
| #81 | **Missing recursive object discovery (PA-2).** The proposal claims free space partitioning will recursively discover new objects. The implementation is a single-pass octree. | ~2-3 days or document as future work | [Spatial Reasoning](Spatial_Reasoning) |
### Tier 3b -- Architectural
| Issue | Description | Effort | Documented In |
|-------|-------------|--------|---------------|
| #93 | **Actions are not generic -- hidden sub-actions in execution.** `sense` contains a hidden move; `pick` contains 3 hidden moves + 3 IK solves + gripper control + constraint weld; `place` same pattern. The planner cannot reason about these sub-actions. | ~1-2 weeks | [Design Decisions](Design_Decisions), [Execution Pipeline](Execution_Pipeline) |
(Empty — no open correctness blockers as of 2026-05-10.)
### Tier 4 -- Deferrable
These are accepted simplifications or fidelity improvements.
### TIER 1 — Uniform / TAMPURA Baseline Placement (correctness blocker)
| Issue | Description | Status | Documented In |
|-------|-------------|--------|---------------|
| #44 | Shadows not recomputed after occluder relocation -- stale registry. | Accepted. Raycasting by body ID compensates. | [Spatial Reasoning](Spatial_Reasoning) |
| #73 | `shadow_occluder_map` goes stale on replan. | Low impact. Derived predicates compensate. | [Planning System](Planning_System) |
| #59 | Objects fly away on first grasp -- constraint-based grasping. | Accepted. Friction grasping is a fidelity improvement, not correctness fix. | [Design Decisions](Design_Decisions) |
| #77 | Object dimensions exceed gripper capacity (0.15 m cubes vs 0.08 m max opening). | Blocks #59. Fix together if pursued. | [Design Decisions](Design_Decisions) |
### Tier 5 -- Dead Code Inventory
| **#66** | `place` fails when an object is larger than a uniform cell. Default 5 cm cells can't fit 6–7 cm occluders → `boxel_fits` empty → PDDLStream stalls. | **Plan A landed on `main` 2026-05-09**: auto-tune cell size to `max_object_AABB + 1 cm`. Plan C (continuous-pose place, real TAMPURA) pending on `audit-64-tampura-binary-grid`. | [Spatial Reasoning](Spatial_Reasoning) |
| **#67** | Phase 1 / Phase 4 hidden-target criterion disagreement — silent run abort. ~16 % of cells aborted before any planner call. | **DONE 2026-05-10** (commit `4c9e4d5`): Phase 4 now uses camera-ray occlusion to match Phase 1. | [Execution Pipeline](Execution_Pipeline) |
Documented and preserved per codebase policy. No action needed.
### TIER 2 — Performance (planner wall-clock)
| Issue | Description | Lines |
|-------|-------------|-------|
| #87 | Dead methods/properties across codebase | ~160 lines |
| #88 | Unused imports (7 statements across 3 files) | ~7 lines |
| #89 | Unused function parameter (`config` in execute_pick/place) | 2 params |
| #90 | `BoxelVisualizer` instantiated but never called (intentionally disabled) | ~145 lines |
| #91 | Dead `CameraObservation` fields (rgb_image, depth_image, point_cloud always None) | 3 fields |
| #92 | `RunLogger` context manager protocol never used | ~5 lines |
| #94 | Dead heuristic config rejection loop (unreachable since #80) | ~8 lines |
---
| Issue | Description | Documented In |
|-------|-------------|---------------|
| **#50** | Planner perf investigation — profile, cache static init, kill redundancies (user-flagged "planner is too slow"). | [Planning System](Planning_System) |
| **#62** | Replan-while-holding still slow after #58. Sub-hotspot of #50 on the held-cube path. Profile first; merge into #50 if dominant cost is branch-agnostic. | [Planning System](Planning_System) |
## Proposal Alignment Gaps
### TIER 3 — Code Quality / Low Priority
These are features described in the thesis proposal that are missing, simplified, or deviate from the proposal's design.
| Issue | Description | Documented In |
|-------|-------------|---------------|
| **#57** | Registry should own cube geometry (audit #55 follow-up; replace PyBullet AABB fallback in `compute_stack_kin_solution`). | [Robot Control and Streams](Robot_Control_and_Streams) |
| **#59** | Scaled-up find-and-stack — N hidden targets + size-compat PDDL predicate (very low priority). | [Design Decisions](Design_Decisions) |
| **#61** | Smart console filter — strip residual startup/shutdown garbage; add Claude-Code-style thinking-animation during long planner / RRT / stream-sample calls. | [Execution Pipeline](Execution_Pipeline) |
| **#63** | README + wiki final doc sweep. **Gated on remaining open issues; explicitly the last commit before thesis submission.** | This page |
| **#65** | Rename `boxel` → `hanixel` (HANI = Heterogeneous Axis-aligned Named Interval; XEL inherited from voxel). Mechanical case-preserved sweep on its own branch (`rename-boxel-to-hanixel`). Land after `audit-64-tampura-binary-grid` merges to avoid rename noise. | Multiple |
| ID | Gap | Proposal Section | Status | Documented In |
|----|-----|-----------------|--------|---------------|
| PA-1 | **No evaluation framework.** No metrics, baselines, batch runner, or scene variation. | Section 5 | MISSING -- CRITICAL (#76) | Above |
| PA-2 | **No recursive object discovery.** Free space partitioning is a single-pass octree. | Section 4.2 | MISSING | [Spatial Reasoning](Spatial_Reasoning) |
| PA-3 | **KIF design differs.** Single predicate instead of two-predicate `K(p)` / `K(not p)`. | Section 4.4.1 | Documented deviation | [Design Decisions](Design_Decisions) |
| PA-4 | **Fixed camera for sensing.** Proposal uses robot-mounted sensor with `stream_find_sensing_config`. | Section 4.4.2 | Accepted simplification | [Design Decisions](Design_Decisions) |
| PA-5 | **No conditional sensing outcomes.** Proposal uses `when found/not_found` branches. | Section 4.4.2 | Accepted deviation (#61) | [Design Decisions](Design_Decisions) |
| PA-6 | **Only top-down grasps.** No lateral or angled grasps. Limits evaluation scenarios. | Implicit | Known limitation | [Robot Control and Streams](Robot_Control_and_Streams) |
**Open count: 8 active items** (#50, #57, #59, #61, #62, #63, #65, #66 Plan C). #67 closed since the previous summary.
---
## Future Work
### Evaluation Framework (PA-1, #76)
The highest-priority missing component. Required deliverables:
- **Batch evaluation runner**: Execute the pipeline across multiple scene configurations with different random seeds.
- **Metric collection**: Success rate, planning time per plan, total plan cost (number of actions), number of replans.
- **Scene variation**: Use the existing `scalability_scene()` preset with varying `n_occluders` and `n_targets`.
- **Baselines**: At minimum, compare against uniform voxelization (fixed-resolution grid) and fixed semantic regions (hand-defined zones).
## Recently Resolved
### Recursive Object Discovery (PA-2, #81)
The following issues closed in the 2026-05-07 → 2026-05-10 window. Bodies preserved in `archive/CODEBASE_AUDIT_RESOLVED.txt`.
The proposal describes a recursive process: after free-space partitioning, if new objects are found within a partition, the process repeats. Implementing this would require:
1. A perception step inside `FreeSpaceGenerator` to detect objects within free cells.
2. Re-running shadow calculation for newly detected objects.
3. Re-running free-space generation with the updated boxel set.
Alternatively, document this as an accepted simplification for the single-camera tabletop scenario where all objects are detected upfront.
| Issue | Description | Resolution |
|-------|-------------|------------|
| #67 | Phase 1/4 hidden-target criterion disagreement | Phase 4 switched to camera-ray occlusion (commit `4c9e4d5`, 2026-05-10). |
| #10 | Uniform voxelization baseline | `UniformGridGenerator` swap behind `--baseline uniform` (commit `afdaadd`, 2026-05-08). PDDL domain unchanged; OBJECT + SHADOW boxels untouched. |
| #9 | Evaluation framework | `eval_runner.py` (matrix sweeps + master-seed plumbing) and `eval_plotter.py` (semantic-vs-uniform autosplit) landed in commit `e8237b6` (2026-05-08). |
| #60 | Post-action lift fails to prevent collisions | Option B (sync + replan) — commit `c08dc65`, 2026-05-08. Closed the pre-eval correctness chain. |
| #58 | Preserve grasp across replans | Planner `held_obj` kwarg + `handempty/holding` swap, commits `690f86d` + `b248c76`, 2026-05-08. |
| #56 | `--post-action-lift` CLI flag | Added then **REVERTED** 2026-05-08 (no observable effect at any value tested; underlying mechanism tracked as #60). |
| #55 | Hardcoded scene → no scenario variation | Resolved by the eval framework (#9) consuming the existing `SceneConfig` parameterization. |
| #36 | Hardcoded post-action lift workaround | `_apply_post_action_lift` helper in `execution.py` (10 cm for place/stack, 5 cm cosmetic for pick), commit 2026-05-07. |
| #41 | Explicit `(on_table ?o)` predicate for stack support chains | Added to PDDL domain + `_build_init` + goal builders + verifiers, 2026-05-07. |
| #48 | Pick from a stack does not clear `on_relations[X]` | Mirror the pick conditional effect in the Python handler, 2026-05-07. |
| #49 | Find-hidden-then-stack-on-tray goal | Tray entity in scene + `--goal find-and-tray-stack` + per-target sense loop, landed in commits across 2026-05-07. |
| #31 | Comment/code drift sweep | 17 drift edits across 9 files, 2026-05-07. Pure narration kept (per user direction); only actively-wrong comments touched. |
### Friction-Based Grasping (#59, #77)
---
Replacing constraint-based grasping with friction-based grasping requires:
## Deferred / Out of Scope
1. Resizing objects to fit in the Panda's 0.08 m gripper opening (#77).
2. Adjusting grasp Z offsets so fingers contact the object sides.
3. Setting appropriate `lateralFriction` on objects and gripper shapes.
4. Testing grip stability across different object shapes.
| Issue | Status |
|-------|--------|
| #2, #14 | Long-standing accepted simplifications. |
| #42 | Logged collisions do not trigger replan — accepted ("no need to track knocked-down objects"). |
| #47 | Sense give-up after 3 still-blocked outcomes. Log-only band-aid is the accepted level of correctness for the thesis (2026-05-06). |
| #51 | Stale shadow blocker fallback after occluder relocation. Fix attempt preserved on branch `audit-51-shadow-blocker-fix` on both remotes (2026-05-07). |
### Grasp Diversity (PA-6)
---
Extending `sample_grasp()` to produce lateral and angled grasps would enable handling scenes with:
## Roadmap to Thesis Submission
- Objects near walls or edges.
- Stacked objects.
- Objects in narrow gaps.
The remaining work is sequenced as:
### Incremental Boxel Recomputation
1. **#66 Plan C** — finish continuous-pose place on `audit-64-tampura-binary-grid`, merge.
2. **#50 / #62** — planner-perf profiling pass; only land fixes that move the wall-clock needle on eval sweeps.
3. **Eval sweep run** — execute the `default` and `scalability` matrices on WSL; regenerate plots for the thesis Results chapter.
4. **#65** — boxel → hanixel rename sweep (mechanical, on its own branch).
5. **#63** — README + wiki final doc sweep (this page is the canonical staging ground).
After relocating an occluder, only the affected shadows and free-space cells need recomputation. Currently the system uses a stale registry (#44). Incremental updates would improve accuracy without the cost of full recomputation.
Per the audit, #63 is "explicitly the last commit before thesis submission" — every other open issue should land or be deferred before this page is rewritten one final time.
### Action Decomposition (#93)
---
Decomposing pick/place into sub-actions in the PDDL domain (approach, grasp, lift as separate actions with their own streams) would allow the planner to:
## Proposal Alignment Gaps
- Optimize approach/retreat paths.
- Detect collisions on approach waypoints.
- Reason about post-action configurations.
These are features described in the thesis proposal that are simplified or absent in the implementation. They are documented as design decisions rather than open issues; see [Design Decisions](Design_Decisions) for the rationale.
The trade-off is 3x more stream evaluations and a more complex domain.
| ID | Gap | Status |
|----|-----|--------|
| PA-3 | KIF design uses a single predicate instead of two-predicate `K(p)` / `K(not p)`. | Documented deviation (Bonet & Geffner-style simplification). |
| PA-4 | Fixed overhead camera for sensing rather than a robot-mounted sensor + `stream_find_sensing_config`. | Accepted simplification. |
| PA-5 | No conditional sensing outcomes (`when found/not_found` branches). Optimistic + reactive replanning instead. | Accepted deviation; convergence proof in [Design Decisions](Design_Decisions). |
| PA-6 | Only top-down grasps (no lateral or angled). | Known limitation; documented in [Robot Control and Streams](Robot_Control_and_Streams). |
| (Recursive object discovery from free-space partitioning) | Killed [WONTFIX] 2026-05-06: dead under fixed-camera + no-concave-geometry setup. Documented in `THESIS_NOTES.md` §20. | Closed. |
---
**See Also:**
- [Design Decisions](Design_Decisions) -- Rationale for the accepted deviations listed here.
- [Planning System](Planning_System) -- The replanning architecture that compensates for some of these limitations.
- [Scene Environment](Scene_Environment) -- Scene parameterization infrastructure (#55).
- [Architecture Overview](Architecture_Overview) -- Overall system structure.
- [Design Decisions](Design_Decisions) — Rationale for the accepted simplifications above.
- [Architecture Overview](Architecture_Overview) — Module list including the eval / baseline modules added during the 2026-05 push.
- `CODEBASE_AUDIT.txt` (repo root) — Canonical, up-to-the-minute issue tracker with full bodies and changelog.
- `THESIS_NOTES.md` (repo root) — Per-decision rationale at thesis-chapter granularity.
---
......
......