Hierarchical Object-Oriented POMDP Planning for Object Rearrangement

Multi-object rearrangement with egocentric vision requires integrating perception, planning, navigation, and manipulation, especially in partially observable, multi-room settings common in everyday life.

Simple greedy planners or fixed skill sequences can't determine the optimal order to move objects. They fail when spatial reasoning is required, such as moving one object to unblock another's path or goal.

Existing work often assumes single rooms or high initial visibility (e.g., >60%). Realistic tasks, like tidying a house, require agents to actively search for objects that are not initially visible.

Reinforcement Learning (RL) methods are not well-suited for the long-horizon, sparse-reward nature of multi-object rearrangement and struggle to scale to complex, multi-room scenarios.

These approaches often use predetermined skill sequences or simple greedy planners. This restricts their ability to find optimal object interaction orders or handle new challenges like blocked paths without extensive retraining.

Current datasets are often limited to single-room settings with only a few objects. They fail to test for challenges like severe partial observability, large object counts (10-20), or complex spatial dependencies (blocked paths).

A high-level POMDP planner handles strategic, abstract decisions (e.g., 'search for object X', 'move object Y to goal'). This frees it from low-level details, which are handled by specialized policies (e.g., 'navigate to (x,y)', 'pick object').

Instead of one massive, intractable belief state, the system maintains a separate, independent belief (a probability distribution over locations) for each object. This leverages Object-Oriented POMDPs (OO-POMDPs) to make planning scalable.

The high-level planner can optimally decide whether it's more efficient to continue searching for a missing object or to rearrange an object it has already found, based on its current belief state.

The agent executes a low-level action, gets a new RGB/Depth observation, and uses it to update its object-factored belief state. This belief is then abstracted into a discrete state for the high-level POMDP planner, which chooses the next sub-goal (abstract action).

The belief state (probabilities over all grid cells) is abstracted into a discrete state for the planner. This includes sampling reachable `pick` locations (based on belief) and `place` locations (near goals/receptacles).

Abstract actions like `Move_AB` or `Rotate` are executed by an A* pathfinder. The `PickPlace` action is a 3-stage policy: an RL model to Pick, A* to navigate, and another RL model to Place.

Our principled planning approach significantly outperforms baselines. VRR (pure RL) fails to scale. FHC (heuristic) fails to handle detector uncertainty. MSS (rigid explore-then-rearrange) cannot handle blocked paths.

We introduce the MultiRoomR dataset (400 configurations) to test scenarios existing datasets don't: severe partial observability (10-30% initial visibility), larger object counts, and complex spatial dependencies (50% of scenes have blocked paths).

Despite imperfect perception, HOOP solves many problems. The small performance gap between HOOP and the Perfect Detector (PD) oracle shows that our POMDP planner effectively handles partial observability and detector failures.

The `HOOP-HP` ablation, which removes the hierarchy and plans directly in the low-level action space, shows a 'substantial performance gap'. Table 1 shows its Scene Success drops to near-zero on complex problems, proving the low-level action space is intractable.

As shown in Table 3, an MCTS search depth of 1 (HOOP-MCTS 1) is too myopic and fails to get a useful reward signal. Our full model uses a depth of 12, showing that lookahead is necessary to make strategic decisions.

The `FHC` (Frontier Exploration) baseline uses a simple 'explore-then-rearrange' heuristic. It fails to handle object detection failures because it's not belief-based, and it cannot reason about complex spatial dependencies like blocked paths.

We trained a YoloV10-Large (25M parameters) for 500 epochs. The resulting True Positive (TP) and False Positive (FP) rates for each object class (see Table 2 in paper) are used to build the probabilistic observation model for the POMDP.

The planner samples a state from the current belief, builds a search tree using UCT action selection, and runs rollouts to estimate value. The full algorithm is provided in the paper's appendix (Algorithm 3).

The RL policies were trained for 5 million steps using PPO. The Pick and Place policies are given navigation actions so they can finely adjust the agent's position before an interaction, making them more robust.

We introduce this new benchmark to test more realistic scenarios. Key features include:

• Severe Partial Observability: Only 10-30% of objects are visible at the start.
• Complex Dependencies: 50% of scenes contain blocked paths, and others include blocked goals and object swaps.
• Scalability: Tasks involve 10, 15, or 20 objects, forcing planners to be efficient.

This provides a practical and scalable method for multi-object rearrangement, a key skill for any future home assistant robot. It robustly handles uncertainty and complexity where pure RL and simple heuristics fail.

Our work shows how to effectively combine high-level, strategic planning under uncertainty (POMDP) with low-level continuous control (RL/A* policies). The hierarchy and abstraction are key to making the problem tractable.

By releasing MultiRoomR, we move the community beyond simplistic single-room datasets. We provide a benchmark to test systems on more realistic challenges, including severe partial observability and complex spatial dependencies.