Home Robotics

Abstract

Home Robotics is a simulation platform for manipulation with two Franka Panda robots on independent carriers of a shared linear rail. MuJoCo supplies physics and authoritative object state; ROS 2 Jazzy and ros2_control integrate the runtime and controllers; MoveIt 2 provides motion planning and collision checking. The current validated baseline is Panda1 grasping, lifting, transporting, placing, and releasing a cube with grasp and placement verification. Phase 5 exposes this sequence through a single-task PickAndPlace ROS Action. All reported manipulation results are from simulation.

1. Project Scope

The platform establishes a reproducible path from simulated physics to controller execution, collision-aware planning, and a task interface. A shared rail gives two manipulators access to an extended workspace while introducing explicit carriage-separation constraints.

Current implementation: Phases 0–5 are complete within the documented scope. Panda1 is the primary manipulation baseline; cube placement at surface_left_2 is an example validated task. Panda2 has its own control and planning chain, but coordinated manipulation is not implemented.

Future direction: demonstration-based task reproduction and broader sensing and coordination remain future work. Phase 6 and later are on hold.

2. System Architecture

ROS 2 Runtime Graph

MuJoCo owns physics, contact, and authoritative object state. MoveIt owns planning and collision reasoning. ros2_control is the execution boundary. The Task Executor provides the high-level ROS Action interface without owning MuJoCo or MoveIt internals. Static environment geometry and synchronized object poses supply the planning scene.

Architecture notes describe these boundaries in more detail.

Main ROS 2 Nodes and Runtime Components

Names describe the normal launch configuration. The runtime reuses the bridge implementation from home_robotics_control; this does not require a second simulator or a separate legacy bridge node. Controllers are listed in Section 5.

3. Simulation Environment

The MuJoCo environment contains the shared rail, two robot carriers, six work surfaces, a floor and arena boundary, and configured objects. Scene geometry, robot setup, and physics parameters are maintained centrally in config/.

An object's presence in the scene or registry does not establish manipulation support. The cube is the validated end-to-end object; apple, bowl, and pan manipulation are not supported.

4. Dual Panda and Shared Rail

Each Franka Panda combines seven arm joints with one independently controlled prismatic rail joint: 8 DOF per manipulator planning group. Each robot has a separate Franka Hand gripper, outside the eight-joint chain. The carriers share a physical rail and must maintain at least 0.70 m separation; crossing is prohibited.

Shared-rail separation safety at the hardware/control boundary and MoveIt geometric collision checking are separate layers. Rail separation alone does not establish collision-free arm motion. Panda2 remains available as a second robot; coordinated dual-arm manipulation has not been validated or implemented.

5. ROS 2 Control Stack

The ROS 2 Jazzy runtime exposes a custom MuJoCo hardware interface, home_robotics_mujoco_hardware/MujocoTopicSystem, to ros2_control. Standard trajectory controllers command each rail-and-arm chain, while separate gripper controllers command the Franka Hands. The joint_state_broadcaster publishes measured joint state.

Controller actions pass through the hardware interface to the authoritative MuJoCo runtime. Simulation time and reset belong to that runtime; MoveIt does not write directly to the physics state. The demo instructions start the Phase 2 stack, MoveIt, static scene loader, and dynamic object synchronizer in separate terminals.

Command and State Flow

Command: MoveIt trajectory → panda1_trajectory_controller → ros2_control position command interfaces → MujocoTopicSystem.write() → /mujoco/command → mujoco_runtime → simulated joint actuators. Panda2 has an equivalent independent controller path.

Feedback: MuJoCo joint state → /mujoco/joint_states → MujocoTopicSystem.read() → ros2_control position/velocity state interfaces → joint_state_broadcaster → /joint_states → MoveIt and monitoring. The private transport uses sensor_msgs/msg/JointState in both directions. Controller manager is configured for 100 Hz; ROS consumers use use_sim_time=true.

Configured Controllers

Key ROS 2 Interfaces

Controller Action paths combine the configured controller names with action_ns in home_robotics_moveit_config/config/moveit_controllers.yaml. Both robots' endpoints are configured; the validated manipulation task uses Panda1.

6. Motion Planning with MoveIt 2

The two manipulator groups are panda1_manipulator (shared_rail → panda1_tcp) and panda2_manipulator (shared_rail → panda2_tcp). Each uses KDL inverse kinematics and OMPL planning with collision checks against the PlanningScene.

The static PlanningScene represents fixed environment geometry; synchronized CollisionObjects represent configured objects. There is currently no coordinated 16-DOF dual-arm planning group.

7. Dynamic Object Synchronization

mujoco_runtime publishes geometry_msgs/msg/PoseStamped on /mujoco/objects/{object_id}/pose for configured planning-scene objects, including /mujoco/objects/cube/pose. The dynamic_object_scene_sync node combines these poses with ObjectRegistry geometry to construct MoveIt CollisionObjects and apply PlanningScene diffs. Its launch sets sync_rate=20.0; MuJoCo remains the authority and MoveIt receives a planning representation.

The planning-object lifecycle is WORLD → ATTACHED → WORLD. The synchronizer queries /get_planning_scene for attached IDs and excludes them from world updates. Manipulation pauses/drains synchronization around ownership changes: verified grasp and stabilization precede attachment, and supported placement/release restores one WORLD object with no stale attachment. Contact state is consumed separately by manipulation verification.

8. Manipulation Pipeline

The pipeline combines a physical simulated grasp with temporary MuJoCo stabilization during handling. Stabilization is conditional on successful grasp verification. MoveIt attachment separately represents the held object for collision reasoning. After placement support checks, stabilization is removed, the gripper opens, and final pose, support, and scene ownership are verified.

The cube baseline includes transfer from its configured source to a named surface such as surface_left_2. A separate purple_ball trial demonstrated grasp, lift, and return; it does not establish named-target sphere placement or reliability and is outside the Phase 5 Action scope.

9. Task Executor API

Endpoint: /home_robotics/pick_and_place
Interface: home_robotics_interfaces/action/PickAndPlace.action

Only one task can be active; a concurrent goal is rejected. Requests are checked for empty or unknown IDs, supported manipulation scope, and valid named targets. The current Action accepts only cube with Panda1; the target must differ from the source surface. Structured results preserve both task-level and manipulation failure reasons. An unavailable final pose is represented by an all-zero pose, including quaternion W.

execute=false is the default node/launch safety setting, not a Goal field. Planning-only preflight issues no physical motion commands; its success does not mean that a placement occurred. execute=true enables simulated execution.

The executor currently delegates Phase 4 through a ros2 launch subprocess. Planning-only cancellation is supported at safe boundaries; execution-mode cancellation requests are rejected because arbitrary interruption is not supported. A direct reusable manipulation API remains a possible future refactor.

Full implemented API contract.

End-to-End Task Execution

For the validated cube → surface_left_2 task with execute=true:

1. The client sends a PickAndPlace goal; the executor admits it only if no task is active.
2. task_executor validates the cube capability and named target against the registry and scene configuration.
3. The adapter starts cube_pick_place_demo.launch.py as a ros2 launch subprocess, passing the object, target, execution flag and result-file path.
4. The manipulation process obtains fresh joint/object state and checks controllers, scene synchronization, rail separation and collision preconditions.
5. MoveIt and the manipulation layer plan/check pre-grasp and Cartesian stages, including hypothetical attached transport. Planning-only mode would finish here without motion.
6. The gripper opens; pre-grasp is replanned from measured state. Pre-grasp and approach execute through MoveIt, the Panda1 trajectory controller and the hardware bridge while MuJoCo publishes state feedback.
7. The grasp observer starts, the gripper closes via /panda1_gripper_controller/gripper_cmd, and physical grasp verification must pass.
8. Temporary MuJoCo stabilization activates; after settling checks, the PlanningScene object changes from WORLD to ATTACHED.
9. The arm lifts and transports the cube to the target, checking lift and placement support/region conditions.
10. Stabilization is removed, WORLD ownership is restored, the gripper opens, and the arm retreats.
11. Final pose, support, open width, ownership and Panda2 drift checks determine the Phase 4 result.
12. The adapter reads the process report and exit status; the executor returns the Action result, preserving manipulation failure details when present.

10. Validation and Results

These are recorded repository results. The test layers check different properties and should not be combined into a single score.

Retained failure: benchmark trial 2 failed before motion during planning/retiming. MoveIt TOTG reported negative path velocity on a hypothetical transport trajectory. The failed trial remains in the evidence; it was not replaced by a later successful run. Successful trials recorded final position errors of 1.416–1.923 mm, with a mean of 1.598 mm.

Runtime integration checks compose the control stack, planning, scene synchronization, and manipulation or Action entry point. Manual RViz checks inspect geometry and state visually; they are not automated tests. The focused Action suite uses a fake adapter and does not substitute for a full-stack run.

Validation summary | Benchmark interpretation | Per-trial benchmark data (JSON) | Phase 5 acceptance

11. Development Environment

Primary development used an Apple MacBook with an Apple M4 chip, 16 GB unified memory, and a 512 GB SSD. The robotics runtime was developed primarily in an Ubuntu 24.04 ARM64 virtual machine, using ROS 2 Jazzy, MuJoCo, MoveIt 2, and ros2_control.

This describes development hardware, not a performance benchmark. Local compute and memory constraints have influenced scope; VLA/LLM-heavy work is deferred.

12. Limitations

• Simulation only; no real-robot deployment or hardware validation.
• Panda1 is the primary validated manipulation baseline. The cube is the validated named-placement object; other scene objects do not establish generic manipulation.
• Object poses and contacts come from ground-truth simulation state. No perception pipeline or semantic vision is implemented.
• No coordinated dual-arm manipulation, generic task scheduler, or autonomous robot allocation.
• Task Executor execution uses process-level Phase 4 delegation. Execution-mode cancellation is rejected; planning-only cancellation has supported safe boundaries.
• No VLA/LLM runtime. The retained planning/retiming failure limits any claim of repeatable success.

13. Future Work

Not implemented; Phase 6+ is on hold. Directions include Logitech F710 teleoperation of Panda2, human demonstration recording, semantic/task extraction from demonstrations, and Panda1 autonomous reproduction of demonstrated tasks. Later work may address perception, coordinated dual-arm manipulation, VLA/LLM integration, and sim-to-real transfer.

14. Repository Structure

Home-Robotics/
├── config/
├── simulation/
├── tests/
├── ros2_ws/src/
│   ├── home_robotics_control/
│   ├── home_robotics_description/
│   ├── home_robotics_mujoco_hardware/
│   ├── home_robotics_bringup/
│   ├── home_robotics_moveit_config/
│   ├── home_robotics_manipulation/
│   ├── home_robotics_interfaces/
│   └── home_robotics_task_executor/
└── docs/

Configuration and simulation are separate from the ROS packages. Technical documentation is in docs/; historical phase notes and measured evidence are preserved under docs/archive/.

15. Project Links

• GitHub Repository — source, configuration, and project history.
• README — current capability and quick start.
• Architecture — runtime ownership and interfaces.
• Validation — evidence and reproduction commands.
• Demo — build, stack startup, planning-only and execution modes.

Project license has not yet been finalized. Third-party Panda source and mesh attribution remain with the assets in the repository.