Drone Aerial Guidance System Utilizing a Robotic Arm

Open original video ↗

I developed a system that guides the movement and landing of a drone by utilizing an ArUco marker attached to a ground robotic arm as a visual reference point. I integrated an agent that executes user commands by linking the drone's marker tracking policy with the robotic arm's movement to target positions.

Participating as the project team leader, I took charge of the drone's reinforcement learning and the agent system design. I implemented the linkage between the robotic arm and the agent, connecting individual control functions into a single demonstration system.

1. Core Implementation

• PPO-based drone marker tracking
  I configured the tracking policy utilizing the relative position, orientation, and movement state with respect to the ArUco marker. The drone was trained to track the target by incorporating position and orientation errors, changes in control inputs, and marker visibility into the reward function.

• Imitation learning-based robotic arm guidance
  The marker was moved to the target position utilizing the ACT-based robotic arm policy implemented by the team. I utilized the ground robotic arm as a flight guidance reference by linking the drone to track this marker.

• LangChain-based agent integration
  I configured the control functions of the drone and robotic arm as tools callable by the agent. I implemented an integrated execution flow that connects user commands to the specific operations of each device.

2. Validation and Achievements

I demonstrated marker tracking and movement/landing guidance operations using an actual physical drone and robotic arm. This project received the Korean Air Capstone Design Award at the 7th National Undergraduate Capstone Design Competition hosted by the Society of Aerospace System Engineering (SASE).