---
title: "Dobot"
space: "Vignettes"
url: "https://factory.uga.edu/vignettes/robotics/dobot"
updated: "2026-11-19"
---

# Dobot Vignette 1

## Introduction

In manufacturing, defect detection has always been a very important part. In the past, most defect detection was done by people. Today, defect detection based on machine vision is already very popular. However, to train a model, we need to collect a large amount of data. The emergence of the Vision-Language Model (VLM) provides a new solution for this problem. Unlike traditional vision models, a VLM can understand image content. It can also use natural language prompts to analyze images at the semantic level. Users can directly describe the inspection target and the inspection requirements in text. For example, they can say, “check whether this part has cracks,” “judge whether there are obvious scratches on the surface,” or “check whether this assembly has missing parts.” This method reduces the dependence on a specific training dataset. It also allows the system to adapt to different types of inspection tasks in a more flexible way.

For this reason, we designed this platform. It can use a robotic arm with a camera to perform all-around defect detection on the target object.

This project allows users to control the robotic arm with their own natural language. It also allows the robotic arm to perform tasks such as defect detection and target object grasping.

Overall, this platform shows a more natural, flexible, and scalable human-robot interactive method for industrial inspection. The system combines natural language understanding, a vision-language model, robot motion control, and a web-based operation interface. It allows users to control the robot in a more intuitive way to complete inspection and operation tasks. This method can lower the barrier to using robotic systems. It also provides a new path for intelligent inspection, flexible automation, and human-robot collaboration in future manufacturing scenarios.





## Hardware Implementation

### Robotic Arm

In this project, the robotic arm we use is the Dobot E6 robotic arm. This robotic arm has six degrees of freedom. It is very suitable for grasping tasks and defect detection tasks.

![Dobot Arm](https://assets.gaa.im/uploads/dobot_1_figures/dobot_arm_wide.png)

To achieve object detection and defect detection tasks, we need a camera. We use a monocular camera to achieve this goal.

![Dobot Arm Up Close](https://assets.gaa.im/images/dobot_1_figures/dobot_arm_closeup.png)

To grasp small cubes, we use an electric suction cup. This suction cup is connected to the I/O port of the robotic arm. This connection allows the system to control the electric suction cup.

### Computer

We use a Mac mini M4 as the computing unit. The robotic arm communicates with the Mac mini through an Ethernet cable.



### Software Implementation

The software part is designed to control the robotic arm by using a VLM. It also supports grasping the target object and performing defect detection. Users can also interact with the robotic arm through the GUI that we designed.





## Overall System Architecture

The software system mainly has three parts: the web interaction interface, the task planning and execution engine, and the low-level robot control interface.

The web interaction interface is implemented based on NiceGUI. It receives user input, shows camera images, displays robot execution results, saves system logs, and visualizes the task plan and the VLM analysis results. Users can use buttons for manual control. They can also directly enter natural language commands. The system then automatically parses the commands and executes the corresponding tasks.

![Dobot Web Interface](https://assets.gaa.im/images/dobot_1_figures/dobot_web_interface.png)

The task planning and execution engine connects user commands with robot actions. When a user enters a natural language command, the system first converts the command into a structured action plan. Then it calls different execution modules based on the action type. For example, movement commands are converted into robotic arm motion commands. Inspection commands trigger multi-view image capture. Defect analysis commands call the VLM to judge the images. Grasping commands call visual recognition and the pick-and-place process.

The low-level robot control interface communicates with the Dobot robotic arm through TCP/IP. Through this interface, the system sends commands such as power on, clear error, enable, motion, speed setting, and suction cup control. It also reads the status information returned by the robotic arm. In this way, the system controls the real robot.

## Visual Inspection and VLM Defect Analysis

In the visual inspection process, the robotic arm observes the target object from multiple views based on preset positions. It also captures images. The captured images are saved and shown on the web interface. Users can then call the VLM module to analyze defects in these images. During defect analysis, the VLM observes the captured images. It then performs defect analysis based on the prompt.

![Dobot Analysis Screen](https://assets.gaa.im/images/dobot_1_figures/dobot_analysis_screen.png)

The whole defect analysis process is divided into two parts: “image capture” and “image understanding.” The robotic arm first obtains visual information in a stable way. Then the VLM performs semantic analysis on the image content. This method is useful for debugging. It also makes it easier for users to review each image and its corresponding judgment result.

## Colored Cube Recognition and Grasping

The grasping and placing function is implemented based on the external camera and the calibration result from camera coordinates to robot coordinates. The system first uses the camera to capture the current workspace image. It then calls the visual detection module to recognize red, yellow, green, and blue cubes. For each cube, the system detects its color, center pixel coordinates, and pose angle.

![Dobot Cube Recognition](https://assets.gaa.im/images/dobot_1_figures/dobot_color_cubes.png)

After the target cube is detected, the system reads the calibration file from the camera coordinate system to the robot coordinate system. It also uses a homography matrix to convert the pixel coordinates in the image into robot plane coordinates. In this way, the system can convert the “position of the cube in the image” into the “position that the robotic arm can move to.”



Before the system sends a real motion command, it checks whether the target point is within the calibrated range. This check includes both the image space and the robot space. It prevents the robotic arm from moving to a dangerous position because of a recognition error or because the target is outside the workspace. If the target is not within the safe range, the system stops the execution and returns an error message.

After the position of the target cube is confirmed as valid, the system executes the complete pick-and-place process. The robotic arm first moves to a safe height above the target. Then it moves down to the grasping height and turns on the suction cup to grasp the cube. After grasping is completed, the robotic arm rises to a safe height. It then moves to the preset placing position. After moving down, it turns off the suction cup and releases the object. Finally, it returns to the observation position.

![Dobot Example](https://assets.gaa.im/images/dobot_1_figures/dobot_arm_example.png)

## Dobot Robot Communication Interface

The low-level robot control is completed through the Dobot TCP/IP interface. For example, the system can send commands to enable the robot, clear alarms, set speed, execute joint motion, execute linear motion, read the robot mode, and control digital output. The related secondary development code can be found here: [https://github.com/Dobot-Arm/TCP-IP-Python-V4](https://github.com/Dobot-Arm/TCP-IP-Python-V4)

The information returned by the robot is parsed by the program. It is used to judge whether the command is executed successfully. If the returned result shows an error, the system stops the current process and reports the error. In this way, the upper-level interface and the task planning module do not need to directly handle complex robot communication details. They complete control through the packaged interface.

## Safety Mechanism

This system has a safety mechanism. We have an emergency stop button. The emergency stop button can be used to stop the robotic arm when it moves into an unsafe area.

![Dobot Emergency Stop](https://assets.gaa.im/images/dobot_1_figures/dobot_stop.png)

After the emergency stop button is pressed, users can click the clear error button to clear the error. Then they can click the enable button to start the robotic arm again.

![Dobot Error Clear](https://assets.gaa.im/images/dobot_1_figures/dobot_clear_screen.png)

## Conclusion

Overall, this vignette implements a complete human-robot interactive robot control process. The system starts from the user’s natural language or voice input. It then goes through structured task planning, visual perception, robot coordinate conversion, motion control, and result feedback. Finally, it completes operation tasks on the real robotic arm.

It shows the application of VLM in our future manufacturing work. It also shows that VLM can be used to understand user intent and to perform tasks such as defect detection.