Combining Object Detection and Hand Gesture Recognition for Automatic Lighting System Control

Abstract

Recently, smart lighting systems are the combination between sensors and lights. These systems turn on/off and adjust the brightness of lights based on the motion of object and the brightness of environment. These systems are often applied in places such as buildings, rooms, garages and parking lot. However, these lighting systems are controlled by lighting sensors, motion sensors based on illumination environment and motion detection. In this paper, we propose an automatic lighting control system using one single camera for buildings, rooms and garages. The proposed system is one integration the results of digital image processing as motion detection, hand gesture detection to control and dim the lighting system. The experimental results showed that the proposed system work very well and could consider to apply for automatic lighting spaces.

I. INTRODUCTION

In recent years, smart lighting systems have been increasing. It is widely used in lighting system to substitute fluorescent lamps which have dis-advantages such as high cost, unstable and difficult to integrate with other devices. But general control systems are often controlled by hand or semi-automatic [1], [2]. And the simplest lighting systems often use one light driver to control turn on/off. In fact, these systems only control turn on/off all lights without select each individual light. Thus, it is requirement for developers to give an intelligent control system that can be automatically controlled by software that user can program to change system if it is necessary.

At this point, smart lighting systems are used motion sensors and light sensors. These systems control on/off, and the brightness of lights based on the motion of objects and the brightness of environment [3], [4]. In the proposed system, we only used one single camera to replace the role of light sensors and motion sensors that is used in previous systems. Our system controls to automatically turn on/off, and dims the brightness of lights following the position of moving object [5], the hand gesture of moving object [6] and the illumination of environment based on video processing. In section 2, we will present the proposed system, the implementation and experimental results are shown in Section 3. The conclusion is described in the Section 4.

 

II. THE PROPOSED SYSTEM

The proposed system includes one single camera, computer, one LED board which consists of the LED array and LED driver as shown in Fig. 1. When an object appears in camera zone, the camera will send video sequence to computer to perform video processing. The purpose of video processing is to detect the position of moving objects, recognize hand gesture and calculate the brightness of environment to control turn on/off and dimming LEDs.

Our system operates on computer and the LED board. On computer, camera will get video sequence to perform moving object detection and hand gesture recognition. And the system then finds corresponding LEDs to the position of moving object and hand gesture, and it then calculates the brightness of LEDs based on the brightness of the received images. The position and brightness of LEDs will be transferred to the LED board. On LED board: the data that is received from computer then checked the content of data. If data is different null, it will be decoded to determine the brightness and position of LEDs. Finally, it sends control signal to de-multiplexer to control dimming LEDs.

 

Fig. 1. The proposed system.

 

III. IMPLEMENTATION AND EXPERIMENTAL RESULTS

 

Fig. 2. LED board implementation.

 

As description in section 2, our system is implemented and experimented on computer and LED board. We designed and implemented LED board as shown in Fig. 2. The hardware structure of LED board includes 5 parts: LED array (1), De-multiplexer (2), AVR micro controller (3), Supply power (4), USB-RS232 transceiver (5)

We would like to propose a scenario of the proposed system as shown in Fig. 3. Experimental environment is one room or one class. Camera is connected to computer, and computer interfaces with the LED board by the USBRS232 transceiver. The LED board is divided into blocks. When a person enters room, the system always detects his motion and recognizes his hand gesture to control LEDs. Firstly, he entered room and did not has any hand gesture. System only detected motion and controlled turn on, dimming LED following his position. Then he has hand gesture, system recognized and controlled dimming LEDs following his hand gestures. The brightness level of LED is dimmed based on the brightness of the received image. When he goes to position where has low brightness, the LED will be has high dimming level and reverse. We also experimented with two peoples to prove that our system operate well with multi moving objects. All experimental results are shown in Fig. 4.

 

Fig. 3. The scenario of the proposed system.

 

Fig. 4. Experimental results, (a) with single objects, and (b) with multi objects.

 

IV. CONCLUSION

In this paper, we designed and experimented an automatic lighting control system using single camera, based on moving object detection, hand gesture recognition and the brightness of the received image. Our system is implemented and experimented on LED board. Future, we will implement and experiment with a large capacity LEDs system, and evaluate the level of energy consumption, and compare with current others.

ACKNOWLEGDEMENT

This paper is supported by the AI Committee; and Advanced Analytics Center; FPT Software Co., Ltd., Hanoi, Vietnam.