VNU Journal of Science: Comp. Science & Com. Eng., Vol. 32, No. 2 (2016) 23–30
Kinect Based Character Navigation in VR GameI
Thi Chau Ma, Minh Duong Hoang
VNU University of Engineering and Technology,
E3 Building, 144 Xuan Thuy Street, Cau Giay, Hanoi, Vietnam
VR game is a hot field because it provides practical experience for the player.
However, in the game, players
are usually required attaching special equipments. Those equipments are often very expensive. Moreover, many
devices make players less flexible because they are attached to players. VR game using Kinect is a good choice
because Kinect is a ordable and not attached directly to the player.
In this paper, we propose a technique
of Kinect based character navigation in VR Game.
As environment is supported by Kinect, we implement
character navigation by player’s hand actions.
Besides, we use markers to increase the e ectiveness of hand
action recognition. In our experiments, character’s rotations reaches 70% in accuracy. Straight movement and
turning reach 98% in accuracy.
Received 30 November 2015, revised 09 January 2016, accepted 14 January 2016
Kinect application, hand recognition, VR games.
1. Introduction
In the last two years, in Vietnam, the game
attracted a lot of investments and researches.
According to [1], Vietnam is the largest market
in Southeast Asia Game. However, restrictions
still exist, obviously, the solutions are still
awaited. An important factor that make the
game to be attractive is the ability of character
movements. To control the movement of the
characters, players have to touch and navigate
specialized devices to control the movements of
the characters. These devices were typically
gamepad, keyboardist, Wiimote. Besides, with
technology advances, in virtual reality games,
players no need to touch game devices. In
these games, body actions or speech were used
to control the character movement through the
special devices, such as Kinect, Virtuix Omni.
because the type of these games gives players
the flexibility and comfort while playing without
exposure Kinect. Moreover, Kinect relatively
low cost comparing with other devices such as,
head mounted devices or data gloves. Kinect-
supported games can be classified into two main
categories. In the first type, entire bodies of
players are identified to contribute and play
games like audition games [2], or collision
ball games [3]. In the second type, hand
gestures are recognized to control games [4].
In our research, we want to make use of hand
actions obtained by Kinect to navigate character
in order to give players the freedom and comfort
in playing games. However, the accuracy of the
hand action recognition and the number of hand
actions based on Kinect were limited because of
identical when recognizing di erent bone joints.
I This work is dedicated to the 20th Anniversary of the IT
Faculty of VNU-UET.
 Corresponding author. Email:
In this paper, we propose a solution of Kinect
based character navigation in VR Games. We
recognize hand action to navigate characters in
T.C. Ma, M.D. Hoang / VNU Journal of Science: Comp. Science & Com. Eng., Vol. 32, No. 2 (2016) 23–30
VR Games.
Marker is glued to the hands of
to see the above Kinect applications are based on
players to obtain more number of hand actions
recognition of human skeleton.
and increase accuracy in hand action recognition.
Another type of Kinect applications is based
The use of glued makers does not reduce the
degree of flexibility of players.
authors accurately reconstructed complex hand
poses across a variety of subjects using Kinect.
2. Related Works
The system allowed continually recovering from
tracking failures. Matthew Tang [13] showed
With the available and a ordable Kinect,
Kinect and its applications appear a lot in the life.
First of all, we mention the applications in the
medical. KinectprovideaSoftwareDevelopment
Kit (SDK) which gives access to skeletal tracking
data and can be used directly in rehabilitation
game developments. So, there are lots of
physical therapy and rehabilitation applications
[5][6][7] using Kinect. These applications are
classified into 3 groups. Fall detection and
Fall risk reduction, Stroke Rehabilitation and
exercise games. In [8], authors presented a
low-cost system to surgical training for computer
assisted surgeries by utilizing dual sensors and
marker based tracking components. Markers
were patterns in the form of checkboard. By
using markers, they implemented calibration of
dual infrared sensors and point correspondence
establishment in a stereo configuration. Then
they accomplished 3D localization by multiple
back projection lines. The system was e ectively
and traditional tracking algorithm.
Secondly, in the VR environment, a hot and
a novel method for recognizing hand gestures
using Kinect sensor. In the approach, they
recognize ’grasp’ and ’drop’ gestures with over
90% accuracy. In her thesis [14], Yi li used
K-means clustering and Graham Scan algorithm
to recognize nine popular gestures and counting
numbers. Kinect data range was between 0.5m
to 0.8m. Hands were distinguished from the
background by depth information.
In Kinect applications in particular and
recognitions in general, Marker is an e ective
tool. Marker is a very simple sticker, usually
in the form of checker board pattern. Marker is
popular in modeling by its simple manipulation
and eciency. Checker board pattern has been
using in camera calibration [15][16][17] to
reconstruct 3D model from 2D images. Checker
board pattern is also use in Kinect calibration
[18] to align depth camera and RGB camera.
Marker help robot to detect path to navigate.
In this paper, we want use hand actions from
Kinect to control the movement of character in
VR game. We utilize marker to detect hand
actions easily and accurately.
exciting field, Kinect attracted much attention.
Kinect is one of easy ways of 3D reconstruction.
In the environment, data recognized by Kinect is
3. Proposed method
divided in two types: Human skeleton and hand
[9] is an open source
software providing a simple toolkit to manipulate
Kinect data.
In this software, 3D model of an
player’s hand actions.
The players navigates
object would be rebuilt when the object are taken
characters using actions only on one hand.
with markers.
3D builder app
[10] and Kinect
character’s actions include straight movement,
sensor are used to create a 3D reconstruction
turning (left/right) and rotation. We chose these
for a 3D printer.
Ultra Seven
[11] is a game
action in implementation because hand gestures
used Kinect in AR environment.
In the game,
the player is turned into a character and shot
using hand gestures are easy.
Moreover, in the
his power.
The character was modeled game
game, most free parts of body but hand makes
player by body recognition with Kinect.
player more comfortable.
In technique term,
T.C. Ma, M.D. Hoang / VNU Journal of Science: Comp. Science & Com. Eng., Vol. 32, No. 2 (2016) 23–30
the problem is more focus.
Meanings of these
actions are described clearly as follow.
movement is character goes forward/backward.
Turning is character turns left/ right. Rotation is
character rotates his body left/right or up/down to
observe the scene. This action changes eye view
to make decision of other actions.
The player’s
thumb finger is worn marker for determining
the exact location, and recognizing of complex
hand gestures as well.
In addition to character
to perform other actions in the game such as,
shooting, and selection.
Firstly, we define some parameters later used
in the process of navigation. Root is player’s
Fig. 1: Kinect skeleton.
neck joint (Fig. 1) on player’s.
Rootdepth is the
the Kinect sensor for a while to obtain some
distance from Kinect sensor to root.
measurements such as, root position and hand
is the distance from Kinect sensor to the wrist
joint of the hand.
HandX is the wrist joint in
calculate character’s speed and acceleration.
horizontal direction in acquired image.
hand marker tracking,
we use emguCV
is root in horizontal direction.
and POSIT algorithm
[20] to determine marker
and its’ rotation.
Then, above defined actions
OrthVector is a vector orthogonal to
are recognized.
Finally, we perform navigation
straightVector on the left.
StraightSpeed and
estimation by recognized actions.
OrthSpeed are speed vectors in movements.
We defined three types of navigation: straight
movement, turning (left/right) and rotation
4. Implementation
corresponding to the following actions:
4.1. Parameter setup
 Straight movement: move the wrist towards
or away from the root. The change of the
distance from the wrist to the root decides
the direction (forward/backward) and speed
of movement.
At the beginning process, player is required
to stand in front of Kinect to collect initial data
about root and wrist joint position. Player simply
need to be in the pose such that Kinect can see
his root and wrist joint position. Player wears a
 Turning: change the wrist to left or right
while distance between the wrist and root
does not change.
 Rotation: rotate the wrist. The changed
marker in front of fist. In order to track rotation
easily, we should use marker in the form of
checker board. We define some stand distances
by the initial depth data as follows:
angle decides the speed of rotation.
 InitDistStraight = HandDepth RootDepth
These actions are determined by comparing
the position of wrist with root and rotation of
 InitDistOrth = HandX RootX
marker glued on a thumb finger. In detail,
navigation technique includes three main steps
(Fig. 2): parameter setup, hand marker tracking,
and navigation estimation. In parameter setup,
players are asked to stand or sit in front of

Kinect Based Character Navigation in VR Game

Đăng ngày | Thể loại: | Lần tải: 0 | Lần xem: 0 | Page: 8 | FileSize: M | File type: PDF
0 lần xem

Kinect Based Character Navigation in VR Game. In this paper, we propose a technique of Kinect based character navigation in VR Game. As environment is supported by Kinect, we implement character navigation by player’s hand actions. Besides, we use markers to increase the effectiveness of hand action recognition. In our experiments, character’s rotations reaches 70% in accuracy. Straight movement and turning reach 98% in accuracy.. Cũng như những tài liệu khác được thành viên giới thiệu hoặc do sưu tầm lại và chia sẽ lại cho các bạn với mục đích học tập , chúng tôi không thu phí từ thành viên ,nếu phát hiện nội dung phi phạm bản quyền hoặc vi phạm pháp luật xin thông báo cho website ,Ngoài tài liệu này, bạn có thể download Tải tài liệu luận văn,bài tập phục vụ nghiên cứu Một ít tài liệu tải về mất font không xem được, nguyên nhân máy tính bạn không hỗ trợ font củ, bạn download các font .vntime củ về cài sẽ xem được.

Nội dung