VNU Journal of Science: Comp. Science & Com. Eng., Vol. 32, No. 2 (2016) 23–30
Kinect Based Character Navigation in VR GameI
Thi Chau Ma, Minh Duong Hoang
VNU University of Engineering and Technology,
E3 Building, 144 Xuan Thuy Street, Cau Giay, Hanoi, Vietnam
Abstract
VR game is a hot field because it provides practical experience for the player.
However, in the game, players
are usually required attaching special equipments. Those equipments are often very expensive. Moreover, many
devices make players less flexible because they are attached to players. VR game using Kinect is a good choice
because Kinect is a ordable and not attached directly to the player.
In this paper, we propose a technique
of Kinect based character navigation in VR Game.
As environment is supported by Kinect, we implement
character navigation by player’s hand actions.
Besides, we use markers to increase the e ectiveness of hand
action recognition. In our experiments, character’s rotations reaches 70% in accuracy. Straight movement and
turning reach 98% in accuracy.
Received 30 November 2015, revised 09 January 2016, accepted 14 January 2016
Keywords:
Kinect application, hand recognition, VR games.
1. Introduction
We
are
interested
in
games
using
Kinect
In the last two years, in Vietnam, the game
attracted a lot of investments and researches.
According to [1], Vietnam is the largest market
in Southeast Asia Game. However, restrictions
still exist, obviously, the solutions are still
awaited. An important factor that make the
game to be attractive is the ability of character
movements. To control the movement of the
characters, players have to touch and navigate
specialized devices to control the movements of
the characters. These devices were typically
gamepad, keyboardist, Wiimote. Besides, with
technology advances, in virtual reality games,
players no need to touch game devices. In
these games, body actions or speech were used
to control the character movement through the
special devices, such as Kinect, Virtuix Omni.
because the type of these games gives players
the flexibility and comfort while playing without
exposure Kinect. Moreover, Kinect relatively
low cost comparing with other devices such as,
head mounted devices or data gloves. Kinect-
supported games can be classified into two main
categories. In the first type, entire bodies of
players are identified to contribute and play
games like audition games [2], or collision
ball games [3]. In the second type, hand
gestures are recognized to control games [4].
In our research, we want to make use of hand
actions obtained by Kinect to navigate character
in order to give players the freedom and comfort
in playing games. However, the accuracy of the
hand action recognition and the number of hand
actions based on Kinect were limited because of
identical when recognizing di erent bone joints.
I This work is dedicated to the 20th Anniversary of the IT
Faculty of VNU-UET.
 Corresponding author. Email: chaumt@vnu.edu.vn
In this paper, we propose a solution of Kinect
based character navigation in VR Games. We
recognize hand action to navigate characters in
23
24
T.C. Ma, M.D. Hoang / VNU Journal of Science: Comp. Science & Com. Eng., Vol. 32, No. 2 (2016) 23–30
VR Games.
Marker is glued to the hands of
to see the above Kinect applications are based on
players to obtain more number of hand actions
recognition of human skeleton.
and increase accuracy in hand action recognition.
Another type of Kinect applications is based
The use of glued makers does not reduce the
on
hand
actions
[12][13][14].
In
[12],
degree of flexibility of players.
authors accurately reconstructed complex hand
poses across a variety of subjects using Kinect.
2. Related Works
The system allowed continually recovering from
tracking failures. Matthew Tang [13] showed
With the available and a ordable Kinect,
Kinect and its applications appear a lot in the life.
First of all, we mention the applications in the
medical. KinectprovideaSoftwareDevelopment
Kit (SDK) which gives access to skeletal tracking
data and can be used directly in rehabilitation
game developments. So, there are lots of
physical therapy and rehabilitation applications
[5][6][7] using Kinect. These applications are
classified into 3 groups. Fall detection and
Fall risk reduction, Stroke Rehabilitation and
exercise games. In [8], authors presented a
low-cost system to surgical training for computer
assisted surgeries by utilizing dual sensors and
marker based tracking components. Markers
were patterns in the form of checkboard. By
using markers, they implemented calibration of
dual infrared sensors and point correspondence
establishment in a stereo configuration. Then
they accomplished 3D localization by multiple
back projection lines. The system was e ectively
thecombinationofinexpensiveelectronicdevices
and traditional tracking algorithm.
Secondly, in the VR environment, a hot and
a novel method for recognizing hand gestures
using Kinect sensor. In the approach, they
recognize ’grasp’ and ’drop’ gestures with over
90% accuracy. In her thesis [14], Yi li used
K-means clustering and Graham Scan algorithm
to recognize nine popular gestures and counting
numbers. Kinect data range was between 0.5m
to 0.8m. Hands were distinguished from the
background by depth information.
In Kinect applications in particular and
recognitions in general, Marker is an e ective
tool. Marker is a very simple sticker, usually
in the form of checker board pattern. Marker is
popular in modeling by its simple manipulation
and eciency. Checker board pattern has been
using in camera calibration [15][16][17] to
reconstruct 3D model from 2D images. Checker
board pattern is also use in Kinect calibration
[18] to align depth camera and RGB camera.
Marker help robot to detect path to navigate.
In this paper, we want use hand actions from
Kinect to control the movement of character in
VR game. We utilize marker to detect hand
actions easily and accurately.
exciting field, Kinect attracted much attention.
Kinect is one of easy ways of 3D reconstruction.
In the environment, data recognized by Kinect is
3. Proposed method
divided in two types: Human skeleton and hand
In
this
paper,
we
propose
a
Kinect-based
recognitions.
RGBDemo
[9] is an open source
navigation
technique
in
VR
game.
In
the
software providing a simple toolkit to manipulate
game,
we
use
a
Kinect
sensors
to
obtain
Kinect data.
In this software, 3D model of an
player’s hand actions.
The players navigates
object would be rebuilt when the object are taken
characters using actions only on one hand.
The
with markers.
3D builder app
[10] and Kinect
character’s actions include straight movement,
sensor are used to create a 3D reconstruction
turning (left/right) and rotation. We chose these
for a 3D printer.
Ultra Seven
[11] is a game
action in implementation because hand gestures
used Kinect in AR environment.
In the game,
are
meaningful
to
people.
So,
learning
and
the player is turned into a character and shot
using hand gestures are easy.
Moreover, in the
his power.
The character was modeled game
game, most free parts of body but hand makes
player by body recognition with Kinect.
Easy
player more comfortable.
In technique term,
T.C. Ma, M.D. Hoang / VNU Journal of Science: Comp. Science & Com. Eng., Vol. 32, No. 2 (2016) 23–30
25
the problem is more focus.
Meanings of these
actions are described clearly as follow.
Straight
movement is character goes forward/backward.
Turning is character turns left/ right. Rotation is
character rotates his body left/right or up/down to
observe the scene. This action changes eye view
to make decision of other actions.
The player’s
thumb finger is worn marker for determining
the exact location, and recognizing of complex
hand gestures as well.
In addition to character
navigation,
the
player
can
also
use
the
hand
to perform other actions in the game such as,
shooting, and selection.
Firstly, we define some parameters later used
in the process of navigation. Root is player’s
Fig. 1: Kinect skeleton.
neck joint (Fig. 1) on player’s.
Rootdepth is the
the Kinect sensor for a while to obtain some
distance from Kinect sensor to root.
Handdepth
measurements such as, root position and hand
is the distance from Kinect sensor to the wrist
depth.
These
measurements
are
utilized
to
joint of the hand.
HandX is the wrist joint in
calculate character’s speed and acceleration.
In
horizontal direction in acquired image.
RootX
hand marker tracking,
we use emguCV
[19]
is root in horizontal direction.
StraightVector
and POSIT algorithm
[20] to determine marker
is
a
vector
showing
straight
direction
of
the
and its’ rotation.
Then, above defined actions
character.
OrthVector is a vector orthogonal to
are recognized.
Finally, we perform navigation
straightVector on the left.
StraightSpeed and
estimation by recognized actions.
OrthSpeed are speed vectors in movements.
We defined three types of navigation: straight
movement, turning (left/right) and rotation
4. Implementation
corresponding to the following actions:
4.1. Parameter setup
 Straight movement: move the wrist towards
or away from the root. The change of the
distance from the wrist to the root decides
the direction (forward/backward) and speed
of movement.
At the beginning process, player is required
to stand in front of Kinect to collect initial data
about root and wrist joint position. Player simply
need to be in the pose such that Kinect can see
his root and wrist joint position. Player wears a
 Turning: change the wrist to left or right
while distance between the wrist and root
does not change.
 Rotation: rotate the wrist. The changed
marker in front of fist. In order to track rotation
easily, we should use marker in the form of
checker board. We define some stand distances
by the initial depth data as follows:
angle decides the speed of rotation.
 InitDistStraight = HandDepth RootDepth
These actions are determined by comparing
the position of wrist with root and rotation of
 InitDistOrth = HandX RootX
marker glued on a thumb finger. In detail,
navigation technique includes three main steps
(Fig. 2): parameter setup, hand marker tracking,
and navigation estimation. In parameter setup,
players are asked to stand or sit in front of
 HandRangeStraight
Min(HandDepth)
 HandRangeOrth
Min(HandX)
=
=
Max(HandDepth)
Max(HandX)
-
-
HƯỚNG DẪN DOWNLOAD TÀI LIỆU

Bước 1:Tại trang tài liệu slideshare.vn bạn muốn tải, click vào nút Download màu xanh lá cây ở phía trên.
Bước 2: Tại liên kết tải về, bạn chọn liên kết để tải File về máy tính. Tại đây sẽ có lựa chọn tải File được lưu trên slideshare.vn
Bước 3: Một thông báo xuất hiện ở phía cuối trình duyệt, hỏi bạn muốn lưu . - Nếu click vào Save, file sẽ được lưu về máy (Quá trình tải file nhanh hay chậm phụ thuộc vào đường truyền internet, dung lượng file bạn muốn tải)
Có nhiều phần mềm hỗ trợ việc download file về máy tính với tốc độ tải file nhanh như: Internet Download Manager (IDM), Free Download Manager, ... Tùy vào sở thích của từng người mà người dùng chọn lựa phần mềm hỗ trợ download cho máy tính của mình  

Kinect Based Character Navigation in VR Game

Đăng ngày | Thể loại: | Lần tải: 0 | Lần xem: 4 | Page: 8 | FileSize: 0.00 M | File type: PDF
4 lần xem

Kinect Based Character Navigation in VR Game. In this paper, we propose a technique of Kinect based character navigation in VR Game. As environment is supported by Kinect, we implement character navigation by player’s hand actions. Besides, we use markers to increase the effectiveness of hand action recognition. In our experiments, character’s rotations reaches 70% in accuracy. Straight movement and turning reach 98% in accuracy..

Nội dung

VNU Journal of Science: Comp. Science & Com. Eng., Vol. 32, No. 2 (2016) 23–30 Kinect Based Character Navigation in VR GameI Thi Chau Ma, Minh Duong Hoang VNU University of Engineering and Technology, E3 Building, 144 Xuan Thuy Street, Cau Giay, Hanoi, Vietnam Abstract VR game is a hot field because it provides practical experience for the player. However, in the game, players are usually required attaching special equipments. Those equipments are often very expensive. Moreover, many devices make players less flexible because they are attached to players. VR game using Kinect is a good choice because Kinect is aordable and not attached directly to the player. In this paper, we propose a technique of Kinect based character navigation in VR Game. As environment is supported by Kinect, we implement character navigation by player’s hand actions. Besides, we use markers to increase the eectiveness of hand action recognition. In our experiments, character’s rotations reaches 70% in accuracy. Straight movement and turning reach 98% in accuracy. Received 30 November 2015, revised 09 January 2016, accepted 14 January 2016 Keywords: Kinect application, hand recognition, VR games. 1. Introduction We are interested in games using Kinect In the last two years, in Vietnam, the game attracted a lot of investments and researches. According to [1], Vietnam is the largest market in Southeast Asia Game. However, restrictions still exist, obviously, the solutions are still awaited. An important factor that make the game to be attractive is the ability of character movements. To control the movement of the characters, players have to touch and navigate specialized devices to control the movements of the characters. These devices were typically gamepad, keyboardist, Wiimote. Besides, with technology advances, in virtual reality games, players no need to touch game devices. In these games, body actions or speech were used to control the character movement through the special devices, such as Kinect, Virtuix Omni. I This work is dedicated to the 20th Anniversary of the IT Faculty of VNU-UET. Corresponding author. Email: chaumt@vnu.edu.vn because the type of these games gives players the flexibility and comfort while playing without exposure Kinect. Moreover, Kinect relatively low cost comparing with other devices such as, head mounted devices or data gloves. Kinect-supported games can be classified into two main categories. In the first type, entire bodies of players are identified to contribute and play games like audition games [2], or collision ball games [3]. In the second type, hand gestures are recognized to control games [4]. In our research, we want to make use of hand actions obtained by Kinect to navigate character in order to give players the freedom and comfort in playing games. However, the accuracy of the hand action recognition and the number of hand actions based on Kinect were limited because of identical when recognizing dierent bone joints. In this paper, we propose a solution of Kinect based character navigation in VR Games. We recognize hand action to navigate characters in 23 24 T.C. Ma, M.D. Hoang / VNU Journal of Science: Comp. Science & Com. Eng., Vol. 32, No. 2 (2016) 23–30 VR Games. Marker is glued to the hands of to see the above Kinect applications are based on players to obtain more number of hand actions and increase accuracy in hand action recognition. recognition of human skeleton. Another type of Kinect applications is based The use of glued makers does not reduce the on hand actions [12][13][14]. In [12], degree of flexibility of players. 2. Related Works With the available and aordable Kinect, Kinect and its applications appear a lot in the life. First of all, we mention the applications in the medical. KinectprovideaSoftwareDevelopment Kit (SDK) which gives access to skeletal tracking data and can be used directly in rehabilitation game developments. So, there are lots of physical therapy and rehabilitation applications [5][6][7] using Kinect. These applications are classified into 3 groups. Fall detection and Fall risk reduction, Stroke Rehabilitation and exercise games. In [8], authors presented a low-cost system to surgical training for computer assisted surgeries by utilizing dual sensors and marker based tracking components. Markers were patterns in the form of checkboard. By using markers, they implemented calibration of dual infrared sensors and point correspondence establishment in a stereo configuration. Then they accomplished 3D localization by multiple back projection lines. The system was eectively thecombinationofinexpensiveelectronicdevices and traditional tracking algorithm. Secondly, in the VR environment, a hot and exciting field, Kinect attracted much attention. Kinect is one of easy ways of 3D reconstruction. In the environment, data recognized by Kinect is authors accurately reconstructed complex hand poses across a variety of subjects using Kinect. The system allowed continually recovering from tracking failures. Matthew Tang [13] showed a novel method for recognizing hand gestures using Kinect sensor. In the approach, they recognize ’grasp’ and ’drop’ gestures with over 90% accuracy. In her thesis [14], Yi li used K-means clustering and Graham Scan algorithm to recognize nine popular gestures and counting numbers. Kinect data range was between 0.5m to 0.8m. Hands were distinguished from the background by depth information. In Kinect applications in particular and recognitions in general, Marker is an eective tool. Marker is a very simple sticker, usually in the form of checker board pattern. Marker is popular in modeling by its simple manipulation and eciency. Checker board pattern has been using in camera calibration [15][16][17] to reconstruct 3D model from 2D images. Checker board pattern is also use in Kinect calibration [18] to align depth camera and RGB camera. Marker help robot to detect path to navigate. In this paper, we want use hand actions from Kinect to control the movement of character in VR game. We utilize marker to detect hand actions easily and accurately. 3. Proposed method divided in two types: Human skeleton and hand In this paper, we propose a Kinect-based recognitions. RGBDemo [9] is an open source navigation technique in VR game. In the software providing a simple toolkit to manipulate game, we use a Kinect sensors to obtain Kinect data. In this software, 3D model of an player’s hand actions. The players navigates object would be rebuilt when the object are taken characters using actions only on one hand. The with markers. 3D builder app [10] and Kinect character’s actions include straight movement, sensor are used to create a 3D reconstruction turning (left/right) and rotation. We chose these for a 3D printer. Ultra Seven [11] is a game action in implementation because hand gestures used Kinect in AR environment. In the game, are meaningful to people. So, learning and the player is turned into a character and shot using hand gestures are easy. Moreover, in the his power. The character was modeled game game, most free parts of body but hand makes player by body recognition with Kinect. Easy player more comfortable. In technique term, T.C. Ma, M.D. Hoang / VNU Journal of Science: Comp. Science & Com. Eng., Vol. 32, No. 2 (2016) 23–30 25 the problem is more focus. Meanings of these actions are described clearly as follow. Straight movement is character goes forward/backward. Turning is character turns left/ right. Rotation is character rotates his body left/right or up/down to observe the scene. This action changes eye view to make decision of other actions. The player’s thumb finger is worn marker for determining the exact location, and recognizing of complex hand gestures as well. In addition to character navigation, the player can also use the hand to perform other actions in the game such as, shooting, and selection. Firstly, we define some parameters later used in the process of navigation. Root is player’s neck joint (Fig. 1) on player’s. Rootdepth is the distance from Kinect sensor to root. Handdepth is the distance from Kinect sensor to the wrist joint of the hand. HandX is the wrist joint in horizontal direction in acquired image. RootX is root in horizontal direction. StraightVector is a vector showing straight direction of the character. OrthVector is a vector orthogonal to straightVector on the left. StraightSpeed and OrthSpeed are speed vectors in movements. We defined three types of navigation: straight movement, turning (left/right) and rotation corresponding to the following actions: Straight movement: move the wrist towards or away from the root. The change of the distance from the wrist to the root decides the direction (forward/backward) and speed of movement. Turning: change the wrist to left or right while distance between the wrist and root does not change. Rotation: rotate the wrist. The changed angle decides the speed of rotation. These actions are determined by comparing the position of wrist with root and rotation of Fig. 1: Kinect skeleton. the Kinect sensor for a while to obtain some measurements such as, root position and hand depth. These measurements are utilized to calculate character’s speed and acceleration. In hand marker tracking, we use emguCV [19] and POSIT algorithm [20] to determine marker and its’ rotation. Then, above defined actions are recognized. Finally, we perform navigation estimation by recognized actions. 4. Implementation 4.1. Parameter setup At the beginning process, player is required to stand in front of Kinect to collect initial data about root and wrist joint position. Player simply need to be in the pose such that Kinect can see his root and wrist joint position. Player wears a marker in front of fist. In order to track rotation easily, we should use marker in the form of checker board. We define some stand distances by the initial depth data as follows: InitDistStraight = HandDepth RootDepth InitDistOrth = HandX RootX marker glued on a thumb finger. In detail, navigation technique includes three main steps (Fig. 2): parameter setup, hand marker tracking, and navigation estimation. In parameter setup, players are asked to stand or sit in front of HandRangeStraight = Min(HandDepth) HandRangeOrth = Min(HandX) Max(HandDepth) - Max(HandX) - 26 T.C. Ma, M.D. Hoang / VNU Journal of Science: Comp. Science & Com. Eng., Vol. 32, No. 2 (2016) 23–30 into body parts, this information is pooled to generate the position of 3D skeletal joints. The meanshiftalgorithmisusedtocomputemodesof probabilitydistributions. Byusingthisalgorithm, the densest region of body part is extracted. For more detail information, please read [21]. 4.2.2. Marker tracking The marker detection has three main steps. First, the contour detection algorithm[22] is used to detect all quadrangle. To normalize the marker image, the input image must be unwarp by using perspective transformation. The estimated transformation will transform the marker the square form to extract data from it. In the Fig. 2: Overview method process. 4.2. Hand and marker tracking implementation, we use simple marker 5x5 bit. For each normalized quadrangle, the image is dividedintocells. Thedataofeachcellisdecided by the number of white pixels; if the number of white pixels is greater than the number of black pixels then the cell’s value is 1, otherwise, Hand tracking is processed by using Kinect the value is 0. After comparing the candidate SDK. To detect marker, we process RGB image from Kinect to get contours fitting for 4 vertices. After that, camera is calibrated in stardard form, and data of marker is extracted in the form of a 2D matrix. 4.2.1. Skeleton tracking The skeleton tracking feature of Kinect is used to get hand position. This process has two stage: first computes a depth map (using structed light), then infers body position (using machine learning). At first stage, the depth images are all computed by the PrimeSense hardware built into Kinect. Pixels in a depth image indicated calibrated depth in the scene, rather than a measure of intensity of color. Depth images oer several benefits, such as working in low light levels, giving a calibrated scale estimate, being color and texture invariant and resolving silhouette ambiguities in pose. The depth image received from previous stage is transformed to body part image by using marker’s value with the real marker’s value, the marker is recognized. 4.3. Navigation estimation 4.3.1. Forward, backward When playing, player do hand actions. While distance between wrist joint changes, the character moves straight away. If the distance increases, character moves forward. Otherwise, character moves backward. The estimataion of straight speed is shown in Algorithm 1. At each frame, the depth distance between Hand and Root depthDistance is calculated and the dierence between current pose and initial pose rawVertical is obtained. If rawVertical is greater than minimum distance minDeltaStraight, rawVertical is converted into range [0;1] and assigned to scaledVertical. Otherwise, scaledVertical is 0. After that, the speed vector of forward movement is calculated. 4.3.2. Left, right randomized decision forest. This tree is learned We use horizontal distance between root from 100,000 depth images with known skeleton and wrist joint to control character turning and theirs derivations. After classifying image (Algorithm 2). T.C. Ma, M.D. Hoang / VNU Journal of Science: Comp. Science & Com. Eng., Vol. 32, No. 2 (2016) 23–30 27 Algorithm 1: Estimate forward/backward 1 depthDistance RootDepth - HandDepth 2 rawVertical (depthDistance InitDistStraight) 3 if rawVertical > minDeltaStraight then 4 scaledVertical rawVertical / HandRangeStraight 5 else 6 scaledVertical 0 7 StraightSpeed StraightVector playerMaxS peed scaledVertical Algorithm 2: Estimate Left/right movement 1 orthDistance HandX - RootX 2 rawHorizontal orthDistance InitDistOrth 3 if rawHorizontal > minDeltaOrth then 4 scaledHorizontal rawHorizontal / HandRangeOrth 5 else 6 scaledHorizontal 0 7 OrthoSpeed OrthoVector playerMaxS peed scaledHorizontal The horizontal distance between Hand and translation matrix HomoMat is calculated by and Root orthDistance is obtained and the coplanar POSIT algorithm. After extract yaw, dierence between current pose and initial pose rawHorizontal is calculated by subtract current orthDistance to InitDistOrth. If rawHorizontal is greater than minimum distance minDeltaOrth, rawHorizontal is converted into range [0;1] and assigned to scaledHorizontal. Otherwise, scaledHorizontal is 0. After that, the speed vector of left movement is calculated. 4.3.3. Rotation Player rotates the wrist to control character rotation. The marker is rotated follow the hand. By using Coplanar POSIT algorithm [20], the estimated angle can be inferred into 3 axis angles (ie. yaw, pitch, roll). The estimated roll angle is used to rotate left, right, and the pitch angle decide to rotate player up or down. After that, each frame add amount of angle to user’s quaternion to change the rotation angle of player (Algorithm 3). pitch, roll angle, the markerState is checked. If marker is in FRONT state and roll angle is greater than minimum angle to rotate minAngleX, axisX is obtained by normalize roll angle into range [0;1]. If marker is in LEFT, RIGHT state, the axisX is respectively 1 and 1. The up/down rotation is measured by using pitch angle instead of yaw angle because of the instability of yaw angle estimation. If pitch angle is too small then player does not rotate. Because the value of pitch is the same when marker rotate up and down, the value of roll and yaw is checked to infer whether player want to rotate up or down. If roll and yaw less than 90, player wants to rotate up; and otherwise, player rotates down. After normalized axisX and axisY, the rotation speed of each frame angleX and angleY are calculated. 5. Evaluation Firstly, the left/right rotation is measured. To prove the eectiveness of the proposed By tracking marker, the marker rotation state method, we experimented on several routes markerState is estimated. The estimated rotation (Fig. 3). These routes are in the shapes of lines, 28 T.C. Ma, M.D. Hoang / VNU Journal of Science: Comp. Science & Com. Eng., Vol. 32, No. 2 (2016) 23–30 Algorithm 3: Estimate rotation 1 axisX 0, axisY 0 2 markerState Extract(markerData) % detect marker on thumb figure % 3 HomoMat CoPOSIT(source;des) % extract homography matrix % 4 (yaw; pitch;roll) Extract(HomoMat) % determine yaw, pitch or roll % 5 if markerState==FRONT then 6 if Abs(roll) > minAngleX then 7 axisX roll=maxAngleX 8 else Fig. 3: A test map. 9 axisX 0 10 else 11 if markerState==LEFT then 12 axisX 1 13 else 14 axisX 1 15 pitch pitch+90 16 if pitch < minAngleY then 17 axisY 0 18 else 19 if roll < 90 and yaw < 90 then 20 axisY Abs(pitch)=maxAngleY 21 else 22 axisY Abs(pitch)=maxAngleY Fig. 4: A test. backward-left, etc. According to table 2, the accuaracy of up/down rotation is lowest because of shadow and errors of estimation algorithm. Each test is defined by a sequence of movements and rotations. An error is defined 23 angleX 24 angleY axisX damplingTimeFrame axisY damplingTimeFrame as follow: ”when user sends control a signal, the responding result is not corrected or the responding time is over a time threshold, then an error is occurred”. In order to calculate triangulations, rectangles, hexagons. Each route estimatedminimumnumberofmovementactions (ie. forward, backward, left, right) and rotation (ie. left/right, up/down) actions to complete each test (Fig. 4). We compare the time to complete test of this method to keyboard/mouse method (Table1). Though,ourproposal’sruntimehasnot reached those of using keyboard/mouse. In small games including mainly character’s movement, our runtime is acceptable. The proposed method works well with simple movement action, such as forward, backward, left, right; however, the accuracy of movement is decrease a bit with the accuracy, we experimented each test several (around 20) times. The experimenters do the sequence of actions as definition of the test in order to reach the final destination. For each sending signal, if the responding movement/rotation is incorrect or the responding time is too long (the time threshold is 1 second), then it is counted as an error. The accuracy is calculated by the percentage of the number of corrected actions overall the number of all actions,wherenumberofcorrectedactionsequals to the subtraction between the number of actions and the number of errors. We applied Kinect based character navigation compound movement actions like forward-left, in Maze Game and Haunted House Game. In T.C. Ma, M.D. Hoang / VNU Journal of Science: Comp. Science & Com. Eng., Vol. 32, No. 2 (2016) 23–30 29 Maze, we navigate character finding his way to some realtime games such as finding short route get destination. In Haunted House Game, we in Mazes, finding way to escape the haunted navigate character escaping from a ghost’s chase. house. In the near future, we study deeply recognition Table 1: Comparison of detection time reductions techniques recognizing to enhance the accuracy of rotation, especially the accuracy Route name Movement Test Rotate Left/Rightt Rotate Up/Down Triangle Rectangular Hexagons Movement / Rotation 12 / 0 0 / 2 0 / 2 2 / 2 4 / 4 8 / 8 Time (keyboard/ our method) 55:3s=38:2s 6:2s=4:6s 20:1s=4:7s 44:8s=21:1s 25:6s=16:5s 34:0s=20:1s of rotate up/down. We also think about how to improve running time to apply our method in many kinds of VR games. Beside that, more gestures can be defined to control not only the navigation but also game actions (ex. jumping, shooting, interacting, etc.) by using hand finger. Moreover, by using Kinect, the action in game can be defined by body action; this makes the games more interactive and interesting. Acknowledgments This work has been supported by VNU Table 2: Accuracy of character navigation University of Engineering and Technology, under Project No. CN.15.02 Route name Movement Rotation Error Movement 12 0 3% Test Rotate 0 2 2% Left/Rightt Rotate 0 2 30% Up/Down Triangle 2 2 4% Rectangular 4 4 2% Hexagons 8 8 3% 6. Conclusions In the paper, we proposed a technique to navigate characters by hand actions in VR game. A Kinect was used to obtain the hand actions. In this technique, we also suggested using the marker attached to the player’s hand to increase the accuracy of hand action recognition. So, we not only take advantage of low-cost device but also enhance flexibility for players. accuracy of straightmovementisupto98%. Rotateleft/right test also gives high accuracy, but rotate up/down have low accuracy. We deploy our solution in References [1] http://www.vnu-itp.edu.vn/en/news/market/178-viet-nam-hien-nay-la-thi-truong-game-lon-nhat-khu-vuc-dong-nam-a.html. [2] D. C. (series), Mtv games, Microsoft Game Studios, 2011. [3] Kinect adventures, . Microsoft Game Studios, 2010. [4] http://www.metacritic.com/game/xbox-360/angry- birds-trilogy. [5] A. M. L. D. C. V. L. M. Khademi, H. Mousavi Hondori, S. C. Cramer, Comparing direct and indirect interaction in stroke rehabilitation, .in Proceedings of the Extended Abstracts of the 32nd Annual ACM Conference on Human Factors in Computing Systems 2014 1639–1644. [6] L. D. S. C. M. Khademi, H. M. Hondori, C. V. Lopes, Comparing ’pick and place’ task in spatial augmentedrealityversusnon-immersivevirtualreality for rehabilitation setting, .inAnnual International Conference of the IEEE Engineering in Medicine and Biology Society 2013 4613–4616. [7] H. M. Hondori, M. Khademi., A review on technical and clinical impact of microsoft kinect on physicaltherapyandrehabilitation,JournalofMedical Engineering 2014. [8] W. L. Hongliang Ren, A. Lim, Marker-based surgical instrument tracking using dual kinect sensors, IEEE transactions on automation science and engineering. 2014. [9] http://rgbdemo.org/index.php. 30 T.C. Ma, M.D. Hoang / VNU Journal of Science: Comp. Science & Com. Eng., Vol. 32, No. 2 (2016) 23–30 [10] http://windows.microsoft.com/en-us/windows-8/apps-windows-store-tutorial. [11] https://code.google.com/p/kinect-ultra/. [12] T. Sharp, et. al., Accurate, robust, and flexible real-time hand tracking, Best of CHI Honorable Mention Award 2012. [13] M. Tang, Recognizing hand gestures with microsoft’s kinect, CS228 2011. [14] Y. Li, Hand gesture recogniton using kinect, Master thesis 2012. [15] Z. Zhang, A flexible new technique for camera calibration,IEEETransactionsonPatternAnalysisand Machine Intelligence 22 (2000) 1330–1334. [16] P. F. Sturm, S. J. Maybank, On plane-based camera calibration: A general algorithm, singularities, applications, International Conference on Computer Vision 1999. [17] J. Heikkila, O. Silven, A four-step camera calibration procedure with implicit image correction, CVPR, IEEE Computer Society 1997 1106–1112.