| Name: | Description: | Size: | Format: | |
|---|---|---|---|---|
| 8.03 MB | Adobe PDF |
Authors
Advisor(s)
Abstract(s)
A captura de movimento é uma técnica fundamental na área dos videojogos e cinematografia
para animar uma personagem virtual de forma realista e num curto
espaço tempo. Através de um só sensor óptico é possível captar os movimentos
do jogador para interagir com o mundo virtual, no entanto pode haver ruído durante
a captura e partes do corpo ficarem ocluídas, reduzindo a experiência do
utilizador.
Este projecto propõem uma solução para corrigir erros na captura de movimento
do sensor Microsoft Kinect, através de redes neuronais profundas (deep neural
networks), treinadas em um conjunto de poses disponibilizadas pela Carnegie
Mellon Graphics (CMU) previamente processadas. É também implementado um
filtro temporal que suaviza um movimento dado por um conjunto de poses devolvidas
pela rede neuronal.
O sistema é implementado utilizando a linguagem de programação Python, em
conjunto com a API Tensorflow para suportar as técnicas de aprendizagem automática
e o motor de jogo Unity para visualizar e interagir com os esqueletos
obtidos.
Os resultados foram avaliados através de um conjunto de métricas e um questionário
com 12 participantes.
Motion capture is fundamental technique in video games development and film production to animate a virtual character based on actor’s movements, creating more realistic animations in a short time. It is possible to detect the player’s movements through one optical sensor to interact with the virtual world, however there may be some noise and certain parts of the body become occluded, decreasing the user experience. This project present a solution to correct the motion capture errors from Microsoft Kinect sensor, through a deep neural network, trained with a pre-processed dataset of poses offered by Carnegie Mellon Graphics (CMU). A temporal filter is implemented to smooth the movement, given by a set of poses returned by the deep neural network. This system is implemented in Python with the Tensorflow API which supports the machine learning techniques and the Unity game engine to visualize and interact with the obtained skeletons. The results were evaluated with a set of metrics and a questionnaire with 12 participants.
Motion capture is fundamental technique in video games development and film production to animate a virtual character based on actor’s movements, creating more realistic animations in a short time. It is possible to detect the player’s movements through one optical sensor to interact with the virtual world, however there may be some noise and certain parts of the body become occluded, decreasing the user experience. This project present a solution to correct the motion capture errors from Microsoft Kinect sensor, through a deep neural network, trained with a pre-processed dataset of poses offered by Carnegie Mellon Graphics (CMU). A temporal filter is implemented to smooth the movement, given by a set of poses returned by the deep neural network. This system is implemented in Python with the Tensorflow API which supports the machine learning techniques and the Unity game engine to visualize and interact with the obtained skeletons. The results were evaluated with a set of metrics and a questionnaire with 12 participants.
Description
Projecto final para obtenção do grau de mestre em Engenharia de Redes de Comunicação e Multimédia
Keywords
Captura de movimento Motion capture Microsoft Kinect Denoising Aprendizagem automática Machine learning Redes neuronais Neural networks Realidade Mista Mixed Reality
Citation
GONÇALVES, André Correia - Captura de movimento em tempo real para realidade mista. Lisboa: Instituto Superior de Engenharia de Lisboa, 2020. Dissertação de mestrado.
Publisher
Instituto Superior de Engenharia de Lisboa