Responsibilties:
Solo Project
Project dates:
August 2022 - Present
Tools:
Unity
ONNX Neural Networks
Platforms:
Oculus Quest 2
Arcane Algebra is a first-person virtual-reality game designed for students who are beginning to learn basic algebraic operators (Addition, Multiplication, and Subtraction). Using their controller as a wand, players cast mathematical spells to defeat the group of angry goblins attacking the village.
Integrating Learning Into Gameplay
Math Agar
As technology continues to embed itself in society, kids are being exposed to new an exicting ways to learn. They can travel to far off places using VR, and visualize complex 3D models with ease. Webites like CoolMathGames.com are used as fun replacements for the tedious repition of memorizing mathematical rules, and iPads are becoming more commonly available in most schools.
Yet, from my experience playing these games, I haven’t found much excitement in playing them. Instead of being transported to a new realm with each game, it feels like you’re doing Math, but in a new arbitrary setting that the game sets up for you. An easy example of this is Math Agar, a reskinned version of the popular web game agar.io. Yet where Math Agar differs from it’s more popular counterpart, is the addition of an equation. Eating special orbs with the answer to this equation will grant you extra mass, while eating an incorrect answer will decrease your size.
While this may be fun for the first few minutes, it quickly becomes apparent that Math Agar can never equate to it’s original inspiration. It doesn’t feel like the Math aspect has been correctly integrated into the game. You’re not playing a Math game, you’re just playing a game and also doing mathematics on top of it. There isn’t any integration between the two. Not to mention the fact that kids can simply avoid doing any math, as there is no explicit penalty to not going after those special orbs!
An ogre from Timez Attack
Yet, educational games don’t have to be like this at all. Growing up I had the chance to play a 2007 game called Timez Attack.
Timez Attack uses the natural progression of learning a multiplication table (starting at lower numbers and adding higher ones slowly) and maps it to a progression of levels. Each level is centered around multiplication by a certain number (starting with 1, then 2, and so on). Players have to fight 12 different ogres, and a boss ogre, on these levels to help them learn and memorize their mutiplication tables.
This adds a larger sense of progression into the game, as each test of multiplication is a reenforcement of previous tests.
Timez Attack also provides students with different ways to visualize the problems they are asked to solve. Before fighting each ogre in a level, players are provided with a visualization of the problem they’re asked to solve. This is done through a locked door, that each ogre sits behind. Each one of these doors has a formula marked on it (2x3 for example). Before that door can unlock, players must run around and catch 3 groups of snails, giving them a visual of what 2x3 actually represents. This “catching-phase” further reinforces the learning process, and adds diversity into the core gameplay loop.
Handwriting Classification
Confidence Interval for a hand-drawn 4
For Arcane Algebra, I wanted to utilize the immersive capabilities of VR to make players feel powerful while writing out their solutions. To do this, I used a convolutional neural network model trained on the MNIST dataset, to parse hand-written digits.
When given a 26x26 greyscale image of a number, the model will return a confidence value of what that number might be. By finding the value with the heighest confidence interval, we can fairly accurately interpret a user’s handwriting.
By projecting the set of world coordinates that make up the player’s gesture onto a 2D plane, and then scaling them to fit onto a 26x26 texture, I was able to map gestures into images the the MNIST model could understand.
I did initially run into a lot of issues with the accuracy of the model, despite it having a 99% accuracy rate with the training data. I intended on disgusing this inaccuracy as an “unstable spell”, which would occur when the model did not meet a certain level of confidence for a number. When players cast an unstable spell, the monster they were facing would be sent backwards, giving them additional time to try writing their solution again.
Thankfully, with guidance from some of my peers, I found that this inaccuracy was caused from the way my images were being created. While my images had thick lines and often took up the whole image, the training dataset always had the image centered and written with thin lines. This difference in how the data was presented caused the model to behave unpredictably. Yet, it was easily resolved by adding some padding and decreasing the stroke width of the drawings.
Turning a hand-drawn number into a texture, and then running it through the MNIST model
Moving Forward
As I continue to work on Arcane Algebra, I want to make sure I’m putting an emphasis on supporting different learning styles. I think something that Timez Attack excels in is in the different ways it presents problems to players. No one processes things the same way, so we can’t expect to have one game that works for everyone.
As such, I’m interested in finding new ways to present information to students, and making the learning process more enjoyable for everyone.
I’d love to play around different modes for each one of the mathematical operations. Maybe a fruit-ninja-inspired mode for visualizing divisions, or a real-time-strategy minigame focused on multiplying your troops to form larger ones. The posibilities are endless!
