Portfolio Projects

Magical Pencil

Magical Pencil is a video game telling a story about a journey of a deserter who has a magical pencil by which whatever is drawn becomes real. In the game, players can create whatever they want by hand drawing to help the character gets to his home. The game can recognize what the player is drawing, and spawn a corresponding item in the game world, with an appearance of the player’s doodle, to interact with the player.


We all remember how cool it is when we saw Neo in Matrix saying “Guns, lots of guns.” Thus what if we get whatever we need in a video game just as simple as that? Then the purpose of the game switches from managing to obtain the game item to figure out what item is one of the solutions.

Moreover, hand drawing recognized by the game is a huge “Wow” moment, which is powered by the magic of Machine Learning. Riding on a motorcycle that was drawing by the player is another level of mind-blowing.


Most interactions in video games are done by pressing some buttons or by clicking some virtual buttons, while quite a few of them like Kinect games by gestures or other sensors. My research was focusing on new game inputs since traditional game inputs are hard to satisfy what an ITP-ish game needs. Line Rider is the first one that caught my attention, in which players can draw a slope freehandedly and the game character sleds down the slop.

Scribblenauts is a vocabulary based video game allowing the player to create any in-game objects they want by typing in the names of the items, hence solving game puzzles in creative ways.

The idea of drawing in games is intriguing because of its simplicity and expressiveness. Creating whatever players want is unusual and neat. What if I combine them, allow the player to draw whatever they need by doodling, and also give the game object an appearance identical to their sketch?

Next, I did more research on drawing related games, found out that at this time point, none of them is allowing players to create game objects by drawing freely.

When Google introduces the Quick Draw project, my mind was blown. This technology will play a crucial role in my game, so the next step is researching to figure out if Doodle Classification can be integrated into my project. Luckily, Google open sourced its Quick Draw dataset, so it is possible to train an ML model. Benefited from the Kaggle community, I did it.

Further reading

The storytelling part of this game is oriented by the game mechanism and my personality. The protagonist is a soldier but has no nation; thus I molded him a cat, to implying the detachment of our real world. Besides, he decided to be a deserter because I hate wars and the best thing I can imagine is staying with families peacefully, just like everyone on this earth.

On his journey, trails and challenges are lurking and ready to rive him in pieces. There’re mountains blocking his way, deserts roasting him, dark woods trying to trap him and home guards wanting him dead. Blessed by his faith and the magical pencil, he managed to go forward. The game includes five scenes, yet the story has not ended, but we all know he will eventually see his family again.

Parallel to the storyline, there is a level called “Another World.” How to enter it is by drawing a door, then stepping into it. This level is an empty white room, in which players can purely enjoy interacting with their own creations, and exploring various effect brought by combination of different game objects..

Technical Details

The game client is powered by the Unity Engine and the doodle recognition by Tensorflow and Google Quick Draw Dataset.
Animations are hand drawn frame by frame.


  1. Art
  2. Machine learning
  3. Narrative / Storytelling
  4. Play/Games
  5. Product Design


Leave a Reply

Your email address will not be published. Required fields are marked *