Capstone Project Deep Gaming

PROJECT OVERVIEW

DeepGaming - Browser Plugin for Playing Gomoku

Artificial intelligence is becoming more and more prominent in the gaming industry since IBM’s Deep Blue famously beat Garry Kasparov in chess. The recent development of AlphaGo by Alphabet’s Google DeepMind beat Lee Sdeol in March 2016 for the first time.

DeepGaming for Gomoku is an exploratory artificial intelligence board game player that integrates with your browser which plays Gomoku (and in the future can be extended to play other games) on the popular BoardGameArena website. The technology it leverages are similar in concept to those built by other industry leading companies. In particular, it leverages reinforcement learning algorithms with neural networks, where the AI plays against itself to improve its techniques in the game by playing against itself. Feature engineering is used to narrow down and simplify the complexities of the board game states.

We look forward to extending this to future games.

Goals

Build end to end product
Learn about state-of-the-art reinforcement learning techniques
Lay groundwork for generic AI framework

Impact

This is fun!
Self education
Extension can be applied to any other game on BoardGameArena site

SYSTEM DESIGN

We chose to build a browser plug-in so there won't be any policy violation on BGA site. On frontend the user with the plug-in will play with another user in real time. On backend, the system will pass current game state to the AI, AI will predict the next move and then pass it back to the game board.

MODEL

Q Learning Approach

AI employs reinforcement learning algorithm along with neural networks
Feature engineering of board game state to reduce noise and improve training
Reduction of number of states in the action space to general moves such as attack and defend to optimize performance and training
Self play algorithm allows AI to play with itself to improve artificial intelligence

EVALUATION

Benchmark model performance against random play

Addition of low reward on ties (e.g. +10) has adverse effects on performance
Removal of rewards on ties allows model to focus on winning instead of tying the games
Tested against a 5x5 grid (model is scalable) allows for deeper analysis of results and model performance

(/100 games)	No reward on ties	Positive reward on ties
Wins	83	35
Losses	5	5
Draws	12	60

OTHER GAME

One of our goals was to build a generic AI framework for board games: Given the right agent, our browser extension code and system design should be extensible to any of the110+ different games in BoardGameArena’s catalog.

Although we primarily focused only on Gomoku during our project, we tested the extensibility of our browser extension and system design by applying it to the game of Hive on BoardGameArena. We chose the game of Hive because A) none of the members of our team knew the rules or how to play, and B) we found a number of good Hive AIs on GitHub. We chose to go with https://github.com/cmelchior/hivemind because it had the highest star rating.

With this AI, we built a working Hive browser extension even when we ourselves possessed little knowledge of the rules of Hive on our part. The development took only a fraction of the time it took to make the original Gomoku extension because we were able to reuse our code. We imagine that development of browser extensions for other games on BoardGameArena will also be relatively straightforward, provided we have the right AI for each game.

OUR TEAM

Chan, Ying

Lane, Jackson

Liu, Dongzi

Song, Minghu