MXNet: Efficient and Flexible Deep Learning
This is a project created in collaboration with researchers from CMU, NYU, NUS, MIT and developed with many others.
MXNet stands for mix and maximize. The idea is to combine the power of declartive programming together with imperative programming. In its core, a dynamic dependency scheduler that automatically parallelizes both symbolic and imperative operations on the fly. A graph optimization layer on top of that makes symbolic execution fast and memory efficient. The library is portable and lightweight, and it scales to multiple GPUs and multiple machines.
Reference
Tianqi Chen, Mu Li, Yutian Li, Min Lin, Naiyan Wang, Minjie Wang, Tianjun Xiao, Bing Xu, Chiyuan Zhang, and Zheng Zhang. MXNet: A Flexible and Efficient Machine Learning Library for Heterogeneous Distributed Systems. In Neural Information Processing Systems, Workshop on Machine Learning Systems, 2015
Backbone Projects
MXNet as well some other large-scale scalable machine learning projects are backed by the following backbone libraries, which is also created and mainly developed by researchers from UW.
MShadow: A Unified CPU/GPU Matrix Template Library in C++/CUDA
In order to support deep learning, we need to be able to write efficient computations on GPU. MShadow is an efficient, device invariant and simple tensor library for machine learning project that aims for both simplicity and performance. Basically, it allows you to write expressions and translate them to GPU code during compilation. This is the backbone library behind many deep learning platforms, including MXNet, CXXNet and Apache Singa.
DMLC-Core: Distributed Machine Learning Common Codebase
When you write distributed machine learning. There are many common non-trivial utility functions that you need. Load the data from distributed filesystem in a sharded way; parse input file fast enough so your bigdata program do not load too slowly; launch jobs on various environment; define and check parameters. dmlc-core is the C++ library that solves all these common pains in distributed machine learning.