A Window-Based Approach to Training Deep Neural Networks for Predictive Sequence Modeling.

Abstract

Innovations in machine learning have stimulated extensive research interests in applied computational intelligence in the backdrop of enormous amounts of data generated in different fields of application. By its nature, such data is overwhelming and mostly go to waste due to capacity and scale restrictions of traditional statistical data modeling as well as legacy machine learning techniques. The specialized area of deep machine learning potentially holds the key to unlocking the door to modern applied computational intelligence. Presently, it is becoming progressively possible to process high volumes of data whether static or arriving in streams of varying velocities using deep learning models. Applications are innumerably many ranging from time series data modeling, signal processing, image analysis, natural language processing to object recognition among others. The critical area of predictive data modeling requires efficient and carefully selected algorithms and models for effective and accurate predictions. In this paper, we present a novel deep machine learning Neural Network for predictive tasks based on a fixed size window of time steps, tested on a well-known data set on customer arrivals to an airline. At the core of the architecture is a Multi-Layer Perceptron – a classical deep learning Neural Network optimized on a number of dimensions that include training algorithm, batch size, number of iterations, loss function among others. We finally present the experimental results and make conclusions on its applicability to such class of computational problems.