Meta-Informed Output Initialization (MIOI): A Novel Context-Aware Deep Learning Architecture with Adaptive Weight Initialization

Deep learning models have been extremely successful in many domains, however, these methods still suffer from expensive training costs, slow convergence rates, as well as lack of generalization to different data source. In this paper, we propose a new Meta-Informed Output Initialization (MIOI) which is a context-aware deep learning model that utilizes dataset meta-information to influence the weight initialization and adaptive training. Our method will derive structural properties from the input, such as entropy content, variance, sparsity and statistical moments, that can be exploited to learn better neural network initialization and optimization. We test MIOI on three domains: face detection with the WIDER FACE dataset, medical image classification with ChestX-ray14 and natural language processing sentiment analysis on IMDB. Extensive experiments show that MIOI consistently surpasses the state-of-the-art baselines with accuracy gains of 3.8-5.8% at the cost of a 54-58% reduction of the training time. Statistical significance testing ensures these improvements are significant (p < 0.001). And the proposed method has exhibited strong cross-domains generalization, which can be applied to various woodworking machine learning tasks where training efficiency and effectiveness are required.