1. Voice Acquisition Module: In this module, the microphone audio input is acquired and expressed as a digitized voice signal. For example, a 16-bit digital voice signal with a 16k sampling rate means that each second of speech is represented as 16,000 16-bit integers.
2. Feature Extraction Module: This module is primarily tasked with converting the digital voice signal into a feature vector and supplying it to the acoustic model for processing. The most common acoustic features extracted from speech input signals are Mel-Frequency Cepstral Coefficients, or MFCCs, although there are other methods, including Linear Predictive Coding (LPC) and Perceptual Linear Prediction (PLP).
   
3. Acoustic Model: The acoustic model represents the relationship between the audio signal and basic units of the language (feature -> phoneme) to reconstruct what was actually uttered. Traditionally, speech recognition used Hidden Markov Model-Gaussian Mixture Model (HMM-GMM), but in recent years, many systems have used Hidden Markov Model-Deep Neural Network (HMM-DNN) or other improved models.
   
4. Lexicon: The lexicon or pronunciation dictionary contains all the words and pronunciations that can be handled by a speech recognition system. It provides data about words and their pronunciations, and links acoustic model and language model.
   
5. Language Model: The language model constrains the decoder's choice of words to those that are most likely to make sense. It predicts the next words based on the last words. Statistical language models (typically n-grams) are compiled from large corpora of text, usually from specific "domains" or subject areas.
   
6. Decoder: The decoder is tasked with receiving the input feature vector and searching for the word string that most probably outputs this feature vector based on the acoustic and language models. Generally, this search process is completed using a beam search-based Viterbi algorithm.

1: Deeper Networks

Deeper networks lead to improved inference accuracies.

Deeper networks means increasing the number of parameters, incresing the model sizes.

2: Mathematical Transforms

Mathematical transforms lead to optimizations. For example, Winograd transformations can be applied to 3x3 filters. Fast Fourier Transforms (FFT) have been shown to be amenable to larger filters (5x5 and above).

3: Compact Data Types

Many researchers have shown that representing data in less than 32 bit (FP32) leads to only a small reduction in accuracy.
The lates GPUs provides support for FP16 and INT8. Research in binarized neural networks (BNN) used 1-bit data types, restricted to +1 or -1.

There are also work in ternary neural networks (TNN) with weights constrained to +1, 0 and -1.

4: Exploiting Sparsity

import numpy as np
import matplotlib.pyplot as plot
time        = np.arange(0, 10, 0.01);
amplitude   = 65536*np.sin(time)
clipped     = np.clip(amplitude, -32768, 32767)
#aliased    = np.where(amplitude>32767,amplitude-65536,amplitude)
#aliased2   = np.where(aliased<-32768,aliased+65536,aliased)
print (clipped)
# Plot a sine wave using time and amplitude obtained for the sine wave
plot.plot(time, clipped)
plot.grid(True, which='both')
plot.axhline(y=0, color='k')
plot.show()