FIGURE 14 Channel-wise Saab transform – PixelHop++
Channel-wise Saab transform was applied in PixelHop as the building block, and the new architecture was named PixelHop++. The structural overview is shown in Figure 14. This technique reduced the filter size and memory requirement for filter computation. For feature learning, a novel tree-decomposed method was proposed whose leaf node provides 1D features, and the features are sorted from lowest to highest by their cross-entropy. The features of lower cross-entropy have higher discriminative power. The filter size was chosen 5x5, inherited from the channel-wise Saab architecture. The number of filters was also increased for convolution layers 6 &16, 16 &32, and 32 &64 for the first and second layers, respectively, and applied for MNIST, Fashion MNIST, and CIFAR-10 datasets. An energy-based hyperparameter Threshold (T) was added to determine the model’s size. The threshold was an excellent parameter for trading minimal efficiency with fewer connections. For example, for MNIST, the change of T from 0.005 to 0.00001, the efficiency decreased by just 0.51%, but parameters were reduced by four times. To make the PixelHop++ lighter, Enhanced-PixelHop (E-PixelHop) was proposed, in which multiple PixelHop++ units were used as the building block. To reduce the number of parameters, only two components were considered in the PCA analysis. However, there was no change in the size and number of filters from the original PixelHop++ architecture. A few other models named FaceHop, VoxelHop, and DefakeHop were proposed. However, they use Saab architecture with minor modifications, but the basic design of filters is the same.
4.3 | Contour Detection architectures