FIGURE
14 Channel-wise Saab transform – PixelHop++
Channel-wise Saab transform was applied in PixelHop as the building
block, and the new architecture was named PixelHop++. The structural
overview is shown in Figure 14. This technique reduced the filter size
and memory requirement for filter computation. For feature learning, a
novel tree-decomposed method was proposed whose leaf node provides 1D
features, and the features are sorted from lowest to highest by their
cross-entropy. The features of lower cross-entropy have higher
discriminative power. The filter size was chosen 5x5, inherited from the
channel-wise Saab architecture. The number of filters was also increased
for convolution layers 6 &16, 16 &32, and 32 &64 for the first and
second layers, respectively, and applied for MNIST, Fashion MNIST, and
CIFAR-10 datasets. An energy-based hyperparameter Threshold (T) was
added to determine the model’s size. The threshold was an excellent
parameter for trading minimal efficiency with fewer connections. For
example, for MNIST, the change of T from 0.005 to 0.00001, the
efficiency decreased by just 0.51%, but parameters were reduced by four
times. To make the PixelHop++ lighter, Enhanced-PixelHop (E-PixelHop)
was proposed, in which multiple PixelHop++ units were used as the
building block. To reduce the number of parameters, only two components
were considered in the PCA analysis. However, there was no change in the
size and number of filters from the original PixelHop++ architecture. A
few other models named FaceHop, VoxelHop, and DefakeHop were proposed.
However, they use Saab architecture with minor modifications, but the
basic design of filters is the same.
4.3 | Contour Detection architectures