Filters in convolutional networks are typically parameterized in a pixel basis, that does not take prior knowledge about the visual world into account. We investigate the generalized notion of frames designed with image properties in mind, as alternatives to this parametrization. We show that frame-based ResNets and Densenets can improve performance on Cifar-10+ consistently, while having additional pleasant properties like steerability. By exploiting these transformation properties explicitly, we arrive at dynamic steerable blocks. They are an extension of residual blocks, that are able to seamlessly transform filters under pre-defined transformations, conditioned on the input at training and inference time. Dynamic steerable blocks learn the degree of invariance from data and locally adapt filters, allowing them to apply a different geometrical variant of the same filter to each location of the feature map. When evaluated on the Berkeley Segmentation contour detection dataset, our approach outperforms all competing approaches that do not utilize pre-training. Our results highlight the benefits of image-based regularization to deep networks.
@cite_4: We study the problem of synthesizing a number of likely future frames from a single input image. In contrast to traditional methods, which have tackled this problem in a deterministic or non-parametric way, we propose a novel approach that models future frames in a probabilistic manner. Our probabilistic model makes it possible for us to sample and synthesize many possible future frames from a single input image. Future frame synthesis is challenging, as it involves low- and high-level image and motion understanding. We propose a novel network structure, namely a Cross Convolutional Network to aid in synthesizing future frames; this network structure encodes image and motion information as feature maps and convolutional kernels, respectively. In experiments, our model performs well on synthetic data, such as 2D shapes and animated game sprites, as well as on real-wold videos. We also show that our model can be applied to tasks such as visual analogy-making, and present an analysis of the learned network representations.
@cite_8: We introduce a complete parameterization of the family of two-dimensional steerable wavelets that are polar-separable in the Fourier domain under the constraint of self-reversibility. These wavelets are constructed by multiorder generalized Riesz transformation of a primary isotropic bandpass pyramid. The backbone of the transform (pyramid) is characterized by a radial frequency profile function @math , while the directional wavelet components at each scale are encoded by an @math shaping matrix @math , where @math is the number of wavelet channels and @math the order of the Riesz transform. We provide general conditions on @math and @math for the underlying wavelet system to form a tight frame of @math (with a redundancy factor @math ). The proposed framework ensures that the wavelets are steerable and provides new degrees of freedom (shaping matrix @math ) that can be exploited for designing specific wavelet systems. It encompasses many known transforms as part...
@cite_21: We describe an architecture for efficient and accurate linear decomposition of an image into scale and orientation subbands. The basis functions of this decomposition are directional derivative operators of any desired order. We describe the construction and implementation of the transform.
@cite_19: 
@cite_31: 
@cite_20: Steerable functions find application in numerous problems in image processing, computer vision and computer graphics. As such, it is important to develop the appropriate mathematical tools to analyze them. In this paper, we introduce the mathematics of Lie group theory in the context of steerable functions and present a canonical decomposition of these functions under any transformation group. The theory presented in this paper can be applied and extended in various ways.
Steerable frames are a concept established early for signal processing. Initially introduced by @cite_19 , the concept was extended to the Steerable Pyramid by @cite_21 and to a Lie-group formulation by @cite_20 @cite_31 . Further, steerability has recently been related to tight frames, presenting Simoncelli's Steerable Pyramid and multiple other Wavelets arising as a special case of the non-orthogonal Riesz transform @cite_8 . Steerable pyramids have been applied to CNNs as a pre-processing step @cite_4 , but have not yet been learnable. We incorporate steerable frames in CNNs to increase their de facto expressiveness and to allow them to learn their configurations, rather than picking them a priori.