In this work, we introduce the Porous Material Transformer (PMTransformer), which is a multimodal Transformer architecture based on the MOFTransformer and is pre-trained with 1.9 million hypothetical porous materials, including MOFs, COFs, PPNs, and zeolites. The model showcases excellent transfer learning capability across various properties of porous materials, thereby achieving state-of-the-art performance in predicting multiple different properties. To address the challenge of asymmetry data aggregation in porous materials, we propose cross-material few-shot learning to improve predictions of materials lacking available data for their properties by exploiting the uniform characteristics in porous materials. Moreover, we obtain cross-relationships in porous materials by predicting unseen properties of other classes of porous materials. Our approach provides a novel perspective for understanding the underlying and uniform relationships between various classes of porous materials, allowing for the prediction of previously unexplored properties across these material classes.