How does Pinterest decipher what's on unmarked pictures and categorize them?
One of the Pinterest's white paper about Human Curation and Convnets powering item-to-item recommendationsarxiv describes implementation of convolutional neural network (CNN) based visual features (VGG2014, Faster R-CNN). This demonstrates the effectiveness of it (such image or object representations) which can improve user engagement. The visual features are computed using the process described in the previous study about visual search at Pinterest and can be used for more targeted features to be computed for related pins.

Here are the examples of detected visual objects from Pinterest's object detection pipeline:



Image source: Human Curation and Convnets: Powering
Item-to-Item Recommendations on Pinterest, Page 4, Fig. 6

The images are categorized by using dominant visual objects (individual objects seen in the image which passes a confidence threshold in Faster R-CNN) using fine-tuned VGG reranking variant. This allows Pinterest to introduce features such real-time recommendations for the users.

Check also this blog entry: Building a scalable machine vision pipeline.