1. A device comprising: a processing unit; a memory coupled to the processing unit and storing instructions thereon, the instructions, when executed by the processing unit, causing the device to: obtain joint locations for a skeleton representation of an observed entity in a frame of a video and a predefined action label for the frame; determine, based on the joint locations and the predefined action label, first weights for mapping the joint locations to a first feature for the frame generated by a first recurrent neural network (RNN) element in a learning network and second weights for mapping the joint locations to a second feature for the frame generated by a second RNN element in the learning network, by increasing a first correlation between the first feature and a first subset of the joint locations and a second correlation between the second feature and the first subset of the joint locations; and determine, based on the joint locations and the predefined action label, a parameter for a classification element included in the learning network by increasing a probability of the frame being associated with the predefined action label; the probability being generated by the classification element at least based on the first and second features.