Designing efficient and effective keypoint descriptors for an image plays a vital role in many computer vision tasks. The traditional binary descriptors such as local binary pattern and its variants directly perform a binarization operation on the intensity differences of the local affine covariant regions, thus their performance usually drops a lot because of the limited distinctiveness. In this paper, we propose a novel image keypoint descriptor, namely local derivative quantized binary pattern for object recognition. To incorporate the spatial information, we first divide the local affine covariant region into several subregions according to the intensity orders. For each sub region, we quantize the intensity differences between the central pixels and their neighbors in an adaptive way, and then we order the differences and use a hash function to map the differences into binary codes. The binary codes are histogramed to form the feature of each subregion. Furthermore, we utilize multi-scale support regions and pool the histograms together to represent the features of the image. Our approach does not need prior codebook training and hence it is more flexible and easy to be implemented. Moreover, our descriptor can preserve more local brightness and edge information than the traditional binary descriptors. Also, our descriptor is robust to rotation, illumination variations and other geometric transformations. Finally we conduct extensive experiments on three challenging datasets (i.e., 53 Objects, ZuBuD, and Kentucky) for object recognition and the experimental results show that our descriptor outperforms the existing state-of-the-art descriptors.