Fractional Local Neighborhood Intensity Pattern for
Image Retrieval using Genetic Algorithm
a

Avirup Bhattacharyya, bAyan Kumar Bhunia, cPrithaj Banerjee, dPartha Pratim Roy*,
e
Subrahmanyam Murala

a

Dept. of ECE, Institute of Engineering & Management, Kolkata, India Email- aavirupiem@gmail.com
b
Dept. of ECE, Institute of Engineering & Management, Kolkata, India. Email- bayanbhunia007@gmail.com
c
Dept. of CSE, Institute of Engineering & Management, Kolkata, India. Email-cprithajtutanbanerjee@gmail.com
d
Dept. of CSE, Indian Institute of Technology Roorkee, India. Email- dproy.fcs@iitr.ac.in
e
Dept. of EE, Indian Institute of Technology Ropar, India. Email- esubbumurala@iitrpr.ac.in
d*
email: proy.fcs@iitr.ac.in, TEL: +91-1332-284816*

Abstract
In this paper, a new texture descriptor named "Fractional Local Neighborhood Intensity
Pattern" (FLNIP) has been proposed for content based image retrieval (CBIR). It is an
extension of the Local Neighborhood Intensity Pattern (LNIP)[1]. FLNIP calculates the
relative intensity difference between a particular pixel and the center pixel of a 3×3 window
by considering the relationship with adjacent neighbors. In this work, the fractional change in
the local neighborhood involving the adjacent neighbors has been calculated first with respect
to one of the eight neighbors of the center pixel of a 3×3 window. Next, the fractional change
has been calculated with respect to the center itself. The two values of fractional change are
next compared to generate a binary bit pattern. Both sign and magnitude information are
encoded in a single descriptor as it deals with the relative change in magnitude in the adjacent
neighborhood i.e., the comparison of the fractional change. The descriptor is applied on four
multi-resolution images- one being the raw image and the other three being filtered gaussian
images obtained by applying gaussian filters of different standard deviations on the raw
image to signify the importance of exploring texture information at different resolutions in an
image. The four sets of distances obtained between the query and the target image are then
combined with a genetic algorithm based approach to improve the retrieval performance by
minimizing the distance between similar class images. The performance of the method has
been tested for image retrieval on four databases, including texture image databases such as
Brodatz texture image database and Salzburg texture database as well as a biomedical image
database namely the OASIS database and one face database - AT&T face database. The
precision and recall values observed on these databases have been compared with recent
state-of-art local patterns. The proposed method has shown a significant improvement over
many other existing methods.

Keywords- Local Neighborhood Intensity Pattern, Local Binary Pattern, Feature Extraction,
Texture Feature.

1. Introduction:
The advancement of technology has resulted in large volume of digital images available both
offline and online which in turn has led to huge image databases. For retrieving information
from this huge database based on a query image, many different types of retrieval methods
have been proposed over the years. Some of the earlier methods used text based image
retrieval. However, it suffers from some serious drawbacks. These are inappropriate
metadata, lack of adequate information necessary for describing the image and very high time
complexity. Thus, it has become less acceptable in the present context and led to the
emergence of content based image retrieval.
1.1 Motivation
Rather than storing the huge image database, storing their extracted features in a feature
database is more convenient. The features are used to represent and index the database.
Feature extraction thus plays an important role in content based image retrieval (CBIR).
Features can be of different types such as color, texture, shape and domain specific features
such as human faces and fingerprint. Combining two or more types of features proves to be
useful for content based image retrieval. The feature extraction method for CBIR should be
designed in such a way that it is able to retrieve information from images taken under
different conditions with variations in illumination and intensity and capture the local
microstructures present in an image as well as represent the local structural differences. A
number of texture patterns have been developed for efficient image retrieval [2]–[9] in the
recent past. However, most of these patterns have focused on comparing the center pixel of a
3×3 window of an image with only one of its neighbors at a time and ignored the remaining.
Citing some drawbacks of this approach, some later works [4], [10] have highlighted the need
of having separate sign and magnitude patterns to improve the discriminative power.
However, they did not consider the mutual relationship among the neighboring pixels of the
center which resulted in comparatively weak performance. The work done by Verma et al.
[3], [11]

stressed on this need which proved the importance of including the mutual

relationship of adjacent neighbors of a pixel for encoding its relationship with the center.
They considered the effect of two adjacent neighbors of a given neighboring pixel of the
center. The work in [11] extended the idea using more adjacent neighbors and claimed a

better performance. However, both [1] and [11] considered separate sign and magnitude
patterns to encode each of them separately.
In this paper, we propose a new feature descriptor for image retrieval task. The striking
advantage of our method over other methods is that we do not calculate separate sign and
magnitude patterns. Instead, we focus on developing a descriptor that takes into account both
the magnitude and sign information in a single descriptor. Thus, it reduces the feature
dimension and computational complexity. Further, we explore the multi-resolution approach
by applying the proposed descriptor on three different filtered versions of the same image
obtained with gaussian filters of different standard deviations. The four sets of distances have
been combined with genetic algorithm to improve the retrieval performance and make it more
robust.
The remainder of the paper is organized in the following manner. The existing literature for
content based image retrieval has been discussed in section 1.2. In section 1.3, we have
discussed the main contributions of our work and its advantages with respect to existing
techniques. In section 2, the traditional LBP feature and the proposed pattern have been
discussed. Section 3 describes the proposed system framework. Section 4 describes the
datasets used in our work and the results obtained on those. The concluding part of the paper
has been provided in the last section.
1.2 Related Work
Extensive research work has been done on content based image retrieval in the recent past.
Wavelet transform based feature extraction has been performed in [12] . This technique
suffers from serious drawbacks as it extracts information from an image in three different
directions such as horizontal, vertical and diagonal. In order to overcome these drawbacks, a
number of techniques have been proposed. For example, the Gabor wavelet feature based
texture analysis was used in [13]. The concept of wavelet correlogram was used by
Moghhadam et al. [14]. The concept of two dimensional rotated wavelet filters was used by
Kokare et al. in [15]. The method which uses a combination of dual tree rotated complex
wavelet filter and DT-CWT was proposed by the authors in [16].
A number of local patterns have been developed for CBIR over the years. The earliest texture
descriptor that was used is Local Binary Pattern (LBP) [17]. It has also been used for a
number of other tasks such as texture retrieval and classification in rotation invariant [17] and
uniform versions [18]. The original LBP method generates an 8 bit binary string by

considering pixels lying on the circumference of a circle of radius r and thresholding them
with the center pixel. The main drawback of LBP is its inability to represent anisotropic
structures due to circular sampling. This prompted the researchers to work on extending the
operator as found in Dominant Local Binary Pattern (DLBP)[19], Center Symmetric Local
Binary Pattern (CSLBP)[2], Block based local binary pattern (BLK LBP) [20], Completed
Local Binary Pattern(CLBP)[21]. The Multi-Structure local binary pattern (Ms-LBP)[22] has
been proposed to overcome the inefficiency of LBP in representing anisotropic structures.
Another drawback of LBP is its sensitivity to noise. In order to overcome this drawback, Tan
and Triggs proposed the local ternary pattern(LTP) [23] in which a ternary encoding is used
to overcome the noise. Local Derivative pattern (LDP) [24] was invented by Zhang et al. for
the purpose of face recognition. In this work, LDP has been shown to be the first order
derivate of LBP. LDP was used in two separate forms, Local Edge Pattern for segmentation
(LEPSEG) and Local Edge Pattern for image retrieval (LEPINV) [25] for different tasks.
Peak Valley edge pattern (PVEP) [7] was obtained by calculating the first order derivative in
four different directions. Local Maximum Edge Binary Pattern(LMEBP) [5] was developed
by the researchers in for the purpose of image retrieval and object tracking. Murala et al.
developed Local Tetra patterns (LTrP) [4] which represents the spatial structure of image
texture in different directions and also calculated a magnitude pattern. The local Gabor XOR
pattern was used for the purpose of face recognition in [26]. Noise-Resistant LBP (NRLBP)[27] and a Robust Local Binary Pattern (RLBP) that contains both sign and magnitude
information was also proposed in [28] to reduce noise of LBP feature. Gaussian filters and
RGB color space was used by Murala et al in [29]. In another work, the authors extracted the
LTP feature from different directions. This was named as Spherical Symmetric 3D Local
Ternary Patterns [30]. Content based image retrieval also has tremendous applications for
biomedical image retrieval where it has proved to be useful in handling a large image
database. Although a number of local pattern based texture features have been developed for
image retrieval in the recent past, not many have considered separate magnitude and sign
patterns. The directional binary wavelet patterns were developed by Murala et al. for
biomedical image retrieval and indexing [31]. Besides all these uses, there are other
implementations which used the concept of LBP and have made further improvements on it.
Among these, Local Bit-plane Decoded Pattern(LBDP)[32], Local Mesh Pattern (LMP) [33],
Average Local Binary Pattern(ALBP) [34], Ellipse Topology [35], three and four patch
LBP[36] etc. are worth mentioning. In Directional Local Extrema Pattern [37], the feature
was mainly obtained using the edge details. In this feature extraction technique by edge

information, four directions of the image were utilised and further improvement on it was
done in [38][5] which is used for retrieving images. The authors proposed a novel technique
for texture image retrieval based on tetrolet transforms [39]. A work on query by saliency
content retrieval (QSCR) image based on human visual attention models has been proposed
by Papushoy et al. in [40].
The concept of Gray Level Co-occurrence matrix (GLCM) was used by Haralick for
classification of the image where some statistical features are used to consider the intensity of
the pixels of images and mainly focuses on the occurrences of a particular kind of pattern. In
many cases, Prewitt operator is operated on the image and then GLCM is applied to calculate
statistical parameters, rather than directly applying GLCM operator on the real image[41].
Again GLCM is implemented for extracting the feature vector in Center Symmetric Local
Binary Co-occurrence Pattern [2], where the feature is extracted based on the diagonally
symmetric elements about the center. In Local Neighborhood Difference Pattern(LNDP) [3],
only the nearest element to a particular pixel is considered for the feature extraction and the
ultimate feature extraction for image retrieval is performed by concatenating this LNDP
feature with LBP feature.

1.3 Main Contributions
The main contributions of our work may be stated in brief as: First, we develop a novel
texture descriptor for content based image retrieval named FLNIP which is based on
comparing the fractional change in intensity in the adjacent neighborhood between the center
pixel of a 3×3 window and its neighboring pixel (Ij ) where j= {1,2,3….8}. Second, we
perform a multi-resolution analysis of the proposed pattern using gaussian filters of different
standard deviations to explore the texture information at different scales. Third, the distances
obtained by applying the descriptor on raw and filtered images have been optimally
combined by using genetic algorithm.

2. Local Patterns
2.1 Local Binary Patterns
Ojala et al. [42] originally proposed local binary pattern for texture classification. Due to
ease of computation, its use was extended to object tracking[43], facial expression
recognition [44] and medical imaging [45]. It has also been popularly used for the purpose of
image classification. This method considers a small window of an image and the intensity

difference between the center pixel and its K neighbors lying on the circumference of a circle
of radius R is encoded in binary form. Considering the center pixel as the threshold, a binary
value (0 or 1) is assigned to each of the surrounding pixels (as given in eqn. 2). These binary
bits are then multiplied by some specific weights which sum up to produce a decimal value
which replaces the center pixel. This has been explained diagrammatically in Fig. 1. Thus, a
local binary map of the image is generated by replacing each center pixel with its binary
pattern value in a similar manner. The feature vector is calculated by creating a histogram of
this local binary map. The formula for local binary pattern and the histogram is defined in
eqn. (1)-(4).
K

LBP(K, R) = ∑ 2m−1 × ∅1 (Im , Ic ) … … … (1)
m=1

∅1 (Im , Ic ) = {

X

1,
0,

Im ≥ Ic
otherwise … … … (2)

Y

His(K)| LBP = ∑ ∑ ∅2 (LBP(m, n), L); L ∈ [0, (2K − 1)] … … … (3)
m=1 n=1

∅2 (c1 , c2 ) = {

1,
0,

c1 = c2
otherwise

… … … (4)

Here, K and R represent the number of neighboring pixels and radius respectively. Im denotes
the mth surrounding pixel and Ic denotes center pixel. Eqn. (3) is used to compute the final
histogram of the pattern map. An example of LBP calculation is shown in Fig. 1.

2.2 Proposed Pattern
2.2.1 Fractional Local Neighborhood Intensity Pattern
The descriptor named fractional local neighborhood intensity pattern is based on comparing
the intensities of the center pixel of a 3×3 window with its neighboring pixels. It is inspired
from the adjacent neighbor concept in LNIP [1]. In LNIP, it was shown that adjacent
neighbors of a particular pixel contain useful texture information and it must be taken into
account for calculating its relative intensity difference with respect to the center pixel of a
3x3 window rather than performing a direct comparison as used in LBP (Local Binary
Pattern). It first calculates the sum of the absolute difference of pixel intensities of the
adjacent neighbors of a particular neighbor (Ik ) of the center pixel, where k={1,2,3,4,….,8} .

In eqn. 5 and 6, αk denotes the set of adjacent neighbors of a given pixel Ik for k={1,3,5,7}.
The variable M indicates the number of adjacent neighbors. For k={1,3,5,7}, M=4 and for
K={2,4,6,8}, M=2. The term μk in eqn. 7 represents the average sum of intensity differences
of the adjacent neighbors of Ik divided by Ik for M neighbors. We call it ‘fractional change’
with respect to (Ik ) in this work. Following the same set of steps, we calculate the fractional
change with respect to the center pixel as μc as given in eqn. 8. If the ‘fractional change’
with respect to the particular neighbor (μk ) is more than the fractional change with respect to
the center(μc ), it is encoded as 1, otherwise it is encoded as 0. Sign(a,b) represents the sign
operation in eqn. 10 which has been used to calculate the thresholding relation given in eqn.
9. This is repeated for all the remaining neighbors of the center pixel. Thus, we generate an 8
bit binary pattern of 0’s and 1’s. The decimal value for the pattern is given by eqn. 11. This
gives a feature descriptor having a dimensionality of 256. The diagram shown in Fig. 2 shows
the pattern calculation procedure for I1. The process can be repeated to obtain bit pattern for
remaining positions. Fig.2 shows an example of calculation of the bit pattern using fractional
local neighborhood intensity pattern. To avoid complexity, only the calculation of the bit
value for I1 has been shown. The fractional change with respect to Ic in our example is 2.53
while that for I1 is 2.31. Since 2.53 is greater than 2.31, the bit for I1 is 0.
αk = {I1+mod(k+5,7) , I1+mod(k+6,9) , Ik+1 , Imod(k+2,8) } ∀ k = 1,3,5,7
αk = { Ik−1 , Imod(k+1,8) } ∀ k = 2,4,6,8
1

μ k = M ∑M
m=1

|αk (m)−Ik |

1

Ik

μ c = M ∑M
m=1

|αk (m)−Ic |
Ic

… … … (7)
… … … (8)

μ(Ik , Ic ) = Sign(μk , μc ) … … … (9)
Sign(a, b) = {

1, a ≥ b
. ... … … (10)
0, otherwise

8

FLNIP (Ic ) = ∑ 2k−1 × μ(Ik , Ic ) … … … (11)
k=1

… … … (6)

… … … (5)

Fig.1. An example to demonstrate the calculation of Local Binary Pattern (LBP) (i) a 3×3 window with
general notations of the center and its neighboring elements (ii) an example of a 3×3 window (iii)-(v)
threshold neighboring pixels with center and assigning binary values accordingly.

Fig.2. Example showing the calculation of the proposed Fractional Local Neighborhood Intensity Pattern
(FLNIP) for a 3×3 window with general notations of the center and its neighboring elements. Diagrams
(b) through (h) illustrate the calculation of the proposed pattern for the neighbor I1 of the center pixel.

2.2.2. Gaussian filters for multi-resolution analysis
All the different textures existing in an image may not be prominent at a given resolution. A
particular texture may be more prominent over others at a given resolution. This motivates us
to increase the robustness of the proposed descriptor by applying a set of different isotropic
gaussian filters for performing multi-resolution analysis. The filters are designed choosing
three different values of standard deviation. The gaussian filter may be expressed as:

G(x, y, σ) =

1
2πσ2

e−(x

2 +y2 )/2σ2

… … … (12)

The filtered image can be represented as:
H(x, y) = G(x, y, σ) ∗ I(x, y) … … … (13)
In eqn. 12, x and y represent the spatial coordinates of the image. The standard deviation is
represented by 𝜎. The convolution operation in eqn.13 is represented by ∗ . In this paper, we
consider three different scales to obtain three filtered images. The three different values of
standard deviation considered for gaussian filtering are 0.5, 0.8 and 1. These values are
selected based on experimental results.We obtain three different histograms for filtered
images. On performing feature extraction after filtering, four sets of distance values between
query and database images (three sets of distance for filtered images and one set of distance
values for raw image) are combined in a weighted manner using genetic algorithm to produce
a better retrieval accuracy.
2.2.3. Genetic Algorithm for distance combination
In this paper, we propose to combine the different distances (between the query and the
database images) obtained by applying the descriptor on filtered images of different standard
deviations and the raw image itself using genetic algorithm. In our problem we aim to
improve the retrieval performance by maximizing precision with the help of genetic
algorithm. For this, the intra class similarity must be made higher and inter class similarity
must be made lower. To begin with, we randomly initialize a set of possible solutions which
constitutes a population. Each individual of the population is called a chromosome. For our
case, a chromosome is a set of four weights learned with the algorithm. Each chromosome
consists of genes, which in this case are the individual weights assigned to each distance.
Each chromosome in our approach has four genes. In eqn.14 d(Q, F) is the distance between
a query image feature and the feature of the database image where Q is the feature vector
representation of the query image and Fi is the feature vector of the database image. The
length of the feature vector is p. As mentioned earlier, we have to combine four distances
using genetic algorithm. This implies that four weights will have to be learned. Table 1 shows
the different parameters used in our algorithm implementation. A brief outline of the
algorithm is presented in algorithm 1.

p
d(Q, F) = ∑4j=1 Wj ( ∑j=1 |

Q− Fi

1+Q+Fi

| ) … …. … (14)

Table 1: Different parameters for the Genetic Algorithm used in our work

Selection
Mutation Rate
Crossover
Roulette Wheel
0.01 (1%)
Two-point crossover.
Algorithm1. Algorithm for GA based distance combination
Input: Set of four distances obtained with raw and filtered images.
Output: A set of four weights, one for each distance.
Preprocessing : Randomly initialize a number of solutions S1, S2, S3,…Sj,…, S999, S1000
where Sj =[ Wj1, Wj2, Wj3, Wj4].

for each generation Gi of G1...GN do
Step 1: Calculate the fitness values of every solution S1, S2, S3……., S1000 using
eqn.24.
Step 2: Select r individuals among S1, S2, S3,…., S1000 having high fitness to serve as
parent solutions for next generation( where r < 1000).
Step 3: Perform crossover between different pairs of parent solutions( r inviduals)
obtained in step 2 using two point crossover approach to produce offspring S1’, S2’,
S3’,……, Sk’ such that k+r =1000.
Step 4: Mutate the new offspring at each locus with the given mutation probability
(0.01).
Step 5: Test the new generation by evaluating their fitness using eqn. 24.
Step 6: Check for stopping condition, else repeat step 2-5.
end for
The precision in eqn. 22 refers to the precision for each category. Eqn. 24 represents the
average precision of all the categories and serves as the fitness function of our problem.
2.2.4 Advantages over other methods
1. In this work, we have proposed a new texture feature named fractional local neighborhood
intensity pattern. The descriptor is capable of capturing both the sign and magnitude
information in one pattern which has not been achieved earlier.

2. A multi-resolution analysis has been performed using gaussian filters of different standard
deviations on the original image before applying the texture feature. This has been done to
explore the texture information present in the image at different resolutions.
3. To better explore the discriminative power of the proposed approach, the distance
measures obtained by applying the texture feature on filtered gaussian images and raw image
have been combined with genetic algorithm based approach in order to obtain a single
distance to improve precision. This ensures that the intra class distance is reduced and the
inter class distance is increased. This approach clearly outperforms the state-of-the-art
descriptors for content based image retrieval and gives promising results on both texture and
color image datasets.

3. Framework for the proposed system:
A flow diagram of the proposed method is presented in Fig. 3 and algorithm for the same has
been described below. The proposed framework has been divided into two sections. The first
part describes the system framework. Here, an image is given as the input. Different sets of
feature representations are obtained after application of FLNIP on raw and filtered images. In
part 2, a query image is given as input. Retrieval of image begins by applying FLNIP on raw
and filtered versions of the query image. The different distances between query and retrieved
images obtained after applying FLNIP on the raw and filtered versions of the query image are
combined by performing a weighted combination using genetic algorithm. The images are
retrieved based on similarity measure.

Fig. 3. Proposed system framework.

3.1 System Framework Algorithm:
Section 1: Construction of feature vector
Input: An Image from the database.
Output: Feature vector.
1) Take an image from the image database and convert it into gray scale image if the image is
colored.
2) Apply gaussian filters of different standard deviations on the original image.
3) Apply the proposed descriptor on raw and filtered images of different standard deviations.
Compute the respective histograms.
Section 2: Image retrieval using the proposed method
Input: Query image from the database.

Output: Images retrieved based on similarity metric.
4) Take the query image as input.
5) Perform step-2 and step- 3 in part 1 for every query image of each database.
6) Obtain the four sets of distances between query and each database image.
7) Combine the distances by using a weighted approach using genetic algorithm to obtain an
optimal distance.
8) Evaluate the performance using the metrics like precision and recall.

3.2. Similarity Measure:
For retrieving the images in content based image retrieval system, apart from texture feature
representation, similarity measure plays a pivotal role. The similarity measure or metric is
used to calculate the distance between the query image and every image from the database in
the feature space and then rank the images accordingly. Based on this ranking, indexing is
done and indices with lower measures are sorted out as the set of retrieved images. A set of
five distance measures have been used in our work for calculation of similarity matching as
represented in eqn.15-19.

a. d1 distance:
l

τD ,qm = ∑ |
n=1

∩m
d (n) − ∩qm (n)
| … … … (15)
1 +∩m
d (n) +∩qm (n)

b. Euclidean Distance:
1/2

l

2
τD ,qm = (∑|(∩m
d (n) − ∩qm (n)) |)

… … … (16)

n=1

c. Manhattan Distance:
l

τD ,qm = ∑|∩m
d (n) − ∩qm (n)| … … … (17)
n=1

d. Canberra Distance:

l

τD ,qm

∩m
d (n) − ∩qm (n)
= ∑| m
| … … … (18)
∩d (n) +∩qm (n)
n=1

e. Chi-square Distance :
l

τD ,qm

2
(∩m
1
d (n) − ∩qm (n))
= ∑
… … … (19)
2
∩m
d (n) +∩qm (n)
n=1

Here, τD ,qm represents the distance function for database D and query image

qm , l

th
represents the length of the feature vector. ∩m
d (n) and ∩qm (n) are feature vector of m

database image and query image respectively.

4. Experimental Results and Analysis:
The suitability of the proposed descriptor for CBIR (Content Based Image Retrieval) is
verified by the results obtained on different datasets for image retrieval including texture
datasets like Stex and Brodatz texture database. One biomedical image and one face dataset
have also been considered for evaluation, namely the OASIS database and ORL face
database. For each dataset, we have evaluated a number of different parameters including
precision, recall, F-score and Average retrieval Rate (ARR). The precision is defined as the
ratio of the true positive rate to the sum of the true positive and false positive rate. For an
image retrieval system, it is a ratio of the total number of correct images retrieved from the
database to the total number of images retrieved from the database. Correct images implies
the number of images relevant for a given query image. On the other hand, recall is a function
of the true positive images and false negative images in the retrieval system. For any image
retrieval system, precision and recall generally share an inverse relationship. In other words,
precision generally decreases and recall generally increases with increase in the number of
images retrieved. If the total no. of images retrieved from the database is n, then the precision
and recall may be computed as give in eqn.20 and eqn. 21:
Precision Rate(q, Pi ) =

Total no of correct images retrieved from the database
Total no of images retrieved from the databases(n)

… … … (20)

where q represents the query image and Pi represents the precision value for the ith query
image.

Similarly, the eqn. for recall may be represented as:
Total no of relevant images retrieved from the database
Recall Rate(q, R i ) =
… … … (21)
Total no of relevant images present in the database(Ni )
Here, Ni indicates the total number of correct images in the database, i.e., number of images
in each category of the dataset.
The average precision rate may be calculated as follows:
m

1
Pavg (K) = ∑ Pi … … … (22 )
m
i=1

In eqn. 22, Pavg (K) represents the average precision rate for category (K), where m is the
total no of images in that category. Similarly, the recall rate for each category may be
expressed as given in eqn. 23
m

1
R avg (K) = ∑ R i … … … (23)
m
i=1

The total precision rate and total recall rate can be computed for our experiment using eqn.
24 and 25.
K

1
Ptot (K) = ∑ Pavg (K) … … … ( 24)
K
c=1

K

1
R tot (K) = ∑ R avg (K) … … … ( 25)
K
c=1

For our experiments, we have also determined the F-measure as given in eqn.26. The Fmeasure is mathematically expressed as the harmonic mean of the precision and recall. We
can write the F-measure mathematically as:
F − score =

2×Precision×Recall
Precision+Recall

… … … (26)

The performance of the method has been shown for all datasets by measuring the precision,
recall and F-score and comparing with state-of-the-art-methods. We have performed the
experiments using three different approaches. First one is simple feature extraction using
FLNIP from query image and database images and ranking based on similarity. The second is
the linear combination of different distances obtained between query and database image

using raw and filtered versions of the image. The third approach is a weighted combination of
the same set of distances using genetic algorithm based optimization. The third approach
clearly performs better than the rest. The distance metric plays a pivotal role in determining
the suitability of the proposed method. Different distance metrics have been considered for
evaluating the proposed method. The list of the different distance metrics considered in our
work is shown in Table 5. The Feature vector length and extraction time has been determined
for the proposed method and compared with existing techniques for image retrieval in Table
4.

Fig. 4. Sample images from different datasets.

Database 1:
For the first experiment, we have used the Brodatz[46] texture database. The database
consists of images of size 640×640 divided into sub images of size 128×128. Thus, there are
25 sub images of each image. There are a total of 112 categories of images in this dataset.
The total number of images for this dataset is 2800. Keeping consistency with the other
datasets, the same metrics have been used to evaluate the performance of the proposed
approach for this dataset. We have plotted the precision, recall and F-score curves for this
dataset as shown in Fig. 5. Some sample images from this database have been shown in Fig
4(b). Fig. 6 shows images retrieved for the given query image.

Fig. 5. Precision, recall and F-score with number of images retrieved for Brodatz database

Fig. 6. Query image and retrieved images from Brodatz database

Database 2:
We have evaluated the performance of our method on The Open Access Series of Imaging
Studies (OASIS)[47] dataset. This is a biomedical image dataset which is widely used to
verify the performance of a biomedical image retrieval system. The dataset consists of a
collection of 421 subjects with ages between 18 and 96 years. There are a total of 124, 102,
89 and 106 images in each category. The grouping has been done based on the shape of the
ventricular. Fig.4(d) represents some sample images from this database. Precision with
varying number of retrieved images and precision for different groups of images for this
dataset are shown in Fig. 7. The query and retrieved images have been shown in Fig. 8. Some
details regarding the dataset are given in Table 2. The Average Retrieval Rate (ARR) has
been calculated for this dataset similar to other datasets. Table 3 shows the results. The
results obtained with different distance metrics are shown in Table 5.

Fig.7. Precision with number of images retrieved and precision with Group-ID for OASIS database.

Fig. 8 Query image and retrieved images from OASIS database.

Table 2: Details of MRI Data Acquisition.

Sequence
TR (msec)
TI (msec)
Flip angle (o)
TE (msec)
Resolution (pixels)
Orientation
TD (msec)
Thickness, gap (mm)

MP-RAGE
9.7
20
10
4.0
176,208
Sagittal
200
1.25, 0

Database 3:
The performance of the proposed method has been evaluated on the ORL[48] database of
facial images for face image retrieval. There are a total of 40 users in this dataset and each
user has 10 images. The database has been created by AT&T laboratories, Cambridge. Thus,
there are a total of 400 images in this dataset. For some users, the images were taken at
different times and under varying illumination and expression. Sample images from this
database are shown in Fig. 4(c). The results of precision, recall and F-score on various
methods are shown in Fig 9. Query and retrieved images corresponding to them are shown in
Fig.10. The Average Retrieval Rate (ARR) has been calculated for this dataset similar to
other datasets. Table 3 shows the results. The results obtained with different distance metrics
are shown in Table 5 while Table 4 shows the time required for feature extraction for our
framework and those for other methods.

Fig. 9. Precision, recall and F-score with number of images retrieved for the ORL database.

Fig. 10 Query image and retrieved images from ORL database.

Database 4:
The fourth database used for the experimental study in our work is the Salzburg texture
database. The Salzburg1 texture Database contains images of size 128×128. There are a total
of 7616 images. There are a total of 476 categories and each category contains 16 images.
Different types of textures like wood, rubber, etc. are presented in the database. Sample
images from the database are presented in Fig. 4(a). Each image of the database is treated as a
query image. Fig. 12 shows some query images and their corresponding retrieved images.
The number of images retrieved for each category for this experiment is initially considered
as 16. This is increased in small steps of 16 images. The maximum number of images for this
dataset that have been retrieved in our experiment is 112. The precision, recall and F-score
have been plotted in Fig.11.

1

http://www.wavelab.at/sources/STex/

Fig 11. Precision, recall and F-score with number of images retrieved for Salzburg texture database

Fig. 12 Query image and retrieved images from Salzburg Texture database.

Table 3: Average Retrieval Rate for different datasets

Different Feature
Brodatz OASIS-MRI ORL STex
CSLBP
57.08
37.25
43.45 37.14
LEPINV
58.05
37.75
27.23 36.64
LEPSEG
65.12
38.35
37.85 27.14
LBP
70.45
44.23
41.45 52.17
CSLBCoP
75.45
50.45
62.15 59.14
LTriDP
77.85
50.95
55.43 57.45
FLNIP(On Single Raw Image)
79.80
53.87
63.89 61.80
FLNIP(Simple Combination)
82.32
57.85
66.14 64.43
FLNIP(Weighted Combination using GA) 84.30
58.84
68.23 67.14

Table 4 Runtime performance and feature length using different methods

Different Feature
CSLBP
LEPINV
LEPSEG
LBP
CSLBCoP
LTriDP
FLNIP(On Single Raw Image)

Feature
Length
16
72
512
256
1024
768
256

Feature
Extraction Time
0.0185
0.0709
0.0360
0.0181
0.0311
0.0302
0.0305

Retrieval
Time
0.0313
0.0312
0.0321
0.0316
0.0325
0.0321
0.0328

FLNIP(Simple Combination)
FLNIP(Weighted Combination)

1024
1024

0.0312
0.0356

0.0330
0.0352

Table 5: Comparative study with different distance matrics

Distance measure Brodatz OASIS-MRI ORL STex
d1
84.54
58.84
68.23 67.14
Euclidean
73.16
53.54
58.00 56.77
Manhattan
79.43
54.75
65.28 63.38
Canberra
70.00
55.95
65.58 52.63
Chi Square
82.69
57.47
65.68 64.42

5. Conclusion
In this paper, we proposed a novel feature descriptor for content based image retrieval named
Fractional Local Neighborhood Intensity Pattern(FLNIP). We have adopted a multiresolution approach by applying the feature on a set of filtered images obtained by applying
gaussian filters on the raw image to find out dominant texture features at different
resolutions. This has been followed by using genetic algorithm based learning to perform
weighted combination of the different distances obtained between query and database images
to reduce the distance between images of similar classes. The novelty in our method lies in
the fact that we calculate the sign and magnitude information in one pattern and do not
propose separate sign and magnitude components unlike previously developed approaches for
image retrieval thus reducing the feature dimension. The method has been evaluated on
standard databases and compared with existing techniques by calculating the precision, recall
and F-score values for all of them. The feature extraction and image retrieval time is also
competitive with most approaches. Thus, the technique can be adapted for efficient image
retrieval in real time systems.

References
[1]

P. Banerjee, A. K. Bhunia, A. Bhattacharyya, P. P. Roy, and S. Murala, “Local
Neighborhood Intensity Pattern: A new texture feature descriptor for image retrieval,”
arXiv Prepr. arXiv1709.02463, 2017.

[2]

M. Verma and B. Raman, “Center symmetric local binary co-occurrence pattern for
texture, face and bio-medical image retrieval,” J. Vis. Commun. Image Represent., vol.
32, pp. 224–236, 2015.

[3]

M. Verma and B. Raman, “Local neighborhood difference pattern: A new feature
descriptor for natural and texture image retrieval,” Multimedia Tools and Applications.
pp. 1–24, 2017.

[4]

S. Murala, R. P. Maheshwari, and R. Balasubramanian, “Local tetra patterns: A new
feature descriptor for content-based image retrieval,” IEEE Trans. Image Process., vol.
21, no. 5, pp. 2874–2886, 2012.

[5]

M. Subrahmanyam, R. P. Maheshwari, and R. Balasubramanian, “Local maximum
edge binary patterns: A new descriptor for image retrieval and object tracking,” Signal
Processing, vol. 92, no. 6, pp. 1467–1479, 2012.

[6]

S. Murala and Q. M. Jonathan Wu, “MRI and CT image indexing and retrieval using
local mesh peak valley edge patterns,” Signal Process. Image Commun., vol. 29, no. 3,
pp. 400–409, 2014.

[7]

S. MuralA and Q. M. J. Wu, “Peak valley edge patterns: A new descriptor for
biomedical image indexing and retrieval,” in IEEE Computer Society Conference on
Computer Vision and Pattern Recognition Workshops, 2013, pp. 444–449.

[8]

H. Müller et al., “A reference data set for the evaluation of medical image retrieval
systems.,” Comput. Med. Imaging Graph., vol. 28, no. 6, pp. 295–305, 2004.

[9]

I. Jeena Jacob, K. G. Srinivasagan, and K. Jayapriya, “Local Oppugnant Color Texture
Pattern for image retrieval system,” Pattern Recognit. Lett., vol. 42, no. 1, pp. 72–78,
2014.

[10] Z. Guo, L. Zhang, and D. Zhang, “A {Completed} {Modeling} of {Local} {Binary}
{Pattern} {Operator} for {Texture} {Classification},” IEEE Trans. Image Process.,
vol. 19, no. 6, pp. 1657–1663, 2010.
[11] M. Verma and B. Raman, “Local tri-directional patterns: A new texture feature
descriptor for image retrieval,” Digit. Signal Process. A Rev. J., vol. 51, pp. 62–72,
2016.
[12] J. LIN and L. QU, “FEATURE EXTRACTION BASED ON MORLET WAVELET
AND ITS APPLICATION FOR MECHANICAL FAULT DIAGNOSIS,” J. Sound
Vib., vol. 234, no. 1, pp. 135–148, 2000.
[13] D. Zhang, A. Wong, M. Indrawan, and G. Lu, “Content-based Image Retrieval Using
Gabor Texture Features,” IEEE Trans. PAMI, vol. 3656 LNCS, pp. 13–15, 2000.
[14] H. A. Moghaddam, T. T. Khajoie, A. H. Rouhi, and M. S. Tarzjan, “Wavelet
correlogram: A new approach for image indexing and retrieval,” Pattern Recognit.,
vol. 38, no. 12, pp. 2506–2518, 2005.
[15] M. Kokare, P. K. Biswas, and B. N. Chatterji, “Texture image retrieval using rotated
wavelet filters,” Pattern Recognit. Lett., vol. 28, no. 10, pp. 1240–1249, 2007.
[16] P. R. Hill, D. R. Bull, and C. N. Canagarajah, “Rotationally invariant texture features
using the dual-tree complex\nwavelet transform,” Proc. 2000 Int. Conf. Image
Process. (Cat. No.00CH37101), vol. 3, pp. 0–3, 2000.
[17] T. Ojala, M. Pietikäinen, and T. Mäenpää, “Multiresolution gray-scale and rotation
invariant texture classification with local binary patterns,” IEEE Trans. Pattern Anal.

Mach. Intell., vol. 24, no. 7, pp. 971–987, 2002.
[18] Y. Fang, J. Luo, and C. Lou, “Fusion of multi-directional rotation invariant uniform
LBP features for face recognition,” in 3rd International Symposium on Intelligent
Information Technology Application, IITA 2009, 2009, vol. 2, pp. 332–335.
[19] S. Liao, M. W. K. Law, and A. C. S. Chung, “Dominant local binary patterns for
texture classification,” IEEE Trans. Image Process., vol. 18, no. 5, pp. 1107–1118,
2009.
[20] V. Takala, T. Ahonen, and M. Pietikainen, “Block-based methods for image retrieval
using Local Binary Patterns,” in Lecture Notes in Computer Science, 2005, vol. 3540,
pp. 882–891.
[21] Y. Zhao, W. Jia, R. X. Hu, and H. Min, “Completed robust local binary pattern for
texture classification,” Neurocomputing, vol. 106, pp. 68–76, 2013.
[22] Y. He, N. Sang, and C. Gao, “Multi-structure local binary patterns for texture
classification,” Pattern Anal. Appl., pp. 1–13, 2012.
[23] X. Tan and B. Triggs, “Enhanced local texture feature sets for face recognition under
difficult lighting conditions,” IEEE Trans. Image Process., vol. 19, no. 6, pp. 1635–
1650, 2010.
[24] B. Zhang, Y. Gao, S. Zhao, and J. Liu, “Local derivative pattern versus local binary
pattern: Face recognition with high-order local pattern descriptor,” IEEE Trans. Image
Process., vol. 19, no. 2, pp. 533–544, 2010.
[25] C. H. Yao and S. Y. Chen, “Retrieval of translated, rotated and scaled color textures,”
Pattern Recognit., vol. 36, no. 4, pp. 913–929, 2002.
[26] S. Xie, S. Shan, X. Chen, and J. Chen, “Fusing local patterns of gabor magnitude and
phase for face recognition,” IEEE Trans. Image Process., vol. 19, no. 5, pp. 1349–
1361, 2010.
[27] J. Ren, X. Jiang, and J. Yuan, “Noise-resistant local binary pattern with an embedded
error-correction mechanism,” IEEE Trans. Image Process., vol. 22, no. 10, pp. 4049–
4060, 2013.
[28] J. Chen and V. Kellokumpu, “RLBP: Robust local binary pattern,” Proc. Br. Mach.
Vis. Conf., no. 2016, pp. 1–10, 2013.
[29] S. Murala, A. B. Gonde, and R. P. Maheshwari, “color and texture features for image
indexing and retrieval,” in 2009 IEEE International Advance Computing Conference,
IACC 2009, 2009, pp. 1411–1416.
[30] S. Murala and Q. M. Jonathan Wu, “Spherical symmetric 3D local ternary patterns for
natural, texture and biomedical image indexing and retrieval,” Neurocomputing, vol.
149, no. PC, pp. 1502–1514, 2015.
[31] S. Murala, R. P. Maheshwari, and R. Balasubramanian, “Directional binary wavelet
patterns for biomedical image indexing and retrieval,” J. Med. Syst., vol. 36, no. 5, pp.
2865–2879, 2012.
[32] S. R. Dubey, S. K. Singh, and R. K. Singh, “Local Bit-Plane Decoded Pattern: A

Novel Feature Descriptor for Biomedical Image Retrieval,” IEEE J. Biomed. Heal.
Informatics, vol. 20, no. 4, pp. 1139–1147, 2016.
[33] S. Murala and Q. M. J. Wu, “Local mesh patterns versus local binary patterns:
Biomedical image indexing and retrieval,” IEEE J. Biomed. Heal. Informatics, vol. 18,
no. 3, pp. 929–938, 2014.
[34] I. Hamouchene and S. Aouat, “A New Texture Analysis Approach for Iris
Recognition,” AASRI Procedia, vol. 9, pp. 2–7, 2014.
[35] S. Liao and A. Chung, “Face Recognition by Using Elongated Local Binary Patterns
with Average Maximum Distance Gradient Magnitude,” Accv, vol. 4844, pp. 672–679,
2007.
[36] L. Wolf, T. Hassner, and Y. Taigman, “Descriptor based methods in the wild,” in In
Faces in Real-Life Images Workshop, 2008, vol. 6, pp. 1–14.
[37] S. Murala, R. P. Maheshwari, and R. Balasubramanian, “Directional local extrema
patterns: a new descriptor for content based image retrieval,” Int. J. Multimed. Inf.
Retr., vol. 1, no. 3, pp. 191–203, 2012.
[38] P. Vijaya Bhaskar Reddy and A. Rama Mohan Reddy, “Content based image indexing
and retrieval using directional local extrema and magnitude patterns,” AEU - Int. J.
Electron. Commun., vol. 68, no. 7, pp. 637–643, 2014.
[39] G. Raghuwanshi and V. Tyagi, “Texture image retrieval using adaptive tetrolet
transforms,” Digit. Signal Process., vol. 48, pp. 50–57, 2016.
[40] A. Papushoy and A. G. Bors, “Image retrieval based on query by saliency content,”
Digit. Signal Process., vol. 36, pp. 156–173, 2015.
[41] J. Zhang, G. L. Li, and S. W. He, “Texture-based image retrieval by edge detection
matching GLCM,” in Proceedings - 10th IEEE International Conference on High
Performance Computing and Communications, HPCC 2008, 2008, pp. 782–786.
[42] T. Ojala, M. Pietikainen, and T. Maenpaa, “Multiresolution gray-scale and rotation
invariant texture classification with local binary patterns,” IEEE Trans. Pattern Anal.
Mach. Intell., vol. 24, no. 7, pp. 971–987, 2002.
[43] J. Ning, L. Zhang, D. Zhang, and C. Wu, “Robust Object Tracking Using Joint ColorTexture Histogram,” Int. J. Pattern Recognit. Artif. Intell., vol. 23, no. 7, pp. 1245–
1263, 2009.
[44] S. Moore and R. Bowden, “Local binary patterns for multi-view facial expression
recognition,” Comput. Vis. Image Underst., vol. 115, no. 4, pp. 541–558, 2011.
[45] L. Nanni, A. Lumini, and S. Brahnam, “Local binary patterns variants as texture
descriptors for medical image analysis,” Artif. Intell. Med., vol. 49, no. 2, pp. 117–125,
2010.
[46] S. Abdelmounaime and H. Dong-Chen, “New Brodatz-Based Image Databases for
Grayscale Color and Multiband Texture Analysis,” ISRN Mach. Vis., vol. 2013, pp. 1–
14, 2013.
[47] D. S. Marcus, T. H. Wang, J. Parker, J. G. Csernansky, J. C. Morris, and R. L.

Buckner, “Open Access Series of Imaging Studies (OASIS): Cross-sectional MRI Data
in Young, Middle Aged, Nondemented, and Demented Older Adults,” J. Cogn.
Neurosci., vol. 19, no. 9, pp. 1498–1507, 2007.
[48] AT&T Laboratories Cambridge, “Database of Faces,” www.cl.cam.ac.uk/, 2002. .

