<p dir="auto">Firstly, show the code:</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="km = joblib.load(filename)
cluster_labels = km.predict(X)
silhouette_avg = silhouette_score(X, cluster_labels)
# Compute the silhouette scores for each sample
sample_silhouette_values = silhouette_samples(X, cluster_labels)"><pre class="notranslate"><code class="notranslate">km = joblib.load(filename)
cluster_labels = km.predict(X)
silhouette_avg = silhouette_score(X, cluster_labels)
# Compute the silhouette scores for each sample
sample_silhouette_values = silhouette_samples(X, cluster_labels)
</code></pre></div>
<p dir="auto">‘km’ is the model that I trained using training data. see details:</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="km = KMeans(n_clusters=num_k, init='k-means++', max_iter=300, n_init=1, verbose=False)
km.fit(X)"><pre class="notranslate"><code class="notranslate">km = KMeans(n_clusters=num_k, init='k-means++', max_iter=300, n_init=1, verbose=False)
km.fit(X)
</code></pre></div>
<p dir="auto">When the amount of X less than 30 thousands rows, both of silhouette_score and silhouette_samples are OK and can get expected results. But when the amount of X more than 100 thousands, the program crashed and get "Segmentation fault (core dumped)".  See the detail error information:</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="Traceback (most recent call last):
  File &quot;test19_statistic_silhouette_score.py&quot;, line 87, in &lt;module&gt;  out()
  File &quot;test19_statistic_silhouette_score.py&quot;, line 63, in out sample_silhouette_values = silhouette_samples(X, cluster_labels)
  File &quot;/home/supermicro/.local/lib/python2.7/site-packages/sklearn/metrics/cluster/unsupervised.py&quot;, line 153, in silhouette_samples
    distances = pairwise_distances(X, metric=metric, **kwds)
  File &quot;/home/supermicro/.local/lib/python2.7/site-packages/sklearn/metrics/pairwise.py&quot;, line 1112, in pairwise_distances
    return _parallel_pairwise(X, Y, func, n_jobs, **kwds)
  File &quot;/home/supermicro/.local/lib/python2.7/site-packages/sklearn/metrics/pairwise.py&quot;, line 962, in _parallel_pairwise
    return func(X, Y, **kwds)
  File &quot;/home/supermicro/.local/lib/python2.7/site-packages/sklearn/metrics/pairwise.py&quot;, line 207, in euclidean_distances
    distances = safe_sparse_dot(X, Y.T, dense_output=True)
  File &quot;/home/supermicro/.local/lib/python2.7/site-packages/sklearn/utils/extmath.py&quot;, line 178, in safe_sparse_dot
    ret = a * b
  File &quot;/usr/lib/python2.7/dist-packages/scipy/sparse/base.py&quot;, line 303, in __mul__
    return self._mul_sparse_matrix(other)
  File &quot;/usr/lib/python2.7/dist-packages/scipy/sparse/compressed.py&quot;, line 528, in _mul_sparse_matrix
    return self.__class__((data,indices,indptr),shape=(M,N))
  File &quot;/usr/lib/python2.7/dist-packages/scipy/sparse/compressed.py&quot;, line 84, in __init__
    self.check_format(full_check=False)
  File &quot;/usr/lib/python2.7/dist-packages/scipy/sparse/compressed.py&quot;, line 144, in check_format raise ValueError(&quot;Last value of index pointer should be less than &quot;
ValueError: Last value of index pointer should be less than the size of index and data arrays
*** Error in `python': munmap_chunk(): invalid pointer: 0x00007f9249d68010 ***
Aborted (core dumped)"><pre class="notranslate"><code class="notranslate">Traceback (most recent call last):
  File "test19_statistic_silhouette_score.py", line 87, in &lt;module&gt;  out()
  File "test19_statistic_silhouette_score.py", line 63, in out sample_silhouette_values = silhouette_samples(X, cluster_labels)
  File "/home/supermicro/.local/lib/python2.7/site-packages/sklearn/metrics/cluster/unsupervised.py", line 153, in silhouette_samples
    distances = pairwise_distances(X, metric=metric, **kwds)
  File "/home/supermicro/.local/lib/python2.7/site-packages/sklearn/metrics/pairwise.py", line 1112, in pairwise_distances
    return _parallel_pairwise(X, Y, func, n_jobs, **kwds)
  File "/home/supermicro/.local/lib/python2.7/site-packages/sklearn/metrics/pairwise.py", line 962, in _parallel_pairwise
    return func(X, Y, **kwds)
  File "/home/supermicro/.local/lib/python2.7/site-packages/sklearn/metrics/pairwise.py", line 207, in euclidean_distances
    distances = safe_sparse_dot(X, Y.T, dense_output=True)
  File "/home/supermicro/.local/lib/python2.7/site-packages/sklearn/utils/extmath.py", line 178, in safe_sparse_dot
    ret = a * b
  File "/usr/lib/python2.7/dist-packages/scipy/sparse/base.py", line 303, in __mul__
    return self._mul_sparse_matrix(other)
  File "/usr/lib/python2.7/dist-packages/scipy/sparse/compressed.py", line 528, in _mul_sparse_matrix
    return self.__class__((data,indices,indptr),shape=(M,N))
  File "/usr/lib/python2.7/dist-packages/scipy/sparse/compressed.py", line 84, in __init__
    self.check_format(full_check=False)
  File "/usr/lib/python2.7/dist-packages/scipy/sparse/compressed.py", line 144, in check_format raise ValueError("Last value of index pointer should be less than "
ValueError: Last value of index pointer should be less than the size of index and data arrays
*** Error in `python': munmap_chunk(): invalid pointer: 0x00007f9249d68010 ***
Aborted (core dumped)
</code></pre></div>
<p dir="auto">With scikit-learn 0.15.2, numpy 1.9.1, python 2.7.8 (on OS X), the following code segfaults:</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="import numpy
import sklearn.cluster

numpy.random.seed(1)
X = numpy.random.random((50000, 100))
model = sklearn.cluster.KMeans(n_clusters=3, random_state=1)
model.fit_predict(X)
print sklearn.metrics.silhouette_score(X, model.labels_, metric='euclidean')"><pre class="notranslate"><code class="notranslate">import numpy
import sklearn.cluster

numpy.random.seed(1)
X = numpy.random.random((50000, 100))
model = sklearn.cluster.KMeans(n_clusters=3, random_state=1)
model.fit_predict(X)
print sklearn.metrics.silhouette_score(X, model.labels_, metric='euclidean')
</code></pre></div>
<p dir="auto">Results in:</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="Segmentation fault: 11"><pre class="notranslate"><code class="notranslate">Segmentation fault: 11
</code></pre></div>
<p dir="auto">Dropping the rows down to 30000, and the above completes fine. Dropping rows to 40000, and the script takes a very long amount of time, but didn't appear to segfault.</p>