title: WAS not processing request hence web are not available
body: <p>I'm new to supporting WAS (Websphere Application Server), currently I'm having issue with my WAS, my WAS was installed under AIX under 2 servers/nodes.
  While investigating it, I found in our application log that there are some activity which is "Performing Cache Maintenance":</p>
  
  <pre><code>===================================
  2017-01-14 01:31:52,619: [Cache Maintenance] com.ibm.srm.util.db.ServerCache refreshed 2017-01-14 01:31:53,314: [Cache Maintenance] Memory: available=[6884mb] used=[9500mb] %used avail=[58%] max=[16384mb] %used max=[58%] total=[16384mb] free=[6884mb] used by doMaintenance=[-251,201,3 92bytes] Time=[22,818ms] 2017-01-14 01:51:53,325: -------- Performing Cache Maintenance -------- 2017-01-14 01:51:53,325: null : QN=319 Select * from perform.cache_timestamps where row_class_name not like '%Cache' and row_class_name not like '%(SRM 6.0)' 2017-01-14 01:51:53,333: Returning 19 data records, QN=319,2 columns, Time: 8ms conn/query time: 5ms
  2017-01-14 01:51:53,333: [Cache Maintenance] Memory: available=[5492mb] used=[10892mb] %used avail=[66%] max=[16384mb] %used max=[66%] total=[16384mb] free=[5492mb] used by doMaintenance=[532kb] Time=[8ms]
  ===================================
  </code></pre>
  
  <p>After this activity triggered, I found that mpmstats value for 'bsy' are keep increasing until reach MaxClient maximum value which is '4000':</p>
  
  <pre><code>===================================
  [Sat Jan 14 01:38:58 2017] [notice] mpmstats: rdy 166 bsy 234 rd 0 wr 234 ka 0 log 0 dns 0 cls 0 [Sat Jan 14 01:38:58 2017] [notice] mpmstats: bsy: 234 in mod_was_ap22_http.c [Sat Jan 14 01:48:58 2017] [notice] mpmstats: rdy 195 bsy 505 rd 0 wr 505 ka 0 log 0 dns 0 cls 0 [Sat Jan 14 01:48:58 2017] [notice] mpmstats: bsy: 505 in mod_was_ap22_http.c [Sat Jan 14 01:58:58 2017] [notice] mpmstats: rdy 180 bsy 720 rd 0 wr 720 ka 0 log 0 dns 0 cls 0 [Sat Jan 14 01:58:58 2017] [notice] mpmstats: bsy: 720 in mod_was_ap22_http.c [Sat Jan 14 02:08:59 2017] [notice] mpmstats: rdy 105 bsy 895 rd 1 wr 894 ka 0 log 0 dns 0 cls 0 [Sat Jan 14 02:08:59 2017] [notice] mpmstats: bsy: 894 in mod_was_ap22_http.c [Sat Jan 14 02:18:59 2017] [notice] mpmstats: rdy 112 bsy 1088 rd 1 wr 1087 ka 0 log 0 dns 0 cls 0 [Sat Jan 14 02:18:59 2017] [notice] mpmstats: bsy: 1087 in mod_was_ap22_http.c
  [Sat Jan 14 02:28:59 2017] [notice] mpmstats: rdy 158 bsy 1242 rd 1 wr 1241 ka 0 log 0 dns 0 cls 0
  ----
  [Sat Jan 14 04:55:34 2017] [notice] mpmstats: rdy 0 bsy 4000 rd 0 wr 4000 ka 0 log 0 dns 0 cls 0 [Sat Jan 14 04:55:34 2017] [notice] mpmstats: bsy: 4000 in mod_was_ap22_http.c [Sat Jan 14 04:57:04 2017] [notice] mpmstats: reached MaxClients (4000/4000) [Sat Jan 14 04:57:04 2017] [notice] mpmstats: rdy 0 bsy 4000 rd 0 wr 4000 ka 0 log 0 dns 0 cls 0 [Sat Jan 14 04:57:04 2017] [notice] mpmstats: bsy: 4000 in mod_was_ap22_http.c [Sat Jan 14 04:58:34 2017] [notice] mpmstats: reached MaxClients (4000/4000) [Sat Jan 14 04:58:34 2017] [notice] mpmstats: rdy 0 bsy 4000 rd 0 wr 4000 ka 0 log 0 dns 0 cls 0
  [Sat Jan 14 04:58:34 2017] [notice] mpmstats: bsy: 4000 in mod_was_ap22_http.c
  ===================================
  </code></pre>
  
  <p>It seem WAS are not processing the client request until it reached the Maximum value.</p>
  
  <p>The questions are:</p>
  
  <ol>
  <li>Is there any log that I can check about why WAS are not processing the Client request until it reached to the max value?</li>
  <li>Does the "Cache Maintenance" activity hold WAS from processing the Client request? Because as mentioned from our developer this activity should not lead to this issue.</li>
  <li>What is the procedure that I can take to identify/resolve this issue?</li>
  </ol>
  
  <p>Appreciate if can help me for this thing as this issue already occurred for a long time but still not resolve.</p>