title: Thread and message sequence handling in WebSocket load testing with JMeter
body: <p>First of all, I'm very new to JMeter. 
  I'm trying to do load testing of websocket service but I haven't been able to figure out how to send multiple messages within one connection sequentially (and not parallel with other threads) after response to previous message has been received, in limited number of concurrent connections. </p>
  
  <p>My connection also does not get closed as close pattern is not recognized if response pattern is found.</p>
  
  <h2>Update</h2>
  
  <p>After modification of WebSocketSampler code both message-specific matching patterns and match &amp; close do work; code changes in the end.</p>
  
  <h1>Goal for this example</h1>
  
  <ul>
  <li>Open total 12 connections (3 concurrent, open new connection after previous has been closed)</li>
  <li>send 5 messages to each connection
  <ul>
  <li>Each message contains key-value pair in format "messageIx":n (where n is integer from 1 to 5)</li>
  <li>Wait until first response containing same key is received</li>
  <li>Send next message</li>
  </ul></li>
  <li>When response containing "messageIx":5 is received, close connection</li>
  </ul>
  
  <p>Actual application will send up to 4 responses for each message. But I haven't been able to find out a way to do this even with WS echo service.
  I'm using <a href="https://github.com/maciejzaleski/JMeter-WebSocketSampler" rel="nofollow">https://github.com/maciejzaleski/JMeter-WebSocketSampler</a> as plugin for JMeter.</p>
  
  <h2>Current setup</h2>
  
  <ul>
  <li>Thread group
  <ul>
  <li>3 threads</li>
  <li>ramp-up 1 second</li>
  <li>loop count 5</li>
  </ul></li>
  <li>WebSocket Sampler
  <ul>
  <li>Connection timeout 5000ms</li>
  <li>Response timeout 5000ms</li>
  <li>Request data ${request}</li>
  <li>Response pattern ${response}</li>
  <li>Close connection pattern \"messageIx\":5</li>
  </ul></li>
  <li>CSV Data Set Config
  <ul>
  <li>Variable names request,response</li>
  <li>Sharing mode current thread</li>
  <li>delimiter ;</li>
  <li>Recycle on EOF False</li>
  <li>Stop thread on EOF: True</li>
  </ul></li>
  </ul>
  
  <p>My CSV file content:</p>
  
  <pre><code> {"messageType":0,"messageIx":1,"action":"None"};"messageIx":1
   {"messageType":0,"messageIx":2,"action":"None"};"messageIx":2
   {"messageType":0,"messageIx":3,"action":"None"};"messageIx":3
   {"messageType":0,"messageIx":4,"action":"None"};"messageIx":4
   {"messageType":0,"messageIx":5,"action":"Close"};"messageIx":5
  </code></pre>
  
  <p>Running this configuration against SimpleWebSocketServer (python) implementation output is</p>
  
  <pre><code> 52682 :  connected
   52682 &lt;--  {"messageType":0,"messageIx":1,"action":"None"}
   52691 :  connected
   52691 &lt;--  {"messageType":0,"messageIx":1,"action":"None"}
   52700 :  connected
   52700 &lt;--  {"messageType":0,"messageIx":1,"action":"None"}
   52682 &lt;--  {"messageType":0,"messageIx":2,"action":"None"}
   52691 &lt;--  {"messageType":0,"messageIx":2,"action":"None"}
   52700 &lt;--  {"messageType":0,"messageIx":2,"action":"None"}
   52682 &lt;--  {"messageType":0,"messageIx":3,"action":"None"}
   52691 &lt;--  {"messageType":0,"messageIx":3,"action":"None"}
   52700 &lt;--  {"messageType":0,"messageIx":3,"action":"None"}
   52682 &lt;--  {"messageType":0,"messageIx":4,"action":"None"}
   52691 &lt;--  {"messageType":0,"messageIx":4,"action":"None"}
   52700 &lt;--  {"messageType":0,"messageIx":4,"action":"None"}
   52682 &lt;--  {"messageType":0,"messageIx":5,"action":"Close"}
   52691 &lt;--  {"messageType":0,"messageIx":5,"action":"Close"}
   52700 &lt;--  {"messageType":0,"messageIx":5,"action":"Close"}
   52682 &lt;--  &lt;EOF&gt;
   52691 &lt;--  &lt;EOF&gt;
   52700 &lt;--  &lt;EOF&gt;
   52682 &lt;--  &lt;EOF&gt;
   52691 &lt;--  &lt;EOF&gt;
   52700 &lt;--  &lt;EOF&gt;
   52691 :  closed
   52682 :  closed
   52700 :  closed
  </code></pre>
  
  <p>From View Result Tree Listener for first message for each thread I get</p>
  
  <pre><code> [Execution Flow]
  - Opening new connection
  - Using response message pattern ""messageIx":1"
  - Using disconnect pattern "Close"
  - Waiting for the server connection for 5000 MILLISECONDS
  - WebSocket conection has been opened
  - Connection established
  - Waiting for messages for 5000 MILLISECONDS
  - Received message #1 (47 bytes); matched response pattern
  - Leaving streaming connection open
  </code></pre>
  
  <p>but for following messages there is no match</p>
  
  <pre><code>[Execution Flow]
   - Reusing exising connection
   - Waiting for messages for 5000 MILLISECONDS
   - Received message #2 (47 bytes); didn't match any pattern
   - Leaving streaming connection open
  </code></pre>
  
  <p>For the last message for a thread data is the same except client closing the connection (streaming connection is left open though)</p>
  
  <pre><code> - WebSocket conection has been successfully closed by the server
   - WebSocket session closed by the client
   - Leaving streaming connection open
  </code></pre>
  
  <p>However if I modify response pattern to be generic \"messageIx\": all messages are matched but Close Connection is not matched with either \"messageIx\":5 or Close pattern;</p>
  
  <pre><code>Response data:
  [Message 5]
  {"messageType":0,"messageIx":5,"action":"Close"}
  
  Sampler request:
  [Execution Flow]
   - Reusing exising connection
   - Waiting for messages for 5000 MILLISECONDS
   - Received message #5 (48 bytes); matched response pattern
   - Leaving streaming connection open
  </code></pre>
  
  <p>Last, threading. If I set up 12 threads they will all get started with ramp-up delay in the beginning.
  Adding loop controller will just multiply messages for the same connection and increasing loops in thread group (and using recycle in csv reader) will fail as connection has already been closed (if close connection pattern matches).</p>
  
  <h2>Help I'm looking for</h2>
  
  <ol>
  <li>How can I match each messages with unique matching pattern?</li>
  <li>How I can close message while matching pattern is found as well?</li>
  <li>How should I create amount of threads, each with new connection, but restricting concurrent amount to lower number?</li>
  </ol>
  
  <p>Please excuse me if I'm asking something very silly here as I'm very new to JMeter :)</p>
  
  <h2>Update: Modified WebSocketSampler code</h2>
  
  <p>From Sampler's GitHub page I found issue similar to my message matching problem: <a href="https://github.com/maciejzaleski/JMeter-WebSocketSampler/issues/19" rel="nofollow">https://github.com/maciejzaleski/JMeter-WebSocketSampler/issues/19</a></p>
  
  <p>I downloaded 1.0.2-SNAPSHOT sources, added modifications from 2nd reply for issue above and also on ServiceSocket.java file changed code (from line 68) as follows:</p>
  
  <pre><code>boolean matchFound = false;
  if (responseExpression == null || responseExpression.matcher(msg).find()) {
      logMessage.append("; matched response pattern");
      closeLatch.countDown();
      matchFound = true;
  }
  if (!disconnectPattern.isEmpty() &amp;&amp; disconnectExpression.matcher(msg).find()) {
      logMessage.append("; matched connection close pattern").append("\n");
      closeLatch.countDown();
      matchFound = true;
      close(StatusCode.NORMAL, "JMeter closed session.");
  }
  
  if (!matchFound)
  {
      logMessage.append("; didn't match any pattern").append("\n");
  } else {
      logMessage.append("\n");
  }
  </code></pre>
  
  <p>I haven't ever written Java previously - I would highly appreciate someone more experienced to comment if changes I applied are sound.</p>
  
  <p>Now I'm able to match messages individually and also close connection. Still looking for help on limiting amount of concurrent threads without reusing same <em>connection</em> for messages.</p>
  
  <h2>Update 2: Reconnecting</h2>
  
  <p>One way to keep concurrent number of users (connections) limited would be to use CSV file for source and set recycle on EOF to true, set threads to concurrent target and loop count to number of connections per thread multiplied by number of messages in CSV file.</p>
  
  <p>On 1.0.2 this won't work directly as connectionId doesn't change and plugin tries to use old, closed connection. On WebSocketSamples.java, line 69:</p>
  
  <pre><code>    if (isStreamingConnection()) {
           if (connectionList.containsKey(connectionId)) {
               socket = connectionList.get(connectionId);
  
               if (socket.isConnected()){
                  socket.initialize(this);
                  return socket;
               } 
               connectionList.remove(connectionId, socket);
           }
  </code></pre>
  
  <p>modification seems to do the trick. Messages still seem to be sent parallel and performance compared to for example Node.JS based Thor seem pretty poor :(</p>