title: stream analytics: Event processing delay, Performance Issues
body: <p>I am facing some delay in event processing through stream analytics. I am working on an IoT Project and gateway is sending messages to IoTHub. Since we are dealing with different type of messages, through event hub end point we are routing the messages. Our Stream analytics job will pick the events from event hub and will start processing and pushing to Service Bus Queue..</p>
  
  <p>After starting the job for the next minute on wards messages are pushing to service bus queue. So we compared the time difference between event hub utc time and stream analytics process time. Initially it will be executing within 5 to 10 seconds.</p>
  
  <p>But after 1 hour this delay will be getting incresaed to 15 seconds, and it will continue. So after 6 or 7 hours, the message produced and message deliver to Service Bus queue gap will be more than 1 minute. Since our application is directly based on realtime data, I may need to restart the job after 7 hours(Which is not a permanent solution).</p>
  
  <p>In the event hub I am using 4 partition and 3 streaming units are using. Since streaming unit utilization is very low(16%), so increasing streaming unit is not a solution for this problem.</p>
  
  <p>Kindly help on this issue.</p>
  
  <p>My Stream Analytics Query Giving Below:</p>
  
  <p>WITH </p>
  
  <p>rawmessage AS</p>
  
  <p>( </p>
  
  <p>SELECT </p>
  
  <p>digitaleventhubstreaminputonlineclassaforrawdata.*,</p>
  
  <p>GetMetadataPropertyValue(digitaleventhubstreaminputonlineclassaforrawdata,</p>
  
  <p>'EventHub.IoTConnectionDeviceId')
  as iotdevice</p>
  
  <p>FROM digitaleventhubstreaminputonlineclassaforrawdata</p>
  
  <p>Partition 
  By PartitionId</p>
  
  <p>) </p>
  
  <p>, </p>
  
  <p>messagetoprocess AS</p>
  
  <p>( </p>
  
  <p>SELECT </p>
  
  <p>rawmessage.*, </p>
  
  <p>digitalblobreferenceinputnmea.*, </p>
  
  <p>digitalblobreferenceinputwidget.*, </p>
  
  <p>digitalblobreferenceinputscalingfactor.* </p>
  
  <p>FROM rawmessage
  Partition 
  By PartitionId</p>
  
  <p>LEFT JOIN digitalblobreferenceinputnmea ON rawmessage.vessel_id=digitalblobreferenceinputnmea.vessel_id</p>
  
  <p>LEFT JOIN digitalblobreferenceinputwidget 
  ON rawmessage.vessel_id=digitalblobreferenceinputwidget.vessel_id</p>
  
  <p>LEFT JOIN digitalblobreferenceinputscalingfactor
  ON rawmessage.vessel_id=digitalblobreferenceinputscalingfactor.vessel_id</p>
  
  <p>WHERE rawmessage.sensorval IS NOT NULL</p>
  
  <p>) </p>
  
  <p>, </p>
  
  <p>processedmessage AS</p>
  
  <p>( </p>
  
  <p>SELECT</p>
  
  <p>event.vessel_id as vessel_id_fk,</p>
  
  <p>event.iotdevice as device_id_fk,</p>
  
  <p>event.PartitionId as partitionId,
  UDF.getEpochTime(event.EventProcessedUtcTime)as EventProcessedUtcTime,</p>
  
  <p>UDF.getAnalyticsProcessTime('arg')
  as AnalyticsProcessTime</p>
  
  <p>FROM messagetoprocess
  as event 
  Partition By PartitionId</p>
  
  <p>) </p>
  
  <p>--output is writing into service bus queue for socket push</p>
  
  <p>SELECT * 
  INTO digitalqueueoutputonlineclassasocketdata
  FROM processedmessage
  Partition 
  By PartitionId</p>