title: Is there a way to sync up live audio and live video using Python?
body: <p>I am trying to write a bit of code to create a 2 person live chat application using Python 3. The code is working in the sense of having live audio and live video, (as tested by running the server and 2 clients on the same device), with the Client-Side code running most of the logic and running the camera and audio feeds to send through the server, (which also works as a form of handshake between the 2 clients with little logic aside from if a client sends video and/or audio to send it to the other client), to have the client-side code to get the audio and video to play.</p>
  
  <p><em>fair warning, this code is a bit messy and is not optimized for fast performance</em></p>
  
  <p><strong>Client-Side Code:</strong></p>
  
  <pre><code>import cv2
  from socket import socket, AF_INET, SOCK_STREAM
  from imutils.video import WebcamVideoStream
  import pyaudio
  from array import array
  from threading import Thread
  import numpy as np
  import zlib
  import struct
  
  HOST = input("Enter Server IP\n")
  PORT_VIDEO = 3000
  PORT_AUDIO = 4000
  
  BufferSize = 4096
  CHUNK=1024
  lnF = 640*480*3
  FORMAT=pyaudio.paInt16
  CHANNELS=2
  RATE=44100
  
  def SendAudio():
      while True:
          data = stream.read(CHUNK)
          dataChunk = array('h', data)
          vol = max(dataChunk)
          clientAudioSocket.sendall(data)
  
  def RecieveAudio():
      while True:
          data = recvallAudio(BufferSize)
          stream.write(data)
  
  def recvallAudio(size):
      databytes = b''
      while len(databytes) != size:
          to_read = size - len(databytes)
          if to_read &gt; (4 * CHUNK):
              databytes += clientAudioSocket.recv(4 * CHUNK)
          else:
              databytes += clientAudioSocket.recv(to_read)
      return databytes
  
  def SendFrame():
      while True:
          try:
              frame = wvs.read()
              cv2_im = cv2.cvtColor(frame, cv2.COLOR_BGR2RGB)
              frame = cv2.resize(frame, (640, 480))
              frame = np.array(frame, dtype = np.uint8).reshape(1, lnF)
              jpg_as_text = bytearray(frame)
  
              databytes = zlib.compress(jpg_as_text, 9)
              length = struct.pack('!I', len(databytes))
              bytesToBeSend = b''
              clientVideoSocket.sendall(length)
              while len(databytes) &gt; 0:
                  if (5000 * CHUNK) &lt;= len(databytes):
                      bytesToBeSend = databytes[:(5000 * CHUNK)]
                      databytes = databytes[(5000 * CHUNK):]
                      clientVideoSocket.sendall(bytesToBeSend)
                  else:
                      bytesToBeSend = databytes
                      clientVideoSocket.sendall(bytesToBeSend)
                      databytes = b''
              # ~ print("##### Data Sent!! #####")
          except:
              continue
  
  
  def RecieveFrame():
      while True:
          try:
              lengthbuf = recvallVideo(4)
              length, = struct.unpack('!I', lengthbuf)
              databytes = recvallVideo(length)
              img = zlib.decompress(databytes)
              if len(databytes) == length:
                  # ~ print("Recieving Media..")
                  # ~ print("Image Frame Size:- {}".format(len(img)))
                  img = np.array(list(img))
                  img = np.array(img, dtype = np.uint8).reshape(480, 640, 3)
                  cv2.imshow("Stream", img)
                  if cv2.waitKey(1) == 27:
                      cv2.destroyAllWindows()
              else:
                  print("Data CORRUPTED")
          except:
              continue
  
  
  def recvallVideo(size):
      databytes = b''
      while len(databytes) != size:
          to_read = size - len(databytes)
          if to_read &gt; (5000 * CHUNK):
              databytes += clientVideoSocket.recv(5000 * CHUNK)
          else:
              databytes += clientVideoSocket.recv(to_read)
      return databytes
  
  
  
  clientVideoSocket = socket(family=AF_INET, type=SOCK_STREAM)
  clientVideoSocket.connect((HOST, PORT_VIDEO))
  wvs = WebcamVideoStream(0).start()
  
  clientAudioSocket = socket(family=AF_INET, type=SOCK_STREAM)
  clientAudioSocket.connect((HOST, PORT_AUDIO))
  
  audio=pyaudio.PyAudio()
  stream=audio.open(format=FORMAT,channels=CHANNELS, rate=RATE, input=True, output = True,frames_per_buffer=CHUNK)
  
  initiation = clientVideoSocket.recv(5).decode()
  
  if initiation == "start":
      SendFrameThread = Thread(target=SendFrame).start()
      SendAudioThread = Thread(target=SendAudio).start()
      RecieveFrameThread = Thread(target=RecieveFrame).start()
      RecieveAudioThread = Thread(target=RecieveAudio).start()
  </code></pre>
  
  <p><strong>Server-Side Code:</strong></p>
  
  <pre><code>from socket import socket, AF_INET, SOCK_STREAM
  from threading import Thread
  import struct
  
  HOST = input("Enter Host IP\n")
  PORT_VIDEO = 3000
  PORT_AUDIO = 4000
  lnF = 640*480*3
  CHUNK = 1024
  BufferSize = 4096
  addressesAudio = {}
  addresses = {}
  threads = {}
  
  def ConnectionsVideo():
      while True:
          try:
              clientVideo, addr = serverVideo.accept()
              print("{} is connected!!".format(addr))
              addresses[clientVideo] = addr
              if len(addresses) &gt; 1:
                  for sockets in addresses:
                      if sockets not in threads:
                          threads[sockets] = True
                          sockets.send(("start").encode())
                          Thread(target=ClientConnectionVideo, args=(sockets, )).start()
              else:
                  continue
          except:
              continue
  
  def ConnectionsSound():
      while True:
          try:
              clientAudio, addr = serverAudio.accept()
              print("{} is connected!!".format(addr))
              addressesAudio[clientAudio] = addr
              Thread(target=ClientConnectionSound, args=(clientAudio, )).start()
          except:
              continue
  
  def ClientConnectionVideo(clientVideo):
      while True:
          try:
              lengthbuf = recvall(clientVideo, 4)
              length, = struct.unpack('!I', lengthbuf)
              recvall(clientVideo, length)
          except:
              continue
  
  def ClientConnectionSound(clientAudio):
      while True:
          try:
              data = clientAudio.recv(BufferSize)
              broadcastSound(clientAudio, data)
          except:
              continue
  
  def recvall(clientVideo, BufferSize):
          databytes = b''
          i = 0
          while i != BufferSize:
              to_read = BufferSize - i
              if to_read &gt; (1000 * CHUNK):
                  databytes = clientVideo.recv(1000 * CHUNK)
                  i += len(databytes)
                  broadcastVideo(clientVideo, databytes)
              else:
                  if BufferSize == 4:
                      databytes += clientVideo.recv(to_read)
                  else:
                      databytes = clientVideo.recv(to_read)
                  i += len(databytes)
                  if BufferSize != 4:
                      broadcastVideo(clientVideo, databytes)
          # ~ print("YES!!!!!!!!!" if i == BufferSize else "NO!!!!!!!!!!!!")
          if BufferSize == 4:
              broadcastVideo(clientVideo, databytes)
              return databytes
  
  def broadcastVideo(clientSocket, data_to_be_sent):
      for clientVideo in addresses:
          if clientVideo != clientSocket:
              clientVideo.sendall(data_to_be_sent)
  
  def broadcastSound(clientSocket, data_to_be_sent):
      for clientAudio in addressesAudio:
          if clientAudio != clientSocket:
              clientAudio.sendall(data_to_be_sent)
  
  serverVideo = socket(family=AF_INET, type=SOCK_STREAM)
  try:
      serverVideo.bind((HOST, PORT_VIDEO))
  except OSError:
      print("Server Busy")
  
  serverAudio = socket(family=AF_INET, type=SOCK_STREAM)
  try:
      serverAudio.bind((HOST, PORT_AUDIO))
  except OSError:
      print("Server Busy")
  
  serverAudio.listen(2)
  print("Waiting for audio connection..")
  AcceptThreadAudio = Thread(target=ConnectionsSound)
  AcceptThreadAudio.start()
  
  
  serverVideo.listen(2)
  print("Waiting for video connection..")
  AcceptThreadVideo = Thread(target=ConnectionsVideo)
  AcceptThreadVideo.start()
  AcceptThreadVideo.join()
  serverVideo.close()
  </code></pre>
  
  <p>Both of these codes are what I have put together in a bit longer of time than I would have liked, but I noticed when doing a bit of testing with these codes that the audio and video were off by anywhere between 3 and 7 seconds. I tried a bit of troubleshooting but couldn't find any issues with the code, aside from the fact that it looks like a kid threw it all together. I am fine with importing a new Python package, but I would prefer to use the Python Packages that I already have in place. Keep in mind I am thinking about eventually converting both the codes into separate .exe files to allow for easy access to send the codes to individual people.</p>