I would like to synchronize my video and voltage data using Python. (I obtain these data using Bonsai and Open Ephys for an open field exploration experiment.)

I sent $150\text{ ms}$ long TTL pulses to a computer recording the voltage data, at the same time as to an LED in the field of view of my camera to turn it on. I sent the pulses at random times with 20-60 second gaps. I would like to align the LED pulses with the TTLs to synchronize my datasets.

The two recordings did not necessarily start and end at the same time, so it was possible for one of the datasets to have fewer pulses than the other one.

The voltage data was recorded at $30000\text{ Hz}$ sampling rate, and the video at about $30\text{ Hz}$.

I tried to downsample the voltage data, correlate the two arrays and find the maximum correlation to calculate the lag and then shift the video data by the offset.

My first problem is downsampling the voltage data. I calculated the real sampling rates from the average step size, and found that the camera is about $30.08207\text{ Hz}$.

I tried to use [resampy's resample](https://resampy.readthedocs.io/en/stable/example.html):

```
resampy.resample(voltage_data, voltage_data_sampling_rate, video_sampling_rate)

```

where voltage\_data\_sampling rate was $30000\text{ Hz}$ and video\_sampling\_rate $30.08207\text{ Hz}$. This returns an array that is the same size as the original voltage array.

***1. How could I get a signal from this that has the same sampling
 rate as my video?***

To resample my data without using this I generated an array with multiples of the rate of sampling rates ($\frac{30000}{30.082074} = 997.271$)), converted these to integers and took these indices. I understand that this is very basic, but I thought it would allow me to check if I can write the rest of this code.

I calculated the correlation using `numpy.correlate`:

```
correlation = np.correlate(video, voltage, "full")

```

I set a threshold based on standard deviation and the median and replaced all values with $0s$ below threshold to remove noise.

I checked which array was shorter and padded the end with $0s$.

```
short_array = np.pad(short_array, (0, len(long_array)-len(short_array)), 'constant')

```

I calculated the lag between the two signals based on the maximum correlation:

```
lag = (np.argmax(corr) - (corr.size + 1)/2)/avg_sampling_rate_video

```

This result is about right. It reduces my 2-5 second lags to about $100\text{ ms}$.

```
plt.plot(downsampled_voltage_time, downsampled_voltage_TTL)
plt.plot(video_time, video_sync_pulses)

```

[![example synchronized peak](https://i.stack.imgur.com/NtUkF.png)](https://i.stack.imgur.com/NtUkF.png)
[![enter image description here](https://i.stack.imgur.com/VcRyE.png)](https://i.stack.imgur.com/VcRyE.png)

I need this to be better than that, I would like the lag to be near $30\text{ ms}$. 

***2. Is the downsampling causing this problem, or are there other problems with this approach?***