# Issues encoding image captured using cv2.VideoCapture(). Goal is to display image in an iPywidget

Hello;

In my use case, I'm using a Jetson Nano running Jupyter Lab. I ssh into the nano, and run Jupyter Lab on a web browser on the host machine, a laptop.

As such, I can't use cv2.imshow() to display the image. This only works when run locally.

Therefore, I've learned to use cv2.VideoCapture() to capture an image, but I'm having trouble encoding this image into the format required by ipywidgets.

Here is my code for capturing one camera frame:

import cv2

import ipywidgets
from IPython.display import display
from IPython.display import Image

gst_pipeline = "nvarguscamerasrc sensor-id=0 ! video/x-raw(memory:NVMM), width=(int)1280, height=(int)720, format=(string)NV12, framerate=(fraction)30/1 ! nvvidconv ! video/x-raw ! jpegenc ! image/jpeg ! appsink"

camera = cv2.VideoCapture(gst_pipeline, cv2.CAP_GSTREAMER)

print(capture_state)

print(captured_image)


The output of this code produces these results:

capture_state = True

captured_image = [[255 216 255 ... 179 255 217]]

However, when I run the next code sequence, I get the following error:

error: OpenCV(4.1.1) /home/nvidia/host/build_opencv/nv_opencv/modules/imgcodecs/src/grfmt_base.cpp:145: error: (-10:Unknown error code -10) Raw image encoder error: Maximum supported image dimension is 65500 pixels in function 'throwOnEror'


This is the code I run, that produces that error:

encode_parameters = [int(cv2.IMWRITE_JPEG_QUALITY),20]

encode_state, encoded_image = bytes(cv2.imencode('.jpg', captured_image, encode_parameters))


In looking around the internt, it seems there is a need to do some sort of transform. In this code example, they are using a numpy array as shown here:

def __init__(self, *args, **kwargs):
super(Camera, self).__init__(*args, **kwargs)
if self.format == 'bgr8':
self.value = np.empty((self.height, self.width, 3), dtype=np.uint8)
self._running = False


I would really appreciate if someone could help me figure out what I need to do to get my "capture_image" into the right format so that I can do the encoding.

Once this step is good, I'll be able to view this image in an iPywidget - accomplishing my overall goal.

Cheers!

edit retag close merge delete

U can't declared variable encode_parameters = . See the answer cv2.imencode

Thanks for your comment, I checked out what you sent me but it didn't work in my case. However; I did find a solution. I'll post it on the main quesiton though.

Sort by » oldest newest most voted

I found the solution to my issue.

The problem is that iPywidgets is expecting a binary value, while OpenCV stores the image in an arrary format. You have to do a conversion of an integer array to a binary stream in order to use as in an iPywidget. Here is how you do this in Jupyter.

Start off by importing the correct libraries:

import cv2
import numpy as np

import ipywidgets
from IPython.display import display
from IPython.display import Image


Next build your image capture pipline and prove it works:

gst_pipeline = "nvarguscamerasrc sensor-id=0 ! video/x-raw(memory:NVMM), width=(int)1280, height=(int)720, format=(string)NV12, framerate=(fraction)30/1 ! nvvidconv ! video/x-raw, format=(string)NV12, width=(int)640, height=(int)480  ! videoconvert ! appsink"

camera = cv2.VideoCapture(gst_pipeline, cv2.CAP_GSTREAMER)

print(capture_state)

print(captured_image.shape)

print(type(captured_image.shape))

print(captured_image.size)

print(captured_image.dtype)


Here are the outputs I get from these print statements:

True
(480, 640, 3)
<class 'tuple'>
921600
uint8


To me this is proves an image was captured and that it's stored in an array.

Next encode the image:

encode_parameters = [int(cv2.IMWRITE_JPEG_QUALITY),75]

encode_state, encoded_image = cv2.imencode('.jpg', captured_image, encode_parameters)

print(encode_state)

print(encoded_image.shape)

print(type(encoded_image.shape))

print(encoded_image.size)

print(captured_image.dtype)


Here are the outputs I got:

True
(26978, 1)
<class 'tuple'>
26978
uint8


Next is the critical step. This is the step that converts the integer array into a binary stream:

binary_image = encoded_image.tobytes()


This will do the conversion for you.

Next, link the binary_image object to an Ipywidget, and viola! You get the displayed image from your CSI camera!

image_widget = ipywidgets.Image(value=binary_image, format='jpeg', width=224, height=224)

display_window = ipywidgets.HBox([image_widget])

display(display_window)


Following these steps produced and image widget in Jupyter Lab, and works either local or remote (using ssh into my device).

I still need to refine the steps for capturing a video stream, but it's going to be very similar to these steps.

Cheers!

more

Official site

GitHub

Wiki

Documentation
{"ft_lang_label":"__label__en","ft_lang_prob":0.7554168,"math_prob":0.73519164,"size":2521,"snap":"2021-43-2021-49","text_gpt3_token_len":641,"char_repetition_ratio":0.106475964,"word_repetition_ratio":0.0,"special_character_ratio":0.25862753,"punctuation_ratio":0.1875,"nsfw_num_words":0,"has_unicode_error":false,"math_prob_llama3":0.9547965,"pos_list":[0],"im_url_duplicate_count":[null],"WARC_HEADER":"{\"WARC-Type\":\"response\",\"WARC-Date\":\"2021-10-16T03:47:36Z\",\"WARC-Record-ID\":\"<urn:uuid:49199864-4524-4561-ac6f-e6ebe46200f4>\",\"Content-Length\":\"61453\",\"Content-Type\":\"application/http; msgtype=response\",\"WARC-Warcinfo-ID\":\"<urn:uuid:bf998f12-5574-461b-bee3-3df53f49564e>\",\"WARC-Concurrent-To\":\"<urn:uuid:9f401754-a4d1-4b07-b297-2ea404006e28>\",\"WARC-IP-Address\":\"104.21.24.86\",\"WARC-Target-URI\":\"https://answers.opencv.org/question/237210/issues-encoding-image-captured-using-cv2videocapture-goal-is-to-display-image-in-an-ipywidget/\",\"WARC-Payload-Digest\":\"sha1:R4S2H7DM2EAW3UQV7OFNXOQ5Z4R3NPOL\",\"WARC-Block-Digest\":\"sha1:GSGVDBLBTFWN22Q3YM72AT275CS2TZW6\",\"WARC-Identified-Payload-Type\":\"application/xhtml+xml\",\"warc_filename\":\"/cc_download/warc_2021/CC-MAIN-2021-43/CC-MAIN-2021-43_segments_1634323583408.93_warc_CC-MAIN-20211016013436-20211016043436-00675.warc.gz\"}"}