Geek asks and answers

Failed to route video from webcam to virtual video device in Linux (via OpenCV)

I would like to put a video overlay on the webcam's input stream through OpenCV. As a first step, I am trying to transfer streaming video from a webcam to /dev/video0a virtual video device in /dev/video1(v4l2loopback). Unfortunately, I cannot redirect the webcam stream to the v4l2loopback device.

I found this topic: How to write / connect to a virtual webcam created by the V4L2loopback module? but the links there really didn't help me.

Receiving and viewing a stream from a webcam works well with a small demo code on the OpenCV page. Also sample code from v4l2loopback page, for example. playing a static video file up /dev/video1works well. I can watch the video when I connect the VLC to/dev/video1

I already read that the v4l2loopback device should be controlled using the usual Linux driver commands (open, ioctl, write, etc.). Thus, in OpenCV there is no wrapper class to write to the loopback device. My streams are webcams with 640x480 and as MJPG. The interesting thing is that I can connect to the loopback device with VLC, and I can see the correct resolution, Codec and FPS displayed when I click on play. The time counter on the progress bar also starts to work. But the screen remains black (with the VLC logo).

What I'm rudely doing is (BTW: the code may not compile ... didn't want everyone to clutter here ... let me know if you need more details):

int main ( int argc, char **argv ) {
    cv::VideoCapture cap;
    struct v4l2_format vid_format;
    size_t framesize = 640 * 480 * 3; // 3 Bytes per  pixel
    __u8 *buffer = null;
    int fd = null;

    cap.open  ( "/dev/video0" );
    fd = open ( "/dev/video1", O_RDWR );

    memset ( &vid_format, 0, sizeof(vid_format) );
    vid_format.type                = V4L2_BUF_TYPE_VIDEO_OUTPUT;
    vid_format.fmt.pix.width       = cap.get(CV_CAP_PROP_FRAME_WIDTH);
    vid_format.fmt.pix.height      = cap.get(CV_CAP_PROP_FRAME_HEIGHT);
    vid_format.fmt.pix.pixelformat = { 'M', 'J', 'P', 'G' };
    vid_format.fmt.pix.sizeimage   = framesize;
    vid_format.fmt.pix.field       = V4L2_FIELD_NONE;
    vid_format.fmt.pix.colorspace  = V4L2_COLORSPACE_SRGB;
    ioctl ( fd, VIDIOC_S_FMT, &vid_format );

    buffer = (__u8*) malloc ( sizeof(__u8) *framesize );
    memset ( buffer, 0, framesize );

    for(;;) {
        cv::Mat frame;
        cap >> frame;
        write ( fd, &frame.data, framesize );
    }
}

It would be great if someone could give me a hint on how I need to convert wecam data to get him to accept VLC.

0
source share
1 answer

After much research, I was finally able to develop a working solution. There are many steps that need to be completed and which I will discuss in detail below:

Are common

, , - , , , , VLC. , .

1) v4l2loopback

v4l2loopback - ( ) Linux. https://github.com/umlaeute/v4l2loopback. :

make
sudo make install
sudo modprobe v4l2loopback

Chrome (WebRTC), :

sudo modprobe v4l2loopback exclusive_caps=1

: , v4l2loopback , , . FullHD, . , , 800x600 v4l2loopback , cmd max_width max_height. 640x480. , :

modinfo v4l2loopback

2) OpenCV

OpenCV - , . OpenCV http://docs.opencv.org/3.0-beta/doc/tutorials/introduction/linux_install/linux_install.html, .

3)

/ :

g++ -ggdb `pkg-config --cflags --libs opencv` sample.cpp -o sample
./sample

:

#include <stdio.h>
#include <fcntl.h>
#include <errno.h>
#include <unistd.h>
#include <sys/ioctl.h>
#include <linux/videodev2.h>
#include "opencv2/opencv.hpp"

#define VIDEO_OUT "/dev/video0" // V4L2 Loopack
#define VIDEO_IN  "/dev/video1" // Webcam

#define WIDTH  1920
#define HEIGHT 1080


int main ( int argc, char **argv ) {
    cv::VideoCapture cap;
    struct v4l2_format vid_format;
    size_t framesize = WIDTH * HEIGHT * 3;
    int fd = 0;

    if( cap.open ( VIDEO_IN ) ) {
        cap.set ( cv::CAP_PROP_FRAME_WIDTH , WIDTH  );
        cap.set ( cv::CAP_PROP_FRAME_HEIGHT, HEIGHT );
    } else {
        std::cout << "Unable to open video input!" << std::endl;
    }

    if ( (fd = open ( VIDEO_OUT, O_RDWR )) == -1 )
        printf ("Unable to open video output!");

    memset ( &vid_format, 0, sizeof(vid_format) );
    vid_format.type = V4L2_BUF_TYPE_VIDEO_OUTPUT;

    if ( ioctl ( fd, VIDIOC_G_FMT, &vid_format ) == -1 )
        printf ( "Unable to get video format data. Errro: %d\n", errno );

    vid_format.fmt.pix.width       = cap.get ( CV_CAP_PROP_FRAME_WIDTH  );
    vid_format.fmt.pix.height      = cap.get ( CV_CAP_PROP_FRAME_HEIGHT );
    vid_format.fmt.pix.pixelformat = V4L2_PIX_FMT_RGB24;
    vid_format.fmt.pix.sizeimage   = framesize;
    vid_format.fmt.pix.field       = V4L2_FIELD_NONE;

    if ( ioctl ( fd, VIDIOC_S_FMT, &vid_format ) == -1 )
        printf ( "Unable to set video format! Errno: %d\n", errno );

    cv::Mat frame ( cap.get(CV_CAP_PROP_FRAME_HEIGHT), 
    cap.get(CV_CAP_PROP_FRAME_WIDTH), CV_8UC3 );

    printf ( "Please open the virtual video device (/dev/video<x>) e.g. with VLC\n" );

    while (1) {
        cap >> frame;
        cv::cvtColor ( frame, frame, cv::COLOR_BGR2RGB ); // Webcams sometimes deliver video in BGR not RGB. so we need to convert
        write ( fd, frame.data, framesize );
    }
}
0

More articles:


All Articles