Image 1 of 1
Overview
The VStreamerMediaMtx C++ library provides RTSP, WebRTC, SRT (optional, not tested in version 1.1.1), RTPM (optional, not tested in version 1.1.1) and HLS (optional, not tested in version 1.1.1) video streaming without audio. The library inherits interface from VStreamer. The library depends on: OpenCV (version >= 4.5) library (running GStreamer pipeline and video scaling, linked, Apache 2.0 license), GStreamer (video encoding and streaming to mediamtx, run as external process by OpenCV, not included in code), mediamtx (provides video server, run as external process, not included in code, MIT license), VCodec (not used, defines video codec interface, source code included, Apache 2.0 license), VOverlay (defines video overlay interface, source code included, Apache 2.0 license), FormatConverterOpenCv (provides methods to convert pixel formats, source code included), ConfigReader (provides functions to read / write JSON files, source code included) and ChildProcess (provides functions to run child process, source code included). The library supports changing RTSP parameters such as: RTSP server port, IP, stream name, user name, password etc. The library works frame-by-frame and accepts video frames in formats: RGB24, BGR24, YUYV, UYVY, GRAY, YUV24, NV12, NV21, YU12 and YV12. Principle of operation: when user sends frame (call sendFrame(…) function) the library copies video frame to internal buffer. After initialization the library runs internal thread. Internal thread is taking new frame according to FPS (set by user in parameters). After taking new frame the library does frame scaling (according to scaling mode if input frame size different from video stream size) and put overlay information (if overlay engine is initialized by user). After preparing frame the library runs (once) GStreamer pipeline and sends video frame to it. GStreamer pipeline includes video encoder (software or hardware, set by user). GStreamer pipeline streams data to external process mediamtx (user should run it in advance or using auto run provided by the library) which provides video server service for clients. All instances of the library should use the same RTSP port, user name and password, RTSP multicast IP and port. The version 1.1.1 of the library support H264, H265 and JPEG video encoding on CPU, Intel GPU and Nvidia Jetson platforms using GStreamer. Bellow the working principle:
Key features:
All instances of VStreamerMediaMtx class use the same RTSP server port, user name, password, multicast port and multicast IP. If user change one of this parameters in one instance the all other VStreamerMediaMtx instances in user application will be restarted automatically with new parameters. The mediamtx process provides RTSP server on the same port for all streams (Difference only in stream name). For example: rtsp://127.0.0.1:7031/Camera1Stream1 (first instance of VStreamerMediaMtx), rtsp://127.0.0.1:7031/Camera1Stream2 (second instance of VStreamerMediaMtx) etc.
RTSP server port, user name, password, multicast IP and multicast port are configurable in mediamtx configuration file (mediamtx.yml). To change RTSP server port, user name, password, multicast IP or multicast port user must stop mediamtx process, change mediamtx.yml file and run mediamtx process again. The library provides automatic mediamtx configuration if user provides path to mediamtx executable file.
The VStreamerMediaMtx supports video overlay (user must initialize VOverlay object if overlay required).
VCodec interface not used. The library compatible with Linux operating systems only. The video server compatible with all popular RTSP clients: ffmpeg, gstreamer, VLC, Milestone etc.
IMPORTANT: the library uses GStreamer pipeline for video encoding. The library can use software encoders or hardware encoders. Type of codec set by user in parameters. Version 1.1.1 of the library supports following GStreamer software codecs (on CPU): x264enc for H264 video encoding, x265enc for H265 video encoding and jpegenc for JPEG video encoding. The library supports following hardware codecs: vaapih264enc or nvv4l2h264enc for H264 video encoding and vaapih265enc or nvv4l2h265enc for H265 video encoding. For JPEG encoding it uses nvjpegenc in NVDIA JETSON platform or jpegenc software codec for all other platforms.
Documentation
Documentation: GO TO DOCUMENTATION
Simple interface
class VStreamerMediaMtx : public cr::video::VStreamer
{
public:
/// Get string of current library version.
static std::string getVersion();
/// Init video streamer by set of parameters.
bool initVStreamer(cr::video::VStreamerParams ¶ms,
cr::video::VCodec *codec = nullptr,
cr::video::VOverlay *overlay = nullptr) override;
/// Get init status.
bool isVStreamerInit() override;
/// Close video streamer.
void closeVStreamer() override;
/// Send frame to video streamer.
bool sendFrame(cr::video::Frame& frame) override;
/// Set video streamer parameter.
bool setParam(cr::video::VStreamerParam id, float value) override;
/// Set video streamer parameter.
bool setParam(cr::video::VStreamerParam id, std::string value) override;
/// Get all video streamer parameters.
void getParams(cr::video::VStreamerParams& params) override;
/// Execute action command.
bool executeCommand(cr::video::VStreamerCommand id) override;
/// Set media mtx path.
static void setMediaMtxPath(std::string path, bool isSingleMediaMTX = true);
};
Simple example
#include <iostream>
#include <chrono>
#include <opencv2/opencv.hpp>
#include "VStreamerMediaMtx.h"
int main(int argc, char **argv)
{
// Set suffix (stream name).
std::string suffix = "live";
std::cout << "Set suffix: ";
std::cin >> suffix;
// Set RTSP server port.
int port = 8554;
std::cout << "Set RTSP server port: ";
std::cin >> port;
// Set mediamtx path
cr::video::VStreamerMediaMtx::setMediaMtxPath("/home/ahmet/Projects/");
// Init video streamer params.
cr::video::VStreamerParams params;
params.enable = true;
params.width = 640;
params.height = 480;
params.fps = 30.0f;
params.bitrateKbps = 7000;
params.gop = 30;
params.port = port;
params.codec = "H264";
params.h264Profile = 1;
params.jpegQuality = 58;
params.custom1 = 0.0f; // 0 - Software, 1 - Intel hardware, 2 - Jetson hardware
cr::video::VStreamerMediaMtx streamer;
if (!streamer.initVStreamer(params))
{
std::cout << "Video streamer init failed" << std::endl;
return -1;
}
// Create frame.
cr::video::Frame frame =
cr::video::Frame(params.width, params.height, cr::video::Fourcc::YUV24);
// Moving object parameters.
int objectX = 200;
int objectY = 200;
int objectVx = 1;
int objectVy = 1;
int border = 100;
int objectSize = 50;
uint8_t color = 0;
// Main loop.
while (true)
{
// Move object.
objectX += objectVx;
objectY += objectVy;
if (objectX < border || objectX > params.width - border - objectSize)
objectVx = -objectVx;
if (objectY < border || objectY > params.height - border - objectSize)
objectVy = -objectVy;
// Draw object.
memset(frame.data, color++, frame.size);
cv::Mat image(params.height, params.width, CV_8UC3, frame.data);
cv::rectangle(image, cv::Rect(objectX, objectY, objectSize, objectSize),
cv::Scalar(0, 0, 255), -1);
// Add frame to streamer.
if (!streamer.sendFrame(frame))
{
std::cout << "Send frame failed" << std::endl;
break;
}
std::this_thread::sleep_for(std::chrono::milliseconds(30));
}
return 0;
}
