Bug in Videotexturing in BGE #41004

New Issue

Jean-Marc Vezien · 2014-07-09T17:08:26+02:00

Jean-Marc Vezien commented

2014-07-09 17:08:26 +02:00

System Information
Windows:64bit details here: system-info.txt

Blender Version
version 2.70 (sub 0),

Short description of error
Bugs in VideoTexturing a plane via FFMPEG
Exact steps for others to reproduce the error

The attached test blend file (video_texture_bge_debug.blend) video_texture_bge_debug.blend with a cube and a plane, exhibits two possible bugs in BGE related to videotexturing. See python script Movie.py.

Bug 1: Videotexturing of a plane with VideoFFMpeg call seems to work with file inputs ou webcam, but fails to display anything
with a rtsp stream as input. Note: An RTSP stream server can be generated with VLC:

vlc -vvv your_file.avi --sout "#rtp{dst=dest_adress,port=1234,sdp=rtsp://your_ip_adress:8080/test.sdp}"

In the script code attached to the plane in the example, the plane is textured by default with a file which name is inputed in the game logic properties
of the plane (sample: Video_jmv_right.avi and Video_jmv_left.avi, not included - you can put any decent avi files instead). Works fine.
Just below in the code (commented) are the corresponding rtsp calls, which DO NOT work on my machine.

My hunch is a problem in FFMPEG default settings with RTSP in Blender, which requires some buffering time (say 500 ms). For example VLC handles RTSP quite well in the "open stream" Ctrl-N option of the VLC GUI, with a default 1000ms buffering.

Bug 2: In stereo mode, the provided example should texture the plane with two different videos, one for each eye. I provided a pre-draw routine which is applied
(See outputs on console), but does not result in the appropriate rendering in stereo.
Obviously one needs a stereo capable hardware to test this. I have Nvidia Quadro cards.

Not sure this is a bug, maybe I do this wrong, I need suggestions on this.

**System Information** Windows:64bit details here: [system-info.txt](https://archive.blender.org/developer/F97191/system-info.txt) **Blender Version** version 2.70 (sub 0), **Short description of error** Bugs in VideoTexturing a plane via FFMPEG **Exact steps for others to reproduce the error** The attached test blend file (video_texture_bge_debug.blend) [video_texture_bge_debug.blend](https://archive.blender.org/developer/F97190/video_texture_bge_debug.blend) with a cube and a plane, exhibits two possible bugs in BGE related to videotexturing. See python script Movie.py. Bug 1: Videotexturing of a plane with VideoFFMpeg call seems to work with file inputs ou webcam, but fails to display anything with a rtsp stream as input. Note: An RTSP stream server can be generated with VLC: vlc -vvv your_file.avi --sout "#rtp{dst=dest_adress,port=1234,sdp=rtsp://your_ip_adress:8080/test.sdp}" In the script code attached to the plane in the example, the plane is textured by default with a file which name is inputed in the game logic properties of the plane (sample: Video_jmv_right.avi and Video_jmv_left.avi, not included - you can put any decent avi files instead). Works fine. Just below in the code (commented) are the corresponding rtsp calls, which DO NOT work on my machine. My hunch is a problem in FFMPEG default settings with RTSP in Blender, which requires some buffering time (say 500 ms). For example VLC handles RTSP quite well in the "open stream" Ctrl-N option of the VLC GUI, with a default 1000ms buffering. Bug 2: In stereo mode, the provided example should texture the plane with two different videos, one for each eye. I provided a pre-draw routine which is applied (See outputs on console), but does not result in the appropriate rendering in stereo. Obviously one needs a stereo capable hardware to test this. I have Nvidia Quadro cards. Not sure this is a bug, maybe I do this wrong, I need suggestions on this.

Jean-Marc Vezien commented

2014-07-09 17:08:26 +02:00

Changed status to: 'Open'

Jean-Marc Vezien commented

2014-07-09 17:08:26 +02:00

Added subscriber: @occamjmv

Jean-Marc Vezien commented

2014-07-09 17:10:12 +02:00

I reported two bugs in one , as they are tightly related (video texturing). A split in two trackers is possible.

Inês Almeida commented

2014-07-11 22:40:50 +02:00

Added subscribers: @kupoman, @dfelinto, @brita

Inês Almeida commented

2014-07-11 22:40:50 +02:00

I don't feel a need to split this in two bugs for now.
@dfelinto can you take a look at the bug 2 stereo part?

As to bug 1.. @occamjmv you can display a video, but not a rtsp stream?
I am not sure this is supported. It probably was never tested, but if it is a matter of using ffmpeg with the correct settings maybe it is doable? I don't know nearly enough about this.. @kupoman any idea of whom to talk to?

I don't feel a need to split this in two bugs for now. @dfelinto can you take a look at the bug 2 stereo part? As to bug 1.. @occamjmv you can display a video, but not a rtsp stream? I am not sure this is supported. It probably was never tested, but if it is a matter of using ffmpeg with the correct settings maybe it is doable? I don't know nearly enough about this.. @kupoman any idea of whom to talk to?

Dalai Felinto commented

2014-07-14 23:32:22 +02:00

I'm investigating that, the rtsp seems simply the lack of building ffmpeg for blender with that support " --enable-network --enable-protocol=tcp --enable-demuxer=rtsp --enable-decoder=h264". I'm getting the ffmpeg sources to see if that fixes it

Dalai Felinto commented

2014-07-15 14:52:56 +02:00

Re: rtsp
@occamjmv testing with the new OSX libraries fixed the initial issue, but now Blender freezes when the script calls for refresh(True).

Could you possibly build ffmpeg in your OS (windows, though it may be easier if you have a linux station and build there first) to see if it works?

Re: rtsp @occamjmv testing with the new OSX libraries fixed the initial issue, but now Blender freezes when the script calls for **refresh**(True). Could you possibly build ffmpeg in your OS (windows, though it may be easier if you have a linux station and build there first) to see if it works?

Dalai Felinto commented

2014-07-15 15:11:54 +02:00

Added subscriber: @BenoitBolsee

Dalai Felinto commented

2014-07-15 15:11:54 +02:00

@BenoitBolsee could you take look at this (the rtsp part)? or at least pointing to some directions to try a fix. I think the problem is on VideoFFmpeg.cpp:1004

*find the correct frame, in case of streaming and no cache, it means just* return the next frame. This is not quite correct, may need more work

	while (av_read_frame(m_formatCtx, &packet) >= 0)

It's where it is hanging for me at least. (you need a recompiled ffmpeg libraries with the updated flags but Sergey said he was to update the libs for Windows soon).

@BenoitBolsee could you take look at this (the rtsp part)? or at least pointing to some directions to try a fix. I think the problem is on VideoFFmpeg.cpp:1004 *find the correct frame, in case of streaming and no cache, it means just* return the next frame. This is not quite correct, may need more work ``` while (av_read_frame(m_formatCtx, &packet) >= 0) ``` It's where it is hanging for me at least. (you need a recompiled ffmpeg libraries with the updated flags but Sergey said he was to update the libs for Windows soon).

Dalai Felinto commented

2014-07-15 15:59:15 +02:00

re: stereo
@occamjmv what is the output you get from this file? Since I don't have a quadbuffer card I only always get the 'BACK' buffer, is the buffer swapping prints working as expected? I would expect this to be an issue of pre_draw() not being called once per stereo eye, and not an issue in videotexture

re: stereo @occamjmv what is the output you get from this file? Since I don't have a quadbuffer card I only always get the 'BACK' buffer, is the buffer swapping prints working as expected? I would expect this to be an issue of pre_draw() not being called once per stereo eye, and not an issue in videotexture

Benoit Bolsee commented

2014-07-16 17:37:08 +02:00

Don't remember if I tested rstp streaming, probably not (I tested http streaming though). In the case of rtsp streaming it is possible that this function call is missing:
av_read_play(m_formatCtx)
It seems necessary to kick off the streaming. The most logical place is to call it in VideoFFmpeg::play(), only is m_isStreaming is true of course.
The corresponding av_read_pause() should be used in VideoFFmpeg::pause()

To be checked: opening a rstp input should cause the m_isStreaming flag to be set (line 554)

Don't remember if I tested rstp streaming, probably not (I tested http streaming though). In the case of rtsp streaming it is possible that this function call is missing: av_read_play(m_formatCtx) It seems necessary to kick off the streaming. The most logical place is to call it in VideoFFmpeg::play(), only is m_isStreaming is true of course. The corresponding av_read_pause() should be used in VideoFFmpeg::pause() To be checked: opening a rstp input should cause the m_isStreaming flag to be set (line 554)

Dalai Felinto commented

2014-07-16 22:32:21 +02:00

@BenoitBolsee Thanks. Only forcing m_isStreaming works already (P89), but the play/pause changes don't seem to interfere (P90). Which one should I push 89 or 90?

@BenoitBolsee Thanks. Only forcing m_isStreaming works already ([P89](https://archive.blender.org/developer/P89.txt)), but the play/pause changes don't seem to interfere ([P90](https://archive.blender.org/developer/P90.txt)). Which one should I push 89 or 90?

Benoit Bolsee commented

2014-07-17 14:03:29 +02:00

Added subscriber: @dalai-1

Benoit Bolsee commented

2014-07-17 14:03:29 +02:00

@dalai-1 looking at ffmpeg code, av_read_play is implemented for the rtsp decoder (and only for it), so P90 is the correct one.

@dalai-1 looking at ffmpeg code, av_read_play is implemented for the rtsp decoder (and only for it), so [P90](https://archive.blender.org/developer/P90.txt) is the correct one.

Dalai Felinto commented

2014-07-17 15:34:54 +02:00

Thanks @BenoitBolsee, committed on 6f846da2 (windows and linux need to build libraries with this settings still, I'm debating on the bf-committers mailing list to see if the official libs adopt that, otherwise they need to be custom build when rtsp is needed).

I'll wait @occamjmv reply to investigate the other issue further. It's hard to debug this without having their Quadbuffer setup at hand.

Thanks @BenoitBolsee, committed on 6f846da2 (windows and linux need to build libraries with this settings still, I'm debating on the bf-committers mailing list to see if the official libs adopt that, otherwise they need to be custom build when rtsp is needed). I'll wait @occamjmv reply to investigate the other issue further. It's hard to debug this without having their Quadbuffer setup at hand.

Dalai Felinto commented

2014-07-17 15:35:09 +02:00

Removed subscriber: @dalai-1