[Bf-blender-cvs] SVN commit: /data/svn/bf-blender [19200] trunk/blender/source/gameengine: VideoTexture: reactivate VideoTexture for scons/cmake/ makefile compilation systems, fix video streaming, fix camera support in Linux, add multi-thread cache service, fix crash when a VideoFFmpeg object could not be created.
Benoit Bolsee
benoit.bolsee at online.be
Thu Mar 5 16:16:44 CET 2009
Revision: 19200
http://projects.blender.org/plugins/scmsvn/viewcvs.php?view=rev&root=bf-blender&revision=19200
Author: ben2610
Date: 2009-03-05 16:16:43 +0100 (Thu, 05 Mar 2009)
Log Message:
-----------
VideoTexture: reactivate VideoTexture for scons/cmake/makefile compilation systems, fix video streaming, fix camera support in Linux, add multi-thread cache service, fix crash when a VideoFFmpeg object could not be created.
The multi-thread cache service is activated only on multi-core processors.
It consists in loading, decoding and caching the video frames in a
separate thread. The cache size is 5 decoded frames and 30 raw frames.
Note that the opening of video file/stream/camera is not multi-thread:
you will still experience a delay at the VideoFFmpeg object creation.
Processing of the video frame (resize, loading to texture) is still done
in the main thread. Caching is automatically enabled for video file,
video streaming and video camera.
Video streaming now works correctly: the videos frames are loaded
at the correct rate. Network delays and frequency drifts are automatically
compensated.
Note: an http video source is always treated as a streaming source,
even though the http protocol allows seeking. For the user it means that
he cannot define start/stop range and cannot restart the video except
by reopening the source. Pause/play is however possible.
Video camera is now correctly handled on Linux: it will not slow down the BGE.
A video camera is treated as a streaming source.
Modified Paths:
--------------
trunk/blender/source/gameengine/BlenderRoutines/CMakeLists.txt
trunk/blender/source/gameengine/BlenderRoutines/Makefile
trunk/blender/source/gameengine/BlenderRoutines/SConscript
trunk/blender/source/gameengine/VideoTexture/CMakeLists.txt
trunk/blender/source/gameengine/VideoTexture/Makefile
trunk/blender/source/gameengine/VideoTexture/SConscript
trunk/blender/source/gameengine/VideoTexture/VideoFFmpeg.cpp
trunk/blender/source/gameengine/VideoTexture/VideoFFmpeg.h
Modified: trunk/blender/source/gameengine/BlenderRoutines/CMakeLists.txt
===================================================================
--- trunk/blender/source/gameengine/BlenderRoutines/CMakeLists.txt 2009-03-05 14:30:36 UTC (rev 19199)
+++ trunk/blender/source/gameengine/BlenderRoutines/CMakeLists.txt 2009-03-05 15:16:43 UTC (rev 19200)
@@ -38,5 +38,9 @@
${PYTHON_INC}
)
+IF(WITH_FFMPEG)
+ ADD_DEFINITIONS(-DWITH_FFMPEG)
+ENDIF(WITH_FFMPEG)
+
BLENDERLIB(bf_blroutines "${SRC}" "${INC}")
#env.BlenderLib ( 'bf_bloutines', sources, Split(incs), [], libtype=['game', 'game2', 'player'], priority=[0, 0, 55] , compileflags=cxxflags)
Modified: trunk/blender/source/gameengine/BlenderRoutines/Makefile
===================================================================
--- trunk/blender/source/gameengine/BlenderRoutines/Makefile 2009-03-05 14:30:36 UTC (rev 19199)
+++ trunk/blender/source/gameengine/BlenderRoutines/Makefile 2009-03-05 15:16:43 UTC (rev 19200)
@@ -76,3 +76,6 @@
CPPFLAGS += -I$(NAN_PYTHON)/include/python$(NAN_PYTHON_VERSION)
+ifeq ($(WITH_FFMPEG), true)
+ CPPFLAGS += -DWITH_FFMPEG
+endif
Modified: trunk/blender/source/gameengine/BlenderRoutines/SConscript
===================================================================
--- trunk/blender/source/gameengine/BlenderRoutines/SConscript 2009-03-05 14:30:36 UTC (rev 19199)
+++ trunk/blender/source/gameengine/BlenderRoutines/SConscript 2009-03-05 15:16:43 UTC (rev 19200)
@@ -23,6 +23,9 @@
incs += ' ' + env['BF_SOLID_INC']
defs.append('USE_SUMO_SOLID')
+if env['WITH_BF_FFMPEG']:
+ defs.append('WITH_FFMPEG')
+
incs += ' ' + env['BF_PYTHON_INC']
incs += ' ' + env['BF_BULLET_INC']
incs += ' ' + env['BF_OPENGL_INC']
Modified: trunk/blender/source/gameengine/VideoTexture/CMakeLists.txt
===================================================================
--- trunk/blender/source/gameengine/VideoTexture/CMakeLists.txt 2009-03-05 14:30:36 UTC (rev 19199)
+++ trunk/blender/source/gameengine/VideoTexture/CMakeLists.txt 2009-03-05 15:16:43 UTC (rev 19200)
@@ -52,7 +52,7 @@
)
IF(WITH_FFMPEG)
- SET(INC ${INC} ${FFMPEG_INC})
+ SET(INC ${INC} ${FFMPEG_INC} ${PTHREADS_INC})
ADD_DEFINITIONS(-DWITH_FFMPEG)
ADD_DEFINITIONS(-D__STDC_CONSTANT_MACROS)
ENDIF(WITH_FFMPEG)
Modified: trunk/blender/source/gameengine/VideoTexture/Makefile
===================================================================
--- trunk/blender/source/gameengine/VideoTexture/Makefile 2009-03-05 14:30:36 UTC (rev 19199)
+++ trunk/blender/source/gameengine/VideoTexture/Makefile 2009-03-05 15:16:43 UTC (rev 19200)
@@ -60,6 +60,9 @@
ifeq ($(WITH_FFMPEG),true)
CPPFLAGS += -DWITH_FFMPEG
CPPFLAGS += $(NAN_FFMPEGCFLAGS)
+ ifdef NAN_PTHREADS
+ CPPFLAGS += -I$(NAN_PTHREADS)/include
+ endif
endif
Modified: trunk/blender/source/gameengine/VideoTexture/SConscript
===================================================================
--- trunk/blender/source/gameengine/VideoTexture/SConscript 2009-03-05 14:30:36 UTC (rev 19199)
+++ trunk/blender/source/gameengine/VideoTexture/SConscript 2009-03-05 15:16:43 UTC (rev 19200)
@@ -27,7 +27,7 @@
if env['WITH_BF_FFMPEG']:
defs += ' WITH_FFMPEG'
- incs += ' ' + env['BF_FFMPEG_INC']
+ incs += ' ' + env['BF_FFMPEG_INC'] + ' ' + env['BF_PTHREADS_INC']
defs += ' __STDC_CONSTANT_MACROS'
env.BlenderLib ( 'bf_videotex', sources, Split(incs), Split(defs), libtype=['game','player'], priority=[25, 72], cxx_compileflags = cxxflags )
Modified: trunk/blender/source/gameengine/VideoTexture/VideoFFmpeg.cpp
===================================================================
--- trunk/blender/source/gameengine/VideoTexture/VideoFFmpeg.cpp 2009-03-05 14:30:36 UTC (rev 19199)
+++ trunk/blender/source/gameengine/VideoTexture/VideoFFmpeg.cpp 2009-03-05 15:16:43 UTC (rev 19200)
@@ -55,7 +55,8 @@
m_frame(NULL), m_frameDeinterlaced(NULL), m_frameRGB(NULL), m_imgConvertCtx(NULL),
m_deinterlace(false), m_preseek(0), m_videoStream(-1), m_baseFrameRate(25.0),
m_lastFrame(-1), m_eof(false), m_curPosition(-1), m_startTime(0),
-m_captWidth(0), m_captHeight(0), m_captRate(0.f), m_isImage(false)
+m_captWidth(0), m_captHeight(0), m_captRate(0.f), m_isImage(false),
+m_isThreaded(false), m_stopThread(false), m_cacheStarted(false)
{
// set video format
m_format = RGB24;
@@ -63,6 +64,12 @@
setFlip(true);
// construction is OK
*hRslt = S_OK;
+ m_thread.first = m_thread.last = NULL;
+ pthread_mutex_init(&m_cacheMutex, NULL);
+ m_frameCacheFree.first = m_frameCacheFree.last = NULL;
+ m_frameCacheBase.first = m_frameCacheBase.last = NULL;
+ m_packetCacheFree.first = m_packetCacheFree.last = NULL;
+ m_packetCacheBase.first = m_packetCacheBase.last = NULL;
}
// destructor
@@ -75,6 +82,7 @@
bool VideoFFmpeg::release()
{
// release
+ stopCache();
if (m_codecCtx)
{
avcodec_close(m_codecCtx);
@@ -112,6 +120,29 @@
return true;
}
+AVFrame *VideoFFmpeg::allocFrameRGB()
+{
+ AVFrame *frame;
+ frame = avcodec_alloc_frame();
+ if (m_format == RGBA32)
+ {
+ avpicture_fill((AVPicture*)frame,
+ (uint8_t*)MEM_callocN(avpicture_get_size(
+ PIX_FMT_RGBA,
+ m_codecCtx->width, m_codecCtx->height),
+ "ffmpeg rgba"),
+ PIX_FMT_RGBA, m_codecCtx->width, m_codecCtx->height);
+ } else
+ {
+ avpicture_fill((AVPicture*)frame,
+ (uint8_t*)MEM_callocN(avpicture_get_size(
+ PIX_FMT_RGB24,
+ m_codecCtx->width, m_codecCtx->height),
+ "ffmpeg rgb"),
+ PIX_FMT_RGB24, m_codecCtx->width, m_codecCtx->height);
+ }
+ return frame;
+}
// set initial parameters
void VideoFFmpeg::initParams (short width, short height, float rate, bool image)
@@ -122,6 +153,7 @@
m_isImage = image;
}
+
int VideoFFmpeg::openStream(const char *filename, AVInputFormat *inputFormat, AVFormatParameters *formatParams)
{
AVFormatContext *formatCtx;
@@ -189,7 +221,6 @@
m_videoStream = videoStream;
m_frame = avcodec_alloc_frame();
m_frameDeinterlaced = avcodec_alloc_frame();
- m_frameRGB = avcodec_alloc_frame();
// allocate buffer if deinterlacing is required
avpicture_fill((AVPicture*)m_frameDeinterlaced,
@@ -207,12 +238,6 @@
{
// allocate buffer to store final decoded frame
m_format = RGBA32;
- avpicture_fill((AVPicture*)m_frameRGB,
- (uint8_t*)MEM_callocN(avpicture_get_size(
- PIX_FMT_RGBA,
- m_codecCtx->width, m_codecCtx->height),
- "ffmpeg rgba"),
- PIX_FMT_RGBA, m_codecCtx->width, m_codecCtx->height);
// allocate sws context
m_imgConvertCtx = sws_getContext(
m_codecCtx->width,
@@ -227,12 +252,6 @@
{
// allocate buffer to store final decoded frame
m_format = RGB24;
- avpicture_fill((AVPicture*)m_frameRGB,
- (uint8_t*)MEM_callocN(avpicture_get_size(
- PIX_FMT_RGB24,
- m_codecCtx->width, m_codecCtx->height),
- "ffmpeg rgb"),
- PIX_FMT_RGB24, m_codecCtx->width, m_codecCtx->height);
// allocate sws context
m_imgConvertCtx = sws_getContext(
m_codecCtx->width,
@@ -244,19 +263,247 @@
SWS_FAST_BILINEAR,
NULL, NULL, NULL);
}
+ m_frameRGB = allocFrameRGB();
+
if (!m_imgConvertCtx) {
avcodec_close(m_codecCtx);
+ m_codecCtx = NULL;
av_close_input_file(m_formatCtx);
+ m_formatCtx = NULL;
av_free(m_frame);
+ m_frame = NULL;
MEM_freeN(m_frameDeinterlaced->data[0]);
av_free(m_frameDeinterlaced);
+ m_frameDeinterlaced = NULL;
MEM_freeN(m_frameRGB->data[0]);
av_free(m_frameRGB);
+ m_frameRGB = NULL;
return -1;
}
return 0;
}
+/*
+ * This thread is used to load video frame asynchronously.
+ * It provides a frame caching service.
+ * The main thread is responsible for positionning the frame pointer in the
+ * file correctly before calling startCache() which starts this thread.
+ * The cache is organized in two layers: 1) a cache of 20-30 undecoded packets to keep
+ * memory and CPU low 2) a cache of 5 decoded frames.
+ * If the main thread does not find the frame in the cache (because the video has restarted
+ * or because the GE is lagging), it stops the cache with StopCache() (this is a synchronous
+ * function: it sends a signal to stop the cache thread and wait for confirmation), then
+ * change the position in the stream and restarts the cache thread.
+ */
+void *VideoFFmpeg::cacheThread(void *data)
+{
+ VideoFFmpeg* video = (VideoFFmpeg*)data;
+ // holds the frame that is being decoded
+ CacheFrame *currentFrame = NULL;
+ CachePacket *cachePacket;
+ bool endOfFile = false;
+ int frameFinished = 0;
+
+ while (!video->m_stopThread)
+ {
+ // packet cache is used solely by this thread, no need to lock
+ // In case the stream/file contains other stream than the one we are looking for,
+ // allow a bit of cycling to get rid quickly of those frames
+ frameFinished = 0;
+ while ( !endOfFile
+ && (cachePacket = (CachePacket *)video->m_packetCacheFree.first) != NULL
+ && frameFinished < 25)
+ {
+ // free packet => packet cache is not full yet, just read more
+ if (av_read_frame(video->m_formatCtx, &cachePacket->packet)>=0)
+ {
+ if (cachePacket->packet.stream_index == video->m_videoStream)
+ {
+ // make sure fresh memory is allocated for the packet and move it to queue
+ av_dup_packet(&cachePacket->packet);
+ BLI_remlink(&video->m_packetCacheFree, cachePacket);
+ BLI_addtail(&video->m_packetCacheBase, cachePacket);
+ break;
+ } else {
+ // this is not a good packet for us, just leave it on free queue
+ // Note: here we could handle sound packet
+ av_free_packet(&cachePacket->packet);
+ frameFinished++;
+ }
+
+ } else {
+ if (video->m_isFile)
+ // this mark the end of the file
+ endOfFile = true;
+ // if we cannot read a packet, no need to continue
+ break;
+ }
+ }
+ // frame cache is also used by main thread, lock
+ if (currentFrame == NULL)
+ {
+ // no current frame being decoded, take free one
+ pthread_mutex_lock(&video->m_cacheMutex);
+ if ((currentFrame = (CacheFrame *)video->m_frameCacheFree.first) != NULL)
+ BLI_remlink(&video->m_frameCacheFree, currentFrame);
+ pthread_mutex_unlock(&video->m_cacheMutex);
+ }
+ if (currentFrame != NULL)
+ {
+ // this frame is out of free and busy queue, we can manipulate it without locking
+ frameFinished = 0;
+ while (!frameFinished && (cachePacket = (CachePacket *)video->m_packetCacheBase.first) != NULL)
+ {
+ BLI_remlink(&video->m_packetCacheBase, cachePacket);
+ // use m_frame because when caching, it is not used in main thread
@@ Diff output truncated at 10240 characters. @@
More information about the Bf-blender-cvs
mailing list