[Bf-committers] Sound stuff
joe
joeedh at gmail.com
Wed Jun 3 03:29:50 CEST 2009
How would this work with 3d sound in the BGE?
Joe
On Tue, Jun 2, 2009 at 5:58 AM, neXyon <nexyon at gmail.com> wrote:
> Greetings,
>
> as Ton already announced in the meeting minutes mail, I'm working on sound
> stuff.
>
> --- Short about:
>
> A already two months old proposal is here:
> http://wiki.blender.org/index.php/User:NeXyon/Sound
>
> To say it short:
>
> current sound stuff in blender is a pain, there are at least 3 more or less
> seperated code parts that handle sound:
>
> * the sequencer, handles sound completely on its own (SDL)
> * editsound.c, handles the sound structures bSound and bSample, has nothing
> to do with sound really, except showing the wave in the audio window
> * SoundSystem, the GE sound system (OpenAL)
>
> Why not use TinySND (the old GSoC sound project): because I personally think
> it's half-baked, has to much dependencies and doesn't really fit the needs
> properly. (Anyone wants to discuss about that?)
>
> ---- More sound formats for 2.4x BGE:
>
> Campell suggested me to write a loading function for the old sound system
> (SoundSystem) to make it possible to play other formats in BGE, not only
> wav. So I did a quick (but - as far as I tested - working flawlessly)
> implementation of that function and it works perfectly with wav files. But
> then I noticed that for other files, the editsound.c sound code disturbs.
> It's really desperating working with this - sorry for the term - crap of
> code, so I only bypass the sound format checking routines in editsound.c to
> get the function work.
>
> The result of this bypass is, that you can load any file resulting in simply
> no playback and the wave isn't displayed in the audio window.
>
> Nevertheless I have attached the patch which includes the function and the
> bypassing changes, if you test it, remember to switch on WITH_FFMPEG. The
> internal SoundSystem doesn't seem to get that define during compilation, so
> I #defined it myself there, so if you try to compile without ffmpeg and
> forget to remove the define, you'll get a compilation error.
>
> I'm not really motivated to also refactor editsound.c to fix the problems of
> my hacks, to get any sound format working in future blender 2.4x. I more
> like to concentrate on a new sound system for 2.5, but if anyone of you is
> interested to continue that 2.4x BGE sound work, feel free, I can give
> support if you want so, simply contact me!
>
> ---- Sound in 2.5:
>
> So, I'm currently working on a new sound library, that should be able to
> handle every sound feature needed by blender. For the integration in
> blender, I think the old sound structures will have to be removed, they're
> simply unusable at the moment.
>
> The library will be in C++ like the current SoundSystem and with a C backend
> to use it. It will use FFMPEG to read sound files (I think we shouldn't
> include an own wav loading function, so if blender's compiled without FFMPEG
> => no sound) and for the output I'm currently working with SDL, but this is
> pluggable. My target is to let the user decide which output driver he wants,
> just like every better multimedia app, as soon as we have programmed other
> output drivers.
>
> Also should the library satisfy Peter Schlaile, because it's possible to let
> the sequencer do as much audio processing as it wants to, the api design is
> very flexible in this respect.
>
> ---- Current state and prospects:
>
> Currently I program the library separately in a C++ project: sound output
> works through SDL, and I'm currently having trouble with audio resampling of
> ffmpeg (something the function provided in the patch, doesn't do, so don't
> worry!), as soon I have that working, I'll finish the class design of the
> library, update it in the proposal and then I'll report again!
>
> Meanwhile, I'd like to hear all your thoughts, recommendation and
> constructive criticism.
>
> Regards,
>
> Jörg
>
>
> Index: intern/SoundSystem/openal/SND_OpenALDevice.cpp
> ===================================================================
> --- intern/SoundSystem/openal/SND_OpenALDevice.cpp (Revision 20555)
> +++ intern/SoundSystem/openal/SND_OpenALDevice.cpp (Arbeitskopie)
> @@ -157,6 +157,193 @@
> p_i= (char *)&(a); \
> s_i=p_i[0]; p_i[0]=p_i[1]; p_i[1]=s_i; }
>
> +#define WITH_FFMPEG
> +
> +#ifdef WITH_FFMPEG
> +
> +extern "C" {
> +#include <libavcodec/avcodec.h>
> +#include <libavformat/avformat.h>
> +#include <libavutil/avutil.h>
> +}
> +
> +#define SND_SIZE(rate, channels, format)
> ((((float)rate)*((float)channels)*((float)format)))
> +
> +int p_input_callback(void *opaque, uint8_t *buf, int buf_size)
> +{
> + if(**(char***)opaque == (char*)buf)
> + {
> + **(char***)opaque += buf_size;
> + return buf_size;
> + }
> + else
> + return 0;
> +}
> +
> +int FFMPEG_FMT_LEN(SampleFormat fmt)
> +{
> + if(fmt == SAMPLE_FMT_S16)
> + return 2;
> + return 1;
> +}
> +
> +uint8_t* resizeBuffer(uint8_t* original, int orig_size, int size)
> +{
> + uint8_t* buffer = (uint8_t*)malloc(size);
> + memcpy(buffer, original, size < orig_size ? size : orig_size);
> + free(original);
> + return buffer;
> +}
> +
> +ALvoid SND_alutLoadWAVMemory(ALbyte *memory,ALsizei memsize,ALenum
> *format,ALvoid **data,ALsizei *size,ALsizei *freq,ALboolean *loop)
> +{
> + *format = AL_FORMAT_MONO16;
> + *data = NULL;
> + *size = 0;
> + *freq = 22050;
> + *loop = AL_FALSE;
> +
> + av_register_all();
> +
> + AVProbeData probe_data, *pd = &probe_data;
> +
> + pd->filename = "";
> + pd->buf = (unsigned char*)memory;
> + pd->buf_size = memsize;
> +
> + AVInputFormat *fmt = av_probe_input_format(pd, 1);
> + if(!fmt)
> + return;
> +
> + ByteIOContext m_io;
> + char** opaque = &memory;
> +
> + if(init_put_byte(&m_io, (unsigned char*)memory, memsize, 0, &opaque,
> p_input_callback, NULL, NULL) < 0)
> + return;
> + m_io.is_streamed = 1;
> +
> + AVFormatContext *pFormatCtx;
> + if(av_open_input_stream(&pFormatCtx, &m_io, "", fmt, NULL))
> + return;
> +
> + if(av_find_stream_info(pFormatCtx)<0)
> + return;
> +
> + dump_format(pFormatCtx, 0, "", 0);
> +
> + int streamID = -1;
> +
> + for(int i = 0; i < pFormatCtx->nb_streams; i++)
> + if(pFormatCtx->streams[i]->codec->codec_type ==
> CODEC_TYPE_AUDIO && streamID < 0)
> + streamID=i;
> + if(streamID == -1)
> + return;
> +
> + AVCodecContext* aCodecCtx=pFormatCtx->streams[streamID]->codec;
> +
> +// printf("Duration: %d\n",
> (int)(pFormatCtx->duration*44100.0f/AV_TIME_BASE));
> + int aBuffer_size;
> + if(pFormatCtx->duration > 0)
> + aBuffer_size =
> (int)ceil(((float)pFormatCtx->duration)*SND_SIZE(aCodecCtx->sample_rate,
> aCodecCtx->channels,
> FFMPEG_FMT_LEN(aCodecCtx->sample_fmt))/((float)AV_TIME_BASE)+((float)AVCODEC_MAX_AUDIO_FRAME_SIZE*6));
> + else
> + aBuffer_size = (int)ceil(SND_SIZE(aCodecCtx->sample_rate,
> aCodecCtx->channels,
> FFMPEG_FMT_LEN(aCodecCtx->sample_fmt))+((float)AVCODEC_MAX_AUDIO_FRAME_SIZE*6));
> + int aBuffer_index = 0;
> + uint8_t* aBuffer = (uint8_t*)malloc(aBuffer_size);
> +
> + printf("Samplerate: %d\n", aCodecCtx->sample_rate);
> + printf("Duration: %d\n", pFormatCtx->duration);
> + printf("Channels: %d\n", aCodecCtx->channels);
> + printf("Allocated %d Bytes for Buffer\n", aBuffer_size);//*/
> +
> + if((aCodecCtx->channels > 2) || ((aCodecCtx->sample_fmt !=
> SAMPLE_FMT_S16) && (aCodecCtx->sample_fmt != SAMPLE_FMT_S16)))
> + return;
> +
> + AVCodec *aCodec = avcodec_find_decoder(aCodecCtx->codec_id);
> + if(!aCodec)
> + return;
> +
> + avcodec_open(aCodecCtx, aCodec);
> +
> + AVPacket packet;
> + int audio_pkt_size;
> + uint8_t *audio_pkt_data;
> + int data_size, len;
> + printf("Entering loop!\n");
> +
> + int READ = 0;
> +
> + while(av_read_frame(pFormatCtx, &packet)>=0)
> + {
> + if(packet.stream_index==streamID)
> + {
> + audio_pkt_data = packet.data;
> + audio_pkt_size = packet.size;
> + printf("Packet Size: %d, %d!\n", audio_pkt_size,
> READ);
> + READ += audio_pkt_size;
> + while(audio_pkt_size > 0)
> + {
> + data_size = aBuffer_size - aBuffer_index;
> + if(data_size < AVCODEC_MAX_AUDIO_FRAME_SIZE)
> + {
> + printf("Resizing!\n");
> + aBuffer = resizeBuffer(aBuffer,
> aBuffer_size, aBuffer_size+AVCODEC_MAX_AUDIO_FRAME_SIZE * 32);
> + aBuffer_size +=
> AVCODEC_MAX_AUDIO_FRAME_SIZE * 32;
> + data_size = aBuffer_size -
> aBuffer_index;
> + }
> + printf("Step 1!\n");
> + len = avcodec_decode_audio2(aCodecCtx,
> (int16_t *)(aBuffer + aBuffer_index), &data_size, audio_pkt_data,
> audio_pkt_size);
> + printf("Step 2!\n");
> + if(len < 0)
> + {
> + audio_pkt_size = 0;
> + break;
> + }
> + audio_pkt_data += len;
> + audio_pkt_size -= len;
> + if(data_size <= 0)
> + continue;
> + aBuffer_index += data_size;
> + }
> + }
> + av_free_packet(&packet);
> + }
> + printf("Finished loop!\n");
> +
> + aBuffer = resizeBuffer(aBuffer, aBuffer_size, aBuffer_index);
> +
> + *data = aBuffer;
> + *size = aBuffer_index;
> + *freq = aCodecCtx->sample_rate;
> + if(aCodecCtx->channels == 1)
> + {
> + switch(aCodecCtx->sample_fmt)
> + {
> + case SAMPLE_FMT_U8:
> + *format = AL_FORMAT_MONO8;
> + break;
> + case SAMPLE_FMT_S16:
> + *format = AL_FORMAT_MONO16;
> + break;
> + }
> + }
> + else
> + {
> + switch(aCodecCtx->sample_fmt)
> + {
> + case SAMPLE_FMT_U8:
> + *format = AL_FORMAT_STEREO8;
> + break;
> + case SAMPLE_FMT_S16:
> + *format = AL_FORMAT_STEREO16;
> + break;
> + }
> + }
> +
> + av_close_input_stream(pFormatCtx);
> +}
> +
> +#else
> +
> static int stream_read(void *out, ALbyte **stream, ALsizei size, ALsizei
> *memsize)
> {
> if(size <= *memsize) {
> @@ -293,6 +480,8 @@
> }
> }
>
> +#endif
> +
> ALvoid SND_alutUnloadWAV(ALenum format,ALvoid *data,ALsizei size,ALsizei
> freq)
> {
> if (data)
> @@ -480,8 +669,10 @@
> waveslot = m_wavecache->GetWaveSlot(samplename);
>
> /* do we support this sample? */
> +#ifndef WITH_FFMPEG
> if (SND_IsSampleValid(name, memlocation))
> {
> +#endif
> if (waveslot)
> {
> bool freemem = false;
> @@ -546,6 +737,7 @@
> /* and free the original stuff (copy was made
> in openal) */
> SND_alutUnloadWAV(sampleformat, data,
> numberofsamples, samplerate);
> }
> +#ifndef WITH_FFMPEG
> }
> else
> {
> @@ -553,6 +745,7 @@
> m_wavecache->RemoveSample(waveslot->GetSampleName(),
> waveslot->GetBuffer());
> waveslot = NULL;
> }
> +#endif
> }
> return waveslot;
> }
> Index: source/blender/src/editsound.c
> ===================================================================
> --- source/blender/src/editsound.c (Revision 20555)
> +++ source/blender/src/editsound.c (Arbeitskopie)
> @@ -366,6 +366,16 @@
> bSample *sample = NULL;
> int channels, rate, bits, len;
>
> +#ifdef WITH_FFMPEG
> + sample = sound->sample;
> + sample->channels = 2;
> + sample->rate = 44100;
> + sample->bits = 16;
> + sample->len = 0;
> + sample->data = malloc(0);
> + sample->type = SAMPLE_WAV;
> + return;
> +#endif
> /* prepare for the worst... */
> sound->sample->type = SAMPLE_INVALID;
>
> @@ -499,6 +509,9 @@
> /* ugly, but it works (for now) */
> static int sound_get_filetype_from_header(bSound *sound, PackedFile *pf)
> {
> +#ifdef WITH_FFMPEG
> + return SAMPLE_WAV;
> +#endif
> int filetype = SAMPLE_INVALID;
> int i;
> char buffer[25];
>
>
> _______________________________________________
> Bf-committers mailing list
> Bf-committers at blender.org
> http://lists.blender.org/mailman/listinfo/bf-committers
>
>
More information about the Bf-committers
mailing list