[Bf-committers] Sound stuff
Dalai Felinto
dfelinto at gmail.com
Wed Jun 3 06:21:55 CEST 2009
Just to add some noise to this:
Some people are having fun implementing AutoCLAM (3D system) in BGE:
http://domejunky.blogspot.com/
http://clam-project.org/
>From Pete Carss at Plymouth University:
"I've just got CLAM going, so I'm about to get our 10.2 sound system
synced up with the dome..."
http://lad.linuxaudio.org/events/2009_cdm/Thursday/05_Arumi/index.html
In fact he is using a dome + dome BGE + CLAM. It's triple cool.
Cheers,
Dalai
http://blenderecia.orgfree.com
2009/6/2 joe <joeedh at gmail.com>
> How would this work with 3d sound in the BGE?
>
> Joe
>
> On Tue, Jun 2, 2009 at 5:58 AM, neXyon <nexyon at gmail.com> wrote:
> > Greetings,
> >
> > as Ton already announced in the meeting minutes mail, I'm working on
> sound
> > stuff.
> >
> > --- Short about:
> >
> > A already two months old proposal is here:
> > http://wiki.blender.org/index.php/User:NeXyon/Sound
> >
> > To say it short:
> >
> > current sound stuff in blender is a pain, there are at least 3 more or
> less
> > seperated code parts that handle sound:
> >
> > * the sequencer, handles sound completely on its own (SDL)
> > * editsound.c, handles the sound structures bSound and bSample, has
> nothing
> > to do with sound really, except showing the wave in the audio window
> > * SoundSystem, the GE sound system (OpenAL)
> >
> > Why not use TinySND (the old GSoC sound project): because I personally
> think
> > it's half-baked, has to much dependencies and doesn't really fit the
> needs
> > properly. (Anyone wants to discuss about that?)
> >
> > ---- More sound formats for 2.4x BGE:
> >
> > Campell suggested me to write a loading function for the old sound system
> > (SoundSystem) to make it possible to play other formats in BGE, not only
> > wav. So I did a quick (but - as far as I tested - working flawlessly)
> > implementation of that function and it works perfectly with wav files.
> But
> > then I noticed that for other files, the editsound.c sound code disturbs.
> > It's really desperating working with this - sorry for the term - crap of
> > code, so I only bypass the sound format checking routines in editsound.c
> to
> > get the function work.
> >
> > The result of this bypass is, that you can load any file resulting in
> simply
> > no playback and the wave isn't displayed in the audio window.
> >
> > Nevertheless I have attached the patch which includes the function and
> the
> > bypassing changes, if you test it, remember to switch on WITH_FFMPEG. The
> > internal SoundSystem doesn't seem to get that define during compilation,
> so
> > I #defined it myself there, so if you try to compile without ffmpeg and
> > forget to remove the define, you'll get a compilation error.
> >
> > I'm not really motivated to also refactor editsound.c to fix the problems
> of
> > my hacks, to get any sound format working in future blender 2.4x. I more
> > like to concentrate on a new sound system for 2.5, but if anyone of you
> is
> > interested to continue that 2.4x BGE sound work, feel free, I can give
> > support if you want so, simply contact me!
> >
> > ---- Sound in 2.5:
> >
> > So, I'm currently working on a new sound library, that should be able to
> > handle every sound feature needed by blender. For the integration in
> > blender, I think the old sound structures will have to be removed,
> they're
> > simply unusable at the moment.
> >
> > The library will be in C++ like the current SoundSystem and with a C
> backend
> > to use it. It will use FFMPEG to read sound files (I think we shouldn't
> > include an own wav loading function, so if blender's compiled without
> FFMPEG
> > => no sound) and for the output I'm currently working with SDL, but this
> is
> > pluggable. My target is to let the user decide which output driver he
> wants,
> > just like every better multimedia app, as soon as we have programmed
> other
> > output drivers.
> >
> > Also should the library satisfy Peter Schlaile, because it's possible to
> let
> > the sequencer do as much audio processing as it wants to, the api design
> is
> > very flexible in this respect.
> >
> > ---- Current state and prospects:
> >
> > Currently I program the library separately in a C++ project: sound output
> > works through SDL, and I'm currently having trouble with audio resampling
> of
> > ffmpeg (something the function provided in the patch, doesn't do, so
> don't
> > worry!), as soon I have that working, I'll finish the class design of the
> > library, update it in the proposal and then I'll report again!
> >
> > Meanwhile, I'd like to hear all your thoughts, recommendation and
> > constructive criticism.
> >
> > Regards,
> >
> > Jörg
> >
> >
> > Index: intern/SoundSystem/openal/SND_OpenALDevice.cpp
> > ===================================================================
> > --- intern/SoundSystem/openal/SND_OpenALDevice.cpp (Revision 20555)
> > +++ intern/SoundSystem/openal/SND_OpenALDevice.cpp (Arbeitskopie)
> > @@ -157,6 +157,193 @@
> > p_i= (char *)&(a); \
> > s_i=p_i[0]; p_i[0]=p_i[1]; p_i[1]=s_i; }
> >
> > +#define WITH_FFMPEG
> > +
> > +#ifdef WITH_FFMPEG
> > +
> > +extern "C" {
> > +#include <libavcodec/avcodec.h>
> > +#include <libavformat/avformat.h>
> > +#include <libavutil/avutil.h>
> > +}
> > +
> > +#define SND_SIZE(rate, channels, format)
> > ((((float)rate)*((float)channels)*((float)format)))
> > +
> > +int p_input_callback(void *opaque, uint8_t *buf, int buf_size)
> > +{
> > + if(**(char***)opaque == (char*)buf)
> > + {
> > + **(char***)opaque += buf_size;
> > + return buf_size;
> > + }
> > + else
> > + return 0;
> > +}
> > +
> > +int FFMPEG_FMT_LEN(SampleFormat fmt)
> > +{
> > + if(fmt == SAMPLE_FMT_S16)
> > + return 2;
> > + return 1;
> > +}
> > +
> > +uint8_t* resizeBuffer(uint8_t* original, int orig_size, int size)
> > +{
> > + uint8_t* buffer = (uint8_t*)malloc(size);
> > + memcpy(buffer, original, size < orig_size ? size : orig_size);
> > + free(original);
> > + return buffer;
> > +}
> > +
> > +ALvoid SND_alutLoadWAVMemory(ALbyte *memory,ALsizei memsize,ALenum
> > *format,ALvoid **data,ALsizei *size,ALsizei *freq,ALboolean *loop)
> > +{
> > + *format = AL_FORMAT_MONO16;
> > + *data = NULL;
> > + *size = 0;
> > + *freq = 22050;
> > + *loop = AL_FALSE;
> > +
> > + av_register_all();
> > +
> > + AVProbeData probe_data, *pd = &probe_data;
> > +
> > + pd->filename = "";
> > + pd->buf = (unsigned char*)memory;
> > + pd->buf_size = memsize;
> > +
> > + AVInputFormat *fmt = av_probe_input_format(pd, 1);
> > + if(!fmt)
> > + return;
> > +
> > + ByteIOContext m_io;
> > + char** opaque = &memory;
> > +
> > + if(init_put_byte(&m_io, (unsigned char*)memory, memsize, 0,
> &opaque,
> > p_input_callback, NULL, NULL) < 0)
> > + return;
> > + m_io.is_streamed = 1;
> > +
> > + AVFormatContext *pFormatCtx;
> > + if(av_open_input_stream(&pFormatCtx, &m_io, "", fmt, NULL))
> > + return;
> > +
> > + if(av_find_stream_info(pFormatCtx)<0)
> > + return;
> > +
> > + dump_format(pFormatCtx, 0, "", 0);
> > +
> > + int streamID = -1;
> > +
> > + for(int i = 0; i < pFormatCtx->nb_streams; i++)
> > + if(pFormatCtx->streams[i]->codec->codec_type ==
> > CODEC_TYPE_AUDIO && streamID < 0)
> > + streamID=i;
> > + if(streamID == -1)
> > + return;
> > +
> > + AVCodecContext* aCodecCtx=pFormatCtx->streams[streamID]->codec;
> > +
> > +// printf("Duration: %d\n",
> > (int)(pFormatCtx->duration*44100.0f/AV_TIME_BASE));
> > + int aBuffer_size;
> > + if(pFormatCtx->duration > 0)
> > + aBuffer_size =
> > (int)ceil(((float)pFormatCtx->duration)*SND_SIZE(aCodecCtx->sample_rate,
> > aCodecCtx->channels,
> >
> FFMPEG_FMT_LEN(aCodecCtx->sample_fmt))/((float)AV_TIME_BASE)+((float)AVCODEC_MAX_AUDIO_FRAME_SIZE*6));
> > + else
> > + aBuffer_size = (int)ceil(SND_SIZE(aCodecCtx->sample_rate,
> > aCodecCtx->channels,
> >
> FFMPEG_FMT_LEN(aCodecCtx->sample_fmt))+((float)AVCODEC_MAX_AUDIO_FRAME_SIZE*6));
> > + int aBuffer_index = 0;
> > + uint8_t* aBuffer = (uint8_t*)malloc(aBuffer_size);
> > +
> > + printf("Samplerate: %d\n", aCodecCtx->sample_rate);
> > + printf("Duration: %d\n", pFormatCtx->duration);
> > + printf("Channels: %d\n", aCodecCtx->channels);
> > + printf("Allocated %d Bytes for Buffer\n", aBuffer_size);//*/
> > +
> > + if((aCodecCtx->channels > 2) || ((aCodecCtx->sample_fmt !=
> > SAMPLE_FMT_S16) && (aCodecCtx->sample_fmt != SAMPLE_FMT_S16)))
> > + return;
> > +
> > + AVCodec *aCodec = avcodec_find_decoder(aCodecCtx->codec_id);
> > + if(!aCodec)
> > + return;
> > +
> > + avcodec_open(aCodecCtx, aCodec);
> > +
> > + AVPacket packet;
> > + int audio_pkt_size;
> > + uint8_t *audio_pkt_data;
> > + int data_size, len;
> > + printf("Entering loop!\n");
> > +
> > + int READ = 0;
> > +
> > + while(av_read_frame(pFormatCtx, &packet)>=0)
> > + {
> > + if(packet.stream_index==streamID)
> > + {
> > + audio_pkt_data = packet.data;
> > + audio_pkt_size = packet.size;
> > + printf("Packet Size: %d, %d!\n", audio_pkt_size,
> > READ);
> > + READ += audio_pkt_size;
> > + while(audio_pkt_size > 0)
> > + {
> > + data_size = aBuffer_size - aBuffer_index;
> > + if(data_size <
> AVCODEC_MAX_AUDIO_FRAME_SIZE)
> > + {
> > + printf("Resizing!\n");
> > + aBuffer = resizeBuffer(aBuffer,
> > aBuffer_size, aBuffer_size+AVCODEC_MAX_AUDIO_FRAME_SIZE * 32);
> > + aBuffer_size +=
> > AVCODEC_MAX_AUDIO_FRAME_SIZE * 32;
> > + data_size = aBuffer_size -
> > aBuffer_index;
> > + }
> > + printf("Step 1!\n");
> > + len = avcodec_decode_audio2(aCodecCtx,
> > (int16_t *)(aBuffer + aBuffer_index), &data_size, audio_pkt_data,
> > audio_pkt_size);
> > + printf("Step 2!\n");
> > + if(len < 0)
> > + {
> > + audio_pkt_size = 0;
> > + break;
> > + }
> > + audio_pkt_data += len;
> > + audio_pkt_size -= len;
> > + if(data_size <= 0)
> > + continue;
> > + aBuffer_index += data_size;
> > + }
> > + }
> > + av_free_packet(&packet);
> > + }
> > + printf("Finished loop!\n");
> > +
> > + aBuffer = resizeBuffer(aBuffer, aBuffer_size, aBuffer_index);
> > +
> > + *data = aBuffer;
> > + *size = aBuffer_index;
> > + *freq = aCodecCtx->sample_rate;
> > + if(aCodecCtx->channels == 1)
> > + {
> > + switch(aCodecCtx->sample_fmt)
> > + {
> > + case SAMPLE_FMT_U8:
> > + *format = AL_FORMAT_MONO8;
> > + break;
> > + case SAMPLE_FMT_S16:
> > + *format = AL_FORMAT_MONO16;
> > + break;
> > + }
> > + }
> > + else
> > + {
> > + switch(aCodecCtx->sample_fmt)
> > + {
> > + case SAMPLE_FMT_U8:
> > + *format = AL_FORMAT_STEREO8;
> > + break;
> > + case SAMPLE_FMT_S16:
> > + *format = AL_FORMAT_STEREO16;
> > + break;
> > + }
> > + }
> > +
> > + av_close_input_stream(pFormatCtx);
> > +}
> > +
> > +#else
> > +
> > static int stream_read(void *out, ALbyte **stream, ALsizei size, ALsizei
> > *memsize)
> > {
> > if(size <= *memsize) {
> > @@ -293,6 +480,8 @@
> > }
> > }
> >
> > +#endif
> > +
> > ALvoid SND_alutUnloadWAV(ALenum format,ALvoid *data,ALsizei size,ALsizei
> > freq)
> > {
> > if (data)
> > @@ -480,8 +669,10 @@
> > waveslot = m_wavecache->GetWaveSlot(samplename);
> >
> > /* do we support this sample? */
> > +#ifndef WITH_FFMPEG
> > if (SND_IsSampleValid(name, memlocation))
> > {
> > +#endif
> > if (waveslot)
> > {
> > bool freemem = false;
> > @@ -546,6 +737,7 @@
> > /* and free the original stuff (copy was
> made
> > in openal) */
> > SND_alutUnloadWAV(sampleformat, data,
> > numberofsamples, samplerate);
> > }
> > +#ifndef WITH_FFMPEG
> > }
> > else
> > {
> > @@ -553,6 +745,7 @@
> >
> m_wavecache->RemoveSample(waveslot->GetSampleName(),
> > waveslot->GetBuffer());
> > waveslot = NULL;
> > }
> > +#endif
> > }
> > return waveslot;
> > }
> > Index: source/blender/src/editsound.c
> > ===================================================================
> > --- source/blender/src/editsound.c (Revision 20555)
> > +++ source/blender/src/editsound.c (Arbeitskopie)
> > @@ -366,6 +366,16 @@
> > bSample *sample = NULL;
> > int channels, rate, bits, len;
> >
> > +#ifdef WITH_FFMPEG
> > + sample = sound->sample;
> > + sample->channels = 2;
> > + sample->rate = 44100;
> > + sample->bits = 16;
> > + sample->len = 0;
> > + sample->data = malloc(0);
> > + sample->type = SAMPLE_WAV;
> > + return;
> > +#endif
> > /* prepare for the worst... */
> > sound->sample->type = SAMPLE_INVALID;
> >
> > @@ -499,6 +509,9 @@
> > /* ugly, but it works (for now) */
> > static int sound_get_filetype_from_header(bSound *sound, PackedFile *pf)
> > {
> > +#ifdef WITH_FFMPEG
> > + return SAMPLE_WAV;
> > +#endif
> > int filetype = SAMPLE_INVALID;
> > int i;
> > char buffer[25];
> >
> >
> > _______________________________________________
> > Bf-committers mailing list
> > Bf-committers at blender.org
> > http://lists.blender.org/mailman/listinfo/bf-committers
> >
> >
> _______________________________________________
> Bf-committers mailing list
> Bf-committers at blender.org
> http://lists.blender.org/mailman/listinfo/bf-committers
>
More information about the Bf-committers
mailing list