Learning Core Audio

Chris Adamson, Kevin Avila

Mentioned 26

Describes the Core Audio framework, covering such topics as recording, playback, format conversion, MIDI conectivity, and audio units.

More on Amazon.com

Mentioned in questions and answers.

I am trying to learn iOS Core Audio for Swift. I started out with Swift right away and I got no understanding of Objective-C. I am an Engineer with training in Sound engineering so I don't need to learn the basics of "sound" (samplerate, bitdepth, etc.) just the way it is approached in Core Audio, iOS, Swift.

Can you recommend any tutorials, guides, documentations or books (free or paid doesn't matter) that explain how to use Core Audio in Swift?

Thanks for your support, Tobias

Chris Adamson's book is in Objective-C, but covers Core Audio quite well. Ask The Google for his name and you'll find some of his articles. Many things transfer to Swift fairly easily. My blog has several examples.

Core MIDI is another thing though. Swift support of Core MIDI is still problematic.

I'm trying to learn about manipulating audio on iOS and I have been reading a lot on Apple's developer pages. However, I have reached a point at which I have an AudioBuffer and I'm not sure what to do with it. I know it contains an mData buffer but I have no idea what that contains. I have looked around for what a "buffer of audio data," as the reference describes it, but I still do not seem to understand what that is.

Also, the mData seems to be of type void which I gather may be cast as the type for the specific audio contained. I guess I'm not certain how to know what to cast this as either.

If you're serious about learning Core Audio, do yourself a favour and get this book. It got me started, and Core Audio is not easy by all means! http://www.amazon.com/Learning-Core-Audio-Hands-Programming/dp/0321636848

Pier.

I have a user recording and another mp3 file in my app, and I want the user to be able to export both of these as one, meaning the two files will be merged or laid over each other in some way.

In case that wasn't understood, both mp3 files are to be played at the same time just as any app where a user can record say, a song, over an instrumental.

The recording and the instrumental are two separate mp3 files that need to be exported as one.

How do I go about doing this? From what I've read, I can't find the solution. I see a lot on concatenating the two audio files, but I don't want them to play one after another, but rather at the same time.

Thanks.

EDIT: I know this is late, but in case anyone stumbles by this and was looking for sample code, it's in my answer here: How can I overlap audio files and combine for iPhone in Xcode?

If I get you right, you are asking for a audio mixer feature. This is not a trivial task. Take a look at Core Audio. A good book to start with is this one. One solution would be, to create a GUI-less Audio Unit (mixer unit) that plays, mixes and renders both signal (mp3s).

Besides the programming aspect, there is also an audio engineering aspect here: you should take care of the level of the signals. Imagine you have 2 identical mp3s at a level of 0dB. If you sum them up, your level will be +3dB. This don't exist in the digital world (0dB ist the maximum). Because of that, you would have to reduce the input levels before mixing.

EDIT: sorry for the late input, but maybe this helps someone in the future: Apple has an example for the Audio Mixer that I just stumpled upon.

iOS has various audio frameworks from the higher-level that lets you simply play a specified file, to the lower level that lets you get at the raw PCM data, and everything in between. For our app, we just need to play external files (WAV, AIFF, MP3) but we need to do so in response to pressing a button and we need that latency to be as small as possible. (It's for queueing in live productions.)

Now the AVAudioPlayer and such work to play simple file assets (via their URL), but its latency in actually starting the sound is too great. With larger files of more than five minutes in length, the delay to start the sound can be over a second long which renders it all but useless for timing in a live performance.

Now I know things like openAL can be used for a very-low-latency playback, but then you're waist-deep into audio buffers, audio sources, listeners, etc.

That said, does anyone know of any frameworks that work at a higher-level (i.e. play 'MyBeddingTrack.mp3') with a very low latency? Pre-buffering is fine. It's just the trigger has to be fast.

Bonus if we can do things like set the start and end points of the playback within the file, or to change the volume or even to perform ducking, etc.

Although Audio Queue framework is relatively easy to use.. it packs a lot of DSP heavy lifting behind the scenes (ie if you supply it with VBR/compressed audio.. it automatically converts it to PCM before playing it on the speaker.. it also handles a lot of the threading issues for the end user opaquely).. which is good news someone doing a light weight non-real time application.

You mentioned that you need it for queuing in live productions. I'm not sure if that means that your app is real-time.. because if it is.. then Audio Queue's will struggle to meed your needs. A good article to read about this is Ross Bencina's. The take away is that you can't afford to let third party frameworks or libraries do anything that can be potentially expensive behind the scenes like thread locking or mallocing or deallocing etc etc.. that's simply too expensive and risky for developing real time audio apps.

That's where the Audio Unit framework come in. Audio Queue's are actually built on top of the Audio Unit framework (it automates a lot of it's work).. but Audio Units bring you as close to the metal as it gets with iOS. It as responsive as you want it to be, and can do a real time app easy. Audio Unit has a huge learning curve though. There are some Open Source wrappers around it that simplifies it though (see novocaine).

If I were you.. I'd at least skim through Learning Core Audio.. it's the go to book for any iOS core-audio developer.. it talks in detail about Audio Queues, Audio Units etc and has excellent code examples..

From my own experience.. I worked on a real-time audio app that had some intensive audio requirements.. i found the Audio Queue framework and thought it was too good to be true.. my app worked when i prototyped it with light restrictions.. but it simply choked upon stress testing it.. that's when i had to dive deep into audio units and change the architecture etc etc (it wasn't pretty). my advice: work with audio queue at least as an introduction to Audio Units.. stick with it if it meets your needs, but then don't be afraid to use Audio Units if it becomes clear that Audio Queue no longer meets your app's demands.

I've being doing some reading up on core audio for ios 4 with the aim of building a little test app.

I'm pretty confused at this point in reseach with all the api's. Ideally what I want to know how to do is to extract a number of samples from two mp3s into arrays.

Then in a callback loop I want to mix these samples together and send them to the speaker.

There are examples on the apple dev site but I'm finding them difficult to disect and digest. Is anybody aware of a nice stripped down example somewhere?

Also I can't determine which api's to be using.

There extendedaudiofile and audio file. These seem to be the one's for extracting audio. Which one should I use?

It it absoultly neccessary to use the mix unit or would I be as well off to do my own mixing code (I want as much sample control as possible).

Do I need to use audio queue services? I've heard that they provide poor latency, is this true?

Finally do I have to use an audio session service. Would an audio app work without it? How would the audio session fit in to the whole audio extraction and callback? Is it purely just to handle interruptions?

There extendedaudiofile and audio file. These seem to be the one's for extracting audio. Which one should I use?

Neither of those would work if you are accessing audio files stored in the iPod library. You will have to use AVAssetReader. (Note: in the AVAssetReader documentation.. it states that AVAssetReader is not intended for use with real-time sources, and its performance is not guaranteed for real-time operations. All I can say that it worked fine for me.. and I've created several real time applications using just AVAssetReader.. here is a sample.

Please see my answer here for more general tips on iOS audio programming as well.

Finally, the book learning core audio is obviously released by now. I strongly recommend you patiently go through the chapters and play with the sample code.. It's best you take your time with the examples and have the concepts sink in before you jump your more complex scenario.. copying and pasting sample code from the web and/or following high level advice of people on the web may work at the beginning, but you'll get into really hairy problems later on that no one else will help you fix.. trust me I learned the hard way!

Currently, I'm doing a little test project to see if I can get samples from an AVAssetReader to play back using an AudioQueue on iOS.

I've read this: ( Play raw uncompressed sound with AudioQueue, no sound ) and this: ( How to correctly read decoded PCM samples on iOS using AVAssetReader -- currently incorrect decoding ),

Which both actually did help. Before reading, I was getting no sound at all. Now, I'm getting sound, but the audio is playing SUPER fast. This is my first foray into audio programming, so any help is greatly appreciated.

I initialize the reader thusly:

NSDictionary * outputSettings = [NSDictionary dictionaryWithObjectsAndKeys:
                                         [NSNumber numberWithInt:kAudioFormatLinearPCM], AVFormatIDKey,
                                         [NSNumber numberWithFloat:44100.0], AVSampleRateKey,
                                         [NSNumber numberWithInt:2], AVNumberOfChannelsKey,
                                         [NSNumber numberWithInt:16], AVLinearPCMBitDepthKey,
                                         [NSNumber numberWithBool:NO], AVLinearPCMIsNonInterleaved,
                                         [NSNumber numberWithBool:NO], AVLinearPCMIsFloatKey,
                                         [NSNumber numberWithBool:NO], AVLinearPCMIsBigEndianKey,

                                         nil];

        output = [[AVAssetReaderAudioMixOutput alloc] initWithAudioTracks:uasset.tracks audioSettings:outputSettings];
        [reader addOutput:output];
...

And I grab the data thusly:

CMSampleBufferRef ref= [output copyNextSampleBuffer];
    // NSLog(@"%@",ref);
    if(ref==NULL)
        return;
    //copy data to file
    //read next one
    AudioBufferList audioBufferList;
    NSMutableData *data = [NSMutableData data];
    CMBlockBufferRef blockBuffer;
    CMSampleBufferGetAudioBufferListWithRetainedBlockBuffer(ref, NULL, &audioBufferList, sizeof(audioBufferList), NULL, NULL, 0, &blockBuffer);
    // NSLog(@"%@",blockBuffer);

    if(blockBuffer==NULL)
    {
        [data release];
        return;
    }
    if(&audioBufferList==NULL)
    {
        [data release];
        return;
    }

    //stash data in same object
    for( int y=0; y<audioBufferList.mNumberBuffers; y++ )
    {
//        NSData* throwData;
        AudioBuffer audioBuffer = audioBufferList.mBuffers[y];
        [self.delegate streamer:self didGetAudioBuffer:audioBuffer];
        /*
        Float32 *frame = (Float32*)audioBuffer.mData;
        throwData = [NSData dataWithBytes:audioBuffer.mData length:audioBuffer.mDataByteSize];
        [self.delegate streamer:self didGetAudioBuffer:throwData];
        [data appendBytes:audioBuffer.mData length:audioBuffer.mDataByteSize];
         */
    }

which eventually brings us to the audio queue, set up in this way:

//Apple's own code for canonical PCM
    audioDesc.mSampleRate       = 44100.0;
    audioDesc.mFormatID         = kAudioFormatLinearPCM;
    audioDesc.mFormatFlags      = kAudioFormatFlagsAudioUnitCanonical;
    audioDesc.mBytesPerPacket   = 2 * sizeof (AudioUnitSampleType);    // 8
    audioDesc.mFramesPerPacket  = 1;
    audioDesc.mBytesPerFrame    = 1 * sizeof (AudioUnitSampleType);    // 8
    audioDesc.mChannelsPerFrame = 2;
    audioDesc.mBitsPerChannel   = 8 * sizeof (AudioUnitSampleType);    // 32


err = AudioQueueNewOutput(&audioDesc, handler_OSStreamingAudio_queueOutput, self, NULL, NULL, 0, &audioQueue);
    if(err){
#pragma warning  handle error
//never errs, am using breakpoint to check
        return;
    }

and we enqueue thusly

while (inNumberBytes)
        {
            size_t bufSpaceRemaining = kAQDefaultBufSize - bytesFilled;
            if (bufSpaceRemaining < inNumberBytes)
            {
                AudioQueueBufferRef fillBuf = audioQueueBuffer[fillBufferIndex];
        fillBuf->mAudioDataByteSize = bytesFilled;
        err = AudioQueueEnqueueBuffer(audioQueue, fillBuf, 0, NULL);
            }


                bufSpaceRemaining = kAQDefaultBufSize - bytesFilled;
                size_t copySize;
                if (bufSpaceRemaining < inNumberBytes)
                {
                    copySize = bufSpaceRemaining;
                }
                else
                {
                    copySize = inNumberBytes;
                }

                if (bytesFilled > packetBufferSize)
                {
                    return;
                }

                AudioQueueBufferRef fillBuf = audioQueueBuffer[fillBufferIndex];
                memcpy((char*)fillBuf->mAudioData + bytesFilled, (const char*)(inInputData + offset), copySize);


                bytesFilled += copySize;
                packetsFilled = 0;
                inNumberBytes -= copySize;
                offset += copySize;
            }
        }

I tried to be as code inclusive as possible so as to make it easy for everyone to point out where I'm being a moron. That being said, I have a feeling my problem occurs either in the output settings declaration of the track reader or in the actual declaration of the AudioQueue (where I describe to the queue what kind of audio I'm going to be sending it). The fact of the matter is, I don't really know mathematically how to actually generate those numbers (bytes per packet, frames per packet, what have you). An explanation of that would be greatly appreciated, and thanks for the help in advance.

Not sure how much of an answer this is, but there will be too much text and links for a comment and hopefully it will help (maybe guide you to your answer).

First off I know with my current project adjusting the sample rate will effect the speed of the sound, so you can try to play with those settings. But 44k is what I see in most default implementation including the apple example SpeakHere. However I would spend some time comparing your code to that example because there are quite a few differences. like checking before enqueueing.

First check out this posting http://stackoverflow.com/a/4299665/530933 It talks about how you need to know the audio format, specifically how many bytes in a frame, and casting appropriately

also good luck. I have had quite a few questions posted here, apple forums, and the ios forum (not the official one). With very little responses/help. To get where I am today (audio recording & streaming in ulaw) I ended up having to open an Apple Dev Support Ticket. Which prior to tackling the audio I never knew existed (dev support). One good thing is that if you have a valid dev account you get 2 incidents for free! CoreAudio is not fun. Documentation is sparse, and besides SpeakHere there are not many examples. One thing I did find is that the framework headers do have some good info and this book. Unfortunately I have only started the book otherwise I may be able to help you further.

You can also check some of my own postings which I have tried to answer to the best of my abilities. This is my main audio question which I have spent alot of time on to compile all pertinent links and code.

using AQRecorder (audioqueue recorder example) in an objective c class

trying to use AVAssetWriter for ulaw audio (2)

Does anyone have a good example of using CARingBuffer to buffer a large audio file and how to read it in a callback?

Should it be reading the audio file in a secondary thread? How do I pause loading the audio file until the loaded buffers have been played (how do I pre-queue the audio file)? CAPlayThrough seems close but is only streaming audio from a microphone.

Thanks!

You can find an example that uses this ring buffer if you download the example code of the book Learning Core Audio here (under the downloads tab). Jump to the chapter 8 example in a folder called CH08_AUGraphInput.

However, if you are simply reading audio from a file, then using an (extra) ring buffer seems like an overkill.. A ring buffer comes in handy when you are having real time (or near real time) input and output (read chapter 8 in the said book for a more detailed explanation of when a ring buffer is necessary.. note that the example in chapter 8 is about playing audio immediately after recording it by a mic, which isn't what you want to do).

The reason why I said extra ring buffer, is because in core Audio there is already an audio Queue (which can be thought of as a ring buffer.. or at least it in your case it replaces the need for a ring buffer: you populate it with data, it plays the data, then fires a callback that informs you that the data you supplied has been played). The apple documentation offers a good explanation on this one.

In your case, if you are simply reading audio from a file, then you can easily control the throughput of the audio from the file. You can pause it by blocking the thread that reads data from the audio file for example.

For a simple example of what I'm talking about, see this example I created on github. For a more advanced example, see Matt Gallagher's famous example.

I am building a game that lets users remix songs. I have built a mixer (based upon the apple sample code MixerHost (creating an audioGraph with a mixer audioUnit), but expanded to load 12 tracks. everything is working fine, however it takes a really long time for the songs to load when the gamer selects the songs they want to remix. This is because the program has to load 12 separate mp4 files into memory before I can start playing the music.

I think what I need is to create a AUFilePlayer audioUnit that is in charge of loading the file into the mixer. If the AUFilePlayer can handle loading the file on the fly then the user will not have to wait for the files to load 100% into memory. My two questions are, 1. can an AUFilePlayer be used this way? 2. The documentation on AUFilePlayer is very very very thin. Where can I find some example code demonstrated how to implement a AUFilePlayer properly in IOS (not in MacOS)?

Thanks

I think you're right - in this case a 'direct-from-disk' buffering approach is probably what you need. I believe the correct AudioUnit subtype is AudioFilePlayer. From the documentation:

The unit reads and converts audio file data into its own internal buffers. It performs disk I/O on a high-priority thread shared among all instances of this unit within a process. Upon completion of a disk read, the unit internally schedules buffers for playback.

A working example of using this unit on Mac OS X is given in Chris Adamson's book Learning Core Audio. The code for iOS isn't much different, and is discussed in this thread on the CoreAudio-API mailing list. Adamson's working code example can be found here. You should be able to adapt this to your requirements.

I am trying to stream music from one device to another over a GKSession such that the music is perfectly in sync. So far I know how to grab the audio using this, but I am unsure about a few things. I am pretty sure that I have to use AudioFileReadPackets to transfer the data.

  1. How should I package and send the packet data from the memory buffer as a stream?
  2. On the other end how do I convert the received packets back into an audio stream?
  3. What is the best way to ensure the packets are played in sync?

Important update: I just created a repo on github that pretty much does what you're asking for. Check it out!

below is my older answer:

  1. a. first you want to get audio buffer data from your music, similar to how I did in the github project.

    b. you want to get this data and chunk it up into packets, with each packet containing header information that you will later on use to manage their delivery and esnure that they are synced. So for example the time stamp of each packet, the order of it, the music file hash etc.. see this answer to get an idea, and then take a look at this excellent tutorial that shows you in detail how to package a packet for any purpose.

  2. on the other side of the network, you unpack the packet in a way that's very similar to packaging it. again refer to the above tutorial that discusses at length how to do it.

  3. this is the hardest part. fortunately, you are only dealing with two devices.. so you can do it with GKSession.. otherwise you will have to drop a level lower (ie using bonjour.. wich GKSession is a wrapper of.. and which adds a lot of overhead that you can't afford for a real time app.. but don't panic.. this book has a nice and easy example for using bonjour.. )

the idea is that you want to gather enough packets on the other end to fill an audio queue buffer, then you want to send an unreliable packet (read: udp protocol) to jump start the playback of the music. with a udp send, it's near instant delivery (i'd argue the human ear wouldn't distinguish the delay between just two devices).. once in a while an unreliable packet will be lost though.. so you will have to either live with it or figure a way to handle that.

and last but not least you definitely want to grab a copy of learning core audio.. anyone doing core audio in iOS needs this. Also take a look at the discussion following this answer.. it will bring your attention to a lot of things to deal with when it comes to core audio /real time iOS apps.

good luck!

I'm a total noob when it comes to core audio so bear with me. Basically what I want to do is record audio data from a machine's default mic, record until the user decides to stop, and then do some analysis on the entire recording. I've been learning from the book "Learning Core Audio" by Chis Adamson and Kevin Avila (which is an awesome book, found it here: http://www.amazon.com/Learning-Core-Audio-Hands-On-Programming/dp/0321636848/ref=sr_1_1?ie=UTF8&qid=1388956621&sr=8-1&keywords=learning+core+audio ). I see how the AudioQueue works, but I'm not sure how to get data as it's coming from the buffers and store it in a global array.

The biggest problem is that I can't allocate an array a priori because we have no idea how long the user wants to record for. I'm guessing that a global array would have to be passed to the AudioQueue's callback where it would then append data from the latest buffer, however I'm not exactly sure how to do that, or if that's the correct place to be doing so.

If using AudioUnits I'm guessing that I would need to create two audio units, one a remote IO unit to get the microphone data and one generic output audio unit that would do the data appending in the unit's (I'm guessing here, really not sure) AudioUnitRender() function.

If you know where I need to be doing these things or know any resources that could help explain how this works, that would be awesome.

I eventually want to learn how to do this in iOS and Mac OS platforms. For the time being I'm just working in the Mac OS.

Ok everyone, so I figured out the answer to my own question, and I only had to add about 10 lines of code to get it to work. Basically what I did was in the user data struct, or as Apple calls it the client data struct I added a variable to keep track of the total number of recorded samples, the sample rate (just so I would have access to the value inside the callback) and a pointer that would point to the audio data. In the callback function, I reallocated memory for the pointer and then copied the contents of the buffer into the newly allocated memory.

I'll post the code for my client recorder struct and the lines of code inside the callback function. I would like to post code for the entire program, but much of it was borrowed from the book "Learning Core Audio" by Chis Adamson and Kevin Avila and I don't want to infringe on any copyrights held by the book (can someone tell me if it's legal to post that here or not? if it is I'd be more than happy to post the whole program).

the client recorder struct code:

//user info struct for recording audio queue callbacks
typedef struct MyRecorder{
    AudioFileID recordFile;
    SInt64 recordPacket;
    Boolean running;
    UInt32 totalRecordedSamples;
    Float32 * allData;
    Float64 sampleRate;
}MyRecorder;

This struct needs to be initialized in the main loop of the program. It would look something like this:

MyRecorder recoder = {0};

I know I spelled "recorder" incorrectly.

Next, what I did inside the callback function:

//now let's also write the data to a buffer that can be accessed in the rest of the program
//first calculate the number of samples in the buffer
Float32 nSamples = RECORD_SECONDS * recorder->sampleRate;
nSamples = (UInt32) nSamples;

//so first we need to reallocate memory that the recorder.allData pointer is pointing to
//pretty simple, just add the amount of samples we just recorded to the number that we already had
//to get the current total number of samples, and the size of each sample, which we get from
//sizeof(Float32).
recorder->allData = realloc(recorder->allData, sizeof(Float32) * (nSamples + recorder->totalRecordedSamples));


//now we need to copy what's in the current buffer into the memory that we just allocated
//however, rememeber that we don't want to overwrite what we already just recorded
//so using pointer arith, we need to offset the recorder->allData pointer in memcpy by
//the current value of totalRecordedSamples
memcpy((recorder->allData) + (recorder->totalRecordedSamples), inBuffer->mAudioData, sizeof(Float32) * nSamples);

//update the number of total recorded samples
recorder->totalRecordedSamples += nSamples;

And of course at the end of my program I freed the memory in the recoder.allData pointer.

Also, my experience with C is very limited, so if I'm making some mistakes especially with memory management, please let me know. Some of the malloc, realloc, memcpy etc. type functions in C sort of freak me out.

EDIT: Also I'm now working on how to do the same thing using AudioUnits, I'll post the solution to that when done.

I realize this question has been asked before, but I haven't seen an answer that works for my needs. Basically I have full length songs in .caf format but I need to be able to upload/download them from a server. Is it viable to do compression (to something like .mp3 or .wav) on the iPhone and then send them to the server? How would I go about doing this?

Refer to AudioConverterServices.. the function that does the magic trick is AudioConverterNew.. however note that you can only convert from a variable bit rate (VBR) file to PCM (lossless format), and from PCM to VBR.. so in your case you will have to convert the file from .caf to PCM, then from PCM to .mp3 or whatever format you want.

For a comprehensive example that illustrates all the steps necessary in doing an audio conversion.. download the free sample code for the book Learning Core Audio. You want chapter 6 sample code for audio conversion. (anyone who is serious about core audio should buy that book).

It's the code from Learning Core Audio http://www.amazon.com/Learning-Core-Audio-Hands-On-Programming/dp/0321636848
Just like the book says ,generating stereo should set asbd.mBitsPerChannel to 8 and asbd.mChannelsPerFrame to 2 . But the audio I get only has one channel . I don't know what's wrong with the code ,please help me . Thanks

#import <Foundation/Foundation.h>
#import <AudioToolbox/AudioToolbox.h>

#define SAMPLE_RATE 44100
#define DURATION 5.0
#define FILENAME_FORMAT @"%0.3f-square.aif"

int main(int argc, const char * argv[])
{

@autoreleasepool {
    if (argc < 2) {
        printf("Usage: CAToneFileGenerator n\n(where n is tone in Hz)");
        return -1;
    }

    double hz = atof(argv[1]);
    assert(hz > 0);
    NSLog(@"generating %f hz tone", hz);

    NSString *fileName = [NSString stringWithFormat:FILENAME_FORMAT, hz];
    NSString *filePath = [[[NSFileManager defaultManager] currentDirectoryPath] stringByAppendingPathComponent:fileName];
    NSLog(@"%@", filePath);
    NSURL *fileURL = [NSURL fileURLWithPath:filePath];

    AudioStreamBasicDescription asbd;
    memset(&asbd, 0, sizeof(asbd));
    asbd.mSampleRate = SAMPLE_RATE;
    asbd.mFormatID = kAudioFormatLinearPCM;
    asbd.mFormatFlags = kAudioFormatFlagIsBigEndian | kAudioFormatFlagIsSignedInteger | kAudioFormatFlagIsPacked;
    asbd.mBitsPerChannel = 8;
    asbd.mChannelsPerFrame = 2;
    asbd.mFramesPerPacket = 1;
    asbd.mBytesPerFrame = 2;
    asbd.mBytesPerPacket = 2;

    AudioFileID audioFile;
    OSStatus audioErr = noErr;
    audioErr = AudioFileCreateWithURL((CFURLRef) fileURL, kAudioFileAIFFType, &asbd, kAudioFileFlags_EraseFile, &audioFile);
    assert(audioErr == noErr);

    long maxSampleCount = SAMPLE_RATE * DURATION;
    long sampleCount = 0;
    UInt32 bytesToWrite = 2;
    double wavelengthInSamples = SAMPLE_RATE / hz;

    while (sampleCount < maxSampleCount) {
        for (int i = 0; i < wavelengthInSamples; i++) {
            SInt16 sample;
            if (i < wavelengthInSamples /2) {
                sample = CFSwapInt16HostToBig(SHRT_MAX);
            }else{
                sample = CFSwapInt16HostToBig(SHRT_MIN);
            }

            audioErr = AudioFileWriteBytes(audioFile, false, sampleCount*2, &bytesToWrite, &sample);
            assert(audioErr == noErr);
            sampleCount ++;
        }
    }
    audioErr = AudioFileClose(audioFile);
    assert(audioErr == noErr);
    NSLog(@"wrote %ld samples", sampleCount);
}
return 0;
}

Just changing the ASBD from the book code doesn't magically fix everything. You haven't accounted for how you're writing the samples to the file. Also, 8-bit is going to sound like ass.

Go back to mBitsPerChannel = 16, and then account for the fact you're writing two channels per frame, meaning that mBytesPerFrame and mBytesPerPacket will now be 4 (they were 2 in the book). Think about why this is.

Then you should just be able to add a second call to AudioFileWriteBytes() -- or do a loop where you count over mChannelsPerFrame -- right after the first one. But you'll have to account for the different offsets in the file, since you're writing 4 bytes each pass instead of 2. I think this is right:

  audioErr = AudioFileWriteBytes(audioFile, false, sampleCount*4, &bytesToWrite, &sample); // left
  audioErr = AudioFileWriteBytes(audioFile, false, (sampleCount*4)+2, &bytesToWrite, &sample); // right

You need to figure out some of this stuff on your own in order for it to sink in.

So, basically I want to play some audio files (mp3 and caf mostly). But the callback never gets called. Only when I call them to prime the queue.

Here's my data struct:

struct AQPlayerState
{
    CAStreamBasicDescription        mDataFormat;
    AudioQueueRef                   mQueue;
    AudioQueueBufferRef             mBuffers[kBufferNum];
    AudioFileID                     mAudioFile;
    UInt32                          bufferByteSize;
    SInt64                          mCurrentPacket;
    UInt32                          mNumPacketsToRead;
    AudioStreamPacketDescription   *mPacketDescs;
    bool                            mIsRunning;
};

Here's my callback function:

static void HandleOutputBuffer (void *aqData, AudioQueueRef inAQ, AudioQueueBufferRef inBuffer)
{
    NSLog(@"HandleOutput");
    AQPlayerState *pAqData = (AQPlayerState *) aqData;

    if (pAqData->mIsRunning == false) return;

    UInt32 numBytesReadFromFile;
    UInt32 numPackets = pAqData->mNumPacketsToRead;
    AudioFileReadPackets (pAqData->mAudioFile,
                          false,
                          &numBytesReadFromFile,
                          pAqData->mPacketDescs,
                          pAqData->mCurrentPacket,
                          &numPackets,
                          inBuffer->mAudioData);
    if (numPackets > 0) {    
        inBuffer->mAudioDataByteSize = numBytesReadFromFile;
        AudioQueueEnqueueBuffer (pAqData->mQueue,
                                 inBuffer,
                                 (pAqData->mPacketDescs ? numPackets : 0),
                                 pAqData->mPacketDescs);
        pAqData->mCurrentPacket += numPackets;
    } else {
//        AudioQueueStop(pAqData->mQueue, false);
//        AudioQueueDispose(pAqData->mQueue, true);
//        AudioFileClose (pAqData->mAudioFile);
//        free(pAqData->mPacketDescs);
//        free(pAqData->mFloatBuffer);
        pAqData->mIsRunning = false;
    }
}

And here's my method:

- (void)playFile
{
    AQPlayerState aqData;

    // get the source file
    NSString *p = [[NSBundle mainBundle] pathForResource:@"1_Female" ofType:@"mp3"];
    NSURL *url2 = [NSURL fileURLWithPath:p];
    CFURLRef srcFile = (__bridge CFURLRef)url2;

    OSStatus result = AudioFileOpenURL(srcFile, 0x1/*fsRdPerm*/, 0/*inFileTypeHint*/, &aqData.mAudioFile);
    CFRelease (srcFile);

    CheckError(result, "Error opinning sound file");

    UInt32 size = sizeof(aqData.mDataFormat);
    CheckError(AudioFileGetProperty(aqData.mAudioFile, kAudioFilePropertyDataFormat, &size, &aqData.mDataFormat),
               "Error getting file's data format");

    CheckError(AudioQueueNewOutput(&aqData.mDataFormat, HandleOutputBuffer, &aqData, CFRunLoopGetCurrent(), kCFRunLoopCommonModes, 0, &aqData.mQueue),
               "Error AudioQueueNewOutPut");

    // we need to calculate how many packets we read at a time and how big a buffer we need
    // we base this on the size of the packets in the file and an approximate duration for each buffer
    {
        bool isFormatVBR = (aqData.mDataFormat.mBytesPerPacket == 0 || aqData.mDataFormat.mFramesPerPacket == 0);

        // first check to see what the max size of a packet is - if it is bigger
        // than our allocation default size, that needs to become larger
        UInt32 maxPacketSize;
        size = sizeof(maxPacketSize);
        CheckError(AudioFileGetProperty(aqData.mAudioFile, kAudioFilePropertyPacketSizeUpperBound, &size, &maxPacketSize),
                   "Error getting max packet size");

        // adjust buffer size to represent about a second of audio based on this format
        CalculateBytesForTime(aqData.mDataFormat, maxPacketSize, 1.0/*seconds*/, &aqData.bufferByteSize, &aqData.mNumPacketsToRead);

        if (isFormatVBR) {
            aqData.mPacketDescs = new AudioStreamPacketDescription [aqData.mNumPacketsToRead];
        } else {
            aqData.mPacketDescs = NULL; // we don't provide packet descriptions for constant bit rate formats (like linear PCM)
        }

        printf ("Buffer Byte Size: %d, Num Packets to Read: %d\n", (int)aqData.bufferByteSize, (int)aqData.mNumPacketsToRead);
    }

    // if the file has a magic cookie, we should get it and set it on the AQ
    size = sizeof(UInt32);
    result = AudioFileGetPropertyInfo(aqData.mAudioFile, kAudioFilePropertyMagicCookieData, &size, NULL);

    if (!result && size) {
        char* cookie = new char [size];
        CheckError(AudioFileGetProperty(aqData.mAudioFile, kAudioFilePropertyMagicCookieData, &size, cookie),
                   "Error getting cookie from file");
        CheckError(AudioQueueSetProperty(aqData.mQueue, kAudioQueueProperty_MagicCookie, cookie, size),
                   "Error setting cookie to file");
        delete[] cookie;
    }

    aqData.mCurrentPacket = 0;
    for (int i = 0; i < kBufferNum; ++i) {
        CheckError(AudioQueueAllocateBuffer (aqData.mQueue,
                                             aqData.bufferByteSize,
                                             &aqData.mBuffers[i]),
                   "Error AudioQueueAllocateBuffer");

        HandleOutputBuffer (&aqData,
                            aqData.mQueue,
                            aqData.mBuffers[i]);
    }

    // set queue's gain

    Float32 gain = 1.0;
    CheckError(AudioQueueSetParameter (aqData.mQueue,
                                       kAudioQueueParam_Volume,
                                       gain),
               "Error AudioQueueSetParameter");

    aqData.mIsRunning = true;
    CheckError(AudioQueueStart(aqData.mQueue,
                               NULL),
               "Error AudioQueueStart");

}

And the output when I press play:

Buffer Byte Size: 40310, Num Packets to Read: 38
HandleOutput start
HandleOutput start
HandleOutput start

I tryed replacing CFRunLoopGetCurrent() with CFRunLoopGetMain() and CFRunLoopCommonModes with CFRunLoopDefaultMode, but nothing.

Shouldn't the primed buffers start playing right away I start the queue? When I start the queue, no callbacks are bang fired. What am I doing wrong? Thanks for any ideas

What you are basically trying to do here is a basic example of audio playback using Audio Queues. Without looking at your code in detail to see what's missing (that could take a while) i'd rather recommend to you to follow the steps in this basic sample code that does exactly what you're doing (without the extras that aren't really relevant.. for example why are you trying to add audio gain?)

Somewhere else you were trying to play audio using audio units. Audio units are more complex than basic audio queue playback, and I wouldn't attempt them before being very comfortable with audio queues. But you can look at this example project for a basic example of audio queues.

In general when it comes to Core Audio programming in iOS, it's best you take your time with the basic examples and build your way up.. the problem with a lot of tutorials online is that they add extra stuff and often mix it with obj-c code.. when Core Audio is purely C code (ie the extra stuff won't add anything to the learning process). I strongly recommend you go over the book Learning Core Audio if you haven't already. All the sample code is available online, but you can also clone it from this repo for convenience. That's how I learned core audio. It takes time :)

Here is my code for this. I copied some of them from iOS library.

- (void)getSamplesWithFileName:(NSString *)fileName
                storeToPointer:(SInt16*)pointer
           withNumberOfSamples:(UInt32*)numberOfSamples
{
    NSFileManager* fileManager = [NSFileManager defaultManager];
    NSURL* url = [[fileManager URLsForDirectory:NSCachesDirectory inDomains:NSUserDomainMask] lastObject];
    NSLog(@"%@",url);
    NSString* directory = [url path];
    NSString* nsFilePath = [directory stringByAppendingPathComponent:fileName];
    const char* filePath = [nsFilePath cStringUsingEncoding:[NSString defaultCStringEncoding]];
    CFURLRef audioFileURL =
    CFURLCreateFromFileSystemRepresentation (           // 1
                                             NULL,                                           // 2
                                             (const UInt8 *) filePath,                       // 3
                                             strlen (filePath),                              // 4
                                             false                                           // 5
                                             );
    struct AQPlayerState aqData;                                   // 1
    aqData.mDataFormat.mFormatID         = kAudioFormatLinearPCM; // 2
    aqData.mDataFormat.mSampleRate       = 8000.0;               // 3
    aqData.mDataFormat.mChannelsPerFrame = 1;                     // 4
    aqData.mDataFormat.mBitsPerChannel   = 16;                    // 5
    aqData.mDataFormat.mBytesPerPacket   =                        // 6
    aqData.mDataFormat.mBytesPerFrame =
    aqData.mDataFormat.mChannelsPerFrame * sizeof (SInt16);
    aqData.mDataFormat.mFramesPerPacket  = 1;                     // 7

    OSStatus result =
    AudioFileOpenURL (                                  // 2
                      audioFileURL,                                   // 3
                      kAudioFileReadPermission,//fsRdPerm                                       // 4
                      0,                                              // 5
                      &aqData.mAudioFile                              // 6
                      );
    NSLog(@"Play back open status:%ld", result);
    CFRelease (audioFileURL);                               // 7
    UInt32 dataFormatSize = sizeof (aqData.mDataFormat);    // 1
    AudioFileGetProperty (                                  // 2
                          aqData.mAudioFile,                                  // 3
                          kAudioFilePropertyDataFormat,                       // 4
                          &dataFormatSize,                                    // 5
                          &aqData.mDataFormat                                 // 6
                          );
    UInt32 numBytesReadFromFile;
    UInt32 numPackets = 100000;
//    pointer = malloc(numPackets * sizeof(SInt16));
    AudioFileReadPackets(aqData.mAudioFile, false, &numBytesReadFromFile, NULL, 0, &numPackets, pointer);
    printf("%ld", numBytesReadFromFile);
    *numberOfSamples = numBytesReadFromFile;    
}

But it seems that I get the wrong data! I tested the voice 'Ahhhhhh', and get really high zero-crossing rate. How exactly can I read this audio file?

Use Extended Audio File Services

Open files with ExtAudioFileOpenURL, read them into buffers with ExtAudioFileRead. Works with all formats Core Audio supports.

I can recommend the book Learning Core Audio: A Hands-On Guide to Audio Programming for Mac and iOS if you want to get into Core Audio programming.

Edit: The docs at the first link has sample code that'll probably help you.

I have found many related questions on SO, but or because I am very new, or because it doesn't exactly fit my problem, I can't find a way to succeed.

So, all I would like to do is playing an audio file stored on an exernal server directly on the iOS device (iPhone in my case)

I have tried differents solution on the net, and this is what I get until now.

NSString* resourcePath = @"http://www.chiptape.com/chiptape/sounds/medium/laugh.mp3"; //your url
NSURL *url = [[NSURL alloc]initWithString:resourcePath];

NSError *error;

audioPlayer = [[AVAudioPlayer alloc] initWithURL:url error:&error];

audioPlayer.delegate = self;
[url release];

[audioPlayer prepareToPlay];
[audioPlayer play];

The compiler show me this error:

No visible @interface for 'AVAudioPlayer' declares the selector 'initWithURL:error:'

I am not sure at all about my code, and would like your guys to know I have started iOS coding yesterday, so it's not impossible I'm missing something so obvious that tutorials/posts don't mention it.

Also, I have imported the library AVFoundation and imported it on my .h file. I have also declared this on my .h file

AVAudioPlayer *audioPlayer;

I hope there is an easy way to stream audio file on iOS as provide android/java

Thank you and if you need more details comment and I'll update my post.

I'm working with it recently.

AVAudioPlayer can't work with audio stream. In fact, there isn't an easy way to do that.

You need to learn some Core Audio things, get the raw data of media, play them in audioQueue (audioToolBox framework). Real-time audio processing is not a simple work, it's full of c-level api and so many dirty work, don't rush, have fun :)

Check out the book Learning Core Audio and this open-source audio streamer.

Let me know if you need more help.

I'm making something like streaming the audio from iPod library, send the data via network or bluetooth, and playback using audio queue.

Thanks for this question and code. Help me a lot.

I have two question about it.

  1. what should I send from one device to another? CMSampleBufferRef? AudioBuffer? mData? AudioQueueBuffer? packet? I have no idea.

  2. When the app finished the playing, it crashed, and I got error (-12733). I just want to know how to handle the errors instead of letting it crash. (check the OSState? When the error happened, stop it?)

    Error: could not read sample data (-12733)

I will answer your second question first - don't wait for the app to crash, you can stop pulling audio from the track by checking the number of samples that are available in the CMSampleBufferRef you are reading; for example (this code will also be included in the 2nd half of my answer):

CMSampleBufferRef sample;
sample = [readerOutput copyNextSampleBuffer];

CMItemCount numSamples = CMSampleBufferGetNumSamples(sample);

if (!sample || (numSamples == 0)) {
  // handle end of audio track here
  return;
}

Regarding your first question, it depends on the type of audio you are grabbing - it could be wither PCM (non-compressed) or VBR (compressed) format. I'm not even going to bother addressing the PCM part because it's simply not smart to send uncompressed audio data from one phone to another over the network - it's unnecessarily expensive and will clog your networking bandwidth. So we're left with VBR data. For that you've got to send the contents of AudioBuffer and AudioStreamPacketDescription you pulled from the sample. But then again, it's probably best to explain what I'm saying by code:

-(void)broadcastSample
{
    [broadcastLock lock];

CMSampleBufferRef sample;
sample = [readerOutput copyNextSampleBuffer];

CMItemCount numSamples = CMSampleBufferGetNumSamples(sample);

if (!sample || (numSamples == 0)) {
    Packet *packet = [Packet packetWithType:PacketTypeEndOfSong];
    packet.sendReliably = NO;
    [self sendPacketToAllClients:packet];
    [sampleBroadcastTimer invalidate];
    return;
}


        NSLog(@"SERVER: going through sample loop");
        Boolean isBufferDataReady = CMSampleBufferDataIsReady(sample);



        CMBlockBufferRef CMBuffer = CMSampleBufferGetDataBuffer( sample );                                                         
        AudioBufferList audioBufferList;  

        CheckError(CMSampleBufferGetAudioBufferListWithRetainedBlockBuffer(
                                                                           sample,
                                                                           NULL,
                                                                           &audioBufferList,
                                                                           sizeof(audioBufferList),
                                                                           NULL,
                                                                           NULL,
                                                                           kCMSampleBufferFlag_AudioBufferList_Assure16ByteAlignment,
                                                                           &CMBuffer
                                                                           ),
                   "could not read sample data");

        const AudioStreamPacketDescription   * inPacketDescriptions;

        size_t                               packetDescriptionsSizeOut;
        size_t inNumberPackets;

        CheckError(CMSampleBufferGetAudioStreamPacketDescriptionsPtr(sample, 
                                                                     &inPacketDescriptions,
                                                                     &packetDescriptionsSizeOut),
                   "could not read sample packet descriptions");

        inNumberPackets = packetDescriptionsSizeOut/sizeof(AudioStreamPacketDescription);

        AudioBuffer audioBuffer = audioBufferList.mBuffers[0];



        for (int i = 0; i < inNumberPackets; ++i)
        {

            NSLog(@"going through packets loop");
            SInt64 dataOffset = inPacketDescriptions[i].mStartOffset;
            UInt32 dataSize   = inPacketDescriptions[i].mDataByteSize;            

            size_t packetSpaceRemaining = MAX_PACKET_SIZE - packetBytesFilled - packetDescriptionsBytesFilled;
            size_t packetDescrSpaceRemaining = MAX_PACKET_DESCRIPTIONS_SIZE - packetDescriptionsBytesFilled;        

            if ((packetSpaceRemaining < (dataSize + AUDIO_STREAM_PACK_DESC_SIZE)) || 
                (packetDescrSpaceRemaining < AUDIO_STREAM_PACK_DESC_SIZE))
            {
                if (![self encapsulateAndShipPacket:packet packetDescriptions:packetDescriptions packetID:assetOnAirID])
                    break;
            }

            memcpy((char*)packet + packetBytesFilled, 
                   (const char*)(audioBuffer.mData + dataOffset), dataSize);

            memcpy((char*)packetDescriptions + packetDescriptionsBytesFilled, 
                   [self encapsulatePacketDescription:inPacketDescriptions[i]
                                         mStartOffset:packetBytesFilled
                    ],
                   AUDIO_STREAM_PACK_DESC_SIZE);  


            packetBytesFilled += dataSize;
            packetDescriptionsBytesFilled += AUDIO_STREAM_PACK_DESC_SIZE; 

            // if this is the last packet, then ship it
            if (i == (inNumberPackets - 1)) {          
                NSLog(@"woooah! this is the last packet (%d).. so we will ship it!", i);
                if (![self encapsulateAndShipPacket:packet packetDescriptions:packetDescriptions packetID:assetOnAirID])
                    break;

            }

        }

    [broadcastLock unlock];
}

Some methods that I've used in the above code are methods you don't need to worry about, such as adding headers to each packet (I was creating my own protocol, you can create your own). For more info see this tutorial.

- (BOOL)encapsulateAndShipPacket:(void *)source
              packetDescriptions:(void *)packetDescriptions
                        packetID:(NSString *)packetID
{

    // package Packet
    char * headerPacket = (char *)malloc(MAX_PACKET_SIZE + AUDIO_BUFFER_PACKET_HEADER_SIZE + packetDescriptionsBytesFilled);

    appendInt32(headerPacket, 'SNAP', 0);    
    appendInt32(headerPacket,packetNumber, 4);    
    appendInt16(headerPacket,PacketTypeAudioBuffer, 8);   
    // we use this so that we can add int32s later
    UInt16 filler = 0x00;
    appendInt16(headerPacket,filler, 10);    
    appendInt32(headerPacket, packetBytesFilled, 12);
    appendInt32(headerPacket, packetDescriptionsBytesFilled, 16);    
    appendUTF8String(headerPacket, [packetID UTF8String], 20);


    int offset = AUDIO_BUFFER_PACKET_HEADER_SIZE;        
    memcpy((char *)(headerPacket + offset), (char *)source, packetBytesFilled);

    offset += packetBytesFilled;

    memcpy((char *)(headerPacket + offset), (char *)packetDescriptions, packetDescriptionsBytesFilled);

    NSData *completePacket = [NSData dataWithBytes:headerPacket length: AUDIO_BUFFER_PACKET_HEADER_SIZE + packetBytesFilled + packetDescriptionsBytesFilled];        



    NSLog(@"sending packet number %lu to all peers", packetNumber);
    NSError *error;    
    if (![_session sendDataToAllPeers:completePacket withDataMode:GKSendDataReliable error:&error])   {
        NSLog(@"Error sending data to clients: %@", error);
    }   

    Packet *packet = [Packet packetWithData:completePacket];

    // reset packet 
    packetBytesFilled = 0;
    packetDescriptionsBytesFilled = 0;

    packetNumber++;
    free(headerPacket);    
    //  free(packet); free(packetDescriptions);
    return YES;

}

- (char *)encapsulatePacketDescription:(AudioStreamPacketDescription)inPacketDescription
                          mStartOffset:(SInt64)mStartOffset
{
    // take out 32bytes b/c for mStartOffset we are using a 32 bit integer, not 64
    char * packetDescription = (char *)malloc(AUDIO_STREAM_PACK_DESC_SIZE);

    appendInt32(packetDescription, (UInt32)mStartOffset, 0);
    appendInt32(packetDescription, inPacketDescription.mVariableFramesInPacket, 4);
    appendInt32(packetDescription, inPacketDescription.mDataByteSize,8);    

    return packetDescription;
}

receiving data:

- (void)receiveData:(NSData *)data fromPeer:(NSString *)peerID inSession:(GKSession *)session context:(void *)context
{

    Packet *packet = [Packet packetWithData:data];
    if (packet == nil)
    {
         NSLog(@"Invalid packet: %@", data);
        return;
    }

    Player *player = [self playerWithPeerID:peerID];

    if (player != nil)
    {
        player.receivedResponse = YES;  // this is the new bit
    } else {
        Player *player = [[Player alloc] init];
        player.peerID = peerID;
        [_players setObject:player forKey:player.peerID];
    }

    if (self.isServer)
    {
        [Logger Log:@"SERVER: we just received packet"];   
        [self serverReceivedPacket:packet fromPlayer:player];

    }
    else
        [self clientReceivedPacket:packet];
}

notes:

  1. There are a lot of networking details that I didn't cover here (ie, in the receiving data part. I used a lot of custom made objects without expanding on their definition). I didn't because explaining all of that is beyond the scope of just one answer on SO. However, you can follow the excellent tutorial of Ray Wenderlich. He takes his time in explaining networking principles, and the architecture I use above is almost taken verbatim from him. HOWEVER THERE IS A CATCH (see next point)

  2. Depending on your project, GKSession may not be suitable (especially if your project is realtime, or if you need more than 2-3 devices to connect simultaneously) it has a lot of limitations. You will have to dig down deeper and use Bonjour directly instead. iPhone cool projects has a nice quick chapter that gives a nice example of using Bonjour services. It's not as scary as it sounds (and the apple documentation is kinda overbearing on that subject).

  3. I noticed you use GCD for your multithreading. Again, if you are dealing with real time then you don't want to use advanced frameworks that do the heavy lifting for you (GCD is one of them). For more on this subject read this excellent article. Also read the prolonged discussion between me and justin in the comments of this answer.

  4. You may want to check out MTAudioProcessingTap introduced in iOS 6. It can potentially save you some hassle while dealing with AVAssets. I didn't test this stuff though. It came out after I did all my work.

  5. Last but not least, you may want to check out the learning core audio book. It's a widely acknowledged reference on this subject. I remember being as stuck as you were at the point you asked the question. Core audio is heavy duty and it takes time to sink in. SO will only give you pointers. You will have to take your time to absorb the material yourself then you will figure out how things work out. Good luck!

I'm quite new to Core Audio development. I did many google searches but so far I couldn't come up with something useful for bootstrapping the process.I looked through the Apple Audio Unit Guide but it is outdated and uses Core Audio SDK which I even cant find for downloading. Is there any useful resources with audio Unit development apart from the horrible Apple guide?

I feel that this book has a good beginning overview of Audio Units (better than any Apple provides) and equips the reader with the tools to build more complicated programs.

http://www.amazon.com/Learning-Core-Audio-Hands-On-Programming/dp/0321636848

Chapter 7 in the book has an example of adding a reverb effect. So you can use that to guide you on whatever audio effects you want to add. As for the equalizer functions, there are audio units for it, also located in Chapter 7. If none of these accomplish what you want, you can always intercept the PCM audio data in the callback and manipulate the raw data. Though that does require you to know some DSP, which is too advanced for me.

Here are some useful posts I found:

http://www.deluge.co/?q=content/coreaudio-iphone-creating-graphic-equalizer

http://www.musicdsp.org/files/Audio-EQ-Cookbook.txt

Getting started with programmatic audio

I would like to modulate the signal from the mic input with a sine wave at 200HZ (FM only). Anyone know of any good tutorials/articles that will help get me started?

Any info is very welcome Thanks

The standard way to do audio processing in iOS or OSX is Core Audio. Here's Apple's overview of the framework.

However, Core Audio has a reputation of being very difficult to learn, especially if you don't have experience with C. If you're still wanting to learn Core Audio, then this book is the way to go: Learning Core Audio.

There are simpler ways to work with audio on iOS and OSX, one of them being AudioKit, which was developed specifically so developers can quickly prototype audio without having to deal with lower-level memory management, buffers, and pointer arithmetic.

There are examples showing both FM synthesis and audio input via the microphone, so you should have everything you need :)

Full disclosure: I am one of the developers of AudioKit.

Hello I am trying to learn core audio farm this book:http://www.amazon.com/Learning-Core-Audio-Hands-On-Programming/dp/0321636848

But when I try to run this code:

      #import <Foundation/Foundation.h>
     #import <AudioToolbox/AudioToolbox.h>

#define SAMPLE_RATE 44100
 #define DURATION 5
  #define FILENAME_FORMAT @"%0.03f-test.aif"

  int main(int argc, const char * argv[])
 {

@autoreleasepool
{
    if(argc<2)
        return -1;

    double hz = 44;
    assert(hz>0);
    NSLog(@"Generating hz tone:%f",hz);

    NSString* fileName = [NSString stringWithFormat:FILENAME_FORMAT, hz];
    NSString* filePath = [[[NSFileManager defaultManager]currentDirectoryPath]
                          stringByAppendingPathComponent:fileName];
    NSURL* fileURL = [NSURL fileURLWithPath:filePath];


    AudioStreamBasicDescription asbd;
    memset(&asbd, 0, sizeof(asbd));
    asbd.mSampleRate = SAMPLE_RATE;
    asbd.mFormatID = kAudioFormatLinearPCM;
    asbd.mFormatFlags = kAudioFormatFlagIsBigEndian | kAudioFormatFlagIsSignedInteger | kAudioFormatFlagIsPacked;
    asbd.mBitsPerChannel = 16;
    asbd.mChannelsPerFrame = 1;
    asbd.mFramesPerPacket = 1;
    asbd.mBytesPerFrame = 2;
    asbd.mBytesPerPacket = 2;


    AudioFileID audioFile;
    OSStatus audioErr = noErr;
    audioErr = AudioFileCreateWithURL((__bridge CFURLRef)fileURL, kAudioFileAIFFType, &asbd, kAudioFileFlags_EraseFile, &audioFile);
    assert(audioErr == noErr);

    long maxSampleCount = SAMPLE_RATE * DURATION;
    long sampleCount = 0;
    UInt32 bytesToWrite = 2;
    double waveLengthInSamples = SAMPLE_RATE / hz;

    while(sampleCount < maxSampleCount)
    {
        for(int i=0;i<waveLengthInSamples;i++)
        {
            SInt16 sample;
            if(i<waveLengthInSamples/2)
                sample = CFSwapInt16BigToHost(SHRT_MAX);
            else
                sample = CFSwapInt16BigToHost(SHRT_MIN);

            audioErr = AudioFileWriteBytes(audioFile, false, sampleCount*2, &bytesToWrite, &sample);
            assert(audioErr = noErr);
            sampleCount++;
        }
    }

    audioErr = AudioFileClose(audioFile);
    assert(audioErr = noErr);


}
return 0;

}

The program exit with this error code: Program ended with exit code: 255

Can anyone help me? I downloaded the sample code and the the same error occurs. I am using xcode 5 and a 64bit macbook. Thanks for your help.

I am getting a -50 (general param error) from a call to AudioQueueGetProperty. Please help me as it has been several months since I've touched XCode and any iPhone work. This is likely a simple goof on my behalf but I cannot resolve it. My code leading to the -50:

//Setup format
AudioStreamBasicDescription recordFormat;
memset(&recordFormat, 0, sizeof(recordFormat));
recordFormat.mFormatID = kAudioFormatMPEG4AAC;
recordFormat.mChannelsPerFrame = 2;
CCGetDefaultInputDeviceSampleRate(&recordFormat.mSampleRate);
UInt32 propSize = sizeof(recordFormat);
AQ(AudioFormatGetProperty(kAudioFormatProperty_FormatInfo, 0, NULL,
                                  &propSize, &recordFormat), 
           "AudioFormatGetProperty throws unexpected errors.");

//Setup Queue
//listing 4.8-4.9
AudioQueueRef theQueue = {0};
self->queue = theQueue;
AQ(AudioQueueNewInput(&recordFormat, CCAudioRecordingCallback, 
                              self, NULL, NULL, 0, &self->queue),
           "AudioQueueNewInput throws unexpected errors.");
UInt32 size = sizeof(recordFormat);
AQ(AudioQueueGetProperty(self->queue,
                                 kAudioConverterCurrentOutputStreamDescription, 
                                 &recordFormat, 
                                 &size), 
           "Getting audio property kAudioConverterCurrentOutputStreamDescription throws unexpected errors.");

I have verified that I have a valid queue just as I make the call to AudioQueueGetProperty. I've tried both ways of passing the queue "self->queue", and "self.queue" and they both result in the same error. The queue is defined as follows:

@interface CCAudioRecorder()
//...
@property (nonatomic, assign) AudioQueueRef queue;
//...

@end

@implementation CCAudioRecorder
@synthesize queue;

AQ is a #def:

#define AQ(expr, msg) if(nil!=CheckError((expr), msg)) [NSException raise:@"AudioException" format:@"Unexpected exception occured."];

Which resolves to the following error checking function:

static NSString* CheckError(OSStatus error, const char* operation)
{
    if (noErr == error) return nil;

    NSString *errorMessage = nil;
    char errorString[20];
    //See if it appears to be a 4-char code
    *(UInt32 *)(errorString+1) = CFSwapInt32HostToBig(error);
    if ( isprint(errorString[1]) && isprint(errorString[2]) 
        && isprint(errorString[3]) && isprint(errorString[4]) ) 
    {
        errorString[0] = errorString[5] = '\'';
        errorString[6] = '\0';
    } else {
        sprintf(errorString, "%d", (int)error);
    }
    errorMessage = [NSString stringWithFormat:
                    @"Audio Error: %@ (%@)\n", 
                    [NSString stringWithUTF8String:operation], 
                    [NSString stringWithUTF8String:errorString]];
    NSLog(@"%@", errorMessage);
    return errorMessage;
}

I've also tried calling AudioFormatGetProperty on a queue created with a local variable, avoiding the class instance level iVar and still get the same error:

AudioQueueRef theQueue = {0};
AQ(AudioQueueNewInput(&recordFormat, CCAudioRecordingCallback, 
                              self, NULL, NULL, 0, &theQueue),
           "AudioQueueNewInput throws unexpected errors.");
UInt32 size = sizeof(recordFormat);
AQ(AudioQueueGetProperty(theQueue,
                                 kAudioConverterCurrentOutputStreamDescription, 
                                 &recordFormat, 
                                 &size), 
           "Getting audio property kAudioConverterCurrentOutputStreamDescription throws unexpected errors.");

** Update ** I have the following code which works on the simulator and not on the device. (I have not cross referenced it with what I posted earlier but I believe it's either similar or the exact.)

AudioQueueRef theQueue = {0};
self->queue = theQueue;
AQ(AudioQueueNewInput(&recordFormat, CCAudioRecordingCallback, 
                              self, NULL, NULL, 0, &self->queue),
           "AudioQueueNewInput throws unexpected errors.");
UInt32 size = sizeof(recordFormat);
AQ(AudioQueueGetProperty(self->queue, 
                                 kAudioConverterCurrentOutputStreamDescription, 
                                 &recordFormat, 
                                 &size), 
           "Getting audio property kAudioConverterCurrentOutputStreamDescription throws unexpected errors.");

Running it on device I get a crash in the same spot with error -50 general param error. My device is an iPhone 4S running iOS6 I'm working with XCode 4.5.

You are following the code from the learning core audio book right? Chapter 4 about implementing a recorder? I noticed a difference between your code and the book's code, in the book they simply initialize the Queue and use it as is:

AudioQueueRef queue = {0};
UInt32 size = sizeof(recordFormat);
CheckError(AudioQueueGetProperty(queue, kAudioConverterCurrentOutputStreamDescription,
                                 &recordFormat, &size), "couldn't get queue's format");

I'm not sure why you're throwing self in the mix. But that's definitely what's causing your bug. If all fails simply download the complete code for that chapter here and see where you can identify your mistake.

I just saw an iPhone app which uses wavetables to generate sounds. I wish to know how it is possible to implement. I am pretty much sure that core audio have to be used, but any other idea where to go for some other info will be appreciated.

Audio file I/O: I found a great resource here. This guy created an excellent API for using ExtAudioFileServices.

A must read is Learning Core Audio. Chris Adamson and company have really put together a great resource. Chris's blog can also be found here

Also, sign up for the Core Audio mailing list.

Michael Tyson's blog/ resources are great too A Tasty Pixel.

Hope this helps!

I'm programming an app on ios with xcode and I after a lot of work I found out that the functionality is really dependant on the accuracy of the methods calls. Calling them one line after the other in the code doesn't help, they're still called at up tp 150 ms difference after each other. So I need to make two methods run at the minimal time difference, hence "at the same time". These two tasks I'm performing are actually audio and video ones so I understand it might include also in-process latency and delays, so I was wondering maybe you guys would have any isight on how to sync an audio task and a video task so that they start running together, with a very tiny time gap. I tried using dispatch queues and stuff like that, but they don't work.

I'll be happy to elaborate if I wasn't clear enough. Thanks!

Obviously calling methods sequentially (serially) won't do what you are asking since each method call will take some finite amount of time, during which time the subsequent methods will be blocked.

To run multiple general-purpose tasks concurrently (potentially truly simultaneously on a modern multi-core device) you do need threading via "dispatch queues and stuff like that." GCD most certainly does work, so you are going to need to elaborate on what you mean by "they don't work."

But this is all probably for naught, because you aren't talking about general-purpose tasks. If you are handling audio and video tasks, you really shouldn't do all this with your own code on the CPU; you need hardware acceleration, and Apple provides frameworks to help. I'd probably start by taking a look at the AV Foundation framework, then, depending on what you are trying to do (you didn't really say in your question...) take a look at, variously, OpenAL, Core Video, and Core Audio. (There's a book out on the latter that's getting rave reviews, but I haven't seen it myself and, truth be told, I'm not an A/V developer.)

I'm trying to generate a square wave using Core Audio with the code below. The AIFF-file generates nothing whatever I pass any arguments.The sample is from Learning Core Audio by Kevin Avila http://www.amazon.com/Learning-Core-Audio-Hands-On-Programming/dp/0321636848

Any Ideas? Please help!

The main.m looks as follows;

#import <Foundation/Foundation.h>
#import <AudioToolbox/AudioToolbox.h>
#define SAMPLE_RATE 44100
#define DURATION 50.0
#define FILENAME_FORMAT @"%0.3f-square.aif"
int main (int argc, const char * argv[]) {
@autoreleasepool {
    if (argc < 2) {
        printf ("Usage: CAToneFileGenerator n\n(where n is tone in Hz)");
        return -1;
        }
    double hz = atof(argv[1]);
    assert (hz > 0);
    NSLog (@"generating %f hz tone", hz);
    NSString *fileName = [NSString stringWithFormat: FILENAME_FORMAT, hz];
    NSString *filePath = [[[NSFileManager defaultManager] currentDirectoryPath]
                          stringByAppendingPathComponent: fileName]; NSURL *fileURL = [NSURL fileURLWithPath: filePath];
    // Prepare the format
    AudioStreamBasicDescription asbd;
    memset(&asbd, 0, sizeof(asbd));
    asbd.mSampleRate = SAMPLE_RATE;
    asbd.mFormatID = kAudioFormatLinearPCM;
    asbd.mFormatFlags = kAudioFormatFlagIsBigEndian | kAudioFormatFlagIsSignedInteger | kAudioFormatFlagIsPacked;
    asbd.mBitsPerChannel = 16;
    asbd.mChannelsPerFrame = 1;
    asbd.mFramesPerPacket = 1;
    asbd.mBytesPerFrame = 2; asbd.mBytesPerPacket = 2;
    // Set up the file
    AudioFileID audioFile;
    OSStatus audioErr = noErr;
    audioErr = AudioFileCreateWithURL((__bridge CFURLRef)fileURL,
                                      kAudioFileAIFFType,
                                      &asbd,
                                      kAudioFileFlags_EraseFile,
                                      &audioFile);
    assert (audioErr == noErr);
    // Start writing samples
    long maxSampleCount = SAMPLE_RATE;
    long sampleCount = 0;
    UInt32 bytesToWrite = 2;
    double wavelengthInSamples = SAMPLE_RATE / hz;
    while (sampleCount < maxSampleCount) {
        for (int i=0; i<wavelengthInSamples; i++) {
            // Square wave
            SInt16 sample;
            if (i < wavelengthInSamples/2) {
                sample = CFSwapInt16HostToBig (SHRT_MAX); } else {
                    sample = CFSwapInt16HostToBig (SHRT_MIN); }
            audioErr = AudioFileWriteBytes(audioFile, false,
                                           sampleCount*2, &bytesToWrite, &sample);
            assert (audioErr == noErr); sampleCount++;
            audioErr = AudioFileClose(audioFile); assert (audioErr == noErr);
            NSLog (@"wrote %ld samples", sampleCount);
        }
        return 0;
    }
}

}

Pretty sure your AudioFileClose() should not be inside the for-loop. That should go outside the loop, after you've written all the samples.

Please check against the downloadable code at the book's home page. I just downloaded and ran it, and it works as described in the book (although, since it was built for Lion, you will have to update the target SDK in the project settings if you're on Mountain Lion or Mavericks)

--Chris (invalidname)

I started reading the apple developer documentation for audio units here, but I realized that the docs are very outdated and there appears to be no audio unit template available in xcode 5. I looked for quite some time yesterday and today for an up-to-date guide on building audio units, but to no avail. If anyone has a guide for building audio units in xcode 5 on OSX 10.9 I would be eternally grateful. Even better, if someone knows how to get the old audio unit templates into xcode 5, that would be really helpful.

The best resource I've found is Adamson and Avila's book.

Highly recommended. You may need to tweak the code a bit to work in Xcode 5 but it shouldn't be too much work.

I've been trying to figure out how to implement Core Audio development kit into the current version of Xcode but have had no luck. I can't find where to download the SDK or where it would be located inside Xcode. The tutorial that apple offers only brings you to their website which doesn't have it, they have audio tools and examples but no SDK. Am I missing something? I have been searching my computer to see where any of the source files would be but have had no luck.

You don't need to download anything actually, it comes with XCode.

Import the CoreAudio header where you need it with

#import <CoreAudio/CoreAudio.h>

Then head over to your targets build phases to section "Link Binary With Libraries".

enter image description here

There you need to add CoreAudio and you are done with it.

enter image description here

In case you are new to CoreAudio I can highly recommend the book "Learning Core Audio". It helped me a ton.

I want to create an iOS app that records or takes audio from a file, gives the use the choice to apply a filter and plays said audio back with the filter applied.

I'm however completely unsure about how to approach this project. How do I best modify the sound of an audio file?

Should I go with Core Audio, OpenAL or are there other audio engines out there that make this even easier? Where would be a good place to start learning about stuff like this?

Thanks for any advice you can give me! :)

OpenAL is not designed for this purpose. It is primarily for playing back audio without any effects other than pitch-shifting (which also results in speed up or speed down) and other very basic manipulations of audio. If you want to apply more robust audio filters and effects, Core Audio and the Audio Unit technology is the way to go. There is a good (and one of the few) books released last year on learning Core Audio, but also recently released is the audio engine already mentioned in your answers.