Audio Unit and Writing to file - macos

I'm creating real-time audio sequencer app on OS X.
Real-time synth part is implemented by using AURenderCallback.
Now I'm making function to write rendered result to Wave File (44100Hz 16bit Stereo).
Format for render-callback function is 44100Hz 32bit float Stereo interleaved.
I'm using ExtAudioFileWrite to write to file.
But ExtAudioFileWrite function returns error code 1768846202;
I searched 1768846202 but I couldn't get information.
Would you give me some hints?
Thank you.
Here is code.
outFileFormat.mSampleRate = 44100;
outFileFormat.mFormatID = kAudioFormatLinearPCM;
outFileFormat.mFormatFlags =
kAudioFormatFlagIsSignedInteger | kAudioFormatFlagIsPacked;
outFileFormat.mBitsPerChannel = 16;
outFileFormat.mChannelsPerFrame = 2;
outFileFormat.mFramesPerPacket = 1;
outFileFormat.mBytesPerFrame =
outFileFormat.mBitsPerChannel / 8 * outFileFormat.mChannelsPerFrame;
outFileFormat.mBytesPerPacket =
outFileFormat.mBytesPerFrame * outFileFormat.mFramesPerPacket;
AudioBufferList *ioList;
ioList = (AudioBufferList*)calloc(1, sizeof(AudioBufferList)
+ 2 * sizeof(AudioBuffer));
ioList->mNumberBuffers = 2;
ioList->mBuffers[0].mNumberChannels = 1;
ioList->mBuffers[0].mDataByteSize = allocByteSize / 2;
ioList->mBuffers[0].mData = ioDataL;
ioList->mBuffers[1].mNumberChannels = 1;
ioList->mBuffers[1].mDataByteSize = allocByteSize / 2;
ioList->mBuffers[1].mData = ioDataR;
...
while (1) {
//Fill buffer by using render callback func.
RenderCallback(self, nil, nil, 0, frames, ioList);
//i want to create one sec file.
if (renderedFrames >= 44100) break;
err = ExtAudioFileWrite(outAudioFileRef, frames , ioList);
if (err != noErr){
NSLog(#"ERROR AT WRITING TO FILE");
goto errorExit;
}
}

Some of the error codes are actually four character strings. The Core Audio book provides a nice function to handle errors.
static void CheckError(OSStatus error, const char *operation)
{
if (error == noErr) return;
char str[20];
// see if it appears to be a 4-char-code
*(UInt32 *)(str + 1) = CFSwapInt32HostToBig(error);
if (isprint(str[1]) && isprint(str[2]) && isprint(str[3]) && isprint(str[4])) {
str[0] = str[5] = '\'';
str[6] = '\0';
} else
// no, format it as an integer
sprintf(str, "%d", (int)error);
fprintf(stderr, "Error: %s (%s)\n", operation, str);
exit(1);
}
Use it like this:
CheckError(ExtAudioFileSetProperty(outputFile,
kExtAudioFileProperty_CodecManufacturer,
sizeof(codec),
&codec), "Setting codec.");

Before you can do any sort of debugging, you probably need to figure out what that error message actually means. Have you tried passing that status code to GetMacOSStatusErrorString() or GetMacOSStatusCommentString()? They aren't documented so well, but they are declared in CoreServices/CarbonCore/Debugging.h.

Related

GetRawInputDeviceInfo indicates a buffer size of 1 character for RIDI_DEVICENAME

I'm getting ridiculous behavior from RIDI_DEVICENAME. According to the documentation,
Return value
Type: UINT
If successful, this function returns a non-negative number indicating the number of bytes copied to pData.
If pData is not large enough for the data, the function returns -1. If pData is NULL, the function returns a value of zero. In both of these cases, pcbSize is set to the minimum size required for the pData buffer.
Call GetLastError to identify any other errors.
Ignoring the obvious problem that -1 is not a representable value in the UINT return type, it seems that the function should tell me the required size of the buffer, and if I supply a buffer of this size, the function should either succeed or at least follow its own rules for failure.
However, I'm not seeing this at all. On Windows 10, the Unicode version of the function sets pcbSize to 1 when pData is null and leaves it alone otherwise, failing in all cases. The ANSI version of the function sets pcbSize to 2 when pData is null, and otherwise doubles whatever value was passed in, and still fails.
Headers used for either version of test code:
#define WIN32_EXTRA_LEAN 1
#include <iomanip>
#include <iostream>
#include <string>
#include <vector>
#include <windows.h>
ANSI test code:
std::string GetRawInputDeviceName( HANDLE hRaw )
{
UINT numChars = 0u;
INT validChars;
validChars = static_cast<INT>(::GetRawInputDeviceInfoA(hRaw, RIDI_DEVICENAME, nullptr, &numChars));
auto lasterror = ::GetLastError();
if (lasterror != ERROR_INSUFFICIENT_BUFFER) {
std::wcerr << L"Failed to get length of name of raw input device, retcode = " << validChars << L", last error = " << lasterror << L"\n";
return {};
}
std::string name;
name.resize(numChars);
validChars = static_cast<INT>(::GetRawInputDeviceInfoA(hRaw, RIDI_DEVICENAME, &name[0], &numChars));
lasterror = ::GetLastError();
if (validChars > 0) {
name.resize(validChars);
return name;
}
else {
std::wcerr << L"Failed to get name of raw input device, retcode = " << validChars << L", last error = " << lasterror << L"\n";
return {};
}
}
Unicode test code:
std::wstring GetRawInputDeviceName( HANDLE hRaw )
{
UINT numChars = 0u;
INT validChars;
validChars = static_cast<INT>(::GetRawInputDeviceInfoW(hRaw, RIDI_DEVICENAME, nullptr, &numChars));
auto lasterror = ::GetLastError();
if (lasterror != ERROR_INSUFFICIENT_BUFFER) {
std::wcerr << L"Failed to get length of name of raw input device, retcode = " << validChars << L", last error = " << lasterror << L"\n";
return {};
}
std::wstring name;
name.resize(numChars);
validChars = static_cast<INT>(::GetRawInputDeviceInfoW(hRaw, RIDI_DEVICENAME, &name[0], &numChars));
lasterror = ::GetLastError();
if (validChars > 0) {
name.resize(validChars);
return name;
}
else {
std::wcerr << L"Failed to get name of raw input device, retcode = " << validChars << L", last error = " << lasterror << L"\n";
return {};
}
}
On Windows 10 through RDP I'm getting ERROR_INSUFFICIENT_BUFFER consistently.
On Windows 8.1 running as a local user, I get ERROR_INSUFFICIENT_BUFFER if pData is null, and when I provide a buffer I get back failure ((UINT)-1) and GetLastError() returns zero.
I've also just tried proposing a likely-large-enough buffer size, and got failures as well.
What is going on, what is the right way to get the interface path name, and do I need administrative rights or to call some other APIs first? I don't seem to be having any problems calling GetRawInputDeviceList or using RIDI_DEVICEINFO mode of GetRawInputDeviceInfo... but I need the interface path in order to go further.
Windows HID Device Name Format
https://stackoverflow.com/a/64320052/103167
the GetRawInputDeviceName have several errors in declaration / implementation / documentation
by fact more correct declare return value as signed ( LONG or INT) but not UINT
exist 3 case:
1. function return negative value (or if want -1) : this is error
case, and by design - last error must be set. but really it not
always set (implementation error).
most common errors:
pcbSize or pData point to invalid or read only memory location. usual error in this case ERROR_NOACCESS (translated from
STATUS_ACCESS_VIOLATION)
hDevice not valid handle - ERROR_INVALID_HANDLE is returned
uiCommand not valid RIDI_XXX constant - ERROR_INVALID_PARAMETER
*pcbSize is not large enough for the data - in this case *pcbSize is set to the minimum size required for the pData buffer. ERROR_INSUFFICIENT_BUFFER
again - only in this case (-1) exist sense call GetLastError();
2. function return 0 this possible only in case when pData is NULL.
*pcbSize is set to the minimum size required for the pData buffer.
3. function return positive value ( > 0) this mean that this count of
bytes (in case RIDI_PREPARSEDDATA or RIDI_DEVICEINFO ) or
characters (in case RIDI_DEVICENAME) written to buffer
so documentation is wrong here:
pcbSize
[in, out]
Pointer to a variable that contains the size, in bytes, of the data in
pData.
in case RIDI_DEVICENAME in characters
so already visible very serious problems with design (type of return value - unsigned) and mixed bytes/characters. many different cases.
but then exist critical error in implementation. in begin of function handle hDevice converted to pointer.
PDEVICEINFO pDeviceInfo = HMValidateHandle(hDevice, TYPE_DEVICEINFO);
(if 0 returned - we got -1 on exit with ERROR_INVALID_HANDLE).
in DEVICEINFO exist UNICODE_STRING ustrName - this name and copied to user mode
switch (uiCommand) {
case RIDI_DEVICENAME:
/*
* N.b. UNICODE_STRING counts the length by the BYTE count, not by the character count.
* Our APIs always treat the strings by the character count. Thus, for RIDI_DEVICNAME
* only, cbOutSize holds the character count, not the byte count, in spite of its
* name. Confusing, but cch is the way to be consistent.
*/
cbOutSize = pDeviceInfo->ustrName.Length / sizeof(WCHAR) + 1; // for Null terminator
break;
//...
}
required cbOutSize compared with cbBufferSize = *pcbSize;
and if (cbBufferSize >= cbOutSize) api begin copy operation
exist next code
case RIDI_DEVICENAME:
if (cbOutSize <= 2) { // !!!! error !!!!
retval = -1;
goto leave;
}
RtlCopyMemory(pData, pDeviceInfo->ustrName.Buffer, pDeviceInfo->ustrName.Length);
((WCHAR*)pData)[1] = '\\'; // convert nt prefix ( \??\ ) to win32 ( \\?\ )
((WCHAR*)pData)[cbOutSize - 1] = 0; // make it null terminated
break;
cbOutSize here - is (len + 1) of device name (which we not control). so if name is zero length - always -1 is returned (error #1) but last error not set ( error #2 )
of course exist and error #3 - why is device name is 0 length ? this must not be. but in case terminal service devices - (virtual mouse/ keyboard device created on UMB bus ) - exist this result.
full code for api ( in kernel)
UINT NtUserGetRawInputDeviceInfo(
HANDLE hDevice,
UINT uiCommand,
LPVOID pData,
PUINT pcbSize)
{
UINT cbOutSize = 0;
UINT cbBufferSize;
int retval = 0;
EnterCrit(0, UserMode);
UserAtomicCheck uac;
try {
ProbeForRead(pcbSize, sizeof(UINT), sizeof(DWORD));
cbBufferSize = *pcbSize;
} except (EXCEPTION_EXECUTE_HANDLER) {
UserSetLastError(RtlNtStatusToDosError(GetExceptionCode()));// ERROR_NOACCESS
retval = -1;
goto leave1;
}
EnterDeviceInfoListCrit_();
PDEVICEINFO pDeviceInfo = HMValidateHandle(hDevice, TYPE_DEVICEINFO);
if (pDeviceInfo == NULL) {
UserSetLastError(ERROR_INVALID_HANDLE);
retval = -1;
goto leave;
}
/*
* Compute the size of the output and evaluate the uiCommand.
*/
switch (uiCommand) {
case RIDI_PREPARSEDDATA:
if (pDeviceInfo->type == DEVICE_TYPE_HID) {
cbOutSize = pDeviceInfo->hid.pHidDesc->hidCollectionInfo.DescriptorSize;
} else {
cbOutSize = 0;
}
break;
case RIDI_DEVICENAME:
/*
* N.b. UNICODE_STRING counts the length by the BYTE count, not by the character count.
* Our APIs always treat the strings by the character count. Thus, for RIDI_DEVICNAME
* only, cbOutSize holds the character count, not the byte count, in spite of its
* name. Confusing, but cch is the way to be consistent.
*/
cbOutSize = pDeviceInfo->ustrName.Length / sizeof(WCHAR) + 1; // for Null terminator
break;
case RIDI_DEVICEINFO:
cbOutSize = sizeof(RID_DEVICE_INFO);
break;
default:
UserSetLastError(ERROR_INVALID_PARAMETER);
retval = -1;
goto leave;
}
if (pData == NULL) {
/*
* The app wants to get the required size.
*/
try {
ProbeForWrite(pcbSize, sizeof(UINT), sizeof(DWORD));
*pcbSize = cbOutSize;
} except (EXCEPTION_EXECUTE_HANDLER) {
UserSetLastError(RtlNtStatusToDosError(GetExceptionCode()));// ERROR_NOACCESS
retval = -1;
goto leave;
}
retval = 0;
} else {
if (cbBufferSize >= cbOutSize) {
try {
ProbeForWrite(pData, cbBufferSize, sizeof(DWORD));
switch (uiCommand) {
case RIDI_PREPARSEDDATA:
if (pDeviceInfo->type == DEVICE_TYPE_HID) {
RtlCopyMemory(pData, pDeviceInfo->hid.pHidDesc->pPreparsedData, cbOutSize);
}
break;
case RIDI_DEVICENAME:
if (cbOutSize <= 2) { // !!!!
retval = -1;
goto leave;
}
RtlCopyMemory(pData, pDeviceInfo->ustrName.Buffer, pDeviceInfo->ustrName.Length);
((WCHAR*)pData)[1] = '\\'; // make it null terminated
((WCHAR*)pData)[cbOutSize - 1] = 0; // make it null terminated
break;
case RIDI_DEVICEINFO:
{
PRID_DEVICE_INFO prdi = (PRID_DEVICE_INFO)pData;
ProbeForRead(prdi, sizeof(UINT), sizeof(DWORD));
if (prdi->cbSize != cbOutSize) {
MSGERRORCLEANUP(ERROR_INVALID_PARAMETER);
}
ProbeForWrite(prdi, sizeof(RID_DEVICE_INFO), sizeof(DWORD));
RtlZeroMemory(prdi, sizeof(RID_DEVICE_INFO));
prdi->cbSize = cbOutSize;
switch (pDeviceInfo->type) {
case DEVICE_TYPE_HID:
prdi->dwType = RIM_TYPEHID;
prdi->hid.dwVendorId = pDeviceInfo->hid.pHidDesc->hidCollectionInfo.VendorID;
prdi->hid.dwProductId = pDeviceInfo->hid.pHidDesc->hidCollectionInfo.ProductID;
prdi->hid.dwVersionNumber = pDeviceInfo->hid.pHidDesc->hidCollectionInfo.VersionNumber;
prdi->hid.usUsagePage = pDeviceInfo->hid.pHidDesc->hidpCaps.UsagePage;
prdi->hid.usUsage = pDeviceInfo->hid.pHidDesc->hidpCaps.Usage;
break;
case DEVICE_TYPE_MOUSE:
prdi->dwType = RIM_TYPEMOUSE;
prdi->mouse.dwId = pDeviceInfo->mouse.Attr.MouseIdentifier;
prdi->mouse.dwNumberOfButtons = pDeviceInfo->mouse.Attr.NumberOfButtons;
prdi->mouse.dwSampleRate = pDeviceInfo->mouse.Attr.SampleRate;
break;
case DEVICE_TYPE_KEYBOARD:
prdi->dwType = RIM_TYPEKEYBOARD;
prdi->keyboard.dwType = GET_KEYBOARD_DEVINFO_TYPE(pDeviceInfo);
prdi->keyboard.dwSubType = GET_KEYBOARD_DEVINFO_SUBTYPE(pDeviceInfo);
prdi->keyboard.dwKeyboardMode = pDeviceInfo->keyboard.Attr.KeyboardMode;
prdi->keyboard.dwNumberOfFunctionKeys = pDeviceInfo->keyboard.Attr.NumberOfFunctionKeys;
prdi->keyboard.dwNumberOfIndicators = pDeviceInfo->keyboard.Attr.NumberOfIndicators;
prdi->keyboard.dwNumberOfKeysTotal = pDeviceInfo->keyboard.Attr.NumberOfKeysTotal;
break;
}
}
break;
default:
__assume(false);
}
} except (EXCEPTION_EXECUTE_HANDLER) {
UserSetLastError(RtlNtStatusToDosError(GetExceptionCode()));// ERROR_NOACCESS
retval = -1;
goto leave;
}
retval = cbOutSize;
} else {
/*
* The buffer size is too small.
* Returns error, storing the required size in *pcbSize.
*/
retval = -1;
try {
ProbeForWrite(pcbSize, sizeof(UINT), sizeof(DWORD));
*pcbSize = cbOutSize;
UserSetLastError(ERROR_INSUFFICIENT_BUFFER);
} except (EXCEPTION_EXECUTE_HANDLER) {
UserSetLastError(RtlNtStatusToDosError(GetExceptionCode()));// ERROR_NOACCESS
retval = -1;
goto leave;
}
}
}
leave:
LeaveDeviceInfoListCrit_();
leave1:
UserSessionSwitchLeaveCrit();
return retval;
}
then GetRawInputDeviceInfoA add additional errors compare GetRawInputDeviceInfoW - the value from *pcbSize by some reason multiple on 2. but again - this error in all case.
note that DeviceName (formatted from strings returned from driver on IRP_MN_QUERY_ID have very strict restrictions:
If a driver returns an ID with an illegal character, the system will
bug check. Characters with the following values are illegal in an ID
for this IRP:
Less than or equal to 0x20 (' ')
Greater than 0x7F
Equal to 0x2C (',')
so even after covert unicode to ansi - length of device name will be the same ( all symbols < 0x80 ). so not need *2 buffer size for Ansi version.
then i already view error in your code - you call ::GetLastError(); unconditionally after GetRawInputDeviceInfoW - but returned value have sense only in case api return -1
explain for observed behavior:
for local devices api in general work correct (if no mistakes in our code)
for terminal service devices - was 0 length ustrName. as result if we pass NULL in pData - return value will be
pDeviceInfo->ustrName.Length / sizeof(WCHAR) + 1;
because pDeviceInfo->ustrName.Length == 0 - 1 will be returned inside *pcbSize
in case A version - -by mistake - 2*1==2 will be returned.
but when e pass not NULL in pData - we trap in this
if (cbOutSize <= 2) { // !!!! error !!!!
retval = -1;
goto leave;
}
so you can pass any by size buffer, anyway, because (cbOutSize <= 2) - -1 will be returned and last error not set
possible solution - at first - never use ansi version - GetRawInputDeviceInfoA
use this wrapper function.
ULONG GetRawInputDeviceInfoExW(_In_opt_ HANDLE hDevice,
_In_ UINT uiCommand,
_Inout_updates_bytes_to_opt_(*pcbSize, *pcbSize) LPVOID pData,
_Inout_ PUINT pcbSize)
{
switch (int i = GetRawInputDeviceInfoW(hDevice, uiCommand, pData, pcbSize))
{
case 0:
return ERROR_INSUFFICIENT_BUFFER;
case 1:
return ERROR_INVALID_NAME;
default:
if (0 > i)
{
return GetLastError();
}
*pcbSize = i;
return NOERROR;
}
}
example of usage: (/RTCs must be disabled )
void Demo()
{
PRAWINPUTDEVICELIST pRawInputDeviceList = 0;
UINT uiNumDevices = 0;
UINT cch, cchAllocated = 0;
union {
PVOID buf;
PWSTR name;
};
buf = 0;
while (0 <= (int)GetRawInputDeviceList(pRawInputDeviceList, &uiNumDevices, sizeof(RAWINPUTDEVICELIST)))
{
if (pRawInputDeviceList)
{
do
{
HANDLE hDevice = pRawInputDeviceList->hDevice;
ULONG dwError;
while (ERROR_INSUFFICIENT_BUFFER == (dwError =
GetRawInputDeviceInfoExW(hDevice, RIDI_DEVICENAME, name, &(cch = cchAllocated))))
{
if (cch > cchAllocated)
{
cchAllocated = RtlPointerToOffset(buf = alloca((cch - cchAllocated) * sizeof(WCHAR)),
pRawInputDeviceList) / sizeof(WCHAR);
}
else
{
__debugbreak();
}
}
if (dwError == NOERROR)
{
DbgPrint("[%p, %x %S]\n", hDevice, pRawInputDeviceList->dwType, name);
}
else
{
DbgPrint("error = %u\n", dwError);
}
} while (pRawInputDeviceList++, --uiNumDevices);
break;
}
pRawInputDeviceList = (PRAWINPUTDEVICELIST)alloca(uiNumDevices * sizeof(RAWINPUTDEVICELIST));
}
}
This code is working fine on my PC. Not sure, but it indeed could be RDP issue.
UINT result = ::GetRawInputDeviceInfoW(m_Handle, RIDI_DEVICENAME, nullptr, &size);
if (result == static_cast<UINT>(-1))
{
//PLOG(ERROR) << "GetRawInputDeviceInfo() failed";
return false;
}
DCHECK_EQ(0u, result);
std::wstring buffer(size, 0);
result = ::GetRawInputDeviceInfoW(m_Handle, RIDI_DEVICENAME, buffer.data(), &size);
if (result == static_cast<UINT>(-1))
{
//PLOG(ERROR) << "GetRawInputDeviceInfo() failed";
return false;
}
DCHECK_EQ(size, result);

Replace Bento4 with libav / ffmpeg

We use Bento4 - a really well designed SDK - to demux mp4 files in .mov containers. Decoding is done by an own codec, so only the raw (intraframe) samples are needed. By now this works pretty straightforward
AP4_Track *test_videoTrack = nullptr;
AP4_ByteStream *input = nullptr;
AP4_Result result = AP4_FileByteStream::Create(filename, AP4_FileByteStream::STREAM_MODE_READ, input);
AP4_File m_file (*input, true);
//
// Read movie tracks, and metadata, find the video track
size_t index = 0;
uint32_t m_width = 0, m_height = 0;
auto item = m_file.GetMovie()->GetTracks().FirstItem();
auto track = item->GetData();
if (track->GetType() == AP4_Track::TYPE_VIDEO)
{
m_width = (uint32_t)((double)test_videoTrack->GetWidth() / double(1 << 16));
m_height = (uint32_t)((double)test_videoTrack->GetHeight() / double(1 << 16));
std::string codec("unknown");
auto sd = track->GetSampleDescription(0);
AP4_String c;
if (AP4_SUCCEEDED(sd->GetCodecString(c)))
{
codec = c.GetChars();
}
// Find and instantiate the decoder
AP4_Sample sample;
AP4_DataBuffer sampleData;
test_videoTrack->ReadSample(0, sample, sampleData);
}
For several reasons we would prefer replacing Bento4 with libav/ffmpeg (mainly because we already have in the project and want to reduce dependencies)
How would we ( preferrably in pseudo-code ) replace the Bento4-tasks done above with libav? Please remember that the used codec is not in the ffmpeg library, so we cannot use the standard ffmpeg decoding examples. Opening the media file simply fails. Without decoder we got no size or any other info so far. What we need would
open the media file
get contained tracks (possibly also audio)
get track size / length info
get track samples by index
It turned out to be very easy:
AVFormatContext* inputFile = avformat_alloc_context();
avformat_open_input(&inputFile, filename, nullptr, nullptr);
avformat_find_stream_info(inputFile, nullptr);
//Get just two streams...First Video & First Audio
int videoStreamIndex = -1, audioStreamIndex = -1;
for (int i = 0; i < inputFile->nb_streams; i++)
{
if (inputFile->streams[i]->codec->codec_type == AVMEDIA_TYPE_VIDEO && videoStreamIndex == -1)
{
videoStreamIndex = i;
}
else if (inputFile->streams[i]->codec->codec_type == AVMEDIA_TYPE_AUDIO && audioStreamIndex == -1)
{
audioStreamIndex = i;
}
}
Now test for the correct codec tag
// get codec id
char ct[64] = {0};
static const char* codec_id = "MPAK";
av_get_codec_tag_string( ct, sizeof(ct),inputFile->streams[videoStreamIndex]->codec->codec_tag);
assert(strncmp( ct , codec_id, strlen(codec_id)) == 0)
I did not know that the sizes are set even before a codec is chosen (or even available).
// lookup size
Size2D mediasize(inputFile->streams[videoStreamIndex]->codec->width, inputFile->streams[videoStreamIndex]->codec->height);
Seeking by frame and unpacking (video) is done like this:
AVStream* s = m_file->streams[videoStreamIndex];
int64_t seek_ts = (int64_t(frame_index) * s->r_frame_rate.den * s->time_base.den) / (int64_t(s->r_frame_rate.num) * s->time_base.num);
av_seek_frame(m_hap_file, videoStreamIndex, seek_ts, AVSEEK_FLAG_ANY);
AVPacket pkt;
av_read_frame(inputFile, &pkt);
Now the packet contains a frame ready to unpack with own decoder.

from xcode5 to xcode6-beta6, sounds are now distorted on iOS7

I have a remoteIO application that loads and plays samples on iOS. It works fine when built with xcode5. I use iOS7 as a deployment target.
My application was originally built using the AudioUnitSampleType audio format and the kAudioFormatFlagsCanonical format flags. My sample files are 16 bits/44100Hz/Mono/Caf files.
Now I want to run it on iOS8.
Building my app with its original code on xcode6, the app runs fine on an iOS7 device but it produces no sounds on an iOS8 device.
As AudioUnitSampleType and kAudioFormatFlagsCanonical are deprecated in iOS8, I replaced them, after some researches, with float and kAudioFormatFlagIsFloat | kAudioFormatFlagIsPacked | kAudioFormatFlagIsNonInterleaved.
Now my app runs fine on iOS8 but the sounds are saturated on iOS7.
Has anyone experiences this? Any help ? Thanks, I am stuck here.
Pascal
ps : here is my sample loading method
#define AUDIO_SAMPLE_TYPE float
#define AUDIO_FORMAT_FLAGS kAudioFormatFlagIsFloat | kAudioFormatFlagIsPacked | kAudioFormatFlagIsNonInterleaved
-(void) load:(NSURL*)fileNameURL{
if (frameCount>0){
if (leftChannel!= NULL){
free (leftChannel);
leftChannel = 0;
}
if (rightChannel != NULL){
free (rightChannel);
rightChannel = 0;
}
}
soundFileURLRef=(CFURLRef)fileNameURL;
//----------------------------------------------
// 1.[OPEN AUDIO FILE] and associate it with the extended audio file object.
//----------------------------------------------
ExtAudioFileRef audioFileExtendedObject = 0;
log_if_err(ExtAudioFileOpenURL((CFURLRef)soundFileURLRef,
&audioFileExtendedObject),
#"ExtAudioFileOpenURL failed");
//----------------------------------------------
// 2.[AUDIO FILE LENGTH] Get the audio file's length in frames.
//----------------------------------------------
UInt64 totalFramesInFile = 0;
UInt32 frameLengthPropertySize = sizeof (totalFramesInFile);
log_if_err(ExtAudioFileGetProperty(audioFileExtendedObject,
kExtAudioFileProperty_FileLengthFrames,
&frameLengthPropertySize,
&totalFramesInFile),
#"ExtAudioFileGetProperty (audio file length in frames) failed");
frameCount = totalFramesInFile;
//----------------------------------------------
// 3.[AUDIO FILE FORMAT] Get the audio file's number of channels. Normally CAF.
//----------------------------------------------
AudioStreamBasicDescription fileAudioFormat = {0};
UInt32 formatPropertySize = sizeof (fileAudioFormat);
log_if_err(ExtAudioFileGetProperty(audioFileExtendedObject,
kExtAudioFileProperty_FileDataFormat,
&formatPropertySize,
&fileAudioFormat),
#"ExtAudioFileGetProperty (file audio format) failed");
//----------------------------------------------
// 4.[ALLOCATE AUDIO FILE MEMORY] Allocate memory in the soundFiles instance
// variable to hold the left channel, or mono, audio data
//----------------------------------------------
UInt32 channelCount = fileAudioFormat.mChannelsPerFrame;
// DLog(#"fileNameURL=%# | channelCount=%d",fileNameURL,(int)channelCount);
if (leftChannel != NULL){
free (leftChannel);
leftChannel = 0;
}
leftChannel =(AUDIO_UNIT_SAMPLE_TYPE *) calloc (totalFramesInFile, sizeof(AUDIO_UNIT_SAMPLE_TYPE));
AudioStreamBasicDescription importFormat = {0};
if (2==channelCount) {
isStereo = YES;
if (rightChannel != NULL){
free (rightChannel);
rightChannel = 0;
}
rightChannel = (AUDIO_UNIT_SAMPLE_TYPE *) calloc (totalFramesInFile, sizeof (AUDIO_UNIT_SAMPLE_TYPE));
importFormat = stereoStreamFormat;
} else if (1==channelCount) {
isStereo = NO;
importFormat = monoStreamFormat;
} else {
ExtAudioFileDispose (audioFileExtendedObject);
return;
}
//----------------------------------------------
// 5.[ASSIGN THE MIXER INPUT BUS STREAM DATA FORMAT TO THE AUDIO FILE]
// Assign the appropriate mixer input bus stream data format to the extended audio
// file object. This is the format used for the audio data placed into the audio
// buffer in the SoundStruct data structure, which is in turn used in the
// inputRenderCallback callback function.
//----------------------------------------------
UInt32 importFormatPropertySize = (UInt32) sizeof (importFormat);
log_if_err(ExtAudioFileSetProperty(audioFileExtendedObject,
kExtAudioFileProperty_ClientDataFormat,
importFormatPropertySize,
&importFormat),
#"ExtAudioFileSetProperty (client data format) failed");
//----------------------------------------------
// 6.[SET THE AUDIBUFFER LIST STRUCT] which has two roles:
//
// 1. It gives the ExtAudioFileRead function the configuration it
// needs to correctly provide the data to the buffer.
//
// 2. It points to the soundFiles[soundFile].leftChannel buffer, so
// that audio data obtained from disk using the ExtAudioFileRead function
// goes to that buffer
//
// Allocate memory for the buffer list struct according to the number of
// channels it represents.
//----------------------------------------------
AudioBufferList *bufferList;
bufferList = (AudioBufferList *) malloc(sizeof(AudioBufferList)+sizeof(AudioBuffer)*(channelCount-1));
if (NULL==bufferList){
NSLog(#"*** malloc failure for allocating bufferList memory");
return;
}
//----------------------------------------------
// 7.initialize the mNumberBuffers member
//----------------------------------------------
bufferList->mNumberBuffers = channelCount;
//----------------------------------------------
// 8.initialize the mBuffers member to 0
//----------------------------------------------
AudioBuffer emptyBuffer = {0};
size_t arrayIndex;
for (arrayIndex = 0; arrayIndex < channelCount; arrayIndex++) {
bufferList->mBuffers[arrayIndex] = emptyBuffer;
}
//----------------------------------------------
// 9.set up the AudioBuffer structs in the buffer list
//----------------------------------------------
bufferList->mBuffers[0].mNumberChannels = 1;
bufferList->mBuffers[0].mDataByteSize = totalFramesInFile * sizeof (AUDIO_UNIT_SAMPLE_TYPE);
bufferList->mBuffers[0].mData = leftChannel;
if (channelCount==2){
bufferList->mBuffers[1].mNumberChannels = 1;
bufferList->mBuffers[1].mDataByteSize = totalFramesInFile * sizeof (AUDIO_UNIT_SAMPLE_TYPE);
bufferList->mBuffers[1].mData = rightChannel;
}
//----------------------------------------------
// 10.Perform a synchronous, sequential read of the audio data out of the file and
// into the "soundFiles[soundFile].leftChannel" and (if stereo) ".rightChannel" members.
//----------------------------------------------
UInt32 numberOfPacketsToRead = (UInt32) totalFramesInFile;
OSStatus result = ExtAudioFileRead (audioFileExtendedObject,
&numberOfPacketsToRead,
bufferList);
free (bufferList);
if (noErr != result) {
log_if_err(result,#"ExtAudioFileRead failure");
//
// If reading from the file failed, then free the memory for the sound buffer.
//
free (leftChannel);
leftChannel = 0;
if (2==channelCount) {
free (rightChannel);
rightChannel = 0;
}
frameCount = 0;
}
//----------------------------------------------
// Dispose of the extended audio file object, which also
// closes the associated file.
//----------------------------------------------
ExtAudioFileDispose (audioFileExtendedObject);
return;
}

CoreAudio AudioQueue stop issue

I'm making a CoreAudio based FLAC player, and ran into a naughty issue with AudioQueues.
I'm initializing my stuff like this (variables beginning with an underscore are instance variables):
_flacDecoder = FLAC__stream_decoder_new();
AudioStreamBasicDescription asbd = {
.mFormatID = kAudioFormatLinearPCM,
.mFormatFlags = kAudioFormatFlagIsSignedInteger | kAudioFormatFlagIsPacked,
.mSampleRate = 44100,
.mChannelsPerFrame = 2,
.mBitsPerChannel = 16,
.mBytesPerPacket = 4,
.mFramesPerPacket = 1,
.mBytesPerFrame = 4,
.mReserved = 0
};
AudioQueueNewOutput(&asbd, HandleOutputBuffer, (__bridge void *)(self), CFRunLoopGetCurrent(), kCFRunLoopDefaultMode, 0, &_audioQueue);
for (int i = 0; i < kNumberBuffers; ++i) {
AudioQueueAllocateBuffer(_audioQueue, 0x10000, &_audioQueueBuffers[i]);
}
AudioQueueSetParameter(_audioQueue, kAudioQueueParam_Volume, 1.0);
16 bit stereo PCM at 44.1 kHz, pretty basic setup. kNumberBuffers is 3, and each buffer is 0x10000 bytes.
I populate the buffers with these callbacks:
static void HandleOutputBuffer(void *inUserData, AudioQueueRef inAQ, AudioQueueBufferRef inBuffer){
FLACPlayer * self = (__bridge FLACPlayer*)inUserData;
UInt32 largestBlockSizeInBytes = self->_currentStreamInfo.max_blocksize * self->_currentStreamInfo.channels * self->_currentStreamInfo.bits_per_sample/8;
inBuffer->mAudioDataByteSize = 0;
self->_buffer = inBuffer;
while(inBuffer->mAudioDataByteSize <= inBuffer->mAudioDataBytesCapacity - largestBlockSizeInBytes){
FLAC__bool result = FLAC__stream_decoder_process_single(self->_flacDecoder);
assert(result);
if(FLAC__stream_decoder_get_state(self->_flacDecoder) == FLAC__STREAM_DECODER_END_OF_STREAM){
AudioQueueStop(self->_audioQueue, false);
break;
}
}
AudioQueueEnqueueBuffer(inAQ, inBuffer, 0, NULL);
}
static FLAC__StreamDecoderWriteStatus flacDecoderWriteCallback(const FLAC__StreamDecoder *decoder, const FLAC__Frame *frame, const FLAC__int32 * const buffer[], void *client_data){
FLACPlayer * self = (__bridge FLACPlayer *)client_data;
assert(frame->header.bits_per_sample == 16); // TODO
int16_t * bufferWritePosition = (int16_t*)((uint8_t*)self->_buffer->mAudioData + self->_buffer->mAudioDataByteSize);
for(int i = 0; i < frame->header.blocksize; i++){
for(int j = 0; j < frame->header.channels; j++){
*bufferWritePosition = (int16_t)buffer[j][i];
bufferWritePosition++;
}
}
int totalFramePayloadInBytes = frame->header.channels * frame->header.blocksize * frame->header.bits_per_sample/8;
self->_buffer->mAudioDataByteSize += totalFramePayloadInBytes;
return FLAC__STREAM_DECODER_WRITE_STATUS_CONTINUE;
}
static void flacDecoderMetadataCallback(const FLAC__StreamDecoder *decoder, const FLAC__StreamMetadata *metadata, void *client_data){
FLACPlayer * self = (__bridge FLACPlayer*) client_data;
if(metadata->type == FLAC__METADATA_TYPE_STREAMINFO){
self->_currentStreamInfo = metadata->data.stream_info;
}
}
Basically when the queue requests a new buffer, I fill the buffer from the FLAC__stream_decoder, then I enqueue it. Just like everyone else would do. When libFLAC tells me that I've reached the end of my file, I tell the AudioQueue to stop asynchronously, until it had consumed all the buffers' contents. However, instead of playing through the end, the playback stops a tiny bit before it should. If I remove this line:
AudioQueueStop(self->_audioQueue, false);
everything works fine; the audio plays end-to-end, although my queue keeps running till the end of time. If I change that line to this:
AudioQueueStop(self->_audioQueue, true);
then the playback stops immediately/synchronously, as you'd expect from Apple's documentation:
If you pass true, stopping occurs immediately (that is,
synchronously). If you pass false, the function returns immediately,
but the audio queue does not stop until its queued buffers are played
or recorded (that is, the stop occurs asynchronously). Audio queue
callbacks are invoked as necessary until the queue actually stops.
My questions are:
- am I doing anything wrong?
- how can I play my audio until the end, and shut down the queue appropriately?
Of course, after struggling with this stuff for hours, I've found the solution minutes after posting this question...
The problem was that the AudioQueue doesn't care about buffers enqueued after calling AudioQueueStop(..., false). So now I'm feeding the queue like this, and everything works like charm:
static void HandleOutputBuffer(void *inUserData, AudioQueueRef inAQ, AudioQueueBufferRef inBuffer){
FLACPlayer * self = (__bridge FLACPlayer*)inUserData;
UInt32 largestBlockSizeInBytes = self->_currentStreamInfo.max_blocksize * self->_currentStreamInfo.channels * self->_currentStreamInfo.bits_per_sample/8;
inBuffer->mAudioDataByteSize = 0;
self->_buffer = inBuffer;
bool shouldStop = false;
while(inBuffer->mAudioDataByteSize <= inBuffer->mAudioDataBytesCapacity - largestBlockSizeInBytes){
FLAC__bool result = FLAC__stream_decoder_process_single(self->_flacDecoder);
assert(result);
if(FLAC__stream_decoder_get_state(self->_flacDecoder) == FLAC__STREAM_DECODER_END_OF_STREAM){
shouldStop = true;
break;
}
}
AudioQueueEnqueueBuffer(inAQ, inBuffer, 0, NULL);
if(shouldStop){
AudioQueueStop(self->_audioQueue, false);
}
}

Images saved with D3DXSaveSurfaceToFile will open in Paint, not Photoshop

I'm using D3DXSaveSurfaceToFile to save windowed Direct3D 9 surfaces to PNG, BMP and JPG files. There are no errors returned from the D3DXSaveSurfaceToFile call and all files open fine in Windows Photo Viewer and Paint. But they will not open in a higher end image editing program such as Paint Shop Pro or Photoshop. The error messages from these programs basically say that the file is corrupted. If I open the files in Paint and then save them in the same file format with a different file name, then they'll open fine in the other programs.
This leads me to believe that D3DXSaveSurfaceToFile is writing out non-standard versions of these file formats. Is there some way I can get this function to write out files that can be opened in programs like Photoshop without the intermediate step of resaving the files in Paint? Or is there another function I should be using that does a better job of saving a Direct3D surfaces to an image?
Take a look at the file in a image meta viewer. What does it tell you?
Unfortunately D3DXSaveSurfaceToFile() isn't the most stable (it's also exceptionally slow). Personally I do something like the below code. It works even on Anti-aliased displays by doing an offscreen render to take the screenshot then getting it into a buffer. It also supports only the most common of the pixel formats. Sorry for any errors in it, pulled it out of an app I used to work on.
You can then, in your code and probably in another thread, then convert said 'bitmap' to anything you like using a variety of different code.
void HandleScreenshot(IDirect3DDevice9* device)
{
DWORD tcHandleScreenshot = GetTickCount();
LPDIRECT3DSURFACE9 pd3dsBack = NULL;
LPDIRECT3DSURFACE9 pd3dsTemp = NULL;
// Grab the back buffer into a surface
if ( SUCCEEDED ( device->GetBackBuffer(0, 0, D3DBACKBUFFER_TYPE_MONO, &pd3dsBack) ))
{
D3DSURFACE_DESC desc;
pd3dsBack->GetDesc(&desc);
LPDIRECT3DSURFACE9 pd3dsCopy = NULL;
if (desc.MultiSampleType != D3DMULTISAMPLE_NONE)
{
if (SUCCEEDED(device->CreateRenderTarget(desc.Width, desc.Height, desc.Format, D3DMULTISAMPLE_NONE, 0, FALSE, &pd3dsCopy, NULL)))
{
if (SUCCEEDED(device->StretchRect(pd3dsBack, NULL, pd3dsCopy, NULL, D3DTEXF_NONE)))
{
pd3dsBack->Release();
pd3dsBack = pd3dsCopy;
}
else
{
pd3dsCopy->Release();
}
}
}
if (SUCCEEDED(device->CreateOffscreenPlainSurface(desc.Width, desc.Height, desc.Format, D3DPOOL_SYSTEMMEM, &pd3dsTemp, NULL)))
{
DWORD tmpTimeGRTD = GetTickCount();
if (SUCCEEDED(device->GetRenderTargetData(pd3dsBack, pd3dsTemp)))
{
D3DLOCKED_RECT lockedSrcRect;
if (SUCCEEDED(pd3dsTemp->LockRect(&lockedSrcRect, NULL, D3DLOCK_READONLY | D3DLOCK_NOSYSLOCK | D3DLOCK_NO_DIRTY_UPDATE)))
{
int nSize = desc.Width * desc.Height * 3;
BYTE* pixels = new BYTE[nSize +1];
int iSrcPitch = lockedSrcRect.Pitch;
BYTE* pSrcRow = (BYTE*)lockedSrcRect.pBits;
LPBYTE lpDest = pixels;
LPDWORD lpSrc;
switch (desc.Format)
{
case D3DFMT_A8R8G8B8:
case D3DFMT_X8R8G8B8:
for (int y = desc.Height - 1; y >= 0; y--)
{
lpSrc = reinterpret_cast<LPDWORD>(lockedSrcRect.pBits) + y * desc.Width;
for (unsigned int x = 0; x < desc.Width; x++)
{
*reinterpret_cast<LPDWORD>(lpDest) = *lpSrc;
lpSrc++; // increment source pointer by 1 DWORD
lpDest += 3; // increment destination pointer by 3 bytes
}
}
break;
default:
ZeroMemory(pixels, nSize);
}
pd3dsTemp->UnlockRect();
BITMAPINFOHEADER header;
header.biWidth = desc.Width;
header.biHeight = desc.Height;
header.biSizeImage = nSize;
header.biSize = sizeof(BITMAPINFOHEADER);
header.biPlanes = 1;
header.biBitCount = 3 * 8; // RGB
header.biCompression = 0;
header.biXPelsPerMeter = 0;
header.biYPelsPerMeter = 0;
header.biClrUsed = 0;
header.biClrImportant = 0;
BITMAPFILEHEADER bfh = {0};
bfh.bfType = 0x4d42;
bfh.bfOffBits = sizeof(BITMAPFILEHEADER) + sizeof(BITMAPINFOHEADER);
bfh.bfSize = bfh.bfOffBits + nSize;
unsigned int rough_size = sizeof(BITMAPINFOHEADER) + sizeof(BITMAPFILEHEADER) + nSize;
unsigned char* p = new unsigned char[rough_size]
memcpy(p, &bfh, sizeof(BITMAPFILEHEADER));
p += sizeof(BITMAPFILEHEADER);
memcpy(p, &header, sizeof(BITMAPINFOHEADER));
p += sizeof(BITMAPINFOHEADER);
memcpy(p, pixels, nSize);
delete [] pixels;
/**********************************************/
// p now has a full BMP file, write it out here
}
}
pd3dsTemp->Release();
}
pd3dsBack->Release();
}
}
Turns out that it was a combination of a bug in my code and Paint being more forgiving than Photoshop when it comes to reading files. The bug in my code caused the files to be saved with the wrong extension (i.e. Image.bmp was actually saved using D3DXIFF_JPG). When opening a file that contained a JPG image, but had a BMP extension, Photoshop just failed the file. I guess Paint worked since it ignored the file extension and just decoded the file contents.
Looking at a file in an image meta viewer helped me to see the problem.

Resources