Android原生回音消除API分析(一)音效支持判断分析

AcousticEchoCanceler
谷歌文档对其进行的描述为：
回声消除器 (AEC) 是一种音频预处理器，可从捕获的音频信号中去除从远程方接收到的信号的影响。

AEC 用于语音通信应用程序（语音聊天、视频会议、SIP 呼叫），其中从远程方接收到的信号中存在显着延迟的回声非常令人不安。AEC 通常与噪声抑制 (NS) 结合使用。
应用程序创建一个 AcousticEchoCanceler 对象来实例化和控制音频捕获路径中的 AEC 引擎。
要将 AcousticEchoCanceler 附加到特定android.media.AudioRecord，请在创建 AcousticEchoCanceler 时指定此 AudioRecord 的音频会话 ID。通过调用android.media.AudioRecord#getAudioSessionId()AudioRecord 实例来检索音频会话。

在某些设备上，平台可以根据android.media.MediaRecorder.AudioSource使用情况在捕获路径中默认插入 AEC 。应用程序应在创建 AEC 后调用 AcousticEchoCanceler.getEnable() 以检查特定 AudioRecord 会话上的默认 AEC 激活状态。
通过阅读文档可以得知，应用层使用AcousticEchoCanceler流程大致如下

...
//我们可以通过以下渠道获取AudioSessionId
int audioSession = ((AudioRecord)obj).getAudioSessionId();
int audioSession =((VideoView)obj).getAudioSessionId();
int audioSession = ((AudioTrack)obj).getAudioSessionId();
int audioSession = ((MediaPlayer)obj).getAudioSessionId();
//获取AcousticEchoCanceler对象
AcousticEchoCanceler canceler = AcousticEchoCanceler.create(audioSession);

if(canceler.isAvailable()){//判断是否支持回音消除
//开启回音消除
canceler.setEnabled(true);
//获取回音消除状态
Log.d("swlog","AEC isEnabled:"+canceler.getEnabled());
//关闭回音消除
canceler.setEnabled(false);
//释放回音消除
canceler.release();
}
...

AcousticEchoCanceler源码位置位于:
/frameworks/base/media/java/android/media/audiofx/AcousticEchoCanceler.java

public class AcousticEchoCanceler extends AudioEffect {
    //检查设备是否执行回声消除
    public static boolean isAvailable() {
        return AudioEffect.isEffectTypeAvailable(AudioEffect.EFFECT_TYPE_AEC);
    }

    /**
     * 创建AcousticEchoCanceler并将其附加到指定音频会话上的录音。
     * @param audioSession系统范围内唯一的音频会话标识符。
     *AcousticeCoocanceler将应用于具有相同音频会话的音频记录。
     * @return 如果设备未实现AEC，则返回null。
     */
    public static AcousticEchoCanceler create(int audioSession) {
        AcousticEchoCanceler aec = null;
        try {
            aec = new AcousticEchoCanceler(audioSession);
        }catch (Exception e) {
            Log.w(TAG, "not enough memory");
        }
        return aec;
    }
    private AcousticEchoCanceler(int audioSession)
            throws IllegalArgumentException, UnsupportedOperationException, RuntimeException {
        super(EFFECT_TYPE_AEC, EFFECT_TYPE_NULL, 0, audioSession);
    }
}

通过源码可以得知AcousticEchoCanceler继承于AudioEffect，所有的方法也是通过AudioEffect来实现，因此我们需要对AudioEffect源码进行阅读。

Android对其描述为：
AudioEffect是android音频框架提供的控制音频效果的基类。

应用程序不应直接使用 AudioEffect 类，而应使用其派生类之一来控制特定效果：
android.media.audiofx.Equalizer
android.media.audiofx.Virtualizer
android.media.audiofx.BassBoost
android.media.audiofx.PresetReverb
android.media.audiofx.EnvironmentalReverb
android.media.audiofx.DynamicsProcessing
android.media.audiofx.HapticGenerator
要将音频效果应用于特定的 AudioTrack 或 MediaPlayer 实例，应用程序必须在创建 AudioEffect 时指定该实例的音频会话 ID。
注意：不推荐使用session 0 将插入效果（均衡器、低音增强、虚拟器）附加到全局音频输出混合。
如果指定音频会话中不存在相同效果类型的实例，则创建 AudioEffect 对象将在音频框架中创建相应的效果引擎。如果存在，将使用此实例。
根据优先级参数，创建 AudioEffect 对象（或派生类）的应用程序将接收或不接收效果引擎的控制权。如果优先级高于当前效果引擎所有者使用的优先级，则控制将转移到新对象。否则，控制权将保留在前一个对象上。在这种情况下，新应用程序将通过适当的侦听器通知有效引擎状态或控制所有权的更改。
该类方法比较多，我们先进行抽丝剥茧研究AudioEffect.isEffectTypeAvailable(AudioEffect.EFFECT_TYPE_AEC);方法。

    public static boolean isEffectTypeAvailable(UUID type) {
        AudioEffect.Descriptor[] desc = AudioEffect.queryEffects();
        if (desc == null) {
            return false;
        }

        for (int i = 0; i < desc.length; i++) {
            if (desc[i].type.equals(type)) {
                return true;
            }
        }
        return false;
    }

指向了AudioEffect.queryEffects()方法。

    static public Descriptor[] queryEffects() {
        return (Descriptor[]) native_query_effects();
    }

指向了jni方法。
改os的加载方式为

    static {
        System.loadLibrary("audioeffect_jni");
        native_init();
    }

java层最终指向了private static native Object[] native_query_effects();。
其中源码实现位置为:
/frameworks/base/media/jni/audioeffect/android_media_AudioEffect.cpp
通过native_query_effects通过AndroidRuntime::registerNativeMethods进行显式native注册。其中对应到的函数为{"native_query_effects", "()[Ljava/lang/Object;", (void *)android_media_AudioEffect_native_queryEffects},因此我们阅读android_media_AudioEffect_native_queryEffects函数即可。
对于该方法存在较多的Java对象构建方法，因此我们只需要关心数据获取相关方法即可

    if (AudioEffect::queryNumberEffects(&totalEffectsCount) != NO_ERROR) {
        return NULL;
    }

//构建返回数据类型等
......
effect_descriptor_t desc;
//通过迭代对desc进行赋值
for (i = 0; i < totalEffectsCount; i++) {
        if (AudioEffect::queryEffect(i, &desc) != NO_ERROR) {
            goto queryEffects_failure;
        }
    //数据转换添加到返回值
    .......
    }

因此我们只需要对AudioEffect::queryNumberEffects和AudioEffect::queryEffect阅读即可。通过#include "media/AudioEffect.h"可以得知AudioEffect应该位于media库。
对应的源码位置为：
/frameworks/av/media/libmedia/AudioEffect.cpp

status_t AudioEffect::queryNumberEffects(uint32_t *numEffects)
{
    const sp<IAudioFlinger>& af = AudioSystem::get_audio_flinger();
    if (af == 0) return PERMISSION_DENIED;
    return af->queryNumberEffects(numEffects);
}

status_t AudioEffect::queryEffect(uint32_t index, effect_descriptor_t *descriptor)
{
    const sp<IAudioFlinger>& af = AudioSystem::get_audio_flinger();
    if (af == 0) return PERMISSION_DENIED;
    return af->queryEffect(index, descriptor);
}

其中均指向了AudioSystem::get_audio_flinger()方法。我们来看一下audio_flinger在谷歌文档中的定义：
AudioFlinger：音频策略的执行者，负责输入输出流设备的管理及音频流数据的处理传输。
通过阅读
/frameworks/av/media/libmedia/AudioSystem.cpp

const sp<IAudioFlinger> AudioSystem::get_audio_flinger()
{
    sp<IAudioFlinger> af;
    sp<AudioFlingerClient> afc;
    {
        Mutex::Autolock _l(gLock);
        if (gAudioFlinger == 0) {
            sp<IServiceManager> sm = defaultServiceManager();
            sp<IBinder> binder;
            do {
                binder = sm->getService(String16("media.audio_flinger"));
                if (binder != 0)
                    break;
                usleep(500000); // 0.5 s
            } while (true);
            if (gAudioFlingerClient == NULL) {
                gAudioFlingerClient = new AudioFlingerClient();
            } else {
                if (gAudioErrorCallback) {
                    gAudioErrorCallback(NO_ERROR);
                }
            }
            binder->linkToDeath(gAudioFlingerClient);
            gAudioFlinger = interface_cast<IAudioFlinger>(binder);
            LOG_ALWAYS_FATAL_IF(gAudioFlinger == 0);
            afc = gAudioFlingerClient;
        }
        af = gAudioFlinger;
    }
    if (afc != 0) {
        af->registerClient(afc);
    }
    return af;
}

我们可以得知通过C/S架构从Client去调用Service端的实现，因此我们需要依据 binder = sm->getService(String16("media.audio_flinger"));去查看AudioFlingerService端的实现。
源码位置：
/frameworks/av/services/audioflinger/AudioFlinger.cpp

status_t AudioFlinger::queryNumberEffects(uint32_t *numEffects) const
{
    Mutex::Autolock _l(mLock);
    return EffectQueryNumberEffects(numEffects);
}

于是我们定位到
/frameworks/av/media/libeffects/factory/EffectsFactory.c

int EffectQueryNumberEffects(uint32_t *pNumEffects)
{
    int ret = init();
    //一些不太相关的操作
    .....
    if (ignoreFxConfFiles) {
        ALOGI("Audio effects in configuration files will be ignored");
    } else {
        if (access(AUDIO_EFFECT_VENDOR_CONFIG_FILE, R_OK) == 0) {
            loadEffectConfigFile(AUDIO_EFFECT_VENDOR_CONFIG_FILE);
        } else if (access(AUDIO_EFFECT_DEFAULT_CONFIG_FILE, R_OK) == 0) {
            loadEffectConfigFile(AUDIO_EFFECT_DEFAULT_CONFIG_FILE);
        }
    }
    *pNumEffects = gNumEffects;
    return ret;
}

该代码的意义为先进行初始化，初始化完成后讲全局变量gNumEffects赋值给指针，ret为状态码，*pNumEffects为数量指针。init方法中调用了updateNumEffects()函数对gNumEffects变量进行了赋值。

static list_elem_t *gLibraryList; // list of lib_entry_t: all currently loaded libraries
uint32_t updateNumEffects() {
    list_elem_t *e;
    uint32_t cnt = 0;
    resetEffectEnumeration();
    e = gLibraryList;
    while (e) {
        lib_entry_t *l = (lib_entry_t *)e->object;
        list_elem_t *efx = l->effects;
        while (efx) {
            cnt++;
            efx = efx->next;
        }
        e = e->next;
    }
    gNumEffects = cnt;
    gCanQueryEffect = 0;
    return cnt;
}

从源码中可以看得出来cnt由gLibraryList决定，gLibraryList在loadLibrary进行初始化和赋值,最终我们定位到了init方法中的loadEffectConfigFile方法，可以看得出来是通过加载配置文件形式进行初始化。此时我应该查询AUDIO_EFFECT_VENDOR_CONFIG_FILE或者AUDIO_EFFECT_DEFAULT_CONFIG_FILE变量即可。
最终找到常量定义
./system/media/audio_effects/include/audio_effects/audio_effects_conf.h

#define AUDIO_EFFECT_DEFAULT_CONFIG_FILE "/system/etc/audio_effects.conf"
#define AUDIO_EFFECT_VENDOR_CONFIG_FILE "/vendor/etc/audio_effects.conf"

对应到的文件如下
./frameworks/av/media/libeffects/data/audio_effects.conf

# List of effect libraries to load. Each library element must contain a "path" element
# giving the full path of the library .so file.
#    libraries {
#        <lib name> {
#          path <lib path>
#        }
#    }
libraries {
# This is a proxy library that will be an abstraction for
# the HW and SW effects

  #proxy {
    #path /system/lib/soundfx/libeffectproxy.so
  #}

# This is the SW implementation library of the effect
  #libSW {
    #path /system/lib/soundfx/libswwrapper.so
  #}

# This is the HW implementation library for the effect
  #libHW {
    #path /system/lib/soundfx/libhwwrapper.so
  #}

  bundle {
    path /system/lib/soundfx/libbundlewrapper.so
  }
  reverb {
    path /system/lib/soundfx/libreverbwrapper.so
  }
  visualizer {
    path /system/lib/soundfx/libvisualizer.so
  }
  downmix {
    path /system/lib/soundfx/libdownmix.so
  }
  loudness_enhancer {
    path /system/lib/soundfx/libldnhncr.so
  }
}

# Default pre-processing library. Add to audio_effect.conf "libraries" section if
# audio HAL implements support for default software audio pre-processing effects
#
#  pre_processing {
#    path /system/lib/soundfx/libaudiopreprocessing.so
#  }

# list of effects to load. Each effect element must contain a "library" and a "uuid" element.
# The value of the "library" element must correspond to the name of one library element in the
# "libraries" element.
# The name of the effect element is indicative, only the value of the "uuid" element
# designates the effect.
# The uuid is the implementation specific UUID as specified by the effect vendor. This is not the
# generic effect type UUID.
#    effects {
#        <fx name> {
#            library <lib name>
#            uuid <effect uuid>
#        }
#        ...
#    }

effects {

# additions for the proxy implementation
# Proxy implementation
  #effectname {
    #library proxy
    #uuid  xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx

    # SW implemetation of the effect. Added as a node under the proxy to
    # indicate this as a sub effect.
      #libsw {
         #library libSW
         #uuid  yyyyyyyy-yyyy-yyyy-yyyy-yyyyyyyyyyyy
      #} End of SW effect

    # HW implementation of the effect. Added as a node under the proxy to
    # indicate this as a sub effect.
      #libhw {
         #library libHW
         #uuid  zzzzzzzz-zzzz-zzzz-zzzz-zzzzzzzzzzzz
      #}End of HW effect
  #} End of effect proxy

  bassboost {
    library bundle
    uuid 8631f300-72e2-11df-b57e-0002a5d5c51b
  }
  virtualizer {
    library bundle
    uuid 1d4033c0-8557-11df-9f2d-0002a5d5c51b
  }
  equalizer {
    library bundle
    uuid ce772f20-847d-11df-bb17-0002a5d5c51b
  }
  volume {
    library bundle
    uuid 119341a0-8469-11df-81f9-0002a5d5c51b
  }
  reverb_env_aux {
    library reverb
    uuid 4a387fc0-8ab3-11df-8bad-0002a5d5c51b
  }
  reverb_env_ins {
    library reverb
    uuid c7a511a0-a3bb-11df-860e-0002a5d5c51b
  }
  reverb_pre_aux {
    library reverb
    uuid f29a1400-a3bb-11df-8ddc-0002a5d5c51b
  }
  reverb_pre_ins {
    library reverb
    uuid 172cdf00-a3bc-11df-a72f-0002a5d5c51b
  }
  visualizer {
    library visualizer
    uuid d069d9e0-8329-11df-9168-0002a5d5c51b
  }
  downmix {
    library downmix
    uuid 93f04452-e4fe-41cc-91f9-e475b6d1d69f
  }
  loudness_enhancer {
    library loudness_enhancer
    uuid fa415329-2034-4bea-b5dc-5b381c8d1e2c
  }
}

# Default pre-processing effects. Add to audio_effect.conf "effects" section if
# audio HAL implements support for them.
#
#  agc {
#    library pre_processing
#    uuid aa8130e0-66fc-11e0-bad0-0002a5d5c51b
#  }
#  aec {
#    library pre_processing
#    uuid bb392ec0-8d4d-11e0-a896-0002a5d5c51b
#  }
#  ns {
#    library pre_processing
#    uuid c06c8400-8e06-11e0-9cb6-0002a5d5c51b
#  }

# Audio preprocessor configurations.
# The pre processor configuration consists in a list of elements each describing
# pre processor settings for a given input source. Valid input source names are:
# "mic", "camcorder", "voice_recognition", "voice_communication"
# Each input source element contains a list of effects elements. The name of the effect
# element must be the name of one of the effects in the "effects" list of the file.
# Each effect element may optionally contain a list of parameters and their
# default value to apply when the pre processor effect is created.
# A parameter is defined by a "param" element and a "value" element. Each of these elements
# consists in one or more elements specifying a type followed by a value.
# The types defined are: "int", "short", "float", "bool" and "string"
# When both "param" and "value" are a single int, a simple form is allowed where just
# the param and value pair is present in the parameter description
#    pre_processing {
#        <input source name> {
#            <fx name> {
#                <param 1 name> {
#                    param {
#                        int|short|float|bool|string <value>
#                        [ int|short|float|bool|string <value> ]
#                        ...
#                    }
#                    value {
#                        int|short|float|bool|string <value>
#                        [ int|short|float|bool|string <value> ]
#                        ...
#                    }
#                }
#                <param 2 name > {<param> <value>}
#                ...
#            }
#            ...
#        }
#        ...
#    }

#
# TODO: add default audio pre processor configurations after debug and tuning phase
#

依据/frameworks/base/media/java/android/media/audiofx/AudioEffect.java

    /**
     * UUID for Acoustic Echo Canceler (AEC)
     */
    public static final UUID EFFECT_TYPE_AEC = UUID
            .fromString("7b491460-8d4d-11e0-bd61-0002a5d5c51b");

很遗憾，貌似当前方案貌似不支持AEC，需要自己进行实现。