Audio support with PulseAudio and AudioFlinger
During the Ubuntu bootstrap for the Touch images, we needed a way to easily use the audio subsystem by interfacing with AudioFlinger (to provide easy access to the proper device audio configuration). As a solution we decided to extend the support for Alsa/PulseAudio done for the Motorola webtop project, which creates an Alsa PCM that connects with AudioFlinger via Binder (by making Pulse just another audio track).
Some issues with this solution:
* Audio latency (we end up having 2 sound servers running at the same time)
* Lack of control of the audio routes using the available APIs (mic, speaker, earpiece, headset)
* Limited amount of mixer control
As PulseAudio is already the main sound server used by the desktop, it'd be desired if it could also be the main sound server for Touch. Then to still be able to reuse the sound related components from Android, we could reuse the AudioTrack API support at libpulse done by Arun Raghavan (for Galaxy Nexus).
Goals with this session:
* Discuss and understand the limitations with both solutions
* Evaluate what would be the effort needed to port PulseAudio to a new Android device
* If going with Pulse is a desired solution, come to a plan to reuse our current solution as a transition (until more devices are available for Pulse to control natively).
Links:
* https:/
* http://
Blueprint information
- Status:
- Started
- Approver:
- Bill Filler
- Priority:
- Undefined
- Drafter:
- Ricardo Salveti
- Direction:
- Approved
- Assignee:
- Ricardo Salveti
- Definition:
- Approved
- Series goal:
- Accepted for saucy
- Implementation:
-
Started
- Milestone target:
-
ubuntu-13.05
- Started by
- Ricardo Salveti
- Completed by
Whiteboard
Notes from the session:
For ease of porting, UbuntuTouch re-uses AudioFlinger & MediaService from AOSP.
We use the AudioTrack and AudioRecord ( same API that AudioFlinger exposes ); PCM audio is routed through this API via the Binder.
Question, can we move to Pulse/Native ALSA support?
Have we measured the latency?
Some devices may not support ALSA directly ( all of the Nexus devices do support ALSA ).
What were the original overall Goals?
* ease of porting to new devices
* ability to use the MediaService
* support of Android applications is *not* an end-goal
AudioTrack API -> libpulse -> PulseAudio -> ALSA working on GalaxyNexus
AudioSystem exposes mixer controls
The audio configuration is usually configured in the HAL, or via an XML description that the HAL reads. There used to be different types of HALs, but the direction now is to have a single HAL that uses XML descriptions. Not whether or not this will catch on...
PulseAudio requires drivers to be in good shape. If AudioFlinger has lesser requirements... we could be in for trouble wrt to hardware enablement.
Would be good to run a testsuite against drivers we want to support; at least checking expected API is there and works to some basic extent
There is an option to write a PulseAudio sink that talks directly to HAL if the device doesn't native ALSA support. This should work, but we'd end up with less fine grained control ( ie. latency adjustment ) than if we had native ALSA.
We think the two possibilities that cover most of the case are:
* native ALSA
* PulseAudio HAL Sink
Audio FX are something that AudioFlinger supports. Something that we should think about for the future.
Question: where does the echo cancellation happen? Most likely this happens on the DSP itself, so not much for us to do.
Question: do we want to abstract the sound API in the platform API ( mixer control, ... ). TBD
Question: do we lose any codec support by moving to Pulse. No, we actually gain the possiblity to DTS or other non-PCM formats by using Pulse.
What about phone calls?
* HAL handles audio
* What happens with system sounds during a call?
* What about Bluetooth audio ( headsets )? Needs more investigation, depends
Audio latency using Pulse/ALSA will be much better than AudioFlinger ( typically ~100ms latency ). This means we shouldn't have a problem with audio skew when video playback is being done via MediaService and audio is routed to Pulse.
Work Items
Work items:
[rsalveti] Get Arun's code to work on Ubuntu Touch ( Galaxy Nexus ): TODO
[diwic] Write guide on how to port the mixer (from xml file to either UCM or PulseAudio profile): DONE
[diwic] make the Native ALSA work for a selective range of devices (galaxy nexus and nexus 4 initially): DONE
[diwic] Investigate the effort needed to have a pulseaudio HAL sink: DONE
Investigate how echo cancellation works on Android (footnote from Arun: this is mostly relevant for VoIP calls. The GSM bits should take care of it themselves): TODO
Dependency tree
* Blueprints in grey have been implemented.
