Skip to content

Commit

Permalink
Added support for audio source types (param audioSourceType) and the …
Browse files Browse the repository at this point in the history 
…following constants: audioinput.FORMAT, audioinput.CHANNELS, audioinput.SAMPLERATE and audioinput.AUDIOSOURCE_TYPE. Also squashed some bugs in the native layers for iOS and Android and the js bridge.
  • Loading branch information
@edimuj
edimuj committed Jul 15, 2016
1 parent b200179 commit 1990182
Show file tree
Hide file tree
Showing 15 changed files with 486 additions and 332 deletions.
114 changes: 63 additions & 51 deletions README.md
View file
Original file line number Diff line number Diff line change
@@ -1,51 +1,49 @@
# cordova-plugin-audioinput

This Cordova/PhoneGap plugin enables audio capture from the device microphone, by in near real-time forwarding raw audio data to the web layer of your application.
A typical usage scenario for this plugin would be to use the captured audio as source for a web audio node chain, where it then can be manipulated or analyzed with or without playback through the speakers.
This Cordova plugin enables audio capture from the device microphone, by in (near) real-time forwarding raw audio data to the web layer of your application.
A typical usage scenario for this plugin would be to use the captured microphone audio as a source for a Web audio API based applications.

Since `Navigator.getUserMedia()` still isn't supported by all browsers, this plugin provides similar functionality.
This is especially true for Safari mobile on iOS devices, where the Web Audio API is supported, but currently has no support for `getUserMedia`.

The plugin supports two different methods for capture:

1. Using the `audioinput` object as an Audio Node, which can be connected to your Web Audio node chain.
2. Using events to receive chunks of raw audio data, which then can be processed by your app.
The plugin supports two different methods for microphone capture:

When using the second method, the following `window` events are used:
1. Let the plugin handle the encoding of raw data by using the `audioinput` object as an [AudioNode](https://developer.mozilla.org/en-US/docs/Web/API/AudioNode), which can be connected to your Web audio API node chain.
2. Subscribing to `audioinput` events in order to receive chunks of raw audio data, which then can be processed by your app. Using this method doesn't require Web audio support on the device.

* audioinput
* audioinputerror
## Supported Platforms
* Android
* iOS

## Installation
From the Cordova Plugin Repository:

```
cordova plugin add cordova-plugin-audioinput
```

or by using the GitHub project URL:

```
cordova plugin add https://github.com/edimuj/cordova-plugin-audioinput.git
```

Building with the Intel XDK is also supported. I haven't tested the plugin with PhoneGap build, so feel free to message me if you tried it there.
Building with the Intel XDK is also supported. I haven't tested the plugin with PhoneGap build, so feel free to message me if you tried it with success there.

## Supported Platforms
* Android
* iOS
## Events
When using the event based approach, the plugin sends the following `window` events:

## Basic Usage Example - Audio Node
After the Cordova `deviceready` event has fired:
* `audioinput`
* `audioinputerror`

## Basic Usage Example - AudioNode
After the Cordova `deviceready` event has fired:
```javascript

// Start with default values and let the plugin handle conversion of raw data to web audio and therefore will not send any events.
// Start with default values and let the plugin handle conversion of raw data, and therefore will not send any audioinput events.
// If an audio context is not provided, the plugin will create one for you.
audioinput.start({
streamToWebAudio: true
});

// Then connect the audioinput to the device speakers in order to hear the captured sound. If an audio context is not provided, the plugin will create one for you.
// Connect the audioinput to the device speakers in order to hear the captured sound.
audioinput.connect(audioinput.getAudioContext().destination);

```
Expand All @@ -56,10 +54,10 @@ Use the event based method if you need more control over the capture process.
Subscribe to `audioinput` events: The event will continuously be fired during capture, allowing the application to receive chunks of raw audio data.

You can also subscribe to `audioinputerror` error events as seen in the example below.

```javascript

function onAudioInput( evt ) {
// 'evt.data' is an integer array containing normalized audio data
// 'evt.data' is an integer array containing raw audio data
//
console.log( "Audio data received: " + evt.data.length + " samples" );

Expand All @@ -70,7 +68,7 @@ function onAudioInput( evt ) {
window.addEventListener( "audioinput", onAudioInput, false );

var onAudioInputError = function( error ) {
alert( "onAudioInputError event recieved: " + error );
alert( "onAudioInputError event recieved: " + JSON.stringify(error) );
};

// Listen to audioinputerror events
Expand All @@ -79,49 +77,53 @@ window.addEventListener( "audioinputerror", onAudioInputError, false );
```

After the Cordova `deviceready` event has fired:

```javascript
var captureCfg = {
bufferSize: 8192 // Here we've changed the bufferSize from the default to 8192 bytes
};

// Start capturing audio from the microphone
audioinput.start( captureCfg );
audioinput.start({
bufferSize: 8192 // Here we've changed the bufferSize from the default to 8192 bytes
});

// Stop capturing audio input
audioinput.stop()

```

## Demo
The `demo` folder contains examples showing both basic and advanced (events) usage, where the captured microphone audio data is used to playback the audio to the device speaker using the Web Audio API.
Remember that unfiltered microphone output likely will create a nasty audio feedback loop so lower the volume before trying them out!
## Demos
The `demo` folder contains some usage examples.

## API
**Start capturing audio** from the microphone:
Remember that unfiltered microphone output likely will create a nasty audio feedback loop so lower the volume before trying out the demos!

basicdemo - How to use the audioinput object as a Web Audio API AudioNode that can be connected to your own chain of AudioNodes.
advanceddemo - How to subscribe to the audioinput events to get and handle chunks of raw audio data.
wav-demo - How to encode recorded data to WAV format and use the resulting file as a source for Audio elements.

## API
**Start capturing audio** from the microphone.
If your app doesn't have recording permission on the users device, the plugin will ask for permission when start is called. And the new Android 6.0 runtime permissions are also supported.
```javascript
audioinput.start( captureCfg );
```

Where `captureCfg` can either be empty, null or contain any of the following parameters.
Please note that not all audio configuration combinations are supported by all devices, and that all of them have default values:

Where `captureCfg` can either be empty, null or contain/override any of the following parameters and their default values.
Please note that not all audio configuration combinations are supported by all devices, the default settings seems to work on most devices though:
```javascript
var captureCfg = {

// The Sample Rate in Hz.
sampleRate: 44100,
// For convenience, use the audioinput.SAMPLERATE constants to set this parameter.
sampleRate: audioinput.SAMPLERATE.CD_AUDIO_44100Hz,

// Maximum size in bytes of the capture buffer.
bufferSize: 16384,

// The number of channels to use: Mono (1) or Stereo (2).
channels: 1,
// For convenience, use the audioinput.CHANNELS constants to set this parameter.
channels: audioinput.CHANNELS.MONO,

// The audio format. Currently PCM_16BIT and PCM_8BIT are supported.
// For convenience, use the audioinput.FORMAT constant to access the possible formats that the plugin supports. For example:
// format: audioinput.FORMAT.PCM_16BIT
format: 'PCM_16BIT',
// For convenience, use the audioinput.FORMAT constant to access the possible formats that the plugin supports.
format: audioinput.FORMAT.PCM_16BIT,

// Specifies if the audio data should be normalized or not.
normalize: true,
Expand All @@ -138,47 +140,57 @@ var captureCfg = {

// Defines how many chunks will be merged each time, a low value means lower latency but requires more CPU resources.
concatenateMaxChunks: 10

/*
Specifies the type of the type of source audio your app requires.
For convenience, use the audioinput.AUDIOSOURCE_TYPE constants to set this parameter.
audioinput.AUDIOSOURCE_TYPE.DEFAULT
audioinput.AUDIOSOURCE_TYPE.CAMCORDER - Microphone audio source with same orientation as camera if available.
audioinput.AUDIOSOURCE_TYPE.MIC - Microphone audio source. (Android only)
audioinput.AUDIOSOURCE_TYPE.UNPROCESSED - Unprocessed sound if available.
audioinput.AUDIOSOURCE_TYPE.VOICE_COMMUNICATION - Tuned for voice communications such as VoIP.
audioinput.AUDIOSOURCE_TYPE.VOICE_RECOGNITION - Tuned for voice recognition if available (Android only)
*/
audioSourceType: audioinput.AUDIOSOURCE_TYPE.DEFAULT

};

```

**Stop capturing audio** from the microphone:

```javascript
audioinput.stop();
```

**Check if the plugin is capturing**, i.e. if it is started or not:

```javascript
audioinput.isCapturing(); // Returns true if it is started
```

**Get the current configuration** from the plugin:

```javascript
audioinput.getCfg();
```

When using the `streamToWebAudio` option, you can **connect the plugin** to your own web audio node chain:

When using the `streamToWebAudio` option, you can **connect the plugin** to your own Web audio node chain:
```javascript
audioinput.connect( audioNode );
```

When using `streamToWebAudio` you can **disconnect the previously connected plugin** from your your own web audio node chain:

When using `streamToWebAudio` you can **disconnect the previously connected plugin** from your your own Web audio node chain:
```javascript
audioinput.disconnect();
```

When using `streamToWebAudio`, and have not supplied the plugin with an audio context, the following method is used to **get the internally created audio context**:

When using `streamToWebAudio`, and have not supplied the plugin with an Audio context, the following method is used to **get the internally created Web Audio context**:
```javascript
audioinput.getAudioContext();
```

## Contributing
This project is open-source, so contributions are welcome. Just ensure that your changes doesn't break backward compatibility.
This project is open-source, so contributions are welcome. Just ensure that your changes doesn't break backward compatibility!

1. Fork the project.
2. Create your feature branch (git checkout -b my-new-feature).
Expand Down
11 changes: 11 additions & 0 deletions demo/advanceddemo.html
View file
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@ <h3>Cordova AudioInput Plugin<br>
<div id="infoMessage">Not ready</div>
<br>
<form class="w3-container">

<label for="sampleRate">SampleRate</label>
<input id="sampleRate" type="number" value="44100" style="width: 5em;" size="5"/> Hz
<br>
Expand All @@ -31,6 +32,16 @@ <h3>Cordova AudioInput Plugin<br>
<label for="format8">Format</label>
<input type="radio" id="format8" name="format" value="PCM_8BIT"> PCM 8BIT
<input type="radio" id="format16" name="format" value="PCM_16BIT" checked> PCM 16BIT<br>
<label for="audioSource">Audio Source</label>
<select id="audioSource">
<option value="0">DEFAULT (Android/iOS)</option>
<option value="5">CAMCORDER (Android/iOS)</option>
<option value="1">MIC (Android)</option>
<option value="9">UNPROCESSED (Android/iOS)</option>
<option value="7">VOICE_COMMUNICATION (Android/iOS)</option>
<option value="6">VOICE_RECOGNITION (Android)</option>
</select>
<br>
<br>
<a class="w3-btn w3-green" href="#" id="startCapture">Start Capture</a>
<a class="w3-btn w3-red" href="#" id="stopCapture">Stop Capture</a>
Expand Down
Loading

0 comments on commit 1990182

Please sign in to comment.