keras-io / english-speaker-accent-recognition-using-transfer-learning

Model description

This model classifies UK & Ireland accents using feature extraction from Yamnet .

Yamnet Model

Yamnet is an audio event classifier trained on the AudioSet dataset to predict audio events from the AudioSet ontology. It is available on TensorFlow Hub. Yamnet accepts a 1-D tensor of audio samples with a sample rate of 16 kHz.
As output, the model returns a 3-tuple:

• Scores of shape (N, 521) representing the scores of the 521 classes.
• Embeddings of shape (N, 1024) .
• The log-mel spectrogram of the entire audio frame.

We will use the embeddings, which are the features extracted from the audio samples, as the input to our dense model.

For more detailed information about Yamnet, please refer to its TensorFlow Hub page.

Dense Model

The dense model that we used consists of:

• An input layer which is embedding output of the Yamnet classifier.
• 4 dense hidden layers and 4 dropout layers.
• An output dense layer.

Training and evaluation data

The dataset used is the Crowdsourced high-quality UK and Ireland English Dialect speech data set which consists of a total of 17,877 high-quality audio wav files.

This dataset includes over 31 hours of recording from 120 vounteers who self-identify as native speakers of Southern England, Midlands, Northern England, Wales, Scotland and Ireland.

For more info, please refer to the above link or to the following paper: Open-source Multi-speaker Corpora of the English Accents in the British Isles

Training procedure

Training hyperparameters

Training Metrics

Model Plot

Validation Results

The model achieved the following results on the validation dataset:

And the confusion matrix for the validation set is:

Credits

Author: Fadi Badine .
Based on the following Keras example English speaker accent recognition using Transfer Learning by Fadi Badine
Check out the demo space here