Skip to content

Repository files navigation

ASR Benchmark – Low-Resource South African Languages

Paper: Algorithm Design and Mathematical Modeling for Efficient Automatic Speech Recognition in Low-Resource African Languages Author: Simanga Comfort Mchunu (47483571@mylife.unisa.ac.za), University of South Africa


Overview

This project benchmarks ASR models on three South African languages:

Language HuggingFace config ISO
isiZulu fleurs/zu_za zu
Setswana fleurs/tn_za tn
Sesotho fleurs/st_za st

Three compression techniques are evaluated:

  1. Low-Rank Factorization – W ≈ U·Vᵀ, reduces parameters O(p·q) → O(r·(p+q))
  2. Knowledge Distillation – teacher → student transfer
  3. Post-training 8-bit Quantization – 4× memory reduction

Project Structure

asr_benchmark/
├── src/
│   ├── benchmark.py        # Main benchmark runner
│   ├── compression.py      # Mathematical compression implementations
│   └── visualize.py        # Result tables (LaTeX + Markdown)
├── results/
│   └── benchmark_results.json   # Auto-generated after run
├── configs/
│   └── default.json             # Configurable run parameters
├── requirements.txt
└── README.md

Quick Start

1. Install dependencies

python -m venv .venv && source .venv/bin/activate   # (recommended)
pip install -r requirements.txt

2. Run benchmark (fast, offline / mock mode)

# Works without GPU or internet – uses synthetic audio
python src/benchmark.py --samples 5

# With a real HuggingFace model (downloads ~300 MB)
python src/benchmark.py \
    --model openai/whisper-tiny \
    --samples 10

# With compression enabled
python src/benchmark.py \
    --model openai/whisper-tiny \
    --samples 10 \
    --low-rank \
    --rank-ratio 0.25 \
    --quantize

3. Generate paper tables

python src/visualize.py

Outputs a Markdown table and a ready-to-paste LaTeX \table{} block.


Command-Line Reference

Flag Default Description
--model facebook/wav2vec2-base HuggingFace model ID
--samples 10 Samples per language
--quantize off Apply 8-bit quantization
--low-rank off Apply LRF (W ≈ UVᵀ)
--rank-ratio 0.25 Fraction of singular values kept
--output results Output directory
--device cpu cpu or cuda

Recommended Models (HuggingFace)

Model Size Notes
openai/whisper-tiny ~75 MB Best for replicating paper baseline
openai/whisper-base ~145 MB Higher accuracy
facebook/wav2vec2-base ~360 MB Requires fine-tuning for SA languages
mbzuai-paris/Whisper-tiny-ZU ~75 MB Fine-tuned isiZulu

Metrics Reported

Metric Definition
WER Word Error Rate (%)
CER Character Error Rate (%)
RTF Real-Time Factor (latency / audio duration; <1 = faster than real-time)
Throughput 1/RTF – multiples of real-time
Model size Estimated MB from parameter count
Compression Ratio vs 85 MB baseline

Mathematical Background

Low-Rank Factorization

W ∈ ℝ^(p×q)
Truncated SVD: W ≈ U_r · diag(σ₁...σ_r) · Vᵀ_r
Parameters: p·q  →  r·(p+q)  where r << min(p,q)

Sample Complexity Bound

m ≥ O(k log n)  samples sufficient for PAC learning
k = vocabulary size,  n = feature dimension

Quantization Perturbation

‖W - W_q‖_F / ‖W‖_F  ≤  ε  (bounded perturbation)
8-bit: 4× memory reduction from fp32

Citation

@misc{mchunu2026asr,
  title  = {Algorithm Design and Mathematical Modeling for Efficient 
             Automatic Speech Recognition in Low-Resource African Languages},
  author = {Mchunu, Simanga Comfort},
  year   = {2026},
  institution = {University of South Africa}
}

About

No description, website, or topics provided.

Resources

Stars

Watchers

Forks

Releases

Packages

Contributors

Languages