When Vambo AI, a Johannesburg-based artificial intelligence (AI) model builder, set out to build MORENA, a 1.5-billion-parameter model covering 12 African languages, it had access to a supercomputer through a United Nations Development Programme (UNDP)-backed programme. It had the graphics processing units (GPUs), specialised chips that power AI computing, but struggled to find enough real African-language text.
“Finding that much real data was close to impossible, so a lot of it was synthetic,” Isheanesu Misi, Vambo’s co-founder and chief technology officer (CTO), told TechCabal in an interview on Monday.
The shortage of African-language text data threatens to limit the development of AI tools that reflect the continent’s linguistic diversity. Without enough high-quality data, developers risk building AI products that perform poorly in local languages, leaving millions of potential users underserved.
MORENA was trained on 251.7 billion tokens, including 65.6 billion African-language tokens, according to its model documentation. The model covers languages such as ChiShona, Kiswahili, Hausa, Yorùbá, Igbo, isiZulu, isiXhosa, Kinyarwanda, Setswana, Afrikaans, isiNdebele and Nigerian Pidgin, alongside English, French and code.
Originally published by TechCabal.