We investigated the genetic diversity and historic relationships among southern African sheep as well as the relationships between them and sheep outside the continent by sourcing both archaeological and modern sheep samples. Archaeological sheep samples derived from the site Die Kelders 1, near Cape Town, date to approximately 1500 years ago. The modern samples were taken as ear snips from Damara, Namaqua Afrikaner, and Ronderib Afrikaner sheep on a farm in Prieska in the Northern Cape. Illumina sequencing libraries were constructed for both ancient and modern specimens. Ancient specimens were enriched for the mitochondrial genome using an in-solution hybridization protocol and modern specimens were subjected to shotgun sequencing. Sequences were mapped to the Ovis aries reference genome, assigned to haplogroups and subhaplogroups, and used to calculate a phylogenetic tree using previously published, geographically dispersed mitochondrial genome sheep sequences. Genetic diversity statistics show that southern African sheep have lower diversity than sheep in other regions. Phylogenetic analysis reveals that many modern southern African sheep are likely descended from prehistoric indigenous sheep populations and not from sheep imported from Europe during the historic period.