The pardon will see the release of all female prisoners except those serving life terms, as well as all disabled and juvenile prisoners.
Terminally-ill prisoners and those above the age of 60 who have served a third of their time will also be freed.
“The president in terms of the constitution of Zimbabwe has granted amnesty to prisoners,” the Zimbabwe Prisons and Correctional Service (ZPCS) said in a statement.
“The exercise has… gone a long way in decongesting our prisons.”
About 3,000 prisoners will be released, the ZPCS said, bringing the prison population down to 17,000.
The statement also announced that prisoners on death row for more than 10 years will not face the death penalty, but will spend life in jail.
The country last executed a prisoner in 2005, and Mnangagwa is a strong opponent of the death penalty.
Prisoners convicted of murder, treason, rape, carjacking, armed robbery and those serving a sentence imposed by a court martial were excluded from the amnesty.
Zimbabwe pardoned thousands of prisoners in recent years under Mugabe.
The cash-strapped government has battled to feed, provide clothing and adequate health care to inmates. In 2013 over 100 prisoners died in the country’s jails.
In 2015, four inmates were killed after prisoners went on riot over poor living conditions.
Mugabe, 94, who ruled for 37 years, was forced to resign after a brief military takeover and the threat of impeachment by once loyal ZANU-PF lawmakers.
Download our app and read this and other great stories on the move. Available for Android and iOS.
