Both have been held for more than a decade and have never been charged.
Ghana has given permission for the men to stay for two years subject to security clearances, Foreign Minister Hanna Tetteh said.
The West African nation has not previously taken any Guantanamo prisoners.
The jail was set up following the 11 September 2001 attacks in the US to detain what Washington called "enemy combatants".
US President Barack Obama has said he wants to close the jail down before he leaves office in 2017.
'Bright young man'
According to military records, Mahmoud Omar Bin Atef is a Yemeni citizen born in Saudi Arabia in 1979 and was handed over to the US following his capture by Northern Alliance Forces in Afghanistan in 2001.
His US lawyer, George Clark, described him as a "smart, bright young man" and said his client intended to start a family and find employment.
US official documents say Khalid al-Dhuby was born in Saudi Arabia in 1981 and travelled to Afghanistan to fight. They say he was assessed as "a probable member" of al-Qaeda.
Dozens of countries have received former Guantanamo Bay detainees, including other African states such as Uganda and Cape Verde.
A total of 780 men have been held at Guantanamo since 2002, the vast majority without charge or criminal trial.
The US navy base now has 105 detainees, nearly 50 of whom have been cleared for release.