Ethiopia had announced plans to withdraw its estimated 3,000 troops from Somalia last November.
Hussein Awale, a Mogadishu resident, told Reuters: "We were chanting praise be to Allah, who made the troops leave our area."
Suleiman Olad Roble, a spokesman of the Alliance for the Re-Liberation of Somalia, said: "Ethiopian troops have left their strategic main bases in Mogadishu and the others will withdraw today."
But Yvonne Ndege, AL Jazeera's correspondent, said the news of the withdrawal was being treated with some degree of scepticism in view of the poor security situation in the country.
"The theory is that Ethiopian troops are rebasing into other parts of the country and the Ethiopian foreign ministry have been very quick to point out that this withdrawal is not something worth to be undertaken with haste," she said.
'Unpopular intervention'
Ethiopian troops arrived in Somalia in 2006 to help the interim government oust rebel fighters from the capital, an intervention that remained deeply unpopular among many Somalis.
Efforts by the African Union (AU) to strengthen a small peacekeeping force has not yielded much results, with member nations not living up to their pledges of making extra deployments.
The ill-equipped and under-funded AU force expected to eventually number 8,000 soldiers currently comprises only 3,400 troops from Uganda and Burundi.
Meanwhile, the United States has been circulating a draft UN Security Council resolution that would call for a UN peacekeeping force in Somalia.
The draft would renew the mandate of AU peacekeepers currently in Somalia, but eventually replace them with a stronger international force.
The draft calls for a council decision by June 1.