The fighting began on Tuesday between forces of the Sudanese army and the Sudan People's Liberation Army (SPLA) and spread through the town's market, hospital and other areas.
Edward Lino, a SPLA political adviser, said a number of houses had been burnt down, leaving hundreds of Sudanese displaced.
More than 340 people are still trapped amid infighting in the centre of Abyei, Abdul Rasul Noor, a leader of a local Sudanese tribe, said.
Sudanese government officials said the clashes were because of "personal disputes".
Oil-rich Abyei has become a potential flashpoint that could wreck the peace between the ethnic African south and Sudan's Arab-dominated government.
For more than two decades the two sides fought a civil war, which ended in a 2005 peace agreement.
UN involvement
A UN-headed committee on Thursday met members of the Sudanese armed forces and the SPLA to try to contain the violence.
The officials said that it was difficult to assess the number of casualties because of the tension on the ground.
A ceasefire was reached, but sporadic gunfire was reported after the truce.
The UN said it had evacuated 250 staff members from the region.
Khaled Mansour, a spokesman for the UN mission in Sudan, said: "We believe that their [staff] safety and security was threatened." He said about 400 UN peacekeepers remained in Abyei.
The Abyei clashes come at a particularly sensitive time for the Khartoum government.
Fighters from a separate conflict in the western Sudanese region of Darfur assaulted the outskirts of Khartoum over the weekend in the first attack on the capital in decades.
The attack was repelled, but at least 200 security members, rebel fighters and civilians were killed.