"The relationship has been moving in a good direction for a number of years now and I think tonight does mark a new phase," Rice said after meeting Gaddafi.
"It is only a start, but I think, after many, many years, I think it's a very good thing that the United States and Libya are establishing a way forward."
Rice travelled to Tunisia's capital, Tunis, on Saturday following the landmark meeting with Gaddafi.
Military ties
An AFP news agency correspondent said she arrived in Tunis on Saturday - her second stop on a North African tour that will also take her to Algeria and Morocco.
The United States has key military ties with Tunisia and is also seeking to conclude a free trade accord with the Maghreb nation.
Rice, the first US top diplomat to visit Tunis since Colin Powell in 2003, was greeted at the El Aouina military base near Tunis by Abdelwaheb Abdallah, the Tunisian foreign minister.
Rice's meeting in Libya was held in Gaddafi's compound, which includes his former home, kept in ruins since it was bombed by US jets in 1986 - an attack which killed about 40 people, including one of Gaddafi's daughters.
Rice's trip to Libya is the first by a US secretary of state in 55 years. Rice is the most senior US diplomat to visit Libya since 1953.
Rice stressed that talks with Gaddafi had been wide ranging, encompassing the possibility of greater investment in Libya by American companies and increasing the number of Libyan students studying in the US.
But the secretary of state could not avoid questions about human rights.
"Our values are different from American values," Abdelrahman Mohammed Shalgam, Libya's foreign minister, who spoke to the press alongside Rice, said. "We don't need anybody to lecture us on how to behave."
Asked about the case of Fathi el-Jahmi, who rights group Amnesty International has classified as a prisoner of conscience held in Libya, Shalgam said that el-Fathi "has been released but he is now being treated at a hospital".
"He's not under any pressure. He's never been subjected to any kind of pressure," Shalgam said.
El-Fathi was arrested by Libyan authorities in 2002. According to Amnesty, his arrest was for demanding free speech and political reform in Libya and the rights group asserts he is being held in a "a special facility ... on the outskirts of Tripoli".
Rice said that it was important to "talk about these issues in a respectful way and that is what I've done during my visit".
US-Libya relations
Washington's relations with Tripoli began to warm after Libya gave up an arms programme in 2003, but Rice had held back on visiting the country until a compensation package was signed last month to cover legal claims involving victims of US and Libyan bombings.
Ronald Regan, US president between 1981 and 1989, notably refered to Gaddafi as "mad dog".
Al Jazeera correspondent Amr al-Kahky said Libya was, for the US, "an excellent choice for oil supplies, being nearer to them than the Gulf countries.
"Libya now wants to be a modern country and develop in science, technology and education.
"The US wants to send a strong message to Iran and North Korea .. that the US has 'no permanent enemies' that it is only an enemy when it is threatened and is a friend when that is over."
Guma al-Gamaty, a writer on Libyan affairs, told Al Jazeera: "The Americans are in [this relationship] for the long term just for their strategic interests.
"This has all been done at the expense of human rights ... there are no rights in Libya and no democracy - the Americans are doing business with a dictator and it discredits them.
"Gaddafi is a ruthless, totalitarian ruler ... and he is preparing to pass his power on to his son.
"This visit says to other countries, especially in the Middle East, the US is only interested in oil and nothing else."