"Quite frankly I never thought I would be visiting Libya so it is quite something," Rice told reporters travelling with her.
"It is a beginning, it is an opening, it is not, I think, the end of the story."
Rice will meet Muammar Gaddafi, the Libyan leader, and may sign a trade and investment deal during her visit.
Experts say the main thrust of the visit is to signal a new era in US-Libyan relations that had been marked by decades of mistrust and violence.
Relations with Tripoli began to warm after Libya gave up its weapons of mass destruction programme in 2003 but Rice held back on visiting the country until a compensation package was signed last month to cover legal claims involving victims of US and Libyan bombings.
US officials said Rice was looking forward to meeting Gaddafi and planned, among other issues, to raise human rights concerns as well as regional conflicts in Chad, Sudan and the recent coup in Mauritania.
Rice is also expected to push Gaddafi on a compensation package signed last month.
Compensation
No money has been paid into the compensation fund yet but the lead US negotiator with Libya, David Welch, said he was optimistic it would happen soon.
Libya finalised legalities to set up the fund on Wednesday and one senior US official said it would take "more than days" before enough money was in the account and payments could be made to both sides.
No details have been given over who will put money into the fund or how much it will amount to but outstanding legal claims could run into billions.
US victims covered include those who died in the 1988 bombing of Pan Am Flight 103 over Lockerbie, Scotland, which killed 270 people, and the 1986 attack on a Berlin disco that killed three people and wounded 229.
It should also compensate Libyans killed in 1986 when US aircraft bombed Tripoli and Benghazi, killing 40 people.
Rice has come under criticism at home for making the trip before the money is paid out.
And families of dissidents have also criticised her for giving legitimacy to Gaddafi.
'Benefits for West'
Sean McCormack, a state department spokesman, called the trip to Libya a "historic stop" - the last secretary of state to visit Libya was John Foster Dulles in May 1953, before Rice was even born.
But the view from Tripoli is not necessarily one of overwhelming gratitude that the West was now allowing a former pariah state back into the international community.
"The West seeks its own interests. They'll benefit from improving relations much more than Libya," one woman Tripoli resident told Al Jazeera.
Another resident said "of course there's plenty of economic benefits for the West".
Miloud Lemhadabi, a professor of international law in Tripoli, told Al Jazeera's Amr El-Kahky that both sides had much to gain from rapprochement.
"Libya wanted to jump over the Lockerbie reprisals to build a modern state after long years of sanctions. The West also saw a big opportunity in business and trade partnership with a rich nation. Good cheap and easily accessible energy is a main target for the West and Libya is an ideal choice to reach that target."
Rice is also set to visit Tunisia, Algeria and Morocco before returning to Washington on September 7.