And, speaking after the first day of Nato meetings in Budapest about the war in Afghanistan, Gates said any settlement with the Taliban would be on the Afghan government's terms and would require the group to subject itself to Kabul's rule.
"There has to be ultimately, and I'll underscore ultimately, reconciliation as part of a political outcome to this," he said.
"That's ultimately the exit strategy for all of us."
Gates also said that reconciliation efforts could not include anyone belonging to al-Qaeda, the group blamed for the September 11, 2001 attacks which killed almost 3,000 people in the US.
A US-led coalition invaded Afghanistan and deposed the Taliban in 2001 shortly after the attacks.
There are about 33,000 US troops in Afghanistan, while Nato has about 40,000 stationed in the country.
'Downward spiral'
Gates's comments come as a draft US intelligence report on Thursday said the situation in Afghanistan is now at its worst since the US-led invasion.
The almost completed National Intelligence Estimate said the country is in danger of a "downward spiral" into violence and chaos.
Brigadier Mark Carleton-Smith, the top British military commander in Afghanistan, said on Sunday that the war there cannot be won militarily and that talks with the Taliban are crucial.
Several Nato commanders and diplomats have argued that negotiations with the Taliban are necessary.
Hamid Karzai, Afghanistan's president, said last week that he had asked Saudi Arabia to mediate negotiations with the Taliban and called on Mullah Omar, the group's leader, to make peace.