"Falling for short-term gains while ignoring our long-term interest is not the right way forward," he said on Wednesday.
"The sovereignty and territorial integrity of the country will be defended at all cost and no external force is allowed to conduct operations inside Pakistan."
He added that Pakistan and US-led forces in Afghanistan had no agreement covering border operations.
Kayani's first public criticism of US policy is a measure of the sensitivity surrounding US military action on Pakistani soil.
Cross-border assaults
The US has long felt that Pakistan is not doing enough to combat the Taliban and al-Qaeda forces now operating on its soil.
According to the American newspaper the New York Times, George Bush, the US president, "secretly approved orders" to allow US special forces to carry out attacks inside Pakistan without the approval of the Pakistani government.
But Pakistani officials warn that any cross-border assaults by the US would only inflame tensions.
Kamal Matinuddin, a former lieutenant-general with the Pakistani Army, told Al Jazeera: "Any crossing of the border would mean the pressure on the Pakistani army to retaliate, the pressure on the newly elected government to retaliate would be there because no one is going to accept the presence of foreign forces on his soil.
"Although we may not retaliate in kind, certainly we can do so by stopping the logistics that are provided to the American forces through Pakistan.
"We can also perhaps find ways and means of not sharing the intelligence that the Americans very badly need."
Missile attack
On Monday, a US missile attack in the North Waziristan tribal region destroyed a school and houses associated with a veteran Taliban commander, killing 20 people, including some women and children.
Pakistani intelligence officials said four foreign fighters were also killed in the raid, three of them al-Qaeda commanders in Pakistan identified as Abu Qasim, Abu Hamza and Abu Haris, a newly-appointed leader for the group in the country.
The tribal belt is considered a possible hiding place for Osama bin Laden and Ayman al-Zawahiri, the al-Qaeda leaders.
The Pentagon has said that the US military was working closely with the Pakistanis on border security.
"We have a shared common interest with respect to terrorism and terrorist activities," Bryan Whitman, a Pentagon spokesman, said.
"Pakistan recognises the challenges that they have, and the United States is committed to helping allies counter terrorism."
Asked if, under Pervez Musharraf, the former Pakistani president and a US ally, Pakistan had ever turned a blind-eye to US military operations in Pakisatan, General Rashid Qureshi, who was a spokesperson for Musharraf, said there had never been any such tacit agreement.
"There were no actions that the United States forces could take in Pakistani territory. All actions that were to be taken - direct or indirect - inside Pakistani territory were to be taken by Pakistani forces," he said.