Russia and China, two of the five permanent members of the UN security council, joined South Africa, Libya and Vietnam in opposing the measures on Friday.
Indonesia abstained from voting.
If the resolution had been passed it would also have called for a UN special envoy for Zimbabwe to be appointed.
Mugabe was sworn in for a sixth term as president after a June 27 presidential run-off which saw the opposition candidate withdraw amid widespread reports of violence and intimidation by ruling Zanu-PF supporters.
Vitaly Churkin, Russia's ambassador to the UN, argued that sanctions would have taken the body beyond its mandate to deal with threats to international peace and security.
US 'disappointed'
The vote in New York came as representatives of Mugabe's Zanu-PF and the opposition Movement for Democratic Change (MDC) met in South Africa for a second consecutive day.
The United States said it was "disappointed" by the veto as the resolution would have provided "an incentive [for Mugabe] to negotiate seriously".
Zalmay Khalilzad, the US ambassador to the UN, said that Russia's "U-turn" on the sanctions was particularly "disturbing" and raises questions about Moscow's "reliability as a G8 partner".
He said that Dmitry Medvedev, Russia's president, had earlier this week supported a G8 statement that promised actions, including targeted "financial measures" against Mugabe and his allies.
China, which is major trading partner of Zimbabwe, said that the African nation was best left to conduct its own talks on how to resolve its political crisis.
The development of the situation in Zimbabwe until now has not exceeded the context of domestic affairs," Wang Guangya, China's envoy, said. "It will unavoidably interfere with the negotiation process."
South Africa talks
Before the meeting, other countries opposed to the restrictions had said that they could jeopardise attempts to get the political rivals to negotiate.
The MDC has said that the talks in Pretoria are merely to present conditions under which future negotiations to resolve the political crisis could be held. It has insisted that substantive negotiations will only take place if violence is halted and more than 1,500 "political prisoners" are released.
The main opposition party has also called for an expanded mediation team, including an African Union permanent envoy, and the swearing in of MPs as the opposition won control of parliament in recent elections.
Zanu-PF was represented at the meeting by Patrick Chinamasa, the justice minister, and Nicholas Goche, the labour minister.
Tendai Biti, the secretary-general accused of treason at home in Zimbabwe, and Elton Mangoma, the deputy treasurer-general, took part on behalf of the MDC.