"If my remarks were seen as lacking regard for the feelings of atomic bomb victims, then I am sorry," he said on Sunday.
Kyuma had said in a speech a day earlier: "My understanding is that it ended the war and that it couldn't be helped ... I don't hold a grudge against the United States."
The remarks drew condemnation from victims of the bombing of Hiroshima on August 6, 1945, and the attack on Nagasaki three days later which together killed more than 210,000 people by the end of the year.
Resignation demand
Some opposition parties have demanded Kyuma's resignation.
Shinzo Abe, the Japanese prime minister, defended his minister on Saturday, but ruling party executives urged Kyuma to apologise in a bid to minimise the damage ahead of the July 29 upper house election.
"Nuclear weapons are absolute evil," said Tetsuo Saito, a lawmaker of the New Komeito, the sole coalition partner of Abe's Liberal Democratic Party (LDP).
"The remarks run against the grain of the Japanese people," Saito said. "They are the remarks any state minister must not make."
'Explain and apologise'
Shoichi Nakagawa, policy chief of the ruling Liberal Democratic Party, said: "If the comments were misunderstood, then he should explain and apologise."
Abe has seen his support ratings drop to around 30 per cent recently largely due to voter anger over the government's mishandling of pension records.
Officials in Japan - the only nation to suffer an atomic bomb attack - typically express sympathy for the victims, but most avoid criticising the attacks out of consideration for Tokyo's ties with Washington, its closest ally.
Defenders of the US bombings argue the attacks had convinced Japan to surrender and saved lives that would otherwise have been lost had fighting continued.