The former US president was due to tour areas destroyed by Israel's 22-day war against the Hamas-ruled territory earlier this year. The offensive killed more than 1,400 Palestinians, many of them women and children, and left large areas of the impoverished strip in ruins.
Thirteen Israelis died during the conflict.
Carter told students at a graduation ceremony in the Strip: "I have to hold back tears when I see the deliberate destruction that has been raged against your people."
Carter is also scheduled to meet Ismail Haniya, a Hamas leader, during his visit.
"Hamas leaders want peace and they want to have reconciliation not only with their Fatah brothers but also eventually with Israelis to live side by side, with two nations, both sovereign nations, recognised by each other and living in peace. That's what I hope to see in the future," Carter said.
'No hope'
Ayman Mohyeldin, Al Jazeera's correspondent in Gaza, said: "This is a historic occasion. The fact that the former US president is coming to Gaza is certainly significant on a symbolic and practical level, the symbolism being that he has spoken to Hamas before and that was considered a taboo by American, Western and Israeli standards, but the fact that he is coming to Gaza to speak to Hamas is certainly a significant development and one that officials here are embracing to the utmost.
"They want to make sure that Carter hears from the people of Gaza about their suffering. He will be met by the families of Palestinian prisoners detained by Israel including an eight-year-old girl who's never met her parents. She will be giving him a letter asking for him to go back to the Israelis and demand that Palestinian prisoners be released," he said.
Mohyeldin said: "These types of high-profile visits raise the visibility of the suffering and puts immediate pressure on Israel to lift the siege.
"In a very limited capacity, Carter is working as an intermediary and as a negotiator even though it is an unofficial visit."
Rebuilding Gaza is being hampered by Israel's blockade which dates back to June 2007 when Hamas took control of the territory.
Since then, Israel and Egypt, which control Gaza's only border crossing that bypasses the Jewish state, have kept the territory of 1.5 million aid-dependent people sealed to all but essential humanitarian supplies.
Israel has insisted that the blockade is necessary to prevent Hamas from arming itself, but human rights groups say it is a collective punishment.
Carter told the Haaretz, an Israeli newspaper, in an interview published on Sunday: "To me, the most grievous circumstance is the maltreatment of the people in Gaza, who are literally starving and have no hope at this time.
"They're being treated like savages. The alleviation of their plight ... I think, would be the most important [thing] the Israeli PM could do."