Mardhi has been replaced by Adbel Basit Sabderat, who was the federal affairs minister.
Mardhi has had recent differences with the media over many reporting bans said to be illegal by rights lawyers.
The energy ministry will now be headed by al-Zubeir Ahmed al-Hassan, the former finance minister.
The energy and finance positions are important in Sudan due to the more than 500,000 barrels of oil the country produces per day.
A Sudanese analyst who preferred not to be named said the reshuffle was just "pushing papers".
"The key people are still in key positions."
But Mardi's removal was a step forward to improve Sudan's image abroad, he said.
Ibrahim Ahmed Omer was appointed minister for science and technology.
A senior figure in President Omar Hassan al-Bashir's National Congress Party, Omer came out of retirement to advise Bashir on media matters.
Kassala Ibrahim Mahmoud Hamad, a former governor, was made minister of interior. He replaces al-Zubayr Bashir Taha, who was moved to the agriculture and forestry portfolio.