In one of two incidents that caused the deaths, smugglers tipped the refugees into the sea at night after noticing lights on land and fearing they would be spotted by the coastguard, MSF quoted survivors as saying.
"They forced us into the sea, even if the water was too deep. Several people did not know how to swim and they drowned," one survivor said. A woman eight months pregnant was injured by the boat's propellor after being forced overboard.
In a second incident, MSF workers discovered a group who had made it to shore after their boat capsized. They said they had buried 23 fellow passengers.
"The boat was stuck almost upside down in the sand, not far from the beach. The fishermen were trying to find survivors underneath but they could not," Said, an MSF worker, said.
"So I had to dive under. I managed to get in the hull and with God's help, we got two women and a man out safe."
'Horrific' conditions
According to the UNHCR, the UN refugee agency, about 32,000 people got safely to Yemen from Somalia between the start of the year and October. At least 230 people had died, and 365 were missing, the agency said last month.
"A lot of attention has been paid lately to tackling the issue of piracy in the waters off the horn of Africa," Francis Coteur, MSF's head of mission in Yemen, said.
"Unfortunately, little attention is paid to the drama of the refugees crossing the same waters in horrific conditions. Much more needs to be done to address this issue."
Conflict in Somalia, drought, and food price rises, have worsened hardships across the horn of Africa, already one of the world's poorest regions.