Eleven bodies were recovered from the water according to David Wimhurst, a spokesman for the UN in Haiti, and Mayor Evil Lavilette of Pestel, the ferry's place of departure.
At least five of those who died were children. Wimhurst said most of the 100 people on board the vessel swam to safety.
The boat was on a two-day journey along Haiti's southern peninsula, transporting passengers, food and charcoal to Port-au-Prince, Haiti's capital.
It made several stops to take on passengers and cargo and was "overcrowded," according to Wimhurst.
Lavilette said rescuers were not able to reach the stricken vessel quickly because the closest rescue boat had run out of fuel.
Many people on Haiti's coastline use ferries as the country's roads are in a poor condition.