The Spanish newspaper El Mundo reported that the assassin had asked another patient who was sharing the Colombian's room if he was Vargas.
When the man said no, he took out a gun fitted with a silencer and shot Vargas, who was asleep.
Neither police or the newspaper confirmed whether Vargas, 60, had been under police guard at the 12 October hospital in the Spanish capital.
In the 1980s Vargas, allegedly a former partner of Pablo Escobar, the late Medellin cartel drug chief, ran his own cocaine manufacturing and smuggling operation out of a remote, jungle-covered area of Colombia's southwestern Caqueta province.
A Colombian court sentenced him in 1995 to 26 years in prison for illicit enrichment related to drugs, but he was freed in 2002.
Spanish authorities arrested him for carrying a fake Venezuelan passport in 2006 and he was awaiting trial in connection with a 500kg cocaine haul.
His trial had been delayed and prison authorities freed him after he came down with what Spanish media said was a lung-related illness.
Police believe his killing was a settling of scores by drug rivals.