Roger Bergman, a plant spokesman, said a second suspect was arrested at the scene because "there is some uncertainty about who owns the bag".
A bomb squad was dispatched to the plant, located about 250km south of Stockholm, Sweden's capital.
OKG, the plant's operator, downplayed the incident, saying there was no threat to the safety of the plant.
Anders Osterberg, a plant spokesman, said traces of the substance were only found on the bag's handle, suggesting it may have rubbed off from the man's hands.
Osterberg said: "The bag contained toiletries, but a test found traces of the substance on the bag's handle."
He said there was no threat to the plant, but that the incident was being taken seriously.
"It's not something you use at home. We're not dealing with toys here," he said.
Security perimeter
Karlsson said police set up a security perimeter with a 300m radius around the plant, but workers already inside were not evacuated.
The plant was continuing to operate normally.
Karlsson declined to give details on the apprehended welder, and could not say if he was suspected of a crime.
TATP is highly explosive, and a tiny amount would be enough to blow off a person's hand, said Svante Karlsson, a weapons expert at the Swedish Defense Research Agency.
Karlsson said the substance had no civilian use and "it is very unstable, very sensitive to both friction and shocks."
The Oskarshamn plant has three nuclear reactors, which account for about 10 per cent of electricity produced in Sweden.
Operator OKG is jointly owned by E.ON of Germany.