None of the trinkets have any disadvantages, but some of them are noticeably better than others.
Only ones I would say are worth remembering are;
Flat penny- VERY good chance of spawning a golden key when collecting a coin
Counterfeit coin- Good chance coins are worth double on pick-up
CANCER-- Doubles tear fire rate (Can exceed normal limits )
Most of the trinkets just increase drop rates of different items.