I've always had incredibly strong scent memory. To the point where catching a waft of a perfume I haven't smelt in years can literally make me upset if its associated with a bad memory, or time in my life. I wanted to find out why that is. According to an article from Health Impact News it could be because there are roughly 1000 scent receptors as opposed to sight and touch which only possess a handful. This allows us to tell the difference between types of smells.
Additionally, smells get routed through the olfactory bulb, which is connected closest with the amygdala and hippocampus which are areas of the brain that deal with memory and emotion.
More so than any other scents such as hearing and touch, scents are tied to memories, particularly ones from your younger years that can be vividly recalled when that particular odor is present again.
For example, one study done on older adults found that memories cued by odor tended to be from the first ten years, and induce a stronger feeling of physically "going back in time." Also, doubly as many scent memories were recalled than those associated with words or pictures.
So if your ex's signature scent from five years ago still makes you cringe when you smell it in public, this could be why.