The Take That star turned to food and comfort ate to a staggering 232lbs during his years in the pop wilderness after his solo career flopped in 1998 and he admits he could find nothing to like about himself.
He said: "It's one thing to loathe yourself in the head. But then to look in the mirror and to loathe yourself physically where do you go?
"At least hate yourself and look good. Or look s**t and feel great. But to have both, it's a double whammy."
While Gary, 40, now maintains a slim physique, he admits food will always be his "enemy".
He added in an interview with the new edition of Esquire magazine: "Food has been my enemy all my life. I've always loved it and it's always made me feel guilty and I've always starved myself.
"I've never eaten a mouthful of food for probably 20 years and not thought, 'I shouldn't be eating this.' "
Gary – who has three children, Daniel, 11, Emily, nine, and two-year-old Daisy with wife Dawn – also admitted fame is like a "drug" to him and understands why stars like Sir Elton John never want to stop performing.
He said: "I recognise it in me what I see in Elton and all those other people. We're chasing the buzz. It's a little drug.
"I think once you've tasted it, especially when you've tasted it a number of times, you never want to lose it.
"It'll annoy my wife for years, that, but there you go."