It is also claimed the timing of the marriage breakdown has not come as a surprise. It has coincided with the couple’s daughter Winnie turning 18.
The insider said: “Fern and Phil went through periods when they weren’t even speaking to one another.
“There were some really rough patches and it looked like deep cracks had developed. Things were fraught.
“They seemed to think it was something that happened to everyone, and it was just the natural ups and downs of any relationship.
“But those around them worried the damage might be irreparable.”
TV presenter Fern, 62, and celebrity chef Phil, 58, announced the break-up on Twitter this week, saying they had decided to “go our separate ways”.
The couple got hitched 20 years ago. Fern, who also has three kids from her previous marriage, admitted recently that she had concerns about life when the children left home.
She said: “These are big hurdles. You’ve got to give it thought.
“It’ll be only me and Phil, and I did talk to him about it, asking ‘Are we going to be all right?’ He went, ‘Yeah, we’ll be fine’.”
The announcement about them separating said they had “more than 20 happy years together”. It added: “We will always share a great friendship and our lovely children.”
The pair met while appearing on BBC programme Ready Steady Cook.
They also began working together on ITV show This Morning.
In a 2018 interview, Fern said someone on set had mentioned to her that Phil fancied her but at first she was not sure who he was.
After going out for the first time she said Phil baked a gooseberry pie, which sealed their romance.
She added: “He is my life’s great love, a really amazing husband, partner, friend. Yes, we have humdingers, but then we’re normal.
“Anyway, that’s good for the children to see. This is a real relationship.”
A representative for both Fern and Phil declined to comment.