However, the Queen has ensured Andrew will keep his honorary military titles despite stepping back from public duties, the Sunday Times reported.
But now, Richard Palmer, Daily Express royal correspondent, has warned that allowing Andrew to keep his patronages would risk “serious damage to the monarchy”.
Mr Palmer tweeted: “It was my understanding earlier this week that Prince Andrew would lose all of his patronages eventually but the wheels often turn slowly at the palace.
“Allowing Andrew to keep patronages after refusing to cooperate with legal inquiries would risk serious damage to the monarchy.”
Andrew is currently the colonel of the Grenadier Guards regiment of the British Army, a role he took over from Prince Philip in 2017.
A source, claiming to be a senior military personnel, told the Sunday Times: “The Queen has let it be known to the regiment that she wants the Duke of York to remain as colonel and the feeling is that nobody wants to do anything that could cause upset to the colonel-in-chief.
“It is a very difficult, unsatisfactory situation.
“For the brief time he was in post, he was a good colonel, but the feeling across the regiment is that it’s not appropriate to retain him.
“You can’t have a colonel who can’t do public duties.”
Ms Giuffre is suing the prince for “punitive damages” which could amount to millions of pounds if she wins her case.
The accusations added that “on multiple occasions, Andrew intentionally touched the Plaintiff in an offensive and sexual manner without her consent”.
Andrew has said he has “no recollection” of meeting Ms Giuffre, despite the existence of a photograph appearing to show him with his arm around her waist.
Andrew is also accused of later abusing her at Epstein’s mansion in New York and on his private island in the Caribbean.
In his disastrous interview on Newsnight back in 2019, the Duke denied anything untoward had happened in London and sought to use a trip to Pizza Express in Woking as an alibi.
The Queen was furious after his disastrous interview and demanded her son not to release a statement of innocence.
Andrew has previously issued a statement saying he “continues to unequivocally regret my ill-judged association with Jeffrey Epstein”.
He later confirmed he is “willing to help any appropriate law enforcement agency with their investigations if required”.
Source: EXPRESS CO UK