The 58-year-old writer will watch the happy couple tie the knot at St George's Chapel at Windsor Castle, Windsor, on May 19 after she was invited to the main event, but she's said to be "unhappy" that she's been missed off the guest list for the pair's private party - which will be attended by 250 guests - at Frogmore House.
A source told the Daily Mail newspaper: "Numbers are limited to the evening party, she is not a member of the Royal Family any more and Prince Charles doesn't have time for her. He just can't see why she is still such a big part of his brother's life."
The charity patron - who is also known as Sarah Ferguson - fell out with the royal family after her divorce from Queen Elizabeth's second son Prince Andrew, with whom she has daughters Beatrice, 29, and Eugenie, 28, in 1996.
However, she was invited back to Balmoral last year as a guest of the queen.
This isn't the first time Sarah hasn't made the guest list of a royal event as she wasn't invited to Prince William and Duchess Catherine's wedding back in 2011 at Westminster Abbey, London, and, instead, went on holiday to Thailand instead.
However, although Prince Charles and his siblings aren't keen on Sarah, it's believed Harry was the one who insisted she was invited to the wedding ceremony.
An insider explained: "He absolutely, 100 per cent wants her there and has told palace officials that in no uncertain terms."
The 33-year-old royal will tie the knot with the former 'Suits' actress, 36, in front of the whole world in just under two weeks' time - eight months after they got engaged.