The site of the old Bishop's Palace is said to be haunted by the figure of a tall, willowy woman wearing the clothes of the Norman period. The figure has been seen in broad daylight drifting across the grounds of Mount House in the direction of where the Bishop's Palace once stood.

The figure is believed to be the ghost of Queen Emma of Normandy (984 - 1052), husband to King Æthelred the Unready, and father to King Edward the Confessor. Queen Emma's connection to Witney is strong. She is remembered in the street name Queen Emma's Dyke and the nearby Queen Emma's primary school.

Although the first Bishop's Palace was not completed until after Queen Emma's death, a popular legend about her is said to be connected to its construction.

Queen Emma was not always popular in court. She was a strong-willed woman and wielded considerable influence over her son, King Edward. As the story goes, the Bishop of Winchester was a close advisor to King Edward and saw Queen Emma as an impediment to gaining further power. In the hope of ruining the Queen's good name, he accused her of 'unchastity', and as a result, Queen Emma was taken to Winchester Cathedral to undergo a 'trial by fiery ordeal' to prove her marital fidelity.

This trial involved walking across plowshares that had been heated to a burning hot temperature and places on the floor of the Cathedral nave. To the Bishop's surprise, Queen Emma passed this trial with flying colours. She walked across the blistering heat of the plowshares without any discomfort or injury, and thus proved her innocence.

According to the story, King Edward was so relieved that his mother had passed her ordeal that he awarded the Bishop of Winchester twenty-one manors, including the one at Witney where the Bishop built his Palace. Why the king should choose to reward the person who had just been proved to have wrongly accused his mother of adultery is anyone's guess!