Finally in 1796 the two invalids were moved to Norfolk in the hope that a change of scene would help.
In fact the elderly couple were virtually forced to move to Norfolk. It became obvious that Mary Unwin could no longer manage the household and Lady Hesketh, who lived mostly in London, could not keep spending months at a time in Weston Underwood. They lived in various lodgings with Cowper's maternal cousin, John Johnson before finding a home with him in East Dereham.
Lady Hesketh never visited Cowper again. However this did not prevent her from interfering in Cowper's affairs. It was at her instigation that Cowper's long standing servants Sam Roberts and his wife were sent back to Olney. New servants were installed. Lady Hesketh was also responsible for Hannah Wilson, Mary Unwin's niece being removed to Norwich and apprenticed there.
This had a detrimental effect on Mary Unwin's condition and to Cowper's despair in December 1796 she died. Although Cowper had sat with her a few hours before her death, she was buried at night without her companion Cowper being told. Johnson was fearful of the effect of the burial service on Cowper's mental state.
He did manage to complete some work for the Second Edition of Homer 1799 and just before Christmas that year he wrote his last verse The Castaway. His despair is almost palpable in these lines.