The novel is set in the mid-19th century, although it includes glimpses into the history of the house, which was built in the late 17th century. The house of the title is a gloomy New England mansion, haunted since its construction by fraudulent dealings, accusations of witchcraft, and sudden death. The current resident, the dignified but desperately poor Hepzibah Pyncheon, opens a shop in a side room to support her brother Clifford, who is about to leave prison after serving thirty years for murder. She refuses all assistance from her wealthy but unpleasant cousin, Judge Jaffrey Pyncheon. A distant relative, the lively and pretty young Phoebe, turns up and quickly becomes invaluable, charming customers and rousing Clifford from depression. A delicate romance grows between Phoebe and the mysterious attic lodger Holgrave, who is writing a history of the Pyncheon family.