Under the Spanish in the 18th century, the property's structure had housed the gunpowder used by the soldiers at the nearby fort . The Henry J. Morton sketch (circa 1865) shown above is one of the few evidences of the Powder House - no photographs are known to exist. Morton's title of the sketch, "The Oldest House", suggests that although the building was no longer in use as an arsenal at the time of the drawing, Morton believed it to be the oldest inhabited structure of the time. All evidence leads to the conclusion that it is certainly the Old Powder House that once stood in an open field west of Cordova Street. Morton's drawing fits the Powder House in two ways: He shows the building standing alone, not along a street, and the ornamentation on its front suggests that it was a government building.

An 1870 description by one-time slave Jack Smith adds to the likelihood that this was indeed the Powder House. Smith described it as having two rooms. "The easternmost was with a window on the north, and another on the south side; on the front, with a window on each side, was an old double door turning on several pairs of staples. It had evidently not all been built at once and may have gone from a powder house to a dwelling....Three years ago (1867) it was torn down, the stone beaten up and the shell thrown into a walk."

Thereafter the property was referred to as "The Old Powder House Lot". The present structure which houses our inn was built in 1899. It still retains much of the original character of the Flagler era in its victorian architecture: high ceilings, verandas, and elaborate wood work.