Ghosts of Sylvester Manor

The old plantation on Shelter Island, once home to slaves and slave-owners, is a place where voices from American history echo through stairwells and attics, and the fragrance of the past grows thick among the boxwood and wildflowers. Philippe Cheng recorded his impressions in words and photographs


When you step past or over the footsteps that have come before, over the same stones, the same earth; when you walk past trees whose bark has absorbed the birdsong of hundreds of years and the cadences of thousands of human voices; when you step past these things without knowing what has come before, how can you possibly reconcile history with the present day? Without knowing what has come before, how can you understand what we as a nation have become? At Sylvester Manor on Shelter Island these questions are commonplace. 

The history of Sylvester Manor begins with the fact that, like all of our area, Shelter Island was long settled by indigenous peoples. It was first colonized in 1636 and subsequently sold to Nathaniel Sylvester, a sugar merchant with plantations on Barbados. He turned the property into a provisioning plantation for his Caribbean operation. Working the land were enslaved Africans and indigenous peoples, as well as English indentured servants. The farm and house that were at the center of this settlement are what we now know as Sylvester Manor.

It is a unique and quietly startling place to visit. 

The house itself has remained, to the credit of the Sylvester family, as true to its original form as time and nature would allow — true in its surroundings, its wallpaper, its furniture, even the layers of paint on the walls and the portraits echoing through time that glance at you from the walls of each room. It is a living, breathing relic of what was. And, of course, the obvious challenge is how to continue to preserve such a place without disrupting its integrity. 

As you enter the house on the first floor, you see sitting rooms to right and left, with kitchen, dining room, library, and a long, elegant stairway with ornate banisters leading to the second floor and its bedrooms. The layout is graceful, linear, and designed for lives well-lived.

Donnamarie Barnes, the curator and archivist at Sylvester Manor, eloquent and inquisitive, has been my guide. As we explored the house, the conversation went directly to a hidden staircase in a passageway between the sitting and dining rooms, as inelegant and rough as the entry staircase is grand and gracious. Curling and made inhumanly steep by design, this is where the slaves carried up the trays, the chamber pots, the linens, the lives of others. An entry point to their servitude. Concealed behind a wall, it went all the way up to the third-floor attic where, amid the rafters, they made their living quarters.

Barely visible and easily mistaken for an arbitrary marking in the attic is a little sketch, drawn by the hand of a child, in the wood rafters, showing boats with sails. 

Dreams? Or memories of the Middle Passage? We cannot know.

I left the house with more knowledge than I had had when I entered. The staircase became my metaphor, my entry point through which I felt I could hear echoes of imagined voices and unimaginable thoughts. 

I walked the grounds, adrift among the lawns and paths, some obscured, leading to gardens with stone borders where plants and trees groped outward from their intended beds. Ancient boxwoods whispered. 

The estate is explored through a road that winds through meadows and fields with cattle, hogs, chickens, goats, and greenhouses. It is clear there is a commitment to sustainable farming practices here today, and, simply, a vision of creating a complete agricultural environment that speaks not only to the history here but also to the future of farming at Sylvester Manor. Adding to this picturesque landscape is a windmill in the middle of a field planted with rows of flowers and vegetables; it transports you back to another landscape, another country, another time. 

In the driveway leading back to the house, tall trees line the road. It feels woody yet settled. Unless someone pointed it out, you wouldn’t take notice and turn to look at it as you pass: to your right, a large, pale, etched rock that did not arrive the glacial way but was placed with intention. Research has confirmed that at least 200 slaves are buried here. No headstones, no names, no dates, yet they are known. 

At Sylvester Manor, history is being rediscovered, unearthed, documented, preserved, and archived. Within its story, our collective history is being reexamined and retold, too, our understanding of the past becoming reconciled with the past as it was.It echoes, it enters and it leaves, it returns home — a story that remains, but in shards. Here, history is always reflecting back at us. • 


The manor was built nearly 300 years ago. The estate now is home to a community-supported agriculture program and farmstand, as well as serving as a community gathering place and living memorial to America's racially divided past and present.


Original wallpaper in the main hall. The bed-hangings and fabrics are original, too; there are no reproductions in the manor.
In the East Parlor is a portrait of Ezra L’Hommideau, a Sylvester descendant born in Southold in 1734 who served in the Continental Congress. The original, painted by Ralph Earl in 1792, hangs in the Museum of Fine Arts, Boston.
The Library was the lair of the last full-time lord of the manor, Andrew Fiske, who died in 1992. He inherited it in 1949 and was dedicated to preserving his family’s history. The wall of books, running the gamut from "Pride and Prejudice" to tomes on gardening, includes many first editions. 
Cornelia Horsford’s bedroom on the second floor; the stand holds a needlepoint screen she worked late in life.
The West Parlor walls are covered in the original Zuber wallpaper bought in Paris in the 1880s. 

The main stairs, used by the proprietors and guests.
The rear stairs, used by servants and slaves, rises steeply to the attic.
A faintly scratched ship in the attic eaves by a child's hand. A merchant ship, in Sag Harbor? Dreams? Memories of the Middle Passage? No one can know.

A mirror in the Poet’s Bedroom. The peeling paint on the pine-panelled walls reveals the original Prussian-blue beneath. (The room has been painted only twice since 1735.) Everywhere, here, the past is reflected back at us.
Cattle and passing birds on a warm day. Hogs and chickens also call Sylvester Manor Educational Farm home. 

Sylvester Manor can be found at 80 North Ferry Road, Shelter Island. The grounds are open for self-guided tours Monday through Friday from 10 a.m. to 4 p.m. The proprietors ask that visitors park at the yellow manor house and sign in at the office.

Philippe Cheng