In the summer of 1927 three young men of means ran into a neighbor from back home in Winnetka, Illinois, on the streets of Paris, who was stranded with a penniless delegation of labor leaders and educators intended for Moscow. Fairfield Porter and his brothers agreed to wire for money to bail the men out, and soon Porter, an aspiring painter, found himself in Moscow, in a meeting with Leon Trotsky, “smooth and charming and clever,” as Porter observed, on the verge of his expulsion from the Soviet Union. Porter sketched Trotsky holding forth at a conference table, proselytizing to the American bureaucrats. The drawing is of no special distinction: it was done, like a doodle, to pass the time. But it suggests the kind of casual power that Porter threw on and off like a cotton sweater: the Harvard junior could boast that his pocket change had made a meeting of nations possible.

Porter went on to become an art world insider, publishing influential reviews for years; he helped establish the early reputation of Willem de Kooning, among others. But Porter’s own paintings are outsider art: DIY worlds put together with some of the earnestness of a person very passionately embracing a hobby, leaving some of the amateur’s awkwardness in. “I predict,” the critic Peter Schjeldahl wrote, “that a bizarre sense, via Porter, of fully inhabiting a parallel or shadow twentieth century will appeal to more and more people as time goes by.” What would such a “shadow twentieth century” look like, on the evidence of the artist’s sun-splashed, cheerful canvases? His mature work provides no evidence of historical turbulence, no public themes, no expression of a divided or tormented twentieth-century psyche. His subjects are few, and repetitive, and apparently banal: a cluttered breakfast table, an afternoon iced coffee, a doubles match, a little girl on a bicycle. All up and down the Atlantic coast, in gift shops and frame shops catering to summer visitors, paintings superficially like Porter’s can be found next to “BEACH THIS WAY” signs and hostess gifts; the closer one gets to Porter’s turf in Penobscot Bay, the more likely it is that these canvases will bear the stamp of his influence.

The work immediately appealed to “solid citizens” who disparaged what was known as modern art, as John Ashbery wrote. Lean and hale, a father of five (he was mocked in the art world as “the man with 8 children”), Porter and his wife, Anne Channing, a New England aristocrat, owned an old sea captain’s house on the main street of Southampton, New York, where his neighbors saw him not as an artist but, according to Ashbery, as one of them, a “kind and concerned citizen.” The largest collection of Porter’s paintings, 250 or so, remains in situ, at the nearby Parrish Art Museum in Water Mill, still soaking up the famous painterly light that they captured decades ago as it crossed the walls of Porter’s porch and studio, just across the potato fields.

The difficulty of breaking this closed circuit of apparent ease and privilege has set a high bar, over the decades, for Porter’s entry into the ranks of “serious” artists. Those who have made the case have sometimes been under Porter’s spell. Ashbery moonlighted as a judicious professional art critic for years, but he formed his assessment of Porter as perhaps “the major American artist” of the twentieth century partly while visiting his friend’s houses in Southampton and on Great Spruce Head Island, Maine. Porter was the subject of important profiles by the poets James Schuyler, who lived with the Porters for a good part of eleven years, and was Fairfield Porter’s on-and-off lover; and Frank O’Hara, a favorite guest of the children in the family, whose classic poem “The Day Lady Died” depicts a frantic lunch hour checking off errands before catching the “4:19 [to] Easthampton” for a bash at the Porters’. Porter “will be acknowledged as a major twentieth-century artist when we can figure out how,” Schjeldahl wrote in 2000. As a guest in Southampton, Schjeldahl once sat in Porter’s studio and “observed the progress of a large, not very successful painting of a lawn party.”

A concise but revelatory exhibition of Porter’s works, “Fairfield Porter: What Everyone Knows,” is on display at the Phyllis Harriman Mason Gallery at the Art Students League in Manhattan until mid-August, and everyone is invited to the party. Its setting is part of the argument. This very unusual school and atelier, where Porter studied for two years in the late 1920s, occupies a scuffed Renaissance Revival hôtel particulier on 57th Street. When I visited, I shared the freight elevator to the second-floor gallery with a delivery man, his hand truck piled with tubes and boxes. Students streamed in and out of the airy room, copying and taking notes and mixing with the gallery’s curators, who work in an adjacent space and sometimes come up for air, like chefs emerging from a busy kitchen.

The paintings have subjects but not really subject matter; their conventional themes—Porter described them as “normal” themes, “not sought after”—establish the ground of reality on which art builds its difference. And what a difference: Porter’s brushstrokes seem at times still wet to the touch, trailing off like vapor before suddenly detonating a bluster of peach or indigo. Paint plays out luxuriously and abstractly within the idea of summer things like Bermuda shorts, clay courts, lawns, and tablecloths, as it does for other painters only in a stormy sea or roiling sky. But Porter’s Tretorn and grosgrain lyricism belies the “hard information,” as Schjeldahl put it, encoded in his successive painterly choices, each one reframing and qualifying the others, about line, shade, color, and perspective. Spend long enough in front of a Porter canvas, and you “can figure out how” to reconstruct these shrewd, ecstatic formal moves. Sanford Schwartz put it succinctly in a mixed verdict in these pages: these paintings seem to say to us, “Why don’t you try making one?”¹

The hard information did not come early to Porter. It took nearly two decades as a painter for him to find his style. His gift was not easily adaptable to the available visual rhetorics of the Thirties and Forties, despite his emerging Marxism (a lifelong commitment, though like many rich Marxists, he worried that he had honored it in the breach). A mural, Turn Imperialist War into Civil War (1934–1935), featured as models for a fearsome band of advancing rebels an accountant, a housepainter, and Anne Porter, who is pictured busting down a door. The dreamed, or feared, figure depicted in Self-Portrait in the Studio (1950), the earliest painting in this show, is forty-three years old but appears in an outgrown blazer and billowing breeches, like a prep school first-former late for baccalaureate. His fists are clenched as though in refusal: you couldn’t pry a brush into them if you tried. It’s a pity, because the rest of the picture is badly drained of color. Barren canvases in the background match the dreary walls and the floor, which tilts, like a funhouse floor, a little in our direction; Porter seems braced, as though he feared he might spill out of the painting and collapse into the gallery. His eyes are blanked out by shadow, in an ostentatious touch that seems a kind of mea culpa for insufficient insight; once he’s learned to look properly, maybe he’ll go back and add in the eyes. The painting is clumsy but enormously touching. Porter’s eldest son, sixteen or so at the time, had symptoms consistent with severe autism; caring for him had baffled the family. Tensed and anxious, Porter appears to self-soothe (the clinical term is “stim”), using strategies observed from, perhaps passed on to, his boy.

The paintings in “What Everyone Knows” are arranged not chronologically or thematically but more like a volume of poems, by rich, implicit correspondences and associations. We work through them by hunches, not certainties. Looking Through the House (1961), a taciturn study of windows, hangs near Porter’s apprehensive self-portrait. In the foreground of the painting, a darkened table and formal dining chairs call to mind the delicate intimacy of Pierre Bonnard and Édouard Vuillard, artists Porter first encountered during his Moscow sojourn back in 1927, as part of a private collection seized by the Soviets. But nearly all the light and movement in the painting is hoarded by three big windows looking out on the islands and blue water of Penobscot Bay. The divided panes of glass don’t quite splice together a consistent seascape; each little rectangle is, after all, a new start, like paintings on a jammed studio or gallery wall, or like sequential images on a contact sheet. (The photographer Eliot Porter, Fairfield’s brother, might be the picture’s implied interlocutor.) One big island seems sloppily reassembled, its halves mismatched after being chopped in two by the window jamb.

Porter’s titles, which can seem so nonchalant and descriptive, often embed important claims. Here we receive a significant education in what is meant by the word “through” in a medium with no literal depth: as in many of Porter’s works, the background is more thickly painted, and tangibly closer, than elements in the foreground. The word has other resonances, too, all of them relevant to the painting: you “look through” images like this one in an old photo album; or you look “through” the house, as though by its inspiration or mentorship. “Through” also indicates the time it took to paint the painting, the duration of that process. Porter usually worked episodically “through” his compositions, without the benefit, or the handicap, of a sketch or a linear plan. My guess is he painted those individual panes left to right, one at a time, like toenails.

The right question to ask of Porter’s pictures is usually the simplest one of all: What’s he up to here? Looking Through the House is front matter for an exhibition in which likenesses cluster, and what Emily Dickinson called “internal difference”—within paintings, between paintings—is usually “where the meanings, are.” Three images of James Schuyler hang near one another in the gallery. Schuyler and John Ashbery appear in Jimmy and John (1957–1958), a profoundly weird double portrait painted in Porter’s Southampton studio. Both men had caught Porter’s eye in the early 1950s, when the artist, a generation or so older than the poets, began to socialize on forays into the city with members of what was soon known as the New York School. Porter, Eagle Scout–ish and starchy, had an instant crush on both of these rather schoolboyish, impish men; by the time of the painting, he had begun a decades-long sexual relationship with Schuyler. (I once asked Ashbery about the painting. He laughed and noted Porter’s keen interest in his and Schuyler’s crotches: “We called that painting the battle of the bulge.”)

The figures occupy entirely different planes, as well as two distinct time signatures: in the background, Schuyler hogs the floral sofa that Porter often used for portraits, installing himself for another long summer afternoon, while Ashbery, the more restless man and talent, sits on a wooden chair, crossing his arms in an expression of good-natured impatience and pique. John’s on his way somewhere; Jimmy, as Anne Porter once put it, “came for a visit and stayed eleven years.”

Porter painted Schuyler several times, almost always capturing the poet absorbed in thought or buried in a book. At the exhibition, the Schuyler of Jimmy and John gazes determinedly out of the frame, at another, more monumental image of himself in a large group portrait hung nearby, The Screen Porch (1964). Schuyler, in khaki shorts and a light blue oxford, takes up the one chair in Porter’s porch studio in Maine. Porter’s daughters stand awkwardly nearby; a fidgeting Katie, fifteen, wears socks and blue tennis sneakers to match those worn by Schuyler, her sometime playmate and confidante, who looks about fifteen himself. The younger daughter, Elizabeth, stands between them, her back turned, her head angled as though to spy into Schuyler’s reading: give the poor girl a book, or anything at all to do! A far-from-amused-looking Anne Porter occupies the right corner of the picture, approaching the group from the other side of the screen as though to spoil all the fun, if fun this be.

But it does not appear to be much fun; instead, lines of insinuation cross every mien, and only Schuyler, the interloper on whose complicated charisma the family rose and fell for years, seems to know how to pass the endless hours in the gorgeous but cold and spartan barracks. (“Screens” in Maine have one meaning: they keep mosquitoes and black flies out. Anne Porter, who complained that she sometimes was treated in her own home as “someone in another generation or of another social class,” is relegated here to the bugs’ realm, and perhaps to their status.) As in Diego Velázquez’s enigmatic group portrait Las Meninas, a favorite of Porter’s and a model for The Screen Porch, we’re seeing a painting of painting: in the lower-left foreground, a jug of thinner, a palette knife, and a few tubes of pigment celebrate the artist’s own ingenious method of passing the time—no help, alas, to his housemates. As in Jimmy and John, the strain of being a subject, of being, as it were, subjected, is one of the painting’s themes. The finished work hung for years in Maine, near where it was painted. The family had a nickname for it: “Four Ugly People.”

A third view of Schuyler, A Day Indoors (1962), hangs beside the ugly people. Schuyler has again commandeered the best seat: on this cold summer day in Maine, he warms himself by the roaring hearth in “the big house” on Great Spruce Head Island. Three figures, Schuyler and the two girls, are smaller, and somewhat indistinct, and seem included partly for scale and contrast; the star of the painting is the room, with its poppy-red painted floor and deteriorating, high-kitsch New England decor, including a model sloop and, above the fire, a reproduction of one of the Elgin Marbles. The accent is on a colorful, braided oval rug, a standard L.L. Bean item, so familiar you can almost smell the must. The chairs and furniture in the room jut at sharp angles and form a visual chute that creates a measure of suspense about the closed door at its terminus. (Ominous music: Is Anne Porter about to enter?) The details in the painting seem to hang on the vertical axis, like buoys in a fishing net. But the area rug is perfectly flush with the frame in the lower-right corner, where a mapmaker would locate his key. This summer cottage staple, repeating the spectrum of colors in the overall composition, is a kind of palette: you could imagine Porter dipping a brush into one of those vivid stripes to touch up Schuyler’s khakis or pale red button-down. Or, with its suggestion of dynamic, spiraling depth, perhaps it’s a drain, threatening to suck this moody group back out to sea, like bilge.

As Sam Sackeroff, the show’s chief curator, argues in his fine catalog essay, we find in Porter “an experience of looking…that pays off over time.” That’s a defense of this show, which presents Porter’s work for the first time outside brittle twentieth-century art spats. What was “abstraction,” you wonder, if not the brushwork in Porter’s exuberant skies and seas? What pure color field immerses us like one of Porter’s lawns? The paintings have deepened for me, over the decades, as they apparently have for others. At the show’s opening, Sackeroff told me, he introduced himself to a woman scrutinizing a portrait of an adamant-looking Katie Porter, age fifteen or so. The woman looking so carefully at the portrait was, it turned out, Katie Porter.

Those who knew Fairfield Porter say that he entirely lacked small talk; to pay him a visit was to be blasted at close range by “unorthodox opinions” delivered, according to Ashbery, “with the most charming vehemence I have ever known.” Vehemence is not a word one would associate with the paintings that round out this astonishing show: views of rocky coast, studies of yard trees, still lifes, amateur-seeming portraits of odd-looking people, the kind you can pick up in an antiques mall along any of the old New England byways, their chins a little long, eyes slightly crossed, foreheads flat as a sunfish. More than one of Porter’s canvases has been discovered in such a shop, among the earnest hobby paintings of the long dead, dispatched as part of a cottage clean-out to the local appraiser and carted off in her Buick. One early Porter image of a pet crow was salvaged from the garbage on Great Spruce Head Island, where it now hangs in an unwinterized cottage, sharing space with old photographs and thumb-tacked nautical maps.

But Porter’s insider-outsider art indeed makes an argument, if not a vehement one. The artist, who studied at Harvard with the philosopher Alfred North Whitehead, liked to quote Wittgenstein, from Philosophical Investigations: “Every sentence in our language ‘is in order as it is.’” Porter made his own casual translation, in pictorial terms: “The truest order is what you find there.” Porter opposed the imposition of order, by all political and aesthetic means, that is, by force or by manifesto. It tempered his early firebrand socialism, which flowered modestly as his and Anne’s generosity to friends, as well as many friends of friends and some strangers, at the family’s two fascinating homes. It is becoming clear that this improvised summer house monde was one of the most exciting and strange of the twentieth century (it was captured brilliantly in a recent biography of Schuyler by Nathan Kernan), and Porter had an easel set up in the middle of it.²

The work of this companionable man, the scion of a real estate fortune, nevertheless belongs to the tradition of American amateurs and recluses like Dickinson and Joseph Cornell (who once presented Porter with a box, as a thank you gift for a favorable review). Porter shares even their relationships to scavenged or improvised materials: he developed his own pigments and prepared his paints using beeswax and linseed oil, and sometimes built his stretcher frames from felled trees. That he was a painter of the patron class only makes these DIY techniques more interesting, and that he was a great and influential critic, attuned to every aspect of the art world, more interesting still. He painted the sunset of WASP power and culture, its last light showing up as eccentricity. Porter’s canvases sometimes feel like artifacts from the coastal cold-roast elite’s secret psychedelic period.

Not that the world of these paintings has vanished. Great Spruce Head Island is a few miles off Deer Isle, Maine, in Penobscot Bay. I met David Porter, the painter’s grandson, and a few of his friends one morning in mid-July at the Deer Isle Yacht Club for the twenty-minute crossing. We passed Scott Island, which belonged to Robert McCloskey, the author of Blueberries for Sal, and Butter Island, owned by the Cabots of Boston—the ones who, according to legend, “speak only to God.” David, who looks uncannily like his grandfather, is a retired professor of marine biology in his mid-eighties; one of his main passions these days is the preservation of the island’s rockweed, a foundation algae used as a fertilizer, which commercial harvesters rake up at high tide. An entire underwater ecosystem—David called it a “forest”—is damaged when their big boats appear offshore.

The island is a human ecosystem where processes like painting or writing poetry cannot be meaningfully separated from other kinds of work. Nearly everyone here paints; Porter’s canvases hang on the walls of the old unwinterized cottages, next to work in the same key by visitors and family. On the old tractor road running by Eliot Porter’s photography studio we met a young man, a guest of the family, carrying an easel and a nearly finished seascape.

Approaching “the big house,” a child bounded out to greet us, reporting with some satisfaction that there were no waffles left. The interior depicted in A Day Indoors remains unchanged from Porter’s depiction of it, down to the poppy-colored floors. (Floors in several bathrooms in houses on the island are done in the same color: there was “surplus paint,” David conjectured.) Upstairs, “Jimmy’s room” is just as it was the day Schuyler wrote “Buried at Springs,” his elegy for Frank O’Hara, while fending off a hornet. The poem is partly about writing O’Hara’s tribute at the desk where O’Hara himself had sat eleven years before, and “saw and heard it all.” It’s all still there, the desk, the bed, the spruces and sound of lapping water, as both poets saw and heard it; before we left, I sat down in Schuyler’s rocking chair, still positioned closest to the hearth.