In colonial times, there was an ordinance (in Massachusetts at least–I don’t know about the other colonies) requiring that every town have at least one tavern or inn for the accommodation of travelers.
Most of the surviving early tavern buildings aren’t currently operational but are private residences or are preserved as museums. Those listed below fall into various categories–and a few are places that were originally private residences but have been turned into taverns in modern times:
Three Chimneys Inn, Durham, NH 1649 Built as a private residence, converted to an inn and tavern in modern times.
Old Constitution House, Windsor, VT c.1770 This house, which served as a tavern in the 1770s, has been moved twice since then and is now preserved as a museum commemorating the signing of the constitution of the Republic of Vermont in 1777.
The Manning Manse, Billerica, MA c. 1696 A seventeenth-century farmhouse, carefully preserved by the nonprofit Manning Association and presently leased out for use as a restaurant.
The Wayside Inn, Sudbury, MA 1716 This inn was the setting for Henry Wadsworth Longfellow’s Tales of a Wayside Inn, the series of narrative poems (loosely based on Chaucer’s Canterbury Tales) that includes his well-known “Midnight Ride of Paul Revere.” The two front rooms (the taproom with working fireplace and a parlor set up as a historical exhibit) are where the main interest lies; further back in the building are big, more modern dining rooms.
Warren Tavern, Charlestown (Boston) MA 1780 Dating to the last years of the American Revolution, the Warren Tavern is located in an exceptionally picturesque neighborhood at the foot of Bunker Hill.
Blanchard’s Bard of Avon Tavern, Avon, MA 1780 Blanchard’s is only open as a tavern on Saturday evenings from 8 to 11 PM; during the week it’s a museum. Saturday evenings tend to attract a crowd of local regulars, and anyone who wanders in for the first time may feel a little out of place. They usually have live folk music, and they have a working fireplace, and a menu of drinks and light snacks that’s modeled on the cuisine of colonial New England.
Salem Cross Inn, West Brookfield, MA Not built as a tavern, but as a large private farmhouse. Nonetheless, the building is about 300 years old and merits inclusion for its historic interest even if it’s only been serving meals for 50 years or so.
Red Lion Inn, Stockbridge, MA c. 1773 Most of the original inn burned down in the 1890s; the building was reconstructed at that point, and almost all of what’s there today dates from that reconstruction.
Old Tavern Farm, Greenfield, MA c. 1760 Currently operated as a bed & breakfast, not as a tavern.
The Old Ordinary, Hingham, MA 1688 Operated as a museum by the Hingham Historical Society.
Munroe Tavern, Lexington, MA 1731 Operated as a museum by the Lexington Hostorical Society.
Buckman Tavern, Lexington, MA c. 1690 Operated as a museum by the Lexington Hostorical Society.
Hartwell Tavern, Lexington, MA 1732 Preserved as part of Minuteman National Historic Park.
Concord’s Colonial Inn, Concord, MA 1716 This structure has served various purposes in its history, but has been a hotel only since the late 19th century.
Rider Tavern, Charlton, MA 1797 Owned by the Charlton Historical Society, this impressive three-story wood-frame building is not, alas, open as a tavern.
Griswold Inn, Essex, CT 1776 Claimed to be the oldest continuously operating inn in America.
The White Horse, Newport, RI 1652 Formerly the meeting place for the Rhode Island provincial assembly, the White Horse has a pleasant taproom with a working fireplace.
Atlas of New England 
There’s a still-operating tavern on Chelmsford Road in Billerica, right next to the town forest. It had been called “The Manning Manse” for most of my life, but now it’s an Italian restaurant called Villagio. The building is from 1696. I have no idea if it’s been serving food continuously since then.
The Manning Association’s website (http://www.manningassociation.org/gen3.html) describes Mr. Manning as a prosperous farmer, selectman, provincial legislator, and former juvenile delinquent, but doesn’t say anything about his having kept a tavern. Presumably it was just a private residence back then. As for its current incarnation, the restaurant’s website says the tavern has “eight new HDTVs for your enjoyment,” which is somewhat unappealing to me. But it’s a very nice old building, judging from the pictures. Thanks for pointing it out!
Free WiFi!