But wouldn't it be interesting to have a few Northeast Regionals to/from Washington and Philadelphia run through Penn Station and on to the Hamptons?  Especially maybe Sunday night/Monday morning to "town", and Friday nights out to the Hamptons ... ?

Sure, but the problem here is that everybody  envisions a service built around his or her sepcific wants ..., and the passenger network that  existed before the highways and airlines took most of the business is just not flexible enough to accommodate them, particularly when business travel is an impossibility on most Amtrak routes, and demand is buiit around a summer-season peak, and there's insufficient equipmet to run additional trains ... etc, etc, etc