But all that has been in the days before.  By the time that the Fifth Avenue Hotel had been firmly established on the site of the Buck-horn, the corner had become the centre of the new town.  Across the Square, at the northeast angle, on the site of the building now capped by the figure of Diana, was a low, sordid shed.  It was the Harlem Railroad Station.  There, from one side started the cars for Boston, and from the other, the cars for Albany.  Cars, not trains, for horses were the motive power as far as Thirty-second Street.  There engines were attached in the open street.  Later, the horses ran through the tunnel as far as Forty-second Street where the Grand Central Station now stands.  In the Square the Worth Monument had been erected in 1857, and on the east side of the park, then enclosed by a high railing, was the brown church which dated from 1854.  That decade from 1860 to 1870 was one of constant changes and shiftings.  The New England soldier who marched through the town on his way to the front in 1861 rubbed his eyes a little when he passed through it again homeward bound after the surrender of Lee’s army at Appomattox Court House had brought the War of Secession to a close.  The last vestige of Knickerbocker life had disappeared forever.

It had been, and still was, an era of extravagant speculation.  Mushroom fortunes were springing up, and their possessors, as socially ambitious as they were socially inept, invaded Fifth Avenue strong in the belief in the all-conquering power of the Almighty Dollar.  In most cases they did not last long.  But they served a purpose.  They erected the splendid houses on the Avenue that a few years later the clubs were to occupy and enjoy.  Of the clubs that were on the Avenue in 1868, a contemporary chronicler wrote that nearly every one recorded the brief life of a New York aristocrat.  “A lucky speculation, a sudden rise in real estate,” so runs the rhetorical statement, “a new turn of the wheel-of-fortune, lifts the man who yesterday could not be trusted for his dinner, and gives him a place among men of wealth.  He buys a lot on Fifth Avenue, puts up a palatial residence, outdoing all who have gone before him; sports his gay team in Central Park, carpets his sidewalk, gives two or three parties, and disappears from society.  His family return to the sphere from which they were taken, and the mansion, with its gorgeous furniture, becomes a club-house.”  Perhaps this picture should be regarded with a certain restraint.  The observer was an up-state minister, looking for the excesses, wickednesses, and extravagances of the great city.  His judgment may have been as faulty as his style.