The draft offices were surrounded by a mob, and as the first name was called a stone crashed through a window.  That was the signal.  The offices were rushed and the building soon in flames.  The police were routed, and a squad of soldiers sent to their aid disarmed and badly beaten.  Then the mob ranged, pillaging the house of William Turner on Lexington Avenue, firing the Bull’s Head Hotel at Forty-fourth Street, and the Croton Cottage opposite the Reservoir, plundering the Provost Marshal’s office at 1148 Broadway, and destroying an arms factory at Seventh Avenue and Twenty-first Street.  Then some one in the mob cried out that the war was being fought on account of the negroes and the rioters started in the direction of the Asylum.  When they reached the spot they found an empty building, for the alarm had been given and the children taken to the Police Station and later conducted under guard to the Almshouse on Blackwell’s Island.  But the structure they destroyed, and when they came upon a coloured man in the neighbourhood they hanged him to the nearest tree or lamp-post.

During the riot the draft-rioters made their headquarters at the Willow Tree Inn, which stood near the south-east corner of Fifth Avenue and Forty-fourth Street, and which at one time was run by Tom Hyer, of prize-ring fame.  A photograph shows it as it was in 1880, with the tree from which it took its name in front, and the Henry W. Tyson Fifth Avenue Market adjoining it.  “Fifth Avenue” quotes from Mr. John T. Mills, Jr., whose father owned the cottage:  “My mother planted the old willow tree,” said Mr. Mills, “and I remember distinctly the Orphan Asylum fire.  The only reason our home was not destroyed was that father ran the Bull’s Head stages which carried people downtown for three cents, and the ruffians did not care to destroy the means of transportation.”  There were many vacant lots in this section of Fifth Avenue at the time of the Civil War, and a small shanty below the Willow Cottage was the only building that stood between Madison Avenue and Fifth Avenue.  On the north-west corner of Fifth Avenue and Forty-fourth Street, then considered far north, stood a three-story brick building.  The stockyards were between Fifth Avenue and Fourth Avenue from Forty-fourth to Forty-sixth Street, and Madison Avenue was not then cut through.  The stockyards were divided into pens of fifty by one hundred feet, into which the cattle were driven from runs between the yards.  On the east side of Fifth Avenue, just above Forty-second Street, stood four high brown-stone-front houses, the first to be built in this neighbourhood.  In the rear of these were stables that had entrances on Fifth Avenue.  “Fifth Avenue” points to the Willow Tree Inn as illustrating the appreciation of Fifth Avenue real estate.  “In 1853 this corner was the extreme south-west angle of the Fair and Lockwood farm, and was sold for eight thousand five hundred dollars.  Here in 1905 a twelve-story office building was erected, replacing Tyson’s meat market and the old Willow Tree Inn.  The corner was then held at two million dollars.  The property was bought in 1909 for one million nine hundred thousand dollars by the American Real Estate Company.”