On the 22d the Confederates moved to Charlestown and pushed well up to my position at Halltown.  Here for the next three days they skirmished with my videttes and infantry pickets, Emory and Cook receiving the main attention; but finding that they could make no impression, and judging it to be an auspicious time to intensify the scare in the North, on the 25th of August Early despatched Fitzhugh Lee’s cavalry to Williamsport, and moved all the rest of his army but Anderson’s infantry and McCausland’s cavalry to Kerneysville.  This same day there was sharp picket firing along the whole front of my infantry line, arising, as afterward ascertained, from a heavy demonstration by Anderson.  During this firing I sent Torbert, with Merritt’s and Wilson’s divisions, to Kerrteysville, whence he was to proceed toward Leetown and learn what had become of Fitz.  Lee.

About a mile from Leetown Torbert met a small force of Confederate cavalry, and soon after encountering it, stumbled on Breckenridge’s corps of infantry on the march, apparently heading for Shepherdstown.  The surprise was mutual, for Torbert expected to meet only the enemy’s cavalry, while the Confederate infantry column was anticipating an unobstructed march to the Potomac.  Torbert attacked with such vigor as at first to double up the head of Breckenridge’s corps and throw it into confusion, but when the Confederates realized that they were confronted only by cavalry, Early brought up the whole of the four infantry divisions engaged in his manoeuvre, and in a sharp attack pushed Torbert rapidly back.

All the advantages which Torbert had gained by surprising the enemy were nullified by this counter-attack, and he was obliged to withdraw Wilson’s division toward my right, to the neighborhood of Duffield’s Station, Merritt drawing back to the same point by way of the Shepherdstown ford.  Custer’s brigade becoming isolated after the fight while assisting the rear guard, was also obliged to retire, which it did to Shepherdstown and there halted, picketing the river to Antietam ford.

When Torbert reported to me the nature of his encounter, and that a part of Early’s infantry was marching to the north, while Fitzhugh Lee’s cavalry had gone toward Martinsburg, I thought that the Confederate general meditated crossing his cavalry into Maryland, so I sent Wilson by way of Harper’s Ferry to watch his movements from Boonesboro’, and at the same time directed Averell, who had reported from West Virginia some days before, to take post at Williamsport and hold the crossing there until he was driven away.  I also thought it possible that Early might cross the Potomac with his whole army, but the doubts of a movement like this outweighed the probabilities favoring it.  Nevertheless, to meet such a contingency I arranged to throw my army on his rear should the occasion arise, and deeming my position at Halltown the most advantageous in which to await developments, my infantry was retained there.