Ball’s Bluff and the Fall of Charles Stone

Second in a series

When Union forces tumbled into the Potomac River on the evening of October 21, 1861, following their rout at Ball’s Bluff, the disaster was just beginning.

The ripples from that plunge would be felt all the way in Washington, dozens of miles downriver. One could argue, in fact, that those ripples would be felt for the rest of the war.

Seventeen hundred Union soldiers had been routed, and more than half ended up killed, wounded, missing, or captured. Many of their bodies floated through the capital. Among the dead: a U.S. Senator, Colonel Edward Baker of Oregon, one of Abraham Lincoln’s best friends.

Congress formed the Joint Committee on the Conduct of the War—an entity that would overshadow all Union military efforts until the end of the war—and one of their first orders of business was to figure out why Ball’s Bluff had been such a disaster.

“There were scapegoats,” says James Morgan.

Morgan, author of A Little Short of Boats: The Civil War Battles of Ball’s Bluff and Edwards Ferry, October 21–22, 1861, says the formation of the Committee was the single most consequential result of the battle.

“The organization of the Joint Committee on the Conduct of the War came about six weeks after this battle—not specifically in reaction to this battle, but this battle was the last straw,” Morgan explains. “The Federals had lost at First Manassas in July, they had lost at Wilson’s Creek in Missouri in August, and they lost here in October—so three really significant defeats in a row. And all this time, they’re putting in all this effort. They’re trying to save the country. The army’s building up. They’ve got all this equipment and everything, but then they lose. What’s going on?”

Politicians had another reason to make a close interrogation of what happened at Ball’s Bluff, too. “Of course you had a senator killed here,” Morgan says, “and the defeat was really his fault more than anybody else’s—and his friends in the Senate didn’t want to hear that.”

And so, the battle that began at Ball’s Bluff turned into a witchhunt against the Federal commander, Charles Stone.

“I have a lot of respect for Stone,” Morgan says. “Stone was a seriously good soldier. High-ranking West Point graduate. Excellent Mexican War service record. Very highly thought of by his professional peers. Everybody who worked with him thought highly of him. Phil Kearny, who was a serious soldier himself, called Stone ‘the ablest man in the army.’”

Ironically, Stone wasn’t even on the field at Ball’s Bluff when the fight broke out or his troops got routed. He sent Baker to take charge while he stayed at headquarters and tried to coordinate between the raiding party at the bluff, the diversion that supported it at Edward’s Ferry, and the movements of a division of Union soldiers in Dranesville that he thought—incorrectly, as it turned out—might be moving in support of him. Oh, and he had to keep the lines of communication open with army commander Major General George B. McClellan.

“Put yourself in his position,” Morgan suggests: “You had this little accidental battle start. It was a reconnaissance, but the guys who came over, they bump into some Confederates, send in a few more troops. You say, ‘Baker, okay. I put him in charge down there. Send in some more guys. Not a problem. Maybe we’ve got an opportunity there. Maybe something will develop.’ So he starts re-positioning his resources, crosses a bunch of other troops at Edwards Ferry in case he needs them.”

Stone also thinks he has 12,000 troops just down the road in Dranesville to support him. “And if McClellan says ‘Go,’ then not only will he have his guys from Ball’s Bluff and from Edward’s Ferry, he’ll have these other guys who will come from Dranesville.” Morgan explains. “It would’ve just taken advantage of an opportunity—but it was a great opportunity.”

Confederates, meanwhile, had been expecting something to happen. Stone’s movement, accidental as it was, more or less mirrored what the Confederates had expected. “Would’ve done so accidentally and unintentionally—but it would have mirrored that,” Morgan says.

Stone had no way of knowing whether the Confederates were in position to counter the Federal moves. So, when he reported to McClellan in the early afternoon, Stone told his commander that everything is fine. “‘Yeah, everything’s good,’” Morgan paraphrases. “‘We’re a little short of boats. We hadn’t really planned on doing this, but we can take advantage. We can adapt.’ He’s a West-Pointer. He’s trained. He understands this kind of stuff.

“At the time he said that, that was around midday, 1:00 in the afternoon, so there really wasn’t any problem yet,” Morgan says. “But when you read these messages back and forth between General Stone and General McClellan, there’s no sense of urgency because it’s not really a problem. He’s at Edward’s Ferry. Baker’s down at Ball’s Bluff and apparently doing okay. I don’t think he was really trying to understate a problem. I think he was just letting McClellan know this opportunity had arisen. ‘We’re doing what we can. We’re a little short of boats, but, y’know, not to worry, we’ll take care of it.’”

But things for Baker were not going okay. He mismanaged the placement of his own troops, leaving them pinned against the river. It led to a rout. Baker himself was shot down. Chaos ensued. Things got ugly. Stone, behind the lines, did not know things were unraveling until after they had unraveled.

“McClellan, to his credit, initially tried to help General Stone—because people immediately went after Stone,” Morgan says. “He was the commanding officer. Even though he wasn’t even here, a lot of people went after him.”

McClellan went to Ball’s Bluff three days after the battle and conducted his own investigation. “And his report basically said the defeat was not the fault of General Stone but of the commander on the field, Senator Baker,” Morgan says. “And McClellan was right. But that didn’t endear McClellan to the people in the Senate who were the hardcore abolitionists who didn’t much like him anyway because he was not a hardcore abolitionist. So, the politics got all tangled up, which is what messed up General Stone moreso than the battle.”

The subsequent investigation—conducted with no real due process and driven by political agenda—led to Stone’s fall from grace. He was imprisoned for six months. McClellan and Joe Hooker both asked for Stone’s reinstatement but were denied by Secretary of War Stanton. By 1863, the Committee finally released Stone. He returned to service under Nathaniel Banks, serving creditably for ten months as Banks’ chief of staff in Louisiana.

“It completely destroyed the career of a man who was a rising star and probably would have had at least a chance to exercise some kind of high command—but he’s destroyed. Virtually destroyed,” Morgan says.

After the war, Stone served with distinction in the Egyptian army and, after his return to America, served as chief engineer for the construction of the Statue of Liberty. Following his death, General-in-Chief of the Army, William T. Sherman, served as one of his pallbearers. Stone was laid to rest, with full military honors, at West Point.

While not a terrible ending, it’s not the ending Stone might’ve had had the Committee not come knocking on his door after Ball’s Bluff.

“The ripples are huge,” Morgan says. “That’s the reason why Grant, when he becomes commander of the army in 1864, doesn’t establish an office in Washington and operate out of it. He goes with the Army of the Potomac precisely because he doesn’t want to be back there in Washington where he’s got all these people, bureaucrats in Congress, looking over his shoulder and fussing at him.”

Stone, by contrast, became no more than a footnote because of Ball’s Bluff, Morgan says. “If it hadn’t been for this, Stone could be the name we think of instead of Grant,” he suggests. “We don’t know that for sure, of course. But Grant himself, years later, said that he thought early in the war, when he was just an obscure colonel, that Stone might likely be one of the two or three guys in the army to rise to the highest levels of command. He knew him from West Point. They overlapped by two years. Stone was two years his junior. Grant respected Stone. Very high opinion.”