The Worst Civil War Map You’ll Ever See: Asking AI About the 1862 New Mexico Campaign

For all the talk of how AI will revolutionize society, or start wars, or maybe end the world entirely, the way that the most people are using it right now is also one of the most banal: Answer quick questions about basic facts, the same way many of us have long used Google or Wikipedia. 

Trying to remember if Antietam was fought on September 17 or September 18? How many casualties were there at Gettysburg? Who was in command of the armies at the battle of Stones’ River? Just ask ChatGPT, or a similar AI-powered chatbot.

And for the big, well-documented battles and campaigns, with vast quantities of material online for these models to train off of, you’ll probably get pretty decent answers.

But how useful is AI as a research tool, especially for a more obscure campaign like Sibley’s 1862 invasion of New Mexico? We’ll start with some easy questions:
That’s a pretty good top-line summary, and helpfully provides good, reliable sources. It does incorrectly imply that Canby was at Fort Union and the battle of Glorieta Pass. A similar answer about the battle of Valverde, while unsourced, was also reasonably accurate, but asking for more details about Glorieta gave AI a lot more trouble:

Glorieta is unusual for being fought on March 26 and 28, with no action in between on March 27. That nuance, and the Confederate setbacks on March 26, are conspicuously missing here, along with any explanation for why Union troops were in the pass to begin with. 

There’s also a strange inference at the very end: Glorieta Pass is called the “Gettysburg of the West” for being the highwater mark of Confederate efforts in the region, not because of a captured supply train. 

Those are errors of omission and interpretation, so let’s try to get back to easier ground – who commanded each side at the major battles of the campaign?

Where the chart is cut off in the screenshot, AI is accurately noting that “Union forces were commanded by Col. John P. Slough and Maj. John Chivington.” However, this response incorrectly places Tom Green at Glorieta Pass, and simply calling Sibley “ill” is papering over some rampant alcohol abuse. Like the previous questions and answers, it’s in the right ballpark, but we have some odd omissions and errors.

Now let’s get into some trickier territory, and ask for interpretation and analysis – specifically, why was the Union successful in the campaign?

In my view, this conclusion is actually rock solid, and the longer version of the answer was backed up by a lengthy explanation that hits the key reasons why Sibley was defeated. I’m pleasantly surprised here – the earlier struggles with basic facts, like which commanders were present at which battles, didn’t give me much hope for good analysis. And while there’s points I’d quibble with or add to, this is better than a lot of what I see on, say, social media accounts about the Civil War.

Interestingly, only the first answer in the exchange included sources. That means there’s no easy opportunity to dig in further myself, or to challenge the interpretation here. If I push, what sources does AI recommend if I want to do more research myself? I was steered toward Don Frazier’s Blood and Treasure and Martin Hardwick Hall’s Sibley’s New Mexico Campaign, along with some additional accounts, biographies and primary sources.

Maybe I should be peeved that my own book doesn’t make the cut, but Frazier and Hall are fantastic starting points if you want to get into this campaign. The recommendations go on to suggest turning the Official Records and other primary sources, “If you want to go from ‘reading history’ to actually doing historical research, I’d add these.”

On balance, so far this is going much better than I expected, especially in terms of analysis of the campaign. AI is putting up a middling performance on specific facts, in ways that may or may not get you into trouble, depending on why you’re doing this research. You would probably still be better off with reliable sources like the National Park Service, Emerging Civil War’s many posts on the subject, or even Wikipedia, but this will at least point a casual student of the war in the right direction.

But we aren’t done yet, folks. Buckle up. 

If I’ve learned one thing about Emerging Civil War readers, it’s that we all share a love of maps. And having just gotten a series of pretty good answers about the campaign, I thought AI would spit out something at least resembling reality.

Instead, I got this abomination:

I’ve been sitting with this for a few minutes, and I’m still not sure where to begin. The dates and names of the battles are correct. The geographic landmarks exist, but not particularly close to how they’re laid out here, and the distances are remarkably inaccurate. Apparently you have to go to Santa Fe on your way to Santa Fe? Maybe there’s something philosophical there.

The closer you look, the less sense it makes. Colonel James Carleton is an actual Union officer, commanding the California Column in the region, but he was nowhere near Valverde. Grierson, Sterling Price, Henry Lane and SD Lee are real officers as well, but none of them came within hundreds of miles of New Mexico during this campaign, if ever. The insets summarizing the battles have no bearing whatsoever on actual events.

This has to be the worst Civil War map that I’ve ever seen. And coming on the heels of good answers from the AI, a casual student of the war would easily think this is summarizing the same information with the same level of accuracy, when that couldn’t be further from the truth. Even worse, this is the kind of visual content that thrives on social media these days.

But the real question, that no chatbot can answer for us, is: Why do we study the Civil War? Depending on who you ask, it can be a fun hobby or a serious profession. For some, it’s an effort to understand our country and our world more clearly. And increasingly, maybe we need to do it so that we can put actual information out there, to counter the pervasive AI slop.

Part of a series.