The Iowa Democratic caucus was supposed to provide some clarity for the very tight race for the Democratic nomination. Unfortunately, it spiraled into a tech-related disaster: The app meant to help caucus organizers collect and report the results, created by a small firm auspiciously named Shadow, melted down at a crucial moment. Days later, Iowa Democratic Party officials are still counting votes manually, trying to apply a 19th century solutions to fix a uniquely 21st century problem.
So what happened?
Speculation abounds, and we’re all left to poke and prod at what really went down. Here’s what we know, what the evidence shows, and what likely happened.
What We Know
As democratic voters in Iowa pledged support for candidates, Shadow’s app (slapped with the ultra-boring moniker IowaReporterApp) was provided to caucus leaders in all 1,765 precincts. It was supposed to make the process of collecting and sending results faster: caucus leaders could record and report results in real time, then provide those results in an accurate, secure way to Democratic Party leadership.
Heading into this caucus, party officials were reportedly terrified of outside interference or cyberattacks on the process. Instead of allowing external security experts and QA testers to pick at the app for bugs, they tried to protect their code by keeping it from as many people as possible—a “security through obscurity” tactic that generally doesn’t work very well.
On caucus night, things quickly devolved into chaos. Many caucus leaders claimed they had trouble reporting via the app. Meanwhile, the ‘old’ method of calling in results also failed to work; one leader simply posted their caucus results on Twitter after being on-hold with the Iowa Democratic Party for over an hour.
Iowa Democratic Party Chair Troy Price claimed that the app recorded the results correctly: "The bottom line is that we hit a stumbling block on the back end of the reporting of the data," he said. "But the one thing I want you to know: We know this data is accurate."
Gerard Niemira, the CEO of Shadow (again, probably a bad name for a company that deals specifically in handling app and data services for political parties) said: “I’m really disappointed that some of our technology created an issue that made the caucus difficult.” Just in case that came off as underplaying a disaster, he added:
“The app was sound and good. All the data that was produced by calculations performed by the app was correct. It did the job it was supposed to do, which is help precinct chairs in the field do the math correctly. The problem was caused by a bug in the code that transmits results data into the state party’s data warehouse.”
As it stands, tech failed democracy—but how?
The Speculation
One accusation supported by fact is that Shadow was only given a (roughly) two-month window to complete the app, suggesting caucus leaders were working with an MVP app more than finished product. Moreover, there are claims the app wasn’t tested at scale, instead using the caucus as its testing ground.
The Facts
Shadow’s app was built atop React Native, a cross-platform app-development framework. React Native has been intensely popular for years, despite attempts by Google and Apple to crack down on apps built using cross-platform tools. While React Native allows developers to build things quickly, taking an off-the-shelf framework and plugging in some elements can sometimes result in problems—for example, a software team might not take the latest APIs or security protocols for iOS or Android into account.
Caucus leaders were asked to sideload Shadow’s app, according to Vice. For Android, Shadow utilized TestFairy, but apparently didn’t even pay for deployment, instead opting for its free tier. This left Shadow and caucus leaders without services such as single sign-on authentication, unlimited data retention, and end-to-end encryption (the number of beta testers is also limited to 200).
For iOS, Shadow utilized TestFlight, which can support up to 10,000 external testers via a public link, or roughly 750 testers in an internal group (though the internal testing is actually limited to 25 members and 30 devices).
We don’t know how many of the 1,765 caucus leaders were actually using the app, or how many used iOS vs. Android. Many complained that they weren’t able to download the app.
Shadow claims the data collected was accurate and transmitting that data was faulty, which points to its proprietary Lightrail system. According to Shadow: “Lightrail is the universal adapter for political data. It’s a data integration and automation tool that makes it easier for users to get your organization’s data where it needs to go, and makes complicated workflows simple, repeatable, and scalable.”
Oops.
Shadow frames Lightrail as a user-friendly SQL database or CRM for automating actions such as texting backers or updating profile information. A core feature of Lightrail is its communication feature with databases, which is where the real hiccup seems to be, and why we may never truly get an answer from Shadow on what really went wrong.
(Shadow also has a messaging service for campaigns who want to reach out to voters directly. It’s probably a good thing that it didn’t start spamming random texts to caucus leaders, on top of everything else.)
According to Shadow’s LinkedIn profile, management of the entire tech stack was left to a CTO with limited experience and an intern, as well as two front-end developers. Is that a big enough team to handle a big, complicated project? That depends on the skills and experience of all involved, but a very tight deadline probably didn’t help—the Iowa Democratic Party paid Shadow $63,000 in late 2019, according to the Chicago Tribune, proving the team had just a few months.
None of Shadow’s staff are versed or experienced in native app development, and the firm’s job posting for a contract front-end engineer asks for experience in React, Python, Node.js, Typescript, as well as services such as Kubernetes, Firebase, and Google Cloud Platform. That’s a lot to put on a single technologist, no matter how long their tenure in the technology industry.
So, What (Probably) Happened in Iowa?
At the most basic level: A small crew of fairly inexperienced developers, no testing, short delivery window, and mostly-untested proprietary tech—likely communicating with other untested technology. Those are the ingredients in a recipe for app disaster.
We have to wonder if Lightrail was communicating with a cloud storage system such as Firebase, and simply failed, or if Lightrail was unable to connect properly to the Iowa Democratic Party’s own database. It’s also possible the Iowa Democrats had a faulty service that couldn’t handle the volume of data being thrown at it; if all precincts were reporting at or around the same time, it could have been a server-load issue.
Shadow only says there was a “coding issue” with its software, which hints that the firm’s relatively inexperienced staffers were working with a database service they had little or no experience with. If the results are accurate, that tells us the apps collected and recorded data just fine—sending it off-device was the issue.
It’s a nightmare scenario for any tech company. Unfortunately for Shadow, it all happened under the bright lights of a hotly contested and eagerly anticipated caucus.
