The 2020 election was unique in more ways than not, and will forever change the expectations and understanding of how a successful presidential campaign operates. Even barring the fact that we faced the most divisive and irresponsible incumbent in the general election, I have no doubt the process we followed to our ultimate victory will be the roadmap for how to win elections in the future. Everything has been reshaped by this campaign — from delegate strategy to travel, from online fundraising to field organizing, from research to digital, and everything in between.
Tech was a continuum across all dimensions of the campaign and played a critical role in how we architected the movements of the campaign. In this article, I plan to discuss a few of the key ways I see tech in national politics shaping out in the future. What I think to be the most critical piece in all of this is that the role of chief technologist on the campaign reports directly to the campaign manager. On Biden for President, my role as CTO did exactly that, and I cannot imagine a future where that isn’t just the expectation. Everything during the campaign is immediate priority, and making sure there was a channel for important technology decisions to share precedence was immensely important to our success.
Cybersecurity is Everything
When I’m asked to reflect on my role with the campaign, I tell people one of the ways it was unique was that it was a 300% job. For the entirety of the primary (and the period where we’d cleared the primary field but it wasn’t yet the general), I filled the roles of CTO, CIO, and CISO. One hundred percent of my time went to all three roles, but most days I was also 150% focused on cybersecurity.
The fallout from the 2016 hacks had most staffers on the campaign hyper-aware of what would happen if they didn’t follow good cyber hygiene, so we didn’t have a lot of resistance to getting most of the basics in place. It’s important in this environment to remember that every additional bit of complexity introduces a potential risk to the campaign’s cybersecurity posture, and needs to be thoroughly interrogated before it is implemented. It’s also important to remember that the opposition is not just script kiddies hoping to score admin access to your Wordpress — it’s also the world’s most sophisticated hackers from nation-states across the world.
Arguably the most important thing we did on the campaign was very early on to establish a cybersecurity checklist for any vendor we onboarded. The core threat to the campaign is a “supply chain” attack, where a vendor we work with is compromised and therefore our data is as well. We made it a priority that every vendor we worked with that would be handling our data (regardless of the classification of the data) had to be approved from a cybersecurity standpoint. From conversations I had with industry experts throughout the campaign, I think the prevailing mindset is that cyber-vetting of vendors is more “security theater” than practical cybersecurity, but I cannot tell you how many times going through the 18-point questionnaire saved us from working with easily compromised vendors. I had the support of legal, accounting, and campaign leadership to carry out this process, and this is something that will be critical to the future as well.
Beyond cyber-vetting, there were extensive efforts overall to limit moving parts and keep our technology architecture confined to a realm we could reason about easily between a few of us on the team. What that means is that first and foremost we did not have an on-premise footprint for anything. It was very important that we build a truly zero-trust environment, where the vast majority of the business of the campaign was able to be conducted in GSuite or AWS. This didn’t come without its challenges. For example, we resisted the urge repeatedly to deploy an Active Directory environment on-premise, which probably would have helped us immensely with group policy and making the printers work. Ultimately we felt like deploying AD would have introduced additional complexity and made HQ a “trusted” environment, which we very much did not want. (Our approach to building a zero-trust environment is arguably the only way we were able to move the entire campaign to fully remote at the beginning of quarantine.)
There’s a lot we could talk about with cybersecurity on the campaign. For example: deploying security keys, enforcing 2FA everywhere, how we secured edge APIs, how we managed monitoring and alerting, how we handled incident response, and so forth, and maybe I’ll do a more detailed write-up in the future just on that subject. But the last point I want to hit on is cybersecurity education.
Users need to know why they’re being asked to do something, so one of the biggest recurring initiatives we had was a monthly series on cybersecurity awareness. Every month, we would choose a topic to highlight, we’d do a write-up explaining the subject, and we’d share it out with the entire campaign. Topics ranged from good email hygiene and sharing documents to raising awareness about locking your laptop when you step away. I think getting out the message about cybersecurity on a recurring basis was two-fold important. Firstly, we actually wanted to highlight these issues, but secondarily it made sure that everyone on the campaign knew there were resources within the campaign focused on this and they could come and find us for any question or concern.
The last thing I’ll say on cybersecurity is the fact that we had no breaches, leaks, hacks, or anything of the sort throughout the entire cycle is a point of incredible pride for me. These short few paragraphs do not do justice to the amount of attention and effort all of us on the tech team gave to making sure this election was focused on the candidate and not on a data leak.
Remote First
Campaigns of the future may go back to the old ways of working, where a thousand people cram into an office building and run 18-hour days, but tech teams will never be able to go back to that. The simple fact is the tech talent that is needed to run a safe technology footprint is distributed across the country, and 2020 was the model for how to make a remote team work.
When we started out, I obviously did not know that the coronavirus pandemic would force the entire organization to fully remote, but it was a personal goal of mine to hire remote team members onto the tech team. I also knew the extensive challenges of having a partially on-site and partially remote team. Communication is the biggest one that comes to mind, and there must be some forcing function that makes communicating on remote-friendly mediums the desirable path for the non-remote team.
A quick note on the hybrid team: arguably one of the greatest facets of working on a campaign is the camaraderie you build with your co-workers as you work relentlessly to change the world. The atmosphere and vibe of being in HQ (especially when you’re winning) is irreplaceable and was one of the greatest feelings I’ve had in my professional life. People are going to opt to work in the office, that’s just how it’s going to be. So, it can be expected that the tech teams of the future will be some hybrid of on-site and remote, because that’s exactly what we made work in 2020.
I decided from the very beginning that I would hire the campaign Engineering Director as a fully remote position. They would have engineers reporting to them who may or may not be on site at HQ, and so we needed to find a way to make it work. I hired Matt Hodges in July 2019 to fill that role, and I have no doubt that his implementation of the role will be the prototype for every remote leader on a campaign in the future.
We did a couple of things to make sure the remote/on-site team split worked for everybody:
- We quickly became over communicators, and on a variety of mediums. For reasons not worth getting into beyond “it was expensive”, we didn’t have a campaign Slack for most of the cycle, which made it difficult to communicate quickly and directly with people outside the tech team. On the tech team, however, we used Keybase as a stand-in, and created a set of shared chats between the different teams. This worked pretty well, and we got to a point where even when we were sitting right next to each other we would talk on chat as well.
- We set up a “perpetual” Google Hangout and we had a TV at the end of the row where the tech team sat. Matt could easily drop-in at any time to just be a part of the environment and participate in side-bar conversations that maybe didn’t raise to the level of chat (e.g. where are we going to get lunch?).
- Daily stand-ups and tracking work items in GitHub. We created project boards for everything we were doing and wrote down even the most inconsequential task so that we could track it. Anybody on the team could quickly and easily get a grasp of what was going on, who was working on what and so forth. This helped a lot as we had conversations with people outside the tech team.
- Bringing a laptop or a phone to unplanned events and making sure anybody remote could be dialed in to participate. When the boss would visit the office, there would often be a group huddle in the middle of HQ, and I wanted to make sure anybody who wasn’t on-site that day could participate. I’d often just FaceTime or setup a Google Hangout and hold my phone up so they could see and hear everything that was going on.
Lastly, I took it as a point of personal responsibility to make sure Matt had all the information he could handle to guide the direction of work for the engineering team. This meant that often my days would start with a phone call to Matt (sometime around 7am) and end with more of the same (sometime around 9pm). Ultimately, putting in this little bit of extra effort helped us stay extremely connected as events of the campaign transpired and changed, and it never felt to me like “extra work”.
Campaign tech teams of the future are going to be remote-first and our execution on this strategy in 2020 is how we can expect the model to work in the future.
Tech, Research, and Comms in Lock Step
I can’t emphasize enough how during the 2020 cycle it was so important that the research and comms teams had a direct line of communication with the tech team, and that their priorities were our priorities. There are a lot of specifics to the work between the three teams, but it is suffice it to say that some of the biggest challenges we face on the technology front now and in the future has to do with how the opposition disseminates dis- and mis-information.
The 2020 election cycle was littered with misinformation for months on end, and we were in the unfortunate position that we had to combat it on every front. Although the platforms are now taking some responsibility for how they contributed to the problem, that was most certainly not the case throughout the cycle.
Where the tech team contributed the most was in helping research and comms get at the answers they needed very quickly. We built a system for collating articles and transcribing and indexing videos that could be quickly and easily searched. The simple fact was that by the time we disproved one conspiracy theory from the right-wing, they were already on to promoting another one, and we needed to be one step ahead of them at all time.
The system we were able to build made it so that files that would have once been in a shared folder or on one person’s laptop were sorted and searchable through a Google-esque interface. We even took it a step farther and used AWS’s machine learning services to extract and index entities from bodies of text, which gave the research team even more granular controls in how they found the resources they needed. There’s probably room for an entire conversation on how we built this system that gave such good perspective on things, but for now I think it’s enough to say that we spent a lot of our time building a product for research and comms that fit their needs specifically.
Having a world class rapid response comms and research team is ultimately, I think, what got us through this election cycle. In practical terms, political tech of the future will need to have a similar always-on communication channel with these teams. It is more critical than I can emphasize in words. What it means for political tech leadership in the future is there will be a lot of interrupt-driven work alongside the regular day-to-day work, and it’s important to find a way to balance these things.
Microservices
Every bit of advice that was given to me coming onto the job is that you should not focus on building software, you should focus on building the glue that ties together vendors. That’s a good idealistic mentality, but the truth is there are a lot of things that don’t exist in the world, are not well suited for the campaign, don’t conform to the cybersecurity standards, or so forth. We ended up writing a lot of software that was more than just glue. But, it’s important to say up front that this wasn’t innovation simply for the sake of innovation — we built software because there was a very real need for us to build it.
In practice, what this means is we were the first campaign to build a microservice architecture. But we won’t be the last. The reality is the business requirements of the campaign change and evolve on a day-to-day basis, and much of the vendor space that exists is built on an understanding on things that have already happened. While we were able to leverage vendors for most of the critical and important aspects of the campaign, there were too many areas where a vendor solution didn’t work. The other piece to it was that we could actually build a solution for a lot cheaper than what a vendor could offer.
Advice that I would give to political tech teams of the future is to expect that you’re going to build and deploy software. Get your systems and infrastructure setup from the outset. Have a plan up front for things like user authentication, API security, and UI deployments. If I could change one thing about how we approached engineering on the 2020 campaign, I would have acknowledged the spirit of “building the glue” and would have set the environment up for the realization that we, in fact, will be building a microservice architecture.
As a second point of personal privilege in this article, I take a great deal of pride in the fact that we were able to build a robust microservice footprint, enabled by a high velocity team, with quickly changing requirements, and a lot of interrupt-driven work. And we did it all while still writing tests, documentation, and following CI/CD best practices (mostly).
Conclusion
In the future, we should expect more of the same for what we did in 2020. The way we approached every aspect of tech’s role in the entirety of the campaign is the model for how it should be going forward. A strong emphasis on cybersecurity, a channel to build a remote-first tech team, and an acknowledgement of the expectations for how software is built on a campaign are now the basis for how to build a successful tech operation.
It is important that I say this was (obviously) by no means a one person operation. There was a team of incredibly talented and motivated people who worked tirelessly over the course of the entire campaign to make sure that Joe Biden was elected president.
I don’t by any means take credit for the plethora of work that was done here, but I do want to highlight one thing I think I brought to the table, which was deep technical knowledge. I am an engineer before I am a diplomat, and I don’t think you’d find many people who would argue with that statement. What this means in practical terms is having the benefit of experience and a developed intuition about technology to know quickly whether a solution is going to work or not.
I think it’s important for political technology leaders of the future to bring that deeply technical experience to the job every single day. The cycle moves too quickly and the landscape changes too fast to not have someone in that leadership position who can deeply understand the mechanics of the technology architecture.
I could honestly write an entire book on tech in the 2020 election, and it still would probably only scratch the surface on certain subjects. I think it’s enough to say here in conclusion that we found the sweet spot with tech in politics, and the fact that tech was never the story is a testament to our success.
