“These are the times that try men’s souls.” —Thomas Paine, The American Crisis
Many of you, reading this blog, weathered the systemic disruptions of the dot-com bubble burst and the great financial crisis. The current environment is fundamentally different. While scientists had previously identified the possibility of such a pandemic, a 2020 business plan could not with certainty have “planned” for the speed and scale of the economic disruption we are experiencing.
In this time of uncertainty, when the extent and timing of COVID-19 impacts is unknown, rapid and frequent decision making is critical to crisis management. Business leaders have to take action in the fight against COVID-19, to protect their people and their organizations, and to prepare for the future. There is no precedent or consensus on the path forward, but here’s a proposal:
- Focus on What Matters Right Now
- Make Better Decisions, Faster
- Prepare for What Comes Next
What Matters Right Now
What matters right now is protecting the population and protecting the nation. And companies that have a role to play must step up to that challenge.
At the beginning of the lockdown, we received a letter from one of our customers, a Fortune 100 food manufacturing company. StreamSets helps this customer ensure continuous operations of their data supply chain in support of the food manufacturing supply chain, and ultimately put food on the shelves of your local grocery store. Because of the critical role StreamSets plays in helping this company maintain the nation’s food supply, we have been designated as essential critical infrastructure by the U.S. Department of Homeland Security.
The COVID-19 Open Research Dataset consists of more than 29,000 machine-readable coronavirus articles ripe for analysis. When a “COVID-19 task force” formed to address the data integration challenges of tracking and responding to the crisis, StreamSets dedicated engineers and technology to the cause. This consortium of data analytics technology partners and global systems integrators are building solutions that enable coronavirus test scheduling and administration, and data pipelines to integrate state and local government data with census, demographic, weather and other open data sets to enable better response planning. Now is the time for epidemiologists, physicians, and data scientists to analyze and learn from this information and shorten this crisis. We are proud to be of help, and do not take these responsibilities lightly.
Make Better Decisions Faster
In recent research, Gartner found that 81% of decision makers believe that the quality of their decisions could be equally as good with less spend, and 81% believe they could have made the same or higher-quality decisions in less time. (Gartner) We no longer have weeks or months to plan. We are making decisions in days or even hours.
At StreamSets, we have been applying DataOps principles since our inception. DataOps is the set of technologies and practices that ensures that the latest data is continuously available for decision making, even in the face of change. Knowing that your data is keeping up with the unprecedented pace and scale of change gives you confidence to make better decisions fast.
Our Head of DataOps reports directly to me, and is responsible for understanding the needs of our business leaders, building data pipelines for application and data integration, creating dashboards, and enabling self-service for advanced analytics. But most importantly, our environment has been designed for resiliency. I can focus on the most important data points to make better decisions confidently, and reduce the time it takes to get that analysis.
This Too Shall Pass
I am confident that, in the coming months, tests will become broadly available, we will figure out an immunization strategy, our children will go back to school, and we will go back to work. But the world will look different on the other side. Companies that will thrive in the post-coronavirus world are the ones that do what it takes to survive, and instead of settling for good enough, they take this opportunity to do things right.
For our part, StreamSets was founded on the belief that the next wave in data integration needed to focus on both speed and confidence in the face of data drift. Organizations that modernize data integration and adopt a DataOps mindset during this pandemic will ultimately emerge stronger, faster and better able to respond to the next challenge or crisis.
Conclusion
So here we are today, making real-time decisions in the midst of so much uncertainty. As CEO of StreamSets, I am focusing my attention on helping the global response to the pandemic, continuing to serve our customers, improving our decision making abilities, and applying it to things in our control.
In the midst of upheaval 2 centuries ago, Thomas Paine reminded us that the “harder the conflict, the more glorious the triumph.” He was right.