Driving Data Driven Decisions

So in the spirit of transparency; I snore.  Apparently, I have snored for a long time.  My snoring was the cause of some deep relationship issues in previous lives!  About 20 years ago I went to a sleep lab and started using a CPAP (continuous positive airway pressure) machine that keeps my throat open as I sleep and lowers the incidence of snoring and related medical side effects tremendously.  It has improved my personal relationships remarkably as well.  A positive impact any way you look at it!

Recently, I came to the conclusion that I need a new sleep machine.  Mine is now over 5 years old and the progress in sleep technology is amazing.  About a week ago I called my sleep machine provider and we met to review my needs.  The Ontario Government provides grants to offset the cost of these machines, but I need a prescription from the sleep lab to get a new machine.  

In the past, I would have had to book an overnight visit to the sleep clinic where they would monitor my sleep through several dozen electrodes and provide me with a diagnosis on my sleep patterns and behaviour.  It often took several months to get an appointment. With COVID there are no overnight sleep tests being done.  Instead they are using advance digital technology. This is where the data part comes in.

It seems that the current generation of sleep machines now have "smart" technology. The CPAP units now have a built in microchip that tracks your sleep patterns, an ability to automatically monitor and adjust air pressures and have a wireless cell phone chip to allow remote access to the data.  The machine has sufficient onboard memory to record your sleep data over multiple nights. 

To get my new sleep machine, I will be loaned one of these "smart" CPAP machines, use it for two weeks, have the data remotely accessed, analysed by the sleep lab and a prescription given for a new upgraded machine.  In other words, the prescription decision is being driven by remote data gathering without the need for an inconvenient overnight lab visit.  Furthermore, that data is aggregated into longitudinal studies on sleep patterns experienced by patients over time, contributing to better diagnosis and better health outcomes. 

The lesson here is that processes that used to take high levels of dedicated infrastructure and personal inconvenience, have been completely transformed through data and digital technology.  The ability to contribute to better health outcomes at the individual patient level has major implications for enhancing health in general.  How many processes do we interact with on a daily basis that could be enhanced through data collection and analysis that would contribute to more positive impacts?  What processes are we working with today that we need to examine and improve with data driven decisions?  

Next steps: look around your world and find three areas in your life and work that could benefit from better data analytics and contribute to better decisions and impact. 

____________________________________-

Feel free to contact me by email or follow me on LinkedIn.  For further information on data driven impact and data driven government, visit my website at www.datadrivengovernment.ca.

Challenge 1: Data is Hard To Find

 Ever had one of those days....

The other day I was looking for a report that had impressed me about a year ago.  I knew that I had downloaded it and kept it but I wasn't sure where to find it. I ended up doing searches through my computer hard drive, my external hard drive and my cloud storage.  I found it after about 10 minutes of work, but the experience reminded me of some statistics I have read about how much time employees spend looking for data. More recently the increasing use of datasets on cloud storage has created a whole new level of data access challenges.  I recently discovered that my data analytics tool could not connect to a Databricks cloud storage because the enterprise VPN would not let the connection go through.  I had to disconnect from the VPN and then connect to the data store through plain Internet.

This comes from a 2019 Forbes article: 

Numerous studies of "knowledge worker" productivity have shown that we spend too much time gathering information instead of analyzing it. In 2001, IDC published its venerable white paper, "The High Cost of Not Finding Information," noting that knowledge workers were spending two and a half hours a day searching for information.

Since then, we have seen the rise of the cloud, ubiquitous computing, connectivity and everything else that was science fiction when we were kids becoming a reality — including the imminent emergence of AI. Yet in 2012, a decade after the IDC report, a study conducted by McKinsey found that knowledge workers still spend 19% of their time searching for and gathering information, and a 2018 IDC study found that "data professionals are losing 50% of their time every week" — 30% searching for, governing and preparing data plus 20% duplicating work.

If approximately twenty percent of time our working time is spent searching for and gathering information that translates into one day out of five.  If you sum up the total compensation cost for your organization and take 20% of that total - that is the financial investment you (and your organization) are making to find and discover information. We need to do better than that.  

The causes for this include data silos - data being held by one office or individual with limited access by others.  Other causes are lack of integrated data inventories - we don't even know what we have so we cannot find it. Multiple data stores where we may have different information in different places.  Difficult to use document management systems.  It is great to have an enterprise document management system, but if the user interface and the document storage structure is too complicated - it takes forever to find a relevant document. Cloud data adds a whole other level of data silos. 

So what's the solution? Here are some things we can do:

1. Recognize that we have a data silo issue.
2. Evaluate the cost of finding data - check with your team members on their experiences in locating data. 
3. Identify the key data bottlenecks - where is it the hardest to find and retrieve data.
4. What data governance issues like metadata, data inventories, search tools are necessary to resolve the bottleneck?
5. Do it, fix it, try it:  change something, have a test or trial to see what works and then deploy. Often it is a small change that can make a big difference. 

Ultimately, data is one of our most strategic assets.  Helping our teams and analysts get to the data and documents they need quicker is going to help everyone do their job better.  We can do this!  Ultimately the fixes are not technical - often they are about better use of what we already have. 

What has been your experience in data bottlenecks and data silos?  What solutions have you seen work?  Feel free to share best practices in the comment section.

Feel free to contact me or connect with me on LinkedIn if you want more information on solutions and options.