Pages

Showing posts with label data silos. Show all posts
Showing posts with label data silos. Show all posts

Friday, 25 June 2021

The Nature of Measurement

Curiosity 

I have always been curious.  If curiosity killed the cat - I have been dead a long time! I now think that curiosity is linked to a desire to learn and I have been a proud life long learner.  It was with some excitement that a few months ago I signed up for Curiosity Stream as part of my Prime firestick subscription. Curiosity Stream is an online streaming service that specializes in documentaries from around the world.  This past week I was watching a BBC commissioned series on the history of measurement called Precision: The Measure of All Things.  It is a fascinating three part series on how we have developed greater precision in our core measurements and the tremendous impact such increasing precision has had on economic and social development. At one point the host says; "Without measurement, there is chaos."  It stopped me cold and may become the motto of this blog.  

Without measurement there is chaos.

What is so incredible about this thought is how true it is.  We need standards of measurement in all facets of life to be able to guide our activities, produce consistent outputs and contribute to outcomes that have impact.  Imagine trying to follow a recipe if we did not have standard measurements?  The recipe would not come out at all if we don't have consistent and standard messages. 

In Chaos there is Measurement

I shared this measurement quote with a colleague, who is a specialist in public sector performance measurement reporting, to get the reply; "In government it is the opposite: in chaos there is measurement!" How true in so many cases.  In the public sector we use a plethora of measurements to report on what we do.  Government is expected to be accountable and transparent - yet the measurement systems we use are anything but.  When we go to evaluate government programs and policies, how often does the report read that there is insufficient evidence or data to support any conclusions about the effectiveness or efficiency of an initiative?

Data, data where's the data?

The fact is that government is awash in data - it exists everywhere and is growing at an exponential rate. Government lags other sectors in the economy in understanding and analysing that data. Furthermore, because analysing data is sometimes hard to do, we tend to focus on analysing the data that is readily available and hope that will suffice.  Government is amazingly good at measuring outputs - the stuff we produce - but has greater difficulty measuring the impact on society and stakeholders an intervention is designed to achieve.  We need to do a better job and move from "in chaos there is measurement" to mastering the chaos through effective measurement and data analytics.

Feel free to contact me by email or follow me on LinkedIn.


Friday, 18 June 2021

Challenge 1: Data is Hard To Find

 Ever had one of those days....

The other day I was looking for a report that had impressed me about a year ago.  I knew that I had downloaded it and kept it but I wasn't sure where to find it. I ended up doing searches through my computer hard drive, my external hard drive and my cloud storage.  I found it after about 10 minutes of work, but the experience reminded me of some statistics I have read about how much time employees spend looking for data. More recently the increasing use of datasets on cloud storage has created a whole new level of data access challenges.  I recently discovered that my data analytics tool could not connect to a Databricks cloud storage because the enterprise VPN would not let the connection go through.  I had to disconnect from the VPN and then connect to the data store through plain Internet.

This comes from a 2019 Forbes article: 

Numerous studies of "knowledge worker" productivity have shown that we spend too much time gathering information instead of analyzing it. In 2001, IDC published its venerable white paper, "The High Cost of Not Finding Information," noting that knowledge workers were spending two and a half hours a day searching for information.

Since then, we have seen the rise of the cloud, ubiquitous computing, connectivity and everything else that was science fiction when we were kids becoming a reality — including the imminent emergence of AI. Yet in 2012, a decade after the IDC report, a study conducted by McKinsey found that knowledge workers still spend 19% of their time searching for and gathering information, and a 2018 IDC study found that "data professionals are losing 50% of their time every week" — 30% searching for, governing and preparing data plus 20% duplicating work.

If approximately twenty percent of time our working time is spent searching for and gathering information that translates into one day out of five.  If you sum up the total compensation cost for your organization and take 20% of that total - that is the financial investment you (and your organization) are making to find and discover information. We need to do better than that.  

The causes for this include data silos - data being held by one office or individual with limited access by others.  Other causes are lack of integrated data inventories - we don't even know what we have so we cannot find it. Multiple data stores where we may have different information in different places.  Difficult to use document management systems.  It is great to have an enterprise document management system, but if the user interface and the document storage structure is too complicated - it takes forever to find a relevant document. Cloud data adds a whole other level of data silos. 

So what's the solution? Here are some things we can do:
  1. Recognize that we have a data silo issue.
  2. Evaluate the cost of finding data - check with your team members on their experiences in locating data. 
  3. Identify the key data bottlenecks - where is it the hardest to find and retrieve data.
  4. What data governance issues like metadata, data inventories, search tools are necessary to resolve the bottleneck?
  5. Do it, fix it, try it:  change something, have a test or trial to see what works and then deploy. Often it is a small change that can make a big difference. 
Ultimately, data is one of our most strategic assets.  Helping our teams and analysts get to the data and documents they need quicker is going to help everyone do their job better.  We can do this!  Ultimately the fixes are not technical - often they are about better use of what we already have. 

What has been your experience in data bottlenecks and data silos?  What solutions have you seen work?  Feel free to share best practices in the comment section.

Feel free to contact me or connect with me on LinkedIn if you want more information on solutions and options. 

Data Literacy for Leaders

One of the key process blockages in undertaking digital transformation is data literacy. A particular challenge is understanding the needs o...