The life-cycle of good quality data

It’s not too weird to think about data like a tree. Once planted, a dataset keeps growing, taps into new information and, more importantly, its value also goes through different seasons.

Information analysts describe this as the “lifecycle” of data. The same dataset might be useful today and useless tomorrow, but then suddenly become useful again ten years later.

Like any other asset (both tangible and intangible), data can decay in value depending on the contextual circumstances of the business and the wider market, according to a recent paper by the International Valuation Standards Council (IVSC).

The IVSC explained that most data is deployed ultimately to inform decision-making. But sometimes a leader can’t wait for the perfect data. In times of urgency, limited data can be better than no data at all. But that’s no excuse for laziness. Even limited data needs to be high-quality.

An example used in the report was that to begin exploration in the oil fields in the North Sea in 2001, perfect data was not required. Even broad data that showed a fraction of the full picture was sufficient for CEOs of oil majors to plan how they might get the energy out of the ground.

“The initial expectations for oil fields in the North Sea were that only about 20-40% of oil would be recovered. Technological progress allowed those rates to be much higher, and for the reserves to be revised upwards. Initial estimates were doubly wrong: oil companies succeeded in extracting a higher proportion of much larger reserves of oil in place than had been initially expected,” said the IVSC report.

This is an interesting point by the IVSC. It implies that although data can decay in value over time, it never quite reaches zero.

Obviously, the most current data for North Sea oil reserves is the most valuable to investors and oil companies. But the first datasets ever captured about the oil fields still play a critical part in painting the full picture of the oil fields, how they have been exhausted over the intervening years and where the best locations for future drilling might be.

Like a deciduous tree, the early data had its Summer months, before slipping into Winter and then finally being reborn again with a new Spring to be useful once more. In other words, treating data as a snapshot in time can extend its value long after its “best by” date.

Another example floated by the IVSC was occurring in the interaction of artificial intelligence (AI) and data.

In the race to build robust AI systems, companies may not realise they have plenty of useful data lying around that could be mutually beneficial.  Even if your company isn’t dabbling in AI, other businesses that are would likely pay a pretty penny for access to that data.

Weather data, for instance, is routinely gathered by local governments and also by small companies and NGOs. At the government level, most of this kind of data is taxpayer-funded, so it is public by default. Who might be interested in this data? Retail companies, for example, may want to use it for teaching their AIs about all the factors that predict customer traffic in shopping malls – weather being one of these factors.

Autonomous car companies might also be interested in high-quality weather data to help train their vehicles to predict road conditions. There could be plenty of valuable information sitting idle in the computers of thousands of unsuspecting companies. Even if there’s no longer a use for it in your business doesn’t mean other firms can’t give it new life.

Of course, because AI tends to be fantastic at picking up patterns, mixing databases together might reveal that some data isn’t as valuable as its owners originally thought. A company could spend years building a database, only to find that it isn’t as good as an external database that was relatively easy to access.

Weather data gathered from satellites, for example, may be superseded by data from thousands of small wind turbines on rooftops or sun-hours data gathered from solar panels. Imagine if your company had spent millions putting satellites into space. All that effort for nothing!

Would that mean the satellite data has no value? Not at all. There may be another use for that data in the future. Different companies might come up with intriguing ways to put that data to work, and pay for it. Maybe your own company will discover new use cases.

Suddenly, green shoots could appear on the seemingly dead branches.

The key point here is that the value of data never drops to zero. So long as the data is high-quality, there will be a market for it somewhere, somewhen. Datasets that sit idle for years can be reinvigorated when a fresh idea pops into an engineer’s head or a joint venture is signed.

But don’t start collecting all kinds of data on the off-chance it might be valuable someday. That would definitely be a waste of resources. Having data doesn’t automatically make a company valuable. It needs to be gathered for a specific purpose with a clear strategy. Otherwise, it’s all just ones and zeroes clogging up the server space. No one wants that.

High-quality data is like a deciduous tree moving through the seasons. The leaves could stay lush for years and suddenly turn brown as a different technology emerges to make it all redundant.

But perhaps someday, that dataset could see the green shoots of Spring as it discovers new and greater value than ever before. That’s the life cycle of data.