Why Big Data Can Be Like A Jelly Fish: No Bones & You Might Get Stung

Jellyfish metaphor for big data without modelingWhat is it helpful to think of big data as "like a jelly fish"?

Both big data and jelly fish can be very beautiful. But if not treated with respect, you might get stung. And how you get stung, at least where data is concerned, relates to the other similarity between jellyfish and big data, which is a lack of bones, or structure.

To be in any way useful, big data needs to be used alongside an interpretive structure or model – the “bones” if you will, without which big data is as amorphous and useless as a jellyfish. The necessity of having this model is a critical challenge for any organization seeking to derive benefit from big data.

Let's prove the point about the necessity of interpretive structure with the simplest possible model.

Consider the results of a query on a joined accounts receivable table. You may have columns representing company names, owed amounts and due dates. And the meaning of the table is completely clear, but only because you know the column headers and table name, which comprise a simple "model" providing the meta-data meaning of the table. Without this "interpretive model" the table could just as easily imply accounts payables as account receivables!

This example is trivial. A more elaborate example might involve predicting service engineer staffing requirements over a holiday period. For a large service organization, lots of historical data may be available concerning call volumes during holidays. And yet the portfolio of maintained products may have evolved since previous years, requiring an adjusted update to the prediction. You can see that we now have a "predictive model", where the inputs are previous year's call volumes overlaid with new service contract support profiles.

Business professionals in any area have always worked with implicit models, and sometimes with explicit ones. The advent of big data now demands that models be made explicit; otherwise big data analytics just sits there like a jellyfish. You can think of the model as "providing the bones for the jellyfish". And if you have a good model, you shouldn't get stung.

A lot has been written on the subject of big data and "algorithms". Truly algorithms are wonderful things, as long as they aren’t erroneous (which can be the topic of another post).

But you can think of an algorithm as just one part of an overall function or business model. For example, the service organization might have algorithms to determine required headcount, given certain assumptions. Or a retailer will have algorithms for SKU re-order quantities. But the use of both of these algorithms, no matter how good they are individually, are of little use outside an overall business management context. A full business function model will contain one or more algorithms, plus the business assumptions on the use and context of the algorithm (and there are always assumptions), plus inputs and outputs of the objects to which the algorithm(s) apply. Models can focus on just one object, or perhaps many. A logistics model for example may include multiple warehouse resources, shipping channels, SKUs etc.

Whenever you purchase a software application, you are usually buying an implicit model concerning how your business should operated. Often the implementation of such software will allow you to customize the parameters of the models and algorithms that drive the work for which your software was acquired. It's worth noting however that your software will likely have some fundamental modelling assumptions "built-in" to the software that you cannot change. It's often the case that casual users are unaware of the assumptions of the software they are using.  (Achieving freedom from other people's modeling assumptions is one motivation for stepping up on modeling.)

Models can also be built explicitly in spreadsheets (very common) or in special purpose modelling software (for example Lanner).

Now, with the arrival of big data, the days of complacency about the business models and algorithms that run your business may be drawing to a close.

Big data puts new responsibilities on business executives to explicitly understand the assumptions and models on which decisions are made. Super powerful software and dramatically less expensive hardware have made big data available to most businesses, along with the capabilities of analyzing that data.

To benefit from this wonderful development, business executives need to step up on modeling. There is enough pressure on management that the arrival of new responsibilities may not always be welcomed. However, the power provided by big data with analytics, interpreted in the context of a developing library of business models, have the potential to make other areas of management purview easier.

Here's a big data modelling checklist. Note that you start with business, and then define what you want to do around the business, before big data even appears!  Notice that you don't actually start doing any analytics until Step No. 7.  And that the whole purpose of the exercise is Step No. 8, "Support Decisions".  Lastly note that much of the preparatory effort from Step No.s 1 thru 6 are more business-oriented than IT-oriented.

Big Data & Business Modeling Check List

  1. Identify P&L business domain.
  2. Define subject business function domain.
  3. Define key actors and responsibilities.
  4. Create business model with KPIs and scorecard.
  5. Define algorithms supporting model.
  6. Define sources and stores of big data. Assure QA. Execute.
  1. Define and execute analytics on big data in support of model.
  2. Use resulting model scenarios to test options and support decisions.

Big data can be beautiful. Make sure you add the modeling bones and you won't get stung.


Why Your Analytics are Failing You - HBR Item by Michael Schrage

In "Why Your Analytics are Failing You" by Michael Schrage, a research fellow at MIT Sloan School’s Center for Digital Business, there's a great analysis concerning the successful use of analytics.  Putting in bluntly, unless you use analytics to drive behaviour, you're wasting your time.  Analytics can help you understand your world.  But you're in business.  Understanding only counts if it drives behaviour.  Leave pure understanding to academia.


Best Contextualization Of Big Data Challenge by Chris Taylor

Tibco's Chris Taylor has written an excellent contextualization of the "big data" story, found on the popular Successful Workplace site. In the marvellously named Big Data must not be an elephant riding a bicycle Mr. Taylor highlights the key requirements for taking advantage of big data.  Also provided are a number of great links to additional resources for various analysts and other writers.

Of particular interest is the reference to Forrester's Clay Richardsonwho makes an excellent connection between big data and big process, in his blog posting Big Data Ain't Worth Diddly Without Big Process.  This is worth exploring separately.

McKinsey Says Business Leaders Need Baseline Analytics Skills

An excellent resource on the effective exploitation of the big data opportunity is found in McKinsey's "Big data: The next frontier for innovation, competition and productivity" (June 2011, McKinsey Global Institute) (available without charge, with registration).

"But having a core set of deep analytical talent is not enough to transform an organization, especially if the key business leaders and analysts do not know how to take advantage of this big data capability. All of the business leaders in an organization will have to develop a baseline understanding of analytical techniques in order to become effective users of these types of analyses."

[Page 115, Paragraph 2, emphasis added.]

Having a baseline understanding of analytics is not exactly the same thing as understanding and using modeling.  The document does allude to modelling a few pages earlier:

"The fourth and highest level is applying advanced analytics, such as the automated algorithms and real-time data analysis that often can create radical new business insight and models. They allow new levels of experimentation to develop optimal approaches to targeting customers and operations, and opening new big data opportunities with third parties. Leveraging big data at this level often requires the expertise of deep analytical talent."

[Page 115, Paragraph 2, emphasis added.]

In some domains, for example that of a product installed base ecosystem, building and maintaining an in-place model will become de rigueur.  Many organizations maintain such models now, usually in spreadsheets.  Although, having seen models where the sum of all items don't add to 100%, the quality of these models may be questionable.  

Big data and analytics will be especially useful to feed and validate business models, which in turn contribute to the generation of the best possible decision options.