If you are an executive charged with a "big data project", here's a prediction. On your first week on the job you'll likely be surprised. And the surprise concerns what you'll actually spend your time doing.

What might be the surprise on your first week on the job? The surprise is that your biggest responsibility won't concern what you thought you signed up for, which is likely something like "generating analytical insights". Rather, your biggest responsibility is almost the opposite, which is focusing on everything but generating analytical insights.

And if you've been dreaming of fishing in sea of data, with a big net to scoop up any of the myriad insights just swimming by your boat, your surprise might even be one of disappointment. Because this blue ocean metaphor, lovely to imagine, is also seriously misleading about the nature of the world of big data. It's more likely that your ocean will consist of nothing but fish, all of which have three eyes and which are inedible! You'll have lots of data but nothing to take home that you can call a great insight.

Here's the problem, which could define the basis of your surprise, your job responsibility and the sort of metaphor that you should use to describe the world of big data. The problem with big data concerns the very definition of big data, which is that it is "big" -- but although big is subjective, there's a very practical and non-trivial definition of "big". Your set of big data is defined as such if it is too big for any of your machines to process in one chunk in a reasonable amount of time. In other words, the world of big data is defined by the system capacity constraints. . . . read more

What is it helpful to think of big data as "like a jelly fish"?

Both big data and jelly fish can be very beautiful. But if not treated with respect, you might get stung. And how you get stung, at least where data is concerned, relates to the other similarity between jellyfish and big data, which is a lack of bones, or structure.

To be in any way useful, big data needs to be used alongside an interpretive structure or model – the “bones” if you will, without which big data is as amorphous and useless as a jellyfish. The necessity of having this model is a critical challenge for any organization seeking to derive benefit from big data.

Let's prove the point about the necessity of interpretive structure with the simplest possible model.

Consider the results of a query on a joined accounts receivable table. You may have columns representing company names, owed amounts and due dates. And the meaning of the table is completely clear, but only because you know the column headers and table name, which comprise a simple "model" providing the meta-data meaning of the table. Without this "interpretive model" the table could just as easily imply accounts payables as account receivables! . . . read more

Lost business. The phrase strikes a chill into any CEO or line-of-business ("LOB") executive.

Lost business is also a special data management challenge.

So, let's see how a story about the "business of lost business" plays out between LOB execs and IT.  There is a data management angle on this story, and eventually we might even find some leverage for master data management (MDM) and analytics, but first we need to understand the all-important business drivers behind the scenes.

In our example, based on real events, imagine a technology services business employing close to a 1,000 people, and with accelerating product life-cycles. In only a half decade, the field refresh period for installed systems has fallen from seven years to five. This change means system sales reps must be on their toes for the upgrade sale, and ensuring the upgrade is even harder when smaller systems are sold through channels where warranty registration isn't complete.

What kinds of lost business events are there? First you have to understand that "lost business" as defined by Field Service means a "lost service contract" -- not a "lost system sale", although of course the two are closely linked. And in the distinction between services sales and systems sales we see the collision between three huge organizations: engineering, systems sales and services sales, each with different metrics and P&Ls! And to make matters more interesting, the engineering bill-of-materials ("BOM") has never been shared with customer services! . . . read more

On his popular ebizQ blog, Peter Schooff, asks (December, 2012) "How central of a role should BPM play in customer experience?"  Readers interested in this question should visit the original discussion as there are some terrific contributions and insights; the question concerning the centrality of the role that BPM should play in customer experience is a good one.  (In the context, "BPM" means "business process management software and practices".)

A short answer is this: "In a competitive world, and for both technical and economic reasons, BPM is likely to play an increasingly important role in customer experience".

This blog entry explores this proposition in more detail.  But first let's qualify our short answer above by defining the terms "customer experience" and "BPM". (For the purposes of this discussion, we'll focus on BPM as "technology" or "software" and on customer experience as "customer experience system"). . . . read more

No business person really cares about technology except insofar as that technology helps get the job done. But there's one technology that's different.

Unlike most or all other technologies, BPM software technology directly and explicitly concerns the work of business and can be therefore directly of concern to business-side executives. Ultimately, business process management software (BPM) is THE technology for getting work done.

Let's explore this proposition in more detail, and then see why the question of BPM is exciting in the context cloud technology. (This posting was originally featured on the Canadian LinkedIn discussion group Cloudfingr.)

There are many important business-enabling technologies, including technologies "further down the stack" (e.g. "SOA") or which hard-code business semantics (most "ERP" systems and purpose-built applications).

. . . read more

On his ebizQ blog, Peter Schooff, asks (September 2012) "what is the number one reason a BPM project fails?"

Asking this question is audacious. But important.

Here's the short answer: BPM projects fail at a rate higher than tolerable (thus the question) because BPM projects, being fundamentally different than all other IT projects, are not yet sufficiently supported culturally, organizationally and economically.  

In particular, a BPM projects puts pressure on business executives for detailed process leadership, a time-based pressure without precedent and for which many or even most executives are not ready.

The first response to ebizQ's question, from Emiel Kelly, alludes to these issues with the statement that BPM is seen as "a project, not as daily business". Subsequent comments by other contributors elaborate in worthwhile ways. But it's worth making Kelly's "not as daily business" explanation more explicit.

Specifically, from the original answer above, what does it mean that BPM projects are "fundamentally different", and why is this difference important? And what is "cultural, organizational and economic" support?

. . . read more

Geoffrey Moore of "Crossing The Chasm" fame, asked a question on his LinkedIn page concerning possible limitations to the growth of market gorillas ("Why Gorillas Don't Rule Everything").  Understanding the effect of corporate size and concentration is important for both individuals and organizations that must operate in a world defined by monster organizations.   The purpose of this note is to identify the top five factors limiting the optimum size of organizations.

Note that that your host is not making any assumption that corporate size is necessarily a bad thing.  Although it's a separate topic, and acknowledging that there are clearly many downsides to large organizations, there are all also examples, supported by research, showing that large organzations can also have many positive attributes, and that by

. . . read more

On his ebizQ blog, Peter Schooff, based on a posting in the the MWD Blog, asked for a definition of  systems robustness.  (July 2012): "What makes a business process robust?"  The question of robustness is much more important than you might expect.  The definition of robustness gets to the heart of what it means to build successful systems for business.  Addressing the question of robustness is one antidote to the "magical thinking" which is sometimes found in the executive suite.  But to think about robustness, one must have an understanding of the technical meaning of the term.  The term is sometimes bandied about along with other feel-nice business jargon, including the word "agile", with the result that the word loses it's power and your dialogue is robbed of meaning.  Here are your host's comments, mirrored from ebizQ:

In the context of several recent banking debacles, a question on the definition of system robustness is a good question. It's a good question because at least one of the UK banking failures appear to have happen at the intersection of technology, business process outsourcing and business process management. The failure was very high profile because thousands of customers were reported to have been locked out of their accounts for extended periods of time.   (This failure was not the same issue which was the trigger for the ebizQ thread; however interestingly, both issues conceivably involve business process.)

Why is "robustness" a good question in the circumstances? Because the ostensible failure of consumer banking services could correctly be characterized as a situation where the system "lacked robustness under conditions of outsourcing." The statement "conditions of outsourcing" gets to the formal definition of robustness.

. . . read more

According to the analysts at Aberdeen Group, on average 26% of world workforce headcount is considered "contingent", including contract and temporary staff.  Clearly contingent labour-force outlays account for a huge portion of total spend.  But Aberdeen asks if this spend is well-managed. 

There are in fact very significant differences between best-in-class and laggard organizations concerning how contingent work is managed.  Best-in-class managers get much better results (over 50% higher reporting program objectives achieved), better contingent workforce cost control and most important, significantly better overall organizational efficiency.  This last benefit gets to the heart of the whole contingent workforce business case. 

Why bother with all the effort and management time to organize contingent workforce scaling if your organization does enjoy overall improved efficiency as a result?

In a world of intense competition, contingent workforce scaling makes intuitive sense, and it's not surprising the Aberdeen Group has identified characteristics of the organizations that "do contingent" better.  But why highlight these insights in this Decision Models forum on business process management technology?

. . . read more

BPM software advocates need look no further than the regular reports from the analysts at Aberdeen Group for terrific examples of BPM in action.  The latest example, by Analyst William Jan, is AP Invoice Management in a Networked Economy (you can acquire this report without charge for a limited time via the embedded URL; registration is required).

The world of business process is about the processes at the core of any business.  And for this reason unless you are an insider in any given function or vertical market, it's difficult to acquire in-depth knowledge about business processes in real life.  Organizations tend to be reticent about revealing the secrets about how they do business; and as well, in any given function the processes reflect the complexity of corporate life and one is not likely to master that complexity over night. 

So, for these reasons, the work by Aberdeen Group is very welcome.  Their analysis work focuses especially on identifying best practices in various corporate functions, such as sales, accounts payables, inventory management, and so on.  And although Aberdeen Group includes technology in its analyses, their work is refreshingly "business first".

The case of Accounts Payable is a nice example of an end-to-end process analysis of an important corporate function.  Using A/P practices as a measure, and compared to "laggards", best-in-class organizations manage their A/P to deliver much better cash flow, which can have a huge impact on bottom lines. 

. . . read more

On his ebizQ blog, Peter Schooff asked an important question (June 2012): "Has social BPM fallen short of expectations so far?"  If you are exploring the promise of social BPM, the answers to Peter's questions are worth reading.  Your host believes there are two key challenges before we will realize the promise of social BPM: (1) a technology challenge and (2) a governance challenge.  Here are your host's comments, mirrored from ebizQ:

Some of the challenge around social BPM is associated with expectations and hype contrasted with the immaturity of social BPM software technology. There is a huge amount of research around "work", "narrative", "story" and "annotation", but that research has not truly been engineered yet into social BPM products.  The result is that most current social products are not built on a solid model of how narrative works in the human mind and as communication transactions between actors. And typically, a model of "work", i.e. what should be the subject of conversation, is also missing. But, over time we should see these challenges addressed, and surely the result will be very exciting.

However, I believe there's another challenge beyond technology, which may be more difficult to solve. This is the challenge of "social technology governance".

. . . read more

Some of my best friends are BPM-istas!  And evangelists-of-BPM as the second coming of application development.  I say bravo!  But then I have a few questions, such as . . .

1.       How many managers does it take to build a business process in BPM?  

“None.  Managers aren’t allowed to build a process in BPM.” . . . read more

Your host started his IT career at IDC, in the previous millennium.  Working for IDC gives one a perspective on analyst relations with vendors and end-users.  Analysts are little bit like movie reviewers -- they like technology as much as movie reviewers like movies.  And the economics of the technology analysis business is such that it's difficult for analysts not to be cheerleaders for technology, in the same way that movie reviewers find it difficult not to be cheerleaders for movies, or, considering the subject of this website, cheerleading for BPM software. . . . read more

 Is it possible to more completely grasp the idea of “enterprise”? 

And thereby submit that enterprise to the will of executive leadership?

John Zachman, well-known evangelist for enterprise architecture and originator of the Zachman “Framework for Enterprise Architecture”, says “yes”.

Canada’s DAMA affiliate IRMAC scored a coup last week by hosting Mr. Zachman on his road show for an update of the famous Zachman Framework.   Mr. Zachman gave a comprehensive tour of the Framework, the reasoning behind it and the advantages that adopting organizations might enjoy.

Mr. Zachman’s key message was that the application of normalization and ontological modeling to low-order, high-entropy organizations – i.e. organizations which are failing due to high cost structures and sclerotic inflexibility -- would reverse that state.  The sciences of organizational normalization and ontological modeling, defined by the Zachman Framework, unlock enormous benefits for organizational stakeholders. 

. . . read more

The promise of BPM technology is only realized within the context of traditional management skills and discipline.  Ironically, it is an erroneous common pattern of “magical thinking” that impedes success in both traditional- and new "BPM technology"-enabled management environments.

Intervention Warning Against Overselling BPM Technology

On a discussion hosted by the BP Group on LinkedIn, member Mr. Ajit Kapoor has made an excellent intervention in this discussion.  Our root discussion concerns an “experimental BPM technology sales pitch” which posited that “for the first time in history, we have a technology that is explicitly about taking your vision about how your business operates, and building tools that directly make it possible to run your business, according to that vision."

Specifically Mr. Kapoor has articulated a powerful position that new technologies such as BPM software should not be pitched as "nirvana".  And his riposte comes with powerful credibility based on his achievements and career.

. . . read more

On the LinkedIn BP Group, in response to your host's item Selling BPM Reveals Essence Of BPM, one of the participants contributed an excellent response questioning the emphasis on the idea of technology when selling BPM.  This dialogue is a great opportunity to highlight the importance of seeing BPM as a business technology. . . . read more

Below is a draft BPM sales pitch which is also an inventory of BPM concepts.  It's all about business motivation, and revealing the power of BPM.  Is this how you see BPM? If not, why not?

BPM Provides The Tools For The Job, Now

Business process management is about the technology of the work you do. 

For the first time in history, we have a technology that is explicitly about taking your vision about how your business operates, and building tools that directly make it possible to run your business, according to your vision. 

All other technology is really mostly about technology and bits and bytes and machines talking to machines. 

. . . read more

The boffins at McKinsey have just issued a stirring call to "free the reps"!

According to the consulting company, at one representative global firm, 75% of inside sales reps' time was spent not selling!

This frustrating sales situation is not uncommon, despite what McKinsey says is "the guiding principle of all sales operations", which is "to maximize time for selling and relationship building".  Of course sales people and sales executives, and probably even general management, all know that sales people should be selling.  But given that sales people everywhere are facing similar issues, it's helpful to have a spotlight on the situation.

As a professional B2B sales person focused on BPM, your host is naturally interested in the subject of the McKinsey article -- and how BPM is one point of leverage for improving sales operations.  The McKinsey article also raises larger questions about sales management; your host has now commented on these issues in the letter below. 

You can read the whole McKinsey Quarterly article and follow up reader comments including your hosts' comment, at the following URL. (Please note you will need to register, although there is no charge.)

. . . read more

BPM prospects often ask a question about "simulation".  Our standard answer is "simulation is best done by a best-of-breed business simulation product, outside of BPM".  This answer is usually delivered after some qualification to discover what the prospect means by "simulation".  Some of the time the prospect is concerned about technical simulations and the regular process of software QA.  But the majority of the time the prospect wants to be able to do business what-if simulations to answer questions such as "how many warehouse staff should we have" or "should we add a new warehouse in the Midwest".

Why is the BPM business simulation question so frequently asked?  The reason is that the question is directly related the two main business cases for BPM.  BPM is justified either on efficiency terms or on business model terms.  The BPM efficiency business case is the same IT efficiency business case that has driven most IT investments for two generations.  Efficiency in the best of situations is about dramatically reducing costs for a given business process; in the worst of situations, it's about "paving the cow path"!

. . . read more

You'd expect the BPM-savvy practitioners and evangelists such as found on LinkedIn's "BP Group" to be able to easily come up with a good definition of BPM   .  A specific and actionable definition.  You'd be wrong.

In a BP Group forum discussion entitled "Can Anyone Make One Sentence Describing BPM", most of the answers were generic and non-actionable and often sounded like mission statements -- the kind of feel-good mission statements that are ridiculed by cynical business writers -- or worse the statements were self-referential ("BPM is about improving your processes").  In fairness. participants shared many worthwhile insights.  It's just that the there was a general and disappointing failure to answer the question in a useful way.

Let's look at what would be a good top-level definition of business process management -- and then why a good definition is important.

On the forum, Kenneth Beard came the closest to a good description of BPM with his "scientific management of work activity to enable informed decision-making", although I would make the case that final phrase in this definition is outside the scope of a definition of BPM.

Your host proposed the that BPM can be simply defined as "the modelling and management of repetitive work", which is certainly not original, but this concise definition emphasizes a fundamental concept, specifically the centrality of the question of work to the definition of BPM.

. . . read more

Is it possible that even with today's excitement and real achievement in software and technology, especially around mobile, M2M ("machine-to-machine"), the IOT ("Internet of Things"), analytics, so-called "big data" and machine learning, just to name a few hot topics du jour, that there is a major roadblock to further easy progress in technology?

After decades of achievement in the development of software technologies and software engineering, the software industry is rightly acknowledged as having contributed enormously to every aspect of business, social and personal life.  It is a general belief, fostered by both science and culture that a "long revolution" based on IT will continue on, bringing ever more amazing, delightful and useful innovations.

This expectation of progress can probably be depicted as a linear function with a nice upward slope.  While "Whiggish" expectations of continual progress are nice, the reality of software engineering is less rosy.  The realization of future progress based on software technology may not be so easily achieved and the immediate future of software development may be disappointing.

Why is there a potential for disappointment?

The current state of software engineering and data management is characterized by what could be called a "semantic ceiling".  On the software engineering side, the newest software products and software development are, while often quite wonderful, still rather limited in what they accomplish:  mashups, social applications, situational applications, modeling tools, more SOA, point business applications etc.  The scope of these new applications is typically either siloed or trivial in some sense. 

Especially on the data management side, the growth of data resources has exacerbated the data chaos that confronts both business and individual trying to make use of technology.  For this reason, it is not surprising that master data management (MDM) is a hot area in the software business.

The idea of a semantic ceiling is the idea that further progress in software engineering will only be possible with the development and deployment of a new layer of semantic technology.

. . . read more

Takeaway Summary:   According to the theory of Nobel Prize winning economist Ronald Coase, corporations exist to manage the transaction costs involved in organizing work.  And the size of a corporation is determined by the optimal management of those costs.  However, new information technologies have changed that transaction cost landscape for business processes.  It is now more than ever possible to disaggregate the work of the firm, and still maintain corporate identity and control.  And for the intercompany integration technology business, the good news is that integration technologies have a leading role to play in this evolut . . . read more

Dynamic BPM, along with Adaptive Case Management, is the name given to a growing body of practical knowledge about building better software for business process.  The trouble with much of existing BPM is that poorly done, the implementation of BPM can contribute to a reduction in organizational flexibility.  The wags have it as "pouring concrete over your business" -- as if enough concrete wasn't already there from the acquisition of ERP systems.  It's fair to say that no one is to blame for this -- and ER . . . read more