The purpose of this article is to introduce the reader to process mining, an innovative analytical approach to learn about any process in an objective and exhaustive manner. It covers of course the key concepts and definitions, as well as the benefits, technical requirements, and success criteria, which hopefully will give you the inspiration to apply such techniques to your own professional reality.
Let’s jump and learn about this fascinating innovative solution for complex processes exploration!
- What is process mining?
- How does it work?
- How does process mining heavily improve process discovery ?
- A word on required data
- Success criteria
WHAT IS PROCESS MINING?
Most organizations create business processes which are sometimes difficult to control and comprehend. Understanding these processes are however an absolute prerequisite prior to taking on any improvement initiative.
Process mining is a discipline supported by a powerful analytical technique which makes it easy and fast to get a complete and objective picture of your processes, and as a consequence to control and continuously improve them. This is made possible by analyzing vast quantities of data available in IT systems. Mainly which activities are performed, when, and by whom.
Process mining is a highly valuable solution for insights generation: did you ever ask yourself what really occurred in a particular process, what deviations from the initial designed flow have been observed, where are the bottlenecks, how can the process be optimized? If so, then you should definitively consider process mining as part of your team’s toolkit.
As process mining tools are very intuitive to use and highly visual, I encourage you to have a look at the video below, before deep diving into more details, as an illustration of the power of process mining, applied to a banking process. This video has been created by Fluxicon.
Using process mining provides several benefits to business owners and organizations. We list hereunder the most significant ones:
- Understanding how a process is actually performed.
Most of the time, business owners know very well their processes from a theoretical perspective: what is supposed to happen, when, who is supposed to do what, under which condition.
However, they usually do not have a way to investigate what is really happening throughout the process lifecycle. Traditional reporting, Business Intelligence (BI), statistical tools have difficulties in revealing both the big and the very detailed picture. Such a back and forth navigation from the big picture and the detail is actually what is the most efficient way to comprehend the real life situation. Process mining solutions such as Disco focuses on making it easy to digest and exploit.
- Improving the process flow by knowing the actual flows, frequencies, and delays in the process.
IT system logs track great volumes of useful information, making it feasible to compute paths and lead times to switch from one step in the process to another. One can leverage this valuable information to detect bottlenecks, highlight sub efficiencies, reveal most frequent paths, etc.
- Cost savings by improving the productivity of employees.
Speed up investigation time, and free up time of your analysts to improve processes rather than to analyze the as-is. Reduce overall process time by focusing on the pain points.
- Harmonizing processes in different regions by understanding how people work in different ways to see what the best practices are and to align processes.
- Improving the quality and increasing the efficiency of auditing.
Conduct audits in an efficient manner, leveraging data science and avoiding opinion-based discussion (data is agnostic!).
- Verifying that implemented process changes have had the expected effect by comparing the old process to the new process.
Want to observe the impact of any change in the process? Conduct pre/post analysis, and quantify your impact and added value to the new process!
HOW DOES IT WORK?
One of the great advantage of process mining is that it analyzes any process in an agnostic way. This is made possible thanks to data, namely the footprint of any action being performed in a IT system and captured in its log.
To understand how this works, let’s consider a process made of 5 activities A, B, C, D, E, and analyze how 3 past items going through this process can help learn more about it.
In the animation below, you can observe that some cases go through different activities, or through the same activities but in different order. Some others loop over the same activity.
Process mining consists in analyzing all possible patterns based on historical data and in creating a highly visual process map for investigations.
HOW DOES PROCESS MINING HEAVILY IMPROVE PROCESS DISCOVERY EXERCISES?
The value of process mining is twofold: generating valuable insights to improve current processes, and speeding up the insight generation time, thus improving team’s productivity. Let’s compare and highlight these benefits:
Without process mining
The traditional way of process discovery is carried out manually through workshops and interviews.
۱٫ Time consuming:for the people who do the interviews and process mapping. If they could do their work faster, they could do more projects and therefore deliver more value for their organization.
۲٫ Bind resources: it binds people (interviewees) from productive work into discussions about how things are currently done just to understand the ‘as-is’ process.
۳٫ Subjective results: based on what people think about how the processes are, not necessarily based on how the processes really are.
۴٫ Risk of political deadlock: there may be different opinions that cannot be resolved (political deadlock). In situations, where the participating people cannot agree on a unified view of the ‘As-is’ process and the problems with the process, the risk is high that the project fails and no process improvement is carried out at all.
۵٫ Incomplete picture: you get only a sample view, not the complete picture. By asking people to spell out the process, the manual picture will never be complete.
۶٫ Measurement is costly: manual tracking and measurement is costly, biased, and also provides only sample data. This is about the practice of measuring process steps with a stopwatch to collect objective evidence: (1) Doing this manual work is very time-consuming, (2) People who are observed behave differently than normally, and (3) Only a limited sample can be obtained (e.g., tracing 30 cases over a few weeks).
۷٫ Difficult to replicate: because the cost of process diagnosis and information collection is so high, it cannot be easily repeated.
With process mining
The starting point for any process improvement project is the so-called ‘As-is’ process analysis, in which the current state and all the deficiencies of the process are mapped out and improvement opportunities are identified. Process mining significantly lowers the cost of understanding the current process by limiting people interviews and extracting the necessary information out of the existing data from the IT systems.
With process mining, the previously mentioned pain points are resolved:
۱٫ Time-efficient: The analyst spends less time on interviews and workshops. Instead, the ‘As-is’ process is automatically reconstructed from the IT data in the organization.
۲٫ Frees resources: workshop sessions can be much more productive, because the real ‘As-is’ process can be already taken as a starting point.
۳٫ Objective results: The data provides a factual basis for the diagnosis and improvement decisions.
۴٫ Freedom from political deadlock: There is no room for opinions as far as the process reality is concerned.
۵٫ Complete picture: All the exceptions are brought into the picture along with the normal process. For example, all process variations and actual delays from the past year can be analyzed to get a complete picture.
۶٫ Measurement is less costly: The actual durations and delays between process steps can be measured for thousands of transactions.
A WORD ON REQUIRED DATA
As for all data driven technologies, data preparation plays an important role in order to be able to fully benefit from process mining.
In this section, I briefly introduce the reader to the data requirements which consist of creating an event log that can be fed into a process mining tool.
What are the key required components to any process analysis? Actually only 3 main pieces of information are required:
- Activities (process step): the several steps that compose the process.
- Case ID: the unique identifier of any item going through the process.
- Timestamp (for performance evaluation): flag of when each item arrived and left any activity.
For more advanced analysis, I strongly encourage you to complement these prerequisites with additional (optional) information, such as resource, country, department etc. being involved.
Leveraging such additional data will enable far more advanced and detailed investigations (territory specificities, products being treated differently depending on their category etc.)
Interested in starting a process mining initiative? Before you start, remind that process mining is a discipline (not just a tool) that requires skills people need to learn (e.g. how to detect data quality issues, how to interpret the results, etc.) and that process mining is just a starting point for the discussion with the domain experts (so the purpose is not to automatically generate definitive answers and stop talking to people, but to start talking to them on a different level).
Make sure success criteria are identified from the beginning.
- Focus on the business value: define the business value in terms of effectiveness (customer experience and revenue), efficiency (costs) and risk (reliability). Determine into which process aspects you want to gain insights. To which business driver does this insight contribute? Better customer experience, cost reduction, risk mitigation?
- Start small, think big: connect the business driver to a specific business domain. Choose a process where the beginning and the end are clearly defined. Each manager of such a process will benefit from insights that help to reduce costs or increase the effectiveness. And while you think about the possible use cases and application possibilities, also make sure to communicate what process mining is not. By indicating clear boundaries, you can manage expectations on what it is.
- Work hypothesis-driven and in short cycles: divide the main business driver into sub hypotheses that you can confirm or disprove with a process mining analysis. For example: There is a gut feeling that a particular service process takes too long. How long does the process really take? How much does it deviate from the expectations? Where are the bottlenecks that cause the delays in this process? In addition, you can then indicate where exactly the delays are in the process. Take your business stakeholders from insight to insight. Stimulate them to ask questions. Explore, analyze and innovate. Time-box the intermediate results and the project. Eight weeks for the first project is usually a good aim.
- Facts don’t lie: process mining allows you to analyze processes based on facts instead of subjective opinions. Speak openly and transparently about the data that you use and about the facts that come out of this analysis. This can be confrontational and for some people even unwelcome. Put a change management team together that has the competency to handle resistance. Process mining can be the perfect assistance in this truth finding. Always use experts from the business process domain and the IT-domain for a sanity check of the data and the analysis. Use process mining as a constructive starting point to ask the right questions and avoid too quick judgments.
I’ve discovered process mining a few years ago, following the online course from process mining’s guru Wil van der Aalst, and this was really eye-opening to me. The benefits of it are numerous, and should definitively part of any data scientist’s toolkit!
I’m happy to receive feedback of any kind so please feel free to comment and share your perspective (opinion, use case, etc.) on this capability.