The
Phases of the KDD Process
A
KDD process consists of several tasks. Indeed, the actual mining,
that is to say the application of a data mining algorithm to a
dataset, is only one of these steps. Following the CRISP-Data Mining
model [9,31] we distinguish the following tasks:
1. Business
Understanding
The very first step of a KDD project should be a close look from the
business point of view. The goal of this phase is to gain a deeper
understanding of the project objectives and further circumstances
strictly from the business perspective. Finally the insights from
this initial phase are to be turned into a data mining problem
definition.
2. Data
Understanding
Based on the results from the business point of view the second step
is to get familiar with the available data. The goal is to understand
the attributes and the corresponding attribute values and to find out
hidden semantics possibly in the data. Furthermore at this stage one
should figure out what exactly the available data offers. That is to
say, whether it has the potential to answer our mining questions or
not, and if possible to select promising subsets of the data.
3. Data Preparation
The next step is to construct the dataset where the mining algorithm
is to be run on. This phase covers both syntactic aspects – format
transformations for the employed mining algorithm – and semantic
aspects like table, record and attribute selection. Last but not
least this phase also includes deriving new attributes that contain
higher information only implicitly contained in the raw data (e.g.
deriving “day of the week” from “date”).
4. Modeling (or
Mining)
In the modeling phase the actual data mining takes place. Based on
the identified business goals and the assessment of the available
data an appropriatemining algorithm is chosen and run on the prepared
data.
5. Evaluation
Evaluating the results of the mining run mainly covers three aspects.
First of all, it is necessary to ensure whether everything went right
from the technical point of view. Was the mining algorithms finally
able to read and interpret the prepared dataset correctly? Were all
designated information actually given to the algorithm? Etc. Second,
one needs to investigate whether the mining results are sound from
the mining methods point of view. Some methods directly support this
decision by computing certain significance measures whereas others
leave this aspect completely to the analyst and his experience.
Third, a key objective of the evaluation phase is to determine if all
important business issues have been considered adequately.
6. Deployment
After mining the data and assessing the data mining results one needs
to transfer the results back into the business environment. This can
be rather straight forward like preparing the results in form of a
report that is understandable by business people (who of course
typically are non data mining experts). Or, as the other extreme, can
be quite complex like implementing a repeatable data mining process
across the enterprise.
