Question

Research the role of ETL tools in providing clean and purposely transformed data as part of data mining processes. Then explain the role of ETL in data mining and statistical analysis.

Answer 1

1.

To Clean data:-

To ensure data quality in data warehouse which can be done using data unification rules like:-

• Making unique identifiers. Example:- sex categories Male/Female/Unknown can be given as M/F/null, Man/Woman/Not Available.
• To Convert null values into standardized Not Available/Not Provided values.
• To Convert phone numbers and ZIP codes into a standardized form.
• To validate address fields and to convert them into proper naming.
• To Validate address fields against each other Example:-State/Country or City/State or City/ZIP code.

To Transform data:-

Here set of rules are applied to transform data from source to target.

• converting any measured data to same dimension using same units in order to join them later.
• To join data from several sources, generating surrogate keys, generating aggregates, sorting, and to apply advanced validation rules.

2.

ETL stands for extract, transform, load. These three functions are combined into one tool in order to pull data from one database and transfer it to another database.

ETL tools helps in bringing data from diverse sources to gather them in a single, accessible structure. Then load that data into data marts or data warehouse. Data mining tools include techniques such as neural networks, advanced statistics in order to locate patterns within data and to develop hypotheses from them.