AI-generated illustration created to represent the article’s subject. It does not depict an actual EPW course, trainer, participant, client, event or venue.
A data preparation and feature engineering checklist should protect the validity of a machine-learning decision, not simply make a dataset tidy. The crucial questions are whether each row represents the right unit, whether each feature would have existed at prediction time, whether transformations are learned only from training data and whether exactly the same logic can run in production.
Missing these controls can produce an impressive validation score that collapses after deployment. This checklist gives data scientists, analysts and machine-learning engineers ten evidence-based gates from problem definition to production monitoring. It is designed for structured organisational data, but the principles also apply to text, images and other model inputs.
Key takeaways
- Define the prediction moment before selecting or engineering features.
- Split data before fitting imputers, encoders, scalers or feature selectors.
- Investigate missingness and outliers; do not treat them as automatic deletion tasks.
- Build transformations as one reproducible pipeline for training and inference.
- Approve features with evidence about lineage, leakage, stability, cost and monitoring.
Why data preparation determines model reliability
A model learns the relationships present in its training inputs. It cannot repair an incorrectly defined target, an unreliable join or information that would not exist when a real prediction is made. Data preparation therefore begins with the decision and the operating process, not with a cleaning function.
The scikit-learn common-pitfalls guidance identifies two particularly important failures: inconsistent preprocessing and data leakage. It recommends applying the same transformations to later data and learning preprocessing only from the training subset. Pipelines help enforce both requirements.
| Gate | Question | Evidence to retain | Failure prevented |
|---|---|---|---|
| 1. Decision | What will be predicted, for whom and when? | Target, unit of analysis, prediction timestamp and action | Solving the wrong problem |
| 2. Provenance | Where did each field originate? | Source owner, definitions, extraction logic and lineage | Untraceable or inconsistent inputs |
| 3. Availability | Would this value exist at prediction time? | Event and availability timestamps | Target or temporal leakage |
| 4. Population | Does the sample represent intended use? | Coverage, exclusions, time range and subgroup profile | Biased or obsolete learning data |
| 5. Quality | Are missing, invalid, duplicate and extreme values understood? | Profiling report and treatment rules | Silent corruption or arbitrary cleaning |
| 6. Split | Is evaluation data protected from learned transformations? | Split strategy, keys, dates and random-state policy | Optimistic validation results |
| 7. Features | Does every transformation have a valid rationale? | Feature definition, owner, cost and expected direction | Noise, proxies and unstable complexity |
| 8. Selection | Do features add robust out-of-sample value? | Ablation, cross-validation and stability results | Overfitting and redundancy |
| 9. Pipeline | Can identical logic run during training and inference? | Versioned pipeline tests and schema contract | Training-serving skew |
| 10. Monitoring | How will changes and failures be detected? | Quality thresholds, drift measures and response owners | Undetected production deterioration |

The ten-point data preparation and feature engineering checklist
1. Define the target, unit and prediction moment
Write one sentence: “For each [unit], at [time], predict [outcome] within [horizon] so that [owner] can [action].” A row might represent a machine-day, customer-month or service request—not merely whatever the source table contains.
Define how and when the label is observed. If the target is “failure within 30 days”, decide what counts as failure, how repeated events are handled and how long labels take to mature. Without this, features and evaluation windows cannot be judged.
2. Record provenance, definitions and reliable joins
Create a compact data contract for each source: owner, field meaning, key, granularity, update frequency, retention and known quality limits. Check that joins preserve the intended row count. Many-to-many joins can silently duplicate examples and over-weight some entities.
Trace derived fields back to source events. A variable called “open balance” may mean current balance in one system and month-end balance in another. A model cannot resolve conflicting business definitions.
3. Test point-in-time availability
For every candidate feature, ask whether the value would have been recorded and available to the production service at the prediction timestamp. Separate event time from processing or database-update time. A repair code entered after equipment failure is not a valid pre-failure feature even if it appears in the same historical record.
Point-in-time logic matters for aggregates. A 30-day transaction count must use only events preceding each historical prediction. Recalculating all rows from today’s completed data creates a view of the past that the model could never have possessed.
4. Design representative and protected splits
Split according to the way the model will encounter novelty. Random splitting may work for independent records, but time-based splitting is often more realistic for forecasting, and group-based splitting may be necessary when repeated observations belong to the same customer, patient, asset or site.
Create the split before learning any transformation. The test set must not influence imputation values, vocabulary, scaling, principal components or selected features. Use cross-validation within the training data for development, and reserve the holdout for an honest final estimate.
5. Profile quality before choosing treatments
Measure types, ranges, categories, uniqueness, missingness, duplicates and relationships across time and subgroups. TensorFlow Data Validation’s official workflow illustrates how descriptive statistics, schemas and anomaly detection can turn assumptions into executable checks.
Investigate why values are missing. Absence may reflect a process, eligibility rule or system outage, so a missing indicator can sometimes be informative. Likewise, an extreme value may be a genuine rare case, a unit error or a sensor fault. Treatment should follow cause and model sensitivity rather than one universal rule.
6. Encode and transform without hiding meaning
Choose transformations to match data and model requirements. Scaling can help distance-based and gradient methods; robust scaling may reduce sensitivity to extremes. Categorical encoding must consider cardinality, unseen values and ordering. Target encoding requires especially careful out-of-fold construction to avoid leakage.
Logarithmic transforms can stabilise skewed positive variables, but zeros, negatives and interpretability need explicit handling. Preserve units and inverse transformations where stakeholders must understand outputs.
7. Engineer domain-valid features
Good features express information available to the decision: recency, frequency, ratios, lags, rolling windows, interactions or domain states. Each definition should state its window, denominator, treatment of absence and update schedule.
Avoid proxies for protected or unavailable information without governance review. A variable can be statistically powerful and still be operationally invalid, unfair, expensive or unstable. Record why the feature should generalise beyond the development sample.
8. Select features through controlled comparison
Begin with a defensible baseline. Add feature groups through ablation tests and compare performance across folds, time periods and important segments. Filter, wrapper and embedded methods answer different questions; none replaces domain review.
Check redundancy and multicollinearity where interpretation or coefficient stability matters. Dimensionality reduction and embeddings may improve representation, but they increase lineage and explainability requirements. Retain a feature only when its benefit justifies its operational cost.
9. Package transformations as a reproducible pipeline
Version code, dependencies, feature definitions and schemas. The scikit-learn pipeline and composite-estimator documentation shows how transformations can be combined with estimators so that fitting and prediction follow the same sequence. The principle applies beyond one library.
Test training and inference parity with known inputs and expected outputs. Validate column presence, types, categories and ranges before prediction. Decide how unknown categories, missing sources and delayed features are handled; production should fail safely rather than improvise.
10. Monitor data and features after release
Monitor schema violations, missingness, range breaches, category growth, feature distributions and freshness. Compare production inputs with an appropriate reference, but do not assume every distribution shift damages performance. Connect alerts to investigation and outcome evidence.
Assign owners and response thresholds. A feature-quality incident may require fallback values, model rollback, pipeline suspension or source-system correction. Store sufficient lineage to identify which predictions and model versions used the affected data.
Worked example: maintenance-failure prediction
Suppose a maintenance team predicts whether a pump will fail within 30 days. Historical tables include sensor readings, work orders, inspections and repair outcomes. The first version achieves excellent validation results, but a leakage audit finds that “final work-order status” is updated after failure.
| Candidate feature | Point-in-time finding | Decision | Replacement or control |
|---|---|---|---|
| Final work-order status | Usually completed after the outcome | Reject | Use status recorded at prediction time |
| Seven-day vibration mean | Available from timestamped sensor events | Retain | Test sensor freshness and minimum coverage |
| Days since last inspection | Knowable, but missing for new assets | Retain with control | Add missing indicator and new-asset rule |
| Site failure rate | Risk of using future failures in the aggregate | Revise | Compute point-in-time expanding estimate from prior events |
| Asset serial number | High-cardinality identifier with weak generalisation | Reject as raw field | Use governed asset-class and age features |

After removing or rebuilding the leaking features, performance will probably fall. That is a useful correction: the lower result is a more credible estimate of future behaviour. The team can now compare valid feature groups, test recent time periods and design monitoring for sensor freshness and unseen asset classes.
Release checklist for a model-ready dataset
- The target, unit, horizon and prediction timestamp are approved.
- Every source and feature has an owner, definition and lineage record.
- Joins, duplicates, exclusions and label maturity are tested.
- Feature availability is verified at historical prediction time.
- Training, validation and holdout separation precedes learned preprocessing.
- Missing, invalid and extreme values have reasoned treatment rules.
- Feature groups demonstrate stable out-of-sample value over a baseline.
- The complete preprocessing pipeline is versioned and reproducible.
- Training and inference transformations pass parity tests.
- Production quality thresholds, owners and response actions are documented.
Develop production-ready data capability
EPW’s five-day Data Preparation and Feature Engineering for Machine Learning course covers prediction targets, provenance, sampling, missing data, outliers, leakage, numerical and categorical features, temporal features, selection, dimensionality reduction, reproducible pipelines, point-in-time accuracy and monitoring.
Professionals can also explore EPW’s Artificial Intelligence and Machine Learning Courses and the AI and machine learning article hub. To apply TIMECHECK to a real dataset, review the course outline, available dates and locations, or request tailored course details.
Sources and references
- scikit-learn, Common pitfalls and recommended practices, version 1.9 documentation, accessed 2 September 2026.
- scikit-learn, Pipelines and composite estimators, version 1.9 documentation, accessed 2 September 2026.
- TensorFlow, Get started with TensorFlow Data Validation, accessed 2 September 2026.
- National Institute of Standards and Technology, AI RMF Playbook, accessed 2 September 2026.
- EPW Training, Data Preparation and Feature Engineering for Machine Learning Course, accessed 2 September 2026.
