A dataset is an orderly collection of data gathered for a specific purpose. It can be a spreadsheet with thousands of customer rows, a folder of labeled photos, or a collection of news texts.
The structure of a dataset#
In a tabular dataset, each row is a sample and each column is a feature. For example, a housing-listings dataset might have columns for area, neighborhood, building age and price. In an image dataset, each photo is a sample and its label (for example "cat") is the correct answer.
The usual split#
In machine learning projects, a dataset is usually split into three parts:
- Training: for the model to learn.
- Validation: for tuning and comparing versions.
- Test: for the final evaluation on data the model has never seen.
This separation prevents self-deception, because if you evaluate the model on the same training data, the result will be overly optimistic.
Where to find datasets#
Public data-sharing sites, research repositories and open government data are well-known sources. Read the dataset's license before using it, since not all allow commercial use.
Common mistake#
Some treat a dataset and a database as the same. A dataset is usually a fixed collection for a specific task, while a database is a live system for continuously storing and managing information.