Data to feed a model

Lesson 2 of 19 in Coddy's Introduction to Machine Learning course.

In Machine Learning, data plays a crucial role in training and developing models. To feed data into a model, it is typically organized in a structured format known as a dataset (a DataFrame in pandas).

A dataset consists of two main parts:

Features - A bunch of columns that signify a set of data
Target variable - Represents the output or the value that the model aims to predict or estimate.

For example, Here is a Weather table

id	degrees (C)	humidity (%)	wind (KM/H)	is_rain
1	23	5	14	True
2	30	10	17	False
3	15	3	12	False
4	20	12	5	False
5	26	57	23	True

Assuming we want to know if it will rain, the features are degrees, humidity, wind, and the target variable is is_rain.

The features are denoted as X, and the target is denoted as y.
The target variable (y) can hold any type of data - number, boolean, string, etc.

During the training process, the model uses the features as input, and the target variable as the ground truth or the correct output.

Without a target variable, the model would have no way of knowing whether its predictions are correct or not.

There are many different variations in the structure of datasets:
Many features, single target variable
Single feature, single target variable
Many features, many target variables
Few features, single or multiple target variables
No target variable
Time series data
etc.

Supervised learning involves datasets with input-output pairs, where the algorithm learns to map inputs to corresponding target variables.

Unsupervised learning doesn't involve target variables; instead, it seeks patterns or structure in the input data without labeled examples.

it’s crucial to ensure that the target variable is independent of the other columns, as any dependency could lead to data leakage and result in misleadingly high performance during training. A truly predictive model must learn from features that are not directly related to the target, thus preserving the integrity and generalizability of the model’s predictions.
An independent target variable cannot be directly calculated or derived from the other columns in your dataset; it’s what you aim to predict.

Test yourself

This lesson includes a short quiz. Start the lesson to answer it and track your progress.

Test yourself

This lesson includes a short quiz. Start the lesson to answer it and track your progress.

Test yourself

This lesson includes a short quiz. Start the lesson to answer it and track your progress.

Challenge

Easy

You are given a CSV file called galactic_zoo_no_target.csv. This dataset does not have a target variable.

In this challenge, you will create your own target variable. Create a column named ComplexityIndex derived from the formula: WaterCoverage * HabitatDiversity / PlanetMass. Then, remove the WaterCoverage, HabitatDiversity, and PlanetMass columns, as they are now represented by the ComplexityIndex.

To pass the challenge don't print anything, we check the df variable internally.

Try it yourself

# pandas as pd is already imported
df = pd.read_csv("galactic_zoo_no_target.csv")
# Write your code below

All lessons in Introduction to Machine Learning

1The Beginning of ML Journey

ML IntroductionData to feed a modelWhat is a ML model

4K-Means Algorithm

K-Means Introduction fit method predict method

2Machine Learning Overview

Supervised learning Unsupervised learning

5Naive Bayes

Naive Bayes Introduction fit method predict method

3K-Nearest-Neighbors (KNN)

KNN Introduction fit method predict method

6Other Models

Logistic Regression Linear Regression Decision Tree Support Vector Machine (SVM)Models quiz