Neural Networks (Supervised Learning)
Supervised learning — someone must have looked and labeled things beforehand. That person is usually underpaid.
Sharlock's Detection Laboratory
Moriarty's Ring
★★★★★Diabolical! The genuine clues form a ring around the crime scene, the red herrings lie outside. A straight line will not do — the network needs to bend.
Learning goal: Observe how the hidden layer creates non-linearity. Expect ~90%, not 100%. (expected ~90%)
Help Sharlock Helmes learn to distinguish genuine clues from red herrings by training a neural network. Click on the canvas to add data points and watch how the network learns from your examples. The visualization shows how Sharlock's artificial neural network processes evidence: input neurons receive coordinates, hidden neurons analyze patterns, and the output neuron determines whether a piece of evidence is a clue or a red herring from Dr. Moriarty.
What you see on the investigation field
Here Sharlock Helmes learns to tell real clues from red herrings — not from fixed rules, but from known examples. Here is what the picture shows.
- What you see
- A square investigation field with points in two classes: blue clues and red herrings. The background is filled as a coloured area and splits the field into two regions — one blue, one red. Where blue turns into red, the dividing line runs. Next to it, a small diagram shows the neural network that makes this split.
- What happens
- From a few points with a known answer the model draws a first rough border. With every training step the coloured area shifts, until the line cleanly separates the blue points from the red ones. The more examples you give, the sharper and better-fitting the transition between the colours becomes.
- What you can do
- Pick an evidence type at the top and click to place new points on the field. Train in single steps or automatically, adjust the learning rate and speed, or load a ready-made case from Sharlock's case file. Reset clears the field.
- What to watch for
- Supervised learning means drawing a rule from known examples that come with an answer. Watch that the final border also sorts new, not-yet-placed points correctly — not just the ones it learned from.
Definition
Neural Networks (Supervised Learning)
Analogy
Definition
This interactive demonstration shows supervised learning in action. You provide labeled examples (clues vs. red herrings), and the neural network learns to classify new evidence. The system uses backpropagation to adjust weights and improve its decision-making over time.
Important Parameters
- Learning Rate: Controls how quickly the network adapts to new information. Higher values learn faster but may overshoot optimal solutions.
- Training Speed: Determines how many learning steps per second are executed during automatic training. Adjust to observe learning patterns at different speeds.
- Evidence Types: Three categories: Clues (genuine evidence), Red Herrings (misleading clues), and Uncertain (unlabeled data for testing predictions).
Visualization Elements
The demo offers multiple ways to understand how the neural network makes decisions and learns from your data.
- Data Points: Blue circles represent clues, red circles are red herrings, and gray circles show uncertain evidence with prediction borders.
- Decision Boundary: The colored regions show where the network would classify evidence as clues (blue) or red herrings (red), with opacity indicating confidence.
- Neural Network: Visual representation of the 2-3-1 network architecture showing nodes, connections, and real-time weight updates during training.
Tips for Effective Training
- Start with some clear examples in each category to establish basic patterns.
- Add uncertain points to test the network's predictions on unlabeled data.
- Watch the accuracy indicator - it should improve as you add more diverse training examples.
- Experiment with different learning rates to see how they affect convergence speed and stability.
Learning with a Teacher
Supervised Learning is like learning with a teacher: The model receives labeled training data - examples with the correct answer. It learns to map inputs (features) to outputs (labels) by recognizing patterns in the training data.
Training occurs iteratively: The model makes predictions, compares them with the true labels, calculates the error, and adjusts its parameters to minimize the error. This process repeats until the model makes good predictions.
Classification vs. Regression
Classification: Predicting categories (e.g. spam/not-spam, cat/dog). Regression: Predicting continuous values (e.g. house prices, temperature).
Advantages
- Precise predictions: Very accurate with good training data
- Well understood: Established methods and best practices
- Measurable performance: Clear metrics (Accuracy, F1-Score, etc.)
- Broad applicability: Works for many problem types
Challenges
Requires many labeled data - labeling is expensive and time-consuming. Overfitting (too good adaptation to training data) and underfitting (too simple models) are common problems. Data quality is crucial.
Practical Applications
Image and speech recognition, spam filters, medical diagnosis, credit risk assessment, stock price prediction, recommendation systems, and autonomous driving.
Try the demo! Train a model and see how it learns from examples.
1
# Supervised Learning Algorithm
2
function train_model(data, labels):
3
# Main function: Learns patterns from labeled data
4
X, y = load_data() # Features and target values
5
X_train, X_test, y_train, y_test = split_data(80/20)
6
7
# Initialize model
8
model = create_neural_network(layers=[100, 50, 10])
9
learning_rate = 0.001, batch_size = 32, epochs = 100
10
11
for each epoch in epochs:
12
# One epoch = One pass through all training data
13
for each batch in training_data:
14
# Forward pass: Make predictions
15
input = batch.features
16
predictions = model.forward(input)
17
18
# Calculate loss: How wrong were we?
19
loss = calculate_error(predictions, batch.labels)
20
21
# Backward pass: Learn from mistakes
22
gradients = calculate_gradients(loss)
23
model.update_weights(gradients, learning_rate)
24
25
# Validation: How good are we?
26
val_predictions = model.predict(X_val)
27
accuracy = calculate_accuracy(val_predictions, y_val)
28
29
if accuracy > best_accuracy:
30
save_model('best_model.pkl')
31
32
# Final testing on unseen data
33
test_accuracy = test_model(X_test, y_test)
34
return model, test_accuracy
📊 Prepare Data
Load labeled data: inputs (X) and correct answers (y). Split into training (80%) and test data (20%) for later evaluation.
X, y = load_data() # Features and target values
X_train, X_test, y_train, y_test = split_data(80/20)
💾 Data Collection
Gather labeled examples: images with categories, texts with sentiment, numbers with predictions. Data quality determines success!
🔧 Preprocessing
Prepare data: Normalization (0-1 scaling), fill missing values, categories to numbers. Clean data = better learning.
🎓 Training
Iterative learning: Show examples, make predictions, calculate errors, adjust. Repeat hundreds of times until model is good.
📋 Validation
Continuous checking during training. Prevents overfitting (memorization). Stop early if validation gets worse.
🚀 Deployment
Trained model in production: Predict new, unseen data. Monitor performance and retrain if needed.
Check
Supervised Learning Quiz
What is the main goal of supervised learning?
Select all that apply
1. What is the main goal of supervised learning?
2. What kind of data does supervised learning require?
3. What distinguishes classification from regression?
4. You train a model with 1000 cat images and 10 dog images. What will happen?
5. Why is it called 'supervised' learning?
6. In the demo, you see that your trained model has 100% accuracy on training data, but only 60% on new test data. What happened?