Homework Assignment

HW 35 — Neural Network Classification

📘 Related: Lesson 35 🛠 MATLAB + Deep Learning Toolbox required 📁 Dataset: sat_data.mat

🌍 Background

Your supervisor at the Combined Space Operations Center (CSpOC) has been following developments in neural networks and asks you to evaluate their use for the GEO object identification problem. You remember from Lesson 35 that MATLAB's Deep Learning Toolbox provides straightforward interfaces for designing and training neural networks. For this task you will design, train, and evaluate a shallow neural network classifier using the same satellite dataset from HW 34.

📁 Dataset: sat_data.mat

The file sat_data.mat contains observations of 300 known objects in geosynchronous orbit. Load it with: 

load('sat_data.mat');
% Variables:
%   Y   — 300 × 2  feature matrix: [avg visual magnitude, avg rotation rate]
%   ID  — 300 × 1  class labels (integers 1–4)

The four object classes are:

LabelObject Type
1Active Payload
2Inactive Payload
3Rocket Body
4Debris

Problem 1 — Prepare Data and Define the Network

  1. Split the data into a training set (first 240 samples) and test set (remaining 60): 
    N_train = 240;
Y_train = Y(1:N_train, :);       ID_train = ID(1:N_train);
Y_test  = Y(N_train+1:end, :);   ID_test  = ID(N_train+1:end);
  2. Convert the training labels into the one-hot format that patternnet expects, and define a shallow network with a single hidden layer. You may choose the number of hidden neurons. See the Lesson 35 worked example for the relevant functions and data layout.

Problem 2 — Train the Network

  1. Train your network on the training data using the approach from the Lesson 35 example. Remember that patternnet expects features as a \(d \times N\) matrix (columns = samples).
  2. From the training visualization window that opens, click the Performance button. Include this plot in your submission. What does it tell you about the training process? What does it say about this specific scenario?

Problem 3 — Classify and Evaluate

  1. Apply the trained network to the test data. Convert the network output back to integer class labels and compute the test accuracy. How does it compare to the KNN and SVM results from HW 34?
  2. Generate a confusion matrix using confusionchart(ID_test, predicted). Include the chart in your submission. Which classes are most often confused with each other? Does this make physical sense given what the features measure?

Problem 4 — Reproducibility and Architecture Exploration

  1. Run your training script 3–5 times without fixing the random seed. Does the test accuracy change between runs? Why?
  2. Add rng('default') at the top of your script and rerun several times. Do results stabilize? Explain why fixing the random seed is useful when comparing architectures.
  3. With rng('default') set once at the top of your script, train networks with at least three different hidden layer sizes (e.g., 5, 10, 25 neurons) in sequence. Record the test accuracy for each. Summarize your results in a table and recommend an architecture. Is bigger always better?