Galaxy augmentation explorer

Below: where the data and the method come from, then the settings and the split stored with the saved results that every chart on this page reads.

Data source. Galaxy10 DECaLS — centred 64 × 64 × 3 colour images in ten human morphology classes. Detailed dataset provenance and usage conditions are on the Dataset page.

Notebook lineage

Author: The PC5251 course notebook Galaxy2.ipynb; this demo repeats its comparison.
Source / context: PC5251 teaching demo: it repeats the notebook's own augmentation comparison on the full ten-class dataset.
Notebook: Galaxy2.ipynb

What the saved results record

Each row is one setting or quantity stored with the saved results. The Produced by column names the stage of the original pipeline that produced it.

Property	Recorded value	Produced by

In the Embeddings row, a galaxy's embedding is the 32 numbers the network's last hidden layer produces for it, just before the ten class scores.

The fixed split, class by class

Part	Galaxies	Share	0	1	2	3	4	5	6	7	8	9	What it is for
Train	12,415	70.0%	762	1,295	1,857	1,443	226	1,393	1,284	1,832	990	1,333	fit weights; rotation applies only here
Validation	1,777	10.0%	102	188	263	212	31	227	177	248	152	177	inspect learning during training
Test	3,544	20.0%	217	370	525	372	77	423	368	548	281	363	one untouched final comparison

Column headings 0–9 are the class indices used throughout this page; the class list in the Design matrix section gives their names.

What a row and a column are here

Size of the data table (design matrix X) given to the CNN

—

One row is one galaxy image, 64 × 64 × 3 (pixels × pixels × colour channels), laid out as a single line of numbers. The class label is kept separately.

17,736

Galaxies

10

Classes in y

12,288

Values per image

42

Split seed

How a row is built

Each image arrives as 64 × 64 pixels in 3 colour channels — 12,288 values in all.
Those values are the row, kept on the original 0–255 brightness scale as float32 numbers; nothing is standardised, centred or rescaled.
The morphology class is stored in a separate target vector y with 10 possible values, never as a column of X.
The three parts of the split are cut once, at seed 42, before either model exists, so both models are scored on the same 3,544 test rows.

The matrix itself

Row X[i,:] is one complete galaxy image flattened to pixel-channel values; column X[:,j] is one pixel-channel read down every galaxy; their intersection x_ij is one 0–255 channel value. The target y is a separate vector of human morphology labels — it is never part of X.

The highlighted row is the selected galaxy (choose one under “What does each galaxy class look like?”). Its 12,288 values form one row of X; its morphology class is stored separately in y, as class 2 · Round smooth.

How to read this panel. It is a sketch of the shape of X and of where the selected galaxy's row sits in it. The cell shading is a fixed pattern, not the galaxy's pixel values; the galaxy's real pixels appear in the galaxy images elsewhere on this page. Hover over a band to see which rows it covers.

The ten target classes are imbalanced

number of galaxies of each class in the complete dataset

0 · Disturbed1,081

1 · Merging1,853

2 · Round smooth2,645

3 · In-between round smooth2,027

4 · Cigar-shaped smooth334

5 · Barred spiral2,043

6 · Unbarred tight spiral1,829

7 · Unbarred loose spiral2,628

8 · Edge-on without bulge1,423

9 · Edge-on with bulge1,873

The same counts as numbers, by part of the split

#	Class	In the dataset	Train	Validation	Test
0	Disturbed	1,081	762	102	217
1	Merging	1,853	1,295	188	370
2	Round smooth	2,645	1,857	263	525
3	In-between round smooth	2,027	1,443	212	372
4	Cigar-shaped smooth	334	226	31	77
5	Barred spiral	2,043	1,393	227	423
6	Unbarred tight spiral	1,829	1,284	177	368
7	Unbarred loose spiral	2,628	1,832	248	548
8	Edge-on without bulge	1,423	990	152	281
9	Edge-on with bulge	1,873	1,333	177	363

One fixed split, made once

the notebook's two train_test_split calls, at the recorded seed

Train · 12,415 (70.0%)fit weights; rotation only here

Validation · 1,777 (10.0%)inspect learning during training

Test · 3,544 (20.0%)one untouched final comparison

One split for both models. It is cut once at seed 42 and reused by both models, so their test scores are measured on the same galaxies. Rotation is applied only to training mini-batches.

What does each galaxy class look like?

one real stored example per class · select one to follow it through the workflow

Each image is a real test galaxy from the stored cases. Click one to select it: the highlighted matrix row, the two rotation images, the ring on the embedding map and the two models' predictions in the Evaluation section all switch to that galaxy.

Transformation — original views and rotated views

What this step does. A galaxy's shape class does not depend on which way its picture is turned. So one model is trained on the training images as supplied, and a second on copies turned by a new random angle each time they are used. If the idea holds, the second model should do better on galaxies it has never seen. Validation and test images are the same for both models and are never rotated.

No augmentationUse every training image in its supplied orientation.

Training rowRead the supplied 64 × 64 × 3 image.

No spatial changeThe same orientation is seen whenever this row is sampled.

Label preservedThe original class label y is the training target.

Rotation augmentationTurn each training image by a new uniform random angle from -90° to +90° every epoch.

Same training rowStart from the same unique galaxy.

Random rotationA new view is generated inside the training loop.

Same labelRotation changes orientation, not morphology.

How to read this figure. Both models train on the same galaxies; rotation adds new orientations, not new galaxies. The images show the selected galaxy, which is a test image, turned by the angle stored with it to illustrate the operation. Test images were never rotated in training or evaluation.

Modelling & training — two fresh copies of one architecture

Both models use the network drawn below. Its three convolution blocks learn spatial features of the image; three dense layers (64 → 32 → 10) then turn those features into one score per class. As in the notebook, the model trained with rotation starts from its own random weights and has its own batch order, dropout masks and rotation angles. The 32 values from the last hidden layer are the embedding used for the maps further down.

64×64×3 image

Conv 8BatchNorm · ReLU · MaxPool · Dropout

Conv 16BatchNorm · ReLU · MaxPool · Dropout

Conv 32BatchNorm · ReLU · MaxPool · Dropout

Flatten 2,048

Dense 64 → Dense 32 → 10 logits

Softmax probabilities

Galaxy2 three-block CNN · 140,074 trainable parameters · Adam · cross-entropy

The two stored training histories

Each chart shows one model's accuracy after every epoch (one pass through the training images): solid for training images, dashed for validation images. Drag the epoch slider below, or click a curve, to read both models at one epoch; hover over a curve for exact values.

No augmentation

same supplied views

train 62.6%validation 43.5%train–validation gap +19.1%

Rotation augmentation

training views change each epoch

train 55.1%validation 55.3%train–validation gap −0.2%

Epoch to read on both training curves 100

Both training histories as numbers at the chosen epoch

Stored quantity at epoch 100	No augmentation	Rotation augmentation
Training loss	1.0387	1.2597
Training accuracy	62.62%	55.06%
Validation loss	1.5900	1.2157
Validation accuracy	43.50%	55.26%
Seconds in this epoch	17.14	22.55

The learned embedding before the final classifier

Just before it gives its ten class scores, each model describes a galaxy with 32 numbers, the output of its last hidden layer. These numbers are the model's embedding of that galaxy: galaxies the model treats as alike get similar numbers. PCA, t-SNE and UMAP each squeeze the 32 numbers into two, so that all 3,544 test galaxies can be drawn as dots; each method was run separately for each model. Choose a model, a method and its parameters below, and click a dot to see that galaxy's image.

1 · Model

2 · Method

3 · Parameters

Select a point

Click a point in the embedding to inspect its original test image.

0 · Disturbed1 · Merging2 · Round smooth3 · In-between round smooth4 · Cigar-shaped smooth5 · Barred spiral6 · Unbarred tight spiral7 · Unbarred loose spiral8 · Edge-on without bulge9 · Edge-on with bulge

Rotation augmentation · t-SNE · perplexity = 15 · learning_rate = auto. Local trustworthiness@10 is 0.995. Truth-label silhouette is -0.043 (higher is more separated; a negative value means overlap). Nonlinear local-neighbour view. Cluster spacing, size and axis values are not globally meaningful. Colour is the human-assigned class; the ring marks the selected galaxy.

How to read the axes. Each choice of model, method and parameters shows a different stored map of the same test galaxies. In t-SNE and UMAP maps the axes have arbitrary units: distances can be compared within one map, but not between two maps.

Every stored map as numbers, both models

Method	Parameters	Model	trustworthiness@10	silhouette (truth)	2-component variance
PCA	n_components = 2	No augmentation	0.913	-0.096	66.2%
PCA	n_components = 2	Rotation augmentation	0.911	-0.059	63.9%
t-SNE	perplexity = 15 · learning_rate = auto	No augmentation	0.989	-0.081	not defined
t-SNE	perplexity = 15 · learning_rate = auto	Rotation augmentation	0.995	-0.043	not defined
t-SNE	perplexity = 30 · learning_rate = auto	No augmentation	0.989	-0.084	not defined
t-SNE	perplexity = 30 · learning_rate = auto	Rotation augmentation	0.995	-0.043	not defined
t-SNE	perplexity = 50 · learning_rate = auto	No augmentation	0.988	-0.091	not defined
t-SNE	perplexity = 50 · learning_rate = auto	Rotation augmentation	0.995	-0.042	not defined
UMAP	n_neighbors = 15 · min_dist = 0.05	No augmentation	0.976	-0.095	not defined
UMAP	n_neighbors = 15 · min_dist = 0.05	Rotation augmentation	0.987	-0.069	not defined
UMAP	n_neighbors = 15 · min_dist = 0.4	No augmentation	0.948	-0.094	not defined
UMAP	n_neighbors = 15 · min_dist = 0.4	Rotation augmentation	0.959	-0.054	not defined
UMAP	n_neighbors = 60 · min_dist = 0.05	No augmentation	0.967	-0.096	not defined
UMAP	n_neighbors = 60 · min_dist = 0.05	Rotation augmentation	0.980	-0.059	not defined
UMAP	n_neighbors = 60 · min_dist = 0.4	No augmentation	0.942	-0.094	not defined
UMAP	n_neighbors = 60 · min_dist = 0.4	Rotation augmentation	0.955	-0.050	not defined

Negative silhouette values mean the ten human-assigned classes overlap in the learned 2-D view, even where a nonlinear method makes local neighbourhoods easier to see.

Evaluation — the same untouched test set, both models

Both models are scored on the same 3,544 test galaxies, which neither saw in training and which are never rotated. The comparison is made four ways: headline metrics, confusion matrices (where each class's galaxies end up), reliability curves (whether confidence matches accuracy), and the same real test galaxies side by side.

Metric on the untouched test set	No augmentation	Rotation augmentation	Change
Test accuracy	44.61%	55.25%	+10.64%
Weighted precision	57.06%	59.62%	+2.56%
Weighted recall	44.61%	55.25%	+10.64%
Weighted F1	40.74%	50.53%	+9.80%
Expected calibration error (lower is better)	10.05%	3.09%	−6.96%

Change is the Rotation augmentation value minus the No augmentation value, in percentage points. These two models come from the independent-start run: each started from its own random weights and saw its own random batch order, as in the notebook.

Confusion matrices

row-normalized · rows truth, columns prediction

No augmentation

Rotation augmentation

0 % of the true class100 % of the true class

Each row is one true class; its cells show where the model sent those galaxies, as a share of the row. Darker cells on the diagonal mean more galaxies put in their true class. Hover over a cell for its share and count.

Both matrices as raw counts

No augmentation — raw counts, rows are the true class
True class	→ 0	→ 1	→ 2	→ 3	→ 4	→ 5	→ 6	→ 7	→ 8	→ 9	row total
0 · Disturbed	18	1	63	52	0	1	0	75	4	3	217
1 · Merging	6	108	85	113	1	1	0	42	3	11	370
2 · Round smooth	0	2	501	13	0	1	0	8	0	0	525
3 · In-between round smooth	3	2	163	194	0	0	0	9	0	1	372
4 · Cigar-shaped smooth	1	1	5	63	1	0	0	4	0	2	77
5 · Barred spiral	4	6	91	104	0	60	6	147	2	3	423
6 · Unbarred tight spiral	0	2	103	42	0	6	30	179	4	2	368
7 · Unbarred loose spiral	4	5	98	93	0	7	3	319	14	5	548
8 · Edge-on without bulge	3	3	1	14	3	0	0	27	168	62	281
9 · Edge-on with bulge	0	9	14	136	2	1	0	11	8	182	363
Rotation augmentation — raw counts, rows are the true class
True class	→ 0	→ 1	→ 2	→ 3	→ 4	→ 5	→ 6	→ 7	→ 8	→ 9	row total
0 · Disturbed	11	5	45	39	1	11	0	81	7	17	217
1 · Merging	1	187	51	40	0	20	0	17	9	45	370
2 · Round smooth	0	12	479	6	0	11	2	15	0	0	525
3 · In-between round smooth	0	10	115	220	0	3	0	17	0	7	372
4 · Cigar-shaped smooth	0	0	4	23	6	1	0	5	1	37	77
5 · Barred spiral	2	5	35	39	0	254	0	68	8	12	423
6 · Unbarred tight spiral	0	3	39	6	1	170	15	122	6	6	368
7 · Unbarred loose spiral	3	13	39	48	2	137	1	264	24	17	548
8 · Edge-on without bulge	1	1	1	1	0	0	0	5	198	74	281
9 · Edge-on with bulge	1	3	7	13	0	2	0	2	11	324	363

Reliability curves

confidence versus actual accuracy on the test set

No augmentation

Rotation augmentation

Test predictions are grouped by the model's confidence; each point plots a group's average confidence against the share it got right. On the diagonal, confidence matches accuracy; below it, the model is overconfident. Expected calibration error (ECE) sums up the gap; lower is better. Hover over a point for its numbers.

The bins as numbers — including the empty ones

Confidence bin	No augmentation			Rotation augmentation
Confidence bin	n	confidence	accuracy	n	confidence	accuracy
0.0000 – 0.0667	0	—	—	0	—	—
0.0667 – 0.1333	0	—	—	0	—	—
0.1333 – 0.2000	10	18.71%	0.00%	13	18.94%	0.00%
0.2000 – 0.2667	128	24.27%	28.91%	150	23.87%	28.00%
0.2667 – 0.3333	369	30.32%	30.35%	371	30.31%	32.88%
0.3333 – 0.4000	454	36.71%	30.62%	484	36.73%	38.84%
0.4000 – 0.4667	522	43.36%	34.67%	535	43.22%	42.06%
0.4667 – 0.5333	486	49.75%	39.71%	459	49.79%	52.51%
0.5333 – 0.6000	346	56.46%	43.35%	333	56.51%	59.46%
0.6000 – 0.6667	276	62.93%	45.65%	292	63.26%	66.78%
0.6667 – 0.7333	242	69.95%	50.00%	275	69.68%	68.73%
0.7333 – 0.8000	193	76.41%	54.40%	274	76.58%	79.56%
0.8000 – 0.8667	184	83.10%	66.30%	206	83.26%	92.23%
0.8667 – 0.9333	227	90.16%	87.67%	138	89.60%	98.55%
0.9333 – 1.0000	107	95.55%	89.72%	14	94.84%	100.00%

2 of the 15 bins are empty for both models: no prediction fell in them, so they show a dash and the curves skip them.

The same real test galaxy, both predictions

Showing 129 of the 129 stored test galaxies (out of 3,544). Filter them with the buttons, then click a tile to compare the two models' predictions.

Human label: 2 · Round smooth
held-out source row 3,013

No augmentation

Prediction: 1 · Merging

Merging52.3%

Truth · Round30.6%

In-between12.7%

Rotation augmentation

Prediction: 2 · Round smooth

Truth · Round58.5%

Merging16.5%

In-between12.8%

Rotation augmentation fixed this held-out case: the baseline was wrong and the augmented model matched the human label.

The selected case as numbers — all ten stored probabilities, both models

Class	No augmentation	Rotation augmentation
0 · Disturbed	1.18%	8.92%
1 · Merging	52.32%	16.54%
2 · Round smooth (truth)	30.55%	58.49%
3 · In-between round smooth	12.73%	12.80%
4 · Cigar-shaped smooth	0.01%	0.23%
5 · Barred spiral	0.12%	0.22%
6 · Unbarred tight spiral	0.21%	0.33%
7 · Unbarred loose spiral	2.82%	2.31%
8 · Edge-on without bulge	0.01%	0.00%
9 · Edge-on with bulge	0.04%	0.16%

Does the result survive a paired control?

In a second, separately saved run (the paired control), the model trained with rotation started from the same weights as the baseline and saw the training images in the same order, so rotation was the only difference. In that run, rotation lowered test accuracy from 44.61% to 32.45%.

Saved experiment	No rotation accuracy	Rotation accuracy
Independent start (as in the notebook)	44.61%	55.25%
Paired start (paired control)	44.61%	32.45%

What the analysis shows

In the independent-start run, rotation augmentation improves every headline metric

Both models are scored on the same unrotated test galaxies, each after its final (100th) epoch.

accuracy 44.61% → 55.25% (+10.64%) · weighted F1 40.74% → 50.53% (+9.80%)

This reproduces the source notebook's favourable augmentation result.

In the same run, the calibration gap is the larger effect

Expected calibration error (ECE) measures how far a model's confidence is from its actual accuracy, regardless of how often it is right.

ECE 10.05% → 3.09% (−6.96%)

The augmented model is not merely more often right; it is much less overconfident when it is wrong.

A paired control puts rotation behind the baseline

In the independent-start run the two models differ in more than rotation: initial weights, batch order, dropout masks and rotation angles are all independent. The paired control starts both models from the same weights and the same batch order.

paired initialisation no · paired batch order no · augmented seed 0 · one run per model

With the starting point held fixed, rotation lowers test accuracy instead (44.61% → 32.45%), so the gain above is this notebook's result, not an effect of rotation alone.

Neither model separates the ten classes in the space it learned

The silhouette measures how well galaxies of the same human-assigned class group together, apart from other classes, in each 2-D map of the embedding. It is negative for every stored method, parameter setting and model.

t-SNE · perplexity = 15 · learning_rate = auto · Rotation augmentation — silhouette -0.043 · trustworthiness@10 0.995

A classifier can be right more often without pulling the human categories apart in its embedding. The table under the embedding map lists every stored value.

Where these galaxy images and labels come from

How a survey image becomes a labelled training example, and which kinds of galaxy the CNN has to tell apart.

From survey image to supervised-learning sample

DECaLS imaging — an optical survey records each galaxy.
Galaxy Zoo — volunteers answer questions about visible morphology.
Galaxy10 — those answers are grouped into ten broad classes.
This experiment — centred 64 × 64 colour images, one per galaxy.

The target is a human judgement. It is a visual-morphology category, not a spectroscopic measurement, and the ten classes are not equally frequent.

What must the CNN distinguish?

disturbed and merging systems;
three kinds of smooth galaxies;
barred and unbarred spirals;
edge-on disks with or without a bulge.

Why rotation at all. Rotating an image should not change its morphology label. That is the reason for the rotation experiment on this page.

Glossary

Every term this page uses in a specific sense, with what it means here — on this dataset, in this experiment — rather than in general.

design matrix X, and y: as used here: one row of X is one complete galaxy image flattened to its pixel-channel values; y is a separate vector holding that galaxy's human morphology class. The label is never a column of X.
rotation augmentation: as used here: an independently sampled rotation angle applied to a training image every time that row is drawn into a training mini-batch, with the label left unchanged. Validation and test images are never rotated.
epoch: one pass of the optimiser over the whole training set. Because the model trained with rotation draws a fresh angle each pass, it sees a different version of the same images in every epoch, while the baseline sees the identical images each time.
held-out test set: as used here: the portion of the split that neither model was trained or validated on. It is fixed once, before either model exists, and is identical for both models, so their two test scores can be compared directly.
confusion matrix, row-normalized: a table whose rows are true classes and whose columns are predicted classes. Row-normalized means each row is divided by its own total, so a row reads "of the galaxies that really are this class, where did the model send them?" — and class imbalance cannot make a large class look better than a small one.
weighted F1: the per-class F1 scores averaged with each class weighted by how many test galaxies it actually has. On an imbalanced set it therefore says more about the common classes than the rare ones.
reliability curve: predictions are put into confidence bins; each bin is plotted at its mean confidence against the share of it the model actually got right. The diagonal is perfect calibration; below it means overconfident.
expected calibration error (ECE): the average distance between a bin's confidence and its accuracy, weighted by how many predictions fall in that bin. Lower is better. It says nothing about whether the model is accurate — only about whether its confidence can be believed.
logits, and softmax probabilities: the ten raw output numbers of the network, and the same ten numbers after softmax turns them into a distribution that sums to one. The probability bars in the Evaluation section are the latter.
the learned embedding: as used here: the 32 values produced by the last hidden dense layer, after its ReLU and immediately before the ten output logits. It is what the classifier actually sees, which is why the maps are drawn from it rather than from the pixels.
PCA: a linear projection onto the directions of largest variance. Its axes and its explained variance are defined quantities, but a nonlinear structure in the 32-D space can be invisible in two linear components.
t-SNE, and perplexity: a nonlinear method that places points so that local neighbourhoods survive. Perplexity is roughly how many neighbours each point is asked to keep. Cluster spacing, cluster size and the axis values are not globally meaningful.
UMAP, n_neighbors and min_dist: a nonlinear neighbourhood-graph method. n_neighbors trades local detail against broader structure; min_dist controls how tightly points may pack. As with t-SNE, the axes carry no units.
trustworthiness@10: how much of each point's ten nearest neighbours in the original 32-D space survives into the 2-D picture, on a 0–1 scale. High trustworthiness says the picture is a fair local map; it says nothing about whether the classes separate.
silhouette (against the truth labels): as used here: how much closer a point sits to its own human class than to the nearest other class, in the 2-D view. A negative value means the classes overlap. Every value stored on this page is negative, for both models and every method.
stored case: as used here: one of 129 test galaxies whose image, true label and both models' full sets of class probabilities were saved for display. The filter counts in the Evaluation section count within these 129, not the whole test set.
fixed by rotation / broken by rotation: as used here: a stored case the baseline got wrong and the augmented model got right, and the reverse.
independent start, paired start: as used here: the two saved runs of the comparison. In the independent-start run, which is the one shown in most sections of this page, the model trained with rotation starts from its own random weights and batch order, as in the notebook. In the paired-start run (the paired control), it starts from the same weights and batch order as the baseline. The Insight page calls them Independent start and Paired start.

Source evidencewhere the data and the method come from

Notebook lineage

What the saved results record

Design matrixwhat one row, column and cell mean

What a row and a column are here

How a row is built

The matrix itself

The ten target classes are imbalanced

One fixed split, made once

What does each galaxy class look like?

Transformation and modellingrandom rotations · three-block CNN with a dense classification head

Transformation — original views and rotated views

Modelling & training — two fresh copies of one architecture

The two stored training histories

No augmentation

Rotation augmentation

The learned embedding before the final classifier

Evaluationaccuracy 44.61% → 55.25% · paired control reverses the result

Evaluation — the same untouched test set, both models

Confusion matrices

Reliability curves

The same real test galaxy, both predictions

No augmentation

Rotation augmentation

Does the result survive a paired control?

Conclusionswhat the analysis shows

What the analysis shows

In the independent-start run, rotation augmentation improves every headline metric

In the same run, the calibration gap is the larger effect

A paired control puts rotation behind the baseline

Neither model separates the ten classes in the space it learned

Glossaryterms as used in this workflow

Where these galaxy images and labels come from

From survey image to supervised-learning sample

What must the CNN distinguish?

Glossary