A good prediction can still miss the materials we want
Can composition reveal which materials need less cooling?
The question
Given only 81 numbers computed from each superconductor's chemical formula, can a computer predict its critical temperature (below which it loses all electrical resistance) for 6,379 materials it was not trained on, including the rare ones that stay superconducting at 90 K and above?
The answer
Mostly, but not for the hottest materials. The best of five models (a random forest on all 81 features) misses by 5.4 K on average, yet predicts the 649 materials measured at 90 K or above 7.9 K too low on average. Press “Highlight higher temperatures” to see that group.
A superconductor carries current without electrical resistance below its critical temperature. The higher that temperature, the less cooling the material needs, so the hottest materials are the ones worth finding.
How close is the computer’s prediction?
Each dot is one test material, kept out of training: measured temperature across, predicted temperature up. Dots on the dashed diagonal are predicted exactly; dots below it are predicted too low.
Measured below 90 KMeasured 90 K and above
All 6,379 test materials. Click a dot, or pick one under “Inspect an example”, to compare its five predictions.
One material, five predictions
Each bar runs from 0 K to one model's prediction; the vertical line marks the measured temperature. A bar that stops short of the line is a prediction that is too low. The selected model's bar is highlighted.
Play steps through the five examples, one every 1.8 seconds; Previous and Next move one at a time.
How the models were trained and scored
The 21,263 materials were split once at random: 14,884 for training and 6,379 for testing. Each of the five models was trained on the training materials and then predicted every test material; choosing a model in “Prediction model” shows that model's predictions. Two models use all 81 features. The other three use 17 components from principal component analysis (PCA), which compresses the 81 features into 17 combinations that keep 95.5% of their variation in the training data. Polynomial regression is run only on the 17 components.
The feature scaling and the PCA were fitted on the training materials only. The 90 K boundary is the top of the five temperature bands used to score the models. In the table, linear regression and random forest both miss by more with 17 components than with all 81 features. Materials are identified by their row number in the dataset file, counted from 0.
Model
Mean absolute error, all test materials (K)
Average prediction minus measurement, materials ≥ 90 K (K)