Every run gives a different picture, and changing perplexity changes the clusters completely. Which plot is correct?
How it works
Perplexity roughly sets how many neighbors each point considers. A low perplexity focuses on very local structure and can split real groups into small fragments. A high perplexity considers more global structure but can merge distinct groups. The optimization also starts from a random layout, so different runs can end in different arrangements.
Answer 1: Try several perplexity values and trust patterns that persist
Typical values are between 5 and 50; the default is 30. Plot a few side by side. Structure that appears across several values is likely real. Structure that appears at only one value may be an artifact.
from sklearn.manifold import TSNE
import matplotlib.pyplot as plt
fig, axes = plt.subplots(1, 4, figsize=(20, 5))
for ax, perp in zip(axes, [5, 15, 30, 50]):
emb = TSNE(perplexity=perp,
random_state=42).fit_transform(X_scaled)
ax.scatter(emb[:, 0], emb[:, 1], c=y, s=5, cmap="tab10")
ax.set_title(f"perplexity={perp}")
plt.show()
The perplexity must be smaller than the number of samples, which is the cause of the ValueError above. With a small dataset, use a small perplexity, such as 5–10.
