{"version":1,"lectureId":"01M14TYKN0KG2QE2M697DD03FR","attempt":0,"publication":{"slug":"clustering-k-means-and-where-it-fails","title":"K-Means, Its Failure Modes, and Gaussian Mixture Models","subject":"machine-learning","summary":"A practical visual treatment of clustering for working data analysts. The lecture runs k-means from initialization through nearest-centre assignment, mean updates, repetition, and convergence, then tests the method against unlucky starts, elongated groups, unequal cluster sizes, and an imposed value of k. Soft responsibilities lead naturally to Gaussian mixtures, where component weights, means, and covariance matrices model population share, location, spread, elongation, and orientation. The closing workflow emphasizes restarts, scaling, stability, model selection, and domain validation.","metaDescription":"A practical visual guide to k-means iteration, its key failure modes, soft membership, and Gaussian mixtures with learned covariance.","transcript":"Clustering looks for useful groups when the data has no group label. K-means is often the first method an analyst tries because its loop is fast, concrete, and easy to inspect. But its answer contains assumptions. Today we will run that loop, break those assumptions on purpose, and then see what a Gaussian mixture adds. Think of each dot as one row of a data set and each axis as one measured feature. Clustering asks whether rows that sit near one another should be treated as members of the same group. K-means gives a hard answer. Each observation belongs to exactly one cluster, represented by one centre. The rule is simply: choose the nearest centre. A Gaussian mixture can give a softer answer. Instead of one cluster label, it gives one probability for every component. Those probabilities express uncertainty about observations near an overlap. That distinction matters operationally. A hard label is convenient for routing or summarizing records. A probability is useful when a borderline case should remain borderline rather than being forced across a sharp boundary. We will not treat the more flexible model as automatically better. K-means has fewer parameters and is often exactly the right baseline. The point is to know what geometry it can represent, what failures to expect, and when extra probability and covariance parameters have a real job to do. Here is our concrete problem. We have twelve observations, two features, and a requested value k equals two. The algorithm must return two clusters. First put the data on the plane. At this stage the points have no fitted cluster identity. K-means needs starting centres. These two are deliberately away from the middle of the groups. They are guesses, not observations and not yet fitted means. The assignment step examines every observation. For each centre, compute squared Euclidean distance across the features, then give the observation to the centre with the smaller total. After that comparison, these six points belong to the red centre and these six belong to the green centre. The colour is the hard cluster label z. Now freeze the assignments and update one centre at a time. The new red centre is the coordinate-wise mean of the six red observations. The green centre is calculated in exactly the same way from the green observations. A centre is therefore a fitted average, not necessarily a row that actually occurs in the data. Move both centres to those means. As the centres travel, the assigned observations stay fixed. When the centres arrive, one full k-means pass is complete. Then repeat. With the centres in their new positions, calculate every nearest-centre assignment again. In this example every point keeps its colour. Recomputing the two means therefore returns the same two centres. The second pass changes neither the assignments nor the centres. That is convergence. Implementations may stop when assignments are identical, when centre movement is below a tolerance, or when a maximum iteration count is reached. The loop is alternating optimization. With centres fixed, nearest-centre assignment cannot increase the within-cluster sum of squares. With assignments fixed, replacing each centre by its mean cannot increase it either. So the objective falls or stays level on every pass. Because only finitely many hard assignments exist, the procedure eventually stops. What this does not prove is that it found the best possible clustering. It may have stopped at a merely local solution. K-means can finish at different answers on the same data because its objective is not generally convex in all assignments and centres together. Here are two runs with k equal to three. On the left, the starting centres are spread across the visible groups. On the right, an unlucky draw crowds all three starting centres into the left-hand group. Run the same assignment and update loop on both sides. The favourable start settles with one centre in each visible group. The unlucky run spends two centres splitting the left group. Its remaining centre absorbs both right-hand groups, even though they are visibly separated. Another pass changes neither answer. Both are local solutions, but the right-hand solution has a larger within-cluster sum of squares and a much less useful interpretation. The practical remedy is not to trust one random start. K-means++ spreads seeds by favouring observations far from centres already chosen. It reduces bad starts, but does not abolish local minima. For routine analysis, make initialization part of the specification. Use a dispersed seeding rule, then run many independent starts. Retain the run with the lowest objective, but still inspect its groups. A numerical improvement can be immaterial, and several nearly tied runs can imply genuine ambiguity in the data. Record the number of starts, the random seed, and the spread of objective values. Reproducibility here is not clerical detail. It is evidence about how strongly the data supports the fitted partition. The nearest-centre rule creates Voronoi cells: regions separated by straight boundaries. That simple geometry explains several important failures. First, consider two long natural groups. Their gray outlines run diagonally across the feature space. With these fitted centres, nearest-centre assignment makes a nearly vertical cut. Red and green divide both long groups across their length rather than following the two elongated densities. K-means has a centre for location, but no parameter for orientation or different spread in different directions. Squared Euclidean distance therefore prefers compact, roughly spherical clusters. Second, place one small dense cluster beside one large diffuse cluster. The gray circles show the intended populations. The large group contributes many more squared distances to the objective. A centre can reduce that cost by moving toward it, while the boundary gives part of the large group to the small cluster. The result is mathematically consistent with the objective, but unfair to the smaller group as a density. K-means does not model cluster population or variance separately. Third, this is one connected cloud without three clear density peaks. We nevertheless request k equals three. K-means does not answer that the data contains one group. It returns exactly three nonempty partitions because three centres were requested. This is not a software defect. The value k is an input to the problem, not a conclusion reached by the algorithm. A tidy colour map does not prove that the corresponding groups exist. Before believing a k-means result, ask whether compact clusters are plausible in the scaled feature space, and whether their spreads and populations are comparable. Then test stability across initializations and resampled data. Finally, ask what k means in the business or scientific setting. If it has no interpretation, treat it as a modelling choice to validate, not a fact discovered by the colours. To relax hard assignment, keep one membership number for every component. For observation i and component j, call that number the responsibility r i j. Consider this observation near the overlap. A hard rule must choose one side. The mixture instead evaluates how plausible the observation is under both fitted components. Suppose the resulting responsibilities are zero point three five and zero point six five. The second component is more plausible, but the first remains credible. The two values add to one. These are model-based probabilities, not calibrated truth supplied by the data. They depend on the fitted component shapes, their population weights, and the Gaussian mixture assumptions. A Gaussian mixture says that the observed density is a weighted sum of Gaussian component densities. The extra parameters each answer a distinct question. The mixture weight pi records the expected share of observations generated by a component. Unlike k-means, the model can explicitly represent one large component and one small component. The mean mu gives location, much like a k-means centre. Here the two means sit near the middle of their respective densities. The covariance matrix Sigma is the crucial addition. Its overall scale controls spread. Its unequal directional variances create elongation, and its off-diagonal relationship rotates that elongation. These ellipses are equal-density contours. They can follow the long axes of the two groups, which is exactly the geometric freedom missing from nearest-centre k-means. The usual fitting procedure alternates two weighted steps. In the E step, use the current weights, means, and covariances to compute every responsibility. In the M step, treat those responsibilities as fractional counts. Update each mixture weight from its total responsibility. Update each mean as a responsibility-weighted average. Then update each covariance from weighted deviations around that mean. Repeat the two steps until log likelihood no longer improves materially. Like k-means, this alternating fit can reach local optima, so restarts still matter. Return to the elongated example. On the left, k-means uses a straight nearest-centre boundary and slices across both natural groups. On the right, the Gaussian mixture learns two covariance ellipses. Their orientations follow the groups, and observations in the overlap can retain intermediate responsibilities instead of being treated as certain. The gain is real, but so is the cost. More parameters require more data, covariance estimates can become unstable, and a Gaussian component can chase a tiny group unless the fit is regularized and checked. K-means and Gaussian mixtures answer related but different modelling questions. K-means asks for a hard partition that minimizes squared distance to centres. A Gaussian mixture asks for a probability density assembled from weighted components. It returns probabilistic membership and can learn a different covariance shape for every component. Choose k-means when a fast, interpretable hard partition is useful and compact groups are plausible after sensible scaling. Choose a Gaussian mixture when overlap, unequal populations, or elliptical covariance structure is part of the question. Neither model determines the scientifically correct number of groups by itself. K-means returns the k requested. A mixture with k components also fits the requested count, even if some fitted components become tiny or redundant. A defensible workflow begins before fitting. Scale features according to what distance should mean, then use several starts and check whether the result is stable. Compare several credible values of k. For k-means, inspect objective curves and resampling stability. For mixtures, likelihood criteria such as B I C can help penalize unnecessary parameters, but none of these replaces domain judgment. Inspect the fitted objects themselves. Look for clusters created only by one scale, uncertain responsibilities, tiny components, extreme covariance estimates, and solutions that change sharply across restarts. Finally, judge the clustering by the decision it supports. A useful segment, anomaly group, or population subtype must remain interpretable and stable where it will actually be used. The durable lesson is simple. K-means alternates nearest-centre assignment with mean updates, and that simplicity creates both its speed and its geometric limits. Gaussian mixtures replace certainty with responsibilities and add weights and covariance, but those extra freedoms deserve validation because they can fit both real structure and noise.","watch":{"version":1,"scenes":[{"title":"Two Kinds of Membership","start":0,"end":95.01412499999999,"objects":{"ambiguous":"a Point [yellow] labelled \"x_i\" drawn in scatter (location=(0.1, 0.2))","card":"a Title that says \"Practical Unsupervised Learning — K-Means, Its Failure Modes, and Gaussian Mixture Models\"","centre_1":"a Point [red] labelled \"mu_1\" drawn in scatter (location=(-2.2, 0.1))","centre_2":"a Point [green] labelled \"mu_2\" drawn in scatter (location=(2.1, 0.0))","hard_rule":"a Math [text] that says \"$upright(\"k-means:\") thin x_i arrow.r upright(\"one nearest centre\")$\"","heading":"a Heading that says \"What Does It Mean to Belong?\"","left_points":"a Point [red] drawn in scatter (location=(-3.1, -0.2))","left_points_2":"a Point [red] drawn in scatter (location=(-2.8, 0.7))","left_points_3":"a Point [red] drawn in scatter (location=(-2.4, -0.8))","left_points_4":"a Point [red] drawn in scatter (location=(-2.0, 0.3))","left_points_5":"a Point [red] drawn in scatter (location=(-1.6, 1.0))","left_points_6":"a Point [red] drawn in scatter (location=(-1.2, -0.4))","question":"a Panel that says \"Given only feature measurements, which observations form useful groups?\"","right_points":"a Point [green] drawn in scatter (location=(1.0, 0.3))","right_points_2":"a Point [green] drawn in scatter (location=(1.5, -0.8))","right_points_3":"a Point [green] drawn in scatter (location=(1.9, 0.9))","right_points_4":"a Point [green] drawn in scatter (location=(2.4, -0.1))","right_points_5":"a Point [green] drawn in scatter (location=(2.8, 0.7))","right_points_6":"a Point [green] drawn in scatter (location=(3.1, -0.7))","scatter":"an Axes (x_range=(-4.0, 4.0), y_range=(-3.0, 3.0), include_ticks=False)","soft_definition":"a Panel that says \"A responsibility $r_(i j)$ is the fitted probability that observation $i$ belongs to component $j$. For one observation, the responsibilities add to one.\"","soft_rule":"a Math [text] that says \"$upright(\"mixture:\") thin x_i arrow.r (r_(i 1), dots, r_(i k))$\"","to_first":"a Line [gray] drawn in scatter (start=(0.1, 0.2), end=(-2.2, 0.1), dashed=True)","to_second":"a Line [gray] drawn in scatter (start=(0.1, 0.2), end=(2.1, 0.0), dashed=True)"},"beats":[{"start":0,"say":"Clustering looks for useful groups when the data has no group label. K-means is often the first method an analyst tries because its loop is fast, concrete, and easy to inspect. But its answer contains assumptions. Today we will run that loop, break those assumptions on purpose, and then see what a Gaussian mixture adds.","live":[],"does":[[0,"card is shown on the screen, written out."],[1.5,"card: enter:write-left-to-right."],[20.77,"card is hidden from the screen — left the board."]]},{"start":21.97,"say":"Think of each dot as one row of a data set and each axis as one measured feature. Clustering asks whether rows that sit near one another should be treated as members of the same group.","live":null,"does":[[21.97,"heading is shown on the screen, written out."],[21.97,"question is shown on the screen, written out."],[22.933,"scatter is shown on the screen, written out."],[29.365,"left_points is shown on the screen, written out."],[29.365,"left_points_2 is shown on the screen, written out."],[29.365,"left_points_3 is shown on the screen, written out."],[29.365,"left_points_4 is shown on the screen, written out."],[29.365,"left_points_5 is shown on the screen, written out."],[29.365,"left_points_6 is shown on the screen, written out."],[29.365,"right_points is shown on the screen, written out."],[29.365,"right_points_2 is shown on the screen, written out."],[29.365,"right_points_3 is shown on the screen, written out."],[29.365,"right_points_4 is shown on the screen, written out."],[29.365,"right_points_5 is shown on the screen, written out."],[29.365,"right_points_6 is shown on the screen, written out."]]},{"start":34.3425,"say":"K-means gives a hard answer. Each observation belongs to exactly one cluster, represented by one centre. The rule is simply: choose the nearest centre.","live":["question","scatter","heading","left_points","left_points_2","left_points_3","left_points_4","left_points_5","left_points_6","right_points","right_points_2","right_points_3","right_points_4","right_points_5","right_points_6"],"does":[[41.099000000000004,"centre_1 is shown on the screen, written out."],[41.099000000000004,"centre_2 is shown on the screen, written out."],[44.18800000000001,"hard_rule is shown on the screen, written out."],[44.18800000000001,"hard_rule (the \"upright(\"one nearest centre\")\" part) is emphasized."],[45.3835,"hard_rule (the \"upright(\"one nearest centre\")\" part) is no longer emphasized."]]},{"start":45.9835,"say":"A Gaussian mixture can give a softer answer. Instead of one cluster label, it gives one probability for every component. Those probabilities express uncertainty about observations near an overlap.","live":["question","hard_rule","scatter","heading","left_points","left_points_2","left_points_3","left_points_4","left_points_5","left_points_6","right_points","right_points_2","right_points_3","right_points_4","right_points_5","right_points_6","centre_1","centre_2"],"does":[[47.701,"hard_rule is hidden from the screen — left the board."],[47.701,"question is hidden from the screen — left the board."],[47.701,"soft_rule is shown on the screen, written out."],[52.019999999999996,"soft_definition is shown on the screen, written out."],[56.699,"ambiguous is shown on the screen, written out."],[57.79,"to_first is shown on the screen, written out."],[57.79,"to_second is shown on the screen, written out."]]},{"start":59.284,"say":"That distinction matters operationally. A hard label is convenient for routing or summarizing records. A probability is useful when a borderline case should remain borderline rather than being forced across a sharp boundary.","live":["soft_rule","scatter","heading","left_points","left_points_2","left_points_3","left_points_4","left_points_5","left_points_6","right_points","right_points_2","right_points_3","right_points_4","right_points_5","right_points_6","centre_1","centre_2","soft_definition","ambiguous","to_first","to_second"],"does":[[68.213,"ambiguous is indicated — a transient flash."]]},{"start":73.80499999999999,"say":"We will not treat the more flexible model as automatically better. K-means has fewer parameters and is often exactly the right baseline. The point is to know what geometry it can represent, what failures to expect, and when extra probability and covariance parameters have a real job to do.","live":null,"does":[[78.193,"centre_1 is indicated — a transient flash."],[78.393,"centre_2 is indicated — a transient flash."],[93.97245833333332,"heading is hidden from the screen — left the board."],[93.97245833333332,"scatter is hidden from the screen — left the board."],[93.97245833333332,"left_points is hidden from the screen — scatter left the board."],[93.97245833333332,"left_points_2 is hidden from the screen — scatter left the board."],[93.97245833333332,"left_points_3 is hidden from the screen — scatter left the board."],[93.97245833333332,"left_points_4 is hidden from the screen — scatter left the board."],[93.97245833333332,"left_points_5 is hidden from the screen — scatter left the board."],[93.97245833333332,"left_points_6 is hidden from the screen — scatter left the board."],[93.97245833333332,"right_points is hidden from the screen — scatter left the board."],[93.97245833333332,"right_points_2 is hidden from the screen — scatter left the board."],[93.97245833333332,"right_points_3 is hidden from the screen — scatter left the board."],[93.97245833333332,"right_points_4 is hidden from the screen — scatter left the board."],[93.97245833333332,"right_points_5 is hidden from the screen — scatter left the board."],[93.97245833333332,"right_points_6 is hidden from the screen — scatter left the board."],[93.97245833333332,"centre_1 is hidden from the screen — scatter left the board."],[93.97245833333332,"centre_2 is hidden from the screen — scatter left the board."],[93.97245833333332,"ambiguous is hidden from the screen — scatter left the board."],[93.97245833333332,"to_first is hidden from the screen — scatter left the board."],[93.97245833333332,"to_second is hidden from the screen — scatter left the board."],[93.97245833333332,"soft_definition is hidden from the screen — left the board."],[93.97245833333332,"soft_rule is hidden from the screen — left the board."]]}]},{"title":"Run K-Means","start":95.01412499999999,"end":242.11149999999998,"objects":{"assign_rule":"a Math [text] that says \"$z_i = op(\"arg min\")_j sum_l (x_(i l) - mu_(j l))^2$\"","assigned_a":"a Point [red] drawn in plot (location=(-3.2, 1.0))","assigned_a_2":"a Point [red] drawn in plot (location=(-2.8, 1.5))","assigned_a_3":"a Point [red] drawn in plot (location=(-2.5, 0.8))","assigned_a_4":"a Point [red] drawn in plot (location=(-3.4, 0.2))","assigned_a_5":"a Point [red] drawn in plot (location=(-2.7, 0.0))","assigned_a_6":"a Point [red] drawn in plot (location=(-2.0, 1.1))","assigned_b":"a Point [green] drawn in plot (location=(1.5, -1.0))","assigned_b_2":"a Point [green] drawn in plot (location=(2.1, -1.4))","assigned_b_3":"a Point [green] drawn in plot (location=(2.8, -0.7))","assigned_b_4":"a Point [green] drawn in plot (location=(1.9, -0.2))","assigned_b_5":"a Point [green] drawn in plot (location=(3.2, -1.5))","assigned_b_6":"a Point [green] drawn in plot (location=(2.7, 0.1))","centre_1":"a Point [red] labelled \"mu_1\" drawn in plot (location=(<VariableNumber centre_1_x = -2.7666666667>, <VariableNumber c…, marker_radius=0.11)","centre_1_x":"a VariableNumber (initial_value=-3.5)","centre_1_y":"a VariableNumber (initial_value=-1.5)","centre_2":"a Point [green] labelled \"mu_2\" drawn in plot (location=(<VariableNumber centre_2_x = 2.3666666667>, <VariableNumber ce…, marker_radius=0.11)","centre_2_x":"a VariableNumber (initial_value=3.5)","centre_2_y":"a VariableNumber (initial_value=1.5)","history":"a Table [text] that says \"Pass Assignments Centres 1 red or green move 2 unchanged unchanged\" (rows=(('Pass', 'Assignments', 'Centres'), ('1', 'red or green', 'mov…, header=True)","mean_1":"a Point [yellow] labelled \"overline(x)_1\" drawn in plot (location=(-2.7666666667, 0.7666666667))","mean_2":"a Point [yellow] labelled \"overline(x)_2\" drawn in plot (location=(2.3666666667, -0.7833333333))","move_1":"a Vector [yellow] drawn in plot (start=(-3.5, -1.5), end=(-2.7666666667, 0.7666666667))","move_2":"a Vector [yellow] drawn in plot (start=(3.5, 1.5), end=(2.3666666667, -0.7833333333))","neutral_points":"a Point [gray] drawn in plot (location=(-3.2, 1.0))","neutral_points_10":"a Point [gray] drawn in plot (location=(1.9, -0.2))","neutral_points_11":"a Point [gray] drawn in plot (location=(3.2, -1.5))","neutral_points_12":"a Point [gray] drawn in plot (location=(2.7, 0.1))","neutral_points_2":"a Point [gray] drawn in plot (location=(-2.8, 1.5))","neutral_points_3":"a Point [gray] drawn in plot (location=(-2.5, 0.8))","neutral_points_4":"a Point [gray] drawn in plot (location=(-3.4, 0.2))","neutral_points_5":"a Point [gray] drawn in plot (location=(-2.7, 0.0))","neutral_points_6":"a Point [gray] drawn in plot (location=(-2.0, 1.1))","neutral_points_7":"a Point [gray] drawn in plot (location=(1.5, -1.0))","neutral_points_8":"a Point [gray] drawn in plot (location=(2.1, -1.4))","neutral_points_9":"a Point [gray] drawn in plot (location=(2.8, -0.7))","objective":"a Math [text] that says \"$J = sum_i sum_l (x_(i l) - mu_(z_i l))^2$\"","plot":"an Axes (x_range=(-4.5, 4.5), y_range=(-2.5, 2.5), include_ticks=False)","question":"a Panel that says \"Starting from two centres, how does k-means turn this scatter into two clusters?\"","stop_rule":"a Math [text] that says \"$upright(\"stop when\") thin z thin upright(\"or\") thin mu thin upright(\"does not change\")$\"","update_rule":"a Math [text] that says \"$mu_j = frac(1, n_j) sum_(i: z_i=j) x_i$\""},"beats":[{"start":95.01412499999999,"say":"Here is our concrete problem. We have twelve observations, two features, and a requested value k equals two. The algorithm must return two clusters.","live":[],"does":[[95.01412499999999,"question is shown on the screen, written out."],[105.38162499999999,"question moves to a new place on the board."]]},{"start":105.981625,"say":"First put the data on the plane. At this stage the points have no fitted cluster identity.","live":["question"],"does":[[105.981625,"plot is shown on the screen, written out."],[109.51112499999999,"neutral_points is shown on the screen, written out."],[109.51112499999999,"neutral_points_2 is shown on the screen, written out."],[109.51112499999999,"neutral_points_3 is shown on the screen, written out."],[109.51112499999999,"neutral_points_4 is shown on the screen, written out."],[109.51112499999999,"neutral_points_5 is shown on the screen, written out."],[109.51112499999999,"neutral_points_6 is shown on the screen, written out."],[109.51112499999999,"neutral_points_7 is shown on the screen, written out."],[109.51112499999999,"neutral_points_8 is shown on the screen, written out."],[109.51112499999999,"neutral_points_9 is shown on the screen, written out."],[109.51112499999999,"neutral_points_10 is shown on the screen, written out."],[109.51112499999999,"neutral_points_11 is shown on the screen, written out."],[109.51112499999999,"neutral_points_12 is shown on the screen, written out."]]},{"start":112.51412499999999,"say":"K-means needs starting centres. These two are deliberately away from the middle of the groups. They are guesses, not observations and not yet fitted means.","live":["question","plot","neutral_points","neutral_points_2","neutral_points_3","neutral_points_4","neutral_points_5","neutral_points_6","neutral_points_7","neutral_points_8","neutral_points_9","neutral_points_10","neutral_points_11","neutral_points_12"],"does":[[114.66212499999999,"centre_1 is shown on the screen, written out."],[114.66212499999999,"centre_2 is shown on the screen, written out."]]},{"start":124.28312499999998,"say":"The assignment step examines every observation. For each centre, compute squared Euclidean distance across the features, then give the observation to the centre with the smaller total.","live":["question","plot","neutral_points","neutral_points_2","neutral_points_3","neutral_points_4","neutral_points_5","neutral_points_6","neutral_points_7","neutral_points_8","neutral_points_9","neutral_points_10","neutral_points_11","neutral_points_12","centre_1","centre_2"],"does":[[124.94512499999999,"plot moves to a new place on the board."],[124.94512499999999,"assign_rule is shown on the screen, written out."],[129.600125,"assign_rule (the \"sum_l (x_(i l) - mu_(j l))^2\" part) is emphasized."],[135.649125,"assign_rule (the \"sum_l (x_(i l) - mu_(j l))^2\" part) is no longer emphasized."]]},{"start":136.249125,"say":"After that comparison, these six points belong to the red centre and these six belong to the green centre. The colour is the hard cluster label z.","live":["question","plot","assign_rule","neutral_points","neutral_points_2","neutral_points_3","neutral_points_4","neutral_points_5","neutral_points_6","neutral_points_7","neutral_points_8","neutral_points_9","neutral_points_10","neutral_points_11","neutral_points_12","centre_1","centre_2"],"does":[[137.120125,"neutral_points is hidden from the screen."],[137.120125,"neutral_points_2 is hidden from the screen."],[137.120125,"neutral_points_3 is hidden from the screen."],[137.120125,"neutral_points_4 is hidden from the screen."],[137.120125,"neutral_points_5 is hidden from the screen."],[137.120125,"neutral_points_6 is hidden from the screen."],[137.120125,"neutral_points_7 is hidden from the screen."],[137.120125,"neutral_points_8 is hidden from the screen."],[137.120125,"neutral_points_9 is hidden from the screen."],[137.120125,"neutral_points_10 is hidden from the screen."],[137.120125,"neutral_points_11 is hidden from the screen."],[137.120125,"neutral_points_12 is hidden from the screen."],[139.500125,"assigned_a is shown on the screen, written out."],[139.500125,"assigned_a_2 is shown on the screen, written out."],[139.500125,"assigned_a_3 is shown on the screen, written out."],[139.500125,"assigned_a_4 is shown on the screen, written out."],[139.500125,"assigned_a_5 is shown on the screen, written out."],[139.500125,"assigned_a_6 is shown on the screen, written out."],[141.613125,"assigned_b is shown on the screen, written out."],[141.613125,"assigned_b_2 is shown on the screen, written out."],[141.613125,"assigned_b_3 is shown on the screen, written out."],[141.613125,"assigned_b_4 is shown on the screen, written out."],[141.613125,"assigned_b_5 is shown on the screen, written out."],[141.613125,"assigned_b_6 is shown on the screen, written out."],[144.782125,"assign_rule (the \"z_i\" part) is emphasized."],[145.85062499999998,"assign_rule (the \"z_i\" part) is no longer emphasized."]]},{"start":146.450625,"say":"Now freeze the assignments and update one centre at a time. The new red centre is the coordinate-wise mean of the six red observations.","live":["question","plot","assign_rule","centre_1","centre_2","assigned_a","assigned_a_2","assigned_a_3","assigned_a_4","assigned_a_5","assigned_a_6","assigned_b","assigned_b_2","assigned_b_3","assigned_b_4","assigned_b_5","assigned_b_6"],"does":[[148.250125,"update_rule is shown on the screen, written out."],[150.92012499999998,"move_1 is shown on the screen, written out."],[152.59212499999998,"mean_1 is shown on the screen, written out."]]},{"start":155.63062499999998,"say":"The green centre is calculated in exactly the same way from the green observations. A centre is therefore a fitted average, not necessarily a row that actually occurs in the data.","live":["question","plot","assign_rule","update_rule","centre_1","centre_2","assigned_a","assigned_a_2","assigned_a_3","assigned_a_4","assigned_a_5","assigned_a_6","assigned_b","assigned_b_2","assigned_b_3","assigned_b_4","assigned_b_5","assigned_b_6","mean_1","move_1"],"does":[[156.153125,"move_2 is shown on the screen, written out."],[158.498125,"mean_2 is shown on the screen, written out."],[162.60812499999997,"update_rule (the \"frac(1, n_j)\" part) is emphasized."],[166.520625,"update_rule (the \"frac(1, n_j)\" part) is no longer emphasized."]]},{"start":167.120625,"say":"Move both centres to those means. As the centres travel, the assigned observations stay fixed. When the centres arrive, one full k-means pass is complete.","live":["question","plot","assign_rule","update_rule","centre_1","centre_2","assigned_a","assigned_a_2","assigned_a_3","assigned_a_4","assigned_a_5","assigned_a_6","assigned_b","assigned_b_2","assigned_b_3","assigned_b_4","assigned_b_5","assigned_b_6","mean_1","move_1","mean_2","move_2"],"does":[[170.835125,"centre_1 is redrawn as the numbers it depends on change."],[170.835125,"centre_2 is redrawn as the numbers it depends on change."],[170.835125,"centre_1_x ticks to -2.7666666667."],[170.835125,"centre_1_y ticks to 0.7666666667."],[170.835125,"centre_2_x ticks to 2.3666666667."],[170.835125,"centre_2_y ticks to -0.7833333333."],[178.17262499999998,"update_rule moves to a new place on the board."],[178.17262499999998,"assign_rule is hidden from the screen — left the board."]]},{"start":178.772625,"say":"Then repeat. With the centres in their new positions, calculate every nearest-centre assignment again.","live":["question","plot","update_rule","centre_1","centre_2","assigned_a","assigned_a_2","assigned_a_3","assigned_a_4","assigned_a_5","assigned_a_6","assigned_b","assigned_b_2","assigned_b_3","assigned_b_4","assigned_b_5","assigned_b_6","mean_1","move_1","mean_2","move_2"],"does":[[178.772625,"mean_1 is hidden from the screen."],[178.772625,"mean_2 is hidden from the screen."],[178.772625,"move_1 is hidden from the screen."],[178.772625,"move_2 is hidden from the screen."],[179.469125,"stop_rule is shown on the screen, written out."],[179.469125,"history is shown on the screen, written out."],[181.88412499999998,"history is shown on the screen, written out."]]},{"start":186.257625,"say":"In this example every point keeps its colour. Recomputing the two means therefore returns the same two centres. The second pass changes neither the assignments nor the centres.","live":["question","plot","update_rule","centre_1","centre_2","assigned_a","assigned_a_2","assigned_a_3","assigned_a_4","assigned_a_5","assigned_a_6","assigned_b","assigned_b_2","assigned_b_3","assigned_b_4","assigned_b_5","assigned_b_6","stop_rule"],"does":[[188.405125,"assigned_a is indicated — a transient flash."],[188.405125,"assigned_a_2 is indicated — a transient flash."],[188.405125,"assigned_a_3 is indicated — a transient flash."],[188.405125,"assigned_a_4 is indicated — a transient flash."],[188.405125,"assigned_a_5 is indicated — a transient flash."],[188.405125,"assigned_a_6 is indicated — a transient flash."],[188.405125,"assigned_b is indicated — a transient flash."],[188.405125,"assigned_b_2 is indicated — a transient flash."],[188.405125,"assigned_b_3 is indicated — a transient flash."],[188.405125,"assigned_b_4 is indicated — a transient flash."],[188.405125,"assigned_b_5 is indicated — a transient flash."],[188.405125,"assigned_b_6 is indicated — a transient flash."],[194.19912499999998,"history is shown on the screen, written out."]]},{"start":198.177125,"say":"That is convergence. Implementations may stop when assignments are identical, when centre movement is below a tolerance, or when a maximum iteration count is reached.","live":null,"does":[[201.823125,"stop_rule (the \"z\" part) is emphasized."],[203.854125,"stop_rule (the \"mu\" part) is emphasized."],[203.854125,"stop_rule (the \"z\" part) is no longer emphasized."],[208.835625,"stop_rule moves to a new place on the board."],[208.835625,"history is hidden from the screen — left the board."],[208.835625,"update_rule is hidden from the screen — left the board."],[208.835625,"stop_rule (the \"mu\" part) is no longer emphasized."]]},{"start":209.435625,"say":"The loop is alternating optimization. With centres fixed, nearest-centre assignment cannot increase the within-cluster sum of squares. With assignments fixed, replacing each centre by its mean cannot increase it either.","live":["question","plot","centre_1","centre_2","assigned_a","assigned_a_2","assigned_a_3","assigned_a_4","assigned_a_5","assigned_a_6","assigned_b","assigned_b_2","assigned_b_3","assigned_b_4","assigned_b_5","assigned_b_6","stop_rule"],"does":[[217.07512499999999,"objective is shown on the screen, written out."],[217.07512499999999,"objective (the \"J\" part) is emphasized."],[223.762625,"objective (the \"J\" part) is no longer emphasized."]]},{"start":224.36262499999998,"say":"So the objective falls or stays level on every pass. Because only finitely many hard assignments exist, the procedure eventually stops. What this does not prove is that it found the best possible clustering. It may have stopped at a merely local solution.","live":["question","plot","centre_1","centre_2","assigned_a","assigned_a_2","assigned_a_3","assigned_a_4","assigned_a_5","assigned_a_6","assigned_b","assigned_b_2","assigned_b_3","assigned_b_4","assigned_b_5","assigned_b_6","stop_rule","objective"],"does":[[225.72112499999997,"A box is drawn around objective."],[241.06983333333335,"objective is hidden from the screen — left the board."],[241.06983333333335,"plot is hidden from the screen — left the board."],[241.06983333333335,"centre_1 is hidden from the screen — plot left the board."],[241.06983333333335,"centre_2 is hidden from the screen — plot left the board."],[241.06983333333335,"assigned_a is hidden from the screen — plot left the board."],[241.06983333333335,"assigned_a_2 is hidden from the screen — plot left the board."],[241.06983333333335,"assigned_a_3 is hidden from the screen — plot left the board."],[241.06983333333335,"assigned_a_4 is hidden from the screen — plot left the board."],[241.06983333333335,"assigned_a_5 is hidden from the screen — plot left the board."],[241.06983333333335,"assigned_a_6 is hidden from the screen — plot left the board."],[241.06983333333335,"assigned_b is hidden from the screen — plot left the board."],[241.06983333333335,"assigned_b_2 is hidden from the screen — plot left the board."],[241.06983333333335,"assigned_b_3 is hidden from the screen — plot left the board."],[241.06983333333335,"assigned_b_4 is hidden from the screen — plot left the board."],[241.06983333333335,"assigned_b_5 is hidden from the screen — plot left the board."],[241.06983333333335,"assigned_b_6 is hidden from the screen — plot left the board."],[241.06983333333335,"question is hidden from the screen — left the board."],[241.06983333333335,"stop_rule is hidden from the screen — left the board."]]}]},{"title":"Initialization Changes the Answer","start":242.11149999999998,"end":349.1395,"objects":{"bad":"a Figure (x_range=(-4.2, 3.5), y_range=(-2.6, 2.6), aspect=(7.7, 5.2))","bad_centres":"a Point [red] labelled \"mu_1\" drawn in bad (location=(-3.13, -0.43))","bad_centres_2":"a Point [green] labelled \"mu_2\" drawn in bad (location=(-2.77, 0.8))","bad_centres_3":"a Point [blue] labelled \"mu_3\" drawn in bad (location=(1.98, 0.07))","bad_left_high":"a Point [green] drawn in bad (location=(-3.0, 0.8))","bad_left_high_2":"a Point [green] drawn in bad (location=(-2.4, 0.4))","bad_left_high_3":"a Point [green] drawn in bad (location=(-2.9, 1.2))","bad_left_low":"a Point [red] drawn in bad (location=(-3.5, -0.8))","bad_left_low_2":"a Point [red] drawn in bad (location=(-3.3, 0.0))","bad_left_low_3":"a Point [red] drawn in bad (location=(-2.6, -0.5))","bad_neutral":"a Point [gray] drawn in bad (location=(-3.5, -0.8))","bad_neutral_10":"a Point [gray] drawn in bad (location=(2.3, 2.0))","bad_neutral_11":"a Point [gray] drawn in bad (location=(2.6, 1.2))","bad_neutral_12":"a Point [gray] drawn in bad (location=(1.2, -1.0))","bad_neutral_13":"a Point [gray] drawn in bad (location=(1.6, -1.7))","bad_neutral_14":"a Point [gray] drawn in bad (location=(2.1, -1.3))","bad_neutral_15":"a Point [gray] drawn in bad (location=(2.5, -1.9))","bad_neutral_16":"a Point [gray] drawn in bad (location=(2.8, -0.8))","bad_neutral_2":"a Point [gray] drawn in bad (location=(-3.3, 0.0))","bad_neutral_3":"a Point [gray] drawn in bad (location=(-3.0, 0.8))","bad_neutral_4":"a Point [gray] drawn in bad (location=(-2.6, -0.5))","bad_neutral_5":"a Point [gray] drawn in bad (location=(-2.4, 0.4))","bad_neutral_6":"a Point [gray] drawn in bad (location=(-2.9, 1.2))","bad_neutral_7":"a Point [gray] drawn in bad (location=(1.2, 1.1))","bad_neutral_8":"a Point [gray] drawn in bad (location=(1.5, 1.7))","bad_neutral_9":"a Point [gray] drawn in bad (location=(2.0, 1.4))","bad_right_merged":"a Point [blue] drawn in bad (location=(1.2, 1.1))","bad_right_merged_10":"a Point [blue] drawn in bad (location=(2.8, -0.8))","bad_right_merged_2":"a Point [blue] drawn in bad (location=(1.5, 1.7))","bad_right_merged_3":"a Point [blue] drawn in bad (location=(2.0, 1.4))","bad_right_merged_4":"a Point [blue] drawn in bad (location=(2.3, 2.0))","bad_right_merged_5":"a Point [blue] drawn in bad (location=(2.6, 1.2))","bad_right_merged_6":"a Point [blue] drawn in bad (location=(1.2, -1.0))","bad_right_merged_7":"a Point [blue] drawn in bad (location=(1.6, -1.7))","bad_right_merged_8":"a Point [blue] drawn in bad (location=(2.1, -1.3))","bad_right_merged_9":"a Point [blue] drawn in bad (location=(2.5, -1.9))","bad_seeds":"a Point [yellow] labelled \"1\" drawn in bad (location=(-3.6, -0.9))","bad_seeds_2":"a Point [yellow] labelled \"2\" drawn in bad (location=(-3.1, 0.1))","bad_seeds_3":"a Point [yellow] labelled \"3\" drawn in bad (location=(-2.5, 0.9))","good":"a Figure (x_range=(-4.2, 3.5), y_range=(-2.6, 2.6), aspect=(7.7, 5.2))","good_bottom":"a Point [blue] drawn in good (location=(1.2, -1.0))","good_bottom_2":"a Point [blue] drawn in good (location=(1.6, -1.7))","good_bottom_3":"a Point [blue] drawn in good (location=(2.1, -1.3))","good_bottom_4":"a Point [blue] drawn in good (location=(2.5, -1.9))","good_bottom_5":"a Point [blue] drawn in good (location=(2.8, -0.8))","good_centres":"a Point [red] labelled \"mu_1\" drawn in good (location=(-2.95, 0.18))","good_centres_2":"a Point [green] labelled \"mu_2\" drawn in good (location=(1.92, 1.48))","good_centres_3":"a Point [blue] labelled \"mu_3\" drawn in good (location=(2.04, -1.34))","good_left":"a Point [red] drawn in good (location=(-3.5, -0.8))","good_left_2":"a Point [red] drawn in good (location=(-3.3, 0.0))","good_left_3":"a Point [red] drawn in good (location=(-3.0, 0.8))","good_left_4":"a Point [red] drawn in good (location=(-2.6, -0.5))","good_left_5":"a Point [red] drawn in good (location=(-2.4, 0.4))","good_left_6":"a Point [red] drawn in good (location=(-2.9, 1.2))","good_neutral":"a Point [gray] drawn in good (location=(-3.5, -0.8))","good_neutral_10":"a Point [gray] drawn in good (location=(2.3, 2.0))","good_neutral_11":"a Point [gray] drawn in good (location=(2.6, 1.2))","good_neutral_12":"a Point [gray] drawn in good (location=(1.2, -1.0))","good_neutral_13":"a Point [gray] drawn in good (location=(1.6, -1.7))","good_neutral_14":"a Point [gray] drawn in good (location=(2.1, -1.3))","good_neutral_15":"a Point [gray] drawn in good (location=(2.5, -1.9))","good_neutral_16":"a Point [gray] drawn in good (location=(2.8, -0.8))","good_neutral_2":"a Point [gray] drawn in good (location=(-3.3, 0.0))","good_neutral_3":"a Point [gray] drawn in good (location=(-3.0, 0.8))","good_neutral_4":"a Point [gray] drawn in good (location=(-2.6, -0.5))","good_neutral_5":"a Point [gray] drawn in good (location=(-2.4, 0.4))","good_neutral_6":"a Point [gray] drawn in good (location=(-2.9, 1.2))","good_neutral_7":"a Point [gray] drawn in good (location=(1.2, 1.1))","good_neutral_8":"a Point [gray] drawn in good (location=(1.5, 1.7))","good_neutral_9":"a Point [gray] drawn in good (location=(2.0, 1.4))","good_seeds":"a Point [yellow] labelled \"1\" drawn in good (location=(-3.4, 0.1))","good_seeds_2":"a Point [yellow] labelled \"2\" drawn in good (location=(1.7, 1.8))","good_seeds_3":"a Point [yellow] labelled \"3\" drawn in good (location=(1.8, -1.5))","good_top":"a Point [green] drawn in good (location=(1.2, 1.1))","good_top_2":"a Point [green] drawn in good (location=(1.5, 1.7))","good_top_3":"a Point [green] drawn in good (location=(2.0, 1.4))","good_top_4":"a Point [green] drawn in good (location=(2.3, 2.0))","good_top_5":"a Point [green] drawn in good (location=(2.6, 1.2))","heading":"a Heading that says \"Same Data, Different Starts\"","left_label":"a Tex [text] that says \"Centres spread across the data\"","local_note":"a Text [text] that says \"Both outcomes are stable under another assignment and mean-update pass.\"","objective":"a Math [text] that says \"$J = sum_i sum_l (x_(i l) - mu_(z_i l))^2$\"","remedy_1":"a Text [text] that says \"Use k-means++ or another dispersed seeding rule.\"","remedy_2":"a Text [text] that says \"Run many independent starts, not one.\"","remedy_3":"a Text [text] that says \"Keep the lowest objective, then inspect the clustering.\"","remedy_4":"a Text [text] that says \"Report instability when several credible answers compete.\"","remedy_heading":"a Heading that says \"Treat Initialization as Part of the Fit\"","right_label":"a Tex [text] that says \"Centres crowded on the left\""},"beats":[{"start":242.11149999999998,"say":"K-means can finish at different answers on the same data because its objective is not generally convex in all assignments and centres together. Here are two runs with k equal to three.","live":[],"does":[[242.11149999999998,"heading is shown on the screen, written out."],[244.78149999999997,"good_neutral is shown on the screen, written out."],[244.78149999999997,"good_neutral_2 is shown on the screen, written out."],[244.78149999999997,"good_neutral_3 is shown on the screen, written out."],[244.78149999999997,"good_neutral_4 is shown on the screen, written out."],[244.78149999999997,"good_neutral_5 is shown on the screen, written out."],[244.78149999999997,"good_neutral_6 is shown on the screen, written out."],[244.78149999999997,"good_neutral_7 is shown on the screen, written out."],[244.78149999999997,"good_neutral_8 is shown on the screen, written out."],[244.78149999999997,"good_neutral_9 is shown on the screen, written out."],[244.78149999999997,"good_neutral_10 is shown on the screen, written out."],[244.78149999999997,"good_neutral_11 is shown on the screen, written out."],[244.78149999999997,"good_neutral_12 is shown on the screen, written out."],[244.78149999999997,"good_neutral_13 is shown on the screen, written out."],[244.78149999999997,"good_neutral_14 is shown on the screen, written out."],[244.78149999999997,"good_neutral_15 is shown on the screen, written out."],[244.78149999999997,"good_neutral_16 is shown on the screen, written out."],[244.78149999999997,"bad_neutral is shown on the screen, written out."],[244.78149999999997,"bad_neutral_2 is shown on the screen, written out."],[244.78149999999997,"bad_neutral_3 is shown on the screen, written out."],[244.78149999999997,"bad_neutral_4 is shown on the screen, written out."],[244.78149999999997,"bad_neutral_5 is shown on the screen, written out."],[244.78149999999997,"bad_neutral_6 is shown on the screen, written out."],[244.78149999999997,"bad_neutral_7 is shown on the screen, written out."],[244.78149999999997,"bad_neutral_8 is shown on the screen, written out."],[244.78149999999997,"bad_neutral_9 is shown on the screen, written out."],[244.78149999999997,"bad_neutral_10 is shown on the screen, written out."],[244.78149999999997,"bad_neutral_11 is shown on the screen, written out."],[244.78149999999997,"bad_neutral_12 is shown on the screen, written out."],[244.78149999999997,"bad_neutral_13 is shown on the screen, written out."],[244.78149999999997,"bad_neutral_14 is shown on the screen, written out."],[244.78149999999997,"bad_neutral_15 is shown on the screen, written out."],[244.78149999999997,"bad_neutral_16 is shown on the screen, written out."],[251.5965,"good is shown on the screen, written out."],[251.5965,"bad is shown on the screen, written out."]]},{"start":254.79749999999999,"say":"On the left, the starting centres are spread across the visible groups. On the right, an unlucky draw crowds all three starting centres into the left-hand group.","live":["good","bad","heading","good_neutral","good_neutral_2","good_neutral_3","good_neutral_4","good_neutral_5","good_neutral_6","good_neutral_7","good_neutral_8","good_neutral_9","good_neutral_10","good_neutral_11","good_neutral_12","good_neutral_13","good_neutral_14","good_neutral_15","good_neutral_16","bad_neutral","bad_neutral_2","bad_neutral_3","bad_neutral_4","bad_neutral_5","bad_neutral_6","bad_neutral_7","bad_neutral_8","bad_neutral_9","bad_neutral_10","bad_neutral_11","bad_neutral_12","bad_neutral_13","bad_neutral_14","bad_neutral_15","bad_neutral_16"],"does":[[255.4945,"left_label is shown on the screen, written out."],[257.2125,"good_seeds is shown on the screen, written out."],[257.2125,"good_seeds_2 is shown on the screen, written out."],[257.2125,"good_seeds_3 is shown on the screen, written out."],[260.1615,"right_label is shown on the screen, written out."],[261.82149999999996,"bad_seeds is shown on the screen, written out."],[261.82149999999996,"bad_seeds_2 is shown on the screen, written out."],[261.82149999999996,"bad_seeds_3 is shown on the screen, written out."]]},{"start":265.823,"say":"Run the same assignment and update loop on both sides. The favourable start settles with one centre in each visible group.","live":["left_label","good","right_label","bad","heading","good_neutral","good_neutral_2","good_neutral_3","good_neutral_4","good_neutral_5","good_neutral_6","good_neutral_7","good_neutral_8","good_neutral_9","good_neutral_10","good_neutral_11","good_neutral_12","good_neutral_13","good_neutral_14","good_neutral_15","good_neutral_16","bad_neutral","bad_neutral_2","bad_neutral_3","bad_neutral_4","bad_neutral_5","bad_neutral_6","bad_neutral_7","bad_neutral_8","bad_neutral_9","bad_neutral_10","bad_neutral_11","bad_neutral_12","bad_neutral_13","bad_neutral_14","bad_neutral_15","bad_neutral_16","good_seeds","good_seeds_2","good_seeds_3","bad_seeds","bad_seeds_2","bad_seeds_3"],"does":[[267.99449999999996,"good_seeds is hidden from the screen."],[267.99449999999996,"good_seeds_2 is hidden from the screen."],[267.99449999999996,"good_seeds_3 is hidden from the screen."],[267.99449999999996,"good_neutral is hidden from the screen."],[267.99449999999996,"good_neutral_2 is hidden from the screen."],[267.99449999999996,"good_neutral_3 is hidden from the screen."],[267.99449999999996,"good_neutral_4 is hidden from the screen."],[267.99449999999996,"good_neutral_5 is hidden from the screen."],[267.99449999999996,"good_neutral_6 is hidden from the screen."],[267.99449999999996,"good_neutral_7 is hidden from the screen."],[267.99449999999996,"good_neutral_8 is hidden from the screen."],[267.99449999999996,"good_neutral_9 is hidden from the screen."],[267.99449999999996,"good_neutral_10 is hidden from the screen."],[267.99449999999996,"good_neutral_11 is hidden from the screen."],[267.99449999999996,"good_neutral_12 is hidden from the screen."],[267.99449999999996,"good_neutral_13 is hidden from the screen."],[267.99449999999996,"good_neutral_14 is hidden from the screen."],[267.99449999999996,"good_neutral_15 is hidden from the screen."],[267.99449999999996,"good_neutral_16 is hidden from the screen."],[271.80249999999995,"good_centres is shown on the screen, written out."],[271.80249999999995,"good_centres_2 is shown on the screen, written out."],[271.80249999999995,"good_centres_3 is shown on the screen, written out."],[272.5915,"good_left is shown on the screen, written out."],[272.5915,"good_left_2 is shown on the screen, written out."],[272.5915,"good_left_3 is shown on the screen, written out."],[272.5915,"good_left_4 is shown on the screen, written out."],[272.5915,"good_left_5 is shown on the screen, written out."],[272.5915,"good_left_6 is shown on the screen, written out."],[272.5915,"good_top is shown on the screen, written out."],[272.5915,"good_top_2 is shown on the screen, written out."],[272.5915,"good_top_3 is shown on the screen, written out."],[272.5915,"good_top_4 is shown on the screen, written out."],[272.5915,"good_top_5 is shown on the screen, written out."],[272.5915,"good_bottom is shown on the screen, written out."],[272.5915,"good_bottom_2 is shown on the screen, written out."],[272.5915,"good_bottom_3 is shown on the screen, written out."],[272.5915,"good_bottom_4 is shown on the screen, written out."],[272.5915,"good_bottom_5 is shown on the screen, written out."]]},{"start":274.30649999999997,"say":"The unlucky run spends two centres splitting the left group. Its remaining centre absorbs both right-hand groups, even though they are visibly separated.","live":["left_label","good","right_label","bad","heading","bad_neutral","bad_neutral_2","bad_neutral_3","bad_neutral_4","bad_neutral_5","bad_neutral_6","bad_neutral_7","bad_neutral_8","bad_neutral_9","bad_neutral_10","bad_neutral_11","bad_neutral_12","bad_neutral_13","bad_neutral_14","bad_neutral_15","bad_neutral_16","bad_seeds","bad_seeds_2","bad_seeds_3","good_left","good_left_2","good_left_3","good_left_4","good_left_5","good_left_6","good_top","good_top_2","good_top_3","good_top_4","good_top_5","good_bottom","good_bottom_2","good_bottom_3","good_bottom_4","good_bottom_5","good_centres","good_centres_2","good_centres_3"],"does":[[274.88649999999996,"bad_seeds is hidden from the screen."],[274.88649999999996,"bad_seeds_2 is hidden from the screen."],[274.88649999999996,"bad_seeds_3 is hidden from the screen."],[274.88649999999996,"bad_neutral is hidden from the screen."],[274.88649999999996,"bad_neutral_2 is hidden from the screen."],[274.88649999999996,"bad_neutral_3 is hidden from the screen."],[274.88649999999996,"bad_neutral_4 is hidden from the screen."],[274.88649999999996,"bad_neutral_5 is hidden from the screen."],[274.88649999999996,"bad_neutral_6 is hidden from the screen."],[274.88649999999996,"bad_neutral_7 is hidden from the screen."],[274.88649999999996,"bad_neutral_8 is hidden from the screen."],[274.88649999999996,"bad_neutral_9 is hidden from the screen."],[274.88649999999996,"bad_neutral_10 is hidden from the screen."],[274.88649999999996,"bad_neutral_11 is hidden from the screen."],[274.88649999999996,"bad_neutral_12 is hidden from the screen."],[274.88649999999996,"bad_neutral_13 is hidden from the screen."],[274.88649999999996,"bad_neutral_14 is hidden from the screen."],[274.88649999999996,"bad_neutral_15 is hidden from the screen."],[274.88649999999996,"bad_neutral_16 is hidden from the screen."],[276.7795,"bad_left_low is shown on the screen, written out."],[276.7795,"bad_left_low_2 is shown on the screen, written out."],[276.7795,"bad_left_low_3 is shown on the screen, written out."],[276.7795,"bad_left_high is shown on the screen, written out."],[276.7795,"bad_left_high_2 is shown on the screen, written out."],[276.7795,"bad_left_high_3 is shown on the screen, written out."],[279.3795,"bad_centres is shown on the screen, written out."],[279.3795,"bad_centres_2 is shown on the screen, written out."],[279.3795,"bad_centres_3 is shown on the screen, written out."],[279.9375,"bad_right_merged is shown on the screen, written out."],[279.9375,"bad_right_merged_2 is shown on the screen, written out."],[279.9375,"bad_right_merged_3 is shown on the screen, written out."],[279.9375,"bad_right_merged_4 is shown on the screen, written out."],[279.9375,"bad_right_merged_5 is shown on the screen, written out."],[279.9375,"bad_right_merged_6 is shown on the screen, written out."],[279.9375,"bad_right_merged_7 is shown on the screen, written out."],[279.9375,"bad_right_merged_8 is shown on the screen, written out."],[279.9375,"bad_right_merged_9 is shown on the screen, written out."],[279.9375,"bad_right_merged_10 is shown on the screen, written out."]]},{"start":284.763,"say":"Another pass changes neither answer. Both are local solutions, but the right-hand solution has a larger within-cluster sum of squares and a much less useful interpretation.","live":["left_label","good","right_label","bad","heading","good_left","good_left_2","good_left_3","good_left_4","good_left_5","good_left_6","good_top","good_top_2","good_top_3","good_top_4","good_top_5","good_bottom","good_bottom_2","good_bottom_3","good_bottom_4","good_bottom_5","good_centres","good_centres_2","good_centres_3","bad_left_low","bad_left_low_2","bad_left_low_3","bad_left_high","bad_left_high_2","bad_left_high_3","bad_right_merged","bad_right_merged_2","bad_right_merged_3","bad_right_merged_4","bad_right_merged_5","bad_right_merged_6","bad_right_merged_7","bad_right_merged_8","bad_right_merged_9","bad_right_merged_10","bad_centres","bad_centres_2","bad_centres_3"],"does":[[288.2695,"local_note is shown on the screen, written out."],[289.7555,"bad_right_merged is indicated — a transient flash."],[291.9385,"objective is shown on the screen, written out."]]},{"start":296.16049999999996,"say":"The practical remedy is not to trust one random start. K-means++ spreads seeds by favouring observations far from centres already chosen. It reduces bad starts, but does not abolish local minima.","live":["left_label","good","right_label","bad","objective","local_note","heading","good_left","good_left_2","good_left_3","good_left_4","good_left_5","good_left_6","good_top","good_top_2","good_top_3","good_top_4","good_top_5","good_bottom","good_bottom_2","good_bottom_3","good_bottom_4","good_bottom_5","good_centres","good_centres_2","good_centres_3","bad_left_low","bad_left_low_2","bad_left_low_3","bad_left_high","bad_left_high_2","bad_left_high_3","bad_right_merged","bad_right_merged_2","bad_right_merged_3","bad_right_merged_4","bad_right_merged_5","bad_right_merged_6","bad_right_merged_7","bad_right_merged_8","bad_right_merged_9","bad_right_merged_10","bad_centres","bad_centres_2","bad_centres_3"],"does":[[310.534,"bad is hidden from the screen — left the board."],[310.534,"bad_left_low is hidden from the screen — bad left the board."],[310.534,"bad_left_low_2 is hidden from the screen — bad left the board."],[310.534,"bad_left_low_3 is hidden from the screen — bad left the board."],[310.534,"bad_left_high is hidden from the screen — bad left the board."],[310.534,"bad_left_high_2 is hidden from the screen — bad left the board."],[310.534,"bad_left_high_3 is hidden from the screen — bad left the board."],[310.534,"bad_right_merged is hidden from the screen — bad left the board."],[310.534,"bad_right_merged_2 is hidden from the screen — bad left the board."],[310.534,"bad_right_merged_3 is hidden from the screen — bad left the board."],[310.534,"bad_right_merged_4 is hidden from the screen — bad left the board."],[310.534,"bad_right_merged_5 is hidden from the screen — bad left the board."],[310.534,"bad_right_merged_6 is hidden from the screen — bad left the board."],[310.534,"bad_right_merged_7 is hidden from the screen — bad left the board."],[310.534,"bad_right_merged_8 is hidden from the screen — bad left the board."],[310.534,"bad_right_merged_9 is hidden from the screen — bad left the board."],[310.534,"bad_right_merged_10 is hidden from the screen — bad left the board."],[310.534,"bad_centres is hidden from the screen — bad left the board."],[310.534,"bad_centres_2 is hidden from the screen — bad left the board."],[310.534,"bad_centres_3 is hidden from the screen — bad left the board."],[310.534,"good is hidden from the screen — left the board."],[310.534,"good_left is hidden from the screen — good left the board."],[310.534,"good_left_2 is hidden from the screen — good left the board."],[310.534,"good_left_3 is hidden from the screen — good left the board."],[310.534,"good_left_4 is hidden from the screen — good left the board."],[310.534,"good_left_5 is hidden from the screen — good left the board."],[310.534,"good_left_6 is hidden from the screen — good left the board."],[310.534,"good_top is hidden from the screen — good left the board."],[310.534,"good_top_2 is hidden from the screen — good left the board."],[310.534,"good_top_3 is hidden from the screen — good left the board."],[310.534,"good_top_4 is hidden from the screen — good left the board."],[310.534,"good_top_5 is hidden from the screen — good left the board."],[310.534,"good_bottom is hidden from the screen — good left the board."],[310.534,"good_bottom_2 is hidden from the screen — good left the board."],[310.534,"good_bottom_3 is hidden from the screen — good left the board."],[310.534,"good_bottom_4 is hidden from the screen — good left the board."],[310.534,"good_bottom_5 is hidden from the screen — good left the board."],[310.534,"good_centres is hidden from the screen — good left the board."],[310.534,"good_centres_2 is hidden from the screen — good left the board."],[310.534,"good_centres_3 is hidden from the screen — good left the board."],[310.534,"heading is hidden from the screen — left the board."],[310.534,"left_label is hidden from the screen — left the board."],[310.534,"local_note is hidden from the screen — left the board."],[310.534,"objective is hidden from the screen — left the board."],[310.534,"right_label is hidden from the screen — left the board."]]},{"start":311.734,"say":"For routine analysis, make initialization part of the specification. Use a dispersed seeding rule, then run many independent starts.","live":[],"does":[[311.734,"remedy_heading is shown on the screen, written out."],[317.2955,"remedy_1 is shown on the screen, written out."],[319.33849999999995,"remedy_2 is shown on the screen, written out."]]},{"start":321.67999999999995,"say":"Retain the run with the lowest objective, but still inspect its groups. A numerical improvement can be immaterial, and several nearly tied runs can imply genuine ambiguity in the data.","live":["remedy_1","remedy_2","remedy_heading"],"does":[[323.1775,"remedy_3 is shown on the screen, written out."],[332.37249999999995,"remedy_4 is shown on the screen, written out."]]},{"start":334.633,"say":"Record the number of starts, the random seed, and the spread of objective values. Reproducibility here is not clerical detail. It is evidence about how strongly the data supports the fitted partition.","live":["remedy_1","remedy_2","remedy_3","remedy_4","remedy_heading"],"does":[[335.9565,"remedy_2 is indicated — a transient flash."],[344.22249999999997,"remedy_4 is indicated — a transient flash."],[348.0978333333333,"remedy_1 is hidden from the screen — left the board."],[348.0978333333333,"remedy_2 is hidden from the screen — left the board."],[348.0978333333333,"remedy_3 is hidden from the screen — left the board."],[348.0978333333333,"remedy_4 is hidden from the screen — left the board."],[348.0978333333333,"remedy_heading is hidden from the screen — left the board."]]}]},{"title":"Geometry and Forced Clusters","start":349.1395,"end":487.92993749999994,"objects":{"caption_forced":"a Tex [text] that says \"One cloud, but k equals three\"","caption_long":"a Tex [text] that says \"Elongated groups\"","caption_size":"a Tex [text] that says \"Very different sizes\"","diagnostic_1":"a Text [text] that says \"Are groups roughly compact in the chosen feature scale?\"","diagnostic_2":"a Text [text] that says \"Are their spreads and populations comparable?\"","diagnostic_3":"a Text [text] that says \"Do assignments remain stable across starts and samples?\"","diagnostic_4":"a Text [text] that says \"Does the requested k have a domain meaning?\"","diagnostic_heading":"a Heading that says \"Questions to Ask Before Believing the Colours\"","forced_blue":"a Point [blue] drawn in forced_case (location=(1.0, -0.2))","forced_blue_2":"a Point [blue] drawn in forced_case (location=(1.4, 1.0))","forced_blue_3":"a Point [blue] drawn in forced_case (location=(1.8, -1.0))","forced_blue_4":"a Point [blue] drawn in forced_case (location=(2.1, 0.3))","forced_case":"a Figure (x_range=(-3.2, 3.2), y_range=(-3.2, 3.2), aspect=(1.0, 1.0))","forced_centres":"a Point [red] labelled \"mu_1\" drawn in forced_case (location=(-1.5, 0.0))","forced_centres_2":"a Point [green] labelled \"mu_2\" drawn in forced_case (location=(0.1, 0.2))","forced_centres_3":"a Point [blue] labelled \"mu_3\" drawn in forced_case (location=(1.5, 0.0))","forced_green":"a Point [green] drawn in forced_case (location=(-0.2, -0.9))","forced_green_2":"a Point [green] drawn in forced_case (location=(0.0, 0.5))","forced_green_3":"a Point [green] drawn in forced_case (location=(0.4, 1.8))","forced_green_4":"a Point [green] drawn in forced_case (location=(0.7, -1.7))","forced_outline":"a Circle [gray] drawn in forced_case (radius=2.5)","forced_red":"a Point [red] drawn in forced_case (location=(-2.1, -0.6))","forced_red_2":"a Point [red] drawn in forced_case (location=(-1.8, 0.8))","forced_red_3":"a Point [red] drawn in forced_case (location=(-1.3, -1.4))","forced_red_4":"a Point [red] drawn in forced_case (location=(-1.0, 0.1))","forced_red_5":"a Point [red] drawn in forced_case (location=(-0.7, 1.6))","heading":"a Heading that says \"Three Ways the Geometry Can Mislead\"","large_kept":"a Point [blue] drawn in size_case (location=(-2.8, -0.2))","large_kept_2":"a Point [blue] drawn in size_case (location=(-2.2, 1.0))","large_kept_3":"a Point [blue] drawn in size_case (location=(-2.0, -1.2))","large_kept_4":"a Point [blue] drawn in size_case (location=(-1.5, 0.3))","large_kept_5":"a Point [blue] drawn in size_case (location=(-1.0, 1.5))","large_kept_6":"a Point [blue] drawn in size_case (location=(-0.8, -1.4))","large_kept_7":"a Point [blue] drawn in size_case (location=(-0.3, 0.7))","large_kept_8":"a Point [blue] drawn in size_case (location=(0.1, -0.5))","large_outline":"a Circle [gray] drawn in size_case (center=(-1.0, 0.0), radius=2.0)","large_stolen":"a Point [red] drawn in size_case (location=(0.6, 0.4))","large_stolen_2":"a Point [red] drawn in size_case (location=(0.9, -0.9))","long_case":"a Figure (x_range=(-4.0, 4.0), y_range=(-2.8, 3.2), aspect=(8.0, 6.0))","long_centre_1":"a Point [red] labelled \"mu_1\" drawn in long_case (location=(-2.0, 0.0))","long_centre_2":"a Point [green] labelled \"mu_2\" drawn in long_case (location=(1.6, 1.2))","long_cut":"a Line [yellow] drawn in long_case (start=(-0.2, -2.2), end=(-0.2, 3.0), dashed=True)","long_left":"a Point [red] drawn in long_case (location=(-3.0, -1.9))","long_left_2":"a Point [red] drawn in long_case (location=(-2.1, -1.4))","long_left_3":"a Point [red] drawn in long_case (location=(-1.2, -0.9))","long_left_4":"a Point [red] drawn in long_case (location=(-0.2, -0.3))","long_left_5":"a Point [red] drawn in long_case (location=(-3.0, -0.5))","long_left_6":"a Point [red] drawn in long_case (location=(-2.1, 0.0))","long_left_7":"a Point [red] drawn in long_case (location=(-1.2, 0.5))","long_left_8":"a Point [red] drawn in long_case (location=(-0.2, 1.1))","long_outline_1":"a Polygon [gray] drawn in long_case (vertices=((2.890483548894609, 1.1673607282593166), (2.8250451244373482, …, filled=False)","long_outline_2":"a Polygon [gray] drawn in long_case (vertices=((2.890483548894609, 2.5673607282593167), (2.8250451244373482, …, filled=False)","long_right":"a Point [green] drawn in long_case (location=(0.8, 0.2))","long_right_2":"a Point [green] drawn in long_case (location=(1.8, 0.8))","long_right_3":"a Point [green] drawn in long_case (location=(2.8, 1.3))","long_right_4":"a Point [green] drawn in long_case (location=(0.8, 1.6))","long_right_5":"a Point [green] drawn in long_case (location=(1.8, 2.2))","long_right_6":"a Point [green] drawn in long_case (location=(2.8, 2.7))","size_boundary":"a Line [yellow] drawn in size_case (start=(0.35, -2.4), end=(0.35, 2.4), dashed=True)","size_case":"a Figure (x_range=(-4.0, 4.0), y_range=(-3.0, 3.0), aspect=(8.0, 6.0))","size_centre_1":"a Point [blue] labelled \"mu_1\" drawn in size_case (location=(-1.2, 0.1))","size_centre_2":"a Point [red] labelled \"mu_2\" drawn in size_case (location=(1.7, -0.2))","small_assigned":"a Point [red] drawn in size_case (location=(1.7, -0.2))","small_assigned_2":"a Point [red] drawn in size_case (location=(1.9, 0.3))","small_assigned_3":"a Point [red] drawn in size_case (location=(2.1, -0.4))","small_assigned_4":"a Point [red] drawn in size_case (location=(2.3, 0.2))","small_assigned_5":"a Point [red] drawn in size_case (location=(2.5, -0.1))","small_outline":"a Circle [gray] drawn in size_case (center=(2.1, 0.0), radius=0.7)"},"beats":[{"start":349.1395,"say":"The nearest-centre rule creates Voronoi cells: regions separated by straight boundaries. That simple geometry explains several important failures.","live":[],"does":[[349.1395,"heading is shown on the screen, written out."]]},{"start":359.968,"say":"First, consider two long natural groups. Their gray outlines run diagonally across the feature space.","live":["heading"],"does":[[360.3165,"long_case is shown on the screen, written out."],[361.8025,"caption_long is shown on the screen, written out."],[364.2405,"long_outline_1 is shown on the screen, written out."],[364.2405,"long_outline_2 is shown on the screen, written out."]]},{"start":368.195,"say":"With these fitted centres, nearest-centre assignment makes a nearly vertical cut. Red and green divide both long groups across their length rather than following the two elongated densities.","live":["long_case","caption_long","heading","long_outline_1","long_outline_2"],"does":[[369.4605,"long_centre_1 is shown on the screen, written out."],[369.4605,"long_centre_2 is shown on the screen, written out."],[372.2705,"long_cut is shown on the screen, written out."],[373.7915,"long_left is shown on the screen, written out."],[373.7915,"long_left_2 is shown on the screen, written out."],[373.7915,"long_left_3 is shown on the screen, written out."],[373.7915,"long_left_4 is shown on the screen, written out."],[373.7915,"long_left_5 is shown on the screen, written out."],[373.7915,"long_left_6 is shown on the screen, written out."],[373.7915,"long_left_7 is shown on the screen, written out."],[373.7915,"long_left_8 is shown on the screen, written out."],[374.1745,"long_right is shown on the screen, written out."],[374.1745,"long_right_2 is shown on the screen, written out."],[374.1745,"long_right_3 is shown on the screen, written out."],[374.1745,"long_right_4 is shown on the screen, written out."],[374.1745,"long_right_5 is shown on the screen, written out."],[374.1745,"long_right_6 is shown on the screen, written out."]]},{"start":380.614,"say":"K-means has a centre for location, but no parameter for orientation or different spread in different directions. Squared Euclidean distance therefore prefers compact, roughly spherical clusters.","live":["long_case","caption_long","heading","long_outline_1","long_outline_2","long_left","long_left_2","long_left_3","long_left_4","long_left_5","long_left_6","long_left_7","long_left_8","long_right","long_right_2","long_right_3","long_right_4","long_right_5","long_right_6","long_centre_1","long_centre_2","long_cut"],"does":[[389.2405,"long_cut is indicated — a transient flash."]]},{"start":394.2065,"say":"Second, place one small dense cluster beside one large diffuse cluster. The gray circles show the intended populations.","live":null,"does":[[394.5085,"size_case is shown on the screen, written out."],[395.8895,"caption_size is shown on the screen, written out."],[400.4405,"large_outline is shown on the screen, written out."],[400.4405,"small_outline is shown on the screen, written out."]]},{"start":403.479,"say":"The large group contributes many more squared distances to the objective. A centre can reduce that cost by moving toward it, while the boundary gives part of the large group to the small cluster.","live":["long_case","caption_long","size_case","caption_size","heading","long_outline_1","long_outline_2","long_left","long_left_2","long_left_3","long_left_4","long_left_5","long_left_6","long_left_7","long_left_8","long_right","long_right_2","long_right_3","long_right_4","long_right_5","long_right_6","long_centre_1","long_centre_2","long_cut","large_outline","small_outline"],"does":[[403.9785,"large_kept is shown on the screen, written out."],[403.9785,"large_kept_2 is shown on the screen, written out."],[403.9785,"large_kept_3 is shown on the screen, written out."],[403.9785,"large_kept_4 is shown on the screen, written out."],[403.9785,"large_kept_5 is shown on the screen, written out."],[403.9785,"large_kept_6 is shown on the screen, written out."],[403.9785,"large_kept_7 is shown on the screen, written out."],[403.9785,"large_kept_8 is shown on the screen, written out."],[408.2505,"size_centre_1 is shown on the screen, written out."],[408.2505,"size_centre_2 is shown on the screen, written out."],[411.45550000000003,"size_boundary is shown on the screen, written out."],[411.9315,"large_stolen is shown on the screen, written out."],[411.9315,"large_stolen_2 is shown on the screen, written out."],[413.4635,"small_assigned is shown on the screen, written out."],[413.4635,"small_assigned_2 is shown on the screen, written out."],[413.4635,"small_assigned_3 is shown on the screen, written out."],[413.4635,"small_assigned_4 is shown on the screen, written out."],[413.4635,"small_assigned_5 is shown on the screen, written out."]]},{"start":415.3055,"say":"The result is mathematically consistent with the objective, but unfair to the smaller group as a density. K-means does not model cluster population or variance separately.","live":["long_case","caption_long","size_case","caption_size","heading","long_outline_1","long_outline_2","long_left","long_left_2","long_left_3","long_left_4","long_left_5","long_left_6","long_left_7","long_left_8","long_right","long_right_2","long_right_3","long_right_4","long_right_5","long_right_6","long_centre_1","long_centre_2","long_cut","large_outline","small_outline","large_kept","large_kept_2","large_kept_3","large_kept_4","large_kept_5","large_kept_6","large_kept_7","large_kept_8","large_stolen","large_stolen_2","small_assigned","small_assigned_2","small_assigned_3","small_assigned_4","small_assigned_5","size_centre_1","size_centre_2","size_boundary"],"does":[[419.1015,"large_stolen is indicated — a transient flash."],[419.1015,"large_stolen_2 is indicated — a transient flash."]]},{"start":427.15549999999996,"say":"Third, this is one connected cloud without three clear density peaks. We nevertheless request k equals three.","live":null,"does":[[427.5035,"forced_case is shown on the screen, written out."],[429.0135,"caption_forced is shown on the screen, written out."],[429.4545,"forced_outline is shown on the screen, written out."]]},{"start":435.395,"say":"K-means does not answer that the data contains one group. It returns exactly three nonempty partitions because three centres were requested.","live":["long_case","caption_long","size_case","caption_size","forced_case","caption_forced","heading","long_outline_1","long_outline_2","long_left","long_left_2","long_left_3","long_left_4","long_left_5","long_left_6","long_left_7","long_left_8","long_right","long_right_2","long_right_3","long_right_4","long_right_5","long_right_6","long_centre_1","long_centre_2","long_cut","large_outline","small_outline","large_kept","large_kept_2","large_kept_3","large_kept_4","large_kept_5","large_kept_6","large_kept_7","large_kept_8","large_stolen","large_stolen_2","small_assigned","small_assigned_2","small_assigned_3","small_assigned_4","small_assigned_5","size_centre_1","size_centre_2","size_boundary","forced_outline"],"does":[[441.25849999999997,"forced_red is shown on the screen, written out."],[441.25849999999997,"forced_red_2 is shown on the screen, written out."],[441.25849999999997,"forced_red_3 is shown on the screen, written out."],[441.25849999999997,"forced_red_4 is shown on the screen, written out."],[441.25849999999997,"forced_red_5 is shown on the screen, written out."],[441.25849999999997,"forced_green is shown on the screen, written out."],[441.25849999999997,"forced_green_2 is shown on the screen, written out."],[441.25849999999997,"forced_green_3 is shown on the screen, written out."],[441.25849999999997,"forced_green_4 is shown on the screen, written out."],[441.25849999999997,"forced_blue is shown on the screen, written out."],[441.25849999999997,"forced_blue_2 is shown on the screen, written out."],[441.25849999999997,"forced_blue_3 is shown on the screen, written out."],[441.25849999999997,"forced_blue_4 is shown on the screen, written out."],[443.6035,"forced_centres is shown on the screen, written out."],[443.6035,"forced_centres_2 is shown on the screen, written out."],[443.6035,"forced_centres_3 is shown on the screen, written out."]]},{"start":445.7365,"say":"This is not a software defect. The value k is an input to the problem, not a conclusion reached by the algorithm. A tidy colour map does not prove that the corresponding groups exist.","live":["long_case","caption_long","size_case","caption_size","forced_case","caption_forced","heading","long_outline_1","long_outline_2","long_left","long_left_2","long_left_3","long_left_4","long_left_5","long_left_6","long_left_7","long_left_8","long_right","long_right_2","long_right_3","long_right_4","long_right_5","long_right_6","long_centre_1","long_centre_2","long_cut","large_outline","small_outline","large_kept","large_kept_2","large_kept_3","large_kept_4","large_kept_5","large_kept_6","large_kept_7","large_kept_8","large_stolen","large_stolen_2","small_assigned","small_assigned_2","small_assigned_3","small_assigned_4","small_assigned_5","size_centre_1","size_centre_2","size_boundary","forced_outline","forced_red","forced_red_2","forced_red_3","forced_red_4","forced_red_5","forced_green","forced_green_2","forced_green_3","forced_green_4","forced_blue","forced_blue_2","forced_blue_3","forced_blue_4","forced_centres","forced_centres_2","forced_centres_3"],"does":[[457.95,"caption_forced is hidden from the screen — left the board."],[457.95,"caption_long is hidden from the screen — left the board."],[457.95,"caption_size is hidden from the screen — left the board."],[457.95,"forced_case is hidden from the screen — left the board."],[457.95,"forced_outline is hidden from the screen — forced_case left the board."],[457.95,"forced_red is hidden from the screen — forced_case left the board."],[457.95,"forced_red_2 is hidden from the screen — forced_case left the board."],[457.95,"forced_red_3 is hidden from the screen — forced_case left the board."],[457.95,"forced_red_4 is hidden from the screen — forced_case left the board."],[457.95,"forced_red_5 is hidden from the screen — forced_case left the board."],[457.95,"forced_green is hidden from the screen — forced_case left the board."],[457.95,"forced_green_2 is hidden from the screen — forced_case left the board."],[457.95,"forced_green_3 is hidden from the screen — forced_case left the board."],[457.95,"forced_green_4 is hidden from the screen — forced_case left the board."],[457.95,"forced_blue is hidden from the screen — forced_case left the board."],[457.95,"forced_blue_2 is hidden from the screen — forced_case left the board."],[457.95,"forced_blue_3 is hidden from the screen — forced_case left the board."],[457.95,"forced_blue_4 is hidden from the screen — forced_case left the board."],[457.95,"forced_centres is hidden from the screen — forced_case left the board."],[457.95,"forced_centres_2 is hidden from the screen — forced_case left the board."],[457.95,"forced_centres_3 is hidden from the screen — forced_case left the board."],[457.95,"heading is hidden from the screen — left the board."],[457.95,"long_case is hidden from the screen — left the board."],[457.95,"long_outline_1 is hidden from the screen — long_case left the board."],[457.95,"long_outline_2 is hidden from the screen — long_case left the board."],[457.95,"long_left is hidden from the screen — long_case left the board."],[457.95,"long_left_2 is hidden from the screen — long_case left the board."],[457.95,"long_left_3 is hidden from the screen — long_case left the board."],[457.95,"long_left_4 is hidden from the screen — long_case left the board."],[457.95,"long_left_5 is hidden from the screen — long_case left the board."],[457.95,"long_left_6 is hidden from the screen — long_case left the board."],[457.95,"long_left_7 is hidden from the screen — long_case left the board."],[457.95,"long_left_8 is hidden from the screen — long_case left the board."],[457.95,"long_right is hidden from the screen — long_case left the board."],[457.95,"long_right_2 is hidden from the screen — long_case left the board."],[457.95,"long_right_3 is hidden from the screen — long_case left the board."],[457.95,"long_right_4 is hidden from the screen — long_case left the board."],[457.95,"long_right_5 is hidden from the screen — long_case left the board."],[457.95,"long_right_6 is hidden from the screen — long_case left the board."],[457.95,"long_centre_1 is hidden from the screen — long_case left the board."],[457.95,"long_centre_2 is hidden from the screen — long_case left the board."],[457.95,"long_cut is hidden from the screen — long_case left the board."],[457.95,"size_case is hidden from the screen — left the board."],[457.95,"large_outline is hidden from the screen — size_case left the board."],[457.95,"small_outline is hidden from the screen — size_case left the board."],[457.95,"large_kept is hidden from the screen — size_case left the board."],[457.95,"large_kept_2 is hidden from the screen — size_case left the board."],[457.95,"large_kept_3 is hidden from the screen — size_case left the board."],[457.95,"large_kept_4 is hidden from the screen — size_case left the board."],[457.95,"large_kept_5 is hidden from the screen — size_case left the board."],[457.95,"large_kept_6 is hidden from the screen — size_case left the board."],[457.95,"large_kept_7 is hidden from the screen — size_case left the board."],[457.95,"large_kept_8 is hidden from the screen — size_case left the board."],[457.95,"large_stolen is hidden from the screen — size_case left the board."],[457.95,"large_stolen_2 is hidden from the screen — size_case left the board."],[457.95,"small_assigned is hidden from the screen — size_case left the board."],[457.95,"small_assigned_2 is hidden from the screen — size_case left the board."],[457.95,"small_assigned_3 is hidden from the screen — size_case left the board."],[457.95,"small_assigned_4 is hidden from the screen — size_case left the board."],[457.95,"small_assigned_5 is hidden from the screen — size_case left the board."],[457.95,"size_centre_1 is hidden from the screen — size_case left the board."],[457.95,"size_centre_2 is hidden from the screen — size_case left the board."],[457.95,"size_boundary is hidden from the screen — size_case left the board."]]},{"start":459.15,"say":"Before believing a k-means result, ask whether compact clusters are plausible in the scaled feature space, and whether their spreads and populations are comparable.","live":[],"does":[[459.15,"diagnostic_heading is shown on the screen, written out."],[462.87649999999996,"diagnostic_1 is shown on the screen, written out."],[467.2655,"diagnostic_2 is shown on the screen, written out."]]},{"start":470.373,"say":"Then test stability across initializations and resampled data. Finally, ask what k means in the business or scientific setting. If it has no interpretation, treat it as a modelling choice to validate, not a fact discovered by the colours.","live":["diagnostic_1","diagnostic_2","diagnostic_heading"],"does":[[471.37149999999997,"diagnostic_3 is shown on the screen, written out."],[477.3975,"diagnostic_4 is shown on the screen, written out."],[486.8882708333333,"diagnostic_1 is hidden from the screen — left the board."],[486.8882708333333,"diagnostic_2 is hidden from the screen — left the board."],[486.8882708333333,"diagnostic_3 is hidden from the screen — left the board."],[486.8882708333333,"diagnostic_4 is hidden from the screen — left the board."],[486.8882708333333,"diagnostic_heading is hidden from the screen — left the board."]]}]},{"title":"From Soft Membership to Mixtures","start":487.92993749999994,"end":682.4885416666666,"objects":{"compare_heading":"a Heading that says \"Return to the Elongated Case\"","density":"a Math [text] that says \"$p(x_i) = sum_j pi_j upright(\"Normal\")(x_i | mu_j, Sigma_j)$\"","e_step":"a Text [text] that says \"E step: compute every responsibility from the current parameters.\"","em_heading":"a Heading that says \"Fit by Alternating Two Weighted Steps\"","em_stop":"a Text [text] that says \"Repeat until the log likelihood stops improving materially.\"","g_case":"a Figure (x_range=(-4.0, 4.0), y_range=(-3.0, 3.0), aspect=(8.0, 6.0))","g_ellipse_a":"a Polygon [red] drawn in g_case (vertices=((2.890483548894609, 1.1173607282593165), (2.834109584128165, 1…, filled=False)","g_ellipse_b":"a Polygon [green] drawn in g_case (vertices=((2.990483548894609, 2.6173607282593165), (2.9341095841281652, …, filled=False)","g_label":"a Tex [text] that says \"Gaussian mixture: fitted ellipses\"","g_points_a":"a Point [red] drawn in g_case (location=(-3.0, -1.8))","g_points_a_2":"a Point [red] drawn in g_case (location=(-2.1, -1.3))","g_points_a_3":"a Point [red] drawn in g_case (location=(-1.2, -0.9))","g_points_a_4":"a Point [red] drawn in g_case (location=(-0.3, -0.4))","g_points_a_5":"a Point [red] drawn in g_case (location=(0.6, 0.1))","g_points_a_6":"a Point [red] drawn in g_case (location=(1.5, 0.6))","g_points_a_7":"a Point [red] drawn in g_case (location=(2.5, 1.2))","g_points_b":"a Point [green] drawn in g_case (location=(-2.8, -0.3))","g_points_b_2":"a Point [green] drawn in g_case (location=(-1.9, 0.2))","g_points_b_3":"a Point [green] drawn in g_case (location=(-1.0, 0.7))","g_points_b_4":"a Point [green] drawn in g_case (location=(-0.1, 1.2))","g_points_b_5":"a Point [green] drawn in g_case (location=(0.8, 1.7))","g_points_b_6":"a Point [green] drawn in g_case (location=(1.7, 2.2))","g_points_b_7":"a Point [green] drawn in g_case (location=(2.6, 2.7))","gmm_ellipse_a":"a Polygon [red] drawn in gmm_plot (vertices=((2.890483548894609, 1.1173607282593165), (2.834109584128165, 1…, filled=False)","gmm_ellipse_b":"a Polygon [green] drawn in gmm_plot (vertices=((2.990483548894609, 2.6173607282593165), (2.9341095841281652, …, filled=False)","gmm_mean_a":"a Point [red] labelled \"mu_1\" drawn in gmm_plot (location=(-0.2, -0.3))","gmm_mean_b":"a Point [green] labelled \"mu_2\" drawn in gmm_plot (location=(-0.1, 1.2))","gmm_plot":"an Axes (x_range=(-4.0, 4.0), y_range=(-3.0, 3.0), include_ticks=False)","gmm_points_a":"a Point [red] drawn in gmm_plot (location=(-3.0, -1.8))","gmm_points_a_2":"a Point [red] drawn in gmm_plot (location=(-2.1, -1.3))","gmm_points_a_3":"a Point [red] drawn in gmm_plot (location=(-1.2, -0.9))","gmm_points_a_4":"a Point [red] drawn in gmm_plot (location=(-0.3, -0.4))","gmm_points_a_5":"a Point [red] drawn in gmm_plot (location=(0.6, 0.1))","gmm_points_a_6":"a Point [red] drawn in gmm_plot (location=(1.5, 0.6))","gmm_points_a_7":"a Point [red] drawn in gmm_plot (location=(2.5, 1.2))","gmm_points_b":"a Point [green] drawn in gmm_plot (location=(-2.8, -0.3))","gmm_points_b_2":"a Point [green] drawn in gmm_plot (location=(-1.9, 0.2))","gmm_points_b_3":"a Point [green] drawn in gmm_plot (location=(-1.0, 0.7))","gmm_points_b_4":"a Point [green] drawn in gmm_plot (location=(-0.1, 1.2))","gmm_points_b_5":"a Point [green] drawn in gmm_plot (location=(0.8, 1.7))","gmm_points_b_6":"a Point [green] drawn in gmm_plot (location=(1.7, 2.2))","gmm_points_b_7":"a Point [green] drawn in gmm_plot (location=(2.6, 2.7))","k_boundary":"a Line [yellow] drawn in k_case (start=(0.0, -2.5), end=(0.0, 3.0), dashed=True)","k_case":"a Figure (x_range=(-4.0, 4.0), y_range=(-3.0, 3.0), aspect=(8.0, 6.0))","k_label":"a Tex [text] that says \"K-means: nearest-centre cells\"","k_left":"a Point [red] drawn in k_case (location=(-3.0, -1.8))","k_left_2":"a Point [red] drawn in k_case (location=(-2.1, -1.3))","k_left_3":"a Point [red] drawn in k_case (location=(-1.2, -0.9))","k_left_4":"a Point [red] drawn in k_case (location=(-0.3, -0.4))","k_left_5":"a Point [red] drawn in k_case (location=(-2.8, -0.3))","k_left_6":"a Point [red] drawn in k_case (location=(-1.9, 0.2))","k_left_7":"a Point [red] drawn in k_case (location=(-1.0, 0.7))","k_left_8":"a Point [red] drawn in k_case (location=(-0.1, 1.2))","k_outline_a":"a Polygon [gray] drawn in k_case (vertices=((2.890483548894609, 1.1173607282593165), (2.834109584128165, 1…, filled=False)","k_outline_b":"a Polygon [gray] drawn in k_case (vertices=((2.990483548894609, 2.6173607282593165), (2.9341095841281652, …, filled=False)","k_right":"a Point [green] drawn in k_case (location=(0.6, 0.1))","k_right_2":"a Point [green] drawn in k_case (location=(1.5, 0.6))","k_right_3":"a Point [green] drawn in k_case (location=(2.5, 1.2))","k_right_4":"a Point [green] drawn in k_case (location=(0.8, 1.7))","k_right_5":"a Point [green] drawn in k_case (location=(1.7, 2.2))","k_right_6":"a Point [green] drawn in k_case (location=(2.6, 2.7))","m_step":"a Text [text] that says \"M step: update weights, means, and covariances using those responsibilities.\"","parameter_heading":"a Heading that says \"What the Extra Parameters Do\"","parameters":"a Block [text] that says \"$pi_j$: the component's expected population share. $mu_j$: the component's centre. $Sigma_j$: its spread, elongation, and orientation.\"","probabilities":"a Table [text] that says \"Component Responsibility 1 0.35 2 0.65\" (rows=(('Component', 'Responsibility'), ('1', '0.35'), ('2', '0.65')), header=True)","responsibility":"a Math [text] that says \"$r_(i j) = frac(pi_j upright(\"Normal\")(x_i | mu_j, Sigma_j), sum_l pi_l upright(\"Normal\")(x_i | mu_l, Sigma_l))$\"","soft_centre_1":"a Point [red] labelled \"mu_1\" drawn in soft_plot (location=(-1.8, 0.1))","soft_centre_2":"a Point [green] labelled \"mu_2\" drawn in soft_plot (location=(1.8, 0.1))","soft_cloud_1":"a Point [red] drawn in soft_plot (location=(-2.8, -0.5))","soft_cloud_1_2":"a Point [red] drawn in soft_plot (location=(-2.2, 0.5))","soft_cloud_1_3":"a Point [red] drawn in soft_plot (location=(-1.7, -0.2))","soft_cloud_1_4":"a Point [red] drawn in soft_plot (location=(-1.2, 0.8))","soft_cloud_2":"a Point [green] drawn in soft_plot (location=(1.0, -0.5))","soft_cloud_2_2":"a Point [green] drawn in soft_plot (location=(1.5, 0.6))","soft_cloud_2_3":"a Point [green] drawn in soft_plot (location=(2.1, -0.2))","soft_cloud_2_4":"a Point [green] drawn in soft_plot (location=(2.7, 0.7))","soft_definition":"a Panel that says \"The responsibility $r_(i j)$ is the conditional probability that component $j$ generated observation $i$, under the current fitted model.\"","soft_heading":"a Heading that says \"Replace a Label with Responsibilities\"","soft_link_1":"a Line [gray] drawn in soft_plot (start=(0.2, 0.2), end=(-1.8, 0.1), dashed=True)","soft_link_2":"a Line [gray] drawn in soft_plot (start=(0.2, 0.2), end=(1.8, 0.1), dashed=True)","soft_observation":"a Point [yellow] labelled \"x_i\" drawn in soft_plot (location=(0.2, 0.2))","soft_plot":"an Axes (x_range=(-4.0, 4.0), y_range=(-3.0, 3.0), include_ticks=False)"},"beats":[{"start":487.92993749999994,"say":"To relax hard assignment, keep one membership number for every component. For observation i and component j, call that number the responsibility r i j.","live":[],"does":[[487.92993749999994,"soft_heading is shown on the screen, written out."],[487.92993749999994,"soft_plot is shown on the screen, written out."],[491.77293749999995,"soft_cloud_1 is shown on the screen, written out."],[491.77293749999995,"soft_cloud_1_2 is shown on the screen, written out."],[491.77293749999995,"soft_cloud_1_3 is shown on the screen, written out."],[491.77293749999995,"soft_cloud_1_4 is shown on the screen, written out."],[491.77293749999995,"soft_cloud_2 is shown on the screen, written out."],[491.77293749999995,"soft_cloud_2_2 is shown on the screen, written out."],[491.77293749999995,"soft_cloud_2_3 is shown on the screen, written out."],[491.77293749999995,"soft_cloud_2_4 is shown on the screen, written out."],[491.77293749999995,"soft_centre_1 is shown on the screen, written out."],[491.77293749999995,"soft_centre_2 is shown on the screen, written out."],[497.15993749999996,"soft_plot moves to a new place on the board."],[497.15993749999996,"soft_definition is shown on the screen, written out."]]},{"start":500.37193749999994,"say":"Consider this observation near the overlap. A hard rule must choose one side. The mixture instead evaluates how plausible the observation is under both fitted components.","live":["soft_definition","soft_plot","soft_heading","soft_cloud_1","soft_cloud_1_2","soft_cloud_1_3","soft_cloud_1_4","soft_cloud_2","soft_cloud_2_2","soft_cloud_2_3","soft_cloud_2_4","soft_centre_1","soft_centre_2"],"does":[[501.41693749999996,"soft_observation is shown on the screen, written out."],[508.70793749999996,"responsibility is shown on the screen, written out."],[510.49593749999997,"soft_link_1 is shown on the screen, written out."],[510.49593749999997,"soft_link_2 is shown on the screen, written out."],[512.2029375,"responsibility moves to a new place on the board."],[512.2029375,"soft_definition is hidden from the screen — left the board."]]},{"start":512.8029375,"say":"Suppose the resulting responsibilities are zero point three five and zero point six five. The second component is more plausible, but the first remains credible. The two values add to one.","live":["responsibility","soft_plot","soft_heading","soft_cloud_1","soft_cloud_1_2","soft_cloud_1_3","soft_cloud_1_4","soft_cloud_2","soft_cloud_2_2","soft_cloud_2_3","soft_cloud_2_4","soft_centre_1","soft_centre_2","soft_observation","soft_link_1","soft_link_2"],"does":[[514.2309375,"probabilities is shown on the screen, written out."],[515.9719375,"probabilities is shown on the screen, written out."],[517.5629375,"probabilities is shown on the screen, written out."],[524.1109375,"probabilities (the \"column=2\" part) is emphasized."],[525.8634374999999,"probabilities (the \"column=2\" part) is no longer emphasized."]]},{"start":526.4634374999999,"say":"These are model-based probabilities, not calibrated truth supplied by the data. They depend on the fitted component shapes, their population weights, and the Gaussian mixture assumptions.","live":null,"does":[[538.3524375,"probabilities is hidden from the screen — left the board."],[538.3524375,"responsibility is hidden from the screen — left the board."],[538.3524375,"soft_heading is hidden from the screen — left the board."],[538.3524375,"soft_plot is hidden from the screen — left the board."],[538.3524375,"soft_cloud_1 is hidden from the screen — soft_plot left the board."],[538.3524375,"soft_cloud_1_2 is hidden from the screen — soft_plot left the board."],[538.3524375,"soft_cloud_1_3 is hidden from the screen — soft_plot left the board."],[538.3524375,"soft_cloud_1_4 is hidden from the screen — soft_plot left the board."],[538.3524375,"soft_cloud_2 is hidden from the screen — soft_plot left the board."],[538.3524375,"soft_cloud_2_2 is hidden from the screen — soft_plot left the board."],[538.3524375,"soft_cloud_2_3 is hidden from the screen — soft_plot left the board."],[538.3524375,"soft_cloud_2_4 is hidden from the screen — soft_plot left the board."],[538.3524375,"soft_centre_1 is hidden from the screen — soft_plot left the board."],[538.3524375,"soft_centre_2 is hidden from the screen — soft_plot left the board."],[538.3524375,"soft_observation is hidden from the screen — soft_plot left the board."],[538.3524375,"soft_link_1 is hidden from the screen — soft_plot left the board."],[538.3524375,"soft_link_2 is hidden from the screen — soft_plot left the board."]]},{"start":539.5524374999999,"say":"A Gaussian mixture says that the observed density is a weighted sum of Gaussian component densities. The extra parameters each answer a distinct question.","live":[],"does":[[539.5524374999999,"parameter_heading is shown on the screen, written out."],[543.1049374999999,"density is shown on the screen, written out."],[544.9859375,"gmm_plot is shown on the screen, written out."]]},{"start":550.6829375,"say":"The mixture weight pi records the expected share of observations generated by a component. Unlike k-means, the model can explicitly represent one large component and one small component.","live":["density","gmm_plot","parameter_heading"],"does":[[551.5889374999999,"parameters is shown on the screen, written out."],[551.8669375,"parameters (the \"$pi_j$\" part) is emphasized."],[563.3729374999999,"parameters (the \"$pi_j$\" part) is no longer emphasized."]]},{"start":563.9729375,"say":"The mean mu gives location, much like a k-means centre. Here the two means sit near the middle of their respective densities.","live":["parameters","density","gmm_plot","parameter_heading"],"does":[[564.9599374999999,"parameters (the \"$mu_j$\" part) is emphasized."],[569.4989374999999,"gmm_mean_a is shown on the screen, written out."],[569.4989374999999,"gmm_mean_b is shown on the screen, written out."],[573.0284374999999,"parameters (the \"$mu_j$\" part) is no longer emphasized."]]},{"start":573.6284374999999,"say":"The covariance matrix Sigma is the crucial addition. Its overall scale controls spread. Its unequal directional variances create elongation, and its off-diagonal relationship rotates that elongation.","live":["parameters","density","gmm_plot","parameter_heading","gmm_mean_a","gmm_mean_b"],"does":[[574.2089374999999,"gmm_ellipse_a is shown on the screen, written out."],[574.2089374999999,"gmm_ellipse_b is shown on the screen, written out."],[575.2999374999999,"parameters (the \"$Sigma_j$\" part) is emphasized."],[583.3459375,"gmm_points_a is shown on the screen, written out."],[583.3459375,"gmm_points_a_2 is shown on the screen, written out."],[583.3459375,"gmm_points_a_3 is shown on the screen, written out."],[583.3459375,"gmm_points_a_4 is shown on the screen, written out."],[583.3459375,"gmm_points_a_5 is shown on the screen, written out."],[583.3459375,"gmm_points_a_6 is shown on the screen, written out."],[583.3459375,"gmm_points_a_7 is shown on the screen, written out."],[583.3459375,"gmm_points_b is shown on the screen, written out."],[583.3459375,"gmm_points_b_2 is shown on the screen, written out."],[583.3459375,"gmm_points_b_3 is shown on the screen, written out."],[583.3459375,"gmm_points_b_4 is shown on the screen, written out."],[583.3459375,"gmm_points_b_5 is shown on the screen, written out."],[583.3459375,"gmm_points_b_6 is shown on the screen, written out."],[583.3459375,"gmm_points_b_7 is shown on the screen, written out."],[588.4304374999999,"parameters (the \"$Sigma_j$\" part) is no longer emphasized."]]},{"start":589.0304375,"say":"These ellipses are equal-density contours. They can follow the long axes of the two groups, which is exactly the geometric freedom missing from nearest-centre k-means.","live":["parameters","density","gmm_plot","parameter_heading","gmm_mean_a","gmm_mean_b","gmm_ellipse_a","gmm_ellipse_b","gmm_points_a","gmm_points_a_2","gmm_points_a_3","gmm_points_a_4","gmm_points_a_5","gmm_points_a_6","gmm_points_a_7","gmm_points_b","gmm_points_b_2","gmm_points_b_3","gmm_points_b_4","gmm_points_b_5","gmm_points_b_6","gmm_points_b_7"],"does":[[589.6929375,"gmm_ellipse_a is indicated — a transient flash."],[589.6929375,"gmm_ellipse_b is indicated — a transient flash."],[599.8974374999999,"density is hidden from the screen — left the board."],[599.8974374999999,"gmm_plot is hidden from the screen — left the board."],[599.8974374999999,"gmm_mean_a is hidden from the screen — gmm_plot left the board."],[599.8974374999999,"gmm_mean_b is hidden from the screen — gmm_plot left the board."],[599.8974374999999,"gmm_ellipse_a is hidden from the screen — gmm_plot left the board."],[599.8974374999999,"gmm_ellipse_b is hidden from the screen — gmm_plot left the board."],[599.8974374999999,"gmm_points_a is hidden from the screen — gmm_plot left the board."],[599.8974374999999,"gmm_points_a_2 is hidden from the screen — gmm_plot left the board."],[599.8974374999999,"gmm_points_a_3 is hidden from the screen — gmm_plot left the board."],[599.8974374999999,"gmm_points_a_4 is hidden from the screen — gmm_plot left the board."],[599.8974374999999,"gmm_points_a_5 is hidden from the screen — gmm_plot left the board."],[599.8974374999999,"gmm_points_a_6 is hidden from the screen — gmm_plot left the board."],[599.8974374999999,"gmm_points_a_7 is hidden from the screen — gmm_plot left the board."],[599.8974374999999,"gmm_points_b is hidden from the screen — gmm_plot left the board."],[599.8974374999999,"gmm_points_b_2 is hidden from the screen — gmm_plot left the board."],[599.8974374999999,"gmm_points_b_3 is hidden from the screen — gmm_plot left the board."],[599.8974374999999,"gmm_points_b_4 is hidden from the screen — gmm_plot left the board."],[599.8974374999999,"gmm_points_b_5 is hidden from the screen — gmm_plot left the board."],[599.8974374999999,"gmm_points_b_6 is hidden from the screen — gmm_plot left the board."],[599.8974374999999,"gmm_points_b_7 is hidden from the screen — gmm_plot left the board."],[599.8974374999999,"parameter_heading is hidden from the screen — left the board."],[599.8974374999999,"parameters is hidden from the screen — left the board."]]},{"start":601.0974375,"say":"The usual fitting procedure alternates two weighted steps. In the E step, use the current weights, means, and covariances to compute every responsibility.","live":[],"does":[[601.0974375,"em_heading is shown on the screen, written out."],[606.0779375,"e_step is shown on the screen, written out."]]},{"start":612.9939374999999,"say":"In the M step, treat those responsibilities as fractional counts. Update each mixture weight from its total responsibility. Update each mean as a responsibility-weighted average.","live":["e_step","em_heading"],"does":[[613.7949375,"m_step is shown on the screen, written out."],[619.0769375,"m_step (the \"weights\" part) is emphasized."],[622.8739374999999,"m_step (the \"means\" part) is emphasized."],[622.8739374999999,"m_step (the \"weights\" part) is no longer emphasized."],[625.5094374999999,"m_step (the \"means\" part) is no longer emphasized."]]},{"start":626.1094374999999,"say":"Then update each covariance from weighted deviations around that mean. Repeat the two steps until log likelihood no longer improves materially. Like k-means, this alternating fit can reach local optima, so restarts still matter.","live":["e_step","m_step","em_heading"],"does":[[627.4209374999999,"m_step (the \"covariances\" part) is emphasized."],[630.8929374999999,"em_stop is shown on the screen, written out."],[630.8929374999999,"m_step (the \"covariances\" part) is no longer emphasized."],[642.2009374999999,"e_step is hidden from the screen — left the board."],[642.2009374999999,"em_heading is hidden from the screen — left the board."],[642.2009374999999,"em_stop is hidden from the screen — left the board."],[642.2009374999999,"m_step is hidden from the screen — left the board."]]},{"start":642.8009374999999,"say":"Return to the elongated example. On the left, k-means uses a straight nearest-centre boundary and slices across both natural groups.","live":[],"does":[[642.8009374999999,"compare_heading is shown on the screen, written out."],[646.4469374999999,"k_label is shown on the screen, written out."],[646.4469374999999,"k_case is shown on the screen, written out."],[649.4419374999999,"k_boundary is shown on the screen, written out."],[650.0569374999999,"k_left is shown on the screen, written out."],[650.0569374999999,"k_left_2 is shown on the screen, written out."],[650.0569374999999,"k_left_3 is shown on the screen, written out."],[650.0569374999999,"k_left_4 is shown on the screen, written out."],[650.0569374999999,"k_left_5 is shown on the screen, written out."],[650.0569374999999,"k_left_6 is shown on the screen, written out."],[650.0569374999999,"k_left_7 is shown on the screen, written out."],[650.0569374999999,"k_left_8 is shown on the screen, written out."],[650.0569374999999,"k_right is shown on the screen, written out."],[650.0569374999999,"k_right_2 is shown on the screen, written out."],[650.0569374999999,"k_right_3 is shown on the screen, written out."],[650.0569374999999,"k_right_4 is shown on the screen, written out."],[650.0569374999999,"k_right_5 is shown on the screen, written out."],[650.0569374999999,"k_right_6 is shown on the screen, written out."],[651.2759374999999,"k_outline_a is shown on the screen, written out."],[651.2759374999999,"k_outline_b is shown on the screen, written out."]]},{"start":653.3509374999999,"say":"On the right, the Gaussian mixture learns two covariance ellipses. Their orientations follow the groups, and observations in the overlap can retain intermediate responsibilities instead of being treated as certain.","live":["k_label","k_case","compare_heading","k_outline_a","k_outline_b","k_left","k_left_2","k_left_3","k_left_4","k_left_5","k_left_6","k_left_7","k_left_8","k_right","k_right_2","k_right_3","k_right_4","k_right_5","k_right_6","k_boundary"],"does":[[654.2559375,"g_label is shown on the screen, written out."],[654.2559375,"g_case is shown on the screen, written out."],[657.3679374999999,"g_ellipse_a is shown on the screen, written out."],[657.3679374999999,"g_ellipse_b is shown on the screen, written out."],[659.9569374999999,"g_points_a is shown on the screen, written out."],[659.9569374999999,"g_points_a_2 is shown on the screen, written out."],[659.9569374999999,"g_points_a_3 is shown on the screen, written out."],[659.9569374999999,"g_points_a_4 is shown on the screen, written out."],[659.9569374999999,"g_points_a_5 is shown on the screen, written out."],[659.9569374999999,"g_points_a_6 is shown on the screen, written out."],[659.9569374999999,"g_points_a_7 is shown on the screen, written out."],[659.9569374999999,"g_points_b is shown on the screen, written out."],[659.9569374999999,"g_points_b_2 is shown on the screen, written out."],[659.9569374999999,"g_points_b_3 is shown on the screen, written out."],[659.9569374999999,"g_points_b_4 is shown on the screen, written out."],[659.9569374999999,"g_points_b_5 is shown on the screen, written out."],[659.9569374999999,"g_points_b_6 is shown on the screen, written out."],[659.9569374999999,"g_points_b_7 is shown on the screen, written out."]]},{"start":667.5809374999999,"say":"The gain is real, but so is the cost. More parameters require more data, covariance estimates can become unstable, and a Gaussian component can chase a tiny group unless the fit is regularized and checked.","live":["k_label","k_case","g_label","g_case","compare_heading","k_outline_a","k_outline_b","k_left","k_left_2","k_left_3","k_left_4","k_left_5","k_left_6","k_left_7","k_left_8","k_right","k_right_2","k_right_3","k_right_4","k_right_5","k_right_6","k_boundary","g_ellipse_a","g_ellipse_b","g_points_a","g_points_a_2","g_points_a_3","g_points_a_4","g_points_a_5","g_points_a_6","g_points_a_7","g_points_b","g_points_b_2","g_points_b_3","g_points_b_4","g_points_b_5","g_points_b_6","g_points_b_7"],"does":[[671.5169374999999,"g_ellipse_a is indicated — a transient flash."],[671.7169374999999,"g_ellipse_b is indicated — a transient flash."],[681.4468749999999,"compare_heading is hidden from the screen — left the board."],[681.4468749999999,"g_case is hidden from the screen — left the board."],[681.4468749999999,"g_ellipse_a is hidden from the screen — g_case left the board."],[681.4468749999999,"g_ellipse_b is hidden from the screen — g_case left the board."],[681.4468749999999,"g_points_a is hidden from the screen — g_case left the board."],[681.4468749999999,"g_points_a_2 is hidden from the screen — g_case left the board."],[681.4468749999999,"g_points_a_3 is hidden from the screen — g_case left the board."],[681.4468749999999,"g_points_a_4 is hidden from the screen — g_case left the board."],[681.4468749999999,"g_points_a_5 is hidden from the screen — g_case left the board."],[681.4468749999999,"g_points_a_6 is hidden from the screen — g_case left the board."],[681.4468749999999,"g_points_a_7 is hidden from the screen — g_case left the board."],[681.4468749999999,"g_points_b is hidden from the screen — g_case left the board."],[681.4468749999999,"g_points_b_2 is hidden from the screen — g_case left the board."],[681.4468749999999,"g_points_b_3 is hidden from the screen — g_case left the board."],[681.4468749999999,"g_points_b_4 is hidden from the screen — g_case left the board."],[681.4468749999999,"g_points_b_5 is hidden from the screen — g_case left the board."],[681.4468749999999,"g_points_b_6 is hidden from the screen — g_case left the board."],[681.4468749999999,"g_points_b_7 is hidden from the screen — g_case left the board."],[681.4468749999999,"g_label is hidden from the screen — left the board."],[681.4468749999999,"k_case is hidden from the screen — left the board."],[681.4468749999999,"k_outline_a is hidden from the screen — k_case left the board."],[681.4468749999999,"k_outline_b is hidden from the screen — k_case left the board."],[681.4468749999999,"k_left is hidden from the screen — k_case left the board."],[681.4468749999999,"k_left_2 is hidden from the screen — k_case left the board."],[681.4468749999999,"k_left_3 is hidden from the screen — k_case left the board."],[681.4468749999999,"k_left_4 is hidden from the screen — k_case left the board."],[681.4468749999999,"k_left_5 is hidden from the screen — k_case left the board."],[681.4468749999999,"k_left_6 is hidden from the screen — k_case left the board."],[681.4468749999999,"k_left_7 is hidden from the screen — k_case left the board."],[681.4468749999999,"k_left_8 is hidden from the screen — k_case left the board."],[681.4468749999999,"k_right is hidden from the screen — k_case left the board."],[681.4468749999999,"k_right_2 is hidden from the screen — k_case left the board."],[681.4468749999999,"k_right_3 is hidden from the screen — k_case left the board."],[681.4468749999999,"k_right_4 is hidden from the screen — k_case left the board."],[681.4468749999999,"k_right_5 is hidden from the screen — k_case left the board."],[681.4468749999999,"k_right_6 is hidden from the screen — k_case left the board."],[681.4468749999999,"k_boundary is hidden from the screen — k_case left the board."],[681.4468749999999,"k_label is hidden from the screen — left the board."]]}]},{"title":"Choose and Check","start":682.4885416666666,"end":822.6397916666666,"objects":{"comparison":"a Table [text] that says \"Model Membership Geometry Main fit criterion K-means hard compact cells squared distance Gaussian mixture probabilistic covariance ellipses likelihood\" (rows=(('Model', 'Membership', 'Geometry', 'Main fit criterion'), ('K…, header=True)","comparison_heading":"a Heading that says \"What Each Model Is Claiming\"","step_1":"a Text [text] that says \"Scale features according to their analytical meaning.\"","step_2":"a Text [text] that says \"Fit several starts and inspect stability.\"","step_3":"a Text [text] that says \"Compare credible values of k, not just one requested value.\"","step_4":"a Text [text] that says \"Inspect assignments, responsibilities, sizes, and covariance shapes.\"","step_5":"a Text [text] that says \"Validate against domain meaning and the downstream decision.\"","workflow_heading":"a Heading that says \"A Practical Analyst's Workflow\""},"beats":[{"start":682.4885416666666,"say":"K-means and Gaussian mixtures answer related but different modelling questions. K-means asks for a hard partition that minimizes squared distance to centres.","live":[],"does":[[682.4885416666666,"comparison_heading is shown on the screen, written out."],[682.4885416666666,"comparison is shown on the screen, written out."],[682.5585416666667,"comparison is shown on the screen, written out."],[689.3155416666666,"comparison (the \"hard\" part) is emphasized."],[691.6715416666666,"comparison (the \"hard\" part) is no longer emphasized."]]},{"start":693.8510416666666,"say":"A Gaussian mixture asks for a probability density assembled from weighted components. It returns probabilistic membership and can learn a different covariance shape for every component.","live":["comparison_heading"],"does":[[694.4195416666666,"comparison is shown on the screen, written out."],[700.9795416666666,"comparison (the \"probabilistic\" part) is emphasized."],[703.7305416666666,"comparison (the \"probabilistic\" part) is no longer emphasized."]]},{"start":706.7920416666666,"say":"Choose k-means when a fast, interpretable hard partition is useful and compact groups are plausible after sensible scaling. Choose a Gaussian mixture when overlap, unequal populations, or elliptical covariance structure is part of the question.","live":null,"does":[[707.5115416666666,"comparison (the \"row=2\" part) is emphasized."],[715.9865416666667,"comparison (the \"row=2\" part) is no longer emphasized."],[715.9865416666667,"comparison (the \"row=3\" part) is emphasized."],[723.9860416666666,"comparison (the \"row=3\" part) is no longer emphasized."]]},{"start":724.5860416666666,"say":"Neither model determines the scientifically correct number of groups by itself. K-means returns the k requested. A mixture with k components also fits the requested count, even if some fitted components become tiny or redundant.","live":null,"does":[[740.5960416666666,"comparison is hidden from the screen — left the board."],[740.5960416666666,"comparison_heading is hidden from the screen — left the board."]]},{"start":741.7960416666666,"say":"A defensible workflow begins before fitting. Scale features according to what distance should mean, then use several starts and check whether the result is stable.","live":[],"does":[[741.7960416666666,"workflow_heading is shown on the screen, written out."],[745.4295416666666,"step_1 is shown on the screen, written out."],[749.5515416666666,"step_2 is shown on the screen, written out."]]},{"start":752.7175416666666,"say":"Compare several credible values of k. For k-means, inspect objective curves and resampling stability. For mixtures, likelihood criteria such as B I C can help penalize unnecessary parameters, but none of these replaces domain judgment.","live":["step_1","step_2","workflow_heading"],"does":[[753.1235416666666,"step_3 is shown on the screen, written out."]]},{"start":770.6390416666666,"say":"Inspect the fitted objects themselves. Look for clusters created only by one scale, uncertain responsibilities, tiny components, extreme covariance estimates, and solutions that change sharply across restarts.","live":["step_1","step_2","step_3","workflow_heading"],"does":[[770.9875416666666,"step_4 is shown on the screen, written out."]]},{"start":786.1000416666666,"say":"Finally, judge the clustering by the decision it supports. A useful segment, anomaly group, or population subtype must remain interpretable and stable where it will actually be used.","live":["step_1","step_2","step_3","step_4","workflow_heading"],"does":[[786.6575416666666,"step_5 is shown on the screen, written out."]]},{"start":799.3085416666665,"say":"The durable lesson is simple. K-means alternates nearest-centre assignment with mean updates, and that simplicity creates both its speed and its geometric limits. Gaussian mixtures replace certainty with responsibilities and add weights and covariance, but those extra freedoms deserve validation because they can fit both real structure and noise.","live":["step_1","step_2","step_3","step_4","step_5","workflow_heading"],"does":[[817.7685416666666,"step_2 is indicated — a transient flash."],[817.9685416666666,"step_4 is indicated — a transient flash."],[818.3685416666666,"step_5 is indicated — a transient flash."],[821.598125,"step_1 is hidden from the screen — left the board."],[821.598125,"step_2 is hidden from the screen — left the board."],[821.598125,"step_3 is hidden from the screen — left the board."],[821.598125,"step_4 is hidden from the screen — left the board."],[821.598125,"step_5 is hidden from the screen — left the board."],[821.598125,"workflow_heading is hidden from the screen — left the board."]]}]}]},"durationSeconds":823,"chapters":[{"title":"Two Kinds of Membership","startSeconds":0,"narration":"Clustering looks for useful groups when the data has no group label. K-means is often the first method an analyst tries because its loop is fast, concrete, and easy to inspect. But its answer contains assumptions. Today we will run that loop, break those assumptions on purpose, and then see what a Gaussian mixture adds. Think of each dot as one row of a data set and each axis as one measured feature. Clustering asks whether rows that sit near one another should be treated as members of the same group. K-means gives a hard answer. Each observation belongs to exactly one cluster, represented by one centre. The rule is simply: choose the nearest centre. A Gaussian mixture can give a softer answer. Instead of one cluster label, it gives one probability for every component. Those probabilities express uncertainty about observations near an overlap. That distinction matters operationally. A hard label is convenient for routing or summarizing records. A probability is useful when a borderline case should remain borderline rather than being forced across a sharp boundary. We will not treat the more flexible model as automatically better. K-means has fewer parameters and is often exactly the right baseline. The point is to know what geometry it can represent, what failures to expect, and when extra probability and covariance parameters have a real job to do."},{"title":"Run K-Means","startSeconds":95.01412499999999,"narration":"Here is our concrete problem. We have twelve observations, two features, and a requested value k equals two. The algorithm must return two clusters. First put the data on the plane. At this stage the points have no fitted cluster identity. K-means needs starting centres. These two are deliberately away from the middle of the groups. They are guesses, not observations and not yet fitted means. The assignment step examines every observation. For each centre, compute squared Euclidean distance across the features, then give the observation to the centre with the smaller total. After that comparison, these six points belong to the red centre and these six belong to the green centre. The colour is the hard cluster label z. Now freeze the assignments and update one centre at a time. The new red centre is the coordinate-wise mean of the six red observations. The green centre is calculated in exactly the same way from the green observations. A centre is therefore a fitted average, not necessarily a row that actually occurs in the data. Move both centres to those means. As the centres travel, the assigned observations stay fixed. When the centres arrive, one full k-means pass is complete. Then repeat. With the centres in their new positions, calculate every nearest-centre assignment again. In this example every point keeps its colour. Recomputing the two means therefore returns the same two centres. The second pass changes neither the assignments nor the centres. That is convergence. Implementations may stop when assignments are identical, when centre movement is below a tolerance, or when a maximum iteration count is reached. The loop is alternating optimization. With centres fixed, nearest-centre assignment cannot increase the within-cluster sum of squares. With assignments fixed, replacing each centre by its mean cannot increase it either. So the objective falls or stays level on every pass. Because only finitely many hard assignments exist, the procedure eventually stops. What this does not prove is that it found the best possible clustering. It may have stopped at a merely local solution."},{"title":"Initialization Changes the Answer","startSeconds":242.11149999999998,"narration":"K-means can finish at different answers on the same data because its objective is not generally convex in all assignments and centres together. Here are two runs with k equal to three. On the left, the starting centres are spread across the visible groups. On the right, an unlucky draw crowds all three starting centres into the left-hand group. Run the same assignment and update loop on both sides. The favourable start settles with one centre in each visible group. The unlucky run spends two centres splitting the left group. Its remaining centre absorbs both right-hand groups, even though they are visibly separated. Another pass changes neither answer. Both are local solutions, but the right-hand solution has a larger within-cluster sum of squares and a much less useful interpretation. The practical remedy is not to trust one random start. K-means++ spreads seeds by favouring observations far from centres already chosen. It reduces bad starts, but does not abolish local minima. For routine analysis, make initialization part of the specification. Use a dispersed seeding rule, then run many independent starts. Retain the run with the lowest objective, but still inspect its groups. A numerical improvement can be immaterial, and several nearly tied runs can imply genuine ambiguity in the data. Record the number of starts, the random seed, and the spread of objective values. Reproducibility here is not clerical detail. It is evidence about how strongly the data supports the fitted partition."},{"title":"Geometry and Forced Clusters","startSeconds":349.1395,"narration":"The nearest-centre rule creates Voronoi cells: regions separated by straight boundaries. That simple geometry explains several important failures. First, consider two long natural groups. Their gray outlines run diagonally across the feature space. With these fitted centres, nearest-centre assignment makes a nearly vertical cut. Red and green divide both long groups across their length rather than following the two elongated densities. K-means has a centre for location, but no parameter for orientation or different spread in different directions. Squared Euclidean distance therefore prefers compact, roughly spherical clusters. Second, place one small dense cluster beside one large diffuse cluster. The gray circles show the intended populations. The large group contributes many more squared distances to the objective. A centre can reduce that cost by moving toward it, while the boundary gives part of the large group to the small cluster. The result is mathematically consistent with the objective, but unfair to the smaller group as a density. K-means does not model cluster population or variance separately. Third, this is one connected cloud without three clear density peaks. We nevertheless request k equals three. K-means does not answer that the data contains one group. It returns exactly three nonempty partitions because three centres were requested. This is not a software defect. The value k is an input to the problem, not a conclusion reached by the algorithm. A tidy colour map does not prove that the corresponding groups exist. Before believing a k-means result, ask whether compact clusters are plausible in the scaled feature space, and whether their spreads and populations are comparable. Then test stability across initializations and resampled data. Finally, ask what k means in the business or scientific setting. If it has no interpretation, treat it as a modelling choice to validate, not a fact discovered by the colours."},{"title":"From Soft Membership to Mixtures","startSeconds":487.92993749999994,"narration":"To relax hard assignment, keep one membership number for every component. For observation i and component j, call that number the responsibility r i j. Consider this observation near the overlap. A hard rule must choose one side. The mixture instead evaluates how plausible the observation is under both fitted components. Suppose the resulting responsibilities are zero point three five and zero point six five. The second component is more plausible, but the first remains credible. The two values add to one. These are model-based probabilities, not calibrated truth supplied by the data. They depend on the fitted component shapes, their population weights, and the Gaussian mixture assumptions. A Gaussian mixture says that the observed density is a weighted sum of Gaussian component densities. The extra parameters each answer a distinct question. The mixture weight pi records the expected share of observations generated by a component. Unlike k-means, the model can explicitly represent one large component and one small component. The mean mu gives location, much like a k-means centre. Here the two means sit near the middle of their respective densities. The covariance matrix Sigma is the crucial addition. Its overall scale controls spread. Its unequal directional variances create elongation, and its off-diagonal relationship rotates that elongation. These ellipses are equal-density contours. They can follow the long axes of the two groups, which is exactly the geometric freedom missing from nearest-centre k-means. The usual fitting procedure alternates two weighted steps. In the E step, use the current weights, means, and covariances to compute every responsibility. In the M step, treat those responsibilities as fractional counts. Update each mixture weight from its total responsibility. Update each mean as a responsibility-weighted average. Then update each covariance from weighted deviations around that mean. Repeat the two steps until log likelihood no longer improves materially. Like k-means, this alternating fit can reach local optima, so restarts still matter. Return to the elongated example. On the left, k-means uses a straight nearest-centre boundary and slices across both natural groups. On the right, the Gaussian mixture learns two covariance ellipses. Their orientations follow the groups, and observations in the overlap can retain intermediate responsibilities instead of being treated as certain. The gain is real, but so is the cost. More parameters require more data, covariance estimates can become unstable, and a Gaussian component can chase a tiny group unless the fit is regularized and checked."},{"title":"Choose and Check","startSeconds":682.4885416666666,"narration":"K-means and Gaussian mixtures answer related but different modelling questions. K-means asks for a hard partition that minimizes squared distance to centres. A Gaussian mixture asks for a probability density assembled from weighted components. It returns probabilistic membership and can learn a different covariance shape for every component. Choose k-means when a fast, interpretable hard partition is useful and compact groups are plausible after sensible scaling. Choose a Gaussian mixture when overlap, unequal populations, or elliptical covariance structure is part of the question. Neither model determines the scientifically correct number of groups by itself. K-means returns the k requested. A mixture with k components also fits the requested count, even if some fitted components become tiny or redundant. A defensible workflow begins before fitting. Scale features according to what distance should mean, then use several starts and check whether the result is stable. Compare several credible values of k. For k-means, inspect objective curves and resampling stability. For mixtures, likelihood criteria such as B I C can help penalize unnecessary parameters, but none of these replaces domain judgment. Inspect the fitted objects themselves. Look for clusters created only by one scale, uncertain responsibilities, tiny components, extreme covariance estimates, and solutions that change sharply across restarts. Finally, judge the clustering by the decision it supports. A useful segment, anomaly group, or population subtype must remain interpretable and stable where it will actually be used. The durable lesson is simple. K-means alternates nearest-centre assignment with mean updates, and that simplicity creates both its speed and its geometric limits. Gaussian mixtures replace certainty with responsibilities and add weights and covariance, but those extra freedoms deserve validation because they can fit both real structure and noise."}]}}