The Right to Be Unpredictable

A recommendation system begins from a practical limitation: there is too much available. No one can inspect every film, product, song, post, route, restaurant or job. The system therefore ranks possibilities. It uses information about the item, information about other users and information about your previous behaviour to estimate what deserves to appear first.

This can feel uncannily personal because good prediction often resembles recognition. The system remembers that you prefer documentaries, buy the same coffee, read about architecture and abandon videos after thirty seconds. It appears to have learned who you are. But its knowledge has a particular shape. It knows you through recorded differences in behaviour. It contains traces of choices made under earlier circumstances.

That distinction matters because a history is not the same thing as a person. A person can acquire a new interest before enough behavioural evidence exists to make that interest statistically legible. They can decide to stop drinking before their shopping history changes. They can become serious about a subject they previously ignored. They can leave a profession, a political belief, a relationship, a city or a style of life while databases continue to contain evidence of the person for whom those things were normal.

The model is not necessarily defective when this happens. It is doing what models do: using what has already happened to reduce uncertainty about what happens next. The problem appears when the prediction is allowed to determine too much of the field in which the next behaviour will occur.

Suppose a music service predicts that you prefer one kind of music and therefore places more of it in front of you. You play some of what appears. Those plays become new evidence that the original prediction was correct. There is nothing mysterious about the loop. The service influenced exposure; exposure influenced the set of possible responses; those responses returned as data.

Researchers studying recommender systems describe versions of this problem as feedback loops. Recommendations are not neutral measurements taken from outside behaviour. They are interventions in the information environment. Studies and simulations have shown that repeated recommendation and response can amplify popularity bias, reduce the diversity of exposure and alter the representation a system builds of user taste. The size and direction of these effects vary by system, user and setting, but the underlying point is simple: observed preference is partly preference observed after something was made available to prefer.

This is easy to miss because the loop can still produce accurate recommendations. If you genuinely enjoy what the system shows you, the fact that it influenced the opportunity does not make the enjoyment false. Nor does every recommendation manufacture a preference. People ignore suggestions, search independently, change platforms and surprise models constantly.

The conceptual shift is smaller and more important. Prediction is no longer only an attempt to foresee the next event. Once a prediction changes exposure, price, ranking, opportunity or attention, it becomes one of the conditions from which the next event emerges. The model has entered the world it is modelling.

A system that repeatedly predicts from its own consequences can become very good at recognising the version of you that exists inside the environment it helped create. That is not quite the same as discovering every version you might become outside it.

History has inertia. A single contradictory action is usually weak evidence against a long pattern, and often it should be. Someone who watches one horror film has not necessarily developed a passion for horror. A bank should not infer a permanent change of circumstances from one unusual payment. Useful models need continuity or they would chase noise.

But continuity creates a lag. Change often begins as a small number of events that look anomalous when judged against the past. The first vegetarian meal sits inside years of meat purchases. The first application for a different kind of work sits inside an employment history pointing elsewhere. The first month without alcohol can look, to a retailer, like an interruption in an established pattern. A new person necessarily begins with less evidence than the old one.

This creates an asymmetry. The past arrives with data. The future arrives initially as intention. If a system trusts only accumulated behaviour, the old identity has an evidential advantage simply because it has existed longer.

Sometimes that is harmless. Sometimes it becomes consequential. Credit, insurance, employment, fraud detection, advertising and automated eligibility systems can use historical categories or proxies to make decisions about present people. Different systems have different legal duties, data sources and levels of consequence, so they should not be collapsed into one machine. Yet they share a design question: how quickly can new evidence revise an old inference, and what happens when the person cannot see or challenge the inference being used?

A profile becomes constraining when it is easier for the system to preserve its model than for the person to demonstrate that the model is stale. The issue is not that memory is bad. It is that memory without sufficient revision can turn biography into destiny.

Then there is a more intimate version of the same problem. Tell people who have known you for years that you are changing something important and one response appears with remarkable frequency: “That’s not you.”

Sometimes the sentence is affectionate. A friend knows you hate early mornings and laughs when you announce a plan to run at six. A parent remembers ten abandoned hobbies and does not immediately reorganise their picture of you around the eleventh. Familiarity contains useful prediction. Relationships would be exhausting if every conversation began with complete uncertainty about who the other person had become overnight.

But people also build models. We remember temperament, loyalties, weaknesses, tastes, roles, past promises and repeated behaviour. We use those memories to anticipate reactions. The quiet sibling will probably stay quiet. The unreliable friend will probably be late. The ambitious colleague will want the promotion. The person who always reconciles the family will probably do it again.

Social psychology has studied what happens when expectations enter interaction. Interpersonal expectancy effects are real: what one person expects can change how they behave towards another, and that treatment can sometimes increase the chance of an expectancy-confirming response. The evidence also gives an important warning against turning this into a total theory of human behaviour. Expectations are often accurate, people resist them, and self-fulfilling effects are generally more limited than popular accounts imply.

That limitation strengthens the useful claim. Other people do not write our personalities for us. But their expectations become part of the environment in which our behaviour occurs. A person trying to become more confident encounters people accustomed to speaking for them. Someone trying to drink less attends gatherings organised around the version of them who drank more. A person leaving a familiar career meets relatives who continue sending vacancies from the old one. The model does not control the person. It changes the resistance surrounding the change.

There is a reason this resistance can be stronger among people who know us well. Our identities are not stored privately inside separate bodies. Relationships acquire structure around repeated expectations. One person plans because the other improvises. One earns while the other provides care. One is the difficult one, the sensible one, the funny one, the successful one, the one who needs rescuing or the one who rescues everyone else.

When one person changes, the relationship may need to change with them. If the friend who always agrees begins saying no, someone loses the convenience of agreement. If the child treated as irresponsible becomes organised, a parent may lose a familiar role as supervisor. If a partner becomes more independent, routines built around dependence no longer fit. Updating your model of another person can therefore require more than changing an opinion. It can require changing your own behaviour, status, expectations and story about the relationship.

This does not mean resistance is always selfish or malicious. Old knowledge can be protective. Someone who remembers an earlier crisis may have good reasons to distrust sudden confidence. A family may recognise a recurring pattern that the person inside it cannot see. Being known by others can rescue us from our own selective memory.

The problem is not prediction itself. It is the refusal to let present evidence acquire enough weight to revise prediction. “I know you” can mean: I have paid attention to your history. It can also mean: your history gives me authority over what counts as authentic for you now.

That is where intimacy can become strangely conservative. The people with the richest archive of your past may have the most material from which to argue that a new version of you is out of character. The stranger has no such archive. They meet the evidence that is standing in front of them.

We need prediction. A world in which every system and every relationship discarded history each morning would not be freer; it would be incoherent. Trust depends partly on remembered behaviour. Expertise depends on learned regularities. Recommendation is useful because yesterday often does tell us something about tomorrow.

The better distinction is between using the past to orient towards a person and using the past to replace them. Prediction says: given what I know, this is what I expect. Recognition adds a second movement: this is what is happening now.

Those two forms of knowledge can cooperate. A good friend can expect you to hate the party and still notice that you are enjoying yourself. A doctor can use medical history without allowing it to erase a new symptom. A recommendation system can exploit known preferences while reserving some space for exploration. A user profile can decay, reset, expose its assumptions or allow corrections. A decision system can give recent, relevant evidence a route to overcome an older proxy.

The point is not to make systems deliberately stupid. It is to make uncertainty visible in the model. A prediction should contain, conceptually if not always literally, the possibility of being revised by the event it is predicting.

This is also why unpredictability should not be confused with randomness. A random person is not necessarily a free person. Acting against every expectation merely gives the expectation another form of control: it still determines the direction, only now by reversal. The meaningful freedom is authorship — the capacity to continue a pattern because it still serves you, or depart from it because something has changed.

Someone may remain perfectly predictable for years because they keep choosing the same values. The right at stake is not an obligation to surprise. It is the right for surprise, when it arrives, to count.

Return to the streaming service. It has not done anything wrong by remembering what you liked. Your friends have not betrayed you by remembering who you have been. A history is one of the things that makes a person intelligible across time. Without continuity, promises, responsibility, learning and intimacy would become difficult to sustain.

But continuity is not ownership. The record of a person is evidence about them, not a claim on their future. The distinction becomes more important as predictive systems spread into more consequential parts of life and as personal archives become longer, cheaper to store and easier to analyse. A past that can always be retrieved can begin to feel more authoritative than a present that has only just begun.

Calling this a right to be unpredictable is therefore slightly misleading on purpose. The valuable thing is not unpredictability itself. It is the freedom that unpredictability reveals: the fact that a person can produce an action for which the best explanation is not simply that they have done similar things before. They can learn. They can decide. They can encounter something new. They can revise a value. They can become tired of a role. They can discover that an old description was never very good. They can make a promise and build enough new history for the promise to become a pattern.

Systems that know us should be designed with this possibility in mind. They need memory, but also forgetting, exploration, correction, expiry, appeal and routes through which recent evidence can matter. The correct mechanism will differ with the stakes. A film recommender can afford playful uncertainty. A lending or employment decision requires stronger transparency and safeguards. The common principle is that accumulated prediction should not become an invisible veto on revision.

People need a version of the same humility. To know someone well is not to possess the final draft of them. It is to hold a detailed model while remaining capable of being surprised by the person from whom the model was learned. Sometimes love is accurate prediction. Sometimes it is noticing that the prediction has stopped being accurate and allowing the relationship to change.

A free life will contain continuity. Most mornings we will resemble the person who went to sleep the night before. The danger begins when resemblance is treated as a requirement.

The systems that know you will predict you. The people who know you will predict you too. Both may often be right. Freedom requires that neither is entitled to stay right after you have changed.