# Gemini ## Introduction to Ethical Frameworks The lecture commences by setting a clear agenda for the session and the subsequent weeks. The primary goal is to introduce the first of three major ethical theories that will form the analytical backbone of the course: utilitarianism. An ethical theory is a systematic framework that attempts to provide a rational basis for determining what is morally right and wrong. These theories are not merely collections of opinions; they are structured systems of principles designed to guide decision-making in complex situations, especially those involving moral dilemmas or challenges. The lecture aims to first build a comprehensive understanding of the theory of utilitarianism itself, exploring its core principles, historical development, and internal variations. Following this theoretical foundation, the second objective is to begin applying this framework to practical, real-world cases involving artificial intelligence (AI). This application is a critical skill that will be honed in tutorials and assessed in the first essay, making the content of this lecture directly relevant to the course's evaluation structure. The lecturer notes that while a few students may have encountered the term "utilitarianism" before, the majority have not, and thus the lecture will proceed from the fundamentals, assuming no prior knowledge. This approach ensures that all students can build a solid and complete mental model of the theory. The lecture will be followed in subsequent weeks by explorations of two other major ethical theories, deontology and virtue ethics, which will provide alternative and often contrasting perspectives on moral reasoning. Before delving into the formal theory of utilitarianism, the lecture employs a well-known philosophical thought experiment to prime the audience's ethical intuitions: the trolley problem. This serves as a practical entry point into the kind of thinking required for ethical analysis. A thought experiment is a hypothetical scenario designed to isolate specific variables and explore the implications of a particular idea or principle. The trolley problem, in its various forms, is specifically designed to be an ethical challenge that forces us to confront difficult choices and, more importantly, to examine the *reasons* behind our choices. By engaging with this problem, we begin to uncover the implicit principles that guide our own moral judgments, setting the stage for a more formal examination of how a theory like utilitarianism would approach such a dilemma. ## The Trolley Problem: An Intuitive Exploration of Ethical Dilemmas The trolley problem is a thought experiment in ethics, with its modern formulation largely attributed to philosophers Philippa Foot and Judith Jarvis Thomson in the mid-20th century. It is not a puzzle with a single correct answer, but rather a tool designed to stimulate thinking about the principles that underpin our moral judgments. The classic scenario is presented as follows: a runaway trolley car is hurtling down a railway track, its brakes having failed. Ahead on the track, five people are tied up and unable to move. If the trolley continues on its current path, it will inevitably strike and kill all five of them. You, as an observer, find yourself standing next to a lever. If you pull this lever, you can switch the trolley to a different track, a side track. However, there is one person tied to this side track. Therefore, you are faced with a stark choice. You can choose to do nothing, in which case the trolley proceeds straight and kills the five people. Alternatively, you can choose to act by pulling the lever, which will divert the trolley, resulting in the death of the one person on the side track but saving the five. This scenario is intentionally bleak and removes many real-world complexities to focus the mind on a core ethical conflict. There is no "happy" solution where everyone is saved; the choice is between two tragic outcomes. The question posed is not just what one *would* do, but what one *should* do. When polled, the class's response to this scenario was notable. A significant majority, nearly 60%, indicated that they would *not* pull the lever, choosing to let the five people die rather than actively intervening to cause the death of one. Approximately 30-40% of the class stated they would pull the lever, and a very small number were unsure. The lecturer observes that this result is somewhat different from previous years, where a larger proportion of students typically opted to pull the lever. This variation highlights that intuitions about even this classic problem are not universal or static. The crucial next step, which the lecture will facilitate, is to move beyond the initial intuitive response and begin to articulate the *reasons* for that choice. Why might it be considered morally preferable to let five die rather than to cause one death? Or conversely, why might saving the greater number be the overriding moral imperative? To further probe these intuitions and reveal their potential inconsistencies, a second version of the trolley problem is introduced. This variation, often called the "footbridge" or "fat man" problem, maintains the same numerical outcome but changes the nature of the action required. In this scenario, the runaway trolley is again heading towards five people on the track. This time, you are standing on a bridge overlooking the track, and next to you is a very large person. You realize that if you were to push this person off the bridge and onto the track below, their body is large enough to stop the trolley, thereby saving the five people. The person would, of course, be killed by the fall and the impact. The choice is again between one death and five deaths. When presented with this scenario, the class's response shifted dramatically. Whereas 30-40% were willing to pull the lever in the first case, almost no one was willing to physically push the person off the bridge. The overwhelming majority felt it would be wrong to do so. This stark difference in response to two problems with identical consequences—one person dies to save five—is the central puzzle that these thought experiments are designed to expose. If the outcome is the same, why does our moral intuition treat the two cases so differently? This forces a deeper reflection on the nature of moral action. The difference seems to lie in the *means* by which the outcome is achieved. Pulling a lever feels indirect, a diversion of an existing threat. Pushing a person feels direct, a physical and intentional use of another human being as a mere tool to achieve an end. This distinction between killing someone and letting someone die, or between redirecting a threat and creating a new one, lies at the heart of many ethical debates. These intuitions, while powerful, are not yet a coherent theory. They are the raw data that ethical theories like utilitarianism attempt to explain, systematize, or in some cases, challenge and correct. ## From Thought Experiments to Real-World AI: The Moral Machine The abstract nature of the trolley problem has found a surprisingly direct and urgent application in the field of artificial intelligence, specifically in the programming of autonomous vehicles. The "Moral Machine" project, developed by researchers at MIT, translates the core dilemma of the trolley problem into the context of self-driving cars. An autonomous vehicle, through its AI, may one day face an unavoidable accident scenario where it must make a choice that will result in harm. The ethical challenge shifts from a person's split-second decision to a programmer's pre-meditated choice: how should the car be programmed to behave in such situations? The Moral Machine is an online platform that presents users with a series of complex crash scenarios. For example, a self-driving car with brake failure might have to choose between continuing straight and hitting a group of pedestrians, or swerving and crashing, potentially killing its own passengers. The scenarios are made more complex by introducing variables about the individuals involved: their age (e.g., elderly vs. young), gender, social status (e.g., doctor vs. criminal), and even species (e.g., humans vs. pets). By collecting millions of responses from people around the world, the project aims to map out global moral preferences and identify both commonalities and cultural differences in how people weigh these different factors. The lecture presents a few of these scenarios to the class to illustrate the complexity. For instance, should a car in an unavoidable crash preferentially save a 30-year-old over a 50-year-old? The class was divided, with a majority favoring a random choice (flipping a coin), but a small number favoring saving the younger person, perhaps based on an intuition about "years of life remaining." In another scenario, choosing between hitting a man and a pregnant woman, the class was far more decisive, with a large majority opting to save the pregnant woman, implicitly valuing two lives (the woman and the fetus) over one. A final, more provocative example involved choosing between a known serial killer and a military service dog. Here, the class overwhelmingly chose to sacrifice the serial killer, demonstrating a strong intuition that a person's perceived moral character or past actions can affect their right to be saved in a crisis. These exercises demonstrate that when we move from the abstract "person" in the trolley problem to individuals with specific characteristics, our moral calculations become more nuanced and, in some ways, more biased. We begin to make judgments about the relative value of different lives. The question for AI ethics is profound: should these human biases be encoded into our machines? Or should an AI be programmed to be perfectly impartial, perhaps by always prioritizing the saving of the greatest number of people, regardless of who they are? This is precisely the kind of question that requires a formal ethical theory for a consistent and justifiable answer. Relying on a patchwork of gut feelings, as the Moral Machine data shows, can lead to contradictory and potentially discriminatory outcomes. ## The Need for a Systematic Approach: Consequentialism and Utilitarianism The trolley problems and Moral Machine scenarios reveal that our untutored moral intuitions can be inconsistent and difficult to justify. We might feel strongly that pushing the man off the bridge is wrong, but struggle to articulate a principle that explains why it's different from pulling the lever. This is where formal ethical theories become invaluable. They provide a structured, principled way to analyze and resolve such dilemmas. The lecture introduces three major families of theories: utilitarianism, which focuses on the **consequences** of actions; deontology, which focuses on moral **rules and duties**; and virtue ethics, which focuses on the **character** of the moral agent. The focus of this lecture is the first of these: utilitarianism. Utilitarianism is a specific form of a broader ethical approach known as **consequentialism**. Consequentialism is the view that the moral rightness or wrongness of an action is determined *solely* by its consequences. Nothing else matters—not the actor's intentions, not their feelings, and not the intrinsic nature of the act itself. If an action produces a good outcome, it is a good action; if it produces a bad outcome, it is a bad action. Philosophers refer to this as a "teleological" theory, from the Greek word *telos*, meaning "end," "purpose," or "goal." The morality of an action is judged by the end it achieves. This is the foundation of the often-misunderstood phrase, "the ends justify the means." For a pure consequentialist, if the ends are sufficiently good, they will indeed justify the means taken to achieve them. However, consequentialism is a broad category. One could be a consequentialist in a very selfish way. For example, **ethical egoism** is a consequentialist theory which states that the right action is the one that maximizes good consequences *for oneself*. An ethical egoist believes their only moral obligation is to promote their own self-interest. Utilitarianism stands in stark opposition to this. It is a consequentialist theory, but it is defined by its commitment to **impartiality**. Utilitarianism asserts that when we calculate the consequences of an action, we must not give special weight to our own interests or the interests of our friends or family. Instead, every individual's interests must be given equal consideration. The core idea is that "all interests matter equally." If an action will cause a certain amount of pain, it is equally bad whether that pain is experienced by me, a stranger on the other side of the world, or any other sentient being. This principle of equal consideration is a radical and foundational component of the theory. Therefore, utilitarianism is not about maximizing *my* good, but about maximizing the *total* or *overall* good in the world, considering everyone affected by the action equally. ## The Foundations of Utilitarianism: Bentham, Mill, and the Principle of Utility Modern utilitarianism was most famously and systematically articulated by the English philosopher and social reformer Jeremy Bentham in the late 18th and early 19th centuries. His work emerged during the Enlightenment, a period of intellectual history that championed reason over tradition and religious dogma. Before this time, morality in the Western world was largely dictated by religious authority, with strict rules about behavior derived from sacred texts and church doctrine. Utilitarianism was revolutionary because it proposed a secular, reason-based foundation for morality. It argued that moral rules are not divine commands to be blindly obeyed; rather, morality is a tool "made for man, not man for morality." Its purpose is to improve the human condition by increasing well-being and reducing suffering. This progressive spirit led early utilitarians like Bentham and his intellectual successor, John Stuart Mill, to advocate for radical social reforms, including the expansion of women's rights, the abolition of slavery, and the reform of the legal system, all based on the argument that these changes would lead to better overall consequences for society. The core of utilitarianism is encapsulated in the **Principle of Utility**. This principle provides the criterion for right and wrong action. As John Stuart Mill formulated it, the principle states that "actions are right in proportion as they tend to promote happiness, wrong as they tend to produce the reverse of happiness." More precisely, the right action in any given situation is the one that produces the **best possible consequences** among all available options. It is not enough for an action to produce *some* good; it must produce the *most* good, or the maximum possible amount of "utility." Utility is the technical term for the good that we are trying tomaximize, which is generally understood as happiness or well-being. To apply this principle, a utilitarian must engage in what is sometimes called the **utilitarian calculus**. This involves a three-step process. First, for any proposed action, one must identify all the good consequences (positive utility) and all the bad consequences (negative utility) that are likely to result from it. Second, one must weigh and aggregate these consequences, summing up the positives and subtracting the negatives to determine the *net utility* of that action. Third, one must repeat this process for all alternative actions available and then choose the action that yields the highest net utility. Consider a simple numerical example: an action is predicted to cause 5 units of pain (negative utility) but also 20 units of pleasure (positive utility). The net utility of this action would be +15 units (20 - 5). If an alternative action would only produce a net utility of +10 units, the utilitarian would be obligated to choose the first action. The "poisoned soup" example from the lecture illustrates this in a more qualitative way. The act of poisoning the soup has several consequences: Caitlin experiences a mild stomach ache (a small negative), Bill is hospitalized with severe dehydration (a large negative), and Abby, the poisoner, feels moderately pleased (a moderate positive). To determine if this action was wrong, a utilitarian must sum these outcomes. The severe suffering of Bill and the mild suffering of Caitlin almost certainly outweigh the moderate pleasure Abby derives from their pain, resulting in a large net negative utility. Therefore, the action is morally wrong. Crucially, a utilitarian must, in principle, count Abby's sadistic pleasure as a positive in the calculation, even if it seems morally repugnant. The theory is concerned only with the sum total of utility, not with the moral character of the sources of that utility. This commitment to aggregation, regardless of how the good and bad are distributed or their source, is a defining and sometimes controversial feature of the theory. ## Defining Utility: Hedonism vs. Preference Satisfaction If utilitarianism commands us to maximize utility, we must have a clear understanding of what "utility" is. The classical utilitarians, Jeremy Bentham and John Stuart Mill, were **hedonistic utilitarians**. Hedonism is the view that the only thing that is intrinsically good—good in and of itself—is pleasure, and the only thing that is intrinsically bad is pain. For a hedonist, happiness is simply the presence of pleasure and the absence of pain. However, Bentham and Mill had a significant disagreement about the nature of pleasure. Bentham held a purely quantitative view. He famously declared that, "prejudice apart, the game of push-pin is of equal value with the arts and sciences of music and poetry." What he meant was that if a simple child's game (push-pin) provides the same intensity and duration of pleasure as listening to a sublime piece of poetry, then the two experiences are of equal moral value. For Bentham, all that matters is the *quantity* of pleasure. The source or type of pleasure is irrelevant. Pain is pain, and pleasure is pleasure. John Stuart Mill found this view unsatisfactory, arguing that it reduced human life to the level of animalistic sensation. He proposed a qualitative distinction between different types of pleasure. Mill argued that there are **higher pleasures** and **lower pleasures**. Higher pleasures are those that engage our intellectual, creative, and moral faculties—the pleasures of philosophy, art, friendship, and helping others. Lower pleasures are the more bodily or sensual pleasures—those of eating, drinking, and other physical sensations. Mill argued that the higher pleasures are intrinsically more valuable than the lower ones, even if the lower pleasures are more intense. He famously wrote, "It is better to be a human being dissatisfied than a pig satisfied; better to be Socrates dissatisfied than a fool satisfied." For Mill, someone who has experienced both types of pleasure will invariably prefer the higher ones. Therefore, when calculating utility, we should give greater weight to the higher-quality intellectual and moral pleasures. This distinction, while intuitively appealing to some, creates a theoretical problem for Mill. If he is a true hedonist, claiming that only pleasure is good, on what basis can he claim that one type of pleasure is "better" than another, other than by the amount of pleasure it contains? This has led some critics to argue that Mill secretly smuggled in a non-hedonistic value, like intellectual development or dignity, into his theory. In response to these and other problems with hedonism, many contemporary utilitarians have adopted a different theory of well-being: **preference utilitarianism**. This view holds that utility is not about pleasure and pain, but about the satisfaction of our desires or preferences. The right action is the one that satisfies the most preferences, weighted by the strength of those preferences, for the greatest number of people. This approach avoids having to define a single substance like "pleasure" and instead focuses on what people actually want for themselves. The "experience machine" thought experiment, a concept similar to the film *The Matrix*, helps to clarify the difference. Imagine a machine that could give you any experience you desired by stimulating your brain. You could have the experience of writing a great novel, falling in love, or achieving world peace, all while floating in a tank. A hedonistic utilitarian might have to conclude that plugging into this machine for life is the best possible choice, as it would guarantee maximum pleasure and zero pain. However, many people feel a strong resistance to this idea. We don't just want the *feeling* of accomplishing things; we want to *actually accomplish* them. We have a preference for living in reality. Preference utilitarianism can account for this. If you have a strong preference to actually live your life and not just experience a simulation of it, then plugging into the machine would frustrate that preference, and would therefore be a bad outcome for you, even if it produced more pleasure. This makes preference utilitarianism a more flexible and, for many, a more plausible account of what makes a life go well. ## The Expanding Moral Circle and Act vs. Rule Utilitarianism One of the most powerful and radical implications of utilitarianism is its ability to expand the circle of moral concern. The contemporary philosopher Peter Singer, a prominent utilitarian, has famously applied the theory to the treatment of non-human animals. His argument is a direct extension of utilitarian principles. If the basis for moral consideration is the capacity to have interests—to experience pleasure and pain, or to have preferences—then any being with this capacity deserves to have its interests counted. Since it is scientifically undeniable that many animals are sentient and can feel pain, their interests must be included in our utilitarian calculus. Furthermore, the principle of equal consideration demands that we give their interests equal weight to our own *similar* interests. This means that the severe pain an animal experiences in a factory farm, for example, should be weighed equally against the trivial human pleasure of eating a certain type of food. When the calculation is performed, Singer argues, it becomes clear that many of our current practices involving animals are profoundly unethical because they cause immense suffering for the sake of minor human benefits. This demonstrates how utilitarianism, taken seriously, can challenge deeply ingrained societal norms and demand significant changes in our behavior. As we apply utilitarianism to more complex cases, a critical distinction emerges between two major variants of the theory: **act utilitarianism** and **rule utilitarianism**. This distinction is crucial for addressing some of the most common and powerful objections to the theory. **Act utilitarianism** is the more straightforward version. It states that we should apply the principle of utility directly to each individual action. In any situation, the right thing to do is the specific act that will produce the greatest good in that particular instance. This is the version that leads to some of the most counter-intuitive conclusions. For example, in the "doctor and the organs" scenario—where a doctor could kill one healthy patient to harvest their organs and save five dying patients—an act utilitarian might be forced to conclude that killing the healthy patient is the morally right thing to do, as it maximizes utility (five lives saved for the cost of one). This conclusion strikes most people as morally monstrous, as it seems to violate fundamental rights and principles of justice. **Rule utilitarianism** was developed to address such problems. It is a two-step theory. First, we do not apply the principle of utility to individual acts, but to potential *moral rules*. We ask: "What set of general rules, if consistently followed by everyone, would lead to the greatest overall utility?" For example, we would compare the consequences of a society that follows the rule "Doctors must never kill healthy patients to harvest their organs" with a society that follows the rule "Doctors should kill healthy patients whenever it would save more lives." It is overwhelmingly clear that the first rule would produce a much better society. People would trust doctors, seek medical care without fear, and the entire healthcare system would function effectively. The second rule would lead to terror and the collapse of medical trust. Therefore, a rule utilitarian would establish "Do not kill innocent patients" as a justified moral rule. The second step is then to follow these justified rules, even in specific cases where breaking the rule might seem to produce more good in that one instance. So, in the organ transplant case, the rule utilitarian doctor would refuse to kill the healthy patient, not because of the consequences of that single act, but because they are obligated to follow the rule that has the best consequences overall. Rule utilitarianism thus aligns more closely with our common-sense moral intuitions about rights, promises, and justice. However, act utilitarians criticize it for what they call "rule worship." They argue that if the ultimate goal is to maximize utility, it is irrational to stick to a rule when you know that breaking it in a specific instance would lead to a better outcome. The debate between these two forms of utilitarianism is ongoing and highlights a central tension within the theory between maximizing good on a case-by-case basis and upholding the general principles that support a flourishing society. ## Applying Utilitarianism to Artificial Intelligence The final part of the lecture brings the theoretical discussion back to the practical domain of AI ethics, using the example of AI-powered headbands designed to monitor student concentration in a classroom. To analyze this technology from a utilitarian perspective, one must perform the utilitarian calculus, identifying and weighing all the potential good and bad consequences for all stakeholders involved. The potential **positive utilities (benefits)** are numerous. For students, the feedback could genuinely help them improve their focus and learning outcomes, leading to better grades and future opportunities. For teachers, the real-time data could help them identify struggling students and tailor their teaching methods more effectively. For the school, improved student performance could enhance its reputation. For the company that developed the technology, there are clear financial benefits and the opportunity to refine their product. Society at large might even benefit from a more educated and productive citizenry. However, the potential **negative utilities (harms)** are equally significant and must be weighed against the benefits. For students, the constant monitoring could create immense stress, anxiety, and performance pressure. It could lead to public embarrassment if their "concentration scores" are displayed. This technology represents a profound invasion of privacy, monitoring a person's internal mental state. There is also the risk of the data being misused, perhaps to label students or limit their future opportunities. The system might also stifle creativity and different learning styles by enforcing a single, narrow definition of "concentration." An **act utilitarian** would have to weigh all these specific, predictable harms and benefits in this particular school and decide if, on balance, the introduction of the headbands would produce more good than harm. The answer is not obvious and would depend on a detailed empirical assessment. A **rule utilitarian**, on the other hand, would ask a broader question: "What would be the consequences if we adopted a general rule allowing for the constant AI-monitoring of students' mental states in schools?" This perspective forces us to consider the long-term societal impact. Such a rule might lead to a society where surveillance is normalized from a young age, where individual privacy is eroded, and where people are constantly being measured and judged by opaque algorithms. The potential for a chilling effect on free thought and the creation of a high-pressure, conformist educational environment could represent a massive net negative utility for society. Therefore, a rule utilitarian might conclude that the practice is unethical, even if it could be shown to produce some local benefits in a specific classroom. This example demonstrates the power of utilitarianism as an analytical tool. It forces us to move beyond a simple "this is good" or "this is bad" reaction and to conduct a systematic, impartial, and comprehensive assessment of the consequences of a technology on everyone it affects. While the theory has its own complexities and controversies, it provides an indispensable framework for thinking through the profound ethical challenges posed by the development and deployment of artificial intelligence. The lecture concludes by noting that this is just the first of several theoretical tools that will be explored, setting the stage for the introduction of deontology in the following week, which will offer a fundamentally different approach to these same problems.