Superintelligence
Bostrom examines possible advanced AI systems and the difficulty of ensuring their objectives remain compatible with human interests.
Superintelligence is defined through capabilities exceeding humans across important domains.
Being very capable is not the same as wanting what humans want. Bostrom separates those questions so that impressive performance does not become automatic reassurance about a system's objectives.
Different technological paths could produce such capabilities.
A transition's speed affects the opportunities for preparation and oversight.
Intelligence does not logically guarantee benevolent goals.
Different final goals can generate similar instrumental pressures.
The scenarios are uncertain, not news from an inevitable future. Their purpose is to make preparation discussable before power and dependence make correction harder, while keeping assumptions open to challenge.
Specifying intentions precisely is harder than describing them informally.
Control and value alignment become central research problems.
Institutional decisions matter alongside technical design.
The scenarios are uncertain arguments, not established forecasts.
The conclusion prioritises preparation before systems become difficult to govern.