AI alignment

Here I specifically mean the technical alignment problem: creating a system which does what its operator would intend.

This is often discussed interchangeably with “AI safety”, or as “the name for the solution to AI risk”, but that’s obviously misleading. Aligned AI still poses terrible risk.

• Is it even possible to align an AI?
  • People often call for more investment or people working on AI alignment, but I worry that work on AI alignment may be accelerating capabilities more than it accelerates safety.
  • Emergent capabilities of large foundation models often surprise their creators

Of course, there’s also the very serious problem of “aligned with what?!”.