Re. "When I workshopped the first draft of the story I was asked ‘Why would the AI work in that way? It’s impossible for it to have come to such a utility function on accident, and literally no one has the motivation to make it have this utility function.’ ‘Why do these two aspects of the AI act like this, that’s not how a functional entity would act given X-Z.’" :
Where do you workshop your stories? Most of the people I was in workshops in wouldn't know what a utility function is.
>literally no one has the motivation to make it have this utility function
It *sort* of makes sense. You program it to Maximize something humans produce, specifically fanfiction, and not only that, but fanfiction written by humans so it can’t just tile the light cone with chatbot “authors”, so in a roundabout way it maximizes humans and not only that, humans with a society relatively like ours insofar as they have free time and hobbies and artistic creativity and so forth and so on.
For this particular story I workshopped it at the Effective Altruism fiction retreat/workshop in 2022. So lots of rats and adjacents. Normally I wouldn't get that sort of comment either, yeah. :)
Re. "When I workshopped the first draft of the story I was asked ‘Why would the AI work in that way? It’s impossible for it to have come to such a utility function on accident, and literally no one has the motivation to make it have this utility function.’ ‘Why do these two aspects of the AI act like this, that’s not how a functional entity would act given X-Z.’" :
Where do you workshop your stories? Most of the people I was in workshops in wouldn't know what a utility function is.
>literally no one has the motivation to make it have this utility function
It *sort* of makes sense. You program it to Maximize something humans produce, specifically fanfiction, and not only that, but fanfiction written by humans so it can’t just tile the light cone with chatbot “authors”, so in a roundabout way it maximizes humans and not only that, humans with a society relatively like ours insofar as they have free time and hobbies and artistic creativity and so forth and so on.
For this particular story I workshopped it at the Effective Altruism fiction retreat/workshop in 2022. So lots of rats and adjacents. Normally I wouldn't get that sort of comment either, yeah. :)