The context compaction failure mode is the one I keep thinking about. You can write the most careful system prompt in the world and an aggressive memory summarization pass just... discards it.
The reversible action rule you lay out is the right mental model - not "is this agent smart enough" but "if this goes sideways and I can't undo it, how bad is that?"
Running autonomous agents that touch real systems means the question of blast radius comes before capability. Most people configure for capability first and learn the blast radius lesson the hard way.
The context compaction failure mode is the one I keep thinking about. You can write the most careful system prompt in the world and an aggressive memory summarization pass just... discards it.
The reversible action rule you lay out is the right mental model - not "is this agent smart enough" but "if this goes sideways and I can't undo it, how bad is that?"
Running autonomous agents that touch real systems means the question of blast radius comes before capability. Most people configure for capability first and learn the blast radius lesson the hard way.
That's why I'm going low and slow, Pawel.
Thank-you again Nicolle
I do not want to be the one running across the apartment.
I just want it to go find stuff for me that I have in my phone somewhere
That would include email
Im not ready for that
I’ll keep reading and learning from you
Thank-you
Glad you found something use in the piece, Cynthia! I'd encourage you to do the exercises and see how you feel, get more comfortable, and then try!
Thank-you for reiterating exercises. I’ll go reread that now.
Love and appreciate your articles Nicolle.