Summarization at the leader's ear (or readme)
I’ve been thinking a lot about leader or manager readmes for a while. They seem to be good documents to set expectations with the people that work with them but when I end up reading one I’m not so sure they will help. The problem is that they are saying things that they know about themselves usually rather than what the reality is.
When was the last time someone wrote “I’ll only focus on the projects that make the most money and pretty much ignore the ones that don’t?”
In thinking through the way we do status at an organization I wonder if we can be taking a better advantage of these artifacts to help get them the information they really care about. And maybe help them tilt them towards reality to get what they need.
Status and summarization in today’s world
With the ability of various LLMs to summarize, I’ve been experimenting with summarization of status. The prompt matters so much when putting this together because you want to highlight the parts that are really most important.
The results are getting better but it has required a fair bit of tuning to take 10-15 pages of PM status into something “helpful.” Part of that tuning was to gear it towards the format that was expected but it missed a little bit in what was actually thought to be important by the person using the summary to send it to a leader.
There is still a bit of telephone each layer that summarizes and moves up. Sometimes this means that things are softened to the point that the high up leader doesn’t hear about something until it is too late (see “watermelon status” that is green on the outside but red on the inside).
After getting some side-channel feedback on what a leader wanted to hear, I started including that in the long-ish prompt and got better results.
Summary at the ear not at the mouth
What if we could stop summarizing way before it gets to where it needs to go?
The readme for a leader seems like an awesome thing to include in the prompt and allows the leader to actually choose what they care about over time. If they aren’t getting the right information maybe their prompt is out of date.
Axioms:
Everyone, including leaders, should get the information that is most appropriate for them.
Summaries, like meeting summaries, are contextual to the person not to the summarizer.
This reminds me of the Presidential Daily Briefing that got so famous around 9/11. They say Trump didn't read these, that the agency-heads turned the summaries into colorful pie-charts and cartoons to to grad his attention. Separating the wheat from the chaff is hard. It is always a political process, advocates of different directions pulling each way. Focusing on one thing and not another can often a strong factor that changes the environment - we aren't always just reacting to facts or digging out the subtle hints of a future coming at us from the "outside."