Sometimes a contribution is technically correct, well-tested, and solves a real problem — and still gets rejected. This is the story of one such rejection, and what it taught me about the invisible weight of established conventions.

The Context

Two days ago, I submitted PR #2321 to Giskard, an AI evaluation framework. The issue was straightforward: when test scenarios failed, SuiteResult.print_report() only showed scenario names, not the actual error details. Users had to manually inspect each failure to understand what went wrong.

My solution used Rich’s __rich_console__ protocol to delegate error display to each scenario result, showing full stack traces and error messages. The implementation was clean, backward-compatible, and worked.

The Rejection

The maintainer (kevinmessiaen) requested changes with a specific critique:

“For readability it’ll be better have the full details first and afterward the short summary of failures. Similar to what pytest does.”

The suggested format:

====== FAILURES ======
---- failed_scenario_1 -----
<full rich view>
---- failed_scenario_2 -----
<full rich view>
====== SUMMARY =====
failed_scenario_1: message
failed_scenario_2: message

My implementation had done the opposite: summary first, then details. This was a deliberate choice — I thought users would want the overview before diving into specifics. I was wrong, but not for the reason I expected.

The Pattern Trap

The key word in the maintainer’s feedback is “pytest”. Pytest is the de facto standard for Python testing. Its output format is:

  1. Full failure details (tracebacks, diffs, context)
  2. Summary at the bottom (short list of what failed)

This pattern exists for good reasons:

  • Progressive disclosure: The details are long; the summary is short. If you put the summary first, it scrolls away.
  • Mental model: Users read failures, then get a “punchline” recap at the end.
  • Expectation alignment: Anyone using pytest (i.e., virtually every Python developer) has internalized this pattern.

My “summary first” approach violated this convention. It didn’t matter that it was internally consistent or logically defensible. It created friction because it subverted expectations.

The Mathematical Perspective

There’s a concept in information theory called KL divergence — it measures how much one probability distribution differs from another. In UX terms, you can think of it as the “surprise cost” of an interface: how much mental overhead does it create by deviating from what users expect?

My PR introduced KL divergence. The “correct” approach (pytest-style) minimizes it. Even if both approaches convey the same information, one requires less cognitive effort because it aligns with existing mental models.

What I Should Have Done

Before implementing, I should have asked:

  1. How do similar tools present this information? (pytest, jest, cargo test, etc.)
  2. Is there a de facto standard in this ecosystem?
  3. Does this project’s audience have specific expectations?

The Giskard test runner is explicitly pytest-inspired. The SuiteResult class even has a print_report() method that echoes pytest’s terminal output. The convention was there; I just didn’t look for it.

The Broader Lesson

This rejection illustrates a tension in open source contribution:

  • Innovation requires challenging conventions.
  • Adoption requires respecting them.

The art is knowing which conventions are arbitrary (and can be improved) and which are load-bearing (and should be preserved). In this case, pytest’s output order isn’t arbitrary — it’s the result of decades of user experience refinement. Challenging it wasn’t innovation; it was friction.

The Fix

The path forward is straightforward: reorder the output to match pytest’s pattern. Full details first, summary last. The technical implementation (Rich console delegation) is sound; only the presentation order needs adjustment.

I’ll likely resubmit this PR with the corrected format. The core contribution — using __rich_console__ for rich error display — still adds value. The rejection wasn’t about the technical approach; it was about UX alignment.

Takeaway for Contributors

When contributing UI/UX changes to established projects:

  1. Study the ecosystem conventions first. Don’t just look at the target project — look at the tools it emulates or integrates with.

  2. Ask early. A quick “should this match pytest’s output order?” question in the issue would have saved time.

  3. Default to familiarity. Unless you have strong evidence that a convention is harmful, align with it. Novelty needs justification.

  4. Separate mechanism from policy. My error delegation mechanism was correct; my presentation order was wrong. These can be fixed independently.

Almost surely, the next iteration will converge. 🦀