The Case for Cross-Layer Literacy in Responsible AI: A Model Team's Tradeoff Is a Product Team's Constraint
Imagine a toxicity detector, a product with a specialized use case built on top of a general-use frontier model. Its intended use case is automated comment moderation on media sites. Imagine, then, that Responsible AI testing uncovers disparities in how the detector works for users from marginalized identities.
In this instance, the product team could implement controls like adding classifiers that better filter detector ratings, setting confidence thresholds for when outputs are shown to users, or routing ambiguous cases for human review. Meanwhile, the team that developed the model could implement controls like model retraining with Reinforcement Learning from Human Feedback (RLHF), adding or improving retrieval-augmented generation (RAG), or fine-tuning the model on targeted examples.
This delineation is a reasonable division of labor based on role scope and where in the product life cycle a particular team is interacting with the product. But the most effective mitigations and controls at either layer require working literacy in the other layer's toolkit: model teams that understand downstream mitigation constraints make better calibration decisions, and product teams that understand model-level levers make clearer asks and better buying decisions.
The Risk of Single-Layer Literacy
Understanding enough about the other layer to recognize when a problem extends beyond your own role, is what I call cross-layer literacy. Both model teams and teams building on top of models need a decent command of this concept. Without it, a team's understanding of how the other layer operates and what interventions are possible can be limited.
For example, a product team that sees a disparity in model behavior may immediately reach for the controls it knows: a stricter threshold, additional UX friction, or a human review step. While these may be appropriate mitigations, they can also obscure the fact that the team is compensating for a model-level failure that could have been better addressed upstream.
The reverse is also true. A model team can make a reasonable calibration decision based on aggregate performance or known evaluation results. But, behavior that looks like an acceptable tradeoff at the model layer can become a much harder constraint downstream.
Returning to the toxicity detector example, a model team may calibrate the detector to reduce false negatives, accepting a higher rate of false positives. Once embedded in a comment moderation product, however, that tradeoff may require the product team to add additional filtering or human review. This is especially true if those false positives disproportionately affect certain groups.
What Model Teams Need to See Downstream And Product Teams Need to See Upstream
Model teams can benefit from more exposure to various downstream manifestations of their model during pre-development. Concrete examples of how a model is adapted downstream can give model teams a clearer understanding of the challenges created by its current configuration and inform where tuning or calibration may be needed.
The same logic applies in the other direction and is actually twofold. Increased upstream literacy can empower product teams to have more productive conversations with model teams and make better decisions when evaluating models before buying or building on them.
Product teams often know their users, workflows, and product constraints better than anyone else in the organization. But they aren’t necessarily as conversant with model development elements.This means they may not be able to easily decipher model behaviors: which ones are configurable versus learned versus best suited for fine-tuning. In short, product teams aren’t necessarily well equipped to pinpoint the fundamental limitations of the underlying model. And without this literacy, a product team's escalation can stop at diagnosing the problem.
Returning to the toxicity detector example, imagine that the product team discovers the detector generates disproportionately high false positives (FPs) for certain groups in its moderation workflow. They try to address this by adding additional classifiers and a human review element to the workflow, but the FP rate is still high. Because they possess cross-layer fluency, the product team explains how the model’s behavior manifests in the actual moderation system and sets up a shadowing session with the model team.
As for the model team, the session is helpful for witnessing the workarounds needed to make the model applicable to this use case. They are also immediately able to connect the current model behavior to a tradeoff they made during model development. At that time they found that mitigations for reducing false positives at the model level tended to result in disproportionately high false negative rates. In light of this reality, the model team made a deliberate calibration decision to prioritize catching true toxicity over minimizing false flags. The session with the product team, then, helps the model team understand the downstream impact of that decision.
Coming out of this experience, the model team uses their newly gained downstream context to target mitigations they couldn’t have scoped based on evaluation data alone. As a result, the team’s targeted model fine-tuning and additional training examples cut the FP rate by one third. Meanwhile, the product team, now aware of the calibration fine line the model team navigated, builds on the improved model with lighter touch controls. They create simplified downstream filtering and reserve human review for select contexts.This reduces the FP rate by an additional one third.
As the toxicity detector example highlights, the value here runs in both directions. Exposure to how product teams adapt and supplement models across different use cases is vital: it can reveal patterns that are difficult to identify from the model layer alone. And those patterns, made clear earlier, can create a feedback loop that informs model development. This feedback loop then gives model teams a stronger basis for deciding where tuning and calibration could reduce downstream challenges.
Upstream literacy can also matter before a model is selected. Model selection is a decision that usually revolves around issues like latency, cost, or benchmark performance. But it should also be, in part, a decision about the mitigation options available once the model is embedded in a product. Understanding which model behaviors are configurable, learned, or otherwise difficult to change can therefore be important before a team commits to a model. For example, imagine a team is picking between two models with similar capabilities. If one model allows the team to more easily configure behaviors that matter to its use case and the other does not, they offer very different practical value propositions.
Making Cross-Layer Literacy Operational
How can we operationalize cross-layer literacy? What this looks like in practice depends on the relationship between the team building the model and the team deploying it.
If an organization owns both the model and the products built on top of it, the goal should be to create formal relationships, escalation paths, and feedback loops across the two layers. These processes should make it easier for: 1. Information about deployment context to flow upstream, and 2. Information about model behavior, known limitations, existing mitigations, and what has already been evaluated to flow downstream.
While the underlying need is the same when a team deploys a third-party model, the mechanisms for attaining cross-layer literacy are different. In this context, product teams have less access to the upstream model team. And so, some of the cross-layer literacy that might otherwise develop through ongoing internal collaboration needs to be established before the model is selected. This reality makes procurement an important part of operationalizing cross-layer literacy.
Before taking in an outside model, product teams need to ask questions that go beyond general model safety and give insight into the bounds of their specific use case. Questions such as:
Which model behaviors can we configure for our use case?
What guardrails already exist, and in what contexts were they evaluated?
What limitations or known disparities should we expect to inherit?
What is the process for escalating behavior that we cannot mitigate at the product layer?,
will help teams understand a model’s core functionality. They also help clarify what teams will be able to change, what they will have to engineer around, and where responsibility will sit when problems arise.
Once a model has been selected, the need for cross-layer communication doesn’t disappear. Teams still need escalation paths and feedback loops. Information about product behavior needs to flow up to the model provider and information about model behavior and known limitations needs to flow downstream.
Conclusions: Towards Cross-Layer Literacy and RAI Expertise
Model deployment amplifies the downstream impact of model team decisions. It’s at that point that the model’s constraints, tradeoffs, and mitigations become the inheritance of every product built on top of it. Likewise, a product team's mitigations don’t exist in isolation: The controls they build are inextricably tied to the foundational model’s features.
Cross-layer literacy gives RAI teams a way to account for this dynamic. It provides a way for teams to understand how both the interventions available at their layer and the decisions made at another layer shape their work in practice.
As AI systems continue to become more complex, this kind of understanding should become part of how we think about RAI’s remit. The goal is not for every team to become expert in every layer. Instead, it’s for teams to be sufficiently fluent across those boundaries and make decisions with the entirety of the model-product ecosystem in mind.
