Evidence tiers and thin product evidence

The relationship between capability and confidence

The current records attach a rationale, evidence links, capture date and confidence to each capability assessment. These fields answer different questions. The rationale explains the behaviour being assessed; the evidence points to the supporting material; the capture date locates the assessment; and confidence indicates how firmly the record treats the claim. None should be replaced by a headline position. [newcore-c3-1]

The current dataset describes documented claims rather than hands-on tests. The distinction is important when a source gives a clear architectural description but leaves deployment details unresolved. A reader can recognise the relevance of the documented control while still requiring a demonstration of its operation in the intended environment. The evidence flag is not a statement that the capability is worthless. [newcore-c7-1]

The evidence tiers as reading aids

The useful meaning of an evidence tier is the strength and specificity of support for the exact claim being made. A general product page, detailed implementation documentation and an observed workflow do not answer identical questions. This article does not assign a new tier system to the index. The existing confidence labels and source records remain the reference points for the published assessments. [aembit-c8-1]

The buyer should read a claim at the narrowest level supported by its source. A page documenting access to enrolled servers can substantiate that capability without substantiating enterprise-wide discovery. A page describing a pre-approved access profile can support provisioning controls without proving that a human reviews a live tool action. These boundaries are visible in the current records. [teleport-c2-1] [clutch-security-c7-1]

• claim: identify the precise behaviour being asserted
• source: open the material supporting that behaviour
• scope: retain integration and deployment boundaries
• confidence: distinguish documented support from verification gaps
• validation: write the unresolved question for a demonstration

The difference between missing and negative evidence

The phrase not verified means that the cited material did not establish the capability for this assessment. It should not be expanded into a universal statement that a vendor cannot provide it. The current Aembit record, for example, documents gateway and delegated credential behaviour but does not verify a human approval workflow in the cited documentation. The positive and unresolved findings can both remain true. [aembit-c4-1] [aembit-c7-1]

The same care applies to discovery. Keycard’s record describes registered application identities and does not verify broad discovery of unknown agents. This is a meaningful boundary for an inventory project, but it does not erase the documented delegated access and step-up approval capabilities. The decision should follow the buyer’s requirement rather than turning every missing evidence item into a general product verdict. [keycard-c2-1] [keycard-c5-1] [keycard-c7-1]

The specificity of evidence before breadth

SailPoint Entro provides a useful example of a narrow documented control. The registry describes a secret scanner that can block exposure in a particular coding workflow. It explicitly warns against generalising that blocking to all agents or all tool deployments. The useful assessment retains the verified scope instead of relabelling a specific scanner as a general authorization gateway. [sailpoint-c3-1] [sailpoint-c8-1]

Teleport documents enrolled resources, server access controls and per-tool audit events, with agent identity entitlements differing between tiers. That evidence can be useful for an enrolled infrastructure requirement. The record does not verify broad shadow-agent discovery, and trusted runtimes are described as beta. An evaluation should preserve both the documented functionality and the entitlement or availability boundary. [teleport-c1-1] [teleport-c2-1] [teleport-c6-1]

The claims needing separate validation

NewCore says its split-key approach divides token-signing authority between the customer and NewCore so that neither side can sign alone. The source is a vendor announcement, not a cryptographic test by this site. The statement can be reported with that attribution while leaving protocol-level key custody and operational behaviour for further examination. [n2] [newcore-c4-1]

NewCore also describes policy checks before tool execution and human approval for sensitive actions. It operates alongside the existing identity provider, and pricing is not public. The product’s direct command-line tool calls currently bypass the gateway. A relevant documented capability and an important coverage limitation belong in the same decision record, because the buyer needs both to assess the proposed deployment. [newcore-c3-1] [newcore-c7-1] [newcore-c8-1]

A practical evidence register

The shortlist should carry unresolved questions forward rather than hiding them behind a single result. The buyer can record the required action, supporting source, covered integration, remaining question and proposed validation. This makes a thin-evidence flag actionable: the next step becomes a request for specific proof rather than a debate about whether the overall product description sounds credible. [newcore-c8-1]

The same register should distinguish product claims from recognition announcements. NewCore announced awards in its cited releases, but the current data explicitly says awards do not contribute points. The recognition therefore does not resolve an implementation question about revocation, retention or key custody. The evidence for the required behaviour must still address that behaviour directly. [n2] [n3]

What readers ask