Protocol premise

The observation unit is one invoice-receipt difference identified before study coding; missing and conflicted evidence remain explicit outcomes.

Research question and boundary

Short-Pay Deduction Reason Code Agreement protocol note 1: This protocol will measure whether qualified reviewers assign the same primary reason code to short-pay evidence. It evaluates local administrative evidence and does not authorize posting, credit, collection, tax, legal, refund, payment, or customer decisions.

Short-Pay Deduction Reason Code Agreement protocol note 2: Register the question, systems, workflow versions, dates, population rules, exclusions, primary outcome, subgroup plan, and stop criteria before viewing results. Preserve dated amendments.

Population and sampling frame

Short-Pay Deduction Reason Code Agreement protocol note 3: The observation unit is one invoice-receipt difference identified before study coding. Build the frame from stable identifiers and retain canceled, superseded, failed, unresolved, corrected, and restricted records.

Short-Pay Deduction Reason Code Agreement protocol note 4: If a census is impractical, use a reproducible random sample with predeclared strata and a published seed. Do not replace an incomplete selected record with a cleaner example.

Data dictionary and lineage

Short-Pay Deduction Reason Code Agreement protocol note 5: Capture study key, invoice, original amount, receipt, difference, customer wording, referenced evidence, first code, second code, disagreement, and adjudication. Define allowed values, authoritative source, observation time, transformation, missing-state codes, and adjudication for every field.

Short-Pay Deduction Reason Code Agreement protocol note 6: Keep absent, inaccessible, disputed, not applicable, overwritten, and indeterminate as distinct states. Link derived values to the read-only extract and transformation log.

Reviewer reliability

Short-Pay Deduction Reason Code Agreement protocol note 7: Train reviewers on cases outside the scored sample, lock the dictionary, and independently double-code a blinded overlap set. Retain confidence and initial choices before discussion.

Short-Pay Deduction Reason Code Agreement protocol note 8: Publish raw agreement, category-specific disagreement, chance-adjusted agreement where suitable, adjudication counts, and every rubric clarification. Weak agreement is a measurement result.

Primary analysis

Short-Pay Deduction Reason Code Agreement protocol note 9: Compare customer segment, amount band, channel, evidence completeness, proposed code, and reviewer pair. Report counts, denominators, missingness, and uncertainty with rates; for elapsed time show medians, quartiles, age bands, and unresolved observations.

Short-Pay Deduction Reason Code Agreement protocol note 10: The primary outcome is primary-code raw agreement and chance-adjusted agreement with code prevalence reported. Keep evidence quality separate from operating performance because missing documentation cannot prove that an event occurred or failed.

Sensitivity checks

Short-Pay Deduction Reason Code Agreement protocol note 11: Recalculate under strict source-only evidence and the declared adjudication rule. Bound inaccessible units once as missing and once as adverse, then repeat after removing linked duplicates.

Short-Pay Deduction Reason Code Agreement protocol note 12: Preserve the registered result when alternatives look more favorable. Explain the changed assumption, number of units moved, and whether the operational recommendation reverses.

Privacy and limitations

Short-Pay Deduction Reason Code Agreement protocol note 13: The central limit is that one difference may contain pricing, quantity, tax, service, and payment-timing issues simultaneously. Also assess selection, channel-specific missingness, system clocks, retrospective corrections, workflow changes, correlated accounts, and reviewer learning.

Short-Pay Deduction Reason Code Agreement protocol note 14: Replace customer, employee, bank, and transaction identifiers with study keys. Restrict the linkage table, minimize free text, log access, and suppress sensitive small cells.

Evidence quality checks

Short-Pay Deduction Reason Code Agreement protocol note 15: Trace a stratified quality sample back to study key, invoice, original amount, receipt, difference, customer wording, referenced evidence, first code, second code, disagreement, and adjudication. Confirm that extraction preserved the source event, actor, version, time zone, and later correction; a derivative spreadsheet is not independent confirmation.

Short-Pay Deduction Reason Code Agreement protocol note 16: Record every failed link, inaccessible artifact, conflicting value, and reviewer query in an evidence-quality log. Retain the affected observation in its missing or disputed category.

Governance and replication

Short-Pay Deduction Reason Code Agreement protocol note 17: Separate data custody, coding, adjudication, and operational approval where practical. Disclose conflicts and require an authorized owner for any use beyond the registered purpose.

Short-Pay Deduction Reason Code Agreement protocol note 18: Give a second analyst the frozen extract, dictionary, and transformation log without private notes. Compare replicated counts and exceptions, explain differences, and version the final package.

Reporting and responsible use

Short-Pay Deduction Reason Code Agreement protocol note 19: Publish frame construction, observation count, exclusions, coverage, dictionary, missingness, reliability, primary estimate, groups, sensitivities, counterexamples, and limitations.

Short-Pay Deduction Reason Code Agreement protocol note 20: Use findings for a bounded process trial with an owner, duration, review date, rollback condition, and privacy check. Do not use this study alone for discipline, surveillance, or automated consequential decisions.

Sources

  1. GAO Standards for Internal Control
  2. IRS Recordkeeping
  3. FTC Debt Collection FAQs
  4. CFPB Debt Collection Rule
  5. NIST Privacy Framework
  6. CISA Recognize and Report Phishing
  7. National Archives Records Management
  8. SEC EDGAR Access
  9. Census Quarterly Financial Report
  10. Treasury Fiscal Service Debt Collection
First Review Completeness of Customer Dispute Packets

Put the protocol into operational context.

Discuss the queue, evidence, ownership, and safeguards that apply to your workflow.

Discuss your workflow