Vendor evaluation scorecard: a template with a worked example

Write one sentence that names the work, users, constraint, and decision owner. For example: “Choose a support platform for 12 agents that can import our ticket history and operate within an approved first-year budget.” This prevents a polished demo from changing the assignment halfway through evaluation. Record the current process as an alternative: keeping it may be reasonable when every proposal introduces more work than it removes.

Use the same evaluation packet for every vendor: representative tasks, sample data, expected outputs, pricing assumptions, and a deadline for clarifications. Agree what counts as proof. A feature mentioned on a pricing page, a successful trial using your test data, and a contractual commitment answer different questions. Attach a dated reference to each observation so a second reviewer can follow the reasoning.

Scotland’s Procurement Journey describes evaluation matrices as a way to score supplier responses against predefined criteria. It also emphasizes that results depend on the chosen criteria and weights, and that cost assessment should consider acquisition, operation, and end-of-life costs. The small-team worksheet here uses those general principles with its own scoring model.^[1]

A mandatory requirement is a condition whose failure makes the purchase unsuitable. Give it a pass, fail, or pending status before the weighted comparison. Examples might include importing a required file format, an approved deployment region, or fitting a fixed budget ceiling. Each requirement needs a precise check and an owner. “Enterprise ready” is too vague to be a useful gate.

Keep the gate list short enough to defend. A preference becomes a gate only when the business cannot accept an alternative. If a supplier misses a gate, exclude it or explicitly revise the requirement for everyone before reconsidering the shortlist. A high score for attractive secondary features cannot compensate for a failed mandatory condition.

Pending means the evidence is incomplete. If a vendor has not demonstrated an export, record “pending: trial export required,” the person responsible, and a response date. Do not silently convert that uncertainty into a pass or a low capability score. A vendor that demonstrated a weak export and one that has not demonstrated any export pose different follow-up questions.

Use a 1–5 scale for criteria that have enough evidence to judge. Write anchors for each criterion before viewing totals. For workflow fit, a 3 might mean completing all core tasks with documented manual workarounds; a 5 might mean completing the same tasks without those workarounds. A 4 represents a demonstrated improvement between those anchors. Missing evidence stays blank, with an open question attached.

For this worksheet, weighted points = weight × (score − 1) ÷ 4. A score of 1 receives none of that criterion’s points, 3 receives half, and 5 receives all. The maximum total is 100 when weights sum to 100. This normalization is a chosen convention, not a probability of success. Scores still depend on judgment, and equal gaps between scores are an assumption.

Separate criteria that measure different things. Workflow fit can measure task completion; implementation effort can measure migration and training. Do not award the same easy-onboarding benefit under five labels. For cost, define the user count, usage, term, support, migration, and exit assumptions. In the example, first-year cost bands are 5 at $12,000 or below, 4 above $12,000 to $15,000, 3 above $15,000 to $18,000, 2 above $18,000 to $21,000, and 1 above $21,000. These are fictional budget choices.

The following vendors, observations, and prices are invented to demonstrate the method. All three have passed the mandatory checks. Vendor A completes the test workflow most cleanly; Vendor B costs less; Vendor C offers the easiest implementation. Assumed first-year totals are $16,800 for A, $11,400 for B, and $19,200 for C, producing cost scores of 3, 5, and 2 under the published bands.

Now move 10 weight points from cost to workflow fit: the weights become 45, 15, 20, 10, and 10. Keep all observed scores unchanged. A rises to 77.50, B falls to 73.75, and C reaches 70.00. The winner changes because A’s workflow advantage matters more and B’s cost advantage matters less. The team needs to settle that tradeoff before signing.

Also test disputed scores. If B’s workflow score falls from 4 to 3 after a realistic trial, its baseline total loses 8.75 points and becomes 67.50. This identifies a valuable next test: resolve the workflow evidence before spending another week comparing minor features. Preserve the baseline and sensitivity versions so the decision record explains both.

NASA’s decision-analysis guidance stresses documenting assumptions, uncertainty, and the limits of the evaluation method when recommending among alternatives. For a vendor choice, this means keeping the unresolved questions and the conditions that could reverse the recommendation beside the final score.^[2]

Finish with a short recommendation: preferred vendor, decisive evidence, accepted compromises, unresolved conditions, and the person who can approve the commitment. If rankings are close, name the additional test that would clarify the choice. If evidence remains missing, withhold a final total rather than rescaling completed criteria to 100; rescaling makes an incomplete proposal appear more complete than it is.

Carry the accepted requirements into implementation. Assign an owner to confirm the import, workflow, support contact, and export behavior after setup. Compare the actual first invoice with the evaluation assumptions. The scorecard becomes useful again when a renewal arrives because the original expectations remain visible.

Kona can help organize approved proposals into an evidence table and draft the comparison. Ask it to preserve blanks, cite the provided document and section for every score suggestion, and show the calculation. Review the source evidence yourself before accepting a score. The downloadable template below can also be completed without AI.