The core risk
Not that a partner fails to deliver. That they deliver something which runs, looks plausible, and is wrong — and nobody finds out until an engineering decision has been made on its output. Your entire evaluation should be built around preventing that.
Commissioning numerical software is different from commissioning application software, and the usual procurement instincts do not transfer. Here is what actually discriminates between partners.
Why normal vendor evaluation fails here
| Usual signal | Why it fails for solvers |
|---|---|
| Portfolio screenshots | Anyone can produce a contour plot |
| Client logos | Says nothing about numerical practice |
| Team size | This work is done by individuals, not headcount |
| Years in business | Usually years of unrelated IT work |
| "We follow best practices" | Meaningless without naming which |
The six questions that discriminate
- "How will you verify the discretisation?" You want to hear method of manufactured solutions and observed order of accuracy. This is the standard technique and a team that does not mention it has probably not used it.
- "Show me a convergence plot from previous work." Error decreasing at the expected rate under mesh refinement is the single most informative artefact in this field.
- "What validation cases would you propose for our problem?" A capable partner names specific benchmarks relevant to your physics, not generic reassurance.
- "How will you demonstrate parallel scaling?" Strong and weak scaling measured on your hardware, reported as a deliverable.
- "What are the limits of applicability?" Every solver has an envelope. A partner who cannot describe where their approach stops being valid has not thought hard enough.
- "When would you tell us to buy commercial instead?" A partner who cannot name those conditions is selling rather than advising.
Listen for specificity, not confidence. The failure mode here is not a partner who admits uncertainty — it is one who is fluently confident about everything. Numerical software has genuine unknowns, and a team that names them is telling you they have done this before.
Scoping the work properly
Ambiguity in a solver specification becomes cost and dispute later. Define:
- Governing equations and assumptions — precisely, including what is neglected.
- Mesh types the solver must handle — structured, unstructured, polyhedral.
- Boundary conditions required, including any unusual ones.
- Target hardware — workstation, cluster size, GPU, and the scaling requirement on it.
- Input and output formats — mesh import, result files, restart capability.
- Validation cases with acceptance tolerances — this is your acceptance test, agreed in advance.
Deciding what "correct" means after delivery is how solver projects turn into disputes. Agree the validation cases before development starts, or you have no contractual definition of success.
Structure that de-risks the engagement
- Paid feasibility phase — time-boxed, ending in a prototype benchmark and a written recommendation. Cheap, and it reveals capability better than any interview.
- Core solver with verification — the numerics, verified for order of accuracy before physics is layered on.
- Physics models — added incrementally, each validated as it lands.
- Parallelisation and performance — with scaling measured on your hardware.
- Handover — documentation, regression suite, and knowledge transfer to your engineers.
Milestone payments should attach to verifiable deliverables, including the validation evidence — not to elapsed time.
What must be in the contract
- Full IP assignment covering source, build scripts, test suite and documentation.
- Validation report as a deliverable, with error metrics — not a claim of correctness.
- Documented numerical methods and assumptions, so results can be defended later.
- Your repository from day one, not a delivery at the end.
- Regression suite that runs the validation cases automatically.
- Third-party licence inventory — critical if you will distribute the solver.
- Knowledge-transfer milestone before final payment.
The regression suite is the deliverable people forget to ask for. Without it you have a solver you dare not modify, because any change might silently break accuracy. With it, the code remains genuinely maintainable after the partner leaves — which is the difference between an asset and a liability.
Warning signs
- A fixed price quoted without discussing your governing equations.
- Verification described as "testing" with no methodology named.
- Claimed experience across every physics domain.
- Reluctance to name the engineers who will do the work.
- No mention of validation until you raise it.
- Unwillingness to propose a feasibility phase for genuinely uncertain work.
Evaluating partners for solver development? Send us your governing equations and target hardware — and ask us the six questions above. See our CFD solver service, the cost breakdown, and the CAx buyer's guide.