Custom AI Tools for Recruitment: Build vs Buy

TL;DR: Most recruitment agencies that say they want custom AI need custom automation built on top of an existing model's API, not a bespoke trained model. Buy off the shelf when a SaaS tool covers 80% or more of your use cases; build when the 20% it misses is your highest-volume or highest-value workflow and you have someone who can maintain prompts and API changes afterwards. A meaningful custom build is roughly £8,000-15,000 to scope, build and test, plus £300-500 a month to run, and only beats SaaS on cost somewhere around the 18 to 24 month mark. Whichever route you take, the obligations under GDPR Article 22, ICO guidance and the Equality Act 2010 stay with you as the data controller, not with the vendor.

What does 'custom AI' actually mean in a recruitment context?

Custom AI in recruitment covers three quite different things: configuring the AI features already built into Bullhorn, HubSpot or LinkedIn Recruiter, building your own automation on top of an OpenAI or Anthropic Claude API with the orchestration logic in n8n or Make, and fine-tuning a model on your own historical data. Only the second and third are genuinely custom, because only those give you control over the logic, the prompt and the data flow. Most agencies need the second one and get sold the third.

Before you can make a sensible build vs buy decision, you need to be clear on which type of custom ai tools for recruitment you're actually talking about. There are three distinct use cases, and they have very different cost profiles, complexity levels, and risk surfaces.

Use case 1 - Configuration: Turning on AI features inside an existing platform. Bullhorn Automation rules, HubSpot AI content tools, LinkedIn Recruiter's AI-assisted search filters. You're not building anything; you're adjusting settings inside a vendor's model. This is what most agencies mean when they say "we use AI." It's fine for what it is, but it's not custom - you have no control over the logic, the prompts, or what data the model has been trained on.

Use case 2 - Custom automation on top of APIs: Connecting OpenAI, Anthropic Claude, or a similar model to your existing data via an API, and building orchestration logic around it in a tool like n8n or Make. A concrete example: a workflow that pulls a new Bullhorn candidate record, sends the CV text to GPT-4o with a structured prompt, returns a scored summary back into a Bullhorn custom field, and flags it for recruiter review. This is genuinely custom - you define the logic, the prompt, and the data flow. You own the output format and can adjust it when job types or requirements change.

Use case 3 - Bespoke model training: Fine-tuning or training a model on your proprietary data - historical placements, your job description corpus, your internal scoring rubrics. Significantly higher cost and complexity. Rarely the right answer for most agencies unless you're at serious scale (hundreds of thousands of candidate records and a clearly defined ML problem to solve). Worth being clear: this is what some vendors are selling when they charge enterprise fees. Most recruitment agencies need use case 2, not use case 3.

When do off-the-shelf AI tools stop being enough?

Off-the-shelf AI tools stop being enough at four specific points: when your ATS holds custom fields the tool's screening logic ignores, when your roles are niche or regulated enough that a general model shortlists badly, when the workflow spans two systems no single vendor connects properly, and when per-seat pricing at your volume overtakes the cost of a build. Any one of those is enough to make a subscription poor value. The pricing one is worth modelling early, because it is the easiest to predict and the one agencies notice last.

Generic SaaS AI tools are built around assumptions that don't hold for a large proportion of UK recruitment agencies. The failure modes are specific and worth understanding before you commit to a subscription.

ATS schema mismatch: Generic AI screening tools are built around standard CV structures. If your Bullhorn setup has 30 custom fields capturing niche candidate attributes - security clearance levels, right-to-work categories, specific certifications for engineering or healthcare roles - off-the-shelf tools either ignore them or error. The screening logic was written for a generic sales or tech hire, not a multi-discipline staffing agency with complex candidate profiles.

Niche job type failure: A generic AI shortlister handles "Python developer, five years' experience" reasonably well. It handles "CCTV engineer with NSI Gold accreditation, working within 20 miles of a specific postcode, willing to work nights" badly. The model hasn't been trained on the specificity of trade, specialist, or regulated industry roles. You end up with AI-assisted shortlists that a recruiter discards because they're not useful.

Multi-system workflow problems: Most agencies run Bullhorn for ATS and HubSpot or Salesforce for CRM. No single SaaS AI vendor covers both cleanly. You end up with data living in two systems, no vendor willing to bridge them properly, and recruiter time spent copying information between platforms manually - which defeats the purpose entirely. A concrete version of this: an agency running perm and contract desks, with Bullhorn on the candidate side and HubSpot on the client side, trying to use a single AI screening tool that has no Bullhorn connector and no awareness of client-specific requirements stored in HubSpot deal records.

Per-seat pricing at volume: An agency running 50+ active roles simultaneously, or a high-volume temp desk processing hundreds of applicants a week, will hit a point where per-seat or per-use SaaS pricing becomes more expensive than a custom build. Worth modelling this early rather than discovering it after sign-up.

Should you build or buy AI tools for recruitment?

Buy when a SaaS tool's standard logic covers 80% or more of your use cases. Build when the 20% it cannot handle is your highest-volume or highest-value workflow, and when there is someone who can own prompt maintenance and API changes after go-live. On cost, a custom build runs roughly £11,600-21,000 over 12 months against £3,600-9,600 for SaaS, and the maths only starts to shift in the build's favour around the 18 to 24 month mark.

The honest answer is that building is often the wrong choice, and buying is often the wrong choice, depending on three variables that most agencies don't think through clearly before making the decision.

Variable 1 - Workflow-specific vs data-specific problem: If the problem is "I need AI to do X in a specific sequence across specific systems," that's a workflow problem - solvable with custom automation (use case 2 above). If the problem is "I need AI that understands my particular sector's language and norms better than a general model," that's a data problem - significantly harder, and potentially requires fine-tuning. Most agencies have workflow problems, not data problems, even if they describe them as the latter.

Variable 2 - Internal maintenance capacity: Custom builds need someone who can update the prompt library when job types change, handle API version deprecations, and debug when outputs degrade. If that person is an external consultant, budget for ongoing retainer time. If there's no one internally who can own this, a SaaS subscription is almost always lower risk.

Variable 3 - Integration complexity: How many systems does the workflow need to touch? One ATS and one AI model - manageable. ATS, CRM, job boards, calendar, compliance document store - each additional integration multiplies the maintenance surface area.

Cost of ownership, directly:

• Custom build: £8,000-15,000 to scope, build, and test a meaningful recruitment AI workflow. Add £300-500 per month in API costs and ongoing maintenance time - roughly four to six hours per month to keep prompts current and monitor output quality.

• SaaS subscription: £300-800 per month depending on seat count and tier. Zero build cost. The vendor absorbs API changes, model updates, and infrastructure.

• Over 12 months: a custom build totals roughly £11,600-21,000 (build plus running costs). SaaS totals £3,600-9,600.

• Over 24 months: custom build running costs flatten (£8,000-15,000 build, plus £7,200-12,000 running) vs SaaS doubling (£7,200-19,200). The maths starts to shift at the 18-24 month mark if the build is stable and the SaaS tool is at the higher end of pricing.

The caveat worth flagging: most custom builds I've seen fail to account for the first six months of prompt iteration. The initial build is rarely the cost that bites - it's the two days a month of refinement that nobody budgeted for.

A rough guide: buy when your problem is generic enough that a SaaS tool's standard logic covers 80% or more of your use cases. Build when the 20% it can't handle is your highest-volume or highest-value workflow.

What UK compliance obligations apply to AI in recruitment?

Three obligations apply whether you build or buy. GDPR Article 22 gives candidates the right not to be subject to decisions based solely on automated processing, so shortlisting needs genuine human review rather than a rubber stamp; ICO guidance requires you to explain that processing in plain language; and the Equality Act 2010 leaves you exposed if the tool systematically produces different outcomes for candidates with protected characteristics. None of that transfers to the vendor, because you are the data controller and the employer.

This section is almost entirely absent from competitor content on custom AI tools for recruitment. That's a problem, because the regulatory exposure is real and sits with you regardless of what tooling you use.

GDPR Article 22 - Automated decision-making: Article 22 gives individuals the right not to be subject to decisions based solely on automated processing that significantly affects them. Candidate shortlisting and scoring potentially qualifies. "AI-assisted" (human makes the final call with AI as one input) and "AI-decided" (AI output determines who advances without meaningful human review) are treated differently under the regulation. If your workflow produces a score that a recruiter rubber-stamps without genuinely reviewing, that's closer to "AI-decided" than you probably want it to be.

ICO guidance on AI and data protection: The ICO has published specific guidance requiring controllers to be able to explain their automated processing in plain language. If a candidate asks why they were rejected, "the AI scored you low" is not a compliant response. You need to be able to say what criteria the AI applied and why those criteria are relevant to the role. If you can't articulate this clearly, the system isn't ready to go live.

Equality Act 2010 - Bias auditing: If your AI tool systematically produces different shortlisting outcomes for candidates based on characteristics that correlate with protected characteristics - age, gender, ethnicity, disability - you have an indirect discrimination exposure. Generic models trained on historical hiring data often embed existing biases. The liability sits with the agency as the data controller and employer. The vendor's terms do not transfer that obligation.

The critical point: off-the-shelf tools do not make you compliant. The vendor's AI is processing your candidates under your instruction. What you need to document, regardless of whether you build or buy: the criteria the AI applies, who reviews the output, what the human review checkpoint actually checks, and how you would explain a rejection to a candidate who asks.

Where custom AI tools actually fail in recruitment

These are the failure modes that no product review covers, because they happen after implementation rather than during the sales process.

Hallucination in CV summarisation: LLMs confidently produce summaries that misrepresent what a CV actually says. A candidate with "exposure to" a technology gets summarised as "experienced in" it. A contract role gets described as a permanent placement. At low volume this gets caught in review; at high volume it slips through. The fix is structured output formats and strict prompting - but this requires someone who knows what they're doing to write and maintain the prompts.

Output degradation from inconsistent job descriptions: The quality of AI screening output is almost entirely dependent on the quality of the input job description. Agencies where 15 different recruiters write job descriptions in 15 different formats - some with salary, some without, some with detailed requirements, some with two lines - get wildly inconsistent AI outputs. This isn't a model problem; it's a process problem that the model exposes. You will not fix it by adjusting the AI.

Recruiter distrust of AI summaries: Hiring managers and senior recruiters distrust AI-generated candidate summaries unless they understand exactly how they were produced. "The AI said this person is a strong match" lands badly if no one has explained what criteria were applied. The summary gets ignored, the recruiter goes back to reading CVs manually, and the custom build becomes shelfware. The fix is transparency at the point of output - show the criteria, not just the conclusion.

The key person dependency: If the person who built and maintained the prompt library leaves, the system degrades silently. Prompts go stale, API versions change, output quality drops. No one notices until a recruiter mentions that the summaries have been "a bit off for months."

Most failures in custom AI tools for recruitment happen at the process layer, not the model layer. The model does what it's told. If what it's told is vague, inconsistent, or untested, the output reflects that.

A practical implementation approach

If you've decided a custom build is the right call, here's how to approach it without creating the problems described above.

Step 1 - Map the process before touching any tooling: Write out exactly what a recruiter does manually, in sequence, for the specific task you're automating. What data do they look at? What decision do they make? What do they produce? If you can't describe this precisely, you can't prompt an AI to do it. Start here, not with the API.

Step 2 - Choose the right API layer: OpenAI GPT-4o is the default for most use cases - widely supported, good at structured output, strong at summarisation and classification. Anthropic Claude is worth considering for longer document processing, particularly full CVs with extensive context. Avoid fine-tuning unless you have a specific, well-defined classification problem and thousands of labelled examples. For most recruitment automation, a well-crafted system prompt on a general model outperforms a poorly fine-tuned custom one.

Step 3 - Connect to ATS data via Bullhorn REST API: Bullhorn has a documented REST API. You can pull candidate records, CV text (stored in the description or parsedResume fields depending on your configuration), and custom field values. Push structured AI output back into custom text or numeric fields. The API is functional but not elegant - expect to handle pagination, authentication token refresh, and occasional rate limiting.

Step 4 - Build orchestration logic in n8n: n8n handles the workflow: trigger on new candidate application, fetch the record from Bullhorn, send to OpenAI with a structured prompt, parse the response, write back to Bullhorn, flag for human review. Self-hosted n8n gives you data residency control, which matters for UK GDPR compliance when processing candidate personal data.

Step 5 - Build the human review checkpoint: Before AI output reaches a recruiter's shortlist decision, a designated reviewer should check a sample of outputs for accuracy. Define what "accurate" means before you start - against the job description criteria, not gut feel.

For rollout: start with one job type and one recruiter. Run for two weeks with the recruiter manually reviewing every AI output against their own judgement. Measure agreement rate. Investigate every disagreement. Only expand when you have evidence it's working. Use real candidate data from closed roles (where you know the outcome) to test before going live - synthetic CVs don't surface the edge cases that matter.

How do you measure whether a custom AI tool is working?

Four numbers tell you: shortlist-to-interview conversion on AI-assisted shortlists compared with manual ones, recruiter time on the specific task you automated, the percentage of AI outputs that needed correction at the human review checkpoint, and reply and opt-out rates if the automation touches candidate outreach. Define all four and take a baseline before you build. Without that baseline you have no basis for deciding whether to keep maintaining the system or retire it.

Define these metrics before you build. If you don't, you'll have no basis for deciding whether to maintain the system or retire it six months in.

Shortlist-to-interview conversion rate: Measure the percentage of AI-assisted shortlists that convert to a client interview, compared to manually produced shortlists from the same period or the same recruiter. If this doesn't improve - or drops - the AI is adding no value at the shortlisting stage regardless of how fast it runs.

Recruiter time on the specific task: Don't measure general time-to-hire. Measure the time a recruiter spends on the specific task you automated - CV review and initial shortlisting for a defined role type. Log this before you build and track it after. If it hasn't changed, the workflow isn't working as intended, or the human review checkpoint is absorbing all the time you saved.

Error rate at the review checkpoint: Every human review of an AI output is a data point. Track the percentage of outputs that required correction. If this is above 15-20% consistently, the prompt needs work or the input data quality is the problem.

Candidate experience signals if automation touches outbound: If the AI is generating outreach messages or acknowledgements, track reply rates and opt-out rates. Degraded reply rates after implementation is a signal worth taking seriously.

"It feels useful" is not a justification for ongoing maintenance cost.

If you're at the point of deciding whether to build or buy, or you've already built something that isn't performing the way you expected, an audit is usually the right starting point - mapping what your current stack is actually doing versus what you think it's doing, before any new tooling gets added. Details are on the custom integrations page.