Native PHP 8.3: Add Website Tech Summaries to CRM Leads with AI Detector

Mihajlo
Aug 15, 2026
Tutorials

A lead’s website often reveals more than a sparse contact form does. A concise note such as “WordPress 6.5, WooCommerce, Cloudflare; high-confidence evidence; one redirect” gives an agency immediate context for discovery calls, estimates, and technical audits.

This tutorial builds that feature for a small CRM using PHP 8.3, native cURL, SQLite, and a command suitable for cron. The integration validates the API boundary, retries only appropriate failures, records structured states, and remains testable without making network requests.

Get access to the detector

First, register an account, or sign in if you already have one.

Open the Website Technology Detector service page, choose an available Free, Plus, or Pro plan, and complete activation. Then open the official service documentation. Find the Service token panel and copy its service-scoped token.

This service requires authentication. It accepts a Bearer token, an X-API-Token header, or a token query parameter. We will use the Bearer header because query parameters can leak into access logs and browser history.

Regenerating the service token revokes the previously active token. Treat rotation as a deployment operation: install the replacement everywhere the worker runs, verify it, and then regenerate or replace the old credential according to your rollout plan.

Confirm the exact endpoint

The integration uses POST https://ai.mihajlo.mk/api/website-technology-detector/v1/detect-technologies. Its JSON request contains one required value, url. Test it without placing the token in shell history:

read -rsp "Service token: " WEBSITE_DETECTOR_TOKEN
curl --fail-with-body \
  --connect-timeout 3 \
  --max-time 15 \
  -X POST \
  -H "Authorization: Bearer ${WEBSITE_DETECTOR_TOKEN}" \
  -H "Content-Type: application/json" \
  --data '{"url":"https://example.com"}' \
  https://ai.mihajlo.mk/api/website-technology-detector/v1/detect-technologies
unset WEBSITE_DETECTOR_TOKEN

A successful response contains confidence-scored detections, versions, evidence, and redirect information. Preserve the response from this test only long enough to compare it with the current documentation; public sites can still expose operational details you may not want in logs.

Now store the credential in local environment configuration. Never commit this file:

# .env
WEBSITE_DETECTOR_TOKEN=YOUR_SERVICE_TOKEN
CRM_DSN=sqlite:/srv/agency-crm/var/crm.sqlite

# .gitignore
.env
.phpunit.cache/
vendor/

Architecture for a small CRM

The design is deliberately modest. An existing leads table supplies a lead ID and public website URL. A command selects pending work, calls the detector, maps the response into domain objects, writes a readable summary, and records whether a failure is retryable.

API adapter: owns authentication, timeouts, JSON decoding, and retry policy.
Domain mapper: validates detections, versions, evidence, confidence, and redirects before CRM code sees them.
Batch command: enriches a bounded number of leads and can run under cron.
Database state: makes successes idempotent and distinguishes temporary failures from permanent ones.

Background execution is preferable to enriching inside a form submission. Detection requires an external request, so making sales staff wait would couple CRM responsiveness to network latency. A queue would be justified at larger scale, but a locked, bounded cron command is easier to operate for a small agency.

Create the PHP 8.3 project

You need PHP 8.3 or newer, Composer, native cURL, JSON, PDO, and PDO SQLite. The only runtime package loads local environment files; production can inject the same variables directly.

{
  "require": {
    "php": "^8.3",
    "ext-curl": "*",
    "ext-json": "*",
    "ext-pdo": "*",
    "ext-pdo_sqlite": "*",
    "vlucas/phpdotenv": "^5.6"
  },
  "require-dev": {
    "phpunit/phpunit": "^11.0"
  },
  "autoload": {
    "psr-4": {
      "AgencyCrm\\": "src/"
    }
  },
  "autoload-dev": {
    "psr-4": {
      "AgencyCrm\\Tests\\": "tests/"
    }
  }
}

composer install
composer dump-autoload
mkdir -p src/Detector src/Http bin database tests var

The CRM is assumed to have leads(id, website_url). Apply this migration once through your normal migration process:

ALTER TABLE leads ADD COLUMN technology_summary TEXT;
ALTER TABLE leads ADD COLUMN technology_status TEXT;
ALTER TABLE leads ADD COLUMN technology_error TEXT;
ALTER TABLE leads ADD COLUMN technology_checked_at TEXT;

CREATE INDEX leads_technology_status_idx
ON leads (technology_status);

Build the bounded cURL transport

The transport fixes the endpoint in application code, uses short connection and total timeouts, and returns only the data the client needs. Keeping arbitrary endpoints out of environment configuration prevents a compromised configuration value from redirecting the service token elsewhere.

<?php
// src/Http/HttpResponse.php
declare(strict_types=1);

namespace AgencyCrm\Http;

final readonly class HttpResponse
{
    public function __construct(
        public int $status,
        public string $body,
        public array $headers
    ) {}
}

<?php
// src/Http/CurlTransport.php
declare(strict_types=1);

namespace AgencyCrm\Http;

use RuntimeException;

final class CurlTransport
{
    public function __invoke(
        string $url,
        array $headers,
        string $json
    ): HttpResponse {
        $handle = curl_init($url);

        if ($handle === false) {
            throw new RuntimeException('Could not initialize cURL');
        }

        $responseHeaders = [];
        $headerLines = [];

        foreach ($headers as $name => $value) {
            $headerLines[] = $name . ': ' . $value;
        }

        curl_setopt_array($handle, [
            CURLOPT_POST => true,
            CURLOPT_RETURNTRANSFER => true,
            CURLOPT_CONNECTTIMEOUT_MS => 3000,
            CURLOPT_TIMEOUT_MS => 12000,
            CURLOPT_HTTPHEADER => $headerLines,
            CURLOPT_POSTFIELDS => $json,
            CURLOPT_HEADERFUNCTION => static function (
                $handle,
                string $line
            ) use (&$responseHeaders): int {
                $length = strlen($line);
                $parts = explode(':', $line, 2);

                if (count($parts) === 2) {
                    $responseHeaders[strtolower(trim($parts[0]))]
                        = trim($parts[1]);
                }

                return $length;
            },
        ]);

        $body = curl_exec($handle);

        if ($body === false) {
            $message = curl_error($handle);
            curl_close($handle);
            throw new RuntimeException('Detector transport failed: ' . $message);
        }

        $status = (int) curl_getinfo($handle, CURLINFO_RESPONSE_CODE);
        curl_close($handle);

        return new HttpResponse($status, $body, $responseHeaders);
    }
}

Map the service response at one boundary

The API response must never be trusted merely because JSON decoding succeeded. The mapper below requires a list of detections, skips malformed entries, validates scalar fields, retains evidence and redirect information, and ignores unfamiliar additions. If the official documentation changes its response contract, this is the one class to update.

<?php
// src/Detector/TechnologyReport.php
declare(strict_types=1);

namespace AgencyCrm\Detector;

use UnexpectedValueException;

final readonly class Detection
{
    public function __construct(
        public string $name,
        public ?float $confidence,
        public array $versions,
        public array $evidence
    ) {}
}

final readonly class TechnologyReport
{
    public function __construct(
        public array $detections,
        public array $redirects
    ) {}

    public static function fromPayload(array $payload): self
    {
        $items = $payload['detections'] ?? null;

        if (!is_array($items) || !array_is_list($items)) {
            throw new UnexpectedValueException(
                'Detector response has no valid detections list'
            );
        }

        $detections = [];

        foreach ($items as $item) {
            if (!is_array($item)) {
                continue;
            }

            $name = $item['name'] ?? null;

            if (!is_string($name) || trim($name) === '') {
                continue;
            }

            $confidence = $item['confidence'] ?? null;
            $confidence = is_int($confidence) || is_float($confidence)
                ? (float) $confidence
                : null;

            $versions = is_array($item['versions'] ?? null)
                ? array_values(array_filter(
                    $item['versions'],
                    static fn (mixed $value): bool =>
                        is_string($value) && $value !== ''
                ))
                : [];

            $evidence = is_array($item['evidence'] ?? null)
                ? $item['evidence']
                : [];

            $detections[] = new Detection(
                trim($name),
                $confidence,
                $versions,
                $evidence
            );
        }

        $redirects = is_array($payload['redirects'] ?? null)
            ? $payload['redirects']
            : [];

        return new self($detections, $redirects);
    }

    public function summary(): string
    {
        if ($this->detections === []) {
            return 'No technologies detected';
        }

        $parts = array_map(
            static function (Detection $detection): string {
                $text = $detection->name;

                if ($detection->versions !== []) {
                    $text .= ' ' . implode(', ', $detection->versions);
                }

                $details = [];

                if ($detection->confidence !== null) {
                    $details[] = 'confidence '
                        . rtrim(rtrim(
                            number_format($detection->confidence, 2, '.', ''),
                            '0'
                        ), '.');
                }

                $details[] = count($detection->evidence)
                    . ' evidence item(s)';

                return $text . ' (' . implode('; ', $details) . ')';
            },
            $this->detections
        );

        if ($this->redirects !== []) {
            $parts[] = count($this->redirects) . ' redirect(s) observed';
        }

        return implode('; ', $parts);
    }
}

Confidence is intentionally rendered without assuming that its scale is a percentage. Versions and evidence are optional at the local boundary because malformed partial data should not crash an entire batch.

Add retries without creating a retry storm

Detection is read-only, so a small retry budget is reasonable for transport errors, rate limiting, and selected server failures. Validation and authentication failures are not retried. A Retry-After value is honored only when it is an integer and is capped at five seconds.

<?php
// src/Detector/WebsiteTechnologyClient.php
declare(strict_types=1);

namespace AgencyCrm\Detector;

use AgencyCrm\Http\HttpResponse;
use Closure;
use JsonException;
use RuntimeException;
use Throwable;

final class DetectorException extends RuntimeException
{
    public function __construct(
        string $message,
        public readonly bool $retryable,
        public readonly ?int $httpStatus = null
    ) {
        parent::__construct($message);
    }
}

final class WebsiteTechnologyClient
{
    private const ENDPOINT =
        'https://ai.mihajlo.mk/api/website-technology-detector/v1/detect-technologies';

    public function __construct(
        private readonly string $token,
        private readonly Closure $transport,
        private readonly Closure $sleep
    ) {}

    public function detect(string $url): TechnologyReport
    {
        if (!filter_var($url, FILTER_VALIDATE_URL)) {
            throw new DetectorException('Lead URL is invalid', false);
        }

        $scheme = strtolower((string) parse_url($url, PHP_URL_SCHEME));

        if (!in_array($scheme, ['http', 'https'], true)) {
            throw new DetectorException('Lead URL must use HTTP or HTTPS', false);
        }

        try {
            $json = json_encode(['url' => $url], JSON_THROW_ON_ERROR);
        } catch (JsonException $exception) {
            throw new DetectorException('Could not encode request', false);
        }

        for ($attempt = 1; $attempt <= 3; $attempt++) {
            try {
                $response = ($this->transport)(
                    self::ENDPOINT,
                    [
                        'Authorization' => 'Bearer ' . $this->token,
                        'Content-Type' => 'application/json',
                        'Accept' => 'application/json',
                    ],
                    $json
                );
            } catch (Throwable $exception) {
                if ($attempt === 3) {
                    throw new DetectorException(
                        'Detector transport unavailable',
                        true
                    );
                }

                ($this->sleep)(250 * (2 ** ($attempt - 1)));
                continue;
            }

            if ($response->status >= 200 && $response->status < 300) {
                return $this->mapResponse($response);
            }

            $retryable = $response->status === 429
                || in_array($response->status, [500, 502, 503, 504], true);

            if (!$retryable || $attempt === 3) {
                throw new DetectorException(
                    'Detector returned HTTP ' . $response->status,
                    $retryable,
                    $response->status
                );
            }

            $retryAfter = $response->headers['retry-after'] ?? null;
            $delay = is_string($retryAfter) && ctype_digit($retryAfter)
                ? min(5000, max(250, (int) $retryAfter * 1000))
                : 250 * (2 ** ($attempt - 1));

            ($this->sleep)($delay);
        }

        throw new DetectorException('Retry loop exhausted', true);
    }

    private function mapResponse(HttpResponse $response): TechnologyReport
    {
        try {
            $payload = json_decode(
                $response->body,
                true,
                512,
                JSON_THROW_ON_ERROR
            );
        } catch (JsonException $exception) {
            throw new DetectorException(
                'Detector returned invalid JSON',
                false,
                $response->status
            );
        }

        if (!is_array($payload)) {
            throw new DetectorException(
                'Detector returned an invalid document',
                false,
                $response->status
            );
        }

        try {
            return TechnologyReport::fromPayload($payload);
        } catch (Throwable $exception) {
            throw new DetectorException(
                'Detector response did not match its contract',
                false,
                $response->status
            );
        }
    }
}

Enrich pending CRM leads

The command limits each run, updates one lead at a time, and logs identifiers and states instead of credentials, response bodies, evidence, or complete URLs. Permanent failures remain visible for correction; temporary failures can be selected again.

<?php
// bin/enrich-leads.php
declare(strict_types=1);

use AgencyCrm\Detector\DetectorException;
use AgencyCrm\Detector\WebsiteTechnologyClient;
use AgencyCrm\Http\CurlTransport;
use Dotenv\Dotenv;

require dirname(__DIR__) . '/vendor/autoload.php';

Dotenv::createImmutable(dirname(__DIR__))->safeLoad();

$token = $_ENV['WEBSITE_DETECTOR_TOKEN']
    ?? getenv('WEBSITE_DETECTOR_TOKEN')
    ?: null;
$dsn = $_ENV['CRM_DSN'] ?? getenv('CRM_DSN') ?: null;

if (!is_string($token) || $token === '' || !is_string($dsn) || $dsn === '') {
    fwrite(STDERR, "Missing WEBSITE_DETECTOR_TOKEN or CRM_DSN\n");
    exit(1);
}

$transport = new CurlTransport();
$sleep = static fn (int $milliseconds): int =>
    usleep($milliseconds * 1000);

$client = new WebsiteTechnologyClient(
    $token,
    Closure::fromCallable($transport),
    $sleep(...)
);

$pdo = new PDO($dsn, null, null, [
    PDO::ATTR_ERRMODE => PDO::ERRMODE_EXCEPTION,
    PDO::ATTR_DEFAULT_FETCH_MODE => PDO::FETCH_ASSOC,
]);

$select = $pdo->prepare(
    "SELECT id, website_url
     FROM leads
     WHERE website_url IS NOT NULL
       AND (
           technology_status IS NULL
           OR technology_status IN ('pending', 'retryable')
       )
     ORDER BY id
     LIMIT :batch_size"
);
$select->bindValue(':batch_size', 25, PDO::PARAM_INT);
$select->execute();

$update = $pdo->prepare(
    "UPDATE leads
     SET technology_summary = :summary,
         technology_status = :status,
         technology_error = :error,
         technology_checked_at = :checked_at
     WHERE id = :id"
);

foreach ($select as $lead) {
    try {
        $report = $client->detect((string) $lead['website_url']);

        $update->execute([
            'summary' => $report->summary(),
            'status' => 'complete',
            'error' => null,
            'checked_at' => gmdate('c'),
            'id' => $lead['id'],
        ]);

        error_log(json_encode([
            'event' => 'technology_detection_complete',
            'lead_id' => $lead['id'],
            'detection_count' => count($report->detections),
        ], JSON_THROW_ON_ERROR));
    } catch (DetectorException $exception) {
        $status = $exception->retryable ? 'retryable' : 'failed';

        $update->execute([
            'summary' => null,
            'status' => $status,
            'error' => $exception->getMessage(),
            'checked_at' => gmdate('c'),
            'id' => $lead['id'],
        ]);

        error_log(json_encode([
            'event' => 'technology_detection_failed',
            'lead_id' => $lead['id'],
            'status' => $status,
            'http_status' => $exception->httpStatus,
        ], JSON_THROW_ON_ERROR));
    }
}

Test without calling the external service

A deterministic fake transport lets the test exercise mapping, retry timing, and authentication behavior without consuming quota or depending on the network.

<?php
// tests/WebsiteTechnologyClientTest.php
declare(strict_types=1);

namespace AgencyCrm\Tests;

use AgencyCrm\Detector\DetectorException;
use AgencyCrm\Detector\WebsiteTechnologyClient;
use AgencyCrm\Http\HttpResponse;
use PHPUnit\Framework\TestCase;

final class WebsiteTechnologyClientTest extends TestCase
{
    public function testItBuildsReadableSummary(): void
    {
        $transport = static fn (): HttpResponse => new HttpResponse(
            200,
            json_encode([
                'detections' => [[
                    'name' => 'Example CMS',
                    'confidence' => 92,
                    'versions' => ['6.5'],
                    'evidence' => ['generator metadata', 'asset path'],
                ]],
                'redirects' => [['from' => 'http', 'to' => 'https']],
            ], JSON_THROW_ON_ERROR),
            []
        );

        $client = new WebsiteTechnologyClient(
            'test-token',
            $transport(...),
            static fn (int $milliseconds): null => null
        );

        $summary = $client->detect('https://example.com')->summary();

        self::assertSame(
            'Example CMS 6.5 (confidence 92; 2 evidence item(s)); '
            . '1 redirect(s) observed',
            $summary
        );
    }

    public function testItRetriesRateLimitOnce(): void
    {
        $responses = [
            new HttpResponse(429, '{}', ['retry-after' => '1']),
            new HttpResponse(200, '{"detections":[],"redirects":[]}', []),
        ];
        $delays = [];

        $transport = static function () use (&$responses): HttpResponse {
            return array_shift($responses);
        };

        $sleep = static function (int $milliseconds) use (&$delays): void {
            $delays[] = $milliseconds;
        };

        $client = new WebsiteTechnologyClient(
            'test-token',
            $transport(...),
            $sleep(...)
        );

        $client->detect('https://example.com');

        self::assertSame([1000], $delays);
    }

    public function testItDoesNotRetryAuthenticationFailure(): void
    {
        $calls = 0;

        $transport = static function () use (&$calls): HttpResponse {
            $calls++;
            return new HttpResponse(401, '{}', []);
        };

        $client = new WebsiteTechnologyClient(
            'test-token',
            $transport(...),
            static fn (int $milliseconds): null => null
        );

        try {
            $client->detect('https://example.com');
            self::fail('Expected DetectorException');
        } catch (DetectorException $exception) {
            self::assertFalse($exception->retryable);
            self::assertSame(401, $exception->httpStatus);
            self::assertSame(1, $calls);
        }
    }
}

vendor/bin/phpunit --testdox tests

Security, observability, and deployment

Only submit public websites that your CRM is authorized to process. Validate URLs again when they enter the CRM, restrict schemes to HTTP and HTTPS, and do not expose this command as a user-controlled proxy. Keep evidence and raw responses out of general application logs.

Grant the worker only the database permissions it needs. Protect .env with restrictive filesystem permissions locally; in production, prefer the deployment platform’s secret store or injected environment variables. Token values must never appear in fixtures, exception messages, screenshots, or monitoring labels.

Track counts for completed, retryable, and permanent failures, plus request duration and HTTP status class. Alert on sustained authentication failures, repeated rate limiting, or a sudden rise in contract-mapping errors. These signals distinguish an expired token from quota pressure or an upstream response change.

Deploy dependencies with composer install --no-dev --classmap-authoritative, apply the migration, inject the token and DSN, and run one lead manually. Then schedule the worker with an operating-system lock so overlapping cron invocations cannot process the same batch:

flock -n /var/lock/agency-crm-enrichment.lock \
  php /srv/agency-crm/bin/enrich-leads.php

Common failures worth designing for

HTTP 401 or 403: confirm plan activation and the service-scoped token. If it was regenerated, the previous token is revoked. Do not retry automatically.
HTTP 400 or 422: inspect the stored lead URL and current documentation. Correct the input instead of retrying it.
HTTP 429: honor bounded backoff, reduce batch frequency, and check the active plan or quota.
Timeouts or selected 5xx responses: allow the three-attempt budget, then retain retryable for a later scheduled run.
Contract-mapping errors: compare a sanitized response with the official documentation and update only the boundary mapper.
Empty detections: treat this as a valid result, not an operational failure. A public page may provide insufficient deterministic evidence.

Final verification checklist

The account and Free, Plus, or Pro service plan are active.
The current service token is stored only in environment-backed configuration.
The minimal POST request succeeds with a JSON body containing url.
PHPUnit passes without making a real network request.
A pending lead becomes complete with a readable technology summary.
A 401 becomes failed after one attempt, while a temporary outage becomes retryable.
Logs contain lead IDs, outcome states, and status codes, but no token, raw evidence, response body, or full URL.
The production command has bounded batches, timeouts, retries, and an overlap lock.

The useful part of this feature is not merely that it detects a CMS or CDN. It turns external technical evidence into dependable CRM context: concise enough for a lead record, structured enough to operate safely, and isolated enough to evolve when the service contract does.