A lead’s website often reveals more than a sparse contact form does. A technology summary can tell a small agency whether a prospect runs WordPress, uses a particular analytics platform, or has an application stack that matches the team’s expertise. The useful version of this feature is not a browser extension or a manual lookup: it is enrichment that appears directly beside each lead.

This tutorial builds that workflow in Native PHP 8.3. A command reads unenriched leads from an existing SQLite-backed CRM, calls the Website Technology Detector API, converts its confidence-scored results into a concise summary, and records structured success or failure states. The API boundary uses native cURL, bounded timeouts, selective retries, defensive JSON mapping, and a transport abstraction that makes PHPUnit tests deterministic.

Get access and create a service token

1. Open the registration page and create an account, or use the sign-in page if you already have one.
2. Visit the Website Technology Detector service page. Choose an available Free, Plus, or Pro plan and complete its activation.
3. Open the official service documentation, locate the Service token panel, and copy its service-scoped token.
4. Store that value in the project’s environment configuration. Never commit it to source control.

Regenerating the service token revokes the previously active token. Treat rotation as a deployment change: update every environment that uses the old value, restart the relevant PHP workers or command processes, and verify authentication before removing your rollback path.

This service requires authentication. It accepts a Bearer token, an X-API-Token header, or a token query parameter. The implementation below uses the Bearer form because it keeps the credential out of URLs, access logs, browser history, and proxy analytics.

Verify the exact endpoint first

The detector call is POST https://ai.mihajlo.mk/api/website-technology-detector/v1/detect-technologies. Its JSON request body contains url. Before writing application code, make one minimal request from a trusted terminal:

export WEBSITE_DETECTOR_TOKEN='YOUR_SERVICE_TOKEN'

curl --fail-with-body \
  --connect-timeout 3 \
  --max-time 20 \
  -X POST \
  'https://ai.mihajlo.mk/api/website-technology-detector/v1/detect-technologies' \
  -H "Authorization: Bearer ${WEBSITE_DETECTOR_TOKEN}" \
  -H 'Accept: application/json' \
  -H 'Content-Type: application/json' \
  --data '{"url":"https://example.com"}'

For local development, create an untracked .env file and a committed placeholder file named .env.example:

# .env.example
WEBSITE_DETECTOR_TOKEN=YOUR_SERVICE_TOKEN
CRM_DATABASE_PATH=var/crm.sqlite

cp .env.example .env
# Replace only the placeholder in .env, then load it into the process:
set -a
. ./.env
set +a

Add .env and var/ to .gitignore. Native PHP does not automatically read dotenv files, so the shell explicitly exports the file before running the command. In production, inject the same variables through the process manager or secret store instead of sourcing a repository file.

Architecture that fits a small CRM

Lead enrichment is external I/O, so it should not delay the screen where a user creates a lead. A scheduled command is enough for a small CRM and avoids introducing a queue system solely for one feature. It selects a bounded batch, validates each website, invokes the detector, and writes a readable summary.

The API client owns HTTP behavior and authentication. A mapper owns the untrusted response boundary. The command owns CRM policy: which leads qualify, what happens after a temporary failure, and whether an authentication error should stop the batch.

bin/enrich-leads
src/Detector/CurlTransport.php
src/Detector/DetectorClient.php
src/Detector/DetectionReport.php
src/Detector/HttpResponse.php
src/Detector/HttpTransport.php
src/Detector/ServiceFailure.php
tests/DetectorClientTest.php
composer.json
.env.example

Install only the test dependency; production uses PHP’s cURL and PDO extensions:

{
  "require": {
    "php": "^8.3",
    "ext-curl": "*",
    "ext-json": "*",
    "ext-pdo": "*",
    "ext-pdo_sqlite": "*"
  },
  "require-dev": {
    "phpunit/phpunit": "^11.0"
  },
  "autoload": {
    "psr-4": {
      "App\\": "src/"
    }
  },
  "scripts": {
    "test": "phpunit tests"
  }
}

Build a replaceable native cURL transport

Keeping cURL behind a tiny interface prevents network calls from leaking into tests. It also gives connection and total-response timeouts distinct meanings: the first bounds connection establishment, while the second bounds the complete exchange.

<?php
// src/Detector/HttpTransport.php
namespace App\Detector;

interface HttpTransport
{
    public function post(
        string $url,
        array $headers,
        string $body,
        int $connectTimeoutMs,
        int $timeoutMs
    ): HttpResponse;
}

final readonly class HttpResponse
{
    public function __construct(
        public int $status,
        public array $headers,
        public string $body
    ) {}
}

final class TransportFailure extends \RuntimeException {}

// src/Detector/CurlTransport.php
final class CurlTransport implements HttpTransport
{
    public function post(
        string $url,
        array $headers,
        string $body,
        int $connectTimeoutMs,
        int $timeoutMs
    ): HttpResponse {
        $responseHeaders = [];
        $handle = curl_init($url);

        if ($handle === false) {
            throw new TransportFailure('Unable to initialize cURL');
        }

        curl_setopt_array($handle, [
            CURLOPT_POST => true,
            CURLOPT_POSTFIELDS => $body,
            CURLOPT_HTTPHEADER => $headers,
            CURLOPT_RETURNTRANSFER => true,
            CURLOPT_CONNECTTIMEOUT_MS => $connectTimeoutMs,
            CURLOPT_TIMEOUT_MS => $timeoutMs,
            CURLOPT_HEADERFUNCTION => static function (
                $curl,
                string $line
            ) use (&$responseHeaders): int {
                $separator = strpos($line, ':');
                if ($separator !== false) {
                    $name = strtolower(trim(substr($line, 0, $separator)));
                    $responseHeaders[$name] = trim(substr($line, $separator + 1));
                }
                return strlen($line);
            },
        ]);

        $bodyText = curl_exec($handle);
        if ($bodyText === false) {
            $message = curl_error($handle);
            curl_close($handle);
            throw new TransportFailure($message);
        }

        $status = (int) curl_getinfo($handle, CURLINFO_RESPONSE_CODE);
        curl_close($handle);

        return new HttpResponse($status, $responseHeaders, $bodyText);
    }
}

Map the API response at one defensive boundary

Remote JSON is untrusted even after a successful HTTP status. Technology entries may contain confidence, evidence, and version information, while redirect data describes how the requested address resolved. Optional or malformed values must not produce notices or corrupt the CRM.

The mapper below accepts the documented technology concepts only when their runtime types are usable. It retains normalized evidence for future diagnostics, but the lead-facing summary stays compact.

<?php
// src/Detector/DetectionReport.php
namespace App\Detector;

final readonly class DetectionReport
{
    public function __construct(
        public array $technologies,
        public array $redirects
    ) {}

    public static function fromJson(array $document): self
    {
        $payload = is_array($document['data'] ?? null)
            ? $document['data']
            : $document;

        $rawTechnologies = $payload['technologies'] ?? [];
        $technologies = [];

        if (is_array($rawTechnologies)) {
            foreach ($rawTechnologies as $item) {
                if (!is_array($item) || !is_string($item['name'] ?? null)) {
                    continue;
                }

                $confidence = $item['confidence'] ?? null;
                $technologies[] = [
                    'name' => trim($item['name']),
                    'version' => is_string($item['version'] ?? null)
                        ? trim($item['version'])
                        : null,
                    'confidence' => is_numeric($confidence)
                        ? max(0.0, min(100.0, (float) $confidence))
                        : null,
                    'evidence' => is_array($item['evidence'] ?? null)
                        ? $item['evidence']
                        : [],
                ];
            }
        }

        usort(
            $technologies,
            static fn (array $a, array $b): int =>
                ($b['confidence'] ?? -1) <=> ($a['confidence'] ?? -1)
        );

        $redirects = is_array($payload['redirects'] ?? null)
            ? $payload['redirects']
            : [];

        return new self($technologies, $redirects);
    }

    public function summary(int $limit = 6): string
    {
        if ($this->technologies === []) {
            return 'No technologies detected';
        }

        $parts = array_map(
            static function (array $technology): string {
                $label = $technology['name'];

                if ($technology['version'] !== null &&
                    $technology['version'] !== '') {
                    $label .= ' ' . $technology['version'];
                }

                if ($technology['confidence'] !== null) {
                    $label .= sprintf(
                        ' (%.0f%% confidence)',
                        $technology['confidence']
                    );
                }

                return $label;
            },
            array_slice($this->technologies, 0, $limit)
        );

        $suffix = count($this->technologies) > $limit
            ? sprintf(' and %d more', count($this->technologies) - $limit)
            : '';

        return implode(', ', $parts) . $suffix;
    }
}

Keep the raw evidence in memory or a purpose-built audit store only when the CRM needs it. Evidence can be verbose, and saving an entire provider response indefinitely creates unnecessary storage and privacy obligations.

Add selective retries and structured failures

A retry is appropriate for a connection failure, HTTP 429, or a server-side 5xx response. It is inappropriate for invalid input or rejected credentials. Retrying those responses merely consumes quota and hides configuration defects.

<?php
// src/Detector/DetectorClient.php
namespace App\Detector;

final class ServiceFailure extends \RuntimeException
{
    public function __construct(
        public readonly string $reason,
        public readonly bool $retryable,
        public readonly ?int $status = null
    ) {
        parent::__construct($reason);
    }
}

final readonly class DetectorClient
{
    private const ENDPOINT =
        'https://ai.mihajlo.mk/api/website-technology-detector/v1/detect-technologies';

    public function __construct(
        private HttpTransport $transport,
        private string $token,
        private \Closure $sleep
    ) {
        if ($token === '') {
            throw new \InvalidArgumentException('Detector token is missing');
        }
    }

    public function detect(string $url): DetectionReport
    {
        $body = json_encode(['url' => $url], JSON_THROW_ON_ERROR);

        for ($attempt = 1; $attempt <= 3; $attempt++) {
            try {
                $response = $this->transport->post(
                    self::ENDPOINT,
                    [
                        'Authorization: Bearer ' . $this->token,
                        'Accept: application/json',
                        'Content-Type: application/json',
                    ],
                    $body,
                    3000,
                    20000
                );
            } catch (TransportFailure $exception) {
                if ($attempt === 3) {
                    throw new ServiceFailure('transport_error', true);
                }
                ($this->sleep)($attempt === 1 ? 250 : 750);
                continue;
            }

            if ($response->status >= 200 && $response->status < 300) {
                try {
                    $json = json_decode(
                        $response->body,
                        true,
                        512,
                        JSON_THROW_ON_ERROR
                    );
                } catch (\JsonException) {
                    throw new ServiceFailure(
                        'invalid_response',
                        false,
                        $response->status
                    );
                }

                if (!is_array($json)) {
                    throw new ServiceFailure(
                        'invalid_response',
                        false,
                        $response->status
                    );
                }

                return DetectionReport::fromJson($json);
            }

            if (in_array($response->status, [401, 403], true)) {
                throw new ServiceFailure(
                    'authentication_failed',
                    false,
                    $response->status
                );
            }

            if (in_array($response->status, [400, 422], true)) {
                throw new ServiceFailure(
                    'invalid_request',
                    false,
                    $response->status
                );
            }

            $retryable = $response->status === 429
                || $response->status >= 500;

            if (!$retryable || $attempt === 3) {
                throw new ServiceFailure(
                    $response->status === 429
                        ? 'rate_limited'
                        : 'upstream_error',
                    $retryable,
                    $response->status
                );
            }

            $retryAfter = $response->headers['retry-after'] ?? null;
            $delayMs = ctype_digit((string) $retryAfter)
                ? min(10000, (int) $retryAfter * 1000)
                : ($attempt === 1 ? 250 : 750);

            ($this->sleep)($delayMs);
        }

        throw new ServiceFailure('unexpected_state', false);
    }
}

Enrich leads without losing good data

Add four nullable columns to the CRM’s existing leads table. Adapt the migration syntax if your application uses a different database:

ALTER TABLE leads ADD COLUMN tech_summary TEXT NULL;
ALTER TABLE leads ADD COLUMN tech_status TEXT NULL;
ALTER TABLE leads ADD COLUMN tech_checked_at TEXT NULL;
ALTER TABLE leads ADD COLUMN tech_error TEXT NULL;

The command processes a bounded batch. A temporary failure records a retryable state but preserves any previous summary. An authentication failure stops immediately because every later request would fail for the same reason.

<?php
// bin/enrich-leads
declare(strict_types=1);

require dirname(__DIR__) . '/vendor/autoload.php';

use App\Detector\CurlTransport;
use App\Detector\DetectorClient;
use App\Detector\ServiceFailure;

$token = getenv('WEBSITE_DETECTOR_TOKEN');
$databasePath = getenv('CRM_DATABASE_PATH') ?: 'var/crm.sqlite';

$client = new DetectorClient(
    new CurlTransport(),
    is_string($token) ? $token : '',
    static fn (int $milliseconds) => usleep($milliseconds * 1000)
);

$pdo = new PDO('sqlite:' . $databasePath, null, null, [
    PDO::ATTR_ERRMODE => PDO::ERRMODE_EXCEPTION,
]);

$leads = $pdo->query(
    "SELECT id, website FROM leads
     WHERE website IS NOT NULL
       AND website != ''
       AND (tech_status IS NULL OR tech_status = 'retry')
     ORDER BY id
     LIMIT 50"
)->fetchAll(PDO::FETCH_ASSOC);

$success = $pdo->prepare(
    "UPDATE leads
     SET tech_summary = :summary,
         tech_status = 'complete',
         tech_checked_at = :checked,
         tech_error = NULL
     WHERE id = :id"
);

$failure = $pdo->prepare(
    "UPDATE leads
     SET tech_status = :status,
         tech_checked_at = :checked,
         tech_error = :error
     WHERE id = :id"
);

foreach ($leads as $lead) {
    $url = filter_var($lead['website'], FILTER_VALIDATE_URL);
    $scheme = is_string($url) ? parse_url($url, PHP_URL_SCHEME) : null;

    if ($url === false || !in_array($scheme, ['http', 'https'], true)) {
        $failure->execute([
            'status' => 'invalid_url',
            'checked' => gmdate(DATE_ATOM),
            'error' => 'Website must be an HTTP or HTTPS URL',
            'id' => $lead['id'],
        ]);
        continue;
    }

    try {
        $report = $client->detect($url);
        $success->execute([
            'summary' => $report->summary(),
            'checked' => gmdate(DATE_ATOM),
            'id' => $lead['id'],
        ]);

        error_log(json_encode([
            'event' => 'lead_technology_enriched',
            'lead_id' => $lead['id'],
            'technology_count' => count($report->technologies),
            'redirect_count' => count($report->redirects),
        ], JSON_THROW_ON_ERROR));
    } catch (ServiceFailure $exception) {
        if ($exception->reason === 'authentication_failed') {
            throw $exception;
        }

        $failure->execute([
            'status' => $exception->retryable ? 'retry' : 'failed',
            'checked' => gmdate(DATE_ATOM),
            'error' => $exception->reason,
            'id' => $lead['id'],
        ]);

        error_log(json_encode([
            'event' => 'lead_technology_enrichment_failed',
            'lead_id' => $lead['id'],
            'reason' => $exception->reason,
            'http_status' => $exception->status,
            'retryable' => $exception->retryable,
        ], JSON_THROW_ON_ERROR));
    }
}

Test without contacting the service

The fake transport returns queued responses, while the injected sleeper records backoff instead of pausing the test suite. Fixtures contain no credential.

<?php
// tests/DetectorClientTest.php
declare(strict_types=1);

use App\Detector\DetectorClient;
use App\Detector\HttpResponse;
use App\Detector\HttpTransport;
use PHPUnit\Framework\TestCase;

final class FakeTransport implements HttpTransport
{
    public int $calls = 0;

    public function __construct(private array $responses) {}

    public function post(
        string $url,
        array $headers,
        string $body,
        int $connectTimeoutMs,
        int $timeoutMs
    ): HttpResponse {
        $response = $this->responses[$this->calls] ?? null;
        $this->calls++;

        if (!$response instanceof HttpResponse) {
            throw new RuntimeException('Fake response queue exhausted');
        }

        return $response;
    }
}

final class DetectorClientTest extends TestCase
{
    public function testMapsAndSummarizesDetections(): void
    {
        $transport = new FakeTransport([
            new HttpResponse(200, [], json_encode([
                'data' => [
                    'technologies' => [
                        [
                            'name' => 'Example CMS',
                            'version' => '8',
                            'confidence' => 96,
                            'evidence' => ['header marker'],
                        ],
                    ],
                    'redirects' => [['status' => 301]],
                ],
            ], JSON_THROW_ON_ERROR)),
        ]);

        $client = new DetectorClient(
            $transport,
            'test-token',
            static fn (int $milliseconds) => null
        );

        $report = $client->detect('https://example.com');

        self::assertSame(
            'Example CMS 8 (96% confidence)',
            $report->summary()
        );
        self::assertCount(1, $report->redirects);
    }

    public function testRetriesRateLimitThenSucceeds(): void
    {
        $transport = new FakeTransport([
            new HttpResponse(429, ['retry-after' => '1'], '{}'),
            new HttpResponse(200, [], '{"technologies":[]}'),
        ]);
        $delays = [];

        $client = new DetectorClient(
            $transport,
            'test-token',
            static function (int $milliseconds) use (&$delays): void {
                $delays[] = $milliseconds;
            }
        );

        self::assertSame(
            'No technologies detected',
            $client->detect('https://example.com')->summary()
        );
        self::assertSame(2, $transport->calls);
        self::assertSame([1000], $delays);
    }
}

Run composer install, then composer test. Add separate tests for malformed JSON, missing technology fields, 401 without retry, three consecutive 5xx responses, and a transport exception. Those cases protect the policies most likely to regress during maintenance.

Security, operations, and deployment

Because this feature fetches a lead-supplied URL through an external service, validate the scheme before sending it. Avoid logging full URLs: they can contain query strings, customer identifiers, or embedded credentials. Logs should contain the lead ID, outcome, HTTP status, retryability, detection count, and redirect count—not the token, request headers, response body, or raw evidence.

Deploy the schema change before the command. Inject WEBSITE_DETECTOR_TOKEN and CRM_DATABASE_PATH, confirm the cURL and PDO SQLite extensions are enabled, run tests, and execute a one-lead canary. Schedule the command only after that canary produces a readable summary.

Use a scheduler lock so two command instances cannot enrich the same batch simultaneously. Start with a conservative frequency and batch size appropriate to the activated plan. Treat repeated 429 responses as a capacity signal: reduce scheduling pressure or review the plan instead of increasing retry counts.

Common failure patterns

• 401 or 403: the token is missing, revoked, incorrectly copied, or not valid for the service. Replace it and restart the process; do not retry automatically.
• 400 or 422: inspect URL validation and JSON construction. The command deliberately marks these failures as non-retryable.
• 429: honor a numeric Retry-After value within the configured ten-second cap, then leave the lead eligible for a later scheduled run if attempts are exhausted.
• 5xx or connection failure: retry briefly, record retry after exhaustion, and preserve an existing summary.
• Successful status with malformed JSON: classify it as an invalid response. Do not guess at data or replace a previously useful summary.
• Empty detections: store “No technologies detected.” An empty result is a valid business result, not automatically a transport failure.

Final verification checklist

• The service plan is active and the service-scoped token comes from the documentation page’s Service token panel.
• The token is environment-backed, absent from source control, fixtures, logs, and URLs.
• The client sends POST to the exact detector endpoint with a JSON url and Bearer authentication.
• Connection and total timeouts are bounded, and only transient failures are retried.
• Confidence, version, evidence, and redirect information cross a defensive mapping boundary.
• A successful lead shows a concise technology summary; temporary failures preserve prior good data.
• Automated tests use a deterministic fake transport and never call the live API.
• Production scheduling prevents overlapping runs and exposes structured success, failure, and rate-limit events.

The polished part of this integration is not the HTTP request. It is the discipline around it: credentials remain contained, uncertain JSON is normalized once, temporary failures do not destroy useful data, and the CRM receives language a person can scan. That is what turns website detection from an interesting API response into a dependable lead-enrichment feature.