Symfony CRM: Auto-Tag Leads with Website Tech Stack Insights

Mihajlo
Aug 11, 2026
Tutorials

A lead’s website often reveals more than a free-form “notes” field ever will. A small agency can use its technology stack to spot likely maintenance work, tailor discovery questions, and route opportunities to the right specialist. The useful outcome is not a dump of fingerprints; it is a concise CRM field such as “WordPress 6.x (98%), WooCommerce (94%), PHP (88%).”

This tutorial builds that feature in a Symfony application. New leads are scanned asynchronously through the Website Technology Detector API, normalized at the application boundary, and updated with a readable summary. The implementation includes bounded timeouts, Messenger retries, structured failure states, safe logging, and deterministic tests.

Get service access and a scoped token

Before writing integration code, create or access an account:

Register at https://ai.mihajlo.mk/register, or sign in at https://ai.mihajlo.mk/login.
Open the Website Technology Detector service page.
Choose an available Free, Plus, or Pro plan and complete its activation.
Open the official service documentation.
Find the Service token panel and copy the service-scoped token.

The service requires authentication. It accepts a Bearer token, an X-API-Token header, or a token query parameter. We will use the Bearer form because Symfony supports it directly and it keeps the credential out of URLs, browser history, proxy access logs, and analytics systems.

Regenerating the service token revokes the previously active token. Treat rotation as a coordinated deployment: update the secret in every environment, restart workers, verify a request, and only then consider the rollout complete.

Confirm the exact API request

The integration calls POST https://ai.mihajlo.mk/api/website-technology-detector/v1/detect-technologies with a JSON object containing url. Test the token without involving Symfony:

curl --fail-with-body \
  --request POST \
  'https://ai.mihajlo.mk/api/website-technology-detector/v1/detect-technologies' \
  --header 'Authorization: Bearer YOUR_SERVICE_TOKEN' \
  --header 'Content-Type: application/json' \
  --data '{"url":"https://example.com"}'

Do not paste a real token into shell history on shared machines or CI logs. A local environment variable or secret-backed command runner is safer for routine diagnostics.

Store the credential in .env.local, which should remain uncommitted:

WEBSITE_TECH_TOKEN=YOUR_SERVICE_TOKEN
MESSENGER_TRANSPORT_DSN=doctrine://default?queue_name=technology_scan

In production, define the same variables through the hosting platform’s secret manager. Keep only non-secret placeholders in the repository.

Choose a deliberately small architecture

Technology detection performs external network work, so it should not delay the request that creates a lead. The controller persists the lead and dispatches a small Messenger message containing its identifier. A worker loads the current lead, calls a dedicated API client, maps the response into domain values, and updates the summary.

This adds a queue, but the trade-off is worthwhile: lead creation remains responsive, transient upstream failures can be retried, and worker concurrency can be limited independently. The database-backed Doctrine transport is adequate for a small CRM. A dedicated broker becomes useful only when traffic or operational requirements justify it.

The relevant project structure is:

src/Technology/WebsiteTechnologyClient.php for the HTTP boundary
src/Technology/DetectionResult.php for defensive response mapping
src/Message/EnrichLeadTechnology.php for the queue message
src/MessageHandler/EnrichLeadTechnologyHandler.php for orchestration
tests/Technology/WebsiteTechnologyClientTest.php for deterministic transport tests

Install and configure Symfony components

The example assumes PHP 8.3 or later, a working Symfony application, Doctrine ORM, and an existing Lead entity with an identifier and website URL.

composer require symfony/http-client symfony/messenger symfony/doctrine-messenger
composer require --dev symfony/test-pack

php bin/console doctrine:migrations:diff
php bin/console doctrine:migrations:migrate
php bin/console messenger:setup-transports

Add nullable summary and error fields plus a status to Lead. A string-backed enum prevents controllers and workers from inventing incompatible status values:

<?php
// src/Entity/LeadTechnologyStatus.php
namespace App\Entity;

enum LeadTechnologyStatus: string
{
    case Pending = 'pending';
    case Ready = 'ready';
    case Rejected = 'rejected';
    case Failed = 'failed';
}

// Relevant additions to src/Entity/Lead.php
#[ORM\Column(length: 20, enumType: LeadTechnologyStatus::class)]
private LeadTechnologyStatus $technologyStatus = LeadTechnologyStatus::Pending;

#[ORM\Column(length: 1000, nullable: true)]
private ?string $technologySummary = null;

#[ORM\Column(length: 255, nullable: true)]
private ?string $technologyError = null;

public function markTechnologyReady(string $summary): void
{
    $this->technologyStatus = LeadTechnologyStatus::Ready;
    $this->technologySummary = $summary;
    $this->technologyError = null;
}

public function markTechnologyRejected(string $reason): void
{
    $this->technologyStatus = LeadTechnologyStatus::Rejected;
    $this->technologyError = mb_substr($reason, 0, 255);
}

# config/services.yaml
services:
  App\Technology\WebsiteTechnologyClient:
    arguments:
      $token: '%env(WEBSITE_TECH_TOKEN)%'

# config/packages/messenger.yaml
framework:
  messenger:
    failure_transport: failed
    transports:
      async:
        dsn: '%env(MESSENGER_TRANSPORT_DSN)%'
        retry_strategy:
          max_retries: 3
          delay: 1000
          multiplier: 2
          max_delay: 8000
      failed: 'doctrine://default?queue_name=failed'
    routing:
      App\Message\EnrichLeadTechnology: async

Normalize the API response at the boundary

The API supplies confidence-scored technology detections, evidence, versions, and redirect information. External values still need defensive validation: fields can be absent, nullable, unexpectedly typed, or added in later service versions. The CRM should depend on a stable object rather than passing arbitrary arrays through the domain.

<?php
// src/Technology/DetectionResult.php
namespace App\Technology;

final readonly class TechnologyDetection
{
    public function __construct(
        public string $name,
        public ?string $version,
        public float $confidence,
        public array $evidence,
    ) {}
}

final readonly class DetectionResult
{
    public function __construct(
        public array $technologies,
        public array $redirects,
    ) {}

    public static function fromPayload(array $payload): self
    {
        $rows = $payload['technologies'] ?? $payload['detections'] ?? null;

        if (!is_array($rows)) {
            throw new PermanentTechnologyApiException(
                'The API response contains no technology detection list.'
            );
        }

        $technologies = [];

        foreach ($rows as $row) {
            if (!is_array($row)) {
                continue;
            }

            $name = $row['name'] ?? $row['technology'] ?? null;
            $confidence = $row['confidence'] ?? null;

            if (!is_string($name) || trim($name) === '' || !is_numeric($confidence)) {
                continue;
            }

            $version = $row['version'] ?? null;
            $version = is_string($version) && trim($version) !== ''
                ? mb_substr(trim($version), 0, 80)
                : null;

            $evidence = is_array($row['evidence'] ?? null)
                ? array_values($row['evidence'])
                : [];

            $technologies[] = new TechnologyDetection(
                mb_substr(trim($name), 0, 120),
                $version,
                max(0.0, min(100.0, (float) $confidence)),
                $evidence,
            );
        }

        $redirects = $payload['redirects'] ?? $payload['redirect_information'] ?? [];

        return new self(
            $technologies,
            is_array($redirects) ? array_values($redirects) : [],
        );
    }

    public function summary(int $limit = 8): string
    {
        $items = $this->technologies;
        usort(
            $items,
            static fn ($a, $b) => $b->confidence <=> $a->confidence
        );

        $labels = array_map(
            static fn (TechnologyDetection $item): string => sprintf(
                '%s%s (%d%%)',
                $item->name,
                $item->version === null ? '' : ' '.$item->version,
                (int) round($item->confidence),
            ),
            array_slice($items, 0, $limit),
        );

        return $labels === [] ? 'No technologies detected' : implode(', ', $labels);
    }
}

The mapper preserves evidence and redirect data for future domain decisions without placing potentially large or unstable structures in the human-facing summary. If the official documentation changes its response envelope, this is the one class that should change.

Build the bounded HTTP client

The client uses a 5-second inactivity timeout and a 15-second total request budget. It distinguishes permanent failures from retryable ones. Validation and authentication failures must not be retried; rate limits, timeouts, and temporary server failures may be.

<?php
// src/Technology/PermanentTechnologyApiException.php
namespace App\Technology;
final class PermanentTechnologyApiException extends \RuntimeException {}

// src/Technology/TransientTechnologyApiException.php
namespace App\Technology;
final class TransientTechnologyApiException extends \RuntimeException {}

// src/Technology/WebsiteTechnologyClient.php
namespace App\Technology;

use Symfony\Contracts\HttpClient\Exception\TransportExceptionInterface;
use Symfony\Contracts\HttpClient\HttpClientInterface;

final readonly class WebsiteTechnologyClient
{
    private const ENDPOINT =
        'https://ai.mihajlo.mk/api/website-technology-detector/v1/detect-technologies';

    public function __construct(
        private HttpClientInterface $httpClient,
        private string $token,
    ) {
        if (trim($token) === '') {
            throw new \InvalidArgumentException('Website technology token is missing.');
        }
    }

    public function detect(string $url): DetectionResult
    {
        try {
            $response = $this->httpClient->request('POST', self::ENDPOINT, [
                'auth_bearer' => $this->token,
                'json' => ['url' => $url],
                'timeout' => 5.0,
                'max_duration' => 15.0,
                'headers' => ['Accept' => 'application/json'],
            ]);

            $status = $response->getStatusCode();
            $body = $response->getContent(false);
        } catch (TransportExceptionInterface $exception) {
            throw new TransientTechnologyApiException(
                'Technology service transport failure.',
                previous: $exception,
            );
        }

        if (in_array($status, [408, 425, 429], true) || $status >= 500) {
            throw new TransientTechnologyApiException(
                sprintf('Technology service temporarily returned HTTP %d.', $status)
            );
        }

        if ($status < 200 || $status >= 300) {
            throw new PermanentTechnologyApiException(
                sprintf('Technology service rejected the request with HTTP %d.', $status)
            );
        }

        try {
            $payload = json_decode($body, true, 512, JSON_THROW_ON_ERROR);
        } catch (\JsonException $exception) {
            throw new PermanentTechnologyApiException(
                'Technology service returned invalid JSON.',
                previous: $exception,
            );
        }

        if (!is_array($payload)) {
            throw new PermanentTechnologyApiException(
                'Technology service returned an unexpected JSON value.'
            );
        }

        return DetectionResult::fromPayload($payload);
    }
}

Dispatch and process scans asynchronously

The message carries only a database identifier. That keeps serialized messages small and ensures the worker sees the current URL rather than a stale entity snapshot.

<?php
// src/Message/EnrichLeadTechnology.php
namespace App\Message;

final readonly class EnrichLeadTechnology
{
    public function __construct(public int $leadId) {}
}

// In the controller, after validating and persisting a new Lead:
$entityManager->persist($lead);
$entityManager->flush();

$messageBus->dispatch(new EnrichLeadTechnology($lead->getId()));

The handler validates that the submitted value is an ordinary HTTP or HTTPS URL with a hostname. Because the CRM does not fetch the target itself, the external service remains responsible for safely inspecting public websites. Rejecting credentials and IP literals still reduces accidental misuse.

<?php
// src/MessageHandler/EnrichLeadTechnologyHandler.php
namespace App\MessageHandler;

use App\Message\EnrichLeadTechnology;
use App\Repository\LeadRepository;
use App\Technology\PermanentTechnologyApiException;
use App\Technology\WebsiteTechnologyClient;
use Doctrine\ORM\EntityManagerInterface;
use Psr\Log\LoggerInterface;
use Symfony\Component\Messenger\Attribute\AsMessageHandler;

#[AsMessageHandler]
final readonly class EnrichLeadTechnologyHandler
{
    public function __construct(
        private LeadRepository $leads,
        private WebsiteTechnologyClient $client,
        private EntityManagerInterface $entityManager,
        private LoggerInterface $logger,
    ) {}

    public function __invoke(EnrichLeadTechnology $message): void
    {
        $lead = $this->leads->find($message->leadId);

        if ($lead === null) {
            $this->logger->notice('Technology scan skipped: lead no longer exists.', [
                'lead_id' => $message->leadId,
            ]);
            return;
        }

        $url = $lead->getWebsiteUrl();

        if (!$this->isAcceptableUrl($url)) {
            $lead->markTechnologyRejected('Website must be a public HTTP(S) hostname.');
            $this->entityManager->flush();
            return;
        }

        try {
            $result = $this->client->detect($url);
            $lead->markTechnologyReady($result->summary());
            $this->entityManager->flush();

            $this->logger->info('Lead technology summary updated.', [
                'lead_id' => $lead->getId(),
                'technology_count' => count($result->technologies),
                'redirect_count' => count($result->redirects),
            ]);
        } catch (PermanentTechnologyApiException $exception) {
            $lead->markTechnologyRejected($exception->getMessage());
            $this->entityManager->flush();

            $this->logger->warning('Technology scan permanently rejected.', [
                'lead_id' => $lead->getId(),
                'error_class' => $exception::class,
            ]);
        }
    }

    private function isAcceptableUrl(?string $url): bool
    {
        if (!is_string($url) || filter_var($url, FILTER_VALIDATE_URL) === false) {
            return false;
        }

        $parts = parse_url($url);
        $scheme = strtolower((string) ($parts['scheme'] ?? ''));
        $host = $parts['host'] ?? null;

        return in_array($scheme, ['http', 'https'], true)
            && is_string($host)
            && $host !== ''
            && !isset($parts['user'], $parts['pass'])
            && filter_var($host, FILTER_VALIDATE_IP) === false;
    }
}

Transient exceptions are intentionally not caught. Messenger retries them with bounded exponential delays and eventually places the message in the failure transport. This avoids nested retry loops that could multiply traffic during an outage.

Test without calling the live service

MockHttpClient provides a deterministic transport. Tests should assert both the outgoing contract and failure classification; they must never contain a production token.

<?php
// tests/Technology/WebsiteTechnologyClientTest.php
namespace App\Tests\Technology;

use App\Technology\PermanentTechnologyApiException;
use App\Technology\WebsiteTechnologyClient;
use PHPUnit\Framework\TestCase;
use Symfony\Component\HttpClient\MockHttpClient;
use Symfony\Component\HttpClient\Response\MockResponse;

final class WebsiteTechnologyClientTest extends TestCase
{
    public function testItBuildsAReadableSummary(): void
    {
        $http = new MockHttpClient(function ($method, $url, $options) {
            self::assertSame('POST', $method);
            self::assertStringEndsWith('/v1/detect-technologies', $url);
            self::assertSame(
                ['url' => 'https://example.com'],
                json_decode($options['body'], true, 512, JSON_THROW_ON_ERROR)
            );

            return new MockResponse(json_encode([
                'technologies' => [
                    [
                        'name' => 'Example CMS',
                        'version' => '4.2',
                        'confidence' => 97,
                        'evidence' => ['response marker'],
                    ],
                    [
                        'name' => 'Example Runtime',
                        'confidence' => 81.4,
                        'evidence' => [],
                    ],
                ],
                'redirects' => [],
            ], JSON_THROW_ON_ERROR), [
                'http_code' => 200,
                'response_headers' => ['content-type: application/json'],
            ]);
        });

        $result = (new WebsiteTechnologyClient($http, 'test-token'))
            ->detect('https://example.com');

        self::assertSame(
            'Example CMS 4.2 (97%), Example Runtime (81%)',
            $result->summary()
        );
    }

    public function testAuthenticationFailureIsPermanent(): void
    {
        $http = new MockHttpClient(new MockResponse('{}', ['http_code' => 401]));
        $client = new WebsiteTechnologyClient($http, 'invalid-test-token');

        $this->expectException(PermanentTechnologyApiException::class);
        $client->detect('https://example.com');
    }
}

php bin/phpunit
php bin/console messenger:consume async --time-limit=3600 --memory-limit=128M
php bin/console messenger:failed:show

Security, observability, and deployment

Never log the Bearer token, complete response payload, evidence values, or headers. Evidence may contain website content, while URLs can contain customer information. Prefer lead identifiers, status codes, exception classes, durations, and detection counts. Configure production logging to alert on sustained rate-limit responses, authentication failures, and growth in the failed queue.

Limit worker concurrency according to the activated plan rather than maximizing throughput. A 429 is a capacity signal, not an invitation to retry immediately. The configured delays spread attempts over time, while the failed transport prevents endless processing.

Generate and review the Doctrine migration locally, commit it, and run doctrine:migrations:migrate --no-interaction during deployment. Inject the token before starting workers. Restart workers after code or secret changes so long-running processes receive the new container and environment.

Common failures are usually straightforward: 401 or 403 indicates a missing, revoked, or wrong service token; 400 or 422 points to an unacceptable URL; 429 means the plan or request rate needs attention; repeated 5xx responses belong in the retry and failure queues. Invalid JSON or a changed response shape should fail visibly at the boundary rather than silently producing misleading CRM data.

Final verification checklist

The service plan is active and the scoped token is supplied through environment-backed configuration.
A minimal authenticated POST request succeeds against the exact detection endpoint.
Creating a lead commits it before dispatching the Messenger message.
The worker writes a concise, confidence-aware technology summary.
Authentication and validation failures are not retried.
Timeouts, rate limits, and temporary server failures receive bounded retries.
Failed messages are observable and recoverable through the failure transport.
Logs contain operational context but no credentials, evidence payloads, or sensitive headers.
Tests pass with MockHttpClient and never contact the live service.

The strongest part of this design is its restraint. One API boundary understands an external response, one background handler owns the workflow, and the CRM stores the small piece of information its users actually need. The result turns a website URL from passive contact data into a practical conversation starter without turning a modest agency application into an integration maze.