Tutorials

Symfony: Detect Tech Stack Changes on Client Websites with Website Technology Detector API

Symfony: Detect Tech Stack Changes on Client Websites with Website Technology Detector API

A client website can change underneath you without a deployment from your side. A redesign may replace its CMS, a hosting migration may remove a CDN, or an analytics tag may disappear during an otherwise harmless theme update. Those changes matter when you maintain integrations, monitor performance, or advise the client on security.

This tutorial builds a production-oriented Symfony command that scans one important client website, records a baseline, and emails a developer when technologies are added, removed, or report a different version. It uses the Website Technology Detector API as the detection boundary and deliberately keeps scheduling, comparison, persistence, and notification inside the application.

Get access and create a service token

Register through the registration page, or use the sign-in page if you already have an account.

  1. Open the Website Technology Detector service page.
  2. Choose the available Free, Plus, or Pro plan and complete activation.
  3. Open the official service documentation.
  4. Find the Service token panel and copy its service-scoped token.

The service requires authentication. It accepts a Bearer token, an X-API-Token header, or a token query parameter. This project uses the Bearer form so the credential stays out of URLs and access logs.

Regenerating the service token revokes the previously active token. Treat rotation as a coordinated deployment: update the application secret immediately after regeneration, then run a verification scan.

Confirm the exact API call

The integration sends POST requests to https://ai.mihajlo.mk/api/website-technology-detector/v1/detect-technologies. Its JSON body contains the required url value.

curl --request POST \
  --url https://ai.mihajlo.mk/api/website-technology-detector/v1/detect-technologies \
  --header "Authorization: Bearer YOUR_SERVICE_TOKEN" \
  --header "Accept: application/json" \
  --header "Content-Type: application/json" \
  --data '{"url":"https://client.example"}'

Use a public website you are authorized to monitor. The response contains confidence-scored detections, evidence, version data, and redirect information. We will validate those values instead of assuming that every optional field is present.

Store the credential before writing application code

For local development, put the token in .env.local, which should remain outside version control. Production should inject the same variables through the hosting platform or secret manager.

TECH_DETECTOR_TOKEN=YOUR_SERVICE_TOKEN
TECH_DETECTOR_ENDPOINT=https://ai.mihajlo.mk/api/website-technology-detector/v1/detect-technologies
CLIENT_WEBSITE_URL=https://client.example
[email protected]
[email protected]
MAILER_DSN=smtp://USERNAME:[email protected]:587

URL-encode reserved characters in SMTP credentials. Never place the real service token in source code, fixtures, logs, screenshots, or exception messages.

Create the Symfony project

This implementation targets PHP 8.3 or later and uses first-party Symfony components. A scheduled command is preferable to Messenger here: each scan is a small, periodic unit of work, so a permanently running worker would add operational cost without improving the outcome.

composer create-project symfony/skeleton tech-watch
cd tech-watch
composer require symfony/console symfony/http-client symfony/mailer symfony/monolog-bundle
composer require --dev symfony/test-pack
mkdir -p var/data var/lock

The relevant project structure is intentionally compact:

src/
  Command/WatchTechnologyStackCommand.php
  Domain/DetectionReport.php
  Domain/StackDiff.php
  Infrastructure/TechnologyDetector.php
  Infrastructure/ReportStore.php
  Infrastructure/DeveloperNotifier.php
tests/
  Infrastructure/TechnologyDetectorTest.php
config/
  services.yaml
var/data/
  technology-report.json

Configure scalar constructor arguments while leaving Symfony to autowire framework services:

# config/services.yaml
services:
    _defaults:
        autowire: true
        autoconfigure: true

    App\:
        resource: '../src/'

    App\Infrastructure\TechnologyDetector:
        arguments:
            $apiToken: '%env(TECH_DETECTOR_TOKEN)%'
            $endpoint: '%env(TECH_DETECTOR_ENDPOINT)%'

    App\Infrastructure\DeveloperNotifier:
        arguments:
            $recipient: '%env(DEVELOPER_EMAIL)%'
            $sender: '%env(NOTIFICATION_FROM)%'

    App\Command\WatchTechnologyStackCommand:
        arguments:
            $websiteUrl: '%env(CLIENT_WEBSITE_URL)%'

Map the response at the application boundary

Detection payloads describe external observations, not trusted domain objects. The mapper below requires a detections collection, accepts both list and name-keyed entries, validates confidence values, normalizes version strings, preserves evidence, and collects top-level redirect-related fields without depending on one optional redirect shape.

<?php
// src/Domain/DetectionReport.php
namespace App\Domain;

final readonly class DetectionReport
{
    public function __construct(
        public array $detections,
        public array $redirectInfo,
    ) {}

    public static function fromPayload(array $payload): self
    {
        $rows = $payload['detections'] ?? null;

        if (!is_array($rows)) {
            throw new \UnexpectedValueException(
                'The API response has no valid detections collection.'
            );
        }

        $detections = [];

        foreach ($rows as $key => $row) {
            if (!is_array($row)) {
                continue;
            }

            $name = is_string($row['name'] ?? null)
                ? trim($row['name'])
                : (is_string($key) ? trim($key) : '');

            if ($name === '') {
                continue;
            }

            $confidence = $row['confidence'] ?? null;
            $confidence = is_int($confidence) || is_float($confidence)
                ? $confidence
                : null;

            $versionInput = $row['versions'] ?? ($row['version'] ?? []);
            $versions = is_array($versionInput)
                ? $versionInput
                : [$versionInput];

            $versions = array_values(array_unique(array_filter(
                array_map(
                    static fn (mixed $value): string =>
                        is_scalar($value) ? trim((string) $value) : '',
                    $versions
                ),
                static fn (string $value): bool => $value !== ''
            )));
            sort($versions, SORT_NATURAL | SORT_FLAG_CASE);

            $evidence = $row['evidence'] ?? [];
            $evidence = is_array($evidence) ? array_values($evidence) : [$evidence];

            $detections[] = [
                'name' => $name,
                'confidence' => $confidence,
                'versions' => $versions,
                'evidence' => $evidence,
            ];
        }

        $redirectInfo = [];
        foreach ($payload as $key => $value) {
            if (is_string($key)
                && str_contains(strtolower($key), 'redirect')) {
                $redirectInfo[$key] = $value;
            }
        }

        return new self($detections, $redirectInfo);
    }

    public function toArray(): array
    {
        return [
            'detections' => $this->detections,
            'redirect_info' => $this->redirectInfo,
            'observed_at' => gmdate(DATE_ATOM),
        ];
    }
}

Confidence and evidence remain available for diagnosis, but they do not trigger alerts. Evidence can change when markup changes, and confidence fluctuations alone can create noisy notifications. The meaningful stack comparison is based on normalized technology names and versions.

Build a resilient HTTP client

The client uses bounded connection and overall response timeouts. It retries transport failures, HTTP 429 responses, and server-side 5xx responses up to three attempts. Authentication and other client errors fail immediately because repeating an invalid request only consumes time and quota.

<?php
// src/Infrastructure/TechnologyDetector.php
namespace App\Infrastructure;

use App\Domain\DetectionReport;
use Psr\Log\LoggerInterface;
use Symfony\Contracts\HttpClient\HttpClientInterface;
use Symfony\Contracts\HttpClient\ResponseInterface;
use Symfony\Contracts\HttpClient\Exception\TransportExceptionInterface;

final class DetectorException extends \RuntimeException {}

final class TechnologyDetector
{
    private \Closure $sleep;

    public function __construct(
        private HttpClientInterface $http,
        private LoggerInterface $logger,
        private string $apiToken,
        private string $endpoint,
        ?callable $sleeper = null,
    ) {
        $this->sleep = $sleeper === null
            ? static fn (int $microseconds) => usleep($microseconds)
            : \Closure::fromCallable($sleeper);
    }

    public function detect(string $url): DetectionReport
    {
        if (filter_var($url, FILTER_VALIDATE_URL) === false
            || !in_array(parse_url($url, PHP_URL_SCHEME), ['http', 'https'], true)) {
            throw new \InvalidArgumentException('A valid HTTP(S) URL is required.');
        }

        for ($attempt = 1; $attempt <= 3; $attempt++) {
            try {
                $response = $this->http->request('POST', $this->endpoint, [
                    'headers' => [
                        'Authorization' => 'Bearer '.$this->apiToken,
                        'Accept' => 'application/json',
                    ],
                    'json' => ['url' => $url],
                    'timeout' => 5.0,
                    'max_duration' => 20.0,
                ]);

                $status = $response->getStatusCode();

                if ($status >= 200 && $status < 300) {
                    $payload = json_decode(
                        $response->getContent(false),
                        true,
                        512,
                        JSON_THROW_ON_ERROR
                    );

                    if (!is_array($payload)) {
                        throw new \UnexpectedValueException(
                            'The API response is not a JSON object.'
                        );
                    }

                    return DetectionReport::fromPayload($payload);
                }

                $retryable = $status === 429 || $status >= 500;

                if (!$retryable || $attempt === 3) {
                    throw new DetectorException(
                        sprintf('Technology detection failed with HTTP %d.', $status)
                    );
                }

                $this->logger->warning('technology_detector.retry', [
                    'status' => $status,
                    'attempt' => $attempt,
                    'host' => parse_url($url, PHP_URL_HOST),
                ]);

                $this->pause($response, $attempt);
            } catch (TransportExceptionInterface $exception) {
                if ($attempt === 3) {
                    throw new DetectorException(
                        'Technology detection failed after transport retries.',
                        0,
                        $exception
                    );
                }

                $this->logger->warning('technology_detector.transport_retry', [
                    'attempt' => $attempt,
                    'host' => parse_url($url, PHP_URL_HOST),
                ]);

                ($this->sleep)((2 ** ($attempt - 1)) * 1_000_000);
            } catch (\JsonException|\UnexpectedValueException $exception) {
                throw new DetectorException(
                    'Technology detector returned an invalid response.',
                    0,
                    $exception
                );
            }
        }

        throw new \LogicException('Retry loop terminated unexpectedly.');
    }

    private function pause(ResponseInterface $response, int $attempt): void
    {
        $header = $response->getHeaders(false)['retry-after'][0] ?? null;
        $seconds = is_string($header) && ctype_digit($header)
            ? max(1, min(15, (int) $header))
            : 2 ** ($attempt - 1);

        ($this->sleep)($seconds * 1_000_000);
    }
}

The exception deliberately omits response bodies and credentials. Logs identify the event, attempt, status, and target host without leaking the token.

Persist and compare snapshots

A single watched site does not justify a database. An atomic JSON file is adequate, easy to inspect, and simple to back up. If you later monitor many clients, move the same repository contract to a database with one row per canonical URL.

<?php
// src/Infrastructure/ReportStore.php
namespace App\Infrastructure;

final class ReportStore
{
    private string $path;

    public function __construct(string $projectDir)
    {
        $this->path = $projectDir.'/var/data/technology-report.json';
    }

    public function load(): ?array
    {
        if (!is_file($this->path)) {
            return null;
        }

        $data = json_decode(
            (string) file_get_contents($this->path),
            true,
            512,
            JSON_THROW_ON_ERROR
        );

        if (!is_array($data)) {
            throw new \UnexpectedValueException('Stored report is invalid.');
        }

        return $data;
    }

    public function save(array $report): void
    {
        $directory = dirname($this->path);
        if (!is_dir($directory) && !mkdir($directory, 0770, true)) {
            throw new \RuntimeException('Cannot create report directory.');
        }

        $temporary = tempnam($directory, 'report-');
        if ($temporary === false) {
            throw new \RuntimeException('Cannot create temporary report.');
        }

        $json = json_encode($report, JSON_PRETTY_PRINT | JSON_THROW_ON_ERROR);

        if (file_put_contents($temporary, $json, LOCK_EX) === false
            || !rename($temporary, $this->path)) {
            @unlink($temporary);
            throw new \RuntimeException('Cannot save technology report.');
        }
    }
}
<?php
// src/Domain/StackDiff.php
namespace App\Domain;

final class StackDiff
{
    public static function between(array $before, array $after): array
    {
        $old = self::index($before);
        $new = self::index($after);

        $added = array_values(array_diff_key($new, $old));
        $removed = array_values(array_diff_key($old, $new));
        $versionChanges = [];

        foreach (array_intersect_key($new, $old) as $key => $current) {
            if ($old[$key]['versions'] !== $current['versions']) {
                $versionChanges[] = [
                    'name' => $current['name'],
                    'before' => $old[$key]['versions'],
                    'after' => $current['versions'],
                ];
            }
        }

        return array_filter([
            'added' => $added,
            'removed' => $removed,
            'version_changes' => $versionChanges,
        ]);
    }

    private static function index(array $detections): array
    {
        $indexed = [];

        foreach ($detections as $detection) {
            if (!is_array($detection)
                || !is_string($detection['name'] ?? null)) {
                continue;
            }

            $key = strtolower(trim($detection['name']));
            if ($key !== '') {
                $indexed[$key] = [
                    'name' => trim($detection['name']),
                    'versions' => is_array($detection['versions'] ?? null)
                        ? array_values($detection['versions'])
                        : [],
                ];
            }
        }

        ksort($indexed);
        return $indexed;
    }
}

Send the notification and run the command

<?php
// src/Infrastructure/DeveloperNotifier.php
namespace App\Infrastructure;

use Symfony\Component\Mailer\MailerInterface;
use Symfony\Component\Mime\Email;

final class DeveloperNotifier
{
    public function __construct(
        private MailerInterface $mailer,
        private string $recipient,
        private string $sender,
    ) {}

    public function send(string $url, array $changes, array $report): void
    {
        $body = "Public technology stack changes were detected for {$url}.\n\n"
            ."Changes:\n"
            .json_encode($changes, JSON_PRETTY_PRINT | JSON_THROW_ON_ERROR)
            ."\n\nCurrent validated report:\n"
            .json_encode($report, JSON_PRETTY_PRINT | JSON_THROW_ON_ERROR);

        $this->mailer->send(
            (new Email())
                ->from($this->sender)
                ->to($this->recipient)
                ->subject('Client website technology stack changed')
                ->text($body)
        );
    }
}
<?php
// src/Command/WatchTechnologyStackCommand.php
namespace App\Command;

use App\Domain\StackDiff;
use App\Infrastructure\DeveloperNotifier;
use App\Infrastructure\ReportStore;
use App\Infrastructure\TechnologyDetector;
use Psr\Log\LoggerInterface;
use Symfony\Component\Console\Attribute\AsCommand;
use Symfony\Component\Console\Command\Command;
use Symfony\Component\Console\Input\InputInterface;
use Symfony\Component\Console\Output\OutputInterface;
use Symfony\Component\Console\Style\SymfonyStyle;

#[AsCommand(
    name: 'app:watch-tech-stack',
    description: 'Detects and reports client website technology changes.'
)]
final class WatchTechnologyStackCommand extends Command
{
    public function __construct(
        private TechnologyDetector $detector,
        private ReportStore $store,
        private DeveloperNotifier $notifier,
        private LoggerInterface $logger,
        private string $websiteUrl,
    ) {
        parent::__construct();
    }

    protected function execute(
        InputInterface $input,
        OutputInterface $output
    ): int {
        $io = new SymfonyStyle($input, $output);

        try {
            $current = $this->detector->detect($this->websiteUrl)->toArray();
            $previous = $this->store->load();

            if ($previous === null) {
                $this->store->save($current);
                $io->success('Initial technology baseline stored.');
                return Command::SUCCESS;
            }

            $changes = StackDiff::between(
                $previous['detections'] ?? [],
                $current['detections']
            );

            if ($changes !== []) {
                $this->notifier->send(
                    $this->websiteUrl,
                    $changes,
                    $current
                );

                $this->logger->notice('technology_stack.changed', [
                    'host' => parse_url($this->websiteUrl, PHP_URL_HOST),
                    'change_groups' => array_keys($changes),
                ]);
            }

            // Save after notification so a mail failure is retried next run.
            $this->store->save($current);
            $io->success(
                $changes === [] ? 'No stack change detected.' : 'Developer notified.'
            );

            return Command::SUCCESS;
        } catch (\Throwable $exception) {
            $this->logger->error('technology_watch.failed', [
                'exception' => $exception,
                'host' => parse_url($this->websiteUrl, PHP_URL_HOST),
            ]);
            $io->error('Technology watch failed; inspect application logs.');
            return Command::FAILURE;
        }
    }
}

The first successful run creates a baseline without emailing anyone. Subsequent runs save ordinary observations silently. When a meaningful change appears, the command sends the email first and only then advances the snapshot, ensuring a temporary mail outage does not permanently suppress the alert.

Test without calling the real service

MockHttpClient makes the API boundary deterministic. The tests verify the request contract, defensive mapping, and bounded retry behavior without consuming quota or exposing a token.

<?php
// tests/Infrastructure/TechnologyDetectorTest.php
namespace App\Tests\Infrastructure;

use App\Infrastructure\TechnologyDetector;
use PHPUnit\Framework\TestCase;
use Psr\Log\NullLogger;
use Symfony\Component\HttpClient\MockHttpClient;
use Symfony\Component\HttpClient\Response\MockResponse;

final class TechnologyDetectorTest extends TestCase
{
    public function testItSendsTheContractAndMapsTheReport(): void
    {
        $http = new MockHttpClient(
            function (string $method, string $url, array $options): MockResponse {
                self::assertSame('POST', $method);
                self::assertSame('https://service.test/detect', $url);
                self::assertSame(
                    ['url' => 'https://client.example'],
                    json_decode($options['body'], true, 512, JSON_THROW_ON_ERROR)
                );

                return new MockResponse(json_encode([
                    'detections' => [[
                        'name' => 'Example CMS',
                        'confidence' => 0.95,
                        'versions' => ['6.1'],
                        'evidence' => ['public marker'],
                    ]],
                    'redirects' => ['https://client.example/'],
                ], JSON_THROW_ON_ERROR), ['http_code' => 200]);
            }
        );

        $detector = new TechnologyDetector(
            $http,
            new NullLogger(),
            'test-token',
            'https://service.test/detect',
            static fn (int $microseconds) => null
        );

        $report = $detector->detect('https://client.example');

        self::assertSame('Example CMS', $report->detections[0]['name']);
        self::assertSame(['6.1'], $report->detections[0]['versions']);
        self::assertArrayHasKey('redirects', $report->redirectInfo);
    }

    public function testItRetriesAServiceFailure(): void
    {
        $http = new MockHttpClient([
            new MockResponse('', ['http_code' => 503]),
            new MockResponse('{"detections":[]}', ['http_code' => 200]),
        ]);

        $detector = new TechnologyDetector(
            $http,
            new NullLogger(),
            'test-token',
            'https://service.test/detect',
            static fn (int $microseconds) => null
        );

        self::assertSame(
            [],
            $detector->detect('https://client.example')->detections
        );
        self::assertSame(2, $http->getRequestsCount());
    }
}
php bin/phpunit
php bin/console app:watch-tech-stack -vv

Deploy, schedule, and observe it

Deploy with production dependencies, provide environment variables through the platform’s secret facility, verify that var/data is persistent and writable, and configure a real mail transport. Containers with ephemeral filesystems need a mounted volume or a database-backed report store.

On a conventional Linux host, run the command from cron and use flock to prevent overlapping scans:

17 6 * * * cd /srv/tech-watch && /usr/bin/flock -n var/lock/tech-watch.lock /usr/bin/php bin/console app:watch-tech-stack --env=prod >> var/log/tech-watch-cron.log 2>&1

Monitor command exit codes and the structured events technology_detector.retry, technology_detector.transport_retry, technology_stack.changed, and technology_watch.failed. A successful “no change” execution should also be visible through scheduler history so silence is not mistaken for health.

Common production failures

  • HTTP 401 or 403: confirm the service-scoped token and whether it was recently regenerated. These failures are intentionally not retried.
  • HTTP 429: reduce scan frequency or review the active plan. The client honors a numeric Retry-After value within a bounded delay.
  • Repeated 5xx or transport failures: check outbound HTTPS, DNS, proxy policy, and service availability. The command exits unsuccessfully after its retry budget.
  • Invalid JSON or missing detections: retain the failure log and compare the response contract with the official documentation. Do not silently store an empty baseline.
  • No email: test the Symfony mail transport independently and inspect mailer logs. Because the snapshot is not advanced after a mail failure, the next run attempts the notification again.
  • Baseline disappears: make var/data persistent across releases and container replacements.

Final verification checklist

  • The service plan is active and the token comes from the documentation page’s Service token panel.
  • The token exists only in environment-backed configuration.
  • The minimal authenticated request succeeds for the configured public URL.
  • php bin/phpunit passes without network access.
  • The first command run creates var/data/technology-report.json.
  • A controlled fixture or test snapshot change produces one developer email.
  • Authentication failures are not retried; 429, 5xx, and transport failures have bounded retries.
  • The scheduler prevents overlapping runs and reports nonzero exit codes.
  • Logs contain operational context but never the token or raw response body.

A useful technology monitor is not merely an HTTP request on a timer. Its value comes from stable comparison rules, careful handling of uncertain external data, reliable notification semantics, and enough observability to distinguish “nothing changed” from “nothing ran.” With those boundaries in place, a quiet command becomes an early-warning system for the website changes that would otherwise surprise you later.

Blog author portrait

Mihajlo

I’m Mihajlo — a developer driven by curiosity, discipline, and the constant urge to create something meaningful. I share insights, tutorials, and free services to help others simplify their work and grow in the ever-evolving world of software and AI.

Search

Categories

Recent Posts

Your Daily Reset: How Intentional Downtime Powers Peak Productivity
4-Week Glutes and Legs Shape Program: Progression Cycle 10
Beyond Microservices: Scaling APIs with Evolving System Needs
The Unseen Engine: How Your Body's Downtime Fuels Peak Mental Performance
Native PHP 8.3: Enrich Leads with Website Tech Insights Using the Detector API