Туториали

Laravel: Monitor Client Tech Stacks with Real-Time AI Alerts

Laravel: Следете ги технолошките стекови на клиентите со AI известувања во реално време

A client’s website can change underneath you without a deployment from your team. A hosting migration may introduce a new proxy, a CMS upgrade may replace plugins, or a hurried contractor may swap an analytics provider. Uptime monitoring will still report green because the pages load. What changed is the site’s operational fingerprint.

This tutorial builds a Laravel monitor that periodically submits an important public website to the Website Technology Detector API, stores a stable snapshot, and emails a developer when technologies or detected versions change. The design uses Laravel’s HTTP client, scheduler, queue, database, notifications, and test fakes. It also treats the remote response as untrusted data rather than assuming every field will always be present.

Get access and copy the service token

Start by registering an account, or use the sign-in page if you already have one.

  1. Open the Website Technology Detector service page.
  2. Choose an available Free, Plus, or Pro plan and complete its activation.
  3. Open the official service documentation.
  4. Find the Service token panel and copy the service-scoped token.
  5. Store it immediately in your password manager and the deployment platform’s secret store.

This service requires authentication. It accepts a Bearer token, an X-API-Token header, or a token query parameter. We will use the Bearer form because it avoids placing the credential in a URL. Regenerating the service token revokes the previously active token, so token rotation must update every running application instance before old credentials are discarded.

Verify the endpoint before writing application code

The exact operation is POST https://ai.mihajlo.mk/api/website-technology-detector/v1/detect-technologies. Its JSON request contains url. Run this minimal request locally, replacing the placeholder without committing the resulting shell history or token:

curl --request POST \
  --url https://ai.mihajlo.mk/api/website-technology-detector/v1/detect-technologies \
  --header "Authorization: Bearer YOUR_SERVICE_TOKEN" \
  --header "Accept: application/json" \
  --header "Content-Type: application/json" \
  --data '{"url":"https://client.example"}'

The response contains confidence-scored technology detections, supporting evidence, version information, and redirect information. Those details are valuable for an alert, but confidence and evidence can fluctuate without the installed stack changing. Our change fingerprint will therefore use technology names and versions, while retaining confidence, evidence, and redirects in the stored snapshot for diagnosis.

Put the credential in the project’s uncommitted .env file:

MIHAJLO_TECH_DETECTOR_TOKEN=YOUR_SERVICE_TOKEN
MIHAJLO_TECH_DETECTOR_URL=https://ai.mihajlo.mk/api/website-technology-detector/v1/detect-technologies

QUEUE_CONNECTION=database
MAIL_MAILER=smtp
[email protected]

Expose it through Laravel configuration rather than calling env() from application classes:

<?php
// config/services.php

return [
    // Other services...

    'technology_detector' => [
        'url' => env(
            'MIHAJLO_TECH_DETECTOR_URL',
            'https://ai.mihajlo.mk/api/website-technology-detector/v1/detect-technologies'
        ),
        'token' => env('MIHAJLO_TECH_DETECTOR_TOKEN'),
        'connect_timeout' => 3,
        'timeout' => 12,
    ],
];

Architecture: scheduled work without noisy alerts

The scheduler finds enabled client sites and dispatches one unique queue job per site. Each job calls a dedicated API client, maps the response into a domain snapshot, and compares its fingerprint with the previous one. The first successful scan establishes a baseline; it does not send an alarming “everything was added” email.

A queue is justified here because the external call can be slow, rate-limited, or temporarily unavailable. It should not occupy a web request or prevent scans for other sites. The resulting project has these important files:

  • app/Services/TechnologyDetectorClient.php for HTTP and failure classification
  • app/Domain/TechnologySnapshot.php for defensive mapping and fingerprints
  • app/Jobs/ScanClientTechnology.php for comparison and persistence
  • app/Notifications/TechnologyStackChanged.php for developer email
  • app/Console/Commands/ScanClientSites.php for scheduled dispatch

Persist the baseline and audit snapshot

Create a model and migration with php artisan make:model ClientSite -m. The table keeps the latest normalized response, not the token or arbitrary request headers:

<?php
// database/migrations/xxxx_xx_xx_create_client_sites_table.php

use Illuminate\Database\Migrations\Migration;
use Illuminate\Database\Schema\Blueprint;
use Illuminate\Support\Facades\Schema;

return new class extends Migration {
    public function up(): void
    {
        Schema::create('client_sites', function (Blueprint $table): void {
            $table->id();
            $table->string('name');
            $table->string('url', 2048)->unique();
            $table->string('notification_email');
            $table->boolean('enabled')->default(true);
            $table->string('last_fingerprint', 64)->nullable();
            $table->json('last_snapshot')->nullable();
            $table->timestamp('last_checked_at')->nullable();
            $table->timestamps();
        });
    }

    public function down(): void
    {
        Schema::dropIfExists('client_sites');
    }
};

// app/Models/ClientSite.php

namespace App\Models;

use Illuminate\Database\Eloquent\Model;

class ClientSite extends Model
{
    protected $fillable = [
        'name', 'url', 'notification_email', 'enabled',
    ];

    protected function casts(): array
    {
        return [
            'enabled' => 'boolean',
            'last_snapshot' => 'array',
            'last_checked_at' => 'immutable_datetime',
        ];
    }
}

Build a bounded, failure-aware API client

The client makes at most two immediate attempts. It retries connection failures, HTTP 429, and server-side 5xx responses, but never retries authentication or validation failures. Queue-level backoff later provides slower recovery without hammering the service.

<?php
// app/Services/TechnologyDetectorClient.php

namespace App\Services;

use Illuminate\Http\Client\ConnectionException;
use Illuminate\Support\Facades\Http;
use RuntimeException;
use Throwable;

class DetectorException extends RuntimeException
{
    public function __construct(
        public readonly string $kind,
        public readonly bool $retryable,
        public readonly ?int $status = null,
        ?Throwable $previous = null,
    ) {
        parent::__construct("Technology detector failure: {$kind}", 0, $previous);
    }
}

class TechnologyDetectorClient
{
    public function detect(string $url): array
    {
        $endpoint = (string) config('services.technology_detector.url');
        $token = (string) config('services.technology_detector.token');

        if ($token === '') {
            throw new DetectorException('missing_configuration', false);
        }

        for ($attempt = 1; $attempt <= 2; $attempt++) {
            try {
                $response = Http::acceptJson()
                    ->withToken($token)
                    ->connectTimeout((int) config(
                        'services.technology_detector.connect_timeout', 3
                    ))
                    ->timeout((int) config(
                        'services.technology_detector.timeout', 12
                    ))
                    ->post($endpoint, ['url' => $url]);
            } catch (ConnectionException $exception) {
                if ($attempt === 2) {
                    throw new DetectorException(
                        'connection', true, null, $exception
                    );
                }

                usleep(250_000);
                continue;
            }

            if ($response->successful()) {
                $payload = $response->json();

                if (! is_array($payload)) {
                    throw new DetectorException('invalid_json_shape', false);
                }

                return $payload;
            }

            $status = $response->status();
            $retryable = $status === 429 || $status >= 500;

            if ($retryable && $attempt === 1) {
                $retryAfter = ctype_digit(
                    (string) $response->header('Retry-After')
                )
                    ? (int) $response->header('Retry-After')
                    : 1;

                sleep(min(max($retryAfter, 1), 5));
                continue;
            }

            throw new DetectorException(
                match ($status) {
                    401, 403 => 'authentication',
                    422 => 'invalid_request',
                    429 => 'rate_limited',
                    default => 'http_error',
                },
                $retryable,
                $status,
            );
        }

        throw new DetectorException('unexpected_state', false);
    }
}

Notice what is absent from exceptions and logs: the token, response body, and full headers. Remote error bodies can contain reflected input or implementation details, so status and a controlled failure category are safer observability fields.

Map an uncertain response at the boundary

Response envelopes can evolve. The mapper recursively locates the contract’s detection and redirect collections, validates individual entries, and rejects a response that contains no recognizable detection collection. That last rule is important: treating a schema change as an empty stack would generate a false “all technologies removed” alert.

<?php
// app/Domain/TechnologySnapshot.php

namespace App\Domain;

use App\Services\DetectorException;

final readonly class TechnologySnapshot
{
    public function __construct(
        public array $technologies,
        public array $redirects,
    ) {}

    public static function fromPayload(array $payload): self
    {
        $rows = self::findCollection($payload, ['detections', 'technologies']);

        if ($rows === null) {
            throw new DetectorException('unsupported_response_schema', false);
        }

        $technologies = [];

        foreach ($rows as $row) {
            if (! is_array($row)) {
                continue;
            }

            $name = $row['name'] ?? $row['technology'] ?? null;
            $confidence = $row['confidence'] ?? null;

            if (! is_string($name) || $name === '' || ! is_numeric($confidence)) {
                continue;
            }

            $versions = $row['versions'] ?? ($row['version'] ?? []);
            $versions = is_array($versions) ? $versions : [$versions];

            $technologies[] = [
                'name' => $name,
                'confidence' => (float) $confidence,
                'versions' => array_values(array_filter(
                    $versions,
                    static fn (mixed $value): bool => is_string($value)
                )),
                'evidence' => is_array($row['evidence'] ?? null)
                    ? $row['evidence']
                    : [],
            ];
        }

        usort(
            $technologies,
            static fn (array $a, array $b): int =>
                strcasecmp($a['name'], $b['name'])
        );

        return new self(
            $technologies,
            self::findCollection($payload, ['redirects']) ?? [],
        );
    }

    public function fingerprint(): string
    {
        $stable = array_map(
            static fn (array $item): array => [
                'name' => mb_strtolower($item['name']),
                'versions' => $item['versions'],
            ],
            $this->technologies,
        );

        return hash(
            'sha256',
            json_encode($stable, JSON_THROW_ON_ERROR | JSON_UNESCAPED_SLASHES)
        );
    }

    public function toArray(): array
    {
        return [
            'technologies' => $this->technologies,
            'redirects' => $this->redirects,
        ];
    }

    private static function findCollection(
        array $node,
        array $acceptedKeys
    ): ?array {
        foreach ($acceptedKeys as $key) {
            if (array_key_exists($key, $node) && is_array($node[$key])) {
                return $node[$key];
            }
        }

        foreach ($node as $value) {
            if (is_array($value)) {
                $found = self::findCollection($value, $acceptedKeys);

                if ($found !== null) {
                    return $found;
                }
            }
        }

        return null;
    }
}

The two accepted collection names isolate response-shape compatibility in one file. Compare them with the current official documentation when integrating. If the documented envelope differs, change only this boundary mapper rather than spreading array offsets through jobs and controllers.

Compare snapshots in a unique queue job

The job performs the network request before opening a transaction. It then locks the row, preventing two workers from comparing against and overwriting the same baseline concurrently. Notification happens after the transaction commits.

<?php
// app/Jobs/ScanClientTechnology.php

namespace App\Jobs;

use App\Domain\TechnologySnapshot;
use App\Models\ClientSite;
use App\Notifications\TechnologyStackChanged;
use App\Services\DetectorException;
use App\Services\TechnologyDetectorClient;
use Illuminate\Contracts\Queue\ShouldBeUnique;
use Illuminate\Contracts\Queue\ShouldQueue;
use Illuminate\Foundation\Queue\Queueable;
use Illuminate\Support\Facades\DB;
use Illuminate\Support\Facades\Log;
use Illuminate\Support\Facades\Notification;
use Throwable;

class ScanClientTechnology implements ShouldQueue, ShouldBeUnique
{
    use Queueable;

    public int $tries = 3;
    public int $uniqueFor = 3600;

    public function __construct(public readonly int $siteId) {}

    public function uniqueId(): string
    {
        return (string) $this->siteId;
    }

    public function backoff(): array
    {
        return [60, 300];
    }

    public function handle(TechnologyDetectorClient $client): void
    {
        $site = ClientSite::query()->findOrFail($this->siteId);

        try {
            $snapshot = TechnologySnapshot::fromPayload(
                $client->detect($site->url)
            );
        } catch (DetectorException $exception) {
            Log::warning('technology_scan_failed', [
                'site_id' => $site->id,
                'kind' => $exception->kind,
                'status' => $exception->status,
                'retryable' => $exception->retryable,
            ]);

            if ($exception->retryable) {
                throw $exception;
            }

            return;
        }

        $change = DB::transaction(function () use ($snapshot): ?array {
            $locked = ClientSite::query()
                ->lockForUpdate()
                ->findOrFail($this->siteId);

            $previous = $locked->last_snapshot;
            $fingerprint = $snapshot->fingerprint();

            $locked->update([
                'last_snapshot' => $snapshot->toArray(),
                'last_fingerprint' => $fingerprint,
                'last_checked_at' => now(),
            ]);

            if ($previous === null || $locked->getOriginal(
                'last_fingerprint'
            ) === $fingerprint) {
                return null;
            }

            return [
                'before' => $previous['technologies'] ?? [],
                'after' => $snapshot->technologies,
            ];
        });

        if ($change !== null) {
            Notification::route('mail', $site->notification_email)
                ->notify(new TechnologyStackChanged(
                    $site->name,
                    $site->url,
                    $change['before'],
                    $change['after'],
                ));
        }

        Log::info('technology_scan_completed', [
            'site_id' => $site->id,
            'technology_count' => count($snapshot->technologies),
            'changed' => $change !== null,
        ]);
    }

    public function failed(?Throwable $exception): void
    {
        Log::error('technology_scan_exhausted', [
            'site_id' => $this->siteId,
            'exception' => $exception?->getMessage(),
        ]);
    }
}

Create TechnologyStackChanged with php artisan make:notification TechnologyStackChanged. Keep the email concise and include both normalized lists:

<?php
// app/Notifications/TechnologyStackChanged.php

namespace App\Notifications;

use Illuminate\Bus\Queueable;
use Illuminate\Contracts\Queue\ShouldQueue;
use Illuminate\Notifications\Messages\MailMessage;
use Illuminate\Notifications\Notification;

class TechnologyStackChanged extends Notification implements ShouldQueue
{
    use Queueable;

    public function __construct(
        private readonly string $siteName,
        private readonly string $url,
        private readonly array $before,
        private readonly array $after,
    ) {}

    public function via(object $notifiable): array
    {
        return ['mail'];
    }

    public function toMail(object $notifiable): MailMessage
    {
        $render = static fn (array $items): string => collect($items)
            ->map(fn (array $item): string =>
                $item['name'].' '.implode(', ', $item['versions'] ?? [])
            )
            ->implode('; ');

        return (new MailMessage)
            ->subject("Technology change: {$this->siteName}")
            ->line("The public technology stack changed for {$this->url}.")
            ->line('Previous: '.($render($this->before) ?: 'None detected'))
            ->line('Current: '.($render($this->after) ?: 'None detected'))
            ->line('Review the stored evidence before treating this as an incident.');
    }
}

Dispatch scans from Laravel’s scheduler

The command uses chunking so the scheduler does not load every customer record into memory:

<?php
// app/Console/Commands/ScanClientSites.php

namespace App\Console\Commands;

use App\Jobs\ScanClientTechnology;
use App\Models\ClientSite;
use Illuminate\Console\Command;

class ScanClientSites extends Command
{
    protected $signature = 'clients:scan-technologies';
    protected $description = 'Queue technology scans for enabled client sites';

    public function handle(): int
    {
        ClientSite::query()
            ->where('enabled', true)
            ->select('id')
            ->chunkById(100, function ($sites): void {
                foreach ($sites as $site) {
                    ScanClientTechnology::dispatch($site->id);
                }
            });

        return self::SUCCESS;
    }
}

// routes/console.php

use Illuminate\Support\Facades\Schedule;

Schedule::command('clients:scan-technologies')
    ->hourly()
    ->withoutOverlapping();

Hourly checks are a starting point, not a promise. Match frequency to plan quota and client importance. The application needs a continuously supervised queue worker and one scheduler invocation each minute, typically php artisan schedule:run from cron.

Test the real behavior without calling the service

Laravel’s Http::fake() provides deterministic external responses. This feature test proves that the first scan becomes a baseline and a subsequent version change sends an on-demand notification:

<?php
// tests/Feature/TechnologyMonitorTest.php

namespace Tests\Feature;

use App\Jobs\ScanClientTechnology;
use App\Models\ClientSite;
use App\Notifications\TechnologyStackChanged;
use Illuminate\Foundation\Testing\RefreshDatabase;
use Illuminate\Support\Facades\Http;
use Illuminate\Support\Facades\Notification;
use Tests\TestCase;

class TechnologyMonitorTest extends TestCase
{
    use RefreshDatabase;

    public function test_it_alerts_only_after_the_baseline_changes(): void
    {
        Notification::fake();

        Http::fakeSequence()
            ->push(['detections' => [[
                'name' => 'Laravel',
                'confidence' => 0.98,
                'versions' => ['11'],
                'evidence' => ['public signal'],
            ]]], 200)
            ->push(['detections' => [[
                'name' => 'Laravel',
                'confidence' => 0.97,
                'versions' => ['12'],
                'evidence' => ['public signal'],
            ]]], 200);

        $site = ClientSite::create([
            'name' => 'Important client',
            'url' => 'https://client.example',
            'notification_email' => '[email protected]',
        ]);

        app()->call([new ScanClientTechnology($site->id), 'handle']);
        Notification::assertNothingSent();

        app()->call([new ScanClientTechnology($site->id), 'handle']);
        Notification::assertSentOnDemand(TechnologyStackChanged::class);

        Http::assertSent(fn ($request): bool =>
            $request->method() === 'POST'
            && $request->url() === config('services.technology_detector.url')
            && $request->hasHeader('Authorization')
            && $request['url'] === 'https://client.example'
        );
    }
}

Add focused tests for 401 without retry, a 429 followed by success, malformed JSON, missing detection collections, empty valid collections, and reordered detections producing the same fingerprint. Never put the real service token in fixtures or continuous-integration variables unless a separate, explicitly controlled integration test requires it.

Security, deployment, and operational traps

Restrict monitored URLs to administrator-managed public http or https destinations. Reject URLs containing credentials, and do not accept arbitrary scan targets from an unauthenticated form. Although the detector examines public sites, target selection can still consume quota or reveal business-sensitive monitoring choices.

On deployment, run php artisan migrate --force, then php artisan config:cache. Restart queue workers after configuration or code changes with php artisan queue:restart. Ensure the cache driver supports distributed locks if multiple workers rely on ShouldBeUnique, and configure failed-job storage so exhausted jobs are visible.

Common failures are usually diagnosable from the structured category:

  • Authentication: confirm the service-scoped token and remember that regeneration revoked the previous token.
  • Validation: check that the submitted value is a complete public URL and that the JSON property is exactly url.
  • Rate limiting: reduce scan frequency, inspect plan capacity, and respect bounded backoff rather than adding aggressive retries.
  • Unsupported schema: compare the official response documentation with the boundary mapper; do not overwrite a good baseline with guessed data.
  • No email: verify the queue worker, mail transport, failed jobs, and the configured recipient before blaming detection.

Final verification checklist

  • The token exists only in environment-backed secret configuration.
  • The exact POST endpoint succeeds with the minimal request.
  • The first queued scan stores a baseline without notifying anyone.
  • A confidence-only or evidence-only change leaves the fingerprint stable.
  • A technology or version change produces one developer notification.
  • 401, 403, and 422 responses are not retried blindly.
  • 429, connection, and 5xx failures use bounded retries and observable queue failure paths.
  • The production scheduler and queue worker are supervised.

A useful monitor is not the one that reports the most differences. It is the one developers trust enough to act on. By separating volatile evidence from stable technology identity, validating the API boundary, and making failure behavior explicit, this Laravel integration turns a public website fingerprint into a quiet early-warning system rather than another noisy inbox generator.

Портрет на автор на блогот

Mihajlo

Јас сум Михајло - развивач поттикнат од љубопитност, дисциплина и постојаната желба да создадам нешто значајно. Споделувам увиди, упатства и бесплатни услуги за да им помогнам на другите да ја поедностават својата работа и да растат во постојано развивачкиот свет на софтверот и вештачката интелигенција.