A lead’s website often reveals more than a sparse contact form does. Its public technology choices can hint at the work an agency might reasonably propose: a framework upgrade, analytics cleanup, performance improvements, or a CMS migration.

This tutorial adds that context to a small Laravel CRM. When a lead is created or explicitly rescanned, Laravel queues a background job, calls the Website Technology Detector API, validates the response at the application boundary, and stores a concise summary such as “Laravel, Nginx, and Google Analytics” alongside the lead.

The design deliberately keeps remote detection outside the request-response path. A slow upstream service should delay enrichment, not prevent a salesperson from saving a lead.

Prerequisites and project shape

You will need PHP 8.3 or later, a supported Laravel application with its normal queue and database infrastructure, and a Lead model containing a public website URL. The examples assume that attribute is named website_url.

The integration has four responsibilities:

• Laravel’s HTTP client authenticates, applies strict timeouts, and classifies upstream failures.
• A domain mapper converts confidence-scored detections, versions, evidence, and redirects into controlled application data.
• A queue job performs enrichment without slowing lead creation.
• The database stores the readable summary and operational state, but not the complete upstream payload.

This is intentionally modest architecture. A separate microservice would add deployment and tracing overhead without helping a typical agency CRM. A dedicated API client and mapper provide the boundary we need while remaining easy to test.

Get access before writing integration code

1. Open the registration page and create an account, or use the sign-in page if you already have one.
2. Open the Website Technology Detector service page.
3. Choose an available Free, Plus, or Pro plan and complete its activation.
4. Open the official service documentation.
5. Find the Service token panel and copy the service-scoped token shown there.

Regenerating that token revokes the previously active token. Treat regeneration as a credential rotation: update every deployed environment promptly, then restart long-running workers so they reload configuration.

The exact API operation is POST https://ai.mihajlo.mk/api/website-technology-detector/v1/detect-technologies. It accepts JSON containing url. Authentication may use a Bearer token, an X-API-Token header, or a token query parameter. This project uses the Bearer form because it keeps the credential out of URLs, access logs, and copied request links.

Verify access with one minimal request:

curl --request POST \
  --url https://ai.mihajlo.mk/api/website-technology-detector/v1/detect-technologies \
  --header 'Authorization: Bearer YOUR_SERVICE_TOKEN' \
  --header 'Accept: application/json' \
  --header 'Content-Type: application/json' \
  --data '{"url":"https://example.com"}'

Now place the token in the project’s .env file. Never commit that file or copy a real token into fixtures, screenshots, logs, or source code.

WEBSITE_TECH_DETECTOR_TOKEN=YOUR_SERVICE_TOKEN
WEBSITE_TECH_CONNECT_TIMEOUT=3
WEBSITE_TECH_TIMEOUT=12

Expose those values through config/services.php. Reading env() only from configuration keeps the integration compatible with Laravel’s configuration cache.

'website_technology_detector' => [
    'endpoint' => 'https://ai.mihajlo.mk/api/website-technology-detector/v1/detect-technologies',
    'token' => env('WEBSITE_TECH_DETECTOR_TOKEN'),
    'connect_timeout' => (int) env('WEBSITE_TECH_CONNECT_TIMEOUT', 3),
    'timeout' => (int) env('WEBSITE_TECH_TIMEOUT', 12),
],

Persist useful state, not an opaque response

Add fields for the human-readable result, redirect information, processing state, failure classification, and scan time. Keeping the full response is usually unnecessary and can increase retention and schema-coupling risks.

<?php

use Illuminate\Database\Migrations\Migration;
use Illuminate\Database\Schema\Blueprint;
use Illuminate\Support\Facades\Schema;

return new class extends Migration {
    public function up(): void
    {
        Schema::table('leads', function (Blueprint $table): void {
            $table->text('technology_summary')->nullable();
            $table->json('technology_redirects')->nullable();
            $table->string('technology_scan_status', 24)->default('pending');
            $table->string('technology_scan_error', 64)->nullable();
            $table->timestamp('technology_scanned_at')->nullable();
        });
    }

    public function down(): void
    {
        Schema::table('leads', function (Blueprint $table): void {
            $table->dropColumn([
                'technology_summary',
                'technology_redirects',
                'technology_scan_status',
                'technology_scan_error',
                'technology_scanned_at',
            ]);
        });
    }
};

Add technology_redirects as an array cast and technology_scanned_at as a datetime cast on Lead. Run php artisan migrate after reviewing the generated SQL for your database.

Map the response at the domain boundary

Remote JSON is untrusted input even when it comes from a service you control. The mapper below requires a detections array, accepts either keyed or list-shaped detection records, ignores unusable entries, bounds displayed evidence, and preserves only scalar redirect details.

<?php

namespace App\Domain\Leads;

use UnexpectedValueException;

final readonly class TechnologyReport
{
    public function __construct(
        public string $summary,
        public array $redirects,
        public int $detectionCount,
    ) {}

    public static function fromPayload(array $payload): self
    {
        $detections = $payload['detections'] ?? null;

        if (! is_array($detections)) {
            throw new UnexpectedValueException('Missing detections array.');
        }

        $parts = [];

        foreach ($detections as $key => $detection) {
            if (is_string($detection)) {
                $parts[] = trim($detection);
                continue;
            }

            if (! is_array($detection)) {
                continue;
            }

            $name = is_string($detection['name'] ?? null)
                ? trim($detection['name'])
                : (is_string($key) ? trim($key) : '');

            if ($name === '') {
                continue;
            }

            $details = [];

            if (is_numeric($detection['confidence'] ?? null)) {
                $score = (float) $detection['confidence'];
                $percentage = $score <= 1 ? $score * 100 : $score;
                $details[] = round(max(0, min(100, $percentage))).'% confidence';
            }

            $versions = self::scalarStrings($detection['versions'] ?? []);
            if ($versions !== []) {
                $details[] = 'version '.implode(', ', array_slice($versions, 0, 3));
            }

            $evidence = self::scalarStrings($detection['evidence'] ?? []);
            if ($evidence !== []) {
                $details[] = 'evidence: '.implode(', ', array_slice($evidence, 0, 3));
            }

            $parts[] = $details === []
                ? $name
                : $name.' ('.implode('; ', $details).')';
        }

        $redirects = self::scalarStrings($payload['redirects'] ?? []);

        return new self(
            summary: $parts === []
                ? 'No public website technologies were detected.'
                : implode('; ', $parts),
            redirects: array_slice($redirects, 0, 20),
            detectionCount: count($parts),
        );
    }

    private static function scalarStrings(mixed $value): array
    {
        if (is_scalar($value)) {
            $text = trim((string) $value);
            return $text === '' ? [] : [mb_substr($text, 0, 300)];
        }

        if (! is_array($value)) {
            return [];
        }

        $result = [];
        array_walk_recursive($value, function (mixed $item) use (&$result): void {
            if (is_scalar($item)) {
                $text = trim((string) $item);
                if ($text !== '') {
                    $result[] = mb_substr($text, 0, 300);
                }
            }
        });

        return array_values(array_unique($result));
    }
}

Confidence values are normalized whether represented as zero-to-one scores or percentages. More importantly, unexpected structures never flow directly into CRM views or database columns.

Build a failure-aware API client

The client distinguishes permanent caller errors from retryable upstream conditions. Authentication and validation failures are not blindly retried. Network errors and server failures receive one short immediate retry; later recovery belongs to the queue, where waiting does not occupy a web request.

<?php

namespace App\Services;

use App\Domain\Leads\TechnologyReport;
use Illuminate\Http\Client\ConnectionException;
use Illuminate\Support\Facades\Http;
use Throwable;

final class WebsiteTechnologyDetector
{
    public function detect(string $url): DetectionOutcome
    {
        $token = config('services.website_technology_detector.token');

        if (! is_string($token) || $token === '') {
            return DetectionOutcome::failure('configuration', false);
        }

        for ($attempt = 1; $attempt <= 2; $attempt++) {
            try {
                $response = Http::acceptJson()
                    ->withToken($token)
                    ->connectTimeout(config('services.website_technology_detector.connect_timeout'))
                    ->timeout(config('services.website_technology_detector.timeout'))
                    ->post(
                        config('services.website_technology_detector.endpoint'),
                        ['url' => $url],
                    );
            } catch (ConnectionException) {
                if ($attempt === 1) {
                    usleep(200_000);
                    continue;
                }

                return DetectionOutcome::failure('connection', true);
            }

            if (in_array($response->status(), [401, 403], true)) {
                return DetectionOutcome::failure('authentication', false, $response->status());
            }

            if ($response->status() === 422) {
                return DetectionOutcome::failure('invalid_request', false, 422);
            }

            if ($response->status() === 429) {
                $delay = filter_var($response->header('Retry-After'), FILTER_VALIDATE_INT);
                return DetectionOutcome::failure(
                    'rate_limited',
                    true,
                    429,
                    max(10, min(300, $delay ?: 60)),
                );
            }

            if ($response->serverError()) {
                if ($attempt === 1) {
                    usleep(500_000);
                    continue;
                }

                return DetectionOutcome::failure('upstream', true, $response->status());
            }

            if ($response->clientError()) {
                return DetectionOutcome::failure('request_rejected', false, $response->status());
            }

            try {
                $payload = $response->json();

                if (! is_array($payload)) {
                    return DetectionOutcome::failure('invalid_payload', false);
                }

                return DetectionOutcome::success(
                    TechnologyReport::fromPayload($payload),
                );
            } catch (Throwable) {
                return DetectionOutcome::failure('invalid_payload', false);
            }
        }

        return DetectionOutcome::failure('unknown', true);
    }
}

final readonly class DetectionOutcome
{
    private function __construct(
        public ?TechnologyReport $report,
        public ?string $error,
        public bool $retryable,
        public ?int $status,
        public int $retryAfter,
    ) {}

    public static function success(TechnologyReport $report): self
    {
        return new self($report, null, false, 200, 0);
    }

    public static function failure(
        string $error,
        bool $retryable,
        ?int $status = null,
        int $retryAfter = 30,
    ): self {
        return new self(null, $error, $retryable, $status, $retryAfter);
    }
}

Enrich leads in a queue job

Validate URLs before saving leads with Laravel’s url rule and allow only public http or https destinations according to your CRM policy. The detector is intended for public websites; do not use it as a path to internal hosts, loopback addresses, or cloud metadata endpoints.

The job performs no database transaction around the network call. It also logs identifiers and classifications rather than tokens, response bodies, or complete URLs.

<?php

namespace App\Jobs;

use App\Models\Lead;
use App\Services\WebsiteTechnologyDetector;
use Illuminate\Contracts\Queue\ShouldQueue;
use Illuminate\Foundation\Queue\Queueable;
use Illuminate\Support\Facades\Log;

final class DetectLeadTechnologies implements ShouldQueue
{
    use Queueable;

    public int $tries = 4;

    public function __construct(public readonly int $leadId) {}

    public function handle(WebsiteTechnologyDetector $detector): void
    {
        $lead = Lead::find($this->leadId);

        if (! $lead || ! is_string($lead->website_url) || $lead->website_url === '') {
            return;
        }

        $started = hrtime(true);
        $outcome = $detector->detect($lead->website_url);

        if ($outcome->report) {
            $lead->forceFill([
                'technology_summary' => $outcome->report->summary,
                'technology_redirects' => $outcome->report->redirects,
                'technology_scan_status' => 'complete',
                'technology_scan_error' => null,
                'technology_scanned_at' => now(),
            ])->save();

            Log::info('Lead technology scan completed', [
                'lead_id' => $lead->id,
                'detections' => $outcome->report->detectionCount,
                'duration_ms' => (int) ((hrtime(true) - $started) / 1_000_000),
            ]);

            return;
        }

        if ($outcome->retryable && $this->attempts() < $this->tries) {
            Log::warning('Lead technology scan will retry', [
                'lead_id' => $lead->id,
                'failure' => $outcome->error,
                'http_status' => $outcome->status,
            ]);

            $this->release($outcome->retryAfter);
            return;
        }

        $lead->forceFill([
            'technology_scan_status' => 'failed',
            'technology_scan_error' => $outcome->error,
            'technology_scanned_at' => now(),
        ])->save();
    }
}

Dispatch the job after a successful lead insert, preferably after the surrounding database transaction commits:

$lead = Lead::create($validated);

DetectLeadTechnologies::dispatch($lead->id)->afterCommit();

For an explicit rescan action, protect the route with authentication and throttling:

Route::post('/leads/{lead}/technology-scan', function (Lead $lead) {
    $lead->update([
        'technology_scan_status' => 'pending',
        'technology_scan_error' => null,
    ]);

    DetectLeadTechnologies::dispatch($lead->id)->afterCommit();

    return back();
})->middleware(['auth', 'throttle:10,1']);

In a larger application, move this closure into an authorized controller action. Throttling protects accidental repeated clicks, while authorization must ensure the current user may modify that lead.

Test success and failure without calling the service

Http::fake() makes the tests deterministic and verifies the real request boundary. Use the configured endpoint so a later URL change cannot silently invalidate the assertion.

<?php

use App\Jobs\DetectLeadTechnologies;
use App\Models\Lead;
use Illuminate\Support\Facades\Http;

it('stores a readable technology report', function () {
    config(['services.website_technology_detector.token' => 'test-token']);

    Http::fake([
        config('services.website_technology_detector.endpoint') => Http::response([
            'detections' => [
                [
                    'name' => 'Laravel',
                    'confidence' => 0.98,
                    'versions' => ['11'],
                    'evidence' => ['response signature'],
                ],
            ],
            'redirects' => ['https://www.example.com'],
        ], 200),
    ]);

    $lead = Lead::factory()->create([
        'website_url' => 'https://example.com',
    ]);

    app(DetectLeadTechnologies::class, ['leadId' => $lead->id])
        ->handle(app(\App\Services\WebsiteTechnologyDetector::class));

    expect($lead->fresh()->technology_summary)
        ->toContain('Laravel')
        ->and($lead->fresh()->technology_scan_status)->toBe('complete');

    Http::assertSent(fn ($request) =>
        $request->method() === 'POST'
        && $request->url() === config('services.website_technology_detector.endpoint')
        && $request['url'] === 'https://example.com'
        && $request->hasHeader('Authorization', 'Bearer test-token')
    );
});

it('classifies authentication failures without retrying', function () {
    config(['services.website_technology_detector.token' => 'expired-token']);

    Http::fake([
        '*' => Http::response([], 401),
    ]);

    $outcome = app(\App\Services\WebsiteTechnologyDetector::class)
        ->detect('https://example.com');

    expect($outcome->error)->toBe('authentication')
        ->and($outcome->retryable)->toBeFalse();

    Http::assertSentCount(1);
});

Add companion cases for malformed JSON, an absent detections array, empty detections, HTTP 429, server errors, and connection exceptions. Those tests matter more than another happy-path fixture because they protect queue capacity and failure semantics.

Deployment, observability, and common failures

Set the token and timeout values in each deployment environment, run migrations, rebuild configuration with php artisan config:cache, and restart queue workers with php artisan queue:restart. Ensure a worker is processing the queue used by the job.

Monitor completion, failure, retry, duration, and detection-count fields from the structured logs. Alert on sustained authentication failures because they usually indicate a missing, revoked, or stale token. Track rate limiting separately; it may indicate a burst that needs queue pacing or a plan capacity review.

Common problems tend to be operational rather than algorithmic:

• Every scan remains pending: confirm the queue worker is running and that production is not using an unintended queue connection.
• Immediate authentication failures: verify the service-scoped token, refresh cached configuration, and restart workers. If the token was regenerated, the previous value no longer works.
• Validation failures: confirm the body is JSON and contains url with a complete public URL.
• Repeated rate limiting: respect the bounded retry delay, reduce concurrency, and review the activated plan instead of creating an aggressive retry loop.
• Invalid payload failures: compare the response with the official documentation, then adjust only the mapper. Do not spread remote response assumptions through controllers and views.
• Unexpectedly slow workers: retain bounded connection and total timeouts, and examine upstream latency before increasing them.

Final verification checklist

• The exact detector endpoint receives a JSON POST containing url.
• The Bearer token comes from environment-backed configuration and never enters logs or source control.
• Lead creation succeeds even when enrichment is unavailable.
• Successful scans store a readable summary, redirects, completion state, and timestamp.
• Authentication and validation errors do not retry.
• Rate limits, network failures, and server errors use bounded retries.
• Tests fake every external call and cover both mapping and failure classification.
• Production workers were restarted after configuration deployment.

A useful CRM does not merely collect fields; it turns available signals into context people can act on. By keeping detection asynchronous, credentials isolated, responses defensively mapped, and failures visible, this integration adds that context without making the lead workflow dependent on a remote service. The summary is the visible feature, but the carefully designed boundary is what makes it production-ready.