Symfony: Automatically Assess Client Website Tech for Smarter Redesign Quotes
A redesign quote becomes risky when the visible website tells only half the story. A polished homepage might sit on an aging CMS, depend on a page builder, load several analytics products, or redirect through infrastructure that must be preserved during migration. Discovering those details after pricing the work usually means revising the estimate or absorbing unplanned effort.
This tutorial builds a production-oriented Symfony feature that submits a client’s public URL to the Website Technology Detector API, converts its confidence-scored results into a stable domain model, and exposes an internal endpoint for quote preparation. The boundary is deliberately strict: upstream response data is treated as untrusted, transient failures receive bounded retries, and logs contain operational context without leaking credentials or sensitive URL components.
Get access and copy the service token
- Create an account at https://ai.mihajlo.mk/register, or use https://ai.mihajlo.mk/login if you already have one.
- Open the Website Technology Detector service page.
- Choose the available Free, Plus, or Pro plan and complete its activation.
- Open the official service documentation.
- Find the Service token panel and copy the service-scoped token shown there.
This service requires authentication. It accepts a Bearer token, an X-API-Token header, or a token query parameter. The implementation below uses a Bearer token because query parameters are more likely to appear in proxy, access, and browser-history logs.
Regenerating the service token revokes the previously active token. Treat regeneration as a credential rotation: update every deployed environment that uses the old value, deploy or restart those instances, and verify them before assuming the rotation is complete.
The exact API operation is:
POST https://ai.mihajlo.mk/api/website-technology-detector/v1/detect-technologiesBefore writing application code, verify the account and token with one minimal request:
curl --request POST \
--url https://ai.mihajlo.mk/api/website-technology-detector/v1/detect-technologies \
--header 'Authorization: Bearer YOUR_SERVICE_TOKEN' \
--header 'Accept: application/json' \
--header 'Content-Type: application/json' \
--data '{"url":"https://example.com"}'Never commit the real token. Put the endpoint in .env and the local credential in the ignored .env.local file. Production should inject the token through the hosting platform’s secret manager or environment configuration.
# .env
WEBSITE_TECHNOLOGY_DETECTOR_ENDPOINT=https://ai.mihajlo.mk/api/website-technology-detector/v1/detect-technologies
# .env.local — do not commit
WEBSITE_TECHNOLOGY_DETECTOR_TOKEN=YOUR_SERVICE_TOKENProject shape and trade-offs
The example requires PHP 8.3 or newer, Composer, a working Symfony application, and the JSON extension. Install Symfony’s HTTP client and, if the project does not already provide PSR-3 logging, MonologBundle:
composer require symfony/http-client symfony/monolog-bundle
composer require --dev symfony/test-packThe resulting request path is intentionally small: a controller validates the submitted URL, a dedicated gateway owns the remote HTTP contract, and a mapper converts the external payload into application DTOs. Quote-related code consumes those DTOs rather than depending on arbitrary API arrays.
The assessment remains synchronous because a salesperson or developer normally requests it while preparing a quote and expects an immediate result. Messenger would be justified for bulk portfolio scans, scheduled refreshes, or workflows that can continue later. Adding it to a single interactive lookup would introduce queue operations and state management without improving the core outcome.
The relevant structure is:
src/
Controller/TechnologyAssessmentController.php
Technology/Detection.php
Technology/DetectionReport.php
Technology/DetectionReportMapper.php
Technology/DetectorFailure.php
Technology/WebsiteTechnologyDetector.php
tests/
Technology/WebsiteTechnologyDetectorTest.php
config/
services.yamlCreate a defensive domain boundary
Technology detections are useful only when their uncertainty and evidence survive the mapping process. Do not flatten a result into a list of product names. The confidence value helps reviewers distinguish a strong signature from a tentative match, while evidence and version information explain why a migration task belongs in the quote.
The mapper below accepts only values it can safely represent. It tolerates detections keyed by technology name, ignores malformed entries, preserves evidence without guessing its internal shape, and keeps redirect information separate. An empty detection list remains valid: “nothing recognized” is different from “the request failed.”
<?php
// src/Technology/Detection.php
namespace App\Technology;
final readonly class Detection implements \JsonSerializable
{
public function __construct(
public string $technology,
public ?float $confidence,
public array $evidence,
public array $versions,
) {}
public function jsonSerialize(): array
{
return [
'technology' => $this->technology,
'confidence' => $this->confidence,
'evidence' => $this->evidence,
'versions' => $this->versions,
];
}
}
// src/Technology/DetectionReport.php
namespace App\Technology;
final readonly class DetectionReport implements \JsonSerializable
{
/** @param list<Detection> $detections */
public function __construct(
public array $detections,
public array $redirects,
) {}
public function jsonSerialize(): array
{
return [
'detections' => $this->detections,
'redirects' => $this->redirects,
];
}
}
// src/Technology/DetectionReportMapper.php
namespace App\Technology;
final class DetectionReportMapper
{
public function map(array $payload): DetectionReport
{
$source = isset($payload['data']) && is_array($payload['data'])
? $payload['data']
: $payload;
$rows = is_array($source['detections'] ?? null)
? $source['detections']
: [];
$detections = [];
foreach ($rows as $key => $row) {
if (!is_array($row)) {
continue;
}
$candidate = $row['technology'] ?? $row['name'] ?? null;
$name = is_string($candidate) && trim($candidate) !== ''
? trim($candidate)
: (is_string($key) ? trim($key) : '');
if ($name === '') {
continue;
}
$confidence = is_int($row['confidence'] ?? null)
|| is_float($row['confidence'] ?? null)
? (float) $row['confidence']
: null;
$detections[] = new Detection(
$name,
$confidence,
$this->asArray($row['evidence'] ?? []),
$this->asArray($row['versions'] ?? []),
);
}
$redirects = is_array($source['redirects'] ?? null)
? $source['redirects']
: [];
return new DetectionReport($detections, $redirects);
}
private function asArray(mixed $value): array
{
if ($value === null || $value === '') {
return [];
}
return is_array($value) ? $value : [$value];
}
}The two accepted technology-name keys are boundary aliases, not assumptions that every successful response must contain both. If the documented payload evolves, this mapper is the single place to add a verified compatibility rule.
Build the resilient HTTP client
Remote detection involves network access and inspection of another public website, so latency and transient failure are normal operating conditions. The gateway uses separate connection and total-duration limits, retries only transport failures, rate limiting, and selected server errors, and applies bounded backoff. Authentication and validation failures are never retried.
<?php
// src/Technology/DetectorFailure.php
namespace App\Technology;
final class DetectorFailure extends \RuntimeException
{
public function __construct(
public readonly string $kind,
public readonly bool $retryable,
string $message,
public readonly ?int $upstreamStatus = null,
?\Throwable $previous = null,
) {
parent::__construct($message, 0, $previous);
}
}
// src/Technology/WebsiteTechnologyDetector.php
namespace App\Technology;
use Psr\Log\LoggerInterface;
use Symfony\Contracts\HttpClient\Exception\DecodingExceptionInterface;
use Symfony\Contracts\HttpClient\Exception\TransportExceptionInterface;
use Symfony\Contracts\HttpClient\HttpClientInterface;
final class WebsiteTechnologyDetector
{
private \Closure $sleep;
public function __construct(
private HttpClientInterface $http,
private DetectionReportMapper $mapper,
private LoggerInterface $logger,
private string $endpoint,
private string $token,
?\Closure $sleep = null,
) {
$this->sleep = $sleep ?? static fn (int $microseconds) => usleep($microseconds);
}
public function detect(string $url): DetectionReport
{
$host = parse_url($url, PHP_URL_HOST);
$lastError = null;
for ($attempt = 1; $attempt <= 3; $attempt++) {
try {
$response = $this->http->request('POST', $this->endpoint, [
'headers' => [
'Authorization' => 'Bearer '.$this->token,
'Accept' => 'application/json',
],
'json' => ['url' => $url],
'timeout' => 5.0,
'max_duration' => 20.0,
]);
$status = $response->getStatusCode();
if ($status >= 200 && $status < 300) {
try {
$report = $this->mapper->map($response->toArray(false));
} catch (DecodingExceptionInterface $error) {
throw new DetectorFailure(
'invalid_response',
false,
'The detector returned invalid JSON.',
$status,
$error,
);
}
$this->logger->info('Website technology detection completed.', [
'target_host' => $host,
'detection_count' => count($report->detections),
'attempt' => $attempt,
]);
return $report;
}
if (in_array($status, [401, 403], true)) {
throw new DetectorFailure(
'authentication_failed',
false,
'The detector rejected its service token.',
$status,
);
}
if (in_array($status, [400, 422], true)) {
throw new DetectorFailure(
'upstream_validation_failed',
false,
'The detector rejected the submitted URL.',
$status,
);
}
$retryable = $status === 429
|| in_array($status, [502, 503, 504], true);
if (!$retryable) {
throw new DetectorFailure(
'upstream_failure',
false,
'The detector request failed.',
$status,
);
}
$lastError = new DetectorFailure(
$status === 429 ? 'rate_limited' : 'upstream_unavailable',
true,
'The detector is temporarily unavailable.',
$status,
);
$delay = $this->retryDelay($response->getHeaders(false), $attempt);
} catch (TransportExceptionInterface $error) {
$lastError = new DetectorFailure(
'transport_failure',
true,
'The detector could not be reached.',
null,
$error,
);
$delay = 250_000 * (2 ** ($attempt - 1));
}
$this->logger->warning('Website technology detection will retry.', [
'target_host' => $host,
'attempt' => $attempt,
'reason' => $lastError?->kind,
]);
if ($attempt < 3) {
($this->sleep)(min($delay, 2_000_000));
}
}
throw $lastError ?? new DetectorFailure(
'unknown_failure',
false,
'Technology detection failed.',
);
}
private function retryDelay(array $headers, int $attempt): int
{
$retryAfter = $headers['retry-after'][0] ?? null;
if (is_string($retryAfter) && ctype_digit($retryAfter)) {
return min((int) $retryAfter, 2) * 1_000_000;
}
return 250_000 * (2 ** ($attempt - 1));
}
}The retry cap prevents a web request from occupying a worker indefinitely. A 429 may indicate a temporary rate limit or exhausted allowance; after the bounded attempts, the structured rate_limited state reaches the caller instead of being disguised as an empty assessment.
Wire scalar constructor arguments explicitly:
# config/services.yaml
services:
_defaults:
autowire: true
autoconfigure: true
App\:
resource: '../src/'
App\Technology\WebsiteTechnologyDetector:
arguments:
$endpoint: '%env(string:WEBSITE_TECHNOLOGY_DETECTOR_ENDPOINT)%'
$token: '%env(string:WEBSITE_TECHNOLOGY_DETECTOR_TOKEN)%'Expose the quote-assessment endpoint
The controller accepts JSON such as {"url":"https://client.example"}. It permits only public HTTP or HTTPS URLs, rejects embedded credentials, and blocks obvious local or private targets. The detector service is the system fetching the public site, but early validation still protects quota and keeps accidental internal-looking inputs out of the workflow.
<?php
// src/Controller/TechnologyAssessmentController.php
namespace App\Controller;
use App\Technology\DetectorFailure;
use App\Technology\WebsiteTechnologyDetector;
use Symfony\Bundle\FrameworkBundle\Controller\AbstractController;
use Symfony\Component\HttpFoundation\JsonResponse;
use Symfony\Component\HttpFoundation\Request;
use Symfony\Component\Routing\Attribute\Route;
final class TechnologyAssessmentController extends AbstractController
{
#[Route('/quote/technology-assessment', methods: ['POST'])]
public function __invoke(
Request $request,
WebsiteTechnologyDetector $detector,
): JsonResponse {
try {
$input = $request->toArray();
} catch (\JsonException) {
return $this->json(['error' => 'invalid_json'], 400);
}
$url = is_string($input['url'] ?? null) ? trim($input['url']) : '';
if (!$this->isAllowedPublicUrl($url)) {
return $this->json(['error' => 'invalid_public_url'], 422);
}
try {
return $this->json($detector->detect($url));
} catch (DetectorFailure $failure) {
$status = match ($failure->kind) {
'upstream_validation_failed' => 422,
'rate_limited' => 429,
default => 503,
};
return $this->json([
'error' => $failure->kind,
'retryable' => $failure->retryable,
], $status);
}
}
private function isAllowedPublicUrl(string $url): bool
{
if (filter_var($url, FILTER_VALIDATE_URL) === false) {
return false;
}
$parts = parse_url($url);
$scheme = strtolower((string) ($parts['scheme'] ?? ''));
$host = strtolower((string) ($parts['host'] ?? ''));
if (!in_array($scheme, ['http', 'https'], true)
|| $host === ''
|| isset($parts['user'])
|| isset($parts['pass'])
|| $host === 'localhost'
|| str_ends_with($host, '.local')) {
return false;
}
if (filter_var($host, FILTER_VALIDATE_IP) !== false) {
return filter_var(
$host,
FILTER_VALIDATE_IP,
FILTER_FLAG_NO_PRIV_RANGE | FILTER_FLAG_NO_RES_RANGE,
) !== false;
}
return true;
}
}Authentication for this internal route is application-specific and should be enforced before deployment. In a Symfony Security setup, restrict it to the staff role responsible for quotes. Public exposure would let anonymous users consume your service allowance.
Test success, mapping, and retry behavior
MockHttpClient makes the suite deterministic: no network request, real credential, or live quota is involved. This test verifies the exact method and endpoint, exercises defensive mapping, and proves that a temporary 429 is retried before success.
<?php
// tests/Technology/WebsiteTechnologyDetectorTest.php
namespace App\Tests\Technology;
use App\Technology\DetectionReportMapper;
use App\Technology\WebsiteTechnologyDetector;
use PHPUnit\Framework\TestCase;
use Psr\Log\NullLogger;
use Symfony\Component\HttpClient\MockHttpClient;
use Symfony\Component\HttpClient\Response\MockResponse;
final class WebsiteTechnologyDetectorTest extends TestCase
{
public function testMapsAValidDetection(): void
{
$client = new MockHttpClient(function (string $method, string $url): MockResponse {
self::assertSame('POST', $method);
self::assertSame('https://service.test/v1/detect-technologies', $url);
return new MockResponse(json_encode([
'detections' => [[
'technology' => 'Example CMS',
'confidence' => 0.94,
'evidence' => ['header signature'],
'versions' => ['6.x'],
]],
'redirects' => [
['from' => 'http://example.test', 'to' => 'https://example.test'],
],
], JSON_THROW_ON_ERROR), ['http_code' => 200]);
});
$detector = $this->detector($client);
$report = $detector->detect('https://example.test');
self::assertCount(1, $report->detections);
self::assertSame('Example CMS', $report->detections[0]->technology);
self::assertSame(0.94, $report->detections[0]->confidence);
self::assertCount(1, $report->redirects);
}
public function testRetriesRateLimitThenSucceeds(): void
{
$client = new MockHttpClient([
new MockResponse('{}', ['http_code' => 429]),
new MockResponse(
'{"detections":[],"redirects":[]}',
['http_code' => 200],
),
]);
$report = $this->detector($client)->detect('https://example.test');
self::assertSame([], $report->detections);
self::assertSame(2, $client->getRequestsCount());
}
private function detector(MockHttpClient $client): WebsiteTechnologyDetector
{
return new WebsiteTechnologyDetector(
$client,
new DetectionReportMapper(),
new NullLogger(),
'https://service.test/v1/detect-technologies',
'test-token',
static fn (int $microseconds) => null,
);
}
}Add companion tests for malformed JSON, three consecutive 429 responses, transport exceptions, and 401 without retry. Those failure-path tests are more valuable than increasing the number of happy-path fixtures.
Security, operations, and deployment
Keep the raw response out of logs unless a controlled debugging process explicitly requires it. Evidence can contain page fragments, and submitted URLs can carry client identifiers or query secrets. This implementation logs only the hostname, attempt number, outcome, and detection count. The Authorization header and token must never enter log context, exception messages, fixtures, or screenshots.
Monitor successful request count, latency, retries, 429 responses, upstream server failures, and invalid-response failures. A rising retry rate may reveal service trouble or insufficient plan capacity before staff experience persistent failures. Alerting should distinguish configuration errors such as rejected credentials from transient transport errors.
For deployment, inject WEBSITE_TECHNOLOGY_DETECTOR_TOKEN through the runtime secret store, run tests before releasing, and build the production cache with the same environment-variable names available. A conventional sequence is:
php bin/phpunit
composer install --no-dev --optimize-autoloader
APP_ENV=prod APP_DEBUG=0 php bin/console cache:clear
APP_ENV=prod php bin/console aboutIf the application uses long-running workers or persistent containers, restart them after rotating the token. Symfony may resolve configuration during container construction, so updating a secret outside the process does not guarantee that an already-running instance will observe it.
Common failures and final verification
- 401 or 403: confirm that the service-scoped token came from the documentation page’s Service token panel and has not been revoked by regeneration.
- 400 or 422: verify that the request is JSON and contains the
urlfield with an absolute public HTTP or HTTPS URL. - 429: stop aggressive manual retries, review current plan capacity, and preserve the structured failure so the quote workflow can be resumed later.
- Empty detections: treat this as a successful assessment with no recognized technologies, not as proof that the site has no dependencies.
- Intermittent 502, 503, 504, or transport errors: inspect retry and latency logs. Do not expand retries until a web worker can remain occupied indefinitely.
- Unexpected mapping omissions: compare the real payload with the official documentation, then update and test the boundary mapper rather than spreading response-key handling through quote code.
Before making the feature available to staff, verify the following:
- The real token exists only in environment-backed secret configuration.
- The minimal request succeeds against the exact documented POST endpoint.
- The internal route requires appropriate application authentication and authorization.
- Detections retain confidence, evidence, versions, and redirect information.
- Authentication and validation failures are not retried.
- Rate limits, selected server failures, and transport errors receive no more than three bounded attempts.
- Logs exclude tokens, full URLs, response bodies, and page evidence.
- Automated tests pass without contacting the live service.
- A real client site produces an assessment that can be reviewed alongside the redesign quote.
The practical payoff is not a decorative list of logos. It is a disciplined pre-quote checkpoint: detected technologies suggest migration work, confidence prevents overstatement, evidence supports human review, versions highlight upgrade risk, and redirects expose behavior that a redesign must preserve. When that information crosses a carefully designed Symfony boundary, the quote becomes easier to defend—and unpleasant technical surprises become much less likely.
