Symfony: AI однапред ги пополнува CRM потенцијалните клиенти со извлекување податоци за компании од веб-страници
A salesperson should not have to copy a company name, phone number, email address, and contact details from half a dozen pages before creating a lead. A website URL is already a useful identity signal. The practical engineering challenge is turning that signal into trustworthy suggestions without exposing credentials, blocking indefinitely, or allowing an upstream outage to break the CRM.
This tutorial builds a production-oriented Symfony endpoint for that job. The salesperson enters a company website, Symfony requests structured company data, maps the response at a strict application boundary, and returns a stable payload that the CRM form can use to prefill its lead fields. The user remains in control: enrichment proposes values; it does not silently save them.
Get access to the service
Start by registering at https://ai.mihajlo.mk/register. If you already have an account, sign in at https://ai.mihajlo.mk/login.
Open the Website to Company data service page at https://ai.mihajlo.mk/api/website-to-company-data. Choose the available Free, Plus, or Pro plan that fits your expected usage and complete its activation.
Next, visit the official documentation at https://ai.mihajlo.mk/api/website-to-company-data/documentation. Find the Service token panel and copy the service-scoped token shown there. This service is not tokenless: every extraction request requires that credential through the token query parameter.
Regenerating the service token revokes the previously active token. Treat rotation as a deployment operation: update the production secret, deploy or restart every application instance that reads it, verify the new credential, and only then retire any stale configuration.
Confirm the exact HTTP contract
The integration uses GET https://ai.mihajlo.mk/api/website-to-company-data/v1/extract. It sends exactly two query parameters: token for authentication and website for the public company website.
Run one minimal request with a placeholder before writing application code:
curl --get 'https://ai.mihajlo.mk/api/website-to-company-data/v1/extract' \
--data-urlencode 'token=YOUR_SERVICE_TOKEN' \
--data-urlencode 'website=https://example.com'
Replace the placeholder locally, but do not commit the resulting command, paste it into tickets, or leave the real token in shell history. The response supplies the contract fields company, contact, email, phone, and people. Their contents must be validated defensively rather than trusting an upstream response blindly.
Store the credential in environment configuration
Put the token in .env.local, which Symfony projects normally exclude from version control:
# .env.local
WEBSITE_COMPANY_DATA_TOKEN=YOUR_SERVICE_TOKEN
In production, inject the same variable through the hosting platform or secret manager. Do not bake it into a container image, source file, fixture, or frontend bundle.
Create the Symfony project
This implementation targets PHP 8.3 or newer and a supported Symfony application. For a new project, install the framework, HTTP client, logging integration, and test tools:
composer create-project symfony/skeleton crm-enrichment
cd crm-enrichment
composer require symfony/http-client symfony/monolog-bundle
composer require --dev symfony/test-pack
The relevant project structure remains deliberately small:
config/services.yaml
src/CompanyData/CompanyEnrichment.php
src/CompanyData/EnrichmentException.php
src/CompanyData/CompanyDataClient.php
src/Controller/LeadPrefillController.php
tests/CompanyData/CompanyDataClientTest.php
Choose a synchronous boundary, not a queue
Prefilling is interactive: the salesperson expects suggestions while the lead form is still open. A synchronous HTTP request therefore gives a simpler and more useful result than introducing Messenger, persistence, polling, and stale jobs.
That choice needs firm limits. The client below uses bounded connection and overall durations, makes no more than three attempts, and retries only transport failures, rate limiting, and selected transient server responses. Authentication and ordinary client errors are not retried because repeating the same invalid request wastes quota and delays useful feedback.
The browser never receives the service token. It calls a same-origin Symfony route, while the server owns authentication, response validation, retries, logging, and error translation.
Map the response at the application boundary
The service contract names five fields, but a resilient application should not assume undocumented nested keys. The domain object therefore accepts strings, arrays, or null for the general fields and requires people to be an array when present. A response containing none of the contract fields is treated as contract drift, not as an empty company.
<?php
// src/CompanyData/CompanyEnrichment.php
namespace App\CompanyData;
final readonly class CompanyEnrichment
{
public function __construct(
public string|array|null $company,
public string|array|null $contact,
public string|array|null $email,
public string|array|null $phone,
public array $people,
) {}
public static function fromPayload(array $payload): self
{
$keys = ['company', 'contact', 'email', 'phone', 'people'];
if (array_intersect($keys, array_keys($payload)) === []) {
throw new EnrichmentException('invalid_payload');
}
$people = $payload['people'] ?? [];
if (!is_array($people)) {
throw new EnrichmentException('invalid_payload');
}
return new self(
self::field($payload, 'company'),
self::field($payload, 'contact'),
self::field($payload, 'email'),
self::field($payload, 'phone'),
$people,
);
}
public function toArray(): array
{
return [
'company' => $this->company,
'contact' => $this->contact,
'email' => $this->email,
'phone' => $this->phone,
'people' => $this->people,
];
}
private static function field(array $payload, string $key): string|array|null
{
$value = $payload[$key] ?? null;
if ($value === null || is_string($value) || is_array($value)) {
return $value;
}
throw new EnrichmentException('invalid_payload');
}
}
Use a typed exception to keep upstream failure categories separate from customer-facing messages:
<?php
// src/CompanyData/EnrichmentException.php
namespace App\CompanyData;
final class EnrichmentException extends \RuntimeException
{
public function __construct(
public readonly string $kind,
public readonly ?int $retryAfterSeconds = null,
?\Throwable $previous = null,
) {
parent::__construct($kind, 0, $previous);
}
}
Build a bounded, observable HTTP client
Configure a dedicated transport and inject the token by argument name. Creating this transport without the framework logger is intentional: because authentication must appear in the query string, generic HTTP traces could otherwise capture the credential. The application client emits its own sanitized events.
# config/services.yaml
services:
_defaults:
autowire: true
autoconfigure: true
App\:
resource: '../src/'
app.company_data_http_client:
class: Symfony\Contracts\HttpClient\HttpClientInterface
factory: ['Symfony\Component\HttpClient\HttpClient', 'create']
arguments:
- { timeout: 3.0, max_duration: 8.0 }
App\CompanyData\CompanyDataClient:
arguments:
$http: '@app.company_data_http_client'
$serviceToken: '%env(WEBSITE_COMPANY_DATA_TOKEN)%'
The client logs only the destination hostname, attempt number, status, and failure category. It never logs the full request URI, token, or returned personal data.
<?php
// src/CompanyData/CompanyDataClient.php
namespace App\CompanyData;
use Psr\Log\LoggerInterface;
use Symfony\Contracts\HttpClient\Exception\DecodingExceptionInterface;
use Symfony\Contracts\HttpClient\Exception\TransportExceptionInterface;
use Symfony\Contracts\HttpClient\HttpClientInterface;
use Symfony\Contracts\HttpClient\ResponseInterface;
final class CompanyDataClient
{
private const ENDPOINT =
'https://ai.mihajlo.mk/api/website-to-company-data/v1/extract';
private readonly \Closure $sleep;
public function __construct(
private readonly HttpClientInterface $http,
private readonly string $serviceToken,
private readonly LoggerInterface $logger,
?\Closure $sleep = null,
) {
if (trim($serviceToken) === '') {
throw new \LogicException('The company-data service token is missing.');
}
$this->sleep = $sleep
?? static fn (int $milliseconds) => usleep($milliseconds * 1000);
}
public function extract(string $website): CompanyEnrichment
{
$host = parse_url($website, PHP_URL_HOST) ?: 'invalid-host';
for ($attempt = 0; $attempt < 3; ++$attempt) {
try {
$response = $this->http->request('GET', self::ENDPOINT, [
'query' => [
'token' => $this->serviceToken,
'website' => $website,
],
]);
$status = $response->getStatusCode();
} catch (TransportExceptionInterface $exception) {
if ($attempt < 2) {
$this->retry($host, $attempt, null, null);
continue;
}
$this->logger->error('company_enrichment.failed', [
'host' => $host,
'kind' => 'transport',
]);
throw new EnrichmentException(
'temporarily_unavailable',
previous: $exception,
);
}
if ($status >= 200 && $status < 300) {
try {
$result = CompanyEnrichment::fromPayload(
$response->toArray(false),
);
} catch (DecodingExceptionInterface|TransportExceptionInterface $e) {
throw new EnrichmentException('invalid_payload', previous: $e);
}
$this->logger->info('company_enrichment.succeeded', [
'host' => $host,
'attempt' => $attempt + 1,
]);
return $result;
}
$retryable = $status === 429
|| in_array($status, [502, 503, 504], true);
if ($retryable && $attempt < 2) {
$this->retry($host, $attempt, $status, $response);
continue;
}
$kind = match (true) {
$status === 401 || $status === 403 => 'authentication',
$status === 429 => 'rate_limited',
$status >= 400 && $status < 500 => 'request_rejected',
default => 'temporarily_unavailable',
};
$retryAfter = $status === 429
? $this->retryAfterSeconds($response)
: null;
$this->logger->error('company_enrichment.failed', [
'host' => $host,
'status' => $status,
'kind' => $kind,
]);
throw new EnrichmentException($kind, $retryAfter);
}
throw new EnrichmentException('temporarily_unavailable');
}
private function retry(
string $host,
int $attempt,
?int $status,
?ResponseInterface $response,
): void {
$retryAfter = $response
? $this->retryAfterSeconds($response)
: null;
$delay = $retryAfter !== null
? min(5000, $retryAfter * 1000)
: min(2000, 250 * (2 ** $attempt) + random_int(0, 100));
$this->logger->warning('company_enrichment.retry', [
'host' => $host,
'attempt' => $attempt + 1,
'status' => $status,
'delay_ms' => $delay,
]);
($this->sleep)($delay);
}
private function retryAfterSeconds(ResponseInterface $response): ?int
{
$value = $response->getHeaders(false)['retry-after'][0] ?? null;
return is_string($value) && ctype_digit($value)
? (int) $value
: null;
}
}
Expose the CRM prefill route
The controller accepts either a full URL or a hostname and defaults missing schemes to HTTPS. It rejects credentials embedded in URLs, localhost, malformed hosts, and literal private or reserved IP addresses. This prevents obvious misuse of a quota-bearing endpoint while keeping the form convenient.
<?php
// src/Controller/LeadPrefillController.php
namespace App\Controller;
use App\CompanyData\CompanyDataClient;
use App\CompanyData\EnrichmentException;
use Symfony\Bundle\FrameworkBundle\Controller\AbstractController;
use Symfony\Component\HttpFoundation\JsonResponse;
use Symfony\Component\HttpFoundation\Request;
use Symfony\Component\Routing\Attribute\Route;
final class LeadPrefillController extends AbstractController
{
#[Route('/crm/leads/prefill', name: 'crm_lead_prefill', methods: ['POST'])]
public function __invoke(
Request $request,
CompanyDataClient $client,
): JsonResponse {
try {
$input = json_decode(
$request->getContent(),
true,
32,
JSON_THROW_ON_ERROR,
);
} catch (\JsonException) {
return $this->json(['error' => 'invalid_json'], 400);
}
$website = $this->normalizeWebsite($input['website'] ?? null);
if ($website === null) {
return $this->json(['error' => 'invalid_website'], 422);
}
try {
$enrichment = $client->extract($website);
} catch (EnrichmentException $exception) {
$status = match ($exception->kind) {
'request_rejected' => 422,
'rate_limited' => 429,
default => 503,
};
$response = $this->json([
'error' => $exception->kind,
], $status);
if ($exception->retryAfterSeconds !== null) {
$response->headers->set(
'Retry-After',
(string) $exception->retryAfterSeconds,
);
}
return $response;
}
return $this->json([
'data' => [
'website' => $website,
...$enrichment->toArray(),
],
]);
}
private function normalizeWebsite(mixed $value): ?string
{
if (!is_string($value) || trim($value) === '') {
return null;
}
$website = trim($value);
if (!preg_match('~^https?://~i', $website)) {
$website = 'https://' . $website;
}
if (filter_var($website, FILTER_VALIDATE_URL) === false) {
return null;
}
$parts = parse_url($website);
$host = strtolower($parts['host'] ?? '');
if ($host === '' || $host === 'localhost'
|| isset($parts['user']) || isset($parts['pass'])) {
return null;
}
if (filter_var($host, FILTER_VALIDATE_IP) !== false
&& filter_var(
$host,
FILTER_VALIDATE_IP,
FILTER_FLAG_NO_PRIV_RANGE | FILTER_FLAG_NO_RES_RANGE,
) === false) {
return null;
}
return $website;
}
}
The existing CRM page can call this route when the website control loses focus, then place the returned suggestions into matching controls. Preserve the user’s ability to edit every value and require the normal save action; enrichment should not create a lead merely because a URL was entered.
const form = document.querySelector('[data-lead-form]');
const website = form.elements.namedItem('website');
website.addEventListener('blur', async () => {
const response = await fetch('/crm/leads/prefill', {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({ website: website.value })
});
if (!response.ok) return;
const lead = (await response.json()).data;
for (const key of ['company', 'contact', 'email', 'phone', 'people']) {
const control = form.elements.namedItem(key);
if (!control || lead[key] == null || control.value !== '') continue;
control.value = typeof lead[key] === 'string'
? lead[key]
: JSON.stringify(lead[key]);
}
});
Production forms should also protect this POST route with the application’s normal authentication and CSRF strategy. Add per-user throttling so an authenticated account cannot consume the service quota accidentally or deliberately.
Test success, mapping, and retries deterministically
MockHttpClient lets the test verify method, endpoint, query parameters, mapping, and retry behavior without network access. Injecting the sleep closure keeps retry tests fast.
<?php
// tests/CompanyData/CompanyDataClientTest.php
namespace App\Tests\CompanyData;
use App\CompanyData\CompanyDataClient;
use PHPUnit\Framework\TestCase;
use Psr\Log\NullLogger;
use Symfony\Component\HttpClient\MockHttpClient;
use Symfony\Component\HttpClient\Response\MockResponse;
final class CompanyDataClientTest extends TestCase
{
public function testMapsSuccessfulResponse(): void
{
$http = new MockHttpClient(
function (string $method, string $url): MockResponse {
self::assertSame('GET', $method);
self::assertStringContainsString(
'/api/website-to-company-data/v1/extract',
$url,
);
self::assertStringContainsString('token=TEST_TOKEN', $url);
self::assertStringContainsString('website=', $url);
return new MockResponse(json_encode([
'company' => 'Example Company',
'contact' => 'General contact',
'email' => '[email protected]',
'phone' => '+1 555 0100',
'people' => [],
], JSON_THROW_ON_ERROR), ['http_code' => 200]);
},
);
$client = new CompanyDataClient(
$http,
'TEST_TOKEN',
new NullLogger(),
static fn (int $milliseconds) => null,
);
$result = $client->extract('https://example.com');
self::assertSame('Example Company', $result->company);
self::assertSame('[email protected]', $result->email);
self::assertSame([], $result->people);
}
public function testRetriesATransientResponse(): void
{
$http = new MockHttpClient([
new MockResponse('', ['http_code' => 503]),
new MockResponse(json_encode([
'company' => null,
'contact' => null,
'email' => null,
'phone' => null,
'people' => [],
], JSON_THROW_ON_ERROR), ['http_code' => 200]),
]);
$delays = [];
$client = new CompanyDataClient(
$http,
'TEST_TOKEN',
new NullLogger(),
function (int $milliseconds) use (&$delays): void {
$delays[] = $milliseconds;
},
);
$client->extract('https://example.com');
self::assertCount(1, $delays);
self::assertSame(2, $http->getRequestsCount());
}
}
Run the suite with php bin/phpunit. Add controller tests for malformed JSON, invalid URLs, unauthenticated callers, and each stable error response before connecting the endpoint to a real CRM form.
Security and operational details that matter
- Redact query strings. The required authentication format places the token in the URL. Configure proxies, application performance monitoring, exception reporting, and outbound HTTP instrumentation to remove the
tokenparameter. - Minimize retained data. Company contact and people data may still be personal data. Store only fields required by the CRM, apply the normal access controls, and follow the organization’s retention policy.
- Separate suggestion from persistence. Never overwrite values the salesperson has already entered. Make the source of prefilled data apparent and let the user correct it.
- Protect quota. Authenticate the CRM route, rate-limit it per user, and debounce or trigger on blur rather than calling on every keystroke.
- Rotate safely. A regenerated service token immediately invalidates the previous active token, so coordinate configuration rollout across all instances.
Observability, deployment, and common failures
Monitor counts for successful enrichments, sanitized failure kinds, retry attempts, and end-to-end latency. Alert on sustained authentication errors because they usually indicate an expired, regenerated, or incorrectly injected token. A brief rise in transient failures should not page anyone if bounded retries recover it.
Build production dependencies, provide the environment token, and warm the cache using the same environment that will run the application:
composer install --no-dev --classmap-authoritative
APP_ENV=prod APP_DEBUG=0 php bin/console cache:clear
APP_ENV=prod APP_DEBUG=0 php bin/console cache:warmup
php bin/phpunit
Restart long-running PHP workers or application containers after changing the token. Do not make a quota-consuming extraction request part of a high-frequency health check. Verify it once during deployment or through a controlled smoke test.
The most common failures are straightforward to diagnose:
authentication: confirm the plan is active and the deployed token is current. Do not retry until configuration changes.rate_limited: honorRetry-Afterwhen supplied, reduce duplicate browser calls, and review plan capacity.request_rejected: inspect the submitted public URL and validation rules. Repeating it unchanged will not help.invalid_payload: retain sanitized metadata, compare the response with the official documentation, and update the boundary mapper deliberately.temporarily_unavailable: keep the form usable, show a retry action, and allow manual entry instead of blocking lead creation.
Final verification checklist
- The user can enter a bare hostname or an HTTP/HTTPS company URL.
- The browser calls only the local
/crm/leads/prefillroute and never receives the service token. - The server sends the exact GET endpoint with
tokenandwebsitequery parameters. - The application maps
company,contact,email,phone, andpeopleat one defensive boundary. - Timeouts and retries are bounded, and authentication or validation failures are not retried blindly.
- Logs, traces, fixtures, and error reports contain neither the token nor the response payload.
- Automated tests pass without making an external request.
- An upstream failure leaves manual CRM entry available.
The strongest enrichment feature is not the one that fills the most boxes. It is the one that saves routine work while remaining predictable when a website is sparse, a response changes, a quota is reached, or the network simply has a bad minute. With a narrow Symfony boundary, cautious retries, safe logging, and human confirmation, one company URL becomes a useful head start rather than another fragile dependency.