Symfony CRM автоматизација: веднаш пополнете потенцијални клиенти со извлекување податоци од веб-страниците на компаниите
A CRM lead should not begin as a typing exercise. When a salesperson already knows a company’s website, the application can use that public source to prepare the company and contact fields for review. The user supplies one URL; the CRM returns a useful draft instead of an empty form.
This tutorial builds that workflow in PHP 8.3 and Symfony 7.4. The integration calls a Website to Company data service synchronously, maps its response into a typed application boundary, and prefills a lead form. It also handles timeouts, retries, rate limits, malformed responses, logging, tests, and production secrets.
Get access and copy the service token
Complete the service onboarding before writing integration code:
- Register at https://ai.mihajlo.mk/register, or sign in at https://ai.mihajlo.mk/login.
- Open the Website to Company data service page.
- Choose the available Free, Plus, or Pro plan and complete its activation.
- Open the official service documentation.
- Find the Service token panel and copy its service-scoped token.
This service requires a token. Authentication uses the token={serviceToken} query parameter, not an Authorization header. Regenerating the token revokes the previously active token, so token rotation must update every deployed application that uses it.
The exact request is GET https://ai.mihajlo.mk/api/website-to-company-data/v1/extract. It accepts the website and token query parameters. Confirm access with a minimal request:
curl --get \
--data-urlencode "token=YOUR_SERVICE_TOKEN" \
--data-urlencode "website=https://example.com" \
"https://ai.mihajlo.mk/api/website-to-company-data/v1/extract"
Do not paste a real token into shell history on shared machines or commit it to source control. In a local Symfony installation, put it in .env.local, which should remain untracked:
COMPANY_DATA_ENDPOINT=https://ai.mihajlo.mk/api/website-to-company-data/v1/extract
COMPANY_DATA_TOKEN=YOUR_SERVICE_TOKEN
Production deployments should inject these values through the hosting platform’s environment or secret manager rather than uploading .env.local.
Create the Symfony project
The project needs PHP 8.3, Composer, Symfony’s HTTP client, Twig, CSRF protection, Monolog, and the test pack. An existing Symfony CRM can use the same components.
composer create-project symfony/skeleton:"7.4.*" crm-prefill
cd crm-prefill
composer require \
symfony/http-client:^7.4 \
symfony/twig-bundle:^7.4 \
symfony/security-csrf:^7.4 \
symfony/monolog-bundle
composer require --dev symfony/test-pack
The relevant structure will be:
src/
Controller/LeadController.php
Integration/CompanyProfile.php
Integration/EnrichmentException.php
Integration/WebsiteCompanyClient.php
templates/
lead/new.html.twig
tests/
Integration/WebsiteCompanyClientTest.php
config/
services.yaml
The browser posts the website to a same-origin Symfony controller. The controller validates the input and CSRF token, while a dedicated client owns the external API contract. The resulting domain object is returned to the browser, which fills the editable lead form.
This is intentionally synchronous: the salesperson is waiting for an immediate suggestion, and the request has strict time limits. Messenger would add operational complexity without improving this interaction. A background message is more appropriate if enrichment becomes part of bulk imports or may take longer than an interactive request budget.
Wire the endpoint and token only into the integration client:
# config/services.yaml
services:
_defaults:
autowire: true
autoconfigure: true
App\:
resource: '../src/'
App\Integration\WebsiteCompanyClient:
arguments:
$endpoint: '%env(COMPANY_DATA_ENDPOINT)%'
$serviceToken: '%env(COMPANY_DATA_TOKEN)%'
Map the API at one boundary
The supplied response contract contains company, contact, email, phone, and people data. Application code should not pass an unvalidated remote payload throughout the CRM.
The mapper below accepts nullable strings for single-value lead fields and an array for people. Missing fields become neutral values, while incompatible types cause a schema failure. That is safer than guessing how an undocumented nested value should map into a CRM field.
<?php
// src/Integration/CompanyProfile.php
namespace App\Integration;
final readonly class CompanyProfile
{
public function __construct(
public ?string $company,
public ?string $contact,
public ?string $email,
public ?string $phone,
public array $people,
) {
}
public static function fromPayload(array $payload): self
{
return new self(
company: self::nullableString($payload, 'company'),
contact: self::nullableString($payload, 'contact'),
email: self::nullableString($payload, 'email'),
phone: self::nullableString($payload, 'phone'),
people: self::people($payload),
);
}
public function toArray(): array
{
return [
'company' => $this->company,
'contact' => $this->contact,
'email' => $this->email,
'phone' => $this->phone,
'people' => $this->people,
];
}
private static function nullableString(array $payload, string $field): ?string
{
$value = $payload[$field] ?? null;
if ($value === null) {
return null;
}
if (!is_string($value)) {
throw new \UnexpectedValueException(
sprintf('Expected "%s" to be a string or null.', $field)
);
}
$value = trim($value);
return $value === '' ? null : $value;
}
private static function people(array $payload): array
{
$people = $payload['people'] ?? [];
if (!is_array($people)) {
throw new \UnexpectedValueException(
'Expected "people" to be an array.'
);
}
return $people;
}
}
Failures also need an application-level vocabulary. It prevents controllers from exposing transport exception messages or URLs containing the query-string token.
<?php
// src/Integration/EnrichmentException.php
namespace App\Integration;
final class EnrichmentException extends \RuntimeException
{
public function __construct(
public readonly string $kind,
public readonly bool $retryable,
) {
parent::__construct('Company enrichment failed.');
}
}
Build a bounded, retry-aware HTTP client
The client permits three attempts, with short backoff. It retries transport failures, rate limiting, and selected temporary server failures. It never retries authentication or validation failures: repeating the same rejected request only consumes time and quota.
<?php
// src/Integration/WebsiteCompanyClient.php
namespace App\Integration;
use Psr\Log\LoggerInterface;
use Symfony\Contracts\HttpClient\Exception\DecodingExceptionInterface;
use Symfony\Contracts\HttpClient\Exception\TransportExceptionInterface;
use Symfony\Contracts\HttpClient\HttpClientInterface;
final class WebsiteCompanyClient
{
public function __construct(
private HttpClientInterface $httpClient,
private LoggerInterface $logger,
private string $endpoint,
private string $serviceToken,
) {
}
public function extract(string $website): CompanyProfile
{
$siteHash = hash('sha256', (string) parse_url($website, PHP_URL_HOST));
$startedAt = microtime(true);
for ($attempt = 1; $attempt <= 3; $attempt++) {
try {
$response = $this->httpClient->request('GET', $this->endpoint, [
'query' => [
'token' => $this->serviceToken,
'website' => $website,
],
'timeout' => 4.0,
'max_duration' => 8.0,
'headers' => [
'Accept' => 'application/json',
],
]);
$status = $response->getStatusCode();
if (in_array($status, [429, 500, 502, 503, 504], true)) {
if ($attempt === 3) {
throw new EnrichmentException(
$status === 429 ? 'rate_limited' : 'upstream_unavailable',
true,
);
}
$headers = $response->getHeaders(false);
$delayMs = $this->retryDelay(
$status,
$headers['retry-after'][0] ?? null,
$attempt,
);
$this->logger->warning('Company enrichment will be retried.', [
'attempt' => $attempt,
'status' => $status,
'delay_ms' => $delayMs,
'site_hash' => $siteHash,
]);
usleep($delayMs * 1000);
continue;
}
if (in_array($status, [401, 403], true)) {
throw new EnrichmentException('credentials_rejected', false);
}
if (in_array($status, [400, 422], true)) {
throw new EnrichmentException('request_rejected', false);
}
if ($status < 200 || $status >= 300) {
throw new EnrichmentException('upstream_error', false);
}
try {
$profile = CompanyProfile::fromPayload(
$response->toArray(false)
);
} catch (DecodingExceptionInterface|\UnexpectedValueException) {
throw new EnrichmentException('invalid_response', false);
}
$this->logger->info('Company enrichment completed.', [
'attempts' => $attempt,
'duration_ms' => (int) ((microtime(true) - $startedAt) * 1000),
'site_hash' => $siteHash,
]);
return $profile;
} catch (TransportExceptionInterface) {
if ($attempt === 3) {
throw new EnrichmentException('transport_error', true);
}
$delayMs = $attempt === 1 ? 100 : 250;
$this->logger->warning('Company enrichment transport retry.', [
'attempt' => $attempt,
'delay_ms' => $delayMs,
'site_hash' => $siteHash,
]);
usleep($delayMs * 1000);
}
}
throw new EnrichmentException('upstream_unavailable', true);
}
private function retryDelay(
int $status,
?string $retryAfter,
int $attempt,
): int {
if ($status === 429 && $retryAfter !== null && ctype_digit($retryAfter)) {
return min(1000, max(100, (int) $retryAfter * 1000));
}
return $attempt === 1 ? 100 : 250;
}
}
The token never enters a log context, exception message, controller response, or test fixture. Even the hostname is hashed, reducing customer-data exposure while retaining a stable correlation value.
Expose the prefill controller
The controller accepts bare domains such as example.com by adding HTTPS. It rejects malformed URLs, credentials embedded in URLs, localhost names, and literal IP addresses. Symfony never fetches the supplied site directly, but restricting input still prevents surprising data disclosure to the external service.
<?php
// src/Controller/LeadController.php
namespace App\Controller;
use App\Integration\EnrichmentException;
use App\Integration\WebsiteCompanyClient;
use JsonException;
use Symfony\Bundle\FrameworkBundle\Controller\AbstractController;
use Symfony\Component\HttpFoundation\JsonResponse;
use Symfony\Component\HttpFoundation\Request;
use Symfony\Component\HttpFoundation\Response;
use Symfony\Component\Routing\Attribute\Route;
use Symfony\Component\Security\Csrf\CsrfToken;
use Symfony\Component\Security\Csrf\CsrfTokenManagerInterface;
final class LeadController extends AbstractController
{
public function __construct(
private WebsiteCompanyClient $client,
private CsrfTokenManagerInterface $csrf,
) {
}
#[Route('/leads/new', name: 'lead_new', methods: ['GET'])]
public function new(): Response
{
return $this->render('lead/new.html.twig');
}
#[Route('/api/leads/prefill', name: 'lead_prefill', methods: ['POST'])]
public function prefill(Request $request): JsonResponse
{
$token = new CsrfToken(
'lead_prefill',
(string) $request->headers->get('X-CSRF-TOKEN')
);
if (!$this->csrf->isTokenValid($token)) {
return $this->json(['error' => 'invalid_csrf'], 403);
}
try {
$input = json_decode(
$request->getContent(),
true,
512,
JSON_THROW_ON_ERROR
);
} catch (JsonException) {
return $this->json(['error' => 'invalid_json'], 400);
}
if (!is_array($input) || !is_string($input['website'] ?? null)) {
return $this->json(['error' => 'website_required'], 422);
}
$website = $this->normalizeWebsite($input['website']);
if ($website === null) {
return $this->json(['error' => 'invalid_website'], 422);
}
try {
return $this->json([
'data' => $this->client->extract($website)->toArray(),
]);
} catch (EnrichmentException $exception) {
$status = match ($exception->kind) {
'request_rejected' => 422,
'rate_limited' => 429,
default => 503,
};
return $this->json([
'error' => $exception->kind,
'retryable' => $exception->retryable,
], $status);
}
}
private function normalizeWebsite(string $value): ?string
{
$value = trim($value);
if ($value === '' || strlen($value) > 2048) {
return null;
}
if (!str_contains($value, '://')) {
$value = 'https://'.$value;
}
if (filter_var($value, FILTER_VALIDATE_URL) === false) {
return null;
}
$parts = parse_url($value);
$scheme = strtolower((string) ($parts['scheme'] ?? ''));
$host = strtolower((string) ($parts['host'] ?? ''));
if (
!in_array($scheme, ['http', 'https'], true)
|| $host === ''
|| $host === 'localhost'
|| isset($parts['user'])
|| isset($parts['pass'])
|| filter_var($host, FILTER_VALIDATE_IP) !== false
) {
return null;
}
return $value;
}
}
Prefill an editable lead form
Enrichment should suggest values, not silently establish facts. The salesperson must be able to inspect and change every field before saving the lead.
{# templates/lead/new.html.twig #}
<form id="lead-form" method="post" action="/leads">
<label>Company website
<input id="website" name="website" type="url" required>
</label>
<button id="prefill" type="button">Prefill lead</button>
<label>Company
<input id="company" name="company">
</label>
<label>Contact
<input id="contact" name="contact">
</label>
<label>Email
<input id="email" name="email" type="email">
</label>
<label>Phone
<input id="phone" name="phone" type="tel">
</label>
<label>People data
<textarea id="people" name="people"></textarea>
</label>
<p id="prefill-status" role="status"></p>
<button type="submit">Save lead</button>
</form>
<script>
const button = document.querySelector('#prefill');
const status = document.querySelector('#prefill-status');
button.addEventListener('click', async () => {
button.disabled = true;
status.textContent = 'Looking up company data…';
try {
const response = await fetch('/api/leads/prefill', {
method: 'POST',
headers: {
'Content-Type': 'application/json',
'X-CSRF-TOKEN': '{{ csrf_token('lead_prefill') }}'
},
body: JSON.stringify({
website: document.querySelector('#website').value
})
});
const payload = await response.json();
if (!response.ok) {
throw new Error(payload.error ?? 'prefill_failed');
}
for (const field of ['company', 'contact', 'email', 'phone']) {
document.querySelector(`#${field}`).value =
payload.data[field] ?? '';
}
document.querySelector('#people').value =
JSON.stringify(payload.data.people ?? [], null, 2);
status.textContent = 'Draft fields are ready for review.';
} catch (error) {
status.textContent =
'Prefill is unavailable. You can still enter the lead manually.';
} finally {
button.disabled = false;
}
});
</script>
A production CRM should submit this form to its normal lead-creation handler, with server-side validation and authorization independent of enrichment. The external result is untrusted input, just like data typed by a user.
Test without calling the live service
MockHttpClient provides a deterministic transport. These tests verify the exact method, endpoint parameters, response mapping, and the rule that rejected credentials are not retried.
<?php
// tests/Integration/WebsiteCompanyClientTest.php
namespace App\Tests\Integration;
use App\Integration\EnrichmentException;
use App\Integration\WebsiteCompanyClient;
use PHPUnit\Framework\TestCase;
use Psr\Log\NullLogger;
use Symfony\Component\HttpClient\MockHttpClient;
use Symfony\Component\HttpClient\Response\MockResponse;
final class WebsiteCompanyClientTest extends TestCase
{
private const ENDPOINT =
'https://ai.mihajlo.mk/api/website-to-company-data/v1/extract';
public function testItMapsACompanyResponse(): void
{
$http = new MockHttpClient(
function (string $method, string $url): MockResponse {
self::assertSame('GET', $method);
self::assertSame(self::ENDPOINT, strtok($url, '?'));
parse_str((string) parse_url($url, PHP_URL_QUERY), $query);
self::assertSame('test-token', $query['token']);
self::assertSame('https://example.com', $query['website']);
return new MockResponse(json_encode([
'company' => 'Example Company',
'contact' => 'Example Contact',
'email' => '[email protected]',
'phone' => '+1 555 0100',
'people' => [
['name' => 'Example Contact'],
],
], JSON_THROW_ON_ERROR), [
'http_code' => 200,
'response_headers' => ['content-type: application/json'],
]);
},
self::ENDPOINT
);
$client = new WebsiteCompanyClient(
$http,
new NullLogger(),
self::ENDPOINT,
'test-token'
);
$profile = $client->extract('https://example.com');
self::assertSame('Example Company', $profile->company);
self::assertSame('[email protected]', $profile->email);
self::assertCount(1, $profile->people);
}
public function testItDoesNotRetryRejectedCredentials(): void
{
$http = new MockHttpClient(
new MockResponse('{}', ['http_code' => 401]),
self::ENDPOINT
);
$client = new WebsiteCompanyClient(
$http,
new NullLogger(),
self::ENDPOINT,
'expired-token'
);
try {
$client->extract('https://example.com');
self::fail('Expected an enrichment failure.');
} catch (EnrichmentException $exception) {
self::assertSame('credentials_rejected', $exception->kind);
self::assertFalse($exception->retryable);
self::assertSame(1, $http->getRequestsCount());
}
}
}
Run the suite with php bin/phpunit. Keep fake domains, fake people, and placeholder credentials in fixtures. Tests should never depend on service availability or consume a plan’s quota.
Operate the integration safely
Security
Authorize both CRM routes according to the application’s existing access-control rules. Keep the service token server-side, retain CSRF protection, escape displayed values, and validate all enriched fields again when saving the lead. Because authentication is contractually a query parameter, avoid logging complete outbound URLs or raw HTTP exceptions.
Observability
The example records duration, attempt count, status, and a hashed host identifier. Add metrics for successful enrichment, validation rejection, rate limiting, authentication failure, schema failure, and latency. Alerts should distinguish an expired token from a temporary upstream outage: they require different responses.
Common failures
credentials_rejected: confirm plan activation and replace a revoked or regenerated token in the deployment secret.request_rejected: inspect website normalization and confirm that the public URL is valid.rate_limited: respect the plan limit, retain bounded backoff, and avoid automatic browser retry loops.invalid_response: compare the official documentation with the boundary mapper before changing domain types.transport_error: check DNS, outbound HTTPS policy, certificates, and application timeout budgets.
Deployment
Inject COMPANY_DATA_ENDPOINT and COMPANY_DATA_TOKEN into every application instance, warm Symfony’s production cache, then perform one controlled enrichment through the deployed UI. If the token is regenerated, deploy its replacement as one coordinated secret change because the previous token stops working.
Do not make external enrichment a prerequisite for saving a lead. The interface already preserves the manual path when the service is unavailable, which keeps a temporary dependency failure from blocking ordinary CRM work.
Final verification checklist
- The account and selected plan are active.
- The service-scoped token comes from the documentation page’s Service token panel.
- The application sends an HTTPS
GETto the exact/v1/extractendpoint. - Both
tokenandwebsiteare query parameters. - The boundary maps
company,contact,email,phone, andpeople. - Timeouts and retries are bounded, and authentication or validation failures are not retried.
- No token or complete authenticated URL appears in source control, logs, tests, or browser code.
- The salesperson can review, edit, and save the prefilled lead—or continue manually after failure.
The valuable part of this integration is not merely turning a website into data. It is turning uncertain external data into a controlled, observable draft that helps a person move faster without surrendering judgment. One website becomes a useful lead, while the Symfony application remains secure, testable, and honest about failure.