Symfony: Turn Website Lists into Insightful Contact Research with AI
A spreadsheet full of company websites looks useful until someone has to open every row, hunt for contact details, and turn inconsistent pages into comparable research. This tutorial builds a Symfony application that does that repetitive work while keeping a human in control of the result.
The finished command reads a CSV exported from a spreadsheet, sends each public website to the Website to Company data service, maps the returned company, contact, email, phone, and people data into a domain object, and writes a reviewable CSV. It resumes interrupted runs, bounds network activity, retries only transient failures, and prevents untrusted values from becoming spreadsheet formulas.
Get access and copy the service token
This service requires a service-scoped token; there is no tokenless mode in the supplied API contract.
- Register at https://ai.mihajlo.mk/register, or sign in at https://ai.mihajlo.mk/login.
- Open the Website to Company data service page.
- Choose the available Free, Plus, or Pro plan and complete its activation.
- Open the official service documentation.
- Find the Service token panel and copy the service-scoped token shown there.
Regenerating that token revokes the previously active token. Treat rotation as a deployment change: update the application environment promptly, then restart any long-running process that cached the old environment.
Confirm the exact API call
The integration uses GET https://ai.mihajlo.mk/api/website-to-company-data/v1/extract. Authentication is supplied through the token query parameter, while the public company URL goes in website.
Use a temporary shell variable for a minimal test so the token is not pasted into the command itself:
read -rsp "Service token: " SERVICE_TOKEN
curl --get "https://ai.mihajlo.mk/api/website-to-company-data/v1/extract" \
--data-urlencode "token=${SERVICE_TOKEN}" \
--data-urlencode "website=https://example.com"
unset SERVICE_TOKENA successful response should be JSON. Its exact values can vary by website, so the application will validate and normalize the documented company, contact, email, phone, and people fields instead of assuming that every site provides every value.
Store the credential outside source control
Create the Symfony project and install first-party components:
composer create-project symfony/skeleton contact-research
cd contact-research
composer require symfony/http-client symfony/console symfony/dotenv psr/log
composer require --dev symfony/test-packPut the credential in .env.local, which Symfony projects normally exclude from version control:
WEBSITE_COMPANY_TOKEN=YOUR_SERVICE_TOKENUse the real token only in your local or deployed environment. Keep the placeholder in documentation, examples, fixtures, and screenshots.
Choose a deliberately small architecture
This project uses a synchronous console command rather than Messenger. For an ordinary research spreadsheet, sequential processing is easier to operate, naturally limits concurrency, and produces a usable partial file after every completed row. A queue becomes worthwhile when imports must run concurrently or independently of a command session, but it also introduces delivery, idempotency, and quota-coordination concerns.
The important files are:
src/Research/CompanyResearch.php: the application-boundary DTO and defensive mapper.src/Research/CompanyDataClient.php: HTTP transport, retry policy, and failure classification.src/Command/ResearchWebsitesCommand.php: CSV ingestion, validation, resumption, and export.tests/Research/CompanyDataClientTest.php: deterministic HTTP tests.
The input is a normal UTF-8 CSV with a website header:
website
https://example.com
https://www.example.orgMap uncertain JSON at the application boundary
Do not let loosely typed remote JSON spread through the command. This DTO preserves structured values as JSON when a field is not a simple string, retains the people collection, and represents failures explicitly.
<?php
// src/Research/CompanyResearch.php
namespace App\Research;
final readonly class CompanyResearch
{
public function __construct(
public string $website,
public string $status,
public ?string $company,
public ?string $contact,
public ?string $email,
public ?string $phone,
public array $people,
public ?string $error,
) {
}
public static function fromPayload(string $website, array $payload): self
{
$people = $payload['people'] ?? [];
if (!is_array($people)) {
$people = [$people];
} elseif (!array_is_list($people)) {
$people = [$people];
}
return new self(
$website,
'ok',
self::text($payload['company'] ?? null),
self::text($payload['contact'] ?? null),
self::text($payload['email'] ?? null),
self::text($payload['phone'] ?? null),
array_values($people),
null,
);
}
public static function failure(
string $website,
string $status,
string $error,
): self {
return new self(
$website,
$status,
null,
null,
null,
null,
[],
$error,
);
}
private static function text(mixed $value): ?string
{
if (is_string($value)) {
$value = trim($value);
return $value === '' ? null : $value;
}
if (is_int($value) || is_float($value)) {
return (string) $value;
}
if (is_array($value)) {
return json_encode(
$value,
JSON_THROW_ON_ERROR | JSON_UNESCAPED_SLASHES | JSON_UNESCAPED_UNICODE,
);
}
return null;
}
}Missing fields remain empty rather than becoming invented defaults. The mapper also avoids asserting undocumented inner structures for company, contact, or people.
Build the bounded, retry-aware HTTP client
The client allows three total attempts. It retries transport failures and the conventional transient statuses 408, 429, 500, 502, 503, and 504. It does not retry authentication or validation failures. Numeric Retry-After values are respected but capped at 30 seconds.
<?php
// src/Research/CompanyDataClient.php
namespace App\Research;
use Psr\Log\LoggerInterface;
use Symfony\Contracts\HttpClient\Exception\DecodingExceptionInterface;
use Symfony\Contracts\HttpClient\Exception\TransportExceptionInterface;
use Symfony\Contracts\HttpClient\HttpClientInterface;
use Symfony\Contracts\HttpClient\ResponseInterface;
final class CompanyDataClient
{
private const ENDPOINT =
'https://ai.mihajlo.mk/api/website-to-company-data/v1/extract';
public function __construct(
private HttpClientInterface $http,
private LoggerInterface $logger,
private string $serviceToken,
private array $retryDelaysUs = [250_000, 1_000_000],
) {
}
public function research(string $website): CompanyResearch
{
$lastAttempt = count($this->retryDelaysUs);
for ($attempt = 0; $attempt <= $lastAttempt; ++$attempt) {
try {
$response = $this->http->request('GET', self::ENDPOINT, [
'query' => [
'token' => $this->serviceToken,
'website' => $website,
],
'timeout' => 10.0,
'max_duration' => 20.0,
'headers' => ['Accept' => 'application/json'],
]);
$status = $response->getStatusCode();
if ($status >= 200 && $status < 300) {
try {
return CompanyResearch::fromPayload(
$website,
$response->toArray(false),
);
} catch (DecodingExceptionInterface | \JsonException) {
return CompanyResearch::failure(
$website,
'invalid_response',
'The service returned invalid JSON.',
);
}
}
if ($this->isRetryable($status) && $attempt < $lastAttempt) {
$this->logger->warning('Company lookup will be retried.', [
'host' => parse_url($website, PHP_URL_HOST),
'http_status' => $status,
'attempt' => $attempt + 1,
]);
$this->pause($response, $this->retryDelaysUs[$attempt]);
continue;
}
return CompanyResearch::failure(
$website,
match ($status) {
401, 403 => 'authentication_error',
400, 422 => 'invalid_request',
429 => 'rate_limited',
default => $status >= 500
? 'upstream_error'
: 'http_error',
},
"The service returned HTTP {$status}.",
);
} catch (TransportExceptionInterface) {
if ($attempt < $lastAttempt) {
$this->logger->warning('Transport failure; lookup will be retried.', [
'host' => parse_url($website, PHP_URL_HOST),
'attempt' => $attempt + 1,
]);
usleep($this->retryDelaysUs[$attempt]);
continue;
}
return CompanyResearch::failure(
$website,
'transport_error',
'The service could not be reached within the retry budget.',
);
}
}
throw new \LogicException('Unreachable retry state.');
}
private function isRetryable(int $status): bool
{
return in_array($status, [408, 429, 500, 502, 503, 504], true);
}
private function pause(ResponseInterface $response, int $fallbackUs): void
{
$value = $response->getHeaders(false)['retry-after'][0] ?? null;
$delayUs = is_string($value) && ctype_digit($value)
? min((int) $value, 30) * 1_000_000
: $fallbackUs;
usleep($delayUs);
}
}Notice what the logs omit: the token, complete request URL, response body, email, phone, and people data. Because authentication is necessarily in the query string, do not use real credentials with verbose HTTP tracing or a development profiler that records outbound URLs.
Wire environment-backed dependency injection
Bind the constructor argument in config/services.yaml. Autowiring supplies the HTTP client and logger; the environment supplies the secret.
services:
_defaults:
autowire: true
autoconfigure: true
bind:
$serviceToken: '%env(string:WEBSITE_COMPANY_TOKEN)%'
App\:
resource: '../src/'Turn the spreadsheet into a resumable review list
The command locks the output so two operators cannot append simultaneously. Existing output rows are treated as completed, making a restart safe after interruption. Each row is flushed immediately.
It also neutralizes cells beginning with =, +, -, or @. That matters because remote website content is untrusted, and spreadsheet applications may interpret those prefixes as formulas.
<?php
// src/Command/ResearchWebsitesCommand.php
namespace App\Command;
use App\Research\CompanyDataClient;
use App\Research\CompanyResearch;
use Symfony\Component\Console\Attribute\AsCommand;
use Symfony\Component\Console\Command\Command;
use Symfony\Component\Console\Input\InputArgument;
use Symfony\Component\Console\Input\InputInterface;
use Symfony\Component\Console\Output\OutputInterface;
#[AsCommand(
name: 'app:research-websites',
description: 'Build a reviewable company contact-research CSV.',
)]
final class ResearchWebsitesCommand extends Command
{
private const COLUMNS = [
'website', 'status', 'company', 'contact',
'email', 'phone', 'people', 'error',
];
public function __construct(private CompanyDataClient $client)
{
parent::__construct();
}
protected function configure(): void
{
$this
->addArgument('input', InputArgument::REQUIRED, 'Input CSV')
->addArgument('output', InputArgument::REQUIRED, 'Output CSV');
}
protected function execute(
InputInterface $input,
OutputInterface $output,
): int {
$inputPath = (string) $input->getArgument('input');
$outputPath = (string) $input->getArgument('output');
$source = fopen($inputPath, 'rb');
if ($source === false) {
throw new \RuntimeException("Cannot open {$inputPath}.");
}
$target = fopen($outputPath, 'c+b');
if ($target === false || !flock($target, LOCK_EX | LOCK_NB)) {
fclose($source);
throw new \RuntimeException("Cannot lock {$outputPath}.");
}
$errors = 0;
$processed = 0;
try {
$completed = $this->completedWebsites($target);
$header = fgetcsv($source, null, ',', '"', '');
if (!is_array($header)) {
throw new \RuntimeException('The input CSV is empty.');
}
$header = array_map(
static fn (mixed $value): string => trim((string) $value),
$header,
);
$websiteColumn = array_search('website', $header, true);
if ($websiteColumn === false) {
throw new \RuntimeException('The input needs a website header.');
}
fseek($target, 0, SEEK_END);
if (ftell($target) === 0) {
fputcsv($target, self::COLUMNS, ',', '"', '');
}
while (($row = fgetcsv($source, null, ',', '"', '')) !== false) {
$website = trim((string) ($row[$websiteColumn] ?? ''));
if ($website === '' || isset($completed[$website])) {
continue;
}
$result = $this->validPublicUrl($website)
? $this->client->research($website)
: CompanyResearch::failure(
$website,
'invalid_website',
'Expected a public HTTP or HTTPS domain URL.',
);
$this->writeResult($target, $result);
$completed[$website] = true;
++$processed;
if ($result->status !== 'ok') {
++$errors;
}
}
} finally {
fflush($target);
flock($target, LOCK_UN);
fclose($target);
fclose($source);
}
$output->writeln("Processed {$processed}; failures {$errors}.");
return $errors === 0 ? Command::SUCCESS : Command::FAILURE;
}
private function completedWebsites($target): array
{
rewind($target);
$completed = [];
$header = fgetcsv($target, null, ',', '"', '');
if (!is_array($header)) {
return [];
}
$column = array_search('website', $header, true);
if ($column === false) {
throw new \RuntimeException('Output CSV has an unexpected schema.');
}
while (($row = fgetcsv($target, null, ',', '"', '')) !== false) {
$website = (string) ($row[$column] ?? '');
$completed[ltrim($website, "'")] = true;
}
return $completed;
}
private function validPublicUrl(string $url): bool
{
$parts = parse_url($url);
$scheme = strtolower((string) ($parts['scheme'] ?? ''));
$host = strtolower((string) ($parts['host'] ?? ''));
return filter_var($url, FILTER_VALIDATE_URL) !== false
&& in_array($scheme, ['http', 'https'], true)
&& $host !== ''
&& str_contains($host, '.')
&& filter_var($host, FILTER_VALIDATE_IP) === false
&& !isset($parts['user'], $parts['pass']);
}
private function writeResult($target, CompanyResearch $result): void
{
$people = json_encode(
$result->people,
JSON_THROW_ON_ERROR | JSON_UNESCAPED_SLASHES | JSON_UNESCAPED_UNICODE,
);
$row = [
$result->website,
$result->status,
$result->company,
$result->contact,
$result->email,
$result->phone,
$people,
$result->error,
];
$row = array_map([$this, 'spreadsheetSafe'], $row);
if (fputcsv($target, $row, ',', '"', '') === false) {
throw new \RuntimeException('Could not write an output row.');
}
fflush($target);
}
private function spreadsheetSafe(mixed $value): string
{
$value = (string) ($value ?? '');
return preg_match('/^[=+\-@]/u', $value) === 1
? "'".$value
: $value;
}
}Run it with an immutable input file and a dedicated output path:
mkdir -p var/imports var/research
php bin/console app:research-websites \
var/imports/companies.csv \
var/research/contact-research.csvA nonzero exit indicates that at least one row produced a structured failure, while successful and failed rows remain available for review. To retry failed rows later, remove those specific rows from a copy of the output or begin a fresh output file; this avoids silently duplicating research records.
Test transport behavior without calling the service
MockHttpClient makes the contract deterministic. The first test verifies the exact method, endpoint, query parameters, and mapping. The second proves that a transient response is retried.
<?php
// tests/Research/CompanyDataClientTest.php
namespace App\Tests\Research;
use App\Research\CompanyDataClient;
use PHPUnit\Framework\TestCase;
use Psr\Log\NullLogger;
use Symfony\Component\HttpClient\MockHttpClient;
use Symfony\Component\HttpClient\Response\MockResponse;
final class CompanyDataClientTest extends TestCase
{
public function testItSendsAndMapsTheDocumentedContract(): void
{
$http = new MockHttpClient(
function (string $method, string $url): MockResponse {
self::assertSame('GET', $method);
self::assertSame(
'https://ai.mihajlo.mk/api/website-to-company-data/v1/extract',
strtok($url, '?'),
);
parse_str((string) parse_url($url, PHP_URL_QUERY), $query);
self::assertSame('secret-test-token', $query['token']);
self::assertSame('https://example.com', $query['website']);
return new MockResponse(json_encode([
'company' => 'Example Company',
'contact' => 'General enquiries',
'email' => '[email protected]',
'phone' => '+1 555 0100',
'people' => [['name' => 'Alex Example']],
], JSON_THROW_ON_ERROR), ['http_code' => 200]);
},
);
$result = (new CompanyDataClient(
$http,
new NullLogger(),
'secret-test-token',
[],
))->research('https://example.com');
self::assertSame('ok', $result->status);
self::assertSame('Example Company', $result->company);
self::assertSame('[email protected]', $result->email);
self::assertCount(1, $result->people);
}
public function testItRetriesAServiceUnavailableResponse(): void
{
$http = new MockHttpClient([
new MockResponse('{}', ['http_code' => 503]),
new MockResponse(
'{"company":"Recovered Company"}',
['http_code' => 200],
),
]);
$result = (new CompanyDataClient(
$http,
new NullLogger(),
'secret-test-token',
[0],
))->research('https://example.com');
self::assertSame('ok', $result->status);
self::assertSame('Recovered Company', $result->company);
self::assertSame(2, $http->getRequestsCount());
}
}php bin/phpunitOperate it safely in production
Set WEBSITE_COMPANY_TOKEN through the deployment platform’s secret store, not a committed production file. Give the process read access only to the import and write access only to the research directory. Persist that directory if the runtime uses ephemeral containers.
Run one process per output file. The lock rejects accidental concurrency, but separate output files could still consume the same service allowance concurrently. Keep concurrency aligned with the activated plan without assuming undocumented quotas.
Monitor the command’s exit status and the counts it prints. Alert on repeated authentication_error, rate_limited, transport_error, or upstream_error rows. Avoid logging the returned contacts: the CSV is already the controlled record containing that data.
Common failure modes
- Every row reports authentication failure: confirm activation and replace the deployed token. If it was regenerated, the old token is no longer active.
- Rows are rate limited: reduce run frequency or concurrency and review the active plan. The client already honors bounded numeric retry guidance.
- The output is immediately skipped: the same websites already exist in that output file. Choose a new file when starting a new research snapshot.
- A website is rejected locally: supply a complete public
http://orhttps://domain URL without embedded credentials. - Fields are blank despite success: the public site may not expose them, or the returned field may be absent. Blank data is preferable to fabricated certainty.
Final verification checklist
- The account and Free, Plus, or Pro plan are active.
- The service-scoped token is stored only in environment-backed configuration.
- The request is a
GETto the exact/v1/extractendpoint withtokenandwebsitequery parameters. - Company, contact, email, phone, and people data are mapped at the boundary.
- Timeouts, bounded retries, rate-limit handling, and structured failures are enabled.
- Tests pass without making external calls.
- The production output directory is writable and persistent.
- The resulting CSV opens as a reviewable list, with failures visible rather than hidden.
The valuable result is not merely a larger spreadsheet. It is a research list with provenance, explicit uncertainty, resumable processing, and predictable failure behavior. That combination turns a repetitive lookup task into an integration a small team can trust, inspect, and operate without surrendering judgment to automation.
