Туториали

Laravel + AI: Transform Website Lists into Actionable Contact Research

Laravel + AI: Претворете ги листите на веб-страници во применливо истражување на контакти

A spreadsheet full of company websites looks useful until someone has to open every row, find contact details, and turn inconsistent page content into structured research. That manual loop is slow, difficult to audit, and especially frustrating when the list changes every week.

This tutorial builds a production-oriented Laravel pipeline that imports website URLs from CSV, enriches each website through the Website to Company data API, stores normalized results, and exports a reviewable contact-research spreadsheet. Queue workers keep large imports away from request timeouts, while explicit failure states make incomplete records visible instead of silently losing them.

Get access before writing integration code

Create an account at https://ai.mihajlo.mk/register, or use https://ai.mihajlo.mk/login if you already have one.

  1. Open the Website to Company data service page.
  2. Choose the available Free, Plus, or Pro plan and complete its activation.
  3. Open the official service documentation.
  4. Find the Service token panel and copy the service-scoped token.
  5. Store it in environment-backed Laravel configuration. Do not commit it, print it, or include it in test fixtures.

This service requires a token. Regenerating it revokes the previously active token, so token rotation must include updating every deployed environment that uses the old value.

The exact call is GET https://ai.mihajlo.mk/api/website-to-company-data/v1/extract. Authentication uses the token query parameter, and the website is supplied through the website query parameter. Test the credential with a public website you are authorized to research:

curl --get \
  --data-urlencode "token=YOUR_SERVICE_TOKEN" \
  --data-urlencode "website=https://example.com" \
  "https://ai.mihajlo.mk/api/website-to-company-data/v1/extract"

Put the token in the project’s uncommitted .env file:

MIHAJLO_WEBSITE_COMPANY_TOKEN=YOUR_SERVICE_TOKEN
MIHAJLO_WEBSITE_COMPANY_URL=https://ai.mihajlo.mk/api/website-to-company-data/v1/extract

Expose those values through config/services.php. Reading env() only from configuration files ensures the integration continues working after Laravel configuration is cached.

<?php

return [
    // Existing services...

    'website_company' => [
        'token' => env('MIHAJLO_WEBSITE_COMPANY_TOKEN'),
        'url' => env(
            'MIHAJLO_WEBSITE_COMPANY_URL',
            'https://ai.mihajlo.mk/api/website-to-company-data/v1/extract'
        ),
    ],
];

Choose a small, durable architecture

The input will be a UTF-8 CSV file whose header contains website. CSV is intentionally used instead of adding an XLSX package: every mainstream spreadsheet application can export it, PHP can parse it natively, and the resulting import is easy to test.

The pipeline has four responsibilities:

  • An Artisan command validates rows and creates one database record per normalized website.
  • A queue job performs enrichment outside the command process.
  • A dedicated API client owns authentication, timeouts, retries, and response mapping.
  • A second command exports completed and failed rows for human review.

Create the project with PHP 8.3 or newer and a currently supported Laravel release. Configure a database and queue connection, then run php artisan queue:table if your application uses the database queue. Generate the migration, model, commands, job, and service classes with Laravel’s normal make: commands, or create the following files directly.

Persist reviewable states, not just successful data

A research record needs to distinguish pending, processing, completed, and failed work. It should also retain a machine-readable failure code and a safe diagnostic message.

<?php
// database/migrations/xxxx_xx_xx_create_contact_researches_table.php

use Illuminate\Database\Migrations\Migration;
use Illuminate\Database\Schema\Blueprint;
use Illuminate\Support\Facades\Schema;

return new class extends Migration {
    public function up(): void
    {
        Schema::create('contact_researches', function (Blueprint $table) {
            $table->id();
            $table->string('website')->unique();
            $table->string('status')->default('pending')->index();
            $table->json('company')->nullable();
            $table->json('contact')->nullable();
            $table->string('email')->nullable();
            $table->string('phone')->nullable();
            $table->json('people')->nullable();
            $table->string('failure_code')->nullable();
            $table->text('failure_message')->nullable();
            $table->timestamps();
        });
    }

    public function down(): void
    {
        Schema::dropIfExists('contact_researches');
    }
};
<?php
// app/Models/ContactResearch.php

namespace App\Models;

use Illuminate\Database\Eloquent\Model;

final class ContactResearch extends Model
{
    protected $fillable = [
        'website', 'status', 'company', 'contact', 'email', 'phone',
        'people', 'failure_code', 'failure_message',
    ];

    protected function casts(): array
    {
        return [
            'company' => 'array',
            'contact' => 'array',
            'people' => 'array',
        ];
    }
}

Run php artisan migrate after reviewing the migration for your database platform.

Build a defensive API boundary

External JSON should not leak directly into the rest of the application. The supplied contract names company, contact, email, phone, and people, but integrations should still reject a non-object response and tolerate nullable or unexpectedly shaped individual values.

<?php
// app/Services/WebsiteCompany/CompanyResearchData.php

namespace App\Services\WebsiteCompany;

use UnexpectedValueException;

final readonly class CompanyResearchData
{
    public function __construct(
        public array $company,
        public array $contact,
        public ?string $email,
        public ?string $phone,
        public array $people,
    ) {}

    public static function fromPayload(mixed $payload): self
    {
        if (! is_array($payload)) {
            throw new UnexpectedValueException('The API response was not a JSON object.');
        }

        return new self(
            self::mapValue($payload['company'] ?? null),
            self::mapValue($payload['contact'] ?? null),
            self::text($payload['email'] ?? null),
            self::text($payload['phone'] ?? null),
            is_array($payload['people'] ?? null)
                ? array_values($payload['people'])
                : [],
        );
    }

    private static function mapValue(mixed $value): array
    {
        if (is_array($value)) {
            return $value;
        }

        return is_string($value) || is_numeric($value)
            ? ['value' => (string) $value]
            : [];
    }

    private static function text(mixed $value): ?string
    {
        if (! is_string($value) && ! is_numeric($value)) {
            return null;
        }

        $value = trim((string) $value);

        return $value === '' ? null : $value;
    }
}

The HTTP client uses bounded connection and response timeouts. It retries connection failures, rate limits, and server errors with capped backoff, but never retries authentication or request-validation failures. It also avoids logging the request URL because the required credential appears in its query string.

<?php
// app/Services/WebsiteCompany/WebsiteCompanyClient.php

namespace App\Services\WebsiteCompany;

use Illuminate\Http\Client\ConnectionException;
use Illuminate\Support\Facades\Http;
use RuntimeException;
use Throwable;

final class WebsiteCompanyException extends RuntimeException
{
    public function __construct(
        public readonly string $failureCode,
        string $message
    ) {
        parent::__construct($message);
    }
}

final class WebsiteCompanyClient
{
    public function extract(string $website): CompanyResearchData
    {
        $token = (string) config('services.website_company.token');
        $url = (string) config('services.website_company.url');

        if ($token === '') {
            throw new WebsiteCompanyException(
                'configuration',
                'The Website to Company service token is missing.'
            );
        }

        for ($attempt = 1; $attempt <= 3; $attempt++) {
            try {
                $response = Http::acceptJson()
                    ->connectTimeout(5)
                    ->timeout(20)
                    ->get($url, [
                        'token' => $token,
                        'website' => $website,
                    ]);
            } catch (ConnectionException $exception) {
                if ($attempt === 3) {
                    throw new WebsiteCompanyException(
                        'transport',
                        'The enrichment service could not be reached.'
                    );
                }

                usleep(250_000 * (2 ** ($attempt - 1)));
                continue;
            }

            if ($response->successful()) {
                try {
                    return CompanyResearchData::fromPayload($response->json());
                } catch (Throwable $exception) {
                    throw new WebsiteCompanyException(
                        'invalid_response',
                        'The enrichment service returned an unusable response.'
                    );
                }
            }

            if (in_array($response->status(), [401, 403], true)) {
                throw new WebsiteCompanyException(
                    'authentication',
                    'The service token was rejected.'
                );
            }

            if (in_array($response->status(), [400, 422], true)) {
                throw new WebsiteCompanyException(
                    'validation',
                    'The website request was rejected.'
                );
            }

            $retryable = $response->status() === 429 || $response->serverError();

            if (! $retryable || $attempt === 3) {
                throw new WebsiteCompanyException(
                    $response->status() === 429 ? 'rate_limited' : 'http_error',
                    'Enrichment failed with HTTP status '.$response->status().'.'
                );
            }

            $retryAfter = ctype_digit((string) $response->header('Retry-After'))
                ? min(5, (int) $response->header('Retry-After'))
                : 0;

            usleep(max($retryAfter * 1_000_000, 250_000 * (2 ** ($attempt - 1))));
        }

        throw new WebsiteCompanyException('unknown', 'Enrichment did not complete.');
    }
}

Import websites and enrich them in the queue

The import command treats malformed URLs as row-level errors. It accepts only absolute HTTP or HTTPS URLs with a host, preventing values such as local file paths from entering the workflow.

<?php
// app/Console/Commands/ImportContactResearch.php

namespace App\Console\Commands;

use App\Jobs\EnrichContactResearch;
use App\Models\ContactResearch;
use Illuminate\Console\Command;

final class ImportContactResearch extends Command
{
    protected $signature = 'research:import {path}';
    protected $description = 'Import company websites from a CSV file';

    public function handle(): int
    {
        $handle = fopen($this->argument('path'), 'rb');

        if ($handle === false) {
            $this->error('The CSV file could not be opened.');
            return self::FAILURE;
        }

        $headers = fgetcsv($handle);
        $websiteIndex = is_array($headers)
            ? array_search('website', array_map('trim', $headers), true)
            : false;

        if ($websiteIndex === false) {
            fclose($handle);
            $this->error('The CSV header must contain website.');
            return self::FAILURE;
        }

        while (($row = fgetcsv($handle)) !== false) {
            $website = trim((string) ($row[$websiteIndex] ?? ''));
            $parts = parse_url($website);

            if (! is_array($parts)
                || ! in_array($parts['scheme'] ?? '', ['http', 'https'], true)
                || empty($parts['host'])) {
                $this->warn("Skipped invalid website: {$website}");
                continue;
            }

            $record = ContactResearch::firstOrCreate(
                ['website' => $website],
                ['status' => 'pending']
            );

            if ($record->status !== 'completed') {
                $record->update([
                    'status' => 'pending',
                    'failure_code' => null,
                    'failure_message' => null,
                ]);

                EnrichContactResearch::dispatch($record->id);
            }
        }

        fclose($handle);
        $this->info('Import accepted; enrichment jobs were dispatched.');

        return self::SUCCESS;
    }
}
<?php
// app/Jobs/EnrichContactResearch.php

namespace App\Jobs;

use App\Models\ContactResearch;
use App\Services\WebsiteCompany\WebsiteCompanyClient;
use App\Services\WebsiteCompany\WebsiteCompanyException;
use Illuminate\Contracts\Queue\ShouldBeUnique;
use Illuminate\Contracts\Queue\ShouldQueue;
use Illuminate\Foundation\Queue\Queueable;
use Illuminate\Support\Facades\Log;

final class EnrichContactResearch implements ShouldQueue, ShouldBeUnique
{
    use Queueable;

    public int $tries = 1;
    public int $timeout = 90;
    public int $uniqueFor = 300;

    public function __construct(public readonly int $researchId) {}

    public function uniqueId(): string
    {
        return (string) $this->researchId;
    }

    public function handle(WebsiteCompanyClient $client): void
    {
        $record = ContactResearch::findOrFail($this->researchId);

        if ($record->status === 'completed') {
            return;
        }

        $record->update(['status' => 'processing']);

        try {
            $data = $client->extract($record->website);

            $record->update([
                'status' => 'completed',
                'company' => $data->company,
                'contact' => $data->contact,
                'email' => $data->email,
                'phone' => $data->phone,
                'people' => $data->people,
                'failure_code' => null,
                'failure_message' => null,
            ]);
        } catch (WebsiteCompanyException $exception) {
            $record->update([
                'status' => 'failed',
                'failure_code' => $exception->failureCode,
                'failure_message' => $exception->getMessage(),
            ]);

            Log::warning('Company research enrichment failed', [
                'research_id' => $record->id,
                'failure_code' => $exception->failureCode,
            ]);
        }
    }
}

Because the client already performs bounded retries, the job has one queue attempt. This prevents a retry multiplier from unexpectedly turning three HTTP attempts into nine. Re-running the import redispatches failed records while leaving completed research untouched.

Export the list for human review

Export the stored records with a small command that writes status and structured fields to CSV. Before writing cells, prefix values beginning with =, +, -, or @ with an apostrophe. That prevents spreadsheet formula injection when external website content reaches an analyst’s desktop.

<?php
// Core of app/Console/Commands/ExportContactResearch.php

$path = storage_path('app/contact-research.csv');
$handle = fopen($path, 'wb');

fputcsv($handle, [
    'website', 'status', 'company', 'contact', 'email',
    'phone', 'people', 'failure_code', 'failure_message',
]);

$clean = static function (mixed $value): string {
    $text = is_array($value)
        ? json_encode($value, JSON_UNESCAPED_SLASHES | JSON_UNESCAPED_UNICODE)
        : (string) ($value ?? '');

    return preg_match('/^[=+\-@]/', $text) ? "'".$text : $text;
};

\App\Models\ContactResearch::query()
    ->orderBy('id')
    ->chunk(500, function ($records) use ($handle, $clean): void {
        foreach ($records as $record) {
            fputcsv($handle, array_map($clean, [
                $record->website,
                $record->status,
                $record->company,
                $record->contact,
                $record->email,
                $record->phone,
                $record->people,
                $record->failure_code,
                $record->failure_message,
            ]));
        }
    });

fclose($handle);

Test the contract without calling the service

Laravel’s HTTP fake makes tests deterministic and ensures no token leaves the test process. Test both mapping and the non-retryable authentication path.

<?php
// tests/Feature/WebsiteCompanyClientTest.php

namespace Tests\Feature;

use App\Services\WebsiteCompany\WebsiteCompanyClient;
use App\Services\WebsiteCompany\WebsiteCompanyException;
use Illuminate\Support\Facades\Http;
use Tests\TestCase;

final class WebsiteCompanyClientTest extends TestCase
{
    public function test_it_maps_the_service_response(): void
    {
        config()->set('services.website_company', [
            'token' => 'test-token',
            'url' => 'https://ai.mihajlo.mk/api/website-to-company-data/v1/extract',
        ]);

        Http::fake([
            '*' => Http::response([
                'company' => ['name' => 'Example Company'],
                'contact' => ['page' => '/contact'],
                'email' => '[email protected]',
                'phone' => '+1 555 0100',
                'people' => [['name' => 'Alex Example']],
            ], 200),
        ]);

        $data = app(WebsiteCompanyClient::class)->extract('https://example.com');

        $this->assertSame('[email protected]', $data->email);
        $this->assertSame('Example Company', $data->company['name']);

        Http::assertSent(fn ($request) =>
            $request['token'] === 'test-token'
            && $request['website'] === 'https://example.com'
        );
    }

    public function test_authentication_failure_is_not_retried(): void
    {
        config()->set('services.website_company', [
            'token' => 'invalid-token',
            'url' => 'https://ai.mihajlo.mk/api/website-to-company-data/v1/extract',
        ]);

        Http::fake(['*' => Http::response([], 401)]);

        try {
            app(WebsiteCompanyClient::class)->extract('https://example.com');
            $this->fail('Expected an authentication exception.');
        } catch (WebsiteCompanyException $exception) {
            $this->assertSame('authentication', $exception->failureCode);
        }

        Http::assertSentCount(1);
    }
}

Deploy and operate the pipeline

Run php artisan test, then cache production configuration with php artisan config:cache. Start a supervised queue worker using php artisan queue:work --timeout=90. The process supervisor should restart workers after deployments, and php artisan queue:restart lets existing workers exit safely after their current jobs.

Monitor counts grouped by status, the age of pending or processing rows, job throughput, HTTP failure codes, and quota-related rate_limited results. Alert on sustained failures rather than a single transient response. A record left in processing after a worker termination should be returned to pending by an explicit maintenance command or operational runbook.

Protect the token as a secret, restrict access to exported contact data, define a retention period, and research only public websites for a legitimate purpose. Avoid logging raw responses: contact data may be sensitive, and the query-authenticated token must never appear in application logs or monitoring breadcrumbs.

Common failures

  • Authentication failures: confirm plan activation and update the environment after token regeneration.
  • Validation failures: check that the input is an absolute public HTTP or HTTPS URL.
  • Rate limits: reduce worker concurrency or pause imports; do not create an unbounded retry loop.
  • Invalid responses: retain the failed record and investigate without weakening boundary validation.
  • No jobs run: verify the queue connection, worker process, cache backend used for unique-job locks, and deployment supervisor.

Final verification checklist

  • The service plan is active and the service-scoped token is available only through environment configuration.
  • The minimal GET request succeeds with the required token and website query parameters.
  • php artisan migrate and php artisan test complete successfully.
  • A CSV headed by website imports with php artisan research:import companies.csv.
  • The queue worker moves records from pending to completed or to a visible structured failure.
  • The exported CSV contains company, contact, email, phone, people, and review status without exposing the token.

The important result is not merely that an API returned contact data. It is that every spreadsheet row now has a traceable lifecycle: validated input, bounded enrichment work, defensively mapped output, an explainable failure state, and a safe artifact a person can review. That is the difference between a promising integration demo and a research tool a small team can confidently operate.

Портрет на автор на блогот

Mihajlo

Јас сум Михајло - развивач поттикнат од љубопитност, дисциплина и постојаната желба да создадам нешто значајно. Споделувам увиди, упатства и бесплатни услуги за да им помогнам на другите да ја поедностават својата работа и да растат во постојано развивачкиот свет на софтверот и вештачката интелигенција.