Laravel: Transform Website Lists into Actionable Company Insights

Mihajlo
Aug 11, 2026
Tutorials

A spreadsheet full of company websites looks useful, but it is not yet a research tool. Someone still has to open every site, locate contact details, copy company information, and decide which records deserve attention. That repetitive work becomes especially awkward when the list changes every week.

This tutorial builds a production-oriented Laravel pipeline that imports website URLs from CSV, enriches them with structured company and contact data, processes requests safely in the background, and exports a reviewable spreadsheet. The design is intentionally modest: Laravel’s HTTP client, database queue, query builder, and automated test tools are enough.

Get access before writing integration code

Start by creating an account at the registration page, or use the sign-in page if you already have one.

Open the Website to Company data service page.
Choose an available Free, Plus, or Pro plan and complete its activation.
Open the official service documentation.
Find the Service token panel and copy the service-scoped token.
Store it in your project’s environment configuration. Never commit it to source control.

Regenerating the service token revokes the previously active token. Treat regeneration as credential rotation: update every deployed environment promptly, restart long-running workers, and verify a request before considering the rotation complete.

Confirm the exact HTTP contract

The service uses an authenticated GET request to https://ai.mihajlo.mk/api/website-to-company-data/v1/extract. Authentication is supplied through the token query parameter, while the public company URL is supplied through website.

Make one minimal request from a trusted terminal. Be aware that commands containing query-string credentials can enter shell history, so remove the history entry when appropriate and never paste the command into tickets or logs.

curl --get \
  'https://ai.mihajlo.mk/api/website-to-company-data/v1/extract' \
  --data-urlencode 'token=YOUR_SERVICE_TOKEN' \
  --data-urlencode 'website=https://example.com'

Once that succeeds, place the credential in Laravel’s uncommitted .env file:

WEBSITE_COMPANY_SERVICE_TOKEN=YOUR_SERVICE_TOKEN
QUEUE_CONNECTION=database

Expose it through config/services.php. Application code should read configuration, never call env() directly, because deployed Laravel applications commonly cache configuration.

<?php

return [
    // Existing services...

    'website_company' => [
        'token' => env('WEBSITE_COMPANY_SERVICE_TOKEN'),
        'endpoint' => 'https://ai.mihajlo.mk/api/website-to-company-data/v1/extract',
    ],
];

Choose a small, resilient architecture

The input will be a CSV exported from Excel, Google Sheets, or another spreadsheet. It must contain a website column. Supporting native XLSX files would require an additional package without improving the enrichment workflow, so CSV is the cleaner boundary.

The import command validates and deduplicates rows, persists pending work, and dispatches one queue job per website. A dedicated client owns the external API contract. The job stores either normalized data or a structured failure. Finally, an export command produces a CSV that a person can sort, annotate, and review.

app/
  Console/Commands/ImportCompanyResearch.php
  Console/Commands/ExportCompanyResearch.php
  Data/CompanyResearch.php
  Exceptions/ServiceFailure.php
  Jobs/ResearchWebsite.php
  Services/WebsiteCompanyClient.php
database/migrations/
  xxxx_xx_xx_create_company_research_table.php
tests/Feature/
  WebsiteCompanyIntegrationTest.php

This asynchronous design matters for more than speed. A spreadsheet may contain bad URLs, temporary network failures, or more rows than the current plan can process immediately. Queueing keeps the import responsive and gives transient failures a bounded second chance.

Create the persistence boundary

Create the queue migration, application migration, classes, and commands with Artisan. Depending on the Laravel application version, a queue jobs-table migration may already exist; generate it only when it is absent.

php artisan make:queue-table
php artisan make:migration create_company_research_table
php artisan make:job ResearchWebsite
php artisan make:command ImportCompanyResearch
php artisan make:command ExportCompanyResearch
php artisan migrate

The research table retains the source URL, processing state, successful payload, and safe error metadata. A unique website constraint also protects against duplicate spreadsheet rows.

<?php

use Illuminate\Database\Migrations\Migration;
use Illuminate\Database\Schema\Blueprint;
use Illuminate\Support\Facades\Schema;

return new class extends Migration {
    public function up(): void
    {
        Schema::create('company_research', function (Blueprint $table): void {
            $table->id();
            $table->string('website', 2048)->unique();
            $table->string('status', 24)->index();
            $table->json('data')->nullable();
            $table->string('error_kind', 40)->nullable();
            $table->string('error_message')->nullable();
            $table->timestamps();
        });
    }

    public function down(): void
    {
        Schema::dropIfExists('company_research');
    }
};

Map uncertain data at the application boundary

The documented application boundary contains company, contact, email, phone, and people data. Individual values may evolve in shape, so the domain object preserves scalar, array, or null values instead of assuming undocumented nested fields.

<?php
// app/Data/CompanyResearch.php

namespace App\Data;

use JsonException;
use UnexpectedValueException;

final readonly class CompanyResearch
{
    public function __construct(
        public mixed $company,
        public mixed $contact,
        public mixed $email,
        public mixed $phone,
        public mixed $people,
    ) {}

    public static function fromPayload(array $payload): self
    {
        $fields = ['company', 'contact', 'email', 'phone', 'people'];

        if (array_intersect($fields, array_keys($payload)) === []) {
            throw new UnexpectedValueException('Expected research fields are absent.');
        }

        foreach ($fields as $field) {
            $value = $payload[$field] ?? null;

            if (! is_null($value) && ! is_scalar($value) && ! is_array($value)) {
                throw new UnexpectedValueException("Invalid {$field} value.");
            }
        }

        return new self(
            $payload['company'] ?? null,
            $payload['contact'] ?? null,
            $payload['email'] ?? null,
            $payload['phone'] ?? null,
            $payload['people'] ?? null,
        );
    }

    public function toArray(): array
    {
        return [
            'company' => $this->company,
            'contact' => $this->contact,
            'email' => $this->email,
            'phone' => $this->phone,
            'people' => $this->people,
        ];
    }
}

This is an important production habit: map only the contract you possess. Do not make controllers and jobs reach into speculative structures such as a supposed company name or primary email field.

Build a bounded HTTP client

The client below applies separate connection and overall response timeouts. It retries only connection failures, HTTP 429 responses, and server errors. Authentication and validation failures are returned immediately because repeating the same rejected request wastes quota and obscures configuration problems.

<?php
// app/Exceptions/ServiceFailure.php

namespace App\Exceptions;

use RuntimeException;
use Throwable;

final class ServiceFailure extends RuntimeException
{
    public function __construct(
        public readonly string $kind,
        public readonly ?int $status = null,
        ?Throwable $previous = null,
    ) {
        parent::__construct("Website research failed: {$kind}", 0, $previous);
    }
}

// app/Services/WebsiteCompanyClient.php

namespace App\Services;

use App\Data\CompanyResearch;
use App\Exceptions\ServiceFailure;
use Illuminate\Http\Client\ConnectionException;
use Illuminate\Support\Facades\Http;
use Throwable;

final class WebsiteCompanyClient
{
    public function extract(string $website): CompanyResearch
    {
        $token = (string) config('services.website_company.token');
        $endpoint = (string) config('services.website_company.endpoint');

        if ($token === '') {
            throw new ServiceFailure('configuration');
        }

        for ($attempt = 1; $attempt <= 3; $attempt++) {
            try {
                $response = Http::acceptJson()
                    ->connectTimeout(3)
                    ->timeout(15)
                    ->get($endpoint, [
                        'token' => $token,
                        'website' => $website,
                    ]);
            } catch (ConnectionException $exception) {
                if ($attempt === 3) {
                    throw new ServiceFailure('transient', null, $exception);
                }

                usleep((250 * (2 ** ($attempt - 1))) * 1000);
                continue;
            }

            if ($response->successful()) {
                $payload = $response->json();

                if (! is_array($payload)) {
                    throw new ServiceFailure('malformed_response', $response->status());
                }

                try {
                    return CompanyResearch::fromPayload($payload);
                } catch (Throwable $exception) {
                    throw new ServiceFailure(
                        'malformed_response',
                        $response->status(),
                        $exception,
                    );
                }
            }

            $status = $response->status();
            $retryable = $status === 429 || $status >= 500;

            if ($retryable && $attempt < 3) {
                $retryAfter = trim($response->header('Retry-After', ''));
                $delay = ctype_digit($retryAfter)
                    ? min(10_000, ((int) $retryAfter) * 1000)
                    : 250 * (2 ** ($attempt - 1));

                usleep(($delay + random_int(0, 150)) * 1000);
                continue;
            }

            $kind = match (true) {
                $status === 401 || $status === 403 => 'authentication',
                $status === 429 => 'rate_limited',
                $status === 400 || $status === 422 => 'validation',
                $status >= 500 => 'transient',
                default => 'service_response',
            };

            throw new ServiceFailure($kind, $status);
        }

        throw new ServiceFailure('transient');
    }
}

The response body is deliberately absent from exception messages. Upstream bodies can contain contact data or implementation details and should not drift into queue dashboards and centralized logs.

Process each website as an idempotent job

The job has two queue attempts in addition to the client’s short request-level retries. Permanent failures are recorded immediately. Exhausted rate-limit and transient failures reach failed(), producing a reviewable record rather than disappearing.

<?php

namespace App\Jobs;

use App\Exceptions\ServiceFailure;
use App\Services\WebsiteCompanyClient;
use Illuminate\Contracts\Queue\ShouldBeUnique;
use Illuminate\Contracts\Queue\ShouldQueue;
use Illuminate\Foundation\Queue\Queueable;
use Illuminate\Support\Facades\DB;
use Illuminate\Support\Facades\Log;
use Throwable;

final class ResearchWebsite implements ShouldQueue, ShouldBeUnique
{
    use Queueable;

    public int $tries = 2;
    public int $timeout = 30;
    public int $uniqueFor = 3600;
    public bool $failOnTimeout = true;

    public function __construct(public readonly string $website) {}

    public function uniqueId(): string
    {
        return hash('sha256', $this->website);
    }

    public function backoff(): array
    {
        return [120];
    }

    public function handle(WebsiteCompanyClient $client): void
    {
        try {
            $research = $client->extract($this->website);
        } catch (ServiceFailure $failure) {
            if (in_array($failure->kind, ['rate_limited', 'transient'], true)) {
                throw $failure;
            }

            DB::table('company_research')
                ->where('website', $this->website)
                ->update([
                    'status' => 'failed',
                    'error_kind' => $failure->kind,
                    'error_message' => 'The service rejected or could not map this request.',
                    'updated_at' => now(),
                ]);

            Log::warning('Company research rejected', [
                'website' => $this->website,
                'kind' => $failure->kind,
                'status' => $failure->status,
            ]);

            return;
        }

        DB::table('company_research')
            ->where('website', $this->website)
            ->update([
                'status' => 'complete',
                'data' => json_encode($research->toArray(), JSON_THROW_ON_ERROR),
                'error_kind' => null,
                'error_message' => null,
                'updated_at' => now(),
            ]);

        Log::info('Company research completed', ['website' => $this->website]);
    }

    public function failed(?Throwable $exception): void
    {
        DB::table('company_research')
            ->where('website', $this->website)
            ->update([
                'status' => 'failed',
                'error_kind' => 'retries_exhausted',
                'error_message' => 'Temporary failure persisted after bounded retries.',
                'updated_at' => now(),
            ]);
    }
}

Import the spreadsheet and export the result

The import command expects a header named website. It adds https:// when a row contains a bare hostname, rejects non-HTTP schemes, skips duplicates, and leaves invalid rows out of the queue.

<?php

namespace App\Console\Commands;

use App\Jobs\ResearchWebsite;
use Illuminate\Console\Command;
use Illuminate\Support\Facades\DB;

final class ImportCompanyResearch extends Command
{
    protected $signature = 'research:import {file}';
    protected $description = 'Import company websites from a CSV file';

    public function handle(): int
    {
        $path = realpath((string) $this->argument('file'));

        if ($path === false || ! is_readable($path)) {
            $this->error('The CSV file is not readable.');
            return self::FAILURE;
        }

        $stream = fopen($path, 'rb');
        $header = fgetcsv($stream);

        if ($header === false) {
            fclose($stream);
            $this->error('The CSV file is empty.');
            return self::FAILURE;
        }

        $header = array_map(fn ($value) => trim((string) $value), $header);
        $websiteColumn = array_search('website', $header, true);

        if ($websiteColumn === false) {
            fclose($stream);
            $this->error('A website column is required.');
            return self::FAILURE;
        }

        $seen = [];
        $queued = 0;

        while (($row = fgetcsv($stream)) !== false) {
            $website = trim((string) ($row[$websiteColumn] ?? ''));

            if ($website !== '' && ! str_contains($website, '://')) {
                $website = 'https://' . $website;
            }

            $parts = parse_url($website);
            $scheme = strtolower((string) ($parts['scheme'] ?? ''));

            if (
                filter_var($website, FILTER_VALIDATE_URL) === false ||
                ! in_array($scheme, ['http', 'https'], true) ||
                isset($parts['user'], $parts['pass']) ||
                isset($seen[$website])
            ) {
                continue;
            }

            $seen[$website] = true;
            $now = now();

            DB::table('company_research')->upsert(
                [[
                    'website' => $website,
                    'status' => 'pending',
                    'data' => null,
                    'error_kind' => null,
                    'error_message' => null,
                    'created_at' => $now,
                    'updated_at' => $now,
                ]],
                ['website'],
                ['status', 'data', 'error_kind', 'error_message', 'updated_at'],
            );

            ResearchWebsite::dispatch($website);
            $queued++;
        }

        fclose($stream);
        $this->info("Queued {$queued} unique websites.");

        return self::SUCCESS;
    }
}

The export command should iterate with cursor() so a large result set is not loaded into memory. Each contracted field is encoded as JSON when it is structured, preserving data without inventing columns from an undocumented shape.

<?php

namespace App\Console\Commands;

use Illuminate\Console\Command;
use Illuminate\Support\Facades\DB;

final class ExportCompanyResearch extends Command
{
    protected $signature = 'research:export';
    protected $description = 'Export the reviewable company research CSV';

    public function handle(): int
    {
        $path = storage_path('app/contact-research.csv');
        $stream = fopen($path, 'wb');

        if ($stream === false) {
            $this->error('Could not create the export.');
            return self::FAILURE;
        }

        fputcsv($stream, [
            'website', 'status', 'company', 'contact',
            'email', 'phone', 'people', 'error_kind',
        ]);

        foreach (DB::table('company_research')->orderBy('website')->cursor() as $row) {
            $data = is_string($row->data)
                ? json_decode($row->data, true)
                : (array) ($row->data ?? []);

            $cell = static fn (mixed $value): string =>
                is_null($value) ? '' :
                (is_scalar($value) ? (string) $value :
                    json_encode($value, JSON_UNESCAPED_SLASHES | JSON_THROW_ON_ERROR));

            fputcsv($stream, [
                $row->website,
                $row->status,
                $cell($data['company'] ?? null),
                $cell($data['contact'] ?? null),
                $cell($data['email'] ?? null),
                $cell($data['phone'] ?? null),
                $cell($data['people'] ?? null),
                $row->error_kind ?? '',
            ]);
        }

        fclose($stream);
        $this->info("Exported {$path}");

        return self::SUCCESS;
    }
}

Prove the boundary with deterministic tests

Laravel’s Http::fake() prevents real network calls and lets the test assert that both required query parameters were sent. A second test confirms that authentication failures become structured database records instead of being retried blindly.

<?php

namespace Tests\Feature;

use App\Jobs\ResearchWebsite;
use App\Services\WebsiteCompanyClient;
use Illuminate\Foundation\Testing\RefreshDatabase;
use Illuminate\Http\Client\Request;
use Illuminate\Support\Facades\DB;
use Illuminate\Support\Facades\Http;
use Tests\TestCase;

final class WebsiteCompanyIntegrationTest extends TestCase
{
    use RefreshDatabase;

    public function test_it_maps_the_documented_boundary(): void
    {
        config()->set('services.website_company.token', 'test-token');
        config()->set(
            'services.website_company.endpoint',
            'https://ai.mihajlo.mk/api/website-to-company-data/v1/extract',
        );

        Http::fake([
            'https://ai.mihajlo.mk/api/website-to-company-data/v1/extract*' =>
                Http::response([
                    'company' => ['name' => 'Example'],
                    'contact' => null,
                    'email' => ['[email protected]'],
                    'phone' => null,
                    'people' => [],
                ], 200),
        ]);

        $result = app(WebsiteCompanyClient::class)
            ->extract('https://example.com');

        $this->assertSame(['[email protected]'], $result->email);

        Http::assertSent(fn (Request $request): bool =>
            $request['token'] === 'test-token' &&
            $request['website'] === 'https://example.com'
        );
    }

    public function test_authentication_failure_is_recorded(): void
    {
        config()->set('services.website_company.token', 'expired-token');
        Http::fake([
            '*' => Http::response(['message' => 'Unauthorized'], 401),
        ]);

        DB::table('company_research')->insert([
            'website' => 'https://example.com',
            'status' => 'pending',
            'created_at' => now(),
            'updated_at' => now(),
        ]);

        ResearchWebsite::dispatchSync('https://example.com');

        $this->assertDatabaseHas('company_research', [
            'website' => 'https://example.com',
            'status' => 'failed',
            'error_kind' => 'authentication',
        ]);
    }
}

Deploy, observe, and operate the pipeline

On deployment, provide the service token through the hosting platform’s secret manager, then run php artisan config:cache and php artisan migrate --force. Restart queue workers after changing configuration so they load the new token.

php artisan test
php artisan config:cache
php artisan migrate --force
php artisan queue:work --queue=default --tries=2 --timeout=35
php artisan research:import storage/app/companies.csv
php artisan research:export

Run the queue worker under a process monitor that restarts it after crashes and deployments. The worker timeout is slightly longer than the job timeout, which is itself longer than one HTTP response timeout. Those boundaries prevent a stalled request from occupying a worker indefinitely.

Monitor counts of pending, complete, failed, authentication, rate-limited, and exhausted records. Alert on unusual failure ratios or a growing pending backlog. Logs should include the source website, failure category, HTTP status, and job identity, but never the service token, complete request URL, response body, email addresses, phone numbers, or people data.

Because the credential travels in a query parameter, HTTPS is essential. Review proxy and HTTP instrumentation settings to ensure query strings are redacted. Restrict access to the exported CSV because it contains contact research, establish a retention policy, and collect only information appropriate for your lawful business purpose.

Common failures worth recognizing quickly

Every row reports authentication: confirm plan activation, token configuration, and configuration-cache refresh. A regenerated token invalidates the previous one.
Rows remain pending: the queue worker is probably stopped, listening to another connection, or using stale configuration.
Many rows are rate-limited: reduce worker concurrency and confirm plan capacity. Do not increase retries aggressively.
A successful response is marked malformed: inspect the response securely outside normal logs and compare it with the official documentation before changing the boundary mapper.
No rows import: verify that the CSV has an exact website header and contains valid HTTP or HTTPS destinations.
The same company appears repeatedly: canonicalize input URLs according to your own business rules before import; the current unique constraint distinguishes different URL strings.

Final verification checklist

The token exists only in environment-backed configuration and is absent from source control.
The minimal GET request reaches the exact /v1/extract endpoint with token and website.
php artisan test passes without contacting the real service.
The database queue migration and research migration are applied.
A supervised queue worker is running with bounded timeouts and attempts.
A small CSV import progresses from pending to complete or a structured failed state.
storage/app/contact-research.csv opens as a reviewable spreadsheet.
Logs, monitoring, and proxies do not expose tokens or returned contact data.

The finished system does not pretend research is fully automatic. It removes mechanical browsing and transcription while preserving the human judgment that makes a contact list useful. That is the right division of labor: software turns scattered public websites into consistent evidence, and a person decides what deserves action.