From Zero to Staging: A Real-World Laravel + Node.js Docker Deployment (With Every Bug We Hit)

A step-by-step walkthrough of setting up a clean staging server from scratch — SSH hardening, Docker orchestration, Let's Encrypt SSL, database seeding, and debugging a login that refused to work for six different reasons.

The Setup

We had a production server (api.dineshstack.ae) running a Laravel 12 + Node.js microservices stack inside Docker. We needed a staging environment that was an exact mirror of production for mobile testing. The staging server existed but had been used by five different developers over time — it was a graveyard of stale projects, orphaned Docker volumes, and broken nginx configs.

Goal: One clean server. One Docker network. Three domains. Zero stale state.

Teaching note: Before touching anything on a shared server, always survey first. Never assume you know what's there. A five-minute df -h and docker ps -a saves hours of debugging phantom behaviour caused by leftover containers or conflicting ports.

Phase 1 — SSH Hardening

The first step before anything else: lock down the door.

On your local machine

# Generate a dedicated key for this server

ssh-keygen -t ed25519 -C "taggo_staging" -f ~/.ssh/dstack_staging


  
  
  Copy the public key to the server


ssh-copy-id -i ~/.ssh/dstack_staging.pub taggo@<SERVER_IP>

Then add a clean alias to ~/.ssh/config:

Host dstack_staging


    HostName <SERVER_IP>


    User taggo


    IdentityFile ~/.ssh/dstack_staging


    IdentitiesOnly yes

Now you connect with just ssh dstack_staging. No password, no ambiguity about which key to use.

On the server — disable password auth

sudo sed -i 's/^#\?PasswordAuthentication./PasswordAuthentication no/' /etc/ssh/sshd_config


sudo sed -i 's/^#\?PubkeyAuthentication./PubkeyAuthentication yes/' /etc/ssh/sshd_config


sudo systemctl reload ssh

Install fail2ban

sudo apt install fail2ban -y


sudo systemctl enable fail2ban --now

Teaching note: IdentitiesOnly yes is the flag most tutorials skip. Without it, SSH will try every key in your agent, which can confuse servers with strict attempt limits. Always be explicit.

Phase 2 — Server Survey

Before deleting anything, map exactly what exists.

# Who's taking up disk?


du -sh /var/www/* | sort -rh


  
  
  What containers are running?


docker ps -a --format "table {{.Names}}\t{{.Status}}\t{{.Image}}"


  
  
  What networks exist?


docker network ls


  
  
  What volumes exist?


docker volume ls

What we found:

49 GB across 8 stale project folders in /var/www/
Three different Docker Compose setups from three different developers, all creating their own networks
11 nginx site configs, only 1 needed
An erpdstack-mysql container with live data that must not be touched

Teaching note: docker ps -aq --filter name=dstack matches substrings. erpdstack contains dstack. Always preview your filter before passing it to docker rm. We learned this the hard way — the erpdstack container was removed. The data volume survived, but the container had to be recreated. Always verify, never assume.

Phase 3 — Full Server Reset

We removed everything dstack-related except the erpdstack data volume and network.

# Stop and remove all containers (carefully — verified list first) docker stop $(docker ps -aq) && docker rm $(docker ps -aq) Remove stale named volumes docker volume rm dstack_sail-mysql dstack_sail-redis dstack_sail-kafka \ dstack_sail-grafana dstack_sail-prometheus Remove stale networks docker network rm dstack_sail Remove all project folders sudo rm -rf /var/www/dstack /var/www/dstack_api /var/www/dstack-services \ /var/www/dinesh-dstack-testing /var/www/nextjs \ /var/www/dsstack-admin-dash /var/www/dstack-admin-dashboard \ /var/www/dstack-dashboard Purge unused Docker images docker image prune -af Clean apt cache and journal logs

sudo apt-get clean sudo journalctl --vacuum-size=100M

Result: Disk dropped from 70 GB used (73%) to 21 GB (22%). 49 GB recovered in under ten minutes.

Phase 4 — Fresh Deployment

Clone the projects

cd /home/dstack dstack (Laravel) — uses GitHub deploy key 1 git clone git@github-main:orions-it/dstack_api.git dstack cd dstack && git checkout docker-standalone dstack-services (Node.js) — uses GitHub deploy key 2

cd /home/dstack git clone git@github-second:DishKief/taggo-services.git dstack-services

Configure staging .env files

Key differences from production:

# dstack/.env — staging overrides


APP_ENV=staging


APP_DEBUG=true


APP_URL=https://api-staging.dstack.ae


DB_DATABASE=dstack_staging


  
  
  dstack-services/.env


NODE_ENV=staging


LARAVEL_UPSTREAM_URL=http://laravel_dstack:81

Start the stack — order matters

Kafka and MySQL are dependencies. Start the infrastructure side first.

# Start dstack-services first (Kafka, Redis, MySQL, monitoring) cd /home/dstack/dstack-services docker compose up -d --build Wait ~60 seconds for MySQL and Kafka to become healthy, then start Laravel

cd /home/dstack/dstack docker compose --profile kafka up -d --build

Bug we hit: The queue, scheduler, and Reverb containers all have depends_on: service_healthy on MySQL. If you run docker compose up before MySQL is healthy, those containers start and immediately exit. The fix: wait for MySQL to show (healthy) in docker ps, then run docker compose up -d again. Docker Compose is idempotent — it only starts what isn't running.

External volumes must be pre-created

The dstack-services compose declares Prometheus and Grafana volumes as external: true. Docker will refuse to start if they don't exist.

docker volume create --name dstack_sail-prometheus


docker volume create --name dstack_sail-grafana

Teaching note: external: true in a Compose file means "this volume was created outside this file — don't manage its lifecycle." It's a design choice that prevents accidental deletion during docker compose down -v. The trade-off: you must remember to create it manually when deploying fresh.

Run migrations and bootstrap Laravel

cd /home/dstack/dstack Create the staging database docker compose exec mysql_dstack mysql -uroot -p -e \ "CREATE DATABASE IF NOT EXISTS dstack_staging CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;" Install PHP dependencies (without --no-dev — Telescope needs dev packages) docker compose exec laravel_dstack composer install Run all migrations docker compose exec laravel_dstack php artisan migrate --force Bootstrap docker compose exec laravel_dstack php artisan config:cache docker compose exec laravel_dstack php artisan storage:link Generate Passport keys and create password grant client

docker compose exec laravel_dstack php artisan passport:keys docker compose exec laravel_dstack php artisan passport:client --password --name="DStack Password Grant" --no-interaction

Bug we hit: Running composer install --no-dev failed because Laravel Telescope is in require-dev and the TelescopeServiceProvider is registered in bootstrap/providers.php. If the package isn't installed, every artisan command throws a fatal error. Lesson: on staging, always run a full composer install. Only strip --no-dev on production where Telescope genuinely isn't needed.

Phase 5 — SSL with Let's Encrypt

This is the step most tutorials get wrong. You cannot have an nginx config referencing SSL certificates that don't exist yet — nginx will refuse to start.

The correct order:

Step 1 — HTTP-only config first

server {


    listen 80;


    listen [::]:80;


    server_name api-staging.dstack.ae;


    location /.well-known/acme-challenge/ { root /var/www/certbot; }


    location / { return 301 https://$host$request_uri; }


}

sudo ln -s /etc/nginx/sites-available/api-staging.dstack.ae /etc/nginx/sites-enabled/


sudo nginx -t && sudo systemctl reload nginx

Step 2 — Issue the certificate

sudo certbot certonly --nginx -d api-staging.dstack.ae

Step 3 — Replace with full HTTPS config

server {


    listen 443 ssl http2;


    listen [::]:443 ssl;


    server_name api-staging.dstack.ae;

ssl_certificate     /etc/letsencrypt/live/api-staging.dstack.ae/fullchain.pem;
ssl_certificate_key /etc/letsencrypt/live/api-staging.dstack.ae/privkey.pem;
include             /etc/letsencrypt/options-ssl-nginx.conf;
ssl_dhparam         /etc/letsencrypt/ssl-dhparams.pem;

add_header X-Robots-Tag "noindex, nofollow" always;
client_max_body_size 100M;

location /realtime/ {
    proxy_pass         http://127.0.0.1:3010/;
    proxy_http_version 1.1;
    proxy_set_header   Upgrade    $http_upgrade;
    proxy_set_header   Connection "upgrade";
    proxy_set_header   Host              $host;
    proxy_set_header   X-Real-IP         $remote_addr;
    proxy_set_header   X-Forwarded-For   $proxy_add_x_forwarded_for;
    proxy_set_header   X-Forwarded-Proto $scheme;
    proxy_read_timeout 86400s;
}

location /app  { proxy_pass http://127.0.0.1:6001; ... }
location /apps { proxy_pass http://127.0.0.1:6001; ... }

location / {
    proxy_pass http://127.0.0.1:3006;
    proxy_http_version 1.1;
    proxy_set_header Host              $host;
    proxy_set_header X-Real-IP         $remote_addr;
    proxy_set_header X-Forwarded-For   $proxy_add_x_forwarded_for;
    proxy_set_header X-Forwarded-Proto $scheme;
}


}

Teaching note: Notice that / proxies to port 3006, not directly to Laravel. The Node.js gateway sits in front of Laravel and handles auth token validation, correlation ID injection, and request routing. Laravel is never directly exposed to the internet. This is the production architecture mirrored exactly.

Phase 6 — Staging Identity Headers

We added two more subdomains — grafana-staging.dstack.ae (monitoring) and realtime-staging.dstack.ae (WebSocket service). To make it immediately obvious these are staging, every nginx config adds identification headers:

add_header X-Robots-Tag  "noindex, nofollow"    always;


add_header X-Environment "staging"              always;


add_header X-Backend     "api-staging.taggo.ae" always;

You can verify this on any response:

curl -sI https://grafana-staging.dstack.ae/login | grep -E "x-environment|x-backend"


  
  
  x-environment: staging




  

  

  x-backend: api-staging.dstack.ae

Teaching note: X-Robots-Tag: noindex, nofollow prevents search engines from indexing staging. Without it, a staging endpoint once appeared in Google search results — exposing internal API structure. Defence in depth: even if Cloudflare DNS accidentally proxies staging, these headers are a last line.

Phase 7 — Database Seeding

With the stack running, seed the database so there's something to log in with.

cd /home/dstack/dstack Core: permissions, roles, admin + dummy users docker compose exec laravel_dstack php artisan db:seed Module seeders: booking config, locations, currency docker compose exec laravel_dstack php artisan db:seed \ --class="Modules\Locations\Database\Seeders\LocationsDatabaseSeeder" docker compose exec laravel_dstack php artisan db:seed \ --class="Modules\Currency\Database\Seeders\CurrencyDatabaseSeeder"

docker compose exec laravel_dstack php artisan db:seed <br> --class="Modules\BookingConfig\Database\Seeders\BookingConfigDatabaseSeeder"

Bug: Role assignment silently failed

After seeding, the admin user had no role. This is because Spatie Permission in this project is team-scoped — every role assignment requires a team_id. The seeder was running $user->assignRole('sys_admin') without setting the team context first.

// Wrong — team_id would be null, MySQL throws a NOT NULL constraint


$user->assignRole('sys_admin');

// Correct — set team context first


app(PermissionRegistrar::class)->setPermissionsTeamId($user->current_team_id);


$user->assignRole('sys_admin');

You can verify the actual state by checking model_has_roles directly:

docker compose exec laravel_dstack php artisan tinker --execute "


DB::table('model_has_roles')


    ->join('roles', 'roles.id', '=', 'model_has_roles.role_id')


    ->select('model_has_roles.model_id', 'roles.name as role', 'model_has_roles.team_id')


    ->get()->each(fn(\$r) => print(\$r->model_id.' | '.\$r->role.' | team='.\$r->team_id.PHP_EOL));


"

Teaching note: When a relationship returns empty in Eloquent but you're sure the data should be there, always go one level deeper and query the pivot table directly. Eloquent's $user->roles will silently return an empty collection if the team scope isn't set — no error, no warning. Direct SQL never lies.

Phase 8 — The Login That Refused to Work

This phase deserves its own section. Getting the admin login working required diagnosing six separate issues in sequence. Each one was a lesson.

Bug 1: Wrong endpoint

We called POST /api/v1/public/auth/login with the sys_admin credentials and got messages.invalid_user.

Reading the frontend login controller revealed:

$allowedRoles = [


    RoleEnum::agent_admin->name,


    RoleEnum::customer->name,


    RoleEnum::dealer->name,


    RoleEnum::driver->name,


    // sys_admin is intentionally NOT here


];

if (! $user->hasAnyRole($allowedRoles)) {


    return self::error(__('messages.invalid_user'), 401);


}

The admin has a dedicated endpoint: POST /api/v1/admin/auth/login.

Lesson: Before debugging credentials, always verify you're hitting the right endpoint. The route file is the source of truth — grep -n "login" routes/apiAdmin.php takes five seconds.

Bug 2: artisan serve was in FATAL state

The admin endpoint returned upstream_unavailable from the gateway. Testing directly inside the container confirmed port 81 was not listening. The Laravel container showed as running but no PHP process was active.

docker logs dstack-laravel_dstack-1 --tail 20


  
  
  WARN exited: php (exit status 255; not expected)




  

  

  WARN gave up: php entered FATAL state, too many start retries too quickly

Supervisor had given up on the PHP process because it crashed on startup before composer install ran (vendor/autoload.php didn't exist at container first boot). After we ran composer install, the vendor directory appeared — but supervisor had already stopped retrying.

Fix:

docker compose restart laravel_dstack



  

  

  Supervisor retries the PHP process fresh — this time vendor exists → success

Lesson: A container showing as "running" only means PID 1 (supervisor) is alive. The actual application process inside might be dead. Always verify with docker logs before assuming the app is up.

Bug 3: artisan serve visible, but login still returns 500

After restarting, the server was up but login returned "An error occurred. Please try again later.". The global exception handler was swallowing the real error:

// bootstrap/app.php

if (! app()->environment(['local', 'testing']) && $response->getStatusCode() === 500) {


    return response()->json(['status' => false, 'message' => 'An error occurred. Please try again later.'], 500);


}

On staging, every 500 becomes this generic message. We couldn't see the real error. Even Log::error() in the controller wasn't writing because:

Bug 4: Log file was root-owned, sail user couldn't write

ls -la storage/logs/


  
  
  -rw-r--r-- 1 root root 6310 Jul 18 10:57 laravel-2026-07-18.log




  

  

  ^^^^ sail cannot write here

The log file was created by an artisan command that ran as root (inside the container). The artisan serve process runs as the sail user. 644 means owner-write only. The sail user silently failed to write any logs.

docker exec dstack-laravel_dstack-1 chown sail:sail /var/www/html/storage/logs/laravel-2026-07-18.log

Now the real error appeared in the log:

[2026-07-18 16:44:06] staging.ERROR: Admin login token issue

{"error":"Key path \"file:///var/www/html/storage/passport/oauth-private.key\" does not exist or is not readable"}

Lesson: When debugging a silent 500, the very first thing to check is whether the log file is writable by the process that's serving requests. A log that can't write is worse than no log — it creates the illusion that nothing is wrong.

Bug 5: Passport private key was root-owned

passport:keys ran as root and created the keys with -rw------- root ownership. The sail user who runs artisan serve couldn't read the private key to sign JWTs.

docker exec dstack-laravel_dstack-1 ls -la /var/www/html/storage/passport/ -rw------- 1 root root 3322 oauth-private.key ← sail cannot read -rw-rw---- 1 root root 812 oauth-public.key Fix ownership and permissions docker exec dstack-laravel_dstack-1 chown sail:sail \ /var/www/html/storage/passport/oauth-private.key \ /var/www/html/storage/passport/oauth-public.key

docker exec dstack-laravel_dstack-1 chmod 600 /var/www/html/storage/passport/oauth-private.key docker exec dstack-laravel_dstack-1 chmod 644 /var/www/html/storage/passport/oauth-public.key

Teaching note: Passport (and any RSA/EC key-based system) requires strict permissions on the private key. Passport v13 specifically checks and will reject 644 on the private key — it wants 600 or 660. The public key can be world-readable (644) since it's meant to be distributed.

Bug 6: Missing Passport personal access client

After fixing the key permissions, login failed again with a different error. Running the token issuance code manually in tinker revealed:

RuntimeException: Personal access client not found for 'users' user provider. Please create one.

The passport:client --password we ran earlier created a password grant client. But $user->createToken() (which the app uses internally) requires a separate personal access client.

docker compose exec laravel_taggo php artisan passport:client <br>

  --personal <br>

  --name="DStack Personal Access Client" <br>

  --no-interaction

Verify the client was created with the correct grant type:

docker compose exec laravel_dstack php artisan tinker --execute "


DB::table('oauth_clients')->get()->each(fn(\$c) =>


    print(\$c->id.' | '.\$c->name.' | '.\$c->grant_types.PHP_EOL)


);


"




  

  

  019f7515... | DStack Personal Access Client | ["personal_access"]

Teaching note: In Passport v13, there are three distinct client types: password grant (for mobile apps using username+password), personal_access (for server-side token creation via $user->createToken()), and authorization_code (for OAuth flows). Most apps need all three. Running only passport:client --password is a common mistake that only surfaces when you try to programmatically issue tokens.

Final Verification

curl -s -X POST https://api-staging.dstack.ae/api/v1/admin/auth/login \

-H "Content-Type: application/json" <br>

  -H "Accept: application/json" <br>

  -d '{"email":"admin@dineshstack.com","password":"Admin@dstack2027"}'

{


    "status": true,


    "data": {


        "account_status": "active",


        "token": {


            "access_token": "eyJ0eXAiOiJKV1Qi...",


            "token_type": "Bearer",


            "access_token_expires_in": 86400


        },


        "user": {


            "email": "admin@dineshstack.com",


            "role": "sys_admin",


            "permissions": [...]


        }


    }


}

What the Final State Looks Like

URL	Service	Port
https://api-staging.dstack.ae	Laravel API (via Node gateway)	3006 → Laravel:81
https://grafana-staging.dstack.ae	Grafana monitoring dashboard	3003
https://realtime-staging.dstack.ae	WebSocket / node-realtime	3010

Docker: 19 containers, all on a single dstack_sail network.
Disk: 31 GB / 96 GB (32%) — recovered 39 GB from the old state.
SSL: Let's Encrypt on all three domains, auto-renewing.

Lessons Condensed

Survey before you touch anything. df -h, docker ps, docker network ls — know the battlefield.
Docker filter substrings bite. --filter name=dstack matches erpdstack. Always echo your list before piping to rm.
Order matters for SSL. HTTP-only config → certbot → HTTPS config. Never the other way.
depends_on: healthy is not a guarantee. If dependencies weren't healthy when up first ran, run it again. Idempotent.
"Running" container ≠ working app. Supervisor alive ≠ PHP alive. Check docker logs.
Log file permissions are invisible failures. A root-owned log file silently eats every error. Always verify ls -la storage/logs/ against the user running your web process.
Passport needs two clients. --password for mobile auth flows. --personal for server-side $user->createToken(). Both. Always.
RSA private keys must be readable by the web process user — and nothing else. 600 on the private key, owned by the process user. Non-negotiable.
Team-scoped permissions require team context on every operation. setPermissionsTeamId() before any role assign or check. Spatie fails silently without it.
When stuck on a generic 500, reproduce in tinker step by step. Narrow the failure to a single line before reading any framework source.

From Zero to Staging: A Real-World Laravel + Node.js Docker Deployment (With Every Bug We Hit)

From Zero to Staging: A Real-World Laravel + Node.js Docker Deployment (With Every Bug We Hit)

The Setup

Phase 1 — SSH Hardening

On your local machine

Copy the public key to the server

On the server — disable password auth

Install fail2ban

Phase 2 — Server Survey

What containers are running?

What networks exist?

What volumes exist?

Phase 3 — Full Server Reset

Remove stale named volumes

Remove stale networks

Remove all project folders

Purge unused Docker images

Clean apt cache and journal logs

Phase 4 — Fresh Deployment

Clone the projects

dstack (Laravel) — uses GitHub deploy key 1

dstack-services (Node.js) — uses GitHub deploy key 2

Configure staging .env files

dstack-services/.env

Start the stack — order matters

Wait ~60 seconds for MySQL and Kafka to become healthy, then start Laravel

External volumes must be pre-created

Run migrations and bootstrap Laravel

Create the staging database

Install PHP dependencies (without --no-dev — Telescope needs dev packages)

Run all migrations

Bootstrap

Generate Passport keys and create password grant client

Phase 5 — SSL with Let's Encrypt

Step 1 — HTTP-only config first

Step 2 — Issue the certificate

Step 3 — Replace with full HTTPS config

Phase 6 — Staging Identity Headers

x-environment: staging

x-backend: api-staging.dstack.ae

Phase 7 — Database Seeding

Core: permissions, roles, admin + dummy users

Module seeders: booking config, locations, currency

Bug: Role assignment silently failed

Phase 8 — The Login That Refused to Work

Bug 1: Wrong endpoint

Bug 2: artisan serve was in FATAL state

WARN exited: php (exit status 255; not expected)

WARN gave up: php entered FATAL state, too many start retries too quickly

Supervisor retries the PHP process fresh — this time vendor exists → success

Bug 3: artisan serve visible, but login still returns 500

Bug 4: Log file was root-owned, sail user couldn't write

-rw-r--r-- 1 root root 6310 Jul 18 10:57 laravel-2026-07-18.log

^^^^ sail cannot write here

Bug 5: Passport private key was root-owned

-rw------- 1 root root 3322 oauth-private.key ← sail cannot read

-rw-rw---- 1 root root 812 oauth-public.key

Fix ownership and permissions

Bug 6: Missing Passport personal access client

019f7515... | DStack Personal Access Client | ["personal_access"]

Final Verification

What the Final State Looks Like

Lessons Condensed

Tags

Author

Stats

Published

You Might Also Like

Our hourly deploy woke 105 times, found work every time, and did none of it

Rollback Strategy: Questions to Ask First

Lovable Hosting: What You Get and Pay For

Lovable app not deploying or broken after deploy: 4 env causes

Optimizing CI/CD Workflows for Microservices: Image Tagging, Deployment, and Toolchain Strategies Across Environments

GitHub Actions API Bug: Stale Workflow Data Threatens Developer Quality and Deployment Integrity

`x-backend: api-staging.dstack.ae`

`WARN gave up: php entered FATAL state, too many start retries too quickly`

`Supervisor retries the PHP process fresh — this time vendor exists → success`

`^^^^ sail cannot write here`

`019f7515... | DStack Personal Access Client | ["personal_access"]`