neweb.ai
neweb.ai generates your site, deploys it, ranks it on Google, and keeps improving it — all without you lifting a finger.
Overview
neweb.ai — AI-Powered WordPress SaaS Platform
Overview
neweb.ai is a full-stack SaaS platform that automates WordPress website creation and blog content generation using multiple AI models. Users configure their business details, and the platform provisions a live WordPress site, generates SEO-optimized blog content on autopilot, tracks keyword rankings, and audits content performance — all through a real-time dashboard.
Stack: Node.js/Express, MongoDB, PostgreSQL, Redis, BullMQ, React 19, Redux Toolkit, TanStack Query v5, Socket.IO, Stripe, Clerk Auth, Docker, Prometheus + Grafana
Live: dashboard.neweb.ai | landing.neweb.ai
Architecture Monorepo with 5 packages: backend, user dashboard, admin dashboard, landing site, WordPress themes Dual databases: MongoDB (primary data), PostgreSQL (secondary/analytics) Redis powers BullMQ job queues, Socket.IO adapter, and SEO data caching
Core Features
1. WordPress Provisioning Pipeline (12-Step Orchestration)
The standout engineering feat — a 12-step sequential pipeline that provisions a full WordPress site from scratch:
1. Create Subdomain (mybiz-f4d2-a7x9.newebai.com) 2. Install WordPress (admin creds held in-memory) 3. Install Theme (dynamically resolved by business type) 4. Install Plugins 5. Insert Business Details (via WP REST API) 6. Shortcode Modifications 7. Health Check (graceful if DNS hasn't propagated) 8. Add Page Templates 9. Import Global Styles 10. Load Environment Config 11. Core Setup 12. Editor Configuration
Key design decisions: Full-sequence retry, not per-step — if step 8 fails, the entire pipeline retries from step 1 after cleaning up the failed subdomain. This guarantees atomic consistency: you never get a half-built site. Max 3 attempts with full cleanup between retries Real-time progress via Socket.IO (wp:build:progress events streamed per step) Credentials emailed as PDF after successful provisioning — avoids storing plaintext admin passwords in logs
2. AI Blog Generation & Autopilot
Multi-model LLM strategy: Azure OpenAI (gpt-4o-mini-2) — bulk content generation (fast, cost-effective) Claude Haiku 4.5 — content auditing and ranking-drop diagnosis (better at analysis) Runware SDK — AI image generation
The generation pipeline enforces structured JSON output with strict constraints: SEO-optimized titles (<60 chars with focus keyword) Meta descriptions (exactly 150-160 chars) Target keyword density of 1-2% Word count bands: short (500-600), medium (900-1100), long (1400-1600) Suggested internal links and image placements
Autopilot mode: Users configure a schedule (daily/weekly/monthly) with a pre-populated topic pool. The system generates and optionally auto-publishes content, pausing automatically after 3+ consecutive failures.
3. SEO Intelligence DataForSEO integration for keyword rank tracking (SERP depth 100) Backlink analysis — total backlinks, referring domains, domain authority Lighthouse audits — performance, accessibility, SEO, best practices scores On-page SEO scoring — real-time analysis of title, meta, keyword density, headings, word count Weekly content audits (Wednesday 4AM UTC) — Claude Haiku analyzes blogs with ranking drops, diagnoses issues, suggests fixes with priority levels
All SEO APIs use Promise.allSettled() for graceful degradation — missing ranking data never blocks site creation or publishing.
4. Subscription & Marketplace Stripe integration with webhook signature validation Accumulated credits model for yearly plans — monthly credits added to a pool rather than hard per-action limits Add-on system — purchased add-ons expand effective limits via getEffectiveLimits() utility Stripe Connect marketplace — agencies enable Connect to receive a percentage of client billing Usage tracking with canPerformAction() checks before every billable operation
5. Agency & RBAC System
Five user roles: user, agency, admin, moderator, manager
The agency model is elegant: Agency users have agencyId: null (they own the agency) Managed users have agencyId: <agencyClerkId> Resources track both clerkId (direct owner) and agencyClerkId Ownership middleware checks agency relationship before falling back to direct ownership Admins/moderators/managers bypass all ownership checks
Infrastructure & Operations
Job Queue Architecture (BullMQ)
| Schedule | Job | Purpose | |----------|-----|---------| | Hourly @ :15 | processBlogSchedules | Trigger blog autopilot | | Monday @ 03:00 | rank-check | Weekly keyword tracking | | Wednesday @ 04:00 | content-audit | Analyze ranking drops | | Daily @ 00:00 | Subscription housekeeping | Expire trials, reset yearly usage | | Every 5 min | Domain workers | DNS/SSL verification |
Jobs use explicit jobId for idempotency: rank:${siteId}:${timestamp} prevents duplicates if cron fires twice.
Monitoring Stack Prometheus — HTTP request duration histograms, total request counters Grafana — dashboards accessible at /grafana/ Loki — structured log aggregation via Winston Custom error monitor — classifies severity, throttles identical errors for 5 minutes, only alerts on non-operational errors, sanitizes sensitive fields
Real-Time Architecture (Socket.IO) JWT verification via @clerk/backend verifyToken() on handshake Users auto-joined to user:${clerkId} rooms (enables multi-tab sync) Redis adapter for horizontal scaling across server instances Events: wp:build:progress, wp:build:done, wp:build:failed, notification:, chat:message
Learnings
Key Professional Takeaways
1. Full-Sequence Retry > Per-Step Retry for Multi-Step Pipelines
When you have a tightly coupled pipeline (like WordPress provisioning), retrying individual steps creates inconsistent state. Retrying the full sequence with cleanup between attempts guarantees atomicity. The tradeoff is speed — but correctness matters more when you're provisioning infrastructure.
When to apply: Any multi-step process where partial completion leaves the system in an invalid or hard-to-recover state (database migrations, infrastructure provisioning, payment flows).
2. Multi-Model LLM Strategy Optimizes Cost and Quality
Using different AI models for different tasks isn't just about vendor diversification — it's about matching model strengths to use cases. Fast/cheap models (gpt-4o-mini) handle bulk generation; analytical models (Claude) handle auditing where reasoning quality matters more than speed.
When to apply: Any AI-heavy product. Profile your LLM calls by volume, latency requirements, and reasoning complexity, then assign models accordingly.
3. Accumulated Credits > Hard Limits for Subscription Flexibility
Instead of "5 blogs per month" as a hard cap, pooling credits allows users to front-load or spread usage. This reduces churn from users who hit limits early in their billing cycle while still enforcing overall quotas.
When to apply: Any usage-based SaaS where customers have variable usage patterns within billing periods.
4. Graceful Degradation with Promise.allSettled()
Third-party APIs will fail. The pattern of wrapping non-critical API calls in Promise.allSettled() and continuing with partial data keeps your core flow running. DataForSEO being down shouldn't prevent blog publishing.
When to apply: Anywhere you aggregate data from multiple external sources. Identify which data is critical vs. enrichment, and only block on the critical path.
5. Error Throttling Prevents Alert Fatigue
Deduplicating identical errors within a 5-minute window means your team gets one alert for "MongoDB connection timeout" instead of 500. Combined with severity classification (only alert on non-operational errors), this keeps monitoring actionable.
When to apply: Any production system with error alerting. Without throttling, your team starts ignoring alerts — which is worse than no monitoring at all.
6. Job Idempotency via Explicit jobId
BullMQ (and most queue systems) accept a jobId that prevents duplicate enqueuing. Using deterministic IDs like rank:${siteId}:${timestamp} ensures cron-triggered jobs don't double-execute if the scheduler fires twice.
When to apply: Any queue-based system with scheduled or event-triggered jobs. Always ask: "What happens if this job gets enqueued twice?"
7. Agency/Ownership Middleware Pattern
The three-tier ownership check (admin bypass → agency relationship → direct ownership) is a clean, composable pattern for multi-tenant SaaS. It's implemented as middleware, keeping route handlers free of authorization logic.
When to apply: Any platform with B2B features where one organization manages resources on behalf of others (agencies, resellers, white-label products).
8. In-Memory Credential Handling for Provisioning
WordPress admin credentials are generated, used during setup, saved to the database after successful provisioning, then emailed as a PDF — never logged or stored in plaintext during the process. This minimizes the window where credentials exist in an insecure state.
When to apply: Any automated provisioning that generates credentials. Think about the lifecycle: generation → usage → persistence → delivery, and minimize exposure at each stage.
9. Topic Pools Decouple Generation from Scheduling
Rather than generating topics on-the-fly during scheduled runs (which can fail and waste a generation credit), pre-populating a topic pool lets users curate content direction. The scheduler just pulls from the pool. Empty pool? Generate one topic, but the user controls the editorial pipeline.
When to apply: Any automated content or task system. Separating "what to do" from "when to do it" gives users control without sacrificing automation.
10. Metadata Maps for Schema Flexibility
Using metadata: Map on MongoDB documents allows storing arbitrary data without schema migrations. WP setup logs, publish errors, custom fields — all go in metadata. The core schema stays clean while edge cases are handled flexibly.
When to apply: Any document-based system where different entities of the same type need different auxiliary data. Prefer typed fields for common data, metadata maps for the long tail.
Tags: Next.js, React, Express, MongoDB, Redis, BullMQ, Socket.io, Azure OpenAI, Claude API, Stripe, Clerk, WordPress, Vite, Tailwind, Docker, Prometheus