SentinelIndia
AI-powered geopolitical narrative intelligence platform that monitors, classifies, and counters disinformation targeting India across social media using multi-LLM classification, graph-based network detection, and real-time alerting.
Overview
SentinelIndia — AI-Powered Geopolitical Narrative Intelligence
SentinelIndia monitors, classifies, and counters disinformation targeting India across global social media. It surfaces coordinated propaganda networks, state-sponsored narratives, and misinformation in real-time — processing thousands of posts per minute across Twitter, YouTube, and Reddit in 8+ languages.
Built for Indian government bodies (MEA, PIB, embassies), think tanks, MNCs, and diaspora organizations.
Architecture
The platform uses an event-driven microservices pattern with purpose-built databases for each workload: PostgreSQL 16 (pgvector) — transactional data: posts, classifications, alerts. Partitioned by month for query performance (O(n) becomes O(n/12) for time-range scans). Neo4j 5 — graph analysis: account networks, coordinated clusters, temporal linking. Graph queries are 100x faster than relational joins for relationship traversal. Elasticsearch 8 — multilingual full-text search with ICU tokenization across 8+ languages. Redis 7 — BullMQ job queues, rate limiting (sliding window), deduplication (SHA-256 bloom filters with 7-day TTL), caching, and Pub/Sub for real-time dashboard updates.
Frontend is Next.js 14 (App Router) with tRPC for end-to-end type safety — frontend and backend share TypeScript types automatically, eliminating API contract mismatches at compile-time.
Multi-LLM Classification Pipeline
The classification system uses a tiered model strategy: Claude Sonnet — primary classifier. Batches 10 posts per API call, classifies into 5 categories (propaganda, misinformation, coordinated, legitimate, irrelevant) with 15 subcategories, severity scores (1-10), confidence levels, narrative themes, and coordination signals. Ollama (self-hosted) — fallback classifier for cost optimization at high volume (30-50 posts/minute on open-source models like Llama 3 or Mistral). OpenAI text-embedding-3-small — generates 1536-dimensional embeddings for semantic narrative clustering via K-means.
A pre-filtering layer skips clearly irrelevant posts (emoji-only, too short, non-target language) before any LLM call — reducing API costs by 30%. Token budgets enforce daily spend caps ($1000/day default = 300-500K posts/day capacity).
Narrative Clustering & Network Detection
Post embeddings feed into a K-means clusterer with automatic k-selection (sqrt(n/2), clamped 2-20). Clusters are labeled from top themes/entities — "Kashmir Propaganda", "Hindu Extremism", "Military Defeat Narrative" — giving analysts a macro view of active disinformation campaigns.
Neo4j detects coordinated behavior through graph patterns: Temporal linking: accounts posting the same narrative within a 5-minute window (3+ co-occurrences = coordination signal) Community detection: Louvain algorithm partitions account graphs by link density, revealing organized networks Risk scoring: weighted composite of flagged-post ratio (30%), average severity (40%), and coordination score (30%)
Real-Time Alerting Pipeline
End-to-end latency from social media post to analyst alert: 5 minutes.
1. Ingestion worker fetches posts every 5 minutes (Twitter), 10 minutes (Reddit), 15 minutes (YouTube — quota-conscious at 10K units/day) 2. Posts normalized to UnifiedPost schema, deduplicated via SHA-256 content hash, stored in PostgreSQL + Elasticsearch 3. Classification worker processes batches every 2 minutes 4. Alert rules evaluate every 5 minutes: threshold (severity >= X), spike (post volume up >Y%), coordination detected 5. Notifications fire via email, webhook, and in-app (Redis Pub/Sub → WebSocket)
Multilingual Intelligence
Custom language detection using Unicode script ranges — Devanagari, Arabic, CJK, Bengali, Tamil, Telugu, Kannada, Malayalam — with marker-word fallback heuristics. No external API cost. 99% accuracy for Indian languages.
Elasticsearch's ICU analyzer handles multilingual tokenization for full-text search across all detected languages.
Security & Multi-Tenancy RBAC: 4 roles (admin, analyst, viewer, apiconsumer) Prisma middleware automatically scopes all queries to orgId — prevents cross-tenant data leaks without manual where clauses MFA (TOTP) with Google Authenticator/Authy support Account lockout after 5 failed attempts (15-minute cooldown) Audit logging for government compliance (userId, action, resource, IP, userAgent) AWS ap-south-1 (Mumbai) for Indian data residency requirements Encryption: at rest (AWS KMS), in transit (TLS 1.3), secrets in AWS Secrets Manager
Worker Orchestration
BullMQ manages all background processing with repeatable jobs:
| Schedule | Worker | Concurrency | |----------|--------|-------------| | Every 5 min | Twitter keyword search | 3 | | Every 15 min | Twitter account tracking | 1 | | Every 15 min | YouTube search (quota-limited) | 1 | | Every 10 min | Reddit scrape | 1 | | Every 2 min | Classification (rate-limited by LLM budget) | 1 | | Every 5 min | Alert evaluation | 1 | | 06:00 UTC daily | Daily briefing digest | 1 | | 08:00 UTC Monday | Weekly summary | 1 |
Workers scale independently on ECS Fargate — ingestion can run 3-10 tasks while classification runs 2-8, each auto-scaling on queue depth thresholds.
Briefing & Counter-Evidence Engine
Automated intelligence reports generated on schedule: Daily digest (06:00 UTC): top narratives, key findings, severity trends Weekly summary (Monday 08:00 UTC): comprehensive threat assessment with recommendations and talking points Counter-evidence reports: Claude-generated analysis with citations and counter-arguments for specific disinformation claims, exportable as PDF for government distribution
28,800 lines of TypeScript across services, workers, and frontend.
Learnings
1. Polyglot Database Architecture — don't shoehorn everything into one database. PostgreSQL for transactional data, Neo4j for relationship traversal (100x faster than SQL joins for network analysis), Elasticsearch for multilingual full-text search, Redis for queues and ephemeral data. Each database solves the problem it was designed for.
2. Pre-Filtering Before LLM Calls Saves 30%+ Costs — skipping emoji-only posts, short text, and non-target languages before sending to Claude reduces API spend dramatically. False negatives are acceptable because other signals (engagement, hashtags, network patterns) catch them.
3. Token Budget Enforcement — tracking daily LLM spend and hard-stopping classification when the budget is exceeded turns unpredictable AI costs into predictable OpEx. Without this, a viral event could trigger $50K in unexpected API calls.
4. Temporal Coordination Detection via Graph Queries — "accounts posting the same narrative within 5 minutes, 3+ co-occurrences" is a clean heuristic for detecting bot networks. Graph databases make this query trivial; relational databases make it painful.
5. Automatic k-Selection for Clustering — using sqrt(n/2) clamped to 2-20 as a heuristic for K-means avoids the manual tuning problem. Good enough for production narrative grouping, avoids the overhead of more sophisticated methods like HDBSCAN.
6. Prisma Middleware for Multi-Tenant Isolation — automatically injecting orgId into every query via ORM middleware prevents cross-tenant data leaks without relying on developers remembering to add it to every query. Defense in depth at the data layer.
7. Content Hash Deduplication — SHA-256 hashing (platform + content + authorId) with Redis set lookups and 7-day TTL prevents storing the same cross-posted content hundreds of times. Reduces both storage and classification costs.
8. Event-Driven Worker Independence — each worker (ingestion, classification, graph sync, alerting) scales independently. If the alert service is slow, tweets still get ingested. Easy to add new workers without touching existing code.
9. Custom Language Detection via Unicode Ranges — for Indian languages, Unicode script detection (Devanagari, Arabic, CJK, etc.) is free and 99% accurate. No need for expensive language detection APIs when the character set tells you everything.
10. Materialized Views for Dashboard Performance — pre-aggregating daily stats in PostgreSQL materialized views (refreshed every 15 minutes) speeds up dashboard queries 100x. Read-heavy analytics workloads should never hit raw tables.
Tags: Next.js, TypeScript, tRPC, Prisma, PostgreSQL, pgvector, Neo4j, Elasticsearch, Redis, BullMQ, Claude AI, OpenAI Embeddings, K-means Clustering, Docker, AWS ECS Fargate, Prometheus, Grafana