Public-sector persona RAG
An agentic RAG pipeline that scrapes government PDFs, cleans + extracts with LLMs, embeds, and serves a stateful chatbot for persona development.
01
At a glance
24h
refresh cadence
PDF → vec
end-to-end
Stateful
agent memory
Multi-source
ingestion
02
How it works
01ScrapeArgo daily job hits gov portals, downloads new PDFs.
02ExtractLLM extracts structured fields, normalises edge cases.
03EmbedChunk + vectorise + upsert to Qdrant.
04ChatLangGraph agent retrieves + answers with state.
03
Architecture
04
Features
ScrapeScrapy-based ingestion across multiple government portals on schedule.
CleanLLM-driven extraction normalises wildly inconsistent PDF formats.
Chunk + embedSemantic chunking + OpenAI/Gemini embeddings into Qdrant.
AgentLangGraph stateful chatbot with multi-turn memory and tool-use.
05
Full stack
LangGraphOpenAI SDKGemini SDKQdrantFastAPIArgo WorkflowsGKEScrapyPandas
next project →03WhatsApp scam-detection bot↗