Building an LLM Twin (and Accidentally Building Chaos) ☕

I decided to build an LLM Twin using a clean ETL + FTI architecture, thinking it would be structured, scalable, and elegant. It started well. I designed a proper ETL pipeline: extract data from blogs, GitHub, and posts clean and normalize everything store it nicely in a database Simple, right? Then reality happened. My “clean data pipeline” slowly became: random HTML scraping inconsistent formats mysterious edge cases But technically… it was still an ETL pipeline 😅 The idea was smart though: Instead of overcomplicating things, I reduced everything into just three types: articles repositories posts Which meant I could scale easily later without rewriting everything. That part actually worked. But here’s the funny part. I thought I was building a system that understands data. What I really built was a system that shows me: how messy real-world data is how optimistic my assumptions were and how “simple architecture” becomes complex in 2 days Final Thought You don’t build an LLM system in