Show HN: Laminar – Open-Source DataDog + PostHog for LLM Apps, Built in Rust
https://github.com/lmnr-ai/lmnrHey HN, we’re Robert, Din and Temirlan from Laminar (https://www.lmnr.ai), an open-source observability and analytics platform for complex LLM apps. It’s designed to be fast, reliable, and scalable. The stack is RabbitMQ for message queues, Postgres for storage, Clickhouse for analytics, Qdrant for semantic search - all powered by Rust.
How is Laminar different from the swarm of other “LLM observability” platforms?
On the observability part, we’re focused on handling full execution traces, not just LLM calls. We built a Rust ingestor for OpenTelemetry (Otel) spans with GenAI semantic conventions. As LLM apps get more complex (think Agents with hundreds of LLM and function calls, or complex RAG pipelines), full tracing is critical. With Otel spans, we can: 1. Cover the entire execution trace. 2. Keep the platform future-proof 3. Leverage an amazing OpenLLMetry (https://github.com/traceloop/openllmetry), open-source package for span production.
The key difference is that we tie text analytics directly to execution traces. Rich text data makes LLM traces unique, so we let you track “semantic metrics” (like what your AI agent is actually saying) and connect those metrics to where they happen in the trace. If you want to know if your AI drive-through agent made an upsell, you can design an LLM extraction pipeline in our builder (more on it later), host it on Laminar, and handle everything from event requests to output logging. Processing requests simply come as events in the Otel span.
We think it’s a win to separate core app logic from LLM event processing. Most devs don’t want to manage background queues for LLM analytics processing but still want insights into how their Agents or RAGs are working.
Our Pipeline Builder uses graph UI where nodes are LLM and util functions, and edges showing data flow. We built a custom task execution engine with support of parallel branch executions, cycles and branches (it’s overkill for simple pipelines, but it’s extremely cool and we’ve spent a lot of time designing a robust engine). You can also call pipelines directly as API endpoints. We found them to be extremely useful for iterating on and separating LLM logic. Laminar also traces pipeline directly, which removes the overhead of sending large outputs over the network.
One thing missing from all LLM observability platforms right now is an adequate search over traces. We’re attacking this problem by indexing each span in a vector DB and performing hybrid search at query time. This feature is still in beta, but we think it’s gonna be crucial part of our platform going forward.
We also support evaluations. We loved the “run everything locally, send results to a server” approach from Braintrust and Weights & Biases, so we did that too: a simple SDK and nice dashboards to track everything. Evals are still early, but we’re pushing hard on them.
Our goal is to make Laminar the Supabase for LLMOps - the go-to open-source comprehensive platform for all things LLMs / GenAI. In it’s current shape, Laminar is just few weeks old and developing rapidly, we’d love any feedback or for you to give Laminar a try in your LLM projects!
- Everything is LLMs these days. LLMs this, LLMs that. Am I really missing out something from these muted models? Back when it was released, they were so much capable but now everything is muted to the point they are mostly autocomplete on steroids.
How can adding analytics to a system that is designed to act like humans produce any good? What is the goal here? Could you clarify why would some need to analyze LLMs out of all the things?
> Rich text data makes LLM traces unique, so we let you track “semantic metrics” (like what your AI agent is actually saying) and connect those metrics to where they happen in the trace
But why does it matter? Because at the current state these are muted LLMs overseen by the big company. We have very little to control the behavior and whatever we give it, it will mostly be 'politically' correct.
> One thing missing from all LLM observability platforms right now is an adequate search over traces.
Again, why do we need to evaluate LLMs? Unless you are working in a security, I see no purpose because these models aren't as capable as they used to be. Everything is muted.
For context: I don't even need to prompt engineer these days because it just gives similar result by using the default prompt. My prompts these are literally three words because it gets more of the job done that way than giving elaborate prompt with precise example and context.
-- findingMeaning Reply - > Everything is LLMs these days. LLMs this, LLMs that. Am I really missing out something from these muted models? Back when it was released, they were so much capable but now everything is muted to the point they are mostly autocomplete on steroids.
it was my experience, too, then I tried out that cursor thing and turns out a well designed UX around claude 3.5 is the bees knees. it really does work, highly recommend the free trial. YMMV of course depending on what you work on; I tested it strictly on Python.
-- baq Reply - They're not "muted". You just got used to them and figured out that they don't actually generete knew knowledge or information, they only give a statistically average summary of the top Google query. (I.e., they are super bland, boring and predictable.)
-- otabdeveloper4 Reply - Hey there, apologies for the late reply.
> Could you clarify why would some need to analyze LLMs out of all the things?
When you want to understand trends of the output of your Agent / RAG on scale, without looking manually at each trace, you need to another LLM to process the output. For instance, you want to understand what is the most common topic discussed with your agent. You can prompt another LLM to extract this info, Laminar will host everything, and turn this data into metrics.
> Why do we need to evaluate LLMs?
You right, devs who want to evaluate output of the LLM apps, truly care about the quality or some other metric. For this kind of cases evals are invaluable. Good example would be, AI drive-through agents or AI voice agents for mortgages (use cases we've seen on Laminar)
-- skull8888888 Reply - Topic modelling and classifications are real problems in LLM observability and evaluation, glad to see a platform doing this.
I see that you have chained prompts, does that mean I can define agents and functions inside the platform without having it in the code?
-- Oras Reply - Yes! Our pipeline builder is pretty versatile. You can define conditional routing, parallel branches, and cycles. Right now we support LLM node and util nodes (json extractor). If you can defined your logic purely from those nodes (and in majority of cases you will be), then great, you can host everything on Laminar! You follow this guide (https://docs.lmnr.ai/tutorials/control-flow-with-LLM) it's bit outdated by gives you a good idea on how to create and run pipelines.
-- skull8888888 Reply - I’m always game for an LLM observability platform that is potentially affordable, at least during the early phases of development.
I was using DD at work and found it to be incredibly helpful but now that I am on my own, I am much more price sensitive.
Still, having a low friction way to see how things are running, check inputs/outputs is a game changer.
One challenge I have run into is a lack of support for Anthropic models. The platforms that do have support are missing key pieces of info like the system prompt. (Prob a skill issue on my end).
Also they seem to all be tightly coupled to langchain, etc which is a no-go.
Will check this out over the next week or two. Very exciting!
-- 7thpower Reply - Totally agree, observability is a must for LLM apps. We wanted to build something of extremely high quality but to be affordable for solo devs, that's why open-source and why very generous free tier on our managed version.
Regarding Anthropic instrumentation, we support it out of the box! You don't even need to wrap anything, just do laminar initialize and you should see detailed traces. We also support images! Hit me up at [email protected] if you need help onboarding or setting up local version
-- skull8888888 Reply - How will you distinguish Laminar as "the Supabase for LLMOps" from the many LLM observability platforms already claiming similar aims? Is the integration of text analytics into execution traces your secret sauce? Or, could this perceived advantage just add complexity for developers who like their systems simple and their setups minimal?
-- gitroom Reply - Hey there! Good question. Our main distinguishing features are:
* Ingestion of Otel traces
* Semantic events-based analytics
* Semantically searchable traces
* High performance, reliability and efficiency out of the box, thanks to our stack
* High quality FE which is fully open-source
* LLM Pipeline manager, first of it's kind, highly customizable and optimized for performance
* Ability to track progression of locally run evals, combining full flexibility of running code locally without need to manage data infra
* Very generous free tier plan. Our infra is so efficient, that we can accommodate large number of free tier users without scaling it too much.
And many more to come in the coming weeks! On of our biggest next priorities is to focus on high quality docs.
All of these features can be used as standalone products, similar to Supabase. So, devs who prefer keep things lightweight might just use our tracing solution and be very happy with it.
-- skull8888888 Reply - > One thing missing from all LLM observability platforms right now is an adequate search over traces.
Why did you decide to build a whole platform and include this feature on top, rather than adding search to (for example) Grafana Tempo?
-- viraptor Reply - Valid point. For us searchable and especially semantically searchable, traces / spans really make sense only in the context of tracing LLM apps. And then, we view it as a powerful feature, but, primarily in the context of AI/LLM-native observability platform. For us, the ultimate goal is to build the comprehensive platform, with features which are extremely useful for observability and development of LLM/GenAI apps.
-- skull8888888 Reply - >For us searchable and especially semantically searchable, traces / spans really make sense only in the context of tracing LLM apps.
I know LLM is the new shiny thing right now. Why is semantic search of traces only useful for LLMs?
I've been working in CI/CD and at a large enough scale, searchability of logs was always an issue. Especially as many tools produce a lot of output with warnings and errors that mislead you.
Is the search feature only working in an LLM context? If so why?
-- carlmr Reply - Now that you mentioned it,
> warnings and errors that mislead youit really makes sense. I guess what I was pointing into, is that when you have really rich text (in your case it would be error descriptions), searching over them semantically is a must have feature.
But you are right, being an output of LLM is not a requirement.
-- skull8888888 Reply - To my eye this looks quite a bit more serious and useful than the naive buzzword bingo test would suggest.
I really like the stack these folks have chosen.
-- benreesman Reply - Thank you! We thought a lot about what would make a great title but couldn't really find anything else which would convey info as densely as current title. We also love our current stack :). I think Rust is perfect language to handle span ingestion and it marries perfectly with the rest of our stack.
-- skull8888888 Reply
-- Reply- Does it do event sourcing like inngest where I can do the “saga pattern”?
-- bn-l Reply - You mean like triggering another processing pipeline from the output of current processing pipeline?
-- skull8888888 Reply - [flagged]
-- dpflug Reply - Turns out there quite a few companies / projects named Laminar. I really like the name, couldn't buy a .com or .ai domain tho, so settled on lmnr.ai. But it's been growing on me.
-- skull8888888 Reply - Lmnop would have had a ring to it with laminar ops!
-- flashgordon Reply
-- Reply- [flagged]
-- ProphetOfAI Reply