Thanks for this! Easy to read and follow and I like that it's aimed at a level higher than 101 so more technical folks can get to the meat of the post right away.
While the coding is above my paygrade, I absolutely appreciate the clean and hands-on nature of this. Very well done. For the code snippets, it looks like you just make screenshots? Guess you're not a fan of Substack's built-in "code block" template?
Also, the speed of your final app is impressive - there's almost no latency at all. Is that due to Mistral's innate response speed? Definitely faster than my experience with GPT-4.
The code snippets is my best attempt, definitely not perfect (you can't select the text) but better than Substack native code blocks.
Regarding speed, the indexing part is a bit of cheating because I cache the embeddings so the second time I ran it didn't need to compute them at all, but the conversation part, yes, that is the real speed of querying the Mistral API.
They're terrible. No syntax highlight, word wrapping wherever the hell it wants to, it's basically nothing but a mono space font, and an ugly font for the matter.
Oh yeah looks like it's really full featured. In the bottom they're doing something very similar to what I did here but of course the devil is in the details, and adding a good UX layer on top makes a world of difference.
Nice, I was curious as to whether you solved some of the underlying problems a little differently. I don't know any of the specifics under the hood, but I was using ChatPDF for a while there, especially in the earlier days of LLMs.
I don't think, I haven't done anything especially innovative here, this is the barebones implementation that everyone doing LLMs plus retrieval is using. It can improved a lot!
Well, you made this independently and from scratch! That is bad ass. Maybe some of the ideas/features from the existing apps can be used to improve this one, and it can become like best in class... but I also have a feeling the point is to make stuff, not to use stuff (although the utility part is a very nice bonus).
Thanks for this! Easy to read and follow and I like that it's aimed at a level higher than 101 so more technical folks can get to the meat of the post right away.
Thanks for the feedback! I have many more similar topics in the oven but if there's something concrete you'd like to see let me know.
While the coding is above my paygrade, I absolutely appreciate the clean and hands-on nature of this. Very well done. For the code snippets, it looks like you just make screenshots? Guess you're not a fan of Substack's built-in "code block" template?
Also, the speed of your final app is impressive - there's almost no latency at all. Is that due to Mistral's innate response speed? Definitely faster than my experience with GPT-4.
Thanks man!
The code snippets is my best attempt, definitely not perfect (you can't select the text) but better than Substack native code blocks.
Regarding speed, the indexing part is a bit of cheating because I cache the embeddings so the second time I ran it didn't need to compute them at all, but the conversation part, yes, that is the real speed of querying the Mistral API.
Yeah I was mainly referring to the subsequent conversation speed. Very cool!
What's your beef with the code blocks on Substack? I don't have any opinion on the subject, since I never use them at all. Just curious.
They're terrible. No syntax highlight, word wrapping wherever the hell it wants to, it's basically nothing but a mono space font, and an ugly font for the matter.
Yeah makes sense!
Alex, have you played with ChatPDF yet?
I've seen more than a few apps on this same line. Which exactly are you talking about?
https://www.chatpdf.com/
Oh yeah looks like it's really full featured. In the bottom they're doing something very similar to what I did here but of course the devil is in the details, and adding a good UX layer on top makes a world of difference.
Nice, I was curious as to whether you solved some of the underlying problems a little differently. I don't know any of the specifics under the hood, but I was using ChatPDF for a while there, especially in the earlier days of LLMs.
I don't think, I haven't done anything especially innovative here, this is the barebones implementation that everyone doing LLMs plus retrieval is using. It can improved a lot!
Well, you made this independently and from scratch! That is bad ass. Maybe some of the ideas/features from the existing apps can be used to improve this one, and it can become like best in class... but I also have a feeling the point is to make stuff, not to use stuff (although the utility part is a very nice bonus).