Hi Paul, Jan-v1 (4B-Q4_K_M) is what I'm testing and what I used for this example. Works well for me and I can recommend it, though I haven't done extensive comparisons with all the other model options. Curious if you or others have different preferences based on what you've tried.
Thanks! I haven't extensively used local LLMs, but it's a goal for this coming semester, and I have a 19-hour flight this weekend, so perfect timing! Currently limited by hardware - the best machine I have available is a M1 MacBook Air with 8GB of RAM, and it doesn't seem to like the model you just mentioned. :-/
You can actually adjust the model parameters to produce more creative and variable results.
In your prompt you tell the model to “set the temperature” and set it to value between 6 and 2 (max.)
You will have a higher chance of hallucination, but also more unusual model responses.
Say, for your Jan example (https://docs.google.com/document/d/1raFl1tsS4bSW0yjbjtBZg5u7wHN9Ep_IOaIkOkBhafE/edit?tab=t.8ig70slmrtvq) Is there a model you prefer? Do you recall which one was in play for this response?
Hi Paul, Jan-v1 (4B-Q4_K_M) is what I'm testing and what I used for this example. Works well for me and I can recommend it, though I haven't done extensive comparisons with all the other model options. Curious if you or others have different preferences based on what you've tried.
Thanks! I haven't extensively used local LLMs, but it's a goal for this coming semester, and I have a 19-hour flight this weekend, so perfect timing! Currently limited by hardware - the best machine I have available is a M1 MacBook Air with 8GB of RAM, and it doesn't seem to like the model you just mentioned. :-/
Cracking. Love the examples
Very helpful. Thanks!
Awesome article
Loved this. Your great guide makes AI feel playful and liberating rather than mechanical.
I love these tips for tinkering with prompts!
Hey Jeremy really cool tips jere. Gonna try these out tomorrow and see what i van create. 💥🌟