Nostr notes by Markus

And I had a veeery similar idea with autocoder, since i saw the ...

2024-12-25T21:36:32Z

In reply to nevent1q…uvah
_________________________

And I had a veeery similar idea with autocoder, since i saw the neurips autogen 0.4 workshop. It's kind of funny.
Let me guess, you also had the idea to do some "ablation" study (no CoT/deep think, no/simpler system prompt...) :D

From my experience with nim (niche lang), most LLMs generalize ...

2024-12-25T21:33:58Z

In reply to nevent1q…uvah
_________________________

From my experience with nim (niche lang), most LLMs generalize pretty ok over their memory, especially with a COT prompt, where you ask it to do it in a more in-the-data lang first.

but i'm very curious about some full results table for 10+ ...

2024-12-25T20:44:23Z

In reply to nevent1q…2759
_________________________

but i'm very curious about some full results table for 10+ models. i'm rooting for deepseek/qwq.

I considered building a similar thing for advent of code, but i ...

2024-12-25T20:43:04Z

In reply to nevent1q…2759
_________________________

I considered building a similar thing for advent of code, but i think for both, there are just a ton of solutions in the training data - so you are probably testing memorization over actual skill.

I'd also consider async/parallel/batch processing, afaik ollama/llama.cpp supports it since 0.2. I'd assume you could get at least 10x throughput if it's well optimized.