News Summary

• The secret, if you can call it that, is that they aren’t running very fast: four tokens per second was about the max.
• Still, in terms of local LLMs, it certainly beats the pants off toy models running via Llama on the PSP or the C64.
• This is only worth it if you are not concerned with paying for electricity.
• I love to see old hardware getting used for new tasks or being experimented in, but short of the glut of ram in this machine it’s largely a waste if electricity for this particular task.