• 0 Posts
  • 17 Comments
Joined 3 years ago
cake
Cake day: July 23rd, 2023

help-circle
  • Running EUROPA on in-house hardware would take a lot of memory, even with the usual compression to 4 bits per parameter. At that level, its weights alone would take up at least 476 GB by our math, plus working memory. That puts it in the same range as Muse Spark Open, Meta’s open model. That model has 816 billion parameters and runs on two Apple Mac Studio desktops with 512 GB of memory each.

    Although it’s worth noting they haven’t actually trained the thing yet so we’d have to wait to get the actual weights to know more.