Logistics and politics in inferencing
by Mat At work we're facing this ongoing need to offer inference solutions on open models for processing research data. It's been a journey. We got caught up in the university's efforts to get some accounting on personal AI subscriptions. Apparently there were thousands of personal subscriptions billed on corporate credit cards. Anyway, this set up an interesting policy backdrop when it comes to practical model inference.
Averaging public opinion doesn't make good opinion
by Mat I, like many of my friends and colleagues, is blown away by this US Gov blocking of Anthropic's Fabel. This is the kind of story I like to dig into myself and write some guidance for the Orac models to author a deep dive, but I didn't get to it, and Orac chose that story. The deal is, I have until 10:00 AEST to write my directives or Orac rolls with it.
When choosing models is hard
by Mat I had this idea that Orac's Shelf would be responding to current events, from the news, new model releases and so on. The news bit is easy, the new model releases is tricky. OpenRouter's public API doesn't really give us signals to tell what is genuinely new, so we nick one used by their frontend instead.
The one corner of the shelf a human still types by hand.
by Mat I have learned that co-authored (AI + me) content side steps the social capital attached when I write stuff by hand. I find this curious because I assume most stuff people write these days has bounced through Chazza, Gazza or Janky* (my names for leading chatbots).