Anthropic

[pelikhan]

@waltoss Let's use this thread to keep track of the Anthropic integration performance. Here's from the top of my mind:

• support for tags instead of code fences, since Claude is trained for those. To support that, we need to update the code that renders defs (in promptdom.ts) and add support for parsing outputs using tags as well.
• caching of messages

[pelikhan]

You can try it out on 1.66

[Iwixnfusnz]

apt update && upgrade

[pelikhan]

Also support for tokenizer api

[pelikhan]

@waltoss XML tag support with def coming up.

[waltoss]

Looking forward to it !

[pelikhan]

PR is here: #921 . Running evals suite.

[pelikhan]

@waltoss you can side load the .vsix from https://github.com/microsoft/genaiscript/releases/tag/1.82.0 and try it out. it should default to XML; you can check in the trace. Also contains the schema updates.

[waltoss]

Also we should try implementing the caching mechanism https://docs.anthropic.com/en/docs/build-with-claude/prompt-caching it would save a lot of tokens

[pelikhan]

Could you open a bug for it? I started adding "ephemeral" to the def definitions so that the renderer can treat them specially.

[pelikhan]

Caching is available for testing in 1.83.0

[pelikhan]

SDK update is coming; prompt caching is now out of beta.