blsmedia/bot
2
0
Fork 0
Code Activity

✨ Support Vision for compatible AI models

Browse Source 
Closes #1313
This commit is contained in:
Baptiste Arnaud
2024-06-11 10:51:02 +02:00
parent 40ca02df8c
commit ee834999e6
23 changed files with 360 additions and 92 deletions
Download Patch File Download Diff File Expand all files Collapse all files

12
apps/docs/editor/blocks/integrations/anthropic.mdx
View File

@ -39,3 +39,15 @@ Finally, save the response of the assistant to a variable in order to append it
alt="Claude AI assistant message variable"
/>
</Frame>
## Vision support
`Create Chat Message` and `Ask Assistant` blocks support vision. This means that Typebot automatically detects images URL in any user message provided to OpenAI and parse it. The URL needs to be isolated from the rest of the text message to be properly detected. Here is an example of a message with an image URL:
If the selected model is [not compatible with vision](https://docs.anthropic.com/en/docs/vision), the image URL will be parsed as a plain text message.
```
What's in this picture?
https://domain.com/image.png
```

12
apps/docs/editor/blocks/integrations/openai.mdx
View File

@ -84,6 +84,18 @@ I also demonstrate how formatting can be affected by the presence of text before
<LoomVideo id="35dc8af6b9244762acc4a5acf275fb43" />
## Vision support
`Create Chat Message` and `Ask Assistant` blocks support vision. This means that Typebot automatically detects images URL in any user message provided to OpenAI and parse it. The URL needs to be isolated from the rest of the text message to be properly detected. Here is an example of a message with an image URL:
If the selected model is [not compatible with vision](https://platform.openai.com/docs/models), the image URL will be parsed as a plain text message.
```
What's in this picture?
https://domain.com/image.png
```
## Troobleshooting
### Error message: "OpenAI block returned error"