2
0

Support Vision for compatible AI models

Closes #1313
This commit is contained in:
Baptiste Arnaud
2024-06-11 10:51:02 +02:00
parent 40ca02df8c
commit ee834999e6
23 changed files with 360 additions and 92 deletions

View File

@ -11,7 +11,7 @@ import {
sessionOnlySetVariableOptions,
valueTypes,
} from '@typebot.io/schemas/features/blocks/logic/setVariable/constants'
import { TextInput } from '@/components/inputs'
import { TextInput, Textarea } from '@/components/inputs'
import { isDefined } from '@typebot.io/lib'
import { useTypebot } from '@/features/editor/providers/TypebotProvider'
import { isInputBlock } from '@typebot.io/schemas/helpers'
@ -215,7 +215,7 @@ const SetVariableValue = ({
)
}
case 'Append value(s)': {
return <TextInput defaultValue={options.item} onChange={updateItem} />
return <Textarea defaultValue={options.item} onChange={updateItem} />
}
case 'Moment of the day': {
return (

View File

@ -39,3 +39,15 @@ Finally, save the response of the assistant to a variable in order to append it
alt="Claude AI assistant message variable"
/>
</Frame>
## Vision support
`Create Chat Message` and `Ask Assistant` blocks support vision. This means that Typebot automatically detects images URL in any user message provided to OpenAI and parse it. The URL needs to be isolated from the rest of the text message to be properly detected. Here is an example of a message with an image URL:
If the selected model is [not compatible with vision](https://docs.anthropic.com/en/docs/vision), the image URL will be parsed as a plain text message.
```
What's in this picture?
https://domain.com/image.png
```

View File

@ -84,6 +84,18 @@ I also demonstrate how formatting can be affected by the presence of text before
<LoomVideo id="35dc8af6b9244762acc4a5acf275fb43" />
## Vision support
`Create Chat Message` and `Ask Assistant` blocks support vision. This means that Typebot automatically detects images URL in any user message provided to OpenAI and parse it. The URL needs to be isolated from the rest of the text message to be properly detected. Here is an example of a message with an image URL:
If the selected model is [not compatible with vision](https://platform.openai.com/docs/models), the image URL will be parsed as a plain text message.
```
What's in this picture?
https://domain.com/image.png
```
## Troobleshooting
### Error message: "OpenAI block returned error"