-
Notifications
You must be signed in to change notification settings - Fork 4.1k
feat(vertexai): Add Live streaming feature #16991
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from 1 commit
Commits
Show all changes
44 commits
Select commit
Hold shift + click to select a range
f53b00c
init
cynthiajoan 0e5a21d
more testing for the audio player
cynthiajoan cd59d74
organize example page and functions
cynthiajoan 8b3285d
Merge branch 'vertexai/example_update' into vertexai/bidi
cynthiajoan c5bb860
get bidi demo working under new example layout
cynthiajoan 9575c5e
Merge branch 'main' into vertexai/bidi
cynthiajoan 3ff550e
kinda playable
cynthiajoan d2cbaff
working audio output
cynthiajoan 76aa22b
use firebase project for the connection
cynthiajoan c1695e5
record to file and load file to stream, no response yet
cynthiajoan b361009
try different ways to test sending audio
cynthiajoan 9988f72
try realtime input
cynthiajoan 2d37054
Merge branch 'main' into vertexai/bidi
cynthiajoan c9476e0
stream recording, try local file dump
cynthiajoan 89c7190
first kinda working version
cynthiajoan 8b3bc15
much better
cynthiajoan fe2073b
Merge branch 'main' into vertexai/bidi
cynthiajoan d53655f
Make function calling working
cynthiajoan fd128b2
add new autopush backend
cynthiajoan 2541969
Some update after api doc
cynthiajoan 1326935
more update on api
cynthiajoan 7ee188c
more clean up for api
cynthiajoan 0ca9d5a
Let's still keep image modality
cynthiajoan b675e6a
minor json fix
cynthiajoan e5920c2
Use IOWebSocketChannel class to pass in headers
cynthiajoan eff222c
unit test for live_api
cynthiajoan 8275927
more controllable receive logic
cynthiajoan 2a8164b
test fix after model library refactor
cynthiajoan 07e18f9
documentation and try catch in the example
cynthiajoan 202418e
Somehow working continuously conversation
cynthiajoan 6c53ab3
function calling fix
cynthiajoan fdb006a
changes after bugbash
cynthiajoan 04d1b40
Merge branch 'main' into vertexai/bidi
cynthiajoan 22ae77e
fix after merge main
cynthiajoan 9689825
more fixes after merge
cynthiajoan 7de0ebb
fix analyzing issues
cynthiajoan 3d78da0
more fix for analyzer
cynthiajoan 7113e44
fix analyzer
cynthiajoan 104714e
two more analyzer
cynthiajoan bda672a
fix the live test for function call
cynthiajoan d03df57
Address review comment
cynthiajoan ce683ee
More review comments to address
cynthiajoan c8bf5fe
review comments on audio player and recorder
cynthiajoan d2951e7
Update code to handle optional id for FunctionCall parse
cynthiajoan File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
much better
- Loading branch information
commit 8b3bc154db7b739ca6b3f99c5091a37b49612a91
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.