Skip to content

feat: add gemini-2.0-flash-001 and gemini-2.0-flash-lite-001 to fine tune score endponts and multimodal endpoints #1650

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 9 commits into from
Apr 29, 2025

Conversation

shuoweil
Copy link
Contributor

feat: add gemini-2.0-flash-001 and gemini-2.0-flash-lite-001 to fine tune score endponts and multimodal endpoints

@shuoweil shuoweil requested review from a team as code owners April 23, 2025 18:33
@shuoweil shuoweil requested a review from shobsi April 23, 2025 18:33
@product-auto-label product-auto-label bot added the size: s Pull request size is small. label Apr 23, 2025
@product-auto-label product-auto-label bot added the api: bigquery Issues related to the googleapis/python-bigquery-dataframes API. label Apr 23, 2025
@shuoweil shuoweil removed the request for review from shobsi April 23, 2025 19:11
@shuoweil shuoweil assigned shuoweil and unassigned ivansmf Apr 23, 2025
@shuoweil shuoweil marked this pull request as draft April 23, 2025 19:11
@shuoweil shuoweil force-pushed the shuowei-add-llm-tests branch from b93529e to aebca40 Compare April 24, 2025 22:45
@shuoweil shuoweil marked this pull request as ready for review April 24, 2025 22:48
@shuoweil shuoweil force-pushed the shuowei-add-llm-tests branch from 8b60de1 to 4e9a1ac Compare April 25, 2025 02:36
@product-auto-label product-auto-label bot added size: m Pull request size is medium. and removed size: s Pull request size is small. labels Apr 25, 2025
Copy link

Check out this pull request on  ReviewNB

See visual diffs & provide feedback on Jupyter Notebooks.


Powered by ReviewNB

@shuoweil shuoweil force-pushed the shuowei-add-llm-tests branch from 4e9a1ac to 6d8e228 Compare April 28, 2025 21:49
@product-auto-label product-auto-label bot added size: s Pull request size is small. and removed size: m Pull request size is medium. labels Apr 28, 2025
@shuoweil shuoweil force-pushed the shuowei-add-llm-tests branch from 6d8e228 to 106951f Compare April 28, 2025 22:02
@shuoweil shuoweil requested a review from GarrettWu April 28, 2025 22:22
@GarrettWu
Copy link
Contributor

Can you check the result for .predict and .score of gemini-2.0-flash-lite-001? If no quota, they shouldn't have meaningful output.

@GarrettWu
Copy link
Contributor

Please actively followup with BQML, when the quota issue is resolved, then to add fine tuning and other missing stuff.

@shuoweil
Copy link
Contributor Author

Can you check the result for .predict and .score of gemini-2.0-flash-lite-001? If no quota, they shouldn't have meaningful output.

Here is the output when running .score() for gemini-2.0-flash-lite.
message=Internal error in Dremel: INTERNAL: An internal error has occurred.; Unable to find default quota for model name: gemini-2.0-flash-lite. debug=; code: \t INTERNAL_ERROR\ndescription: \"An internal error has occurred.; Unable to find default quota for model name: gemini-2.0-flash-lite-001
Note: this test is blocked by b/413131111

Output when running .predict() for gemini-2.0-flash-lite.
E google.api_core.exceptions.BadRequest: 400 GET https://bigquery.googleapis.com/bigquery/v2/projects/bigframes-load-testing/queries/bdfcd82f-4e59-412d-81e0-2f8d3e411aa8?maxResults=0&location=US&prettyPrint=false: Option(s) DATA_SPLIT_METHOD, MAX_ITERATIONS, INPUT_LABEL_COLS, PROMPT_COL are not supported for GEMINI_1P5_FLASH model training.
Note: this bug is an internal bug, BQML is working on rolling out this feature.

@shuoweil shuoweil merged commit 4fb54df into main Apr 29, 2025
24 checks passed
@shuoweil shuoweil deleted the shuowei-add-llm-tests branch April 29, 2025 17:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
api: bigquery Issues related to the googleapis/python-bigquery-dataframes API. size: s Pull request size is small.
3 participants