Skip to content

Add LLM functionality using auto-model V2#2633

Merged
koesie10 merged 7 commits intomainfrom
koesie10/automodel-v2
Jul 27, 2023
Merged

Add LLM functionality using auto-model V2#2633
koesie10 merged 7 commits intomainfrom
koesie10/automodel-v2

Conversation

@koesie10
Copy link
Copy Markdown
Member

This updates the LLM functionality to use the new request/response format and the query included in codeql when the codeQL.dataExtensions.llmGenerationV2 config setting is enabled.

The first two commits clean up the query resolver for contextual queries so that it's easier to re-use that functionality for resolving non-contextual queries. The last commit uses this to do the following:

  • Resolve the query with tags automodel <application-mode/framework-mode> candidates from the CodeQL libraries (either submodule or query pack)
  • Run the query
  • Convert the results to SARIF
  • Encode the SARIF as JSON
  • GZIP compress the JSON string
  • Base64-encode the GZIPed JSON
  • Send the request
  • Parse the YAML returned from the API
  • Send the modeled methods to the view

Checklist

  • CHANGELOG.md has been updated to incorporate all user visible changes made by this pull request.
  • Issues have been created for any UI or other user-facing changes made by this pull request.
  • [Maintainers only] If this pull request makes user-facing changes that require documentation changes, open a corresponding docs pull request in the github/codeql repo and add the ready-for-doc-review label there.

@koesie10 koesie10 force-pushed the koesie10/automodel-v2 branch from 453cc9a to f3dbb92 Compare July 25, 2023 08:15
@koesie10 koesie10 force-pushed the koesie10/automodel-v2 branch from f3dbb92 to 84de8ad Compare July 25, 2023 08:36
@charisk
Copy link
Copy Markdown
Contributor

charisk commented Jul 25, 2023

Any chance you could have a separate PR for the refactoring (first two commits)? It'd be easier for me personally but it would also probably bring feedback from others too since the query resolver/contextual queries isn't really an area we own/touched on a lot as a team.

@koesie10 koesie10 mentioned this pull request Jul 25, 2023
3 tasks
@koesie10 koesie10 changed the base branch from main to koesie10/cleanup-query-resolver July 25, 2023 09:30
Base automatically changed from koesie10/cleanup-query-resolver to main July 26, 2023 07:36
@koesie10 koesie10 marked this pull request as ready for review July 26, 2023 07:36
@koesie10 koesie10 requested review from a team as code owners July 26, 2023 07:36
Copy link
Copy Markdown
Contributor

@charisk charisk left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for getting this working and for writing a few tests too!

Comment thread extensions/ql-vscode/src/common/zlib.ts Outdated
import { join } from "path";
import { assertNever } from "../common/helpers-pure";

type Options = {
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't quite get why we're calling things "Options" - to me they look like normal method inputs. Also it's hard to know the difference between Options and AutoModelQueryOptions

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These are regular method inputs, but with this many inputs it becomes somewhat harder to reason about the order of the arguments. By using an object, they are at least all named so it becomes easier to reason about them.

I've renamed it to AutoModelQueriesOptions and moved mode into it. I've also moved it closer to the actual function definition so it's easier to distinguish between the two options types.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the changes! The term "options" still seems a bit off to me because it makes me think of optional or variadic arguments, but I don't have a good suggestions atm (only "inputs" comes to mind).

Comment thread extensions/ql-vscode/src/data-extensions-editor/auto-model-codeml-queries.ts Outdated
Comment thread extensions/ql-vscode/src/data-extensions-editor/data-extensions-editor-view.ts Outdated
Copy link
Copy Markdown
Contributor

@charisk charisk left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

@koesie10 koesie10 enabled auto-merge July 27, 2023 07:50
@koesie10 koesie10 merged commit c2ed98e into main Jul 27, 2023
@koesie10 koesie10 deleted the koesie10/automodel-v2 branch July 27, 2023 08:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants