Codeunit "AIT Test Context"
Exposes functions that can be used by the AI evals.
Methods
NextTurn
Sets to next turn.
procedure NextTurn(): BooleanReturns
| Type | Description |
|---|---|
| Boolean |
True if another turn exists |
GetCurrentTurn
Gets the current turn. Turns start from turn 0.
procedure GetCurrentTurn(): IntegerReturns
| Type | Description |
|---|---|
| Integer |
The current turn number. |
GetInput
procedure GetInput()GetTurnSetup
procedure GetTurnSetup()GetTurnSetup
procedure GetTurnSetup(TurnSetup: __MissingTypeSymbol__): BooleanParameters
| Name | Type | Description |
|---|---|---|
| TurnSetup | __MissingTypeSymbol__ |
Returns
| Type | Description |
|---|---|
| Boolean |
GetTestSetup
procedure GetTestSetup()GetContext
procedure GetContext()GetQuery
procedure GetQuery()GetQuestion
Obsolete
This element will become obsolete from version 29.0. Use GetQuery() instead.
[Obsolete(Use GetQuery() instead.,29.0)]
procedure GetQuestion()GetGroundTruth
procedure GetGroundTruth()GetExpectedData
procedure GetExpectedData()GetCanContinueOnFailure
Gets the continue on failure flag for the current turn. If the flag is not set in the test input, it defaults to false.
procedure GetCanContinueOnFailure(): BooleanReturns
| Type | Description |
|---|---|
| Boolean |
True if the eval should continue on failure, false otherwise. |
SetAnswerForQnAEvaluation
Sets the answer for a question and answer evaluation. This will also copy the context, question and ground truth to the output dataset.
procedure SetAnswerForQnAEvaluation(Answer: Text)Parameters
| Name | Type | Description |
|---|---|---|
| Answer | Text |
The answer as text. |
SetQueryResponse
Sets the query and respone for a single-turn evaluation. Optionally, a context can be provided.
procedure SetQueryResponse(Query: Text; Response: Text; Context: Text)Parameters
| Name | Type | Description |
|---|---|---|
| Query | Text |
The query as text. |
| Response | Text |
The response as text. |
| Context | Text |
The context as text. |
SetQueryResponse
Sets the query and response for a single-turn evaluation.
procedure SetQueryResponse(Query: Text; Response: Text)Parameters
| Name | Type | Description |
|---|---|---|
| Query | Text |
The query as text. |
| Response | Text |
The response as text. |
AddMessage
Adds a message to the current eval iteration. This is used for multi-turn evals to add messages to the output.
procedure AddMessage(Content: Text; Role: Text; Context: Text)Parameters
| Name | Type | Description |
|---|---|---|
| Content | Text |
The content of the message. |
| Role | Text |
The role of the message (e.g., 'user', 'assistant'). |
| Context | Text |
The context of the message. |
AddMessage
Adds a message to the current eval iteration. This is used for multi-turn evals to add messages to the output.
procedure AddMessage(Content: Text; Role: Text)Parameters
| Name | Type | Description |
|---|---|---|
| Content | Text |
The content of the message. |
| Role | Text |
The role of the message (e.g., 'user', 'assistant'). |
SetTestOutput
procedure SetTestOutput(TestOutputJson: __MissingTypeSymbol__)Parameters
| Name | Type | Description |
|---|---|---|
| TestOutputJson | __MissingTypeSymbol__ |
SetTestOutput
Sets the test output for the current iteration.
procedure SetTestOutput(TestOutputText: Text)Parameters
| Name | Type | Description |
|---|---|---|
| TestOutputText | Text |
The test output as text. |
SetTestOutput
Sets the test output for the current iteration.
procedure SetTestOutput(Context: Text; Question: Text; Answer: Text)Parameters
| Name | Type | Description |
|---|---|---|
| Context | Text |
The context as text. |
| Question | Text |
The question as text. |
| Answer | Text |
The answer as text. |
SetTestMetric
Sets the test metric for the output dataset.
procedure SetTestMetric(TestMetric: Text)Parameters
| Name | Type | Description |
|---|---|---|
| TestMetric | Text |
The test metric as text. |
SetAccuracy
Sets the accuracy of the eval.
procedure SetAccuracy(Accuracy: Decimal)Parameters
| Name | Type | Description |
|---|---|---|
| Accuracy | Decimal |
The accuracy as a decimal between 0 and 1. |
GetAITTestSuite
Gets the AITTestSuite associated with the run.
procedure GetAITTestSuite(var AITTestSuite: Record "AIT Test Suite")Parameters
| Name | Type | Description |
|---|---|---|
| AITTestSuite | Table System.TestTools.AITestToolkit."AIT Test Suite" |
AITTestSuite associated with the run. |
SetTokenConsumption
Sets the token consumption for the method line run. Useful if external calls are made outside of AI toolkit.
procedure SetTokenConsumption(TokensUsed: Integer)Parameters
| Name | Type | Description |
|---|---|---|
| TokensUsed | Integer |
Number of tokens used externally. |
GetEvalSuiteSetupDataInput
procedure GetEvalSuiteSetupDataInput()SetEvalSuiteSetupCompleted
Marks the per-suite setup as completed on the test suite record. Call this after your suite setup logic has finished successfully.
procedure SetEvalSuiteSetupCompleted()IsSuiteSetupDone
Checks if the per-suite setup has been marked as done on the test suite record.
procedure IsSuiteSetupDone(): BooleanReturns
| Type | Description |
|---|---|
| Boolean |
True if suite setup has been executed. |
Events
OnAfterRunComplete
Integration event that is raised after an eval run is completed.
[IntegrationEvent(False,False)]
internal procedure OnAfterRunComplete(Code: Code[10]; Version: Integer; Tag: Text[20])Parameters
| Name | Type | Description |
|---|---|---|
| Code | Code[10] |
The code of the eval run. |
| Version | Integer |
The version of the eval run. |
| Tag | Text[20] |
The tag of the eval run. |