Codeunit "AIT Test Context"
Exposes functions that can be used by the AI tests.
Methods
NextTurn
Sets to next turn.
procedure NextTurn(): BooleanReturns
| Type | Description |
|---|---|
| Boolean |
True if another turn exists |
GetCurrentTurn
Gets the current turn. Turns start from turn 0.
procedure GetCurrentTurn(): IntegerReturns
| Type | Description |
|---|---|
| Integer |
The current turn number. |
GetInput
procedure GetInput()GetTestSetup
procedure GetTestSetup()GetContext
procedure GetContext()GetQuestion
procedure GetQuestion()GetGroundTruth
procedure GetGroundTruth()GetExpectedData
procedure GetExpectedData()SetAnswerForQnAEvaluation
Sets the answer for a question and answer evaluation. This will also copy the context, question and ground truth to the output dataset.
procedure SetAnswerForQnAEvaluation(Answer: Text)Parameters
| Name | Type | Description |
|---|---|---|
| Answer | Text |
The answer as text. |
SetQueryResponse
Sets the query and respone for a single-turn evaluation. Optionally, a context can be provided.
procedure SetQueryResponse(Query: Text, Response: Text, Context: Text)Parameters
| Name | Type | Description |
|---|---|---|
| Query | Text |
The query as text. |
| Response | Text |
The response as text. |
| Context | Text |
The context as text. |
SetQueryResponse
Sets the query and response for a single-turn evaluation.
procedure SetQueryResponse(Query: Text, Response: Text)Parameters
| Name | Type | Description |
|---|---|---|
| Query | Text |
The query as text. |
| Response | Text |
The response as text. |
AddMessage
Adds a message to the current test iteration. This is used for multi-turn tests to add messages to the output.
procedure AddMessage(Content: Text, Role: Text, Context: Text)Parameters
| Name | Type | Description |
|---|---|---|
| Content | Text |
The content of the message. |
| Role | Text |
The role of the message (e.g., 'user', 'assistant'). |
| Context | Text |
The context of the message. |
AddMessage
Adds a message to the current test iteration. This is used for multi-turn tests to add messages to the output.
procedure AddMessage(Content: Text, Role: Text)Parameters
| Name | Type | Description |
|---|---|---|
| Content | Text |
The content of the message. |
| Role | Text |
The role of the message (e.g., 'user', 'assistant'). |
SetTestOutput
procedure SetTestOutput(TestOutputJson: __MissingTypeSymbol__)Parameters
| Name | Type | Description |
|---|---|---|
| TestOutputJson | __MissingTypeSymbol__ |
SetTestOutput
Sets the test output for the current iteration.
procedure SetTestOutput(TestOutputText: Text)Parameters
| Name | Type | Description |
|---|---|---|
| TestOutputText | Text |
The test output as text. |
SetTestOutput
Sets the test output for the current iteration.
procedure SetTestOutput(Context: Text, Question: Text, Answer: Text)Parameters
| Name | Type | Description |
|---|---|---|
| Context | Text |
The context as text. |
| Question | Text |
The question as text. |
| Answer | Text |
The answer as text. |
SetTestMetric
Sets the test metric for the output dataset.
procedure SetTestMetric(TestMetric: Text)Parameters
| Name | Type | Description |
|---|---|---|
| TestMetric | Text |
The test metric as text. |
SetAccuracy
Sets the accuracy of the test.
procedure SetAccuracy(Accuracy: Decimal)Parameters
| Name | Type | Description |
|---|---|---|
| Accuracy | Decimal |
The accuracy as a decimal between 0 and 1. |
GetAITTestSuite
Gets the AITTestSuite associated with the run.
procedure GetAITTestSuite(var AITTestSuite: Record "AIT Test Suite")Parameters
| Name | Type | Description |
|---|---|---|
| AITTestSuite | Table System.TestTools.AITestToolkit."AIT Test Suite" |
AITTestSuite associated with the run. |
SetTokenConsumption
Sets the token consumption for the method line run. Useful if external calls are made outside of AI toolkit.
procedure SetTokenConsumption(TokensUsed: Integer)Parameters
| Name | Type | Description |
|---|---|---|
| TokensUsed | Integer |
Number of tokens used externally. |
Events
OnAfterRunComplete
Integration event that is raised after a test run is completed.
[IntegrationEvent(False,False)]
internal procedure OnAfterRunComplete(Code: Code[10], Version: Integer, Tag: Text[20])Parameters
| Name | Type | Description |
|---|---|---|
| Code | Code[10] |
The code of the test run. |
| Version | Integer |
The version of the test run. |
| Tag | Text[20] |
The tag of the test run. |