[BUG] [promptflow-evals] Adversarial simulator cannot generate samples with jailbreak correctly if max_simulation_results is large.

[Wix97]

Describe the bug
promptflow-evals version: 0.3.0
Cannot generate jailbreak samples with following code:

import asyncio
from typing import Any, Dict

import pandas as pd
from azure.identity import DefaultAzureCredential
from promptflow.evals.synthetic import AdversarialScenario, AdversarialSimulator

azure_ai_project = {
    "subscription_id": "",
    "resource_group_name": "",
    "project_name": "",
    "credential": DefaultAzureCredential(),
}


async def callback(
    messages: Dict,
    stream: bool = False,
    session_state: Any = None,
) -> dict:
    query = messages["messages"][0]["content"]

    # Add file contents for summarization or re-write
    if "file_content" in messages["template_parameters"]:
        query += messages["template_parameters"]["file_content"]

    # Call your own endpoint and pass your query as input. Make sure to handle your function_call_to_your_endpoint's error responses.

    response = f"I don't know."

    # Format responses in OpenAI message protocol
    formatted_response = {
        "content": response,
        "role": "assistant",
        "context": {},
    }

    messages["messages"].append(formatted_response)
    return {
        "messages": messages["messages"],
        "stream": stream,
        "session_state": session_state,
    }


n_test_cases = 1384


for use_jailbreak in [False, True]:

    scenario = AdversarialScenario.ADVERSARIAL_QA

    stage_name = f"{scenario.value}"

    if use_jailbreak:
        stage_name += "_jailbreak"

    print(f"Current stage:{stage_name}")


    simulator = AdversarialSimulator(azure_ai_project=azure_ai_project)

    outputs = asyncio.run(
        simulator(
            scenario=scenario,  # required adversarial scenario to simulate
            target=callback,  # callback function to simulate against
            max_simulation_results=n_test_cases,  # optional
            jailbreak=use_jailbreak,  # optional
            concurrent_async_task=5,
        )
    )

    print(outputs.to_eval_qa_json_lines())

The code will raise the following error:

jinja2.exceptions.TemplateSyntaxError: expected token ':', got 'message'

If we set the variable n_test_cases lower than 100, the error may not be raised.

[Wix97]

And an addition question, what is the minimum permission required to call AI studio annotation service to generate adversarial samples? Thanks!

[nagkumar91]

Fix is here - #3448 being reviewed ATM

[nagkumar91]

And an addition question, what is the minimum permission required to call AI studio annotation service to generate adversarial samples? Thanks!

Confirmed with an engineer on the service that a user needs to have contributor access to the workspace.

[nagkumar91]

We have fixed this issue in 0.3.1 release. Please reopen the issue if it's not solved.