10 Pull requests merged by 6 people

• [fundamental] Update PR template to emphasize license issue

  #3539 merged Jul 12, 2024

• Make evaluation run a context manager instead of a singleton.

  #3529 merged Jul 11, 2024

• Log performance metrics to the application insights

  #3493 merged Jul 11, 2024

• Update dependency for pf dependences and local import for dependences related to pf-azure

  #3527 merged Jul 10, 2024

• Adding the credential to jailbreak adv sim init to make sure it can b…

  #3523 merged Jul 9, 2024

• Task/jailbreak adv sim

  #3455 merged Jul 9, 2024

• [doc] highlight tracing feature

  #3518 merged Jul 9, 2024

• Fixing Evaluate when no target present

  #3514 merged Jul 8, 2024

• Doc update: Open AI -> OpenAI

  #3509 merged Jul 8, 2024

• Update chat-with-pdf-azure.ipynb (Typo "Azure Open AI"→"Azure OpenAI")

  #3351 merged Jul 5, 2024

14 Pull requests opened by 9 people

• Update connection.ipynb (Typo "Azure Open AI"→"Azure OpenAI")

  #3511 opened Jul 5, 2024

• clarity usage in docstring for evaluator_config parameter

  #3515 opened Jul 8, 2024

• [fundamental] Add CG in compliance check CI to avoid license issue

  #3517 opened Jul 9, 2024

• Replace dag with flex in evals

  #3521 opened Jul 9, 2024

• [Snyk] Security upgrade zipp from 3.15.0 to 3.19.1

  #3524 opened Jul 9, 2024

• [Snyk] Security upgrade zipp from 3.15.0 to 3.19.1

  #3525 opened Jul 9, 2024

• [Snyk] Security upgrade zipp from 3.15.0 to 3.19.1

  #3530 opened Jul 10, 2024

• [Snyk] Security upgrade zipp from 3.15.0 to 3.19.1

  #3531 opened Jul 10, 2024

• [Snyk] Security upgrade zipp from 3.15.0 to 3.19.1

  #3532 opened Jul 10, 2024

• Add local e2e test gate

  #3534 opened Jul 10, 2024

• Task simulator

  #3541 opened Jul 11, 2024

• Leverage async batch run for first async-enabled evaluator - FluencyEvaluator

  #3542 opened Jul 11, 2024

• add category to template_parameters in the output of simulations

  #3544 opened Jul 11, 2024

• Fixing user agent header name

  #3545 opened Jul 11, 2024

8 Issues closed by 7 people

• [BUG] [promptflow-evals] Failed to invoke the Azure CLI when running evaluation

  #3438 closed Jul 12, 2024

• [BUG] Flow is not enabled

  #3543 closed Jul 11, 2024

• [BUG] ValueError: Missing required inputs for target : ['question'] while using evalutor_config

  #3516 closed Jul 9, 2024

• [BUG] [promptflow-evals] Adversarial simulator cannot generate samples with jailbreak correctly if `max_simulation_results` is large.

  #3440 closed Jul 9, 2024

• [BUG] [promptflow-evals] The `AdversarialSimulator` object cannot be reused.

  #3441 closed Jul 9, 2024

• I need to serve to 0.0.0.0

  #3432 closed Jul 9, 2024

• [BUG] promptflow 1.11.0 raises IndexError: list index out of range

  #3355 closed Jul 6, 2024

• Set Trace Destination fails with Cosmos DB is disabled error

  #3418 closed Jul 5, 2024

11 Issues opened by 9 people

• Failed to load trace[BUG]

  #3540 opened Jul 11, 2024

• [BUG] Tracing Contextvar reset

  #3538 opened Jul 11, 2024

• [BUG] prompt eval method does not take credential and creates PFClient without passing in credential

  #3537 opened Jul 11, 2024

• [BUG] Running evaluate from promptflow.evals.evaluate and setting trace.destination="none" or "local" causes evaluations to not be available

  #3535 opened Jul 10, 2024

• [Feature Request] Have option to return trace as part of out put of a PromptFlow when deployed as endpoint

  #3528 opened Jul 9, 2024

• [Feature Request] pass addition inputs to target function when using the evaluate method

  #3526 opened Jul 9, 2024

• [BUG] Evaluator respond with Nan values as the first token is a text

  #3522 opened Jul 9, 2024

• [Performance] how to disable all dump, for ex. _node_run_postprocess in run_tracker.py

  #3520 opened Jul 9, 2024

• Lisence issue for the dependency on docutils(GPL 3.0)

  #3513 opened Jul 8, 2024

• [BUG]calling pf_client.run inside the callable target function for evaluate got "Error: (AssertionError) daemonic processes are not allowed to have children."

  #3512 opened Jul 6, 2024

• Support remote tracing for CMK-enabled ML workspaces

  #3510 opened Jul 5, 2024

22 Unresolved conversations

Sometimes conversations happen on old items that aren’t yet closed. Here is a list of all the Issues and Pull Requests with unresolved conversations.

• docs(docker): link to environment variable config page + configuring run and finish scripts

  #3508 commented on Jul 11, 2024 • 9 new comments

• Add batch timeout override

  #3501 commented on Jul 11, 2024 • 5 new comments

• [Snyk] Security upgrade anyio from 3.7.1 to 4.4.0

  #3471 commented on Jul 10, 2024 • 0 new comments

• [Snyk] Security upgrade anyio from 3.7.1 to 4.4.0

  #3467 commented on Jul 9, 2024 • 0 new comments

• [Snyk] Security upgrade anyio from 3.7.1 to 4.4.0

  #3465 commented on Jul 9, 2024 • 0 new comments

• [BUG] UnicodeDecodeError: 'utf-8' codec can't decode byte 0x82 in position 80: invalid start byte

  #3450 commented on Jul 12, 2024 • 0 new comments

• [BUG] Types for promptflow-evals evaluator classes

  #3201 commented on Jul 11, 2024 • 0 new comments

• [Feature Request] Add docstrings for constructors

  #3202 commented on Jul 11, 2024 • 0 new comments

• [Feature Request] Provide example of evaluator_config for evaluate

  #3203 commented on Jul 11, 2024 • 0 new comments

• [Feature Request] Allow Path objects for evaluate data and output_path parameters

  #3206 commented on Jul 11, 2024 • 0 new comments

• [Feature Request] Allow the data parameter of evaluate to take a list

  #3207 commented on Jul 11, 2024 • 0 new comments

• [Feature Request] Allow the target function in evaluate to be a local function

  #3208 commented on Jul 11, 2024 • 0 new comments

• [BUG] Parsing of tool calls does not support arrays in the "arguments"

  #3388 commented on Jul 11, 2024 • 0 new comments

• [Feature Request] Enable search filters and search params in Index Lookup tool

  #3365 commented on Jul 10, 2024 • 0 new comments

• [BUG] [VSCode Extension] Not able to import modules outside of the flow directory

  #3495 commented on Jul 10, 2024 • 0 new comments

• [Feature Request] Improve compatibility between PromptFlow and Python modules with relative imports

  #3209 commented on Jul 10, 2024 • 0 new comments

• [BUG] Function `evaluate(...)` from promptflow.evals.evaluate can't be used / imported.

  #3382 commented on Jul 9, 2024 • 0 new comments

• [Feature Request] Logging statements are not present in log output

  #1899 commented on Jul 9, 2024 • 0 new comments

• [Feature Request]: Using Managed Identity (RBAC) instead of basic key autentication for Connections

  #2102 commented on Jul 7, 2024 • 0 new comments

• [Feature Request] Is there a way to export the trace reports?

  #3371 commented on Jul 6, 2024 • 0 new comments

• [BUG] Azure PFClient gives error, unable to run the flow on Azure

  #3370 commented on Jul 6, 2024 • 0 new comments

• Invalid instrumentation key

  #3373 commented on Jul 6, 2024 • 0 new comments