[Feature Request]The server shuts down after one hour of inactivity, which is not good

[jiaochenlu]

Is your feature request related to a problem? Please describe.
User feedback: The server shuts down after one hour of inactivity, which is not good! Do I have to restart it every morning before our customers wake up?

Describe the solution you'd like
Can be configured not to shut down automatically

[tyler-suard-parker]

Use case: our chatbot can be used at any time of day by any one of our beta users. If the beta user reports a problem, we want to be able to look at the trace and find out what went wrong. If the tracing server shuts down after one hour of activity, will we lose the trace from any user chats afterward? Will it take 60 seconds for the server to come online, which adds 60 seconds to a chat response?

[YingChen1996]

Hi, @tyler-suard-parker. Thanks for reporting the issue. Local pfs will auto stop in 1 hour if there's no any request to local pfs (We will remove the behavior). Trace will be stored in database even if the tracing server shuts down, so I think you won't lose the trace. And I think PFS startup will not take 60 seconds

[tyler-suard-parker]

Oh, ok, that makes sense. So when I am using prompflow locally, the local server will auto-stop in one hour if there are no requests to it for promptflow tracing. Those traces recorded by the local server are stored locally.

However, when my web app is deployed, the local tracing server does not matter, because all traces get sent to bot the local tracing server, and my AI Studio tracing. The local server (running on my azure web app) receives the traces and saves them, but they get deleted every time I restart the web app, and I never check them anyways. The promptflow instance running on my web app is also sending traces to my AI Studio workspace, where they are saved. This is why the promptflow tracing server shuts down after an hour, is that correct?

[YingChen1996]

Hi, @tyler-suard-parker. What you mean is that when you restart the web app, the previous local traces are lost?

BTW, we have removed prompt flow local service (a local web service) auto-stop feature and it won't shut down after an hour without requests in the latest public version. Maybe you can have a try. Thanks