flutter - How to convert this complex Future returning function to a Stream? - Stack Overflow

admin管理员组

文章数量:1287632

To be clear I don't simply want to convert the Future to a Stream object, I want to change the internal behavior of the function to support streaming the LLM response rather than returning the final result.

I have a method that handles communication with an LLM through the VertexAI API which can make function calls during the conversation. The current implementation returns a Future<String>, but I need to modify it to return a Stream<String> for the final response while preserving the function call handling logic.

To explain what the function does better, it takes the user's query and sends it to the LLM, the LLM might either respond immediately, or if it needs to it might make multiple rounds of function calling before giving a final result. This was pretty easy to handle with Future but now I want to have the UI similar to ChatGPT where the response is streamed gradually to the user as each word comes in so I want to use a Stream.

Note that the api supports a sendMessageStream method instead of the sendMessage method which does the same thing but just returns a Stream instead of a Future.

I need to handle both cases where the LLM might respond with the final response immediately which doesn't contain function calls, in that case that's what should be returned as the stream, or handle if there are function calls then I wait until all are resolved then return the stream.

Future<String> generateResponse(String query) async {
  final chat = model.startChat();

  GenerateContentResponse response = await chat.sendMessage(Content.text(query));

  while (response.functionCalls.isNotEmpty) {
    final List<Map<String, DataRow>> functionCallResults = [];

    for (final functionCall in response.functionCalls) {
      final functionIdentifier = functionCall.name;
      final parameters = functionCall.args;

      final result = await executeFunction(functionIdentifier, parameters);

      functionCallResults.add({
        functionIdentifier: result,
      });
    }

    response = await chat.sendMessage(
      functionCallResults.length == 1
          ? Content.functionResponse(
              functionCallResults.single.entries.single.key,
              functionCallResults.single.entries.single.value,
            )
          : Content.functionResponses([
              for (final entry in functionCallResults)
                FunctionResponse(
                  entry.entries.single.key,
                  entry.entries.single.value,
                ),
            ]),
    );
  }

  return response.text ?? 'Sorry, I am not able to respond to your query at the moment.';
}

本文标签： flutterHow to convert this complex Future returning function to a StreamStack Overflow