Use Mistral AI to build generative AI applications with Go

Mistral has a specific prompt format, where:

• refers to thebeginning of stringtoken
• text for the user role is inside the [INST]...[/INST] tokens
• text outside is the assistant role

type MistralRequest struct { Prompt string `json:"prompt"` MaxTokens int `json:"max_tokens,omitempty"` Temperature float64 `json:"temperature,omitempty"` TopP float64 `json:"top_p,omitempty"` TopK int `json:"top_k,omitempty"` StopSequences []string `json:"stop,omitempty"` }

output, err := brc.InvokeModel(context.Background(), &bedrockruntime.InvokeModelInput{ Body: payloadBytes, ModelId: aws.String(modelID7BInstruct), ContentType: aws.String("application/json"), }) var resp MistralResponse err = json.Unmarshal(output.Body, &resp) fmt.Println("response string:\n", resp.Outputs[0].Text)

Moving on to a simple conversational interaction. This is what Mistral refers to as amulti-turn promptand we will add thewhich is theend of stringtoken.

To run the example:

go run chat/main.go

Here is my interaction:

You can refer to the complete code here

The code itself is overly simplified for the purposes of this example. But, important part is the how the tokens are used to format the prompt. Note that we are using Mixtral 8X7B (mistral.mixtral-8x7b-instruct-v0:1) in this example.

const userMessageFormat = "[INST] %s [/INST]" const modelID8X7BInstruct = "mistral.mixtral-8x7b-instruct-v0:1" const bos = "" const eos = "" var verbose *bool func main() { reader := bufio.NewReader(os.Stdin) first := true var msg string for { fmt.Print("\nEnter your message: ") input, _ := reader.ReadString('\n') input = strings.TrimSpace(input) if first { msg = bos + fmt.Sprintf(userMessageFormat, input) } else { msg = msg + fmt.Sprintf(userMessageFormat, input) } payload := MistralRequest{ Prompt: msg, } response, err := send(payload) fmt.Println("[Assistant]:", response) msg = msg + response + eos + " " first = false } }

Thebeginning of string(bos) token is only neededonceat the start of the conversation, while eos (end of string) marks the end of a single conversation exchange (user and assistant).

Chat with streaming

If you've read my previous blogs, I always like to include a "streaming" example because:

1. It provides a better experience from a client application point of view
2. It's a common mistake to overlook the InvokeModelWithResponseStream function (the async counterpart of InvokeModel)
3. The partial model payload response can be interesting (and tricky at times)

You can refer to the complete code here

Lets try this out. This example uses Mistral Large - simply change the model ID to mistral.mistral-large-2402-v1:0. To run the example:

go run chat-streaming/main.go

Notice the usage of InvokeModelWithResponseStream (instead of Invoke):

output, err := brc.InvokeModelWithResponseStream(context.Background(), &bedrockruntime.InvokeModelWithResponseStreamInput{ Body: payloadBytes, ModelId: aws.String(modelID7BInstruct), ContentType: aws.String("application/json"), }) //...

To process it's output, we use:

//... resp, err := processStreamingOutput(output, func(ctx context.Context, part []byte) error { fmt.Print(string(part)) return nil })

Here are a few bits from the processStreamingOutput function - you can check the code here. The important thing to understand is how the partial responses are collected together to produce the final output (MistralResponse).

func processStreamingOutput(output *bedrockruntime.InvokeModelWithResponseStreamOutput, handler StreamingOutputHandler) (MistralResponse, error) { var combinedResult string resp := MistralResponse{} op := Outputs{} for event := range output.GetStream().Events() { switch v := event.(type) { case *types.ResponseStreamMemberChunk: var pr MistralResponse err := json.NewDecoder(bytes.NewReader(v.Value.Bytes)).Decode(&pr) if err != nil { return resp, err } handler(context.Background(), []byte(pr.Outputs[0].Text)) combinedResult += pr.Outputs[0].Text op.StopReason = pr.Outputs[0].StopReason //... } op.Text = combinedResult resp.Outputs = []Outputs{op} return resp, nil }

Conclusion

Remember - building AI/ML applications using Large Language Models (like Mistral, Meta Llama, Claude, etc.) does not imply that you have to use Python. Managed platforms like Amazon Bedrock provide access to these powerful models using flexible APIs in a variety of programming languages, including Go! Thanks to AWS SDK support, you can use the programming language of your choice to integrate with Amazon Bedrock, and build generative AI solutions.

You can learn more by exploring the official Mistral documentation as well the Amazon Bedrock user guide. Happy building!

The above is the detailed content of Use Mistral AI to build generative AI applications with Go. For more information, please follow other related articles on the PHP Chinese website!