Waiting for AWS Athena query execution in the Go SDK-Golang-php.cn

Waiting for AWS Athena query execution in the Go SDK

王林

Release： 2024-02-09 15:09:18

forward

1087 people have browsed it

在 Go SDK 中等待 AWS Athena 查询执行

php editor Baicao is waiting for the AWS Athena query execution guide in the Go SDK, which aims to help developers optimize query performance and improve application response speed. AWS Athena is a serverless query service that can run SQL queries directly in S3 buckets without pre-defined schema or data loading. However, due to the asynchronous execution nature of the query, developers need to wait for the query to complete before obtaining the results. This guide will explain how to use the wait mechanism in the Go SDK to effectively wait during query execution and get the results after the query is completed, providing a better user experience and application performance.

Question content

I have a working code that runs an athena query and waits for the query by polling getqueryresults for the returned error using the following code Finish:

func getqueryresults(client *athena.client, queryid *string) []types.row {

    params := &athena.getqueryresultsinput{
        queryexecutionid: queryid,
    }

    data, err := client.getqueryresults(context.todo(), params)

    for err != nil {
        println(err.error())
        time.sleep(time.second)
        data, err = client.getqueryresults(context.todo(), params)
    }

    return data.resultset.rows
}

Copy after login

The problem is that I have absolutely no way to break the loop if the query fails.

For example, in python I can do the following:

    while athena.get_query_execution(QueryExecutionId=execution_id)["QueryExecution"][
        "Status"
    ]["State"] in ["RUNNING", "QUEUED"]:
        sleep(2)

Copy after login

I could do a check like strings.contains(err.error(),"failed") inside the for loop, but I'm looking for a cleaner way .

I tried looking for the equivalent for go but without success. Is there any function in go sdk that can return execution status? Is there a better way to further check for errors in go than err != nil ?

Solution

sdk has provided a retry function.

This is an example using aws-sdk-go-v2.

package main

import (
    "context"
    "fmt"
    "time"

    "github.com/aws/aws-sdk-go-v2/aws"
    "github.com/aws/aws-sdk-go-v2/service/athena"
    "github.com/aws/aws-sdk-go-v2/service/athena/types"
)

func main() {
    cfg := aws.NewConfig()
    ath := athena.NewFromConfig(*cfg)

    ctx, cancelFunc := context.WithTimeout(context.Background(), time.Minute*5)
    defer cancelFunc()

    rows, err := GetQueryResults(ctx, ath, aws.String("query-id"), 10)
    if err != nil {
        panic(err) // TODO: handle error
    }

    fmt.Println(rows)
}

func GetQueryResults(ctx context.Context, client *athena.Client, QueryID *string, attempts int) ([]types.Row, error) {
    t := time.NewTicker(time.Second * 5)
    defer t.Stop()

    attemptsFunc := func(o *athena.Options) { o.RetryMaxAttempts = attempts }

WAIT:
    for {
        select {
        case <-t.C:
            out, err := client.GetQueryExecution(ctx, &athena.GetQueryExecutionInput{
                QueryExecutionId: QueryID,
            }, attemptsFunc)
            if err != nil {
                return nil, err
            }

            switch out.QueryExecution.Status.State {
            case types.QueryExecutionStateCancelled,
                types.QueryExecutionStateFailed,
                types.QueryExecutionStateSucceeded:
                break WAIT
            }

        case <-ctx.Done():
            break WAIT
        }
    }

    data, err := client.GetQueryResults(ctx, &athena.GetQueryResultsInput{
        QueryExecutionId: QueryID,
    })
    if err != nil {
        return nil, err
    }

    return data.ResultSet.Rows, nil
}

Copy after login

The above is the detailed content of Waiting for AWS Athena query execution in the Go SDK. For more information, please follow other related articles on the PHP Chinese website!