最初由部落格發佈
Kubernetes 標準庫充滿了寶石，隱藏在生態系統中的許多不同的子包中。我最近發現了一個這樣的範例 k8s.io/client-go/tools/leaderelection，它可用於向 Kubernetes 叢集內執行的任何應用程式新增領導者選舉協定。本文將討論什麼是領導者選舉，它是如何在這個 Kubernetes 套件中實現的，並提供一個範例來說明如何在我們自己的應用程式中使用這個函式庫。
領導者選舉是一個分散式系統概念，是高可用軟體的核心構建塊。它允許多個並發進程相互協調並選舉一個「領導者」進程，然後該進程負責執行同步操作，例如寫入資料儲存。
這在分散式資料庫或快取等系統中非常有用，在這些系統中，多個進程正在運行以針對硬體或網路故障建立冗餘，但無法同時寫入儲存以確保資料一致性。如果領導者進程在未來某個時刻變得無回應，剩餘進程將啟動新的領導者選舉，最終選擇一個新進程作為領導者。
利用這個概念，我們可以創建具有單一領導者和多個備用副本的高可用軟體。
在 Kubernetes 中，controller-runtime 套件使用領導者選舉來使控制器具有高可用性。在控制器部署中，僅當進程是領導者並且其他副本處於等待狀態時才會發生資源協調。如果 Leader Pod 沒有回應，剩餘的副本將選出一個新的 Leader 來執行後續協調並恢復正常運作。
該程式庫使用 Kubernetes Lease 或分散式鎖，可以由進程取得。租約是由單一身分在給定期限內持有的原生 Kubernetes 資源，並具有續約選項。 以下是文件中的範例規格：
apiVersion: coordination.k8s.io/v1
kind: Lease
metadata:
labels:
apiserver.kubernetes.io/identity: kube-apiserver
kubernetes.io/hostname: master-1
name: apiserver-07a5ea9b9b072c4a5f3d1c3702
namespace: kube-system
spec:
holderIdentity: apiserver-07a5ea9b9b072c4a5f3d1c3702_0c8914f7-0f35-440e-8676-7844977d3a05
leaseDurationSeconds: 3600
renewTime: "2023-07-04T21:58:48.065888Z"
k8s 生態係以三種方式使用租約：
現在讓我們透過編寫範例程式來探索租賃的第二個用例，以演示如何在領導者選舉場景中使用它們。
在此程式碼範例中，我們使用 Leaderelection 套件來處理領導者選舉和租約操作細節。
package main
import (
"context"
"fmt"
"os"
"time"
"k8s.io/client-go/tools/leaderelection"
rl "k8s.io/client-go/tools/leaderelection/resourcelock"
ctrl "sigs.k8s.io/controller-runtime"
)
var (
// lockName and lockNamespace need to be shared across all running instances
lockName = "my-lock"
lockNamespace = "default"
// identity is unique to the individual process. This will not work for anything,
// outside of a toy example, since processes running in different containers or
// computers can share the same pid.
identity = fmt.Sprintf("%d", os.Getpid())
)
func main() {
// Get the active kubernetes context
cfg, err := ctrl.GetConfig()
if err != nil {
panic(err.Error())
}
// Create a new lock. This will be used to create a Lease resource in the cluster.
l, err := rl.NewFromKubeconfig(
rl.LeasesResourceLock,
lockNamespace,
lockName,
rl.ResourceLockConfig{
Identity: identity,
},
cfg,
time.Second*10,
)
if err != nil {
panic(err)
}
// Create a new leader election configuration with a 15 second lease duration.
// Visit https://pkg.go.dev/k8s.io/client-go/tools/leaderelection#LeaderElectionConfig
// for more information on the LeaderElectionConfig struct fields
el, err := leaderelection.NewLeaderElector(leaderelection.LeaderElectionConfig{
Lock: l,
LeaseDuration: time.Second * 15,
RenewDeadline: time.Second * 10,
RetryPeriod: time.Second * 2,
Name: lockName,
Callbacks: leaderelection.LeaderCallbacks{
OnStartedLeading: func(ctx context.Context) { println("I am the leader!") },
OnStoppedLeading: func() { println("I am not the leader anymore!") },
OnNewLeader: func(identity string) { fmt.Printf("the leader is %s\n", identity) },
},
})
if err != nil {
panic(err)
}
// Begin the leader election process. This will block.
el.Run(context.Background())
}
leaderelection 包的優點在於它提供了一個基於回調的框架來處理領導者選舉。這樣，您可以以精細的方式對特定的狀態變化採取行動，並在選舉新領導者時適當地釋放資源。透過在單獨的 goroutine 中運行這些回調，該套件利用 Go 強大的並發支援來有效地利用機器資源。
為了測試這一點，讓我們使用 kind 啟動一個測試叢集。
$ kind create cluster
將範例程式碼複製到 main.go 中，建立一個新模組（go mod init Leaderelectiontest）並整理它（go mod tidy）以安裝其依賴項。執行 go run main.go 後，您應該會看到以下輸出：
$ go run main.go I0716 11:43:50.337947 138 leaderelection.go:250] attempting to acquire leader lease default/my-lock... I0716 11:43:50.351264 138 leaderelection.go:260] successfully acquired lease default/my-lock the leader is 138 I am the leader!
The exact leader identity will be different from what's in the example (138), since this is just the PID of the process that was running on my computer at the time of writing.
And here's the Lease that was created in the test cluster:
$ kubectl describe lease/my-lock Name: my-lock Namespace: default Labels: <none> Annotations: <none> API Version: coordination.k8s.io/v1 Kind: Lease Metadata: Creation Timestamp: 2024-07-16T15:43:50Z Resource Version: 613 UID: 1d978362-69c5-43e9-af13-7b319dd452a6 Spec: Acquire Time: 2024-07-16T15:43:50.338049Z Holder Identity: 138 Lease Duration Seconds: 15 Lease Transitions: 0 Renew Time: 2024-07-16T15:45:31.122956Z Events: <none>
See that the "Holder Identity" is the same as the process's PID, 138.
Now, let's open up another terminal and run the same main.go file in a separate process:
$ go run main.go I0716 11:48:34.489953 604 leaderelection.go:250] attempting to acquire leader lease default/my-lock... the leader is 138
This second process will wait forever, until the first one is not responsive. Let's kill the first process and wait around 15 seconds. Now that the first process is not renewing its claim on the Lease, the .spec.renewTime field won't be updated anymore. This will eventually cause the second process to trigger a new leader election, since the Lease's renew time is older than its duration. Because this process is the only one now running, it will elect itself as the new leader.
the leader is 604 I0716 11:48:51.904732 604 leaderelection.go:260] successfully acquired lease default/my-lock I am the leader!
If there were multiple processes still running after the initial leader exited, the first process to acquire the Lease would be the new leader, and the rest would continue to be on standby.
This package is not foolproof, in that it "does not guarantee that only one client is acting as a leader (a.k.a. fencing)". For example, if a leader is paused and lets its Lease expire, another standby replica will acquire the Lease. Then, once the original leader resumes execution, it will think that it's still the leader and continue doing work alongside the newly-elected leader. In this way, you can end up with two leaders running simultaneously.
To fix this, a fencing token which references the Lease needs to be included in each request to the server. A fencing token is effectively an integer that increases by 1 every time a Lease changes hands. So a client with an old fencing token will have its requests rejected by the server. In this scenario, if an old leader wakes up from sleep and a new leader has already incremented the fencing token, all of the old leader's requests would be rejected because it is sending an older (smaller) token than what the server has seen from the newer leader.
Implementing fencing in Kubernetes would be difficult without modifying the core API server to account for corresponding fencing tokens for each Lease. However, the risk of having multiple leader controllers is somewhat mitigated by the k8s API server itself. Because updates to stale objects are rejected, only controllers with the most up-to-date version of an object can modify it. So while we could have multiple controller leaders running, a resource's state would never regress to older versions if a controller misses a change made by another leader. Instead, reconciliation time would increase as both leaders need to refresh their own internal states of resources to ensure that they are acting on the most recent versions.
Still, if you're using this package to implement leader election using a different data store, this is an important caveat to be aware of.
Leader election and distributed locking are critical building blocks of distributed systems. When trying to build fault-tolerant and highly-available applications, having tools like these at your disposal is critical. The Kubernetes standard library gives us a battle-tested wrapper around its primitives to allow application developers to easily build leader election into their own applications.
While use of this particular library does limit you to deploying your application on Kubernetes, that seems to be the way the world is going recently. If in fact that is a dealbreaker, you can of course fork the library and modify it to work against any ACID-compliant and highly-available datastore.
Stay tuned for more k8s source deep dives!
以上是如何將 Kubernetes 支援的領導者選舉添加到您的 Go 應用程式中的詳細內容。更多資訊請關注PHP中文網其他相關文章！
