管理Kubernetes集群的50個開源實用工具

推薦一篇博客，這篇博客裡列舉了50個實用的Kubenretes工具，所有的工具都是開源的，有的提供商業支持，這些工具分為以下幾類：

集群部署工具

監控工具

測試工具

安全性管理

命令行幫助工具

開發工具

CI/CD流水線

Serverless/Function工具

服務發現

可視化&控制

另外我還整理了一份Awsome Cloud Native列表，關注整個雲原生領域的開源軟體——https://jimmysong.io/awesome-cloud-native/

如果僅僅是打造一個高效的Kubernetes管理命令行終端，可以參考我的這篇文章：https://jimmysong.io/posts/configuring-efficient-kubernetes-cli-terminal/

In just two relatively short years, Kubernetes has laid waste to its fellow competitors in the battlefield of container orchestration. Sadly, Docker Swarm hasn’t been a major contender since 2016 and, like AWS, admitted defeat by pledging K8s support and integration.

Since Kubernetes has skyrocketed to popularity as the container solution of choice, here’s a comprehensive list of all the tools that complement K8s to further enhance your development work.

Kubernetes Categories List:Kube Cluster Deployment Tools

1. Kubespray

Kubespray provides a set of Ansible roles for Kubernetes deployment and configuration. Kubespray can use AWS, GCE, Azure, OpenStack or a bare metal Infrastructure as a Service (IaaS) platform. Kubespray is an open-source project with an open development model. The tool is a good choice for people who already know Ansible as there’s no need to use another tool for provisioning and orchestration. Kubespray uses kubeadm under the hood.

Link: https://github.com/kubernetes-incubator/kubespray

Cost: Free

2. Minikube

Minikube allows you to install and try out Kubernetes locally. The tool is a good starting point for Kubernetes exploration. Easily launch a single-node Kubernetes cluster inside a virtual machine (VM) on your laptop. Minikube is available on Windows, Linux, and OSX. In just 5 minutes you will be able to explore Kubernetes』 main features. Launch the Minikube dashboard straight-from-the-box with just one command.

Link: https://github.com/kubernetes/minikube

Cost: Free

3. Kubeadm

Kubeadm is a Kubernetes distribution tool since version 1.4. The tool helps to bootstrap best-practice Kubernetes clusters on existing infrastructure. Kubeadm cannot provision infrastructure for you though. Its main advantage is the ability to launch minimum viable Kubernetes clusters anywhere. Add-ons and networking setup are both out of Kubeadm’s scope though, so you will need to install this manually or using another tool.

Link: https://github.com/kubernetes/kubeadm

Cost: Free

4. Kops

Kops helps you create, destroy, upgrade, and maintain production-grade, highly available Kubernetes clusters from the command line. Amazon Web Services (AWS) is currently officially supported, with GCE in beta support, and VMware vSphere in alpha, and other platform support is planned. Kops allows you to control the full Kubernetes cluster lifecycle; from infrastructure provisioning to cluster deletion.

Link: https://github.com/kubernetes/kops

Cost: Free

5. Bootkube

CoreOS provided the concept of self-hosted Kubernetes cluster with the version 1.4 release. In the center of the self-hosted cluster approach is Bootkube. Bootkube helps you set up a temporary Kubernetes control plane which will operate until the self-hosted control-plane is able to handle requests.

Link: https://github.com/kubernetes-incubator/bootkube

Cost: Free

6. Kubernetes on AWS (Kube-AWS)

Kube-AWS is a console tool provided by CoreOS which deploys a fully-functional Kubernetes cluster using AWS CloudFormation. Kube-AWS allows you to deploy a traditional Kubernetes cluster and automatically provision every K8s service with native AWS features (e.g., ELB, S3, and Auto Scaling, etc.).

Link: https://github.com/kubernetes-incubator/kube-aws

Cost: Free

7. SimpleKube

SimpleKube is a bash script which allows you to deploy a single-node Kubernetes cluster on a Linux server. While Minikube needs a hypervisor (VirtualBox, KVM), SimpleKube will install all K8s binaries into the server itself. Simplekube is tested on Debian 8/9 and Ubuntu 16.x/17.x. It’s a great tool for giving Kubernetes a first try.

Link: https://github.com/valentin2105/Simplekube

Cost: Free

8. Juju

Juju is an orchestrator from Canonical that allows you to remotely operate cloud provider solutions. Juju works at a higher level of abstraction than Puppet/Ansible/Chef and manages services instead of machines/VMs. Canonical made the great effort to provide what they call a suitable 「Kubernetes-core bundle」 in production. Juju is available as a dedicated tool with its own console/UI interface and also as a service (JaaS) which is free during the beta period.

Link: https://jujucharms.com/

Cost:Free Community EditionCommercial Edition – from 200$ per year

9. Conjure-up

Conjure-up is another Canonical product which allows you to deploy 「The Canonical Distribution of Kubernetes on Ubuntu」 with a few simple commands. It supports AWS, GCE, Azure, Joyent, OpenStack, VMware, bare metal, and localhost deployments. Juju, MAAS, and LXD are the underlying technology for Conjure-up.

Link: https://conjure-up.io/

Cost: Free

Monitoring Tools10. Kubebox

Kubebox is a terminal console for Kubernetes cluster which allows you to manage and monitor your cluster-live status with nice, old-school interface. Kubebox shows your pod resource usage, cluster monitoring, and container logs, etc. Additionally, you can easily navigate to the desired namespace and execute into the desired container for fast troubleshooting/recovery.

Link: https://github.com/astefanutti/kubebox

Cost: Free

11. Kubedash

Kubedash offers a performance analytics UI for Kubernetes. Kubedash aggregates and summarizes metrics from different sources and provides high-level analytic data for administrators. Kubedash uses Heapster as a source of data which runs as a service by default in all Kubernetes clusters collecting metrics and analytics for individual containers.

Link: https://github.com/kubernetes-retired/kubedash

Cost: Free

12. Kubernetes Operational View (Kube-ops-view)

Kube-ops-view is a read-only system dashboard for multiple K8s clusters. With Kube-ops-view you can easily navigate between your cluster and monitor nodes as well as your pod’s healthiness. Kube-ops-view animates some Kubernetes processes such as pod creation and termination. It also uses Heapster as a source of data.

Link: https://github.com/hjacobs/kube-ops-view

Cost: Free

13. Kubetail

Kubetail is a small bash script which allows you to aggregate logs from multiple pods into one stream. The initial Kubetail version doesn’t have filtering or highlighting features, but there is an additional Kubetail fork on Github. This can form and perform logs coloring using multitail tools.

Link: https://github.com/johanhaleby/kubetailhttps://github.com/aks/kubetail

Cost: Free

14. Kubewatch

Kubewatch is a Kubernetes watcher which can publish K8s events to the team communication app, Slack. Kubewatch runs as a pod inside Kubernetes clusters and monitors changes that occur in the system. You can specify the notifications you want to receive by editing the configuration file.

Link: https://github.com/bitnami-labs/kubewatch

Cost: Free

15. Weave Scope

Weave Scope is a troubleshooting and monitoring tool for Docker and Kubernetes clusters. It can automatically generate applications and infrastructure topologies which can help you to identify application performance bottlenecks easily. You can deploy Weave Scope as a standalone application on your local server/laptop, or you can choose the Weave Scope Software as a Service (SaaS) solution on Weave Cloud. With Weave Scope, you can easily group, filter or search containers using names, labels, and/or resource consumption.

Link: https://www.weave.works/oss/scope/

Cost:Free in standalone modeStandard mode – 30% per month (free 30-day trial)Enterprise mode – 150$ per node/month

16. Searchlight

Searchlight by AppsCode is a Kubernetes operator for Icinga. Searchlight periodically runs various checks on Kubernetes clusters and alerts you via email, SMS or chat if something goes wrong. Searchlight includes a default suite of checks written specifically for Kubernetes. Also, it can enhance Prometheus monitoring with external black-box monitoring and serves as a fallback in case internal systems completely fail.

Link: https://github.com/appscode/searchlight

Cost: Free

17. Heapster

Heapster enables container cluster monitoring and performance analysis for Kubernetes. Heapster supports Kubernetes natively and can run as a pod on all K8s setups. Heapster’s data then can be pushed to a configurable backend for storage and visualization.

Link: https://github.com/kubernetes/heapster

Cost: Free

Testing18. Kube-monkey

Kube-monkey is the Kubernetes』 version of Netflix’s Chaos Monkey. Kube-monkey is a tool that follows the principles of chaos engineering. It can delete K8s pods at random, check services are failure-resilient, and contribute to your system’s healthiness. Kube-monkey is also configured by a TOML file where you can specify which app is to be killed and when to practice your recovery strategies.

Link: https://github.com/asobti/kube-monkey

Cost: Free

19. K8s-testsuite

K8s-testsuite is made up of 2 Helm charts which work for network bandwidth testing and load testing a single Kubernetes cluster. Load tests emulate simple web-servers with loadbots which run as a Kubernetes microservice based on the Vegeta. Network tests use iperf3 and netperf-2.7.0 internally and run three times. Both sets of tests generate comprehensive log messages with all results and metrics.

Link: https://github.com/mrahbar/k8s-testsuite

Cost: Free

20. Test-infra

Test-infra is a collection of tools for Kubernetes testing and results verification. Test-infra includes a few dashboards for displaying history, aggregating failures, and showing what is currently testing. You can enhance your test-infra suite by creating your own test jobs. Test-infra can perform end-to-end Kubernetes testing with full Kubernetes lifecycle emulation on different providers using the Kubetest tool.

Link: https://github.com/kubernetes/test-infra

Cost: Free

21. Sonobuoy

Sonobuoy allows you to understand your current Kubernetes cluster state by running a set of tests in an accessible and non-destructive manner. Sonobuoy generates informative reports with detailed information about cluster performance. Sonobuoy supports Kubernetes versions 1.8 and on. Sonobuoy Scanner is a browser-based tool which allows you to test Kubernetes clusters in a few clicks, but the CLI version has a bigger set of tests available.

Link: https://github.com/heptio/sonobuoy

Cost: Free

22. PowerfulSeal

PowerfulSeal is a tool similar to Kube-monkey and follows the Principles of Chaos Engineering. PowerfulSeal can kill pods and remove/add VMs from or to your clusters. In contrast to Kube-monkey, PowefulSeal has an interactive mode which allows you to manually break specific cluster components. Also, PowefulSeal doesn’t need external dependencies apart from SSH.

Link: https://github.com/bloomberg/powerfulseal

Cost: Free

Security23. Trireme

Trireme is a flexible and straightforward implementation of the Kubernetes Network Policies. Trireme works in any Kubernetes cluster and allows you to manage traffic between pods from different clusters. The main advantages of Trireme are the lack of a need for any centralized policy management, the ability to easily organize the interaction of the two resources deployed in Kubernetes, and the lack of complexities of SDN, VLAN tags, and subnets (Trireme uses a conventional L3-network).

Link: https://github.com/aporeto-inc/trireme-kubernetes

Cost: Free

24. Aquasec

Aquasec provides full lifecycle security for Kubernetes deployments. Aqua Security deploys a dedicated agent on each and every container instance which works as a firewall and blocks security holes in the container. This agent communicates with the central Aqua Security console which enforces defined security restraints. Aqua Security also helps to organize a flexible security delivery pipeline for cloud and on-premise environments. Kube-Bench is an open-source tool released by AquaSec that checks Kubernetes environments against the long list of tests outlined in the CIS Kubernetes Benchmark.

Link: https://www.aquasec.com/

Cost: 0.29$ per scan

25. Twistlock

Twistlock is another tool which works as a 「cloud-native application firewall」 and analyzes network traffic between containers and services. Twistlock is able to analyze standard container behavior and generate appropriate rules according to this behavior so that administrators don’t have to manually generate rules of policy. Twistlock also supports Kubernetes’s CIS Benchmark from version 2.2.

Link: https://www.twistlock.com/

Cost: from 1700$ per year per license (Free trial available)

26. Sysdig Falco

Sysdig Falco is a behavioral activity monitor designed to detect anomalous activity in your applications. Falco is based on the Sysdig Project, an open source tool (and now a commercial service), built for monitoring container performance by way of tracking kernel system calls. Falco lets you continuously monitor and detect container, application, host, and network activity with one set of rules.

Link: https://sysdig.com/opensource/falco/

Cost:Free as a standalone toolBasic Cloud: $20 per month (free trial)Pro Cloud: $30 per monthPro Software: Custom price

27. Sysdig Secure

Sysdig Secure, part of the Sysdig Container Intelligence Platform, comes out-of-the-box with unmatched container visibility and deep integrations with container orchestration tools. These include Kubernetes, Docker, AWS ECS, and Apache Mesos. With Sysdig Secure you can Implement service-aware policies, block attacks, analyze your history, and monitor cluster performance. Sysdig Secure is available as cloud and on-premise software offerings.

Link: https://sysdig.com/product/secure/

Cost:Free as a standalone toolPro Cloud: Custom pricePro Software: Custom price

28. Kubesec.io

Kubesec.io is a service which allows you to score Kubernetes resources for security feature usage. Kubesec.io verifies resource configuration according to Kubernetes security best-practices. As a result, you will have total control and additional suggestions for how to improve overall system security. The site also contains plenty of external links related to containers and Kubernetes security.

Link: https://kubesec.io

Cost: Free

Helpful CLI Tools29. Cabin

Cabin functions as a mobile dashboard for the remote management of Kubernetes clusters. With Cabin, users can quickly manage applications, scale deployments, and troubleshoot overall K8s cluster from their Android or iOS device. Cabin is a great tool for operators of K8s clusters as it allows you to perform quick remediation actions in case of incidents.

Link: https://github.com/bitnami-labs/cabin

Cost: Free

30. Kubectx/Kubens

Kubectx is a small open-source utility tool which enhances Kubectl functionality with the possibility to switch context easily and connect to a few Kubernetes clusters at the same time. Kubens allows you to navigate between Kubernetes namespaces. Both tools have an auto-completion feature on bash/zsh/fish shells.

Link: https://github.com/ahmetb/kubectx

Cost: Free

31. Kube-shell

Kube-shell increases your productivity when working with kubectl. Kube-shell enables command auto-completion and auto-suggestion. Also, Kube-shell will provide in-line documentation about executed command. Kube-shell even can search and correct commands when wrongly typed. It’s a great tool to increase your performance and productivity in the K8s console.

Link: https://github.com/cloudnativelabs/kube-shell

Cost: Free

32. Kail

Kail is short for Kubernetes tail and works for Kubernetes clusters. With Kail, you can tail Docker logs for all matched pods. Kail allows you to filter pods by service, deployment, labels, and other features. Pods will be added (or removed) automatically to the log after a launch if it matches the criteria.

Link: https://github.com/boz/kail

Cost: Free

Development Tools

33. Telepresence

Telepresence provides the possibility to debug Kubernetes clusters locally by proxy data from your Kubernetes environment to the local process. Telepresence is able to provide access to Kubernetes services and AWS/GCP resources for your local code as it will be deployed to the cluster. With Telepresence, Kubernetes counts local code as a normal pod within your cluster.

Link: https://www.telepresence.io/

Cost: Free

34. Helm

Helm is a package manager for Kubernetes. It is like APT/Yum/Homebrew, but for Kubernetes. Helm operates with Char which is an archive set of Kubernetes resource manifests that make up a distributed application. You can share your application by creating a Helm chart. Helm allows you to create reproducible builds and manage Kubernetes manifests easily.

Link: https://github.com/kubernetes/helm

Cost: Free

35. Keel

Keel allows you to automate Kubernetes deployment updates and can be launched as a Kubernetes service in a dedicated namespace. With such organization, Keel introduces a minimal load on your environment and adds significant robustness. Keel helps to deploy Kubernetes service through labels, annotations, and charts. You just need to specify an update policy for each deployment or Helm release. Keel will automatically update your environment as soon as the new application version is available in the repository.

Link: https://keel.sh/

Cost: Free

36. Apollo

Apollo is an open source application providing teams with self-service UI for creating and deploying their services to Kubernetes. Apollo allows operators to view logs and revert deployments to any point in time with just one click. Apollo has flexible permission models for deployments. Each user can deploy only what he needs to deploy.

Link: https://github.com/logzio/apollo

Cost: Free

37. Draft

Draft is a tool provided by the Azure team that streamlines application development and deployment into any Kubernetes cluster. Draft creates 「inner loops」 between code deployment and code commits which significantly speed up the change verification process. With Draft, developers can prepare application Dockerfiles and Helm charts plus deploy applications to a remote or local Kubernetes cluster with two commands.

Link: https://github.com/azure/draft

Cost: Free

38. Deis Workflow

Deis Workflow is an open source tool. The Platform as a Service (PaaS) creates additional layers of abstraction on top of Kubernetes clusters. These layers allow you to deploy and/or update Kubernetes applications without specific domain knowledge from developers. Workflow builds upon Kubernetes concepts to provide simple, developer-friendly app deployment. Delivered as a set of Kubernetes microservices, operators can easily install the platform. Workflow can deploy new versions of your app with zero downtime.

Link: https://deis.com/workflow/

Cost: Free

39. Kel

Kel is an open source PaaS from Eldarion, Inc. which helps to manage Kubernetes applications through the entire lifecycle. Kel provides two additional layers written in Python and Go on top of Kubernetes. Level 0 allows you to provision Kubernetes resources, and Level 1 helps you to deploy any application on K8s.

Link: http://www.kelproject.com/

Cost: Free

Continuous Integration/ Continuous Delivery Pipeline

40. Cloud 66

A full DevOps toolchain for containerized apps in production, Cloud 66 automates much of the heavy-lifting for Devs through specialized Ops tools. The platform currently runs 4,000 customer workloads on Kubernetes and manages 2,500 lines of config. By offering end-to-end infrastructure management, Cloud 66 enables engineers to build, deliver, deploy, and manage any application on any cloud or server.

Link: www.cloud66.com/

Cost: Free for 14 days

Serverless/Function Tools

41. Kubeless

Kubeless is a Kubernetes-native serverless framework that lets you deploy small bits of code without having to worry about the underlying infrastructure plumbing. Kubeless is aware of Kubernetes resources out-of-the-box and also provides auto-scaling, API routing, monitoring, and troubleshooting. Kubeless fully relies on K8s primitives, so Kubernetes users will also be able to use native K8s API servers and API gateways.

Link: https://github.com/kubeless/kubeless

Cost: Free

42. Fission

Fission is a fast serverless framework for Kubernetes with a focus on developer productivity and high performance. Fission works on a Kubernetes cluster anywhere: on your laptop, in any public cloud, or in a private data-center. You can write your function using Python, NodeJS, Go, C# or PHP, and deploy it on K8s clusters with Fission.Link: https://fission.io/

Cost: Free

43. Funktion

For a long time, there was only one Function as a Service (FaaS) implementation available for Kubernetes: Funktion. Funktion is an open source event-driven lambda-style programming model designed for Kubernetes. Funktion is tightly coupled with the fabric8 platform. With Funktion, you can create flows to subscribe from over 200 event sources to invoke your function, including most databases, messaging systems, social media, and other middleware and protocols.

Link: https://github.com/funktionio/funktion

Cost: Free

44. IronFunction

IronFunctions is an open source serverless platform or FaaS platform that you can run anywhere. IronFunction is written on Golang and really supports functions in any language. The main advantage of IronFunction is that it supports the AWS Lambda format. Import functions directly from Lambda and run them wherever you want.

Link: https://github.com/iron-io/functions

Cost: Free

45. OpenWhisk

Apache OpenWhisk is a robust open source FaaS platform driven by IBM and Adobe. OpenWhisk can be deployed on a local on-premise device or on the cloud. The design of Apache OpenWhisk means it acts as an asynchronous and loosely-coupled execution environment that can run functions against external triggers. OpenWhisk is available as SaaS solution on Bluemix, or you can deploy a Vagrant-based VM locally.

Link: https://console.bluemix.net/openwhisk/

Cost: Free

46. OpenFaaS

The OpenFaaS framework aims to manage serverless functions on Docker Swarm or Kubernetes where it will collect and analyze a wide range of metrics. You can package any process inside your function and use it without repetitive coding or any other routine action. FaaS has Prometheus metrics baked-in, which means it can automatically scale your functions up and down for demand. FaaS natively supports a web-based interface where you can try out your function.

Link: https://github.com/openfaas/faas

Cost: Free

47. Nuclio

Nuclio is a serverless project which aims to proceed with high-performance events and large amounts of data. Nuclio can be launched on an on-premise device as a standalone library or inside a VM/Docker container. Also, Nuclio supports Kubernetes out of the box. Nuclio provides real-time data processing with maximum parallelism and minimum overheads. You can try out Nuclio on the playground page.

Link: https://github.com/nuclio/nuclio

Cost: Free

48. Virtual-Kubelet

Virtual Kubelet is an open source Kubernetes Kubelet implementation that masquerades as a kubelet for the purposes of connecting Kubernetes to other APIs. Virtual Kubelet allows the nodes to be backed by other services like ACI, Hyper.sh, and AWS, etc. This connector features a pluggable architecture and direct use of Kubernetes primitives, making it much easier to build on.

Link: https://github.com/virtual-kubelet/virtual-kubeletCost: Free

49. Fnproject

Fnproject is a container native serverless project which supports practically any language and can run almost everywhere. Fn is written on Go, so it is performance-ready and lightweight. Fnproject supports AWS Lambda format style, so you can easily import your Lambda functions and launch it with Fnproject.

Link: http://fnproject.io/

Cost: Free

Native Service Discovery

50. CoreDNS

CoreDNS is a set of plugins written in Go which perform DNS functions. CoreDNS with additional Kubernetes plugins can replace the default Kube-DNS service and implement the specification defined for Kubernetes DNS-based service discovery. CoreDNS can also listen for DNS requests coming in over UDP/TCP, TLS, and gRPC.

Link: https://coredns.io/

Cost: Free

Native Visualization & Control51. Kubernetes Dashboard

Kubernetes Dashboard is a general purpose, web-based UI for Kubernetes clusters. It is much easier to troubleshoot and monitor K8s clusters with a native dashboard. You need to create a secure proxy channel between your machine and Kubernetes API server to access the dashboard. The native Kubernetes dashboard relies on the Heapster data collector, so it also needs to be installed in the system.

Link: https://github.com/kubernetes/dashboard#kubernetes-dashboard

Cost: Free

And that’s the complete list! As always, we』d love your feedback and suggestions for future articles. (Don’t forget to check out our 50+ Useful Docker Tools too!)

點擊【閱讀原文】可以跳轉到原文頁面。

同時再推薦一個了解Kubernetes原理的課程（圖文），適用於初始Kubenretes或者對Kubernetes原理有更深入追求的人，可以在茶餘飯後，夜深人靜的時候看看。