Go kit
Building an app based on microservices architecture is not trivial. We would need to implement many specialized parts such as RPC safety, system observability, infrastructure integration, program design, and more. Go language's standard library itself isn't adequate to implement these easily. This is where Go kit becomes useful.
Using Go kit means that developers can focus on their application rather than grapple with the challenges of building a distributed system. Go kit is positioned as a toolkit for microservices. It's not a framework. It doesn't force developers to use everything it provides. It's also lightly opinionated about what components or architecture that developers should use. Go kit is really a library. Developers can choose only what they want.
Discussion
-
What's the relevance of Go kit in the context of microservices? 
When an app is composed of dozens or even hundreds of microservices, there's lot more to be done than just defining and building those services. We need consistent logging across services. Metrics have to be collected both at infrastructure layer and application layer. When a request comes in, we may need to trace how this request moves across multiple services. For robustness, we need rate limiting and circuit breaking. Each service will likely be running multiple instances that are dynamically created and destroyed. Service discovery and load balancing are therefore essential.
While Go has a much lower resource footprint compared to Ruby, Scala, Clojure or Node, back in 2014, it didn't have anything for a microservices architecture. Go kit's inventor, Peter Bourgon at SoundCloud, found that teams were choosing Scala and JVM because of better support for "microservices plumbing". In fact, SoundCloud adopted Twitter's Finagle. Go didn't have an equivalent to this. Thus was born the idea for Go kit. It's really,
a set of standards, best practices, and usable components for doing microservices in Go
-
Which are the key components of Go kit? 
Go kit microservices have three layers: Transport, Endpoint, Service. Requests come in at transport layer, move through endpoint layer and arrive at service layer. Responses take the reverse route.
Go kit supports a number of transports. Thus, legacy transports and modern transports can be supported within the same service. Some supported transports include NATS, gRPC, Thrift, HTTP, AMQP and AWS Lambda.
The primary messaging in Go kit is RPC. An endpoint is an RPC method that maps to a service method. A single endpoint can be exposed via multiple transports.
Service layer is where all the business logic resides. This is the concern of application developers who need not know anything about endpoints or transports, which are taken care of by Go kit. A service is modelled as an interface and its implementation contains the business logic.
Due to this separation of concerns, Go kit encourages you to adopt SOLID design principles and clean architecture. In fact, Go kit can be used to build elegant monoliths as well.
-
What's the concept of middleware in Go kit? 
Go kit takes the idea of "separation of concerns" further with the use of middleware. Rather than bloat core implementations with lots of functionality, additional functionality is provided through middleware using the decorator pattern. Go kit provides middleware for endpoint layer while application developers can write middleware for the service layer. We can also chain multiple middleware.
Here's a sample of middleware:
- Endpoint: Load balancing, safety, operational metrics, circuit breaking, rate limiting.
- Service: Includes anything that needs knowledge of business domain including app-level logging, analytics, and instrumentation.
As can be expected of the decorator pattern, an endpoint middleware accepts an endpoint and returns an endpoint. A service middleware accepts a service and returns a service.
Go kit doesn't force you to use all the middleware the come with it. For example, if you're using Istio that already comes with circuit breaking and rate limiting, then you ignore their equivalent Go kit middleware.
-
Which are the packages available in Go kit? Without being exhaustive, here are some Go kit packages:
- Authentication: Basic, casbin, JWT.
- Circuit Breaker: Hystrix, GoBreaker, and HandyBreaker.
- Logging: Provide an interface for structured logging. Recognizes that logs are data. They need context and semantics to be useful for analysis. Supported formats are logfmt and JSON.
- Metrics: Provides uniform interfaces for service instrumentation. Comes with counters, gauges, and histograms. Has adapters for CloudWatch, Prometheus, Graphite, DogStatsD, StatsD, expvar, and more.
- Rate Limit: Uses Go's token bucket implementation.
- Service Discovery: Consul, DNS SRV, etcd, Eureka, ZooKeeper, and more.
- Tracing: OpenCensus, OpenTracing, and Zipkin.
- Transport: AMQP, AWS Lambda, gRPC, HTTP, NATS, Thrift.
-
Could you share some best practices for using Go kit? Don't change your platform or infrastructure to suit Go kit. Remember that Go kit is only lightly opinionated. It will integrate well with whatever platform or infrastructure you're already using.
Go kit adopts domain-driven design and declarative composition. It uses interfaces as contracts. You should too when implementing your services.
Each middleware should have a single responsibility.
Other frameworks might use dependency injection. In Go kit, the preferred approach is to wire up the entire component graph in
func main. This forces you to pass them explicitly to components via constructors. Avoid using global state.Errors can be encoded in two ways: as an error field in response struct, or an error return value. For services, prefer the former method. For endpoints, prefer the latter since these are recognized by middleware such as circuit breaker.
Individual services should not collect or aggregate logs. This is the job of the platform. Your services should write to stdout/stderr.
Access to databases will likely be in a service implementation. Better still, use an interface to model persistence operations while an implementation wraps the database handle.
-
What are some criticisms or limitations of Go kit? It's been said that Go kit is too verbose. Adding an API to a microservice involves a lot of boilerplate code. While Go kit nicely separates the business logic, endpoint and transport layers, this abstraction comes at a cost. The code is harder to understand.
However, Developers who don't wish to write boilerplate code can make use of code generators. Two generators are kitgen and Kujtim Hoxha's GoKit CLI.
Inspired by Go kit, the engineering team at Grab created Grab-Kit. They felt that Go kit still requires a lot of manual work, something Grab-Kit aims for solve. The idea is to codify best practices. Grab-Kit standardizes APIs, SDKs, error handling, etc. It uses a consistent middleware stack across services including automatic profiling and execution traces.
Milestones
2015
2018
2018
References
- Beyang. 2017. "Peter Bourgon on the history of Go kit and what's next." Sourcegraph, July 12. Accessed 2019-08-15.
- Bourgon, Peter. 2015. "Go Kit: A Toolkit for Microservices." GopherCon UK, via YouTube, September 10. Accessed 2019-08-15.
- Bourgon, Peter. 2017. "Go + Microservices = Go kit." Cloud Native Con 2017, Berlin. Accessed 2019-08-15.
- Bryant, Daniel. 2015. "Building Microservices with Go and ‘Go kit’: Peter Bourgon Q&A." InfoQ, September 02. Accessed 2019-08-15.
- Go kit. 2019a. "Homepage." Go kit. Accessed 2019-08-15.
- Go kit. 2019b. "Frequently asked questions." Go kit. Accessed 2019-08-15.
- Go kit GitHub. 2019. "go-kit/kit." August 12. Accessed 2019-08-15.
- Klimenko, Anton. 2018. "Microservices in Go." Seek Blog, via Medium, June 04. Accessed 2019-08-15.
- Kue, Karen and Michael Cartmell. 2018. "Introducing Grab-Kit: Distributed Service Design at Grab." Tech Blog, Grab, June 08. Accessed 2019-08-15.
- Pliutau, Alex. 2018. "Microservices with go-kit. Part 1." July 30. Accessed 2019-08-15.
- Posener, Eyal. 2017. "Why I Recommend to Avoid Using the go-kit Library." Posener, on GitHub Gist, February 11. Updated 2018-01-13. Accessed 2019-08-15.
- Ryan, Fintan. 2018. "Language Framework Popularity: A Look at Go." RedMonk, February 07. Accessed 2019-08-15.
- Varghese, Shiju. 2018. "Go Microservices with Go kit: Introduction." Medium, November 17. Accessed 2019-08-15.
Further Reading
Article Stats
Cite As
See Also
- Go (Language)
- Concurrency in Go
- Microservices
- Service Discovery for Microservices
- Inter-Service Communication for Microservices
- Remote Procedure Call