Go kit

Go kit logo. Source: Go kit 2019a.
Go kit logo. Source: Go kit 2019a.

Building an app based on microservices architecture is not trivial. We would need to implement many specialized parts such as RPC safety, system observability, infrastructure integration, program design, and more. Go language's standard library itself isn't adequate to implement these easily. This is where Go kit becomes useful.

Using Go kit means that developers can focus on their application rather than grapple with the challenges of building a distributed system. Go kit is positioned as a toolkit for microservices. It's not a framework. It doesn't force developers to use everything it provides. It's also lightly opinionated about what components or architecture that developers should use. Go kit is really a library. Developers can choose only what they want.

Go kit is open source and follows the MIT License.

Discussion

  • What's the relevance of Go kit in the context of microservices?
    Requirements or features of a microservices architecture. Source: Bourgon 2015, 15:30.
    Requirements or features of a microservices architecture. Source: Bourgon 2015, 15:30.

    When an app is composed of dozens or even hundreds of microservices, there's lot more to be done than just defining and building those services. We need consistent logging across services. Metrics have to be collected both at infrastructure layer and application layer. When a request comes in, we may need to trace how this request moves across multiple services. For robustness, we need rate limiting and circuit breaking. Each service will likely be running multiple instances that are dynamically created and destroyed. Service discovery and load balancing are therefore essential.

    While Go has a much lower resource footprint compared to Ruby, Scala, Clojure or Node, back in 2014, it didn't have anything for a microservices architecture. Go kit's inventor, Peter Bourgon at SoundCloud, found that teams were choosing Scala and JVM because of better support for "microservices plumbing". In fact, SoundCloud adopted Twitter's Finagle. Go didn't have an equivalent to this. Thus was born the idea for Go kit. It's really,

    a set of standards, best practices, and usable components for doing microservices in Go
  • Which are the key components of Go kit?
    Transport layer does the encoding and decoding. Source: Bourgon 2017, slide 78.
    Transport layer does the encoding and decoding. Source: Bourgon 2017, slide 78.

    Go kit microservices have three layers: Transport, Endpoint, Service. Requests come in at transport layer, move through endpoint layer and arrive at service layer. Responses take the reverse route.

    Go kit supports a number of transports. Thus, legacy transports and modern transports can be supported within the same service. Some supported transports include NATS, gRPC, Thrift, HTTP, AMQP and AWS Lambda.

    The primary messaging in Go kit is RPC. An endpoint is an RPC method that maps to a service method. A single endpoint can be exposed via multiple transports.

    Service layer is where all the business logic resides. This is the concern of application developers who need not know anything about endpoints or transports, which are taken care of by Go kit. A service is modelled as an interface and its implementation contains the business logic.

    Due to this separation of concerns, Go kit encourages you to adopt SOLID design principles and clean architecture. In fact, Go kit can be used to build elegant monoliths as well.

  • What's the concept of middleware in Go kit?
    Separation of concerns in Go kit. Source: Bourgon 2017, slide 51.
    Separation of concerns in Go kit. Source: Bourgon 2017, slide 51.

    Go kit takes the idea of "separation of concerns" further with the use of middleware. Rather than bloat core implementations with lots of functionality, additional functionality is provided through middleware using the decorator pattern. Go kit provides middleware for endpoint layer while application developers can write middleware for the service layer. We can also chain multiple middleware.

    Here's a sample of middleware:

    • Endpoint: Load balancing, safety, operational metrics, circuit breaking, rate limiting.
    • Service: Includes anything that needs knowledge of business domain including app-level logging, analytics, and instrumentation.

    As can be expected of the decorator pattern, an endpoint middleware accepts an endpoint and returns an endpoint. A service middleware accepts a service and returns a service.

    Go kit doesn't force you to use all the middleware the come with it. For example, if you're using Istio that already comes with circuit breaking and rate limiting, then you ignore their equivalent Go kit middleware.

  • Which are the packages available in Go kit?

    Without being exhaustive, here are some Go kit packages:

    • Authentication: Basic, casbin, JWT.
    • Circuit Breaker: Hystrix, GoBreaker, and HandyBreaker.
    • Logging: Provide an interface for structured logging. Recognizes that logs are data. They need context and semantics to be useful for analysis. Supported formats are logfmt and JSON.
    • Metrics: Provides uniform interfaces for service instrumentation. Comes with counters, gauges, and histograms. Has adapters for CloudWatch, Prometheus, Graphite, DogStatsD, StatsD, expvar, and more.
    • Rate Limit: Uses Go's token bucket implementation.
    • Service Discovery: Consul, DNS SRV, etcd, Eureka, ZooKeeper, and more.
    • Tracing: OpenCensus, OpenTracing, and Zipkin.
    • Transport: AMQP, AWS Lambda, gRPC, HTTP, NATS, Thrift.
  • Could you share some best practices for using Go kit?

    Don't change your platform or infrastructure to suit Go kit. Remember that Go kit is only lightly opinionated. It will integrate well with whatever platform or infrastructure you're already using.

    Go kit adopts domain-driven design and declarative composition. It uses interfaces as contracts. You should too when implementing your services.

    Each middleware should have a single responsibility.

    Other frameworks might use dependency injection. In Go kit, the preferred approach is to wire up the entire component graph in func main. This forces you to pass them explicitly to components via constructors. Avoid using global state.

    Errors can be encoded in two ways: as an error field in response struct, or an error return value. For services, prefer the former method. For endpoints, prefer the latter since these are recognized by middleware such as circuit breaker.

    Individual services should not collect or aggregate logs. This is the job of the platform. Your services should write to stdout/stderr.

    Access to databases will likely be in a service implementation. Better still, use an interface to model persistence operations while an implementation wraps the database handle.

  • What are some criticisms or limitations of Go kit?

    It's been said that Go kit is too verbose. Adding an API to a microservice involves a lot of boilerplate code. While Go kit nicely separates the business logic, endpoint and transport layers, this abstraction comes at a cost. The code is harder to understand.

    However, Developers who don't wish to write boilerplate code can make use of code generators. Two generators are kitgen and Kujtim Hoxha's GoKit CLI.

    Inspired by Go kit, the engineering team at Grab created Grab-Kit. They felt that Go kit still requires a lot of manual work, something Grab-Kit aims for solve. The idea is to codify best practices. Grab-Kit standardizes APIs, SDKs, error handling, etc. It uses a consistent middleware stack across services including automatic profiling and execution traces.

Milestones

Feb
2015

Peter Bourgon, while working at SoundCloud, makes the first commit of Go kit code. However, the codebase is not versioned at this point. First tagged version v0.1.0 happens only in June 2016.

Jan
2018
Go kit's popularity based on GitHub stars. Source: Ryan 2018.
Go kit's popularity based on GitHub stars. Source: Ryan 2018.

Based on GitHub stars, Go kit is seen as the most popular toolkit, followed by Micro, Gizmo and Kite.

Mar
2018

Version 0.7.0 is released. This includes kitgen code generator, JSON-RPC transport, and etcdv3 support in service discovery.

Nov
2018

Version 0.8.0 is released. This includes NATS and AMPQ transports.

Jun
2019

Version 0.9.0 is released. This supports AWS Lambda as a transport.

References

  1. Beyang. 2017. "Peter Bourgon on the history of Go kit and what's next." Sourcegraph, July 12. Accessed 2019-08-15.
  2. Bourgon, Peter. 2015. "Go Kit: A Toolkit for Microservices." GopherCon UK, via YouTube, September 10. Accessed 2019-08-15.
  3. Bourgon, Peter. 2017. "Go + Microservices = Go kit." Cloud Native Con 2017, Berlin. Accessed 2019-08-15.
  4. Bryant, Daniel. 2015. "Building Microservices with Go and ‘Go kit’: Peter Bourgon Q&A." InfoQ, September 02. Accessed 2019-08-15.
  5. Go kit. 2019a. "Homepage." Go kit. Accessed 2019-08-15.
  6. Go kit. 2019b. "Frequently asked questions." Go kit. Accessed 2019-08-15.
  7. Go kit GitHub. 2019. "go-kit/kit." August 12. Accessed 2019-08-15.
  8. Klimenko, Anton. 2018. "Microservices in Go." Seek Blog, via Medium, June 04. Accessed 2019-08-15.
  9. Kue, Karen and Michael Cartmell. 2018. "Introducing Grab-Kit: Distributed Service Design at Grab." Tech Blog, Grab, June 08. Accessed 2019-08-15.
  10. Pliutau, Alex. 2018. "Microservices with go-kit. Part 1." July 30. Accessed 2019-08-15.
  11. Posener, Eyal. 2017. "Why I Recommend to Avoid Using the go-kit Library." Posener, on GitHub Gist, February 11. Updated 2018-01-13. Accessed 2019-08-15.
  12. Ryan, Fintan. 2018. "Language Framework Popularity: A Look at Go." RedMonk, February 07. Accessed 2019-08-15.
  13. Varghese, Shiju. 2018. "Go Microservices with Go kit: Introduction." Medium, November 17. Accessed 2019-08-15.

Further Reading

  1. Go kit. 2019. "The stringsvc tutorial." Go kit. Accessed 2019-08-15.
  2. Napoletano, Francesco. 2019. "How to write a microservice in Go with Go kit." Dev.to, April 15. Accessed 2019-08-15.
  3. Klimenko, Anton. 2018. "Microservices in Go." Seek Blog, via Medium, June 04. Accessed 2019-08-15.

Article Stats

Author-wise Stats for Article Edits

Author
No. of Edits
No. of Chats
DevCoins
2
0
1117
1
0
8
1236
Words
2
Likes
5060
Hits

Cite As

Devopedia. 2022. "Go kit." Version 3, February 15. Accessed 2023-11-13. https://devopedia.org/go-kit
Contributed by
2 authors


Last updated on
2022-02-15 11:54:03