Getting "Go"-ing

Over the last year, I started using Go for some work projects, a small bundle of microservices that process and transfer large datasets between our sites and our cloud platform. We have deployed those services and others to our local Kubernetes clusters in each site but also to our EKS cluster.

All of which I had never done before 😬. Sounds like a great set of topics to write about! So, welcome to my series on Go, microservices, and Kubernetes. A bit of fun writing about new topics and re-enforcing what I've just learned. I’m also going to be covering things like logging, metrics, tracing, CI/CD, building containers, etc. By the end of this series, my hope is to demonstrate how to build and design the most over-engineered, production-ready tutorial microservices.

An adventure

Go

Go follows the C style family with its style but removes the need to have semi-colons at the end of a line. Go also makes the formatting of the code easy, with go fmt which automatically stylise your code. This helps avoid some of the arguments about code style, that’s one less thing to worry about. Go was originally designed by Robert Griesemer, Rob Pike, and Ken Thompson. Pike and Thompson are veterans of Plan 9 from Bell Labs, Thompson also being the creator of B, the precursor to C.

Go unlike C is a [garbage collected](https://en.wikipedia.org/wiki/Garbage_collection_(computer_science) (GC) language. I'm a fan of C and C++, I have always enjoyed the challenge of memory management but at times it can nuisance, the contributors to Go have put serious effort into the performance of the Go's GC to make sure it is fast. The argument being we should all stop re-implementing memory management every time we write an application. For most applications, this is a solved problem already.

A Gophers first step

If you haven't used Go before (I hadn’t), A Tour of Go is a great place to start, it’s an interactive tutorial covering the basics of the language. It even has an online playground, allowing you to run Go code from the browser without needing to install anything. We will, however need to install Go, if you’re going to follow along with this series just follow the official instructions for your platform.

Hello World

Now for the obligatory "Hello World" example

package main

import “fmt”
    
func main() {
	fmt.Println(“Hello, World!”)
}

source

We can run this example locally in two ways, with go run and go build.

go run will compile and run our program but will not produce a binary for us to keep ☹️. go build will compile and produce a binary we can execute 👍.

$ go run
Hello, World!

$ go build helloworld.go
$ ./helloworld

Hello World Wide Web

Let's step things up a level, it's time for Hello World as a service (HWaaS). Go provides an easy to use HTTP package as part of its standard library. We just need to write a simple function which takes a httpResponseWriter and a *http.Request. For now, we don't care about the request, we just say want to respond with "Hello, world!" so we just write it to the httpResponseWriter. The net/http package provides http.HandleFunc which you allows us to register a handler function against a URL pattern. Registering our helloWorld function against the root (/) so that visiting localhost:8080/ we will return our message. / is just the top level of our site, we could achieve the same thing by using /hello so that visiting localhost:8080/hello would return Hello, World!.

package main

import (
	"fmt"
	"net/http"
)

func helloWorld(w http.ResponseWriter, r *http.Request) {
	fmt.Fprintf(w, "Hello, World")
}

func main() {
	http.HandleFunc("/", helloWorld)
	http.ListenAndServe(":8080", nil)
}

source

We pass nil to http.ListenAndServe as it uses the libraries default request router.

Basics String Service

Now for something a bit more "useful". Below is a service which handles strings. You can POST some JSON to "/uppercase" and will get a response back with the string back in uppercase. "/count" will return the number of characters in the string.

package main

import (
	"encoding/json"
	"errors"
	"fmt"
	"net/http"
	"strings"
)

var ErrEmptyString = errors.New("empty string")

type stringRequest struct {
	S string `json:"s"`
}

type countResponse struct {
	C int `json:"c"`
}

type uppercaseResponse struct {
	S string `json:"s"`
}

func count(s string) int {
	return len(s)
}

func uppercase(s string) string {
	return strings.ToUpper(s)
}

func countHandler(w http.ResponseWriter, r *http.Request) {
	var request stringRequest

	if err := json.NewDecoder(r.Body).Decode(&request); err != nil {
		fmt.Println(err)
	}
	if request.S == "" {
		json.NewEncoder(w).Encode(ErrEmptyString)
	}

	response := &countResponse{
		C: count(request.S),
	}
	json.NewEncoder(w).Encode(response)
}

func uppercaseHandler(w http.ResponseWriter, r *http.Request) {
	var request stringRequest

	if err := json.NewDecoder(r.Body).Decode(&request); err != nil {
		fmt.Println(err)
	}

	if request.S == "" {
		json.NewEncoder(w).Encode(ErrEmptyString)
	}

	response := &uppercaseResponse{
		S: uppercase(request.S),
	}
	json.NewEncoder(w).Encode(response)
}

func main() {
	http.HandleFunc("/count", countHandler)
	http.HandleFunc("/uppercase", uppercaseHandler)
	http.ListenAndServe(":8080", nil)
}

source

Go's net/http and encoding/json packages makes it very easy to create a simple JSON-RPC service. We define basic functions uppercase and count, both perform simple operations on a string and return a value.

Now that we have our core implementation done, we wrap the functions in handlers. countHandler and uppercaseHandler perform some boilerplate marshalling of data from an http.Request into a stringRequest struct we have defined. We also check to see if the string is empty, no point processing an easy to catch error. The results are stored in a countResponse or uppercaseResponse respectively, which are then marshalled into JSON by the http.ResponseWriter. This sends a response to our end-user.

Just like we did in the hello world example, we will assign the handlers to a particular path to handle those requests. We can test our new service with curl.

$ curl -d '{"s": "Hello, World!"}' http://localhost:8080/count
{"c":13}

$ curl -d '{"s": "Hello, World!"}' http://localhost:8080/uppercase
{"s": "HELLO, WORLD!"}

Go Kit

Go is a great general-purpose language, but microservices require a certain amount of specialized support. RPC safety, system observability, infrastructure integration, even program design — Go kit fills in the gaps left by the standard library, and makes Go a first-class language for writing microservices in any organization.

Go Kit is a microservices framework for Go, it is designed to help provide tooling to create reliable and observable microservices. It is a little verbose in some places but helps provide a structured approach to building a microservice. It is designed to be flexible and not enforce rigid patterns for a service.

For the Go Kit version of our service, we are going to build it up in layers. Go Kit’s model of a service separates code into three layers: service, endpoint, and transport. The core business logic, logging, application analytics, and service metrics live in the service layer. Load balancing, and circuit breaking lives in the endpoint layer, and transport handles the different communication protocols.

The following code is an almost direct copy from the Go Kit's examples code.

Service Layer - Business Logic

package main

import (
	"errors"
	"strings"
)

type StringService interface {
	Uppercase(string) (string, error)
	Count(string) int
}

// stringService is a concrete implementation of StringService
type stringService struct{}

func (stringService) Uppercase(s string) (string, error) {
	if s == "" {
		return "", ErrEmpty
	}
	return strings.ToUpper(s), nil
}

func (stringService) Count(s string) int {
	return len(s)
}

// ErrEmpty is returned when an input string is empty.
var ErrEmpty = errors.New("empty string")

source

This is our core implementation, very much what we wrote previously.

Endpoint Layer - Request & Response

Go kit is primarily based around RPC, this means we need to define a request and response struct for each endpoint. Each endpoint is equivalent to a single API method

// for each method, we define request and response structs
type uppercaserequest struct {
	s string `json:"s"`
}

type uppercaseresponse struct {
	v   string `json:"v"`
	err string `json:"err,omitempty"`
}

type countrequest struct {
	s string `json:"s"`
}

type countresponse struct {
	v int `json:"v"`
}

source

An endpoint receives a request and returns a response and each function in our service will have an endpoint, representing a single RPC method. So, we need to create an endpoint for Uppercase and for Count. An endpoint should contain code to ensure the safety and robustness of our service.

We need to add our new dependency endpoint.Endpoint which under the hood is just:

type Endpoint func(ctx context.Context, request interface{}) (response interface{}, err error)

The use of interface{} will require us to use type assertion but nothing to worry about there.

import (
	"context"
	"github.com/go-kit/kit/endpoint"
)

// Endpoints are a primary abstraction in go-kit. An endpoint represents a single RPC (method in our service interface)
func makeUppercaseEndpoint(svc StringService) endpoint.Endpoint {
	return func(_ context.Context, request interface{}) (interface{}, error) {
		req := request.(uppercaseRequest)
		v, err := svc.Uppercase(req.S)
		if err != nil {
			return uppercaseResponse{v, err.Error()}, nil
		}
		return uppercaseResponse{v, ""}, nil
	}
}

func makeCountEndpoint(svc StringService) endpoint.Endpoint {
	return func(_ context.Context, request interface{}) (interface{}, error) {
		req := request.(countRequest)
		v := svc.Count(req.S)
		return countResponse{v}, nil
	}
}

source

Transports - HTTP + JSON

Now that we have a service and endpoints, to interact with them we're going to need a protocol that other applications will understand. Go kit supports several common transport methods out of the box. We will use its httptransport to do the same as we did for our previous example and use HTTP and JSON.

import (
	"context"
	"encoding/json"
	"log"
	"net/http"
	httptransport "github.com/go-kit/kit/transport/http"
)

func main() {
	svc := stringService{}

	uppercaseHandler := httptransport.NewServer(
		makeUppercaseEndpoint(svc),
		decodeUppercaseRequest,
		encodeResponse,
	)

	countHandler := httptransport.NewServer(
		makeCountEndpoint(svc),
		decodeCountRequest,
		encodeResponse,
	)

	http.Handle("/uppercase", uppercaseHandler)
	http.Handle("/count", countHandler)
	log.Fatal(http.ListenAndServe(":8080", nil))
}

func decodeUppercaseRequest(_ context.Context, r *http.Request) (interface{}, error) {
	var request uppercaseRequest
	if err := json.NewDecoder(r.Body).Decode(&request); err != nil {
		return nil, err
	}
	return request, nil
}

func decodeCountRequest(_ context.Context, r *http.Request) (interface{}, error) {
	var request countRequest
	if err := json.NewDecoder(r.Body).Decode(&request); err != nil {
		return nil, err
	}
	return request, nil
}

func encodeResponse(_ context.Context, w http.ResponseWriter, response interface{}) error {
	return json.NewEncoder(w).Encode(response)
}

source

If we run the program, we can test it against curl.

$ curl -d '{"s": "Hello, World!"}' http://localhost:8080/count
{"v":13}

$ curl -d '{"s": "Hello, World!"}' http://localhost:8080/uppercase
{"v": "HELLO, WORLD!"}

We now have our HWaaS running in the Go Kit fashion!

Summary

That's all for this post, we had a quick introduction to Go and GoKit to write simple microservices. Next time we will start looking into testing for our services.

As always, I appreciate any feedback or if you want to reach out, I’m @neuralsandwich on twitter and most other places.

Getting Go-ing

Getting "Go"-ing

Go

A Gophers first step

Hello World

Hello World Wide Web

Basics String Service

Go Kit

Service Layer - Business Logic

Endpoint Layer - Request & Response

Transports - HTTP + JSON

Summary