Blog Home

Blog /  Tech / Go – from Ideas to Production

Go – from Ideas to Production

May 14, 2020 11:56 am | by Jay | Posted in Tech

Go - from ideas to production

Go – from Ideas to Production

May 14, 2020 11:56 am | by Jay | Posted in Tech

I’ve been praising Go a LOT at work. It’s so damn much that my coworkers started calling me a professional Go preacher.

So I’m here again to talk about writing applications in Go that are ready for this cruel cruel world.

In this blog, I’m going to write a simple “quote service” that gives a random quote from a local source. I’ll be lazy and use lukePeavey/quotable‘s quote database. Supergeneral!

And like always, this project is available on my GitHub.

Starting up


Lets do the ritual to get a new Go module.

$ mkdir ~/quoteservice && cd ~/quoteservice
$ go mod init

I also downloaded the JSON file from lukePeavey/quotable and saved it in ./data/quotes/json. This JSON file is essentially a decent database of quotes. Finally, to get quotes out of this quotes list, I would want a service, so I’ll also create a file in a new quotes directory. We’re going to end up with a directory structure like this:

~/quoteservice
├── data
│   └── quotes.json
├── go.mod
├── go.sum
├── main.go
└── quotes
    └── quotes.go

Defining a service

We can define a single-method interface that gives out a random quote like this:

type Quote struct {
    // ...
}

type Service interface {
    Random() Quote
}

This lets us create an implementation that reads the JSON file and use it as a source…

type quouteSource struct {
    quotes []Quote
}

func NewService() (Service, error) {
	b, err := ioutil.ReadFile("data/quotes.json")
	if err != nil {
		return nil, err
	}

	var q []Quote
	if err := json.Unmarshal(b, &q); err != nil {
		return nil, err
	}

	return quoteSource{q}, nil
}

Sweet!

Spinning up an HTTP Server

It’s pretty straightforward.

func main() {
	qs, err := quotes.NewService()
	if err != nil {
		panic(err)
	}

	log.Fatal(http.ListenAndServe(":8080", createHttpHandler(qs)))
}

func createHttpHandler(qs quotes.Service) http.HandlerFunc {
	return func(w http.ResponseWriter, r *http.Request) {
		w.Header().Add("Content-Type", "application/json")

		q, err := json.Marshal(qs.Random())
		if err != nil {
			w.WriteHeader(http.StatusInternalServerError)
			return
		}

		w.WriteHeader(http.StatusOK)
		w.Write([]byte(q))
	}
}

And to verify that it works, we can run go run . and hit http//localhost:8080 to get a random quote. Now we can build a binary using go build and run it on a machine and it will work… until it doesn’t.


Getting ready for the real world

Before we get giddy and put this service out, there are some things that should be considered.

    • Instrumentation – Will provide important insights on requests and other important stuff.
    • Rate limiting – To save some compute resources.
  • Containerizing – To isolate process and save the host from several vulnerabilities

Instrumentation

From WikiPedia – In the context of computer programming, instrumentation refers to the measure of a product’s performance, to diagnose errors, and to write trace information. Considering our scale, we can write a good logging implementation and that can be sufficient. I’m also going to implement a middleware mechanism to get more control over the service.

Logging

For smaller applications like this, Go’s log is usually sufficient. op/go-logging and google/logger are some examples of logging libraries that work really well. For this application though, I’m going to use op/go-logging for it’s configurable nature. This will let me define a logger like this that I can use instead of panicking or log.Fataling.

import (
    log "github.com/op/go-logging"
)

var (
    logger = log.MustGetLogger("quoteservice")
)

This setup will let us do level-based logging that I can control based on what environment our application is running in. This enables us to log information like this:

func main() {
	// ...

	logger.Info("Starting server on port 8080...")
	logger.Fatal(http.ListenAndServe(":8080", createHttpHandler(qs)))
}

// Other implementations using logger.Debugf, logger.Warnf, and so on...

Middleware

Middleware receives a http.Handler and returns an http.Handler. The returned http.Handler is a closure that can execute some code before and after a request is served. The closure must call ServeHTTP(...) on received http.Handler to continue execution of the handler chain. Which means we can define the middleware chain like this:

type Middleware func(http.Handler) http.Handler

func combineHandlers(root http.Handler, mwf ...Middleware) (handler http.Handler) {
	handler = root
	for _, m := range mwf {
		handler = m(handler)
	}
	return
}

Which let’s us define a middlware that can log execution time of requests being served.

func loggingMiddleware(next http.Handler) http.Handler {
	return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
		start := time.Now()
		next.ServeHTTP(w, r)
		logger.Infof("Request completed in %d ms", time.Since(start).Microseconds())
	})
}

With a little more hacking, this function can also be made aware HTTP status codes and responses! Having a middleware mechanism in place will also be helpful in implementing…

Rate Limiting

Increased web traffic can exhaust a server’s resources when a request needs to perform tasks that involves heavy computation. To get around this problem, developers usually add a rate-limiter to their servers. As the name suggests, a rate limiter is used to control the rate of requests sent or received by a server. It works wonders in preventing DoS attacks.

Go’s x/time/rate package implements rate limiting by a “token bucket” that is refilled at rate r tokens per second. We can implement a Middleware on top of this that limits the server to serve only 200 requests per second.

func rateLimiter(next http.Handler) http.Handler {

	// Limit requests at 200 per second
	limiter := rate.NewLimiter(1, 200)

	return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
		if !limiter.Allow() {
			w.WriteHeader(http.StatusTooManyRequests)
			return
		}

		next.ServeHTTP(w, r)
	})
}

Pretty simple!

Putting it all together…

The main method can be defined like this:

func main() {
	root := createHttpHandler(qs)
	handler := combineHandlers(root, rateLimiter, loggingMiddleware)

	logger.Fatal(http.ListenAndServe(":8080", handler))
}

Containerizing it

This step is pretty trivial. We’re going to make a multi-stage docker image. You can pretty much copy down this in your Dockerfile and expect it to work.

# Build
FROM golang:1.14-alpine3.11 AS build

WORKDIR /quoteservice

COPY . .

RUN go build -o ./app .

# Deployment
FROM alpine:3.11
EXPOSE 8080

WORKDIR /app
RUN mkdir /app/data

COPY --from=build /quoteservice/app ./
COPY ./data/quotes.json ./data/quotes.json

ENTRYPOINT [ "./app" ]

For a quick test, run docker build -t quotes . && docker run --rm -it quotes and hit http://localhost:8080.

Aaaaannnnndddd we’re done!


A word… or two

Though it seems really easy to build applications for production, you might want to consider these points…

  • Writing tests : You want to make sure that you’re not running into it-runs-on-my-machine™ issues. This also makes sure that your application is not running into some weird regressions. For this particular application, it makes sense to write a unit test for rateLimiter.
  • Better instrumentation : You might want to use something like Prometheus to monitor your applications.
  • Exposing the server : Generally it is a decent idea to run the server behind an Nginx reverse proxy since it helps with load balancing and SSL configuration. However, I really recommend checking Go’s standard library. Seriously. It’s a goldmine of great stuff!

As usual, you can find the complete code for this blog on my GitHub. You can use the issues board to contact me. Feel free to raise a PR if something seems out of place 😄

Thanks for reading, here’s a gopher.

Gopher- Golang

Written by Jay

Application Developer

Jay is a Software Engineer and Architect at Sarvika Technologies, who fell in love with coding while developing mods for online games. He believes that an individual is defined by mindset and not by degrees. The software quality is of prime importance to Jay, an approach that helps him look at the bigger picture and build sustainable & sophisticated software. Beware of him, cuz he’s code-blooded!

Related Post

How system upgrades contribute to business growth.

For companies that are into IT services, hardware, and software upgrades play a crucial role in improving the quality of deliveries.

by Aviral Chitkara

May 21, 2020

Capitalizing-the-post-pandemic-economy-in-the-IT-Industry-copy

In this time of the pandemc it is very essential to plan for capitalizing the post-pandemic economy which will be scaling in the IT Industry.

by Aviral Chitkara

April 30, 2020

Business Analysis A roller-coaster ride to having happy clients blog featured image

Business Analysis is a roller-coaster ride to having happy clients. Check out the various KPIs/responsibilities of a BA to-achieve time-line.

by Harish Ramchandani

April 17, 2020

Subscribe to the Newsletter

Get our latest news, tutorials, guides, tips & deals delivered to your inbox.