One Repo to Rule Them All

I spent a good deal of time before/during/after writing the URL shortener itself on infrastructure, so it seems reasonable to discuss it here. Working with subtrees and orphaned branches was a bit annoying on my previous speedrun attempts, and since the final objective was to unite all the projects together anyway…

I decided to do it early.

First, I got my heroku account verified and my own domain name pointing to one of my projects there.

Then I set up a new git repo, and merged the previous two orphaned branches into it.

Then I reorganized my existing code into packages, a main package with that distributed routes to other packages, each of which would handle its own microservice. Here’s what my new main function looks like:

func main() {
	// get bound port of host system
	port := os.Getenv("PORT")

	http.HandleFunc("/", index)
	http.HandleFunc(timestamp.ROUTE, timestamp.Handler)
	http.HandleFunc(headers.ROUTE, headers.Handler)
	http.HandleFunc(tiny.ROUTE, tiny.Handler)
	log.Fatal(http.ListenAndServe(":"+port, nil))
}

Simple and sweet, all the logic internal to the services is contained in those services, including the routes they should be served on!

Persistence Pays

I knew that a critical question was data persistence, and I also knew that I wanted to do this with a minimum of fuss, as befits a “speed run,” so I decided not to skip using a database…

But since heroku pulls things down after inactivity on the free tier, I needed at least some kind of persistence, or the generated short addresses would likely only last as long as a dyno was active. So I decided to do the simplest possible thing, and just write to a file. This would almost certainly be a terrible idea in any kind of production server due to a variety of threading issues, but I determined that it was acceptable for what I viewed as a proof of concept.

For doing this write I used the following function:

func writeDB(sr ShortenerResponse) {
	f, err := os.OpenFile(DB, os.O_APPEND|os.O_WRONLY|os.O_CREATE, 0600)
	if err != nil {
		panic(err)
	}
	defer f.Close()

	data, _ := json.Marshal(sr)
	if _, err = f.Write(append(data, '\n')); err != nil {
		panic(err)
	}
}

In short, this opens a file in “append” mode, creating it if it doesn’t already exist, and writes a JSON object to the file, being sure to close this file before the function ends (the purpose of defer f.Close()).

Another function init, which in go runs only when the package is first loaded, reads from this same file (if it exists) and loads the contents into memory.

Map the World!

The objects themselves were built from a now familiar go struct with JSON tags:

type ShortenerResponse struct {
	Original *string `json:"original_url"`
	Short    *string `json:"short_url"`
}

This data structure is essentially only used when marshaling to JSON for the response to a client, or for unmarshalling the contents of my “database file”.

For actually doing the work of redirect on demand, I used a map, that takes strings and translates them directly into other strings. In short: a requested short URL redirects to the long URL without bothering to iterate over anything.

What Happens?

  1. a request is routed to the handler function
  2. the handler checks the map of URLs, and if a match is found, it just redirects to the long URL
  3. or it calls Shorten with the URL
  4. this function verifies that the URL is valid and sends an error response if it is not
  5. or it calls NewResponse which builds a response, adds an entry in the map, and returns the Response to Shorten
  6. which writes the new Response to the DB…
  7. and returns the JSON representation of it to the handler…
  8. which returns it to the user.

Room for Improvement

  • use a proper database, even just SQLite, instead of a flat file
  • use a better hashing function from long URLs to short URLs, and check for collisions
  • do something to better accomodate thread/process scaling, probably running a separate goroutine to issue writes to the DB/map, using channels