GoLang Dep: The Missing Manual

As our engineering team has grown, and as the number of different applications have grown, it’s helped us stay sane when dealing with changes in external dependencies.

During the transition from a custom vendor management solution to dep, I found some of the important documentation to be scattered around. This post collects the most helpful “extra” information I wish I had known getting started.

What’s dep?

depdep is a dependency management tool for the Go programming language. We elected to use dep because of its close relationship with the official Go toolchain developers and its straightforward, “unmagical” dependency management model.

It has an active and helpful development community over on gophers.slack.com#vendorgophers.slack.com#vendor and an exciting roadmapexciting roadmap. If you start using dep you should join up and get to know the community. It’s a great time to help find edge cases and make this tool better.

Set up dep

Follow the steps in the dep READMEdep README to install dep. The homebrew release is up-to-date and is the recommended way to install dep.

Where to run dep

dep should be run at the project root—the directory just above where your vendor directory sits. depassumes that any packages that cannot be reached in your GOPATH by navigating down from where it’s run are external packages that need to be added to the vendor directory.

To get started, run dep init. This will read your application code and generate a set of constraints based on its best solution for your dependencies. After running this, you may need to edit it to be more specific about constraints, especially if you depend on older versions of some projects.

Gopkg.toml

Read this: Gopkg.toml READMEGopkg.toml README

The Gopkg.toml file describes all the dependencies required for your project. It only describes primary dependencies, not transitive dependencies, leaving the dep constraint solver free to pick transitive dependency versions as long as a version can be chosen that satisfies all constraints.

The most important fields in the file are the constraint entries. Constraints look like this:

[[constraint]]
  name = "github.com/lightstep/lightstep-tracer-go"
  version = "v0.14.0"

Every constraint needs a name, which is actually the URI you would use when go geting the project. Every constraint should also have either a version, branch, or revision, with version being preferred if you can use it.

version fields use SemVer 2.0 [http://semver.org/http://semver.org/ ] and dep assumes that v1.2.0 means ^v1.2.0. If you need to constrain to an exact version, use =v1.2.0.

Never edit the lock file!

Gopkg.lock is really an output of the constraint solver. Editing it does nothing.

dep ensure -v is your friend

It’s possible to describe a set of constraints that cannot be solved—you may have a primary dependency on a version of a package, and one of your primary dependencies may depend on an incompatible version. In that case, dep ensure will fail and print an error message. By running dep ensure -v you will get detailed output from the constraint solver that can help you identify the source of the problem.

dep ensure doesn’t do much if Your constraints are already satisfied!

dep ensure will “ensure” that a package that you import is also installed in your vendor directory and satisfies any described constraints… That’s it! It won’t make sure you have the latest release if your constraints are already satisfied.

Use dep ensure -update pgkname to get the latest version that satisfies constraints.

Gopkg.toml trick: required

Sometimes you need some go code included that your application doesn’t run directly. For example, https://github.com/grpc-ecosystem/grpc-gatewayhttps://github.com/grpc-ecosystem/grpc-gateway generates code which can then be committed to a repository, but a dep ensure will not install it, and a dep prune would remove it if installed.

The required keyword lets you depend on repositories that are not dependencies of your application like so:

required = ["github.com/grpc-ecosystem/grpc-gateway"]

Note that after ensuring it’s installed, you still need to go install your requirements:

$ dep ensure
$ go install vendor/github.com/grpc-ecosystem/grpc-gateway/...

There are other tools that can be used to make these installations project-specific as well, like virtualgo: https://github.com/GetStream/vghttps://github.com/GetStream/vg

Gopkg.toml trick: ignored

The ignored keyword prevents a package and its transitive dependencies from being installed in a project. Why would you want to do that? A typical use case might be to support updating to a new major version of a library that removes a package. Let’s say that you depend on github.org/foo/bar/bazsomewhere in your application, and version 2.0.0 of foo/bar drops this package.

Original Gopkg.toml:

[[constraint]]
  name = "github.com/foo/bar"
  version = "1.2.1"

Updating to 2.0.0 without changing your source code:

ignored = ["github.com/foo/bar/baz"]

[[constraint]]
  name = "github.com/foo/bar"
  version = "2.0.0"

This can help you install the new version of your library, even though it doesn’t have all the packages required by your application. You can then work to transition your application code while having the source for the library version you’re working with installed locally.

Committing vendor

If you’re writing a library, especially an open-source library, it’s not generally a good idea to commit your vendor directory. Any users who go get your code may have an impossible time compiling if they have conflicting dependencies with you. It’s a great idea, however, to commit and share your Gopkg.toml file. This will help other users of dep easily consume your library.

If you’re writing an applicaiton that emits binaries, there are some arguments for committing your vendor directory as an application that builds a binary and some arguments against it.

In favor of committing vendor

• You have all the source and binaries needed to build your application in your repository. This can speed CI builds by avoiding a lot of downloading and dependency resolution when building. It also gives you a repeatable set of source to build from.

• You’re protected from upstream changes breaking your builds—if a dependency unpublishes a previous release, you’ll still have a copy.

Against committing vendor

• You may be storing and handling a lot of code that isn’t directly related to your application in your own repository.

• Changes that include dependency upgrades will have very large diffs and may be unwieldy.

We’ve chosen, for the time being, to commit our vendor directories for our application binaries.

Making your project dep friendly

This section assumes you’re hosting your project as a git repository. Similar rules hold for bzr and hg as well.

Use annotated git tags to mark releases. git tag -a v2.1.3 -m "Release Version 2.1.3"Including release notes is a great idea. Leave off the -m argument to open your editor and add a longer tag message.

Mark them with 3-digit SemVer tags: v2.1.3.

Be honest about what a “breaking change” is

Any change that would cause a build failure if it is installed is considered a “breaking change” by SemVer and should be released with a major revision update. Changes that add new APIs can be considered minor releases, and changes that do not modify the API of the project can be considered a “patch” version.

We’ve found dep to be a straightforward, understandable tool for managing application dependencies. It fits nicely into our CI and development workflows, and it’s easy to understand which version of a given dependency is currently being used in a build. Let us know in the comments about your experiences with dep and share any information you’ve found helpful when using the tool!

Interested in joining our team? See our open positions herehere.