* Captain Git Hook A configurable installation of Git hooks and tools for use in software development teams. [[https://travis-ci.com/jemstep/captain-git-hook][https://travis-ci.com/jemstep/captain-git-hook.svg?branch=master]] ** Build and Install from Source *** Prerequisites Captain Git Hook is written in the Rust programming language. You can download the Rust toolchain from the [[https://www.rust-lang.org/tools/install][Rust website]]. This project targets the latest stable release of the Rust toolchain. *** Test The project can be compiled and the unit tests run using Cargo. As you're adding new functionality, it's recommended to also add unit tests for that functionality. #+BEGIN_SRC shell cargo test #+END_SRC *** Build and Run Captain Git Hook can be built and run directly from Cargo. One important thing to know is that the ~--~ on the command line separates the arguments being passed to Cargo from the arguments being passed to Captain Git Hook. #+BEGIN_SRC shell cargo run -- #+END_SRC This is useful for limited testing, but many of the commands of the CLI expect to be called by Git with certain inputs, so it's recommended to rather install binary to your path for local use. *** Build and Install Most usages of the project assume that you have the project built and installed on your path. This can be done through Cargo, by running the install command. #+BEGIN_SRC shell cargo install --path . #+END_SRC This will add an executable to your path called ~capn~. #+BEGIN_SRC shell capn --version #+END_SRC ** Usage *** Runtime Dependencies Captain Git Hook requires certain command line applications to be installed and on the path. - git - This is used as a binary on the CLI only for cases unsupported by libgit2, such as verifying signatures. - gpg - This is used for verifying signatures. - dirmngr - This is a gpg component that gpg uses as part of fetching gpg keys from a keyserver. On some distros, this is bundled together with gpg. *** Git Hooks Captain Git Hook works by installing hooks in your Git repository. Git will then call out to Captain Git Hook at various points in its lifecycle. **** On a local development machine To get started, go to your Git repo and run the ~install-hooks~ command. #+BEGIN_SRC shell capn install-hooks #+END_SRC **** On a Git Server Some of policies run on the 'pre-receive' hook, as commits are being recieved by a Git server. How to install this will depend on how you administrate your Git server. For example, these are the instructions for GitHub Enterprise: [[https://help.github.com/en/enterprise/2.19/admin/developer-workflow/managing-pre-receive-hooks-on-the-github-enterprise-server-appliance][Managing pre-receive hooks on the GitHub Enterprise Server appliance]]. Some Git servers, like GitHub Enterprise, require specifying a sandbox environment for the pre-receive hook to run in. For convenience, we include a Dockerfile and script for setting up a GitHub Enterprise sandbox. [[./github/readme.org]] *** Policy Configuration The policies that Captain Git Hook will apply for a repo are controlled by a ~.capn~ configuration file in the root of the project's repo. This configuration file is in TOML format. This is an example ~.capn~ file: [[./.capn]] *** Policies **** Git configuration There are some properties that are common across policies. Configuration of how certain Git conventions are followed are grouped into the ~[git]~ section, and may affect multiple policies. #+BEGIN_SRC toml [git] # The set of branches and patterns that are considered the 'mainline' by other policies. # Supports globs and the special symbolic reference "HEAD". # Default is [ "HEAD" ] mainlines = [ "HEAD", "develop", "RC-*" ] #+END_SRC **** Verify Git Commits This policy ensures that all commits come from a trusted source, using GPG keys. For this policy to work, Captain Git Hook must be installed as a ~pre-receive~ hook on your Git server. GPG public keys are fetched from a keyserver. They are checked against a "Team Fingerprints" file that must be checked into the repo. The Team Fingerprints file is a CSV file with the format GPG Fingerprint, Name, Email. For example: #+BEGIN_SRC csv 3FFD FF12 60CC D40C B14F 67E2 9C1E 6C5B 630C 6EE1,Justin Wernick,justin@jemstep.com #+END_SRC This is the config section for this policy: #+BEGIN_SRC toml [verify_git_commits] verify_email_addresses = true # if true, ensure that committers and authors have the specified domain author_domain = "yourdomain.com" # required domain for author email addresses committer_domain = "yourdomain.com" # required domain for committer email addresses verify_commit_signatures = true # if true, ensure that all code changes have a GPG signature keyserver = "hkp://your.preferred.keyserver" # url to the keyserver to fetch public keys from team_fingerprints_file = "gpg/TEAM_FINGERPRINTS" # path to the fingerprints file recv_keys_par = true # run key requests to keyserver in parallel skip_recv_keys = false # if true, do not fetch keys from the keyserver verify_different_authors = true # if true, merge commits to the mainline branch of the repo should have multiple authors in the branch override_tag_pattern = "capn-override-*" # glob used to limit tags that are considered override tags (see Override Tags docs) override_tags_required = 2 # the number of tags required to override signed commit rules #+END_SRC ***** Override Tags Sometimes, you need to override the verification checks for a range of commits. For example, it may be necessary to mark a starting 'good' point when first introducing the policy. This can be done by adding signed tags to the commit. The signatures must belong to people in the Team Fingerprints file, and the number of signed tags required is determined by the config. If there are enough signed tags on a commit, then all ancestors of that commit will not be checked. Signed tags are created in Git using this command: #+BEGIN_SRC shell git tag --sign git push #+END_SRC *** Monitoring By default, logging output is produced to the terminal, following the convention of output to stdout, diagnostics to stderr. Additional diagnostics can be produced to stderr by specifying =-v= or =--verbose= on the command line. For example, =capn -v pre-receive= will produce debug level logging, which =capn pre-receive= will only produce info level logging. Diagnostic logging over TCP is also supported with the =--log-url= command line parameter. Network logs are sent in JSON format. In the case of network logging, it's usually useful to provide some data to contextualise the log. A server side hook using all of the context parameters would look like this: #+BEGIN_SRC rust capn-qa -vv --log-url 10.0.0.123:123 --repo "$GITHUB_REPO_NAME" --user "$GITHUB_USER_LOGIN" --ip "$GITHUB_USER_IP" pre-receive #+END_SRC * Development ** High level architecture On a high level, Captain Git Hook is a collection of Git Hooks, linked to Policies, controlled by Configuration. The Git Hook is the event that runs the application. Which hook is run will depend on what is currently going on in Git. Each hook has a hardcoded list of all the policies that make sense for the hook. For example, it wouldn't ever make sense to verify GPG signatures on commits in a pre-commit hook (there is no commit to check yet), and it wouldn't make sense to create a commit messge template in a pre-receive hook. Each policy gets the config from the .capn file. If the .capn file doesn't have the configuration for the policy, then that policy is disabled and it does not get applied. A hook has been successfully run if all of the policies that were turned on in the configuration passed successfully. #+BEGIN_SRC dot :file architecture.svg :exports results digraph { hooks [ shape="record" label="{Hooks|{pre-receive|pre-push|prepare-commit-msg}}" ] policies [ shape="record" label="{Policies|{verify git commits|verify different authors|prepend branch name}}" ] config [ shape="rect", label="Configuration" ] hooks -> policies [label=" triggers"] policies -> config [label=" controlled by"] } #+END_SRC #+RESULTS: [[file:architecture.svg]] ** Integration tests Some of the end to end tests require a valid Git repository and GPG key to run. To facilitate this, there is a bare Git repo, set up as a test repository, checked in to the tests folder of this repo. It is located at [[./tests/test-repo.git]]. *** GPG keys and the test repo To create valid commits for these tests, you need to sign the commits with the secret key in [[./tests/test-secret-key.asc]]. The password to import this key is 'test'. You can import the key into your GPG keyring with the following command: #+BEGIN_SRC sh # This command will prompt you for the key's password. The password is 'test'. gpg --import ./tests/test-secret-key.asc #+END_SRC *** Cloning the test repo to make changes To add extra testing scenarios, you'll probably need to add additional commits to this bare repo. It's recommended to, as well as cloning the test repo, also set your user inside the test repo to a test user. The test user uses the test GPG key that you imported above. #+BEGIN_SRC sh # run this from somewhere outside of the Capn directory git clone cd test-repo git config user.email "blackhole@jemstep.com" git config user.name "Test User" git config user.signingkey "0xE1F315E39CCCECAA" #+END_SRC Make any required commits, and push the changes back. Then commit the changes in this repo. *** Visualising the test repo The easiest way to visualise the data in the test repo is to use =git log=. #+BEGIN_SRC sh cd git log --graph --decorate --oneline --all #+END_SRC * License This software may be used under the conditions of the Apache License. Copyright 2019 Jemstep Incorporated Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.