Goodbye Travis, hello GitHub Actions

My very cyclical open-source work

For some reason, I only manage to do coding at home every some months - mostly 6 months apart, so twice a year (even worse than my blogging frequency :P). As such, I missed the whole discussion about travis-ci (the .org version) going away, etc.

So when I finally did some work on corydalis a few weeks ago, and had a travis build failure (restoring a large cache simply timed out, either travis or S3 has some hiccup - it cleared by itself a day later), I opened the travis-ci interface to see the scary banner (“travis-ci.org is shutting down”), and asked myself what’s happening. The deadline was in less than a week even 😧…

Long story short, Travis was a good home for many years, but they were bought and are doing significant changes to their OSS support, so it’s time to move on. I anyway wanted to learn GitHub Actions for a while (ahem, free time intervened), so this was a good (forced) “opportunity”.

Proper composable CI

The advantage of Travis’ infrastructure was that the build configuration was really simple. It had very few primitives: pre-steps (before_install), install steps, and the actual things to do to test, post-steps (after_sucess) and a few other small helpers (caching, apt packages, etc.) This made it really easy to just pick up and write a config, plus it had the advantage of allowing to test configs from the web UI without needing to push.

This simplicity was unfortunately also its significant limiter: the way to do complex things in steps was simply to add more shell commands.

GitHub actions, together with its marketplace, changes this entirely. There are no “built-in” actions, the language just defines the build/job/step hierarchy, and one glues together whatever steps they want. This has the disadvantage that even checking out the code needs to be explicitly written in all workflows (so boilerplate, if you don’t need customisation), but it opens up a huge opportunity for composition, since it allows people to publish actions (steps) that you just import, encapsulating all the work.

So, after learning how to write a moderately complicated workflow (complicated as in multiple Python version, some of them needing different OS version, and multi-OS), it was straightforward to port this to all my projects - just somewhat tedious. I’ve now shutdown all builds on Travis, just can’t find a way to delete my account 😅

Better multi-OS, worse (missing) multi-arch

In theory, Travis supports Linux, MacOS, FreeBSD and Windows, but I’ve found that support for non-Linux is not quite as good. Maybe I missed things, but multi-version Python builds on MacOS were not as nicely supported as Linux; Windows is quite early, and very limited; and I haven’t tested FreeBSD.

GitHub is more restrictive - Linux, MacOS and Windows - but I found support for MacOS and Windows better for my use cases. If your use case is testing multiple MacOS versions, Travis wins, if it’s more varied languages/etc. on the single available MacOS version, GitHub works better.

On the multi-arch side, Travis wins hands-down. Four different native architectures, and enabling one more is just adding another key to the arch list. With GitHub, if I understand right, you either have to use docker+emulation, or use self-hosted runners.

So here it really matters what is more important to you. Maybe in the future GitHub will support more arches, but right now, Travis wins for this use-case.

Summary

For my specific use-case, GitHub Actions is a better fit right now. The marketplace has advantages (I’ll explain better in a future post), the actions are a very nice way to encapsulate functionality, and it’s still available/free (up to a limit) for open source projects. I don’t know what the future of Travis for OSS will be, but all I heard so far is very concerning.

However, I’ll still miss a few things.

For example, an overall dashboard for all my projects, like this one:

I couldn’t find any such thing on GitHub, so I just use my set of badges.

Then cache management. Travis allows you to clear the cache, and it does auto-update the cache. GitHub caches are immutable once built, so you have to:

• watch if changed dependencies/dependency chains result in things that are no longer cached;
• if so, need to manually bump the cache key, resulting in a commit for purely administrative purposes.

For languages where you have a clean full chain of dependencies recorded (e.g. node’s package-lock.json, stack’s stack.yaml.lock), this is trivial to achieve, but gets complicated if you add OS dependencies, languages which don’t record all this, etc.

Hmm, maybe I should just embed the year/month in the cache name - cheating, but automated cheating.

With all said and done, I think GHA is much less refined, but with more potential. Plus, the pace of innovation on Travis side was quite slow (likely money problems, hence them being bought, etc.)…

So one TODO done: “learn GitHub Actions”, even if a bit unplanned 😂