Working Offline with Github Pages

Update - 8 May 2017

Whilst the below is a novel solution to this problem, I have since found a far simpler solution (discussed at the end of the article). There are merits for each, and it provides for interesting discussion. Have a read and judge for yourself?

The Problem

When you run a (Jekyll) GitHub pages site locally, how do you ensure that you are using the same gems (plugins) and versions as will be used on GitHub?

Further - what happens if you don’t have internet access, and can’t check what versions Github is currently using?

The [edit: A Potential] Solution

Part A: Using GitHub’s versions locally

Solving the first part of this problem was relatively straightforward;

#Gemfile
source 'https://rubygems.org'
require 'json'

#Check what plugin versions GitHub is using:
github_versions_url = 'https://pages.github.com/versions.json'
versions = JSON.parse(open(github_versions_url).read)
 
#Load the same plugin versions locally:
gem 'github-pages', versions['github-pages']

The above (when internet access is available) shall check to see what plugin versions GitHub is using.

Perfect.

This means that when I run my Jekyll site locally, it will behave exactly in the same way as when Github builds my site.

Except - what happens when I take my dev environment on the road, and I can’t connect to the internet?

Part B: Dealing with internet connectivity issues

Checking for internet connectivity

Solving this part, first happened after I read fotanus’ post on Stackoverflow.

This caused me to include the following function within my Gemfile:

#Function to check if can access a url (i.e. github)
require 'open-uri'
require 'active_support'
require 'active_support/core_ext'

def url_exist?(url_string)
  url = URI.parse(url_string)
  req = Net::HTTP.new(url.host, url.port)
  req.use_ssl = (url.scheme == 'https')
  path = url.path if url.path.present?
  res = req.request_head(path || '/')
  if res.kind_of?(Net::HTTPRedirection)
    url_exist?(res['location']) # Go after any redirect and make sure you can access the redirected URL 
  else
    ! %W(4 5).include?(res.code[0]) # Not from 4xx or 5xx families
  end
rescue
  false #false if can't find the server
end

Reverting to local cache when connectivity is unavailable

However I still needed to write some logic to:

“Cache” the versions page (when internet connectivity exists)
Fall back to this cache when internet connectivity does not exist

In order to “cache” this information, I simply store a ‘versions.json’ in the project root. The file gets updated whenever GitHub’s version becomes changed.

That code looks like this:

github_versions_url = 'https://pages.github.com/versions.json'
local_versions_path  = 'versions.json'

if url_exist?(github_versions_url)
    puts "***************************************"
    puts "ACCESSING GITHUB PAGES PACKAGE VERSIONS"
    puts "***************************************"
    download = open(github_versions_url)
    IO.copy_stream(download, local_versions_path)
    versions = JSON.parse(open(github_versions_url).read)
else
    # If offline will need to use cached local version (below), instead.
    puts "************"
    puts "OFFLINE MODE"
    puts "************"
    versions = JSON.parse(open(local_versions_path).read)
end

gem 'github-pages', versions['github-pages']

Note that I also included some printed output, so that when I run jekyll serve I am advised whether internet connectivity was able to be established.

Finished Gemfile

The completed Gemfile:

#Gemfile
source 'https://rubygems.org'
require 'json'
require 'open-uri'
require 'active_support'
require 'active_support/core_ext'

#Function to check if can access a url (i.e. github)
require "net/http"
def url_exist?(url_string)
  url = URI.parse(url_string)
  req = Net::HTTP.new(url.host, url.port)
  req.use_ssl = (url.scheme == 'https')
  path = url.path if url.path.present?
  res = req.request_head(path || '/')
  if res.kind_of?(Net::HTTPRedirection)
    url_exist?(res['location']) # Go after any redirect and make sure you can access the redirected URL 
  else
    ! %W(4 5).include?(res.code[0]) # Not from 4xx or 5xx families
  end
rescue
  false #false if can't find the server
end

github_versions_url = 'https://pages.github.com/versions.json'
local_versions_path = 'versions.json'

if url_exist?(github_versions_url)
    puts "***************************************"
    puts "ACCESSING GITHUB PAGES PACKAGE VERSIONS"
    puts "***************************************"
    download = open(github_versions_url)
    IO.copy_stream(download, local_versions_path)
    versions = JSON.parse(open(github_versions_url).read)
else
    # If offline will need to use cached local version (below), instead.
    puts "************"
    puts "OFFLINE MODE"
    puts "************"
    versions = JSON.parse(open(local_versions_path).read)
end

# Load github pages gem
gem 'github-pages', versions['github-pages']

# Plugins for local livereload.
gem 'guard'
gem 'guard-jekyll-plus'
gem 'guard-livereload'

An even simpler solution?

Update - 8 May 2017

Per earlier comment, I came across this simpler solution after posting the original article.

It’s actually possible to completely ignore this entire post, and instead make sure that you are running Jekyll via: bundle exec jekyll serve instead of jekyll serve.

But, you also need to make sure that you are:

running bundle update at regular intervals, and
checking Gemfile.lock into source control.

A detailed explanation of this, can be found here: bundler.io/rationale.

Which solution should you use?

Interestingly, the two solutions work in a very similar same way, ie;

Both solutions write a file to the project directory (versions.json / Gemfile.lock), listing out the gem versions that were utilised within the previous successful build.
The files even look loosely the same:

Gemfile.lock versus Versions.json — **Gemfile.lock** versus **Versions.json**

As I alluded to at the start of the article; the Bundler solution is somewhat simpler and clearly more widely recognised. It also tracks changes to your local environment’s other Jekyll dependencies. For these reasons it is currently the solution I would recommend.

A further argument is that the development tenant “DRY”, can be extended to encompass “Don’t Repeat Your Framework”. Clearly the custom solution above breaches this.

That said; Bundler does require you to run bundle update (and you must remember to do this regularly), whereas my custom (read as: “proceed with caution”) solution doesn’t require you to do this. For this reason, I’m leaving this post here as food for thought.

Comments (constructive) are welcome .