Github pages, Kramdown, Rouge, Jekyll 3.0: the final struggle

What this is about…

Github has recently announced Github pages was moving to Jekyll 3.0. Beyond better performances, this change also implies renuncing to Redcarpet and Pygments. From now, only Kramdown will be supported as the markdown engine. And Rouge will be the only syntax highlighter to work.

I have started to do the migration.
Unlike what Github indicated, this is far from being that easy. The most difficult part is about updating your local Jekyll installation to verify it works as expected. To be clear, there are still bugs in the last github-pages gem (in its dependencies). However, this article may help you to be as close as possible waiting for the fixes.

Reset your Ruby installation

First, Jekyll 3.0 requires Ruby 2.0 or higher.
If you are on Ubuntu, you will find Debian packages and instructions on this page.

Then, you may suffer from bad interactions with your previous Jekyll install. If you want to start from a brand new point, uninstall all your gems. This can be done in a single command line.

sudo gem list | cut -d” ” -f1 | sudo xargs gem uninstall -aIx

So far, so good.
Now, the best thing to do is to follow Github instructions and let the github-pages gem download the required dependencies. Verify your have a Gemfile file in your site sources with the following content.

source ‘https://rubygems.org’
gem ‘github-pages’

You may have other gems as well.
Make sure you DO NOT HAVE a Gemfile.lock file in the same directory. If so, delete it. Otherwise, you will keep on downloading the same old gem versions.

Eventually, run sudo bundle install.
This will download github-pages with all the right gems.

Once this is done, update your _config.yml file. Mine used to start with…

encoding: utf-8
markdown: redcarpet
highlighter: pygments
redcarpet:
  extensions: ["tables", "no_intra_emphasis"]

gems: [ "jemoji" ]

Now, it looks like…

encoding: utf-8
markdown: kramdown
highlighter: rouge
kramdown:
  input: GFM
  hard_wrap: false
  syntax_highlighter: rouge

gems: [ "jemoji" ]

And that should do the trick.
OK, on February 12th, it does not compeletly. First, there is a bug with Jekyll 3.1.1. This is why it is better to use github-pages and not installing directly the Jekyll gem. And then, there is also a bug with with the current github-pages. When using fency code blocks…

```properties
# Here in version %v_SNAP%
key = value
```

… the generated HTML is different from what pygments used to generate. Instead of generating…

<div class="highlight"><pre><code class="language-properties" data-lang="properties"><span class="c"># Here in version 0.5
</span><span class="py">key</span><span class="p"> = </span><span class="s">value</span>
</code></pre></div>
</blockquote>

… it generates…

<div class="highlighter-rouge"><pre><code><span class="c"># Here in version 0.5
</span><span class="py">key</span><span class="p"> = </span><span class="s">value</span>
</code></pre></div>
</blockquote>

The language-properties CSS class disappears.
According to this topic, this is fixed in Jekyll 3.1.2. So, let’s wait for it!

Edit from March 4th: in fact, it is a current limitation of Rouge. See jneen/rouge#379 for more details.

Injection Error with Angular JS

Just a quick hint with Angular JS.

I have just got an error like…

Error: [$injector:modulerr] http://errors.angularjs.org&#8230;

I was using the minified Angular script, and the error was hard to understand.
To get a human-readable error, just replace the minified script by the full one. That is to say, replace angular.min.js by angular.js in your HTML page. The error message will be easier to understand. During your development, this will save you a lot of time.

Listing files on a Mediafire account

Here is the story.
I have a free account on Mediafire, where I host some files to be shared. This site provides a nice user experience. This is true for those who download, but also for those whose share files with it (the file management is really good).

The fact is that I would like to be able to list my files from an external application. It is just a way to retrieve my links to share them. Then, people will go on Mediafire to download the files. Mediafire provides an API, but it does not support this feature.

After a quite long search, I finally found a way to list my files.
It was quite hard because authentication is required and because files are not listed in the HTML source but entirely added in the page with Javascript. So, here is the solution (implemented it in Java). For this, I used HTML Unit, but it should also work with Cobra.

/**
 * List file links associated with a given Mediafire account.
 * @author Vincent Zurczak
 */
public class MediafireTest {

	/**
	 * @param args
	 */
	public static void main(String[] args) {

		// Create a web client
		final WebClient webClient = new WebClient( BrowserVersion.FIREFOX_3_6 );
		webClient.setThrowExceptionOnScriptError( false );
		webClient.setCssEnabled( false );
		webClient.setIncorrectnessListener( new IncorrectnessListener() {
			@Override
			public void notify(String arg0, Object arg1) {
				// Nothing
			}
		});

	    try {
			final HtmlPage page = webClient.getPage( "http://www.mediafire.com/" );
			System.getProperties().put( "org.apache.commons.logging.simplelog.defaultlog", "fatal" );

			// Log in
			final HtmlForm form = page.getFormByName( "form_login1" );
			HtmlTextInput emailField = form.getInputByName( "login_email" );
			emailField.setValueAttribute( "your email" );

			HtmlPasswordInput pwdField = form.getInputByName( "login_pass" );
			pwdField.setValueAttribute( "your password" );

			final HtmlImageInput button = form.getInputByName( "submit_login" );
			HtmlPage page2 = (HtmlPage) button.click();

			// Then, get the JS script to extract the links
			// We parse the script and not the modified HTML (which I did not succeed BTW)
			JavaScriptPage jsPage = webClient.getPage( "http://www.mediafire.com/js/myfiles.php/?45144" );
			// 45144: seems to be the same ID for all (anonymous and registered account)

			Pattern pattern = Pattern.compile( "es\\[\\d+\\]=Array\\(([^;]*)\\);" );
			Matcher m = pattern.matcher( jsPage.getContent());
			while( m.find()) {

				String[] parts = m.group( 1 ).split( "," );
				String link = removeSurroundingQuotes( parts[ 3 ]);
				String filename = removeSurroundingQuotes( parts[ 5 ]);
				System.out.println( filename + ": http://mediafire/download.php?" + link );
			}

		} catch( FailingHttpStatusCodeException e ) {
			e.printStackTrace();

		} catch( MalformedURLException e ) {
			e.printStackTrace();

		} catch( IOException e ) {
			e.printStackTrace();

		} finally {
			webClient.closeAllWindows();
		}
	}

	/**
	 * Removes surrounding quotes.
	 * @param s a string
	 * <p>
	 * If the string is null or if its length is lesser than 2, then the original string is returned.
	 * If the first and last characters are different, then the original string is returned.
	 * If the first character is not a single quote or a double quote, then the original string is returned.
	 * Otherwise, the first and last characters are returned.
	 * </p>
	 */
	private static String removeSurroundingQuotes( String s ) {

		String result;
		if( s == null )
			result = s;
		else {
			int length = s.length();
			if( length < 2 )
				result = s;
			else if( s.charAt( 0 ) != s.charAt( length - 1 ))
				result = s;
			else if( s.charAt( 0 ) != '\'' && s.charAt( 0 ) != '"' )
				result = s;
			else
				result = s.substring( 1, s.length() - 1 );
		}

		return result;
	}
}