'Creating paginated archives in Grain' post illustration

Creating paginated archives in Grain

avatar

There are very few things in Grain framework that need a detailed explanation except being described in the docs. However, there's one thing, which I really want to clarify in details for Grain users. I wanted to write an article for the people who seek for simple instructions for implementing tasks that require complex manipulation to site pages, like including information from one page to another, creating or replacing pages, etc. As an example, in this guide I will tell you how to create paginated blog archive for your site.

Resource mapper mechanism

First of all, I'd like to briefly introduce the main mechanism we will use for this task. When it comes to modifying site resources, there's one stop where you should do it. In Grain we have SiteConfig.groovy -> resource_mapper closure that processes all the source files. In resource_mapper closure, initially you being given the list of site resources, represented by maps. And, in its basics, resource mapper looks like this:

1
2
3
resource_mapper = { Map resources ->
    // Do something with site resources
}

The resource maps contain properties, specified in YAML headers, and the set of properties that describe the resource for Grain, like url and location. You can have a look at the full list of these properties at the proper section of the documentation here. I am not duplicating all the list here, as for this task we will only need the url and location. As you might have already guessed, url stands for direct url to the resource, and location represents certain physical location in the site folder.

So we have a mechanism for modifying site resources, what can we do with them using this approach? The answer is, pretty much everything. We can add new pages to our site basing on the information taken from any other resource, add information to already described resource, change pages urls and even change the content manually, etc.

Preparing pages

Ok, so now, as we know that we have a mechanism that does all kinds of resource transformation, what do we do next? Following the main Grain concept, our site have content, theme and the framework layers, where framework layer is represented by Grain. Let's check out what step-by-step work we need to do in content and theme layers. At content layer, we only provide the necessary information, that may be simply modified by people, that doesn't have to know any programming language.

Note: All further logic is implemented on the top of Grain Template version 0.4.0

So here we simply create a folder for the articles, on the top of which we about to create the paginated archive. Let's create posts folder in the /content directory and a few sample markdown posts there. Initially they will look like this:

/content/blog/first-post.markdown
1
2
3
4
5
6
7
---
layout: default
title: My awesome post
published: true
date: 2014-02-01 17:33
---
A few good thoughts about things.

Configuring layouts

After we created some posts, let's start creating our archive. The archive will consist of the set of the paginated pages. Each of that pages will contain a brief information on the several posts and links to other pages of archive. Let's also note that the posts in our archive should be sorted by date.

Firstly, on content layer we will also need to provide the information about the layout that will handle rendering logic on the paginated pages. Looks like we definitely will need to create separate layout for this. And this is where we start to work on changes to theme layer. Before we start working with layouts, I would like to highly recommend to check out Creating a Custom Website or Theme with Grain post, which explains this mechanism in details. Please have a look if you haven't checked it out yet.

So, let's figure out what do we need to have on this layout. As we know, that one layout can use another layout for rendering, let our layout inherit the default one, so we don't need to duplicate header, footer, etc. Speaking about default layout, let's make it just a bit nicer by putting the following in the end of the default header, just for the sake of making navigation easier:

/theme/includes/header.html
1
2
3
4
5
...

<% if (page.url != '/') { %>
<p><a href = "${link '/'}">Go back to the main page</a></p>
<% } %>

The core information we need for rendering the paginated archive page, is the detailed information on posts on this page in the page model. For our page we will use such information as posts creation date, title and the link to this post. Also a brief introduction to the post might come in handy. In order to include it, we will use concept, where post authors decide where to cut the post by putting <!--more--> tag. Also, obviously we will need the links to the next and the previous page. When we put the above together, the layout page, that we may call blog.html, will look like this:

/theme/layouts/blog.html
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
---
layout: default
---
<% page.posts.each { post ->
def brief = post.render().content.split('<!--more-->').head()
%>
<h4>${post.date.format('yyyy/MM/dd')}</h4>
<h2><a href="${link post.url}">${post.title}</a></h2>
<p>${brief}</p>
<% } %>
<% if (page.prev_page) { %>
<a href="${link page.prev_page}">&larr; Newer</a>
<% } %>
<% if (page.next_page) { %>
<a href="${link page.next_page}">Older &rarr;</a>
<% } %>

Ok, so now, after we created a layout, we return to the point where we are letting Grain know which layout to use for paginated pages on content layer. In order to do this, we may simply create an empty html page index.html, in content/blog directory and set this resource layout to layout which we just created. Path of the file, that we will create, will be specified in location property of all the paginated pages, we will use this later for creating resource mapper logic. In the end the file will simply look like this:

/content/blog/index.html
1
2
3
4
5
---
layout: blog
title: Blog
published: true
---

Creating resources with resource mapper

When we have layout ready, we may start to work with resource_mapper in order to create resources for our paginated pages. Initially, in a template theme we have ResourceMapper.groovy class, in which some sample mapping logic already provided. From the start it only have the logic of filling in the creation and update dates and filtering unpublished pages. In version 0.4.0 resource mapping looks like this:

ResourceMapper.groovy
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
package com.example.site

import com.sysgears.grain.taglib.Site

/**
* Change pages urls and extend models.
*/
class ResourceMapper {

    /**
    * Site reference, provides access to site configuration.
    */
    private final Site site

    public ResourceMapper(Site site) {
        this.site = site
    }

    /**
    * This closure is used to transform page URLs and page data models.
    */
    def map = { resources ->

        def refinedResources = resources.findResults(filterPublished).collect {
            Map resource ->
                fillDates << resource
        }

        refinedResources
    }

    /**
    * Excludes resources with published property set to false,
    * unless it is allowed to show unpublished resources in SiteConfig.
    */
    private def filterPublished = { Map it ->
        (it.published != false || site.show_unpublished) ? it : null
    }

    /**
    * Fills in page `date` and `updated` fields
    */
    private def fillDates = { Map it ->
        def update = [
            date: it.date ?
                Date.parse(site.datetime_format, it.date) : new Date(it.dateCreated as Long),
            updated: it.updated ?
                Date.parse(site.datetime_format, it.updated) : new Date(it.lastUpdated as Long)
        ]
        it + update
    }
}

Firstly, before we start creating new resources, let's sort resources by creation date, and let's update urls for our posts. We will need this, as by default resource url is the same as resource location, so we could end up with many *.markdown urls that won't be processed as html by browser. After we make the changes we want, the resource mapper will look like this:

ResourceMapper.groovy
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
...

def map = { resources ->

    def refinedResources = resources.findResults(filterPublished).collect { Map resource ->
        setPostsUrls << // passing a resource with date filled in order to process urls
        fillDates <<
        resource
    }.sort { -it.date.time } // sorting resources by the creation date

    refinedResources
}

/**
* Sets SEO-friendly urls for blog posts.
*/
private def setPostsUrls = { Map it ->
    // Here we are filtering the resources which location starts from /posts/.
    // This way we will catch all the posts
    if (it.location =~ /\/posts\/.*/) {
        // Getting formatted information on date and title
        def date = it.date.format('yyyy/MM/dd/')
        def title = it.title.encodeAsSlug()
        // Setting updated url
        it.url = "/blog/$date$title/"
    }

    it
}

...

Ok, now we finally got to creating resources. Things we will need to do now is replace the resource of our base page, /blog/index.html, with the new resources for paginated pages in resources list. In the end we need resource_mapper to create pages that would have the following in-memory representation:

  • url - will be /blog/page/page-number if not first page, otherwise will be just /blog
  • location - will be the same for all the pages - /blog/index.html, paginated pages will differ only by url, posts and links to other paginated pages
  • posts - list of the posts for this page, each post should at least have url, date and title properties
  • prev_page - url to the previous page(may not be provided)
  • next_page - url to the next page(may not be provided) Except these properties, created resources should include all the properties set in our base page /blog/index.html

As we know exactly which resources we need to create, let's start coding.

ResourceMapper.groovy
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
...

def map = { resources ->

    ...

    // Capturing all the resources, which location starts with /posts/
    def posts = resources.findAll { it.location =~ /\/posts\/.*/ }

    // Reforming the whole resources list
    refinedResources.inject([]) { List updatedResources, Map page ->
        switch (page.url) {
            // Capturing base page resource
            case '/blog/':
                // Replacing base page resource with actual paginated pages
                updatedResources += paginate(posts, 3, page)
                break
            default:
                // Keeping other resources as they are
                updatedResources << page
        }

        updatedResources
    }
}

/**
* Creates paginated page resources out of given base page and the list of posts.
*
* @param pages models of resources , information on which need to be collected to pages
* @param perPage number of posts included in one page
* @param basePage model of the base page
* @return models of paginated pages resources
*/
private static def paginate(pages, perPage, basePage) {
    def pageUrl = { pageNo -> basePage.url + (pageNo > 1 ? "page/$pageNo/" : '') }
    def splitOnPages = pages.collate(perPage)
    def numPages = splitOnPages.size()
    def pageNo = 0
    splitOnPages.collect { itemsOnPage ->
        def model = [url: (pageUrl(++pageNo)), posts: itemsOnPage]
        if (pageNo > 1) {
            model.prev_page = pageUrl(pageNo - 1)
        }
        if (pageNo < numPages) {
            model.next_page = pageUrl(pageNo + 1)
        }
        // Merging base page with the created resource
        basePage + model
    }
}

...

Checking results

Looks like that's it, now in order to check out the results, simply have a look on your site by running ./grainw preview in the command line and the result will appear on http://localhost:4000. If you want to deploy the resulted site, don't forget to check out how simple it is to deploy Grain site to GitHub Pages service, you can have a look here for details. Eventually, after all the procedures, you will see something like this. Note that the complete sources for this guide is available here

Hopefully the provided guide will help you to adapt to Grain easier and faster.

If you're looking for a developer or considering starting a new project,
we are always ready to help!