A Merkur case study: data visualizations for the 2020 Czech senate and regional elections

10 min readMay 13, 2021

Interested in microfrontends? Wondering how it actually works in practice?

In this article I will talk about a specific project: a cross-site data visualization widget/component for the 2020 Czech senate and regional elections. The project was built with Merkur, a microfrontend framework — you might want to read my previous article about it. In this one, I will focus on the technical aspects of building a microfrontend (I will use the term “widget”), the decisions we made and the problems we ran into while working on it.

The plan

As the project was originally conceived, we would build four widgets: one for each election (senate and regional), one to serve as navigation (separate to fit into the host sites’ layout), and one as a promotion/teaser. These widgets would be embedded on three websites: Novinky.cz, Seznam Zprávy, and the Seznam.cz homepage. Two of these run on the IMA.js framework, one is a custom React build. Ideally, the widgets should work even on a site with no React at all.

This isn’t an ad-hoc solution to be discarded after one use, but rather an ongoing project that will be maintained and reused for future elections — after all, unless things go very badly, some elections will continue to be held almost every year.

We decided to use Merkur because… well, to be honest, we already had Merkur (originally created for a different project) and needed to field test its capabilities on something more complex. In our case, much more complex, but you know how it goes. Sometimes the progress for new technology adoption isn’t so much a curve as a cliff face.

The widgets

These are the widgets that were created:

Menu widget

Provides navigation between main sections of the election widget below it. It needs to be separate to make space for the right sidebar, which is part of the host site layout and not the widget.

The menu widget contains a title (for SEO reasons) and two separate menus — one with widget navigation and the other with links to relevant articles.

Senate/regional widgets

The actual data display. They consist of multiple views, a “home view” with a map of regions/districts and total results, a list of candidates, detailed election results for a district, and for the whole region in case of regional elections.

Promo

A smaller, one-view data display. The data is displayed in a carousel, the links lead to the host app page with the widget.

Here you can see the finished widgets live: once on Novinky.cz (courtesy Google Translate link for the content), and here again, this time on Seznamzpravy.cz.

The process

Before we started working on the widgets themselves, one of our backend teams built a dedicated JSON API to serve the election data. The Czech Statistical Office distributes the election datasets as XML and CSV files, but we obviously can’t use those directly — if nothing else, because with our sites’ pageviews, we would have effectively DDoS-ed them.

Since all the widgets are built on Merkur and all have the same design, it would be great if we could share modules and components between them. Our solution to this was to develop them all in a single Lerna monorepo (other options could be git submodules or Bit).

If you haven’t encountered the term yet, “monorepo” refers to the practice of housing multiple projects in one version-controlled repository, as opposed to multiple separate repositories. It’s often used for projects that are mutually dependent, such as plugins. Lerna is a popular monorepo manager within the npm ecosystem.

Lerna’s main strength lies with managing release and versioning of interdependent modules, but it does just as well for our use case. All our widgets use a shared `node_modules` folder — that way, we save on space and can be sure that all widgets use the right module versions.

For the shared code, we decided that simplicity trumps everything and dumped it all into a single lerna package called `shared` (we’re very original, I know). An alternative could be to share code as privately-released npm modules. That sounds neat, but for newly written code it would be more trouble than it’s worth.

To get this setup to work, all we had to do was to properly configure Webpack and Jest aliases. Here is an example from our `webpack.config`:

const widgetDirname = path.resolve('./');
const lernaDirname = path.resolve(__dirname);{
  target: 'web',
  mode: environment,
  resolve: {
    alias: {
      shared: path.resolve(widgetDirname, '../shared')
    }
  },
  entry: {
    merkur: ['./src/client.jsx'],
  },
  // ...

And for Jest, all we need is:

const widgetDirname = path.resolve('./');merkurJestConfig.moduleNameMapper = {
  '^shared/(.+)': `${widgetDirname}/../shared/$1`,
};
merkurJestConfig.modulePaths = [`${widgetDirname}/node_modules`];

After that, we simply include the shared files as `shared/componentFolder/ComponentName`.

I mentioned earlier that the data widgets use multiple views. Merkur uses a `View` key in its config to display content, but there is no built-in way to switch between multiple views. To handle this, we have created a new Merkur plugin, `plugin-router`. (It’s already available on npm but as there is some useful functionality still missing, I recommend treating it as experimental for the time being.)

`plugin-router` uses universal-router under the hood; it defines routes as objects with a name, a path (with parameter placeholders) and an action (a handler function). In `plugin-router`, the action returns an object containing the View to display and new lifecycle functions for the route.

const routes = [
    {
        name: 'home',
        path: '/home',
        action: () => {
            return {
                PageView: HomeView
            };
        }
    },
    {
        name: 'post',
        path: '/post/:id',
        action: () => {
            return {
                PageView: PostView,
                load: async ({params: {postId}}) => {
                    const post = getPostData(postId);
                    return {
                        post
                    }
                }
            };
        }
    }
];

Now, talking about “routing” in a microfrontend context is tricky. Usually, we use the term to describe the process of mapping an URL to a handler, which controls the displayed content — for example, the Controller in MVC architecture. A microfrontend widget, though, should ideally not concern itself with the URL it is displayed on. After all, it can be placed anywhere within its host app. We might want to show two different views on the same URL; we might want to show the same view on two different URLs.

If you didn’t have to work with URLs or URL fragments, you could also pass route parameters directly as objects, instead of parsing them from a string. After all, you’re already communicating with the widget through an object!

In practice, there is one very good reason for the widget to know the absolute URLs of its views: generating links. If you need to navigate between widget’s views by links inside the widget, these links should be absolute URLs. Why?

To make linking work even with missing/disabled JavaScript.
For SEO.

The first one applies only if both the widget and the host app use server-side rendering. If your host app or widget are browser-rendered, there will be no content and therefore no links. But since search engines these days also crawl JS-generated content via headless browsers, even client-side-only apps need to think about having good links.

If we need to map routes to URL fragments anyway, we might as well use web application routing patterns (and a library that can do it for us, universal-router).

It’s all a bit frustrating, honestly.

Speaking of frustrations: in normal functioning we don’t let the browser navigate between widget views through HTTP requests. The host app traps the `<a>` tag clicks and switches between its own views. This does, unfortunately, fetch the widget from the server every time its view is changed, even though Merkur with plugin-router is capable of switching views without full reload. The problem is within the host app: when navigating between views, it does a full unmount/remount cycle for the page, which destroys and rebuilds the widget. We haven’t found the solution to that one yet — it should be possible, it’s just difficult.

To integrate the widgets and the host applications, we developed four modules. Two, `@merkur/integration` and `@merkur/integration-react`, are general Merkur modules available through npm, one is an IMA.js plugin (currently being prepared for release) and the last one is a custom module for our websites. I think this split was a good choice: we got two useful, versatile public modules out of it. Working on three new modules stacked on top of each other though…

In happier news, now that the modules have been tested (…thoroughly…), they’re safe to use if you want to try Merkur in your own app! They certainly make the integration much easier than doing it by yourself.

One important concern we had when building the widgets was performance. When using microfrontends, it’s important to keep in mind that its server will receive comparable load as the host app. The exact proportion will depend on the way you use the widget, but unless you display it only on a very small part of your website, you should pay as much attention to its optimization as you do with the host app. Our widgets run in multiple instances behind a load-balancer, and use in-memory caching as well as a Redis cache.

Last but not least, the evergreen of setting up a new project: what about testing?

We stayed with the setup that comes with Merkur out of the box: unit tests for components and integration tests for the widget API output, both done with Jest.

We have not set up visual regression testing from the start, something we have come to regret in the late stages of development. The data visualisation components and the senate/regional widgets contain a lot of logic that controls what and how things will be displayed, and without regression testing we spent entirely too much time hunting down unwanted changes through all the possible input combinations.

I’m happy to report that Merkur is now fully ready to integrate with Storybook for visual regression testing.

The verdict

So… did we succeed? I could leave you with some flashy numbers and inane positivity, but that doesn’t really say much about a project’s success. Especially considering that in a professional setting, “success” doesn’t have a single definition. Instead I’d like to give you a few different perspectives.

The project owner/project manager

The goal of the project was to offer authoritative and dependable election coverage to our news servers.

In this, the project certainly succeeded. It was delivered and on time, there were no outages and minimum errors that would impact user experience. It placed in top positions for the targeted keywords on multiple search engines as well.

The technical manager

The goal of the project was to develop a service that would fulfil the project owner’s requirements, but also be reusable to save developer time in the future. It should also test a new technology (i.e. Merkur) and fill potential gaps in its capabilities.

In this respect, the project has been a success as well. The critical “live” phase during the counting of the votes went off without any larger issues.

Although we didn’t get to test the widgets with a “vanilla” React host app, we’re certain that too will happen sooner or later. In fact, we’re already preparing for the next election date in early October of 2021.

The future will show how well we can share the components between widgets and how the tech stack will evolve. In general though, we are now confident in Merkur, its stability, and its usability for building even very large microfrontend widgets.

The developers

The goal was to fulfill the goals of the project owner and the technical manager, get it done on time, not have anything major blow up in our faces and not go barking mad in the process. ;)

This goal was also a success. We had four teams in two cities working on this project, 12 developers in total. Everybody survived and from talking with my colleagues, many of them enjoyed working on it quite a bit.

As an unplanned but very welcome side-effect, the cross-team cooperation ensured that almost every of our teams has someone with Merkur experience now.

Finally… oh alright, lets have some numbers.

The widgets add about 150kB of data to the site. Specifically, what gets included in the site is the Merkur core, widget CSS and one version of the widget JS bundle — either an ES9 version for modern browsers and a transpiled ES5 version with an additional polyfill file for older browsers.

Merkur core + plugins: ~5kB minified and GZipped
Largest widget CSS: ~27kB
Largest widget ES9 bundle: 117kB
Largest widget ES5: 248kB and 51kB of polyfills.

This graph displays the median FCP (first contentful paint), request and response times for Novinky.cz before, during and after the elections. This is for all pages on the site, since at least one widget (the promo) was present on every page.

As you can see, the widgets didn’t make any visible change in the numbers.