Open source web analytics

by Marnix Dessing

Open Source Web Analytics (OSWA)

In this post we review a list of Open Source Web Analytics (OSWA) tools. This is to give an indication of what tools are out there and how mature are these tools. As this project targets large organisations with enterprise size websites, we need a tool that is ready for enterprise level usage.

A open source maturity model (OSSM) can be used to objectively determine the maturity of open source products. Different models exist and can be used. A comparison is given in [2]. The model from “Succeeding with Open Source”[1] is also part of the comparison. This models also is used in the following maturity review.

Overview of OSWA tools

First we will have a look at the different OSWA tools that are available. Important to notice is that all these tools can be run without a providing party. In contrast to Google (Analytics), Adobe Analytics or Bing webmaster (Microsoft). Thanks to @berkes@bitcoinhackers.org for contributing to the list of OSWA tools.

* No support.
** Limited features.

Excluded

The products below are excluded from evaluation:

• signal - Seems to be deprecated.
• simpleanalytics - Not open source.

The Open Source Maturity Model

To assess the maturity of the open source web analytics tools an evaluation based on the Open Source Maturity Model (OSSM) is done. The OSSM model is based on the description in [1]. By using the default elements and element weights the table below is constructed.

The first phase of assessing maturity is scoring elements individually. To score all elements individually the OSMM describes four steps:

1. Define requirements for each element
2. Locate resources (not applicable?)
3. Assess maturity w.r.t. requirements from step 1.
4. Assign a maturity score between 0 and 10.

The second phase used for setting the weighting factors for each element. However default weighting factors can be used (see table above for the defaults). In phase three a overall maturity score is calculated based on the weighed sum.

In an evaluation of multiple Open Source Maturity Models “it has been observed that all existing OSS maturity models selection guidelines follow similar criteria.”[2] This evaluation also included the Novicasoft OSSM described in [1]. It is also pointed out that a weakness of the OSMM used is ignoring IT management and administration as a criteria. Which are important criteria, however the OSMM is used as a baseline in this project. If products fail to meet those criteria, it will become apparent in later project stadia (e.g. demoing & testing). The key point of the OSMM as applied in this project is judging the technical maturity of a Open Source Web Analytics product.

Element requirements

Let us define the requirements for each element.

Software

The book mentions requirements w.r.t. software should cover the following aspects:

• Functionality
• Quality of the product
• Longevity of the product
• Quality of the engineering team

Support

In the book is noted that support can come in three forms: community, paid or self-support. The requirements formulated w.r.t. support will focus on the diversity of community support.

Documentation

The need for documentation depends on the organization and departments within the organization. Usage manuals are required for end-users. A lack of technical documentation would hinder implementation of a web analytics tool.

Training

Integration

Service

Evaluation using OSMM

Assessment table: Matomo

Score: 100

Assessment table: Open web analytics

Score: 76

Assessment table: Ackee

Score: 58

Assessment table: Fathom (Lite)

Score: 63

Assessment table: Frequency

Score: 25

Assessment table: Trackingco.de

Score: 7

Assessment table: Picostats

Score: 0

Assessment table: GoAccess

Score: 51

Assessment table: Visitors

Score: 11

Assessment table: ELK stack

The ELK stack exists of 3 large components LogStash, Elasticsearch and Kibana. We will evaluate LogStash below.

Score: 100

Assessment table: Plausible

Score: 50

Score overview

The overview of all products and there scores is displayed below. We can now select the products that are not sufficiently mature to spend any time and research effort on. In Table 11.2 in “Succeeding with Open Source”[1] the minimum maturity scores for production software are: 60 for early adaptors / 70 for pragmatists.

Conclusion

In this post we have reviewed the technical maturity of 10 different open source web analytics tools. This review is based on the Open source maturity model developed by Navicasoft and described in the book “Succeeding with open source”[1]. It can be concluded that a lot of the open source web analytics tools that are reviewed are not mature enough to use in a production environment.

The products that scored higher than the recommended minimum score of 70 [1] are:

• Matomo
• Open web analytics
• ELK stack

We know of these three products that they are mature enough to function in conjunction with an enterprise level website and all complications that are related to such environments. It now remains to be seen which of these products fits best with the needs of the Radboud University.

References

[1] “Making Open Source Ready for the Enterprise: The Open Source Maturity Model”, from “Succeeding with Open Source” by Bernard Golden, Addison-Wesley, 2005, available form http://www.navicasoft.com
[2] Zahoor, Adnan, Khalid Mehboob, and Sarfaraz Natha. “Comparison of open source maturity models.” Procedia computer science 111 (2017): 348-354.