In this document we will go through setting up A/B testing for Trustmary's notifications. The document will be split to three parts:
Setting up the A/B test for notifications in Trustmary
Setting up conversion tracking for A/B test in Trustmary
Setting up Trustmary events to Google analytics, so you can track the effect on Google analytics
In the first part we will go through the basics of setting up the notifications correctly, so you are able to run A/B tests to begin with. The second part will go through setting up conversion tracking for Trustmary and different options for it. On the third part we will go through how you can push the event data to Google Analytics using Google Tag Manager, if you like to measure it there.
1. Setting up notification for the A/B test
Setting up A/B test in notification settings is quite simple:
Navigate to Notifications from the left menu
Click the notification you want to edit (or create new one)
Click the Show on pages from the left tab
4. Scroll down to the Sessions section
5. Define Show to part of visitors to 50%
6. Click save, and you are done
Now, the specified notification will be shown to 50% of the users and 50% of users will be filtered to a group which you can use to compare the effectiveness of the notification. The filtering will be done on the moment of the notification trigger event.
You can also adjust the percentage of how many visitors will see the notification in case it is needed. Most commonly, the percentage is adjusted in case you are quite certain of the test result and want to show the notification to most users, but want to still get some data of what would the results be with out it. Or vice versa, if you are skeptical and want to test with just very small part of the visitors.
2. Setting up Conversion Tracking to Trustmary
Conversion tracking can be found in the Conversions tab from the menu on the left. To to start setting up the conversion tracking, you need to have the Trustmary pixel installed, which you can find under the Install code button. Make sure to install the pixel to all relevant pages of your website before the </head> tag.
Once you have the pixel installed you can get started with setting up the conversion tracking:
First click Create conversion from the top right (of the Conversions page)
Give the the conversion a name
Choose conversion type, there are 4 options for this:
URL - used for tracking conversions from specific thank you page views
Form submit (path) - Used for tracking conversions from form submits from a specific page
Form submit (querySelector) - Used for tracking conversions from form submits that have specified element
Data layer push - Gives you a snippet which you can run to launch conversion code. Mostly used with Google Tag Manager (for this you need to first click "Create new" to get the code)
Setup the conversion (more instructions below)
Click Create new
Setting up conversions with URL conversion type
To use the URL conversion type you need a "Thank you page" to which the users will be directed after the conversion. When you have the URL, you can take the URL path to the specified field, as seen on the example below
Thank you page url:
URL path (from above):
Using the /thank-you/ as seen in picture below:
What about dynamic thank you page urls?
Firstly, the URL path doesn't consider anything after the "?". If the URL would be for example "https://trustmary.io/thank-you/?test=0" it still works with the path "/thank-you/".
If there are other dynamic parameters, that are quite common, (for example everything after "/thank-you/" changes for each customer: https://trustmary.io/thank-you/dynamic-parameter=0204940242849) you can solve the problem by using a *-wildcard (solution for the aforementioned example: /thank-you/*). The *-wildcard works similarly as in the "Show on pages" section of notifications. Basically, everything is counted in after the asterisk.
So if you would use just an asterisk as a path, it would count all the page views to that conversion.
For the example above, it counts all the pages that start with /thank-you/ and that have something after the /thank-you/. It is important to remember that /thank-you/* does not work for for example https://trustmary.io/thank-you/ as it requires there to be something after last slash.
Setting up conversions with Form Submit (path) - COMING SOON
Documentation coming soon, meanwhile please use data layer push or URL if you are having a problem with Form Submit (path).
Setting up conversions with Form Submit (querySelector) - COMING SOON
Documentation coming soon, meanwhile please use data layer push or URL if you are having a problem with Form Submit (querySelector).
Setting up conversion with Data layer push and Google Tag manager
You can also use Data layer push conversion type without Google Tag Manager by running the scripts straight on desired action on your website. But on this documentation we will focus on using them with Google Tag Manager.
To start off you need to create conversion as defined above. After this, click on the conversion and you will see the code for the specific conversion you need to run on the conversion event, example below:
Copy paste the script and you are ready to move in to Google Tag Manager. There are two steps needed to be done to use the code in Tag Manager:
Setting up triggers - trigger when the conversion fires
Setting up tags - the conversion code from Trustmary
1. Setting up triggers
If you have already setup triggers for events you want to measure in Tag Manager you can move straight to tags. In this section, we will only go through the absolute basics of Tag Manager triggers, a more in-depth guide can be found here!
Triggers can be found inside a container from the left navigation from the tab Triggers.
(If you are using Google Tag Manager for the first time you need to confiqure the possible triggers from the Variables sections.)
To create a new trigger, click New on the top right and click Choose a trigger type to begin set-up after that. You will see a menu on the right now which has all the different trigger types. Trigger types are split into 4 different categories:
Page view - (Page view, DOM Ready, Window Loaded)
Click - (All elements, just links)
User Engagement - (Element visibility, form submission, scroll depth, Youtube video)
For our purposes the most useful from these are Page view triggers, click triggers and form submissions under user engagement. As you can also launch conversions based on page views using Trustmary URL - Conversion type, we will focus on click and form submission triggers here. Click events are mostly used for measuring button or link clicks that can work as conversions and form submissions for measuring for submissions. You can also measure form submissions straight in Trustmary using Form submit (path) or (querySelector) conversion types.
Creating click triggers
To setup a click trigger on Tag Manager choose All elements or Just Links and click some clicks or Some link clicks.
After this you get to define on which clicks you want the trigger to work.
The simplest of them all is to define it based on Click URL. In this case you copy the URL of the specific button or link you want to measure leads to and paste it to field on the right.
Creating form submission triggers
To set up form submission triggers in Google Tag Manager, choose the Form Submission and click some forms.
The easiest way to track form submission is to define a page where the form exists, but this only works if there is one form on a specific URL. You can do this by choosing Page URL and writing the specific URL on the right.
If you want to create a more specific trigger it is usually best to use Form ID. To do this, it is necessary for your form to have a specific ID. You can check the form ID by navigating to the specific page that has the URL, right clicking the page and clicking inspect. Use the "select an element in the page to inspect it" and bring the mouse on top of the form. You can find the form ID from the code of the site from the right, as seen on the picture below "gform_9".
After you have put the form ID into the field in Tag Manager you can use the trigger after saving it.
2. Setting up conversion tags
To setup conversion tags:
Navigate to the specific account and to the specific container you have installed to the website or other property
Navigate to Tags from the menu on left.
Click New from the top right
Click Choose a tag type to begin set-up
Choose Custom HTML from the right bar navigation
Copy paste the conversion code you got from Trustmary to the empty field
Select trigger you want the conversion event to launch on
Name the Tag
Publish workspace changes by clicking Submit from the top right and Publish after that.
3. Setting up Trustmary events to Google analytics
To do this you will need Google Tag Manager that is connected to the desired Google analytics accounts. If you have Tag Manager setup and linked to Google analytics, you are ready to go.
First make sure you have "custom event" as a trigger option enabled in variables, as we push events specified below to data layer from which you can use them with "custom event" triggers. After this there are several different events you can push info from to Google Analytics, as listed below:
tmary.activationFiltered - Notification not shown, because it's set in notification settings that notification is shown only to x percent of the visitors (you can create a segment based on this trigger, against which to measure the effectiveness of notifications)
tmary.reactivationFiltered: - Noticication not shown, because user has seen the notification earlier
tmary.modalOpen - modal open
tmary.modalClose - modal close
tmary.notificationOpen - notification open
tmary.notificationClose - notification close
tmary.notificationClick - notification click
tmary.formSubmit - modal form submitted
Usually the most important events to drive data to Google analytics are activationFiltered, notificationOpen, reactivationFiltered and formSubmit, in case you are using a form inside a modal.
To use these events, create a custom event and name it with the event. (For example tmary.activationFiltered, as seen below)
When you have specified trigger like this you can create tags, which trigger with the specified trigger and drive data to different platforms, for example Google analytics.
To push the data to Google analytics, create a new tag and choose Google analytics: Universal Analytics.
After this, choose as track type Event and define at least category and action as in the example below:
After this, make sure that you have chosen the right Google analytics property from the Google Analytics settings and click Save.
After you publish the workspace changes in Tag Manager you should have the data running to Google Analytics and based on it you can create segments which you can compare.