In this documentation we will go through how the data is compiled from notifications and A/B tests in Trustmary. The documentation is written for those who want to know the exact way the data is diplayed and gathered in Trustmary, so that the data can be compared to data from other channels.
1. User loads pixel logic
User session is created or loaded
Page visitation is saved
Page load conversions are tracked
Front end conversions are sent to client
Server will pick first popup (pathname/device filters)
2. Popup activation event-triggered logic
Check if this popup is previously filtered, don't show (event: activationFiltered) - Filtering happens due to showing notification only to x percent of users to split test
Check if this popup isn't previously activated and popup has activate session under 100%
=> test if this session should be activated and save activation/filtered status (if filtered, event: activationFiltered)If popup has reactivation time filter set, check if enough time is lasted from last activation, otherwise don't show (event: reactivationFiltered)
Save last activation time
Activate popup (event: modalOpen/notificationOpen)
3. Events from Notifications you can catch with GTM
tmary.activationFiltered - Notification not shown, because in notification settings set that notification is shown only to x% of the visitors (you can create a segment based on this trigger against which to measure the effectiveness of notifications)
tmary.reactivationFiltered: - Noticication not shown, because user has seen notification earlier
tmary.modalOpen - modal open
tmary.modalClose - modal close
tmary.notificationOpen - notification open
tmary.notificationClose - notification close
tmary.notificationClick - notification click
tmary.formSubmit - modal form submitted
More in-depth documentation of this here!
4. Conversion data in Trustmary
First lets define the 3 categories that can be seen on top of the Conversions page:
Visitors - User or 1 browser that we have been able to identify. Can have multiple sessions during an extended period of time.
Notified - Counts notificationOpen events, maximum one per day per visitor. So if a person visits the website and has the notificationOpen today and tomorrow, it will count as two. If twice today, it will count as one.
Filtered - Counts notificationFiltered events, maximum one per day per visitor. So if a browser visits the website today and tomorrow and gets filtered events on both days, it will count as two. But if event launches 100 times a day it just counts as one.
The above is good to take into account if the reactivation of the notification is more than 1 day, since in this case the segments of notified and filtered are not necessarily completely representative.
The Conversion data itself will be split into three segments:
Notification Activated - Conversion from users to which notificationOpen event has launched in the past 7 days
Notification Filtered - Conversion from users to which activationFiltered event has launched in the past 7 days
Other - Conversions from users which have not met the criteria for Popup activation event so they are not in category of filtered or activated
Important thing to note is that Activated and Filtered events are in relation to a specific notification. So you can be filtered from one notification and still get another notification.
For the specific data from each notification (if you have many) you can export an excel file which will have data related to each notification. Inside Trustmary we will only show the aggregate data from all notification events.
4.1 Conversion data in relation to Google Analytics data
When comparing data from Trustmary to Google analytics it is good to remember that the metrics are not completely identical and the logic is a bit different.
In Google Analytics, conversion rate is calculated conversions/sessions, so the timeframe where the effect of notification is compared by default is one session in Google Analytics (if you push the data to Google Analytics).
In Trustmary the conversion rate is calculated conversions/users (per segment) and the timeframe is past 7 days.
How does this actually affect the data?
The data is just a bit different and needs to be analyzed differently. The most common mistake in analyzing the data in Google Analytics is that you just compare two segments: notificationOpen and activationFiltered, even though you have some reactivation time limit in the notification. In this case if you just compare the two segments in the activationFiltered segment you will get a lot more returning visitors than in the notificationOpen segment.
To get segments which you can compare accurately, you should compare:
notificationOpen + reactivationFiltered
activationFiltered
These two segments will give you the comparison that you are most likely looking for.