Hi I am Aditya Nambiar , a 3rd year undergraduate student pursuing Computer Science and Engineering at IIT Bombay. I am currently working on a GSOC project - Visualization for Mailing stats and A/B testing with Kurund Jalmi as my mentor
This project aims at creating an A/B testing tool for CiviMail so all users of CiviMail can improve their response rate for all their mailings. To make this easier on the end user, this project will also incorporate graphical reporting (such as using charts, graphs etc.) of the mailing statistics. This will allow the admin to interpret and monitor the results of their mailing campaign and hence make a decision based on data.
We (my mentor Kurund Jalmi and I) are currently finalizing how the UI of A/B Testing should look like. To come up with a good UI I have searched through other available A/B testing platforms and my findings are available here http://wiki.civicrm.org/confluence/display/CRM/Research+Work
Several Components were more or less common to all of them such as
- The Test Type :-Subject Line, From Address and Email Content.
- Winner Criteria :- Open Rate and Total Unique Clicks.
One that was missing was the the total unique clicks on a particular link which I believe is required and was implemented by only one websites.
- Slider Bar (Or something similar ) to decide the percentage of subscribers to be tested upon.
- How long the test should run for
A few of the websites gave the option of automated testing but none of them appeared to have used statistics i.e gave the confidence interval of how better version A is than version B. I have planned to use Hypothesis testing in the A/B testing in CiviMail so that the results that which version is better are statistically significant .
Apart from that there would also be a graphical view of the results of version A and version B as it was implemented by a few of the websites. There would also be a preview page, to preview the email created and send test email and also to review all the selected options so that the user could verify them.
The link to the UI mockup is http://wiki.civicrm.org/confluence/display/CRM/UI+Mockup. Below are the first and last tabs in the mockup
Please suggest ideas/changes in the current mockup . Thanks
