{"id":336,"date":"2023-09-01T17:01:17","date_gmt":"2023-09-01T14:01:17","guid":{"rendered":"https:\/\/dsprog.pro\/?p=336"},"modified":"2023-09-10T07:52:31","modified_gmt":"2023-09-10T04:52:31","slug":"a-b-test-again-testing-a-new-design","status":"publish","type":"post","link":"https:\/\/dsprog.pro\/ru\/a-b-test-again-testing-a-new-design\/","title":{"rendered":"A-B \u0442\u0435\u0441\u0442. \u0415\u0449\u0435 \u0440\u0430\u0437. \u0422\u0435\u0441\u0442\u0438\u0440\u0443\u0435\u043c \u043d\u043e\u0432\u044b\u0439 \u0434\u0438\u0437\u0430\u0439\u043d."},"content":{"rendered":"\n<p>\u0410\u0432\u0442\u043e\u0440: \u0415\u0432\u0433\u0435\u043d\u0438\u0439 \u0411\u043e\u0434\u044f\u0433\u0438\u043d, https:\/\/dsprog.pro<\/p>\n\n\n\n<h1 class=\"wp-block-heading\" id=\"Introduction\">\u0412\u0432\u0435\u0434\u0435\u043d\u0438\u0435<\/h1>\n\n\n\n<p>\u041d\u0435\u043e\u0431\u0445\u043e\u0434\u0438\u043c\u043e \u043f\u0440\u043e\u0432\u0435\u0440\u0438\u0442\u044c \u044d\u0444\u0444\u0435\u043a\u0442\u0438\u0432\u043d\u043e\u0441\u0442\u044c \u043d\u043e\u0432\u043e\u0433\u043e \u0434\u0438\u0437\u0430\u0439\u043d\u0430 web-\u043f\u0440\u0438\u043b\u043e\u0436\u0435\u043d\u0438\u044f. \u0415\u0441\u0442\u044c \u043f\u0440\u0435\u0436\u043d\u0438\u0439 \u0434\u0438\u0437\u0430\u0439\u043d &#171;A&#187;, \u0435\u0441\u0442\u044c \u043d\u043e\u0432\u044b\u0439 \u0434\u0438\u0437\u0430\u0439\u043d &#171;B&#187;. \u041f\u0440\u043e\u0432\u0435\u0434\u0435\u043c A-B \u0442\u0435\u0441\u0442. \u041e\u0433\u043e\u0432\u043e\u0440\u0438\u043c\u0441\u044f, \u0447\u0442\u043e \u0435\u0441\u0442\u044c \u043c\u0430\u0441\u0441\u0430 \u0432\u0430\u0440\u0438\u0430\u0446\u0438\u0439 \u043f\u0440\u043e\u0432\u0435\u0434\u0435\u043d\u0438\u044f A-B \u0442\u0435\u0441\u0442\u043e\u0432. \u041d\u0435\u043e\u0431\u0445\u043e\u0434\u0438\u043c\u043e \u0443\u0447\u0438\u0442\u044b\u0432\u0430\u0442\u044c \u043d\u044e\u0430\u043d\u0441\u044b \u0430\u043d\u0430\u043b\u0438\u0437\u0438\u0440\u0443\u0435\u043c\u043e\u0433\u043e \u043e\u0431\u044a\u0435\u043a\u0442\u0430\/\u043f\u0440\u043e\u0446\u0435\u0441\u0441\u0430 \u0432 \u043a\u0430\u0436\u0434\u043e\u043c \u043e\u0442\u0434\u0435\u043b\u044c\u043d\u043e\u043c \u0441\u043b\u0443\u0447\u0430\u0435. \u0412 \u0434\u0430\u043d\u043d\u043e\u043c \u043c\u0430\u0442\u0435\u0440\u0438\u0430\u043b\u0435 \u0431\u0443\u0434\u0443\u0442 \u043f\u043e\u043a\u0430\u0437\u0430\u043d\u044b <em>\u043e\u0441\u043d\u043e\u0432\u043d\u044b\u0435<\/em> \u0448\u0430\u0433\u0438 A-B \u0442\u0435\u0441\u0442\u0430. <\/p>\n\n\n\n<h1 class=\"wp-block-heading\" id=\"Experiment-Design\">\u0414\u0438\u0437\u0430\u0439\u043d \u044d\u043a\u0441\u043f\u0435\u0440\u0438\u043c\u0435\u043d\u0442\u0430<a href=\"http:\/\/localhost:8888\/notebooks\/Users\/targetplanet\/Desktop\/dsprog\/dsprog__witcozud0o\/dsprog__witcozud0o\/a-b-testing__ru__witcozud0o.ipynb#\u0414\u0438\u0437\u0430\u0439\u043d-\u044d\u043a\u0441\u043f\u0435\u0440\u0438\u043c\u0435\u043d\u0442\u0430\"><\/a><a href=\"http:\/\/localhost:8888\/notebooks\/Users\/targetplanet\/Desktop\/dsprog\/dsprog__witcozud0o\/dsprog__witcozud0o\/a-b-testing__en__witcozud0o.ipynb#Experiment-Design\"><\/a><\/h1>\n\n\n\n<ul class=\"wp-block-list\">\n<li><strong>\u041d\u0443\u043b\u0435\u0432\u0430\u044f \u0433\u0438\u043f\u043e\u0442\u0435\u0437\u0430<\/strong>: \u041d\u0435\u0442 \u0440\u0430\u0437\u043d\u0438\u0446\u044b \u0432 \u044d\u0444\u0444\u0435\u043a\u0442\u0438\u0432\u043d\u043e\u0441\u0442\u0438 \u043c\u0435\u0436\u0434\u0443 \u0434\u0438\u0437\u0430\u0439\u043d\u043e\u043c A \u0438 B.<\/li>\n\n\n\n<li><strong>\u0410\u043b\u044c\u0442\u0435\u0440\u043d\u0430\u0442\u0438\u0432\u043d\u0430\u044f \u0433\u0438\u043f\u043e\u0442\u0435\u0437\u0430<\/strong>: \u0415\u0441\u0442\u044c \u0437\u043d\u0430\u0447\u0438\u0442\u0435\u043b\u044c\u043d\u0430\u044f \u0440\u0430\u0437\u043d\u0438\u0446\u0430 \u0432 \u044d\u0444\u0444\u0435\u043a\u0442\u0438\u0432\u043d\u043e\u0441\u0442\u0438 \u043c\u0435\u0436\u0434\u0443 \u0434\u0438\u0437\u0430\u0439\u043d\u043e\u043c A \u0438 B.<\/li>\n\n\n\n<li><strong>\u041e\u0431\u044a\u0435\u043a\u0442 \u043e\u0431\u0440\u0430\u0431\u043e\u0442\u043a\u0438<\/strong>: \u0414\u0432\u0435 \u0432\u044b\u0431\u043e\u0440\u043a\u0438 \u0441\u0435\u0430\u043d\u0441\u043e\u0432 \u043f\u043e\u0441\u0435\u0442\u0438\u0442\u0435\u043b\u0435\u0439 \u0441\u0430\u0439\u0442\u0430 \u043f\u0435\u0440\u0435\u043c\u0435\u0448\u0430\u043d\u043d\u044b\u0435 \u0441\u043b\u0443\u0447\u0430\u0439\u043d\u044b\u043c \u043e\u0431\u0440\u0430\u0437\u043e\u043c. \u0420\u0430\u0437\u043c\u0435\u0440 \u0432\u044b\u0431\u043e\u0440\u043e\u043a \u043e\u0434\u0438\u043d\u0430\u043a\u043e\u0432.<\/li>\n\n\n\n<li><strong>\u0426\u0435\u043b\u0435\u0432\u0430\u044f \u0430\u0443\u0434\u0438\u0442\u043e\u0440\u0438\u044f<\/strong>: \u041f\u043e\u0441\u0435\u0442\u0438\u0442\u0435\u043b\u0438 \u0441\u0430\u0439\u0442\u0430 \u0437\u0430 \u043f\u043e\u0441\u043b\u0435\u0434\u043d\u0438\u0435 30 \u0434\u043d\u0435\u0439<\/li>\n\n\n\n<li><strong>\u041c\u0435\u0442\u0440\u0438\u043a\u0438<\/strong>:<\/li>\n\n\n\n<li><strong>\u0414\u043b\u0438\u0442\u0435\u043b\u044c\u043d\u043e\u0441\u0442\u044c \u043f\u0440\u043e\u0432\u0435\u0434\u0435\u043d\u0438\u044f \u0442\u0435\u0441\u0442\u0430<\/strong>: 30 \u0434\u043d\u0435\u0439<\/li>\n\n\n\n<li><strong>\u0423\u0440\u043e\u0432\u043d\u0438 \u0437\u043d\u0430\u0447\u0438\u043c\u043e\u0441\u0442\u0438<\/strong>:<\/li>\n\n\n\n<li><strong>\u03b1<\/strong> = 0.05<\/li>\n\n\n\n<li>\u041a\u0430\u043c\u043f\u0430\u043d\u0438\u0438 \u0437\u0430\u043f\u0443\u0441\u043a\u0430\u044e\u0442\u0441\u044f \u043e\u0434\u043d\u043e\u0432\u0440\u0435\u043c\u0435\u043d\u043d\u043e, \u0430 \u0430\u0443\u0434\u0438\u0442\u043e\u0440\u0438\u044f \u0434\u0435\u043b\u0438\u0442\u0441\u044f \u0441\u043b\u0443\u0447\u0430\u0439\u043d\u044b\u043c \u043e\u0431\u0440\u0430\u0437\u043e\u043c.<\/li>\n<\/ul>\n\n\n\n<h1 class=\"wp-block-heading\" id=\"Data-structure\">\u0421\u0442\u0440\u0443\u043a\u0442\u0443\u0440\u0430 \u0434\u0430\u043d\u043d\u044b\u0445<a href=\"http:\/\/localhost:8888\/notebooks\/Users\/targetplanet\/Desktop\/dsprog\/dsprog__witcozud0o\/dsprog__witcozud0o\/a-b-testing__ru__witcozud0o.ipynb#\u0421\u0442\u0440\u0443\u043a\u0442\u0443\u0440\u0430-\u0434\u0430\u043d\u043d\u044b\u0445\"><\/a><\/h1>\n\n\n\n<p>\u041a\u0430\u0436\u0434\u0430\u044f \u0441\u0442\u0440\u043e\u043a\u0430 \u0434\u0430\u043d\u043d\u044b\u0445 \u043f\u0440\u0435\u0434\u0441\u0442\u0430\u0432\u043b\u044f\u0435\u0442 \u0438\u043d\u0444\u043e\u0440\u043c\u0430\u0446\u0438\u044e \u043e \u0432\u0438\u0437\u0438\u0442\u0430\u0445 \u043f\u043e\u043b\u044c\u0437\u043e\u0432\u0430\u0442\u0435\u043b\u044f \u043d\u0430 web-\u043f\u0440\u0438\u043b\u043e\u0436\u0435\u043d\u0438\u044f.<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>.<\/li>\n\n\n\n<li><strong>\u041f\u043e\u043b\u044c\u0437\u043e\u0432\u0430\u0442\u0435\u043b\u044c ID (User_ID):<\/strong> ID \u043f\u043e\u043b\u044c\u0437\u043e\u0432\u0430\u0442\u0435\u043b\u044f.<\/li>\n\n\n\n<li><strong>\u041d\u0430\u0437\u0432\u0430\u043d\u0438\u0435 \u043a\u0430\u043c\u043f\u0430\u043d\u0438\u0438 (Campaign_Name):<\/strong> \u041d\u0430\u0437\u0432\u0430\u043d\u0438\u0435 \u043a\u0430\u043c\u043f\u0430\u043d\u0438\u0438 (\u0434\u0438\u0437\u0430\u0439\u043d \u0410 \u043b\u0438\u0431\u043e \u0434\u0438\u0437\u0430\u0439\u043d B).<\/li>\n\n\n\n<li><strong>\u041f\u0440\u043e\u0441\u043c\u043e\u0442\u0440 \u043a\u043e\u043d\u0442\u0435\u043d\u0442\u0430 (View)<\/strong>: \u0421\u043a\u043e\u043b\u044c\u043a\u043e \u0440\u0430\u0437 \u043f\u043e\u043b\u044c\u0437\u043e\u0432\u0430\u0442\u0435\u043b\u044c \u043f\u0440\u043e\u0441\u043c\u043e\u0442\u0440\u0435\u043b \u043a\u043e\u043d\u0442\u0435\u043d\u0442 \u043d\u0430 \u0441\u0430\u0439\u0442\u0435.<\/li>\n\n\n\n<li><strong>\u041a\u043b\u0438\u043a\u0438 (Click):<\/strong> \u041a\u043e\u043b\u0438\u0447\u0435\u0441\u0442\u0432\u043e \u043a\u043b\u0438\u043a\u043e\u0432 \u043d\u0430 \u0431\u0430\u043d\u043d\u0435\u0440. \u041a\u043e\u043b\u0438\u0447\u0435\u0441\u0442\u0432\u043e \u043f\u043e\u043b\u044c\u0437\u043e\u0432\u0430\u0442\u0435\u043b\u0435\u0439 \u043a\u043e\u0442\u043e\u0440\u044b\u0435 \u043f\u043e\u043b\u044c\u0437\u043e\u0432\u0430\u043b\u0438\u0441\u044c \u0434\u0438\u0437\u0430\u0439\u043d\u043e\u043c \u0410 \u0438 \u0434\u0438\u0437\u0430\u0439\u043d\u043e\u043c B \u0440\u0430\u0432\u043d\u044b.<\/li>\n<\/ul>\n\n\n\n<pre class=\"EnlighterJSRAW\" data-enlighter-language=\"python\" data-enlighter-theme=\"dracula\" data-enlighter-highlight=\"\" data-enlighter-linenumbers=\"\" data-enlighter-lineoffset=\"\" data-enlighter-title=\"\" data-enlighter-group=\"\">import pandas as pd\nimport numpy as np\nimport matplotlib.pyplot as plt\nimport seaborn as sns\n%matplotlib inline\nimport sys\n\nimport warnings\nwarnings.filterwarnings('ignore')\nfrom scipy.stats import mannwhitneyu\n\nalpha = 0.05 <\/pre>\n\n\n\n<h2 class=\"wp-block-heading\">\u0417\u0430\u0433\u0440\u0443\u0437\u043a\u0430 \u0434\u0430\u043d\u043d\u044b\u0445<\/h2>\n\n\n\n<pre class=\"EnlighterJSRAW\" data-enlighter-language=\"python\" data-enlighter-theme=\"dracula\" data-enlighter-highlight=\"\" data-enlighter-linenumbers=\"\" data-enlighter-lineoffset=\"\" data-enlighter-title=\"\" data-enlighter-group=\"\">alldata = pd.read_csv('input\/Xab_group.csv')\ndf_userAgg = pd.read_csv('input\/Xab_group.csv')\nalldata.head()<\/pre>\n\n\n\n<figure class=\"wp-block-table\"><table><thead><tr><th><\/th><th>User_ID<\/th><th>Campaign_Name<\/th><th>View<\/th><th>Click<\/th><\/tr><\/thead><tbody><tr><th>0<\/th><td>1<\/td><td>a campaign<\/td><td>4.0<\/td><td>0.0<\/td><\/tr><tr><th>1<\/th><td>2<\/td><td>b campaign<\/td><td>12.0<\/td><td>0.0<\/td><\/tr><tr><th>2<\/th><td>3<\/td><td>a campaign<\/td><td>1.0<\/td><td>0.0<\/td><\/tr><tr><th>3<\/th><td>4<\/td><td>a campaign<\/td><td>5.0<\/td><td>0.0<\/td><\/tr><tr><th>4<\/th><td>5<\/td><td>a campaign<\/td><td>8.0<\/td><td>0.0<\/td><\/tr><\/tbody><\/table><\/figure>\n\n\n\n<h2 class=\"wp-block-heading\">\u041f\u0440\u043e\u0432\u0435\u0440\u043a\u0430 \u043d\u0430 \u0434\u0443\u0431\u043b\u0438\u043a\u0430\u0442\u044b \u0438 null \u0437\u043d\u0430\u0447\u0435\u043d\u0438\u044f<\/h2>\n\n\n\n<pre class=\"EnlighterJSRAW\" data-enlighter-language=\"python\" data-enlighter-theme=\"dracula\" data-enlighter-highlight=\"\" data-enlighter-linenumbers=\"\" data-enlighter-lineoffset=\"\" data-enlighter-title=\"\" data-enlighter-group=\"\">alldata.duplicated().sum()<\/pre>\n\n\n\n<p>0<\/p>\n\n\n\n<pre class=\"EnlighterJSRAW\" data-enlighter-language=\"python\" data-enlighter-theme=\"dracula\" data-enlighter-highlight=\"\" data-enlighter-linenumbers=\"\" data-enlighter-lineoffset=\"\" data-enlighter-title=\"\" data-enlighter-group=\"\">alldata.isnull().sum()<\/pre>\n\n\n\n<p>User_ID 0<br>Campaign_Name 0 <br>View 0 <br>Click 0 <br>dtype: int64<\/p>\n\n\n\n<p>\u0412 \u0438\u0441\u0445\u043e\u0434\u043d\u044b\u0445 \u0434\u0430\u043d\u043d\u044b\u0445 \u043f\u043e\u0432\u0442\u043e\u0440\u044f\u044e\u0449\u0438\u0445\u0441\u044f \u0437\u043d\u0430\u0447\u0435\u043d\u0438\u0439 \u043d\u0435\u0442; null \u0437\u043d\u0430\u0447\u0435\u043d\u0438\u0439 \u0442\u0430\u043a\u0436\u0435 \u043d\u0435\u0442.<\/p>\n\n\n\n<pre class=\"EnlighterJSRAW\" data-enlighter-language=\"python\" data-enlighter-theme=\"dracula\" data-enlighter-highlight=\"\" data-enlighter-linenumbers=\"\" data-enlighter-lineoffset=\"\" data-enlighter-title=\"\" data-enlighter-group=\"\">grouped = alldata.groupby('Campaign_Name')\na_group = grouped.get_group('a campaign')\nb_group = grouped.get_group('b campaign')<\/pre>\n\n\n\n<pre class=\"EnlighterJSRAW\" data-enlighter-language=\"python\" data-enlighter-theme=\"dracula\" data-enlighter-highlight=\"\" data-enlighter-linenumbers=\"\" data-enlighter-lineoffset=\"\" data-enlighter-title=\"\" data-enlighter-group=\"\">desired_palette = [\"#0099cc\", \"#009900\"]\ndata_all = pd.concat([a_group, b_group])\nfig, ax = plt.subplots(ncols=2, figsize=(18,6))\nax1 = sns.barplot(data=data_all, x='Campaign_Name', y='View', errorbar=('ci', False), ax=ax[0], estimator='sum', palette=desired_palette)\nax2 = sns.barplot(data=data_all, x='Campaign_Name', y='Click', errorbar=('ci', False), ax=ax[1], estimator='sum', palette=desired_palette)\n# this content is from dsprog.pro\nax1.set_title('View')\nax2.set_title('Click')\n\nplt.show()\n\na_group_View_sum = a_group['View'].sum()\nb_group_View_sum = b_group['View'].sum()\na_group_Click_sum = a_group['Click'].sum()\nb_group_Click_sum = b_group['Click'].sum()\n\nprint('A group View sum: %s'%(a_group_View_sum))\nprint('B group View sum: %s'%(b_group_View_sum))\nprint('A group Click sum: %s'%(a_group_Click_sum))\nprint('B group Click sum: %s'%(b_group_Click_sum))<\/pre>\n\n\n\n<figure class=\"wp-block-image size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"1496\" height=\"547\" src=\"https:\/\/dsprog.pro\/wp-content\/uploads\/2023\/09\/witcozud0o_0.png\" alt=\"\" class=\"wp-image-321\"\/><\/figure>\n\n\n\n<p>A group View sum: 149422.0<br>B group View sum: 151117.0<br>A group Click sum: 10423.0<br>B group Click sum: 12133.0<\/p>\n\n\n\n<p>\u0414\u0438\u0437\u0430\u0439\u043d B \u044f\u0432\u043b\u044f\u0435\u0442\u0441\u044f \u0431\u043e\u043b\u0435\u0435 \u0440\u0435\u0437\u0443\u043b\u044c\u0442\u0430\u0442\u0438\u0432\u043d\u044b\u043c, \u043f\u043e \u0441\u0440\u0430\u0432\u043d\u0435\u043d\u0438\u044e \u0441 \u0434\u0438\u0437\u0430\u0439\u043d\u043e\u043c A. \u041f\u043e \u043e\u0431\u043e\u0438\u043c \u043f\u043e\u043a\u0430\u0437\u0430\u0442\u0435\u043b\u044f\u043c \u043d\u0430\u0431\u043b\u044e\u0434\u0430\u0435\u0442\u0441\u044f \u043f\u043e\u0437\u0438\u0442\u0438\u0432\u043d\u044b\u0439 \u044d\u0444\u0444\u0435\u043a\u0442. \u041d\u043e&#8230; \u0438\u043c\u0435\u044e\u0442 \u043b\u0438 \u043d\u0430\u0448\u0438 \u0432\u044b\u0432\u043e\u0434\u044b \u0441\u0442\u0430\u0442\u0438\u0441\u0442\u0438\u0447\u0435\u0441\u043a\u0443\u044e \u0437\u043d\u0430\u0447\u0438\u043c\u043e\u0441\u0442\u044c?<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">A-B \u0442\u0435\u0441\u0442<\/h2>\n\n\n\n<p>\u0420\u0430\u0441\u0441\u0447\u0438\u0442\u0430\u0435\u043c \u0441\u0440\u0435\u0434\u043d\u0438\u0435 \u043f\u043e \u043a\u0430\u0436\u0434\u043e\u043c\u0443 \u043f\u043e\u043a\u0430\u0437\u0430\u0442\u0435\u043b\u044e; \u043f\u043e \u043a\u0430\u0436\u0434\u043e\u0439 \u0433\u0440\u0443\u043f\u043f\u0435: A \u0438 B. \u0420\u0430\u0441\u0441\u0447\u0438\u0442\u0430\u0435\u043c \u0442\u0430\u043a\u0436\u0435 \u0440\u0430\u0437\u043d\u0438\u0446\u044b \u0441\u0440\u0435\u0434\u043d\u0438\u0445 \u0437\u043d\u0430\u0447\u0435\u043d\u0438\u0439.<\/p>\n\n\n\n<pre class=\"EnlighterJSRAW\" data-enlighter-language=\"python\" data-enlighter-theme=\"dracula\" data-enlighter-highlight=\"\" data-enlighter-linenumbers=\"\" data-enlighter-lineoffset=\"\" data-enlighter-title=\"\" data-enlighter-group=\"\">columns = ['View', 'Click']\nobs_diffs = {}\nfor column in columns:\n    view_cont= alldata.loc[alldata['Campaign_Name']=='a campaign'][column].mean()\n    view_test= alldata.loc[alldata['Campaign_Name']=='b campaign'][column].mean()\n    print('Mean of %s \"a campaign\": %sn' % (column, round(view_cont,3)))\n    print('Mean of %s \"b campaign\": %sn' % (column, round(view_test,3)))\n    \n    obs_dif_vw= view_test - view_cont\n    obs_diffs[column] = obs_dif_vw\n# this content is from dsprog.pro\n    print('Diff means of %s : %sn' % (column, round(obs_dif_vw,3)))\n    print(\"**************\")<\/pre>\n\n\n\n<p>Mean of View &#171;a campaign&#187;: 4.981<br>Mean of View &#171;b campaign&#187;: 5.037<br>Diff means of View : 0.056<\/p>\n\n\n\n<p>Mean of Click &#171;a campaign&#187;: 0.347<br>Mean of Click &#171;b campaign&#187;: 0.404<br>Diff means of Click : 0.057<\/p>\n\n\n\n<p>\u041f\u0440\u0438\u043c\u0435\u043d\u0438\u043c \u0442\u0435\u0445\u043d\u0438\u043a\u0443 \u0431\u0443\u0442\u0441\u0442\u0440\u044d\u043f\u0430 \u0434\u043b\u044f \u043e\u0446\u0435\u043d\u043a\u0438 \u0440\u0430\u0437\u043d\u043e\u0441\u0442\u0438 \u0441\u0440\u0435\u0434\u043d\u0438\u0445 \u0437\u043d\u0430\u0447\u0435\u043d\u0438\u0439 \u0432\u0430\u0440\u0438\u0430\u0431\u0435\u043b\u044c\u043d\u043e\u0441\u0442\u0438 \u043c\u0435\u0436\u0434\u0443 \u0433\u0440\u0443\u043f\u043f\u0430\u043c\u0438 a campaign \u0438 b campaign \u043f\u0443\u0442\u0435\u043c \u043c\u043d\u043e\u0433\u043e\u043a\u0440\u0430\u0442\u043d\u043e\u0439 \u0433\u0435\u043d\u0435\u0440\u0430\u0446\u0438\u0438 \u0441\u043b\u0443\u0447\u0430\u0439\u043d\u044b\u0445 \u0432\u044b\u0431\u043e\u0440\u043e\u043a \u0438 \u043f\u043e\u0434\u0441\u0447\u0435\u0442\u0430 \u0440\u0430\u0437\u043d\u043e\u0441\u0442\u0438 \u0441\u0440\u0435\u0434\u043d\u0438\u0445 \u0437\u043d\u0430\u0447\u0435\u043d\u0438\u0439. \u0421\u0444\u043e\u0440\u043c\u0438\u0440\u0443\u0435\u043c \u0433\u0438\u0441\u0442\u043e\u0433\u0440\u0430\u043c\u043c\u044b \u043f\u043e \u0440\u0435\u0437\u0443\u043b\u044c\u0442\u0430\u0442\u0430\u043c.<\/p>\n\n\n\n<pre class=\"EnlighterJSRAW\" data-enlighter-language=\"python\" data-enlighter-theme=\"dracula\" data-enlighter-highlight=\"\" data-enlighter-linenumbers=\"\" data-enlighter-lineoffset=\"\" data-enlighter-title=\"\" data-enlighter-group=\"\">columns = ['View', 'Click']\nboxes_data = {}\n\nfor column in columns:\n    n_iterations = 10000\n    group_control = alldata[alldata['Campaign_Name'] == 'a campaign'][column].values\n    group_test = alldata[alldata['Campaign_Name'] == 'b campaign'][column].values\n    \n    \n    diffs_viw = np.empty(n_iterations)\n    np.random.seed(200)\n    # this content is from dsprog.pro\n    for i in range(n_iterations):\n        random_indices = np.random.choice(len(group_control), len(group_control), replace=True)\n        sample_control = group_control[random_indices]\n        sample_test = group_test[random_indices]\n        \n        diffs_viw[i] = np.mean(sample_test) - np.mean(sample_control)\n        \n    diffs_view = np.array(diffs_viw)\n    boxes_data[column] = diffs_view\n    low, high = np.percentile(diffs_view, 2.5), np.percentile(diffs_view, 97.5)\n    \n    plt.figure(figsize=(6,5))\n    plt.xlabel(column + ': proportions of differences', fontsize=15)\n    plt.ylabel(column + ': frequency', fontsize=15)\n    plt.axvline(x= low, color= 'r', linewidth= 2)\n    plt.axvline(x= high, color= 'r', linewidth= 2)\n    plt.hist(diffs_view)<\/pre>\n\n\n\n<figure class=\"wp-block-image size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"555\" height=\"454\" src=\"https:\/\/dsprog.pro\/wp-content\/uploads\/2023\/09\/witcozud0o_1.png\" alt=\"\" class=\"wp-image-328\"\/><\/figure>\n\n\n\n<figure class=\"wp-block-image size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"555\" height=\"454\" src=\"https:\/\/dsprog.pro\/wp-content\/uploads\/2023\/09\/witcozud0o_2.png\" alt=\"\" class=\"wp-image-329\"\/><\/figure>\n\n\n\n<p>\u041f\u043e \u043a\u0430\u0436\u0434\u043e\u043c\u0443 \u043f\u043e\u043a\u0430\u0437\u0430\u0442\u0435\u043b\u044e \u0441\u0434\u0435\u043b\u0430\u0435\u043c \u0441\u043e\u043f\u043e\u0441\u0442\u0430\u0432\u043b\u0435\u043d\u0438\u0435 \u043c\u0435\u0436\u0434\u0443 p-value \u0438 \u0430\u043b\u044c\u0444\u0430 \u0443\u0440\u043e\u0432\u043d\u0435\u043c \u0434\u043b\u044f \u043f\u0440\u0438\u043d\u044f\u0442\u0438\u044f \u043b\u0438\u0431\u043e \u043e\u043f\u0440\u043e\u0432\u0435\u0440\u0436\u0435\u043d\u0438\u044f \u043d\u0443\u043b\u0435\u0432\u043e\u0439 \u0433\u0438\u043f\u043e\u0442\u0435\u0437\u044b.<\/p>\n\n\n\n<figure class=\"wp-block-image size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"628\" height=\"469\" src=\"https:\/\/dsprog.pro\/wp-content\/uploads\/2023\/09\/witcozud0o_3.png\" alt=\"\" class=\"wp-image-332\"\/><\/figure>\n\n\n\n<p>The p-value for View is: 0.1218<br>Unable to reject Null Hypothesis, no statistically difference between &#171;a group&#187; and &#171;b group&#187; 0.1218 &gt; \u03b1 on View<\/p>\n\n\n\n<p>The p-value for Click is: 0.0<br>Reject Null Hypothesis, statistically significant difference between &#171;a group&#187; and &#171;b group&#187; 0.0 &lt; \u03b1 on Click<\/p>\n\n\n\n<h1 class=\"wp-block-heading\" id=\"Conclusions\">\u0412\u044b\u0432\u043e\u0434\u044b<\/h1>\n\n\n\n<p>\u041f\u043e \u043f\u043e\u043a\u0430\u0437\u0430\u0442\u0435\u043b\u044e View \u043c\u044b \u043d\u0435 \u043c\u043e\u0436\u0435\u043c \u043e\u0442\u0432\u0435\u0440\u0433\u043d\u0443\u0442\u044c \u0433\u0438\u043f\u043e\u0442\u0435\u0437\u0443 <em>\ud835\udc3b<\/em>0. \u041d\u043e\u0432\u0430\u044f \u0444\u043e\u0440\u043c\u0430 \u0434\u0438\u0437\u0430\u0439\u043d\u0430 \u043d\u0435 \u043f\u0440\u0438\u0432\u043d\u0435\u0441\u043b\u0430 \u0441\u0442\u0430\u0442\u0438\u0441\u0442\u0438\u0447\u0435\u0441\u043a\u0438 \u0437\u043d\u0430\u0447\u0438\u043c\u044b\u0445 \u0438\u0437\u043c\u0435\u043d\u0435\u043d\u0438\u0439 \u043d\u0430 \u043f\u043e\u043a\u0430\u0437\u0430\u0442\u0435\u043b\u044c \u043f\u0440\u043e\u0441\u043c\u043e\u0442\u0440\u043e\u0432 \u043f\u043e\u043b\u044c\u0437\u043e\u0432\u0430\u0442\u0435\u043b\u044f\u043c\u0438 \u0441\u0442\u0440\u0430\u043d\u0438\u0446 web-\u043f\u0440\u0438\u043b\u043e\u0436\u0435\u043d\u0438\u044f. \u0412\u043c\u0435\u0441\u0442\u0435 \u0441 \u0442\u0435\u043c \u043d\u043e\u0432\u044b\u0439 \u0434\u0438\u0437\u0430\u0439\u043d \u043e\u043a\u0430\u0437\u0430\u043b \u0441\u0442\u0430\u0442\u0438\u0441\u0442\u0438\u0447\u0435\u0441\u043a\u0438 \u0437\u043d\u0430\u0447\u0438\u043c\u043e\u0435 \u0432\u043b\u0438\u044f\u043d\u0438\u0435 \u043d\u0430 \u043f\u043e\u043a\u0430\u0437\u0430\u0442\u0435\u043b\u044c Click. \u0422\u0430\u043a\u0438\u043c \u043e\u0431\u0440\u0430\u0437\u043e\u043c, \u0432 \u0446\u0435\u043b\u043e\u043c \u0437\u043d\u0430\u0447\u0438\u0442\u0435\u043b\u044c\u043d\u0430\u044f \u0440\u0430\u0437\u043d\u0438\u0446\u0430 \u0432 \u044d\u0444\u0444\u0435\u043a\u0442\u0438\u0432\u043d\u043e\u0441\u0442\u0438 \u043c\u0435\u0436\u0434\u0443 \u0434\u0438\u0437\u0430\u0439\u043d\u043e\u043c A \u0438 B \u0432\u0441\u0435 \u0436\u0435 \u0441\u0443\u0449\u0435\u0441\u0442\u0432\u0443\u0435\u0442.<\/p>\n\n\n\n<p>=====================<\/p>\n\n\n\n<p>\u0414\u043b\u044f \u043f\u043e\u043b\u0443\u0447\u0435\u043d\u0438\u044f \u043f\u043e\u043b\u043d\u043e\u0433\u043e \u043a\u043e\u043d\u0442\u0435\u043d\u0442\u0430 \u0434\u0430\u043d\u043d\u043e\u0439 \u0441\u0442\u0430\u0442\u044c\u0438; \u0434\u043b\u044f \u0441\u043e\u0442\u0440\u0443\u0434\u043d\u0438\u0447\u0435\u0441\u0442\u0432\u0430 \u0441 \u0430\u0432\u0442\u043e\u0440\u043e\u043c, \u043f\u0438\u0448\u0438\u0442\u0435 \u0432 \u0422\u0435\u043b\u0435\u0433\u0440\u0430\u043c: cryptosensors<br>\u041f\u043e\u043b\u043d\u044b\u0439 \u043a\u043e\u043d\u0442\u0435\u043d\u0442 \u0434\u0430\u043d\u043d\u043e\u0439 \u0441\u0442\u0430\u0442\u044c\u0438 \u0432\u043a\u043b\u044e\u0447\u0430\u0435\u0442 \u0444\u0430\u0439\u043b\u044b\/\u0434\u0430\u043d\u043d\u044b\u0435:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>\u0444\u0430\u0439\u043b Jupyter Notebook (ipynb),<\/li>\n\n\n\n<li>\u0444\u0430\u0439\u043b\u044b \u0434\u0430\u0442\u0430\u0441\u0435\u0442\u043e\u0432 (csv \u0438\u043b\u0438 xlsx),<\/li>\n\n\n\n<li>\u043d\u0435\u043e\u043f\u0443\u0431\u043b\u0438\u043a\u043e\u0432\u0430\u043d\u043d\u044b\u0435 \u044d\u043b\u0435\u043c\u0435\u043d\u0442\u044b \u043a\u043e\u0434\u0430, \u043d\u044e\u0430\u043d\u0441\u044b.<\/li>\n<\/ul>\n\n\n\n<p>\u0420\u0430\u0441\u043f\u0440\u043e\u0441\u0442\u0440\u0430\u043d\u0435\u043d\u0438\u0435 \u043c\u0430\u0442\u0435\u0440\u0438\u0430\u043b\u043e\u0432 \u0441 dsprog.pro \u0433\u043e\u0440\u044f\u0447\u043e \u043f\u0440\u0438\u0432\u0435\u0442\u0441\u0442\u0432\u0443\u0435\u0442\u0441\u044f (\u0441 \u0443\u043a\u0430\u0437\u0430\u043d\u0438\u0435\u043c \u0441\u0441\u044b\u043b\u043a\u0438 \u043d\u0430 \u0438\u0441\u0442\u043e\u0447\u043d\u0438\u043a).<\/p>\n","protected":false},"excerpt":{"rendered":"<p>\u0410\u0432\u0442\u043e\u0440: \u0415\u0432\u0433\u0435\u043d\u0438\u0439 \u0411\u043e\u0434\u044f\u0433\u0438\u043d, https:\/\/dsprog.pro \u0412\u0432\u0435\u0434\u0435\u043d\u0438\u0435 \u041d\u0435\u043e\u0431\u0445\u043e\u0434\u0438\u043c\u043e \u043f\u0440\u043e\u0432\u0435\u0440\u0438\u0442\u044c \u044d\u0444\u0444\u0435\u043a\u0442\u0438\u0432\u043d\u043e\u0441\u0442\u044c \u043d\u043e\u0432\u043e\u0433\u043e \u0434\u0438\u0437\u0430\u0439\u043d\u0430 web-\u043f\u0440\u0438\u043b\u043e\u0436\u0435\u043d\u0438\u044f. \u0415\u0441\u0442\u044c \u043f\u0440\u0435\u0436\u043d\u0438\u0439 \u0434\u0438\u0437\u0430\u0439\u043d &#171;A&#187;, \u0435\u0441\u0442\u044c \u043d\u043e\u0432\u044b\u0439 \u0434\u0438\u0437\u0430\u0439\u043d &#171;B&#187;. \u041f\u0440\u043e\u0432\u0435\u0434\u0435\u043c A-B \u0442\u0435\u0441\u0442. \u041e\u0433\u043e\u0432\u043e\u0440\u0438\u043c\u0441\u044f, \u0447\u0442\u043e \u0435\u0441\u0442\u044c \u043c\u0430\u0441\u0441\u0430 \u0432\u0430\u0440\u0438\u0430\u0446\u0438\u0439 \u043f\u0440\u043e\u0432\u0435\u0434\u0435\u043d\u0438\u044f A-B \u0442\u0435\u0441\u0442\u043e\u0432. \u041d\u0435\u043e\u0431\u0445\u043e\u0434\u0438\u043c\u043e \u0443\u0447\u0438\u0442\u044b\u0432\u0430\u0442\u044c \u043d\u044e\u0430\u043d\u0441\u044b \u0430\u043d\u0430\u043b\u0438\u0437\u0438\u0440\u0443\u0435\u043c\u043e\u0433\u043e \u043e\u0431\u044a\u0435\u043a\u0442\u0430\/\u043f\u0440\u043e\u0446\u0435\u0441\u0441\u0430 \u0432 \u043a\u0430\u0436\u0434\u043e\u043c \u043e\u0442\u0434\u0435\u043b\u044c\u043d\u043e\u043c \u0441\u043b\u0443\u0447\u0430\u0435. \u0412 \u0434\u0430\u043d\u043d\u043e\u043c \u043c\u0430\u0442\u0435\u0440\u0438\u0430\u043b\u0435 \u0431\u0443\u0434\u0443\u0442 \u043f\u043e\u043a\u0430\u0437\u0430\u043d\u044b \u043e\u0441\u043d\u043e\u0432\u043d\u044b\u0435 \u0448\u0430\u0433\u0438 A-B \u0442\u0435\u0441\u0442\u0430. \u0414\u0438\u0437\u0430\u0439\u043d \u044d\u043a\u0441\u043f\u0435\u0440\u0438\u043c\u0435\u043d\u0442\u0430 \u0421\u0442\u0440\u0443\u043a\u0442\u0443\u0440\u0430 \u0434\u0430\u043d\u043d\u044b\u0445 \u041a\u0430\u0436\u0434\u0430\u044f \u0441\u0442\u0440\u043e\u043a\u0430 \u0434\u0430\u043d\u043d\u044b\u0445 [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":348,"comment_status":"closed","ping_status":"","sticky":false,"template":"","format":"standard","meta":{"_locale":"ru_RU","_original_post":"https:\/\/dsprog.pro\/?p=314","footnotes":""},"categories":[8,3,1,4],"tags":[24,14,23],"class_list":["post-336","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-a-b-test","category-data-processing","category-data-science","category-python","tag-bootstrap","tag-hypothesis-testing","tag-new-design-effective","ru-RU","entry","has-media"],"_links":{"self":[{"href":"https:\/\/dsprog.pro\/wp-json\/wp\/v2\/posts\/336","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/dsprog.pro\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/dsprog.pro\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/dsprog.pro\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/dsprog.pro\/wp-json\/wp\/v2\/comments?post=336"}],"version-history":[{"count":13,"href":"https:\/\/dsprog.pro\/wp-json\/wp\/v2\/posts\/336\/revisions"}],"predecessor-version":[{"id":368,"href":"https:\/\/dsprog.pro\/wp-json\/wp\/v2\/posts\/336\/revisions\/368"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/dsprog.pro\/wp-json\/wp\/v2\/media\/348"}],"wp:attachment":[{"href":"https:\/\/dsprog.pro\/wp-json\/wp\/v2\/media?parent=336"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/dsprog.pro\/wp-json\/wp\/v2\/categories?post=336"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/dsprog.pro\/wp-json\/wp\/v2\/tags?post=336"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}