专题出品人： 敬请期待
专题出品人： 敬请期待
专题：数据科学
在数据科学的时代，“数据科学家”、“数据分析师”等职位在各大公司开始越来越被重视，也开启了数据科学在中国发展的黄金时代。本专题着重探讨如何把数据科学落地化，如何提取数据中的商业价值为公司节省费用或者创造新的商业利益。
本专题下的议题
亿级大数据背后的游戏数据科学解决方案
程序 腾讯 技术专家
所属专题：数据科学
课程概要
背景介绍：
腾讯游戏数据挖掘与应用中心负责腾讯全部200多款游戏、3亿多玩家大数据的深度挖掘。面临数据量大， 数据类型多样化， 不同游戏场景差异化大等问题。经过6年发展，为游戏领域的运营问题定制出多种效果显著的数据解决方案与相关算法，在众多业务中应用取得不错效果。
解决思路/成功要点：
对于数据多样性问题，首先为结构化与非结构化数据设计了提取feature的流程，然后在此基础上提炼了用户画像。游戏的画像也很有挑战性， 它涉及图像，音效，玩法等维度，为此我们构建了游戏的知识图谱， 包括行业游戏图谱与单游戏图谱。有了这些数据基础，我们构建了多样化的应用场景，包括道具推荐，内容推荐等。 但是对于游戏场景，譬如社交推荐， 当遇到海量用户，就涉及到复杂的图计算问题，因为计算效率的问题，我们走了很多弯路，也研究出了很多新的算法，去解决这些问题，同样游戏的知识图谱构建，由于要求准确度高， 本身包括图片，文本，玩法等异构数据，也有别于传统图谱的构建。为此我们定制了游戏领域的多模态图谱构建方案。 基于此也为游戏行业趋势，新游发现，游戏资讯推荐 提供了基于图谱的支持，取得不错效果。在传统推荐上，也引入一些新的方法，譬如增强学习等来提升效果。
成果：
目前整个方案已经成为游戏接入过程中的重要一环，在游戏不同阶段的不同场景中，譬如新游戏预测，用户拉新、留存提升、匹配算法优化、游戏资讯内容推荐、游戏流失干预等场景均取得不错效果，整体平均约有30%效果提升。其中多项成果转化为专利和论文。
听众收益
1、了解游戏数据挖掘的解决方案
2、了解核心难点以及相关处理技术
3、了解一些研究方向的最新进展
Online Controlled Experimentation at Scale
Sophia Liu Microsoft Data scientist
所属专题：数据科学
课程概要
ABSTRACT
Online controlled experimentation (A/B testing) has been proven to be one of the most effective ways to improve products and drive revenue through data science. In this proposal, we summarize lessons learned from running thousands of experiments across multiple products with real examples from Microsoft. Then we present challenges and best practices for executing large-scale online controlled experiments.
AUDIENCE
This is an intermediate technical talk on digital randomized controlled experiments, i.e., A/B testing. Audience are expected to have basic knowledge of probability, statistics and hypothesis testing. We will cover basic concepts, but participants are assumed to have some prior knowledge. With real-world example experiments at Microsoft, we will present challenges, insights and learnings through several user-engaged games and exercises.
解决思路/成功要点：
What is experimentation, i.e., A/B tests?
Experimentation, i.e., A/B tests, is a method for comparing two behaviors—commonly, the existing behavior and a new behavior—against each other to determine which one performs better. Two or more variants are shown to users at random, and statistical analysis is used to determine which variation performs better against various business objectives of the product/organization. Through randomization, all other factors that could impact outcomes are balanced between control and treatment groups. Thus, using data collected from users, statistical tests can establish causal impact between changes in behavior and changes in key metrics.
成果：
Challenges and Best Practices in large-scale online experimentation
Since client-side experimentation requires code to be shipped to customers and data to be collected/sent from client devices, multiple challenges exist for client-side experimentation. In this session, we will discuss the common limitations and challenges in client-side experimentation with suggestions and guidelines for best practices.
1. Shipping cadence
Client-side experimentation requires different behavior to be sent to customers. This imposes two major challenges. First, experimentation need to coincide with shipping cadence. Given that client-software and OS need extensive testing, such updates may not happen as regular as server-side products (e.g. websites) which can quickly revert in the case of failures. Second, update of client updates is neither instantons nor complete (i.e. some users will not update). This limits the client-side experimentation’s agility. Silent updates or increase software shipping cadence using controlled rollouts can help by iterating faster and more efficiently.
2. Coding feature and call back
Client-side experimentation starts with the creation of one or more “variants” code paths, where each variant is an adjustment to the behavior of the system. Coding such features efficiently in the product design phase and collecting users’ assignment requires a centralized tool/platform. Moreover, if feature code introduced bugs/failures during the experiment, a centralized configuration platform needs to be in place to stop the experiment and withdraw those features from the client.
3. Trustworthy data collection
Since users’ telemetry (e.g. usage of the app) need to be sent back to the server, network connection/access is required. Thus, user data can be delayed or even lost. Furthermore, client time can be messy. Those data collected can cause validity issues which can dilute and sometimes invalidate the analysis results. One recommendation is to refine data cooking pipeline, adding data quality metrics, and continuously monitor data collection quality through a feedback system.
4. Metrics monitoring and alerting
Experimenting on thousands of features can be overwhelming. It is unscalable to manually check metrics movement per experiment. This is particularly important when failures occur, and we need to call back the features from the client. Having alerting system in place is key to solve these issues. In addition, we highly recommend experimenters to have a rich set of metrics which can help identify problems such as where we are blind in addition to successes. Three layers of metrics are suggested to use as guidelines for metrics design.
(a) Guardrail metrics. These are a set of organizational metrics that should not regress during the experiment. Some example metrics can be time to load a page for Bing, system usage for Windows etc.
(b) Diagnostic metrics. These are the set of metrics that can signify what the impact of local features brings to the product. Example metrics are funnel metrics for a shopping website.
(c)Key metrics contributing to Overall Evaluation Criteria (OEC). The OEC metrics are the ones that an organization regards as the key criteria for success. It sometimes happens that some teams are optimizing for retention rate while some teams are optimizing for revenue. Then leadership team should union them together as an organization to provide guidance on the OEC design making sure that teams are making efforts in the same direction.
5. Trustworthy data analysis
Running statistical tests and computing correct variances is critical for accurate interpretation of experimental results. Data issues like insufficient sample sizes, highly skewed distributions, and outliers pose serious challenges. Triangulation using multiple metrics and advanced statistical techniques can help in this scenario.
听众收益
The objective of this talk is to educate attendees about
(1) the key concepts in A/B testing,
(2) understand the challenges and best practices for executing large scale online controlled experiments.
(3) learn the full experimentation life cycle from metrics design to statistic
百度云数据工厂与数据科学
杨嘉义 百度 资深研发工程师
所属专题：数据科学
课程概要
百度云数据工厂与数据科学，是集数据可视化、大数据分析、建模、模型部署于一身的可视化开发平台。平台支持Notebook代码、表单、拖拽等交互模式，集成批量和流式的数据处理系统，提供统一作业提交接口和例行调度。平台集成优秀的机器学习和深度学习算法，支持自动机器学习，端到端GPU加速提速13倍，安全建模环境保障了数据和建模过程的安全性。能够帮助用户高效率、低门槛地完成数据科学工作。
听众收益
1. 了解百度云数据工厂与数据科学的整体架构
2. 了解其核心功能
3. 了解其典型应用场景
4. 了解其业务使用中的问题
专题：数据科学
在数据科学的时代，“数据科学家”、“数据分析师”等职位在各大公司开始越来越被重视，也开启了数据科学在中国发展的黄金时代。本专题着重探讨如何把数据科学落地化，如何提取数据中的商业价值为公司节省费用或者创造新的商业利益。
其他相关专题
-
数据科学
专题出品人：敬请期待
在数据科学的时代，“数据科学家”、“数据分析师”等职位在各大公司开始越来越被重视，也开启了数据科学在中国发展的黄金时代。本专题着重探讨如何把数据科学落地化，如何提取数据中的商业价值为公司节省费用或者创造新的商业利益。
-
智能数据分析
专题出品人：敬请期待
运用数据驱动的思想和策略在实践中逐渐成为共识；数据的价值已在科学研究和工商业的不同领域得到充分展现。然而，如果无法从数据中提取出知识和信息并加以有效利用，数据本身并不能驱动和引领数字化转型取得成功。如何让数据发挥它最大的价值？
-
大数据前沿
专题出品人：敬请期待
现今是数据信息时代，每分每秒都在产生数不尽的数据。这些数据如何获取，有什么用途，如何与前沿的深度学习、机器学习等相结合，是众多技术型企业都在关注的问题，本专题将探讨大数据平台的前沿技术，囊括Flink、Clickhouse、Spark Streaming等热门技术。
-
大数据平台架构
专题出品人：李远策
快手 数据架构师
人工智能技术离不开大数据的支撑，大数据近年来也已经渗透到当今每一个行业和业务职能领域，成为重要的生产因素。本专题将突出大数据平台实战，囊括Offline、Online、Real-time等多场景案例。
