Hello, I'm Soheil (Sol) Sadeghi

Thank you for visiting

About Me

Welcome to my website! I'm an applied scientist at Amazon, with a focus on machine learning for AWS. Before joining Amazon, I was a senior data scientist at Netflix, where I specialized in network experimentation. My journey also includes a role at Microsoft, where I focused on experimentation and machine learning for Windows.

Image placeholder

Education

2017 - University of Wisconsin-Madison

PhD in Statistics with a minor in Computer Science


2013 - University of Florida

MS in Statistics


2011 - Sharif University of Technology

BS in Industrial Engineering


Download CV

Online Experimentation

  • All
  • Novelty and Primacy
  • Sliced Designs

Novelty and Primacy: A Long-Term Estimator for Online Experiments

Joint work with Somit Gupta, Stefan Gramatovici, Jiannan Lu, Hao Ai, and Ruhan Zhang
Full Article

Online experiments are the gold standard for evaluating impact on user experience and accelerating innovation in software. However, since experiments are typically limited in duration, observed treatment effects are not always stable, sometimes revealing increasing or decreasing patterns over time. There are multiple causes for a treatment effect to change over time. In this paper, we focus on a particular cause, user-learning, which is primarily associated with novelty or primacy. Novelty describes the desire to use new technology that tends to diminish over time. Primacy describes the growing engagement with technology as a result of adoption of the innovation. Estimating user-learning is critical because it holds experimentation responsible for trustworthiness, empowers organizations to make better decisions by providing a long-term view of expected impact, and prevents user dissatisfaction. In this paper, we propose an observational approach, based on difference-in-differences technique to estimate user-learning at scale. We use this approach to test and estimate user-learning in many experiments at Microsoft. We compare our approach with the existing experimental method to show its benefits in terms of ease of use and higher statistical power, and to discuss its limitation in presence of other forms of treatment interaction with time.

Sliced Designs for Multi-Platform Online Experiments

Joint work with Peter Chien and Neeraj Arora
Full Article

Multivariate testing is a popular method to improve websites, mobile apps, and email campaigns. A unique aspect of testing in the online space is that it needs to be conducted across multiple platforms such as a desktop and a smartphone. The existing experimental design literature does not offer precise guidance for such a multi-platform context. In this article, we introduce a multi-platform design framework that allows us to measure the effect of the design factors for each platform and the interaction effect of the design factors with platforms. Substantively, the resulting designs are of great importance for testing digital campaigns across platforms. We illustrate this in an empirical email application to maximize engagement for a digital magazine. We introduce a novel “sliced effect hierarchy principle” and develop design criteria to generate factorial designs for multi-platform experiments. To help construct such designs, we prove a theorem that connects the proposed designs to the well-known minimum aberration designs. We find that experimental versions made for one platform should be similar to other platforms. From the standpoint of real-world application, such homogeneous subdesigns are cheaper to implement. To assist practitioners, we provide an algorithm to construct the designs that we propose.

A Sliced Design Approach for Conducting an Industry Email Campaign with Four Platforms

Joint work with Tzu-Hsiang Hung, Peter Chien, and Neeraj Arora
Full Article

Multivariate testing is a popular method to improve the effectiveness of digital marketing in industry. Online campaigns are often conducted across multiple platforms, such as desktops, tablets, smart phones, and smart watches. We propose minimum sliced aberration designs to accommodate online experiments with four platforms. This approach provides important insights into how different sets of design factors work differently across the four platforms, which can be used by industry for optimizing many forms of digital marketing. The effectiveness of the proposed approach is illustrated by an industrial email campaign with four platforms.

Get In Touch

Get In Touch

Contact Details