A unified data and machine learning foundation helped a national retailer
- Improve campaign measurement
- Accelerate experimentation
- Operationalize personalization across millions of customer records.
Personalization becomes difficult to scale when customer data, campaign measurement, and machine learning workflows live in separate systems.
A national retailer faced that problem across its digital engagement operations. Data was distributed across legacy infrastructure, marketing teams had limited visibility into the incremental impact of campaigns, and the path from model development to customer activation was too slow for continuous experimentation.
Factored worked as an embedded data engineering and machine learning partner to modernize the underlying data environment, establish more rigorous campaign measurement, and build predictive capabilities for next-best-action use cases.
The result was a shared foundation for analytics and personalization: Databricks-based data infrastructure, causal measurement methods, predictive models, activation workflows, and business-facing reporting operating as part of one connected system.
Fragmented systems limited both analytics and personalization
The retailer’s existing data environment created constraints at several layers.
On-premises systems limited scalability and increased the time required to process customer and campaign data. Pipelines were fragmented across teams and workflows, making it difficult to establish consistent views of customer behavior across channels.
Those limitations carried into marketing measurement. Without a reliable framework for separating campaign effects from underlying customer behavior, teams had difficulty determining which communications were actually contributing incremental orders or revenue.
Machine learning workflows faced a related constraint. Models intended to support next-best-action decisions could be developed, but deployment cycles and disconnected activation systems made it harder to operationalize their outputs quickly.
Knowledge was also distributed across engineering, analytics, and marketing teams. That made experimentation more dependent on individual workflows and slowed the transfer of technical capabilities into day-to-day marketing operations.
The problem was therefore broader than a data migration. The retailer needed an architecture that could support scalable processing, credible experimentation, predictive decisioning, and activation as connected capabilities.
Modernizing the data foundation on Databricks
Factored began by moving core data workloads from the retailer’s on-premises environment to Databricks.
The migration included redesigning data workflows around a Lakehouse architecture and creating modular schemas that could support both analytical workloads and future machine learning use cases.
Existing Pandas-based processing was replaced where appropriate with PySpark and SQL, enabling workloads to operate more efficiently across larger datasets. According to the supplied project results, this reduced processing time by 92%. This figure should be confirmed by the Data Engineering team before publication.
The new environment was designed to support both batch processing and real-time analytical needs rather than creating a separate infrastructure path for every downstream use case.
That architectural decision mattered because personalization depended on more than model quality. Customer behavior, campaign history, transaction data, experimentation results, and activation outputs all needed to move through an environment capable of supporting repeated analysis and model execution at scale.
With the shared Databricks foundation in place, engineering and analytics teams could build subsequent personalization capabilities on a common data layer.
Establishing a more rigorous way to measure campaign impact
Modernizing infrastructure addressed processing constraints, but it did not by itself solve the retailer’s marketing measurement problem.
Factored worked with analytics and marketing stakeholders to build an experimentation and measurement framework for campaigns delivered through channels including SMS, push notifications, and email.
The objective was to distinguish actual campaign lift from customer activity that might have occurred without the intervention.
The team applied causal inference techniques to reduce attribution bias and used propensity score matching to create more comparable audience groups based on purchase likelihood. Automated and manually defined control groups supported A/B testing and statistical comparisons where appropriate.
This gave marketing teams a more defensible basis for evaluating campaign effectiveness.
Instead of relying only on surface-level engagement or aggregate revenue changes, teams could examine the incremental contribution associated with specific campaigns and audience treatments.
The analysis also surfaced behavioral patterns relevant to campaign design. For example, the supplied project notes indicate that signals such as day-of-week preferences were associated with differences in order volume. The Machine Learning team should validate the exact analytical interpretation before publication.
The broader value was operational: experimentation became part of the personalization system rather than a separate reporting exercise performed after campaigns ended.
Turning customer signals into next-best-action decisions
With a unified data environment and stronger measurement framework in place, Factored developed predictive models designed to help determine which offer or interaction was most relevant for a given customer.
The models combined historical engagement, purchase frequency, and spend data to estimate next-order probability.
Factored also applied an Action Value Analysis approach to evaluate the expected value of different actions or offers. One example supplied in the project material was determining whether an incentive such as a discount tied to a subsequent order was appropriate for a specific customer context.
The intent was not simply to predict whether a customer might purchase. The system needed to connect predictions to actions that marketing teams could use.
Model outputs were deployed through Databricks and processed across millions of customer records on a monthly basis. Those outputs were then integrated with Hightouch and Salesforce to support activation through the retailer’s owned channels.
This closed an important gap between model development and production use. Predictions were no longer isolated analytical artifacts; they could become inputs to customer engagement workflows.
The Machine Learning team should validate the precise scoring logic, Action Value Analysis methodology, execution cadence, and activation behavior before publication.
Giving marketing teams direct access to campaign performance
Factored also built Power BI dashboards backed by the Databricks pipelines.
The reporting layer consolidated measures including order lift, ROI, redemption rates, and campaign impact into a shared view for marketing stakeholders.
This reduced dependence on manually assembled reporting and gave teams a faster way to monitor experiments and evaluate campaign performance.
According to the supplied project material, reporting cycles that previously took days could be completed in hours.
More importantly, the dashboards connected the technical work to operational decision-making. Marketing teams could use the same underlying data and measurement framework to launch, monitor, and adjust campaigns with less dependence on separate analytical handoffs.
Faster processing created room for faster experimentation
The combined work changed more than one part of the retailer’s data stack.
The Databricks migration improved the efficiency of large-scale processing. The experimentation framework strengthened campaign measurement. Predictive models connected customer signals to next-best-action decisions. Hightouch and Salesforce integrations created a path to activation, while Power BI gave business teams access to campaign results.
The project material reports several outcomes that require final validation before external publication:
- 92% reduction in data processing time
- 3× faster experimentation cycles
- Millions in incremental revenue associated with personalized campaign optimization
- Reporting cycles reduced from days to hours
Together, these results point to a more important architectural outcome: the retailer established a repeatable system for moving from customer data to measurement, prediction, activation, and learning.
That shortened the distance between an analytical question and an operational decision.
Personalization depends on the system around the model
The reusable lesson from this work is that personalization is not primarily a model-selection problem.
A next-best-action model can only create sustained value when the surrounding system can supply reliable data, measure whether interventions work, deliver predictions into operational tools, and make results visible to the teams responsible for acting on them.
For this retailer, that meant treating data engineering, experimentation, machine learning, activation, and reporting as parts of one production workflow.
Factored’s role was to help build that workflow alongside the retailer’s teams, while transferring the knowledge required to operate and extend it.
The resulting foundation gave the organization a more scalable way to test customer engagement strategies, evaluate their incremental impact, and apply predictive insights across owned channels.
