Notebook-Driven Development with Tecton 0.6: Combating Fraud and Optimizing Dynamic Pricing

Notebook-driven development enables data teams to consolidate tests, experiments, and code snippets within a single, shareable notebook for team collaboration. The outcome is a lightweight and versatile solution for feature definition and testing that facilitates rapid iteration and experimentation with diverse concepts. This post will focus on how it can improve feature engineering workflows, then focus in on a few key use cases that can be improved by integrating notebook development into your team’s workflows.

Accelerating model iteration

This new notebook-driven development capability allows data teams to quickly develop and test features within a familiar notebook environment — significantly speeding up the model iteration process as a result. It’s ideal for teams that have an interest in rapid prototyping and iteration, and are looking for easier ways to test various algorithms and models. 

Improving data collaboration

Notebook-driven development allows data teams to work within a shared environment and facilitate transparent documentation. This approach makes it easy to share, reproduce, and collaborate on code with other team members. By leveraging notebook-driven development in Tecton 0.6, data teams can optimize feature engineering workflows, leading to more accurate and efficient machine learning models.

More efficient engineering workflows

Feature engineering workflows can now be optimized through notebook-driven development and with the addition of new built-in aggregation functions. The construction and implementation of machine learning models now demand less time and effort, resulting in enhanced overall efficiency for data teams.

Enabling data exploration

Notebook development is well-suited for data exploration, particularly from a feature engineering perspective. You can now extract and select relevant features from data that can be used to build machine learning models. Additionally, the use of notebook-driven development allows for easy documentation and replication of results, which is essential for reproducible research.

How Notebook-driven development works with Tecton 0.6 

In Tecton 0.6, the notebook-driven development feature allows users to write feature definitions directly in a notebook, then test, edit, and run them all again from the same notebook. The output can then be made available in a repo before applying it to a Tecton cluster. This capability aims to simplify and accelerate the development and testing of features, making it more efficient for data engineers and data scientists.

Sample notebook tracking user sign-ups:

from tecton import Entity, BatchSource, FileConfig, batch_feature_view, FilteredSource
from datetime import datetime, timedelta

# Fetch the workspace
ws = tecton.get_workspace("prod")

# Fetch objects from the workspace
user = ws.get_entity("user")
user_sign_ups = ws.get_data_source("user_sign_ups")

# Use those objects as dependencies and define objects in a notebook
@batch_feature_view(
	sources=[FilteredSource(user_sign_ups)],
	entities=[user],
	mode="spark_sql",
	batch_schedule=timedelta(days=1),
	ttl=timedelta(days=3650),
)
def user_credit_card_issuer(user_sign_ups):
	return f"""
    	SELECT
        	user_id,
        	signup_timestamp,
        	CASE SUBSTRING(CAST(cc_num AS STRING), 0, 1)
            	WHEN '3' THEN 'AmEx'
            	WHEN '4' THEN 'Visa'
            	WHEN '5' THEN 'MasterCard'
            	WHEN '6' THEN 'Discover'
            	ELSE 'other'
        	END as credit_card_issuer
    	FROM
        	{user_sign_ups}
    	"""

# Validate objects interactively
user_credit_card_issuer.validate()

Fraud Detection Use Case

Notebook-driven development is designed to be used for essentially any use case your team sees fit. This post focuses on fraud detection and dynamic pricing as examples of where notebook development can be put to use.

Fraudulent activity is a major concern for companies that process financial transactions, and implementing machine learning models can be an effective way to prevent and identify fraudulent behavior. However, putting these models into production can be a challenging task, particularly when it comes to feeding the model the right data to make predictions.

Tecton’s feature store streamlines the process of putting a fraud detection model into production and this can be done in real-time, which is very important in fraud detection as fraud attacks can be automated and can lead to a large amount of fraud in a short period of time before batch-based fraud models can detect and shut down bad actors. Notebook-driven development with Tecton 0.6 can significantly enhance the process of building and refining relevant fraud detection models. By using Jupyter notebooks, data teams can easily explore data, test different algorithms, and experiment with various feature engineering techniques. As they iterate on features, they can seamlessly integrate these changes into their machine learning models. This streamlined process allows data scientists and engineers to work together more effectively, resulting in faster development cycles and more accurate fraud detection models.

Dynamic Pricing Use Case

Dynamic pricing is another use case that can greatly benefit from notebook-driven development with Tecton 0.6. Dynamic pricing strategies involve adjusting prices based on factors such as supply and demand, time of day, customer behavior, and market conditions. Building effective dynamic pricing models requires sophisticated feature engineering to capture these factors accurately.

With notebook-driven development, data teams can easily experiment with different features and pricing algorithms within Jupyter notebooks. As they discover new insights and develop more accurate models, they can quickly iterate on these features and incorporate them into their pricing strategies.

Tecton’s feature store can also help with ingesting and processing real-time data, such as customer behavior and market conditions, allowing the dynamic pricing models to make up-to-date and informed decisions. As a result, businesses can optimize their pricing strategies and maximize revenue while maintaining customer satisfaction.

Conclusion

Notebook-driven development allows data scientists and engineers to quickly iterate and experiment with different ideas, making it easier to identify anomalies and improve models over time. To learn more about Tecton 0.6 and its notebook-driven development capabilities, check out our 0.6 announcement and the Tecton 0.6 release notes.