Combining Online Stores for Real-Time Serving

Online stores are low-latency, key-value stores that hold the latest values of pre-computed features. They are used by machine learning models in production to retrieve features for real-time inference. Tecton supports the ability to use DynamoDB or Redis as an online store and offers documentation that may be beneficial in selecting which online store you would like to utilize for online feature serving.

At Tecton, we’ve also seen developers interested in using both DynamoDB and Redis simultaneously or even combining the two within a single Tecton instance for online features. With a few considerations, ML infrastructure teams can take advantage of the performance/cost advantages of both.

DynamoDB & Redis: A winning combination

As is the case with many infrastructure decisions, the choice between using DynamoDB and Redis for an online store comes down to a mixture of cost, performance, and operational overhead. For a succinct overview of these considerations, please see Tecton’s Comparison Summary, where we explain that a better online store can vary widely depending on latency, queries per second, and dataset sizes needed. An ML model’s feature retrieval needs can vary just as widely, and ML infrastructure teams may achieve significant optimizations by assigning an online store to each use model, or dynamically assigning an online store at inference time instead of retrieving every feature through the same online store.

For models requiring the lowest latency: Redis

Compared to DynamoDB, Redis offers lower latency and significantly better tail latencies in our benchmarks. If a model’s only concern is getting a prediction as fast as possible, regardless of price or other considerations, Redis may be the best option.

For models requiring the highest query volumes: Redis

Tecton uses DynamoDB with on-demand capacity mode, which allows DynamoDB to instantly accommodate workloads as they ramp up or down. As query volumes to an online store scaled past 10,000 queries per second, Tecton found in our benchmarking that DynamoDB’s autoscaling was significantly more expensive than adding additional Redis nodes to accommodate increasing query volumes. 

For models requesting features from the largest datasets: DynamoDB

Because Redis keeps everything in-memory, Tecton has noticed significant costs associated with keeping large (in the hundreds of gigabytes) datasets available in a Redis online store. Tecton’s Feature View ttl parameter can help with the rising costs associated with growing datasets, and so can considering DynamoDB as an online store.

For models that need to be online now: DynamoDB

DynamoDB is the default online store for Tecton Feature Views and is managed by Tecton. Because you will have to provision, scale, monitor, and manage a Redis cluster yourself if you use it as an online store, DynamoDB may be the best option if you would like a model to be retrieving features with Tecton as soon as possible.

Segmenting Tecton feature services by online store

In Tecton, a Feature Service groups common Feature Views together so they can be retrieved with a single API call. All Feature Views in a given Feature Service need to have the same online store, and you can select the online store on a per Feature View basis. Online store selection occurs when defining a Feature View via the online_store parameter, making the process of switching between stores or duplicating Feature Views to multiple stores straightforward. The following code takes the same Feature View definition and applies it to both a Feature Service using DynamoDB and a Feature Service using Redis.

from tecton import Entity, BatchSource, FileConfig, batch_feature_view, Aggregation,DynamoConfig, RedisConfig, FeatureService
from datetime import datetime, timedelta

transactions = BatchSource(
    name="transactions",
    batch_config=FileConfig(
        uri="s3://tecton.ai.public/tutorials/fraud_demo/transactions/data.pq",
        file_format="parquet",
        timestamp_field="timestamp",
    ),
)

user = Entity(name="user", join_keys=["user_id"])

@batch_feature_view(
    sources=[transactions],
    entities=[user],
    mode="spark_sql",
    aggregation_interval=timedelta(days=1),
    aggregations=[
        Aggregation(
            function="mean", 
            column="amt",
            time_window=timedelta(days=1)
        ),
    ],
    online=True,
    online_store=DynamoConfig()
)
def dynamo_feature_view(transactions):
    return f"""
        SELECT user_id, timestamp, amt
        FROM {transactions}
        """

@batch_feature_view(
    sources=[transactions],
    entities=[user],
    mode="spark_sql",
    aggregation_interval=timedelta(days=1),
    aggregations=[
        Aggregation(
            function="mean", 
            column="amt",
            time_window=timedelta(days=1)
        ),
    ],
    online=True,
    online_store=RedisConfig(
        primary_endpoint=”REDIS_ENDPOINT”,
        authentication_token=”REDIS_TOKEN”
    )
)
def redis_feature_view(transactions):
    return f"""
        SELECT user_id, timestamp, amt
        FROM {transactions}
        """

dynamo_feature_service = FeatureService(
    name="my_dynamodb_service", features=[dynamo_feature_view]
)
redis_feature_service = FeatureService(
    name="my_redis_service", features=[redis_feature_view]
)

Combining online stores

Feature retrieval needs do not only change from model to model—the same model may have significantly different retrieval requirements based on the time it’s being used. A new song or television series can drastically increase the demands on ML models powering streaming services, and the more retrieval methods can adapt with demands, the more optimized the resulting service will be.

With the two Feature Services defined in the section above, a simple conditional can decide whether to get features from Redis, which is significantly cheaper at high query volumes, or DynamoDB, which is significantly cheaper at low query volumes in our tests. (Note: This example uses Tecton’s Python Client, which is still in development.)

from tecton_client.tecton_client import TectonClient
from tecton_client.requests import GetFeaturesRequest
from tecton_client.requests import GetFeaturesRequestData

tecton_client = TectonClient(url, api_key)

request_data = GetFeaturesRequestData(join_key_map={"user_id": 1234})

if surge:
    request = GetFeaturesRequest(
        "my_workspace",
        "my_redis_service",  
        request_data=request_data
    )
else:
    request = GetFeaturesRequest(
        "my_workspace", 
        "my_dynamodb_service", 
        request_data=request_data
    )
tecton_client.get_features(request)

ML models could easily experience the opposite of the situation described above. For instance, a regional or national bank may have a model that is always surging as it is constantly processing transactions, except during off-hours (nights and weekends) when fewer transactions occur. If the differences in demand are significant enough, this bank could revert from Redis to Tecton’s default online store and realize the significant cost savings DynamoDB provides at lower query volumes during the bank’s off-hours.

from tecton_client.tecton_client import TectonClient
from tecton_client.requests import GetFeaturesRequest
from tecton_client.requests import GetFeaturesRequestData

tecton_client = TectonClient(url, api_key)

request_data = GetFeaturesRequestData(join_key_map={"user_id": 1234})

if off_hours:
    request = GetFeaturesRequest(
        "my_workspace", 
        "my_dynamodb_service",
        request_data=request_data
    )
else:
    request = GetFeaturesRequest(
        "my_workspace", 
        "my_redis_service", 
        request_data=request_data
    )
tecton_client.get_features(request)

There are many more examples in which the same ML model’s feature retrieval needs will vary widely from time to time. If this variance is great enough, implementing multiple online stores and choosing between them at inference time could result in significant performance increases and/or cost reductions.

Fortunately, Tecton’s feature platform makes it easy to define Feature Views and assign them to an online store. Tecton then automates the process of materializing features for real-time retrieval and keeping multiple offline and online stores consistent, while providing transparent performance and cost monitoring, regardless of whether Redis or DynamoDB (or both!) is used.