Skip to content

Multi-layered cache with waterfall hit propagation and built-in storage adapters for DynamoDB, Redis, BigCache, GoLRU etc.

License

Notifications You must be signed in to change notification settings

juliaqiuxy/wfcache

Folders and files

NameName
Last commit message
Last commit date

Latest commit

6d62882 · Nov 22, 2021

History

44 Commits
Jun 8, 2021
May 31, 2021
Jun 5, 2021
Jun 8, 2021
Jun 6, 2021
Jun 8, 2021
Jun 7, 2021
Jun 7, 2021
May 31, 2021
Jun 7, 2021
Nov 22, 2021
Jun 8, 2021
Jun 6, 2021
Jun 6, 2021
Jun 5, 2021
Jun 12, 2021
Nov 22, 2021

Repository files navigation

Waterfall Cache

GoDoc wfcache CI Go Report Card npm

wfcache is a multi-layered cache with waterfall hit propagation and built-in storage adapters for DynamoDB, Redis, BigCache (in-memory), and GoLRU (in-memory)

wfcache is effective for read-heavy workloads and it can be used both as a side-cache or a read-through/write-through cache.

This libary is used in production by an application that receives 8 Million hits a month.

Built-in Storage Adapters

Package Description Eviction strategy
DynamoDB DynamoDB TTL
Redis Redis TTL/LRU
BigCache Performant on heap storage with minimal GC TTL (enforced on add)
GoLRU In-memory storage with approximated LRU similar to Redis TTL/LRU
Basic Basic in-memory storage (not recommended) TTL (enforced on get)

Installation

To retrieve wfcache, run:

$ go get github.com/juliaqiuxy/wfcache

Usage

import (
  "github.com/juliaqiuxy/wfcache"
  basic "github.com/juliaqiuxy/wfcache/basic"
  bigcache "github.com/juliaqiuxy/wfcache/bigcache"
  dynamodb "github.com/juliaqiuxy/wfcache/dynamodb"
)

c, err := wfcache.New(
  basic.Create(5 * time.Minute),
  bigcache.Create(2 * time.Hour),
  dynamodb.Create(dynamodbClient, "my-cache-table", 24 * time.Hour),
)

items, err := c.BatchGet(keys)
if err == wfcache.ErrPartiallyFulfilled {
  fmt.Println("Somethings are missing")
}

Usage with hooks

You can configure wfcache to notify you when each storage operation starts and finishes. This is useful when you want to do performance logging, tracing etc.

import (
  "context"
  "github.com/juliaqiuxy/wfcache"
  "gopkg.in/DataDog/dd-trace-go.v1/ddtrace/tracer"
  basic "github.com/juliaqiuxy/wfcache/basic"
)

func onStartStorageOp(ctx context.Context, opName string) interface{} {
  span, _ := tracer.StartSpanFromContext(ctx, opName)
  return span
}

func onFinishStorageOp(span interface{}) {
  span.(ddtrace.Span).Finish()
}

wfcache.NewWithHooks(
  onStartStorageOp,
  onFinishStorageOp,
  basic.Create(5 * time.Minute),
)

How it works

The following steps outline how reads from wfcache work:

  • When getting a value, wfcache tries to read it from the first storage layer (e.g. BigCache).
  • If the storage layer is not populated with the requested key-value pair (cache miss), transparent to the application, wfcache notes the missing key and moves on to the next layer. This continues until all configured storage options are exhausted.
  • When there is a cache hit, wfcache then primes each storage layer with a previously reported cache miss to make the data available for any subsequent reads.
  • wfcache returns the key-value pair back to the application

If you want to use wfcache as read-through cache, you can implement a custom adapter for your source database and configure it as the last storage layer. In this setup, a cache miss only ever happens in intermediate storage layers (which are then primed as your source storage resolves values) but wfcache would always yield data.

When mutating wfcache, key-value pairs are written and removed from all storage layers. To mutate a specific storage layer in isolation, you can ask wfcache to provide you with a reference to the underlying slice of storages by calling its Storages() method.

Cache eviction

wfcache leaves it up to each storage layer to implement their eviction strategy. Built-in adapters use a combination of Time-to-Live (TTL) and Least Recently Used (LRU) algorithm to decide which items to evict.

Also note that the built-in Basic storage is not meant for production use as the TTL enforcement only happens if and when a "stale" item is requested form the storage layer.

Implementing Custom Adapters

For use cases where:

  • you require a stroge adapter which is not included in wfcache, or
  • you want to use wfcache as a read-through/write-through cache

it is trivial to extend wfcache by implementing the following adapter interface:

type Storage interface {
  Get(ctx context.Context, key string) *CacheItem
  BatchGet(ctx context.Context, keys []string) []*CacheItem
  Set(ctx context.Context, key string, value []byte) error
  BatchSet(ctx context.Context, pairs map[string][]byte) error
  Del(ctx context.Context, key string) error
  BatchDel(ctx context.Context, keys []string) error
}

About

Multi-layered cache with waterfall hit propagation and built-in storage adapters for DynamoDB, Redis, BigCache, GoLRU etc.

Topics

Resources

License

Stars

Watchers

Forks

Packages

No packages published

Languages