Caches

Once a Session has been obtained, it is now possible to begin working with NamedMap and/or NamedCache instances. This is done by calling either Session.get_map(name: str, cache_options: Optional[CacheOptions] = None) or Session.get_cache(name: str, cache_options: Optional[CacheOptions] = None). The name argument is the logical name for this cache. The optional cache_options argument accepts a CacheOptions to configure the default time-to-live (ttl) for entries placed in the cache as well as allowing the configuration of near caching (discussed later in this section).

Here are some examples:

# obtain NamedCache 'person'
cache: NamedCache[int, Person] = await session.get_cache("person")

# obtain NamedCache 'person' with a default ttl of 2000 millis
# any entry inserted into this cache, unless overridden by a put call
# with a custom ttl, will have a default ttl of 2000
options: CacheOptions = CacheOptions(2000)
cache: NamedCache[int, Person] = await session.get_cache("person", options)

Near Caches

Near caches are a local cache within the NamedMap or NamedCache that will store entries as they are obtained from the remote cache. By doing so, it is possible to reduce the number of remote calls made to the Coherence cluster by returning the locally cached value. This local cache also ensures updates made to, or removal of, an entry are properly reflected thus ensuring stale data isn’t mistakenly returned.

Note

Near caching will only work with Coherence CE 24.09 or later. Attempting to use near caching features with older versions will have no effect.

A near cache is configured via NearCacheOptions which provides several options for controlling how entries will be cached locally.

  • ttl - configures the time-to-live of locally cached entries (this has no impact on entries stored within Coherence). If not specified, or the ttl is 0, entries in the near cache will not expire

  • high_units - configures the max number of entries that may be locally

    cached. Once the number of locally cached entries exceeds the configured value, the cache will be pruned down (least recently used entries first) to a target size based on the configured prune_factor (defaults to 0.80 meaning the prune operation would retain 80% of the entries)

  • high_units_memory - configures the maximum memory size, in bytes, the

    locally cached entries may use. If total memory exceeds the configured value, the cache will be pruned down (least recently used entries first) to a target size based on the configured prune_factor (defaults to 0.80 meaning the prune operation would retain 80% the cache memory)

  • prune_factor - configures the target near cache size after exceeding

    either high_units or high_units_memory high-water marks

Note

high_units and high_units_memory are mutually exclusive

Examples of configuring near caching:

# obtain NamedCache 'person' and configure near caching with a local
# ttl of 20_000 millis
near_options: NearCacheOptions = NearCacheOptions(20_000)
cache_options: CacheOptions = CacheOptions(near_cache_options=near_options)
cache: NamedCache[int, Person] = await session.get_cache("person", options)

# obtain NamedCache 'person' and configure near caching with a max
# number of entries of 1_000 and when pruned, it will be reduced
# to 20%
near_options: NearCacheOptions = NearCacheOptions(high_units=1_000, prune_factor=0.20)
cache_options: CacheOptions = CacheOptions(near_cache_options=near_options)
cache: NamedCache[int, Person] = await session.get_cache("person", options)

To verify the effectiveness of a near cache, several statistics are monitored and may be obtained from the CacheStats instance returned by the near_cache_stats property of the NamedMap or NamedCache

The following statistics are available (the statistic name given is the same property name on the CacheStats instance)

  • hits - the number of times an entry was found in the near cache

  • misses - the number of times an entry was not found in the near cache

  • misses_duration - The accumulated time, in millis, spent for a cache miss (i.e., having to make a remote call and update the local cache)

  • hit_rate - the ratio of hits to misses

  • puts - the total number of puts that have been made against the near cache

  • gets - the total number of gets that have been made against the near cache

  • prunes - the number of times the cache was pruned due to exceeding the configured high_units or high_units_memory high-water marks

  • expires - the number of times the near cache’s expiry logic expired entries

  • num_pruned - the total number of entries that were removed due to exceeding the configured high_units or high_units_memory high-water marks

  • num_expired - the total number of entries that were removed due to expiration

  • prunes_duration - the accumulated time, in millis, spent pruning the near cache

  • expires_duration - the accumulated time, in millis, removing expired entries from the near cache

  • size - the total number of entries currently held by the near cache

  • bytes - the total bytes the near cache entries consume

Note

The near_cache_stats option will return None if near caching isn’t configured or available

The following example demonstrates the value that near caching can provide:

 1# Copyright (c) 2024, Oracle and/or its affiliates.
 2# Licensed under the Universal Permissive License v 1.0 as shown at
 3# https://oss.oracle.com/licenses/upl.
 4
 5import asyncio
 6import time
 7from functools import reduce
 8
 9from coherence import CacheOptions, CacheStats, NamedCache, NearCacheOptions, Session
10
11
12async def do_run() -> None:
13    session: Session = await Session.create()
14
15    # obtain the basic cache and configure near caching with
16    # all defaults; meaning no expiry or pruning
17    cache_remote: NamedCache[str, str] = await session.get_cache("remote")
18    cache_near: NamedCache[str, str] = await session.get_cache(
19        "near", CacheOptions(near_cache_options=NearCacheOptions(ttl=0))
20    )
21
22    await cache_remote.clear()
23    await cache_near.clear()
24    stats: CacheStats = cache_near.near_cache_stats
25
26    # these knobs control:
27    #  - how many current tasks to run
28    #  - how many entries will be inserted and queried
29    #  - how many times the calls will be invoked
30    task_count: int = 25
31    num_entries: int = 1_000
32    iterations: int = 4
33
34    # seed data to populate the cache
35    cache_seed: dict[str, str] = {str(x): str(x) for x in range(num_entries)}
36    cache_seed_keys: set[str] = {key for key in cache_seed.keys()}
37    print()
38
39    # task calling get_all() for 1_000 keys
40    async def get_all_task(task_cache: NamedCache[str, str]) -> int:
41        begin = time.time_ns()
42
43        for _ in range(iterations):
44            async for _ in await task_cache.get_all(cache_seed_keys):
45                continue
46
47        return (time.time_ns() - begin) // 1_000_000
48
49    await cache_remote.put_all(cache_seed)
50    await cache_near.put_all(cache_seed)
51
52    print("Run without near caching ...")
53    begin_outer: int = time.time_ns()
54    results: list[int] = await asyncio.gather(*[get_all_task(cache_remote) for _ in range(task_count)])
55    end_outer: int = time.time_ns()
56    total_time = end_outer - begin_outer
57    task_time = reduce(lambda first, second: first + second, results)
58
59    # Example output
60    # Run without near caching ...
61    # [remote] 25 tasks completed!
62    # [remote] Total time: 4246ms
63    # [remote] Tasks completion average: 3755.6
64
65    print(f"[remote] {task_count} tasks completed!")
66    print(f"[remote] Total time: {total_time // 1_000_000}ms")
67    print(f"[remote] Tasks completion average: {task_time / task_count}")
68
69    print()
70    print("Run with near caching ...")
71    begin_outer = time.time_ns()
72    results = await asyncio.gather(*[get_all_task(cache_near) for _ in range(task_count)])
73    end_outer = time.time_ns()
74    total_time = end_outer - begin_outer
75    task_time = reduce(lambda first, second: first + second, results)
76
77    # Run with near caching ...
78    # [near] 25 tasks completed!
79    # [near] Total time: 122ms
80    # [near] Tasks completion average: 113.96
81    # [near] Near cache statistics: CacheStats(puts=1000, gets=100000, hits=99000,
82    #     misses=1000, misses-duration=73ms, hit-rate=0.99, prunes=0,
83    #     num-pruned=0, prunes-duration=0ms, size=1000, expires=0,
84    #     num-expired=0, expires-duration=0ms, memory-bytes=681464)
85
86    print(f"[near] {task_count} tasks completed!")
87    print(f"[near] Total time: {total_time // 1_000_000}ms")
88    print(f"[near] Tasks completion average: {task_time / task_count}")
89    print(f"[near] Near cache statistics: {stats}")
90
91    await session.close()
92
93
94asyncio.run(do_run())