module: context_vars_registry

This is documentation page for the module: contextvars_registry.context_vars_registry

The module is about class ContextVarsRegistry - a container that provides nice @property-like access to context variables.

API summary 

class ContextVarsRegistry

`ContextVarsRegistry._registry_allocate_on_setattr`	Automatically create new context variables when setting attributes?
`ContextVarsRegistry.__call__`(...)	Set attributes temporarily, using context manager (the `with` statement in Python).

Functions

`restore_context_vars_registry`(registry, ...)	Restore ContextVarsRegistry state from dict.
`save_context_vars_registry`(registry)	Dump variables from ContextVarsRegistry as dict.

Exceptions

`RegistryInheritanceError`(args, *kwargs)	Class ContextVarsRegistry must be subclassed, and only one level deep.
`SetClassVarAttributeError`(args, *kwargs)	Can't set ClassVar: '{class_name}.{attr_name}'.

class ContextVarsRegistry 

ContextVarsRegistry is a container that makes context variables behave like @property.

The idea is simple: you create a sub-class, and its attributes magically become context variables:

>>> from contextvars_registry import ContextVarsRegistry

>>> class CurrentVars(ContextVarsRegistry):
...    locale: str = "en"
...    timezone: str = "UTC"
...    user_id: int = None
...    db_session: object

>>> current = CurrentVars()

and then you can work with context variables by just getting/setting the attributes:

>>> current.timezone  # ContextVar.get() is called under the hood
'UTC'

>>> current.timezone = "GMT"  # ContextVar.set() is called under the hood
>>> current.timezone
'GMT'

The underlying ContextVar methods can be reached via class members:

>>> CurrentVars.timezone.get()
'GMT'

>>> token = CurrentVars.timezone.set("Europe/London")
>>> current.timezone
'Europe/London'
>>> CurrentVars.timezone.reset(token)
>>> current.timezone
'GMT'

An important thing to understand is that ContextVarsRegistry is not a classic Python object. It doesn’t have any instance state, and you cannot really mutate its attributes. All its attributes are “virtual” proxies to context variables.

That is, when you’re setting an attribute like this:

current.timezone = "GMT"

Under the hood it really turns into this:

CurrentVars.timezone.set("GMT")

The CurrentVars.timezone above is a magic ContextVarDescriptor object, check this out:

>>> CurrentVars.timezone
<ContextVarDescriptor name='__main__.CurrentVars.timezone'...>

Such ContextVarDescriptor is an extended version of the standard contextvars.ContextVar that behaves like @property when you put it into a class.

Another important note is that ContextVarDescriptor is NOT a subclass of ContextVar. It should have been done a subclass, but unfortunately, Python’s ContextVar cannot be subclassed (a technical limitation), so ContextVarDescriptor is made a wrapper for ContextVar.

If you really need to reach the lower-level ContextVar object, then you just use the .context_var attribute, like this:

>>> CurrentVars.timezone.context_var
<ContextVar name='__main__.CurrentVars.timezone'...>

But in most cases, you don’t need it, because ContextVarDescriptor implements all the same methods and attributes as the standard ContextVar, so it should work as a drop-in replacement in all cases except isinstance() checks.

In addition, ContextVarDescriptor provides some extension methods (not available in the standard ContextVar):

>>> CurrentVars.timezone.is_set()
True

>>> CurrentVars.timezone.delete()

>>> CurrentVars.timezone.is_set()
False

The list of available methods can be found here: module: context_var_descriptor

Attribute Allocation 

As mentioned above, all registry attributes must be descriptors, so when you set (or even just declare) an attribute, then ContextVarsRegistry automatically allocates a new ContextVarDescriptor for each attribute.

But, a little problem is that not all attributes should become context variables. For example, you may want to define some methods in your registry subclass, and then you probably expect methods to remain methods (not converted to context variables).

In most cases it automatically does the right thing, so you don’t need to change anything, but still, there are ways to alter automatic variable allocation, and there are special cases that worth knowing about, so the section below describes the variable allocation procedure in detail.

There are 4 ways to allocate variables in a registry:

by type annotation (recommended)
by value
dynamic
manual creation of ContextVarDescriptor()

by type annotation 

For attributes with type hints the rules are simple and explicit:

if you add ClassVar, then attribute is skipped

otherwise it is converted to context variable

Example:

>>> from typing import ClassVar

>>> class CurrentVars(ContextVarsRegistry):
...     some_registry_setting: ClassVar[str] = "not a context variable"
...
...     user_id: int
...     timezone: str = "UTC"

>>> CurrentVars.some_registry_setting
'not a context variable'

>>> CurrentVars.user_id
<ContextVarDescriptor name='__main__.CurrentVars.user_id'>

>>> CurrentVars.timezone
<ContextVarDescriptor name='__main__.CurrentVars.timezone'>

Because rules for type annotations are so simple and explicit, this is the recommended way to go.

by value 

Without type hints, things become a bit more complicated:
- skipped:
  - methods (regular functions defined via def)
  - @property (and other kinds of descriptors)
  - special attributes (like __doc__)
- all other values are converted to context variables (including: Lambdas, partial() and custom callable() objects - they’re all converted to context variables)

Example:

>>> from functools import partial

>>> class CurrentVars(ContextVarsRegistry):
...     # All regular attributes are converted to context variables
...     # (even "private" attributes are converted!).
...     var1 = "var1 default value"
...     _var2 = "var2 default value"
...     __var3 = "var3 default value"
...
...     # special attributes are skipped
...     __special__ = "special attribute"
...
...     # properties are skipped
...     @property
...     def some_property(self):
...         return self.__var3
...
...     # Methods are skipped.
...     def some_method(self):
...         return self.__var3
...
...     # BUT: lambda/partial functions are converted to context variables!
...     some_lambda = lambda self: self.var1
...     some_partial = partial(some_method)

# All regular attributes are converted to context variables.
>>> CurrentVars.var1
<ContextVarDescriptor ...>

# Even "private" attributes are converted.
>>> CurrentVars._CurrentVars__var3
<ContextVarDescriptor ...>

# @properties are skipped
>>> CurrentVars.some_property
<property object ...>

# Methods are skipped.
>>> CurrentVars.some_method
<function CurrentVars.some_method ...>

# BUT: lambda functions are converted!
>>> CurrentVars.some_lambda
<ContextVarDescriptor ...>

# partial() objects are also converted
>>> CurrentVars.some_partial
<ContextVarDescriptor ...>

So, as you can see, without type annotations rules become somewhat magic, sometimes even fragile. Like, for example, you may apply a decorator to your method, and the decorator returns a partial() object, and then your method suddenly becomes a ContextVarDescriptor, which wasn’t your intent.

To avoid such surprises, just use type hints. They make things safe and explicit.

dynamic 

Registry automatically allocates new descriptors on the fly when setting attributes, check this out:

>>> class CurrentVars(ContextVarsRegistry):
...     pass

current = CurrentVars()

current.timezone = "UTC"

CurrentVars.timezone
<ContextVarDescriptor name='__main__.CurrentVars.timezone'>

That means that you can start with an empty registry subclass, and then just set variables as needed.

This feature is akin to the famous flask.g object, where you set context-dependent global variables like this:

from flask import g

g.timezone = "UTC"

So now you can do a similar thing with pure context variables (not depending on Flask):

>>> class GlobalVars(ContextVarsRegistry):
...     pass

>>> g = GlobalVars()

>>> g.timezone = "UTC"

The dynamic allocation is on by default, but you can turn it off by overriding the ContextVarsRegistry._registry_allocate_on_setattr attribute, like this:

>>> class CurrentVars(ContextVarsRegistry):
...     _registry_allocate_on_setattr: ClassVar[bool] = False

>>> current = CurrentVars()

>>> current.timezone = "UTC"
Traceback (most recent call last):
...
AttributeError: 'CurrentVars' object has no attribute 'timezone'

manual creation of ContextVarDescriptor()

You can also just manually create ContextVarDescriptor objects, like this:

>>> from contextvars_registry import ContextVarDescriptor

>>> class CurrentVars(ContextVarsRegistry):
...     timezone = ContextVarDescriptor(deferred_default=lambda: "UTC")

This is useful when you need to pass some extended constructor arguments, like the deferred_default in the example above.

`@property` support 

ContextVarsRegistry supports classic @property, which is useful for adding extra validation or data transformation when setting variables.

Here is an example of how @property can be used to validate timezone names, and automatically convert them to datetime.tzinfo objects using pytz package:

>>> import pytz
>>> from datetime import tzinfo

>>> class CurrentVars(ContextVarsRegistry):
...     # a "private" context variable that stores the current timezone setting
...     _timezone: tzinfo = "UTC"
...
...     @property
...     def timezone(self):
...         return self._timezone
...
...     @timezone.setter
...     def timezone(self, new_timezone):
...         assert isinstance(new_timezone, tzinfo)
...         self._timezone = new_timezone
...
...     @timezone.deleter
...     def timezone(self):
...         del self._timezone
...
...     @property
...     def timezone_name(self):
...          return self._timezone.zone
...
...     @timezone_name.setter
...     def timezone_name(self, new_timezone_name):
...         assert isinstance(new_timezone_name, str)
...         self._timezone = pytz.timezone(new_timezone_name)
...
...     @timezone_name.deleter
...     def timezone_name(self):
...         del self._timezone

>>> current = CurrentVars()

>>> current.timezone_name = "GMT"

>>> current.timezone
<StaticTzInfo 'GMT'>

`with` Syntax for Setting Attributes 

ContextVarsRegistry can act as a context manager, that allows you to set attributes temporarily, like this:

>>> class CurrentVars(ContextVarsRegistry):
...     locale: str =  "en"
...     timezone: str = "UTC"

>>> current = CurrentVars()

>>> with current(locale="en_GB", timezone="GMT"):
...     print(current.locale)
...     print(current.timezone)
en_GB
GMT

Upon exit from the with block, the attributes are reset to their previous values.

But, keep in mind that it doesn’t restore state of the whole registry. It is only a small syntax sugar over setting attributes, and it restores only attributes that are listed inside the inside the with() parenthesizes, and nothing else.

If you need a full context isolation mechanism, then you should use tools from the context_management module.

Deleting Attributes 

In Python, it is not possible to delete a ContextVar object. (an attempt to do so causes a memory leak, so you shall never really delete context variables).

So, we have to do some trickery to implement deletion…

When you call del or delattr(), we don’t actually delete anything, but instead we write to the variable a special sentinel object called DELETED.

Later on, when the variable is read, there is a if check under the hood, that detects the special sentinel object, and throws an exception.

On the high level, you should never notice this trick. Attribute mechanics works like for a normal Python object, as if its attribute was really deleted, check this out:

>>> class CurrentVars(ContextVarsRegistry):
...    user_id: int = None

>>> current =  CurrentVars()

>>> hasattr(current, 'user_id')
True

>>> delattr(current, 'user_id')

>>> hasattr(current, 'user_id')
False

>>> try:
...     current.user_id
... except AttributeError:
...     print("AttributeError raised")
... else:
...     print("not raised")
AttributeError raised

>>> getattr(current, 'user_id', 'DEFAULT_VALUE')
'DEFAULT_VALUE'

The only case when you see this special DELETED object is when you use some low-level stuff, like save_context_vars_registry(), or the get_raw() method:

>>> CurrentVars.user_id.get_raw()
<DeletionMark.DELETED: 'DELETED'>

So, long story short: once a contextvars.ContextVar object is allocated, it lives forever in the registry. When you delete it, we only mark it as deleted, but never actually delete it. All this thing happens under the hood, and normally you shouln’t notice it.

dict-like Access 

ContextVarsRegistry implements collections.abc.MutableMapping protocol.

That means that you can get/set context variables, as if it was just a dict, like this:

>>> class CurrentVars(ContextVarsRegistry):
...    locale: str = 'en'
...    timezone: str = 'UTC'
...    user_id: int = None

>>> current = CurrentVars()

>>> current['locale'] = 'en_US'
>>> current['locale']
'en_US'

Standard dict operators are supported:

# `in` operator
>>> 'locale' in current
True

# count variables in the dict
>>> len(current)
3

# iterate over keys in the dict
>>> for key in current:
...     print(key)
locale
timezone
user_id

# convert to dict() easily
>>> dict(current)
{'locale': 'en_US', 'timezone': 'UTC', 'user_id': None}

Methods are supported as well:

>>> current.update({
...    'locale': 'en',
...    'timezone': 'UTC',
...    'user_id': 42
... })

>>> list(current.keys())
['locale', 'timezone', 'user_id']

>>> list(current.values())
['en', 'UTC', 42]

>>> current.pop('locale')
'en'

>>> list(current.items())
[('timezone', 'UTC'), ('user_id', 42)]

API reference 

class ContextVarsRegistryMeta(name, bases, attrs)[source]

Metaclass for ContextVarsRegistry and its subclasses.

It automatically adds empty __slots__ to all registry classes.

This metaclass adds empty __slots__ to ContextVarsRegistry and all its subclasses, which means that you can’t set any attributes on a registry instance.

Why? Because a registry doesn’t have any real attributes. It only acts as a proxy, forwarding operations to context variables (which are hosted in the registry class, not in the instance).

Setting regular (non-context-variable) attributes on an instance would almost always lead to nasty race conditions (bugs that occur in production, but not in tests).

To avoid that, we set __slots__ = tuple() for all registry classes, thus ensuring that all the state is stored in context variables.

class ContextVarsRegistry[source]

A collection of ContextVar() objects, with nice @property-like way to access them.

_registry_allocate_on_setattr: ClassVar[bool] = True

Automatically create new context variables when setting attributes?

If set to True (default), missing ContextVar() objects are dynamically allocated when setting attributes. That is, you can define an empty class, and then set arbitrary attributes:

>>> class CurrentVars(ContextVarsRegistry):
...    pass

>>> current = CurrentVars()
>>> current.locale = 'en'
>>> current.timezone = 'UTC'

However, if you find this behavior weak, you may disable it, like this:

>>> class CurrentVars(ContextVarsRegistry):
...     _registry_allocate_on_setattr = False
...     locale: str = 'en'

>>> current = CurrentVars()
>>> current.timezone = 'UTC'
AttributeError: ...

__call__(**attr_names_and_values)[source]

Set attributes temporarily, using context manager (the with statement in Python).

Example of usage:

>>> class CurrentVars(ContextVarsRegistry):
...     timezone: str = 'UTC'
...     locale: str = 'en'
>>> current = CurrentVars()

>>> with current(timezone='Europe/London', locale='en_GB'):
...    print(current.timezone)
...    print(current.locale)
Europe/London
en_GB

On exiting form the with block, the values are restored:

>>> current.timezone
'UTC'
>>> current.locale
'en'

Caution

Only attributes that are present inside with (...) parenthesis are restored:

>>> with current(timezone='Europe/London'):
...   current.locale = 'en_GB'
...   current.user_id = 42
>>> current.timezone  # restored
'UTC'
>>> current.locale  # not restored
'en_GB'
>>> current.user_id  # not restored
42

That is, the with current(...) doesn’t make a full copy of all context variables. It is NOT a scope isolation mechanism that protects all attributes.

It is a more primitive tool, roughly, a syntax sugar for this:

>>> try:
...     saved_timezone = current.timezone
...     current.timezone = 'Europe/London'
...
...     # do_something_useful_with_current_timezone()
... finally:
...     current.timezone = saved_timezone

save_context_vars_registry(registry)[source]

Dump variables from ContextVarsRegistry as dict.

The resulting dict can be used as argument to restore_context_vars_registry().

Parameters:: registry (ContextVarsRegistry) –
Return type:: Dict[str, Any]

restore_context_vars_registry(registry, saved_registry_state)[source]

Restore ContextVarsRegistry state from dict.

The restore_context_vars_registry() restores state of ContextVarsRegistry, using state that was previously dumped by save_context_vars_registry().

Parameters:

registry (ContextVarsRegistry) – a ContextVarsRegistry instance that will be written
saved_registry_state (Dict[str, Any]) – output of save_context_vars_registry() function

Example:

>>> from contextvars_registry.context_vars_registry import (
...    ContextVarsRegistry,
...    save_context_vars_registry,
...    restore_context_vars_registry,
... )

>>> class CurrentVars(ContextVarsRegistry):
...     locale: str = 'en'
...     timezone: str = 'UTC'

>>> current = CurrentVars()
>>> state1 = save_context_vars_registry(current)

>>> current.locale = 'en_US'
>>> current.timezone = 'America/New York'
>>> state2 = save_context_vars_registry(current)

>>> del current.locale
>>> del current.timezone
>>> current.user_id = 42
>>> state3 = save_context_vars_registry(current)

>>> restore_context_vars_registry(current, state1)
>>> dict(current)
{'locale': 'en', 'timezone': 'UTC'}

>>> restore_context_vars_registry(current, state2)
>>> dict(current)
{'locale': 'en_US', 'timezone': 'America/New York'}

>>> restore_context_vars_registry(current, state3)
>>> dict(current)
{'user_id': 42}

A similar result could be achieved by the standard collections.abc.MutableMapping methods, like registry.clear() and registry.update(), but this is not exactly the same. There is still a couple of differences:

save_registry_state() and restore_registry_state() can handle special cases, like DELETED tokens, or lazy initializers.

collections.abc.MutableMapping methods are slow, while save_registry_state() and restore_registry_state() are faster, since they can reach some registry internals directly, avoiding complex methods.

Note

This function is not scalable, it takes O(N) time, where N is the number of variables in the registry.

There is a faster tool, a decorator that saves/restores all context variables on each call, and that takes O(1) time: contextvars_registry.context.bind_to_sandbox_context()

So you prefer that decorator by default, and choose restore_registry_state() only when you can’t use the decorator, or when you need to restore only 1 specific registry, not touching variables outside of the registry.

exception RegistryInheritanceError(*args, **kwargs)[source]

Class ContextVarsRegistry must be subclassed, and only one level deep.

This exception is raised in 2 cases:

When you use ContextVarsRegistry directly, without subclassing:
```
instance = ContextVarsRegistry()
```

When you create a sub-sub-class of ContextVarsRegistry:

class SubRegistry(ContextVarsRegistry):
    pass

class SubSubRegistry(ContextVarsRegistry):
    pass

These limitations are caused by the way we store ContextVar objects on class attributes. Setting a context variable on a base class pollutes all its sub-classes, and setting a variable on sub-class shadows attribute of the base class. Things become messy quickly, so we ensure you define subclasses in a right way.

So, the proper way is to just define a subclass (but not sub-sub-class), and then use it, like this:

class CurrentVars(ContextVarsRegistry):
    var1: str = "default_value"

current = CurrentVars()
current.var1   # => "default_value"

Note

Actually, that could be done in a smarter way: what we really want is to make sure that ContextVar objects are always stored in the leafs of class hierarchy. So, we could forbid subclassing if a class has context variables, and also forbid setting variables on a class that has subclasses, and that should solve the problem.

But, that could add some complexity, so to keep things simple, we just ban deep inheritance. At least for now (may be implemented in the future, or maybe not).

exception SetClassVarAttributeError(*args, **kwargs)[source]

Can’t set ClassVar: ‘{class_name}.{attr_name}’.

This exception is raised when an attribute is declared as typing.ClassVar, like this:

class {class_name}(ContextVarsRegistry):
    {attr_name}: ClassVar[...]

…but you’re trying to set it on instance level, as if it was a context variable.

To solve the issue, you need to either:

Remove ClassVar annotation (and thus convert the attribute to a context variable).
Set the attribute off the class (not instance), like this:
```
{class_name}.{attr_name} = ...
```