prefect.utilities.hashing

file_hash

Given a path to a file, produces a stable hash of the file contents.

Parameters:

Name	Description	Default
path	the path to a file str	required
hash_algo	Hash algorithm from hashlib to use.	<built-in function openssl_md5>

Returns:

Type	Description
str	a hash of the file contents

Source code in prefect/utilities/hashing.py

def file_hash(path: str, hash_algo=hashlib.md5) -> str:
    """Given a path to a file, produces a stable hash of the file contents.

    Args:
        path (str): the path to a file
        hash_algo: Hash algorithm from hashlib to use.

    Returns:
        str: a hash of the file contents
    """
    contents = Path(path).read_bytes()
    return stable_hash(contents, hash_algo=hash_algo)

hash_objects

Attempt to hash objects by dumping to JSON or serializing with cloudpickle. On failure of both, None will be returned

Source code in prefect/utilities/hashing.py

def hash_objects(*args, hash_algo=hashlib.md5, **kwargs) -> Optional[str]:
    """
    Attempt to hash objects by dumping to JSON or serializing with cloudpickle.
    On failure of both, `None` will be returned
    """
    try:
        return stable_hash(
            json.dumps((args, kwargs), sort_keys=True), hash_algo=hash_algo
        )
    except Exception:
        pass

    try:
        return stable_hash(cloudpickle.dumps((args, kwargs)), hash_algo=hash_algo)
    except Exception:
        pass

    return None

stable_hash

Given some arguments, produces a stable 64-bit hash of their contents.

Supports bytes and strings. Strings will be UTF-8 encoded.

Parameters:

Name	Description	Default
*args	Items to include in the hash. Union[str, bytes]	()
hash_algo	Hash algorithm from hashlib to use.	<built-in function openssl_md5>

Returns:

Type	Description
str	A hex hash.

Source code in prefect/utilities/hashing.py

def stable_hash(*args: Union[str, bytes], hash_algo=hashlib.md5) -> str:
    """Given some arguments, produces a stable 64-bit hash of their contents.

    Supports bytes and strings. Strings will be UTF-8 encoded.

    Args:
        *args: Items to include in the hash.
        hash_algo: Hash algorithm from hashlib to use.

    Returns:
        A hex hash.
    """
    h = hash_algo()
    for a in args:
        if isinstance(a, str):
            a = a.encode()
        h.update(a)
    return h.hexdigest()