Hypothesis for the Scientific Stack

numpy

Hypothesis offers a number of strategies for NumPy testing, available in the hypothesis[numpy] extra. It lives in the hypothesis.extra.numpy package.

The centerpiece is the arrays() strategy, which generates arrays with any dtype, shape, and contents you can specify or give a strategy for. To make this as useful as possible, strategies are provided to generate array shapes and generate all kinds of fixed-size or compound dtypes.

hypothesis.extra.numpy.from_dtype(dtype)

Creates a strategy which can generate any value of the given dtype.

hypothesis.extra.numpy.arrays(dtype, shape, elements=None, fill=None, unique=False)

Returns a strategy for generating numpy.ndarrays.

• dtype may be any valid input to dtype (this includes dtype objects), or a strategy that generates such values.
• shape may be an integer >= 0, a tuple of such integers, or a strategy that generates such values.
• elements is a strategy for generating values to put in the array. If it is None a suitable value will be inferred based on the dtype, which may give any legal value (including eg NaN for floats). If you have more specific requirements, you should supply your own elements strategy.
• fill is a strategy that may be used to generate a single background value for the array. If None, a suitable default will be inferred based on the other arguments. If set to nothing() then filling behaviour will be disabled entirely and every element will be generated independently.
• unique specifies if the elements of the array should all be distinct from one another. Note that in this case multiple NaN values may still be allowed. If fill is also set, the only valid values for it to return are NaN values (anything for which numpy.isnan returns True. So e.g. for complex numbers (nan+1j) is also a valid fill). Note that if unique is set to True the generated values must be hashable.

Arrays of specified dtype and shape are generated for example like this:

>>> import numpy as np
>>> arrays(np.int8, (2, 3)).example()
array([[-8,  6,  3],
       [-6,  4,  6]], dtype=int8)

• See What you can generate and how.

>>> import numpy as np
>>> from hypothesis.strategies import floats
>>> arrays(np.float, 3, elements=floats(0, 1)).example()
array([ 0.88974794,  0.77387938,  0.1977879 ])

Array values are generated in two parts:

1. Some subset of the coordinates of the array are populated with a value drawn from the elements strategy (or its inferred form).
2. If any coordinates were not assigned in the previous step, a single value is drawn from the fill strategy and is assigned to all remaining places.

You can set fill to nothing() if you want to disable this behaviour and draw a value for every element.

If fill is set to None then it will attempt to infer the correct behaviour automatically: If unique is True, no filling will occur by default. Otherwise, if it looks safe to reuse the values of elements across multiple coordinates (this will be the case for any inferred strategy, and for most of the builtins, but is not the case for mutable values or strategies built with flatmap, map, composite, etc) then it will use the elements strategy as the fill, else it will default to having no fill.

Having a fill helps Hypothesis craft high quality examples, but its main importance is when the array generated is large: Hypothesis is primarily designed around testing small examples. If you have arrays with hundreds or more elements, having a fill value is essential if you want your tests to run in reasonable time.

hypothesis.extra.numpy.array_shapes(min_dims=1, max_dims=None, min_side=1, max_side=None)

Return a strategy for array shapes (tuples of int >= 1).

hypothesis.extra.numpy.scalar_dtypes()

Return a strategy that can return any non-flexible scalar dtype.

hypothesis.extra.numpy.unsigned_integer_dtypes(endianness='?', sizes=(8, 16, 32, 64))

Return a strategy for unsigned integer dtypes.

endianness may be < for little-endian, > for big-endian, = for native byte order, or ? to allow either byte order. This argument only applies to dtypes of more than one byte.

sizes must be a collection of integer sizes in bits. The default (8, 16, 32, 64) covers the full range of sizes.

hypothesis.extra.numpy.integer_dtypes(endianness='?', sizes=(8, 16, 32, 64))

Return a strategy for signed integer dtypes.

endianness and sizes are treated as for unsigned_integer_dtypes().

hypothesis.extra.numpy.floating_dtypes(endianness='?', sizes=(16, 32, 64))

Return a strategy for floating-point dtypes.

sizes is the size in bits of floating-point number. Some machines support 96- or 128-bit floats, but these are not generated by default.

Larger floats (96 and 128 bit real parts) are not supported on all platforms and therefore disabled by default. To generate these dtypes, include these values in the sizes argument.

hypothesis.extra.numpy.complex_number_dtypes(endianness='?', sizes=(64, 128))

Return a strategy for complex-number dtypes.

sizes is the total size in bits of a complex number, which consists of two floats. Complex halfs (a 16-bit real part) are not supported by numpy and will not be generated by this strategy.

hypothesis.extra.numpy.datetime64_dtypes(max_period='Y', min_period='ns', endianness='?')

Return a strategy for datetime64 dtypes, with various precisions from year to attosecond.

hypothesis.extra.numpy.timedelta64_dtypes(max_period='Y', min_period='ns', endianness='?')

Return a strategy for timedelta64 dtypes, with various precisions from year to attosecond.

hypothesis.extra.numpy.byte_string_dtypes(endianness='?', min_len=1, max_len=16)

Return a strategy for generating bytestring dtypes, of various lengths and byteorder.

While Hypothesis’ string strategies can generate empty strings, string dtypes with length 0 indicate that size is still to be determined, so the minimum length for string dtypes is 1.

hypothesis.extra.numpy.unicode_string_dtypes(endianness='?', min_len=1, max_len=16)

Return a strategy for generating unicode string dtypes, of various lengths and byteorder.

While Hypothesis’ string strategies can generate empty strings, string dtypes with length 0 indicate that size is still to be determined, so the minimum length for string dtypes is 1.

hypothesis.extra.numpy.array_dtypes(subtype_strategy=scalar_dtypes(), min_size=1, max_size=5, allow_subarrays=False)

Return a strategy for generating array (compound) dtypes, with members drawn from the given subtype strategy.

hypothesis.extra.numpy.nested_dtypes(subtype_strategy=scalar_dtypes(), max_leaves=10, max_itemsize=None)

Return the most-general dtype strategy.

Elements drawn from this strategy may be simple (from the subtype_strategy), or several such values drawn from array_dtypes() with allow_subarrays=True. Subdtypes in an array dtype may be nested to any depth, subject to the max_leaves argument.

hypothesis.extra.numpy.valid_tuple_axes(ndim, min_size=0, max_size=None)

Return a strategy for generating permissible tuple-values for the axis argument for a numpy sequential function (e.g. numpy.sum()), given an array of the specified dimensionality.

All tuples will have an length >= min_size and <= max_size. The default value for max_size is ndim.

Examples from this strategy shrink towards an empty tuple, which render most sequential functions as no-ops.

The following are some examples drawn from this strategy.

>>> [valid_tuple_axes(3).example() for i in range(4)]
[(-3, 1), (0, 1, -1), (0, 2), (0, -2, 2)]

valid_tuple_axes can be joined with other strategies to generate any type of valid axis object, i.e. integers, tuples, and None:

any_axis_strategy = none() | integers(-ndim, ndim - 1) | valid_tuple_axes(ndim)

hypothesis.extra.numpy.broadcastable_shapes(shape, min_dims=0, max_dims=None, min_side=1, max_side=None)

Return a strategy for generating shapes that are broadcast-compatible with the provided shape.

Examples from this strategy shrink towards a shape with length min_dims. The size of an aligned dimension shrinks towards size 1. The size of an unaligned dimension shrink towards min_side.

• shape a tuple of integers
• min_dims The smallest length that the generated shape can possess.
• max_dims The largest length that the generated shape can possess. The default-value for max_dims is min(32, max(len(shape), min_dims) + 2).
• min_side The smallest size that an unaligned dimension can possess.
• max_side The largest size that an unaligned dimension can possess. The default value is 2 + ‘size-of-largest-aligned-dimension’.

The following are some examples drawn from this strategy.

>>> [broadcastable_shapes(shape=(2, 3)).example() for i in range(5)]
[(1, 3), (), (2, 3), (2, 1), (4, 1, 3), (3, )]

hypothesis.extra.numpy.mutually_broadcastable_shapes(*, num_shapes=not_set, signature=not_set, base_shape=(), min_dims=0, max_dims=None, min_side=1, max_side=None)

Return a strategy for generating a specified number of shapes, N, that are mutually-broadcastable with one another and with the provided “base-shape”.

The strategy will generate a named-tuple of:

• input_shapes: the N generated shapes
• result_shape: the resulting shape, produced by broadcasting the N shapes with the base-shape

Each shape produced from this strategy shrinks towards a shape with length min_dims. The size of an aligned dimension shrinks towards being having a size of 1. The size of an unaligned dimension shrink towards min_side.

• num_shapes The number of mutually broadcast-compatible shapes to generate.
• base-shape The shape against which all generated shapes can broadcast. The default shape is empty, which corresponds to a scalar and thus does not constrain broadcasting at all.
• min_dims The smallest length that any generated shape can possess.
• max_dims The largest length that any generated shape can possess. It cannot exceed 32, which is the greatest supported dimensionality for a numpy array. The default-value for max_dims is 2 + max(len(shape), min_dims), capped at 32.
• min_side The smallest size that an unaligned dimension can possess.
• max_side The largest size that an unaligned dimension can possess. The default value is 2 + ‘size-of-largest-aligned-dimension’.

The following are some examples drawn from this strategy.

>>> # Draw three shapes, and each shape is broadcast-compatible with `(2, 3)`
>>> for _ in range(5):
...     mutually_broadcastable_shapes(num_shapes=3, base_shape=(2, 3)).example()
BroadcastableShapes(input_shapes=((4, 1, 3), (4, 2, 3), ()), result_shape=(4, 2, 3))
BroadcastableShapes(input_shapes=((3,), (1,), (2, 1)), result_shape=(2, 3))
BroadcastableShapes(input_shapes=((3,), (1, 3), (2, 3)), result_shape=(2, 3))
BroadcastableShapes(input_shapes=((), (), ()), result_shape=(2, 3))
BroadcastableShapes(input_shapes=((3,), (), (3,)), result_shape=(2, 3))

Use with Generalised Universal Function signatures

A universal function (or ufunc for short) is a function that operates on ndarrays in an element-by-element fashion, supporting array broadcasting, type casting, and several other standard features. A generalised ufunc operates on sub-arrays rather than elements, based on the “signature” of the function. Compare e.g. numpy.add (ufunc) to numpy.matmul (gufunc).

To generate shapes for a gufunc, you can pass the signature argument instead of num_shapes. This must be a gufunc signature string; which you can write by hand or access as e.g. np.matmul.signature on generalised ufuncs.

In this case, the side arguments are applied to the ‘core dimensions’ as well, ignoring any frozen dimensions. base_shape and the dims arguments are applied to the ‘loop dimensions’, and if necessary, the dimensionality of each shape is silently capped to respect the 32-dimension limit.

The generated result_shape is the real result shape of applying the gufunc to arrays of the generated input_shapes, even where this is different to broadcasting the loop dimensions.

gufunc-compatible shapes shrink their loop dimensions as above, towards omitting optional core dimensions, and smaller-size core dimensions.

>>> # np.matmul.signature == "(m?,n),(n,p?)->(m?,p?)"
>>> for _ in range(3):
...     mutually_broadcastable_shapes(signature=np.matmul.signature).example()
BroadcastableShapes(input_shapes=((2,), (2,)), result_shape=())
BroadcastableShapes(input_shapes=((3, 4, 2), (1, 2)), result_shape=(3, 4))
BroadcastableShapes(input_shapes=((4, 2), (1, 2, 3)), result_shape=(4, 3))

hypothesis.extra.numpy.basic_indices(shape, *, min_dims=0, max_dims=None, allow_newaxis=False, allow_ellipsis=True)

The basic_indices strategy generates basic indexes for arrays of the specified shape, which may include dimensions of size zero.

It generates tuples containing some mix of integers, slice objects, ... (Ellipsis), and numpy.newaxis; which when used to index a shape-shaped array will produce either a scalar or a shared-memory view.

• shape: the array shape that will be indexed, as a tuple of integers >= 0. This must be at least two-dimensional for a tuple to be a valid basic index; for one-dimensional arrays use slices() instead.
• min_dims: the minimum dimensionality of the resulting view from use of the generated index. When min_dims == 0, scalars and zero-dimensional arrays are both allowed.
• max_dims: the maximum dimensionality of the resulting view. If not specified, it defaults to max(len(shape), min_dims) + 2.
• allow_ellipsis: whether ...` is allowed in the index.
• allow_newaxis: whether numpy.newaxis is allowed in the index.

Note that the length of the generated tuple may be anywhere between zero and min_dims. It may not match the length of shape, or even the dimensionality of the array view resulting from its use!

hypothesis.extra.numpy.integer_array_indices(shape, result_shape=array_shapes(), dtype='int')

Return a search strategy for tuples of integer-arrays that, when used to index into an array of shape shape, given an array whose shape was drawn from result_shape.

Examples from this strategy shrink towards the tuple of index-arrays:

len(shape) * (np.zeros(drawn_result_shape, dtype), )

• shape a tuple of integers that indicates the shape of the array, whose indices are being generated.
• result_shape a strategy for generating tuples of integers, which describe the shape of the resulting index arrays. The default is array_shapes(). The shape drawn from this strategy determines the shape of the array that will be produced when the corresponding example from integer_array_indices is used as an index.
• dtype the integer data type of the generated index-arrays. Negative integer indices can be generated if a signed integer type is specified.

Recall that an array can be indexed using a tuple of integer-arrays to access its members in an arbitrary order, producing an array with an arbitrary shape. For example:

>>> from numpy import array
>>> x = array([-0, -1, -2, -3, -4])
>>> ind = (array([[4, 0], [0, 1]]),)  # a tuple containing a 2D integer-array
>>> x[ind]  # the resulting array is commensurate with the indexing array(s)
array([[-4,  0],
       [0, -1]])

Note that this strategy does not accommodate all variations of so-called ‘advanced indexing’, as prescribed by NumPy’s nomenclature. Combinations of basic and advanced indexes are too complex to usefully define in a standard strategy; we leave application-specific strategies to the user. Advanced-boolean indexing can be defined as arrays(shape=..., dtype=bool), and is similarly left to the user.

pandas

Hypothesis provides strategies for several of the core pandas data types: pandas.Index, pandas.Series and pandas.DataFrame.

The general approach taken by the pandas module is that there are multiple strategies for generating indexes, and all of the other strategies take the number of entries they contain from their index strategy (with sensible defaults). So e.g. a Series is specified by specifying its numpy.dtype (and/or a strategy for generating elements for it).

hypothesis.extra.pandas.indexes(elements=None, dtype=None, min_size=0, max_size=None, unique=True)

Provides a strategy for producing a pandas.Index.

Arguments:

• elements is a strategy which will be used to generate the individual values of the index. If None, it will be inferred from the dtype. Note: even if the elements strategy produces tuples, the generated value will not be a MultiIndex, but instead be a normal index whose elements are tuples.
• dtype is the dtype of the resulting index. If None, it will be inferred from the elements strategy. At least one of dtype or elements must be provided.
• min_size is the minimum number of elements in the index.
• max_size is the maximum number of elements in the index. If None then it will default to a suitable small size. If you want larger indexes you should pass a max_size explicitly.
• unique specifies whether all of the elements in the resulting index should be distinct.

hypothesis.extra.pandas.range_indexes(min_size=0, max_size=None)

Provides a strategy which generates an Index whose values are 0, 1, …, n for some n.

Arguments:

• min_size is the smallest number of elements the index can have.
• max_size is the largest number of elements the index can have. If None it will default to some suitable value based on min_size.

hypothesis.extra.pandas.series(elements=None, dtype=None, index=None, fill=None, unique=False)

Provides a strategy for producing a pandas.Series.

Arguments:

• elements: a strategy that will be used to generate the individual values in the series. If None, we will attempt to infer a suitable default from the dtype.

• dtype: the dtype of the resulting series and may be any value that can be passed to numpy.dtype. If None, will use pandas’s standard behaviour to infer it from the type of the elements values. Note that if the type of values that comes out of your elements strategy varies, then so will the resulting dtype of the series.

• index: If not None, a strategy for generating indexes for the resulting Series. This can generate either pandas.Index objects or any sequence of values (which will be passed to the Index constructor).

  You will probably find it most convenient to use the indexes() or range_indexes() function to produce values for this argument.

Usage:

>>> series(dtype=int).example()
0   -2001747478
1    1153062837

class hypothesis.extra.pandas.column(name=None, elements=None, dtype=None, fill=None, unique=False)

Data object for describing a column in a DataFrame.

Arguments:

• name: the column name, or None to default to the column position. Must be hashable, but can otherwise be any value supported as a pandas column name.
• elements: the strategy for generating values in this column, or None to infer it from the dtype.
• dtype: the dtype of the column, or None to infer it from the element strategy. At least one of dtype or elements must be provided.
• fill: A default value for elements of the column. See arrays() for a full explanation.
• unique: If all values in this column should be distinct.

hypothesis.extra.pandas.columns(names_or_number, dtype=None, elements=None, fill=None, unique=False)

A convenience function for producing a list of column objects of the same general shape.

The names_or_number argument is either a sequence of values, the elements of which will be used as the name for individual column objects, or a number, in which case that many unnamed columns will be created. All other arguments are passed through verbatim to create the columns.

hypothesis.extra.pandas.data_frames(columns=None, rows=None, index=None)

Provides a strategy for producing a pandas.DataFrame.

Arguments:

• columns: An iterable of column objects describing the shape of the generated DataFrame.

• rows: A strategy for generating a row object. Should generate either dicts mapping column names to values or a sequence mapping column position to the value in that position (note that unlike the pandas.DataFrame constructor, single values are not allowed here. Passing e.g. an integer is an error, even if there is only one column).

  At least one of rows and columns must be provided. If both are provided then the generated rows will be validated against the columns and an error will be raised if they don’t match.

  Caveats on using rows:

  • In general you should prefer using columns to rows, and only use rows if the columns interface is insufficiently flexible to describe what you need - you will get better performance and example quality that way.
  • If you provide rows and not columns, then the shape and dtype of the resulting DataFrame may vary. e.g. if you have a mix of int and float in the values for one column in your row entries, the column will sometimes have an integral dtype and sometimes a float.

• index: If not None, a strategy for generating indexes for the resulting DataFrame. This can generate either pandas.Index objects or any sequence of values (which will be passed to the Index constructor).

  You will probably find it most convenient to use the indexes() or range_indexes() function to produce values for this argument.

Usage:

The expected usage pattern is that you use column and columns() to specify a fixed shape of the DataFrame you want as follows. For example the following gives a two column data frame:

>>> from hypothesis.extra.pandas import column, data_frames
>>> data_frames([
... column('A', dtype=int), column('B', dtype=float)]).example()
            A              B
0  2021915903  1.793898e+232
1  1146643993            inf
2 -2096165693   1.000000e+07

If you want the values in different columns to interact in some way you can use the rows argument. For example the following gives a two column DataFrame where the value in the first column is always at most the value in the second:

>>> from hypothesis.extra.pandas import column, data_frames
>>> import hypothesis.strategies as st
>>> data_frames(
...     rows=st.tuples(st.floats(allow_nan=False),
...                    st.floats(allow_nan=False)).map(sorted)
... ).example()
               0             1
0  -3.402823e+38  9.007199e+15
1 -1.562796e-298  5.000000e-01

You can also combine the two:

>>> from hypothesis.extra.pandas import columns, data_frames
>>> import hypothesis.strategies as st
>>> data_frames(
...     columns=columns(["lo", "hi"], dtype=float),
...     rows=st.tuples(st.floats(allow_nan=False),
...                    st.floats(allow_nan=False)).map(sorted)
... ).example()
         lo            hi
0   9.314723e-49  4.353037e+45
1  -9.999900e-01  1.000000e+07
2 -2.152861e+134 -1.069317e-73

(Note that the column dtype must still be specified and will not be inferred from the rows. This restriction may be lifted in future).

Combining rows and columns has the following behaviour:

• The column names and dtypes will be used.
• If the column is required to be unique, this will be enforced.
• Any values missing from the generated rows will be provided using the column’s fill.
• Any values in the row not present in the column specification (if dicts are passed, if there are keys with no corresponding column name, if sequences are passed if there are too many items) will result in InvalidArgument being raised.

Supported Versions

There is quite a lot of variation between pandas versions. We only commit to supporting the latest version of pandas, but older minor versions are supported on a “best effort” basis. Hypothesis is currently tested against and confirmed working with Pandas 0.19, 0.20, 0.21, 0.22, and 0.23.

Releases that are not the latest patch release of their minor version are not tested or officially supported, but will probably also work unless you hit a pandas bug.