Streams

What are streams

A stream means an object that provides the signature or log-signature over any interval. For more information on streams, see Lyons and McLeod [LM24] and Lyons et al. [LCL07].

How do streams fit into RoughPy

Streams are parametrised sequential data viewed via Rough Path theory as a rough path.

How to work with streams

To create a stream, you need data. For example, we can create data using a linearly spaced time array.

>>> times = np.linspace(0.1,1,10)
>>> times2 = 2*times
>>> data = np.concatenate([times.reshape(-1,1),
                           times2.reshape(-1,1),
                           (times*d2t - 2*times*dt).reshape(-1,1)], axis=1)

To create a stream, we should also provide a context. For more information, see Contexts.

For example, we can create the following context, with Width 2, Depth 6, and Real coefficients. The depth specification for constructing the stream is not the depth of the data, but it is the depth of the stream. The depth of the data must be less than or equal to the depth of the stream.

>>> context = roughpy.get_context(width=2, depth=6, coeffs=roughpy.DPReal)

You can then create a stream using your chosen data and context with Lie increments.

>>> stream = roughpy.LieIncrementStream.from_increments(data, indices=times, ctx=context)

You can also compute and display the signature and log signature of any stream.

>>> sig = stream.signature(interval)
>>> print(f"{sig=!s}")
sig={ 1() 4.5(1) 9(2) 10.125(1,1) 20.25(1,2) 20.25(2,1) 40.5(2,2) 15.1875(1,1,1) 30.375(1,1,2) 30.375(1,2,1) 60.75(1,2,2) 30.375(2,1,1) 60.75(2,1,2) 60.75(2,2,1) 121.5(2,2,2) 17.0859(1,1,1,1) 34.1719(1,1,1,2) 34.1719(1,1,2,1) 68.3438(1,1,2,2) 34.1719(1,2,1,1) 68.3438(1,2,1,2) 68.3438(1,2,2,1) 136.688(1,2,2,2) 34.1719(2,1,1,1) 68.3438(2,1,1,2) 68.3438(2,1,2,1) 136.688(2,1,2,2) 68.3438(2,2,1,1) 136.688(2,2,1,2) 136.688(2,2,2,1) 273.375(2,2,2,2) 15.3773(1,1,1,1,1) 30.7547(1,1,1,1,2) 30.7547(1,1,1,2,1) 61.5094(1,1,1,2,2) 30.7547(1,1,2,1,1) 61.5094(1,1,2,1,2) 61.5094(1,1,2,2,1) 123.019(1,1,2,2,2) 30.7547(1,2,1,1,1) 61.5094(1,2,1,1,2) 61.5094(1,2,1,2,1) 123.019(1,2,1,2,2) 61.5094(1,2,2,1,1) 123.019(1,2,2,1,2) 123.019(1,2,2,2,1) 246.038(1,2,2,2,2) 30.7547(2,1,1,1,1) 61.5094(2,1,1,1,2) 61.5094(2,1,1,2,1) 123.019(2,1,1,2,2) 61.5094(2,1,2,1,1) 123.019(2,1,2,1,2) 123.019(2,1,2,2,1) 246.038(2,1,2,2,2) 61.5094(2,2,1,1,1) 123.019(2,2,1,1,2) 123.019(2,2,1,2,1) 246.038(2,2,1,2,2) 123.019(2,2,2,1,1) 246.038(2,2,2,1,2) 246.038(2,2,2,2,1) 492.075(2,2,2,2,2) 11.533(1,1,1,1,1,1) 23.066(1,1,1,1,1,2) 23.066(1,1,1,1,2,1) 46.132(1,1,1,1,2,2) 23.066(1,1,1,2,1,1) 46.132(1,1,1,2,1,2) 46.132(1,1,1,2,2,1) 92.2641(1,1,1,2,2,2) 23.066(1,1,2,1,1,1) 46.132(1,1,2,1,1,2) 46.132(1,1,2,1,2,1) 92.2641(1,1,2,1,2,2) 46.132(1,1,2,2,1,1) 92.2641(1,1,2,2,1,2) 92.2641(1,1,2,2,2,1) 184.528(1,1,2,2,2,2) 23.066(1,2,1,1,1,1) 46.132(1,2,1,1,1,2) 46.132(1,2,1,1,2,1) 92.2641(1,2,1,1,2,2) 46.132(1,2,1,2,1,1) 92.2641(1,2,1,2,1,2) 92.2641(1,2,1,2,2,1) 184.528(1,2,1,2,2,2) 46.132(1,2,2,1,1,1) 92.2641(1,2,2,1,1,2) 92.2641(1,2,2,1,2,1) 184.528(1,2,2,1,2,2) 92.2641(1,2,2,2,1,1) 184.528(1,2,2,2,1,2) 184.528(1,2,2,2,2,1) 369.056(1,2,2,2,2,2) 23.066(2,1,1,1,1,1) 46.132(2,1,1,1,1,2) 46.132(2,1,1,1,2,1) 92.2641(2,1,1,1,2,2) 46.132(2,1,1,2,1,1) 92.2641(2,1,1,2,1,2) 92.2641(2,1,1,2,2,1) 184.528(2,1,1,2,2,2) 46.132(2,1,2,1,1,1) 92.2641(2,1,2,1,1,2) 92.2641(2,1,2,1,2,1) 184.528(2,1,2,1,2,2) 92.2641(2,1,2,2,1,1) 184.528(2,1,2,2,1,2) 184.528(2,1,2,2,2,1) 369.056(2,1,2,2,2,2) 46.132(2,2,1,1,1,1) 92.2641(2,2,1,1,1,2) 92.2641(2,2,1,1,2,1) 184.528(2,2,1,1,2,2) 92.2641(2,2,1,2,1,1) 184.528(2,2,1,2,1,2) 184.528(2,2,1,2,2,1) 369.056(2,2,1,2,2,2) 92.2641(2,2,2,1,1,1) 184.528(2,2,2,1,1,2) 184.528(2,2,2,1,2,1) 369.056(2,2,2,1,2,2) 184.528(2,2,2,2,1,1) 369.056(2,2,2,2,1,2) 369.056(2,2,2,2,2,1) 738.113(2,2,2,2,2,2) }

>>> logsig = stream.log_signature(interval)
>>> print(f"{logsig=!s}")
logsig={ 4.5(1) 9(2) }

The above stream was constructed with Lie increments. You can construct streams in many ways. The six types are:

  1. Lie increment streams

    # Using Lie increments data as above
>>> lie_increment_stream = roughpy.LieIncrementStream.from_increments(data, indices=times, ctx=context)

  2. Tick streams

    # create tick stream data
>>> data = { 1.0: [("first", "increment", 1.0),("second", "increment", 2.0)],
         2.0: [("first", "increment", 1.0)]
         }

# construct stream
>>> tick_stream = TickStream.from_data(data, width=2, depth=2, dtype=DPReal)

  3. Brownian streams

    # Generating on demand from a source of randomness with normal increments that approximate Brownian motion
>>> brownian_stream = BrownianStream.with_generator(width=2, depth=2, dtype=DPReal)

  4. Piecewize abelian streams

    # create piecewise lie data
>>> piecewise_intervals = [RealInterval(float(i), float(i + 1)) for i in range(5)]

>>> piecewise_lie_data = [
        (interval,
        brownian_stream.log_signature(interval))
        for interval in piecewise_intervals
    ]

# construct stream
>>> piecewise_abelian_stream = PiecewiseAbelianStream.construct(piecewise_lie_data, width=2, depth=2, dtype=DPReal)

  5. Function streams

    # create a function to generate a stream from
>>> def func(t, ctx):
...     return Lie(np.array([t, 2*t]), ctx=ctx)

#construct stream
>>> function_stream = rp.FunctionStream.from_function(func, width=2, depth=2, dtype=rp.DPReal)

  6. External source data (two kinds)

    # create a stream from an external source
# here we use a sound file, but other formats are supported
>>> roughpy.ExternalDataStream.from_uri("/path/to/sound_file.mp3", depth=2)

You can also plot your streams. For example, here is a stream from electricity data, which can be downloaded here.

../_images/1.png

Fig 1. Raw signal over a time interval at the specified resolution. Accessing all the level one increments at the fine resolution and using these values to plot the two dimensional over the interval.

To see how these streams were made, queried and then plotted, see Electricity Data Example.

Literature references

[Bou98]

N. Bourbaki. Algebra I: Chapters 1-3. Springer Science & Business Media, August 1998. ISBN 978-3-540-64243-5. Google-Books-ID: STS9aZ6F204C.

[Bou89]

Nicolas Bourbaki. Lie Groups and Lie Algebras: Chapters 1-3. Springer Science & Business Media, 1989. ISBN 978-3-540-64242-8. Google-Books-ID: brSYF_rB2ZcC.

[KMFL20]

Patrick Kidger, James Morrill, James Foster, and Terry Lyons. Neural Controlled Differential Equations for Irregular Time Series. November 2020. arXiv:2005.08926 [cs, stat]. URL: http://arxiv.org/abs/2005.08926 (visited on 2023-12-08), doi:10.48550/arXiv.2005.08926.

[LM24]

Terry Lyons and Andrew D. McLeod. Signature Methods in Machine Learning. January 2024. arXiv:2206.14674 [cs, math, stat]. URL: http://arxiv.org/abs/2206.14674 (visited on 2024-02-14).

[LCL07]

Terry J. Lyons, Michael J. Caruana, and Thierry Lévy. Differential Equations Driven by Rough Paths: Ecole d’Eté de Probabilités de Saint-Flour XXXIV-2004. Springer, April 2007. ISBN 978-3-540-71285-5. Google-Books-ID: hOm5BQAAQBAJ.

[Reu93]

Christophe Reutenauer. Free Lie Algebras. Clarendon Press, 1993. ISBN 978-0-19-853679-6. Google-Books-ID: cBvvAAAAMAAJ.