Geek asks and answers

How to implement a "daisychaining" plugin in python?

here is the problem:

1) suppose I have some measurement data (for example, 1Msample read from my electronics) and I need to process it along the processing chain.

2) this processing chain consists of different operations that can be replaced / omitted / have different parameters. A typical example would be to take this data, first pass it through a lookup table, then perform an exponential fit, and then multiply by some calibration factors

3) now, since I don’t know which algorithm is best, I would like to evaluate at each stage the best possible implementation (as an example, LUTs can be created in 5 ways, and I want to see which one is the best)

4) I would like to describe these functions in such a way that I would build a “class” containing a top-level algorithm and having (that is, pointing to) a child class containing a lower-level algorithm.

I thought to use double-linked-list and generate a sequence, for example:

myCaptureClass.addDataTreatment (pmCalibrationFactor (opt, pmExponentialFit (opt, pmLUT (opt))))

where myCaptureClass is the class responsible for generating the data, and also (after taking the data) starts the top-level data processing module (pm). This processing will first go deep into the lower-child (lut), process the data there, then the middle (expofit), then the upper (califactors) and return the data to the capture, which return data to the requestor.

Now this has a few problems:

1) , python 2) , , , , , .

- , , "-" , X " ", ?

:

, . : HW-, VME. "" python, myCaptureClass.

- , . - ( , ).

"opt", , , , , , , .

, myCaptureClass daisychained ( ), - - "" .

"" , , .

, , , bukzor. , , , ....

+3
3

, . , , . , , :)

class ProcessingPipeline(object):
    def __init__(self, *functions, **kwargs):
        self.functions = functions
        self.data = kwargs.get('data')
    def __call__(self, data):
        return ProcessingPipeline(*self.functions, data=data)
    def __iter__(self):
        data = self.data
        for func in self.functions:
            data = func(data)
        return data

# a few (very simple) operators, of different kinds
class Multiplier(object):
    def __init__(self, by):
        self.by = by
    def __call__(self, data):
        for x in data:
            yield x * self.by

def add(data, y):
    for x in data:
        yield x + y

from functools import partial
by2 = Multiplier(by=2)
sub1 = partial(add, y=-1)
square = lambda data: ( x*x for x in data )

pp = ProcessingPipeline(square, sub1, by2)

print list(pp(range(10)))
print list(pp(range(-3, 4)))

:

$ python how-to-implement-daisychaining-of-pluggable-function-in-python.py 
[-2, 0, 6, 16, 30, 48, 70, 96, 126, 160]
[16, 6, 0, -2, 0, 6, 16]
+1

functional pypi. . .

functool partial .

, .

0

, , , , , , :

l = [myCaptureClass.addDataTreatment(
          pmCalibrationFactor(opt, pmExponentialFit (opt, pmLUT (opt))))
     for opt in data]

will create a new data list that went through compiled functions.

Or you can create a generator expression for cyclization, it will not create a whole new list, it will just create an iterator. I don’t think there is any advantage to doing such things, and not just processing the data in the body of the loop, but it’s interesting to look at:

d = (myCaptureClass.addDataTreatment(
          pmCalibrationFactor(opt, pmExponentialFit (opt, pmLUT (opt))))
     for opt in data)
for thing in d:
    # do something
    pass

Or optdata?

0
source

More articles:


All Articles