How to write a wrapper to fix arbitrary parameters of the jacobian of a function

This is an extension of a previous stack-exchange question I posted. link

Context:

My goal is to fit data to a function f(t, *p) using the scipy.optimize.curve_fit function. I happen to know some parameters pfix = {p_j, ..., p_k} and want to fit f(t, *p) to my data with the parameters in pfix... fixed.

Above I have a link asking how I could write a wrapper for fixing parameters in a function. Now I want to do the same thing, but for the jacobian of f(t, *p), fixing parameters pfix. I am not sure how to do this.

Wrapper for func(x, *p)

Below is the wrapper for my function:

def fix_params(f, fix_pars):
    # fix_pars = ((1, A), (2, B))
    def new_func(x, *pars):
        new_pars = [None]*(len(pars) + len(fix_pars))
        for j, fp in fix_pars:
            new_pars[j] = fp
        for par in pars:
            for j, npar in enumerate(new_pars):
                if npar is None:
                    new_pars[j] = par
                    break
        return f(x, *new_pars)
    return new_func

Issue

Naively I would just use this wrapper for my Jacobian function. However, here is the problem.

Let's say I have N parameters and M values for x. Then my jacobian function returns a (M, N) numpy array. Now this is fine if I don't fix any parameters. However, even when I fix just one parameter, my wrapped jacobian function still returns a (M, N) numpy array. This causes curve_fit to complain, as the number of parameters I use is now less than the parameter dimension of my jacobian. I am not sure how to get around this.

Any suggestions?


Asked by: Ada543 | Posted: 30-11-2021






Answer 1

This should work (using as in my comment scipy.optimize.leatsq)

import matplotlib.pyplot as plt
import numpy as np
from scipy.optimize import leastsq

def arb_func( x, a, b, c, d ):
    return a * np.exp( b * np.sin( c * x + d ) )

def arb_fixed( fixed ):
    def g( x , *pars ):
        locPars = list( pars )
        for f in sorted( list(fixed), key=lambda x: x[0] ):
            locPars.insert( f[0], f[1] )
        return arb_func( x, *locPars)
    return g

def func_da( x, a, b, c, d ):
    return np.exp( b * np.sin( c * x + d ) )

def func_db( x, a, b, c, d ):
    return arb_func( x, a, b, c, d ) * np.sin( c * x + d )

def func_dc( x, a, b, c, d ):
    return arb_func( x, a, b, c, d ) * b * np.cos( c * x + d ) * x

def func_dd( x, a, b, c, d ):
    return arb_func( x, a, b, c, d ) * b * np.cos( c * x + d )

dList = [ func_da, func_db, func_dc, func_dd ]

def jac( pars, x, y ):
    return [ func( x, *pars ) for func in dList ]

def jac_fixed( Fixed = None ):
    def h( pars, x, y, z ):
        funcList = dList[::]
        locFixed = sorted( list(Fixed), key=lambda x: x[0] )
        locPars = list( pars )
        for f in locFixed:
            locPars.insert( f[0], f[1] )
        locFixed.reverse()
        for f in locFixed:
            del funcList[ f[0] ]
        out = [ func( x, *locPars ) for func in funcList ]
        return out
    return h

a0 = +3
b0 = -0.6
c0 = +1.44
d0 = +0.4

xList = np.linspace( -2, 4, 100 )
y0List = np.fromiter( ( arb_func( x, a0, b0, c0, d0 ) for x in xList ), np.float )
yNoiseList = np.fromiter( ( y + .2 * np.random.normal() for y in y0List[::4] ), np.float )
xNoiseList = xList[::4]

def residuals_fixed( params, xList, yList, Fixed=None ):
    if Fixed is None:
        fixedSorted = []
    else:
        fixedSorted = sorted( list(Fixed), key=lambda x: x[0] )
    locParams = list( params )
    for f in fixedSorted:
            locParams.insert( f[0], f[1] )
    diff = [ arb_func( x, *locParams ) - y for x, y in zip( xList, yList )]
    return diff

def leastsq_wrapper( xList, yList, p0, **kwargs ):
    fixed = kwargs.pop( 'Fixed', None )
    if fixed is None:
        locFixed = []
    else:
        locFixed = fixed
        s = np.array( locFixed ).shape
        if len(s) !=2 or s[-1] !=2:
            raise ValueError( 'fixed value list has wrong shape. Must be n by 2, but is {}'.format(s) )
    if len( p0 ) + len( locFixed ) != 4:
            raise TypeError( 'Total number of arguments (variable + fixed) is not 4' )
    fixedSorted = sorted( list( locFixed ), key=lambda x: x[0] )
    if not all( [ ( type( item[0] ) is int )  and ( item[0] > -1 ) and ( item[0] < 4 ) for item in fixedSorted ] ):
        raise ValueError( 'list indices i for fixed values are not int with 0 <= i < 4' )
    my_jac = jac_fixed( Fixed=fixedSorted )
    baseDict = { 'args':( xList, yList, fixed ), 'Dfun':my_jac, 'col_deriv':1}
    baseDict.update(kwargs) ## allows to send e.g.  full_output=True
    out = leastsq( residuals_fixed, p0, **baseDict )
    return out

myFitStd, err = leastsq( residuals_fixed, [ a0, b0 ,c0 , d0 ], args=( xNoiseList, yNoiseList ) )
print myFitStd
myFit0, err = leastsq_wrapper( xNoiseList, yNoiseList,  [ a0, b0 ,c0 , d0 ] )
print myFit0
myFixed1 = [[0,3.3]]
myFit1, err = leastsq_wrapper( xNoiseList, yNoiseList,  [ b0 ,c0 , d0 ], Fixed=myFixed1 )
arb1 = arb_fixed( myFixed1 )
print myFit1

myFixed2 = [ [ 3, .8], [2, 1.2 ] ]
myFit2, err = leastsq_wrapper( xNoiseList, yNoiseList,  [ a0, b0 ], Fixed=myFixed2 )
arb2 = arb_fixed( myFixed2 )
print myFit2

fig = plt.figure()
ax = fig.add_subplot( 1, 1, 1 )

ax.plot( xList, y0List )
ax.plot( xNoiseList, yNoiseList, marker='o', linestyle='' )
ax.plot( xList, np.fromiter( ( arb_func( x, *myFitStd ) for x in xList ), np.float), linestyle='--' )
ax.plot( xList, np.fromiter( ( arb_func( x, *myFit0 ) for x in xList ), np.float), linestyle=':' )
ax.plot( xList, np.fromiter( ( arb1( x, *myFit1 ) for x in xList ), np.float) )
ax.plot( xList, np.fromiter( ( arb2( x, *myFit2 ) for x in xList ), np.float) )

plt.show()

Giving the output

[ 3.03802692 -0.57275564  1.43380277  0.38557492]
[ 3.03802692 -0.57275564  1.43380277  0.38557493]
[-0.49087778  1.422561    0.40503389]
[ 3.31028289 -0.46678563]

enter image description here

Answered by: Wilson555 | Posted: 01-01-2022



Similar questions

python - How do I call a function with arbitrary parameters from another function?

I'd like to know how I would go about doing something like this (in python 2): def myFunction(fn, params): return ("You called " + fn.__name__ + "(" + str(params) ")" + " and got " + str(fn(params))) Say I have a couple functions: def doSomething(s): # manipulation return s def doAnotherThing(someInt, someOtherInt, someString): # do something with ...


Python function that counts negative integers using arbitrary number of parameters

Python newbie here, I need help with working with function with arbitrary number of parameters. I want to implement a function that can take an arbitrary number of parameters, and counts negative integers. I want to try the negative_counter function on the following list of numbers 4,-3,5,6,-7 See my attempt (not sure what I am doing wrong) def negative_counter(*args):...


python - Django: Arbitrary number of unnamed urls.py parameters

I have a Django model with a large number of fields and 20000+ table rows. To facilitate human readable URLs and the ability to break down the large list into arbitrary sublists, I would like to have a URL that looks like this: /browse/&lt;name1&gt;/&lt;value1&gt;/&lt;name2&gt;/&lt;value2&gt;/ .... etc .... where 'name' maps to a model attribute and 'value' is the search criteria for that...


python - How do I call a function with arbitrary parameters from another function?

I'd like to know how I would go about doing something like this (in python 2): def myFunction(fn, params): return ("You called " + fn.__name__ + "(" + str(params) ")" + " and got " + str(fn(params))) Say I have a couple functions: def doSomething(s): # manipulation return s def doAnotherThing(someInt, someOtherInt, someString): # do something with ...


curve fitting - Creating a python lmfit Model with arbitrary number of parameters

Is there a way to construct a an lmfit Model based on a function with an arbitrary number of dependent variables? For example: from lmfit import Model def my_poly(x, *params): func = 0 for i in range(len(params)): func+= params[i]*z**i return func #note: below does not work my_model = Model(my_poly, independent_vars = ['x'], param_names = ['A','B','C']) Something similar to th...


How do Python methods handle arbitrary parameters?

I will use the range function as the main example in this question. I use range a lot to generate a list of numbers. However, I use it often with different parameters, specifically a different NUMBER of parameters. It makes me wonder how range easily handles different numbers of parameters and assigns them to their respective value (if they represent the highest value, or the ...


python - Allowing for an arbitrary number of parameters in a function?

This question already has answers here:


python - Fit an arbitrary number of parameters when calling curve_fit

Closest I found to this question was here: Fitting only one parameter of a function with many parameters in python. I have a multi-parameter function that I want to be able to call with a different subset of parameters being optimised in different parts of the code (useful because for some data...


scipy - Python: Finding number of fitting parameters for arbitrary curve

Is there a way to return the number of arguments an arbitrary function has which is defined in some other file? I have tried using the Signature class in inspect as follows: from foo import func1, func2, func3 from inspect import signature from scipy.optimize import curve_fit func_list = [ func1, func2, func3 ] n, bins, patches = hist( array ) for f in func_list: sig = signature(f) args = sig.par...


python - How to create a function with arbitrary parameters based on an arbitrary string

My end goal is: I want to create a set of truth tables, where each truth table corresponds to an arbitrarily defined boolean expression, which is originally stored as a string (like: "(var_1 and not var_2) or var_3" ). The string could have any number of operators. This is easily achievable if I have a particular boolean expression in mind: def evaluator(var_1,var_2,var_3): return (var_1 and no...


python - How to allow any arbitrary query parameters using FastAPI and Swagger?

Note: This question is different from the one here, in that I need it to work with Swagger. Given a FastAPI GET endpoint, I want to allow any arbitrary set of URL parameters, while maintaining Swagger support. My use case is that I want to support a JSON API-like set of query parameters s...


Python function that counts negative integers using arbitrary number of parameters

Python newbie here, I need help with working with function with arbitrary number of parameters. I want to implement a function that can take an arbitrary number of parameters, and counts negative integers. I want to try the negative_counter function on the following list of numbers 4,-3,5,6,-7 See my attempt (not sure what I am doing wrong) def negative_counter(*args):...


python - Django: Arbitrary number of unnamed urls.py parameters

I have a Django model with a large number of fields and 20000+ table rows. To facilitate human readable URLs and the ability to break down the large list into arbitrary sublists, I would like to have a URL that looks like this: /browse/&lt;name1&gt;/&lt;value1&gt;/&lt;name2&gt;/&lt;value2&gt;/ .... etc .... where 'name' maps to a model attribute and 'value' is the search criteria for that...


python - variables as parameters in field options

I want to create a model, that will set editable=False on creation, and editable=True on editing item. I thought it should be something like this: home = models.ForeignKey(Team, editable=lambda self: True if self.id else False) But it doesn't work. Maybe something with overriding the init can help me, but i don't sure what can do the trick. I know i can check for self.id...


c# - How to analyse .exe parameters inside the program?

I have a program that can have a lot of parameters (we have over +30 differents options). Example: myProgram.exe -t alpha 1 -prod 1 2 -sleep 200 This is 3 Commands (from command pattern object at the end) that each contain some parameters. Inside the code we parse all command (start with -) and get a list of string (split all space) for the parameters. So in fact, we have : string-->Collection ...


python - Default parameters to actions with Django

Is there a way to have a default parameter passed to a action in the case where the regex didnt match anything using django? urlpatterns = patterns('',(r'^test/(?P&lt;name&gt;.*)?$','myview.displayName')) #myview.py def displayName(request,name): # write name to response or something I have tried setting the third parameter in the urlpatterns to a dictionary containing ' and giving...


python - Loop function parameters for sanity check

I have a Python function in which I am doing some sanitisation of the input parameters: def func(param1, param2, param3): param1 = param1 or '' param2 = param2 or '' param3 = param3 or '' This caters for the arguments being passed as None rather than empty strings. Is there an easier/more concise way to loop round the function parameters to apply such an expression to ...


python - How can I pass all the parameters to a decorator?

I tried to trace the execution of some methods using a decorator. Here is the decorator code: def trace(func): def ofunc(*args): func_name = func.__name__ xargs = args print "entering %s with args %s" % (func_name,xargs) ret_val = func(args) print "return value %s" % ret_val print "exiting %s" % (func_nam...


parameters - Python Newbie: Returning Multiple Int/String Results in Python

I have a function that has several outputs, all of which "native", i.e. integers and strings. For example, let's say I have a function that analyzes a string, and finds both the number of words and the average length of a word. In C/C++ I would use @ to pass 2 parameters to the function. In Python I'm not sure what's the right solution, because integers and strings are not passed by reference but by value (at leas...


Print out list of function parameters in Python

Is there a way to print out a function's parameter list? For example: def func(a, b, c): pass print_func_parametes(func) Which will produce something like: ["a", "b", "c"]


python - How to create a decorator that can be used either with or without parameters?

I'd like to create a Python decorator that can be used either with parameters: @redirect_output("somewhere.log") def foo(): .... or without them (for instance to redirect the output to stderr by default): @redirect_output def foo(): .... Is that at all possible? Note that I'm not looking for a different solution to the problem of redirectin...


python - Scope of lambda functions and their parameters?

This question already has answers here:






Still can't find your answer? Check out these communities...



PySlackers | Full Stack Python | NHS Python | Pythonist Cafe | Hacker Earth | Discord Python



top