How do you make this code more pythonic?

Could you guys please tell me how I can make the following code more pythonic?

The code is correct. Full disclosure - it's problem 1b in Handout #4 of this machine learning course. I'm supposed to use newton's algorithm on the two data sets for fitting a logistic hypothesis. But they use matlab & I'm using scipy

Eg one question i have is the matrixes kept rounding to integers until I initialized one value to 0.0. Is there a better way?


import os.path
import math
from numpy import matrix
from scipy.linalg import inv #, det, eig

x = matrix( '0.0;0;1'  )
y = 11
grad = matrix( '0.0;0;0'  )
hess = matrix('0.0,0,0;0,0,0;0,0,0')
theta = matrix( '0.0;0;0'  ) 

# run until convergence=6or7
for i in range(1, 6):
  grad = matrix( '0.0;0;0'  )
  hess = matrix('0.0,0,0;0,0,0;0,0,0')

  xfile = open("q1x.dat", "r")
  yfile = open("q1y.dat", "r")

  #over whole set=99 items  
  for i in range(1, 100):    
    xline = xfile.readline()
    s= xline.split("  ")
    x[0] = float(s[1])
    x[1] = float(s[2])
    y = float(yfile.readline())

    hypoth = 1/ (1+ math.exp(-(theta.transpose() * x)))

    for j in range(0,3):
      grad[j] = grad[j] + (y-hypoth)* x[j]      
      for k in range(0,3):
        hess[j,k] = hess[j,k] - (hypoth *(1-hypoth)*x[j]*x[k])

  theta = theta - inv(hess)*grad #update theta after construction


print "done"
print theta

Asked by: Melissa846 | Posted: 06-12-2021

Answer 1

One obvious change is to get rid of the "for i in range(1, 100):" and just iterate over the file lines. To iterate over both files (xfile and yfile), zip them. ie replace that block with something like:

 import itertools

 for xline, yline in itertools.izip(xfile, yfile):
    s= xline.split("  ")
    x[0] = float(s[1])
    x[1] = float(s[2])
    y = float(yline)

(This is assuming the file is 100 lines, (ie. you want the whole file). If you're deliberately restricting to the first 100 lines, you could use something like:

 for i, xline, yline in itertools.izip(range(100), xfile, yfile):

However, its also inefficient to iterate over the same file 6 times - better to load it into memory in advance, and loop over it there, ie. outside your loop, have:

xfile = open("q1x.dat", "r")
yfile = open("q1y.dat", "r")
data = zip([line.split("  ")[1:3] for line in xfile], map(float, yfile))

And inside just:

for (x1,x2), y in data:
    x[0] = x1
    x[1] = x2

Answered by: Rebecca842 | Posted: 07-01-2022

Answer 2

x = matrix([[0.],[0],[1]])
theta = matrix(zeros([3,1]))
for i in range(5):
  grad = matrix(zeros([3,1]))
  hess = matrix(zeros([3,3]))
  [xfile, yfile] = [open('q1'+a+'.dat', 'r') for a in 'xy']
  for xline, yline in zip(xfile, yfile):
    x.transpose()[0,:2] = [map(float, xline.split("  ")[1:3])]
    y = float(yline)
    hypoth = 1 / (1 + math.exp(theta.transpose() * x))
    grad += (y - hypoth) * x
    hess -= hypoth * (1 - hypoth) * x * x.transpose()
  theta += inv(hess) * grad
print "done"
print theta

Answered by: Andrew845 | Posted: 07-01-2022

Answer 3

the matrixes kept rounding to integers until I initialized one value to 0.0. Is there a better way?

At the top of your code:

from __future__ import division

In Python 2.6 and earlier, integer division always returns an integer unless there is at least one floating point number within. In Python 3.0 (and in future division in 2.6), division works more how we humans might expect it to.

If you want integer division to return an integer, and you've imported from future, use a double //. That is

from __future__ import division
print 1//2 # prints 0
print 5//2 # prints 2
print 1/2  # prints 0.5
print 5/2  # prints 2.5

Answered by: Edward456 | Posted: 07-01-2022

Answer 4

You could make use of the with statement.

Answered by: Ryan423 | Posted: 07-01-2022

Answer 5

the code that reads the files into lists could be drastically simpler

for line in open("q1x.dat", "r"):
    x = map(float,line.split("  ")[1:])
y = map(float, open("q1y.dat", "r").readlines())

Answered by: Maddie248 | Posted: 07-01-2022

Similar questions

python - How to make my code more Pythonic?

Closed. This question is off-topic. It is not curre...

python - Should I do anything to make my code more pythonic?

Closed. This question does not meet Stack Overflow guid...

python - Pythonic way for the following code

I have a python dict a = { "Ram":(20,"Male"), "Shyam":(22,"Male"), "Shruti":(19,"Female"), } I want to check whether a name is present in the dict as key and return age based on it otherwise name. So I have written the following function for it def func(val ): if val in a: return ( a[val][ 0 ], True ) # Return age and key in dict? return ( val,...

python how to make this code more pythonic?

Closed. This question does not meet Stack Overflow guid...

python - How can I make this code more pythonic?

I am reading a bunch of daily files and using glob to concatenate them all together into separate dataframes.I eventually join them together and basically create a single large file which I use to connect to a dashboard. I am not too familiar with Python but I used pandas and sklearn often. As you can see, I am basically just reading the last 60 (or more) days worth of data (last 60 files) and creating a dataframe...

python - Please show me how to make this code more "Pythonic"

Closed. This question needs to be more focused. It ...

python - Pythonic ways to use 'else' in a for loop

This question already has answers here:

python - What is the Pythonic way to write this loop?

for jr in json_reports: jr['time_created'] = str(jr['time_created'])

python - Pythonic way to return list of every nth item in a larger list

Say we have a list of numbers from 0 to 1000. Is there a pythonic/efficient way to produce a list of the first and every subsequent 10th item, i.e. [0, 10, 20, 30, ... ]? Yes, I can do this using a for loop, but I'm wondering if there is a neater way to do this, perhaps even in one line?

python - Can this be made more pythonic?

I came across this (really) simple program a while ago. It just outputs the first x primes. I'm embarrassed to ask, is there any way to make it more "pythonic" ie condense it while making it (more) readable? Switching functions is fine; I'm only interested in readability. Thanks from math import sqrt def isprime(n): if n ==2: return True if n % 2 ==0 : # evens return False max = in...

python - What is the pythonic way to unpack tuples?

This question already has answers here:

python - Pythonic way to check if a file exists?

This question already has answers here:

python - Pythonic way to create a numpy array from a list of numpy arrays

I generate a list of one dimensional numpy arrays in a loop and later convert this list to a 2d numpy array. I would've preallocated a 2d numpy array if i knew the number of items ahead of time, but I don't, therefore I put everything in a list. The mock up is below: >>> list_of_arrays = map(lambda x: x*ones(2), range(5)) >>> list_of_arrays [array([ 0., 0.]), array([ 1., 1.]), ar...

python - What is the most pythonic way to check if an object is a number?

Given an arbitrary python object, what's the best way to determine whether it is a number? Here is is defined as acts like a number in certain circumstances. For example, say you are writing a vector class. If given another vector, you want to find the dot product. If given a scalar, you want to scale the whole vector. Checking if something is int, float, ...

python - More pythonic way to write this?

I have this code here: import re def get_attr(str, attr): m = + r'=(\w+)', str) return None if not m else str = 'type=greeting hello=world' print get_attr(str, 'type') # greeting print get_attr(str, 'hello') # world print get_attr(str, 'attr') # None Which works, but I am not particularly fond of this line: return None if not m e...

python - Taking list's tail in a Pythonic way?

from random import randrange data = [(randrange(8), randrange(8)) for x in range(8)] And we have to test if the first item equals to one of a tail. I am curious, how we would do it in most simple way without copying tail items to the new list? Please take into account this piece of code gets executed many times in, say, update() method, and therefore it has to be quick as possible. Using ...

python - Is it pythonic for a function to return multiple values?

In python, you can have a function return multiple values. Here's a contrived example: def divide(x, y): quotient = x/y remainder = x % y return quotient, remainder (q, r) = divide(22, 7) This seems very useful, but it looks like it can also be abused ("Well..function X already computes what we need as an intermediate value. Let's have X return that value also"). W...

python - A pythonic way to insert a space before capital letters

I've got a file whose format I'm altering via a python script. I have several camel cased strings in this file where I just want to insert a single space before the capital letter - so "WordWordWord" becomes "Word Word Word". My limited regex experience just stalled out on me - can someone think of a decent regex to do this, or (better yet) is there a more pythonic way to do this that I'm missing?

python - Is there a more Pythonic way to merge two HTML header rows with colspans?

I am using BeautifulSoup in Python to parse some HTML. One of the problems I am dealing with is that I have situations where the colspans are different across header rows. (Header rows are the rows that need to be combined to get the column headings in my jargon) That is one column may span a number of columns above or below it and the words need to be appended or prepended based on the spanning. Below is a routine to d...

python - What is the pythonic way to share common files in multiple projects?

Lets say I have projects x and y in brother directories: projects/x and projects/y. There are some utility funcs common to both projects in and some db stuff in, etc. Those are minor common goodies, so I don't want to create a single package for them. Questions arise about the whereabouts of such files, possible changes to PYTHONPATH, proper way to import, etc. What is th...

python - Pythonic ways to use 'else' in a for loop

This question already has answers here:

python - pythonic way to compare compound classes?

I have a class that acts as an item in a tree: class CItem( list ): pass I have two trees, each with CItem as root, each tree item has some dict members (like item._test = 1). Now i need to compare this trees. I can suggest to overload a comparison operator for CItem: class CItem( list ): def __eq__( self, other ): # first compare items as lists if not list.__eq...

python - Pythonic URL Parsing

There are a number of questions about how to parse a URL in Python, this question is about the best or most Pythonic way to do it. In my parsing I need 4 parts: the network location, the first part of the URL, the path and the filename and querystring parts.

list - Pythonic way to get some rows of a matrix

I was thinking about a code that I wrote a few years ago in Python, at some point it had to get just some elements, by index, of a list of lists. I remember I did something like this: def getRows(m, row_indices): tmp = [] for i in row_indices: tmp.append(m[i]) return tmp Now that I've learnt a little bit more since then, I'd use a list comprehension like this:

python - What is the Pythonic way to write this loop?

for jr in json_reports: jr['time_created'] = str(jr['time_created'])

python - Pythonic Swap of 2 lists elements

I found that I have to perform a swap in python and I write something like this: arr[first], arr[second] = arr[second], arr[first] I suppose this is not so pythonic. Does somebody know how to do a swap in python more elegant? EDIT: I think another example will show my doubts: self.memberlist[someindexA], self.memberlist[someindexB] = self.memberlist[som...

Still can't find your answer? Check out these communities...

PySlackers | Full Stack Python | NHS Python | Pythonist Cafe | Hacker Earth | Discord Python