Pythonic equivalent of unshift or redo?

I'm learning Python, and I have a situation where I want to consume items from an iterator. The tricky part is that under certain conditions, I want to "un-iterate." That is, put an item back onto the front of the iterator before I loop.

For example, suppose I'm picking apples from a tree. My fruit basket can only hold 10kg before it needs to be emptied. But I have to pick each apple before I can weigh it and determine if this apple would exceed the capacity of the basket.

In a language like Perl, I could unshift() the apple back onto the tree, and then let the loop expression re-pick the apple:

while ($apple = shift(@tree)) {
  $wt = weight($apple);
  if ($wt + weight(@basket) > 10) {
    send(@basket);
    @basket = ();
    unshift(@tree, $apple);
  } else {
    push(@basket, $element);
  }
}

Or else I can also use redo, which resumes processing at the top of block, without evaluating the loop expression. So the same apple can be re-processed, after the basket has been emptied.

while ($apple = shift(@tree)) {
  $wt = weight($apple);
  if ($wt + weight(@basket) > 10) {
    send(@basket);
    @basket = ();
    redo;
  } else {
    push(@basket, $apple);
  }
}

What would be the most pythonic solution for this kind of problem?


Asked by: Sienna294 | Posted: 01-10-2021






Answer 1

I'm learning Python, and I have a situation where I want to consume items from an iterator. The tricky part is that under certain conditions, I want to "un-iterate." That is, put an item back onto the front of the iterator before I loop.

Here's a simple solution:

class MyIterator(object):   # undo-able iterator wrapper
    def __init__(self, iterable):
        super(MyIterator, self).__init__()
        self.iterator = iter(iterable)
        self.stack = []

    def __iter__(self):
        return self

    def next(self):
        if self.stack:
            return self.stack.pop()
        return self.iterator.next()  # Raises StopIteration eventually

    def undo(self, item):
        self.stack.append(item)
for i in  MyIterator(xrange(5)): print i
0
1
2
3
4
rng = MyIterator(xrange(5))
rng.next()
0
rng.next()
1
rng.undo(1)
rng.next()
1

Answered by: Stuart891 | Posted: 02-11-2021



Answer 2

Why bother with unshifting when the else clause should always occur?

for apple in tree:
    if (apple.weight + basket.weight) > 10:
       send(basket)
       basket.clear()
    basket.add(apple)

Anyway, I'm fairly certain that Python doesn't have the sort of behavior you're looking for.

Answered by: Aida578 | Posted: 02-11-2021



Answer 3

I'd say that the most Pythonic solution is the simplest one. Instead of trying to wrap an iterator in a generator expression that allows you to "backtrack" or something similarly complex, use a while loop, as you have in Perl! Iterators don't mix very nicely with mutation, anywho.

Simple translation of your implementation (ignoring @Patrick's optimization):

while tree:
    apple = tree.pop(0)
    if apple.weight + basket.weight > 10:
        basket.send()
        basket.clear()
        tree.insert(0, apple) # Put it back.
    else:
        basket.append(apple)

Or, you could use a peek-like functionality with ordered sequence indices:

while tree:
    apple = tree[0] # Take a peek at it.
    if apple.weight + basket.weight > 10:
        basket.send()
        basket.clear()
    else:
        basket.append(tree.pop(0))

If you don't like the "simple" argument, check out the collections.deque iterators mentioned in the above (linked) thread.

Answered by: Sienna386 | Posted: 02-11-2021



Answer 4

If you don't want to follow the other's suggestion of just removing the else clause, you can write your own unshift function that will work in a way similar to perl's with any iterable:

class UnshiftableIterable(object):
    def __init__(self, iterable):
        self._iter = iter(iterable)
        self._unshifted = [] # empty list of unshifted stuff
    def __iter__(self):
        while True:
            if self._unshifted:
                yield self._unshifted.pop()
            else:
                yield self._iter.next()
    def unshift(self, item):
        self._unshifted.append(item)

Then in your code:

it = UnshiftableIterable(tree)
for apple in tree:
    if weigth(basket) + weight(apple) > MAX_WEIGHT:
        send(basket)
        basket = []
        it.unshift(apple)
    else:
        basket.append(apple)

Some testing of the UnshiftableIterable:

it = UnshiftableIterable(xrange(5))

for i in it:
    print '*',
    if i == 2:
        it.unshift(10)
    else:
        print i,
# output: * 0 * 1 * * 10 * 3 * 4

Answered by: Anna471 | Posted: 02-11-2021



Answer 5

You're looking for a generator, an iterator that can receive modifications to its internal state via the send() method

https://docs.python.org/howto/functional.html#passing-values-into-a-generator

Answered by: Luke489 | Posted: 02-11-2021



Answer 6

By the way, what you really want is list.insert(0,yourObject)

Answered by: Tess200 | Posted: 02-11-2021



Answer 7

While I was writing this @Patrick already suggested the same thing. But since I have written it I will paste the code anyways, with comments in code marking methods from Patrick.

import random

apples=[random.randint(1,3) for j in range(10)]
print 'apples',apples

basket=[]
y=6
baskets=[]

for i in range(len(apples)):
    if sum(basket+[apples[i]])>y:
        #basket is full                                                                                                                                     
        baskets.append(basket)#basket.send()                                                                                                                
        basket=[]#basket.empty()                                                                                                                            
    basket.append(apples[i])#add apple to basket                                                                                                            

print 'baskets',baskets

though this does not pop() the apples from the original iterator. Please remark if that's a desired behavior too.

the output

apples [1, 1, 3, 3, 1, 1, 3, 3, 2, 3]
baskets [[1, 1, 3], [3, 1, 1], [3, 3]]

Answered by: Melissa734 | Posted: 02-11-2021



Answer 8

Currently my upgraded version of pythonizer doesn't handle redo but if I added it, I would probably implement it like this:

while (apple:=(tree.pop(0) if tree else None)):
    while True:
        wt = weight(apple)
        if wt+weight(*basket) > 10:
            sendit(basket)
            basket = []
            continue
        else:
            basket.append(apple)
        break

(Note: I had to change send to sendit because send is predefined in perl.)

Answered by: Kevin780 | Posted: 02-11-2021



Answer 9

Back to the original question about impementing unshift, operator.delitem can be used to implement a simple non-OO function:

from operator import delitem

def unshift(l,idx):
    retval = l[0]
    delitem(l,0)
    return retval

x = [2,4,6,8]

firstval = unshift(x,0)

print firstval,x

2 [4, 6, 8]

Answered by: Anna332 | Posted: 02-11-2021



Answer 10

There is no way general way to push a value into an iterator in python. A stack or linked list is better suited to that.

If you're iterating over a list or something, of course you can add the item manually back to the list. But you can also iterate over objects which can't be manipulated in such a way.

If you want to use python to implement that algorithm, you'll have to choose a data structure that allows the operations you want to use. I suggest the .push() and .pop() methods which let you treat lists as stacks.

Answered by: Elise145 | Posted: 02-11-2021



Similar questions

python - pythonic equivalent this sed command

I have this awk/sed command awk '{full=full$0}END{print full;}' initial.xml | sed 's|</Product>|</Product>\ |g' > final.xml to break an XML doc containing large number of tags such that the new file will have all contents of the product node in a single line I am trying to run it using os.system and subprocess module however this is wrapping all the contents of th...


python - Pythonic equivalent to Matlab's textscan

There are some similar questions to this, but nothing exact that I can find. I have a very odd text-file with lines like the following: field1=1; field2=2; field3=3; field1=4; field2=5; field3=6; Matlab's textscan() function deals with this very neatly, as you can do this: array = textscan(fid, 'field1=%d; field2=%d; field3=%d;' and you will...


python - What is the Pythonic equivalent of Perl's -Sx command line options?

I'm given a task to convert a Perl script to Python. I'm really new to Perl and understanding it where I came across a command line option which is -Sx. There is good documentation provided for these parameters in Perl. But there is no much documentation for the same in python (Didn't find much info in Python official site). My question is are those command line optio...


python - Django equivalent for count and group by

I have a model that looks like this: class Category(models.Model): name = models.CharField(max_length=60) class Item(models.Model): name = models.CharField(max_length=60) category = models.ForeignKey(Category) I want select count (just the count) of items for each category, so in SQL it would be as simple as this: select category_id, count(id) from item group b...


Is there a Python equivalent of Perl's x operator (replicate string)?

In Perl, I can replicate strings with the 'x' operator: $str = "x" x 5; Can I do something similar in Python?


Python's os.execvp equivalent for PHP

I've got a PHP command line program running. And I want to connect to a mysql shell straight from PHP. I've done this before in Python using os.execvp But I can't get the same thing to work in PHP. I've tried the following functions: system passthru exec shell_exec example: system('mysql -u root -pxxxx db_name'); But they all ...


Ruby equivalent of Python's "dir"?

In Python we can "dir" a module, like this: >>> import re >>> dir(re) And it lists all functions in the module. Is there a similar way to do this in Ruby?


Is there a Python news site that's the near equivalent of RubyFlow?

I really like the format and type of links from RubyFlow for Ruby related topics. Is there an equivalent for Python that's active? There is a PythonFlow, but I think it's pretty much dead. I don't really like http://planet.python.org/ because there's lots o...


Java Servlet Filter Equivalent in Ruby [on Rails] and PHP?

Not sure if the terminology is correct, but are there rough equivalents to Java Servlet Filters in Ruby and PHP ? Are they actual concrete classes ? I assume there is also a number of common web app libraries/frameworks in Python. Is there an equivalent there ? Thanks. === ADDENDUM === On the good advice of


python - Is there a Vim equivalent to the Linux/Unix "fold" command?

I realize there's a way in Vim to hide/fold lines, but what I'm looking for is a way to select a block of text and have Vim wrap lines at or near column 80. Mostly I want to use this on comments in situations where I'm adding some text to an existing comment that pushes it over 80 characters. It would also be nice if it could insert the comment marker at the beginning of the line when it wraps too. Also I'd pre...


python - Ruby equivalent of virtualenv?

Is there something similar to the Python utility virtualenv? Basically it allows you to install Python packages into a sandboxed environment, so easy_install django doesn't go in your system-wide site-packages directory, it would go in the virtualenv-created directory. For example: $ virtualenv test New python...


What's the idiomatic Python equivalent to Django's 'regroup' template tag?

http://docs.djangoproject.com/en/dev/ref/templates/builtins/#regroup I can think of a few ways of doing it with loops but I'd particularly like to know if there is a neat one-liner.


Do Python lists have an equivalent to dict.get?

I have a list of integers. I want to know whether the number 13 appears in it and, if so, where. Do I have to search the list twice, as in the code below? if 13 in intList: i = intList.index(13) In the case of dictionaries, there's a get function which will ascertain membership and perform look-up with the same search. Is there something similar for lists?






Still can't find your answer? Check out these communities...



PySlackers | Full Stack Python | NHS Python | Pythonist Cafe | Hacker Earth | Discord Python



top