Free a deepcopy dictionary in python
I have this python class in which I need to do
self.data = copy.deepcopy(raw_data)
raw_data is a dictionary of a dictionary and takes many megabytes in memory. I only need the data once (in which I do some modification to the data thus the need to do a deepcopy) and I would like to destroy the deepcopy data once I'm done with the computation.
What would be the best way to clear the data from the memory?
Would this work?
self.data = None
Note I'm using Python 3.4 if it makes a difference.
Asked by: Robert806 | Posted: 27-01-2022
Answer 1
Some say it's not neccesary that python will do it for you, as long as you don't use the varaible for some time. Some say to use garabage collector library.
According Havenard and to Python Official Documentation, you can force the Garbage Collector to release unreferenced memory with gc.collect()
For more information on this:
How can I explicitly free memory in Python?
Answered by: Haris648 | Posted: 28-02-2022Answer 2
You will have to stop referencing the object both directly and indirectly. It maybe that setting self.data to None is not enough. del self.data does about the same thing, but instead of setting self.data to None it removes the attribute data from self instead.
CPython (the normal implementation of python) uses primary reference counting to determine when an object may be collected, when reference count has dropped to zero it will be collected immediately. You can check the reference count of self.data by using sys.getrefcount(self.data), but that might confuse you as it may report one more reference since the function itself has a reference to the object.
Similar questions
Python dictionary deepcopy
I was wondering in how does exactly deepcopy work in the following context:
from copy import deepcopy
def copyExample:
self.myDict = {}
firstPosition = "First"
firstPositionContent = ["first", "primero"]
secondPosition = "Second"
secondPositionContent = ["second"]
self.myDict[firstPosition] = firstPositionContent
self.myDict[secondPosition] = secondPositionContent
return de...
python - Issue using deepcopy with dictionary inside object
Reading the documentation, I understood that copy.deepcopy(obj) copies recursively any other object inside this one, but when I run:
>>> import copy
>>> class SomeObject:
... a=1
... b={1:1,2:2}
...
>>> o1=SomeObject()
>>> o2=copy.deepcopy(o1)
>>> id(o1)
140041523635624
>>> id(o2)
140041523635912
>>> id(o1.b)
30087968
>&...
python - does deepCopy copies the space left after delete from the old dictionary?
I am trying to delete keys from a dictionary according to Python reclaiming memory after deleting items in a dictionary
. One solution proposed in the latter is:
to create a new dictionary by copying the old dictionary
In source code 1 I did t...
Why doesn't deepcopy work on a python dictionary?
How to achieve deepcopy in dictionaries?
My original code :
li1 = ['Column_6', 'Column_11']
delimiters = ['a','b','c','d']
inner_dict = dict.fromkeys(delimiters,[0,0])
delim_dict = dict.fromkeys(li1 ,None)
for k,v in delim_dict.items():
delim_dict[k] = copy.deepcopy(inner_dict)
print (delim_dict) gives
{'Column_6': {'a': [0, 0], 'b': [0, 0], 'c': [0, 0], 'd': [0, 0...
python - Why use deepcopy dictionary?
I've seen in code examples people use deepcopy, to copy a dictionary, and i've read that dictionaries in some instances might get scrued up, when data is being added to them, and deep copy acts like a new memory zone, which will basically keep populating the dictionary without rewriting the existing memory if that makes sens?
Like c++ pointers to some degree?
Can someone help me understand this functionality ...
python - How to only update one value in a nested dictionary with deepcopy
I am making an adjacency list to eventually make a graph of cities and the distance between them. To do so, the data structure in general looks like this after initialization:
my_dict = { 'city1': {'v1': -1, 'v2': -1, 'v3': -1},
'city2': {'v1': -1, 'v2': -1, 'v3': -1},
'city3': {'v1': -1, 'v2': -1, 'v3': -1}
}
and so on.
Now that it's initialized, I began to...
Python dictionary deepcopy
I was wondering in how does exactly deepcopy work in the following context:
from copy import deepcopy
def copyExample:
self.myDict = {}
firstPosition = "First"
firstPositionContent = ["first", "primero"]
secondPosition = "Second"
secondPositionContent = ["second"]
self.myDict[firstPosition] = firstPositionContent
self.myDict[secondPosition] = secondPositionContent
return de...
python - Issue using deepcopy with dictionary inside object
Reading the documentation, I understood that copy.deepcopy(obj) copies recursively any other object inside this one, but when I run:
>>> import copy
>>> class SomeObject:
... a=1
... b={1:1,2:2}
...
>>> o1=SomeObject()
>>> o2=copy.deepcopy(o1)
>>> id(o1)
140041523635624
>>> id(o2)
140041523635912
>>> id(o1.b)
30087968
>&...
python - does deepCopy copies the space left after delete from the old dictionary?
I am trying to delete keys from a dictionary according to Python reclaiming memory after deleting items in a dictionary
. One solution proposed in the latter is:
to create a new dictionary by copying the old dictionary
In source code 1 I did t...
Why doesn't deepcopy work on a python dictionary?
How to achieve deepcopy in dictionaries?
My original code :
li1 = ['Column_6', 'Column_11']
delimiters = ['a','b','c','d']
inner_dict = dict.fromkeys(delimiters,[0,0])
delim_dict = dict.fromkeys(li1 ,None)
for k,v in delim_dict.items():
delim_dict[k] = copy.deepcopy(inner_dict)
print (delim_dict) gives
{'Column_6': {'a': [0, 0], 'b': [0, 0], 'c': [0, 0], 'd': [0, 0...
python - Why use deepcopy dictionary?
I've seen in code examples people use deepcopy, to copy a dictionary, and i've read that dictionaries in some instances might get scrued up, when data is being added to them, and deep copy acts like a new memory zone, which will basically keep populating the dictionary without rewriting the existing memory if that makes sens?
Like c++ pointers to some degree?
Can someone help me understand this functionality ...
python - How to only update one value in a nested dictionary with deepcopy
I am making an adjacency list to eventually make a graph of cities and the distance between them. To do so, the data structure in general looks like this after initialization:
my_dict = { 'city1': {'v1': -1, 'v2': -1, 'v3': -1},
'city2': {'v1': -1, 'v2': -1, 'v3': -1},
'city3': {'v1': -1, 'v2': -1, 'v3': -1}
}
and so on.
Now that it's initialized, I began to...
sorting - In Python, how can you easily retrieve sorted items from a dictionary?
Dictionaries unlike lists are not ordered (and do not have the 'sort' attribute). Therefore, you can not rely on getting the items in the same order when first added.
What is the easiest way to loop through a dictionary containing strings as the key value and retrieving them in ascending order by key?
For example, you had this:
d = {'b' : 'this is b', 'a': 'this is a' , 'c' : 'this is c'}
Python dictionary from an object's fields
Do you know if there is a built-in function to build a dictionary from an arbitrary object? I'd like to do something like this:
>>> class Foo:
... bar = 'hello'
... baz = 'world'
...
>>> f = Foo()
>>> props(f)
{ 'bar' : 'hello', 'baz' : 'world' }
NOTE: It should not include methods. Only fields.
python - How do you retrieve items from a dictionary in the order that they're inserted?
Is it possible to retrieve items from a Python dictionary in the order that they were inserted?
python - How can I make a dictionary from separate lists of keys and values?
I want to combine these:
keys = ['name', 'age', 'food']
values = ['Monty', 42, 'spam']
Into a single dictionary:
{'name': 'Monty', 'age': 42, 'food': 'spam'}
python - Dictionary or If statements, Jython
I am writing a script at the moment that will grab certain information from HTML using dom4j.
Since Python/Jython does not have a native switch statement I decided to use a whole bunch of if statements that call the appropriate method, like below:
if type == 'extractTitle':
extractTitle(dom)
if type == 'extractMetaTags':
extractMetaTags(dom)
Is a Python dictionary an example of a hash table?
One of the basic data structures in Python is the dictionary, which allows one to record "keys" for looking up "values" of any type. Is this implemented internally as a hash table? If not, what is it?
python - Is there a "one-liner" way to get a list of keys from a dictionary in sorted order?
The list sort() method is a modifier function that returns None.
So if I want to iterate through all of the keys in a dictionary I cannot do:
for k in somedictionary.keys().sort():
dosomething()
Instead, I must:
keys = somedictionary.keys()
keys.sort()
for k in keys:
dosomething()
Is there a pretty way to iterate t...
python - Interface to versioned dictionary
I have an versioned document store which I want to access through an dict like interface.
Common usage is to access the latest revision (get, set, del), but one should be able to access specific revisions too (keys are always str/unicode or int).
from UserDict import DictMixin
class VDict(DictMixin):
def __getitem__(self, key):
if isinstance(key, tuple):
docid, rev = key
e...
python - List all words in a dictionary that start with <user input>
How would a go about making a program where the user enters a string, and the program generates a list of words beginning with that string?
Ex:
User: "abd"
Program:abdicate, abdomen, abduct...
Thanks!
Edit: I'm using python, but I assume that this is a fairly language-independent problem.
python - Check if a given key already exists in a dictionary and increment it
How do I find out if a key in a dictionary has already been set to a non-None value?
I want to increment the value if there's already one there, or set it to 1 otherwise:
my_dict = {}
if my_dict[key] is not None:
my_dict[key] = 1
else:
my_dict[key] += 1
Still can't find your answer? Check out these communities...
PySlackers | Full Stack Python | NHS Python | Pythonist Cafe | Hacker Earth | Discord Python