Latest news about Bitcoin and all cryptocurrencies. Your daily crypto news habit.
Python built-in zip library is commonly used to create archive. However, there is a concern when creating zip files using built-in library. Consider the case which we are zipping files larger than our available memory, we would easily run out of memory.
I was building a feature that requires zipping of files and upload to our Django backend storage. After digging around the internet, I summarize the logic I used to support this feature.
Use NamedTemporaryFile Instead Of Memory
A NamedTemporaryFile resides in secondary memory instead of main memory, thus using it does not consume extra memory.
Pseudocode
1. Create a NamedTempFile2. Create a ZipFile with NamedTempFile as file output3. Write files into ZipFile4. Move the cursor of the NamedTempFile back to the beginning5. Wrap it with Django File 6. Inject a file name7. Upload it to storage
Sample Code
Assume we have a model as in models.py, the logic of using secondary memory to create zip files lies in zipping.py.
I hope this helps. Please leave me a comment if you have better idea on reducing memory load when zipping files.
Your clap will definitely drive me further. Give me a clap if you like this post.
Reducing Memory Footprint While Creating Archive in Django was originally published in Hacker Noon on Medium, where people are continuing the conversation by highlighting and responding to this story.
Disclaimer
The views and opinions expressed in this article are solely those of the authors and do not reflect the views of Bitcoin Insider. Every investment and trading move involves risk - this is especially true for cryptocurrencies given their volatility. We strongly advise our readers to conduct their own research when making a decision.