Latest news about Bitcoin and all cryptocurrencies. Your daily crypto news habit.
TL;DR
Artificial Intelligence(AI) and Machine Learning(ML) are literally on fire these days. Powering a wide spectrum of use-cases ranging from self-driving cars to drug discovery and to God knows what. AI and ML have a bright and thriving future ahead ofĀ them.
On the other hand, Docker revolutionized the computing world through the introduction of ephemeral lightweight containers. Containers basically package all the software required to run inside an image(a bunch of read-only layers) with a COW(Copy On Write) layer to persist theĀ data.
Enough talk letās get started with building a Python data science container.
Python Data ScienceĀ Packages
Our Python data science container makes use of the following super cool python packages:
- NumPy: NumPy or Numeric Python supports large, multi-dimensional arrays and matrices. It provides fast precompiled functions for mathematical and numerical routines. In addition, NumPy optimizes Python programming with powerful data structures for efficient computation of multi-dimensional arrays and matrices.
- SciPy: SciPy provides useful functions for regression, minimization, Fourier-transformation, and many more. Based on NumPy, SciPy extends its capabilities. SciPyās main data structure is again a multidimensional array, implemented by Numpy. The package contains tools that help with solving linear algebra, probability theory, integral calculus, and many moreĀ tasks.
- Pandas: Pandas offer versatile and powerful tools for manipulating data structures and performing extensive data analysis. It works well with incomplete, unstructured, and unordered real-world dataāāāand comes with tools for shaping, aggregating, analyzing, and visualizing datasets.
- SciKit-Learn: Scikit-learn is a Python module integrating a wide range of state-of-the-art machine learning algorithms for medium-scale supervised and unsupervised problems. It is one of the best-known machine-learning libraries for python. The Scikit-learn package focuses on bringing machine learning to non-specialists using a general-purpose high-level language. The primary emphasis is upon ease of use, performance, documentation, and API consistency. With minimal dependencies and easy distribution under the simplified BSD license, SciKit-Learn is widely used in academic and commercial settings. Scikit-learn exposes a concise and consistent interface to the common machine learning algorithms, making it simple to bring ML into production systems.
- Matplotlib: Matplotlib is a Python 2D plotting library, capable of producing publication quality figures in a wide variety of hardcopy formats and interactive environments across platforms. Matplotlib can be used in Python scripts, the Python and IPython shell, the Jupyter notebook, web application servers, and four graphical user interface toolkits.
- NLTK: NLTK is a leading platform for building Python programs to work with human language data. It provides easy-to-use interfaces to over 50 corpora and lexical resources such as WordNet, along with a suite of text processing libraries for classification, tokenization, stemming, tagging, parsing, and semantic reasoning.
Building the Data Science Container
Python is fast becoming the go-to language for data scientists and for this reason we are going to use Python as the language of choice for building our data science container.
The Base Alpine LinuxĀ Image
Alpine Linux is a tiny Linux distribution designed for power users who appreciate security, simplicity and resource efficiency.
As claimed byĀ Alpine:
Small. Simple. Secure. Alpine Linux is a security-oriented, lightweight Linux distribution based on musl libc andĀ busybox.
The Alpine image is surprisingly tiny with a size of no more than 8MB for containers. With minimal packages installed to reduce the attack surface on the underlying container. This makes Alpine an image of choice for our data science container.
Downloading and Running an Alpine Linux container is as simpleĀ as:
$ docker container run --rm alpine:latest cat /etc/os-release
In our, Dockerfile we can simply use the Alpine base imageĀ as:
Talk is cheap letās build the Dockerfile
Now letās work our way through the Dockerfile.
The FROM directive is used to set alpine:latest as the base image. Using the WORKDIR directive we set the /var/www as the working directory for our container. The ENV PACKAGES lists the software packages required for our container like git, blas and libgfortran. The python packages for our data science container are defined in the ENV PACKAGES.
We have combined all the commands under a single Dockerfile RUN directive to reduce the number of layers which in turn helps in reducing the resultant imageĀ size.
Building and tagging theĀ image
Now that we have our Dockerfile defined, navigate to the folder with the Dockerfile using the terminal and build the image using the following command:
$ docker build -t faizanbashir/python-datascience:2.7 -f Dockerfile .
The -t flag is used to name a tag in the 'name:tag' format. The -f tag is used to define the name of the Dockerfile (Default is 'PATH/Dockerfile').
Running the container
We have successfully built and tagged the docker image, now we can run the container using the following command:
$ docker container run --rm -it faizanbashir/python-datascience:2.7 python
Voila, we are greeted by the sight of a python shell ready to perform all kinds of cool data scienceĀ stuff.
Python 2.7.15 (default, Aug 16 2018, 14:17:09) [GCC 6.4.0] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>>
Our container comes with Python 2.7, but donāt be sad if you wanna work with Python 3.6. Lo, behold the Dockerfile for PythonĀ 3.6:
Build and tag the image likeĀ so:
$ docker build -t faizanbashir/python-datascience:3.6 -f Dockerfile .
Run the container likeĀ so:
$ docker container run --rm -it faizanbashir/python-datascience:3.6 python
With this, you have a ready to use container for doing all kinds of cool data scienceĀ stuff.
Serving Puddinā
Figures, you have the time and resources to set up all this stuff. In case you donāt, you can pull the existing images that I have already built and pushed to Dockerās registry Docker HubĀ using:
# For Python 2.7 pull$ docker pull faizanbashir/python-datascience:2.7
# For Python 3.6 pull$ docker pull faizanbashir/python-datascience:3.6
After pulling the images you can use the image and extend in your Dockerfile file or use it as an image in your docker-compose or stackĀ file.
Aftermath
The world of AI, ML is getting pretty exciting these days and will continue to become even more exciting. Big players are investing heavily in these domains. About time you start to harness the power of data, who knows it might lead to something wonderful.
You can check out the codeĀ here.
faizanbashir/python-datascience
I hope this article helped in building containers for your data science projects. Clap if it increased your knowledge, help it reach moreĀ people.
Building Python Data Science Container using Docker was originally published in Hacker Noon on Medium, where people are continuing the conversation by highlighting and responding to this story.
Disclaimer
The views and opinions expressed in this article are solely those of the authors and do not reflect the views of Bitcoin Insider. Every investment and trading move involves risk - this is especially true for cryptocurrencies given their volatility. We strongly advise our readers to conduct their own research when making a decision.