Python Environments

If you’re working on different projects, especially large projects, you should be using Python Environments.

So what is a Python Environment and why should I care?

The best definition is the one from the official documentation:

The venv module provides support for creating lightweight “virtual environments” with their own site directories, optionally isolated from system site directories. Each virtual environment has its own Python binary (which matches the version of the binary that was used to create this environment) and can have its own independent set of installed Python packages in its site directories.

https://docs.python.org/3/library/venv.html

Here are some of the reason to use venv

Think of it as a sandbox where you can play without worrying about other kids in the park. In terms of code, it’s where you can test out ideas and run different versions of the Python modules without affecting the system-wide ones. That’s a general concept, but here are more concrete reasons why:

For Safely Running Different Packages

For example, you have the latest Django module in your system but you want to try out an older version. Maybe this is an old system that you now need to support and/or maintain. Sure you can just pip install the old version on the system but you’d be risking overwriting the existing packages. Virtual Environments allow us to isolate or contain specific versions of Python packages inside this sandbox environment so we don’t affect anything outside of it.

… using different Python versions

If you’re developing using various versions of Python (ex. 2.7 and 3.7), it really makes it easy to work on packages and not worry about mixing or overwriting anything else.

To Easily Recreate Project Environment

When using virtual environment for a single project, all the packages installed in that environment can be easily exported to a pip output file. You can’t do this easily on a system or global environment since it’d be hard to isolate which package belongs to which project, but inside a virtual environment, all packages are for that project alone. Once you have exported the dependencies, you can easily recreate it on a server or different location by importing those dependencies and be sure that it matches what you have.

For Making Build Stages Environments

If you’ve deployed or built a system for a large enterprise (especially those using SOX controls), you might have heard of different testing processes — Coding and Unit Testing (CUT), System Testing (ST), Integration Testing (Integ), QA Testing (QA), User Acceptance Testing (UAT), Production Testing (Prod), etc.

Each of these testing stages represent different…well, environments 🙂

For example, a UAT Environment is meant to replicate or mirror the next (and in this case final) stage of testing which is Production, but it differs from QA, Integ and System and the goal is to really subject the code to various environments, capturing as many errors as possible and fixing them before they get to production. CUT Environment is where the code was written, which is normally the laptop or computer of the developer(s).

So how does this example support the need for a virtual environment?

Well, let’s start at the CUT environment. The developer might have installed some libraries or modules into his/her workstation but forgot to include those in the Release Note/Notice and build artifacts. When this build gets to the next test stage, that build would fail because the target environment simply doesn’t have those new modules. If you have a similar environment as the next stage, you could test your build in it and catch those errors before actually sending it (saving you and the organization huge time and effort that is involved in build release process).

Working in a Virtual Environment

This tutorial is made/tested using Python 3.7.4

If you’re using an older version of Python, you would have to change the command slightly.

To check your Python version, just type:

python -V

Note that it’s a capital “V”.

Another way to check is using “–version” switch:

Creating a virtual environment

So if you’re now convinced or at least curious, how do you actually create one?

# for Python 2.7*
python -m virtualenv environment_name

# for Python 3.3* and upwards
python -m venv environment_name

For a more detailed reference, use the link above, but for this blog, I’ll create one so I can show you what it would look like once you do.

Activating a virtual environment

If you’ve successfully activated the environment, the prompt would have a prefix of the environment you activated and in this example, we saw the “(uat_env)” added to the left side of the command prompt.

Installing packages inside a virtual environment

Now, you can safely install all the packages needed to build that environment without affecting the global/system one.

Listing (freezing) the installed packages

The command “pip freeze” lists all the packages installed in the current environment (see below):

Saving the virtual environment

This exact same command (pip freeze) is what you’ll use to recreate this environment somewhere else, you just have to redirect the output to a file.

Once you have this requirements file, you can send it to someone so they can recreate this environment.

Restoring the virtual environment

Once you’re in a new environment (physical or virtual), you just run pip again, but this time specifying the file as input (using the “r” switch):

You now have replicated the UAT environment and can test any system written under it.

Can’t I just copy the environment folder?

Short answer: Yes, you can.

But, this is a huge folder so just be aware of that especially when moving across the network. However this isn’t really the reason why we use the requirements file to recreate an environment.

Cross-platform recreation

This is really the reason why the requirements file is the better choice.

Imagine if you’re a Windows machine but you want to test your codes in a Linux environment. Copying the env folder won’t work since the environment folder contains files that are specific to Windows.

Creating an environment in a Linux machine (using the same or compatible Python version) and running the “pip -r” will do the recreation properly/correctly on the Linux environment.

You don’t have to be in a team or a large environment to use a virtual environment. Even if you’re a solo coder you’d still benefit a lot from using a virtual environment as it organizes your packages/dependencies in a safe sandbox environment.