When working with Python, often you’ll end up with multiple Python scripts. And later, you may need them all together in a single Python file. While working on a web application development project or any major project, there would be 10s of files for a single project. Knowing how to manage and organize them is a crucial part, as an unorganized project will lead to inconsistency.
In this article, we’re going to see how organizing files in the right way is important for any programming project and also check how can we build a single Python file from multiple independent Python scripts. In the end, we’re also going to learn how to package your project and also learn how we can distribute it. In the process, we’ll also learn to create a setup.py file.
What is a Python module?
A Python script is any block of Pythonic code, i.e., it has some relevance for the Python interpreter. A Python module is a file that contains Python script. It can have Python code of any number of lines. We need a Python module to save a Python script. When you’re writing Python script in the terminal, as soon as you quit the Python interpreter, the script is lost. There’s no way to access the same script again or run it again. Therefore, we need a Python module for this purpose.
Related: Learn about a few popular modules of Python.
What is a Python package?
A Python package is a directory that contains Python modules and an __init__.py file. __init__.py is a constructor of a Python package and is called whenever a Python package has been imported. It is a way to create a folder of Python files and helps in organizing Python files.
A package of Python packages can be referred to as a library. A python library is created by senior programmers to help other programmers in their programming journey and make the hard jobs easy to do.
Related: What Is __init__.py Used For?
Combining multiple Python scripts
Combining multiple Python scripts means taking more than one Python file and creating a single file out of them. To do this, there are various methods. One of the easiest ways is to import all the Python scripts into a single Python file using the import statement. Powerful and efficient imports can be done by combining the import keyword with from keyword.
A little more advanced way of combining multiple Python files is by using a zipfile. A zip file compresses multiple files and creates a single archived file out of them.
How to combine multiple Python files using the Zipfile module
import zipfile
try:
with zipfile.ZipFile('combined_files.zip', 'w') as zipf:
zipf.write('file1.py')
zipf.write('file2.py')
zipf.write('file3.py')
print("created a zipfile succesfully!")
except:
raise Exception("Failed to combine the files!")
Now a zip file would have been created in the same directory where you have this particular Python module located.
When you unzip(uncompress) this zip file, you will find a folder with all the Python files you combined.
There you go, you have combined multiple Python scripts.
Running Python scripts from within a Python file
An alternative to the import statement is provided by the subprocess library. It allows you to run a few shell operations from within a Python script. We can use the subprocess library to run multiple Python files from a single Python script. The advantage of the subprocess library is you can run a Python file dynamically as user input during runtime.
So let’s create a Python file with some Python script in it.
Now that we have created a simple Python file, we just have to run it
So first, we imported the subprocess library. Then we used the run method of the subprocess library to run a Python file. The run method takes the file name as input. This way, you can loop this line of code to run as many files as you want or even mention which files you have to run statically.
Merging multiple Python files into a single Python file using the shutil and pathlib libraries
You can also combine the text of the Python files and create a single Python file out of it using the shutil and pathlib libraries.
import shutil
from pathlib import Path
try:
with open(Path('merged_file.py'), 'wb') as destination_file:
with open(Path('file1.py'), 'rb') as file1:
shutil.copyfileobj(file1, destination_file)
with open(Path('file2.py'), 'rb') as file2:
shutil.copyfileobj(file2, destination_file)
print("Files merged successfully!")
except :
raise exception("Cannot merge files!")
First, we imported both libraries. Then we opened the destination file using the with keyword. Then we copied the first file into the destination file and then the second one in a similar way in the same destination file resulting in a combined Python file with both Python scripts in it.
Packaging and Distributing Your Project
Now you can package and then distribute the project you created. Doing it is a bit of a tedious process, but if you stick to the method, you’ll be able to do it. To do so, you’ll first need to set up your package, and then once you get a wheel file, you can upload this wheel file to PyPI for distribution.
Let’s start with setting up our project with the setup.py file.
How to create a setup.py file for your project?
To set up your project, you would need the setuptools library. The setuptools provide functions to package your project and create a wheel file out of your project. First, you would need to install the setuptools using the following command.
pip install setuptools
Once you’ve installed the setuptools library, you will have to use the setup method to specify the metadata and set up your package.
from setuptools import setup, find_packages
setup(
name = 'my_project',
version = '1.0.0',
author = 'Kundan Singh',
description = 'This is a sample project for demonstration purpose.',
packages = find_packages(),
)
Now just run the following command in the terminal after changing the current directory to the directory where the setup.py file resides.
python setup.py sdist bdist_wheel
This will generate a wheel file which you can later upload to the PyPI for distribution purposes.
How to upload your project to the Python Package Index (PyPI)?
To upload a project on PyPI, you would have to first register an account on the PyPI website.
Once you input the details, click on the Create account button at the bottom. Now you should verify your email address in order to upload your project to the PyPI. After verification, you’ll have to use the twine library to upload your project to PyPI. Install the twine library with the following command.
pip install twine
Now you can upload your distribution project by uploading the wheel file inside of the dist folder you got after the setup using the following command.
twine upload dist/*
Conclusion
Those were the few ways to combine multiple Python files. All of these methods work great, and it doesn’t matter whichever you choose. You can choose the one which suits your requirements. You can even look for more ways, like the PyBreeder tool. You must always keep exploring new stuff as a programmer. That’s what makes a programmer a good programmer.
References
Official Python Documentation.
Stack Overflow thread for the same question.