Contents

7.2. Workflow Automation

This section covers some tools to automate your workflow of your Python project such as scheduling a time to run your code, send notifications when your program finishes, etc.

7.2.1. Schedule: Schedule your Python Functions to Run At a Specific Time

If you want to schedule Python functions to run periodically at a certain day or time of the week, use schedule.

In the code snippet below, I use schedule to get incoming data at 10:30 every day and train the model at 8:00 every Wednesday.

import schedule 
import time 

def get_incoming_data():
    print("Get incoming data")

def train_model():
    print("Retraining model")

schedule.every().day.at("10:30").do(get_incoming_data)
schedule.every().wednesday.at("08:00").do(train_model)

while True:
    schedule.run_pending()
    time.sleep(1)

Link to schedule

7.2.2. notify-send: Send a Desktop Notification after Finishing Executing a File

If you want to receive a desktop notification after finishing executing a file in Linux, use notify-send.

In the code below, after finishing executing file_to_run.py, you will receive a notification on the top of your screen to inform you that the process is terminated.

python file_to_run.py ; notify-send "Process terminated"

7.2.3. knockknock: Receive an Email When Your Code Finishes Executing

It can take hours or days to train a model and you can be away from the computer when your model finishes training. Wouldn’t it be nice to receive an email when your code finishes executing? There is an app for that knock-knock.

All it takes is one line of code specifying your email address.

from knockknock import email_sender 

@email_sender(recipient_emails=['<your_email@address.com>', '<your_second_email@adress.com>'],
sender_email="<grandma's_email@gmail.com>")
def train_your_nicest_model(your_nicest_parameters):
    import time 
    time.sleep(10_000)
    return {'loss': 0.9}

You can even have it send to your slack channel so everybody in your team can see. See the docs of this library here.

7.3. Code Review Automation

This section covers some tools to automatically review and improve your code such as sorting imports, check for missing docstrings, etc.

7.3.1. isort: Automatically Sort your Python Imports in 1 Line of Code

When your code grows bigger, you might need to import a lot of libraries, and it can be confusing to look at. Instead of manually organing your imports, use isort.

isort is a Python library to sort imports alphabetically and automatically separated into sections and by type. You just need to use isort name_of_your_file.py to sort your imports.

Below is how the imports look like before sorting.

from sklearn.metrics import confusion_matrix, fl_score, classification_report, roc_curve
from sklearn.model_selection import train_test_split
from sklearn.model_selection import GridSearchCV, StratifiedKFold
from sklearn import svm
from sklearn.naive_bayes import GaussianNB, MultinomialNB
from sklearn.neighbors import KNeighborsClassifier
from sklearn.tree import DecisionTreeClassifier
from sklearn.model_selection import TimeSeriesSplit

On your terminal, type:

isort name_of_your_file.py

Now the imports are much more organized!

from sklearn import svm
from sklearn.metrics import (classification_report, confusion_matrix, fl_score,
                             roc_curve)
from sklearn.model_selection import (GridSearchCV, StratifiedKFold,
                                     TimeSeriesSplit, train_test_split)
from sklearn.naive_bayes import GaussianNB, MultinomialNB
from sklearn.neighbors import KNeighborsClassifier
from sklearn.tree import DecisionTreeClassifier

Link to isort.

7.3.2. interrogate: Check your Python Code for Missing Docstrings

!pip install interrogate

Sometimes, you might forget to write docstrings for classes and functions. Instead of manually looking at all your functions and classes for missing docstrings, use interrogate instead.

In the code below, I use interrogate to check for missing docstrings in the file interrogate_example.py.

# interrogate_example.py
class Math:
    def __init__(self, num) -> None:
        self.num = num

    def plus_two(self):
        """Add 2"""
        return self.num + 2

    def multiply_three(self):
        return self.num * 3

On your terminal, type:

interrogate interrogate_example.py

Output:

!interrogate interrogate_example.py

RESULT: FAILED (minimum: 80.0%, actual: 20.0%)

To automatically check for missing docstrings whenever you commit new files, add interrogate to your pre-commit hooks. Here is how to do that.

Link to interrogate.

7.3.3. mypy: Static Type Checker for Python

!pip install mypy

Type hinting in Python is useful for other developers to understand which data types to insert into your function. To automatically type check your code, use mypy.

To see how mypy works, start with writing a normal code that uses type hinting. We name this file mypy_example.py

# mypy_example.py
from typing import List, Union

def get_name_price(fruits: list) -> Union[list, tuple]:
    return zip(*fruits)

fruits = [('apple', 2), ('orange', 3), ('grape', 2)]
names, prices = get_name_price(fruits)
print(names)  # ('apple', 'orange', 'grape')
print(prices)  # (2, 3, 2)

On your terminal, type:

mypy mypy_example.py

And you should see something like below:

!mypy mypy_example.py

mypy_example.py:4: error: Incompatible return value type (got "zip[Any]", expected "Union[List[Any], Tuple[Any, ...]]")
Found 1 error in 1 file (checked 1 source file)

Link to mypy.