Run Python using a Shell Script

Let's say we have the following three files: config.py, main.py and experiments/test.sh.

config.py is in charge of (hyper-)parameters in main.py and we'd like to run main.py using test.sh.

# config.py

import argparse

def load_args():
    parser = argparse.ArgumentParser()

    parser.add_argument('--lr', default=1e-3, type=float)
    parser.add_argument('--h_layers', default=[2, 2, 2], type=lambda s: [int(item.strip()) for item in s.split(',')])
    parser.add_argument('--use_ar', default=False, type=lambda s: eval(s))
    args = parser.parse_args()
    return args

# main.py

import os
from config import load_args

print("os.getcwd():", os.getcwd())

args = load_args()
print(args)
print(type(args.h_layers))
print(type(args.use_ar))

## test

#!/bin/sh

cd "$PWD/.."
echo "PWD: $PWD"

echo "start main.py"
python main.py --h_layers "1,2,3,4" --use_ar "True"

You should notice the following things:

• main.py: type argument can take a function to convert 'string' to whatever you want. (reference)
• Note that 'argparse' always receives user input as 'string', and we need to change its type using the type argument.
• 'parser' cannot process 'type=bool'. Therefore, we should use a lambda function as shown.
• This shell programming tutorial is referred to.