Sometimes you only want to profile your Python program part of the time. For this use case, Fil provides a Python API.
Important: This API turns profiling on and off for the whole process! If you want more fine grained profiling, e.g. per thread, please file an issue.
Let’s you have some code that does the following:
def main(): config = load_config() result = run_processing(config) generate_report(result)
You only want to get memory profiling for the
You can do so in the code like so:
from filprofiler.api import profile def main(): config = load_config() result = profile(lambda: run_processing(config), "/tmp/fil-result") generate_report(result)
You could also make it conditional, e.g. based on an environment variable:
import os from filprofiler.api import profile def main(): config = load_config() if os.environ.get("FIL_PROFILE"): result = profile(lambda: run_processing(config), "/tmp/fil-result") else: result = run_processing(config) generate_report(result)
You still need to run your program in a special way. If previously you did:
$ python yourscript.py --config=myconfig
Now you would do:
$ fil-profile python yourscript.py --config=myconfig
Notice that you’re doing
python, rather than
fil-profile run as you would if you were profiling the full script.
Only functions running for the duration of the
filprofiler.api.profile() call will have memory profiling enabled, including of course the function you pass in.
The rest of the code will run at (close) to normal speed and configuration.
Each call to
profile() will generate a separate report.
The memory profiling report will be written to the directory specified as the output destination when calling
profile(); in or example above that was
Unlike full-program profiling:
- The directory you give will be used directly, there won’t be timestamped sub-directories.
If there are multiple calls to
profile(), it is your responsibility to ensure each call writes to a unique directory.
- The report(s) will not be opened in a browser automatically, on the presumption you’re running this in an automated fashion.