pip3 install pillar
import pillar.application
class MyApplication(pillar.application.Application):
@@ -794,6 +915,86 @@ Sample Application app = MyApplication()
app.run()
% python pillar_sample_app.py
+[23:54:32][ERROR]: Is anyone there? TT_TT
+
+% python pillar_sample_app.py Nick
+[23:56:44][INFO]: Hello Nick. It's quite lonely in here isn't it?
+
+% python pillar_sample_app.py Nick Fred
+[23:57:10][INFO]: Hello Nick and Fred.
+
+% python pillar_sample_app.py Nick Fred James
+[23:57:37][INFO]: Hello Nick, Fred, and James
+
+% python pillar_sample_app.py Nick Fred James Alice Bob Malory
+[23:58:18][CRITICAL]: Sorry but there are too many of you 0_0
+% python pillar_sample_app.py --help
+usage: pillar_sample_app.py [-h] [-c CONFIG_PATH] [-v] [--log-dir PATH] [--version] [NAME ...]
+
+This is a sample application to show off Pillar.
+
+The class docstring will be used as the description of the program it it is set.
+
+positional arguments:
+ NAME Name(s) to say hello to.
+
+options:
+ -h, --help show this help message and exit
+ -c CONFIG_PATH, --config CONFIG_PATH
+ Add a config file to parse. Config files are parsed in the order they are
+ added with values being merged into the previously parsed config.
+ -v, --verbose Increase logging verbosity
+ --log-dir PATH Set where log files should be stored. Defaults to /var/tmp
+ --version show program's version number and exit
+
+Copyright (c) The Pillar Authors
+
+% python pillar_sample_app.py --version
+1.2.3
+To view the logs we can use the --verbose argument multiple times, or use the shortened form -v.
% python pillar_sample_app.py -v Nick Fred James Alice Bob Malory
+[23:59:36][DEBUG]: There are 6
+[23:59:36][CRITICAL]: Sorry but there are too many of you 0_0
+
+% python pillar_sample_app.py -vvv Nick Fred James Alice Bob Malory
+[23:59:57][DEBUG]: There are 6
+[23:59:57][VDEBUG]: Our config is {'too_many_names': 5}
+[23:59:57][CRITICAL]: Sorry but there are too many of you 0_0
+We know that we have a default config with a too_many_names set to 5. Let's write our own config file to increase. By default Pillar supports JSON and YAML config files.
Let's call our program again with the new config file.
+% python pillar_sample_app.py -c pillar_sample_config.yaml -vv Nick Fred James Alice Bob Malory
+[00:02:36][DEBUG]: There are 6
+[00:02:36][VDEBUG]: Our config is {'too_many_names': 10}
+[00:02:36][INFO]: Hello Nick, Fred, James, Alice, Bob, and Malory
+All our output so far has been through the console. However Pillar will also have written logs to /var/tmp.
% tail /var/tmp/pillar_sample_app.log
+[20240101T235737+1100][2407242][pillar_sample_app][INFO]: Hello Nick, Fred, and James
+[20240101T235818+1100][2407254][pillar_sample_app][CRITICAL]: Sorry but there are too many of you 0_0
+[20240101T235936+1100][2407292][pillar_sample_app][DEBUG]: There are 6
+[20240101T235936+1100][2407292][pillar_sample_app][CRITICAL]: Sorry but there are too many of you 0_0
+[20240101T235957+1100][2407299][pillar_sample_app][DEBUG]: There are 6
+[20240101T235957+1100][2407299][pillar_sample_app][VDEBUG]: Our config is {'too_many_names': 5}
+[20240101T235957+1100][2407299][pillar_sample_app][CRITICAL]: Sorry but there are too many of you 0_0
+[20240102T000236+1100][2407433][pillar_sample_app][DEBUG]: There are 6
+[20240102T000236+1100][2407433][pillar_sample_app][VDEBUG]: Our config is {'too_many_names': 10}
+[20240102T000236+1100][2407433][pillar_sample_app][INFO]: Hello Nick, Fred, James, Alice, Bob, and Malory
+Pillar will also create a basic log file early during the setup process to capture any errors that occur before full logging is available. Let's check it too:
+ diff --git a/0.1.0/search/search_index.json b/0.1.0/search/search_index.json index e948ae7..ddcec85 100644 --- a/0.1.0/search/search_index.json +++ b/0.1.0/search/search_index.json @@ -1 +1 @@ -{"config":{"lang":["en"],"separator":"[\\s\\-]+","pipeline":["stopWordFilter"]},"docs":[{"location":"","title":"Pillar: Building Blocks for Python Applications","text":""},{"location":"#introduction","title":"Introduction","text":"Pillar provides resusable components for building command-line applications. No more copy-pasting boilerplate for logging or argparse just to get bare-bone features.
In particular Pillar focusses on providing components for applications that are run as services, background tasks, or scheduled jobs (e.g. cron jobs).
Pillar is currently Alpha software and does not have complete documentation, testing, or implementation of certain features.
"},{"location":"#features","title":"Features","text":"Tools like Click, Fire, and Typer are focussed on CLI interfaces and how to get the command-line arguments to the code that runs them. Whilst Pillar can be used for extensive CLIs, it is better suited to single purpose applications like cron jobs. That's not to say that it doesn't support command-line arguments, as it still uses and exposes argparse for you to add your own.
Frameworks like Cement are powerful, but leads to a lot of boilerplate when creating lots of small applications. In contrast Pillar provides most of it's core functionality through a single class that your application subclasses from. You are able to customise and plug-and-play many components, but components are also fairly opinioated so that you don't have to remeber to customise them every time.
"},{"location":"#quick-start","title":"Quick Start","text":"Follow our Quickstart Guide.
tldr.pyfrom pillar.application import Application\n\nclass MyApplication(Application):\n \"Hello World application that prints basic info then exits\"\n\n def main(self):\n self.info(f\"hello world! My name is {self.name}\")\n self.debug(f\"My config is: {self.config}\")\n return\n\nif __name__ == \"__main__\":\n app = MyApplication()\n app.run()\nPlease submit an issue on github.
In the case of bug reports, please help us help you by following best practices 1 2.
In the case of feature requests, please provide background to the problem you are trying to solve so to help find a solution that makes the most sense for the library as well as your usecase.
"},{"location":"#contributing","title":"Contributing","text":"I am still working through open source licencing and contributing, so not taking PRs at this point in time. Instead raise and issue and I'll try get to it as soon a feasible.
"},{"location":"#licence","title":"Licence","text":"This project is licenced under the MIT Licence - see LICENCE.
This project includes other open source licenced software - see NOTICE.
A project by Nicholas Hairs - www.nicholashairs.com.
"},{"location":"changelog/","title":"Change Log","text":""},{"location":"changelog/#010","title":"0.1.0","text":"Recommended
The recommended install target will include libraries to support common usecases of Pillar. This includes a number of libraries that are not pure Python, and thus may have install requirements outside of pip - they are typically included as they provide more performant versions of standard libraries.
To install Pillar with recommended packages use:
pip3 install pillar[recommended]\nMinimal
A minimal installation will only include libraries to perform core functionality. This can be useful if you are intending to only use parts of Pillar in another program without using the Pillar Application.
To perform a minimal install use:
pip3 install pillar\nimport pillar.application\n\nclass MyApplication(pillar.application.Application):\n \"\"\"This is a sample application to show off Pillar.\n\n The class docstring will be used as the description of the program it it is set.\n \"\"\"\n\n # We can add an argparse epilog\n epilog = \"Copyright (c) The Pillar Authors\"\n\n # We can also add an argparse version\n version = \"1.2.3\"\n\n # Set a default config\n default_config = {\"too_many_names\": 5}\n\n # Customise the ArgumentParsrer\n def get_argument_parser(self):\n # First we generate the default one\n parser = super().get_argument_parser()\n\n # Now lets customise it\n parser.add_argument(\n \"names\",\n metavar=\"NAME\",\n nargs=\"*\",\n default=[],\n help=\"Name(s) to say hello to.\"\n )\n return parser\n\n def main(self):\n # We can access the parsed arguments with self.args\n self.debug(f\"There are {len(self.args.names)}\")\n\n # We can access the loaded config using self.config\n self.vdebug(f\"Our config is {self.config}\")\n\n # Let's write a silly little program\n\n if len(self.args.names) >= self.config[\"too_many_names\"]:\n self.critical(\"Sorry but there are too many of you 0_0\")\n # if main returns an integer it will be used for the\n # exit code of the program.\n return 1\n\n if len(self.args.names) == 0:\n self.error(\"Is anyone there? TT_TT\")\n return 0\n\n if len(self.args.names) == 1:\n self.info(f\"Hello {self.args.names[0]}. It's quite lonely in here isn't it?\")\n elif len(self.args.names) == 2:\n self.info(f\"Hello {self.args.names[0]} and {self.args.names[1]}.\")\n else:\n self.info(f\"Hello {', '.join(self.args.names[:-1])}, and {self.args.names[-1]}\")\n\n return 0\n\nif __name__ == \"__main__\":\n app = MyApplication()\n app.run()\nApplication(argv: Optional[List[str]] = None)\n Bases: LoggingMixin
Base class for running applications.
Child classes should override the self.main method with their application logic.
Some attributes should be set on the class while others will be computed and set on the instance. All attributes that are set on the class have default value and are optional. They are tagged as (class) below.
Important: the types in the Attributes table below represent the type on the instance, some of these (e.g. name) shadow the ones set on the class (on purpose).
Attributes:
Name Type Descriptionname str (class) name of application - used in logging.
application_name str (class) argparse prog name.
epilog Optional[str] (class) argparse help text epilog (e.g. copyright info).
version Optional[str] (class) version of application - used in argparse help text.
logging_manifest LoggingManifest (class) configuration of application's logging.
config_args_enabled bool (class) enable collecting config from args.
config_required bool (class) providing a config file via args is mandatory.
default_config Dict[str, Any] (class) default config of the application
config_loader_class Type[ConfigLoader] (class) ConfigLoader to use with the application.
logger Logger logger of application
log_level int computed logging level
args Namespace parsed arguments from argparse
config_loader ConfigLoader the ConfigLoader of the application.
config Dict[str, Any] the config_loader.config dict.
Parameters:
Name Type Description Defaultargv Optional[List[str]] arguments to pass to this application. If None will collect from sys.argv instead.
None"},{"location":"reference/pillar/application/#pillar.application.Application.critical","title":"critical","text":"critical(msg: str, *args, **kwargs) -> None\nLog a CRITICAL message
Something is on fire. We somehow caught it but we probably need to exit now. If we keep going more things may catch on fire.
In a larger system, someone is probably going to get paged over this. An end user is definitely going to get an error message, probably not even a useful one, just a HTTP 500.
"},{"location":"reference/pillar/application/#pillar.application.Application.debug","title":"debug","text":"debug(msg: str, *args, **kwargs) -> None\nLog a DEBUG message
Basic debug messages
"},{"location":"reference/pillar/application/#pillar.application.Application.error","title":"error","text":"error(msg: str, *args, **kwargs) -> None\nLog an ERROR message
Something bad has happened but we caught it. We might be able to continue, but other things might start breaking. We can probably still safely exit.
In a larger system, this will likely cause a gentle alert to be placed somewhere. An end user might receive a useful error message (like a HTTP 4xx 5xx).
"},{"location":"reference/pillar/application/#pillar.application.Application.get_argument_parser","title":"get_argument_parser","text":"get_argument_parser() -> argparse.ArgumentParser\nGet the argument parser for this application.
To add your own arguments you should override this method and call super().get_argument_parser().
info(msg: str, *args, **kwargs) -> None\nLog an INFO message
Print something to the screen/logfile so we know what is happening
"},{"location":"reference/pillar/application/#pillar.application.Application.main","title":"main","text":"main() -> Optional[int]\nMain application entrypoint.
Child classes MUST implement this method.
Returns:
Type DescriptionOptional[int] An integer exit code. None will be converted to 0 during application exit.
run(*, prevent_exit: bool = False) -> int\nRun this application until completion.
Parameters:
Name Type Description Defaultprevent_exit bool Do not call sys.exit and instead allow the method to return.
False Returns:
Type Descriptionint the exit_code that would have been passed to sys.exit
setup(*, suppress_warning: bool = False) -> None\nPrepare the application to be run
In no particular order, this will:
Once called the following instance attributes will be populated:
self.argsself.configself.config_loaderself.log_levelGenerally you will not need to call this method as it is called during self.run.
This method may only be called once per an instance. If it is called multiple times, subsequent calls will have no effect and a RuntimeWarning will be emited.
If this method is overridden in a child class, it should call super().setup() or otherwise set self._setup_called to True to indicate the the application has been correctly setup.
Parameters:
Name Type Description Defaultsupress_warning Suppress the RuntimeWarning if this method is called multiple times.
vdebug(msg: str, *args, **kwargs) -> None\nLog a Verbose Debug (VDEBUG) message.
More than debug, less than everything.
"},{"location":"reference/pillar/application/#pillar.application.Application.vvdebug","title":"vvdebug","text":"vvdebug(msg: str, *args, **kwargs) -> None\nLog a Very Verbose Debug (VVDEBUG) message.
When you're tired of finding the bug and want to log everything.
"},{"location":"reference/pillar/application/#pillar.application.Application.warning","title":"warning","text":"warning(msg: str, *args, **kwargs) -> None\nLog a WARNING message
Something is wrong but we likely can recover or skip it without issue.
In a larger system this will likely just go to centralised logging.
"},{"location":"reference/pillar/application/#pillar.application.LoggingManifest","title":"LoggingManifest","text":"Simplified configuration of an Application's logging.
Be careful when modifying these options. Errors during logging setup may cause the application to error before any logging is setup causing errors to not be sent to the log files (making debugging much harder). The initial logging will use settings from both the console and file logging settings.
Attributes:
Name Type Descriptiondefault_level int Default logging level. Actual log level will be computed from this level and the application's verbosity args.
additional_namespaces List[str] Additional namespaces that should be logged.
initial_log_location str Directory for storing the initial log
initial_log_filename str filename of the initial log. Will be formatted with the application's name.
initial_log_level Union[str, int] logging level of initial log.
console_stream IO stream to output console logging to
console_format str text format for console logging
console_format_style str console_format format style
console_date_format str console_format date format
file_enabled bool enable logging to files.
file_default_location str default log file location. Actual location may be changed from the applications log-dir arg.
file_filename str filename of log. Will be formatted with the applications name.
file_format str text format for file logging
file_format_stle str file_format format style
file_date_format str file_format date format
file_max_size int max size of a log file in bytes (see RotatingFileHandler).
file_backup_count int number of old log files to keep (see RotatingFileHandler).
ConfigLoader(\n default_config: Optional[Dict[str, Any]] = None,\n default_parsers: Optional[\n Dict[str, ConfigParser]\n ] = None,\n)\n Bases: LoggingMixin
Load and merge multiple config files from various locations.
Attributes:
Name Type Descriptionlogger Logger merger Merger deepmerge merger
parsers Dict[str, ConfigParser] loaded parses
config Dict[str, Any] computed config
load_stack Dict[str, Dict[str, Any]] loaded files and their respective config before merging. Because dictionaries are ordered this will also be in order of loading. The only only exception is if the same file is loaded multiple times (don't do that).
load_errors Dict[str, Dict[str, Any]] errors encountered when loading files. If the same file is loaded multiple times, it will only have the latest error encountered.
Parameters:
Name Type Description Defaultdefault_config Optional[Dict[str, Any]] Set initial self.config to this.
None default_parsers Optional[Dict[str, ConfigParser]] Use these parsers instead of the DEFAULT_PARSERS.
None"},{"location":"reference/pillar/config/#pillar.config.ConfigLoader.critical","title":"critical","text":"critical(msg: str, *args, **kwargs) -> None\nLog a CRITICAL message
Something is on fire. We somehow caught it but we probably need to exit now. If we keep going more things may catch on fire.
In a larger system, someone is probably going to get paged over this. An end user is definitely going to get an error message, probably not even a useful one, just a HTTP 500.
"},{"location":"reference/pillar/config/#pillar.config.ConfigLoader.debug","title":"debug","text":"debug(msg: str, *args, **kwargs) -> None\nLog a DEBUG message
Basic debug messages
"},{"location":"reference/pillar/config/#pillar.config.ConfigLoader.error","title":"error","text":"error(msg: str, *args, **kwargs) -> None\nLog an ERROR message
Something bad has happened but we caught it. We might be able to continue, but other things might start breaking. We can probably still safely exit.
In a larger system, this will likely cause a gentle alert to be placed somewhere. An end user might receive a useful error message (like a HTTP 4xx 5xx).
"},{"location":"reference/pillar/config/#pillar.config.ConfigLoader.info","title":"info","text":"info(msg: str, *args, **kwargs) -> None\nLog an INFO message
Print something to the screen/logfile so we know what is happening
"},{"location":"reference/pillar/config/#pillar.config.ConfigLoader.load_config","title":"load_config","text":"load_config(\n path: Union[str, Path], *, suppress_errors: bool = False\n) -> None\nLoad a given config path.
The loaded config will be merged into self.config. The actual config loaded will also be stored in self.load_stack[path]. Any errors during loading will be stored in self.load_errors[path].
Parameters:
Name Type Description Defaultpath Union[str, Path] Path to try load from. Can include scheme (e.g. file://config/dev.json)
suppress_errors bool Prevent errors from being thrown while loading the config. This does not affect errors being stored in self.load_errors.
False Raises:
Type DescriptionValueError Unsupported path scheme
"},{"location":"reference/pillar/config/#pillar.config.ConfigLoader.vdebug","title":"vdebug","text":"vdebug(msg: str, *args, **kwargs) -> None\nLog a Verbose Debug (VDEBUG) message.
More than debug, less than everything.
"},{"location":"reference/pillar/config/#pillar.config.ConfigLoader.vvdebug","title":"vvdebug","text":"vvdebug(msg: str, *args, **kwargs) -> None\nLog a Very Verbose Debug (VVDEBUG) message.
When you're tired of finding the bug and want to log everything.
"},{"location":"reference/pillar/config/#pillar.config.ConfigLoader.warning","title":"warning","text":"warning(msg: str, *args, **kwargs) -> None\nLog a WARNING message
Something is wrong but we likely can recover or skip it without issue.
In a larger system this will likely just go to centralised logging.
"},{"location":"reference/pillar/config/#pillar.config.ConfigParser","title":"ConfigParser","text":"Base class for config loaders.
"},{"location":"reference/pillar/config/#pillar.config.ConfigParser.parse_content","title":"parse_content","text":"parse_content(content: bytes) -> Any\nParse content into usable config.
Parameters:
Name Type Description Defaultcontent bytes Raw content to load from
required"},{"location":"reference/pillar/config/#pillar.config.JsonParser","title":"JsonParser","text":" Bases: ConfigParser
Config parser for JSON
If orjson is installed will use that for parsing. It can be installed using pillar[recommended].
parse_content(content: bytes) -> Any\nParse config into a usable config.
See pillar.config.ConfigParser.parse_content.
"},{"location":"reference/pillar/config/#pillar.config.YamlParser","title":"YamlParser","text":" Bases: ConfigParser
Config parser for YAML
"},{"location":"reference/pillar/config/#pillar.config.YamlParser.parse_content","title":"parse_content","text":"parse_content(content: bytes) -> Any\nParse config into a usable config.
See pillar.config.ConfigParser.parse_content.
"},{"location":"reference/pillar/dataclass/","title":"dataclass_slots_kwargs() -> Dict[str, Any]\nGenerate dataclass slots keyword argument if it is supported by this python version.
If it is not supported contains an empty dictionary.
To use it unpack it like so @dataclass(**dataclass_slots_kwargs())
MissingDependencyError(\n feature: str,\n required_package: str,\n package_name: str,\n optional_dependency: str,\n)\n Bases: Exception
Exception for when an optional dependency is required but not installed
Parameters:
Name Type Description Defaultfeature str Name of the feature. This could be a word, phrase, or longer sentence.
requiredrequired_package str Name of the package we were trying to import. Doesn't need to be the actual import name, could be it's colloquial or PyPI name.
requiredpackage_name str Name of our package. Used to generate install instructions.
requiredoptional_dependency str Name of the optional dependency that should be installed. Used to generate install instructions.
required"},{"location":"reference/pillar/exception/#pillar.exception.PillarException","title":"PillarException","text":"PillarException(*args, **kwargs)\n Bases: Exception
Base class for all Pillar specific exceptions
"},{"location":"reference/pillar/exception/#pillar.exception.PillarMissingDependencyError","title":"PillarMissingDependencyError","text":"PillarMissingDependencyError(\n feature: str,\n required_package: str,\n optional_dependency: str,\n)\n Bases: PillarException, MissingDependencyError
Pillar is missing a required package.
As per MissingDependencyError with package_name set to \"pillar\"
Bases: LoggingMixinBase
Adds shortcut logging methods to a class
Expects that a logging.Logger exists at self.logger
critical(msg: str, *args, **kwargs) -> None\nLog a CRITICAL message
Something is on fire. We somehow caught it but we probably need to exit now. If we keep going more things may catch on fire.
In a larger system, someone is probably going to get paged over this. An end user is definitely going to get an error message, probably not even a useful one, just a HTTP 500.
"},{"location":"reference/pillar/logging/#pillar.logging.LoggingMixin.debug","title":"debug","text":"debug(msg: str, *args, **kwargs) -> None\nLog a DEBUG message
Basic debug messages
"},{"location":"reference/pillar/logging/#pillar.logging.LoggingMixin.error","title":"error","text":"error(msg: str, *args, **kwargs) -> None\nLog an ERROR message
Something bad has happened but we caught it. We might be able to continue, but other things might start breaking. We can probably still safely exit.
In a larger system, this will likely cause a gentle alert to be placed somewhere. An end user might receive a useful error message (like a HTTP 4xx 5xx).
"},{"location":"reference/pillar/logging/#pillar.logging.LoggingMixin.info","title":"info","text":"info(msg: str, *args, **kwargs) -> None\nLog an INFO message
Print something to the screen/logfile so we know what is happening
"},{"location":"reference/pillar/logging/#pillar.logging.LoggingMixin.vdebug","title":"vdebug","text":"vdebug(msg: str, *args, **kwargs) -> None\nLog a Verbose Debug (VDEBUG) message.
More than debug, less than everything.
"},{"location":"reference/pillar/logging/#pillar.logging.LoggingMixin.vvdebug","title":"vvdebug","text":"vvdebug(msg: str, *args, **kwargs) -> None\nLog a Very Verbose Debug (VVDEBUG) message.
When you're tired of finding the bug and want to log everything.
"},{"location":"reference/pillar/logging/#pillar.logging.LoggingMixin.warning","title":"warning","text":"warning(msg: str, *args, **kwargs) -> None\nLog a WARNING message
Something is wrong but we likely can recover or skip it without issue.
In a larger system this will likely just go to centralised logging.
"},{"location":"reference/pillar/logging/#pillar.logging.LoggingMixinBase","title":"LoggingMixinBase","text":"Base class for logging mixins
"},{"location":"reference/pillar/logging/#pillar.logging.NotFilter","title":"NotFilter","text":"NotFilter(name: str)\n Bases: Filter
Ignore the given logger (and all subloggers).
Is the opposite of logging.Filter.
filter(record) -> bool\nAs per logging.Filter.filter
Bases: LoggingMixinBase
Add shortcut logging methods to a class with underscore (_) prefix
Expects that a logging.Logger exists at self._logger.
get_log_level(verbosity: int, default_level: int) -> int\nGet the log level based on the default log level.
This function essentially adds a \"verbosity level\" per \"major log level\" lower than the default.
For example, if default_level == DEBUG, then verbosity == 1 will return VDEBUG and verbosity >= 2 will return VVDBEUG. For any defaul_level > INFO will always use 4 levels of verbosity (INFO, DEBUG, VDEBUG, VVDEBUG).
logging_file_handler_errors_kwargs(\n errors: str,\n) -> Dict[str, str]\nGenerate FileHandler keyword argument if it is supported by this python version.
If it is not supported contains an empty dictionary.
To use it unpack it like so FileHandler(**logging_file_handler_errors_kwargs())
Pillar provides resusable components for building command-line applications. No more copy-pasting boilerplate for logging or argparse just to get bare-bone features.
In particular Pillar focusses on providing components for applications that are run as services, background tasks, or scheduled jobs (e.g. cron jobs).
Pillar is currently Alpha software and does not have complete documentation, testing, or implementation of certain features.
"},{"location":"#features","title":"Features","text":"Tools like Click, Fire, and Typer are focussed on CLI interfaces and how to get the command-line arguments to the code that runs them. Whilst Pillar can be used for extensive CLIs, it is better suited to single purpose applications like cron jobs. That's not to say that it doesn't support command-line arguments, as it still uses and exposes argparse for you to add your own.
Frameworks like Cement are powerful, but leads to a lot of boilerplate when creating lots of small applications. In contrast Pillar provides most of it's core functionality through a single class that your application subclasses from. You are able to customise and plug-and-play many components, but components are also fairly opinioated so that you don't have to remeber to customise them every time.
"},{"location":"#quick-start","title":"Quick Start","text":"Follow our Quickstart Guide.
tldr.pyfrom pillar.application import Application\n\nclass MyApplication(Application):\n \"Hello World application that prints basic info then exits\"\n\n def main(self):\n self.info(f\"hello world! My name is {self.name}\")\n self.debug(f\"My config is: {self.config}\")\n return\n\nif __name__ == \"__main__\":\n app = MyApplication()\n app.run()\nPlease submit an issue on github.
In the case of bug reports, please help us help you by following best practices 1 2.
In the case of feature requests, please provide background to the problem you are trying to solve so to help find a solution that makes the most sense for the library as well as your usecase.
"},{"location":"#contributing","title":"Contributing","text":"I am still working through open source licencing and contributing, so not taking PRs at this point in time. Instead raise and issue and I'll try get to it as soon a feasible.
"},{"location":"#licence","title":"Licence","text":"This project is licenced under the MIT Licence - see LICENCE.
This project includes other open source licenced software - see NOTICE.
A project by Nicholas Hairs - www.nicholashairs.com.
"},{"location":"changelog/","title":"Change Log","text":""},{"location":"changelog/#010","title":"0.1.0","text":"Recommended
The recommended install target will include libraries to support common usecases of Pillar. This includes a number of libraries that are not pure Python, and thus may have install requirements outside of pip - they are typically included as they provide more performant versions of standard libraries.
To install Pillar with recommended packages use:
pip3 install pillar[recommended]\nMinimal
A minimal installation will only include libraries to perform core functionality. This can be useful if you are intending to only use parts of Pillar in another program without using the Pillar Application.
To perform a minimal install use:
pip3 install pillar\nimport pillar.application\n\nclass MyApplication(pillar.application.Application):\n \"\"\"This is a sample application to show off Pillar.\n\n The class docstring will be used as the description of the program it it is set.\n \"\"\"\n\n # We can add an argparse epilog\n epilog = \"Copyright (c) The Pillar Authors\"\n\n # We can also add an argparse version\n version = \"1.2.3\"\n\n # Set a default config\n default_config = {\"too_many_names\": 5}\n\n # Customise the ArgumentParsrer\n def get_argument_parser(self):\n # First we generate the default one\n parser = super().get_argument_parser()\n\n # Now lets customise it\n parser.add_argument(\n \"names\",\n metavar=\"NAME\",\n nargs=\"*\",\n default=[],\n help=\"Name(s) to say hello to.\"\n )\n return parser\n\n def main(self):\n # We can access the parsed arguments with self.args\n self.debug(f\"There are {len(self.args.names)}\")\n\n # We can access the loaded config using self.config\n self.vdebug(f\"Our config is {self.config}\")\n\n # Let's write a silly little program\n\n if len(self.args.names) >= self.config[\"too_many_names\"]:\n self.critical(\"Sorry but there are too many of you 0_0\")\n # if main returns an integer it will be used for the\n # exit code of the program.\n return 1\n\n if len(self.args.names) == 0:\n self.error(\"Is anyone there? TT_TT\")\n return 0\n\n if len(self.args.names) == 1:\n self.info(f\"Hello {self.args.names[0]}. It's quite lonely in here isn't it?\")\n elif len(self.args.names) == 2:\n self.info(f\"Hello {self.args.names[0]} and {self.args.names[1]}.\")\n else:\n self.info(f\"Hello {', '.join(self.args.names[:-1])}, and {self.args.names[-1]}\")\n\n return 0\n\nif __name__ == \"__main__\":\n app = MyApplication()\n app.run()\n% python pillar_sample_app.py\n[23:54:32][ERROR]: Is anyone there? TT_TT\n\n% python pillar_sample_app.py Nick\n[23:56:44][INFO]: Hello Nick. It's quite lonely in here isn't it?\n\n% python pillar_sample_app.py Nick Fred\n[23:57:10][INFO]: Hello Nick and Fred.\n\n% python pillar_sample_app.py Nick Fred James\n[23:57:37][INFO]: Hello Nick, Fred, and James\n\n% python pillar_sample_app.py Nick Fred James Alice Bob Malory\n[23:58:18][CRITICAL]: Sorry but there are too many of you 0_0\n% python pillar_sample_app.py --help\nusage: pillar_sample_app.py [-h] [-c CONFIG_PATH] [-v] [--log-dir PATH] [--version] [NAME ...]\n\nThis is a sample application to show off Pillar.\n\nThe class docstring will be used as the description of the program it it is set.\n\npositional arguments:\n NAME Name(s) to say hello to.\n\noptions:\n -h, --help show this help message and exit\n -c CONFIG_PATH, --config CONFIG_PATH\n Add a config file to parse. Config files are parsed in the order they are\n added with values being merged into the previously parsed config.\n -v, --verbose Increase logging verbosity\n --log-dir PATH Set where log files should be stored. Defaults to /var/tmp\n --version show program's version number and exit\n\nCopyright (c) The Pillar Authors\n\n% python pillar_sample_app.py --version\n1.2.3\nTo view the logs we can use the --verbose argument multiple times, or use the shortened form -v.
% python pillar_sample_app.py -v Nick Fred James Alice Bob Malory\n[23:59:36][DEBUG]: There are 6\n[23:59:36][CRITICAL]: Sorry but there are too many of you 0_0\n\n% python pillar_sample_app.py -vvv Nick Fred James Alice Bob Malory\n[23:59:57][DEBUG]: There are 6\n[23:59:57][VDEBUG]: Our config is {'too_many_names': 5}\n[23:59:57][CRITICAL]: Sorry but there are too many of you 0_0\nWe know that we have a default config with a too_many_names set to 5. Let's write our own config file to increase. By default Pillar supports JSON and YAML config files.
too_many_names: 10\nLet's call our program again with the new config file.
% python pillar_sample_app.py -c pillar_sample_config.yaml -vv Nick Fred James Alice Bob Malory\n[00:02:36][DEBUG]: There are 6\n[00:02:36][VDEBUG]: Our config is {'too_many_names': 10}\n[00:02:36][INFO]: Hello Nick, Fred, James, Alice, Bob, and Malory\nAll our output so far has been through the console. However Pillar will also have written logs to /var/tmp.
% tail /var/tmp/pillar_sample_app.log\n[20240101T235737+1100][2407242][pillar_sample_app][INFO]: Hello Nick, Fred, and James\n[20240101T235818+1100][2407254][pillar_sample_app][CRITICAL]: Sorry but there are too many of you 0_0\n[20240101T235936+1100][2407292][pillar_sample_app][DEBUG]: There are 6\n[20240101T235936+1100][2407292][pillar_sample_app][CRITICAL]: Sorry but there are too many of you 0_0\n[20240101T235957+1100][2407299][pillar_sample_app][DEBUG]: There are 6\n[20240101T235957+1100][2407299][pillar_sample_app][VDEBUG]: Our config is {'too_many_names': 5}\n[20240101T235957+1100][2407299][pillar_sample_app][CRITICAL]: Sorry but there are too many of you 0_0\n[20240102T000236+1100][2407433][pillar_sample_app][DEBUG]: There are 6\n[20240102T000236+1100][2407433][pillar_sample_app][VDEBUG]: Our config is {'too_many_names': 10}\n[20240102T000236+1100][2407433][pillar_sample_app][INFO]: Hello Nick, Fred, James, Alice, Bob, and Malory\nPillar will also create a basic log file early during the setup process to capture any errors that occur before full logging is available. Let's check it too:
% tail /var/tmp/pillar_sample_app.init.log\n<should be empty>\nApplication(argv: Optional[List[str]] = None)\n Bases: LoggingMixin
Base class for running applications.
Child classes should override the self.main method with their application logic.
Some attributes should be set on the class while others will be computed and set on the instance. All attributes that are set on the class have default value and are optional. They are tagged as (class) below.
Important: the types in the Attributes table below represent the type on the instance, some of these (e.g. name) shadow the ones set on the class (on purpose).
Attributes:
Name Type Descriptionname str (class) name of application - used in logging.
application_name str (class) argparse prog name.
epilog Optional[str] (class) argparse help text epilog (e.g. copyright info).
version Optional[str] (class) version of application - used in argparse help text.
logging_manifest LoggingManifest (class) configuration of application's logging.
config_args_enabled bool (class) enable collecting config from args.
config_required bool (class) providing a config file via args is mandatory.
default_config Dict[str, Any] (class) default config of the application
config_loader_class Type[ConfigLoader] (class) ConfigLoader to use with the application.
logger Logger logger of application
log_level int computed logging level
args Namespace parsed arguments from argparse
config_loader ConfigLoader the ConfigLoader of the application.
config Dict[str, Any] the config_loader.config dict.
Parameters:
Name Type Description Defaultargv Optional[List[str]] arguments to pass to this application. If None will collect from sys.argv instead.
None"},{"location":"reference/pillar/application/#pillar.application.Application.critical","title":"critical","text":"critical(msg: str, *args, **kwargs) -> None\nLog a CRITICAL message
Something is on fire. We somehow caught it but we probably need to exit now. If we keep going more things may catch on fire.
In a larger system, someone is probably going to get paged over this. An end user is definitely going to get an error message, probably not even a useful one, just a HTTP 500.
"},{"location":"reference/pillar/application/#pillar.application.Application.debug","title":"debug","text":"debug(msg: str, *args, **kwargs) -> None\nLog a DEBUG message
Basic debug messages
"},{"location":"reference/pillar/application/#pillar.application.Application.error","title":"error","text":"error(msg: str, *args, **kwargs) -> None\nLog an ERROR message
Something bad has happened but we caught it. We might be able to continue, but other things might start breaking. We can probably still safely exit.
In a larger system, this will likely cause a gentle alert to be placed somewhere. An end user might receive a useful error message (like a HTTP 4xx 5xx).
"},{"location":"reference/pillar/application/#pillar.application.Application.get_argument_parser","title":"get_argument_parser","text":"get_argument_parser() -> argparse.ArgumentParser\nGet the argument parser for this application.
To add your own arguments you should override this method and call super().get_argument_parser().
info(msg: str, *args, **kwargs) -> None\nLog an INFO message
Print something to the screen/logfile so we know what is happening
"},{"location":"reference/pillar/application/#pillar.application.Application.main","title":"main","text":"main() -> Optional[int]\nMain application entrypoint.
Child classes MUST implement this method.
Returns:
Type DescriptionOptional[int] An integer exit code. None will be converted to 0 during application exit.
run(*, prevent_exit: bool = False) -> int\nRun this application until completion.
Parameters:
Name Type Description Defaultprevent_exit bool Do not call sys.exit and instead allow the method to return.
False Returns:
Type Descriptionint the exit_code that would have been passed to sys.exit
setup(*, suppress_warning: bool = False) -> None\nPrepare the application to be run
In no particular order, this will:
Once called the following instance attributes will be populated:
self.argsself.configself.config_loaderself.log_levelGenerally you will not need to call this method as it is called during self.run.
This method may only be called once per an instance. If it is called multiple times, subsequent calls will have no effect and a RuntimeWarning will be emited.
If this method is overridden in a child class, it should call super().setup() or otherwise set self._setup_called to True to indicate the the application has been correctly setup.
Parameters:
Name Type Description Defaultsupress_warning Suppress the RuntimeWarning if this method is called multiple times.
vdebug(msg: str, *args, **kwargs) -> None\nLog a Verbose Debug (VDEBUG) message.
More than debug, less than everything.
"},{"location":"reference/pillar/application/#pillar.application.Application.vvdebug","title":"vvdebug","text":"vvdebug(msg: str, *args, **kwargs) -> None\nLog a Very Verbose Debug (VVDEBUG) message.
When you're tired of finding the bug and want to log everything.
"},{"location":"reference/pillar/application/#pillar.application.Application.warning","title":"warning","text":"warning(msg: str, *args, **kwargs) -> None\nLog a WARNING message
Something is wrong but we likely can recover or skip it without issue.
In a larger system this will likely just go to centralised logging.
"},{"location":"reference/pillar/application/#pillar.application.LoggingManifest","title":"LoggingManifest","text":"Simplified configuration of an Application's logging.
Be careful when modifying these options. Errors during logging setup may cause the application to error before any logging is setup causing errors to not be sent to the log files (making debugging much harder). The initial logging will use settings from both the console and file logging settings.
Attributes:
Name Type Descriptiondefault_level int Default logging level. Actual log level will be computed from this level and the application's verbosity args.
additional_namespaces List[str] Additional namespaces that should be logged.
initial_log_location str Directory for storing the initial log
initial_log_filename str filename of the initial log. Will be formatted with the application's name.
initial_log_level Union[str, int] logging level of initial log.
console_stream IO stream to output console logging to
console_format str text format for console logging
console_format_style str console_format format style
console_date_format str console_format date format
file_enabled bool enable logging to files.
file_default_location str default log file location. Actual location may be changed from the applications log-dir arg.
file_filename str filename of log. Will be formatted with the applications name.
file_format str text format for file logging
file_format_stle str file_format format style
file_date_format str file_format date format
file_max_size int max size of a log file in bytes (see RotatingFileHandler).
file_backup_count int number of old log files to keep (see RotatingFileHandler).
ConfigLoader(\n default_config: Optional[Dict[str, Any]] = None,\n default_parsers: Optional[\n Dict[str, ConfigParser]\n ] = None,\n)\n Bases: LoggingMixin
Load and merge multiple config files from various locations.
Attributes:
Name Type Descriptionlogger Logger merger Merger deepmerge merger
parsers Dict[str, ConfigParser] loaded parses
config Dict[str, Any] computed config
load_stack Dict[str, Dict[str, Any]] loaded files and their respective config before merging. Because dictionaries are ordered this will also be in order of loading. The only only exception is if the same file is loaded multiple times (don't do that).
load_errors Dict[str, Dict[str, Any]] errors encountered when loading files. If the same file is loaded multiple times, it will only have the latest error encountered.
Parameters:
Name Type Description Defaultdefault_config Optional[Dict[str, Any]] Set initial self.config to this.
None default_parsers Optional[Dict[str, ConfigParser]] Use these parsers instead of the DEFAULT_PARSERS.
None"},{"location":"reference/pillar/config/#pillar.config.ConfigLoader.critical","title":"critical","text":"critical(msg: str, *args, **kwargs) -> None\nLog a CRITICAL message
Something is on fire. We somehow caught it but we probably need to exit now. If we keep going more things may catch on fire.
In a larger system, someone is probably going to get paged over this. An end user is definitely going to get an error message, probably not even a useful one, just a HTTP 500.
"},{"location":"reference/pillar/config/#pillar.config.ConfigLoader.debug","title":"debug","text":"debug(msg: str, *args, **kwargs) -> None\nLog a DEBUG message
Basic debug messages
"},{"location":"reference/pillar/config/#pillar.config.ConfigLoader.error","title":"error","text":"error(msg: str, *args, **kwargs) -> None\nLog an ERROR message
Something bad has happened but we caught it. We might be able to continue, but other things might start breaking. We can probably still safely exit.
In a larger system, this will likely cause a gentle alert to be placed somewhere. An end user might receive a useful error message (like a HTTP 4xx 5xx).
"},{"location":"reference/pillar/config/#pillar.config.ConfigLoader.info","title":"info","text":"info(msg: str, *args, **kwargs) -> None\nLog an INFO message
Print something to the screen/logfile so we know what is happening
"},{"location":"reference/pillar/config/#pillar.config.ConfigLoader.load_config","title":"load_config","text":"load_config(\n path: Union[str, Path], *, suppress_errors: bool = False\n) -> None\nLoad a given config path.
The loaded config will be merged into self.config. The actual config loaded will also be stored in self.load_stack[path]. Any errors during loading will be stored in self.load_errors[path].
Parameters:
Name Type Description Defaultpath Union[str, Path] Path to try load from. Can include scheme (e.g. file://config/dev.json)
suppress_errors bool Prevent errors from being thrown while loading the config. This does not affect errors being stored in self.load_errors.
False Raises:
Type DescriptionValueError Unsupported path scheme
"},{"location":"reference/pillar/config/#pillar.config.ConfigLoader.vdebug","title":"vdebug","text":"vdebug(msg: str, *args, **kwargs) -> None\nLog a Verbose Debug (VDEBUG) message.
More than debug, less than everything.
"},{"location":"reference/pillar/config/#pillar.config.ConfigLoader.vvdebug","title":"vvdebug","text":"vvdebug(msg: str, *args, **kwargs) -> None\nLog a Very Verbose Debug (VVDEBUG) message.
When you're tired of finding the bug and want to log everything.
"},{"location":"reference/pillar/config/#pillar.config.ConfigLoader.warning","title":"warning","text":"warning(msg: str, *args, **kwargs) -> None\nLog a WARNING message
Something is wrong but we likely can recover or skip it without issue.
In a larger system this will likely just go to centralised logging.
"},{"location":"reference/pillar/config/#pillar.config.ConfigParser","title":"ConfigParser","text":"Base class for config loaders.
"},{"location":"reference/pillar/config/#pillar.config.ConfigParser.parse_content","title":"parse_content","text":"parse_content(content: bytes) -> Any\nParse content into usable config.
Parameters:
Name Type Description Defaultcontent bytes Raw content to load from
required"},{"location":"reference/pillar/config/#pillar.config.JsonParser","title":"JsonParser","text":" Bases: ConfigParser
Config parser for JSON
If orjson is installed will use that for parsing. It can be installed using pillar[recommended].
parse_content(content: bytes) -> Any\nParse config into a usable config.
See pillar.config.ConfigParser.parse_content.
"},{"location":"reference/pillar/config/#pillar.config.YamlParser","title":"YamlParser","text":" Bases: ConfigParser
Config parser for YAML
"},{"location":"reference/pillar/config/#pillar.config.YamlParser.parse_content","title":"parse_content","text":"parse_content(content: bytes) -> Any\nParse config into a usable config.
See pillar.config.ConfigParser.parse_content.
"},{"location":"reference/pillar/dataclass/","title":"dataclass_slots_kwargs() -> Dict[str, Any]\nGenerate dataclass slots keyword argument if it is supported by this python version.
If it is not supported contains an empty dictionary.
To use it unpack it like so @dataclass(**dataclass_slots_kwargs())
MissingDependencyError(\n feature: str,\n required_package: str,\n package_name: str,\n optional_dependency: str,\n)\n Bases: Exception
Exception for when an optional dependency is required but not installed
Parameters:
Name Type Description Defaultfeature str Name of the feature. This could be a word, phrase, or longer sentence.
requiredrequired_package str Name of the package we were trying to import. Doesn't need to be the actual import name, could be it's colloquial or PyPI name.
requiredpackage_name str Name of our package. Used to generate install instructions.
requiredoptional_dependency str Name of the optional dependency that should be installed. Used to generate install instructions.
required"},{"location":"reference/pillar/exception/#pillar.exception.PillarException","title":"PillarException","text":"PillarException(*args, **kwargs)\n Bases: Exception
Base class for all Pillar specific exceptions
"},{"location":"reference/pillar/exception/#pillar.exception.PillarMissingDependencyError","title":"PillarMissingDependencyError","text":"PillarMissingDependencyError(\n feature: str,\n required_package: str,\n optional_dependency: str,\n)\n Bases: PillarException, MissingDependencyError
Pillar is missing a required package.
As per MissingDependencyError with package_name set to \"pillar\"
Bases: LoggingMixinBase
Adds shortcut logging methods to a class
Expects that a logging.Logger exists at self.logger
critical(msg: str, *args, **kwargs) -> None\nLog a CRITICAL message
Something is on fire. We somehow caught it but we probably need to exit now. If we keep going more things may catch on fire.
In a larger system, someone is probably going to get paged over this. An end user is definitely going to get an error message, probably not even a useful one, just a HTTP 500.
"},{"location":"reference/pillar/logging/#pillar.logging.LoggingMixin.debug","title":"debug","text":"debug(msg: str, *args, **kwargs) -> None\nLog a DEBUG message
Basic debug messages
"},{"location":"reference/pillar/logging/#pillar.logging.LoggingMixin.error","title":"error","text":"error(msg: str, *args, **kwargs) -> None\nLog an ERROR message
Something bad has happened but we caught it. We might be able to continue, but other things might start breaking. We can probably still safely exit.
In a larger system, this will likely cause a gentle alert to be placed somewhere. An end user might receive a useful error message (like a HTTP 4xx 5xx).
"},{"location":"reference/pillar/logging/#pillar.logging.LoggingMixin.info","title":"info","text":"info(msg: str, *args, **kwargs) -> None\nLog an INFO message
Print something to the screen/logfile so we know what is happening
"},{"location":"reference/pillar/logging/#pillar.logging.LoggingMixin.vdebug","title":"vdebug","text":"vdebug(msg: str, *args, **kwargs) -> None\nLog a Verbose Debug (VDEBUG) message.
More than debug, less than everything.
"},{"location":"reference/pillar/logging/#pillar.logging.LoggingMixin.vvdebug","title":"vvdebug","text":"vvdebug(msg: str, *args, **kwargs) -> None\nLog a Very Verbose Debug (VVDEBUG) message.
When you're tired of finding the bug and want to log everything.
"},{"location":"reference/pillar/logging/#pillar.logging.LoggingMixin.warning","title":"warning","text":"warning(msg: str, *args, **kwargs) -> None\nLog a WARNING message
Something is wrong but we likely can recover or skip it without issue.
In a larger system this will likely just go to centralised logging.
"},{"location":"reference/pillar/logging/#pillar.logging.LoggingMixinBase","title":"LoggingMixinBase","text":"Base class for logging mixins
"},{"location":"reference/pillar/logging/#pillar.logging.NotFilter","title":"NotFilter","text":"NotFilter(name: str)\n Bases: Filter
Ignore the given logger (and all subloggers).
Is the opposite of logging.Filter.
filter(record) -> bool\nAs per logging.Filter.filter
Bases: LoggingMixinBase
Add shortcut logging methods to a class with underscore (_) prefix
Expects that a logging.Logger exists at self._logger.
get_log_level(verbosity: int, default_level: int) -> int\nGet the log level based on the default log level.
This function essentially adds a \"verbosity level\" per \"major log level\" lower than the default.
For example, if default_level == DEBUG, then verbosity == 1 will return VDEBUG and verbosity >= 2 will return VVDBEUG. For any defaul_level > INFO will always use 4 levels of verbosity (INFO, DEBUG, VDEBUG, VVDEBUG).
logging_file_handler_errors_kwargs(\n errors: str,\n) -> Dict[str, str]\nGenerate FileHandler keyword argument if it is supported by this python version.
If it is not supported contains an empty dictionary.
To use it unpack it like so FileHandler(**logging_file_handler_errors_kwargs())