Application Configuration Management

To keep the interface libraries consistent, we continue to decompose common tasks or utilities into more reusable components. Chief among these is application configuration and secrets management.

Application Configuration Management

The ability to have feature flags or change configuration in real time without deployments is something most of our applications require. Since we use Python and Django to build our service side components, Constance was an obvious solution for real time configuration. Not all of our apps use it or require it, however, so we decided to make constance an optional requirement.

setup(
    name=‘config-loader',
    version=‘0.0.1',
    packages=[‘config_loader'],
    include_package_data=True,
    description='Support optional loading of constance configs over django settings for use in libraries.',
    install_requires=install_requires,
    setup_requires=['pytest-runner'],
    tests_require=['redis', 'pytest', 'pytest-coverage', 'pytest-django', 'pytest-mock'],
    long_description=README,
    author='',
    author_email='',
    classifiers=[
        'Programming Language :: Python :: 3.7',
    ],
    extras_require={
        'constance':  ['django-constance>=2.0.0,<3'],
        'credstash':  ['credstash>=1.0.0,<2'],
        'django': ['Django>=1.11'],
    }
)

By using extras_require in the setup.py for your library, you can specify optional packages that your library supports. To have them installed, install your package like `pip install config-loader[constance]`. Find more details in the [https://setuptools.readthedocs.io/en/latest/setuptools.html#declaring-extras-optional-features-with-their-own-dependencies|setuptools docs].

Most of our applications are deployed with Zappa to AWS Lambda. This makes it convenient to do per environment configuration in a zappa_settings.yml file. To support applications that go this route, our config library also supports reading from environment variables.

In our final solution libraries can declare their settings, along with a namespace in a `Settings` object.

from config_loader import Settings

loader_settings = Settings({'A_SETTING': 'DEFAULT'}, ’NAMESPACE')

And load values from a hierarchy of places:

1. constance CONSTANCE_CONFIG = {’NAMESPACE_A_SETTING': (True, 'A fine setting'),}
2. settings.py NAMESPACE = {'A_SETTING': 'NEW_VALUE’}
3. os.environ export NAMESPACE_A_SETTING=env_Value
4. The configured default

Secrets Management

Another optional requirement, used by applications that need credentials to other systems, is credstash. Credstash leverages AWS KMS and AWS Dynamo to maintain secure storage of secret data. With a little bit of code glue, we are able to specify the name of a secure value in our clear text config, protecting the actual secrets.

Any of the values above can be surrounded by %% eg %%app.API_PASSWORD.dev%% to fetch those values from credstash.