Update DI in Python docs page

Author: rmk135

docs/introduction/di_in_python.rst

@@ -1,137 +1,255 @@
 Dependency injection and inversion of control in Python
--------------------------------------------------------
+=======================================================
 
 .. meta::
- :keywords: Python,DI,Dependency injection,IoC,Inversion of Control
- :description: This article describes benefits of dependency injection and
- inversion of control for Python applications. Also it
- contains some Python examples that show how dependency
- injection and inversion could be implemented. In addition, it
- demonstrates usage of dependency injection framework,
- IoC container and such popular design pattern as Factory.
-
-History
-~~~~~~~
-
-Originally, dependency injection pattern got popular in languages with static
-typing, like Java. Dependency injection framework can
-significantly improve flexibility of the language with static typing. Also,
-implementation of dependency injection framework for language with static
-typing is not something that one can do shortly, it could be quite complex
-thing to be done well.
-
-While Python is very flexible interpreted language with dynamic typing, there
-is a meaning that dependency injection doesn't work for it as well, as it does
-for Java. Also there is a meaning that dependency injection framework is
-something that Python developer would not ever need, cause dependency injection
-could be implemented easily using language fundamentals.
-
-Discussion
-~~~~~~~~~~
-
-It is true.
-
-Partly.
-
-Dependency injection, as a software design pattern, has number of
-advantages that are common for each language (including Python):
-
-+ Dependency Injection decreases coupling between a class and its dependency.
-+ Because dependency injection doesn't require any change in code behavior it
- can be applied to legacy code as a refactoring. The result is clients that
- are more independent and that are easier to unit test in isolation using
- stubs or mock objects that simulate other objects not under test. This ease
- of testing is often the first benefit noticed when using dependency
- injection.
-+ Dependency injection can be used to externalize a system's configuration
- details into configuration files allowing the system to be reconfigured
- without recompilation (rebuilding). Separate configurations can be written
- for different situations that require different implementations of
- components. This includes, but is not limited to, testing.
-+ Reduction of boilerplate code in the application objects since all work to
- initialize or set up dependencies is handled by a provider component.
-+ Dependency injection allows a client to remove all knowledge of a concrete
- implementation that it needs to use. This helps isolate the client from the
- impact of design changes and defects. It promotes reusability, testability
- and maintainability.
-+ Dependency injection allows a client the flexibility of being configurable.
- Only the client's behavior is fixed. The client may act on anything that
- supports the intrinsic interface the client expects.
-
-.. note::
-
- While improved testability is one the first benefits of using dependency
- injection, it could be easily overwhelmed by monkey-patching technique,
- that works absolutely great in Python (you can monkey-patch anything,
- anytime). At the same time, monkey-patching has nothing similar with
- other advantages defined above. Also monkey-patching technique is
- something that could be considered like too dirty to be used in production.
-
-The complexity of dependency injection pattern implementation in Python is
-definitely quite lower than in other languages (even with dynamic typing).
-
-.. note::
-
- Low complexity of dependency injection pattern implementation in Python
- still means that some code should be written, reviewed, tested and
- supported.
-
-Talking about inversion of control, it is a software design principle that
-also works for each programming language, not depending on its typing type.
-
-Inversion of control is used to increase modularity of the program and make
-it extensible.
-
-Main design purposes of using inversion of control are:
-
-+ To decouple the execution of a task from implementation.
-+ To focus a module on the task it is designed for.
-+ To free modules from assumptions about how other systems do what they do and
- instead rely on contracts.
-+ To prevent side effects when replacing a module.
-
-Example
-~~~~~~~
-
-Let's go through next example:
-
-.. image:: /images/miniapps/engines_cars/diagram.png
- :width: 100%
- :align: center
-
-Listing of ``example.engines`` module:
-
-.. literalinclude:: ../../examples/miniapps/engines_cars/example/engines.py
- :language: python
+ :keywords: Python,DI,Dependency injection,IoC,Inversion of Control,Example
+ :description: This page describes a usage of the dependency injection pattern in Python. It
+ contains Python examples that show how to implement dependency injection. It
+ demonstrates a usage of the dependency injection framework
+ Dependency Injector, its container, Factory, Singleton and Configuration
+ providers. The example show how to use Dependency Injector providers overriding
+ feature for testing or configuring project in different environments and explains
+ why it's better then monkey-patching.
 
-Listing of ``example.cars`` module:
+Originally dependency injection pattern got popular in the languages with a static typing,
+like Java. Dependency injection framework can significantly improve flexibility of the language
+with a static typing. Implementation of a dependency injection framework for a language
+with a static typing is not something that one can do quickly. It will be a quite complex thing
+to be done well. And will take time.
 
-.. literalinclude:: ../../examples/miniapps/engines_cars/example/cars.py
- :language: python
+Python is an interpreted language with a dynamic typing. There is an opinion that dependency
+injection doesn't work for it as well as it does for Java. A lot of the flexibility is already
+built in. Also there is an opinion that a dependency injection framework is something that
+Python developer rarely needs. Python developers say that dependency injection can be implemented
+easily using language fundamentals.
 
-Next example demonstrates creation of several cars with different engines:
+This page describes the advantages of the dependency injection usage in Python. It
+contains Python examples that show how to implement dependency injection. It demonstrates a usage
+of the dependency injection framework ``Dependency Injector``, its container, ``Factory``,
+``Singleton`` and ``Configuration`` providers. The example shows how to use ``Dependency Injector``
+providers overriding feature for testing or configuring project in different environments and
+explains why it's better then monkey-patching.
 
-.. literalinclude:: ../../examples/miniapps/engines_cars/example_di.py
- :language: python
+What is dependency injection?
+-----------------------------
 
-While previous example demonstrates advantages of dependency injection, there
-is a disadvantage demonstration as well - creation of car requires additional
-code for specification of dependencies. Nevertheless, this disadvantage could
-be easily avoided by using a dependency injection framework for creation of
-inversion of control container (IoC container).
+Let's see what the dependency injection is.
 
-Example of creation of several inversion of control containers (IoC containers)
-using :doc:`Dependency Injector <../index>`:
+Dependency injection is a principle that helps to decrease coupling and increase cohesion.
 
-.. literalinclude:: ../../examples/miniapps/engines_cars/example_ioc_containers.py
- :language: python
+.. image:: images/coupling-cohesion.png
+
+What is coupling and cohesion?
+
+Coupling and cohesion are about how tough the components are tied.
+
+- **High coupling**. If the coupling is high it's like using a superglue or welding. No easy way
+ to disassemble.
+- **High cohesion**. High cohesion is like using the screws. Very easy to disassemble and
+ assemble back or assemble a different way. It is an opposite to high coupling.
+
+When the cohesion is high the coupling is low.
+
+Low coupling brings a flexibility. Your code becomes easier to change and test.
+
+How to implement the dependency injection?
+
+Objects do not create each other anymore. They provide a way to inject the dependencies instead.
+
+Before:
+
+.. code-block:: python
+
+ import os
+
+
+ class ApiClient:
+
+ def __init__(self):
+ self.api_key = os.getenv('API_KEY') # <-- the dependency
+ self.timeout = os.getenv('TIMEOUT') # <-- the dependency
+
+
+ class Service:
+
+ def __init__(self):
+ self.api_client = ApiClient() # <-- the dependency
+
+
+ if __name__ == '__main__':
+ service = Service()
+
+
+After:
+
+.. code-block:: python
+
+ import os
+
+
+ class ApiClient:
+
+ def __init__(self, api_key: str, timeout: int):
+ self.api_key = api_key # <-- the dependency is injected
+ self.timeout = timeout # <-- the dependency is injected
+
+
+ class Service:
+
+ def __init__(self, api_client: ApiClient):
+ self.api_client = api_client # <-- the dependency is injected
+
+
+ if __name__ == '__main__':
+ service = Service(ApiClient(os.getenv('API_KEY'), os.getenv('TIMEOUT')))
+
+``ApiClient`` is decoupled from knowing where the options come from. You can read a key and a
+timeout from a configuration file or even get them from a database.
+
+``Service`` is decoupled from the ``ApiClient``. It does not create it anymore. You can provide a
+stub or other compatible object.
+
+Flexibility comes with a price.
+
+Now you need to assemble the objects like this::
+
+ service = Service(ApiClient(os.getenv('API_KEY'), os.getenv('TIMEOUT')))
+
+The assembly code might get duplicated and it'll become harder to change the application structure.
+
+Here comes the ``Dependency Injector``.
+
+What does the Dependency Injector do?
+-------------------------------------
+
+With the dependency injection pattern objects lose the responsibility of assembling the
+dependencies. The ``Dependency Injector`` absorbs that responsibility.
+
+``Dependency Injector`` helps to assemble the objects.
+
+It provides a container and providers that help you with the objects assembly. When you
+need an object you get it from the container. The rest of the assembly work is done by the
+framework:
+
+.. code-block:: python
+
+ from dependency_injector import containers, providers
+
+
+ class Container(containers.DeclarativeContainer):
+
+ config = providers.Configuration()
+
+ api_client = providers.Singleton(
+ ApiClient,
+ api_key=config.api_key,
+ timeout=config.timeout.as_int(),
+ )
+
+ service = providers.Factory(
+ Service,
+ api_client=api_client,
+ )
+
+
+ if __name__ == '__main__':
+ container = Container()
+ container.config.api_key.from_env('API_KEY')
+ container.config.timeout.from_env('TIMEOUT')
+
+ service = container.service()
+
+Retrieving of the ``Service`` instance now is done like this::
+
+ service = container.service()
+
+Objects assembling is consolidated in the container. When you need to make a change you do it in
+one place.
+
+When doing a testing you call the ``container.api_client.override()`` to replace the real API
+client with a mock:
+
+.. code-block:: python
+
+ from unittest import mock
+
+
+ with container.api_client.override(mock.Mock()):
+ service = container.service()
+
+You can override any provider by another provider.
+
+It also helps you in configuring project for the different environments: replace an API client
+with a stub on the dev or stage.
+
+Testing, Monkey-patching and dependency injection
+-------------------------------------------------
+
+The testability benefit is opposed to a monkey-patching.
+
+In Python you can monkey-patch
+anything, anytime. The problem with a monkey-patching is that it's too fragile. The reason is that
+when you monkey-patch you do something that wasn't intended to be done. You monkey-patch the
+implementation details. When implementation changes the monkey-patching is broken.
+
+With a dependency injection you patch the interface, not an implementation. This is a way more
+stable approach.
+
+Also monkey-patching is a way too dirty to be used outside of the testing code for
+reconfiguring the project for the different environments.
+
+Conclusion
+---------
+
+Dependency injection brings you 3 advantages:
+
+- **Flexibility**. The components are loosely coupled. You can easily extend or change a
+ functionality of the system by combining the components different way. You even can do it on
+ the fly.
+- **Testability**. Testing is easy because you can easily inject mocks instead of real objects
+ that use API or database, etc.
+- **Clearness and maintainability**. Dependency injection helps you reveal the dependencies.
+ Implicit becomes explicit. And "Explicit is better than implicit" (PEP20 - The Zen of Python).
+ You have all the components and dependencies defined explicitly in the container. This
+ provides an overview and control on the application structure. It is easy to understand and
+ change it.
+
+Is it worth to use a dependency injection in Python?
+
+It depends on what you build. The advantages above are not too important if you use Python as a
+scripting language. The picture is different when you use Python to create an application. The
+larger the application the more significant is the benefit.
+
+Is it worth to use a framework for the dependency injection?
+
+The complexity of the dependency injection pattern implementation in Python is
+lower than in the other languages but it's still in place. It doesn't mean you have to use a
+framework but using a framework is beneficial because the framework is:
+
+- Already implemented
+- Tested on all platforms and versions of Python
+- Documented
+- Supported
+- Known to the other engineers
+
+Few advices at last:
+
+- **Give it a try**. Dependency injection is counter-intuitive. Our nature is that
+ when we need something the first thought that comes to our mind is to go and get it. Dependency
+ injection is just like "Wait, I need to state a need instead of getting something right now".
+ It's like a little investment that will pay-off later. The advice is to just give it a try for
+ two weeks. This time will be enough for getting your own impression. If you don't like it you
+ won't lose too much.
+- **Common sense first**. Use a common sense when apply dependency injection. It is a good
+ principle, but not a silver bullet. If you do it too much you will reveal too much of the
+ implementation details. Experience comes with practice and time.
 
 What's next?
-~~~~~~~~~~~~
+------------
 
 Choose one of the following as a next step:
 
-- Look at application examples:
+- Look at the application examples:
  - :ref:`application-single-container`
  - :ref:`application-multiple-containers`
  - :ref:`decoupled-packages`
@@ -140,11 +258,12 @@ Choose one of the following as a next step:
  - :ref:`aiohttp-tutorial`
  - :ref:`asyncio-daemon-tutorial`
  - :ref:`cli-tutorial`
+- Know more about the ``Dependency Injector`` :ref:`key-features`
 - Know more about the :ref:`providers`
 - Go to the :ref:`contents`
 
 Useful links
-~~~~~~~~~~~~
+------------
 
 There are some useful links related to dependency injection design pattern
 that could be used for further reading: